2024-09-19T04:24:53.960 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T04:24:53.985 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T04:24:54.080 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911638 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/fsx}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911638' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAqEBztEqVJs0fKQMOWWFyDqDaELQa6q6M+CQbiyWhvXaFP/1ApWtro8pYKcs22xCEJI3mUTiJqQ4+XNtL15Zqk= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCcT+SY8PS/RhbLvhf/DszoDdJQT3gfgXQ4aZAnpQetuLqZmC7ZuB01iqs2GxZqxHdn1kYbEa47LbAnT4ZGk1ZQ= smithi193.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDMcEqto6L3NCIdKcOqRcJkdhTFvJHJ9ve3OrpRssuAMvSQjySPXfCDIuOHDSy1EKY90f5hV6BaUAGNty4cwvsY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T04:24:54.080 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T04:24:54.081 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T04:24:54.081 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T04:24:54.082 INFO:teuthology.task.internal:Checking packages... 2024-09-19T04:24:54.104 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T04:24:54.104 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T04:24:54.104 INFO:teuthology.packaging:ref: None 2024-09-19T04:24:54.104 INFO:teuthology.packaging:tag: None 2024-09-19T04:24:54.104 INFO:teuthology.packaging:branch: squid 2024-09-19T04:24:54.104 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:24:54.104 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T04:24:54.338 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T04:24:54.340 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T04:24:54.347 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T04:24:54.348 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T04:24:54.452 INFO:teuthology.task.internal:Saving configuration 2024-09-19T04:24:54.472 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T04:24:54.510 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T04:24:54.536 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911638', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 04:20:00.258145', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAqEBztEqVJs0fKQMOWWFyDqDaELQa6q6M+CQbiyWhvXaFP/1ApWtro8pYKcs22xCEJI3mUTiJqQ4+XNtL15Zqk='} 2024-09-19T04:24:54.555 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911638', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 04:20:00.256739', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCcT+SY8PS/RhbLvhf/DszoDdJQT3gfgXQ4aZAnpQetuLqZmC7ZuB01iqs2GxZqxHdn1kYbEa47LbAnT4ZGk1ZQ='} 2024-09-19T04:24:54.574 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911638', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 04:20:00.259516', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDMcEqto6L3NCIdKcOqRcJkdhTFvJHJ9ve3OrpRssuAMvSQjySPXfCDIuOHDSy1EKY90f5hV6BaUAGNty4cwvsY='} 2024-09-19T04:24:54.574 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T04:24:54.582 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-19T04:24:54.582 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-19T04:24:54.582 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-19T04:24:54.582 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T04:24:54.691 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc1982e3be0>, signals=[15]) 2024-09-19T04:24:54.692 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T04:24:54.698 INFO:teuthology.task.internal:Opening connections... 2024-09-19T04:24:54.699 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T04:24:54.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:24:54.779 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-09-19T04:24:54.779 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:24:54.857 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2024-09-19T04:24:54.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:24:54.957 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T04:24:54.966 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-09-19T04:24:54.984 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-09-19T04:24:54.985 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-09-19T04:24:55.043 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-09-19T04:24:55.043 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-09-19T04:24:55.043 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-09-19T04:24:55.043 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-09-19T04:24:55.043 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-09-19T04:24:55.043 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-09-19T04:24:55.043 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T04:24:55.043 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-09-19T04:24:55.043 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-09-19T04:24:55.043 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T04:24:55.044 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-09-19T04:24:55.044 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T04:24:55.044 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T04:24:55.044 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T04:24:55.044 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-09-19T04:24:55.073 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-09-19T04:24:55.092 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-09-19T04:24:55.092 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-09-19T04:24:55.150 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-09-19T04:24:55.150 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-09-19T04:24:55.150 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-09-19T04:24:55.150 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-09-19T04:24:55.151 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-09-19T04:24:55.151 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-09-19T04:24:55.151 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T04:24:55.151 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-09-19T04:24:55.151 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-09-19T04:24:55.151 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T04:24:55.151 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-09-19T04:24:55.151 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T04:24:55.151 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T04:24:55.151 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T04:24:55.152 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-09-19T04:24:55.173 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2024-09-19T04:24:55.191 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2024-09-19T04:24:55.192 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2024-09-19T04:24:55.249 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream" 2024-09-19T04:24:55.249 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9" 2024-09-19T04:24:55.250 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2024-09-19T04:24:55.250 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2024-09-19T04:24:55.250 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9" 2024-09-19T04:24:55.250 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9" 2024-09-19T04:24:55.250 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T04:24:55.250 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2024-09-19T04:24:55.250 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon" 2024-09-19T04:24:55.250 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T04:24:55.250 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/" 2024-09-19T04:24:55.250 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T04:24:55.250 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T04:24:55.250 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T04:24:55.251 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2024-09-19T04:24:55.273 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T04:24:55.284 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T04:24:55.322 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T04:24:55.322 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-09-19T04:24:55.325 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-09-19T04:24:55.327 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2024-09-19T04:24:55.343 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T04:24:55.351 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T04:24:55.351 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-09-19T04:24:55.381 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-09-19T04:24:55.385 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2024-09-19T04:24:55.653 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T04:24:55.720 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T04:24:55.738 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-19T04:24:55.738 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-19T04:24:55.738 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-19T04:24:55.738 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T04:24:55.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:24:55.756 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-19T04:24:55.918 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-19T04:24:55.918 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-19T04:24:55.918 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-09-19T04:24:55.936 INFO:teuthology.orchestra.run.smithi123.stdout:5.14.0-503.el9.x86_64 2024-09-19T04:24:55.936 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-19T04:24:55.937 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-19T04:24:55.937 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum upgrade -y linux-firmware 2024-09-19T04:24:58.810 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 7.3 MB/s | 8.3 MB 00:01 2024-09-19T04:25:01.086 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 17 MB/s | 20 MB 00:01 2024-09-19T04:25:06.512 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.5 MB 00:01 2024-09-19T04:25:08.221 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-19T04:25:09.091 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-19T04:25:14.547 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-19T04:25:16.567 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-19T04:25:16.577 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-19T04:25:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-09-19T04:25:16.577 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-19T04:25:16.577 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-09-19T04:25:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: linux-firmware noarch 20240905-146.el9 baseos 388 M 2024-09-19T04:25:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:25:16.578 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-19T04:25:16.578 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-19T04:25:16.578 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 1 Package 2024-09-19T04:25:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:25:16.578 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 388 M 2024-09-19T04:25:16.578 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-19T04:25:22.240 INFO:teuthology.orchestra.run.smithi123.stdout:linux-firmware-20240905-146.el9.noarch.rpm 70 MB/s | 388 MB 00:05 2024-09-19T04:25:22.240 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:25:22.240 INFO:teuthology.orchestra.run.smithi123.stdout:Total 69 MB/s | 388 MB 00:05 2024-09-19T04:25:25.349 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-19T04:25:25.520 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-19T04:25:25.520 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-19T04:25:28.833 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-19T04:25:28.833 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-19T04:25:38.417 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-19T04:25:38.588 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-19T04:25:38.668 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-19T04:25:39.460 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: linux-firmware-20240716-145.el9.noarch 2/2 2024-09-19T04:25:39.460 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-19T04:25:40.224 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-19T04:25:40.224 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:25:40.224 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-09-19T04:25:40.224 INFO:teuthology.orchestra.run.smithi123.stdout: linux-firmware-20240905-146.el9.noarch 2024-09-19T04:25:40.224 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:25:40.224 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-19T04:25:40.569 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-19T04:25:40.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-19T04:25:40.721 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-19T04:25:40.721 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-19T04:25:40.724 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi123.front.sepia.ceph.com to complete... 2024-09-19T04:25:48.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19 04:25:48 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-19T04:25:48.082 INFO:teuthology.orchestra.run.smithi123.stderr:FINISHED --2024-09-19 04:25:48-- 2024-09-19T04:25:48.082 INFO:teuthology.orchestra.run.smithi123.stderr:Total wall clock time: 7.3s 2024-09-19T04:25:48.082 INFO:teuthology.orchestra.run.smithi123.stderr:Downloaded: 1 files, 343M in 7.0s (48.7 MB/s) 2024-09-19T04:25:48.085 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-19T04:25:48.086 DEBUG:teuthology.orchestra.run.smithi123:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-19T04:25:50.148 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying... ######################################## 2024-09-19T04:25:50.298 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing... ######################################## 2024-09-19T04:25:50.357 INFO:teuthology.orchestra.run.smithi123.stdout:Updating / installing... 2024-09-19T04:25:58.366 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-19T04:26:38.342 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi123.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-19T04:26:38.342 DEBUG:teuthology.orchestra.run.smithi123:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-19T04:26:38.422 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:26:38.422 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:26:38.422 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:26:38.422 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:26:38.422 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-19T04:26:38.422 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-19T04:26:38.423 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-19T04:26:38.423 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-19T04:26:38.423 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-19T04:26:38.423 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-19T04:26:38.423 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-19T04:26:38.423 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-19T04:26:38.423 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-19T04:26:38.423 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-19T04:26:38.423 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-19T04:26:38.423 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-19T04:26:38.423 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-19T04:26:38.423 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-19T04:26:38.424 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-19T04:26:38.424 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-19T04:26:38.424 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-19T04:26:38.424 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-19T04:26:38.424 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-19T04:26:38.424 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-19T04:26:38.424 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-19T04:26:38.424 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-19T04:26:38.424 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-19T04:26:38.424 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-19T04:26:38.424 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-19T04:26:38.424 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-19T04:26:38.425 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-19T04:26:38.425 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-19T04:26:38.425 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-19T04:26:38.425 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-19T04:26:38.425 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-19T04:26:38.425 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-19T04:26:38.425 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-19T04:26:38.425 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-19T04:26:38.425 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-19T04:26:38.425 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-19T04:26:38.425 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-19T04:26:38.425 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-19T04:26:38.426 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-19T04:26:38.426 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-19T04:26:38.426 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-19T04:26:38.426 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-19T04:26:38.426 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-19T04:26:38.426 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-19T04:26:38.426 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-19T04:26:38.426 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-19T04:26:38.426 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-19T04:26:38.426 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-19T04:26:38.426 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-19T04:26:38.426 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-19T04:26:38.426 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-19T04:26:38.427 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-19T04:26:38.427 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-19T04:26:38.427 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-19T04:26:38.427 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-19T04:26:38.427 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-19T04:26:38.427 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-19T04:26:38.427 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-19T04:26:38.427 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-19T04:26:38.427 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-19T04:26:38.427 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-19T04:26:38.427 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-19T04:26:38.427 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-19T04:26:38.427 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-19T04:26:38.428 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-19T04:26:38.428 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-19T04:26:38.428 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-19T04:26:38.428 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-19T04:26:38.428 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-19T04:26:38.428 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-19T04:26:38.428 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-19T04:26:38.428 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-19T04:26:38.428 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-19T04:26:38.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-19T04:26:38.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-19T04:26:38.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-19T04:26:38.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-19T04:26:38.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-19T04:26:38.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-19T04:26:38.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-19T04:26:38.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-19T04:26:38.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-19T04:26:38.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-19T04:26:38.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-19T04:26:38.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-19T04:26:38.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-19T04:26:38.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-19T04:26:38.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-19T04:26:38.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-19T04:26:38.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-19T04:26:38.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-19T04:26:38.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-19T04:26:38.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-19T04:26:38.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-19T04:26:38.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-19T04:26:38.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-19T04:26:38.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-19T04:26:38.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-19T04:26:38.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-19T04:26:38.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-19T04:26:38.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-19T04:26:38.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-19T04:26:38.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-19T04:26:38.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-19T04:26:38.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-19T04:26:38.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-19T04:26:38.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-19T04:26:38.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-19T04:26:38.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-19T04:26:38.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-19T04:26:38.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-19T04:26:38.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-19T04:26:38.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-19T04:26:38.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-19T04:26:38.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-19T04:26:38.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-19T04:26:38.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-19T04:26:38.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-19T04:26:38.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-19T04:26:38.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-19T04:26:38.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-19T04:26:38.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-19T04:26:38.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-19T04:26:38.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-19T04:26:38.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-19T04:26:38.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-19T04:26:38.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-19T04:26:38.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-19T04:26:38.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-19T04:26:38.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-19T04:26:38.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-19T04:26:38.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-19T04:26:38.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-19T04:26:38.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-19T04:26:38.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-19T04:26:38.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-19T04:26:38.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-19T04:26:38.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-19T04:26:38.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-19T04:26:38.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-19T04:26:38.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-19T04:26:38.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-19T04:26:38.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-19T04:26:38.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-19T04:26:38.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-19T04:26:38.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-19T04:26:38.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-19T04:26:38.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-19T04:26:38.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-19T04:26:38.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-19T04:26:38.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-19T04:26:38.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-19T04:26:38.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-19T04:26:38.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-19T04:26:38.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-19T04:26:38.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-19T04:26:38.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-19T04:26:38.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-19T04:26:38.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-19T04:26:38.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-19T04:26:38.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-19T04:26:38.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-19T04:26:38.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-19T04:26:38.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-19T04:26:38.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-19T04:26:38.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-19T04:26:38.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-19T04:26:38.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-19T04:26:38.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-19T04:26:38.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-19T04:26:38.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-19T04:26:38.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-19T04:26:38.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-19T04:26:38.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-19T04:26:38.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-19T04:26:38.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-19T04:26:38.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-19T04:26:38.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-19T04:26:38.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-19T04:26:38.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-19T04:26:38.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-19T04:26:38.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-19T04:26:38.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-19T04:26:38.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-19T04:26:38.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-19T04:26:38.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-19T04:26:38.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-19T04:26:38.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-19T04:26:38.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-19T04:26:38.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-19T04:26:38.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-19T04:26:38.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-19T04:26:38.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-19T04:26:38.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-19T04:26:38.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-19T04:26:38.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-19T04:26:38.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-19T04:26:38.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-19T04:26:38.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-19T04:26:38.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-19T04:26:38.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-19T04:26:38.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-19T04:26:38.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-19T04:26:38.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-19T04:26:38.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-19T04:26:38.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-19T04:26:38.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-19T04:26:38.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-19T04:26:38.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-19T04:26:38.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-19T04:26:38.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-19T04:26:38.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-19T04:26:38.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-19T04:26:38.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-19T04:26:38.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-19T04:26:38.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-19T04:26:38.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-19T04:26:38.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-19T04:26:38.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-19T04:26:38.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-19T04:26:38.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-19T04:26:38.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-19T04:26:38.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-19T04:26:38.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-19T04:26:38.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-19T04:26:38.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-19T04:26:38.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-19T04:26:38.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-19T04:26:38.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-19T04:26:38.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-19T04:26:38.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-19T04:26:38.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-19T04:26:38.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-19T04:26:38.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-19T04:26:38.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-19T04:26:38.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-19T04:26:38.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-19T04:26:38.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-19T04:26:38.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-19T04:26:38.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-19T04:26:38.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-19T04:26:38.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-19T04:26:38.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-19T04:26:38.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-19T04:26:38.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-19T04:26:38.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-19T04:26:38.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-19T04:26:38.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-19T04:26:38.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-19T04:26:38.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-19T04:26:38.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-19T04:26:38.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-19T04:26:38.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-19T04:26:38.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-19T04:26:38.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-19T04:26:38.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-19T04:26:38.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-19T04:26:38.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-19T04:26:38.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-19T04:26:38.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-19T04:26:38.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-19T04:26:38.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-19T04:26:38.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-19T04:26:38.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-19T04:26:38.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-19T04:26:38.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-19T04:26:38.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-19T04:26:38.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-19T04:26:38.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-19T04:26:38.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-19T04:26:38.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-19T04:26:38.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-19T04:26:38.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-19T04:26:38.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-19T04:26:38.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-19T04:26:38.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-19T04:26:38.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-19T04:26:38.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-19T04:26:38.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-19T04:26:38.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-19T04:26:38.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-19T04:26:38.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-19T04:26:38.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-19T04:26:38.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-19T04:26:38.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-19T04:26:38.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-19T04:26:38.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-19T04:26:38.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-19T04:26:38.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-19T04:26:38.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-19T04:26:38.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-19T04:26:38.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-19T04:26:38.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-19T04:26:38.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-19T04:26:38.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-19T04:26:38.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-19T04:26:38.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-19T04:26:38.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-19T04:26:38.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-19T04:26:38.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-19T04:26:38.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-19T04:26:38.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-19T04:26:38.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-19T04:26:38.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-19T04:26:38.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-19T04:26:38.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-19T04:26:38.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-19T04:26:38.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-19T04:26:38.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-19T04:26:38.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-19T04:26:38.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-19T04:26:38.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-19T04:26:38.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-19T04:26:38.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-19T04:26:38.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-19T04:26:38.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-19T04:26:38.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-19T04:26:38.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-19T04:26:38.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-19T04:26:38.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-19T04:26:38.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-19T04:26:38.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-19T04:26:38.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-19T04:26:38.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-19T04:26:38.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-19T04:26:38.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-19T04:26:38.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-19T04:26:38.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-19T04:26:38.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-19T04:26:38.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-19T04:26:38.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-19T04:26:38.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-19T04:26:38.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-19T04:26:38.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-19T04:26:38.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-19T04:26:38.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-19T04:26:38.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-19T04:26:38.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-19T04:26:38.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-19T04:26:38.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-19T04:26:38.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-19T04:26:38.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-19T04:26:38.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-19T04:26:38.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-19T04:26:38.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-19T04:26:38.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-19T04:26:38.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-19T04:26:38.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-19T04:26:38.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-19T04:26:38.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-19T04:26:38.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-19T04:26:38.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-19T04:26:38.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-19T04:26:38.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-19T04:26:38.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-19T04:26:38.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-19T04:26:38.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-19T04:26:38.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-19T04:26:38.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-19T04:26:38.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-19T04:26:38.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-19T04:26:38.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-19T04:26:38.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-19T04:26:38.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-19T04:26:38.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-19T04:26:38.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-19T04:26:38.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-19T04:26:38.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-19T04:26:38.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-19T04:26:38.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-19T04:26:38.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-19T04:26:38.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-19T04:26:38.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-19T04:26:38.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-19T04:26:38.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-19T04:26:38.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-19T04:26:38.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-19T04:26:38.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-19T04:26:38.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-19T04:26:38.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-19T04:26:38.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-19T04:26:38.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-19T04:26:38.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-19T04:26:38.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-19T04:26:38.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-19T04:26:38.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-19T04:26:38.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-19T04:26:38.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-19T04:26:38.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-19T04:26:38.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-19T04:26:38.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-19T04:26:38.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-19T04:26:38.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-19T04:26:38.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-19T04:26:38.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-19T04:26:38.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-19T04:26:38.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-19T04:26:38.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-19T04:26:38.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-19T04:26:38.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-19T04:26:38.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-19T04:26:38.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-19T04:26:38.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-19T04:26:38.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-19T04:26:38.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-19T04:26:38.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-19T04:26:38.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-19T04:26:38.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-19T04:26:38.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-19T04:26:38.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-19T04:26:38.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-19T04:26:38.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-19T04:26:38.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-19T04:26:38.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-19T04:26:38.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-19T04:26:38.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-19T04:26:38.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-19T04:26:38.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-19T04:26:38.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-19T04:26:38.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-19T04:26:38.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-19T04:26:38.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-19T04:26:38.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-19T04:26:38.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-19T04:26:38.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-19T04:26:38.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-19T04:26:38.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-19T04:26:38.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-19T04:26:38.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-19T04:26:38.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-19T04:26:38.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-19T04:26:38.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-19T04:26:38.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-19T04:26:38.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-19T04:26:38.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-19T04:26:38.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-19T04:26:38.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-19T04:26:38.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-19T04:26:38.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-19T04:26:38.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-19T04:26:38.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-19T04:26:38.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-19T04:26:38.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-19T04:26:38.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-19T04:26:38.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-19T04:26:38.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-19T04:26:38.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-19T04:26:38.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-19T04:26:38.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-19T04:26:38.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-19T04:26:38.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-19T04:26:38.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-19T04:26:38.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-19T04:26:38.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-19T04:26:38.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-19T04:26:38.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-19T04:26:38.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-19T04:26:38.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-19T04:26:38.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-19T04:26:38.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-19T04:26:38.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-19T04:26:38.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-19T04:26:38.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-19T04:26:38.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-19T04:26:38.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-19T04:26:38.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-19T04:26:38.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-19T04:26:38.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-19T04:26:38.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-19T04:26:38.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-19T04:26:38.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-19T04:26:38.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-19T04:26:38.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-19T04:26:38.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-19T04:26:38.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-19T04:26:38.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-19T04:26:38.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-19T04:26:38.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-19T04:26:38.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-19T04:26:38.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-19T04:26:38.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-19T04:26:38.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-19T04:26:38.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-19T04:26:38.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-19T04:26:38.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-19T04:26:38.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-19T04:26:38.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-19T04:26:38.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-19T04:26:38.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-19T04:26:38.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-19T04:26:38.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-19T04:26:38.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-19T04:26:38.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-19T04:26:38.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-19T04:26:38.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-19T04:26:38.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-19T04:26:38.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-19T04:26:38.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-19T04:26:38.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-19T04:26:38.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-19T04:26:38.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-19T04:26:38.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-19T04:26:38.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-19T04:26:38.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-19T04:26:38.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-19T04:26:38.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-19T04:26:38.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-19T04:26:38.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-19T04:26:38.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-19T04:26:38.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-19T04:26:38.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-19T04:26:38.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-19T04:26:38.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-19T04:26:38.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-19T04:26:38.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-19T04:26:38.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-19T04:26:38.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-19T04:26:38.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-19T04:26:38.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-19T04:26:38.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-19T04:26:38.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-19T04:26:38.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-19T04:26:38.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-19T04:26:38.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-19T04:26:38.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-19T04:26:38.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-19T04:26:38.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-19T04:26:38.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-19T04:26:38.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-19T04:26:38.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-19T04:26:38.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-19T04:26:38.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-19T04:26:38.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-19T04:26:38.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-19T04:26:38.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-19T04:26:38.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-19T04:26:38.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-19T04:26:38.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-19T04:26:38.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-19T04:26:38.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-19T04:26:38.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-19T04:26:38.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-19T04:26:38.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-19T04:26:38.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-19T04:26:38.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-19T04:26:38.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-19T04:26:38.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-19T04:26:38.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-19T04:26:38.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-19T04:26:38.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-19T04:26:38.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-19T04:26:38.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-19T04:26:38.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-19T04:26:38.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-19T04:26:38.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-19T04:26:38.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-19T04:26:38.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-19T04:26:38.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-19T04:26:38.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-19T04:26:38.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-19T04:26:38.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-19T04:26:38.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-19T04:26:38.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-19T04:26:38.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-19T04:26:38.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-19T04:26:38.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-19T04:26:38.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-19T04:26:38.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-19T04:26:38.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-19T04:26:38.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-19T04:26:38.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-19T04:26:38.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-19T04:26:38.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-19T04:26:38.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-19T04:26:38.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-19T04:26:38.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-19T04:26:38.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-19T04:26:38.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-19T04:26:38.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-19T04:26:38.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-19T04:26:38.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-19T04:26:38.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-19T04:26:38.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-19T04:26:38.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-19T04:26:38.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-19T04:26:38.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-19T04:26:38.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-19T04:26:38.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-19T04:26:38.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-19T04:26:38.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-19T04:26:38.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-19T04:26:38.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-19T04:26:38.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-19T04:26:38.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-19T04:26:38.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-19T04:26:38.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-19T04:26:38.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-19T04:26:38.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-19T04:26:38.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-19T04:26:38.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-19T04:26:38.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-19T04:26:38.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-19T04:26:38.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-19T04:26:38.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-19T04:26:38.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-19T04:26:38.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-19T04:26:38.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-19T04:26:38.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-19T04:26:38.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-19T04:26:38.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-19T04:26:38.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-19T04:26:38.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-19T04:26:38.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-19T04:26:38.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-19T04:26:38.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-19T04:26:38.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-19T04:26:38.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-19T04:26:38.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-19T04:26:38.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-19T04:26:38.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-19T04:26:38.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-19T04:26:38.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-19T04:26:38.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-19T04:26:38.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-19T04:26:38.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-19T04:26:38.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-19T04:26:38.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-19T04:26:38.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-19T04:26:38.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-19T04:26:38.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-19T04:26:38.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-19T04:26:38.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-19T04:26:38.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-19T04:26:38.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-19T04:26:38.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-19T04:26:38.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-19T04:26:38.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-19T04:26:38.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-19T04:26:38.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-19T04:26:38.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-19T04:26:38.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-19T04:26:38.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-19T04:26:38.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-19T04:26:38.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-19T04:26:38.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-19T04:26:38.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-19T04:26:38.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-19T04:26:38.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-19T04:26:38.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-19T04:26:38.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-19T04:26:38.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-19T04:26:38.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-19T04:26:38.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-19T04:26:38.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-19T04:26:38.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-19T04:26:38.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-19T04:26:38.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-19T04:26:38.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-19T04:26:38.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-19T04:26:38.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-19T04:26:38.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-19T04:26:38.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-19T04:26:38.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-19T04:26:38.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-19T04:26:38.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-19T04:26:38.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-19T04:26:38.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-19T04:26:38.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-19T04:26:38.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-19T04:26:38.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-19T04:26:38.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-19T04:26:38.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-19T04:26:38.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-19T04:26:38.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-19T04:26:38.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-19T04:26:38.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-19T04:26:38.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-19T04:26:38.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-19T04:26:38.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-19T04:26:38.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-19T04:26:38.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-19T04:26:38.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-19T04:26:38.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-19T04:26:38.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-19T04:26:38.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-19T04:26:38.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-19T04:26:38.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-19T04:26:38.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-19T04:26:38.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-19T04:26:38.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-19T04:26:38.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-19T04:26:38.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-19T04:26:38.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-19T04:26:38.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-19T04:26:38.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-19T04:26:38.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-19T04:26:38.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-19T04:26:38.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-19T04:26:38.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-19T04:26:38.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-19T04:26:38.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-19T04:26:38.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-19T04:26:38.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-19T04:26:38.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-19T04:26:38.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-19T04:26:38.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-19T04:26:38.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-19T04:26:38.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-19T04:26:38.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-19T04:26:38.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-19T04:26:38.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-19T04:26:38.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-19T04:26:38.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-19T04:26:38.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-19T04:26:38.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-19T04:26:38.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-19T04:26:38.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-19T04:26:38.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-19T04:26:38.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-19T04:26:38.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-19T04:26:38.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-19T04:26:38.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-19T04:26:38.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-19T04:26:38.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-19T04:26:38.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-19T04:26:38.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-19T04:26:38.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-19T04:26:38.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-19T04:26:38.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-19T04:26:38.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-19T04:26:38.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-19T04:26:38.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-19T04:26:38.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-19T04:26:38.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-19T04:26:38.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-19T04:26:38.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-19T04:26:38.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-19T04:26:38.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-19T04:26:38.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-19T04:26:38.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-19T04:26:38.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-19T04:26:38.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-19T04:26:38.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-19T04:26:38.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-19T04:26:38.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-19T04:26:38.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-19T04:26:38.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-19T04:26:38.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-19T04:26:38.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-19T04:26:38.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-19T04:26:38.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-19T04:26:38.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-19T04:26:38.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-19T04:26:38.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-19T04:26:38.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-19T04:26:38.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-19T04:26:38.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-19T04:26:38.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-19T04:26:38.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-19T04:26:38.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-19T04:26:38.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-19T04:26:38.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-19T04:26:38.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-19T04:26:38.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-19T04:26:38.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-19T04:26:38.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-19T04:26:38.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-19T04:26:38.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-19T04:26:38.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-19T04:26:38.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-19T04:26:38.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-19T04:26:38.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-19T04:26:38.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-19T04:26:38.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-19T04:26:38.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-19T04:26:38.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-19T04:26:38.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-19T04:26:38.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-19T04:26:38.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-19T04:26:38.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-19T04:26:38.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-19T04:26:38.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-19T04:26:38.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-19T04:26:38.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-19T04:26:38.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-19T04:26:38.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-19T04:26:38.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-19T04:26:38.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-19T04:26:38.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-19T04:26:38.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-19T04:26:38.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-19T04:26:38.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-19T04:26:38.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-19T04:26:38.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-19T04:26:38.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-19T04:26:38.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-19T04:26:38.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-19T04:26:38.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-19T04:26:38.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-19T04:26:38.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-19T04:26:38.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-19T04:26:38.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-19T04:26:38.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-19T04:26:38.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-19T04:26:38.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-19T04:26:38.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-19T04:26:38.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-19T04:26:38.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-19T04:26:38.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-19T04:26:38.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-19T04:26:38.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-19T04:26:38.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-19T04:26:38.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-19T04:26:38.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-19T04:26:38.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-19T04:26:38.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-19T04:26:38.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-19T04:26:38.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-19T04:26:38.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-19T04:26:38.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-19T04:26:38.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-19T04:26:38.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-19T04:26:38.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-19T04:26:38.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-19T04:26:38.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-19T04:26:38.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-19T04:26:38.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-19T04:26:38.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-19T04:26:38.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-19T04:26:38.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-19T04:26:38.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-19T04:26:38.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-19T04:26:38.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-19T04:26:38.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-19T04:26:38.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-19T04:26:38.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-19T04:26:38.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-19T04:26:38.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-19T04:26:38.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-19T04:26:38.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-19T04:26:38.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-19T04:26:38.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-19T04:26:38.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-19T04:26:38.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-19T04:26:38.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-19T04:26:38.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-19T04:26:38.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-19T04:26:38.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-19T04:26:38.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-19T04:26:38.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-19T04:26:38.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-19T04:26:38.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-19T04:26:38.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-19T04:26:38.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-19T04:26:38.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-19T04:26:38.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-19T04:26:38.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-19T04:26:38.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-19T04:26:38.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-19T04:26:38.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-19T04:26:38.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-19T04:26:38.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-19T04:26:38.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-19T04:26:38.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-19T04:26:38.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-19T04:26:38.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-19T04:26:38.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-19T04:26:38.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-19T04:26:38.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-19T04:26:38.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-19T04:26:38.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-19T04:26:38.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-19T04:26:38.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-19T04:26:38.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-19T04:26:38.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-19T04:26:38.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-19T04:26:38.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-19T04:26:38.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-19T04:26:38.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-19T04:26:38.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-19T04:26:38.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-19T04:26:38.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-19T04:26:38.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-19T04:26:38.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-19T04:26:38.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-19T04:26:38.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-19T04:26:38.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-19T04:26:38.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-19T04:26:38.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-19T04:26:38.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-19T04:26:38.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-19T04:26:38.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-19T04:26:38.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-19T04:26:38.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-19T04:26:38.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-19T04:26:38.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-19T04:26:38.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-19T04:26:38.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-19T04:26:38.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-19T04:26:38.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-19T04:26:38.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-19T04:26:38.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-19T04:26:38.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-19T04:26:38.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-19T04:26:38.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-19T04:26:38.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-19T04:26:38.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-19T04:26:38.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-19T04:26:38.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-19T04:26:38.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-19T04:26:38.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-19T04:26:38.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-19T04:26:38.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-19T04:26:38.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-19T04:26:38.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-19T04:26:38.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-19T04:26:38.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-19T04:26:38.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-19T04:26:38.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-19T04:26:38.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-19T04:26:38.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-19T04:26:38.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-19T04:26:38.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-19T04:26:38.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-19T04:26:38.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-19T04:26:38.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-19T04:26:38.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-19T04:26:38.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-19T04:26:38.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-19T04:26:38.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-19T04:26:38.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-19T04:26:38.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-19T04:26:38.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-19T04:26:38.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-19T04:26:38.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-19T04:26:38.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-19T04:26:38.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-19T04:26:38.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-19T04:26:38.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-19T04:26:38.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-19T04:26:38.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-19T04:26:38.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-19T04:26:38.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-19T04:26:38.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-19T04:26:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-19T04:26:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-19T04:26:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-19T04:26:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-19T04:26:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-19T04:26:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-19T04:26:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-19T04:26:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-19T04:26:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-19T04:26:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-19T04:26:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-19T04:26:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-19T04:26:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-19T04:26:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-19T04:26:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-19T04:26:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-19T04:26:38.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-19T04:26:38.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-19T04:26:38.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-19T04:26:38.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-19T04:26:38.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-19T04:26:38.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-19T04:26:38.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-19T04:26:38.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-19T04:26:38.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-19T04:26:38.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-19T04:26:38.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-19T04:26:38.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-19T04:26:38.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-19T04:26:38.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-19T04:26:38.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-19T04:26:38.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-19T04:26:38.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-19T04:26:38.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-19T04:26:38.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-19T04:26:38.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-19T04:26:38.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-19T04:26:38.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-19T04:26:38.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-19T04:26:38.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-19T04:26:38.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-19T04:26:38.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-19T04:26:38.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-19T04:26:38.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-19T04:26:38.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-19T04:26:38.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-19T04:26:38.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-19T04:26:38.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-19T04:26:38.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-19T04:26:38.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-19T04:26:38.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-19T04:26:38.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-19T04:26:38.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-19T04:26:38.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-19T04:26:38.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-19T04:26:38.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-19T04:26:38.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-19T04:26:38.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-19T04:26:38.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-19T04:26:38.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-19T04:26:38.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-19T04:26:38.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-19T04:26:38.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-19T04:26:38.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-19T04:26:38.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-19T04:26:38.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-19T04:26:38.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-19T04:26:38.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-19T04:26:38.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-19T04:26:38.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-19T04:26:38.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-19T04:26:38.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-19T04:26:38.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-19T04:26:38.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-19T04:26:38.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-19T04:26:38.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-19T04:26:38.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-19T04:26:38.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-19T04:26:38.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-19T04:26:38.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-19T04:26:38.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-19T04:26:38.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-19T04:26:38.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-19T04:26:38.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-19T04:26:38.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-19T04:26:38.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-19T04:26:38.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-19T04:26:38.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-19T04:26:38.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-19T04:26:38.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-19T04:26:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-19T04:26:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-19T04:26:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-19T04:26:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-19T04:26:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-19T04:26:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-19T04:26:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-19T04:26:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-19T04:26:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-19T04:26:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-19T04:26:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-19T04:26:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-19T04:26:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-19T04:26:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-19T04:26:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-19T04:26:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-19T04:26:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-19T04:26:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-19T04:26:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-19T04:26:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-19T04:26:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-19T04:26:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-19T04:26:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-19T04:26:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-19T04:26:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-19T04:26:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-19T04:26:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-19T04:26:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-19T04:26:38.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-19T04:26:38.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-19T04:26:38.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-19T04:26:38.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-19T04:26:38.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-19T04:26:38.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-19T04:26:38.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-19T04:26:38.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-19T04:26:38.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-19T04:26:38.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-19T04:26:38.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-19T04:26:38.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-19T04:26:38.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-19T04:26:38.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-19T04:26:38.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-19T04:26:38.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-19T04:26:38.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-19T04:26:38.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-19T04:26:38.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-19T04:26:38.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-19T04:26:38.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-19T04:26:38.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-19T04:26:38.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-19T04:26:38.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-19T04:26:38.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-19T04:26:38.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-19T04:26:38.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-19T04:26:38.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-19T04:26:38.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-19T04:26:38.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-19T04:26:38.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-19T04:26:38.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-19T04:26:38.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-19T04:26:38.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-19T04:26:38.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-19T04:26:38.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-19T04:26:38.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-19T04:26:38.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-19T04:26:38.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-19T04:26:38.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-19T04:26:38.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-19T04:26:38.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-19T04:26:38.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-19T04:26:38.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-19T04:26:38.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-19T04:26:38.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-19T04:26:38.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-19T04:26:38.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-19T04:26:38.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-19T04:26:38.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-19T04:26:38.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-19T04:26:38.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-19T04:26:38.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-19T04:26:38.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-19T04:26:38.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-19T04:26:38.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-19T04:26:38.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-19T04:26:38.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-19T04:26:38.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-19T04:26:38.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-19T04:26:38.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-19T04:26:38.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-19T04:26:38.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-19T04:26:38.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-19T04:26:38.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-19T04:26:38.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-19T04:26:38.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-19T04:26:38.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-19T04:26:38.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-19T04:26:38.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-19T04:26:38.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-19T04:26:38.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-19T04:26:38.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-19T04:26:38.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-19T04:26:38.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-19T04:26:38.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-19T04:26:38.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-19T04:26:38.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-19T04:26:38.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-19T04:26:38.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-19T04:26:38.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-19T04:26:38.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-19T04:26:38.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-19T04:26:38.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-19T04:26:38.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-19T04:26:38.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-19T04:26:38.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-19T04:26:38.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-19T04:26:38.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-19T04:26:38.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-19T04:26:38.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-19T04:26:38.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-19T04:26:38.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-19T04:26:38.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-19T04:26:38.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-19T04:26:38.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-19T04:26:38.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-19T04:26:38.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-19T04:26:38.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-19T04:26:38.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-19T04:26:38.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-19T04:26:38.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-19T04:26:38.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-19T04:26:38.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-19T04:26:38.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-19T04:26:38.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-19T04:26:38.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-19T04:26:38.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-19T04:26:38.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-19T04:26:38.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-19T04:26:38.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-19T04:26:38.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-19T04:26:38.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-19T04:26:38.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-19T04:26:38.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-19T04:26:38.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-19T04:26:38.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-19T04:26:38.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-19T04:26:38.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-19T04:26:38.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-19T04:26:38.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-19T04:26:38.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-19T04:26:38.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-19T04:26:38.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-19T04:26:38.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-19T04:26:38.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-19T04:26:38.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-19T04:26:38.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-19T04:26:38.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-19T04:26:38.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-19T04:26:38.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-19T04:26:38.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-19T04:26:38.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-19T04:26:38.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-19T04:26:38.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-19T04:26:38.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-19T04:26:38.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-19T04:26:38.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-19T04:26:38.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-19T04:26:38.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-19T04:26:38.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-19T04:26:38.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-19T04:26:38.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-19T04:26:38.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-19T04:26:38.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-19T04:26:38.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-19T04:26:38.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-19T04:26:38.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-19T04:26:38.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-19T04:26:38.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-19T04:26:38.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-19T04:26:38.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-19T04:26:38.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-19T04:26:38.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-19T04:26:38.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-19T04:26:38.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-19T04:26:38.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-19T04:26:38.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-19T04:26:38.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-19T04:26:38.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-19T04:26:38.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-19T04:26:38.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-19T04:26:38.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-19T04:26:38.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-19T04:26:38.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-19T04:26:38.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-19T04:26:38.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-19T04:26:38.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-19T04:26:38.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-19T04:26:38.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-19T04:26:38.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-19T04:26:38.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-19T04:26:38.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-19T04:26:38.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-19T04:26:38.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-19T04:26:38.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-19T04:26:38.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-19T04:26:38.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-19T04:26:38.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-19T04:26:38.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-19T04:26:38.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-19T04:26:38.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-19T04:26:38.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-19T04:26:38.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-19T04:26:38.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-19T04:26:38.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-19T04:26:38.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-19T04:26:38.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-19T04:26:38.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-19T04:26:38.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-19T04:26:38.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-19T04:26:38.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-19T04:26:38.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-19T04:26:38.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-19T04:26:38.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-19T04:26:38.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-19T04:26:38.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-19T04:26:38.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-19T04:26:38.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-19T04:26:38.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-19T04:26:38.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-19T04:26:38.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-19T04:26:38.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-19T04:26:38.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-19T04:26:38.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-19T04:26:38.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-19T04:26:38.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-19T04:26:38.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-19T04:26:38.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-19T04:26:38.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-19T04:26:38.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-19T04:26:38.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-19T04:26:38.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-19T04:26:38.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-19T04:26:38.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-19T04:26:38.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-19T04:26:38.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-19T04:26:38.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-19T04:26:38.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-19T04:26:38.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-19T04:26:38.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-19T04:26:38.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-19T04:26:38.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-19T04:26:38.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-19T04:26:38.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-19T04:26:38.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-19T04:26:38.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-19T04:26:38.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-19T04:26:38.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-19T04:26:38.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-19T04:26:38.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-19T04:26:38.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-19T04:26:38.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-19T04:26:38.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-19T04:26:38.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-19T04:26:38.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-19T04:26:38.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-19T04:26:38.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-19T04:26:38.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-19T04:26:38.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-19T04:26:38.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-19T04:26:38.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-19T04:26:38.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-19T04:26:38.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-19T04:26:38.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-19T04:26:38.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-19T04:26:38.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-19T04:26:38.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-19T04:26:38.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-19T04:26:38.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-19T04:26:38.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-19T04:26:38.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-19T04:26:38.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-19T04:26:38.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-19T04:26:38.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-19T04:26:38.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-19T04:26:38.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-19T04:26:38.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-19T04:26:38.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-19T04:26:38.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-19T04:26:38.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-19T04:26:38.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-19T04:26:38.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-19T04:26:38.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-19T04:26:38.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-19T04:26:38.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-19T04:26:38.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-19T04:26:38.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-19T04:26:38.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-19T04:26:38.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-19T04:26:38.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-19T04:26:38.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-19T04:26:38.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-19T04:26:38.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-19T04:26:38.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-19T04:26:38.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-19T04:26:38.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-19T04:26:38.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-19T04:26:38.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-19T04:26:38.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-19T04:26:38.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-19T04:26:38.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-19T04:26:38.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-19T04:26:38.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-19T04:26:38.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-19T04:26:38.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-19T04:26:38.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-19T04:26:38.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-19T04:26:38.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-19T04:26:38.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-19T04:26:38.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-19T04:26:38.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-19T04:26:38.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-19T04:26:38.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-19T04:26:38.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-19T04:26:38.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-19T04:26:38.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-19T04:26:38.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-19T04:26:38.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-19T04:26:38.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-19T04:26:38.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-19T04:26:38.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-19T04:26:38.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-19T04:26:38.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-19T04:26:38.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-19T04:26:38.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-19T04:26:38.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-19T04:26:38.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-19T04:26:38.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-19T04:26:38.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-19T04:26:38.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-19T04:26:38.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-19T04:26:38.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-19T04:26:38.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-19T04:26:38.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-19T04:26:38.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-19T04:26:38.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-19T04:26:38.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-19T04:26:38.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-19T04:26:38.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-19T04:26:38.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-19T04:26:38.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-19T04:26:38.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-19T04:26:38.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-19T04:26:38.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-19T04:26:38.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-19T04:26:38.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-19T04:26:38.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-19T04:26:38.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-19T04:26:38.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-19T04:26:38.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-19T04:26:38.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-19T04:26:38.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-19T04:26:38.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-19T04:26:38.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-19T04:26:38.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-19T04:26:38.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-19T04:26:38.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-19T04:26:38.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-19T04:26:38.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-19T04:26:38.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-19T04:26:38.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-19T04:26:38.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-19T04:26:38.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-19T04:26:38.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-19T04:26:38.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-19T04:26:38.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-19T04:26:38.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-19T04:26:38.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-19T04:26:38.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-19T04:26:38.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-19T04:26:38.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-19T04:26:38.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-19T04:26:38.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-19T04:26:38.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-19T04:26:38.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-19T04:26:38.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-19T04:26:38.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-19T04:26:38.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-19T04:26:38.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-19T04:26:38.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-19T04:26:38.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-19T04:26:38.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-19T04:26:38.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-19T04:26:38.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-19T04:26:38.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-19T04:26:38.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-19T04:26:38.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-19T04:26:38.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-19T04:26:38.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-19T04:26:38.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-19T04:26:38.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-19T04:26:38.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-19T04:26:38.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-19T04:26:38.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-19T04:26:38.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-19T04:26:38.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-19T04:26:38.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-19T04:26:38.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-19T04:26:38.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-19T04:26:38.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-19T04:26:38.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-19T04:26:38.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-19T04:26:38.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-19T04:26:38.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-19T04:26:38.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-19T04:26:38.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-19T04:26:38.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-19T04:26:38.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-19T04:26:38.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-19T04:26:38.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-19T04:26:38.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-19T04:26:38.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-19T04:26:38.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-19T04:26:38.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-19T04:26:38.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-19T04:26:38.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-19T04:26:38.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-19T04:26:38.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-19T04:26:38.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-19T04:26:38.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-19T04:26:38.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-19T04:26:38.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-19T04:26:38.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-19T04:26:38.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-19T04:26:38.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-19T04:26:38.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-19T04:26:38.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-19T04:26:38.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-19T04:26:38.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-19T04:26:38.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-19T04:26:38.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-19T04:26:38.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-19T04:26:38.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-19T04:26:38.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-19T04:26:38.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-19T04:26:38.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-19T04:26:38.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-19T04:26:38.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-19T04:26:38.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-19T04:26:38.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-19T04:26:38.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-19T04:26:38.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-19T04:26:38.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-19T04:26:38.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-19T04:26:38.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-19T04:26:38.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-19T04:26:38.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-19T04:26:38.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-19T04:26:38.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-19T04:26:38.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-19T04:26:38.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-19T04:26:38.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-19T04:26:38.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-19T04:26:38.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-19T04:26:38.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-19T04:26:38.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-19T04:26:38.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-19T04:26:38.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-19T04:26:38.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-19T04:26:38.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-19T04:26:38.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-19T04:26:38.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-19T04:26:38.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-19T04:26:38.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-19T04:26:38.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-19T04:26:38.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-19T04:26:38.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-19T04:26:38.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-19T04:26:38.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-19T04:26:38.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-19T04:26:38.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-19T04:26:38.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-19T04:26:38.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-19T04:26:38.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-19T04:26:38.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-19T04:26:38.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-19T04:26:38.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-19T04:26:38.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-19T04:26:38.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-19T04:26:38.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-19T04:26:38.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-19T04:26:38.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-19T04:26:38.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-19T04:26:38.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-19T04:26:38.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-19T04:26:38.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-19T04:26:38.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-19T04:26:38.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-19T04:26:38.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-19T04:26:38.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-19T04:26:38.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-19T04:26:38.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-19T04:26:38.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-19T04:26:38.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-19T04:26:38.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-19T04:26:38.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-19T04:26:38.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-19T04:26:38.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-19T04:26:38.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-19T04:26:38.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-19T04:26:38.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-19T04:26:38.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-19T04:26:38.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-19T04:26:38.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-19T04:26:38.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-19T04:26:38.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-19T04:26:38.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-19T04:26:38.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-19T04:26:38.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-19T04:26:38.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-19T04:26:38.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-19T04:26:38.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-19T04:26:38.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-19T04:26:38.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-19T04:26:38.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-19T04:26:38.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-19T04:26:38.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-19T04:26:38.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-19T04:26:38.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-19T04:26:38.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-19T04:26:38.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-19T04:26:38.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-19T04:26:38.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-19T04:26:38.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-19T04:26:38.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-19T04:26:38.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-19T04:26:38.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-19T04:26:38.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-19T04:26:38.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-19T04:26:38.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-19T04:26:38.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-19T04:26:38.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-19T04:26:38.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-19T04:26:38.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-19T04:26:38.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-19T04:26:38.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-19T04:26:38.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-19T04:26:38.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-19T04:26:38.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-19T04:26:38.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-19T04:26:38.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-19T04:26:38.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-19T04:26:38.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-19T04:26:38.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-19T04:26:38.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-19T04:26:38.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-19T04:26:38.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-19T04:26:38.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-19T04:26:38.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-19T04:26:38.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-19T04:26:38.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-19T04:26:38.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-19T04:26:38.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-19T04:26:38.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-19T04:26:38.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-19T04:26:38.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-19T04:26:38.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-19T04:26:38.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-19T04:26:38.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-19T04:26:38.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-19T04:26:38.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-19T04:26:38.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-19T04:26:38.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-19T04:26:38.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-19T04:26:38.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-19T04:26:38.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-19T04:26:38.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-19T04:26:38.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-19T04:26:38.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-19T04:26:38.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-19T04:26:38.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-19T04:26:38.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-19T04:26:38.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-19T04:26:38.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-19T04:26:38.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-19T04:26:38.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-19T04:26:38.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-19T04:26:38.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-19T04:26:38.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-19T04:26:38.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-19T04:26:38.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-19T04:26:38.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-19T04:26:38.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-19T04:26:38.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-19T04:26:38.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-19T04:26:38.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-19T04:26:38.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-19T04:26:38.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-19T04:26:38.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-19T04:26:38.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-19T04:26:38.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-19T04:26:38.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-19T04:26:38.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-19T04:26:38.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-19T04:26:38.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-19T04:26:38.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-19T04:26:38.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-19T04:26:38.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-19T04:26:38.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-19T04:26:38.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-19T04:26:38.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-19T04:26:38.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-19T04:26:38.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-19T04:26:38.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-19T04:26:38.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-19T04:26:38.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-19T04:26:38.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-19T04:26:38.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-19T04:26:38.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-19T04:26:38.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-19T04:26:38.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-19T04:26:38.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-19T04:26:38.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-19T04:26:38.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-19T04:26:38.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-19T04:26:38.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-19T04:26:38.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-19T04:26:38.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-19T04:26:38.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-19T04:26:38.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-19T04:26:38.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-19T04:26:38.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-19T04:26:38.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-19T04:26:38.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-19T04:26:38.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-19T04:26:38.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-19T04:26:38.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-19T04:26:38.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-19T04:26:38.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-19T04:26:38.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-19T04:26:38.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-19T04:26:38.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-19T04:26:38.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-19T04:26:38.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-19T04:26:38.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-19T04:26:38.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-19T04:26:38.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-19T04:26:38.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-19T04:26:38.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-19T04:26:38.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-19T04:26:38.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-19T04:26:38.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-19T04:26:38.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-19T04:26:38.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-19T04:26:38.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-19T04:26:38.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-19T04:26:38.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-19T04:26:38.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-19T04:26:38.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-19T04:26:38.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-19T04:26:38.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-19T04:26:38.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-19T04:26:38.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-19T04:26:38.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-19T04:26:38.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-19T04:26:38.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-19T04:26:38.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-19T04:26:38.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-19T04:26:38.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-19T04:26:38.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-19T04:26:38.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-19T04:26:38.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-19T04:26:38.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-19T04:26:38.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-19T04:26:38.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-19T04:26:38.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-19T04:26:38.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-19T04:26:38.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-19T04:26:38.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-19T04:26:38.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-19T04:26:38.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-19T04:26:38.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-19T04:26:38.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-19T04:26:38.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-19T04:26:38.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-19T04:26:38.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-19T04:26:38.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-19T04:26:38.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-19T04:26:38.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-19T04:26:38.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-19T04:26:38.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-19T04:26:38.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-19T04:26:38.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-19T04:26:38.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-19T04:26:38.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-19T04:26:38.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-19T04:26:38.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-19T04:26:38.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-19T04:26:38.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-19T04:26:38.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-19T04:26:38.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-19T04:26:38.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-19T04:26:38.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-19T04:26:38.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-19T04:26:38.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-19T04:26:38.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-19T04:26:38.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-19T04:26:38.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-19T04:26:38.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-19T04:26:38.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-19T04:26:38.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-19T04:26:38.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-19T04:26:38.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-19T04:26:38.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-19T04:26:38.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-19T04:26:38.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-19T04:26:38.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-19T04:26:38.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-19T04:26:38.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-19T04:26:38.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-19T04:26:38.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-19T04:26:38.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-19T04:26:38.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-19T04:26:38.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-19T04:26:38.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-19T04:26:38.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-19T04:26:38.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-19T04:26:38.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-19T04:26:38.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-19T04:26:38.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-19T04:26:38.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-19T04:26:38.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-19T04:26:38.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-19T04:26:38.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-19T04:26:38.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-19T04:26:38.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-19T04:26:38.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-19T04:26:38.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-19T04:26:38.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-19T04:26:38.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-19T04:26:38.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-19T04:26:38.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-19T04:26:38.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-19T04:26:38.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-19T04:26:38.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-19T04:26:38.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-19T04:26:38.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-19T04:26:38.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-19T04:26:38.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-19T04:26:38.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-19T04:26:38.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-19T04:26:38.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-19T04:26:38.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-19T04:26:38.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-19T04:26:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-19T04:26:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-19T04:26:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-19T04:26:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-19T04:26:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-19T04:26:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-19T04:26:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-19T04:26:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-19T04:26:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-19T04:26:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-19T04:26:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-19T04:26:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-19T04:26:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-19T04:26:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-19T04:26:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-19T04:26:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-19T04:26:38.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-19T04:26:38.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-19T04:26:38.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-19T04:26:38.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-19T04:26:38.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-19T04:26:38.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-19T04:26:38.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-19T04:26:38.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-19T04:26:38.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-19T04:26:38.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-19T04:26:38.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-19T04:26:38.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-19T04:26:38.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-19T04:26:38.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-19T04:26:38.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-19T04:26:38.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-19T04:26:38.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-19T04:26:38.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-19T04:26:38.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-19T04:26:38.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-19T04:26:38.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-19T04:26:38.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-19T04:26:38.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-19T04:26:38.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-19T04:26:38.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-19T04:26:38.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-19T04:26:38.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-19T04:26:38.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-19T04:26:38.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-19T04:26:38.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-19T04:26:38.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-19T04:26:38.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-19T04:26:38.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-19T04:26:38.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-19T04:26:38.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-19T04:26:38.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-19T04:26:38.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-19T04:26:38.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-19T04:26:38.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-19T04:26:38.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-19T04:26:38.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-19T04:26:38.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-19T04:26:38.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-19T04:26:38.563 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-19T04:26:38.563 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-19T04:26:38.563 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-19T04:26:38.563 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-19T04:26:38.563 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-19T04:26:38.563 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-19T04:26:38.563 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-19T04:26:38.563 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-19T04:26:38.563 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-19T04:26:38.563 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-19T04:26:38.563 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-19T04:26:38.563 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-19T04:26:38.563 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-19T04:26:38.564 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-19T04:26:38.564 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-19T04:26:38.564 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-19T04:26:38.564 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-19T04:26:38.564 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-19T04:26:38.564 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-19T04:26:38.564 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-19T04:26:38.564 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-19T04:26:38.564 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-19T04:26:38.564 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-19T04:26:38.564 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-19T04:26:38.564 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-19T04:26:38.564 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-19T04:26:38.564 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-19T04:26:38.565 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-19T04:26:38.565 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-19T04:26:38.565 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-19T04:26:38.565 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-19T04:26:38.565 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-19T04:26:38.565 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-19T04:26:38.565 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-19T04:26:38.565 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-19T04:26:38.565 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-19T04:26:38.565 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-19T04:26:38.565 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-19T04:26:38.565 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-19T04:26:38.565 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-19T04:26:38.566 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-19T04:26:38.566 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-19T04:26:38.566 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-19T04:26:38.566 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-19T04:26:38.566 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-19T04:26:38.566 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-19T04:26:38.566 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-19T04:26:38.566 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-19T04:26:38.566 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-19T04:26:38.566 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-19T04:26:38.566 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-19T04:26:38.566 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-19T04:26:38.566 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-19T04:26:38.566 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-19T04:26:38.567 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-19T04:26:38.567 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-19T04:26:38.567 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-19T04:26:38.567 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-19T04:26:38.567 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-19T04:26:38.567 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-19T04:26:38.567 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-19T04:26:38.567 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-19T04:26:38.567 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-19T04:26:38.567 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-19T04:26:38.567 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-19T04:26:38.567 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-19T04:26:38.567 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-19T04:26:38.567 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-19T04:26:38.568 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-19T04:26:38.568 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-19T04:26:38.568 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-19T04:26:38.568 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-19T04:26:38.568 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-19T04:26:38.568 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-19T04:26:38.568 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-19T04:26:38.568 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-19T04:26:38.568 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-19T04:26:38.568 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-19T04:26:38.568 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-19T04:26:38.568 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-19T04:26:38.568 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-19T04:26:38.568 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-19T04:26:38.569 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-19T04:26:38.569 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-19T04:26:38.569 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-19T04:26:38.569 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-19T04:26:38.569 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-19T04:26:38.569 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-19T04:26:38.569 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-19T04:26:38.569 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-19T04:26:38.569 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-19T04:26:38.569 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-19T04:26:38.569 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-19T04:26:38.569 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-19T04:26:38.569 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-19T04:26:38.569 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-19T04:26:38.570 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-19T04:26:38.570 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-19T04:26:38.570 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-19T04:26:38.570 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-19T04:26:38.570 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-19T04:26:38.570 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-19T04:26:38.570 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-19T04:26:38.570 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-19T04:26:38.570 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-19T04:26:38.574 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:26:38.574 DEBUG:teuthology.orchestra.run.smithi123:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-19T04:26:38.598 INFO:teuthology.orchestra.run.smithi123.stdout:postinstall scriptlet (using /bin/sh): 2024-09-19T04:26:38.598 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-19T04:26:38.598 INFO:teuthology.orchestra.run.smithi123.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-19T04:26:38.598 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-09-19T04:26:38.598 INFO:teuthology.orchestra.run.smithi123.stdout:for file in vmlinuz System.map config; do 2024-09-19T04:26:38.598 INFO:teuthology.orchestra.run.smithi123.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-19T04:26:38.598 INFO:teuthology.orchestra.run.smithi123.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-19T04:26:38.598 INFO:teuthology.orchestra.run.smithi123.stdout: fi 2024-09-19T04:26:38.598 INFO:teuthology.orchestra.run.smithi123.stdout:done 2024-09-19T04:26:38.598 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-19T04:26:38.598 INFO:teuthology.orchestra.run.smithi123.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-19T04:26:38.599 INFO:teuthology.orchestra.run.smithi123.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-19T04:26:38.599 INFO:teuthology.orchestra.run.smithi123.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-19T04:26:38.599 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-09-19T04:26:38.599 INFO:teuthology.orchestra.run.smithi123.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-19T04:26:38.599 INFO:teuthology.orchestra.run.smithi123.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:26:38.599 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-09-19T04:26:38.599 INFO:teuthology.orchestra.run.smithi123.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-19T04:26:38.599 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-19T04:26:38.599 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:26:38.599 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-09-19T04:26:38.600 DEBUG:teuthology.orchestra.run.smithi123:> sudo rpm -qi grub2-tools 2024-09-19T04:26:38.683 INFO:teuthology.orchestra.run.smithi123.stdout:Name : grub2-tools 2024-09-19T04:26:38.684 INFO:teuthology.orchestra.run.smithi123.stdout:Epoch : 1 2024-09-19T04:26:38.684 INFO:teuthology.orchestra.run.smithi123.stdout:Version : 2.06 2024-09-19T04:26:38.684 INFO:teuthology.orchestra.run.smithi123.stdout:Release : 82.el9 2024-09-19T04:26:38.684 INFO:teuthology.orchestra.run.smithi123.stdout:Architecture: x86_64 2024-09-19T04:26:38.684 INFO:teuthology.orchestra.run.smithi123.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T04:26:38.684 INFO:teuthology.orchestra.run.smithi123.stdout:Group : Unspecified 2024-09-19T04:26:38.684 INFO:teuthology.orchestra.run.smithi123.stdout:Size : 8274670 2024-09-19T04:26:38.684 INFO:teuthology.orchestra.run.smithi123.stdout:License : GPLv3+ 2024-09-19T04:26:38.684 INFO:teuthology.orchestra.run.smithi123.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T04:26:38.684 INFO:teuthology.orchestra.run.smithi123.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T04:26:38.684 INFO:teuthology.orchestra.run.smithi123.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T04:26:38.684 INFO:teuthology.orchestra.run.smithi123.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T04:26:38.684 INFO:teuthology.orchestra.run.smithi123.stdout:Packager : builder@centos.org 2024-09-19T04:26:38.685 INFO:teuthology.orchestra.run.smithi123.stdout:Vendor : CentOS 2024-09-19T04:26:38.685 INFO:teuthology.orchestra.run.smithi123.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T04:26:38.685 INFO:teuthology.orchestra.run.smithi123.stdout:Summary : Support tools for GRUB. 2024-09-19T04:26:38.685 INFO:teuthology.orchestra.run.smithi123.stdout:Description : 2024-09-19T04:26:38.685 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:26:38.685 INFO:teuthology.orchestra.run.smithi123.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T04:26:38.685 INFO:teuthology.orchestra.run.smithi123.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T04:26:38.685 INFO:teuthology.orchestra.run.smithi123.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T04:26:38.685 INFO:teuthology.orchestra.run.smithi123.stdout:hardware devices. 2024-09-19T04:26:38.685 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:26:38.685 INFO:teuthology.orchestra.run.smithi123.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T04:26:38.686 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T04:26:38.686 INFO:teuthology.task.kernel:Updating grub on smithi123 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:26:38.686 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T04:26:39.254 INFO:teuthology.orchestra.run.smithi123.stderr:Generating grub configuration file ... 2024-09-19T04:26:40.531 INFO:teuthology.orchestra.run.smithi123.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T04:26:40.567 INFO:teuthology.orchestra.run.smithi123.stderr:done 2024-09-19T04:26:40.569 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-09-19T04:26:40.584 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.UwdD5HhekV 2024-09-19T04:26:40.585 DEBUG:teuthology.orchestra.run.smithi123:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UwdD5HhekV 2024-09-19T04:26:40.655 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.UwdD5HhekV 2024-09-19T04:26:40.797 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.UwdD5HhekV is 6KB 2024-09-19T04:26:40.846 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.UwdD5HhekV 2024-09-19T04:26:40.861 DEBUG:teuthology.orchestra.run.smithi123:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T04:26:40.923 INFO:teuthology.orchestra.run.smithi123.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T04:26:40.923 INFO:teuthology.orchestra.run.smithi123.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T04:26:40.923 INFO:teuthology.orchestra.run.smithi123.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T04:26:40.923 INFO:teuthology.orchestra.run.smithi123.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-19T04:26:40.925 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:26:41.071 DEBUG:teuthology.orchestra.run.smithi123:> sudo shutdown -r now 2024-09-19T04:27:11.074 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-19T04:27:11.075 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-19T04:27:11.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:27:29.469 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-09-19T04:27:38.470 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-19T04:27:38.471 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:27:41.533 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-09-19T04:27:53.534 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-19T04:27:53.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:28:25.655 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-09-19T04:28:26.171 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-19T04:28:26.171 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-19T04:28:26.171 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-09-19T04:28:26.205 INFO:teuthology.orchestra.run.smithi123.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:28:26.206 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:28:26.206 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T04:28:26.206 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T04:28:27.206 DEBUG:teuthology.parallel:result is None 2024-09-19T04:28:27.207 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T04:28:27.214 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T04:28:27.215 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T04:28:27.219 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T04:28:27.222 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T04:28:27.250 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T04:28:27.272 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T04:28:27.360 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T04:28:27.361 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T04:28:27.365 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T04:28:27.367 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T04:28:27.415 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T04:28:27.427 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T04:28:27.427 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T04:28:27.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:28:27.447 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T04:28:27.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:28:27.464 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T04:28:27.468 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T04:28:27.489 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T04:28:27.522 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:28:27.537 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:28:27.570 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:28:27.587 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:28:27.621 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:28:27.654 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:28:27.657 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T04:28:27.664 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T04:28:27.664 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T04:28:27.668 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T04:28:27.670 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T04:28:27.719 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T04:28:27.728 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T04:28:27.728 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T04:28:27.732 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T04:28:27.738 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T04:28:27.765 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:28:27.857 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:28:27.942 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:28:28.005 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:28:28.074 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:28:28.074 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T04:28:28.131 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:28:28.172 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:28:28.248 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:28:28.290 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:28:28.334 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-19T04:28:28.334 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T04:28:28.400 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:28:28.447 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:28:28.505 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:28:28.589 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:28:28.633 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-19T04:28:28.633 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T04:28:28.700 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-09-19T04:28:28.704 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-09-19T04:28:28.706 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2024-09-19T04:28:28.741 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:28:28.777 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:28:28.797 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:28:29.289 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T04:28:29.297 INFO:teuthology.task.internal:Starting timer... 2024-09-19T04:28:29.297 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T04:28:29.352 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T04:28:29.380 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T04:28:29.380 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-09-19T04:28:29.490 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-09-19T04:28:29.493 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2024-09-19T04:28:29.564 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2024-09-19T04:28:29.566 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2024-09-19T04:28:29.625 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2024-09-19T04:28:29.627 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T04:28:29.628 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-09-19T04:28:29.668 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-09-19T04:28:29.668 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2024-09-19T04:28:29.696 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2024-09-19T04:28:29.696 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2024-09-19T04:28:29.718 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2024-09-19T04:28:29.719 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi123.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2024-09-19T04:28:29.719 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:28:29.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:28:29.778 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:28:29.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:28:29.810 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:28:29.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:28:29.839 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T04:28:29.839 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-09-19T04:28:29.906 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2024-09-19T04:28:29.944 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2024-09-19T04:28:29.978 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T04:28:29.987 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-19T04:28:29.991 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-19T04:28:30.107 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T04:28:30.117 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T04:28:30.118 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi123.front.sepia.ceph.com,smithi183.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T04:33:02.058 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi123.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2024-09-19T04:33:02.059 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-19T04:33:02.060 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:33:02.150 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-09-19T04:33:02.259 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-19T04:33:02.259 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-19T04:33:02.260 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:33:02.335 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-09-19T04:33:02.417 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-19T04:33:02.418 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-19T04:33:02.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:33:02.495 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-09-19T04:33:02.581 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-19T04:33:02.581 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T04:33:02.596 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T04:33:02.597 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T04:33:02.597 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:33:02.601 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T04:33:02.601 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:33:02.604 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T04:33:02.604 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:33:02.633 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T04:33:02.647 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T04:33:02.652 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T04:33:02.659 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T04:33:02.666 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T04:33:02.672 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2024-09-19T04:33:02.683 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2024-09-19T04:33:02.688 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T04:33:02.691 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2024-09-19T04:33:02.697 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T04:33:02.702 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2024-09-19T04:33:02.710 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T04:33:02.716 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T04:33:02.730 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T04:33:02.738 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-09-19T04:33:02.763 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-09-19T04:33:02.764 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:33:02.768 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:33:02.768 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-09-19T04:33:02.768 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:33:02.768 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:33:02.768 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:33:02.768 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:33:02.785 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:33:02.788 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:33:02.788 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-09-19T04:33:02.788 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:33:02.788 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:33:02.788 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:33:02.789 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:33:02.791 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T04:33:02.820 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T04:33:02.925 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:33:02.932 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:33:02.932 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-09-19T04:33:02.932 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:33:02.932 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:33:02.932 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:33:02.932 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:33:02.933 INFO:teuthology.run_tasks:Running task install... 2024-09-19T04:33:02.943 DEBUG:teuthology.task.install:project ceph 2024-09-19T04:33:02.943 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T04:33:02.944 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T04:33:02.944 INFO:teuthology.task.install:Using flavor: default 2024-09-19T04:33:02.949 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T04:33:02.949 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T04:33:02.949 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T04:33:02.950 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:33:02.952 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T04:33:02.952 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:33:02.953 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T04:33:02.953 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:33:03.101 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T04:33:03.101 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T04:33:03.105 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T04:33:03.105 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T04:33:03.113 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T04:33:03.113 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T04:33:03.211 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T04:33:03.212 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:33:03.212 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T04:33:03.224 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T04:33:03.224 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-19T04:33:03.224 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T04:33:03.226 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T04:33:03.227 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-19T04:33:03.227 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T04:33:03.252 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T04:33:03.256 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T04:33:03.257 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T04:33:03.257 DEBUG:teuthology.orchestra.run.smithi193:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T04:33:03.268 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T04:33:03.268 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T04:33:03.319 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T04:33:03.323 DEBUG:teuthology.orchestra.run.smithi193:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T04:33:03.365 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T04:33:03.393 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T04:33:03.398 DEBUG:teuthology.orchestra.run.smithi193:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T04:33:03.444 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T04:33:03.458 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2024-09-19T04:33:03.459 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2024-09-19T04:33:03.462 INFO:teuthology.orchestra.run.smithi193.stdout:check_obsoletes = 1 2024-09-19T04:33:03.464 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum clean all 2024-09-19T04:33:03.498 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-09-19T04:33:03.501 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-09-19T04:33:03.685 INFO:teuthology.orchestra.run.smithi183.stdout:45 files removed 2024-09-19T04:33:03.689 INFO:teuthology.orchestra.run.smithi193.stdout:45 files removed 2024-09-19T04:33:03.712 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T04:33:03.716 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T04:33:03.777 INFO:teuthology.orchestra.run.smithi123.stdout:45 files removed 2024-09-19T04:33:03.809 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T04:33:04.256 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 315 kB/s | 84 kB 00:00 2024-09-19T04:33:04.269 INFO:teuthology.orchestra.run.smithi193.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-19T04:33:04.493 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 295 kB/s | 84 kB 00:00 2024-09-19T04:33:04.540 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T04:33:04.545 INFO:teuthology.orchestra.run.smithi193.stdout:ceph noarch packages 50 kB/s | 12 kB 00:00 2024-09-19T04:33:04.777 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T04:33:04.816 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-19T04:33:04.829 INFO:teuthology.orchestra.run.smithi193.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-19T04:33:05.061 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-19T04:33:05.290 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.3 MB 00:00 2024-09-19T04:33:05.346 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.3 MB 00:00 2024-09-19T04:33:07.050 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-19T04:33:07.638 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 17 MB/s | 20 MB 00:01 2024-09-19T04:33:08.794 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.3 MB 00:03 2024-09-19T04:33:10.837 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-19T04:33:11.973 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-19T04:33:13.317 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-19T04:33:13.540 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 19 kB 00:00 2024-09-19T04:33:14.370 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-19T04:33:14.701 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 117 kB/s | 19 kB 00:00 2024-09-19T04:33:15.530 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-19T04:33:16.707 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.5 MB 00:01 2024-09-19T04:33:18.288 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 19 kB 00:00 2024-09-19T04:33:19.259 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-19T04:33:19.817 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-19T04:33:20.994 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-09-19T04:33:21.927 INFO:teuthology.orchestra.run.smithi193.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:33:21.927 INFO:teuthology.orchestra.run.smithi193.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:33:21.938 INFO:teuthology.orchestra.run.smithi193.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T04:33:21.939 INFO:teuthology.orchestra.run.smithi193.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T04:33:21.940 INFO:teuthology.orchestra.run.smithi193.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T04:33:21.941 INFO:teuthology.orchestra.run.smithi193.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T04:33:21.943 INFO:teuthology.orchestra.run.smithi193.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T04:33:21.945 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T04:33:21.946 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T04:33:22.007 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-19T04:33:22.021 INFO:teuthology.orchestra.run.smithi193.stdout:====================================================================================== 2024-09-19T04:33:22.021 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-09-19T04:33:22.021 INFO:teuthology.orchestra.run.smithi193.stdout:====================================================================================== 2024-09-19T04:33:22.021 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-19T04:33:22.021 INFO:teuthology.orchestra.run.smithi193.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T04:33:22.022 INFO:teuthology.orchestra.run.smithi193.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T04:33:22.022 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T04:33:22.022 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T04:33:22.022 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T04:33:22.022 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T04:33:22.022 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T04:33:22.022 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T04:33:22.022 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T04:33:22.022 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T04:33:22.022 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T04:33:22.022 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T04:33:22.022 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T04:33:22.023 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T04:33:22.023 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T04:33:22.023 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T04:33:22.023 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T04:33:22.023 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T04:33:22.023 INFO:teuthology.orchestra.run.smithi193.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T04:33:22.023 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T04:33:22.023 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T04:33:22.023 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T04:33:22.023 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T04:33:22.024 INFO:teuthology.orchestra.run.smithi193.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T04:33:22.024 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T04:33:22.024 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T04:33:22.024 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T04:33:22.024 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T04:33:22.024 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T04:33:22.024 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T04:33:22.024 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T04:33:22.024 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T04:33:22.024 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T04:33:22.024 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T04:33:22.024 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T04:33:22.025 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T04:33:22.025 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T04:33:22.025 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrading: 2024-09-19T04:33:22.025 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T04:33:22.025 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T04:33:22.025 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T04:33:22.025 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T04:33:22.025 INFO:teuthology.orchestra.run.smithi193.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T04:33:22.025 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T04:33:22.025 INFO:teuthology.orchestra.run.smithi193.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T04:33:22.025 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T04:33:22.025 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-09-19T04:33:22.025 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T04:33:22.026 INFO:teuthology.orchestra.run.smithi193.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T04:33:22.026 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T04:33:22.026 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T04:33:22.026 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T04:33:22.026 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T04:33:22.026 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T04:33:22.026 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T04:33:22.026 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T04:33:22.026 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T04:33:22.026 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T04:33:22.026 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T04:33:22.026 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T04:33:22.026 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T04:33:22.026 INFO:teuthology.orchestra.run.smithi193.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T04:33:22.027 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T04:33:22.027 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T04:33:22.027 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T04:33:22.027 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T04:33:22.027 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T04:33:22.027 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T04:33:22.027 INFO:teuthology.orchestra.run.smithi193.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T04:33:22.027 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T04:33:22.027 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T04:33:22.027 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T04:33:22.027 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T04:33:22.027 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T04:33:22.027 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T04:33:22.028 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T04:33:22.028 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T04:33:22.028 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T04:33:22.028 INFO:teuthology.orchestra.run.smithi193.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T04:33:22.028 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T04:33:22.028 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T04:33:22.028 INFO:teuthology.orchestra.run.smithi193.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T04:33:22.028 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T04:33:22.028 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T04:33:22.028 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T04:33:22.028 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T04:33:22.028 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T04:33:22.028 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T04:33:22.028 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T04:33:22.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T04:33:22.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T04:33:22.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T04:33:22.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T04:33:22.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T04:33:22.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T04:33:22.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T04:33:22.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T04:33:22.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T04:33:22.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T04:33:22.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T04:33:22.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T04:33:22.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T04:33:22.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T04:33:22.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T04:33:22.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T04:33:22.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T04:33:22.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T04:33:22.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T04:33:22.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T04:33:22.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T04:33:22.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T04:33:22.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T04:33:22.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T04:33:22.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T04:33:22.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T04:33:22.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T04:33:22.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T04:33:22.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T04:33:22.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T04:33:22.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T04:33:22.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T04:33:22.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T04:33:22.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T04:33:22.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T04:33:22.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T04:33:22.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T04:33:22.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T04:33:22.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T04:33:22.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T04:33:22.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T04:33:22.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T04:33:22.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T04:33:22.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T04:33:22.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T04:33:22.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T04:33:22.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T04:33:22.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T04:33:22.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T04:33:22.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T04:33:22.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T04:33:22.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T04:33:22.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T04:33:22.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T04:33:22.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T04:33:22.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T04:33:22.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T04:33:22.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T04:33:22.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T04:33:22.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T04:33:22.033 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T04:33:22.033 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T04:33:22.033 INFO:teuthology.orchestra.run.smithi193.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T04:33:22.033 INFO:teuthology.orchestra.run.smithi193.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T04:33:22.033 INFO:teuthology.orchestra.run.smithi193.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T04:33:22.033 INFO:teuthology.orchestra.run.smithi193.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T04:33:22.034 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T04:33:22.034 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T04:33:22.034 INFO:teuthology.orchestra.run.smithi193.stdout:Installing weak dependencies: 2024-09-19T04:33:22.034 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T04:33:22.034 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:33:22.034 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-19T04:33:22.034 INFO:teuthology.orchestra.run.smithi193.stdout:====================================================================================== 2024-09-19T04:33:22.034 INFO:teuthology.orchestra.run.smithi193.stdout:Install 148 Packages 2024-09-19T04:33:22.034 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrade 8 Packages 2024-09-19T04:33:22.034 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:33:22.035 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 226 M 2024-09-19T04:33:22.035 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-19T04:33:22.660 INFO:teuthology.orchestra.run.smithi193.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 48 kB/s | 6.6 kB 00:00 2024-09-19T04:33:22.894 INFO:teuthology.orchestra.run.smithi193.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 3.8 MB/s | 896 kB 00:00 2024-09-19T04:33:22.977 INFO:teuthology.orchestra.run.smithi193.stdout:(3/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-19T04:33:23.002 INFO:teuthology.orchestra.run.smithi193.stdout:(4/156): ceph-immutable-object-cache-19.1.1-361 1.3 MB/s | 145 kB 00:00 2024-09-19T04:33:23.115 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:33:23.116 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:33:23.128 INFO:teuthology.orchestra.run.smithi193.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 16 MB/s | 2.4 MB 00:00 2024-09-19T04:33:23.130 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T04:33:23.131 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T04:33:23.132 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T04:33:23.136 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T04:33:23.137 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T04:33:23.138 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T04:33:23.139 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T04:33:23.170 INFO:teuthology.orchestra.run.smithi193.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 9.1 MB/s | 1.5 MB 00:00 2024-09-19T04:33:23.202 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-09-19T04:33:23.217 INFO:teuthology.orchestra.run.smithi183.stdout:====================================================================================== 2024-09-19T04:33:23.217 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-09-19T04:33:23.217 INFO:teuthology.orchestra.run.smithi183.stdout:====================================================================================== 2024-09-19T04:33:23.217 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-09-19T04:33:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T04:33:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T04:33:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T04:33:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T04:33:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T04:33:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T04:33:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T04:33:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T04:33:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T04:33:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T04:33:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T04:33:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T04:33:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T04:33:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T04:33:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T04:33:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T04:33:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T04:33:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T04:33:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T04:33:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T04:33:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T04:33:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T04:33:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T04:33:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T04:33:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T04:33:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T04:33:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T04:33:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T04:33:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T04:33:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T04:33:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T04:33:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T04:33:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T04:33:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T04:33:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T04:33:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T04:33:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T04:33:23.220 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2024-09-19T04:33:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T04:33:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T04:33:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T04:33:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T04:33:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T04:33:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T04:33:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T04:33:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T04:33:23.221 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-09-19T04:33:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T04:33:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T04:33:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T04:33:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T04:33:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T04:33:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T04:33:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T04:33:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T04:33:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T04:33:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T04:33:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T04:33:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T04:33:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T04:33:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T04:33:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T04:33:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T04:33:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T04:33:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T04:33:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T04:33:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T04:33:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T04:33:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T04:33:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T04:33:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T04:33:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T04:33:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T04:33:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T04:33:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T04:33:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T04:33:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T04:33:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T04:33:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T04:33:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T04:33:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T04:33:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T04:33:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T04:33:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T04:33:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T04:33:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T04:33:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T04:33:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T04:33:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T04:33:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T04:33:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T04:33:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T04:33:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T04:33:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T04:33:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T04:33:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T04:33:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T04:33:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T04:33:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T04:33:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T04:33:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T04:33:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T04:33:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T04:33:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T04:33:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T04:33:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T04:33:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T04:33:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T04:33:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T04:33:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T04:33:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T04:33:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T04:33:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T04:33:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T04:33:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T04:33:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T04:33:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T04:33:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T04:33:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T04:33:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T04:33:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T04:33:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T04:33:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T04:33:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T04:33:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T04:33:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T04:33:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T04:33:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T04:33:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T04:33:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T04:33:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T04:33:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T04:33:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T04:33:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T04:33:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T04:33:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T04:33:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T04:33:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T04:33:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T04:33:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T04:33:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T04:33:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T04:33:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T04:33:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T04:33:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T04:33:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T04:33:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T04:33:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T04:33:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T04:33:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T04:33:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T04:33:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T04:33:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T04:33:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T04:33:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T04:33:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T04:33:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T04:33:23.230 INFO:teuthology.orchestra.run.smithi183.stdout:Installing weak dependencies: 2024-09-19T04:33:23.230 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T04:33:23.230 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:33:23.230 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-09-19T04:33:23.230 INFO:teuthology.orchestra.run.smithi183.stdout:====================================================================================== 2024-09-19T04:33:23.230 INFO:teuthology.orchestra.run.smithi183.stdout:Install 148 Packages 2024-09-19T04:33:23.230 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 8 Packages 2024-09-19T04:33:23.230 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:33:23.231 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 226 M 2024-09-19T04:33:23.231 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-09-19T04:33:23.429 INFO:teuthology.orchestra.run.smithi193.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 16 MB/s | 4.7 MB 00:00 2024-09-19T04:33:23.687 INFO:teuthology.orchestra.run.smithi183.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 52 kB/s | 6.6 kB 00:00 2024-09-19T04:33:23.770 INFO:teuthology.orchestra.run.smithi193.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 17 MB/s | 21 MB 00:01 2024-09-19T04:33:23.829 INFO:teuthology.orchestra.run.smithi193.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 432 kB/s | 25 kB 00:00 2024-09-19T04:33:24.212 INFO:teuthology.orchestra.run.smithi183.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 896 kB 00:00 2024-09-19T04:33:24.287 INFO:teuthology.orchestra.run.smithi183.stdout:(3/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 7.5 MB/s | 5.4 MB 00:00 2024-09-19T04:33:24.304 INFO:teuthology.orchestra.run.smithi193.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 15 MB/s | 17 MB 00:01 2024-09-19T04:33:24.313 INFO:teuthology.orchestra.run.smithi183.stdout:(4/156): ceph-immutable-object-cache-19.1.1-361 1.4 MB/s | 145 kB 00:00 2024-09-19T04:33:24.363 INFO:teuthology.orchestra.run.smithi193.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.8 MB/s | 228 kB 00:00 2024-09-19T04:33:24.438 INFO:teuthology.orchestra.run.smithi193.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 361 kB/s | 27 kB 00:00 2024-09-19T04:33:24.472 INFO:teuthology.orchestra.run.smithi183.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 13 MB/s | 2.4 MB 00:00 2024-09-19T04:33:24.497 INFO:teuthology.orchestra.run.smithi193.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 545 kB/s | 32 kB 00:00 2024-09-19T04:33:24.597 INFO:teuthology.orchestra.run.smithi193.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 7.3 MB/s | 751 kB 00:00 2024-09-19T04:33:24.689 INFO:teuthology.orchestra.run.smithi193.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 1.7 MB/s | 163 kB 00:00 2024-09-19T04:33:24.773 INFO:teuthology.orchestra.run.smithi193.stdout:(16/156): librados-devel-19.1.1-361.g2f599a46.e 1.5 MB/s | 127 kB 00:00 2024-09-19T04:33:24.864 INFO:teuthology.orchestra.run.smithi183.stdout:(6/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-19T04:33:24.864 INFO:teuthology.orchestra.run.smithi193.stdout:(17/156): libradosstriper1-19.1.1-361.g2f599a46 5.4 MB/s | 506 kB 00:00 2024-09-19T04:33:24.922 INFO:teuthology.orchestra.run.smithi183.stdout:(7/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 2.5 MB/s | 1.5 MB 00:00 2024-09-19T04:33:25.198 INFO:teuthology.orchestra.run.smithi193.stdout:(18/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 6.0 MB/s | 11 MB 00:01 2024-09-19T04:33:25.282 INFO:teuthology.orchestra.run.smithi193.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 543 kB/s | 45 kB 00:00 2024-09-19T04:33:25.382 INFO:teuthology.orchestra.run.smithi193.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.3 MB/s | 136 kB 00:00 2024-09-19T04:33:25.466 INFO:teuthology.orchestra.run.smithi193.stdout:(21/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 8.9 MB/s | 5.3 MB 00:00 2024-09-19T04:33:25.491 INFO:teuthology.orchestra.run.smithi193.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.5 MB/s | 161 kB 00:00 2024-09-19T04:33:25.541 INFO:teuthology.orchestra.run.smithi193.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 4.2 MB/s | 320 kB 00:00 2024-09-19T04:33:25.575 INFO:teuthology.orchestra.run.smithi193.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 3.5 MB/s | 300 kB 00:00 2024-09-19T04:33:25.581 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-19T04:33:25.617 INFO:teuthology.orchestra.run.smithi193.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.3 MB/s | 99 kB 00:00 2024-09-19T04:33:25.651 INFO:teuthology.orchestra.run.smithi193.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-19T04:33:25.734 INFO:teuthology.orchestra.run.smithi193.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.0 MB/s | 171 kB 00:00 2024-09-19T04:33:25.810 INFO:teuthology.orchestra.run.smithi193.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 424 kB/s | 32 kB 00:00 2024-09-19T04:33:25.864 INFO:teuthology.orchestra.run.smithi183.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 9.2 MB/s | 21 MB 00:02 2024-09-19T04:33:25.872 INFO:teuthology.orchestra.run.smithi193.stdout:(29/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-19T04:33:25.898 INFO:teuthology.orchestra.run.smithi193.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.6 MB/s | 146 kB 00:00 2024-09-19T04:33:26.039 INFO:teuthology.orchestra.run.smithi183.stdout:(9/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86_ 14 MB/s | 17 MB 00:01 2024-09-19T04:33:26.065 INFO:teuthology.orchestra.run.smithi183.stdout:(10/156): ceph-selinux-19.1.1-361.g2f599a46.el9 126 kB/s | 25 kB 00:00 2024-09-19T04:33:26.098 INFO:teuthology.orchestra.run.smithi193.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 16 MB/s | 3.6 MB 00:00 2024-09-19T04:33:26.157 INFO:teuthology.orchestra.run.smithi183.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 2.4 MB/s | 228 kB 00:00 2024-09-19T04:33:26.165 INFO:teuthology.orchestra.run.smithi193.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 3.7 MB/s | 249 kB 00:00 2024-09-19T04:33:26.207 INFO:teuthology.orchestra.run.smithi183.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 543 kB/s | 27 kB 00:00 2024-09-19T04:33:26.241 INFO:teuthology.orchestra.run.smithi193.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 658 kB/s | 49 kB 00:00 2024-09-19T04:33:26.274 INFO:teuthology.orchestra.run.smithi183.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 476 kB/s | 32 kB 00:00 2024-09-19T04:33:26.316 INFO:teuthology.orchestra.run.smithi193.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 219 kB/s | 16 kB 00:00 2024-09-19T04:33:26.400 INFO:teuthology.orchestra.run.smithi193.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 3.3 MB/s | 285 kB 00:00 2024-09-19T04:33:26.432 INFO:teuthology.orchestra.run.smithi183.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 4.6 MB/s | 751 kB 00:00 2024-09-19T04:33:26.483 INFO:teuthology.orchestra.run.smithi193.stdout:(36/156): cephadm-19.1.1-361.g2f599a46.el9.noar 9.0 MB/s | 765 kB 00:00 2024-09-19T04:33:26.525 INFO:teuthology.orchestra.run.smithi183.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 1.7 MB/s | 163 kB 00:00 2024-09-19T04:33:26.583 INFO:teuthology.orchestra.run.smithi183.stdout:(16/156): librados-devel-19.1.1-361.g2f599a46.e 2.1 MB/s | 127 kB 00:00 2024-09-19T04:33:26.650 INFO:teuthology.orchestra.run.smithi183.stdout:(17/156): libradosstriper1-19.1.1-361.g2f599a46 7.4 MB/s | 506 kB 00:00 2024-09-19T04:33:26.934 INFO:teuthology.orchestra.run.smithi193.stdout:(37/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 16 MB/s | 49 MB 00:03 2024-09-19T04:33:26.959 INFO:teuthology.orchestra.run.smithi193.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 38 kB/s | 18 kB 00:00 2024-09-19T04:33:27.008 INFO:teuthology.orchestra.run.smithi183.stdout:(18/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-19T04:33:27.051 INFO:teuthology.orchestra.run.smithi193.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 6.4 MB/s | 7.4 MB 00:01 2024-09-19T04:33:27.067 INFO:teuthology.orchestra.run.smithi183.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 777 kB/s | 45 kB 00:00 2024-09-19T04:33:27.126 INFO:teuthology.orchestra.run.smithi183.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.3 MB/s | 136 kB 00:00 2024-09-19T04:33:27.184 INFO:teuthology.orchestra.run.smithi183.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.7 MB/s | 161 kB 00:00 2024-09-19T04:33:27.251 INFO:teuthology.orchestra.run.smithi183.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 4.7 MB/s | 320 kB 00:00 2024-09-19T04:33:27.310 INFO:teuthology.orchestra.run.smithi183.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.1 MB/s | 300 kB 00:00 2024-09-19T04:33:27.393 INFO:teuthology.orchestra.run.smithi183.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.2 MB/s | 99 kB 00:00 2024-09-19T04:33:27.452 INFO:teuthology.orchestra.run.smithi183.stdout:(25/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-19T04:33:27.659 INFO:teuthology.orchestra.run.smithi193.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 110 kB/s | 80 kB 00:00 2024-09-19T04:33:27.686 INFO:teuthology.orchestra.run.smithi183.stdout:(26/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 13 MB/s | 3.1 MB 00:00 2024-09-19T04:33:27.843 INFO:teuthology.orchestra.run.smithi193.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 241 kB/s | 191 kB 00:00 2024-09-19T04:33:27.918 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:33:27.919 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:33:27.930 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T04:33:27.931 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T04:33:27.931 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T04:33:27.932 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T04:33:27.934 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T04:33:27.936 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T04:33:27.937 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T04:33:28.003 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-19T04:33:28.026 INFO:teuthology.orchestra.run.smithi123.stdout:====================================================================================== 2024-09-19T04:33:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-19T04:33:28.026 INFO:teuthology.orchestra.run.smithi123.stdout:====================================================================================== 2024-09-19T04:33:28.026 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-19T04:33:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T04:33:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T04:33:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T04:33:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T04:33:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T04:33:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T04:33:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T04:33:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T04:33:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T04:33:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T04:33:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T04:33:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T04:33:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T04:33:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T04:33:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T04:33:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T04:33:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T04:33:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T04:33:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T04:33:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T04:33:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T04:33:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T04:33:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T04:33:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T04:33:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T04:33:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T04:33:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T04:33:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T04:33:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T04:33:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T04:33:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T04:33:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T04:33:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T04:33:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T04:33:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T04:33:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T04:33:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T04:33:28.029 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-09-19T04:33:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T04:33:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T04:33:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T04:33:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T04:33:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T04:33:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T04:33:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T04:33:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T04:33:28.030 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-19T04:33:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T04:33:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T04:33:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T04:33:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T04:33:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T04:33:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T04:33:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T04:33:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T04:33:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T04:33:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T04:33:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T04:33:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T04:33:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T04:33:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T04:33:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T04:33:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T04:33:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T04:33:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T04:33:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T04:33:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T04:33:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T04:33:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T04:33:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T04:33:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T04:33:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T04:33:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T04:33:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T04:33:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T04:33:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T04:33:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T04:33:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T04:33:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T04:33:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T04:33:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T04:33:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T04:33:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T04:33:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T04:33:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T04:33:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T04:33:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T04:33:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T04:33:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T04:33:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T04:33:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T04:33:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T04:33:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T04:33:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T04:33:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T04:33:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T04:33:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T04:33:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T04:33:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T04:33:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T04:33:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T04:33:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T04:33:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T04:33:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T04:33:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T04:33:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T04:33:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T04:33:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T04:33:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T04:33:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T04:33:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T04:33:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T04:33:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T04:33:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T04:33:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T04:33:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T04:33:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T04:33:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T04:33:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T04:33:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T04:33:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T04:33:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T04:33:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T04:33:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T04:33:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T04:33:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T04:33:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T04:33:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T04:33:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T04:33:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T04:33:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T04:33:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T04:33:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T04:33:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T04:33:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T04:33:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T04:33:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T04:33:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T04:33:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T04:33:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T04:33:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T04:33:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T04:33:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T04:33:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T04:33:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T04:33:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T04:33:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T04:33:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T04:33:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T04:33:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T04:33:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T04:33:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T04:33:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T04:33:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T04:33:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T04:33:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T04:33:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T04:33:28.038 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-09-19T04:33:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T04:33:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:33:28.038 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-19T04:33:28.038 INFO:teuthology.orchestra.run.smithi123.stdout:====================================================================================== 2024-09-19T04:33:28.038 INFO:teuthology.orchestra.run.smithi123.stdout:Install 148 Packages 2024-09-19T04:33:28.038 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 8 Packages 2024-09-19T04:33:28.039 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:33:28.039 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 226 M 2024-09-19T04:33:28.039 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-19T04:33:28.111 INFO:teuthology.orchestra.run.smithi183.stdout:(27/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 24 MB/s | 49 MB 00:02 2024-09-19T04:33:28.136 INFO:teuthology.orchestra.run.smithi183.stdout:(28/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 380 kB/s | 171 kB 00:00 2024-09-19T04:33:28.161 INFO:teuthology.orchestra.run.smithi183.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 639 kB/s | 32 kB 00:00 2024-09-19T04:33:28.186 INFO:teuthology.orchestra.run.smithi183.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.9 MB/s | 146 kB 00:00 2024-09-19T04:33:28.304 INFO:teuthology.orchestra.run.smithi183.stdout:(31/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 3.1 MB/s | 11 MB 00:03 2024-09-19T04:33:28.387 INFO:teuthology.orchestra.run.smithi183.stdout:(32/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 16 MB/s | 3.6 MB 00:00 2024-09-19T04:33:28.429 INFO:teuthology.orchestra.run.smithi183.stdout:(33/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.0 MB/s | 249 kB 00:00 2024-09-19T04:33:28.454 INFO:teuthology.orchestra.run.smithi183.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 739 kB/s | 49 kB 00:00 2024-09-19T04:33:28.476 INFO:teuthology.orchestra.run.smithi193.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 528 kB/s | 801 kB 00:01 2024-09-19T04:33:28.480 INFO:teuthology.orchestra.run.smithi183.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 326 kB/s | 16 kB 00:00 2024-09-19T04:33:28.505 INFO:teuthology.orchestra.run.smithi183.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.5 MB/s | 285 kB 00:00 2024-09-19T04:33:28.518 INFO:teuthology.orchestra.run.smithi193.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 295 kB/s | 253 kB 00:00 2024-09-19T04:33:28.555 INFO:teuthology.orchestra.run.smithi183.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 366 kB/s | 18 kB 00:00 2024-09-19T04:33:28.664 INFO:teuthology.orchestra.run.smithi183.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 16 MB/s | 7.4 MB 00:00 2024-09-19T04:33:28.691 INFO:teuthology.orchestra.run.smithi123.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 53 kB/s | 6.6 kB 00:00 2024-09-19T04:33:28.698 INFO:teuthology.orchestra.run.smithi183.stdout:(39/156): cephadm-19.1.1-361.g2f599a46.el9.noar 3.4 MB/s | 765 kB 00:00 2024-09-19T04:33:28.773 INFO:teuthology.orchestra.run.smithi183.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 368 kB/s | 80 kB 00:00 2024-09-19T04:33:28.857 INFO:teuthology.orchestra.run.smithi183.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-19T04:33:28.866 INFO:teuthology.orchestra.run.smithi123.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 5.0 MB/s | 896 kB 00:00 2024-09-19T04:33:28.890 INFO:teuthology.orchestra.run.smithi183.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 996 kB/s | 191 kB 00:00 2024-09-19T04:33:28.933 INFO:teuthology.orchestra.run.smithi123.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.1 MB/s | 145 kB 00:00 2024-09-19T04:33:28.949 INFO:teuthology.orchestra.run.smithi183.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-19T04:33:28.974 INFO:teuthology.orchestra.run.smithi183.stdout:(44/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-19T04:33:29.007 INFO:teuthology.orchestra.run.smithi183.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 8.1 MB/s | 1.2 MB 00:00 2024-09-19T04:33:29.033 INFO:teuthology.orchestra.run.smithi183.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-19T04:33:29.058 INFO:teuthology.orchestra.run.smithi123.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-19T04:33:29.058 INFO:teuthology.orchestra.run.smithi183.stdout:(47/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-19T04:33:29.083 INFO:teuthology.orchestra.run.smithi183.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-19T04:33:29.142 INFO:teuthology.orchestra.run.smithi123.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 11 MB/s | 2.4 MB 00:00 2024-09-19T04:33:29.169 INFO:teuthology.orchestra.run.smithi193.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 154 kB/s | 106 kB 00:00 2024-09-19T04:33:29.192 INFO:teuthology.orchestra.run.smithi123.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 11 MB/s | 1.5 MB 00:00 2024-09-19T04:33:29.227 INFO:teuthology.orchestra.run.smithi193.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 191 kB/s | 135 kB 00:00 2024-09-19T04:33:29.250 INFO:teuthology.orchestra.run.smithi183.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 641 kB/s | 107 kB 00:00 2024-09-19T04:33:29.302 INFO:teuthology.orchestra.run.smithi183.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 422 kB/s | 22 kB 00:00 2024-09-19T04:33:29.336 INFO:teuthology.orchestra.run.smithi183.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-19T04:33:29.369 INFO:teuthology.orchestra.run.smithi183.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-09-19T04:33:29.387 INFO:teuthology.orchestra.run.smithi183.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-09-19T04:33:29.437 INFO:teuthology.orchestra.run.smithi183.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-19T04:33:29.462 INFO:teuthology.orchestra.run.smithi183.stdout:(55/156): flex-2.6.4-9.el9.x86_64.rpm 1.9 MB/s | 313 kB 00:00 2024-09-19T04:33:29.487 INFO:teuthology.orchestra.run.smithi183.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-19T04:33:29.521 INFO:teuthology.orchestra.run.smithi183.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-19T04:33:29.554 INFO:teuthology.orchestra.run.smithi183.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 677 kB/s | 45 kB 00:00 2024-09-19T04:33:29.592 INFO:teuthology.orchestra.run.smithi123.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-09-19T04:33:29.638 INFO:teuthology.orchestra.run.smithi183.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-19T04:33:29.696 INFO:teuthology.orchestra.run.smithi183.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 4.0 MB/s | 581 kB 00:00 2024-09-19T04:33:29.722 INFO:teuthology.orchestra.run.smithi183.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 293 kB/s | 24 kB 00:00 2024-09-19T04:33:29.772 INFO:teuthology.orchestra.run.smithi183.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 955 kB/s | 47 kB 00:00 2024-09-19T04:33:29.797 INFO:teuthology.orchestra.run.smithi183.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-19T04:33:29.848 INFO:teuthology.orchestra.run.smithi183.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 772 kB/s | 38 kB 00:00 2024-09-19T04:33:29.878 INFO:teuthology.orchestra.run.smithi193.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 176 kB/s | 125 kB 00:00 2024-09-19T04:33:29.889 INFO:teuthology.orchestra.run.smithi183.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-19T04:33:29.919 INFO:teuthology.orchestra.run.smithi193.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 605 kB/s | 1.2 MB 00:02 2024-09-19T04:33:30.036 INFO:teuthology.orchestra.run.smithi193.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 266 kB/s | 215 kB 00:00 2024-09-19T04:33:30.042 INFO:teuthology.orchestra.run.smithi123.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 14 MB/s | 21 MB 00:01 2024-09-19T04:33:30.078 INFO:teuthology.orchestra.run.smithi193.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-09-19T04:33:30.129 INFO:teuthology.orchestra.run.smithi193.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-09-19T04:33:30.154 INFO:teuthology.orchestra.run.smithi193.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 289 kB/s | 22 kB 00:00 2024-09-19T04:33:30.179 INFO:teuthology.orchestra.run.smithi193.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-19T04:33:30.204 INFO:teuthology.orchestra.run.smithi193.stdout:(53/156): boost-program-options-1.75.0-8.el9.x8 638 kB/s | 107 kB 00:00 2024-09-19T04:33:30.226 INFO:teuthology.orchestra.run.smithi123.stdout:(9/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86_ 16 MB/s | 17 MB 00:01 2024-09-19T04:33:30.246 INFO:teuthology.orchestra.run.smithi193.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 355 kB/s | 33 kB 00:00 2024-09-19T04:33:30.260 INFO:teuthology.orchestra.run.smithi123.stdout:(10/156): ceph-selinux-19.1.1-361.g2f599a46.el9 116 kB/s | 25 kB 00:00 2024-09-19T04:33:30.272 INFO:teuthology.orchestra.run.smithi193.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-09-19T04:33:30.288 INFO:teuthology.orchestra.run.smithi193.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 265 kB/s | 11 kB 00:00 2024-09-19T04:33:30.318 INFO:teuthology.orchestra.run.smithi123.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.8 MB/s | 228 kB 00:00 2024-09-19T04:33:30.322 INFO:teuthology.orchestra.run.smithi193.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-19T04:33:30.339 INFO:teuthology.orchestra.run.smithi193.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-09-19T04:33:30.373 INFO:teuthology.orchestra.run.smithi193.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-09-19T04:33:30.377 INFO:teuthology.orchestra.run.smithi123.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 467 kB/s | 27 kB 00:00 2024-09-19T04:33:30.407 INFO:teuthology.orchestra.run.smithi183.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.9 MB/s | 3.0 MB 00:01 2024-09-19T04:33:30.423 INFO:teuthology.orchestra.run.smithi193.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-19T04:33:30.440 INFO:teuthology.orchestra.run.smithi193.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 365 kB/s | 24 kB 00:00 2024-09-19T04:33:30.457 INFO:teuthology.orchestra.run.smithi193.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-19T04:33:30.482 INFO:teuthology.orchestra.run.smithi193.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 292 kB 00:00 2024-09-19T04:33:30.509 INFO:teuthology.orchestra.run.smithi123.stdout:(13/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 12 MB/s | 11 MB 00:00 2024-09-19T04:33:30.515 INFO:teuthology.orchestra.run.smithi183.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-19T04:33:30.516 INFO:teuthology.orchestra.run.smithi193.stdout:(64/156): libtool-2.4.6-46.el9.x86_64.rpm 3.2 MB/s | 581 kB 00:00 2024-09-19T04:33:30.536 INFO:teuthology.orchestra.run.smithi123.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 200 kB/s | 32 kB 00:00 2024-09-19T04:33:30.541 INFO:teuthology.orchestra.run.smithi193.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 657 kB/s | 38 kB 00:00 2024-09-19T04:33:30.569 INFO:teuthology.orchestra.run.smithi123.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 12 MB/s | 751 kB 00:00 2024-09-19T04:33:30.590 INFO:teuthology.orchestra.run.smithi183.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-19T04:33:30.594 INFO:teuthology.orchestra.run.smithi123.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.7 MB/s | 163 kB 00:00 2024-09-19T04:33:30.620 INFO:teuthology.orchestra.run.smithi123.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T04:33:30.642 INFO:teuthology.orchestra.run.smithi193.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-09-19T04:33:30.649 INFO:teuthology.orchestra.run.smithi183.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-19T04:33:30.670 INFO:teuthology.orchestra.run.smithi123.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 6.6 MB/s | 506 kB 00:00 2024-09-19T04:33:30.734 INFO:teuthology.orchestra.run.smithi193.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-09-19T04:33:30.745 INFO:teuthology.orchestra.run.smithi123.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 605 kB/s | 45 kB 00:00 2024-09-19T04:33:30.767 INFO:teuthology.orchestra.run.smithi193.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-19T04:33:30.801 INFO:teuthology.orchestra.run.smithi193.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 13 MB/s | 910 kB 00:00 2024-09-19T04:33:30.812 INFO:teuthology.orchestra.run.smithi123.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.0 MB/s | 136 kB 00:00 2024-09-19T04:33:30.824 INFO:teuthology.orchestra.run.smithi183.stdout:(70/156): postgresql-server-13.16-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-09-19T04:33:30.826 INFO:teuthology.orchestra.run.smithi193.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-19T04:33:30.879 INFO:teuthology.orchestra.run.smithi123.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.4 MB/s | 161 kB 00:00 2024-09-19T04:33:30.891 INFO:teuthology.orchestra.run.smithi183.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-19T04:33:30.893 INFO:teuthology.orchestra.run.smithi193.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 13 MB/s | 4.9 MB 00:00 2024-09-19T04:33:30.927 INFO:teuthology.orchestra.run.smithi193.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-09-19T04:33:30.941 INFO:teuthology.orchestra.run.smithi183.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-19T04:33:30.952 INFO:teuthology.orchestra.run.smithi193.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-19T04:33:30.988 INFO:teuthology.orchestra.run.smithi123.stdout:(22/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-19T04:33:31.002 INFO:teuthology.orchestra.run.smithi193.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-19T04:33:31.021 INFO:teuthology.orchestra.run.smithi123.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 2.2 MB/s | 320 kB 00:00 2024-09-19T04:33:31.047 INFO:teuthology.orchestra.run.smithi123.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.0 MB/s | 300 kB 00:00 2024-09-19T04:33:31.072 INFO:teuthology.orchestra.run.smithi123.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T04:33:31.077 INFO:teuthology.orchestra.run.smithi193.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-09-19T04:33:31.097 INFO:teuthology.orchestra.run.smithi123.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T04:33:31.103 INFO:teuthology.orchestra.run.smithi193.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-19T04:33:31.125 INFO:teuthology.orchestra.run.smithi183.stdout:(73/156): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 6.0 MB 00:00 2024-09-19T04:33:31.128 INFO:teuthology.orchestra.run.smithi193.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-19T04:33:31.148 INFO:teuthology.orchestra.run.smithi123.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-19T04:33:31.153 INFO:teuthology.orchestra.run.smithi193.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-09-19T04:33:31.179 INFO:teuthology.orchestra.run.smithi193.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-19T04:33:31.183 INFO:teuthology.orchestra.run.smithi183.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-19T04:33:31.204 INFO:teuthology.orchestra.run.smithi193.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-19T04:33:31.214 INFO:teuthology.orchestra.run.smithi123.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 480 kB/s | 32 kB 00:00 2024-09-19T04:33:31.229 INFO:teuthology.orchestra.run.smithi193.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-19T04:33:31.242 INFO:teuthology.orchestra.run.smithi183.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-19T04:33:31.273 INFO:teuthology.orchestra.run.smithi123.stdout:(29/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 15 MB/s | 3.1 MB 00:00 2024-09-19T04:33:31.292 INFO:teuthology.orchestra.run.smithi183.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-19T04:33:31.298 INFO:teuthology.orchestra.run.smithi123.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.7 MB/s | 146 kB 00:00 2024-09-19T04:33:31.304 INFO:teuthology.orchestra.run.smithi193.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-19T04:33:31.330 INFO:teuthology.orchestra.run.smithi193.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-19T04:33:31.342 INFO:teuthology.orchestra.run.smithi183.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-19T04:33:31.355 INFO:teuthology.orchestra.run.smithi193.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-19T04:33:31.380 INFO:teuthology.orchestra.run.smithi193.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T04:33:31.393 INFO:teuthology.orchestra.run.smithi183.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-19T04:33:31.406 INFO:teuthology.orchestra.run.smithi193.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-19T04:33:31.431 INFO:teuthology.orchestra.run.smithi193.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-19T04:33:31.448 INFO:teuthology.orchestra.run.smithi193.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-09-19T04:33:31.459 INFO:teuthology.orchestra.run.smithi183.stdout:(79/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.1 MB/s | 4.9 MB 00:01 2024-09-19T04:33:31.473 INFO:teuthology.orchestra.run.smithi193.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 6.5 MB/s | 279 kB 00:00 2024-09-19T04:33:31.485 INFO:teuthology.orchestra.run.smithi183.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-09-19T04:33:31.499 INFO:teuthology.orchestra.run.smithi193.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T04:33:31.510 INFO:teuthology.orchestra.run.smithi183.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-09-19T04:33:31.524 INFO:teuthology.orchestra.run.smithi193.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T04:33:31.565 INFO:teuthology.orchestra.run.smithi123.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 12 MB/s | 3.6 MB 00:00 2024-09-19T04:33:31.608 INFO:teuthology.orchestra.run.smithi193.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-19T04:33:31.633 INFO:teuthology.orchestra.run.smithi193.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-09-19T04:33:31.640 INFO:teuthology.orchestra.run.smithi123.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 3.3 MB/s | 249 kB 00:00 2024-09-19T04:33:31.644 INFO:teuthology.orchestra.run.smithi183.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-09-19T04:33:31.658 INFO:teuthology.orchestra.run.smithi193.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-09-19T04:33:31.683 INFO:teuthology.orchestra.run.smithi193.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-19T04:33:31.702 INFO:teuthology.orchestra.run.smithi183.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-19T04:33:31.709 INFO:teuthology.orchestra.run.smithi193.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-09-19T04:33:31.716 INFO:teuthology.orchestra.run.smithi123.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 659 kB/s | 49 kB 00:00 2024-09-19T04:33:31.727 INFO:teuthology.orchestra.run.smithi183.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 250 kB 00:00 2024-09-19T04:33:31.734 INFO:teuthology.orchestra.run.smithi193.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-19T04:33:31.753 INFO:teuthology.orchestra.run.smithi183.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T04:33:31.760 INFO:teuthology.orchestra.run.smithi193.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-19T04:33:31.778 INFO:teuthology.orchestra.run.smithi183.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 839 kB/s | 42 kB 00:00 2024-09-19T04:33:31.785 INFO:teuthology.orchestra.run.smithi193.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-19T04:33:31.808 INFO:teuthology.orchestra.run.smithi123.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 15 MB/s | 7.4 MB 00:00 2024-09-19T04:33:31.810 INFO:teuthology.orchestra.run.smithi193.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-19T04:33:31.812 INFO:teuthology.orchestra.run.smithi183.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-19T04:33:31.833 INFO:teuthology.orchestra.run.smithi123.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 141 kB/s | 16 kB 00:00 2024-09-19T04:33:31.835 INFO:teuthology.orchestra.run.smithi193.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T04:33:31.858 INFO:teuthology.orchestra.run.smithi123.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.6 MB/s | 285 kB 00:00 2024-09-19T04:33:31.861 INFO:teuthology.orchestra.run.smithi193.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-09-19T04:33:31.870 INFO:teuthology.orchestra.run.smithi183.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-19T04:33:31.892 INFO:teuthology.orchestra.run.smithi123.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 13 MB/s | 765 kB 00:00 2024-09-19T04:33:31.895 INFO:teuthology.orchestra.run.smithi183.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-19T04:33:31.917 INFO:teuthology.orchestra.run.smithi123.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 312 kB/s | 18 kB 00:00 2024-09-19T04:33:31.919 INFO:teuthology.orchestra.run.smithi193.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-09-19T04:33:31.921 INFO:teuthology.orchestra.run.smithi183.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T04:33:31.946 INFO:teuthology.orchestra.run.smithi183.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T04:33:31.970 INFO:teuthology.orchestra.run.smithi193.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-19T04:33:31.996 INFO:teuthology.orchestra.run.smithi183.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-09-19T04:33:32.003 INFO:teuthology.orchestra.run.smithi193.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-19T04:33:32.029 INFO:teuthology.orchestra.run.smithi193.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 5.0 MB/s | 863 kB 00:00 2024-09-19T04:33:32.054 INFO:teuthology.orchestra.run.smithi193.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-09-19T04:33:32.063 INFO:teuthology.orchestra.run.smithi183.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 662 kB/s | 44 kB 00:00 2024-09-19T04:33:32.088 INFO:teuthology.orchestra.run.smithi193.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 987 kB/s | 26 kB 00:00 2024-09-19T04:33:32.113 INFO:teuthology.orchestra.run.smithi193.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-19T04:33:32.138 INFO:teuthology.orchestra.run.smithi193.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-19T04:33:32.147 INFO:teuthology.orchestra.run.smithi183.stdout:(94/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 4.4 MB 00:02 2024-09-19T04:33:32.164 INFO:teuthology.orchestra.run.smithi193.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-19T04:33:32.214 INFO:teuthology.orchestra.run.smithi183.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 538 kB/s | 36 kB 00:00 2024-09-19T04:33:32.281 INFO:teuthology.orchestra.run.smithi183.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 712 kB/s | 47 kB 00:00 2024-09-19T04:33:32.322 INFO:teuthology.orchestra.run.smithi193.stdout:(112/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-09-19T04:33:32.364 INFO:teuthology.orchestra.run.smithi193.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.1 MB/s | 839 kB 00:00 2024-09-19T04:33:32.389 INFO:teuthology.orchestra.run.smithi193.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.8 MB/s | 43 kB 00:00 2024-09-19T04:33:32.393 INFO:teuthology.orchestra.run.smithi123.stdout:(39/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 22 MB/s | 49 MB 00:02 2024-09-19T04:33:32.415 INFO:teuthology.orchestra.run.smithi193.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T04:33:32.432 INFO:teuthology.orchestra.run.smithi183.stdout:(97/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-19T04:33:32.447 INFO:teuthology.orchestra.run.smithi123.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 144 kB/s | 80 kB 00:00 2024-09-19T04:33:32.449 INFO:teuthology.orchestra.run.smithi193.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 422 kB/s | 14 kB 00:00 2024-09-19T04:33:32.457 INFO:teuthology.orchestra.run.smithi183.stdout:(98/156): qatlib-24.02.0-1.el9.x86_64.rpm 562 kB/s | 221 kB 00:00 2024-09-19T04:33:32.482 INFO:teuthology.orchestra.run.smithi183.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-09-19T04:33:32.482 INFO:teuthology.orchestra.run.smithi193.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 5.1 MB/s | 172 kB 00:00 2024-09-19T04:33:32.508 INFO:teuthology.orchestra.run.smithi193.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-09-19T04:33:32.508 INFO:teuthology.orchestra.run.smithi183.stdout:(100/156): uuid-1.6.2-55.el9.x86_64.rpm 768 kB/s | 58 kB 00:00 2024-09-19T04:33:32.535 INFO:teuthology.orchestra.run.smithi123.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.8 MB/s | 253 kB 00:00 2024-09-19T04:33:32.541 INFO:teuthology.orchestra.run.smithi183.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-19T04:33:32.541 INFO:teuthology.orchestra.run.smithi193.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-19T04:33:32.566 INFO:teuthology.orchestra.run.smithi183.stdout:(102/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 876 kB/s | 96 kB 00:00 2024-09-19T04:33:32.567 INFO:teuthology.orchestra.run.smithi193.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-19T04:33:32.591 INFO:teuthology.orchestra.run.smithi183.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 534 kB/s | 45 kB 00:00 2024-09-19T04:33:32.592 INFO:teuthology.orchestra.run.smithi193.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T04:33:32.593 INFO:teuthology.orchestra.run.smithi123.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 955 kB/s | 191 kB 00:00 2024-09-19T04:33:32.617 INFO:teuthology.orchestra.run.smithi193.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-19T04:33:32.635 INFO:teuthology.orchestra.run.smithi123.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-09-19T04:33:32.641 INFO:teuthology.orchestra.run.smithi183.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-19T04:33:32.642 INFO:teuthology.orchestra.run.smithi193.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-19T04:33:32.668 INFO:teuthology.orchestra.run.smithi193.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 705 kB/s | 18 kB 00:00 2024-09-19T04:33:32.669 INFO:teuthology.orchestra.run.smithi123.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-19T04:33:32.675 INFO:teuthology.orchestra.run.smithi183.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-19T04:33:32.693 INFO:teuthology.orchestra.run.smithi193.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 927 kB/s | 23 kB 00:00 2024-09-19T04:33:32.694 INFO:teuthology.orchestra.run.smithi123.stdout:(45/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 801 kB 00:00 2024-09-19T04:33:32.700 INFO:teuthology.orchestra.run.smithi183.stdout:(106/156): lua-devel-5.4.4-4.el9.x86_64.rpm 140 kB/s | 22 kB 00:00 2024-09-19T04:33:32.718 INFO:teuthology.orchestra.run.smithi193.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 783 kB/s | 19 kB 00:00 2024-09-19T04:33:32.720 INFO:teuthology.orchestra.run.smithi123.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-19T04:33:32.726 INFO:teuthology.orchestra.run.smithi183.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 386 kB/s | 19 kB 00:00 2024-09-19T04:33:32.735 INFO:teuthology.orchestra.run.smithi193.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-19T04:33:32.751 INFO:teuthology.orchestra.run.smithi183.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-19T04:33:32.761 INFO:teuthology.orchestra.run.smithi123.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-09-19T04:33:32.776 INFO:teuthology.orchestra.run.smithi183.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T04:33:32.786 INFO:teuthology.orchestra.run.smithi123.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-19T04:33:32.794 INFO:teuthology.orchestra.run.smithi193.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 9.0 MB/s | 2.0 MB 00:00 2024-09-19T04:33:32.802 INFO:teuthology.orchestra.run.smithi183.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-19T04:33:32.819 INFO:teuthology.orchestra.run.smithi193.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T04:33:32.835 INFO:teuthology.orchestra.run.smithi183.stdout:(111/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-09-19T04:33:32.869 INFO:teuthology.orchestra.run.smithi183.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-19T04:33:32.878 INFO:teuthology.orchestra.run.smithi193.stdout:(130/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.2 MB/s | 4.4 MB 00:00 2024-09-19T04:33:32.912 INFO:teuthology.orchestra.run.smithi193.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-09-19T04:33:32.919 INFO:teuthology.orchestra.run.smithi183.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-19T04:33:32.936 INFO:teuthology.orchestra.run.smithi183.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.5 MB/s | 43 kB 00:00 2024-09-19T04:33:32.937 INFO:teuthology.orchestra.run.smithi193.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 673 kB/s | 79 kB 00:00 2024-09-19T04:33:32.953 INFO:teuthology.orchestra.run.smithi183.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-19T04:33:32.962 INFO:teuthology.orchestra.run.smithi193.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 665 kB/s | 16 kB 00:00 2024-09-19T04:33:32.970 INFO:teuthology.orchestra.run.smithi183.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 847 kB/s | 14 kB 00:00 2024-09-19T04:33:32.987 INFO:teuthology.orchestra.run.smithi193.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-19T04:33:32.995 INFO:teuthology.orchestra.run.smithi183.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-19T04:33:33.013 INFO:teuthology.orchestra.run.smithi193.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-19T04:33:33.029 INFO:teuthology.orchestra.run.smithi183.stdout:(118/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.2 MB/s | 839 kB 00:00 2024-09-19T04:33:33.038 INFO:teuthology.orchestra.run.smithi193.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-19T04:33:33.062 INFO:teuthology.orchestra.run.smithi183.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-19T04:33:33.063 INFO:teuthology.orchestra.run.smithi193.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-19T04:33:33.087 INFO:teuthology.orchestra.run.smithi183.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-19T04:33:33.097 INFO:teuthology.orchestra.run.smithi193.stdout:(138/156): python3-natsort-7.1.1-5.el9.noarch.r 264 kB/s | 58 kB 00:00 2024-09-19T04:33:33.113 INFO:teuthology.orchestra.run.smithi183.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T04:33:33.122 INFO:teuthology.orchestra.run.smithi193.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-19T04:33:33.138 INFO:teuthology.orchestra.run.smithi183.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-19T04:33:33.148 INFO:teuthology.orchestra.run.smithi193.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 738 kB/s | 36 kB 00:00 2024-09-19T04:33:33.164 INFO:teuthology.orchestra.run.smithi183.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-19T04:33:33.173 INFO:teuthology.orchestra.run.smithi193.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-19T04:33:33.189 INFO:teuthology.orchestra.run.smithi183.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-19T04:33:33.198 INFO:teuthology.orchestra.run.smithi193.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-19T04:33:33.214 INFO:teuthology.orchestra.run.smithi183.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-09-19T04:33:33.224 INFO:teuthology.orchestra.run.smithi193.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T04:33:33.237 INFO:teuthology.orchestra.run.smithi123.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 237 kB/s | 107 kB 00:00 2024-09-19T04:33:33.239 INFO:teuthology.orchestra.run.smithi183.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 771 kB/s | 19 kB 00:00 2024-09-19T04:33:33.249 INFO:teuthology.orchestra.run.smithi193.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-19T04:33:33.265 INFO:teuthology.orchestra.run.smithi183.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 787 kB/s | 20 kB 00:00 2024-09-19T04:33:33.274 INFO:teuthology.orchestra.run.smithi193.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-19T04:33:33.300 INFO:teuthology.orchestra.run.smithi193.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T04:33:33.325 INFO:teuthology.orchestra.run.smithi193.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-19T04:33:33.332 INFO:teuthology.orchestra.run.smithi183.stdout:(128/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 4.4 MB 00:00 2024-09-19T04:33:33.354 INFO:teuthology.orchestra.run.smithi123.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 186 kB/s | 22 kB 00:00 2024-09-19T04:33:33.357 INFO:teuthology.orchestra.run.smithi183.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T04:33:33.375 INFO:teuthology.orchestra.run.smithi193.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-19T04:33:33.382 INFO:teuthology.orchestra.run.smithi183.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T04:33:33.408 INFO:teuthology.orchestra.run.smithi183.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T04:33:33.441 INFO:teuthology.orchestra.run.smithi183.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-19T04:33:33.466 INFO:teuthology.orchestra.run.smithi183.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-19T04:33:33.492 INFO:teuthology.orchestra.run.smithi183.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-19T04:33:33.534 INFO:teuthology.orchestra.run.smithi183.stdout:(135/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.6 MB/s | 2.0 MB 00:00 2024-09-19T04:33:33.537 INFO:teuthology.orchestra.run.smithi123.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 843 kB/s | 689 kB 00:00 2024-09-19T04:33:33.559 INFO:teuthology.orchestra.run.smithi193.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 13 MB/s | 3.4 MB 00:00 2024-09-19T04:33:33.567 INFO:teuthology.orchestra.run.smithi183.stdout:(136/156): python3-kubernetes-26.1.0-2.el9.noar 3.4 MB/s | 1.0 MB 00:00 2024-09-19T04:33:33.593 INFO:teuthology.orchestra.run.smithi183.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 280 kB/s | 28 kB 00:00 2024-09-19T04:33:33.618 INFO:teuthology.orchestra.run.smithi183.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-09-19T04:33:33.637 INFO:teuthology.orchestra.run.smithi123.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 1.1 MB/s | 313 kB 00:00 2024-09-19T04:33:33.643 INFO:teuthology.orchestra.run.smithi183.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-19T04:33:33.651 INFO:teuthology.orchestra.run.smithi193.stdout:(150/156): librbd1-19.1.1-361.g2f599a46.el9.x86 9.7 MB/s | 3.2 MB 00:00 2024-09-19T04:33:33.663 INFO:teuthology.orchestra.run.smithi123.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 261 kB/s | 33 kB 00:00 2024-09-19T04:33:33.668 INFO:teuthology.orchestra.run.smithi183.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-19T04:33:33.694 INFO:teuthology.orchestra.run.smithi183.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-19T04:33:33.696 INFO:teuthology.orchestra.run.smithi123.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-09-19T04:33:33.711 INFO:teuthology.orchestra.run.smithi183.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-19T04:33:33.728 INFO:teuthology.orchestra.run.smithi183.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 2.6 MB/s | 90 kB 00:00 2024-09-19T04:33:33.753 INFO:teuthology.orchestra.run.smithi183.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-19T04:33:33.778 INFO:teuthology.orchestra.run.smithi183.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-19T04:33:33.780 INFO:teuthology.orchestra.run.smithi123.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-09-19T04:33:33.803 INFO:teuthology.orchestra.run.smithi183.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T04:33:33.805 INFO:teuthology.orchestra.run.smithi123.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-09-19T04:33:33.829 INFO:teuthology.orchestra.run.smithi183.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-19T04:33:33.895 INFO:teuthology.orchestra.run.smithi183.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-19T04:33:33.897 INFO:teuthology.orchestra.run.smithi123.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-19T04:33:33.922 INFO:teuthology.orchestra.run.smithi123.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-09-19T04:33:34.056 INFO:teuthology.orchestra.run.smithi123.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 4.3 MB/s | 581 kB 00:00 2024-09-19T04:33:34.173 INFO:teuthology.orchestra.run.smithi123.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 210 kB/s | 24 kB 00:00 2024-09-19T04:33:34.196 INFO:teuthology.orchestra.run.smithi183.stdout:(149/156): librbd1-19.1.1-361.g2f599a46.el9.x86 8.6 MB/s | 3.2 MB 00:00 2024-09-19T04:33:34.198 INFO:teuthology.orchestra.run.smithi123.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.2 MB/s | 662 kB 00:00 2024-09-19T04:33:34.234 INFO:teuthology.orchestra.run.smithi193.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 71 kB/s | 41 kB 00:00 2024-09-19T04:33:34.257 INFO:teuthology.orchestra.run.smithi123.stdout:(62/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.8 MB/s | 3.0 MB 00:00 2024-09-19T04:33:34.282 INFO:teuthology.orchestra.run.smithi123.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-09-19T04:33:34.316 INFO:teuthology.orchestra.run.smithi123.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-19T04:33:34.363 INFO:teuthology.orchestra.run.smithi183.stdout:(150/156): librados2-19.1.1-361.g2f599a46.el9.x 6.1 MB/s | 3.4 MB 00:00 2024-09-19T04:33:34.374 INFO:teuthology.orchestra.run.smithi123.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-19T04:33:34.391 INFO:teuthology.orchestra.run.smithi123.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 353 kB/s | 38 kB 00:00 2024-09-19T04:33:34.488 INFO:teuthology.orchestra.run.smithi183.stdout:(151/156): NetworkManager-libnm-1.51.0-1.el9.x8 6.2 MB/s | 1.8 MB 00:00 2024-09-19T04:33:34.505 INFO:teuthology.orchestra.run.smithi183.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 288 kB/s | 41 kB 00:00 2024-09-19T04:33:34.547 INFO:teuthology.orchestra.run.smithi183.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 4.2 MB/s | 249 kB 00:00 2024-09-19T04:33:34.625 INFO:teuthology.orchestra.run.smithi123.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-09-19T04:33:34.681 INFO:teuthology.orchestra.run.smithi183.stdout:(154/156): NetworkManager-1.51.0-1.el9.x86_64.r 3.0 MB/s | 2.3 MB 00:00 2024-09-19T04:33:34.767 INFO:teuthology.orchestra.run.smithi123.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 6.3 MB/s | 910 kB 00:00 2024-09-19T04:33:34.789 INFO:teuthology.orchestra.run.smithi183.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 9.9 MB/s | 2.4 MB 00:00 2024-09-19T04:33:34.810 INFO:teuthology.orchestra.run.smithi193.stdout:(152/156): NetworkManager-1.51.0-1.el9.x86_64.r 1.6 MB/s | 2.3 MB 00:01 2024-09-19T04:33:34.892 INFO:teuthology.orchestra.run.smithi123.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-19T04:33:34.968 INFO:teuthology.orchestra.run.smithi193.stdout:(153/156): NetworkManager-libnm-1.51.0-1.el9.x8 1.3 MB/s | 1.8 MB 00:01 2024-09-19T04:33:34.993 INFO:teuthology.orchestra.run.smithi193.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 328 kB/s | 249 kB 00:00 2024-09-19T04:33:34.997 INFO:teuthology.orchestra.run.smithi183.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 2.7 MB/s | 1.4 MB 00:00 2024-09-19T04:33:35.005 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:33:35.005 INFO:teuthology.orchestra.run.smithi183.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-19T04:33:35.692 INFO:teuthology.orchestra.run.smithi123.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.6 MB/s | 4.9 MB 00:01 2024-09-19T04:33:35.785 INFO:teuthology.orchestra.run.smithi193.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:00 2024-09-19T04:33:36.002 INFO:teuthology.orchestra.run.smithi193.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-09-19T04:33:36.009 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:33:36.010 INFO:teuthology.orchestra.run.smithi193.stdout:Total 16 MB/s | 226 MB 00:13 2024-09-19T04:33:36.109 INFO:teuthology.orchestra.run.smithi123.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.6 MB/s | 4.4 MB 00:01 2024-09-19T04:33:36.244 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-09-19T04:33:36.309 INFO:teuthology.orchestra.run.smithi123.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 438 kB/s | 87 kB 00:00 2024-09-19T04:33:36.394 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-09-19T04:33:36.394 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-09-19T04:33:37.235 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-19T04:33:37.259 INFO:teuthology.orchestra.run.smithi123.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 659 kB/s | 1.0 MB 00:01 2024-09-19T04:33:37.385 INFO:teuthology.orchestra.run.smithi123.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 490 kB/s | 61 kB 00:00 2024-09-19T04:33:37.385 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-19T04:33:37.385 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-19T04:33:37.769 INFO:teuthology.orchestra.run.smithi123.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 638 kB/s | 245 kB 00:00 2024-09-19T04:33:38.152 INFO:teuthology.orchestra.run.smithi123.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 648 kB/s | 248 kB 00:00 2024-09-19T04:33:38.201 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-09-19T04:33:38.202 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-09-19T04:33:38.269 INFO:teuthology.orchestra.run.smithi123.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 220 kB/s | 26 kB 00:00 2024-09-19T04:33:38.402 INFO:teuthology.orchestra.run.smithi123.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 144 kB/s | 19 kB 00:00 2024-09-19T04:33:38.711 INFO:teuthology.orchestra.run.smithi123.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 558 kB/s | 172 kB 00:00 2024-09-19T04:33:38.828 INFO:teuthology.orchestra.run.smithi123.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 298 kB/s | 35 kB 00:00 2024-09-19T04:33:39.285 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-19T04:33:39.285 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-19T04:33:40.745 INFO:teuthology.orchestra.run.smithi123.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 1.0 MB/s | 5.8 MB 00:05 2024-09-19T04:33:41.120 INFO:teuthology.orchestra.run.smithi123.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 669 kB/s | 250 kB 00:00 2024-09-19T04:33:41.487 INFO:teuthology.orchestra.run.smithi123.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 606 kB/s | 222 kB 00:00 2024-09-19T04:33:41.687 INFO:teuthology.orchestra.run.smithi123.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 386 kB/s | 77 kB 00:00 2024-09-19T04:33:41.804 INFO:teuthology.orchestra.run.smithi123.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 362 kB/s | 42 kB 00:00 2024-09-19T04:33:42.271 INFO:teuthology.orchestra.run.smithi123.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 577 kB/s | 269 kB 00:00 2024-09-19T04:33:42.554 INFO:teuthology.orchestra.run.smithi123.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 561 kB/s | 159 kB 00:00 2024-09-19T04:33:43.021 INFO:teuthology.orchestra.run.smithi123.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 597 kB/s | 279 kB 00:00 2024-09-19T04:33:43.130 INFO:teuthology.orchestra.run.smithi123.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 471 kB/s | 51 kB 00:00 2024-09-19T04:33:43.238 INFO:teuthology.orchestra.run.smithi123.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 495 kB/s | 54 kB 00:00 2024-09-19T04:33:44.659 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-09-19T04:33:44.743 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T04:33:44.790 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T04:33:44.843 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T04:33:44.877 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T04:33:44.902 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T04:33:44.945 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T04:33:45.030 INFO:teuthology.orchestra.run.smithi123.stdout:(91/156): python3-babel-2.9.1-2.el9.noarch.rpm 700 kB/s | 6.0 MB 00:08 2024-09-19T04:33:45.147 INFO:teuthology.orchestra.run.smithi123.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 359 kB/s | 42 kB 00:00 2024-09-19T04:33:45.264 INFO:teuthology.orchestra.run.smithi123.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 378 kB/s | 44 kB 00:00 2024-09-19T04:33:45.266 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T04:33:45.370 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:33:45.474 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:33:45.504 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:33:45.594 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:33:45.625 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:33:45.631 INFO:teuthology.orchestra.run.smithi123.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 602 kB/s | 221 kB 00:00 2024-09-19T04:33:45.748 INFO:teuthology.orchestra.run.smithi123.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 308 kB/s | 36 kB 00:00 2024-09-19T04:33:45.773 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T04:33:45.821 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-19T04:33:45.822 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T04:33:45.850 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:33:45.883 INFO:teuthology.orchestra.run.smithi123.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 352 kB/s | 47 kB 00:00 2024-09-19T04:33:45.931 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T04:33:45.937 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:33:45.959 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:33:45.977 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T04:33:46.022 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:33:46.039 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T04:33:46.044 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:33:46.063 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:33:46.073 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T04:33:46.099 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T04:33:46.141 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T04:33:46.253 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:33:46.290 INFO:teuthology.orchestra.run.smithi123.stdout:(97/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 699 kB/s | 5.1 MB 00:07 2024-09-19T04:33:46.384 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:33:46.390 INFO:teuthology.orchestra.run.smithi123.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 600 kB/s | 304 kB 00:00 2024-09-19T04:33:46.415 INFO:teuthology.orchestra.run.smithi123.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 462 kB/s | 58 kB 00:00 2024-09-19T04:33:46.464 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T04:33:46.534 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T04:33:46.575 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:33:46.583 INFO:teuthology.orchestra.run.smithi123.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 499 kB/s | 96 kB 00:00 2024-09-19T04:33:46.616 INFO:teuthology.orchestra.run.smithi123.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 317 kB/s | 64 kB 00:00 2024-09-19T04:33:46.644 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:33:46.680 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:33:46.708 INFO:teuthology.orchestra.run.smithi123.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 358 kB/s | 45 kB 00:00 2024-09-19T04:33:46.709 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:33:46.733 INFO:teuthology.orchestra.run.smithi123.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-19T04:33:46.735 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:33:46.784 INFO:teuthology.orchestra.run.smithi123.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-19T04:33:46.801 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:33:46.817 INFO:teuthology.orchestra.run.smithi123.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-19T04:33:46.830 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:33:46.851 INFO:teuthology.orchestra.run.smithi123.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 591 kB/s | 19 kB 00:00 2024-09-19T04:33:46.955 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:33:46.976 INFO:teuthology.orchestra.run.smithi123.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-09-19T04:33:46.977 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T04:33:47.001 INFO:teuthology.orchestra.run.smithi123.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-19T04:33:47.027 INFO:teuthology.orchestra.run.smithi123.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T04:33:47.052 INFO:teuthology.orchestra.run.smithi123.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-19T04:33:47.053 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T04:33:47.076 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T04:33:47.077 INFO:teuthology.orchestra.run.smithi123.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-19T04:33:47.081 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:33:47.119 INFO:teuthology.orchestra.run.smithi123.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-19T04:33:47.153 INFO:teuthology.orchestra.run.smithi123.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-19T04:33:47.170 INFO:teuthology.orchestra.run.smithi123.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-19T04:33:47.177 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T04:33:47.185 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:33:47.187 INFO:teuthology.orchestra.run.smithi123.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-19T04:33:47.203 INFO:teuthology.orchestra.run.smithi123.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 847 kB/s | 14 kB 00:00 2024-09-19T04:33:47.207 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:33:47.215 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T04:33:47.229 INFO:teuthology.orchestra.run.smithi123.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-19T04:33:47.259 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T04:33:47.262 INFO:teuthology.orchestra.run.smithi123.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 10 MB/s | 358 kB 00:00 2024-09-19T04:33:47.269 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:33:47.288 INFO:teuthology.orchestra.run.smithi123.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 12 MB/s | 303 kB 00:00 2024-09-19T04:33:47.293 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:33:47.313 INFO:teuthology.orchestra.run.smithi123.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-19T04:33:47.317 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:33:47.331 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T04:33:47.376 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T04:33:47.388 INFO:teuthology.orchestra.run.smithi123.stdout:(121/156): python3-grpcio-1.46.7-10.el9.x86_64. 27 MB/s | 2.0 MB 00:00 2024-09-19T04:33:47.408 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T04:33:47.413 INFO:teuthology.orchestra.run.smithi123.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-19T04:33:47.439 INFO:teuthology.orchestra.run.smithi123.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-19T04:33:47.451 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T04:33:47.464 INFO:teuthology.orchestra.run.smithi123.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-19T04:33:47.489 INFO:teuthology.orchestra.run.smithi123.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 929 kB/s | 23 kB 00:00 2024-09-19T04:33:47.492 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T04:33:47.514 INFO:teuthology.orchestra.run.smithi123.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 782 kB/s | 19 kB 00:00 2024-09-19T04:33:47.517 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:33:47.520 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T04:33:47.540 INFO:teuthology.orchestra.run.smithi123.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 778 kB/s | 20 kB 00:00 2024-09-19T04:33:47.582 INFO:teuthology.orchestra.run.smithi123.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 25 MB/s | 1.0 MB 00:00 2024-09-19T04:33:47.588 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T04:33:47.607 INFO:teuthology.orchestra.run.smithi123.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T04:33:47.632 INFO:teuthology.orchestra.run.smithi123.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T04:33:47.642 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T04:33:47.642 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:33:47.657 INFO:teuthology.orchestra.run.smithi123.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T04:33:47.673 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T04:33:47.691 INFO:teuthology.orchestra.run.smithi123.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 7.9 MB/s | 272 kB 00:00 2024-09-19T04:33:47.708 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T04:33:47.717 INFO:teuthology.orchestra.run.smithi123.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 657 kB/s | 16 kB 00:00 2024-09-19T04:33:47.742 INFO:teuthology.orchestra.run.smithi123.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-19T04:33:47.757 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T04:33:47.767 INFO:teuthology.orchestra.run.smithi123.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-19T04:33:47.786 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T04:33:47.792 INFO:teuthology.orchestra.run.smithi123.stdout:(136/156): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-19T04:33:47.819 INFO:teuthology.orchestra.run.smithi123.stdout:(137/156): python3-routes-2.5.1-5.el9.noarch.rp 7.4 MB/s | 188 kB 00:00 2024-09-19T04:33:47.831 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T04:33:47.865 INFO:teuthology.orchestra.run.smithi123.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-19T04:33:47.866 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:33:47.874 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T04:33:47.893 INFO:teuthology.orchestra.run.smithi123.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.2 MB/s | 36 kB 00:00 2024-09-19T04:33:47.904 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T04:33:47.919 INFO:teuthology.orchestra.run.smithi123.stdout:(140/156): python3-typing-extensions-4.12.2-2.e 2.9 MB/s | 74 kB 00:00 2024-09-19T04:33:47.942 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:33:47.944 INFO:teuthology.orchestra.run.smithi123.stdout:(141/156): python3-webob-1.8.8-2.el9.noarch.rpm 9.0 MB/s | 230 kB 00:00 2024-09-19T04:33:47.968 INFO:teuthology.orchestra.run.smithi123.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.6 MB/s | 90 kB 00:00 2024-09-19T04:33:47.971 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T04:33:48.002 INFO:teuthology.orchestra.run.smithi123.stdout:(143/156): python3-werkzeug-2.0.3-3.el9.1.noarc 13 MB/s | 427 kB 00:00 2024-09-19T04:33:48.017 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T04:33:48.027 INFO:teuthology.orchestra.run.smithi123.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 798 kB/s | 20 kB 00:00 2024-09-19T04:33:48.053 INFO:teuthology.orchestra.run.smithi123.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-19T04:33:48.091 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T04:33:48.111 INFO:teuthology.orchestra.run.smithi123.stdout:(146/156): thrift-0.15.0-2.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-09-19T04:33:48.164 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:33:48.230 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T04:33:48.236 INFO:teuthology.orchestra.run.smithi123.stdout:(147/156): protobuf-compiler-3.14.0-13.el9.x86_ 565 kB/s | 863 kB 00:01 2024-09-19T04:33:48.274 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T04:33:48.286 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T04:33:48.367 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T04:33:48.387 INFO:teuthology.orchestra.run.smithi123.stdout:(148/156): librados2-19.1.1-361.g2f599a46.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-19T04:33:48.401 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T04:33:48.435 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T04:33:48.487 INFO:teuthology.orchestra.run.smithi123.stdout:(149/156): NetworkManager-1.51.0-1.el9.x86_64.r 23 MB/s | 2.3 MB 00:00 2024-09-19T04:33:48.499 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T04:33:48.544 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T04:33:48.562 INFO:teuthology.orchestra.run.smithi123.stdout:(150/156): librbd1-19.1.1-361.g2f599a46.el9.x86 9.7 MB/s | 3.2 MB 00:00 2024-09-19T04:33:48.577 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T04:33:48.612 INFO:teuthology.orchestra.run.smithi123.stdout:(151/156): NetworkManager-libnm-1.51.0-1.el9.x8 14 MB/s | 1.8 MB 00:00 2024-09-19T04:33:48.629 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T04:33:48.654 INFO:teuthology.orchestra.run.smithi123.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 444 kB/s | 41 kB 00:00 2024-09-19T04:33:48.670 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T04:33:48.680 INFO:teuthology.orchestra.run.smithi123.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 3.6 MB/s | 249 kB 00:00 2024-09-19T04:33:48.698 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T04:33:48.765 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T04:33:48.780 INFO:teuthology.orchestra.run.smithi123.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-09-19T04:33:48.800 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T04:33:48.836 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T04:33:48.867 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T04:33:48.890 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T04:33:48.916 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T04:33:48.920 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T04:33:48.954 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T04:33:48.972 INFO:teuthology.orchestra.run.smithi123.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 4.3 MB/s | 1.4 MB 00:00 2024-09-19T04:33:48.986 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T04:33:48.992 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T04:33:49.022 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T04:33:49.034 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T04:33:49.065 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T04:33:49.094 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T04:33:49.172 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T04:33:49.189 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T04:33:49.226 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T04:33:49.245 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T04:33:49.300 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T04:33:49.308 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T04:33:49.381 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T04:33:49.413 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T04:33:49.430 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T04:33:49.449 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T04:33:49.486 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T04:33:49.495 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T04:33:49.532 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T04:33:49.575 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T04:33:49.617 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T04:33:49.755 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T04:33:49.807 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T04:33:49.836 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T04:33:49.938 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T04:33:49.981 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T04:33:49.997 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T04:33:50.020 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T04:33:50.046 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T04:33:50.071 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T04:33:50.082 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T04:33:50.120 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T04:33:50.145 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T04:33:50.163 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T04:33:50.209 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T04:33:50.251 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T04:33:50.400 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T04:33:50.467 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T04:33:50.498 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T04:33:50.522 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T04:33:50.545 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T04:33:50.572 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T04:33:50.602 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T04:33:50.607 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T04:33:50.635 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T04:33:50.646 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T04:33:50.671 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T04:33:50.677 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T04:33:50.707 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T04:33:50.717 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T04:33:50.742 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T04:33:50.754 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T04:33:50.775 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T04:33:50.796 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T04:33:50.808 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T04:33:50.839 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T04:33:50.843 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T04:33:50.929 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T04:33:50.972 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T04:33:50.973 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T04:33:51.004 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T04:33:51.017 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T04:33:51.106 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T04:33:51.132 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T04:33:51.155 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T04:33:51.474 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T04:33:51.517 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T04:33:51.557 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T04:33:51.584 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T04:33:51.607 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T04:33:51.684 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T04:33:51.809 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T04:33:51.856 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T04:33:51.876 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T04:33:51.884 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T04:33:51.918 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T04:33:51.966 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T04:33:51.998 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T04:33:52.028 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T04:33:52.066 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T04:33:52.095 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T04:33:52.096 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T04:33:52.128 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T04:33:52.139 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T04:33:52.174 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T04:33:52.227 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T04:33:52.259 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T04:33:52.274 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T04:33:52.304 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T04:33:52.319 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T04:33:52.335 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T04:33:52.407 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T04:33:52.436 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T04:33:52.462 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T04:33:52.511 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T04:33:52.648 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T04:33:52.679 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T04:33:52.755 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T04:33:53.220 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T04:33:53.268 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T04:33:53.301 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T04:33:53.398 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T04:33:53.400 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T04:33:53.439 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T04:33:53.492 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T04:33:53.509 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T04:33:53.581 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T04:33:53.621 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T04:33:53.803 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T04:33:53.836 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T04:33:53.912 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T04:33:54.547 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T04:33:54.666 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T04:33:54.733 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T04:33:54.807 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T04:33:54.835 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T04:33:54.868 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T04:33:55.177 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T04:33:55.231 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:33:55.305 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:33:55.327 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T04:33:55.363 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T04:33:55.844 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T04:33:55.876 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T04:33:55.884 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:33:55.939 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:33:55.957 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T04:33:55.969 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T04:33:55.985 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T04:33:56.018 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T04:33:56.327 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T04:33:56.382 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:33:56.463 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:33:56.495 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T04:33:56.539 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T04:33:57.018 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T04:33:57.068 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:33:57.140 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:33:57.170 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T04:33:58.278 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:33:58.307 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:33:58.854 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:33:58.894 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:33:58.987 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:33:58.987 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T04:33:58.988 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T04:33:58.988 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:33:59.046 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:33:59.464 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:33:59.542 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:34:00.089 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:34:00.129 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:34:00.220 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:34:00.220 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T04:34:00.220 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T04:34:00.220 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:34:00.247 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:34:07.187 INFO:teuthology.orchestra.run.smithi123.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 695 kB/s | 16 MB 00:23 2024-09-19T04:34:07.191 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:34:07.191 INFO:teuthology.orchestra.run.smithi123.stdout:Total 5.8 MB/s | 226 MB 00:39 2024-09-19T04:34:09.257 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-19T04:34:09.437 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-19T04:34:09.438 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-19T04:34:10.820 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:34:10.820 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2024-09-19T04:34:10.820 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2024-09-19T04:34:10.820 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2024-09-19T04:34:10.820 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2024-09-19T04:34:10.820 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2024-09-19T04:34:10.821 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /root 2024-09-19T04:34:10.821 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2024-09-19T04:34:10.821 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:34:10.999 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:34:11.019 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:34:11.019 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:11.019 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T04:34:11.019 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:34:11.776 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-19T04:34:11.776 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-19T04:34:12.179 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:34:12.179 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /sys 2024-09-19T04:34:12.179 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /proc 2024-09-19T04:34:12.179 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /mnt 2024-09-19T04:34:12.180 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /var/tmp 2024-09-19T04:34:12.180 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /home 2024-09-19T04:34:12.180 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /root 2024-09-19T04:34:12.180 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /tmp 2024-09-19T04:34:12.180 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:34:12.353 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:34:12.370 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:34:12.370 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:12.370 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T04:34:12.370 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:34:12.494 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:34:12.521 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:34:12.521 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:12.521 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T04:34:12.521 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:34:12.522 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:34:12.522 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:34:12.744 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:34:12.768 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:34:12.768 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:12.768 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T04:34:12.768 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:34:12.768 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:34:12.768 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:34:13.204 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:34:13.227 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:34:13.228 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:13.228 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T04:34:13.228 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:34:13.228 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:34:13.228 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:34:13.325 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T04:34:13.373 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T04:34:13.429 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T04:34:13.617 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T04:34:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T04:34:13.740 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T04:34:13.837 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T04:34:13.840 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:34:13.867 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:34:13.867 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:13.867 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T04:34:13.867 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:34:13.867 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:34:13.867 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:34:13.977 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T04:34:14.029 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T04:34:14.061 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T04:34:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T04:34:14.099 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:34:14.122 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:34:14.122 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:14.123 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T04:34:14.123 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:34:14.123 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:34:14.123 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:34:14.526 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:34:14.554 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:34:14.558 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:34:14.581 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:34:14.582 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:14.582 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T04:34:14.582 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:34:14.582 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:34:14.582 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:34:14.585 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T04:34:14.648 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T04:34:14.677 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:34:14.697 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T04:34:14.762 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T04:34:14.825 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T04:34:15.010 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T04:34:15.059 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T04:34:15.137 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T04:34:15.233 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T04:34:15.372 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T04:34:15.418 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T04:34:15.450 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T04:34:15.478 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T04:34:15.729 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:34:15.771 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:34:15.849 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:34:15.876 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:34:15.907 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T04:34:15.978 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T04:34:16.008 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:34:16.732 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:34:16.763 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:34:16.864 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:34:17.024 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T04:34:17.056 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:34:17.057 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:34:17.081 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:34:17.081 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:17.081 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T04:34:17.081 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:34:17.081 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:34:17.081 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:34:17.093 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:34:17.132 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:34:17.144 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:34:17.250 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T04:34:17.306 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T04:34:18.054 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:34:18.085 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:34:18.190 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:34:18.221 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-19T04:34:18.226 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T04:34:18.251 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:34:18.275 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:34:18.276 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:18.276 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T04:34:18.276 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:34:18.276 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:34:18.276 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:34:18.327 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:34:18.348 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T04:34:18.350 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:34:18.350 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:18.350 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T04:34:18.350 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:34:18.375 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T04:34:18.379 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:34:18.403 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:34:18.403 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:34:18.403 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:18.403 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T04:34:18.403 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:34:18.403 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:34:18.404 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:34:18.426 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:34:18.426 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:18.426 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T04:34:18.426 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:34:18.426 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:34:18.426 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:34:18.433 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T04:34:18.463 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:34:18.475 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:34:18.515 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T04:34:18.560 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T04:34:18.572 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T04:34:18.592 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T04:34:18.628 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T04:34:18.667 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T04:34:18.728 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:34:18.752 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:34:18.752 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:18.752 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T04:34:18.752 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:34:18.753 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:34:18.753 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:34:19.027 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T04:34:19.134 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:34:19.279 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:34:19.319 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:34:19.427 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:34:19.466 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:34:19.549 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T04:34:19.583 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:34:19.607 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:34:19.607 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:19.607 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T04:34:19.607 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:34:19.607 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:34:19.607 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:34:19.650 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:34:19.673 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:34:19.673 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:19.673 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T04:34:19.673 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:34:19.688 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T04:34:19.726 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:34:19.749 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:34:19.750 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:19.750 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T04:34:19.750 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:34:19.750 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:34:19.750 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:34:19.758 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T04:34:19.789 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:34:19.906 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:34:19.940 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:34:20.014 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:34:20.041 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:34:20.060 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:34:20.082 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:34:20.083 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:34:20.083 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:20.084 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T04:34:20.084 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:34:20.084 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:34:20.084 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:34:20.293 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:34:20.471 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:34:20.664 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T04:34:20.785 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:34:20.891 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:34:21.173 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:34:21.337 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T04:34:21.459 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T04:34:21.504 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T04:34:21.559 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T04:34:21.649 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T04:34:21.711 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T04:34:21.750 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T04:34:21.805 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T04:34:21.843 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T04:34:21.877 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T04:34:21.965 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T04:34:22.020 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T04:34:22.064 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T04:34:22.119 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T04:34:22.199 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T04:34:22.234 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T04:34:22.278 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T04:34:22.306 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T04:34:22.394 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T04:34:22.450 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T04:34:22.539 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T04:34:22.697 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T04:34:22.760 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T04:34:23.299 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T04:34:23.340 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T04:34:23.381 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T04:34:23.398 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T04:34:23.413 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T04:34:23.440 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T04:34:23.494 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T04:34:23.532 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T04:34:23.590 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T04:34:23.623 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T04:34:23.629 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T04:34:23.670 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T04:34:23.692 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T04:34:23.700 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T04:34:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T04:34:23.785 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T04:34:23.861 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T04:34:23.933 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T04:34:23.991 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T04:34:24.083 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T04:34:24.125 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T04:34:24.170 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T04:34:24.227 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T04:34:24.269 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T04:34:24.314 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T04:34:24.364 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T04:34:24.400 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:34:24.441 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:34:24.462 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:34:24.533 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T04:34:24.588 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T04:34:24.622 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T04:34:24.640 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T04:34:24.663 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T04:34:24.713 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T04:34:24.754 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T04:34:24.756 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T04:34:24.781 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T04:34:24.801 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T04:34:24.851 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T04:34:24.889 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T04:34:24.914 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T04:34:24.962 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T04:34:25.003 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T04:34:25.033 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T04:34:25.034 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T04:34:25.062 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T04:34:25.109 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T04:34:25.188 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T04:34:25.230 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T04:34:25.274 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T04:34:25.310 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T04:34:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T04:34:25.350 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T04:34:25.389 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T04:34:25.404 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T04:34:25.435 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T04:34:25.445 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T04:34:25.477 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T04:34:25.526 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T04:34:25.528 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T04:34:25.557 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T04:34:25.571 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T04:34:25.584 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T04:34:25.603 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T04:34:25.616 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T04:34:25.616 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:34:25.701 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T04:34:25.718 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:34:25.718 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:34:25.762 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T04:34:25.779 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:34:25.801 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:34:25.806 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T04:34:25.815 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:34:25.826 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T04:34:25.832 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:34:25.832 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:34:25.836 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:34:25.929 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T04:34:25.996 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T04:34:26.028 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:34:26.028 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:34:26.112 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:34:26.177 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:34:26.656 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T04:34:26.738 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T04:34:26.778 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T04:34:26.860 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T04:34:26.864 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T04:34:26.900 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T04:34:26.921 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T04:34:26.925 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T04:34:26.951 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T04:34:26.951 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:34:26.954 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T04:34:27.035 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:34:27.036 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:34:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T04:34:27.119 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:34:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T04:34:27.144 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T04:34:27.150 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:34:27.150 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:34:27.200 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T04:34:27.304 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T04:34:27.338 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:34:27.339 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:34:27.348 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T04:34:27.414 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:34:27.477 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:34:27.582 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T04:34:27.621 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T04:34:27.713 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T04:34:28.499 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T04:34:28.632 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T04:34:30.202 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T04:34:30.323 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T04:34:30.362 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T04:34:30.395 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T04:34:30.739 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T04:34:30.790 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:34:30.902 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:34:30.938 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T04:34:30.986 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T04:34:31.513 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T04:34:31.543 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:34:31.644 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:34:31.670 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T04:34:31.897 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:34:31.898 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T04:34:31.898 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T04:34:31.898 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T04:34:31.898 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T04:34:31.898 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T04:34:31.898 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T04:34:31.898 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T04:34:31.898 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:34:31.898 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:34:31.898 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:34:31.898 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T04:34:31.898 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T04:34:31.898 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:34:31.899 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T04:34:31.899 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T04:34:31.899 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T04:34:31.899 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T04:34:31.899 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T04:34:31.899 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T04:34:31.899 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T04:34:31.899 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T04:34:31.899 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T04:34:31.899 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T04:34:31.899 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T04:34:31.899 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T04:34:31.899 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T04:34:31.900 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T04:34:31.900 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T04:34:31.900 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T04:34:31.911 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T04:34:31.912 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T04:34:31.912 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T04:34:31.912 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T04:34:31.912 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T04:34:31.912 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T04:34:31.912 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T04:34:31.912 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T04:34:31.912 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T04:34:31.912 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T04:34:31.912 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T04:34:31.912 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T04:34:31.912 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T04:34:31.913 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T04:34:31.913 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T04:34:31.913 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T04:34:31.913 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T04:34:31.913 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T04:34:31.913 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T04:34:31.913 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T04:34:31.913 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T04:34:31.913 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T04:34:31.913 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T04:34:31.913 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T04:34:31.913 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T04:34:31.914 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T04:34:31.914 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T04:34:31.914 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T04:34:31.914 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T04:34:31.914 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T04:34:31.914 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T04:34:31.914 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T04:34:31.914 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T04:34:31.914 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T04:34:31.914 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T04:34:31.914 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T04:34:31.914 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T04:34:31.914 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T04:34:31.914 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T04:34:31.915 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T04:34:31.915 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T04:34:31.915 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T04:34:31.915 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T04:34:31.915 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T04:34:31.915 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T04:34:31.915 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T04:34:31.915 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T04:34:31.915 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T04:34:31.915 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T04:34:31.915 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T04:34:31.915 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T04:34:31.915 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T04:34:31.916 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T04:34:31.916 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T04:34:31.916 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T04:34:31.916 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T04:34:31.916 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T04:34:31.916 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T04:34:31.916 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T04:34:31.916 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T04:34:31.916 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T04:34:31.916 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T04:34:31.916 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T04:34:31.916 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T04:34:31.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T04:34:31.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T04:34:31.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T04:34:31.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T04:34:31.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T04:34:31.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T04:34:31.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T04:34:31.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T04:34:31.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T04:34:31.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T04:34:31.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T04:34:31.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T04:34:31.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T04:34:31.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T04:34:31.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T04:34:31.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T04:34:31.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T04:34:31.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T04:34:31.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T04:34:31.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T04:34:31.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T04:34:31.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T04:34:31.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T04:34:31.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T04:34:31.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T04:34:31.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T04:34:31.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T04:34:31.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T04:34:31.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T04:34:31.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T04:34:31.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T04:34:31.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T04:34:31.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T04:34:31.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T04:34:31.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T04:34:31.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T04:34:31.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T04:34:31.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T04:34:31.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T04:34:31.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T04:34:31.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T04:34:31.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T04:34:31.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T04:34:31.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T04:34:31.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T04:34:31.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T04:34:31.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T04:34:31.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T04:34:31.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T04:34:31.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T04:34:31.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T04:34:31.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T04:34:31.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T04:34:31.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T04:34:31.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T04:34:31.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T04:34:31.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T04:34:31.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T04:34:31.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T04:34:31.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T04:34:31.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T04:34:31.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T04:34:31.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T04:34:31.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T04:34:31.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T04:34:31.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T04:34:31.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T04:34:31.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T04:34:31.922 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T04:34:31.922 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T04:34:33.335 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:34:33.335 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T04:34:33.335 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T04:34:33.335 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T04:34:33.335 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T04:34:33.335 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T04:34:33.335 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T04:34:33.335 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T04:34:33.335 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:34:33.335 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:34:33.336 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:34:33.336 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T04:34:33.336 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T04:34:33.336 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:34:33.336 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T04:34:33.336 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T04:34:33.336 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T04:34:33.336 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T04:34:33.336 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T04:34:33.336 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T04:34:33.336 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T04:34:33.336 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T04:34:33.336 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T04:34:33.337 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T04:34:33.337 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T04:34:33.337 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T04:34:33.337 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T04:34:33.337 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T04:34:33.337 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T04:34:33.337 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T04:34:33.337 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T04:34:33.337 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T04:34:33.337 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T04:34:33.338 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T04:34:33.338 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T04:34:33.338 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T04:34:33.338 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T04:34:33.338 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T04:34:33.338 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T04:34:33.338 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T04:34:33.338 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T04:34:33.338 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T04:34:33.338 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T04:34:33.338 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T04:34:33.338 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T04:34:33.339 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T04:34:33.339 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T04:34:33.339 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T04:34:33.339 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T04:34:33.339 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T04:34:33.339 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T04:34:33.339 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T04:34:33.339 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T04:34:33.339 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T04:34:33.339 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T04:34:33.339 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T04:34:33.339 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T04:34:33.339 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T04:34:33.339 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T04:34:33.340 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T04:34:33.340 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T04:34:33.340 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T04:34:33.340 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T04:34:33.340 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T04:34:33.340 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T04:34:33.340 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T04:34:33.340 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T04:34:33.340 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T04:34:33.340 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T04:34:33.340 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T04:34:33.340 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T04:34:33.340 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T04:34:33.340 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T04:34:33.341 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T04:34:33.341 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T04:34:33.341 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T04:34:33.341 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T04:34:33.341 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T04:34:33.341 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T04:34:33.342 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T04:34:33.342 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T04:34:33.342 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T04:34:33.342 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T04:34:33.342 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T04:34:33.342 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T04:34:33.343 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T04:34:33.343 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T04:34:33.343 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T04:34:33.343 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T04:34:33.343 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T04:34:33.343 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T04:34:33.343 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T04:34:33.343 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T04:34:33.343 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T04:34:33.343 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T04:34:33.343 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T04:34:33.343 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T04:34:33.343 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T04:34:33.343 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T04:34:33.343 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T04:34:33.344 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T04:34:33.344 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T04:34:33.344 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T04:34:33.344 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T04:34:33.344 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T04:34:33.344 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T04:34:33.345 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T04:34:33.345 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T04:34:33.345 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T04:34:33.345 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T04:34:33.345 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T04:34:33.345 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T04:34:33.345 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T04:34:33.345 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T04:34:33.345 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T04:34:33.346 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T04:34:33.346 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T04:34:33.346 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T04:34:33.346 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T04:34:33.346 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T04:34:33.346 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T04:34:33.346 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T04:34:33.346 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T04:34:33.346 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T04:34:33.346 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T04:34:33.346 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T04:34:33.346 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T04:34:33.346 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T04:34:33.346 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T04:34:33.347 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T04:34:33.347 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T04:34:33.347 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T04:34:33.347 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T04:34:33.347 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T04:34:33.347 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T04:34:33.347 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T04:34:33.347 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T04:34:33.347 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T04:34:33.347 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T04:34:33.347 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T04:34:33.347 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T04:34:33.347 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T04:34:33.347 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T04:34:33.347 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T04:34:33.348 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T04:34:33.348 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T04:34:33.348 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T04:34:33.348 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T04:34:33.348 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T04:34:33.348 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T04:34:33.348 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T04:34:33.348 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T04:34:33.348 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T04:34:33.348 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T04:34:33.348 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T04:34:33.348 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T04:34:33.348 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T04:34:33.348 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T04:34:33.348 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T04:34:33.349 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T04:34:33.349 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T04:34:33.349 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T04:34:33.349 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T04:34:33.349 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T04:34:34.242 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:34:34.290 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:34:34.541 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:34:34.541 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:34:34.541 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2024-09-19T04:34:34.542 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T04:34:34.542 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T04:34:34.542 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T04:34:34.542 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T04:34:34.542 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.542 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.542 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T04:34:34.542 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T04:34:34.542 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-09-19T04:34:34.542 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T04:34:34.542 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T04:34:34.542 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T04:34:34.542 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T04:34:34.543 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.543 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.543 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.543 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.543 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:34.543 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.543 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.543 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.543 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:34.543 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:34.543 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:34.543 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:34.544 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:34.544 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.544 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.544 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:34.544 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.544 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.544 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.544 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:34.544 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:34.544 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.544 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.544 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:34.544 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T04:34:34.544 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T04:34:34.545 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T04:34:34.545 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T04:34:34.545 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T04:34:34.545 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T04:34:34.545 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T04:34:34.545 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T04:34:34.545 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T04:34:34.545 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T04:34:34.545 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T04:34:34.545 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.545 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.545 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.546 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T04:34:34.546 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T04:34:34.546 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T04:34:34.546 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T04:34:34.546 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T04:34:34.546 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.546 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.546 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T04:34:34.546 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.546 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T04:34:34.546 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T04:34:34.547 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T04:34:34.547 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T04:34:34.547 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T04:34:34.547 INFO:teuthology.orchestra.run.smithi183.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T04:34:34.547 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T04:34:34.547 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T04:34:34.547 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T04:34:34.547 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T04:34:34.547 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T04:34:34.547 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T04:34:34.547 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T04:34:34.547 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T04:34:34.547 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T04:34:34.547 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T04:34:34.548 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T04:34:34.548 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T04:34:34.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T04:34:34.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T04:34:34.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T04:34:34.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T04:34:34.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T04:34:34.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T04:34:34.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T04:34:34.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T04:34:34.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T04:34:34.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T04:34:34.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T04:34:34.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T04:34:34.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T04:34:34.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T04:34:34.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T04:34:34.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T04:34:34.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T04:34:34.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T04:34:34.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T04:34:34.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T04:34:34.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T04:34:34.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T04:34:34.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T04:34:34.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T04:34:34.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T04:34:34.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T04:34:34.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T04:34:34.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T04:34:34.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T04:34:34.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T04:34:34.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T04:34:34.551 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T04:34:34.551 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T04:34:34.551 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T04:34:34.551 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T04:34:34.551 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T04:34:34.551 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T04:34:34.551 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T04:34:34.551 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T04:34:34.551 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T04:34:34.551 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T04:34:34.551 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T04:34:34.551 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T04:34:34.551 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T04:34:34.551 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T04:34:34.552 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.552 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.552 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T04:34:34.552 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T04:34:34.552 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T04:34:34.552 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.552 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T04:34:34.552 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T04:34:34.552 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T04:34:34.552 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T04:34:34.552 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T04:34:34.552 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T04:34:34.552 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T04:34:34.553 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T04:34:34.553 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T04:34:34.553 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T04:34:34.553 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T04:34:34.553 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T04:34:34.553 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T04:34:34.553 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T04:34:34.553 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T04:34:34.553 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.553 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.553 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:34.553 INFO:teuthology.orchestra.run.smithi183.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T04:34:34.553 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T04:34:34.554 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T04:34:34.554 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T04:34:34.554 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T04:34:34.554 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T04:34:34.554 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T04:34:34.554 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:34:34.554 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-09-19T04:34:34.927 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:34:34.940 DEBUG:teuthology.parallel:result is None 2024-09-19T04:34:34.978 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:34:35.106 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:34:35.106 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T04:34:35.106 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T04:34:35.106 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:34:35.145 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:34:35.771 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:34:35.772 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:34:35.772 INFO:teuthology.orchestra.run.smithi193.stdout:Upgraded: 2024-09-19T04:34:35.772 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T04:34:35.772 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T04:34:35.772 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T04:34:35.772 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T04:34:35.772 INFO:teuthology.orchestra.run.smithi193.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.772 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.772 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T04:34:35.772 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T04:34:35.772 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-19T04:34:35.772 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T04:34:35.773 INFO:teuthology.orchestra.run.smithi193.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T04:34:35.773 INFO:teuthology.orchestra.run.smithi193.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T04:34:35.773 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T04:34:35.773 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.773 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.773 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.773 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.773 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:35.773 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.773 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.773 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.773 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:35.773 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:35.774 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:35.774 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:35.774 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:35.774 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.774 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.774 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:35.774 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.774 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.774 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.774 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:35.774 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:35.774 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.774 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.775 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:34:35.775 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T04:34:35.775 INFO:teuthology.orchestra.run.smithi193.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T04:34:35.775 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T04:34:35.775 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T04:34:35.775 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T04:34:35.775 INFO:teuthology.orchestra.run.smithi193.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T04:34:35.775 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T04:34:35.775 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T04:34:35.775 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T04:34:35.775 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T04:34:35.775 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T04:34:35.775 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.775 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.776 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.776 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T04:34:35.776 INFO:teuthology.orchestra.run.smithi193.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T04:34:35.776 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T04:34:35.776 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T04:34:35.776 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T04:34:35.776 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.776 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.776 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T04:34:35.776 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.777 INFO:teuthology.orchestra.run.smithi193.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T04:34:35.777 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T04:34:35.777 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T04:34:35.777 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T04:34:35.777 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T04:34:35.777 INFO:teuthology.orchestra.run.smithi193.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T04:34:35.777 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T04:34:35.777 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T04:34:35.777 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T04:34:35.777 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T04:34:35.777 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T04:34:35.777 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T04:34:35.777 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T04:34:35.778 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T04:34:35.778 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T04:34:35.778 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T04:34:35.778 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T04:34:35.778 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T04:34:35.778 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T04:34:35.778 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T04:34:35.778 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T04:34:35.778 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T04:34:35.778 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T04:34:35.778 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.778 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.778 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.778 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T04:34:35.779 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T04:34:35.779 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T04:34:35.779 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T04:34:35.779 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T04:34:35.779 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T04:34:35.779 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T04:34:35.779 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T04:34:35.779 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T04:34:35.779 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T04:34:35.779 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T04:34:35.779 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T04:34:35.779 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T04:34:35.779 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T04:34:35.779 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T04:34:35.780 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T04:34:35.780 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T04:34:35.780 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T04:34:35.780 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T04:34:35.780 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T04:34:35.780 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T04:34:35.780 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T04:34:35.780 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T04:34:35.780 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T04:34:35.780 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T04:34:35.780 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T04:34:35.780 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T04:34:35.780 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T04:34:35.781 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T04:34:35.781 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T04:34:35.781 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T04:34:35.781 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T04:34:35.781 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T04:34:35.781 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T04:34:35.781 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T04:34:35.781 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T04:34:35.781 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T04:34:35.781 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T04:34:35.781 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T04:34:35.781 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T04:34:35.781 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.781 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.782 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T04:34:35.782 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T04:34:35.782 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T04:34:35.782 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.782 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T04:34:35.782 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T04:34:35.782 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T04:34:35.782 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T04:34:35.782 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T04:34:35.782 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T04:34:35.782 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T04:34:35.782 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T04:34:35.783 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T04:34:35.783 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T04:34:35.783 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T04:34:35.783 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T04:34:35.783 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T04:34:35.783 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T04:34:35.783 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T04:34:35.783 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.783 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.783 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:34:35.783 INFO:teuthology.orchestra.run.smithi193.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T04:34:35.783 INFO:teuthology.orchestra.run.smithi193.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T04:34:35.783 INFO:teuthology.orchestra.run.smithi193.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T04:34:35.783 INFO:teuthology.orchestra.run.smithi193.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T04:34:35.783 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T04:34:35.784 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T04:34:35.784 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T04:34:35.784 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:34:35.784 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-19T04:34:36.209 DEBUG:teuthology.parallel:result is None 2024-09-19T04:34:47.474 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:34:47.475 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-09-19T04:34:47.475 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-09-19T04:34:47.475 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-09-19T04:34:47.475 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-09-19T04:34:47.475 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-09-19T04:34:47.475 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-09-19T04:34:47.475 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-09-19T04:34:47.475 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:34:47.689 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:34:47.724 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:34:47.724 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:47.725 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T04:34:47.725 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:34:49.283 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:34:49.341 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:34:49.341 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:49.341 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T04:34:49.341 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:34:49.341 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:34:49.341 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:34:49.584 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:34:49.634 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:34:49.634 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:49.634 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T04:34:49.634 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:34:49.634 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:34:49.634 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:34:50.109 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:34:50.159 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:34:50.159 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:50.159 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T04:34:50.160 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:34:50.160 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:34:50.160 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:34:50.301 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T04:34:50.361 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T04:34:50.424 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T04:34:50.647 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T04:34:50.713 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T04:34:50.799 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T04:34:50.908 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T04:34:51.093 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T04:34:51.147 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T04:34:51.185 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T04:34:51.211 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T04:34:51.692 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:34:51.727 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:34:51.768 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T04:34:51.836 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T04:34:51.891 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:34:53.103 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:34:53.143 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:34:54.214 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:34:54.243 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:34:54.380 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:34:54.549 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T04:34:54.578 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:34:54.630 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:34:54.630 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:54.630 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T04:34:54.631 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:34:54.631 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:34:54.631 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:34:54.687 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:34:54.713 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:34:54.837 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T04:34:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T04:34:55.887 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T04:34:55.922 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:34:55.974 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:34:55.974 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:55.974 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T04:34:55.974 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:34:55.974 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:34:55.974 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:34:56.023 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:34:56.072 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:34:56.072 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:56.072 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T04:34:56.072 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:34:56.124 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:34:56.174 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:34:56.174 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:56.174 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T04:34:56.174 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:34:56.174 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:34:56.175 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:34:56.508 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:34:56.559 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:34:56.559 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:34:56.559 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T04:34:56.559 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:34:56.559 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:34:56.559 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:35:01.492 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T04:35:01.537 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T04:35:01.580 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T04:35:01.609 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T04:35:01.645 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T04:35:01.887 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T04:35:01.933 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T04:35:01.962 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T04:35:01.994 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T04:35:02.038 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T04:35:02.762 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:35:02.810 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:35:02.852 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:35:04.134 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T04:35:04.234 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T04:35:04.289 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T04:35:04.397 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T04:35:04.436 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T04:35:04.461 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T04:35:04.498 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T04:35:04.498 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:35:04.605 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:35:04.605 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:35:04.714 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:35:04.739 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T04:35:04.754 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:35:04.754 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:35:04.991 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:35:04.991 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:35:05.100 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:35:05.173 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:35:05.387 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:35:05.387 INFO:teuthology.orchestra.run.smithi123.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-19T04:35:05.387 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:35:11.095 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T04:35:11.095 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T04:35:11.096 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T04:35:11.096 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T04:35:11.096 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T04:35:11.096 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T04:35:11.096 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T04:35:11.096 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:35:11.096 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:35:11.096 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:35:11.096 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T04:35:11.096 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T04:35:11.096 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:35:11.097 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T04:35:11.097 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T04:35:11.097 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T04:35:11.097 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T04:35:11.097 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T04:35:11.097 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T04:35:11.097 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T04:35:11.097 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T04:35:11.097 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T04:35:11.098 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T04:35:11.098 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T04:35:11.098 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T04:35:11.098 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T04:35:11.098 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T04:35:11.109 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T04:35:11.109 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T04:35:11.109 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T04:35:11.110 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T04:35:11.110 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T04:35:11.110 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T04:35:11.110 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T04:35:11.110 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T04:35:11.110 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T04:35:11.110 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T04:35:11.110 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T04:35:11.110 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T04:35:11.110 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T04:35:11.110 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T04:35:11.110 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T04:35:11.110 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T04:35:11.110 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T04:35:11.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T04:35:11.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T04:35:11.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T04:35:11.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T04:35:11.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T04:35:11.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T04:35:11.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T04:35:11.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T04:35:11.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T04:35:11.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T04:35:11.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T04:35:11.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T04:35:11.112 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T04:35:11.112 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T04:35:11.112 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T04:35:11.112 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T04:35:11.112 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T04:35:11.112 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T04:35:11.112 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T04:35:11.112 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T04:35:11.112 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T04:35:11.112 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T04:35:11.112 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T04:35:11.112 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T04:35:11.112 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T04:35:11.113 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T04:35:11.113 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T04:35:11.113 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T04:35:11.113 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T04:35:11.113 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T04:35:11.113 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T04:35:11.113 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T04:35:11.113 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T04:35:11.113 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T04:35:11.113 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T04:35:11.113 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T04:35:11.113 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T04:35:11.113 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T04:35:11.114 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T04:35:11.114 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T04:35:11.114 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T04:35:11.114 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T04:35:11.114 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T04:35:11.114 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T04:35:11.114 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T04:35:11.114 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T04:35:11.114 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T04:35:11.114 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T04:35:11.114 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T04:35:11.114 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T04:35:11.114 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T04:35:11.115 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T04:35:11.115 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T04:35:11.115 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T04:35:11.115 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T04:35:11.115 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T04:35:11.115 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T04:35:11.115 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T04:35:11.115 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T04:35:11.115 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T04:35:11.115 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T04:35:11.115 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T04:35:11.115 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T04:35:11.115 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T04:35:11.116 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T04:35:11.116 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T04:35:11.116 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T04:35:11.116 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T04:35:11.116 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T04:35:11.116 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T04:35:11.116 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T04:35:11.116 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T04:35:11.116 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T04:35:11.116 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T04:35:11.116 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T04:35:11.116 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T04:35:11.116 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T04:35:11.116 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T04:35:11.117 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T04:35:11.117 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T04:35:11.117 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T04:35:11.117 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T04:35:11.117 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T04:35:11.117 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T04:35:11.117 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T04:35:11.117 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T04:35:11.117 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T04:35:11.117 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T04:35:11.117 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T04:35:11.117 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T04:35:11.117 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T04:35:11.118 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T04:35:11.118 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T04:35:11.118 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T04:35:11.118 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T04:35:11.118 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T04:35:11.118 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T04:35:11.118 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T04:35:11.118 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T04:35:11.118 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T04:35:11.118 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T04:35:11.118 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T04:35:11.118 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T04:35:11.118 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T04:35:11.118 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T04:35:11.119 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T04:35:11.119 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T04:35:11.119 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T04:35:11.119 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T04:35:11.119 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T04:35:11.119 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T04:35:11.119 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T04:35:11.119 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T04:35:11.119 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T04:35:11.119 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T04:35:11.119 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T04:35:11.119 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T04:35:11.119 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T04:35:11.119 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T04:35:13.245 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:35:13.245 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:35:13.246 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-09-19T04:35:13.246 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T04:35:13.246 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T04:35:13.246 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T04:35:13.246 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T04:35:13.246 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.246 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.246 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T04:35:13.246 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T04:35:13.246 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-19T04:35:13.246 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T04:35:13.247 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T04:35:13.247 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T04:35:13.247 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T04:35:13.247 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.247 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.247 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.247 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.247 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:35:13.247 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.247 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.247 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.247 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:35:13.247 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:35:13.248 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:35:13.248 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:35:13.248 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:35:13.248 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.248 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.248 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:35:13.248 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.248 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.248 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.248 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:35:13.248 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:35:13.248 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.248 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.249 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:35:13.249 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T04:35:13.249 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T04:35:13.249 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T04:35:13.249 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T04:35:13.249 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T04:35:13.249 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T04:35:13.249 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T04:35:13.249 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T04:35:13.249 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T04:35:13.249 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T04:35:13.249 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T04:35:13.250 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.250 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.250 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.250 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T04:35:13.250 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T04:35:13.250 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T04:35:13.250 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T04:35:13.250 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T04:35:13.250 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.250 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.250 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T04:35:13.250 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.250 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T04:35:13.250 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T04:35:13.251 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T04:35:13.251 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T04:35:13.251 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T04:35:13.251 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T04:35:13.251 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T04:35:13.251 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T04:35:13.251 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T04:35:13.251 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T04:35:13.251 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T04:35:13.251 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T04:35:13.251 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T04:35:13.251 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T04:35:13.251 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T04:35:13.252 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T04:35:13.252 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T04:35:13.252 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T04:35:13.252 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T04:35:13.252 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T04:35:13.252 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T04:35:13.252 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T04:35:13.252 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T04:35:13.252 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.252 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.252 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.252 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T04:35:13.253 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T04:35:13.253 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T04:35:13.253 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T04:35:13.253 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T04:35:13.253 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T04:35:13.253 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T04:35:13.253 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T04:35:13.253 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T04:35:13.253 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T04:35:13.253 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T04:35:13.253 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T04:35:13.253 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T04:35:13.253 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T04:35:13.254 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T04:35:13.254 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T04:35:13.254 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T04:35:13.254 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T04:35:13.254 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T04:35:13.254 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T04:35:13.254 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T04:35:13.254 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T04:35:13.254 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T04:35:13.254 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T04:35:13.254 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T04:35:13.254 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T04:35:13.254 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T04:35:13.255 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T04:35:13.255 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T04:35:13.255 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T04:35:13.255 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T04:35:13.255 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T04:35:13.255 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T04:35:13.255 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T04:35:13.255 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T04:35:13.255 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T04:35:13.255 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T04:35:13.255 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T04:35:13.255 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T04:35:13.255 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T04:35:13.255 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.256 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.256 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T04:35:13.256 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T04:35:13.256 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T04:35:13.256 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.256 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T04:35:13.256 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T04:35:13.256 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T04:35:13.256 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T04:35:13.256 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T04:35:13.256 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T04:35:13.256 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T04:35:13.257 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T04:35:13.257 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T04:35:13.257 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T04:35:13.257 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T04:35:13.257 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T04:35:13.257 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T04:35:13.257 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T04:35:13.257 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T04:35:13.257 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.257 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.257 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:35:13.257 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T04:35:13.258 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T04:35:13.258 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T04:35:13.258 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T04:35:13.258 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T04:35:13.258 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T04:35:13.258 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T04:35:13.258 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:35:13.258 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-19T04:35:13.872 DEBUG:teuthology.parallel:result is None 2024-09-19T04:35:13.873 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:35:14.015 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T04:35:14.061 INFO:teuthology.orchestra.run.smithi123.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T04:35:14.061 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T04:35:14.062 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T04:35:14.064 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:35:14.203 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T04:35:14.229 INFO:teuthology.orchestra.run.smithi183.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T04:35:14.230 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T04:35:14.230 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T04:35:14.232 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:35:14.369 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T04:35:14.395 INFO:teuthology.orchestra.run.smithi193.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T04:35:14.395 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T04:35:14.395 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T04:35:14.397 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T04:35:14.398 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:35:14.398 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T04:35:14.456 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-19T04:35:14.456 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T04:35:14.489 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-19T04:35:14.489 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T04:35:14.522 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T04:35:14.522 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:35:14.522 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T04:35:14.583 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T04:35:14.679 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-19T04:35:14.680 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T04:35:14.710 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T04:35:14.778 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-19T04:35:14.778 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T04:35:14.808 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T04:35:14.876 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T04:35:14.876 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:35:14.876 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T04:35:14.934 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T04:35:15.033 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-19T04:35:15.033 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T04:35:15.059 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T04:35:15.125 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-19T04:35:15.125 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T04:35:15.150 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T04:35:15.215 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T04:35:15.215 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:35:15.215 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T04:35:15.281 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T04:35:15.378 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-19T04:35:15.378 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T04:35:15.404 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T04:35:15.468 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-19T04:35:15.469 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T04:35:15.494 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T04:35:15.559 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T04:35:15.652 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-19T04:35:15.652 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:35:15.653 INFO:tasks.cephadm:Cluster fsid is 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:35:15.653 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T04:35:15.653 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.123', 'mon.b': '172.21.15.183', 'mon.c': '172.21.15.193'} 2024-09-19T04:35:15.653 INFO:tasks.cephadm:First mon is mon.a on smithi123 2024-09-19T04:35:15.653 INFO:tasks.cephadm:First mgr is x 2024-09-19T04:35:15.653 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T04:35:15.654 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname $(hostname -s) 2024-09-19T04:35:15.711 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname $(hostname -s) 2024-09-19T04:35:15.736 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname $(hostname -s) 2024-09-19T04:35:15.761 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T04:35:15.762 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:35:15.905 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T04:35:16.045 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:35:16.047 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T04:35:16.047 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T04:35:16.047 DEBUG:teuthology.orchestra.run.smithi123:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T04:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 04:35 /home/ubuntu/cephtest/cephadm 2024-09-19T04:35:16.521 DEBUG:teuthology.orchestra.run.smithi183:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T04:35:16.899 INFO:teuthology.orchestra.run.smithi183.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 04:35 /home/ubuntu/cephtest/cephadm 2024-09-19T04:35:16.899 DEBUG:teuthology.orchestra.run.smithi193:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T04:35:17.475 INFO:teuthology.orchestra.run.smithi193.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 04:35 /home/ubuntu/cephtest/cephadm 2024-09-19T04:35:17.475 DEBUG:teuthology.orchestra.run.smithi123:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T04:35:17.517 DEBUG:teuthology.orchestra.run.smithi183:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T04:35:17.537 DEBUG:teuthology.orchestra.run.smithi193:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T04:35:17.572 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T04:35:17.572 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T04:35:17.576 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T04:35:17.580 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T04:35:19.047 INFO:teuthology.orchestra.run.smithi193.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:35:19.138 INFO:teuthology.orchestra.run.smithi183.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:35:19.463 INFO:teuthology.orchestra.run.smithi123.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:35:51.850 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:35:51.850 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:35:51.850 INFO:teuthology.orchestra.run.smithi123.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T04:35:51.850 INFO:teuthology.orchestra.run.smithi123.stdout: "repo_digests": [ 2024-09-19T04:35:51.850 INFO:teuthology.orchestra.run.smithi123.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T04:35:51.851 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T04:35:51.851 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:35:56.671 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-19T04:35:56.671 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:35:56.671 INFO:teuthology.orchestra.run.smithi193.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T04:35:56.671 INFO:teuthology.orchestra.run.smithi193.stdout: "repo_digests": [ 2024-09-19T04:35:56.671 INFO:teuthology.orchestra.run.smithi193.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T04:35:56.671 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2024-09-19T04:35:56.671 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-19T04:36:10.559 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-19T04:36:10.559 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:36:10.559 INFO:teuthology.orchestra.run.smithi183.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T04:36:10.559 INFO:teuthology.orchestra.run.smithi183.stdout: "repo_digests": [ 2024-09-19T04:36:10.559 INFO:teuthology.orchestra.run.smithi183.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T04:36:10.559 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-19T04:36:10.559 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-19T04:36:10.583 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph 2024-09-19T04:36:10.652 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph 2024-09-19T04:36:10.679 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph 2024-09-19T04:36:10.713 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /etc/ceph 2024-09-19T04:36:10.775 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /etc/ceph 2024-09-19T04:36:10.800 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /etc/ceph 2024-09-19T04:36:10.829 INFO:tasks.cephadm:Writing seed config... 2024-09-19T04:36:10.830 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T04:36:10.830 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T04:36:10.830 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T04:36:10.830 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T04:36:10.830 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T04:36:10.830 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-19T04:36:10.831 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T04:36:10.831 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T04:36:10.831 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T04:36:10.831 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-19T04:36:10.831 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-19T04:36:10.831 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-19T04:36:10.831 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-19T04:36:10.831 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T04:36:10.831 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T04:36:10.831 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T04:36:10.831 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T04:36:10.831 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-19T04:36:10.831 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-19T04:36:10.832 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T04:36:10.832 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T04:36:10.832 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T04:36:10.832 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-19T04:36:10.832 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-19T04:36:10.832 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T04:36:10.832 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T04:36:10.832 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T04:36:10.832 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T04:36:10.832 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T04:36:10.832 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T04:36:10.832 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T04:36:10.832 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T04:36:10.833 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T04:36:10.833 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-19T04:36:10.833 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-19T04:36:10.833 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-19T04:36:10.833 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-19T04:36:10.833 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-19T04:36:10.833 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T04:36:10.833 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T04:36:10.833 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-19T04:36:10.833 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-19T04:36:10.833 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-19T04:36:10.833 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-19T04:36:10.833 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-19T04:36:10.834 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-19T04:36:10.834 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T04:36:10.834 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T04:36:10.834 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-19T04:36:10.835 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:36:10.835 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T04:36:10.873 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 911fa052-7640-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T04:36:10.873 DEBUG:teuthology.orchestra.run.smithi123:mon.a> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mon.a.service 2024-09-19T04:36:10.917 DEBUG:teuthology.orchestra.run.smithi123:mgr.x> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mgr.x.service 2024-09-19T04:36:10.961 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T04:36:10.962 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid 911fa052-7640-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.123 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:36:11.217 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:36:11.217 INFO:teuthology.orchestra.run.smithi123.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', '911fa052-7640-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.123', '--skip-admin-label'] 2024-09-19T04:36:11.217 INFO:teuthology.orchestra.run.smithi123.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T04:36:11.217 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying podman|docker is present... 2024-09-19T04:36:11.262 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stdout 5.2.2 2024-09-19T04:36:11.262 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying lvm2 is present... 2024-09-19T04:36:11.262 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying time synchronization is in place... 2024-09-19T04:36:11.276 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T04:36:11.276 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T04:36:11.288 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T04:36:11.288 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-09-19T04:36:11.302 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout enabled 2024-09-19T04:36:11.314 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout active 2024-09-19T04:36:11.315 INFO:teuthology.orchestra.run.smithi123.stdout:Unit chronyd.service is enabled and running 2024-09-19T04:36:11.315 INFO:teuthology.orchestra.run.smithi123.stdout:Repeating the final host check... 2024-09-19T04:36:11.357 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stdout 5.2.2 2024-09-19T04:36:11.357 INFO:teuthology.orchestra.run.smithi123.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T04:36:11.358 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl is present 2024-09-19T04:36:11.358 INFO:teuthology.orchestra.run.smithi123.stdout:lvcreate is present 2024-09-19T04:36:11.370 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T04:36:11.370 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T04:36:11.382 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T04:36:11.383 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-09-19T04:36:11.396 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout enabled 2024-09-19T04:36:11.409 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout active 2024-09-19T04:36:11.409 INFO:teuthology.orchestra.run.smithi123.stdout:Unit chronyd.service is enabled and running 2024-09-19T04:36:11.409 INFO:teuthology.orchestra.run.smithi123.stdout:Host looks OK 2024-09-19T04:36:11.409 INFO:teuthology.orchestra.run.smithi123.stdout:Cluster fsid: 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:36:11.409 INFO:teuthology.orchestra.run.smithi123.stdout:Acquiring lock 140555183740720 on /run/cephadm/911fa052-7640-11ef-bceb-c7b262605968.lock 2024-09-19T04:36:11.409 INFO:teuthology.orchestra.run.smithi123.stdout:Lock 140555183740720 acquired on /run/cephadm/911fa052-7640-11ef-bceb-c7b262605968.lock 2024-09-19T04:36:11.410 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying IP 172.21.15.123 port 3300 ... 2024-09-19T04:36:11.410 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying IP 172.21.15.123 port 6789 ... 2024-09-19T04:36:11.410 INFO:teuthology.orchestra.run.smithi123.stdout:Base mon IP(s) is [172.21.15.123:3300, 172.21.15.123:6789], mon addrv is [v2:172.21.15.123:3300,v1:172.21.15.123:6789] 2024-09-19T04:36:11.417 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.123 metric 100 2024-09-19T04:36:11.417 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.123 metric 100 2024-09-19T04:36:11.422 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T04:36:11.423 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-19T04:36:11.423 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-19T04:36:11.428 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T04:36:11.428 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T04:36:11.429 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T04:36:11.429 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-19T04:36:11.429 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link noprefixroute 2024-09-19T04:36:11.429 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T04:36:11.429 INFO:teuthology.orchestra.run.smithi123.stdout:Mon IP `172.21.15.123` is in CIDR network `172.21.0.0/20` 2024-09-19T04:36:11.429 INFO:teuthology.orchestra.run.smithi123.stdout:Mon IP `172.21.15.123` is in CIDR network `172.21.0.0/20` 2024-09-19T04:36:11.430 INFO:teuthology.orchestra.run.smithi123.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T04:36:11.430 INFO:teuthology.orchestra.run.smithi123.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T04:36:11.431 INFO:teuthology.orchestra.run.smithi123.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:36:11.909 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T04:36:11.909 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:36:11.909 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T04:36:11.909 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T04:36:11.909 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T04:36:11.909 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T04:36:11.909 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T04:36:13.003 INFO:teuthology.orchestra.run.smithi123.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T04:36:13.003 INFO:teuthology.orchestra.run.smithi123.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T04:36:13.003 INFO:teuthology.orchestra.run.smithi123.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T04:36:13.974 INFO:teuthology.orchestra.run.smithi123.stdout:stat: stdout 167 167 2024-09-19T04:36:13.974 INFO:teuthology.orchestra.run.smithi123.stdout:Creating initial keys... 2024-09-19T04:36:14.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph-authtool: stdout AQA+qutmrqEzFRAAexUl8HrnKWBi7LpobRLeqg== 2024-09-19T04:36:16.028 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph-authtool: stdout AQA/qutmBwgLFxAA3TqBMW8MJz5H+G854tFlYw== 2024-09-19T04:36:16.985 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph-authtool: stdout AQBAqutmaZ/zFxAAoN0aepV0HHdahOosENNfNw== 2024-09-19T04:36:16.986 INFO:teuthology.orchestra.run.smithi123.stdout:Creating initial monmap... 2024-09-19T04:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T04:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T04:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T04:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool for a [v2:172.21.15.123:3300,v1:172.21.15.123:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T04:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout:setting min_mon_release = quincy 2024-09-19T04:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: set fsid to 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T04:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout:Creating mon... 2024-09-19T04:36:19.046 INFO:teuthology.orchestra.run.smithi123.stdout:create mon.a on 2024-09-19T04:36:19.363 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T04:36:19.579 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T04:36:19.867 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-911fa052-7640-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-911fa052-7640-11ef-bceb-c7b262605968.target. 2024-09-19T04:36:19.867 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-911fa052-7640-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-911fa052-7640-11ef-bceb-c7b262605968.target. 2024-09-19T04:36:20.189 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-911fa052-7640-11ef-bceb-c7b262605968@mon.a 2024-09-19T04:36:20.189 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Failed to reset failed state of unit ceph-911fa052-7640-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-911fa052-7640-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-19T04:36:20.455 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-911fa052-7640-11ef-bceb-c7b262605968.target.wants/ceph-911fa052-7640-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-911fa052-7640-11ef-bceb-c7b262605968@.service. 2024-09-19T04:36:21.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:21 smithi123 podman[30121]: 2024-09-19 04:36:21.024576221 +0000 UTC m=+0.353176815 container start 8857b33da95400e191311d2e134f2ad84e14488f2d1428a29bc3741d8878441c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T04:36:21.178 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:36:21.178 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout disabled 2024-09-19T04:36:21.192 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:36:21.192 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-09-19T04:36:21.192 INFO:teuthology.orchestra.run.smithi123.stdout:firewalld.service is not enabled 2024-09-19T04:36:21.192 INFO:teuthology.orchestra.run.smithi123.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T04:36:21.192 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mon to start... 2024-09-19T04:36:21.193 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mon... 2024-09-19T04:36:21.344 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:21 smithi123 ceph-mon[30140]: mkfs 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:36:21.344 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:21 smithi123 ceph-mon[30140]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T04:36:21.344 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:21 smithi123 bash[30121]: 8857b33da95400e191311d2e134f2ad84e14488f2d1428a29bc3741d8878441c 2024-09-19T04:36:21.345 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:21 smithi123 systemd[1]: Started Ceph mon.a for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:36:22.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:22 smithi123 ceph-mon[30140]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T04:36:22.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:22 smithi123 ceph-mon[30140]: monmap epoch 1 2024-09-19T04:36:22.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:22 smithi123 ceph-mon[30140]: fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:36:22.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:22 smithi123 ceph-mon[30140]: last_changed 2024-09-19T04:36:17.413126+0000 2024-09-19T04:36:22.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:22 smithi123 ceph-mon[30140]: created 2024-09-19T04:36:17.413126+0000 2024-09-19T04:36:22.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:22 smithi123 ceph-mon[30140]: min_mon_release 19 (squid) 2024-09-19T04:36:22.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:22 smithi123 ceph-mon[30140]: election_strategy: 1 2024-09-19T04:36:22.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:22 smithi123 ceph-mon[30140]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-09-19T04:36:22.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:22 smithi123 ceph-mon[30140]: fsmap 2024-09-19T04:36:22.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:22 smithi123 ceph-mon[30140]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T04:36:22.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:22 smithi123 ceph-mon[30140]: mgrmap e1: no daemons active 2024-09-19T04:36:22.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:22 smithi123 ceph-mon[30140]: from='client.? 172.21.15.123:0/2149651801' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T04:36:22.373 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T04:36:22.380 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout id: 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:36:22.380 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T04:36:22.380 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-09-19T04:36:22.380 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout services: 2024-09-19T04:36:22.380 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.686159s) 2024-09-19T04:36:22.380 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T04:36:22.381 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T04:36:22.381 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-09-19T04:36:22.381 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout data: 2024-09-19T04:36:22.381 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T04:36:22.381 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T04:36:22.381 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T04:36:22.381 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T04:36:22.381 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-09-19T04:36:22.381 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.693+0000 7fdbb4e00640 1 Processor -- start 2024-09-19T04:36:22.381 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.693+0000 7fdbb4e00640 1 -- start start 2024-09-19T04:36:22.381 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.693+0000 7fdbb4e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb01075d0 0x7fdbb01079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:22.381 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.693+0000 7fdbb4e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb0107fa0 con 0x7fdbb01075d0 2024-09-19T04:36:22.382 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.694+0000 7fdbaf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb01075d0 0x7fdbb01079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:22.382 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.694+0000 7fdbaf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb01075d0 0x7fdbb01079d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41950/0 (socket says 172.21.15.123:41950) 2024-09-19T04:36:22.382 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.694+0000 7fdbaf400640 1 -- 172.21.15.123:0/1901430402 learned_addr learned my addr 172.21.15.123:0/1901430402 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:22.382 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.694+0000 7fdbaf400640 1 -- 172.21.15.123:0/1901430402 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb0108780 con 0x7fdbb01075d0 2024-09-19T04:36:22.382 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.697+0000 7fdbaf400640 1 --2- 172.21.15.123:0/1901430402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb01075d0 0x7fdbb01079d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009820 tx=0x7fdba002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e1b4eaf2d33716 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:22.382 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.697+0000 7fdbaea00640 1 -- 172.21.15.123:0/1901430402 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba0037070 con 0x7fdbb01075d0 2024-09-19T04:36:22.382 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.697+0000 7fdbaea00640 1 -- 172.21.15.123:0/1901430402 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba0032440 con 0x7fdbb01075d0 2024-09-19T04:36:22.382 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.697+0000 7fdbaea00640 1 -- 172.21.15.123:0/1901430402 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba0032740 con 0x7fdbb01075d0 2024-09-19T04:36:22.382 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.698+0000 7fdbb4e00640 1 -- 172.21.15.123:0/1901430402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb01075d0 msgr2=0x7fdbb01079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:22.382 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.698+0000 7fdbb4e00640 1 --2- 172.21.15.123:0/1901430402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb01075d0 0x7fdbb01079d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009820 tx=0x7fdba002a270 comp rx=0 tx=0).stop 2024-09-19T04:36:22.382 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.698+0000 7fdbb4e00640 1 -- 172.21.15.123:0/1901430402 shutdown_connections 2024-09-19T04:36:22.383 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.698+0000 7fdbb4e00640 1 --2- 172.21.15.123:0/1901430402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb01075d0 0x7fdbb01079d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:22.383 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.698+0000 7fdbb4e00640 1 -- 172.21.15.123:0/1901430402 >> 172.21.15.123:0/1901430402 conn(0x7fdbb0102da0 msgr2=0x7fdbb01051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:22.383 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.698+0000 7fdbb4e00640 1 -- 172.21.15.123:0/1901430402 shutdown_connections 2024-09-19T04:36:22.383 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.698+0000 7fdbb4e00640 1 -- 172.21.15.123:0/1901430402 wait complete. 2024-09-19T04:36:22.383 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.699+0000 7fdbb4e00640 1 Processor -- start 2024-09-19T04:36:22.383 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.699+0000 7fdbb4e00640 1 -- start start 2024-09-19T04:36:22.383 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.700+0000 7fdbb4e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb01075d0 0x7fdbb019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:22.383 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.700+0000 7fdbb4e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb0108d20 con 0x7fdbb01075d0 2024-09-19T04:36:22.383 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.700+0000 7fdbaf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb01075d0 0x7fdbb019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:22.383 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.700+0000 7fdbaf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb01075d0 0x7fdbb019cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41964/0 (socket says 172.21.15.123:41964) 2024-09-19T04:36:22.383 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.700+0000 7fdbaf400640 1 -- 172.21.15.123:0/2149651801 learned_addr learned my addr 172.21.15.123:0/2149651801 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:22.384 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.700+0000 7fdbaf400640 1 -- 172.21.15.123:0/2149651801 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb019d4d0 con 0x7fdbb01075d0 2024-09-19T04:36:22.384 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.701+0000 7fdbaf400640 1 --2- 172.21.15.123:0/2149651801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb01075d0 0x7fdbb019cf90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdba00090b0 tx=0x7fdba00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:22.384 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.701+0000 7fdbacc00640 1 -- 172.21.15.123:0/2149651801 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba0041070 con 0x7fdbb01075d0 2024-09-19T04:36:22.384 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.701+0000 7fdbacc00640 1 -- 172.21.15.123:0/2149651801 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba003c7a0 con 0x7fdbb01075d0 2024-09-19T04:36:22.384 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.701+0000 7fdbacc00640 1 -- 172.21.15.123:0/2149651801 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba0037040 con 0x7fdbb01075d0 2024-09-19T04:36:22.384 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.701+0000 7fdbb4e00640 1 -- 172.21.15.123:0/2149651801 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb019d760 con 0x7fdbb01075d0 2024-09-19T04:36:22.384 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.701+0000 7fdbb4e00640 1 -- 172.21.15.123:0/2149651801 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb01a0450 con 0x7fdbb01075d0 2024-09-19T04:36:22.384 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.701+0000 7fdbacc00640 1 -- 172.21.15.123:0/2149651801 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdba0044070 con 0x7fdbb01075d0 2024-09-19T04:36:22.384 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.701+0000 7fdbacc00640 1 -- 172.21.15.123:0/2149651801 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdba003d3f0 con 0x7fdbb01075d0 2024-09-19T04:36:22.384 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.702+0000 7fdbb4e00640 1 -- 172.21.15.123:0/2149651801 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb0109370 con 0x7fdbb01075d0 2024-09-19T04:36:22.384 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.705+0000 7fdbacc00640 1 -- 172.21.15.123:0/2149651801 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fdba0046450 con 0x7fdbb01075d0 2024-09-19T04:36:22.384 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.767+0000 7fdbb4e00640 1 -- 172.21.15.123:0/2149651801 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fdbb01a09a0 con 0x7fdbb01075d0 2024-09-19T04:36:22.385 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.767+0000 7fdbacc00640 1 -- 172.21.15.123:0/2149651801 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fdba0046630 con 0x7fdbb01075d0 2024-09-19T04:36:22.385 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.768+0000 7fdbb4e00640 1 -- 172.21.15.123:0/2149651801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb01075d0 msgr2=0x7fdbb019cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:22.385 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.768+0000 7fdbb4e00640 1 --2- 172.21.15.123:0/2149651801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb01075d0 0x7fdbb019cf90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdba00090b0 tx=0x7fdba00046c0 comp rx=0 tx=0).stop 2024-09-19T04:36:22.385 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.768+0000 7fdbb4e00640 1 -- 172.21.15.123:0/2149651801 shutdown_connections 2024-09-19T04:36:22.385 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.768+0000 7fdbb4e00640 1 --2- 172.21.15.123:0/2149651801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbb01075d0 0x7fdbb019cf90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:22.385 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.768+0000 7fdbb4e00640 1 -- 172.21.15.123:0/2149651801 >> 172.21.15.123:0/2149651801 conn(0x7fdbb0102da0 msgr2=0x7fdbb007c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:22.385 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.768+0000 7fdbb4e00640 1 -- 172.21.15.123:0/2149651801 shutdown_connections 2024-09-19T04:36:22.385 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:21.768+0000 7fdbb4e00640 1 -- 172.21.15.123:0/2149651801 wait complete. 2024-09-19T04:36:22.385 INFO:teuthology.orchestra.run.smithi123.stdout:mon is available 2024-09-19T04:36:22.385 INFO:teuthology.orchestra.run.smithi123.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T04:36:23.346 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:23 smithi123 ceph-mon[30140]: from='client.? 172.21.15.123:0/3599585374' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T04:36:23.347 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:23 smithi123 ceph-mon[30140]: from='client.? 172.21.15.123:0/3599585374' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T04:36:23.612 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-09-19T04:36:23.612 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T04:36:23.612 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout fsid = 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:36:23.612 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.123:3300,v1:172.21.15.123:6789] 2024-09-19T04:36:23.612 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T04:36:23.612 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T04:36:23.613 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T04:36:23.613 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T04:36:23.613 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-09-19T04:36:23.613 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T04:36:23.613 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T04:36:23.613 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-09-19T04:36:23.613 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T04:36:23.613 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T04:36:23.613 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T04:36:23.613 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T04:36:23.613 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T04:36:23.613 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T04:36:23.613 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.884+0000 7fa967e00640 1 Processor -- start 2024-09-19T04:36:23.614 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.884+0000 7fa967e00640 1 -- start start 2024-09-19T04:36:23.614 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.884+0000 7fa967e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa960107740 0x7fa960107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:23.614 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.884+0000 7fa967e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa960108110 con 0x7fa960107740 2024-09-19T04:36:23.614 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.885+0000 7fa965400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa960107740 0x7fa960107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:23.614 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.885+0000 7fa965400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa960107740 0x7fa960107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38704/0 (socket says 172.21.15.123:38704) 2024-09-19T04:36:23.614 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.885+0000 7fa965400640 1 -- 172.21.15.123:0/3395181041 learned_addr learned my addr 172.21.15.123:0/3395181041 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:23.614 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.885+0000 7fa965400640 1 -- 172.21.15.123:0/3395181041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa960108940 con 0x7fa960107740 2024-09-19T04:36:23.614 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.886+0000 7fa965400640 1 --2- 172.21.15.123:0/3395181041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa960107740 0x7fa960107b40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa954009820 tx=0x7fa95402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec1b848a54e184f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:23.614 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.886+0000 7fa95fe00640 1 -- 172.21.15.123:0/3395181041 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa954037070 con 0x7fa960107740 2024-09-19T04:36:23.614 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.886+0000 7fa95fe00640 1 -- 172.21.15.123:0/3395181041 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa954032440 con 0x7fa960107740 2024-09-19T04:36:23.614 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.886+0000 7fa95fe00640 1 -- 172.21.15.123:0/3395181041 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa954032740 con 0x7fa960107740 2024-09-19T04:36:23.614 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.887+0000 7fa967e00640 1 -- 172.21.15.123:0/3395181041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa960107740 msgr2=0x7fa960107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:23.614 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.887+0000 7fa967e00640 1 --2- 172.21.15.123:0/3395181041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa960107740 0x7fa960107b40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa954009820 tx=0x7fa95402a270 comp rx=0 tx=0).stop 2024-09-19T04:36:23.615 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.887+0000 7fa967e00640 1 -- 172.21.15.123:0/3395181041 shutdown_connections 2024-09-19T04:36:23.615 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.887+0000 7fa967e00640 1 --2- 172.21.15.123:0/3395181041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa960107740 0x7fa960107b40 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:23.615 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.887+0000 7fa967e00640 1 -- 172.21.15.123:0/3395181041 >> 172.21.15.123:0/3395181041 conn(0x7fa960102f70 msgr2=0x7fa960105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:23.615 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.887+0000 7fa967e00640 1 -- 172.21.15.123:0/3395181041 shutdown_connections 2024-09-19T04:36:23.615 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.888+0000 7fa967e00640 1 -- 172.21.15.123:0/3395181041 wait complete. 2024-09-19T04:36:23.615 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.888+0000 7fa967e00640 1 Processor -- start 2024-09-19T04:36:23.615 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.889+0000 7fa967e00640 1 -- start start 2024-09-19T04:36:23.615 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.889+0000 7fa967e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa960107740 0x7fa96019d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:23.615 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.889+0000 7fa967e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa960108ee0 con 0x7fa960107740 2024-09-19T04:36:23.615 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.889+0000 7fa965400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa960107740 0x7fa96019d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:23.615 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.889+0000 7fa965400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa960107740 0x7fa96019d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38710/0 (socket says 172.21.15.123:38710) 2024-09-19T04:36:23.615 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.889+0000 7fa965400640 1 -- 172.21.15.123:0/3599585374 learned_addr learned my addr 172.21.15.123:0/3599585374 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:23.615 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.890+0000 7fa965400640 1 -- 172.21.15.123:0/3599585374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96019d6b0 con 0x7fa960107740 2024-09-19T04:36:23.615 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.890+0000 7fa965400640 1 --2- 172.21.15.123:0/3599585374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa960107740 0x7fa96019d170 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa9540090b0 tx=0x7fa9540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:23.616 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.890+0000 7fa95e000640 1 -- 172.21.15.123:0/3599585374 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa954041070 con 0x7fa960107740 2024-09-19T04:36:23.616 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.890+0000 7fa95e000640 1 -- 172.21.15.123:0/3599585374 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa95403c7a0 con 0x7fa960107740 2024-09-19T04:36:23.616 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.890+0000 7fa95e000640 1 -- 172.21.15.123:0/3599585374 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa954037040 con 0x7fa960107740 2024-09-19T04:36:23.616 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.891+0000 7fa967e00640 1 -- 172.21.15.123:0/3599585374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96019d940 con 0x7fa960107740 2024-09-19T04:36:23.616 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.891+0000 7fa967e00640 1 -- 172.21.15.123:0/3599585374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9601a0630 con 0x7fa960107740 2024-09-19T04:36:23.616 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.891+0000 7fa95e000640 1 -- 172.21.15.123:0/3599585374 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa954044070 con 0x7fa960107740 2024-09-19T04:36:23.616 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.891+0000 7fa95e000640 1 -- 172.21.15.123:0/3599585374 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa95403d3f0 con 0x7fa960107740 2024-09-19T04:36:23.616 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.892+0000 7fa967e00640 1 -- 172.21.15.123:0/3599585374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa960109530 con 0x7fa960107740 2024-09-19T04:36:23.616 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.895+0000 7fa95e000640 1 -- 172.21.15.123:0/3599585374 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa954046450 con 0x7fa960107740 2024-09-19T04:36:23.616 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.959+0000 7fa967e00640 1 -- 172.21.15.123:0/3599585374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa9601a1d00 con 0x7fa960107740 2024-09-19T04:36:23.616 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.965+0000 7fa95e000640 1 -- 172.21.15.123:0/3599585374 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7fa954046630 con 0x7fa960107740 2024-09-19T04:36:23.616 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.965+0000 7fa95e000640 1 -- 172.21.15.123:0/3599585374 <== mon.0 v2:172.21.15.123:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa9540584b0 con 0x7fa960107740 2024-09-19T04:36:23.616 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.966+0000 7fa967e00640 1 -- 172.21.15.123:0/3599585374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa960107740 msgr2=0x7fa96019d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:23.617 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.966+0000 7fa967e00640 1 --2- 172.21.15.123:0/3599585374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa960107740 0x7fa96019d170 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa9540090b0 tx=0x7fa9540046c0 comp rx=0 tx=0).stop 2024-09-19T04:36:23.617 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.966+0000 7fa967e00640 1 -- 172.21.15.123:0/3599585374 shutdown_connections 2024-09-19T04:36:23.617 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.966+0000 7fa967e00640 1 --2- 172.21.15.123:0/3599585374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa960107740 0x7fa96019d170 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:23.617 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.966+0000 7fa967e00640 1 -- 172.21.15.123:0/3599585374 >> 172.21.15.123:0/3599585374 conn(0x7fa960102f70 msgr2=0x7fa96007c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:23.617 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.967+0000 7fa967e00640 1 -- 172.21.15.123:0/3599585374 shutdown_connections 2024-09-19T04:36:23.617 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:22.967+0000 7fa967e00640 1 -- 172.21.15.123:0/3599585374 wait complete. 2024-09-19T04:36:23.617 INFO:teuthology.orchestra.run.smithi123.stdout:Generating new minimal ceph.conf... 2024-09-19T04:36:24.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:24 smithi123 ceph-mon[30140]: from='client.? 172.21.15.123:0/3103113303' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:36:24.822 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.148+0000 7fadace00640 1 Processor -- start 2024-09-19T04:36:24.822 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.149+0000 7fadace00640 1 -- start start 2024-09-19T04:36:24.822 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.149+0000 7fadace00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada8107790 0x7fada8107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:24.822 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.149+0000 7fadace00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fada8108160 con 0x7fada8107790 2024-09-19T04:36:24.822 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.149+0000 7fada5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada8107790 0x7fada8107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:24.822 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.150+0000 7fada5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada8107790 0x7fada8107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38714/0 (socket says 172.21.15.123:38714) 2024-09-19T04:36:24.822 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.150+0000 7fada5e00640 1 -- 172.21.15.123:0/3729077749 learned_addr learned my addr 172.21.15.123:0/3729077749 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:24.822 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.150+0000 7fada5e00640 1 -- 172.21.15.123:0/3729077749 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada8108940 con 0x7fada8107790 2024-09-19T04:36:24.822 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.151+0000 7fada5e00640 1 --2- 172.21.15.123:0/3729077749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada8107790 0x7fada8107b90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fad98009820 tx=0x7fad9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546daa0905b71ecd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:24.823 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.151+0000 7fada5400640 1 -- 172.21.15.123:0/3729077749 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad98037070 con 0x7fada8107790 2024-09-19T04:36:24.823 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.151+0000 7fada5400640 1 -- 172.21.15.123:0/3729077749 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fad98032900 con 0x7fada8107790 2024-09-19T04:36:24.823 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.151+0000 7fada5400640 1 -- 172.21.15.123:0/3729077749 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad98030850 con 0x7fada8107790 2024-09-19T04:36:24.823 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.152+0000 7fadace00640 1 -- 172.21.15.123:0/3729077749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada8107790 msgr2=0x7fada8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:24.823 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.152+0000 7fadace00640 1 --2- 172.21.15.123:0/3729077749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada8107790 0x7fada8107b90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fad98009820 tx=0x7fad9802a270 comp rx=0 tx=0).stop 2024-09-19T04:36:24.823 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.152+0000 7fadace00640 1 -- 172.21.15.123:0/3729077749 shutdown_connections 2024-09-19T04:36:24.823 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.152+0000 7fadace00640 1 --2- 172.21.15.123:0/3729077749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada8107790 0x7fada8107b90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:24.823 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.152+0000 7fadace00640 1 -- 172.21.15.123:0/3729077749 >> 172.21.15.123:0/3729077749 conn(0x7fada8102fa0 msgr2=0x7fada81053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:24.823 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.152+0000 7fadace00640 1 -- 172.21.15.123:0/3729077749 shutdown_connections 2024-09-19T04:36:24.823 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.152+0000 7fadace00640 1 -- 172.21.15.123:0/3729077749 wait complete. 2024-09-19T04:36:24.823 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.153+0000 7fadace00640 1 Processor -- start 2024-09-19T04:36:24.823 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.153+0000 7fadace00640 1 -- start start 2024-09-19T04:36:24.823 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.153+0000 7fadace00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada8107790 0x7fada807e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:24.824 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.153+0000 7fadace00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fada810bcf0 con 0x7fada8107790 2024-09-19T04:36:24.824 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.153+0000 7fada5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada8107790 0x7fada807e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:24.824 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.154+0000 7fada5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada8107790 0x7fada807e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38730/0 (socket says 172.21.15.123:38730) 2024-09-19T04:36:24.824 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.154+0000 7fada5e00640 1 -- 172.21.15.123:0/3103113303 learned_addr learned my addr 172.21.15.123:0/3103113303 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:24.824 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.154+0000 7fada5e00640 1 -- 172.21.15.123:0/3103113303 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada807e910 con 0x7fada8107790 2024-09-19T04:36:24.824 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.154+0000 7fada5e00640 1 --2- 172.21.15.123:0/3103113303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada8107790 0x7fada807e3d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fad980097f0 tx=0x7fad98030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:24.824 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.154+0000 7fad9ea00640 1 -- 172.21.15.123:0/3103113303 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad98037040 con 0x7fada8107790 2024-09-19T04:36:24.824 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.154+0000 7fad9ea00640 1 -- 172.21.15.123:0/3103113303 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fad98040070 con 0x7fada8107790 2024-09-19T04:36:24.824 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.154+0000 7fadace00640 1 -- 172.21.15.123:0/3103113303 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada807cb00 con 0x7fada8107790 2024-09-19T04:36:24.824 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.154+0000 7fad9ea00640 1 -- 172.21.15.123:0/3103113303 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad9803a390 con 0x7fada8107790 2024-09-19T04:36:24.824 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.155+0000 7fadace00640 1 -- 172.21.15.123:0/3103113303 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada807cfe0 con 0x7fada8107790 2024-09-19T04:36:24.824 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.155+0000 7fad9ea00640 1 -- 172.21.15.123:0/3103113303 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fad9803a5e0 con 0x7fada8107790 2024-09-19T04:36:24.824 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.155+0000 7fad9ea00640 1 -- 172.21.15.123:0/3103113303 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fad98045950 con 0x7fada8107790 2024-09-19T04:36:24.825 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.156+0000 7fadace00640 1 -- 172.21.15.123:0/3103113303 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada8107c10 con 0x7fada8107790 2024-09-19T04:36:24.825 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.158+0000 7fad9ea00640 1 -- 172.21.15.123:0/3103113303 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fad98045460 con 0x7fada8107790 2024-09-19T04:36:24.825 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.218+0000 7fadace00640 1 -- 172.21.15.123:0/3103113303 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fada807d510 con 0x7fada8107790 2024-09-19T04:36:24.825 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.218+0000 7fad9ea00640 1 -- 172.21.15.123:0/3103113303 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7fad9803b430 con 0x7fada8107790 2024-09-19T04:36:24.825 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.219+0000 7fadace00640 1 -- 172.21.15.123:0/3103113303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada8107790 msgr2=0x7fada807e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:24.825 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.219+0000 7fadace00640 1 --2- 172.21.15.123:0/3103113303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada8107790 0x7fada807e3d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fad980097f0 tx=0x7fad98030d80 comp rx=0 tx=0).stop 2024-09-19T04:36:24.825 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.220+0000 7fadace00640 1 -- 172.21.15.123:0/3103113303 shutdown_connections 2024-09-19T04:36:24.825 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.220+0000 7fadace00640 1 --2- 172.21.15.123:0/3103113303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada8107790 0x7fada807e3d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:24.825 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.220+0000 7fadace00640 1 -- 172.21.15.123:0/3103113303 >> 172.21.15.123:0/3103113303 conn(0x7fada8102fa0 msgr2=0x7fada8103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:24.825 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.220+0000 7fadace00640 1 -- 172.21.15.123:0/3103113303 shutdown_connections 2024-09-19T04:36:24.825 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:24.220+0000 7fadace00640 1 -- 172.21.15.123:0/3103113303 wait complete. 2024-09-19T04:36:24.825 INFO:teuthology.orchestra.run.smithi123.stdout:Restarting the monitor... 2024-09-19T04:36:25.161 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:24 smithi123 systemd[1]: Stopping Ceph mon.a for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:36:25.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:25 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-a[30136]: 2024-09-19T04:36:25.159+0000 7f80cb600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:36:25.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:25 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-a[30136]: 2024-09-19T04:36:25.159+0000 7f80cb600640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T04:36:25.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:25 smithi123 podman[30391]: 2024-09-19 04:36:25.395867266 +0000 UTC m=+0.346596733 container died 8857b33da95400e191311d2e134f2ad84e14488f2d1428a29bc3741d8878441c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T04:36:25.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:25 smithi123 podman[30391]: 2024-09-19 04:36:25.755878987 +0000 UTC m=+0.706608456 container cleanup 8857b33da95400e191311d2e134f2ad84e14488f2d1428a29bc3741d8878441c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T04:36:25.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:25 smithi123 bash[30391]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-a 2024-09-19T04:36:26.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:25 smithi123 podman[30404]: 2024-09-19 04:36:25.948367151 +0000 UTC m=+0.538399822 container remove 8857b33da95400e191311d2e134f2ad84e14488f2d1428a29bc3741d8878441c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:36:26.536 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 systemd[1]: ceph-911fa052-7640-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T04:36:26.537 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 systemd[1]: Stopped Ceph mon.a for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:36:26.537 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 systemd[1]: ceph-911fa052-7640-11ef-bceb-c7b262605968@mon.a.service: Consumed 1.200s CPU time. 2024-09-19T04:36:26.537 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 systemd[1]: Starting Ceph mon.a for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:36:26.787 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 podman[30491]: 2024-09-19 04:36:26.439995152 +0000 UTC m=+0.027479103 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:36:26.787 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 podman[30491]: 2024-09-19 04:36:26.540957172 +0000 UTC m=+0.128441098 container create 16e47864cb79fff026808b31823e5a40efacf6d794b75f0381bbff58605755b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T04:36:26.788 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 podman[30491]: 2024-09-19 04:36:26.7348466 +0000 UTC m=+0.322330536 container init 16e47864cb79fff026808b31823e5a40efacf6d794b75f0381bbff58605755b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T04:36:26.788 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 podman[30491]: 2024-09-19 04:36:26.741579004 +0000 UTC m=+0.329062930 container start 16e47864cb79fff026808b31823e5a40efacf6d794b75f0381bbff58605755b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-19T04:36:26.788 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T04:36:26.788 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T04:36:26.788 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: pidfile_write: ignore empty --pid-file 2024-09-19T04:36:26.788 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: load: jerasure load: lrc 2024-09-19T04:36:26.788 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: RocksDB version: 7.9.2 2024-09-19T04:36:26.788 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Git sha 0 2024-09-19T04:36:26.788 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T04:36:26.788 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: DB SUMMARY 2024-09-19T04:36:26.788 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: DB Session ID: LSNTS0RT50HSNTW3G7UZ 2024-09-19T04:36:26.788 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: CURRENT file: CURRENT 2024-09-19T04:36:26.788 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T04:36:26.789 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T04:36:26.789 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T04:36:26.789 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106430 ; 2024-09-19T04:36:26.789 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.error_if_exists: 0 2024-09-19T04:36:26.790 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.create_if_missing: 0 2024-09-19T04:36:26.790 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.paranoid_checks: 1 2024-09-19T04:36:26.790 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T04:36:26.790 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T04:36:26.790 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T04:36:26.790 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.env: 0x56057eca1940 2024-09-19T04:36:26.790 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T04:36:26.790 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.info_log: 0x56058dd176a0 2024-09-19T04:36:26.790 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T04:36:26.791 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.statistics: (nil) 2024-09-19T04:36:26.791 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.use_fsync: 0 2024-09-19T04:36:26.791 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_log_file_size: 0 2024-09-19T04:36:26.791 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T04:36:26.791 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T04:36:26.791 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T04:36:26.791 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T04:36:26.791 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.allow_fallocate: 1 2024-09-19T04:36:26.791 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T04:36:26.791 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T04:36:26.791 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.use_direct_reads: 0 2024-09-19T04:36:26.792 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T04:36:26.792 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T04:36:26.792 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.db_log_dir: 2024-09-19T04:36:26.792 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.wal_dir: 2024-09-19T04:36:26.792 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T04:36:26.792 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T04:36:26.792 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T04:36:26.792 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T04:36:26.792 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T04:36:26.792 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T04:36:26.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T04:36:26.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T04:36:26.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.write_buffer_manager: 0x56058dd26dc0 2024-09-19T04:36:26.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T04:36:26.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T04:36:26.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T04:36:26.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T04:36:26.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T04:36:26.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T04:36:26.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T04:36:26.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T04:36:26.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.unordered_write: 0 2024-09-19T04:36:26.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T04:36:26.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T04:36:26.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T04:36:26.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T04:36:26.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.row_cache: None 2024-09-19T04:36:26.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.wal_filter: None 2024-09-19T04:36:26.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T04:36:26.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T04:36:26.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.two_write_queues: 0 2024-09-19T04:36:26.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T04:36:26.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.wal_compression: 0 2024-09-19T04:36:26.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.atomic_flush: 0 2024-09-19T04:36:26.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T04:36:26.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T04:36:26.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T04:36:26.794 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.log_readahead_size: 0 2024-09-19T04:36:26.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T04:36:26.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T04:36:26.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T04:36:26.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T04:36:26.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T04:36:26.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T04:36:26.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T04:36:26.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_background_jobs: 2 2024-09-19T04:36:26.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_background_compactions: -1 2024-09-19T04:36:26.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_subcompactions: 1 2024-09-19T04:36:26.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T04:36:26.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T04:36:26.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T04:36:26.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T04:36:26.796 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T04:36:26.796 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T04:36:26.796 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T04:36:26.796 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T04:36:26.796 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_open_files: -1 2024-09-19T04:36:26.796 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T04:36:26.796 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T04:36:26.796 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T04:36:26.796 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T04:36:26.796 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_background_flushes: -1 2024-09-19T04:36:26.796 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Compression algorithms supported: 2024-09-19T04:36:26.796 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: kZSTD supported: 0 2024-09-19T04:36:26.796 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: kXpressCompression supported: 0 2024-09-19T04:36:26.797 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: kBZip2Compression supported: 0 2024-09-19T04:36:26.797 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T04:36:26.797 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: kLZ4Compression supported: 1 2024-09-19T04:36:26.797 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: kZlibCompression supported: 1 2024-09-19T04:36:26.797 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T04:36:26.797 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: kSnappyCompression supported: 1 2024-09-19T04:36:26.797 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T04:36:26.797 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T04:36:26.797 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-19T04:36:26.797 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T04:36:26.797 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T04:36:26.797 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.merge_operator: 2024-09-19T04:36:26.797 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compaction_filter: None 2024-09-19T04:36:26.797 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T04:36:26.798 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T04:36:26.798 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T04:36:26.798 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T04:36:26.798 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56058dd17300) 2024-09-19T04:36:26.798 INFO:journalctl@ceph.mon.a.smithi123.stdout: cache_index_and_filter_blocks: 1 2024-09-19T04:36:26.798 INFO:journalctl@ceph.mon.a.smithi123.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T04:36:26.798 INFO:journalctl@ceph.mon.a.smithi123.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T04:36:26.798 INFO:journalctl@ceph.mon.a.smithi123.stdout: pin_top_level_index_and_filter: 1 2024-09-19T04:36:26.798 INFO:journalctl@ceph.mon.a.smithi123.stdout: index_type: 0 2024-09-19T04:36:26.798 INFO:journalctl@ceph.mon.a.smithi123.stdout: data_block_index_type: 0 2024-09-19T04:36:26.798 INFO:journalctl@ceph.mon.a.smithi123.stdout: index_shortening: 1 2024-09-19T04:36:26.798 INFO:journalctl@ceph.mon.a.smithi123.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T04:36:26.798 INFO:journalctl@ceph.mon.a.smithi123.stdout: checksum: 4 2024-09-19T04:36:26.798 INFO:journalctl@ceph.mon.a.smithi123.stdout: no_block_cache: 0 2024-09-19T04:36:26.798 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_cache: 0x56058dd0b1f0 2024-09-19T04:36:26.799 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_cache_name: BinnedLRUCache 2024-09-19T04:36:26.799 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_cache_options: 2024-09-19T04:36:26.799 INFO:journalctl@ceph.mon.a.smithi123.stdout: capacity : 536870912 2024-09-19T04:36:26.799 INFO:journalctl@ceph.mon.a.smithi123.stdout: num_shard_bits : 4 2024-09-19T04:36:26.799 INFO:journalctl@ceph.mon.a.smithi123.stdout: strict_capacity_limit : 0 2024-09-19T04:36:26.799 INFO:journalctl@ceph.mon.a.smithi123.stdout: high_pri_pool_ratio: 0.000 2024-09-19T04:36:26.799 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_cache_compressed: (nil) 2024-09-19T04:36:26.799 INFO:journalctl@ceph.mon.a.smithi123.stdout: persistent_cache: (nil) 2024-09-19T04:36:26.799 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_size: 4096 2024-09-19T04:36:26.799 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_size_deviation: 10 2024-09-19T04:36:26.799 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_restart_interval: 16 2024-09-19T04:36:26.799 INFO:journalctl@ceph.mon.a.smithi123.stdout: index_block_restart_interval: 1 2024-09-19T04:36:26.799 INFO:journalctl@ceph.mon.a.smithi123.stdout: metadata_block_size: 4096 2024-09-19T04:36:26.800 INFO:journalctl@ceph.mon.a.smithi123.stdout: partition_filters: 0 2024-09-19T04:36:26.800 INFO:journalctl@ceph.mon.a.smithi123.stdout: use_delta_encoding: 1 2024-09-19T04:36:26.800 INFO:journalctl@ceph.mon.a.smithi123.stdout: filter_policy: bloomfilter 2024-09-19T04:36:26.800 INFO:journalctl@ceph.mon.a.smithi123.stdout: whole_key_filtering: 1 2024-09-19T04:36:26.800 INFO:journalctl@ceph.mon.a.smithi123.stdout: verify_compression: 0 2024-09-19T04:36:26.800 INFO:journalctl@ceph.mon.a.smithi123.stdout: read_amp_bytes_per_bit: 0 2024-09-19T04:36:26.800 INFO:journalctl@ceph.mon.a.smithi123.stdout: format_version: 5 2024-09-19T04:36:26.800 INFO:journalctl@ceph.mon.a.smithi123.stdout: enable_index_compression: 1 2024-09-19T04:36:26.800 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_align: 0 2024-09-19T04:36:26.800 INFO:journalctl@ceph.mon.a.smithi123.stdout: max_auto_readahead_size: 262144 2024-09-19T04:36:26.800 INFO:journalctl@ceph.mon.a.smithi123.stdout: prepopulate_block_cache: 0 2024-09-19T04:36:26.800 INFO:journalctl@ceph.mon.a.smithi123.stdout: initial_auto_readahead_size: 8192 2024-09-19T04:36:26.800 INFO:journalctl@ceph.mon.a.smithi123.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T04:36:26.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T04:36:26.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T04:36:26.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compression: NoCompression 2024-09-19T04:36:26.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T04:36:26.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T04:36:26.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T04:36:26.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.num_levels: 7 2024-09-19T04:36:26.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T04:36:26.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T04:36:26.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T04:36:26.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T04:36:26.801 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T04:36:26.802 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T04:36:26.802 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T04:36:26.802 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:36:26.802 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T04:36:26.802 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T04:36:26.802 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:36:26.802 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:36:26.802 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T04:36:26.802 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T04:36:26.802 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T04:36:26.802 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T04:36:26.802 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:36:26.802 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:36:26.802 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T04:36:26.803 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T04:36:26.803 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:36:26.803 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T04:36:26.803 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T04:36:26.804 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T04:36:26.804 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T04:36:26.804 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T04:36:26.804 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T04:36:26.804 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T04:36:26.804 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T04:36:26.804 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T04:36:26.804 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T04:36:26.804 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T04:36:26.804 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T04:36:26.804 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T04:36:26.805 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T04:36:26.805 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T04:36:26.805 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T04:36:26.805 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T04:36:26.805 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T04:36:26.805 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T04:36:26.805 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T04:36:26.805 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T04:36:26.805 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T04:36:26.805 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T04:36:26.805 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T04:36:26.805 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T04:36:26.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T04:36:26.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T04:36:26.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T04:36:26.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T04:36:26.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T04:36:26.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T04:36:26.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T04:36:26.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T04:36:26.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.inplace_update_support: 0 2024-09-19T04:36:26.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T04:36:26.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T04:36:26.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T04:36:26.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T04:36:26.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.bloom_locality: 0 2024-09-19T04:36:26.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.max_successive_merges: 0 2024-09-19T04:36:26.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T04:36:26.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T04:36:26.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T04:36:26.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T04:36:26.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.ttl: 2592000 2024-09-19T04:36:26.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T04:36:26.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T04:36:26.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T04:36:26.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.enable_blob_files: false 2024-09-19T04:36:26.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.min_blob_size: 0 2024-09-19T04:36:26.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T04:36:26.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T04:36:26.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T04:36:26.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T04:36:26.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T04:36:26.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T04:36:26.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T04:36:26.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T04:36:26.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T04:36:26.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T04:36:26.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 277f1d91-4f23-42d6-a289-63b078c95a32 2024-09-19T04:36:26.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726720586783480, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T04:36:26.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T04:36:26.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726720586785443, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101450, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 99326, "index_size": 316, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14564, "raw_average_key_size": 50, "raw_value_size": 91401, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726720586, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "277f1d91-4f23-42d6-a289-63b078c95a32", "db_session_id": "LSNTS0RT50HSNTW3G7UZ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T04:36:26.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726720586785621, "job": 1, "event": "recovery_finished"} 2024-09-19T04:36:26.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T04:36:26.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T04:36:26.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56058dd3ae00 2024-09-19T04:36:26.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: DB pointer 0x56058de36000 2024-09-19T04:36:26.809 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T04:36:26.809 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T04:36:26.809 INFO:journalctl@ceph.mon.a.smithi123.stdout: ** DB Stats ** 2024-09-19T04:36:26.809 INFO:journalctl@ceph.mon.a.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:36:26.809 INFO:journalctl@ceph.mon.a.smithi123.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T04:36:26.809 INFO:journalctl@ceph.mon.a.smithi123.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:36:26.809 INFO:journalctl@ceph.mon.a.smithi123.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:36:26.809 INFO:journalctl@ceph.mon.a.smithi123.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T04:36:26.809 INFO:journalctl@ceph.mon.a.smithi123.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:36:26.809 INFO:journalctl@ceph.mon.a.smithi123.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:36:26.809 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-09-19T04:36:26.809 INFO:journalctl@ceph.mon.a.smithi123.stdout: ** Compaction Stats [default] ** 2024-09-19T04:36:26.809 INFO:journalctl@ceph.mon.a.smithi123.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:36:26.809 INFO:journalctl@ceph.mon.a.smithi123.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T04:36:26.810 INFO:journalctl@ceph.mon.a.smithi123.stdout: L0 2/0 100.93 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:36:26.810 INFO:journalctl@ceph.mon.a.smithi123.stdout: Sum 2/0 100.93 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:36:26.810 INFO:journalctl@ceph.mon.a.smithi123.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:36:26.810 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-09-19T04:36:26.810 INFO:journalctl@ceph.mon.a.smithi123.stdout: ** Compaction Stats [default] ** 2024-09-19T04:36:26.810 INFO:journalctl@ceph.mon.a.smithi123.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:36:26.810 INFO:journalctl@ceph.mon.a.smithi123.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T04:36:26.810 INFO:journalctl@ceph.mon.a.smithi123.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 75.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:36:26.810 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-09-19T04:36:26.810 INFO:journalctl@ceph.mon.a.smithi123.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T04:36:26.810 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-09-19T04:36:26.810 INFO:journalctl@ceph.mon.a.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:36:26.810 INFO:journalctl@ceph.mon.a.smithi123.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T04:36:26.810 INFO:journalctl@ceph.mon.a.smithi123.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T04:36:26.810 INFO:journalctl@ceph.mon.a.smithi123.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T04:36:26.811 INFO:journalctl@ceph.mon.a.smithi123.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T04:36:26.811 INFO:journalctl@ceph.mon.a.smithi123.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T04:36:26.811 INFO:journalctl@ceph.mon.a.smithi123.stdout: Cumulative compaction: 0.00 GB write, 13.81 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:36:26.811 INFO:journalctl@ceph.mon.a.smithi123.stdout: Interval compaction: 0.00 GB write, 13.81 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:36:26.811 INFO:journalctl@ceph.mon.a.smithi123.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T04:36:26.811 INFO:journalctl@ceph.mon.a.smithi123.stdout: Block cache BinnedLRUCache@0x56058dd0b1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-19T04:36:26.811 INFO:journalctl@ceph.mon.a.smithi123.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-19T04:36:26.811 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-09-19T04:36:26.811 INFO:journalctl@ceph.mon.a.smithi123.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T04:36:26.811 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: starting mon.a rank 0 at public addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] at bind addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:36:26.846 INFO:teuthology.orchestra.run.smithi123.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T04:36:27.091 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: mon.a@-1(???) e1 preinit fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:36:27.091 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: mon.a@-1(???).mds e1 new map 2024-09-19T04:36:27.091 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: mon.a@-1(???).mds e1 print_map 2024-09-19T04:36:27.091 INFO:journalctl@ceph.mon.a.smithi123.stdout: e1 2024-09-19T04:36:27.092 INFO:journalctl@ceph.mon.a.smithi123.stdout: btime 2024-09-19T04:36:21:083029+0000 2024-09-19T04:36:27.092 INFO:journalctl@ceph.mon.a.smithi123.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T04:36:27.092 INFO:journalctl@ceph.mon.a.smithi123.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:36:27.092 INFO:journalctl@ceph.mon.a.smithi123.stdout: legacy client fscid: -1 2024-09-19T04:36:27.092 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-09-19T04:36:27.092 INFO:journalctl@ceph.mon.a.smithi123.stdout: No filesystems configured 2024-09-19T04:36:27.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T04:36:27.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:36:27.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:36:27.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:36:27.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T04:36:27.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 bash[30491]: 16e47864cb79fff026808b31823e5a40efacf6d794b75f0381bbff58605755b7 2024-09-19T04:36:27.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 systemd[1]: Started Ceph mon.a for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:36:27.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T04:36:27.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: monmap epoch 1 2024-09-19T04:36:27.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:36:27.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: last_changed 2024-09-19T04:36:17.413126+0000 2024-09-19T04:36:27.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: created 2024-09-19T04:36:17.413126+0000 2024-09-19T04:36:27.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: min_mon_release 19 (squid) 2024-09-19T04:36:27.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: election_strategy: 1 2024-09-19T04:36:27.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-09-19T04:36:27.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: fsmap 2024-09-19T04:36:27.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T04:36:27.093 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:26 smithi123 ceph-mon[30511]: mgrmap e1: no daemons active 2024-09-19T04:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.322+0000 7f98aa000640 1 Processor -- start 2024-09-19T04:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.323+0000 7f98aa000640 1 -- start start 2024-09-19T04:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.323+0000 7f98aa000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4107750 0x7f98a4107b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.323+0000 7f98aa000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4108120 con 0x7f98a4107750 2024-09-19T04:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.324+0000 7f98a3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4107750 0x7f98a4107b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.324+0000 7f98a3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4107750 0x7f98a4107b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38740/0 (socket says 172.21.15.123:38740) 2024-09-19T04:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.324+0000 7f98a3400640 1 -- 172.21.15.123:0/3768519052 learned_addr learned my addr 172.21.15.123:0/3768519052 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.324+0000 7f98a3400640 1 -- 172.21.15.123:0/3768519052 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a4108950 con 0x7f98a4107750 2024-09-19T04:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.327+0000 7f98a3400640 1 --2- 172.21.15.123:0/3768519052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4107750 0x7f98a4107b50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9894009820 tx=0x7f989402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27dd4397bf8cbc7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.327+0000 7f98a2a00640 1 -- 172.21.15.123:0/3768519052 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9894037070 con 0x7f98a4107750 2024-09-19T04:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.327+0000 7f98a2a00640 1 -- 172.21.15.123:0/3768519052 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9894032900 con 0x7f98a4107750 2024-09-19T04:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.327+0000 7f98a2a00640 1 -- 172.21.15.123:0/3768519052 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9894030850 con 0x7f98a4107750 2024-09-19T04:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.328+0000 7f98aa000640 1 -- 172.21.15.123:0/3768519052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4107750 msgr2=0x7f98a4107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.328+0000 7f98aa000640 1 --2- 172.21.15.123:0/3768519052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4107750 0x7f98a4107b50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9894009820 tx=0x7f989402a270 comp rx=0 tx=0).stop 2024-09-19T04:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.328+0000 7f98aa000640 1 -- 172.21.15.123:0/3768519052 shutdown_connections 2024-09-19T04:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.328+0000 7f98aa000640 1 --2- 172.21.15.123:0/3768519052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4107750 0x7f98a4107b50 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.328+0000 7f98aa000640 1 -- 172.21.15.123:0/3768519052 >> 172.21.15.123:0/3768519052 conn(0x7f98a4102f40 msgr2=0x7f98a4105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.328+0000 7f98aa000640 1 -- 172.21.15.123:0/3768519052 shutdown_connections 2024-09-19T04:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.328+0000 7f98aa000640 1 -- 172.21.15.123:0/3768519052 wait complete. 2024-09-19T04:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.329+0000 7f98aa000640 1 Processor -- start 2024-09-19T04:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.329+0000 7f98aa000640 1 -- start start 2024-09-19T04:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.329+0000 7f98aa000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4107750 0x7f98a419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.329+0000 7f98aa000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98a410bd00 con 0x7f98a4107750 2024-09-19T04:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.329+0000 7f98a3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4107750 0x7f98a419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.330+0000 7f98a3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4107750 0x7f98a419b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38742/0 (socket says 172.21.15.123:38742) 2024-09-19T04:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.330+0000 7f98a3400640 1 -- 172.21.15.123:0/3139964456 learned_addr learned my addr 172.21.15.123:0/3139964456 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.330+0000 7f98a3400640 1 -- 172.21.15.123:0/3139964456 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a419bd30 con 0x7f98a4107750 2024-09-19T04:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.330+0000 7f98a3400640 1 --2- 172.21.15.123:0/3139964456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4107750 0x7f98a419b7f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f98940097f0 tx=0x7f9894030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.330+0000 7f98a0c00640 1 -- 172.21.15.123:0/3139964456 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9894037040 con 0x7f98a4107750 2024-09-19T04:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.330+0000 7f98a0c00640 1 -- 172.21.15.123:0/3139964456 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9894040070 con 0x7f98a4107750 2024-09-19T04:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.330+0000 7f98a0c00640 1 -- 172.21.15.123:0/3139964456 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f989403a390 con 0x7f98a4107750 2024-09-19T04:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.330+0000 7f98aa000640 1 -- 172.21.15.123:0/3139964456 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a419bfc0 con 0x7f98a4107750 2024-09-19T04:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.331+0000 7f98aa000640 1 -- 172.21.15.123:0/3139964456 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a419ecb0 con 0x7f98a4107750 2024-09-19T04:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.331+0000 7f98a0c00640 1 -- 172.21.15.123:0/3139964456 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f989403a5e0 con 0x7f98a4107750 2024-09-19T04:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.331+0000 7f98a0c00640 1 -- 172.21.15.123:0/3139964456 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9894045950 con 0x7f98a4107750 2024-09-19T04:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.332+0000 7f98aa000640 1 -- 172.21.15.123:0/3139964456 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a410be80 con 0x7f98a4107750 2024-09-19T04:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.335+0000 7f98a0c00640 1 -- 172.21.15.123:0/3139964456 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9894045460 con 0x7f98a4107750 2024-09-19T04:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.395+0000 7f98aa000640 1 -- 172.21.15.123:0/3139964456 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f98a419ee80 con 0x7f98a4107750 2024-09-19T04:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.397+0000 7f98a0c00640 1 -- 172.21.15.123:0/3139964456 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f989403b430 con 0x7f98a4107750 2024-09-19T04:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.398+0000 7f98aa000640 1 -- 172.21.15.123:0/3139964456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4107750 msgr2=0x7f98a419b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.398+0000 7f98aa000640 1 --2- 172.21.15.123:0/3139964456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4107750 0x7f98a419b7f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f98940097f0 tx=0x7f9894030d80 comp rx=0 tx=0).stop 2024-09-19T04:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.398+0000 7f98aa000640 1 -- 172.21.15.123:0/3139964456 shutdown_connections 2024-09-19T04:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.398+0000 7f98aa000640 1 --2- 172.21.15.123:0/3139964456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4107750 0x7f98a419b7f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.398+0000 7f98aa000640 1 -- 172.21.15.123:0/3139964456 >> 172.21.15.123:0/3139964456 conn(0x7f98a4102f40 msgr2=0x7f98a4103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.398+0000 7f98aa000640 1 -- 172.21.15.123:0/3139964456 shutdown_connections 2024-09-19T04:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:27.398+0000 7f98aa000640 1 -- 172.21.15.123:0/3139964456 wait complete. 2024-09-19T04:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T04:36:28.007 INFO:teuthology.orchestra.run.smithi123.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:36:28.007 INFO:teuthology.orchestra.run.smithi123.stdout:Creating mgr... 2024-09-19T04:36:28.008 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T04:36:28.009 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T04:36:28.346 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-911fa052-7640-11ef-bceb-c7b262605968@mgr.x 2024-09-19T04:36:28.346 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Failed to reset failed state of unit ceph-911fa052-7640-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-911fa052-7640-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-19T04:36:28.581 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-911fa052-7640-11ef-bceb-c7b262605968.target.wants/ceph-911fa052-7640-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-911fa052-7640-11ef-bceb-c7b262605968@.service. 2024-09-19T04:36:28.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:28 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3139964456' entity='client.admin' 2024-09-19T04:36:28.677 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:28 smithi123 systemd[1]: Starting Ceph mgr.x for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:36:29.177 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:28 smithi123 podman[30720]: 2024-09-19 04:36:28.820798087 +0000 UTC m=+0.027195184 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:36:29.177 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:28 smithi123 podman[30720]: 2024-09-19 04:36:28.967925465 +0000 UTC m=+0.174322559 container create 1f7915704579ef2041c10b4d6a7373e7748ba49237add44504e48b1134adf480 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-19T04:36:29.177 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:29 smithi123 podman[30720]: 2024-09-19 04:36:29.153736682 +0000 UTC m=+0.360133781 container init 1f7915704579ef2041c10b4d6a7373e7748ba49237add44504e48b1134adf480 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-19T04:36:29.177 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:29 smithi123 podman[30720]: 2024-09-19 04:36:29.159680038 +0000 UTC m=+0.366077135 container start 1f7915704579ef2041c10b4d6a7373e7748ba49237add44504e48b1134adf480 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T04:36:29.293 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:36:29.293 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout disabled 2024-09-19T04:36:29.308 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:36:29.308 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-09-19T04:36:29.308 INFO:teuthology.orchestra.run.smithi123.stdout:firewalld.service is not enabled 2024-09-19T04:36:29.308 INFO:teuthology.orchestra.run.smithi123.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T04:36:29.329 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:36:29.329 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout disabled 2024-09-19T04:36:29.344 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:36:29.344 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-09-19T04:36:29.344 INFO:teuthology.orchestra.run.smithi123.stdout:firewalld.service is not enabled 2024-09-19T04:36:29.344 INFO:teuthology.orchestra.run.smithi123.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-19T04:36:29.344 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mgr to start... 2024-09-19T04:36:29.344 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mgr... 2024-09-19T04:36:29.596 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:29 smithi123 bash[30720]: 1f7915704579ef2041c10b4d6a7373e7748ba49237add44504e48b1134adf480 2024-09-19T04:36:29.596 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:29 smithi123 systemd[1]: Started Ceph mgr.x for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:36:29.596 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:29 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:29.335+0000 7f2d56894040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:36:29.596 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:29 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:29.410+0000 7f2d56894040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:36:29.866 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:29 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:29.603+0000 7f2d56894040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:36:29.866 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:29 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:29.685+0000 7f2d56894040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:36:29.866 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:29 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:29.864+0000 7f2d56894040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:36:30.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:29 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/231883630' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T04:36:30.177 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:30 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:30.042+0000 7f2d56894040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:36:30.499 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:30 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:30.426+0000 7f2d56894040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:36:30.500 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:30 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:30.498+0000 7f2d56894040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:36:30.545 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-09-19T04:36:30.548 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:36:30.548 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsid": "911fa052-7640-11ef-bceb-c7b262605968", 2024-09-19T04:36:30.548 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T04:36:30.548 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T04:36:30.548 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T04:36:30.548 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T04:36:30.548 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:30.548 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T04:36:30.548 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T04:36:30.549 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 0 2024-09-19T04:36:30.549 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:36:30.549 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T04:36:30.549 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T04:36:30.549 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:36:30.549 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-19T04:36:30.549 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T04:36:30.549 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:36:30.549 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T04:36:30.549 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T04:36:30.549 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:30.549 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T04:36:30.549 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:36:30.550 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T04:36:30.550 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T04:36:30.550 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T04:36:30.557 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T04:36:30.557 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T04:36:30.557 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T04:36:30.557 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:30.557 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T04:36:30.557 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T04:36:30.558 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T04:36:30.558 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T04:36:30.558 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T04:36:30.558 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T04:36:30.558 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T04:36:30.558 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T04:36:30.558 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T04:36:30.558 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:30.558 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T04:36:30.558 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:36:30.558 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T04:36:21:083029+0000", 2024-09-19T04:36:30.558 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T04:36:30.558 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T04:36:30.558 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:30.559 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T04:36:30.559 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T04:36:30.559 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T04:36:30.559 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T04:36:30.559 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T04:36:30.559 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T04:36:30.559 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T04:36:30.559 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:36:30.559 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:36:30.559 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:30.559 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T04:36:30.559 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:36:30.559 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T04:36:21.083852+0000", 2024-09-19T04:36:30.559 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:36:30.560 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:30.560 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T04:36:30.560 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:36:30.560 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.822+0000 7ff2ef200640 1 Processor -- start 2024-09-19T04:36:30.560 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.822+0000 7ff2ef200640 1 -- start start 2024-09-19T04:36:30.560 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.822+0000 7ff2ef200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01075f0 0x7ff2f01079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:30.560 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.822+0000 7ff2ef200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f0107fc0 con 0x7ff2f01075f0 2024-09-19T04:36:30.560 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.823+0000 7ff2ede00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01075f0 0x7ff2f01079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:30.560 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.823+0000 7ff2ede00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01075f0 0x7ff2f01079f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38780/0 (socket says 172.21.15.123:38780) 2024-09-19T04:36:30.560 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.823+0000 7ff2ede00640 1 -- 172.21.15.123:0/1501186161 learned_addr learned my addr 172.21.15.123:0/1501186161 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:30.560 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.823+0000 7ff2ede00640 1 -- 172.21.15.123:0/1501186161 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f01087a0 con 0x7ff2f01075f0 2024-09-19T04:36:30.560 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.824+0000 7ff2ede00640 1 --2- 172.21.15.123:0/1501186161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01075f0 0x7ff2f01079f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009820 tx=0x7ff2e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d47870970fc8b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:30.560 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.824+0000 7ff2eca00640 1 -- 172.21.15.123:0/1501186161 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2e0037070 con 0x7ff2f01075f0 2024-09-19T04:36:30.561 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.824+0000 7ff2eca00640 1 -- 172.21.15.123:0/1501186161 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff2e0032900 con 0x7ff2f01075f0 2024-09-19T04:36:30.561 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.824+0000 7ff2eca00640 1 -- 172.21.15.123:0/1501186161 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2e0030850 con 0x7ff2f01075f0 2024-09-19T04:36:30.561 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.825+0000 7ff2ef200640 1 -- 172.21.15.123:0/1501186161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01075f0 msgr2=0x7ff2f01079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:30.561 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.825+0000 7ff2ef200640 1 --2- 172.21.15.123:0/1501186161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01075f0 0x7ff2f01079f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009820 tx=0x7ff2e002a270 comp rx=0 tx=0).stop 2024-09-19T04:36:30.561 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.825+0000 7ff2ef200640 1 -- 172.21.15.123:0/1501186161 shutdown_connections 2024-09-19T04:36:30.561 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.825+0000 7ff2ef200640 1 --2- 172.21.15.123:0/1501186161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01075f0 0x7ff2f01079f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:30.561 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.825+0000 7ff2ef200640 1 -- 172.21.15.123:0/1501186161 >> 172.21.15.123:0/1501186161 conn(0x7ff2f0102d40 msgr2=0x7ff2f01051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:30.561 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.825+0000 7ff2ef200640 1 -- 172.21.15.123:0/1501186161 shutdown_connections 2024-09-19T04:36:30.561 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.825+0000 7ff2ef200640 1 -- 172.21.15.123:0/1501186161 wait complete. 2024-09-19T04:36:30.561 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.826+0000 7ff2ef200640 1 Processor -- start 2024-09-19T04:36:30.561 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.826+0000 7ff2ef200640 1 -- start start 2024-09-19T04:36:30.561 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.827+0000 7ff2ef200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01075f0 0x7ff2f019d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:30.561 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.827+0000 7ff2ef200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f010bb50 con 0x7ff2f01075f0 2024-09-19T04:36:30.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.827+0000 7ff2ede00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01075f0 0x7ff2f019d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:30.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.827+0000 7ff2ede00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01075f0 0x7ff2f019d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38788/0 (socket says 172.21.15.123:38788) 2024-09-19T04:36:30.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.827+0000 7ff2ede00640 1 -- 172.21.15.123:0/231883630 learned_addr learned my addr 172.21.15.123:0/231883630 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:30.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.827+0000 7ff2ede00640 1 -- 172.21.15.123:0/231883630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f019dd10 con 0x7ff2f01075f0 2024-09-19T04:36:30.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.828+0000 7ff2ede00640 1 --2- 172.21.15.123:0/231883630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01075f0 0x7ff2f019d7d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00097f0 tx=0x7ff2e0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:30.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.828+0000 7ff2e6a00640 1 -- 172.21.15.123:0/231883630 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2e0037040 con 0x7ff2f01075f0 2024-09-19T04:36:30.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.828+0000 7ff2e6a00640 1 -- 172.21.15.123:0/231883630 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff2e0040070 con 0x7ff2f01075f0 2024-09-19T04:36:30.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.828+0000 7ff2ef200640 1 -- 172.21.15.123:0/231883630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f019dfa0 con 0x7ff2f01075f0 2024-09-19T04:36:30.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.828+0000 7ff2e6a00640 1 -- 172.21.15.123:0/231883630 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2e003a390 con 0x7ff2f01075f0 2024-09-19T04:36:30.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.828+0000 7ff2ef200640 1 -- 172.21.15.123:0/231883630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f01a0c90 con 0x7ff2f01075f0 2024-09-19T04:36:30.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.828+0000 7ff2e6a00640 1 -- 172.21.15.123:0/231883630 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff2e003a5e0 con 0x7ff2f01075f0 2024-09-19T04:36:30.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.828+0000 7ff2e6a00640 1 -- 172.21.15.123:0/231883630 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff2e0045d50 con 0x7ff2f01075f0 2024-09-19T04:36:30.562 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.829+0000 7ff2ef200640 1 -- 172.21.15.123:0/231883630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f0107a70 con 0x7ff2f01075f0 2024-09-19T04:36:30.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.832+0000 7ff2e6a00640 1 -- 172.21.15.123:0/231883630 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff2e003b430 con 0x7ff2f01075f0 2024-09-19T04:36:30.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.895+0000 7ff2ef200640 1 -- 172.21.15.123:0/231883630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff2f01a0eb0 con 0x7ff2f01075f0 2024-09-19T04:36:30.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.895+0000 7ff2e6a00640 1 -- 172.21.15.123:0/231883630 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7ff2e003b7c0 con 0x7ff2f01075f0 2024-09-19T04:36:30.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.897+0000 7ff2ef200640 1 -- 172.21.15.123:0/231883630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01075f0 msgr2=0x7ff2f019d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:30.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.897+0000 7ff2ef200640 1 --2- 172.21.15.123:0/231883630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01075f0 0x7ff2f019d7d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00097f0 tx=0x7ff2e0030d80 comp rx=0 tx=0).stop 2024-09-19T04:36:30.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.897+0000 7ff2ef200640 1 -- 172.21.15.123:0/231883630 shutdown_connections 2024-09-19T04:36:30.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.897+0000 7ff2ef200640 1 --2- 172.21.15.123:0/231883630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01075f0 0x7ff2f019d7d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:30.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.897+0000 7ff2ef200640 1 -- 172.21.15.123:0/231883630 >> 172.21.15.123:0/231883630 conn(0x7ff2f0102d40 msgr2=0x7ff2f0103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:30.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.897+0000 7ff2ef200640 1 -- 172.21.15.123:0/231883630 shutdown_connections 2024-09-19T04:36:30.563 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:29.897+0000 7ff2ef200640 1 -- 172.21.15.123:0/231883630 wait complete. 2024-09-19T04:36:30.563 INFO:teuthology.orchestra.run.smithi123.stdout:mgr not available, waiting (1/15)... 2024-09-19T04:36:30.927 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:30 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:30.711+0000 7f2d56894040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:36:32.177 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:31 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:31.795+0000 7f2d56894040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:36:32.177 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:31 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:31.873+0000 7f2d56894040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:36:33.067 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:32 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:32.797+0000 7f2d56894040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:36:33.068 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:32 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:32.934+0000 7f2d56894040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:36:33.321 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:33 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:33.102+0000 7f2d56894040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:36:33.322 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:33 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:33.176+0000 7f2d56894040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:36:33.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:33 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2503842451' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T04:36:33.677 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:33 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:33.355+0000 7f2d56894040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:36:33.677 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:33 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:33.441+0000 7f2d56894040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:36:33.677 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:33 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:33.515+0000 7f2d56894040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:36:33.962 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:33 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:33.699+0000 7f2d56894040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:36:33.962 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:33 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:33.779+0000 7f2d56894040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:36:33.962 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:33 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:33.859+0000 7f2d56894040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:36:34.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-09-19T04:36:34.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:36:34.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsid": "911fa052-7640-11ef-bceb-c7b262605968", 2024-09-19T04:36:34.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T04:36:34.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T04:36:34.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T04:36:34.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T04:36:34.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:34.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T04:36:34.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T04:36:34.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 0 2024-09-19T04:36:34.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:36:34.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T04:36:34.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T04:36:34.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:36:34.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-19T04:36:34.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T04:36:34.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:36:34.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T04:36:34.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T04:36:34.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:34.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T04:36:34.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:36:34.169 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T04:36:34.169 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T04:36:34.169 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T04:36:34.176 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T04:36:34.176 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T04:36:34.176 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T04:36:34.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:34.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T04:36:34.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T04:36:34.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T04:36:34.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T04:36:34.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T04:36:34.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T04:36:34.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T04:36:34.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T04:36:34.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T04:36:34.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:34.178 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T04:36:34.178 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:36:34.178 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T04:36:21:083029+0000", 2024-09-19T04:36:34.178 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T04:36:34.178 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T04:36:34.178 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:34.178 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T04:36:34.178 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T04:36:34.178 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T04:36:34.178 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T04:36:34.178 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T04:36:34.178 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T04:36:34.179 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T04:36:34.179 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:36:34.179 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:36:34.179 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:34.179 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T04:36:34.179 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:36:34.179 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T04:36:21.083852+0000", 2024-09-19T04:36:34.179 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:36:34.179 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:34.179 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T04:36:34.179 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:36:34.179 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.335+0000 7f4913800640 1 Processor -- start 2024-09-19T04:36:34.180 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.335+0000 7f4913800640 1 -- start start 2024-09-19T04:36:34.180 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.336+0000 7f4913800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c109fa0 0x7f490c10a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:34.180 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.336+0000 7f4913800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f490c10a970 con 0x7f490c109fa0 2024-09-19T04:36:34.180 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.336+0000 7f4910e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c109fa0 0x7f490c10a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:34.180 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.336+0000 7f4910e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c109fa0 0x7f490c10a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56044/0 (socket says 172.21.15.123:56044) 2024-09-19T04:36:34.180 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.336+0000 7f4910e00640 1 -- 172.21.15.123:0/2118434948 learned_addr learned my addr 172.21.15.123:0/2118434948 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:34.180 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.336+0000 7f4910e00640 1 -- 172.21.15.123:0/2118434948 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c10aaf0 con 0x7f490c109fa0 2024-09-19T04:36:34.180 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.337+0000 7f4910e00640 1 --2- 172.21.15.123:0/2118434948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c109fa0 0x7f490c10a3a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4900009820 tx=0x7f490002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740a3e50c8db35b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:34.180 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.338+0000 7f490b400640 1 -- 172.21.15.123:0/2118434948 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4900037070 con 0x7f490c109fa0 2024-09-19T04:36:34.180 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.338+0000 7f490b400640 1 -- 172.21.15.123:0/2118434948 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4900032900 con 0x7f490c109fa0 2024-09-19T04:36:34.180 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.338+0000 7f490b400640 1 -- 172.21.15.123:0/2118434948 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4900030850 con 0x7f490c109fa0 2024-09-19T04:36:34.180 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.338+0000 7f4913800640 1 -- 172.21.15.123:0/2118434948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c109fa0 msgr2=0x7f490c10a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:34.180 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.338+0000 7f4913800640 1 --2- 172.21.15.123:0/2118434948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c109fa0 0x7f490c10a3a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4900009820 tx=0x7f490002a270 comp rx=0 tx=0).stop 2024-09-19T04:36:34.181 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.338+0000 7f4913800640 1 -- 172.21.15.123:0/2118434948 shutdown_connections 2024-09-19T04:36:34.181 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.338+0000 7f4913800640 1 --2- 172.21.15.123:0/2118434948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c109fa0 0x7f490c10a3a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:34.181 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.338+0000 7f4913800640 1 -- 172.21.15.123:0/2118434948 >> 172.21.15.123:0/2118434948 conn(0x7f490c07cfe0 msgr2=0x7f490c07d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:34.181 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.339+0000 7f4913800640 1 -- 172.21.15.123:0/2118434948 shutdown_connections 2024-09-19T04:36:34.181 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.339+0000 7f4913800640 1 -- 172.21.15.123:0/2118434948 wait complete. 2024-09-19T04:36:34.181 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.339+0000 7f4913800640 1 Processor -- start 2024-09-19T04:36:34.181 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.340+0000 7f4913800640 1 -- start start 2024-09-19T04:36:34.181 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.340+0000 7f4913800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c109fa0 0x7f490c19d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:34.181 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.340+0000 7f4913800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f490c10df80 con 0x7f490c109fa0 2024-09-19T04:36:34.181 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.340+0000 7f4910e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c109fa0 0x7f490c19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:34.181 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.340+0000 7f4910e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c109fa0 0x7f490c19d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56048/0 (socket says 172.21.15.123:56048) 2024-09-19T04:36:34.181 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.340+0000 7f4910e00640 1 -- 172.21.15.123:0/2503842451 learned_addr learned my addr 172.21.15.123:0/2503842451 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:34.181 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.341+0000 7f4910e00640 1 -- 172.21.15.123:0/2503842451 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c19de60 con 0x7f490c109fa0 2024-09-19T04:36:34.182 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.341+0000 7f4910e00640 1 --2- 172.21.15.123:0/2503842451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c109fa0 0x7f490c19d920 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4900009940 tx=0x7f4900032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:34.182 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.341+0000 7f4909600640 1 -- 172.21.15.123:0/2503842451 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4900042070 con 0x7f490c109fa0 2024-09-19T04:36:34.182 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.341+0000 7f4909600640 1 -- 172.21.15.123:0/2503842451 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f49000308c0 con 0x7f490c109fa0 2024-09-19T04:36:34.182 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.341+0000 7f4913800640 1 -- 172.21.15.123:0/2503842451 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490c19e0f0 con 0x7f490c109fa0 2024-09-19T04:36:34.182 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.341+0000 7f4909600640 1 -- 172.21.15.123:0/2503842451 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4900037040 con 0x7f490c109fa0 2024-09-19T04:36:34.182 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.341+0000 7f4913800640 1 -- 172.21.15.123:0/2503842451 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490c1a0de0 con 0x7f490c109fa0 2024-09-19T04:36:34.182 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.342+0000 7f4909600640 1 -- 172.21.15.123:0/2503842451 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4900045070 con 0x7f490c109fa0 2024-09-19T04:36:34.182 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.342+0000 7f4909600640 1 -- 172.21.15.123:0/2503842451 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4900047bc0 con 0x7f490c109fa0 2024-09-19T04:36:34.182 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.342+0000 7f4913800640 1 -- 172.21.15.123:0/2503842451 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f490c10a420 con 0x7f490c109fa0 2024-09-19T04:36:34.182 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.345+0000 7f4909600640 1 -- 172.21.15.123:0/2503842451 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f49000327b0 con 0x7f490c109fa0 2024-09-19T04:36:34.182 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.409+0000 7f4913800640 1 -- 172.21.15.123:0/2503842451 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f490c1a1360 con 0x7f490c109fa0 2024-09-19T04:36:34.182 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.410+0000 7f4909600640 1 -- 172.21.15.123:0/2503842451 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f490003c800 con 0x7f490c109fa0 2024-09-19T04:36:34.183 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.411+0000 7f4913800640 1 -- 172.21.15.123:0/2503842451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c109fa0 msgr2=0x7f490c19d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:34.183 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.411+0000 7f4913800640 1 --2- 172.21.15.123:0/2503842451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c109fa0 0x7f490c19d920 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4900009940 tx=0x7f4900032ce0 comp rx=0 tx=0).stop 2024-09-19T04:36:34.183 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.411+0000 7f4913800640 1 -- 172.21.15.123:0/2503842451 shutdown_connections 2024-09-19T04:36:34.183 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.411+0000 7f4913800640 1 --2- 172.21.15.123:0/2503842451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c109fa0 0x7f490c19d920 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:34.183 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.411+0000 7f4913800640 1 -- 172.21.15.123:0/2503842451 >> 172.21.15.123:0/2503842451 conn(0x7f490c07cfe0 msgr2=0x7f490c107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:34.183 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.411+0000 7f4913800640 1 -- 172.21.15.123:0/2503842451 shutdown_connections 2024-09-19T04:36:34.183 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:33.411+0000 7f4913800640 1 -- 172.21.15.123:0/2503842451 wait complete. 2024-09-19T04:36:34.183 INFO:teuthology.orchestra.run.smithi123.stdout:mgr not available, waiting (2/15)... 2024-09-19T04:36:34.427 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:33 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:33.969+0000 7f2d56894040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:36:34.427 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:34 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:34.042+0000 7f2d56894040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:36:34.427 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:34 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:34.113+0000 7f2d56894040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:36:35.427 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:35.018+0000 7f2d56894040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:36:35.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:35 smithi123 ceph-mon[30511]: Activating manager daemon x 2024-09-19T04:36:35.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:35 smithi123 ceph-mon[30511]: mgrmap e2: x(active, starting, since 0.00426281s) 2024-09-19T04:36:35.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:35 smithi123 ceph-mon[30511]: from='mgr.14100 172.21.15.123:0/3764396207' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:36:35.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:35 smithi123 ceph-mon[30511]: from='mgr.14100 172.21.15.123:0/3764396207' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:36:35.428 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:35 smithi123 ceph-mon[30511]: from='mgr.14100 172.21.15.123:0/3764396207' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:36:35.428 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:35 smithi123 ceph-mon[30511]: from='mgr.14100 172.21.15.123:0/3764396207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:36:35.428 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:35 smithi123 ceph-mon[30511]: from='mgr.14100 172.21.15.123:0/3764396207' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:36:35.428 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:35 smithi123 ceph-mon[30511]: Manager daemon x is now available 2024-09-19T04:36:35.428 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:35 smithi123 ceph-mon[30511]: from='mgr.14100 172.21.15.123:0/3764396207' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:36:35.428 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:35 smithi123 ceph-mon[30511]: from='mgr.14100 172.21.15.123:0/3764396207' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:36:35.428 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:35 smithi123 ceph-mon[30511]: from='mgr.14100 172.21.15.123:0/3764396207' entity='mgr.x' 2024-09-19T04:36:35.428 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:35 smithi123 ceph-mon[30511]: from='mgr.14100 172.21.15.123:0/3764396207' entity='mgr.x' 2024-09-19T04:36:35.428 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:35 smithi123 ceph-mon[30511]: from='mgr.14100 172.21.15.123:0/3764396207' entity='mgr.x' 2024-09-19T04:36:37.220 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:37 smithi123 ceph-mon[30511]: mgrmap e3: x(active, since 1.00951s) 2024-09-19T04:36:37.220 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:37 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/192388553' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T04:36:37.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-09-19T04:36:37.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:36:37.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsid": "911fa052-7640-11ef-bceb-c7b262605968", 2024-09-19T04:36:37.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T04:36:37.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T04:36:37.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T04:36:37.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T04:36:37.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:37.490 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T04:36:37.491 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T04:36:37.491 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 0 2024-09-19T04:36:37.491 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:36:37.491 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T04:36:37.491 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T04:36:37.491 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:36:37.491 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-19T04:36:37.491 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T04:36:37.491 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:36:37.491 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T04:36:37.491 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T04:36:37.491 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:37.491 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T04:36:37.491 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:36:37.492 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T04:36:37.492 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T04:36:37.492 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T04:36:37.492 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T04:36:37.492 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T04:36:37.492 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T04:36:37.492 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:37.492 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T04:36:37.492 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T04:36:37.492 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T04:36:37.492 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T04:36:37.492 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T04:36:37.492 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T04:36:37.493 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T04:36:37.493 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T04:36:37.493 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T04:36:37.493 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:37.493 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T04:36:37.493 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:36:37.493 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T04:36:21:083029+0000", 2024-09-19T04:36:37.493 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T04:36:37.493 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T04:36:37.494 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:37.494 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T04:36:37.494 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T04:36:37.494 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T04:36:37.494 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T04:36:37.494 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T04:36:37.494 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T04:36:37.494 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T04:36:37.494 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:36:37.494 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:36:37.494 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:37.495 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T04:36:37.495 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:36:37.495 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T04:36:21.083852+0000", 2024-09-19T04:36:37.495 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:36:37.495 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:36:37.495 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T04:36:37.495 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:36:37.495 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.686+0000 7f6e0d200640 1 Processor -- start 2024-09-19T04:36:37.495 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.686+0000 7f6e0d200640 1 -- start start 2024-09-19T04:36:37.496 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.687+0000 7f6e0d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e08107790 0x7f6e08107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:37.496 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.687+0000 7f6e0d200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6e08108160 con 0x7f6e08107790 2024-09-19T04:36:37.496 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.687+0000 7f6e05e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e08107790 0x7f6e08107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:37.496 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.687+0000 7f6e05e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e08107790 0x7f6e08107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56130/0 (socket says 172.21.15.123:56130) 2024-09-19T04:36:37.496 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.687+0000 7f6e05e00640 1 -- 172.21.15.123:0/2473442314 learned_addr learned my addr 172.21.15.123:0/2473442314 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:37.496 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.687+0000 7f6e05e00640 1 -- 172.21.15.123:0/2473442314 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e08108940 con 0x7f6e08107790 2024-09-19T04:36:37.496 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.688+0000 7f6e05e00640 1 --2- 172.21.15.123:0/2473442314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e08107790 0x7f6e08107b90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009820 tx=0x7f6df802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1f57eeab122a5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:37.496 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.689+0000 7f6e05400640 1 -- 172.21.15.123:0/2473442314 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df8037070 con 0x7f6e08107790 2024-09-19T04:36:37.496 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.689+0000 7f6e05400640 1 -- 172.21.15.123:0/2473442314 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6df8032900 con 0x7f6e08107790 2024-09-19T04:36:37.496 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.689+0000 7f6e05400640 1 -- 172.21.15.123:0/2473442314 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df8030850 con 0x7f6e08107790 2024-09-19T04:36:37.496 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.689+0000 7f6e0d200640 1 -- 172.21.15.123:0/2473442314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e08107790 msgr2=0x7f6e08107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:37.496 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.689+0000 7f6e0d200640 1 --2- 172.21.15.123:0/2473442314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e08107790 0x7f6e08107b90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009820 tx=0x7f6df802a270 comp rx=0 tx=0).stop 2024-09-19T04:36:37.497 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.689+0000 7f6e0d200640 1 -- 172.21.15.123:0/2473442314 shutdown_connections 2024-09-19T04:36:37.497 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.689+0000 7f6e0d200640 1 --2- 172.21.15.123:0/2473442314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e08107790 0x7f6e08107b90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:37.497 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.689+0000 7f6e0d200640 1 -- 172.21.15.123:0/2473442314 >> 172.21.15.123:0/2473442314 conn(0x7f6e08102fa0 msgr2=0x7f6e081053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:37.497 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.689+0000 7f6e0d200640 1 -- 172.21.15.123:0/2473442314 shutdown_connections 2024-09-19T04:36:37.497 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.689+0000 7f6e0d200640 1 -- 172.21.15.123:0/2473442314 wait complete. 2024-09-19T04:36:37.497 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.690+0000 7f6e0d200640 1 Processor -- start 2024-09-19T04:36:37.497 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.690+0000 7f6e0d200640 1 -- start start 2024-09-19T04:36:37.497 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.691+0000 7f6e0d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e08107790 0x7f6e0819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:37.497 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.691+0000 7f6e0d200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0810bcf0 con 0x7f6e08107790 2024-09-19T04:36:37.497 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.691+0000 7f6e05e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e08107790 0x7f6e0819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:37.497 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.691+0000 7f6e05e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e08107790 0x7f6e0819d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56142/0 (socket says 172.21.15.123:56142) 2024-09-19T04:36:37.497 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.691+0000 7f6e05e00640 1 -- 172.21.15.123:0/192388553 learned_addr learned my addr 172.21.15.123:0/192388553 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:37.497 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.691+0000 7f6e05e00640 1 -- 172.21.15.123:0/192388553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0819deb0 con 0x7f6e08107790 2024-09-19T04:36:37.498 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.691+0000 7f6e05e00640 1 --2- 172.21.15.123:0/192388553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e08107790 0x7f6e0819d970 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6df80097f0 tx=0x7f6df8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:37.498 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.692+0000 7f6dfea00640 1 -- 172.21.15.123:0/192388553 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df8037040 con 0x7f6e08107790 2024-09-19T04:36:37.498 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.692+0000 7f6dfea00640 1 -- 172.21.15.123:0/192388553 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6df8040070 con 0x7f6e08107790 2024-09-19T04:36:37.498 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.692+0000 7f6e0d200640 1 -- 172.21.15.123:0/192388553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0819e140 con 0x7f6e08107790 2024-09-19T04:36:37.498 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.692+0000 7f6dfea00640 1 -- 172.21.15.123:0/192388553 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df803a390 con 0x7f6e08107790 2024-09-19T04:36:37.498 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.692+0000 7f6e0d200640 1 -- 172.21.15.123:0/192388553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e081a0e30 con 0x7f6e08107790 2024-09-19T04:36:37.498 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.693+0000 7f6dfea00640 1 -- 172.21.15.123:0/192388553 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7f6df803b430 con 0x7f6e08107790 2024-09-19T04:36:37.498 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.693+0000 7f6e0d200640 1 -- 172.21.15.123:0/192388553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e08107c10 con 0x7f6e08107790 2024-09-19T04:36:37.498 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.693+0000 7f6dfea00640 1 --2- 172.21.15.123:0/192388553 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f6dec0390a0 0x7f6dec03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:37.498 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.693+0000 7f6dfea00640 1 -- 172.21.15.123:0/192388553 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6df806c1b0 con 0x7f6e08107790 2024-09-19T04:36:37.498 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.694+0000 7f6dffe00640 1 --2- 172.21.15.123:0/192388553 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f6dec0390a0 0x7f6dec03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:37.499 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.694+0000 7f6dffe00640 1 --2- 172.21.15.123:0/192388553 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f6dec0390a0 0x7f6dec03b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6df00098c0 tx=0x7f6df0006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:37.499 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.699+0000 7f6dfea00640 1 -- 172.21.15.123:0/192388553 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6df8032900 con 0x7f6e08107790 2024-09-19T04:36:37.499 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.936+0000 7f6e0d200640 1 -- 172.21.15.123:0/192388553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6e081a1440 con 0x7f6e08107790 2024-09-19T04:36:37.499 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.937+0000 7f6dfea00640 1 -- 172.21.15.123:0/192388553 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f6df8040ca0 con 0x7f6e08107790 2024-09-19T04:36:37.499 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.939+0000 7f6e0d200640 1 -- 172.21.15.123:0/192388553 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f6dec0390a0 msgr2=0x7f6dec03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:37.499 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.939+0000 7f6e0d200640 1 --2- 172.21.15.123:0/192388553 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f6dec0390a0 0x7f6dec03b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6df00098c0 tx=0x7f6df0006db0 comp rx=0 tx=0).stop 2024-09-19T04:36:37.499 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.939+0000 7f6e0d200640 1 -- 172.21.15.123:0/192388553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e08107790 msgr2=0x7f6e0819d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:37.499 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.939+0000 7f6e0d200640 1 --2- 172.21.15.123:0/192388553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e08107790 0x7f6e0819d970 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6df80097f0 tx=0x7f6df8030d80 comp rx=0 tx=0).stop 2024-09-19T04:36:37.499 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.940+0000 7f6e0d200640 1 -- 172.21.15.123:0/192388553 shutdown_connections 2024-09-19T04:36:37.499 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.940+0000 7f6e0d200640 1 --2- 172.21.15.123:0/192388553 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f6dec0390a0 0x7f6dec03b560 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:37.499 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.940+0000 7f6e0d200640 1 --2- 172.21.15.123:0/192388553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e08107790 0x7f6e0819d970 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:37.499 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.940+0000 7f6e0d200640 1 -- 172.21.15.123:0/192388553 >> 172.21.15.123:0/192388553 conn(0x7f6e08102fa0 msgr2=0x7f6e08103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:37.499 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.940+0000 7f6e0d200640 1 -- 172.21.15.123:0/192388553 shutdown_connections 2024-09-19T04:36:37.500 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:36.940+0000 7f6e0d200640 1 -- 172.21.15.123:0/192388553 wait complete. 2024-09-19T04:36:37.500 INFO:teuthology.orchestra.run.smithi123.stdout:mgr is available 2024-09-19T04:36:38.180 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:38 smithi123 ceph-mon[30511]: mgrmap e4: x(active, since 2s) 2024-09-19T04:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-09-19T04:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T04:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout fsid = 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.123:3300,v1:172.21.15.123:6789] 2024-09-19T04:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T04:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T04:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T04:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T04:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-09-19T04:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T04:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T04:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-09-19T04:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T04:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T04:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T04:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T04:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T04:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T04:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:37.997+0000 7f8c19c00640 1 Processor -- start 2024-09-19T04:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:37.997+0000 7f8c19c00640 1 -- start start 2024-09-19T04:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:37.998+0000 7f8c19c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c14107790 0x7f8c14107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:37.998+0000 7f8c19c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c14108160 con 0x7f8c14107790 2024-09-19T04:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:37.998+0000 7f8c12a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c14107790 0x7f8c14107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:37.998+0000 7f8c12a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c14107790 0x7f8c14107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56152/0 (socket says 172.21.15.123:56152) 2024-09-19T04:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:37.998+0000 7f8c12a00640 1 -- 172.21.15.123:0/4294735281 learned_addr learned my addr 172.21.15.123:0/4294735281 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:37.998+0000 7f8c12a00640 1 -- 172.21.15.123:0/4294735281 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c14108940 con 0x7f8c14107790 2024-09-19T04:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:37.999+0000 7f8c12a00640 1 --2- 172.21.15.123:0/4294735281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c14107790 0x7f8c14107b90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009820 tx=0x7f8c0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27db13b020676d0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.000+0000 7f8c12000640 1 -- 172.21.15.123:0/4294735281 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c04037070 con 0x7f8c14107790 2024-09-19T04:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.000+0000 7f8c12000640 1 -- 172.21.15.123:0/4294735281 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8c04032900 con 0x7f8c14107790 2024-09-19T04:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.000+0000 7f8c12000640 1 -- 172.21.15.123:0/4294735281 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c04030850 con 0x7f8c14107790 2024-09-19T04:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.000+0000 7f8c19c00640 1 -- 172.21.15.123:0/4294735281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c14107790 msgr2=0x7f8c14107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.000+0000 7f8c19c00640 1 --2- 172.21.15.123:0/4294735281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c14107790 0x7f8c14107b90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009820 tx=0x7f8c0402a270 comp rx=0 tx=0).stop 2024-09-19T04:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.000+0000 7f8c19c00640 1 -- 172.21.15.123:0/4294735281 shutdown_connections 2024-09-19T04:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.000+0000 7f8c19c00640 1 --2- 172.21.15.123:0/4294735281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c14107790 0x7f8c14107b90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.000+0000 7f8c19c00640 1 -- 172.21.15.123:0/4294735281 >> 172.21.15.123:0/4294735281 conn(0x7f8c14102fa0 msgr2=0x7f8c141053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.000+0000 7f8c19c00640 1 -- 172.21.15.123:0/4294735281 shutdown_connections 2024-09-19T04:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.000+0000 7f8c19c00640 1 -- 172.21.15.123:0/4294735281 wait complete. 2024-09-19T04:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.001+0000 7f8c19c00640 1 Processor -- start 2024-09-19T04:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.001+0000 7f8c19c00640 1 -- start start 2024-09-19T04:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.002+0000 7f8c19c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c14107790 0x7f8c1419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.002+0000 7f8c19c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1410bcf0 con 0x7f8c14107790 2024-09-19T04:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.002+0000 7f8c12a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c14107790 0x7f8c1419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.002+0000 7f8c12a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c14107790 0x7f8c1419b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56164/0 (socket says 172.21.15.123:56164) 2024-09-19T04:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.002+0000 7f8c12a00640 1 -- 172.21.15.123:0/3693586576 learned_addr learned my addr 172.21.15.123:0/3693586576 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.002+0000 7f8c12a00640 1 -- 172.21.15.123:0/3693586576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1419bc80 con 0x7f8c14107790 2024-09-19T04:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.003+0000 7f8c12a00640 1 --2- 172.21.15.123:0/3693586576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c14107790 0x7f8c1419b740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8c040097f0 tx=0x7f8c04030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.003+0000 7f8c0be00640 1 -- 172.21.15.123:0/3693586576 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c04037040 con 0x7f8c14107790 2024-09-19T04:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.003+0000 7f8c0be00640 1 -- 172.21.15.123:0/3693586576 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8c04040070 con 0x7f8c14107790 2024-09-19T04:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.003+0000 7f8c19c00640 1 -- 172.21.15.123:0/3693586576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1419bf10 con 0x7f8c14107790 2024-09-19T04:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.003+0000 7f8c0be00640 1 -- 172.21.15.123:0/3693586576 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c0403a390 con 0x7f8c14107790 2024-09-19T04:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.003+0000 7f8c19c00640 1 -- 172.21.15.123:0/3693586576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1419ec00 con 0x7f8c14107790 2024-09-19T04:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.004+0000 7f8c0be00640 1 -- 172.21.15.123:0/3693586576 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f8c0403a5e0 con 0x7f8c14107790 2024-09-19T04:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.004+0000 7f8c19c00640 1 -- 172.21.15.123:0/3693586576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c14107c10 con 0x7f8c14107790 2024-09-19T04:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.004+0000 7f8c0be00640 1 --2- 172.21.15.123:0/3693586576 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f8bf80391b0 0x7f8bf803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.004+0000 7f8c0be00640 1 -- 172.21.15.123:0/3693586576 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8c0406c4a0 con 0x7f8c14107790 2024-09-19T04:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.005+0000 7f8c0a000640 1 --2- 172.21.15.123:0/3693586576 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f8bf80391b0 0x7f8bf803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.005+0000 7f8c0a000640 1 --2- 172.21.15.123:0/3693586576 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f8bf80391b0 0x7f8bf803b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0098c0 tx=0x7f8bfc006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.010+0000 7f8c0be00640 1 -- 172.21.15.123:0/3693586576 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8c0403a980 con 0x7f8c14107790 2024-09-19T04:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.193+0000 7f8c19c00640 1 -- 172.21.15.123:0/3693586576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f8c1419c390 con 0x7f8c14107790 2024-09-19T04:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.194+0000 7f8c0be00640 1 -- 172.21.15.123:0/3693586576 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f8c0406c200 con 0x7f8c14107790 2024-09-19T04:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.196+0000 7f8c19c00640 1 -- 172.21.15.123:0/3693586576 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f8bf80391b0 msgr2=0x7f8bf803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.196+0000 7f8c19c00640 1 --2- 172.21.15.123:0/3693586576 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f8bf80391b0 0x7f8bf803b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0098c0 tx=0x7f8bfc006db0 comp rx=0 tx=0).stop 2024-09-19T04:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.196+0000 7f8c19c00640 1 -- 172.21.15.123:0/3693586576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c14107790 msgr2=0x7f8c1419b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.196+0000 7f8c19c00640 1 --2- 172.21.15.123:0/3693586576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c14107790 0x7f8c1419b740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8c040097f0 tx=0x7f8c04030d80 comp rx=0 tx=0).stop 2024-09-19T04:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.196+0000 7f8c19c00640 1 -- 172.21.15.123:0/3693586576 shutdown_connections 2024-09-19T04:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.196+0000 7f8c19c00640 1 --2- 172.21.15.123:0/3693586576 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f8bf80391b0 0x7f8bf803b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.196+0000 7f8c19c00640 1 --2- 172.21.15.123:0/3693586576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c14107790 0x7f8c1419b740 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.196+0000 7f8c19c00640 1 -- 172.21.15.123:0/3693586576 >> 172.21.15.123:0/3693586576 conn(0x7f8c14102fa0 msgr2=0x7f8c14103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.197+0000 7f8c19c00640 1 -- 172.21.15.123:0/3693586576 shutdown_connections 2024-09-19T04:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:38.197+0000 7f8c19c00640 1 -- 172.21.15.123:0/3693586576 wait complete. 2024-09-19T04:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout:Enabling cephadm module... 2024-09-19T04:36:39.078 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:39 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3693586576' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T04:36:40.312 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:40 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1675070151' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T04:36:40.312 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: ignoring --setuser ceph since I am not root 2024-09-19T04:36:40.313 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: ignoring --setgroup ceph since I am not root 2024-09-19T04:36:40.313 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:40.257+0000 7f7d827dc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:36:40.602 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:40.337+0000 7f7d827dc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:36:40.602 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:40.519+0000 7f7d827dc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:36:40.653 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.312+0000 7f21c0600640 1 Processor -- start 2024-09-19T04:36:40.653 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.312+0000 7f21c0600640 1 -- start start 2024-09-19T04:36:40.654 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.312+0000 7f21c0600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b8109950 0x7f21b8109d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:40.654 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.312+0000 7f21c0600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21b810a320 con 0x7f21b8109950 2024-09-19T04:36:40.654 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.313+0000 7f21bdc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b8109950 0x7f21b8109d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:40.654 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.313+0000 7f21bdc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b8109950 0x7f21b8109d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56174/0 (socket says 172.21.15.123:56174) 2024-09-19T04:36:40.654 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.313+0000 7f21bdc00640 1 -- 172.21.15.123:0/2656745867 learned_addr learned my addr 172.21.15.123:0/2656745867 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:40.654 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.313+0000 7f21bdc00640 1 -- 172.21.15.123:0/2656745867 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b810ab50 con 0x7f21b8109950 2024-09-19T04:36:40.654 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.314+0000 7f21bdc00640 1 --2- 172.21.15.123:0/2656745867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b8109950 0x7f21b8109d50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009820 tx=0x7f21ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708ff599344c04ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:40.654 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.314+0000 7f21b7e00640 1 -- 172.21.15.123:0/2656745867 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21ac037070 con 0x7f21b8109950 2024-09-19T04:36:40.654 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.314+0000 7f21b7e00640 1 -- 172.21.15.123:0/2656745867 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f21ac032900 con 0x7f21b8109950 2024-09-19T04:36:40.654 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.314+0000 7f21b7e00640 1 -- 172.21.15.123:0/2656745867 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21ac030850 con 0x7f21b8109950 2024-09-19T04:36:40.654 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.315+0000 7f21c0600640 1 -- 172.21.15.123:0/2656745867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b8109950 msgr2=0x7f21b8109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:40.654 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.315+0000 7f21c0600640 1 --2- 172.21.15.123:0/2656745867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b8109950 0x7f21b8109d50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009820 tx=0x7f21ac02a270 comp rx=0 tx=0).stop 2024-09-19T04:36:40.654 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.315+0000 7f21c0600640 1 -- 172.21.15.123:0/2656745867 shutdown_connections 2024-09-19T04:36:40.655 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.315+0000 7f21c0600640 1 --2- 172.21.15.123:0/2656745867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b8109950 0x7f21b8109d50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:40.655 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.315+0000 7f21c0600640 1 -- 172.21.15.123:0/2656745867 >> 172.21.15.123:0/2656745867 conn(0x7f21b807cf30 msgr2=0x7f21b807d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:40.655 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.315+0000 7f21c0600640 1 -- 172.21.15.123:0/2656745867 shutdown_connections 2024-09-19T04:36:40.655 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.315+0000 7f21c0600640 1 -- 172.21.15.123:0/2656745867 wait complete. 2024-09-19T04:36:40.655 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.316+0000 7f21c0600640 1 Processor -- start 2024-09-19T04:36:40.655 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.316+0000 7f21c0600640 1 -- start start 2024-09-19T04:36:40.655 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.316+0000 7f21c0600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b8109950 0x7f21b819d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:40.655 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.316+0000 7f21c0600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21b810df00 con 0x7f21b8109950 2024-09-19T04:36:40.655 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.316+0000 7f21bdc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b8109950 0x7f21b819d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:40.655 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.316+0000 7f21bdc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b8109950 0x7f21b819d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56184/0 (socket says 172.21.15.123:56184) 2024-09-19T04:36:40.655 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.316+0000 7f21bdc00640 1 -- 172.21.15.123:0/1675070151 learned_addr learned my addr 172.21.15.123:0/1675070151 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:40.655 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.317+0000 7f21bdc00640 1 -- 172.21.15.123:0/1675070151 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b819df30 con 0x7f21b8109950 2024-09-19T04:36:40.655 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.317+0000 7f21bdc00640 1 --2- 172.21.15.123:0/1675070151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b8109950 0x7f21b819d9f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0097f0 tx=0x7f21ac030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:40.655 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.317+0000 7f21b6000640 1 -- 172.21.15.123:0/1675070151 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21ac042070 con 0x7f21b8109950 2024-09-19T04:36:40.655 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.317+0000 7f21b6000640 1 -- 172.21.15.123:0/1675070151 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f21ac032ca0 con 0x7f21b8109950 2024-09-19T04:36:40.656 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.317+0000 7f21b6000640 1 -- 172.21.15.123:0/1675070151 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21ac037040 con 0x7f21b8109950 2024-09-19T04:36:40.656 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.317+0000 7f21c0600640 1 -- 172.21.15.123:0/1675070151 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b819e1c0 con 0x7f21b8109950 2024-09-19T04:36:40.656 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.317+0000 7f21c0600640 1 -- 172.21.15.123:0/1675070151 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b81a0eb0 con 0x7f21b8109950 2024-09-19T04:36:40.656 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.318+0000 7f21b6000640 1 -- 172.21.15.123:0/1675070151 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f21ac045070 con 0x7f21b8109950 2024-09-19T04:36:40.656 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.319+0000 7f21c0600640 1 -- 172.21.15.123:0/1675070151 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b810e080 con 0x7f21b8109950 2024-09-19T04:36:40.656 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.319+0000 7f21b6000640 1 --2- 172.21.15.123:0/1675070151 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f21a00391b0 0x7f21a003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:40.656 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.319+0000 7f21b6000640 1 -- 172.21.15.123:0/1675070151 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f21ac06c080 con 0x7f21b8109950 2024-09-19T04:36:40.656 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.320+0000 7f21bd200640 1 --2- 172.21.15.123:0/1675070151 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f21a00391b0 0x7f21a003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:40.656 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.320+0000 7f21bd200640 1 --2- 172.21.15.123:0/1675070151 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f21a00391b0 0x7f21a003b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f21a40098c0 tx=0x7f21a4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:40.656 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.324+0000 7f21b6000640 1 -- 172.21.15.123:0/1675070151 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f21ac04b170 con 0x7f21b8109950 2024-09-19T04:36:40.656 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:39.548+0000 7f21c0600640 1 -- 172.21.15.123:0/1675070151 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f21b81a1520 con 0x7f21b8109950 2024-09-19T04:36:40.656 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:40.035+0000 7f21b6000640 1 -- 172.21.15.123:0/1675070151 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f21ac03d430 con 0x7f21b8109950 2024-09-19T04:36:40.656 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:40.036+0000 7f21b6000640 1 -- 172.21.15.123:0/1675070151 <== mon.0 v2:172.21.15.123:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f21ac03da60 con 0x7f21b8109950 2024-09-19T04:36:40.657 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:40.038+0000 7f21c0600640 1 -- 172.21.15.123:0/1675070151 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f21a00391b0 msgr2=0x7f21a003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:40.657 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:40.038+0000 7f21c0600640 1 --2- 172.21.15.123:0/1675070151 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f21a00391b0 0x7f21a003b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f21a40098c0 tx=0x7f21a4006db0 comp rx=0 tx=0).stop 2024-09-19T04:36:40.657 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:40.038+0000 7f21c0600640 1 -- 172.21.15.123:0/1675070151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b8109950 msgr2=0x7f21b819d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:40.657 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:40.038+0000 7f21c0600640 1 --2- 172.21.15.123:0/1675070151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b8109950 0x7f21b819d9f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0097f0 tx=0x7f21ac030ec0 comp rx=0 tx=0).stop 2024-09-19T04:36:40.657 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:40.039+0000 7f21c0600640 1 -- 172.21.15.123:0/1675070151 shutdown_connections 2024-09-19T04:36:40.657 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:40.039+0000 7f21c0600640 1 --2- 172.21.15.123:0/1675070151 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f21a00391b0 0x7f21a003b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:40.657 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:40.039+0000 7f21c0600640 1 --2- 172.21.15.123:0/1675070151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b8109950 0x7f21b819d9f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:40.657 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:40.039+0000 7f21c0600640 1 -- 172.21.15.123:0/1675070151 >> 172.21.15.123:0/1675070151 conn(0x7f21b807cf30 msgr2=0x7f21b810bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:40.657 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:40.039+0000 7f21c0600640 1 -- 172.21.15.123:0/1675070151 shutdown_connections 2024-09-19T04:36:40.657 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:40.039+0000 7f21c0600640 1 -- 172.21.15.123:0/1675070151 wait complete. 2024-09-19T04:36:40.927 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:40.600+0000 7f7d827dc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:36:40.927 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:40.787+0000 7f7d827dc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:36:41.186 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:41 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1675070151' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T04:36:41.186 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:41 smithi123 ceph-mon[30511]: mgrmap e5: x(active, since 5s) 2024-09-19T04:36:41.187 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:40.962+0000 7f7d827dc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:36:41.442 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:41 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:41.341+0000 7f7d827dc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:36:41.442 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:41 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:41.414+0000 7f7d827dc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:36:41.764 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:41 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:41.630+0000 7f7d827dc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:36:42.055 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:36:42.055 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-19T04:36:42.055 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T04:36:42.055 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T04:36:42.055 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T04:36:42.055 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:36:42.055 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.184+0000 7f0a92800640 1 Processor -- start 2024-09-19T04:36:42.056 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.184+0000 7f0a92800640 1 -- start start 2024-09-19T04:36:42.056 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.185+0000 7f0a92800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c1075d0 0x7f0a8c1079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:42.056 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.185+0000 7f0a92800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c107fa0 con 0x7f0a8c1075d0 2024-09-19T04:36:42.056 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.185+0000 7f0a91400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c1075d0 0x7f0a8c1079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:42.056 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.185+0000 7f0a91400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c1075d0 0x7f0a8c1079d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56204/0 (socket says 172.21.15.123:56204) 2024-09-19T04:36:42.056 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.185+0000 7f0a91400640 1 -- 172.21.15.123:0/3383217707 learned_addr learned my addr 172.21.15.123:0/3383217707 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:42.056 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.186+0000 7f0a91400640 1 -- 172.21.15.123:0/3383217707 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c108780 con 0x7f0a8c1075d0 2024-09-19T04:36:42.056 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.187+0000 7f0a91400640 1 --2- 172.21.15.123:0/3383217707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c1075d0 0x7f0a8c1079d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009820 tx=0x7f0a8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92240cf8b13b592 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:42.056 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.187+0000 7f0a8be00640 1 -- 172.21.15.123:0/3383217707 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a80037070 con 0x7f0a8c1075d0 2024-09-19T04:36:42.056 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.187+0000 7f0a8be00640 1 -- 172.21.15.123:0/3383217707 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0a80032900 con 0x7f0a8c1075d0 2024-09-19T04:36:42.056 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.187+0000 7f0a8be00640 1 -- 172.21.15.123:0/3383217707 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a80030850 con 0x7f0a8c1075d0 2024-09-19T04:36:42.056 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.188+0000 7f0a92800640 1 -- 172.21.15.123:0/3383217707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c1075d0 msgr2=0x7f0a8c1079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:42.057 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.188+0000 7f0a92800640 1 --2- 172.21.15.123:0/3383217707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c1075d0 0x7f0a8c1079d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009820 tx=0x7f0a8002a270 comp rx=0 tx=0).stop 2024-09-19T04:36:42.057 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.188+0000 7f0a92800640 1 -- 172.21.15.123:0/3383217707 shutdown_connections 2024-09-19T04:36:42.057 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.188+0000 7f0a92800640 1 --2- 172.21.15.123:0/3383217707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c1075d0 0x7f0a8c1079d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:42.057 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.188+0000 7f0a92800640 1 -- 172.21.15.123:0/3383217707 >> 172.21.15.123:0/3383217707 conn(0x7f0a8c102da0 msgr2=0x7f0a8c1051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:42.057 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.188+0000 7f0a92800640 1 -- 172.21.15.123:0/3383217707 shutdown_connections 2024-09-19T04:36:42.057 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.188+0000 7f0a92800640 1 -- 172.21.15.123:0/3383217707 wait complete. 2024-09-19T04:36:42.057 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.189+0000 7f0a92800640 1 Processor -- start 2024-09-19T04:36:42.057 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.189+0000 7f0a92800640 1 -- start start 2024-09-19T04:36:42.057 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.189+0000 7f0a92800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c1075d0 0x7f0a8c19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:42.057 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.189+0000 7f0a92800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c10bb30 con 0x7f0a8c1075d0 2024-09-19T04:36:42.057 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.190+0000 7f0a91400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c1075d0 0x7f0a8c19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:42.057 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.190+0000 7f0a91400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c1075d0 0x7f0a8c19d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56212/0 (socket says 172.21.15.123:56212) 2024-09-19T04:36:42.057 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.190+0000 7f0a91400640 1 -- 172.21.15.123:0/2142379725 learned_addr learned my addr 172.21.15.123:0/2142379725 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:42.058 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.190+0000 7f0a91400640 1 -- 172.21.15.123:0/2142379725 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c19dce0 con 0x7f0a8c1075d0 2024-09-19T04:36:42.058 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.190+0000 7f0a91400640 1 --2- 172.21.15.123:0/2142379725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c1075d0 0x7f0a8c19d7a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0a800097f0 tx=0x7f0a80030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:42.058 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.191+0000 7f0a8a000640 1 -- 172.21.15.123:0/2142379725 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a80037040 con 0x7f0a8c1075d0 2024-09-19T04:36:42.058 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.191+0000 7f0a8a000640 1 -- 172.21.15.123:0/2142379725 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0a80040070 con 0x7f0a8c1075d0 2024-09-19T04:36:42.058 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.191+0000 7f0a92800640 1 -- 172.21.15.123:0/2142379725 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8c19df70 con 0x7f0a8c1075d0 2024-09-19T04:36:42.058 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.191+0000 7f0a8a000640 1 -- 172.21.15.123:0/2142379725 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a8003a390 con 0x7f0a8c1075d0 2024-09-19T04:36:42.058 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.191+0000 7f0a92800640 1 -- 172.21.15.123:0/2142379725 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a8c1a0c60 con 0x7f0a8c1075d0 2024-09-19T04:36:42.058 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.192+0000 7f0a8a000640 1 -- 172.21.15.123:0/2142379725 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f0a8003a5e0 con 0x7f0a8c1075d0 2024-09-19T04:36:42.058 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.192+0000 7f0a92800640 1 -- 172.21.15.123:0/2142379725 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a8c107a50 con 0x7f0a8c1075d0 2024-09-19T04:36:42.058 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.192+0000 7f0a8a000640 1 --2- 172.21.15.123:0/2142379725 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f0a74039200 0x7f0a7403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:42.058 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.192+0000 7f0a8a000640 1 -- 172.21.15.123:0/2142379725 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0a8006c500 con 0x7f0a8c1075d0 2024-09-19T04:36:42.058 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.192+0000 7f0a90a00640 1 -- 172.21.15.123:0/2142379725 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f0a74039200 msgr2=0x7f0a7403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2065492981 2024-09-19T04:36:42.058 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.192+0000 7f0a90a00640 1 --2- 172.21.15.123:0/2142379725 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f0a74039200 0x7f0a7403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:36:42.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.197+0000 7f0a8a000640 1 -- 172.21.15.123:0/2142379725 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0a8006c260 con 0x7f0a8c1075d0 2024-09-19T04:36:42.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.393+0000 7f0a90a00640 1 -- 172.21.15.123:0/2142379725 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f0a74039200 msgr2=0x7f0a7403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2065492981 2024-09-19T04:36:42.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.393+0000 7f0a90a00640 1 --2- 172.21.15.123:0/2142379725 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f0a74039200 0x7f0a7403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:36:42.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.424+0000 7f0a92800640 1 -- 172.21.15.123:0/2142379725 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f0a8c1a1330 con 0x7f0a8c1075d0 2024-09-19T04:36:42.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.425+0000 7f0a8a000640 1 -- 172.21.15.123:0/2142379725 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f0a80043760 con 0x7f0a8c1075d0 2024-09-19T04:36:42.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.427+0000 7f0a92800640 1 -- 172.21.15.123:0/2142379725 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f0a74039200 msgr2=0x7f0a7403b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:36:42.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.427+0000 7f0a92800640 1 --2- 172.21.15.123:0/2142379725 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f0a74039200 0x7f0a7403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:42.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.427+0000 7f0a92800640 1 -- 172.21.15.123:0/2142379725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c1075d0 msgr2=0x7f0a8c19d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:42.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.427+0000 7f0a92800640 1 --2- 172.21.15.123:0/2142379725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c1075d0 0x7f0a8c19d7a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0a800097f0 tx=0x7f0a80030d80 comp rx=0 tx=0).stop 2024-09-19T04:36:42.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.427+0000 7f0a92800640 1 -- 172.21.15.123:0/2142379725 shutdown_connections 2024-09-19T04:36:42.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.427+0000 7f0a92800640 1 --2- 172.21.15.123:0/2142379725 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7f0a74039200 0x7f0a7403b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:42.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.427+0000 7f0a92800640 1 --2- 172.21.15.123:0/2142379725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c1075d0 0x7f0a8c19d7a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:42.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.427+0000 7f0a92800640 1 -- 172.21.15.123:0/2142379725 >> 172.21.15.123:0/2142379725 conn(0x7f0a8c102da0 msgr2=0x7f0a8c1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:42.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.427+0000 7f0a92800640 1 -- 172.21.15.123:0/2142379725 shutdown_connections 2024-09-19T04:36:42.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:41.427+0000 7f0a92800640 1 -- 172.21.15.123:0/2142379725 wait complete. 2024-09-19T04:36:42.060 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for the mgr to restart... 2024-09-19T04:36:42.060 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mgr epoch 5... 2024-09-19T04:36:42.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:42 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2142379725' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T04:36:43.177 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:42.781+0000 7f7d827dc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:36:43.177 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:42.862+0000 7f7d827dc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:36:44.051 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:43.753+0000 7f7d827dc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:36:44.052 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:43.888+0000 7f7d827dc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:36:44.052 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:44 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:44.049+0000 7f7d827dc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:36:44.371 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:44 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:44.121+0000 7f7d827dc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:36:44.371 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:44 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:44.288+0000 7f7d827dc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:36:44.371 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:44 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:44.369+0000 7f7d827dc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:36:44.677 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:44 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:44.442+0000 7f7d827dc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:36:44.677 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:44 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:44.619+0000 7f7d827dc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:36:45.010 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:44 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:44.695+0000 7f7d827dc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:36:45.010 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:44 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:44.773+0000 7f7d827dc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:36:45.010 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:44 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:44.869+0000 7f7d827dc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:36:45.010 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:44 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:44.939+0000 7f7d827dc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:36:45.011 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:45 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:45.008+0000 7f7d827dc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:36:46.177 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:45 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:45.910+0000 7f7d827dc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:36:46.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:45 smithi123 ceph-mon[30511]: Active manager daemon x restarted 2024-09-19T04:36:46.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:45 smithi123 ceph-mon[30511]: Activating manager daemon x 2024-09-19T04:36:46.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:45 smithi123 ceph-mon[30511]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T04:36:46.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:45 smithi123 ceph-mon[30511]: mgrmap e6: x(active, starting, since 0.00390435s) 2024-09-19T04:36:46.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:45 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:36:46.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:45 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:36:46.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:45 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:36:46.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:45 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:36:46.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:45 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:36:46.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:45 smithi123 ceph-mon[30511]: Manager daemon x is now available 2024-09-19T04:36:46.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:45 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' 2024-09-19T04:36:46.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:45 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' 2024-09-19T04:36:46.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:45 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:36:46.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:45 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:36:47.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:46 smithi123 ceph-mon[30511]: Found migration_current of "None". Setting to last migration. 2024-09-19T04:36:47.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:46 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:36:47.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:46 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:36:47.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:46 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' 2024-09-19T04:36:47.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:46 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' 2024-09-19T04:36:47.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:46 smithi123 ceph-mon[30511]: mgrmap e7: x(active, since 1.00723s) 2024-09-19T04:36:47.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:36:47.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-19T04:36:47.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T04:36:47.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:36:47.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.751+0000 7ff503e00640 1 Processor -- start 2024-09-19T04:36:47.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.751+0000 7ff503e00640 1 -- start start 2024-09-19T04:36:47.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.752+0000 7ff503e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5041075d0 0x7ff5041079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:47.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.752+0000 7ff503e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff504107fa0 con 0x7ff5041075d0 2024-09-19T04:36:47.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.752+0000 7ff502a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5041075d0 0x7ff5041079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:47.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.752+0000 7ff502a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5041075d0 0x7ff5041079d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43120/0 (socket says 172.21.15.123:43120) 2024-09-19T04:36:47.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.752+0000 7ff502a00640 1 -- 172.21.15.123:0/72532412 learned_addr learned my addr 172.21.15.123:0/72532412 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:47.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.753+0000 7ff502a00640 1 -- 172.21.15.123:0/72532412 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff504108780 con 0x7ff5041075d0 2024-09-19T04:36:47.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.753+0000 7ff502a00640 1 --2- 172.21.15.123:0/72532412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5041075d0 0x7ff5041079d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009820 tx=0x7ff4f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74078b056653bf5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:47.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.754+0000 7ff502000640 1 -- 172.21.15.123:0/72532412 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4f4037070 con 0x7ff5041075d0 2024-09-19T04:36:47.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.754+0000 7ff502000640 1 -- 172.21.15.123:0/72532412 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff4f4032900 con 0x7ff5041075d0 2024-09-19T04:36:47.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.754+0000 7ff502000640 1 -- 172.21.15.123:0/72532412 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4f4030850 con 0x7ff5041075d0 2024-09-19T04:36:47.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.754+0000 7ff503e00640 1 -- 172.21.15.123:0/72532412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5041075d0 msgr2=0x7ff5041079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:47.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.754+0000 7ff503e00640 1 --2- 172.21.15.123:0/72532412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5041075d0 0x7ff5041079d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009820 tx=0x7ff4f402a270 comp rx=0 tx=0).stop 2024-09-19T04:36:47.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.755+0000 7ff503e00640 1 -- 172.21.15.123:0/72532412 shutdown_connections 2024-09-19T04:36:47.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.755+0000 7ff503e00640 1 --2- 172.21.15.123:0/72532412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5041075d0 0x7ff5041079d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:47.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.755+0000 7ff503e00640 1 -- 172.21.15.123:0/72532412 >> 172.21.15.123:0/72532412 conn(0x7ff504102da0 msgr2=0x7ff5041051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:47.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.755+0000 7ff503e00640 1 -- 172.21.15.123:0/72532412 shutdown_connections 2024-09-19T04:36:47.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.755+0000 7ff503e00640 1 -- 172.21.15.123:0/72532412 wait complete. 2024-09-19T04:36:47.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.756+0000 7ff503e00640 1 Processor -- start 2024-09-19T04:36:47.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.756+0000 7ff503e00640 1 -- start start 2024-09-19T04:36:47.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.756+0000 7ff503e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5041075d0 0x7ff50419b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:47.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.756+0000 7ff503e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff50410bb30 con 0x7ff5041075d0 2024-09-19T04:36:47.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.756+0000 7ff502a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5041075d0 0x7ff50419b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:47.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.756+0000 7ff502a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5041075d0 0x7ff50419b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43136/0 (socket says 172.21.15.123:43136) 2024-09-19T04:36:47.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.756+0000 7ff502a00640 1 -- 172.21.15.123:0/2158674116 learned_addr learned my addr 172.21.15.123:0/2158674116 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:47.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.757+0000 7ff502a00640 1 -- 172.21.15.123:0/2158674116 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50419bae0 con 0x7ff5041075d0 2024-09-19T04:36:47.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.757+0000 7ff502a00640 1 --2- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5041075d0 0x7ff50419b5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009940 tx=0x7ff4f4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:47.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.757+0000 7ff4fbe00640 1 -- 172.21.15.123:0/2158674116 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4f4042070 con 0x7ff5041075d0 2024-09-19T04:36:47.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.757+0000 7ff503e00640 1 -- 172.21.15.123:0/2158674116 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50419bd70 con 0x7ff5041075d0 2024-09-19T04:36:47.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.757+0000 7ff4fbe00640 1 -- 172.21.15.123:0/2158674116 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff4f40308c0 con 0x7ff5041075d0 2024-09-19T04:36:47.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.757+0000 7ff4fbe00640 1 -- 172.21.15.123:0/2158674116 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4f4037040 con 0x7ff5041075d0 2024-09-19T04:36:47.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.757+0000 7ff503e00640 1 -- 172.21.15.123:0/2158674116 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50419ea60 con 0x7ff5041075d0 2024-09-19T04:36:47.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.759+0000 7ff4fbe00640 1 -- 172.21.15.123:0/2158674116 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7ff4f4045070 con 0x7ff5041075d0 2024-09-19T04:36:47.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.759+0000 7ff4fbe00640 1 --2- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7ff4e80391b0 0x7ff4e803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:47.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.759+0000 7ff4fbe00640 1 -- 172.21.15.123:0/2158674116 --> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff4e803bd80 con 0x7ff4e80391b0 2024-09-19T04:36:47.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.759+0000 7ff4fbe00640 1 -- 172.21.15.123:0/2158674116 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff4f406c310 con 0x7ff5041075d0 2024-09-19T04:36:47.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.759+0000 7ff4fa000640 1 -- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7ff4e80391b0 msgr2=0x7ff4e803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2065492981 2024-09-19T04:36:47.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.759+0000 7ff4fa000640 1 --2- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7ff4e80391b0 0x7ff4e803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:36:47.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.960+0000 7ff4fa000640 1 -- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7ff4e80391b0 msgr2=0x7ff4e803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2065492981 2024-09-19T04:36:47.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:42.960+0000 7ff4fa000640 1 --2- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7ff4e80391b0 0x7ff4e803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:36:47.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:43.361+0000 7ff4fa000640 1 -- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7ff4e80391b0 msgr2=0x7ff4e803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2065492981 2024-09-19T04:36:47.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:43.361+0000 7ff4fa000640 1 --2- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7ff4e80391b0 0x7ff4e803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T04:36:47.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:44.162+0000 7ff4fa000640 1 -- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7ff4e80391b0 msgr2=0x7ff4e803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2065492981 2024-09-19T04:36:47.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:44.162+0000 7ff4fa000640 1 --2- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7ff4e80391b0 0x7ff4e803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T04:36:47.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:45.763+0000 7ff4fa000640 1 -- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7ff4e80391b0 msgr2=0x7ff4e803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2065492981 2024-09-19T04:36:47.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:45.764+0000 7ff4fa000640 1 --2- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7ff4e80391b0 0x7ff4e803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-19T04:36:47.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:45.914+0000 7ff4fbe00640 1 -- 172.21.15.123:0/2158674116 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7ff4f40327b0 con 0x7ff5041075d0 2024-09-19T04:36:47.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:45.915+0000 7ff4fbe00640 1 -- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7ff4e80391b0 msgr2=0x7ff4e803b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:36:47.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:45.915+0000 7ff4fbe00640 1 --2- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7ff4e80391b0 0x7ff4e803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:47.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.917+0000 7ff4fbe00640 1 -- 172.21.15.123:0/2158674116 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7ff4f4065c90 con 0x7ff5041075d0 2024-09-19T04:36:47.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.917+0000 7ff4fbe00640 1 --2- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7ff4e803cae0 0x7ff4e803eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:47.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.917+0000 7ff4fbe00640 1 -- 172.21.15.123:0/2158674116 --> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff4e803bd80 con 0x7ff4e803cae0 2024-09-19T04:36:47.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.918+0000 7ff4fa000640 1 --2- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7ff4e803cae0 0x7ff4e803eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:47.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.920+0000 7ff4fa000640 1 --2- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7ff4e803cae0 0x7ff4e803eed0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec003980 tx=0x7ff4ec009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:47.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.921+0000 7ff4fbe00640 1 -- 172.21.15.123:0/2158674116 <== mgr.14118 v2:172.21.15.123:6800/3323203947 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7ff4e803bd80 con 0x7ff4e803cae0 2024-09-19T04:36:47.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.927+0000 7ff503e00640 1 -- 172.21.15.123:0/2158674116 --> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff50410bcb0 con 0x7ff4e803cae0 2024-09-19T04:36:47.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.928+0000 7ff4fbe00640 1 -- 172.21.15.123:0/2158674116 <== mgr.14118 v2:172.21.15.123:6800/3323203947 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ff50410bcb0 con 0x7ff4e803cae0 2024-09-19T04:36:47.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.928+0000 7ff503e00640 1 -- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7ff4e803cae0 msgr2=0x7ff4e803eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:47.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.928+0000 7ff503e00640 1 --2- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7ff4e803cae0 0x7ff4e803eed0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec003980 tx=0x7ff4ec009290 comp rx=0 tx=0).stop 2024-09-19T04:36:47.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.928+0000 7ff503e00640 1 -- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5041075d0 msgr2=0x7ff50419b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:47.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.928+0000 7ff503e00640 1 --2- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5041075d0 0x7ff50419b5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009940 tx=0x7ff4f4032ce0 comp rx=0 tx=0).stop 2024-09-19T04:36:47.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.928+0000 7ff503e00640 1 -- 172.21.15.123:0/2158674116 shutdown_connections 2024-09-19T04:36:47.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.928+0000 7ff503e00640 1 --2- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7ff4e803cae0 0x7ff4e803eed0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:47.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.928+0000 7ff503e00640 1 --2- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:6800/2065492981,v1:172.21.15.123:6801/2065492981] conn(0x7ff4e80391b0 0x7ff4e803b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:47.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.928+0000 7ff503e00640 1 --2- 172.21.15.123:0/2158674116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5041075d0 0x7ff50419b5a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:47.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.928+0000 7ff503e00640 1 -- 172.21.15.123:0/2158674116 >> 172.21.15.123:0/2158674116 conn(0x7ff504102da0 msgr2=0x7ff504103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:47.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.928+0000 7ff503e00640 1 -- 172.21.15.123:0/2158674116 shutdown_connections 2024-09-19T04:36:47.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:46.929+0000 7ff503e00640 1 -- 172.21.15.123:0/2158674116 wait complete. 2024-09-19T04:36:47.874 INFO:teuthology.orchestra.run.smithi123.stdout:mgr epoch 5 is available 2024-09-19T04:36:47.874 INFO:teuthology.orchestra.run.smithi123.stdout:Setting orchestrator backend to cephadm... 2024-09-19T04:36:48.145 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:47 smithi123 ceph-mon[30511]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T04:36:48.145 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:47 smithi123 ceph-mon[30511]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T04:36:48.145 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:48.023+0000 7f7cda000640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:04:36:48] ENGINE Error in HTTPServer.serve 2024-09-19T04:36:48.145 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: Traceback (most recent call last): 2024-09-19T04:36:48.145 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:36:48.145 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: self._connections.run(self.expiration_interval) 2024-09-19T04:36:48.145 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:36:48.146 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: self._run(expiration_interval) 2024-09-19T04:36:48.146 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:36:48.146 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:36:48.146 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:36:48.146 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:36:48.146 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:36:48.146 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: s = self.context.wrap_socket( 2024-09-19T04:36:48.146 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:36:48.146 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: return self.sslsocket_class._create( 2024-09-19T04:36:48.146 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:36:48.146 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: self.do_handshake() 2024-09-19T04:36:48.146 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:36:48.146 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: self._sslobj.do_handshake() 2024-09-19T04:36:48.146 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:36:48.146 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:36:49.236 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-mon[30511]: [19/Sep/2024:04:36:47] ENGINE Bus STARTING 2024-09-19T04:36:49.236 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-mon[30511]: [19/Sep/2024:04:36:47] ENGINE Serving on http://172.21.15.123:8765 2024-09-19T04:36:49.236 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-mon[30511]: mgrmap e8: x(active, since 2s) 2024-09-19T04:36:49.237 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-mon[30511]: [19/Sep/2024:04:36:48] ENGINE Serving on https://172.21.15.123:7150 2024-09-19T04:36:49.237 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-mon[30511]: [19/Sep/2024:04:36:48] ENGINE Bus STARTED 2024-09-19T04:36:49.237 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:36:49.237 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-mon[30511]: [19/Sep/2024:04:36:48] ENGINE Error in HTTPServer.serve 2024-09-19T04:36:49.237 INFO:journalctl@ceph.mon.a.smithi123.stdout: Traceback (most recent call last): 2024-09-19T04:36:49.237 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:36:49.237 INFO:journalctl@ceph.mon.a.smithi123.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:36:49.237 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:36:49.237 INFO:journalctl@ceph.mon.a.smithi123.stdout: self._run(expiration_interval) 2024-09-19T04:36:49.237 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:36:49.237 INFO:journalctl@ceph.mon.a.smithi123.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:36:49.237 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:36:49.237 INFO:journalctl@ceph.mon.a.smithi123.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:36:49.238 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:36:49.238 INFO:journalctl@ceph.mon.a.smithi123.stdout: s = self.context.wrap_socket( 2024-09-19T04:36:49.238 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:36:49.238 INFO:journalctl@ceph.mon.a.smithi123.stdout: return self.sslsocket_class._create( 2024-09-19T04:36:49.238 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:36:49.238 INFO:journalctl@ceph.mon.a.smithi123.stdout: self.do_handshake() 2024-09-19T04:36:49.238 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:36:49.238 INFO:journalctl@ceph.mon.a.smithi123.stdout: self._sslobj.do_handshake() 2024-09-19T04:36:49.238 INFO:journalctl@ceph.mon.a.smithi123.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:36:49.238 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' 2024-09-19T04:36:49.238 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:48 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:36:49.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.379+0000 7f4f6a000640 1 Processor -- start 2024-09-19T04:36:49.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.379+0000 7f4f6a000640 1 -- start start 2024-09-19T04:36:49.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.380+0000 7f4f6a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f64109950 0x7f4f64109d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:49.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.380+0000 7f4f6a000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6410a320 con 0x7f4f64109950 2024-09-19T04:36:49.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.380+0000 7f4f63400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f64109950 0x7f4f64109d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:49.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.380+0000 7f4f63400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f64109950 0x7f4f64109d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43202/0 (socket says 172.21.15.123:43202) 2024-09-19T04:36:49.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.380+0000 7f4f63400640 1 -- 172.21.15.123:0/2390524620 learned_addr learned my addr 172.21.15.123:0/2390524620 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:49.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.380+0000 7f4f63400640 1 -- 172.21.15.123:0/2390524620 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f6410ab50 con 0x7f4f64109950 2024-09-19T04:36:49.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.381+0000 7f4f63400640 1 --2- 172.21.15.123:0/2390524620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f64109950 0x7f4f64109d50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009820 tx=0x7f4f5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be252ac8fba60955 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:49.248 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.382+0000 7f4f62a00640 1 -- 172.21.15.123:0/2390524620 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f54037070 con 0x7f4f64109950 2024-09-19T04:36:49.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.382+0000 7f4f62a00640 1 -- 172.21.15.123:0/2390524620 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4f54032900 con 0x7f4f64109950 2024-09-19T04:36:49.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.382+0000 7f4f62a00640 1 -- 172.21.15.123:0/2390524620 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f54030850 con 0x7f4f64109950 2024-09-19T04:36:49.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.382+0000 7f4f6a000640 1 -- 172.21.15.123:0/2390524620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f64109950 msgr2=0x7f4f64109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:49.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.382+0000 7f4f6a000640 1 --2- 172.21.15.123:0/2390524620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f64109950 0x7f4f64109d50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009820 tx=0x7f4f5402a270 comp rx=0 tx=0).stop 2024-09-19T04:36:49.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.382+0000 7f4f6a000640 1 -- 172.21.15.123:0/2390524620 shutdown_connections 2024-09-19T04:36:49.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.382+0000 7f4f6a000640 1 --2- 172.21.15.123:0/2390524620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f64109950 0x7f4f64109d50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:49.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.382+0000 7f4f6a000640 1 -- 172.21.15.123:0/2390524620 >> 172.21.15.123:0/2390524620 conn(0x7f4f6407cf30 msgr2=0x7f4f6407d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:49.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.382+0000 7f4f6a000640 1 -- 172.21.15.123:0/2390524620 shutdown_connections 2024-09-19T04:36:49.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.382+0000 7f4f6a000640 1 -- 172.21.15.123:0/2390524620 wait complete. 2024-09-19T04:36:49.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.383+0000 7f4f6a000640 1 Processor -- start 2024-09-19T04:36:49.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.383+0000 7f4f6a000640 1 -- start start 2024-09-19T04:36:49.249 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.384+0000 7f4f6a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f64109950 0x7f4f6419d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:49.250 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.384+0000 7f4f6a000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6410df00 con 0x7f4f64109950 2024-09-19T04:36:49.250 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.384+0000 7f4f63400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f64109950 0x7f4f6419d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:49.250 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.384+0000 7f4f63400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f64109950 0x7f4f6419d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43206/0 (socket says 172.21.15.123:43206) 2024-09-19T04:36:49.250 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.384+0000 7f4f63400640 1 -- 172.21.15.123:0/2955323056 learned_addr learned my addr 172.21.15.123:0/2955323056 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:49.250 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.384+0000 7f4f63400640 1 -- 172.21.15.123:0/2955323056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f6419df30 con 0x7f4f64109950 2024-09-19T04:36:49.250 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.385+0000 7f4f63400640 1 --2- 172.21.15.123:0/2955323056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f64109950 0x7f4f6419d9f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4f540097f0 tx=0x7f4f54030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:49.250 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.385+0000 7f4f60c00640 1 -- 172.21.15.123:0/2955323056 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f54042070 con 0x7f4f64109950 2024-09-19T04:36:49.250 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.385+0000 7f4f60c00640 1 -- 172.21.15.123:0/2955323056 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4f54032ca0 con 0x7f4f64109950 2024-09-19T04:36:49.250 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.385+0000 7f4f6a000640 1 -- 172.21.15.123:0/2955323056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f6419e1c0 con 0x7f4f64109950 2024-09-19T04:36:49.250 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.385+0000 7f4f60c00640 1 -- 172.21.15.123:0/2955323056 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f54037040 con 0x7f4f64109950 2024-09-19T04:36:49.250 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.385+0000 7f4f6a000640 1 -- 172.21.15.123:0/2955323056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f641a0eb0 con 0x7f4f64109950 2024-09-19T04:36:49.250 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.386+0000 7f4f60c00640 1 -- 172.21.15.123:0/2955323056 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f4f54045070 con 0x7f4f64109950 2024-09-19T04:36:49.251 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.386+0000 7f4f6a000640 1 -- 172.21.15.123:0/2955323056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f6410e080 con 0x7f4f64109950 2024-09-19T04:36:49.251 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.387+0000 7f4f60c00640 1 --2- 172.21.15.123:0/2955323056 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f4f48039200 0x7f4f4803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:49.251 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.387+0000 7f4f60c00640 1 -- 172.21.15.123:0/2955323056 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4f5406c240 con 0x7f4f64109950 2024-09-19T04:36:49.251 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.387+0000 7f4f5aa00640 1 --2- 172.21.15.123:0/2955323056 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f4f48039200 0x7f4f4803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:49.251 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.388+0000 7f4f5aa00640 1 --2- 172.21.15.123:0/2955323056 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f4f48039200 0x7f4f4803b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0098c0 tx=0x7f4f4c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:49.251 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.392+0000 7f4f60c00640 1 -- 172.21.15.123:0/2955323056 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4f5406bb90 con 0x7f4f64109950 2024-09-19T04:36:49.251 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.583+0000 7f4f6a000640 1 -- 172.21.15.123:0/2955323056 --> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f4f6410c4d0 con 0x7f4f48039200 2024-09-19T04:36:49.251 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.589+0000 7f4f60c00640 1 -- 172.21.15.123:0/2955323056 <== mgr.14118 v2:172.21.15.123:6800/3323203947 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4f6410c4d0 con 0x7f4f48039200 2024-09-19T04:36:49.251 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.591+0000 7f4f6a000640 1 -- 172.21.15.123:0/2955323056 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f4f48039200 msgr2=0x7f4f4803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:49.251 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.591+0000 7f4f6a000640 1 --2- 172.21.15.123:0/2955323056 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f4f48039200 0x7f4f4803b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0098c0 tx=0x7f4f4c006db0 comp rx=0 tx=0).stop 2024-09-19T04:36:49.251 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.591+0000 7f4f6a000640 1 -- 172.21.15.123:0/2955323056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f64109950 msgr2=0x7f4f6419d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:49.251 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.591+0000 7f4f6a000640 1 --2- 172.21.15.123:0/2955323056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f64109950 0x7f4f6419d9f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4f540097f0 tx=0x7f4f54030ec0 comp rx=0 tx=0).stop 2024-09-19T04:36:49.252 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.592+0000 7f4f6a000640 1 -- 172.21.15.123:0/2955323056 shutdown_connections 2024-09-19T04:36:49.252 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.592+0000 7f4f6a000640 1 --2- 172.21.15.123:0/2955323056 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f4f48039200 0x7f4f4803b6c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:49.252 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.592+0000 7f4f6a000640 1 --2- 172.21.15.123:0/2955323056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f64109950 0x7f4f6419d9f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:49.252 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.592+0000 7f4f6a000640 1 -- 172.21.15.123:0/2955323056 >> 172.21.15.123:0/2955323056 conn(0x7f4f6407cf30 msgr2=0x7f4f6410bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:49.253 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.592+0000 7f4f6a000640 1 -- 172.21.15.123:0/2955323056 shutdown_connections 2024-09-19T04:36:49.253 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:48.592+0000 7f4f6a000640 1 -- 172.21.15.123:0/2955323056 wait complete. 2024-09-19T04:36:50.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:49 smithi123 ceph-mon[30511]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:36:50.579 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T04:36:50.579 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.726+0000 7f9014a00640 1 Processor -- start 2024-09-19T04:36:50.579 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.726+0000 7f9014a00640 1 -- start start 2024-09-19T04:36:50.579 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.726+0000 7f9014a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9010105de0 0x7f90101061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:50.579 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.726+0000 7f9014a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f901007be20 con 0x7f9010105de0 2024-09-19T04:36:50.579 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.726+0000 7f900e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9010105de0 0x7f90101061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:50.579 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.726+0000 7f900e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9010105de0 0x7f90101061e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43208/0 (socket says 172.21.15.123:43208) 2024-09-19T04:36:50.580 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.726+0000 7f900e800640 1 -- 172.21.15.123:0/4182946678 learned_addr learned my addr 172.21.15.123:0/4182946678 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:50.580 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.727+0000 7f900e800640 1 -- 172.21.15.123:0/4182946678 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9010106720 con 0x7f9010105de0 2024-09-19T04:36:50.580 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.728+0000 7f900e800640 1 --2- 172.21.15.123:0/4182946678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9010105de0 0x7f90101061e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9000009820 tx=0x7f900002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8626132442cf8b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:50.580 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.728+0000 7f900d400640 1 -- 172.21.15.123:0/4182946678 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9000037070 con 0x7f9010105de0 2024-09-19T04:36:50.580 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.728+0000 7f900d400640 1 -- 172.21.15.123:0/4182946678 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9000032900 con 0x7f9010105de0 2024-09-19T04:36:50.580 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.728+0000 7f900d400640 1 -- 172.21.15.123:0/4182946678 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9000030850 con 0x7f9010105de0 2024-09-19T04:36:50.580 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.729+0000 7f9014a00640 1 -- 172.21.15.123:0/4182946678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9010105de0 msgr2=0x7f90101061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:50.580 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.729+0000 7f9014a00640 1 --2- 172.21.15.123:0/4182946678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9010105de0 0x7f90101061e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9000009820 tx=0x7f900002a270 comp rx=0 tx=0).stop 2024-09-19T04:36:50.580 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.729+0000 7f9014a00640 1 -- 172.21.15.123:0/4182946678 shutdown_connections 2024-09-19T04:36:50.580 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.729+0000 7f9014a00640 1 --2- 172.21.15.123:0/4182946678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9010105de0 0x7f90101061e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:50.580 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.729+0000 7f9014a00640 1 -- 172.21.15.123:0/4182946678 >> 172.21.15.123:0/4182946678 conn(0x7f9010101bd0 msgr2=0x7f9010103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:50.580 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.729+0000 7f9014a00640 1 -- 172.21.15.123:0/4182946678 shutdown_connections 2024-09-19T04:36:50.581 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.729+0000 7f9014a00640 1 -- 172.21.15.123:0/4182946678 wait complete. 2024-09-19T04:36:50.581 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.730+0000 7f9014a00640 1 Processor -- start 2024-09-19T04:36:50.581 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.730+0000 7f9014a00640 1 -- start start 2024-09-19T04:36:50.581 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.730+0000 7f9014a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9010105de0 0x7f901019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:50.581 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.730+0000 7f9014a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9010107820 con 0x7f9010105de0 2024-09-19T04:36:50.581 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.730+0000 7f900e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9010105de0 0x7f901019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:50.581 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.730+0000 7f900e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9010105de0 0x7f901019d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43214/0 (socket says 172.21.15.123:43214) 2024-09-19T04:36:50.581 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.730+0000 7f900e800640 1 -- 172.21.15.123:0/617551966 learned_addr learned my addr 172.21.15.123:0/617551966 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:50.581 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.731+0000 7f900e800640 1 -- 172.21.15.123:0/617551966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f901019dc60 con 0x7f9010105de0 2024-09-19T04:36:50.581 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.731+0000 7f900e800640 1 --2- 172.21.15.123:0/617551966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9010105de0 0x7f901019d720 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f90000097f0 tx=0x7f9000030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:50.581 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.731+0000 7f9007400640 1 -- 172.21.15.123:0/617551966 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9000037040 con 0x7f9010105de0 2024-09-19T04:36:50.581 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.731+0000 7f9007400640 1 -- 172.21.15.123:0/617551966 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9000040070 con 0x7f9010105de0 2024-09-19T04:36:50.581 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.731+0000 7f9014a00640 1 -- 172.21.15.123:0/617551966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f901019def0 con 0x7f9010105de0 2024-09-19T04:36:50.582 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.731+0000 7f9007400640 1 -- 172.21.15.123:0/617551966 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f900003a390 con 0x7f9010105de0 2024-09-19T04:36:50.582 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.731+0000 7f9014a00640 1 -- 172.21.15.123:0/617551966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90101a0be0 con 0x7f9010105de0 2024-09-19T04:36:50.582 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.732+0000 7f9007400640 1 -- 172.21.15.123:0/617551966 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f900003b430 con 0x7f9010105de0 2024-09-19T04:36:50.582 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.732+0000 7f9014a00640 1 -- 172.21.15.123:0/617551966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90101061e0 con 0x7f9010105de0 2024-09-19T04:36:50.582 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.733+0000 7f9007400640 1 --2- 172.21.15.123:0/617551966 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f8ff4039200 0x7f8ff403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:50.582 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.733+0000 7f9007400640 1 -- 172.21.15.123:0/617551966 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f900006c7e0 con 0x7f9010105de0 2024-09-19T04:36:50.582 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.733+0000 7f900de00640 1 --2- 172.21.15.123:0/617551966 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f8ff4039200 0x7f8ff403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:50.582 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.734+0000 7f900de00640 1 --2- 172.21.15.123:0/617551966 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f8ff4039200 0x7f8ff403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc0098c0 tx=0x7f8ffc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:50.582 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.738+0000 7f9007400640 1 -- 172.21.15.123:0/617551966 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f900006c130 con 0x7f9010105de0 2024-09-19T04:36:50.582 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.912+0000 7f9014a00640 1 -- 172.21.15.123:0/617551966 --> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f9010062630 con 0x7f8ff4039200 2024-09-19T04:36:50.582 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.913+0000 7f9007400640 1 -- 172.21.15.123:0/617551966 <== mgr.14118 v2:172.21.15.123:6800/3323203947 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f9010062630 con 0x7f8ff4039200 2024-09-19T04:36:50.582 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.915+0000 7f9014a00640 1 -- 172.21.15.123:0/617551966 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f8ff4039200 msgr2=0x7f8ff403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:50.582 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.916+0000 7f9014a00640 1 --2- 172.21.15.123:0/617551966 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f8ff4039200 0x7f8ff403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc0098c0 tx=0x7f8ffc006db0 comp rx=0 tx=0).stop 2024-09-19T04:36:50.583 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.916+0000 7f9014a00640 1 -- 172.21.15.123:0/617551966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9010105de0 msgr2=0x7f901019d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:50.583 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.916+0000 7f9014a00640 1 --2- 172.21.15.123:0/617551966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9010105de0 0x7f901019d720 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f90000097f0 tx=0x7f9000030d80 comp rx=0 tx=0).stop 2024-09-19T04:36:50.583 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.916+0000 7f9014a00640 1 -- 172.21.15.123:0/617551966 shutdown_connections 2024-09-19T04:36:50.583 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.916+0000 7f9014a00640 1 --2- 172.21.15.123:0/617551966 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f8ff4039200 0x7f8ff403b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:50.583 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.916+0000 7f9014a00640 1 --2- 172.21.15.123:0/617551966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9010105de0 0x7f901019d720 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:50.583 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.916+0000 7f9014a00640 1 -- 172.21.15.123:0/617551966 >> 172.21.15.123:0/617551966 conn(0x7f9010101bd0 msgr2=0x7f9010191a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:50.583 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.916+0000 7f9014a00640 1 -- 172.21.15.123:0/617551966 shutdown_connections 2024-09-19T04:36:50.583 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:49.916+0000 7f9014a00640 1 -- 172.21.15.123:0/617551966 wait complete. 2024-09-19T04:36:50.583 INFO:teuthology.orchestra.run.smithi123.stdout:Generating ssh key... 2024-09-19T04:36:51.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:50 smithi123 ceph-mon[30511]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:36:51.893 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: Generating public/private rsa key pair. 2024-09-19T04:36:51.893 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: Your identification has been saved in /tmp/tmpfe98i9s0/key 2024-09-19T04:36:51.894 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: Your public key has been saved in /tmp/tmpfe98i9s0/key.pub 2024-09-19T04:36:51.894 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: The key fingerprint is: 2024-09-19T04:36:51.894 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: SHA256:EhVmDb3N/WcJ/HFGwnP8NnrYn4nEUlk8hU5lbO09NKM ceph-911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:36:51.894 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: The key's randomart image is: 2024-09-19T04:36:51.894 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: +---[RSA 3072]----+ 2024-09-19T04:36:51.894 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: | *= ..=*| 2024-09-19T04:36:51.894 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: | + o *OB| 2024-09-19T04:36:51.894 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: | . +.+=B*| 2024-09-19T04:36:51.894 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: | . . oEo+B| 2024-09-19T04:36:51.894 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: | . S o *+*| 2024-09-19T04:36:51.894 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: | . . = =+| 2024-09-19T04:36:51.894 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: | o o.=| 2024-09-19T04:36:51.894 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: | . o.| 2024-09-19T04:36:51.894 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: | | 2024-09-19T04:36:51.895 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:36:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: +----[SHA256]-----+ 2024-09-19T04:36:52.278 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.040+0000 7fcec3600640 1 Processor -- start 2024-09-19T04:36:52.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.040+0000 7fcec3600640 1 -- start start 2024-09-19T04:36:52.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.041+0000 7fcec3600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcebc107790 0x7fcebc107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:52.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.041+0000 7fcec3600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc108160 con 0x7fcebc107790 2024-09-19T04:36:52.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.041+0000 7fcec0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcebc107790 0x7fcebc107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:52.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.041+0000 7fcec0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcebc107790 0x7fcebc107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43218/0 (socket says 172.21.15.123:43218) 2024-09-19T04:36:52.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.041+0000 7fcec0c00640 1 -- 172.21.15.123:0/821643084 learned_addr learned my addr 172.21.15.123:0/821643084 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:52.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.041+0000 7fcec0c00640 1 -- 172.21.15.123:0/821643084 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcebc108940 con 0x7fcebc107790 2024-09-19T04:36:52.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.042+0000 7fcec0c00640 1 --2- 172.21.15.123:0/821643084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcebc107790 0x7fcebc107b90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fceac009820 tx=0x7fceac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4887195819422ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:52.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.043+0000 7fcebbe00640 1 -- 172.21.15.123:0/821643084 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fceac037070 con 0x7fcebc107790 2024-09-19T04:36:52.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.043+0000 7fcebbe00640 1 -- 172.21.15.123:0/821643084 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fceac032900 con 0x7fcebc107790 2024-09-19T04:36:52.279 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.043+0000 7fcebbe00640 1 -- 172.21.15.123:0/821643084 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fceac030850 con 0x7fcebc107790 2024-09-19T04:36:52.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.043+0000 7fcec3600640 1 -- 172.21.15.123:0/821643084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcebc107790 msgr2=0x7fcebc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:52.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.043+0000 7fcec3600640 1 --2- 172.21.15.123:0/821643084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcebc107790 0x7fcebc107b90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fceac009820 tx=0x7fceac02a270 comp rx=0 tx=0).stop 2024-09-19T04:36:52.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.043+0000 7fcec3600640 1 -- 172.21.15.123:0/821643084 shutdown_connections 2024-09-19T04:36:52.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.043+0000 7fcec3600640 1 --2- 172.21.15.123:0/821643084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcebc107790 0x7fcebc107b90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:52.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.043+0000 7fcec3600640 1 -- 172.21.15.123:0/821643084 >> 172.21.15.123:0/821643084 conn(0x7fcebc102fa0 msgr2=0x7fcebc1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:52.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.043+0000 7fcec3600640 1 -- 172.21.15.123:0/821643084 shutdown_connections 2024-09-19T04:36:52.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.043+0000 7fcec3600640 1 -- 172.21.15.123:0/821643084 wait complete. 2024-09-19T04:36:52.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.044+0000 7fcec3600640 1 Processor -- start 2024-09-19T04:36:52.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.044+0000 7fcec3600640 1 -- start start 2024-09-19T04:36:52.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.045+0000 7fcec3600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcebc107790 0x7fcebc19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:52.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.045+0000 7fcec3600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc10bcf0 con 0x7fcebc107790 2024-09-19T04:36:52.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.045+0000 7fcec0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcebc107790 0x7fcebc19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:52.280 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.045+0000 7fcec0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcebc107790 0x7fcebc19d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43224/0 (socket says 172.21.15.123:43224) 2024-09-19T04:36:52.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.045+0000 7fcec0c00640 1 -- 172.21.15.123:0/3086179115 learned_addr learned my addr 172.21.15.123:0/3086179115 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:52.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.045+0000 7fcec0c00640 1 -- 172.21.15.123:0/3086179115 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcebc19deb0 con 0x7fcebc107790 2024-09-19T04:36:52.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.046+0000 7fcec0c00640 1 --2- 172.21.15.123:0/3086179115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcebc107790 0x7fcebc19d970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fceac0097f0 tx=0x7fceac030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:52.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.046+0000 7fceba000640 1 -- 172.21.15.123:0/3086179115 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fceac037040 con 0x7fcebc107790 2024-09-19T04:36:52.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.046+0000 7fceba000640 1 -- 172.21.15.123:0/3086179115 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fceac040070 con 0x7fcebc107790 2024-09-19T04:36:52.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.046+0000 7fceba000640 1 -- 172.21.15.123:0/3086179115 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fceac03a390 con 0x7fcebc107790 2024-09-19T04:36:52.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.046+0000 7fcec3600640 1 -- 172.21.15.123:0/3086179115 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcebc19e140 con 0x7fcebc107790 2024-09-19T04:36:52.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.046+0000 7fcec3600640 1 -- 172.21.15.123:0/3086179115 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcebc1a0e30 con 0x7fcebc107790 2024-09-19T04:36:52.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.047+0000 7fceba000640 1 -- 172.21.15.123:0/3086179115 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fceac032900 con 0x7fcebc107790 2024-09-19T04:36:52.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.047+0000 7fcec3600640 1 -- 172.21.15.123:0/3086179115 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcebc107c10 con 0x7fcebc107790 2024-09-19T04:36:52.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.047+0000 7fceba000640 1 --2- 172.21.15.123:0/3086179115 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fcea0039200 0x7fcea003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:52.281 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.047+0000 7fceba000640 1 -- 172.21.15.123:0/3086179115 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fceac06c830 con 0x7fcebc107790 2024-09-19T04:36:52.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.048+0000 7fceb3e00640 1 --2- 172.21.15.123:0/3086179115 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fcea0039200 0x7fcea003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:52.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.048+0000 7fceb3e00640 1 --2- 172.21.15.123:0/3086179115 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fcea0039200 0x7fcea003b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcea40098c0 tx=0x7fcea4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:52.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.052+0000 7fceba000640 1 -- 172.21.15.123:0/3086179115 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fceac06c180 con 0x7fcebc107790 2024-09-19T04:36:52.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.229+0000 7fcec3600640 1 -- 172.21.15.123:0/3086179115 --> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fcebc1041c0 con 0x7fcea0039200 2024-09-19T04:36:52.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.566+0000 7fceba000640 1 -- 172.21.15.123:0/3086179115 <== mgr.14118 v2:172.21.15.123:6800/3323203947 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcebc1041c0 con 0x7fcea0039200 2024-09-19T04:36:52.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.569+0000 7fcec3600640 1 -- 172.21.15.123:0/3086179115 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fcea0039200 msgr2=0x7fcea003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:52.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.569+0000 7fcec3600640 1 --2- 172.21.15.123:0/3086179115 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fcea0039200 0x7fcea003b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcea40098c0 tx=0x7fcea4006db0 comp rx=0 tx=0).stop 2024-09-19T04:36:52.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.569+0000 7fcec3600640 1 -- 172.21.15.123:0/3086179115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcebc107790 msgr2=0x7fcebc19d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:52.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.569+0000 7fcec3600640 1 --2- 172.21.15.123:0/3086179115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcebc107790 0x7fcebc19d970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fceac0097f0 tx=0x7fceac030d80 comp rx=0 tx=0).stop 2024-09-19T04:36:52.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.569+0000 7fcec3600640 1 -- 172.21.15.123:0/3086179115 shutdown_connections 2024-09-19T04:36:52.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.569+0000 7fcec3600640 1 --2- 172.21.15.123:0/3086179115 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fcea0039200 0x7fcea003b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:52.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.569+0000 7fcec3600640 1 --2- 172.21.15.123:0/3086179115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcebc107790 0x7fcebc19d970 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:52.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.569+0000 7fcec3600640 1 -- 172.21.15.123:0/3086179115 >> 172.21.15.123:0/3086179115 conn(0x7fcebc102fa0 msgr2=0x7fcebc103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:52.282 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.569+0000 7fcec3600640 1 -- 172.21.15.123:0/3086179115 shutdown_connections 2024-09-19T04:36:52.283 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:51.569+0000 7fcec3600640 1 -- 172.21.15.123:0/3086179115 wait complete. 2024-09-19T04:36:52.774 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:52 smithi123 ceph-mon[30511]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:36:52.775 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:52 smithi123 ceph-mon[30511]: Generating ssh key... 2024-09-19T04:36:52.775 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:52 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' 2024-09-19T04:36:52.775 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:52 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' 2024-09-19T04:36:53.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7OSSNG8zcJefpr+R9V/17bxDTyFPelwJTIF/flxiapM648SuBb+xUh2mOfl+Tl1/AxeADsRqNfoe5pI4zhNLoKOKraKBLfa1dWFMlJnRuMZQjRYELW49+e7ZyerHiguQp4odcwZ5ESEtzyOcKErK9B+WDRCVN62rXB/hwS/xfA60j/X8g7pm6HqWi6zG7E9iUk8DkICwbNRbrmWq4lwatZEaoRvuXMDH8rzhuhX/RDEsu3w5W6Up3VnDwSQuFWsTLSpdJASQFofzpLqGxosRTdvIzAARJVV3slgNDQF0a652Q62rQR0Nw1MVQ78VEfB5Xu5HwjogV2ZhXN/ePKOmhLVYURT3Y7cyQlo5c1AQyVCqAVsYk2cXMNB4ueAwowSHQrqW08otQc/aMwvwrzYKYRHpJ5CC+7zfjtoIN7Orq2u4g23uaCja0fEKvhZTtJwpKBQunKCF2grCeRg1dNMY0tB9OTKnFeMvt3xjIqguP1njKlJeIetwjgND+ErtMNcM= ceph-911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:36:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.759+0000 7fb8ae000640 1 Processor -- start 2024-09-19T04:36:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.759+0000 7fb8ae000640 1 -- start start 2024-09-19T04:36:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.759+0000 7fb8ae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a8107ff0 0x7fb8a810a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.759+0000 7fb8ae000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a807be20 con 0x7fb8a8107ff0 2024-09-19T04:36:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.760+0000 7fb8acc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a8107ff0 0x7fb8a810a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.760+0000 7fb8acc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a8107ff0 0x7fb8a810a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48302/0 (socket says 172.21.15.123:48302) 2024-09-19T04:36:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.760+0000 7fb8acc00640 1 -- 172.21.15.123:0/3935122543 learned_addr learned my addr 172.21.15.123:0/3935122543 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.760+0000 7fb8acc00640 1 -- 172.21.15.123:0/3935122543 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a810a920 con 0x7fb8a8107ff0 2024-09-19T04:36:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.762+0000 7fb8acc00640 1 --2- 172.21.15.123:0/3935122543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a8107ff0 0x7fb8a810a3e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009820 tx=0x7fb89c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69c8a97a42f02eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.762+0000 7fb8a7400640 1 -- 172.21.15.123:0/3935122543 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb89c037070 con 0x7fb8a8107ff0 2024-09-19T04:36:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.762+0000 7fb8a7400640 1 -- 172.21.15.123:0/3935122543 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb89c032900 con 0x7fb8a8107ff0 2024-09-19T04:36:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.762+0000 7fb8a7400640 1 -- 172.21.15.123:0/3935122543 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb89c030850 con 0x7fb8a8107ff0 2024-09-19T04:36:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.763+0000 7fb8ae000640 1 -- 172.21.15.123:0/3935122543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a8107ff0 msgr2=0x7fb8a810a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.763+0000 7fb8ae000640 1 --2- 172.21.15.123:0/3935122543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a8107ff0 0x7fb8a810a3e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009820 tx=0x7fb89c02a270 comp rx=0 tx=0).stop 2024-09-19T04:36:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.763+0000 7fb8ae000640 1 -- 172.21.15.123:0/3935122543 shutdown_connections 2024-09-19T04:36:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.763+0000 7fb8ae000640 1 --2- 172.21.15.123:0/3935122543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a8107ff0 0x7fb8a810a3e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.763+0000 7fb8ae000640 1 -- 172.21.15.123:0/3935122543 >> 172.21.15.123:0/3935122543 conn(0x7fb8a8101bd0 msgr2=0x7fb8a8103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.763+0000 7fb8ae000640 1 -- 172.21.15.123:0/3935122543 shutdown_connections 2024-09-19T04:36:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.763+0000 7fb8ae000640 1 -- 172.21.15.123:0/3935122543 wait complete. 2024-09-19T04:36:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.764+0000 7fb8ae000640 1 Processor -- start 2024-09-19T04:36:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.764+0000 7fb8ae000640 1 -- start start 2024-09-19T04:36:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.765+0000 7fb8ae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a8107ff0 0x7fb8a8107870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.765+0000 7fb8ae000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a810b960 con 0x7fb8a8107ff0 2024-09-19T04:36:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.765+0000 7fb8acc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a8107ff0 0x7fb8a8107870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.765+0000 7fb8acc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a8107ff0 0x7fb8a8107870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48312/0 (socket says 172.21.15.123:48312) 2024-09-19T04:36:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.765+0000 7fb8acc00640 1 -- 172.21.15.123:0/2333753030 learned_addr learned my addr 172.21.15.123:0/2333753030 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.765+0000 7fb8acc00640 1 -- 172.21.15.123:0/2333753030 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a8107db0 con 0x7fb8a8107ff0 2024-09-19T04:36:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.765+0000 7fb8acc00640 1 --2- 172.21.15.123:0/2333753030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a8107ff0 0x7fb8a8107870 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0097f0 tx=0x7fb89c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.766+0000 7fb8a5600640 1 -- 172.21.15.123:0/2333753030 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb89c037040 con 0x7fb8a8107ff0 2024-09-19T04:36:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.766+0000 7fb8a5600640 1 -- 172.21.15.123:0/2333753030 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb89c040070 con 0x7fb8a8107ff0 2024-09-19T04:36:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.766+0000 7fb8a5600640 1 -- 172.21.15.123:0/2333753030 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb89c03a390 con 0x7fb8a8107ff0 2024-09-19T04:36:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.766+0000 7fb8ae000640 1 -- 172.21.15.123:0/2333753030 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a8105fa0 con 0x7fb8a8107ff0 2024-09-19T04:36:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.766+0000 7fb8ae000640 1 -- 172.21.15.123:0/2333753030 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a8106480 con 0x7fb8a8107ff0 2024-09-19T04:36:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.767+0000 7fb8ae000640 1 -- 172.21.15.123:0/2333753030 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a8108450 con 0x7fb8a8107ff0 2024-09-19T04:36:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.767+0000 7fb8a5600640 1 -- 172.21.15.123:0/2333753030 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fb89c03b430 con 0x7fb8a8107ff0 2024-09-19T04:36:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.768+0000 7fb8a5600640 1 --2- 172.21.15.123:0/2333753030 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fb890039200 0x7fb89003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.768+0000 7fb8a5600640 1 -- 172.21.15.123:0/2333753030 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb89c06c840 con 0x7fb8a8107ff0 2024-09-19T04:36:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.768+0000 7fb8a7e00640 1 --2- 172.21.15.123:0/2333753030 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fb890039200 0x7fb89003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.769+0000 7fb8a7e00640 1 --2- 172.21.15.123:0/2333753030 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fb890039200 0x7fb89003b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb8940098c0 tx=0x7fb894006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.773+0000 7fb8a5600640 1 -- 172.21.15.123:0/2333753030 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb89c06c190 con 0x7fb8a8107ff0 2024-09-19T04:36:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.955+0000 7fb8ae000640 1 -- 172.21.15.123:0/2333753030 --> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fb8a8106ce0 con 0x7fb890039200 2024-09-19T04:36:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.956+0000 7fb8a5600640 1 -- 172.21.15.123:0/2333753030 <== mgr.14118 v2:172.21.15.123:6800/3323203947 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fb8a8106ce0 con 0x7fb890039200 2024-09-19T04:36:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.959+0000 7fb8ae000640 1 -- 172.21.15.123:0/2333753030 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fb890039200 msgr2=0x7fb89003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.959+0000 7fb8ae000640 1 --2- 172.21.15.123:0/2333753030 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fb890039200 0x7fb89003b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb8940098c0 tx=0x7fb894006db0 comp rx=0 tx=0).stop 2024-09-19T04:36:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.959+0000 7fb8ae000640 1 -- 172.21.15.123:0/2333753030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a8107ff0 msgr2=0x7fb8a8107870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.959+0000 7fb8ae000640 1 --2- 172.21.15.123:0/2333753030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a8107ff0 0x7fb8a8107870 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0097f0 tx=0x7fb89c030d80 comp rx=0 tx=0).stop 2024-09-19T04:36:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.959+0000 7fb8ae000640 1 -- 172.21.15.123:0/2333753030 shutdown_connections 2024-09-19T04:36:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.959+0000 7fb8ae000640 1 --2- 172.21.15.123:0/2333753030 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fb890039200 0x7fb89003b6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.959+0000 7fb8ae000640 1 --2- 172.21.15.123:0/2333753030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a8107ff0 0x7fb8a8107870 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.959+0000 7fb8ae000640 1 -- 172.21.15.123:0/2333753030 >> 172.21.15.123:0/2333753030 conn(0x7fb8a8101bd0 msgr2=0x7fb8a8102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.959+0000 7fb8ae000640 1 -- 172.21.15.123:0/2333753030 shutdown_connections 2024-09-19T04:36:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:52.959+0000 7fb8ae000640 1 -- 172.21.15.123:0/2333753030 wait complete. 2024-09-19T04:36:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T04:36:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T04:36:53.606 INFO:teuthology.orchestra.run.smithi123.stdout:Adding host smithi123... 2024-09-19T04:36:53.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:53 smithi123 ceph-mon[30511]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:36:55.274 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:55 smithi123 ceph-mon[30511]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "addr": "172.21.15.123", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:36:56.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:56 smithi123 ceph-mon[30511]: Deploying cephadm binary to smithi123 2024-09-19T04:36:58.860 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout Added host 'smithi123' with addr '172.21.15.123' 2024-09-19T04:36:58.860 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.094+0000 7fcc5fc00640 1 Processor -- start 2024-09-19T04:36:58.860 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.094+0000 7fcc5fc00640 1 -- start start 2024-09-19T04:36:58.860 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.095+0000 7fcc5fc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc58107790 0x7fcc58107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:58.860 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.095+0000 7fcc5fc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcc58108160 con 0x7fcc58107790 2024-09-19T04:36:58.860 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.095+0000 7fcc5d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc58107790 0x7fcc58107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:58.860 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.095+0000 7fcc5d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc58107790 0x7fcc58107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48320/0 (socket says 172.21.15.123:48320) 2024-09-19T04:36:58.860 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.095+0000 7fcc5d200640 1 -- 172.21.15.123:0/1753991218 learned_addr learned my addr 172.21.15.123:0/1753991218 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:58.860 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.095+0000 7fcc5d200640 1 -- 172.21.15.123:0/1753991218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc58108940 con 0x7fcc58107790 2024-09-19T04:36:58.860 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.096+0000 7fcc5d200640 1 --2- 172.21.15.123:0/1753991218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc58107790 0x7fcc58107b90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009820 tx=0x7fcc4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e63eeca4c2e9b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:58.860 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.097+0000 7fcc57e00640 1 -- 172.21.15.123:0/1753991218 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc48037070 con 0x7fcc58107790 2024-09-19T04:36:58.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.097+0000 7fcc57e00640 1 -- 172.21.15.123:0/1753991218 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc48032900 con 0x7fcc58107790 2024-09-19T04:36:58.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.097+0000 7fcc57e00640 1 -- 172.21.15.123:0/1753991218 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc48030850 con 0x7fcc58107790 2024-09-19T04:36:58.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.097+0000 7fcc5fc00640 1 -- 172.21.15.123:0/1753991218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc58107790 msgr2=0x7fcc58107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:58.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.097+0000 7fcc5fc00640 1 --2- 172.21.15.123:0/1753991218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc58107790 0x7fcc58107b90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009820 tx=0x7fcc4802a270 comp rx=0 tx=0).stop 2024-09-19T04:36:58.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.097+0000 7fcc5fc00640 1 -- 172.21.15.123:0/1753991218 shutdown_connections 2024-09-19T04:36:58.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.097+0000 7fcc5fc00640 1 --2- 172.21.15.123:0/1753991218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc58107790 0x7fcc58107b90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:58.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.097+0000 7fcc5fc00640 1 -- 172.21.15.123:0/1753991218 >> 172.21.15.123:0/1753991218 conn(0x7fcc58102fa0 msgr2=0x7fcc581053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:58.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.097+0000 7fcc5fc00640 1 -- 172.21.15.123:0/1753991218 shutdown_connections 2024-09-19T04:36:58.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.097+0000 7fcc5fc00640 1 -- 172.21.15.123:0/1753991218 wait complete. 2024-09-19T04:36:58.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.098+0000 7fcc5fc00640 1 Processor -- start 2024-09-19T04:36:58.861 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.098+0000 7fcc5fc00640 1 -- start start 2024-09-19T04:36:58.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.099+0000 7fcc5fc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc58107790 0x7fcc5807e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:58.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.099+0000 7fcc5fc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5810bcf0 con 0x7fcc58107790 2024-09-19T04:36:58.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.099+0000 7fcc5d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc58107790 0x7fcc5807e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:58.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.099+0000 7fcc5d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc58107790 0x7fcc5807e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48336/0 (socket says 172.21.15.123:48336) 2024-09-19T04:36:58.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.099+0000 7fcc5d200640 1 -- 172.21.15.123:0/1917570023 learned_addr learned my addr 172.21.15.123:0/1917570023 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:36:58.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.099+0000 7fcc5d200640 1 -- 172.21.15.123:0/1917570023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5807e910 con 0x7fcc58107790 2024-09-19T04:36:58.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.099+0000 7fcc5d200640 1 --2- 172.21.15.123:0/1917570023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc58107790 0x7fcc5807e3d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcc480097f0 tx=0x7fcc48030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:58.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.100+0000 7fcc56000640 1 -- 172.21.15.123:0/1917570023 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc48037040 con 0x7fcc58107790 2024-09-19T04:36:58.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.100+0000 7fcc56000640 1 -- 172.21.15.123:0/1917570023 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc48040070 con 0x7fcc58107790 2024-09-19T04:36:58.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.100+0000 7fcc56000640 1 -- 172.21.15.123:0/1917570023 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc4803a390 con 0x7fcc58107790 2024-09-19T04:36:58.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.100+0000 7fcc5fc00640 1 -- 172.21.15.123:0/1917570023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5807cb00 con 0x7fcc58107790 2024-09-19T04:36:58.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.100+0000 7fcc5fc00640 1 -- 172.21.15.123:0/1917570023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc5807cfe0 con 0x7fcc58107790 2024-09-19T04:36:58.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.101+0000 7fcc56000640 1 -- 172.21.15.123:0/1917570023 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fcc4803b430 con 0x7fcc58107790 2024-09-19T04:36:58.862 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.101+0000 7fcc5fc00640 1 -- 172.21.15.123:0/1917570023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc58107c10 con 0x7fcc58107790 2024-09-19T04:36:58.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.101+0000 7fcc56000640 1 --2- 172.21.15.123:0/1917570023 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fcc3c039200 0x7fcc3c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:36:58.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.101+0000 7fcc56000640 1 -- 172.21.15.123:0/1917570023 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcc4806c7d0 con 0x7fcc58107790 2024-09-19T04:36:58.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.102+0000 7fcc4fe00640 1 --2- 172.21.15.123:0/1917570023 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fcc3c039200 0x7fcc3c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:36:58.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.102+0000 7fcc4fe00640 1 --2- 172.21.15.123:0/1917570023 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fcc3c039200 0x7fcc3c03b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcc400098c0 tx=0x7fcc40006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:36:58.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.106+0000 7fcc56000640 1 -- 172.21.15.123:0/1917570023 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcc4806c120 con 0x7fcc58107790 2024-09-19T04:36:58.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:54.284+0000 7fcc5fc00640 1 -- 172.21.15.123:0/1917570023 --> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi123", "addr": "172.21.15.123", "target": ["mon-mgr", ""]}) -- 0x7fcc58104220 con 0x7fcc3c039200 2024-09-19T04:36:58.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:58.177+0000 7fcc56000640 1 -- 172.21.15.123:0/1917570023 <== mgr.14118 v2:172.21.15.123:6800/3323203947 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fcc58104220 con 0x7fcc3c039200 2024-09-19T04:36:58.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:58.180+0000 7fcc5fc00640 1 -- 172.21.15.123:0/1917570023 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fcc3c039200 msgr2=0x7fcc3c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:58.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:58.180+0000 7fcc5fc00640 1 --2- 172.21.15.123:0/1917570023 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fcc3c039200 0x7fcc3c03b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcc400098c0 tx=0x7fcc40006db0 comp rx=0 tx=0).stop 2024-09-19T04:36:58.863 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:58.180+0000 7fcc5fc00640 1 -- 172.21.15.123:0/1917570023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc58107790 msgr2=0x7fcc5807e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:36:58.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:58.180+0000 7fcc5fc00640 1 --2- 172.21.15.123:0/1917570023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc58107790 0x7fcc5807e3d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcc480097f0 tx=0x7fcc48030d80 comp rx=0 tx=0).stop 2024-09-19T04:36:58.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:58.180+0000 7fcc5fc00640 1 -- 172.21.15.123:0/1917570023 shutdown_connections 2024-09-19T04:36:58.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:58.180+0000 7fcc5fc00640 1 --2- 172.21.15.123:0/1917570023 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fcc3c039200 0x7fcc3c03b6c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:58.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:58.180+0000 7fcc5fc00640 1 --2- 172.21.15.123:0/1917570023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc58107790 0x7fcc5807e3d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:36:58.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:58.180+0000 7fcc5fc00640 1 -- 172.21.15.123:0/1917570023 >> 172.21.15.123:0/1917570023 conn(0x7fcc58102fa0 msgr2=0x7fcc58103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:36:58.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:58.181+0000 7fcc5fc00640 1 -- 172.21.15.123:0/1917570023 shutdown_connections 2024-09-19T04:36:58.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:58.181+0000 7fcc5fc00640 1 -- 172.21.15.123:0/1917570023 wait complete. 2024-09-19T04:36:58.864 INFO:teuthology.orchestra.run.smithi123.stdout:Deploying unmanaged mon service... 2024-09-19T04:36:59.385 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:59 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' 2024-09-19T04:36:59.385 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:59 smithi123 ceph-mon[30511]: Added host smithi123 2024-09-19T04:36:59.385 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:36:59 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:37:00.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:00 smithi123 ceph-mon[30511]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:37:00.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:00 smithi123 ceph-mon[30511]: Saving service mon spec with placement count:5 2024-09-19T04:37:00.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:00 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' 2024-09-19T04:37:01.456 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T04:37:01.456 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.383+0000 7fdc2b800640 1 Processor -- start 2024-09-19T04:37:01.456 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.383+0000 7fdc2b800640 1 -- start start 2024-09-19T04:37:01.456 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.384+0000 7fdc2b800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc24105a40 0x7fdc24105e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:01.457 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.384+0000 7fdc2b800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24106410 con 0x7fdc24105a40 2024-09-19T04:37:01.457 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.384+0000 7fdc28e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc24105a40 0x7fdc24105e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:01.457 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.384+0000 7fdc28e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc24105a40 0x7fdc24105e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48342/0 (socket says 172.21.15.123:48342) 2024-09-19T04:37:01.457 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.384+0000 7fdc28e00640 1 -- 172.21.15.123:0/1776639442 learned_addr learned my addr 172.21.15.123:0/1776639442 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:01.457 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.385+0000 7fdc28e00640 1 -- 172.21.15.123:0/1776639442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc241067b0 con 0x7fdc24105a40 2024-09-19T04:37:01.457 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.386+0000 7fdc28e00640 1 --2- 172.21.15.123:0/1776639442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc24105a40 0x7fdc24105e40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009820 tx=0x7fdc1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76377e84920208b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:01.457 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.386+0000 7fdc23e00640 1 -- 172.21.15.123:0/1776639442 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc14037070 con 0x7fdc24105a40 2024-09-19T04:37:01.457 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.386+0000 7fdc23e00640 1 -- 172.21.15.123:0/1776639442 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdc14032900 con 0x7fdc24105a40 2024-09-19T04:37:01.457 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.386+0000 7fdc23e00640 1 -- 172.21.15.123:0/1776639442 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc14030850 con 0x7fdc24105a40 2024-09-19T04:37:01.457 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.386+0000 7fdc2b800640 1 -- 172.21.15.123:0/1776639442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc24105a40 msgr2=0x7fdc24105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:01.457 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.386+0000 7fdc2b800640 1 --2- 172.21.15.123:0/1776639442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc24105a40 0x7fdc24105e40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009820 tx=0x7fdc1402a270 comp rx=0 tx=0).stop 2024-09-19T04:37:01.457 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.387+0000 7fdc2b800640 1 -- 172.21.15.123:0/1776639442 shutdown_connections 2024-09-19T04:37:01.457 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.387+0000 7fdc2b800640 1 --2- 172.21.15.123:0/1776639442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc24105a40 0x7fdc24105e40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:01.458 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.387+0000 7fdc2b800640 1 -- 172.21.15.123:0/1776639442 >> 172.21.15.123:0/1776639442 conn(0x7fdc241015f0 msgr2=0x7fdc24103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:01.458 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.387+0000 7fdc2b800640 1 -- 172.21.15.123:0/1776639442 shutdown_connections 2024-09-19T04:37:01.458 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.387+0000 7fdc2b800640 1 -- 172.21.15.123:0/1776639442 wait complete. 2024-09-19T04:37:01.458 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.388+0000 7fdc2b800640 1 Processor -- start 2024-09-19T04:37:01.458 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.388+0000 7fdc2b800640 1 -- start start 2024-09-19T04:37:01.458 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.388+0000 7fdc2b800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc24105a40 0x7fdc2419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:01.458 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.388+0000 7fdc2b800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241079f0 con 0x7fdc24105a40 2024-09-19T04:37:01.458 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.388+0000 7fdc28e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc24105a40 0x7fdc2419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:01.458 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.388+0000 7fdc28e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc24105a40 0x7fdc2419d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48358/0 (socket says 172.21.15.123:48358) 2024-09-19T04:37:01.458 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.388+0000 7fdc28e00640 1 -- 172.21.15.123:0/1141169033 learned_addr learned my addr 172.21.15.123:0/1141169033 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:01.458 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.388+0000 7fdc28e00640 1 -- 172.21.15.123:0/1141169033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2419de60 con 0x7fdc24105a40 2024-09-19T04:37:01.458 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.389+0000 7fdc28e00640 1 --2- 172.21.15.123:0/1141169033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc24105a40 0x7fdc2419d920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdc140097f0 tx=0x7fdc14030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:01.458 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.389+0000 7fdc22000640 1 -- 172.21.15.123:0/1141169033 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc14037040 con 0x7fdc24105a40 2024-09-19T04:37:01.458 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.389+0000 7fdc22000640 1 -- 172.21.15.123:0/1141169033 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdc14040070 con 0x7fdc24105a40 2024-09-19T04:37:01.459 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.389+0000 7fdc2b800640 1 -- 172.21.15.123:0/1141169033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2419e0f0 con 0x7fdc24105a40 2024-09-19T04:37:01.459 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.389+0000 7fdc22000640 1 -- 172.21.15.123:0/1141169033 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc1403a390 con 0x7fdc24105a40 2024-09-19T04:37:01.459 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.389+0000 7fdc2b800640 1 -- 172.21.15.123:0/1141169033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc241a0de0 con 0x7fdc24105a40 2024-09-19T04:37:01.459 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.390+0000 7fdc22000640 1 -- 172.21.15.123:0/1141169033 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fdc1403b430 con 0x7fdc24105a40 2024-09-19T04:37:01.459 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.390+0000 7fdc2b800640 1 -- 172.21.15.123:0/1141169033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc24105e40 con 0x7fdc24105a40 2024-09-19T04:37:01.459 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.391+0000 7fdc22000640 1 --2- 172.21.15.123:0/1141169033 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fdc080391b0 0x7fdc0803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:01.459 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.391+0000 7fdc22000640 1 -- 172.21.15.123:0/1141169033 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdc1406c850 con 0x7fdc24105a40 2024-09-19T04:37:01.459 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.391+0000 7fdc1be00640 1 --2- 172.21.15.123:0/1141169033 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fdc080391b0 0x7fdc0803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:01.459 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.391+0000 7fdc1be00640 1 --2- 172.21.15.123:0/1141169033 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fdc080391b0 0x7fdc0803b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0098c0 tx=0x7fdc0c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:01.459 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.396+0000 7fdc22000640 1 -- 172.21.15.123:0/1141169033 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdc1406c1a0 con 0x7fdc24105a40 2024-09-19T04:37:01.459 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.590+0000 7fdc2b800640 1 -- 172.21.15.123:0/1141169033 --> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fdc24062630 con 0x7fdc080391b0 2024-09-19T04:37:01.459 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.598+0000 7fdc22000640 1 -- 172.21.15.123:0/1141169033 <== mgr.14118 v2:172.21.15.123:6800/3323203947 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdc24062630 con 0x7fdc080391b0 2024-09-19T04:37:01.459 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.600+0000 7fdc2b800640 1 -- 172.21.15.123:0/1141169033 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fdc080391b0 msgr2=0x7fdc0803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:01.460 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.600+0000 7fdc2b800640 1 --2- 172.21.15.123:0/1141169033 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fdc080391b0 0x7fdc0803b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0098c0 tx=0x7fdc0c006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:01.460 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.600+0000 7fdc2b800640 1 -- 172.21.15.123:0/1141169033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc24105a40 msgr2=0x7fdc2419d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:01.460 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.600+0000 7fdc2b800640 1 --2- 172.21.15.123:0/1141169033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc24105a40 0x7fdc2419d920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdc140097f0 tx=0x7fdc14030d80 comp rx=0 tx=0).stop 2024-09-19T04:37:01.460 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.601+0000 7fdc2b800640 1 -- 172.21.15.123:0/1141169033 shutdown_connections 2024-09-19T04:37:01.460 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.601+0000 7fdc2b800640 1 --2- 172.21.15.123:0/1141169033 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fdc080391b0 0x7fdc0803b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:01.460 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.601+0000 7fdc2b800640 1 --2- 172.21.15.123:0/1141169033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc24105a40 0x7fdc2419d920 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:01.460 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.601+0000 7fdc2b800640 1 -- 172.21.15.123:0/1141169033 >> 172.21.15.123:0/1141169033 conn(0x7fdc241015f0 msgr2=0x7fdc24193f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:01.460 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.601+0000 7fdc2b800640 1 -- 172.21.15.123:0/1141169033 shutdown_connections 2024-09-19T04:37:01.460 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:36:59.601+0000 7fdc2b800640 1 -- 172.21.15.123:0/1141169033 wait complete. 2024-09-19T04:37:01.460 INFO:teuthology.orchestra.run.smithi123.stdout:Deploying unmanaged mgr service... 2024-09-19T04:37:02.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:02 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' 2024-09-19T04:37:02.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:02 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' 2024-09-19T04:37:02.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T04:37:02.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.989+0000 7f9b1e200640 1 Processor -- start 2024-09-19T04:37:02.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.989+0000 7f9b1e200640 1 -- start start 2024-09-19T04:37:02.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.990+0000 7f9b1e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b18109fa0 0x7f9b1810a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:02.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.990+0000 7f9b1e200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1810a970 con 0x7f9b18109fa0 2024-09-19T04:37:02.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.990+0000 7f9b17400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b18109fa0 0x7f9b1810a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:02.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.990+0000 7f9b17400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b18109fa0 0x7f9b1810a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34626/0 (socket says 172.21.15.123:34626) 2024-09-19T04:37:02.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.990+0000 7f9b17400640 1 -- 172.21.15.123:0/1673101265 learned_addr learned my addr 172.21.15.123:0/1673101265 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:02.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.991+0000 7f9b17400640 1 -- 172.21.15.123:0/1673101265 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1810aaf0 con 0x7f9b18109fa0 2024-09-19T04:37:02.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.992+0000 7f9b17400640 1 --2- 172.21.15.123:0/1673101265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b18109fa0 0x7f9b1810a3a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009820 tx=0x7f9b0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc032aaf37c59766 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:02.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.992+0000 7f9b16000640 1 -- 172.21.15.123:0/1673101265 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b08037070 con 0x7f9b18109fa0 2024-09-19T04:37:02.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.992+0000 7f9b16000640 1 -- 172.21.15.123:0/1673101265 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b08032920 con 0x7f9b18109fa0 2024-09-19T04:37:02.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.992+0000 7f9b16000640 1 -- 172.21.15.123:0/1673101265 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b08030810 con 0x7f9b18109fa0 2024-09-19T04:37:02.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.993+0000 7f9b1e200640 1 -- 172.21.15.123:0/1673101265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b18109fa0 msgr2=0x7f9b1810a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:02.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.993+0000 7f9b1e200640 1 --2- 172.21.15.123:0/1673101265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b18109fa0 0x7f9b1810a3a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009820 tx=0x7f9b0802a270 comp rx=0 tx=0).stop 2024-09-19T04:37:02.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.993+0000 7f9b1e200640 1 -- 172.21.15.123:0/1673101265 shutdown_connections 2024-09-19T04:37:02.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.993+0000 7f9b1e200640 1 --2- 172.21.15.123:0/1673101265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b18109fa0 0x7f9b1810a3a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:02.854 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.993+0000 7f9b1e200640 1 -- 172.21.15.123:0/1673101265 >> 172.21.15.123:0/1673101265 conn(0x7f9b1807cfb0 msgr2=0x7f9b1807d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:02.854 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.993+0000 7f9b1e200640 1 -- 172.21.15.123:0/1673101265 shutdown_connections 2024-09-19T04:37:02.854 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.993+0000 7f9b1e200640 1 -- 172.21.15.123:0/1673101265 wait complete. 2024-09-19T04:37:02.854 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.994+0000 7f9b1e200640 1 Processor -- start 2024-09-19T04:37:02.854 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.994+0000 7f9b1e200640 1 -- start start 2024-09-19T04:37:02.854 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.995+0000 7f9b1e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b18109fa0 0x7f9b1819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:02.854 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.995+0000 7f9b1e200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1810ddf0 con 0x7f9b18109fa0 2024-09-19T04:37:02.854 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.995+0000 7f9b17400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b18109fa0 0x7f9b1819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:02.854 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.995+0000 7f9b17400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b18109fa0 0x7f9b1819d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34634/0 (socket says 172.21.15.123:34634) 2024-09-19T04:37:02.854 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.995+0000 7f9b17400640 1 -- 172.21.15.123:0/2300304680 learned_addr learned my addr 172.21.15.123:0/2300304680 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:02.854 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.995+0000 7f9b17400640 1 -- 172.21.15.123:0/2300304680 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1819deb0 con 0x7f9b18109fa0 2024-09-19T04:37:02.854 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.996+0000 7f9b17400640 1 --2- 172.21.15.123:0/2300304680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b18109fa0 0x7f9b1819d970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9b080097f0 tx=0x7f9b080327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:02.854 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.996+0000 7f9b0fe00640 1 -- 172.21.15.123:0/2300304680 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b08037040 con 0x7f9b18109fa0 2024-09-19T04:37:02.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.996+0000 7f9b0fe00640 1 -- 172.21.15.123:0/2300304680 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b08030cc0 con 0x7f9b18109fa0 2024-09-19T04:37:02.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.996+0000 7f9b0fe00640 1 -- 172.21.15.123:0/2300304680 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b0803a640 con 0x7f9b18109fa0 2024-09-19T04:37:02.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.996+0000 7f9b1e200640 1 -- 172.21.15.123:0/2300304680 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1819e140 con 0x7f9b18109fa0 2024-09-19T04:37:02.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.996+0000 7f9b1e200640 1 -- 172.21.15.123:0/2300304680 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b181a0e30 con 0x7f9b18109fa0 2024-09-19T04:37:02.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.998+0000 7f9b0fe00640 1 -- 172.21.15.123:0/2300304680 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f9b0803a8f0 con 0x7f9b18109fa0 2024-09-19T04:37:02.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.998+0000 7f9b1e200640 1 -- 172.21.15.123:0/2300304680 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b1810a420 con 0x7f9b18109fa0 2024-09-19T04:37:02.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.998+0000 7f9b0fe00640 1 --2- 172.21.15.123:0/2300304680 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f9afc039200 0x7f9afc03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:02.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.998+0000 7f9b0fe00640 1 -- 172.21.15.123:0/2300304680 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9b0806ca00 con 0x7f9b18109fa0 2024-09-19T04:37:02.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.998+0000 7f9b16a00640 1 --2- 172.21.15.123:0/2300304680 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f9afc039200 0x7f9afc03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:02.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:01.999+0000 7f9b16a00640 1 --2- 172.21.15.123:0/2300304680 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f9afc039200 0x7f9afc03b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9b040098c0 tx=0x7f9b04006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:02.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:02.004+0000 7f9b0fe00640 1 -- 172.21.15.123:0/2300304680 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9b080328c0 con 0x7f9b18109fa0 2024-09-19T04:37:02.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:02.204+0000 7f9b1e200640 1 -- 172.21.15.123:0/2300304680 --> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f9b18108500 con 0x7f9afc039200 2024-09-19T04:37:02.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:02.209+0000 7f9b0fe00640 1 -- 172.21.15.123:0/2300304680 <== mgr.14118 v2:172.21.15.123:6800/3323203947 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9b18108500 con 0x7f9afc039200 2024-09-19T04:37:02.856 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:02.211+0000 7f9b1e200640 1 -- 172.21.15.123:0/2300304680 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f9afc039200 msgr2=0x7f9afc03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:02.856 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:02.211+0000 7f9b1e200640 1 --2- 172.21.15.123:0/2300304680 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f9afc039200 0x7f9afc03b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9b040098c0 tx=0x7f9b04006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:02.856 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:02.211+0000 7f9b1e200640 1 -- 172.21.15.123:0/2300304680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b18109fa0 msgr2=0x7f9b1819d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:02.856 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:02.211+0000 7f9b1e200640 1 --2- 172.21.15.123:0/2300304680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b18109fa0 0x7f9b1819d970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9b080097f0 tx=0x7f9b080327b0 comp rx=0 tx=0).stop 2024-09-19T04:37:02.856 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:02.211+0000 7f9b1e200640 1 -- 172.21.15.123:0/2300304680 shutdown_connections 2024-09-19T04:37:02.856 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:02.211+0000 7f9b1e200640 1 --2- 172.21.15.123:0/2300304680 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f9afc039200 0x7f9afc03b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:02.856 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:02.211+0000 7f9b1e200640 1 --2- 172.21.15.123:0/2300304680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b18109fa0 0x7f9b1819d970 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:02.856 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:02.211+0000 7f9b1e200640 1 -- 172.21.15.123:0/2300304680 >> 172.21.15.123:0/2300304680 conn(0x7f9b1807cfb0 msgr2=0x7f9b18107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:02.856 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:02.212+0000 7f9b1e200640 1 -- 172.21.15.123:0/2300304680 shutdown_connections 2024-09-19T04:37:02.856 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:02.212+0000 7f9b1e200640 1 -- 172.21.15.123:0/2300304680 wait complete. 2024-09-19T04:37:03.381 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:03 smithi123 ceph-mon[30511]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:37:03.381 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:03 smithi123 ceph-mon[30511]: Saving service mgr spec with placement count:2 2024-09-19T04:37:03.382 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:03 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' 2024-09-19T04:37:04.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.701+0000 7f495a800640 1 Processor -- start 2024-09-19T04:37:04.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.701+0000 7f495a800640 1 -- start start 2024-09-19T04:37:04.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.702+0000 7f495a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49541075b0 0x7f49541079b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:04.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.702+0000 7f495a800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4954107f80 con 0x7f49541075b0 2024-09-19T04:37:04.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.702+0000 7f4959400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49541075b0 0x7f49541079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:04.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.702+0000 7f4959400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49541075b0 0x7f49541079b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34642/0 (socket says 172.21.15.123:34642) 2024-09-19T04:37:04.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.702+0000 7f4959400640 1 -- 172.21.15.123:0/2613191047 learned_addr learned my addr 172.21.15.123:0/2613191047 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:04.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.702+0000 7f4959400640 1 -- 172.21.15.123:0/2613191047 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49541087b0 con 0x7f49541075b0 2024-09-19T04:37:04.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.703+0000 7f4959400640 1 --2- 172.21.15.123:0/2613191047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49541075b0 0x7f49541079b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4948009820 tx=0x7f494802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115cb697763ca1c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:04.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.703+0000 7f4953e00640 1 -- 172.21.15.123:0/2613191047 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4948037070 con 0x7f49541075b0 2024-09-19T04:37:04.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.704+0000 7f4953e00640 1 -- 172.21.15.123:0/2613191047 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4948032920 con 0x7f49541075b0 2024-09-19T04:37:04.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.704+0000 7f4953e00640 1 -- 172.21.15.123:0/2613191047 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4948030810 con 0x7f49541075b0 2024-09-19T04:37:04.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.704+0000 7f495a800640 1 -- 172.21.15.123:0/2613191047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49541075b0 msgr2=0x7f49541079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:04.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.704+0000 7f495a800640 1 --2- 172.21.15.123:0/2613191047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49541075b0 0x7f49541079b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4948009820 tx=0x7f494802a270 comp rx=0 tx=0).stop 2024-09-19T04:37:04.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.704+0000 7f495a800640 1 -- 172.21.15.123:0/2613191047 shutdown_connections 2024-09-19T04:37:04.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.704+0000 7f495a800640 1 --2- 172.21.15.123:0/2613191047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49541075b0 0x7f49541079b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:04.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.704+0000 7f495a800640 1 -- 172.21.15.123:0/2613191047 >> 172.21.15.123:0/2613191047 conn(0x7f4954102e80 msgr2=0x7f4954105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:04.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.704+0000 7f495a800640 1 -- 172.21.15.123:0/2613191047 shutdown_connections 2024-09-19T04:37:04.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.704+0000 7f495a800640 1 -- 172.21.15.123:0/2613191047 wait complete. 2024-09-19T04:37:04.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.705+0000 7f495a800640 1 Processor -- start 2024-09-19T04:37:04.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.705+0000 7f495a800640 1 -- start start 2024-09-19T04:37:04.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.706+0000 7f495a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49541075b0 0x7f495419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:04.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.706+0000 7f495a800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f495410bb60 con 0x7f49541075b0 2024-09-19T04:37:04.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.706+0000 7f4959400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49541075b0 0x7f495419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:04.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.706+0000 7f4959400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49541075b0 0x7f495419d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34658/0 (socket says 172.21.15.123:34658) 2024-09-19T04:37:04.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.706+0000 7f4959400640 1 -- 172.21.15.123:0/1629500395 learned_addr learned my addr 172.21.15.123:0/1629500395 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:04.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.706+0000 7f4959400640 1 -- 172.21.15.123:0/1629500395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495419dce0 con 0x7f49541075b0 2024-09-19T04:37:04.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.706+0000 7f4959400640 1 --2- 172.21.15.123:0/1629500395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49541075b0 0x7f495419d7a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f49480097f0 tx=0x7f49480327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:04.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.707+0000 7f4952000640 1 -- 172.21.15.123:0/1629500395 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4948037040 con 0x7f49541075b0 2024-09-19T04:37:04.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.707+0000 7f4952000640 1 -- 172.21.15.123:0/1629500395 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4948030cc0 con 0x7f49541075b0 2024-09-19T04:37:04.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.707+0000 7f495a800640 1 -- 172.21.15.123:0/1629500395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495419df70 con 0x7f49541075b0 2024-09-19T04:37:04.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.707+0000 7f4952000640 1 -- 172.21.15.123:0/1629500395 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f494803a640 con 0x7f49541075b0 2024-09-19T04:37:04.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.707+0000 7f495a800640 1 -- 172.21.15.123:0/1629500395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49541a0c60 con 0x7f49541075b0 2024-09-19T04:37:04.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.708+0000 7f4952000640 1 -- 172.21.15.123:0/1629500395 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f494803b3f0 con 0x7f49541075b0 2024-09-19T04:37:04.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.708+0000 7f495a800640 1 -- 172.21.15.123:0/1629500395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495410bce0 con 0x7f49541075b0 2024-09-19T04:37:04.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.708+0000 7f4952000640 1 --2- 172.21.15.123:0/1629500395 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f493c0391b0 0x7f493c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:04.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.708+0000 7f4952000640 1 -- 172.21.15.123:0/1629500395 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f494806c890 con 0x7f49541075b0 2024-09-19T04:37:04.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.709+0000 7f4958a00640 1 --2- 172.21.15.123:0/1629500395 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f493c0391b0 0x7f493c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:04.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.709+0000 7f4958a00640 1 --2- 172.21.15.123:0/1629500395 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f493c0391b0 0x7f493c03b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f49400098c0 tx=0x7f4940006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:04.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.713+0000 7f4952000640 1 -- 172.21.15.123:0/1629500395 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4948042e10 con 0x7f49541075b0 2024-09-19T04:37:04.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.894+0000 7f495a800640 1 -- 172.21.15.123:0/1629500395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f4954107a30 con 0x7f49541075b0 2024-09-19T04:37:04.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.898+0000 7f4952000640 1 -- 172.21.15.123:0/1629500395 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f494806c1e0 con 0x7f49541075b0 2024-09-19T04:37:04.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.901+0000 7f495a800640 1 -- 172.21.15.123:0/1629500395 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f493c0391b0 msgr2=0x7f493c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:04.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.901+0000 7f495a800640 1 --2- 172.21.15.123:0/1629500395 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f493c0391b0 0x7f493c03b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f49400098c0 tx=0x7f4940006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:04.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.901+0000 7f495a800640 1 -- 172.21.15.123:0/1629500395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49541075b0 msgr2=0x7f495419d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:04.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.901+0000 7f495a800640 1 --2- 172.21.15.123:0/1629500395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49541075b0 0x7f495419d7a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f49480097f0 tx=0x7f49480327b0 comp rx=0 tx=0).stop 2024-09-19T04:37:04.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.901+0000 7f495a800640 1 -- 172.21.15.123:0/1629500395 shutdown_connections 2024-09-19T04:37:04.701 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.901+0000 7f495a800640 1 --2- 172.21.15.123:0/1629500395 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f493c0391b0 0x7f493c03b670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:04.701 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.901+0000 7f495a800640 1 --2- 172.21.15.123:0/1629500395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49541075b0 0x7f495419d7a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:04.701 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.901+0000 7f495a800640 1 -- 172.21.15.123:0/1629500395 >> 172.21.15.123:0/1629500395 conn(0x7f4954102e80 msgr2=0x7f49541049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:04.701 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.901+0000 7f495a800640 1 -- 172.21.15.123:0/1629500395 shutdown_connections 2024-09-19T04:37:04.701 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:03.901+0000 7f495a800640 1 -- 172.21.15.123:0/1629500395 wait complete. 2024-09-19T04:37:04.967 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:04 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1629500395' entity='client.admin' 2024-09-19T04:37:04.968 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:04 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' 2024-09-19T04:37:06.095 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.218+0000 7effc2000640 1 Processor -- start 2024-09-19T04:37:06.095 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.218+0000 7effc2000640 1 -- start start 2024-09-19T04:37:06.095 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.218+0000 7effc2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc07de60 0x7effbc07c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:06.095 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.218+0000 7effc2000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7effbc07c8e0 con 0x7effbc07de60 2024-09-19T04:37:06.095 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.219+0000 7effc0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc07de60 0x7effbc07c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:06.095 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.219+0000 7effc0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc07de60 0x7effbc07c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34664/0 (socket says 172.21.15.123:34664) 2024-09-19T04:37:06.095 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.219+0000 7effc0c00640 1 -- 172.21.15.123:0/1454508346 learned_addr learned my addr 172.21.15.123:0/1454508346 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:06.095 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.219+0000 7effc0c00640 1 -- 172.21.15.123:0/1454508346 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc07ca60 con 0x7effbc07de60 2024-09-19T04:37:06.096 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.220+0000 7effc0c00640 1 --2- 172.21.15.123:0/1454508346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc07de60 0x7effbc07c310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7effb0009820 tx=0x7effb002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3723c35a3d4e8385 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:06.096 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.220+0000 7effbb400640 1 -- 172.21.15.123:0/1454508346 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effb0037070 con 0x7effbc07de60 2024-09-19T04:37:06.096 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.220+0000 7effbb400640 1 -- 172.21.15.123:0/1454508346 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb0032920 con 0x7effbc07de60 2024-09-19T04:37:06.096 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.220+0000 7effbb400640 1 -- 172.21.15.123:0/1454508346 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effb0030810 con 0x7effbc07de60 2024-09-19T04:37:06.096 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.221+0000 7effc2000640 1 -- 172.21.15.123:0/1454508346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc07de60 msgr2=0x7effbc07c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:06.096 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.221+0000 7effc2000640 1 --2- 172.21.15.123:0/1454508346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc07de60 0x7effbc07c310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7effb0009820 tx=0x7effb002a270 comp rx=0 tx=0).stop 2024-09-19T04:37:06.096 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.221+0000 7effc2000640 1 -- 172.21.15.123:0/1454508346 shutdown_connections 2024-09-19T04:37:06.096 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.221+0000 7effc2000640 1 --2- 172.21.15.123:0/1454508346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc07de60 0x7effbc07c310 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:06.096 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.221+0000 7effc2000640 1 -- 172.21.15.123:0/1454508346 >> 172.21.15.123:0/1454508346 conn(0x7effbc102da0 msgr2=0x7effbc1051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:06.096 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.221+0000 7effc2000640 1 -- 172.21.15.123:0/1454508346 shutdown_connections 2024-09-19T04:37:06.096 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.221+0000 7effc2000640 1 -- 172.21.15.123:0/1454508346 wait complete. 2024-09-19T04:37:06.096 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.222+0000 7effc2000640 1 Processor -- start 2024-09-19T04:37:06.097 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.222+0000 7effc2000640 1 -- start start 2024-09-19T04:37:06.097 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.223+0000 7effc2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc07de60 0x7effbc1a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:06.097 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.223+0000 7effc2000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7effbc1097f0 con 0x7effbc07de60 2024-09-19T04:37:06.097 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.223+0000 7effc0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc07de60 0x7effbc1a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:06.097 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.223+0000 7effc0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc07de60 0x7effbc1a1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34678/0 (socket says 172.21.15.123:34678) 2024-09-19T04:37:06.097 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.223+0000 7effc0c00640 1 -- 172.21.15.123:0/1795769558 learned_addr learned my addr 172.21.15.123:0/1795769558 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:06.097 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.223+0000 7effc0c00640 1 -- 172.21.15.123:0/1795769558 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc1a20b0 con 0x7effbc07de60 2024-09-19T04:37:06.097 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.224+0000 7effc0c00640 1 --2- 172.21.15.123:0/1795769558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc07de60 0x7effbc1a1b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7effb0030380 tx=0x7effb0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:06.097 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.224+0000 7effb9600640 1 -- 172.21.15.123:0/1795769558 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effb0037040 con 0x7effbc07de60 2024-09-19T04:37:06.097 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.224+0000 7effb9600640 1 -- 172.21.15.123:0/1795769558 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb0040070 con 0x7effbc07de60 2024-09-19T04:37:06.097 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.224+0000 7effc2000640 1 -- 172.21.15.123:0/1795769558 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effbc1a2340 con 0x7effbc07de60 2024-09-19T04:37:06.097 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.224+0000 7effb9600640 1 -- 172.21.15.123:0/1795769558 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effb003a7b0 con 0x7effbc07de60 2024-09-19T04:37:06.097 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.224+0000 7effc2000640 1 -- 172.21.15.123:0/1795769558 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effbc1a5030 con 0x7effbc07de60 2024-09-19T04:37:06.097 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.225+0000 7effc2000640 1 -- 172.21.15.123:0/1795769558 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effbc07c390 con 0x7effbc07de60 2024-09-19T04:37:06.098 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.225+0000 7effb9600640 1 -- 172.21.15.123:0/1795769558 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7effb003bc00 con 0x7effbc07de60 2024-09-19T04:37:06.098 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.226+0000 7effb9600640 1 --2- 172.21.15.123:0/1795769558 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7effa40391b0 0x7effa403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:06.098 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.226+0000 7effb9600640 1 -- 172.21.15.123:0/1795769558 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7effb006c980 con 0x7effbc07de60 2024-09-19T04:37:06.098 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.226+0000 7effbbe00640 1 --2- 172.21.15.123:0/1795769558 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7effa40391b0 0x7effa403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:06.098 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.227+0000 7effbbe00640 1 --2- 172.21.15.123:0/1795769558 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7effa40391b0 0x7effa403b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7effa80098c0 tx=0x7effa8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:06.098 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.230+0000 7effb9600640 1 -- 172.21.15.123:0/1795769558 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7effb006c2d0 con 0x7effbc07de60 2024-09-19T04:37:06.098 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.413+0000 7effc2000640 1 -- 172.21.15.123:0/1795769558 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7effbc19afe0 con 0x7effbc07de60 2024-09-19T04:37:06.098 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.416+0000 7effb9600640 1 -- 172.21.15.123:0/1795769558 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7effb003b7a0 con 0x7effbc07de60 2024-09-19T04:37:06.098 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.419+0000 7effc2000640 1 -- 172.21.15.123:0/1795769558 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7effa40391b0 msgr2=0x7effa403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:06.098 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.419+0000 7effc2000640 1 --2- 172.21.15.123:0/1795769558 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7effa40391b0 0x7effa403b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7effa80098c0 tx=0x7effa8006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:06.098 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.419+0000 7effc2000640 1 -- 172.21.15.123:0/1795769558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc07de60 msgr2=0x7effbc1a1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:06.099 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.419+0000 7effc2000640 1 --2- 172.21.15.123:0/1795769558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc07de60 0x7effbc1a1b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7effb0030380 tx=0x7effb0032670 comp rx=0 tx=0).stop 2024-09-19T04:37:06.099 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.420+0000 7effc2000640 1 -- 172.21.15.123:0/1795769558 shutdown_connections 2024-09-19T04:37:06.099 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.420+0000 7effc2000640 1 --2- 172.21.15.123:0/1795769558 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7effa40391b0 0x7effa403b670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:06.099 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.420+0000 7effc2000640 1 --2- 172.21.15.123:0/1795769558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc07de60 0x7effbc1a1b70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:06.099 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.420+0000 7effc2000640 1 -- 172.21.15.123:0/1795769558 >> 172.21.15.123:0/1795769558 conn(0x7effbc102da0 msgr2=0x7effbc105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:06.099 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.420+0000 7effc2000640 1 -- 172.21.15.123:0/1795769558 shutdown_connections 2024-09-19T04:37:06.099 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:05.420+0000 7effc2000640 1 -- 172.21.15.123:0/1795769558 wait complete. 2024-09-19T04:37:06.099 INFO:teuthology.orchestra.run.smithi123.stdout:Enabling the dashboard module... 2024-09-19T04:37:06.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:06 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1795769558' entity='client.admin' 2024-09-19T04:37:06.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:06 smithi123 ceph-mon[30511]: from='mgr.14118 172.21.15.123:0/2355888057' entity='mgr.x' 2024-09-19T04:37:06.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:06 smithi123 ceph-mon[30511]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:07 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/160406975' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T04:37:07.677 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:07 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: ignoring --setuser ceph since I am not root 2024-09-19T04:37:07.677 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:07 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: ignoring --setgroup ceph since I am not root 2024-09-19T04:37:07.677 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:07 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:07.664+0000 7f8e099d3040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:37:08.000 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:07 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:07.734+0000 7f8e099d3040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:37:08.000 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:07 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:07.916+0000 7f8e099d3040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:37:08.000 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:07 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:07.998+0000 7f8e099d3040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:37:08.338 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:08 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:08.171+0000 7f8e099d3040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:37:08.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.585+0000 7f45f8600640 1 Processor -- start 2024-09-19T04:37:08.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.586+0000 7f45f8600640 1 -- start start 2024-09-19T04:37:08.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.586+0000 7f45f8600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f0109980 0x7f45f0109d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:08.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.586+0000 7f45f8600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45f010a350 con 0x7f45f0109980 2024-09-19T04:37:08.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.586+0000 7f45f5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f0109980 0x7f45f0109d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:08.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.586+0000 7f45f5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f0109980 0x7f45f0109d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34686/0 (socket says 172.21.15.123:34686) 2024-09-19T04:37:08.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.586+0000 7f45f5c00640 1 -- 172.21.15.123:0/1611522821 learned_addr learned my addr 172.21.15.123:0/1611522821 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:08.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.587+0000 7f45f5c00640 1 -- 172.21.15.123:0/1611522821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f010ab30 con 0x7f45f0109980 2024-09-19T04:37:08.401 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.588+0000 7f45f5c00640 1 --2- 172.21.15.123:0/1611522821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f0109980 0x7f45f0109d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009820 tx=0x7f45e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a5f88c8e2b1fa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:08.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.588+0000 7f45efe00640 1 -- 172.21.15.123:0/1611522821 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45e4037070 con 0x7f45f0109980 2024-09-19T04:37:08.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.588+0000 7f45efe00640 1 -- 172.21.15.123:0/1611522821 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45e4032920 con 0x7f45f0109980 2024-09-19T04:37:08.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.588+0000 7f45efe00640 1 -- 172.21.15.123:0/1611522821 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45e4030810 con 0x7f45f0109980 2024-09-19T04:37:08.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.589+0000 7f45f8600640 1 -- 172.21.15.123:0/1611522821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f0109980 msgr2=0x7f45f0109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:08.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.589+0000 7f45f8600640 1 --2- 172.21.15.123:0/1611522821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f0109980 0x7f45f0109d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009820 tx=0x7f45e402a270 comp rx=0 tx=0).stop 2024-09-19T04:37:08.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.589+0000 7f45f8600640 1 -- 172.21.15.123:0/1611522821 shutdown_connections 2024-09-19T04:37:08.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.589+0000 7f45f8600640 1 --2- 172.21.15.123:0/1611522821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f0109980 0x7f45f0109d80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:08.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.589+0000 7f45f8600640 1 -- 172.21.15.123:0/1611522821 >> 172.21.15.123:0/1611522821 conn(0x7f45f007cfe0 msgr2=0x7f45f007d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:08.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.589+0000 7f45f8600640 1 -- 172.21.15.123:0/1611522821 shutdown_connections 2024-09-19T04:37:08.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.589+0000 7f45f8600640 1 -- 172.21.15.123:0/1611522821 wait complete. 2024-09-19T04:37:08.402 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.590+0000 7f45f8600640 1 Processor -- start 2024-09-19T04:37:08.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.590+0000 7f45f8600640 1 -- start start 2024-09-19T04:37:08.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.591+0000 7f45f8600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f0109980 0x7f45f019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:08.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.591+0000 7f45f8600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45f010dee0 con 0x7f45f0109980 2024-09-19T04:37:08.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.591+0000 7f45f5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f0109980 0x7f45f019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:08.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.591+0000 7f45f5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f0109980 0x7f45f019d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34700/0 (socket says 172.21.15.123:34700) 2024-09-19T04:37:08.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.591+0000 7f45f5c00640 1 -- 172.21.15.123:0/160406975 learned_addr learned my addr 172.21.15.123:0/160406975 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:08.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.591+0000 7f45f5c00640 1 -- 172.21.15.123:0/160406975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f019de70 con 0x7f45f0109980 2024-09-19T04:37:08.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.592+0000 7f45f5c00640 1 --2- 172.21.15.123:0/160406975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f0109980 0x7f45f019d930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f45e40097f0 tx=0x7f45e40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:08.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.592+0000 7f45ee000640 1 -- 172.21.15.123:0/160406975 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45e4037040 con 0x7f45f0109980 2024-09-19T04:37:08.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.592+0000 7f45ee000640 1 -- 172.21.15.123:0/160406975 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45e4030cc0 con 0x7f45f0109980 2024-09-19T04:37:08.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.592+0000 7f45f8600640 1 -- 172.21.15.123:0/160406975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f019e100 con 0x7f45f0109980 2024-09-19T04:37:08.403 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.592+0000 7f45ee000640 1 -- 172.21.15.123:0/160406975 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45e403a640 con 0x7f45f0109980 2024-09-19T04:37:08.404 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.592+0000 7f45f8600640 1 -- 172.21.15.123:0/160406975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f01a0df0 con 0x7f45f0109980 2024-09-19T04:37:08.404 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.593+0000 7f45f8600640 1 -- 172.21.15.123:0/160406975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45f010e060 con 0x7f45f0109980 2024-09-19T04:37:08.404 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.594+0000 7f45ee000640 1 -- 172.21.15.123:0/160406975 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f45e403b3f0 con 0x7f45f0109980 2024-09-19T04:37:08.404 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.595+0000 7f45ee000640 1 --2- 172.21.15.123:0/160406975 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f45d8039200 0x7f45d803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:08.404 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.595+0000 7f45ee000640 1 -- 172.21.15.123:0/160406975 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f45e406d330 con 0x7f45f0109980 2024-09-19T04:37:08.404 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.595+0000 7f45f5200640 1 --2- 172.21.15.123:0/160406975 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f45d8039200 0x7f45d803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:08.404 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.596+0000 7f45f5200640 1 --2- 172.21.15.123:0/160406975 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f45d8039200 0x7f45d803b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0098c0 tx=0x7f45dc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:08.404 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.599+0000 7f45ee000640 1 -- 172.21.15.123:0/160406975 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f45e4038250 con 0x7f45f0109980 2024-09-19T04:37:08.404 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:06.830+0000 7f45f8600640 1 -- 172.21.15.123:0/160406975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f45f01a0fc0 con 0x7f45f0109980 2024-09-19T04:37:08.404 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:07.420+0000 7f45ee000640 1 -- 172.21.15.123:0/160406975 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f45e40328c0 con 0x7f45f0109980 2024-09-19T04:37:08.404 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:07.422+0000 7f45ee000640 1 -- 172.21.15.123:0/160406975 <== mon.0 v2:172.21.15.123:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f45e406bd20 con 0x7f45f0109980 2024-09-19T04:37:08.404 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:07.424+0000 7f45f8600640 1 -- 172.21.15.123:0/160406975 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f45d8039200 msgr2=0x7f45d803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:08.405 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:07.424+0000 7f45f8600640 1 --2- 172.21.15.123:0/160406975 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f45d8039200 0x7f45d803b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0098c0 tx=0x7f45dc006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:08.405 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:07.424+0000 7f45f8600640 1 -- 172.21.15.123:0/160406975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f0109980 msgr2=0x7f45f019d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:08.405 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:07.424+0000 7f45f8600640 1 --2- 172.21.15.123:0/160406975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f0109980 0x7f45f019d930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f45e40097f0 tx=0x7f45e40327b0 comp rx=0 tx=0).stop 2024-09-19T04:37:08.405 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:07.424+0000 7f45f8600640 1 -- 172.21.15.123:0/160406975 shutdown_connections 2024-09-19T04:37:08.405 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:07.424+0000 7f45f8600640 1 --2- 172.21.15.123:0/160406975 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f45d8039200 0x7f45d803b6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:08.405 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:07.424+0000 7f45f8600640 1 --2- 172.21.15.123:0/160406975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f0109980 0x7f45f019d930 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:08.405 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:07.424+0000 7f45f8600640 1 -- 172.21.15.123:0/160406975 >> 172.21.15.123:0/160406975 conn(0x7f45f007cfe0 msgr2=0x7f45f0107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:08.405 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:07.424+0000 7f45f8600640 1 -- 172.21.15.123:0/160406975 shutdown_connections 2024-09-19T04:37:08.405 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:07.424+0000 7f45f8600640 1 -- 172.21.15.123:0/160406975 wait complete. 2024-09-19T04:37:08.597 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:08 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/160406975' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T04:37:08.597 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:08 smithi123 ceph-mon[30511]: mgrmap e9: x(active, since 21s) 2024-09-19T04:37:08.597 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:08 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:08.336+0000 7f8e099d3040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:37:08.883 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:08 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:08.709+0000 7f8e099d3040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:37:08.883 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:08 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:08.784+0000 7f8e099d3040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:37:09.177 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:09 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:09.000+0000 7f8e099d3040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:37:09.575 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:09 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2015693582' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T04:37:09.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:37:09.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T04:37:09.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T04:37:09.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T04:37:09.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T04:37:09.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:37:09.864 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.991+0000 7f2c0bc00640 1 Processor -- start 2024-09-19T04:37:09.865 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.991+0000 7f2c0bc00640 1 -- start start 2024-09-19T04:37:09.865 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.991+0000 7f2c0bc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c04109fa0 0x7f2c0410a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:09.865 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.991+0000 7f2c0bc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0410a970 con 0x7f2c04109fa0 2024-09-19T04:37:09.865 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.991+0000 7f2c09200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c04109fa0 0x7f2c0410a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:09.865 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.991+0000 7f2c09200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c04109fa0 0x7f2c0410a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34730/0 (socket says 172.21.15.123:34730) 2024-09-19T04:37:09.865 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.992+0000 7f2c09200640 1 -- 172.21.15.123:0/3258867377 learned_addr learned my addr 172.21.15.123:0/3258867377 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:09.865 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.992+0000 7f2c09200640 1 -- 172.21.15.123:0/3258867377 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0410aaf0 con 0x7f2c04109fa0 2024-09-19T04:37:09.865 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.993+0000 7f2c09200640 1 --2- 172.21.15.123:0/3258867377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c04109fa0 0x7f2c0410a3a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009820 tx=0x7f2bf802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3266ee9c3ddbc5bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:09.865 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.993+0000 7f2c03400640 1 -- 172.21.15.123:0/3258867377 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bf8037070 con 0x7f2c04109fa0 2024-09-19T04:37:09.865 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.993+0000 7f2c03400640 1 -- 172.21.15.123:0/3258867377 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf8032920 con 0x7f2c04109fa0 2024-09-19T04:37:09.865 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.993+0000 7f2c03400640 1 -- 172.21.15.123:0/3258867377 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bf8030810 con 0x7f2c04109fa0 2024-09-19T04:37:09.865 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.994+0000 7f2c0bc00640 1 -- 172.21.15.123:0/3258867377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c04109fa0 msgr2=0x7f2c0410a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:09.866 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.994+0000 7f2c0bc00640 1 --2- 172.21.15.123:0/3258867377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c04109fa0 0x7f2c0410a3a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009820 tx=0x7f2bf802a270 comp rx=0 tx=0).stop 2024-09-19T04:37:09.866 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.994+0000 7f2c0bc00640 1 -- 172.21.15.123:0/3258867377 shutdown_connections 2024-09-19T04:37:09.866 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.994+0000 7f2c0bc00640 1 --2- 172.21.15.123:0/3258867377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c04109fa0 0x7f2c0410a3a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:09.866 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.994+0000 7f2c0bc00640 1 -- 172.21.15.123:0/3258867377 >> 172.21.15.123:0/3258867377 conn(0x7f2c0407cfe0 msgr2=0x7f2c0407d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:09.866 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.994+0000 7f2c0bc00640 1 -- 172.21.15.123:0/3258867377 shutdown_connections 2024-09-19T04:37:09.866 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.994+0000 7f2c0bc00640 1 -- 172.21.15.123:0/3258867377 wait complete. 2024-09-19T04:37:09.866 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.995+0000 7f2c0bc00640 1 Processor -- start 2024-09-19T04:37:09.866 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.995+0000 7f2c0bc00640 1 -- start start 2024-09-19T04:37:09.866 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.996+0000 7f2c0bc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c04109fa0 0x7f2c0419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:09.866 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.996+0000 7f2c0bc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0410df80 con 0x7f2c04109fa0 2024-09-19T04:37:09.866 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.996+0000 7f2c09200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c04109fa0 0x7f2c0419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:09.866 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.996+0000 7f2c09200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c04109fa0 0x7f2c0419d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34744/0 (socket says 172.21.15.123:34744) 2024-09-19T04:37:09.867 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.996+0000 7f2c09200640 1 -- 172.21.15.123:0/2015693582 learned_addr learned my addr 172.21.15.123:0/2015693582 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:09.867 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.996+0000 7f2c09200640 1 -- 172.21.15.123:0/2015693582 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0419deb0 con 0x7f2c04109fa0 2024-09-19T04:37:09.867 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.997+0000 7f2c09200640 1 --2- 172.21.15.123:0/2015693582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c04109fa0 0x7f2c0419d970 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8030380 tx=0x7f2bf8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:09.867 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.997+0000 7f2c01600640 1 -- 172.21.15.123:0/2015693582 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bf8037040 con 0x7f2c04109fa0 2024-09-19T04:37:09.867 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.997+0000 7f2c01600640 1 -- 172.21.15.123:0/2015693582 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf8040070 con 0x7f2c04109fa0 2024-09-19T04:37:09.867 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.997+0000 7f2c01600640 1 -- 172.21.15.123:0/2015693582 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bf803a770 con 0x7f2c04109fa0 2024-09-19T04:37:09.867 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.997+0000 7f2c0bc00640 1 -- 172.21.15.123:0/2015693582 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0419e140 con 0x7f2c04109fa0 2024-09-19T04:37:09.867 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.997+0000 7f2c0bc00640 1 -- 172.21.15.123:0/2015693582 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c041a0e30 con 0x7f2c04109fa0 2024-09-19T04:37:09.867 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.998+0000 7f2c01600640 1 -- 172.21.15.123:0/2015693582 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f2bf803b420 con 0x7f2c04109fa0 2024-09-19T04:37:09.867 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.998+0000 7f2c0bc00640 1 -- 172.21.15.123:0/2015693582 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c0410a420 con 0x7f2c04109fa0 2024-09-19T04:37:09.867 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.999+0000 7f2c01600640 1 --2- 172.21.15.123:0/2015693582 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f2bec039260 0x7f2bec03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:09.867 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.999+0000 7f2c01600640 1 -- 172.21.15.123:0/2015693582 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2bf806c8f0 con 0x7f2c04109fa0 2024-09-19T04:37:09.867 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.999+0000 7f2c03e00640 1 -- 172.21.15.123:0/2015693582 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f2bec039260 msgr2=0x7f2bec03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3323203947 2024-09-19T04:37:09.867 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:08.999+0000 7f2c03e00640 1 --2- 172.21.15.123:0/2015693582 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f2bec039260 0x7f2bec03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:37:09.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:09.004+0000 7f2c01600640 1 -- 172.21.15.123:0/2015693582 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2bf8043360 con 0x7f2c04109fa0 2024-09-19T04:37:09.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:09.200+0000 7f2c03e00640 1 -- 172.21.15.123:0/2015693582 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f2bec039260 msgr2=0x7f2bec03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3323203947 2024-09-19T04:37:09.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:09.200+0000 7f2c03e00640 1 --2- 172.21.15.123:0/2015693582 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f2bec039260 0x7f2bec03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:37:09.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:09.234+0000 7f2c0bc00640 1 -- 172.21.15.123:0/2015693582 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f2c041a1000 con 0x7f2c04109fa0 2024-09-19T04:37:09.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:09.234+0000 7f2c01600640 1 -- 172.21.15.123:0/2015693582 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f2c041a1000 con 0x7f2c04109fa0 2024-09-19T04:37:09.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:09.237+0000 7f2c0bc00640 1 -- 172.21.15.123:0/2015693582 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f2bec039260 msgr2=0x7f2bec03b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:37:09.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:09.237+0000 7f2c0bc00640 1 --2- 172.21.15.123:0/2015693582 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f2bec039260 0x7f2bec03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:09.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:09.237+0000 7f2c0bc00640 1 -- 172.21.15.123:0/2015693582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c04109fa0 msgr2=0x7f2c0419d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:09.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:09.237+0000 7f2c0bc00640 1 --2- 172.21.15.123:0/2015693582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c04109fa0 0x7f2c0419d970 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8030380 tx=0x7f2bf8032670 comp rx=0 tx=0).stop 2024-09-19T04:37:09.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:09.237+0000 7f2c0bc00640 1 -- 172.21.15.123:0/2015693582 shutdown_connections 2024-09-19T04:37:09.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:09.237+0000 7f2c0bc00640 1 --2- 172.21.15.123:0/2015693582 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7f2bec039260 0x7f2bec03b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:09.868 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:09.237+0000 7f2c0bc00640 1 --2- 172.21.15.123:0/2015693582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c04109fa0 0x7f2c0419d970 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:09.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:09.237+0000 7f2c0bc00640 1 -- 172.21.15.123:0/2015693582 >> 172.21.15.123:0/2015693582 conn(0x7f2c0407cfe0 msgr2=0x7f2c04107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:09.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:09.237+0000 7f2c0bc00640 1 -- 172.21.15.123:0/2015693582 shutdown_connections 2024-09-19T04:37:09.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:09.237+0000 7f2c0bc00640 1 -- 172.21.15.123:0/2015693582 wait complete. 2024-09-19T04:37:09.869 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for the mgr to restart... 2024-09-19T04:37:09.869 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mgr epoch 9... 2024-09-19T04:37:10.382 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:10 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:10.117+0000 7f8e099d3040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:37:10.382 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:10 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:10.201+0000 7f8e099d3040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:37:11.245 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:11 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:11.109+0000 7f8e099d3040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:37:11.644 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:11 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:11.243+0000 7f8e099d3040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:37:11.644 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:11 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:11.404+0000 7f8e099d3040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:37:11.644 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:11 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:11.475+0000 7f8e099d3040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:37:11.644 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:11 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:11.642+0000 7f8e099d3040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:37:11.927 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:11 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:11.722+0000 7f8e099d3040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:37:11.927 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:11 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:11.792+0000 7f8e099d3040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:37:12.291 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:11 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:11.972+0000 7f8e099d3040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:37:12.291 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:12 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:12.047+0000 7f8e099d3040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:37:12.291 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:12 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:12.124+0000 7f8e099d3040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:37:12.291 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:12 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:12.219+0000 7f8e099d3040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:37:12.677 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:12 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:12.289+0000 7f8e099d3040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:37:12.677 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:12 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:12.359+0000 7f8e099d3040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:37:13.677 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:13 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:13.255+0000 7f8e099d3040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:37:13.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:13 smithi123 ceph-mon[30511]: Active manager daemon x restarted 2024-09-19T04:37:13.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:13 smithi123 ceph-mon[30511]: Activating manager daemon x 2024-09-19T04:37:13.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:13 smithi123 ceph-mon[30511]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T04:37:13.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:13 smithi123 ceph-mon[30511]: mgrmap e10: x(active, starting, since 0.00458177s) 2024-09-19T04:37:13.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:37:13.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:37:13.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:37:13.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:37:13.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:37:13.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:13 smithi123 ceph-mon[30511]: Manager daemon x is now available 2024-09-19T04:37:13.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:37:13.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:37:14.667 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:37:14.667 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-mon[30511]: mgrmap e11: x(active, since 1.00803s) 2024-09-19T04:37:14.927 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:14.663+0000 7f8d5f600640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:04:37:14] ENGINE Error in HTTPServer.serve 2024-09-19T04:37:14.927 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: Traceback (most recent call last): 2024-09-19T04:37:14.927 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:37:14.927 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: self._connections.run(self.expiration_interval) 2024-09-19T04:37:14.928 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:37:14.928 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: self._run(expiration_interval) 2024-09-19T04:37:14.928 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:37:14.928 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:37:14.928 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:37:14.928 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:37:14.928 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:37:14.928 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: s = self.context.wrap_socket( 2024-09-19T04:37:14.928 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:37:14.928 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: return self.sslsocket_class._create( 2024-09-19T04:37:14.928 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:37:14.928 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: self.do_handshake() 2024-09-19T04:37:14.928 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:37:14.929 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: self._sslobj.do_handshake() 2024-09-19T04:37:14.929 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:37:14.929 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:37:14 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:37:15.587 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:15 smithi123 ceph-mon[30511]: [19/Sep/2024:04:37:14] ENGINE Bus STARTING 2024-09-19T04:37:15.587 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:15 smithi123 ceph-mon[30511]: [19/Sep/2024:04:37:14] ENGINE Serving on http://172.21.15.123:8765 2024-09-19T04:37:15.602 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:37:15.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T04:37:15.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T04:37:15.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:37:15.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.380+0000 7fc862200640 1 Processor -- start 2024-09-19T04:37:15.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.380+0000 7fc862200640 1 -- start start 2024-09-19T04:37:15.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.380+0000 7fc862200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc85c07de60 0x7fc85c07c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:15.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.380+0000 7fc862200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c07c8e0 con 0x7fc85c07de60 2024-09-19T04:37:15.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.381+0000 7fc860e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc85c07de60 0x7fc85c07c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:15.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.381+0000 7fc860e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc85c07de60 0x7fc85c07c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34758/0 (socket says 172.21.15.123:34758) 2024-09-19T04:37:15.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.381+0000 7fc860e00640 1 -- 172.21.15.123:0/4137139146 learned_addr learned my addr 172.21.15.123:0/4137139146 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:15.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.381+0000 7fc860e00640 1 -- 172.21.15.123:0/4137139146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85c07ca60 con 0x7fc85c07de60 2024-09-19T04:37:15.603 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.382+0000 7fc860e00640 1 --2- 172.21.15.123:0/4137139146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc85c07de60 0x7fc85c07c310 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc850009820 tx=0x7fc85002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3de65a80c848ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:15.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.382+0000 7fc85b400640 1 -- 172.21.15.123:0/4137139146 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc850037070 con 0x7fc85c07de60 2024-09-19T04:37:15.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.383+0000 7fc85b400640 1 -- 172.21.15.123:0/4137139146 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc850032920 con 0x7fc85c07de60 2024-09-19T04:37:15.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.383+0000 7fc85b400640 1 -- 172.21.15.123:0/4137139146 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc850030810 con 0x7fc85c07de60 2024-09-19T04:37:15.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.383+0000 7fc862200640 1 -- 172.21.15.123:0/4137139146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc85c07de60 msgr2=0x7fc85c07c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:15.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.383+0000 7fc862200640 1 --2- 172.21.15.123:0/4137139146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc85c07de60 0x7fc85c07c310 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc850009820 tx=0x7fc85002a270 comp rx=0 tx=0).stop 2024-09-19T04:37:15.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.383+0000 7fc862200640 1 -- 172.21.15.123:0/4137139146 shutdown_connections 2024-09-19T04:37:15.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.383+0000 7fc862200640 1 --2- 172.21.15.123:0/4137139146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc85c07de60 0x7fc85c07c310 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:15.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.383+0000 7fc862200640 1 -- 172.21.15.123:0/4137139146 >> 172.21.15.123:0/4137139146 conn(0x7fc85c102da0 msgr2=0x7fc85c1051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:15.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.383+0000 7fc862200640 1 -- 172.21.15.123:0/4137139146 shutdown_connections 2024-09-19T04:37:15.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.384+0000 7fc862200640 1 -- 172.21.15.123:0/4137139146 wait complete. 2024-09-19T04:37:15.604 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.384+0000 7fc862200640 1 Processor -- start 2024-09-19T04:37:15.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.385+0000 7fc862200640 1 -- start start 2024-09-19T04:37:15.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.385+0000 7fc862200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc85c07de60 0x7fc85c1a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:15.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.385+0000 7fc862200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c1097f0 con 0x7fc85c07de60 2024-09-19T04:37:15.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.385+0000 7fc860e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc85c07de60 0x7fc85c1a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:15.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.385+0000 7fc860e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc85c07de60 0x7fc85c1a1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34766/0 (socket says 172.21.15.123:34766) 2024-09-19T04:37:15.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.385+0000 7fc860e00640 1 -- 172.21.15.123:0/765778337 learned_addr learned my addr 172.21.15.123:0/765778337 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:15.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.386+0000 7fc860e00640 1 -- 172.21.15.123:0/765778337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85c1a20f0 con 0x7fc85c07de60 2024-09-19T04:37:15.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.386+0000 7fc860e00640 1 --2- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc85c07de60 0x7fc85c1a1bb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc8500097f0 tx=0x7fc8500327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:15.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.386+0000 7fc859600640 1 -- 172.21.15.123:0/765778337 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc850037040 con 0x7fc85c07de60 2024-09-19T04:37:15.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.386+0000 7fc859600640 1 -- 172.21.15.123:0/765778337 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc850030cc0 con 0x7fc85c07de60 2024-09-19T04:37:15.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.386+0000 7fc862200640 1 -- 172.21.15.123:0/765778337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85c1a2380 con 0x7fc85c07de60 2024-09-19T04:37:15.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.386+0000 7fc862200640 1 -- 172.21.15.123:0/765778337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc85c1a5070 con 0x7fc85c07de60 2024-09-19T04:37:15.605 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.386+0000 7fc859600640 1 -- 172.21.15.123:0/765778337 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc85003a6b0 con 0x7fc85c07de60 2024-09-19T04:37:15.606 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.388+0000 7fc859600640 1 -- 172.21.15.123:0/765778337 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fc85003baf0 con 0x7fc85c07de60 2024-09-19T04:37:15.606 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.388+0000 7fc859600640 1 --2- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fc844039200 0x7fc84403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:15.606 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.388+0000 7fc859600640 1 -- 172.21.15.123:0/765778337 --> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc84403bdd0 con 0x7fc844039200 2024-09-19T04:37:15.606 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.388+0000 7fc859600640 1 -- 172.21.15.123:0/765778337 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc85006c920 con 0x7fc85c07de60 2024-09-19T04:37:15.606 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.388+0000 7fc85be00640 1 -- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fc844039200 msgr2=0x7fc84403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3323203947 2024-09-19T04:37:15.606 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.388+0000 7fc85be00640 1 --2- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fc844039200 0x7fc84403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:37:15.606 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.589+0000 7fc85be00640 1 -- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fc844039200 msgr2=0x7fc84403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3323203947 2024-09-19T04:37:15.606 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.589+0000 7fc85be00640 1 --2- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fc844039200 0x7fc84403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:37:15.606 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.990+0000 7fc85be00640 1 -- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fc844039200 msgr2=0x7fc84403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3323203947 2024-09-19T04:37:15.606 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:10.990+0000 7fc85be00640 1 --2- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fc844039200 0x7fc84403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T04:37:15.606 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:11.791+0000 7fc85be00640 1 -- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fc844039200 msgr2=0x7fc84403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3323203947 2024-09-19T04:37:15.606 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:11.791+0000 7fc85be00640 1 --2- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fc844039200 0x7fc84403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T04:37:15.607 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:13.260+0000 7fc859600640 1 -- 172.21.15.123:0/765778337 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7fc85006b310 con 0x7fc85c07de60 2024-09-19T04:37:15.607 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:13.260+0000 7fc859600640 1 -- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fc844039200 msgr2=0x7fc84403b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:37:15.607 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:13.260+0000 7fc859600640 1 --2- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fc844039200 0x7fc84403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:15.607 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.261+0000 7fc859600640 1 -- 172.21.15.123:0/765778337 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7fc85003b3f0 con 0x7fc85c07de60 2024-09-19T04:37:15.607 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.261+0000 7fc859600640 1 --2- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc84403ccc0 0x7fc84403f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:15.607 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.261+0000 7fc859600640 1 -- 172.21.15.123:0/765778337 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc85003eec0 con 0x7fc84403ccc0 2024-09-19T04:37:15.607 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.262+0000 7fc85be00640 1 --2- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc84403ccc0 0x7fc84403f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:15.607 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.263+0000 7fc85be00640 1 --2- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc84403ccc0 0x7fc84403f0b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc848003980 tx=0x7fc848006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:15.607 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.264+0000 7fc859600640 1 -- 172.21.15.123:0/765778337 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fc85003eec0 con 0x7fc84403ccc0 2024-09-19T04:37:15.607 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.271+0000 7fc862200640 1 -- 172.21.15.123:0/765778337 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc85c07c390 con 0x7fc84403ccc0 2024-09-19T04:37:15.607 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.271+0000 7fc859600640 1 -- 172.21.15.123:0/765778337 <== mgr.14150 v2:172.21.15.123:6800/1532622003 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fc85c07c390 con 0x7fc84403ccc0 2024-09-19T04:37:15.607 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.272+0000 7fc862200640 1 -- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc84403ccc0 msgr2=0x7fc84403f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:15.607 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.272+0000 7fc862200640 1 --2- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc84403ccc0 0x7fc84403f0b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc848003980 tx=0x7fc848006e10 comp rx=0 tx=0).stop 2024-09-19T04:37:15.608 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.272+0000 7fc862200640 1 -- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc85c07de60 msgr2=0x7fc85c1a1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:15.608 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.272+0000 7fc862200640 1 --2- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc85c07de60 0x7fc85c1a1bb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc8500097f0 tx=0x7fc8500327b0 comp rx=0 tx=0).stop 2024-09-19T04:37:15.608 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.272+0000 7fc862200640 1 -- 172.21.15.123:0/765778337 shutdown_connections 2024-09-19T04:37:15.608 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.272+0000 7fc862200640 1 --2- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc84403ccc0 0x7fc84403f0b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:15.608 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.272+0000 7fc862200640 1 --2- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:6800/3323203947,v1:172.21.15.123:6801/3323203947] conn(0x7fc844039200 0x7fc84403b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:15.608 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.272+0000 7fc862200640 1 --2- 172.21.15.123:0/765778337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc85c07de60 0x7fc85c1a1bb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:15.608 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.272+0000 7fc862200640 1 -- 172.21.15.123:0/765778337 >> 172.21.15.123:0/765778337 conn(0x7fc85c102da0 msgr2=0x7fc85c105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:15.608 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.272+0000 7fc862200640 1 -- 172.21.15.123:0/765778337 shutdown_connections 2024-09-19T04:37:15.608 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:14.272+0000 7fc862200640 1 -- 172.21.15.123:0/765778337 wait complete. 2024-09-19T04:37:15.608 INFO:teuthology.orchestra.run.smithi123.stdout:mgr epoch 9 is available 2024-09-19T04:37:15.608 INFO:teuthology.orchestra.run.smithi123.stdout:Generating a dashboard self-signed certificate... 2024-09-19T04:37:16.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:16 smithi123 ceph-mon[30511]: [19/Sep/2024:04:37:14] ENGINE Serving on https://172.21.15.123:7150 2024-09-19T04:37:16.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:16 smithi123 ceph-mon[30511]: [19/Sep/2024:04:37:14] ENGINE Bus STARTED 2024-09-19T04:37:16.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:16 smithi123 ceph-mon[30511]: [19/Sep/2024:04:37:14] ENGINE Error in HTTPServer.serve 2024-09-19T04:37:16.677 INFO:journalctl@ceph.mon.a.smithi123.stdout: Traceback (most recent call last): 2024-09-19T04:37:16.677 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:37:16.677 INFO:journalctl@ceph.mon.a.smithi123.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:37:16.677 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:37:16.678 INFO:journalctl@ceph.mon.a.smithi123.stdout: self._run(expiration_interval) 2024-09-19T04:37:16.678 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:37:16.678 INFO:journalctl@ceph.mon.a.smithi123.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:37:16.678 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:37:16.678 INFO:journalctl@ceph.mon.a.smithi123.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:37:16.678 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:37:16.678 INFO:journalctl@ceph.mon.a.smithi123.stdout: s = self.context.wrap_socket( 2024-09-19T04:37:16.678 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:37:16.678 INFO:journalctl@ceph.mon.a.smithi123.stdout: return self.sslsocket_class._create( 2024-09-19T04:37:16.678 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:37:16.678 INFO:journalctl@ceph.mon.a.smithi123.stdout: self.do_handshake() 2024-09-19T04:37:16.678 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:37:16.678 INFO:journalctl@ceph.mon.a.smithi123.stdout: self._sslobj.do_handshake() 2024-09-19T04:37:16.678 INFO:journalctl@ceph.mon.a.smithi123.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:37:16.679 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:16 smithi123 ceph-mon[30511]: mgrmap e12: x(active, since 2s) 2024-09-19T04:37:16.679 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:16 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:17.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T04:37:17.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.123+0000 7f8110800640 1 Processor -- start 2024-09-19T04:37:17.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.123+0000 7f8110800640 1 -- start start 2024-09-19T04:37:17.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.124+0000 7f8110800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8108107790 0x7f8108107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:17.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.124+0000 7f8110800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8108108160 con 0x7f8108107790 2024-09-19T04:37:17.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.124+0000 7f810de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8108107790 0x7f8108107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:17.062 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.124+0000 7f810de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8108107790 0x7f8108107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52838/0 (socket says 172.21.15.123:52838) 2024-09-19T04:37:17.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.124+0000 7f810de00640 1 -- 172.21.15.123:0/3806415050 learned_addr learned my addr 172.21.15.123:0/3806415050 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:17.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.125+0000 7f810de00640 1 -- 172.21.15.123:0/3806415050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8108108940 con 0x7f8108107790 2024-09-19T04:37:17.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.126+0000 7f810de00640 1 --2- 172.21.15.123:0/3806415050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8108107790 0x7f8108107b90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009820 tx=0x7f80fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd5ad080350d32b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:17.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.126+0000 7f810ca00640 1 -- 172.21.15.123:0/3806415050 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80fc037070 con 0x7f8108107790 2024-09-19T04:37:17.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.126+0000 7f810ca00640 1 -- 172.21.15.123:0/3806415050 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80fc032920 con 0x7f8108107790 2024-09-19T04:37:17.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.126+0000 7f810ca00640 1 -- 172.21.15.123:0/3806415050 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80fc030810 con 0x7f8108107790 2024-09-19T04:37:17.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.127+0000 7f8110800640 1 -- 172.21.15.123:0/3806415050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8108107790 msgr2=0x7f8108107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:17.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.127+0000 7f8110800640 1 --2- 172.21.15.123:0/3806415050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8108107790 0x7f8108107b90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009820 tx=0x7f80fc02a270 comp rx=0 tx=0).stop 2024-09-19T04:37:17.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.127+0000 7f8110800640 1 -- 172.21.15.123:0/3806415050 shutdown_connections 2024-09-19T04:37:17.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.127+0000 7f8110800640 1 --2- 172.21.15.123:0/3806415050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8108107790 0x7f8108107b90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:17.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.127+0000 7f8110800640 1 -- 172.21.15.123:0/3806415050 >> 172.21.15.123:0/3806415050 conn(0x7f8108102fa0 msgr2=0x7f81081053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:17.063 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.127+0000 7f8110800640 1 -- 172.21.15.123:0/3806415050 shutdown_connections 2024-09-19T04:37:17.064 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.127+0000 7f8110800640 1 -- 172.21.15.123:0/3806415050 wait complete. 2024-09-19T04:37:17.064 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.128+0000 7f8110800640 1 Processor -- start 2024-09-19T04:37:17.064 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.128+0000 7f8110800640 1 -- start start 2024-09-19T04:37:17.064 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.129+0000 7f8110800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8108107790 0x7f810819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:17.064 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.129+0000 7f8110800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f810810bcf0 con 0x7f8108107790 2024-09-19T04:37:17.064 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.129+0000 7f810de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8108107790 0x7f810819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:17.064 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.129+0000 7f810de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8108107790 0x7f810819d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52850/0 (socket says 172.21.15.123:52850) 2024-09-19T04:37:17.064 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.129+0000 7f810de00640 1 -- 172.21.15.123:0/4191061238 learned_addr learned my addr 172.21.15.123:0/4191061238 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:17.064 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.129+0000 7f810de00640 1 -- 172.21.15.123:0/4191061238 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810819dee0 con 0x7f8108107790 2024-09-19T04:37:17.064 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.129+0000 7f810de00640 1 --2- 172.21.15.123:0/4191061238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8108107790 0x7f810819d9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0097f0 tx=0x7f80fc0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:17.064 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.130+0000 7f8102a00640 1 -- 172.21.15.123:0/4191061238 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80fc037040 con 0x7f8108107790 2024-09-19T04:37:17.064 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.130+0000 7f8102a00640 1 -- 172.21.15.123:0/4191061238 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80fc030cc0 con 0x7f8108107790 2024-09-19T04:37:17.064 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.130+0000 7f8102a00640 1 -- 172.21.15.123:0/4191061238 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80fc03a640 con 0x7f8108107790 2024-09-19T04:37:17.065 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.130+0000 7f8110800640 1 -- 172.21.15.123:0/4191061238 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810819e170 con 0x7f8108107790 2024-09-19T04:37:17.065 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.130+0000 7f8110800640 1 -- 172.21.15.123:0/4191061238 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81081a0e60 con 0x7f8108107790 2024-09-19T04:37:17.065 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.131+0000 7f8102a00640 1 -- 172.21.15.123:0/4191061238 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f80fc03b3f0 con 0x7f8108107790 2024-09-19T04:37:17.065 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.131+0000 7f8110800640 1 -- 172.21.15.123:0/4191061238 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f810810be70 con 0x7f8108107790 2024-09-19T04:37:17.065 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.131+0000 7f8102a00640 1 --2- 172.21.15.123:0/4191061238 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f80f0039200 0x7f80f003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:17.065 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.131+0000 7f8102a00640 1 -- 172.21.15.123:0/4191061238 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f80fc06c9e0 con 0x7f8108107790 2024-09-19T04:37:17.065 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.132+0000 7f810d400640 1 --2- 172.21.15.123:0/4191061238 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f80f0039200 0x7f80f003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:17.065 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.133+0000 7f810d400640 1 --2- 172.21.15.123:0/4191061238 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f80f0039200 0x7f80f003b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f80f40098c0 tx=0x7f80f4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:17.065 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.137+0000 7f8102a00640 1 -- 172.21.15.123:0/4191061238 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f80fc042e10 con 0x7f8108107790 2024-09-19T04:37:17.065 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.329+0000 7f8110800640 1 -- 172.21.15.123:0/4191061238 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f81081041c0 con 0x7f80f0039200 2024-09-19T04:37:17.065 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.423+0000 7f8102a00640 1 -- 172.21.15.123:0/4191061238 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f81081041c0 con 0x7f80f0039200 2024-09-19T04:37:17.065 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.425+0000 7f8110800640 1 -- 172.21.15.123:0/4191061238 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f80f0039200 msgr2=0x7f80f003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:17.066 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.425+0000 7f8110800640 1 --2- 172.21.15.123:0/4191061238 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f80f0039200 0x7f80f003b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f80f40098c0 tx=0x7f80f4006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:17.066 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.425+0000 7f8110800640 1 -- 172.21.15.123:0/4191061238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8108107790 msgr2=0x7f810819d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:17.066 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.425+0000 7f8110800640 1 --2- 172.21.15.123:0/4191061238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8108107790 0x7f810819d9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0097f0 tx=0x7f80fc0327b0 comp rx=0 tx=0).stop 2024-09-19T04:37:17.066 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.425+0000 7f8110800640 1 -- 172.21.15.123:0/4191061238 shutdown_connections 2024-09-19T04:37:17.066 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.425+0000 7f8110800640 1 --2- 172.21.15.123:0/4191061238 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f80f0039200 0x7f80f003b6c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:17.066 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.425+0000 7f8110800640 1 --2- 172.21.15.123:0/4191061238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8108107790 0x7f810819d9a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:17.066 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.425+0000 7f8110800640 1 -- 172.21.15.123:0/4191061238 >> 172.21.15.123:0/4191061238 conn(0x7f8108102fa0 msgr2=0x7f8108103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:17.066 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.425+0000 7f8110800640 1 -- 172.21.15.123:0/4191061238 shutdown_connections 2024-09-19T04:37:17.066 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:16.425+0000 7f8110800640 1 -- 172.21.15.123:0/4191061238 wait complete. 2024-09-19T04:37:17.066 INFO:teuthology.orchestra.run.smithi123.stdout:Creating initial admin user... 2024-09-19T04:37:17.630 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:17 smithi123 ceph-mon[30511]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:37:17.631 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:17.631 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:17.631 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:19.299 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$a5hlUJuqsyvNhJFx9wNAIuipAuwT7bc7eDo2myJ2CRqJZGaUQ5bvK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726720638, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T04:37:19.299 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.623+0000 7fe669800640 1 Processor -- start 2024-09-19T04:37:19.299 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.623+0000 7fe669800640 1 -- start start 2024-09-19T04:37:19.299 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.623+0000 7fe669800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe664106040 0x7fe664106440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:19.300 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.623+0000 7fe669800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe66407bd90 con 0x7fe664106040 2024-09-19T04:37:19.300 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.624+0000 7fe662a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe664106040 0x7fe664106440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:19.300 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.624+0000 7fe662a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe664106040 0x7fe664106440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52866/0 (socket says 172.21.15.123:52866) 2024-09-19T04:37:19.300 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.624+0000 7fe662a00640 1 -- 172.21.15.123:0/108212257 learned_addr learned my addr 172.21.15.123:0/108212257 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:19.300 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.624+0000 7fe662a00640 1 -- 172.21.15.123:0/108212257 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe664106980 con 0x7fe664106040 2024-09-19T04:37:19.300 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.625+0000 7fe662a00640 1 --2- 172.21.15.123:0/108212257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe664106040 0x7fe664106440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe654009820 tx=0x7fe65402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddcd557fb01eab01 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:19.300 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.625+0000 7fe661600640 1 -- 172.21.15.123:0/108212257 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe654037070 con 0x7fe664106040 2024-09-19T04:37:19.300 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.625+0000 7fe661600640 1 -- 172.21.15.123:0/108212257 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe654032920 con 0x7fe664106040 2024-09-19T04:37:19.300 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.625+0000 7fe661600640 1 -- 172.21.15.123:0/108212257 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe654030810 con 0x7fe664106040 2024-09-19T04:37:19.300 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.626+0000 7fe669800640 1 -- 172.21.15.123:0/108212257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe664106040 msgr2=0x7fe664106440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:19.300 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.626+0000 7fe669800640 1 --2- 172.21.15.123:0/108212257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe664106040 0x7fe664106440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe654009820 tx=0x7fe65402a270 comp rx=0 tx=0).stop 2024-09-19T04:37:19.300 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.626+0000 7fe669800640 1 -- 172.21.15.123:0/108212257 shutdown_connections 2024-09-19T04:37:19.301 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.626+0000 7fe669800640 1 --2- 172.21.15.123:0/108212257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe664106040 0x7fe664106440 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:19.301 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.626+0000 7fe669800640 1 -- 172.21.15.123:0/108212257 >> 172.21.15.123:0/108212257 conn(0x7fe664101db0 msgr2=0x7fe664104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:19.301 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.626+0000 7fe669800640 1 -- 172.21.15.123:0/108212257 shutdown_connections 2024-09-19T04:37:19.301 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.626+0000 7fe669800640 1 -- 172.21.15.123:0/108212257 wait complete. 2024-09-19T04:37:19.301 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.627+0000 7fe669800640 1 Processor -- start 2024-09-19T04:37:19.301 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.628+0000 7fe669800640 1 -- start start 2024-09-19T04:37:19.301 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.628+0000 7fe669800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe664106040 0x7fe66419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:19.301 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.628+0000 7fe669800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe664107ac0 con 0x7fe664106040 2024-09-19T04:37:19.301 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.628+0000 7fe662a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe664106040 0x7fe66419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:19.301 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.628+0000 7fe662a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe664106040 0x7fe66419d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52880/0 (socket says 172.21.15.123:52880) 2024-09-19T04:37:19.301 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.628+0000 7fe662a00640 1 -- 172.21.15.123:0/2994878769 learned_addr learned my addr 172.21.15.123:0/2994878769 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:19.301 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.629+0000 7fe662a00640 1 -- 172.21.15.123:0/2994878769 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66419dea0 con 0x7fe664106040 2024-09-19T04:37:19.302 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.629+0000 7fe662a00640 1 --2- 172.21.15.123:0/2994878769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe664106040 0x7fe66419d960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe6540097f0 tx=0x7fe6540327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:19.302 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.629+0000 7fe65b400640 1 -- 172.21.15.123:0/2994878769 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe654037040 con 0x7fe664106040 2024-09-19T04:37:19.302 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.630+0000 7fe65b400640 1 -- 172.21.15.123:0/2994878769 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe654030c40 con 0x7fe664106040 2024-09-19T04:37:19.302 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.630+0000 7fe65b400640 1 -- 172.21.15.123:0/2994878769 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe65403a5d0 con 0x7fe664106040 2024-09-19T04:37:19.302 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.630+0000 7fe669800640 1 -- 172.21.15.123:0/2994878769 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66419e130 con 0x7fe664106040 2024-09-19T04:37:19.302 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.630+0000 7fe669800640 1 -- 172.21.15.123:0/2994878769 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6641a0e20 con 0x7fe664106040 2024-09-19T04:37:19.302 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.631+0000 7fe65b400640 1 -- 172.21.15.123:0/2994878769 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fe65403a7c0 con 0x7fe664106040 2024-09-19T04:37:19.302 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.631+0000 7fe65b400640 1 --2- 172.21.15.123:0/2994878769 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe648039260 0x7fe64803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:19.302 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.631+0000 7fe65b400640 1 -- 172.21.15.123:0/2994878769 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe65406c9b0 con 0x7fe664106040 2024-09-19T04:37:19.302 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.631+0000 7fe669800640 1 -- 172.21.15.123:0/2994878769 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe664106440 con 0x7fe664106040 2024-09-19T04:37:19.302 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.632+0000 7fe662000640 1 --2- 172.21.15.123:0/2994878769 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe648039260 0x7fe64803b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:19.303 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.633+0000 7fe662000640 1 --2- 172.21.15.123:0/2994878769 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe648039260 0x7fe64803b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe6500098c0 tx=0x7fe650006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:19.303 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.638+0000 7fe65b400640 1 -- 172.21.15.123:0/2994878769 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe6540392f0 con 0x7fe664106040 2024-09-19T04:37:19.303 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:17.831+0000 7fe669800640 1 -- 172.21.15.123:0/2994878769 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fe6641a0ff0 con 0x7fe648039260 2024-09-19T04:37:19.303 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:18.058+0000 7fe65b400640 1 -- 172.21.15.123:0/2994878769 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fe6641a0ff0 con 0x7fe648039260 2024-09-19T04:37:19.303 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:18.061+0000 7fe669800640 1 -- 172.21.15.123:0/2994878769 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe648039260 msgr2=0x7fe64803b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:19.303 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:18.061+0000 7fe669800640 1 --2- 172.21.15.123:0/2994878769 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe648039260 0x7fe64803b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe6500098c0 tx=0x7fe650006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:19.303 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:18.061+0000 7fe669800640 1 -- 172.21.15.123:0/2994878769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe664106040 msgr2=0x7fe66419d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:19.303 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:18.061+0000 7fe669800640 1 --2- 172.21.15.123:0/2994878769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe664106040 0x7fe66419d960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe6540097f0 tx=0x7fe6540327b0 comp rx=0 tx=0).stop 2024-09-19T04:37:19.303 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:18.061+0000 7fe669800640 1 -- 172.21.15.123:0/2994878769 shutdown_connections 2024-09-19T04:37:19.303 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:18.061+0000 7fe669800640 1 --2- 172.21.15.123:0/2994878769 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe648039260 0x7fe64803b720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:19.303 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:18.061+0000 7fe669800640 1 --2- 172.21.15.123:0/2994878769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe664106040 0x7fe66419d960 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:19.303 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:18.061+0000 7fe669800640 1 -- 172.21.15.123:0/2994878769 >> 172.21.15.123:0/2994878769 conn(0x7fe664101db0 msgr2=0x7fe664191c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:19.303 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:18.061+0000 7fe669800640 1 -- 172.21.15.123:0/2994878769 shutdown_connections 2024-09-19T04:37:19.303 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:18.061+0000 7fe669800640 1 -- 172.21.15.123:0/2994878769 wait complete. 2024-09-19T04:37:19.304 INFO:teuthology.orchestra.run.smithi123.stdout:Fetching dashboard port number... 2024-09-19T04:37:19.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:19 smithi123 ceph-mon[30511]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:37:19.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:19 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:20.329 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:20 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2083095775' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T04:37:20.623 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T04:37:20.623 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.772+0000 7f06e9000640 1 Processor -- start 2024-09-19T04:37:20.623 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.773+0000 7f06e9000640 1 -- start start 2024-09-19T04:37:20.623 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.773+0000 7f06e9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e4107790 0x7f06e4107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:20.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.773+0000 7f06e9000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f06e4108160 con 0x7f06e4107790 2024-09-19T04:37:20.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.773+0000 7f06e1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e4107790 0x7f06e4107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:20.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.773+0000 7f06e1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e4107790 0x7f06e4107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52894/0 (socket says 172.21.15.123:52894) 2024-09-19T04:37:20.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.773+0000 7f06e1e00640 1 -- 172.21.15.123:0/3358477680 learned_addr learned my addr 172.21.15.123:0/3358477680 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:20.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.774+0000 7f06e1e00640 1 -- 172.21.15.123:0/3358477680 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e4108940 con 0x7f06e4107790 2024-09-19T04:37:20.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.775+0000 7f06e1e00640 1 --2- 172.21.15.123:0/3358477680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e4107790 0x7f06e4107b90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009820 tx=0x7f06d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43dec8de9f30f67 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:20.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.775+0000 7f06e0a00640 1 -- 172.21.15.123:0/3358477680 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06d4037070 con 0x7f06e4107790 2024-09-19T04:37:20.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.775+0000 7f06e0a00640 1 -- 172.21.15.123:0/3358477680 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d4032920 con 0x7f06e4107790 2024-09-19T04:37:20.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.775+0000 7f06e0a00640 1 -- 172.21.15.123:0/3358477680 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06d4030810 con 0x7f06e4107790 2024-09-19T04:37:20.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.775+0000 7f06e9000640 1 -- 172.21.15.123:0/3358477680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e4107790 msgr2=0x7f06e4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:20.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.775+0000 7f06e9000640 1 --2- 172.21.15.123:0/3358477680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e4107790 0x7f06e4107b90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009820 tx=0x7f06d402a270 comp rx=0 tx=0).stop 2024-09-19T04:37:20.624 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.775+0000 7f06e9000640 1 -- 172.21.15.123:0/3358477680 shutdown_connections 2024-09-19T04:37:20.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.775+0000 7f06e9000640 1 --2- 172.21.15.123:0/3358477680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e4107790 0x7f06e4107b90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:20.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.775+0000 7f06e9000640 1 -- 172.21.15.123:0/3358477680 >> 172.21.15.123:0/3358477680 conn(0x7f06e4102fa0 msgr2=0x7f06e41053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:20.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.776+0000 7f06e9000640 1 -- 172.21.15.123:0/3358477680 shutdown_connections 2024-09-19T04:37:20.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.776+0000 7f06e9000640 1 -- 172.21.15.123:0/3358477680 wait complete. 2024-09-19T04:37:20.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.776+0000 7f06e9000640 1 Processor -- start 2024-09-19T04:37:20.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.777+0000 7f06e9000640 1 -- start start 2024-09-19T04:37:20.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.777+0000 7f06e9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e4107790 0x7f06e407e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:20.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.777+0000 7f06e9000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f06e410bcf0 con 0x7f06e4107790 2024-09-19T04:37:20.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.777+0000 7f06e1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e4107790 0x7f06e407e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:20.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.777+0000 7f06e1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e4107790 0x7f06e407e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52908/0 (socket says 172.21.15.123:52908) 2024-09-19T04:37:20.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.777+0000 7f06e1e00640 1 -- 172.21.15.123:0/2083095775 learned_addr learned my addr 172.21.15.123:0/2083095775 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:20.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.777+0000 7f06e1e00640 1 -- 172.21.15.123:0/2083095775 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e407e910 con 0x7f06e4107790 2024-09-19T04:37:20.625 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.778+0000 7f06e1e00640 1 --2- 172.21.15.123:0/2083095775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e4107790 0x7f06e407e3d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f06d40097f0 tx=0x7f06d40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:20.626 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.778+0000 7f06daa00640 1 -- 172.21.15.123:0/2083095775 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06d4037040 con 0x7f06e4107790 2024-09-19T04:37:20.626 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.778+0000 7f06daa00640 1 -- 172.21.15.123:0/2083095775 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d4030cc0 con 0x7f06e4107790 2024-09-19T04:37:20.626 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.778+0000 7f06e9000640 1 -- 172.21.15.123:0/2083095775 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e407cb00 con 0x7f06e4107790 2024-09-19T04:37:20.626 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.778+0000 7f06daa00640 1 -- 172.21.15.123:0/2083095775 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06d403a640 con 0x7f06e4107790 2024-09-19T04:37:20.626 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.778+0000 7f06e9000640 1 -- 172.21.15.123:0/2083095775 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e407d040 con 0x7f06e4107790 2024-09-19T04:37:20.626 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.779+0000 7f06daa00640 1 -- 172.21.15.123:0/2083095775 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f06d403b3f0 con 0x7f06e4107790 2024-09-19T04:37:20.626 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.779+0000 7f06e9000640 1 -- 172.21.15.123:0/2083095775 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06e4107c10 con 0x7f06e4107790 2024-09-19T04:37:20.626 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.780+0000 7f06daa00640 1 --2- 172.21.15.123:0/2083095775 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f06c8039260 0x7f06c803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:20.626 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.780+0000 7f06daa00640 1 -- 172.21.15.123:0/2083095775 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f06d406c990 con 0x7f06e4107790 2024-09-19T04:37:20.626 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.780+0000 7f06e1400640 1 --2- 172.21.15.123:0/2083095775 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f06c8039260 0x7f06c803b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:20.626 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.780+0000 7f06e1400640 1 --2- 172.21.15.123:0/2083095775 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f06c8039260 0x7f06c803b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f06d00098c0 tx=0x7f06d0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:20.626 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.784+0000 7f06daa00640 1 -- 172.21.15.123:0/2083095775 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f06d40328c0 con 0x7f06e4107790 2024-09-19T04:37:20.627 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.958+0000 7f06e9000640 1 -- 172.21.15.123:0/2083095775 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f06e407d2c0 con 0x7f06e4107790 2024-09-19T04:37:20.627 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.958+0000 7f06daa00640 1 -- 172.21.15.123:0/2083095775 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f06d4042dc0 con 0x7f06e4107790 2024-09-19T04:37:20.627 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.960+0000 7f06e9000640 1 -- 172.21.15.123:0/2083095775 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f06c8039260 msgr2=0x7f06c803b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:20.627 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.960+0000 7f06e9000640 1 --2- 172.21.15.123:0/2083095775 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f06c8039260 0x7f06c803b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f06d00098c0 tx=0x7f06d0006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:20.627 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.960+0000 7f06e9000640 1 -- 172.21.15.123:0/2083095775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e4107790 msgr2=0x7f06e407e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:20.627 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.960+0000 7f06e9000640 1 --2- 172.21.15.123:0/2083095775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e4107790 0x7f06e407e3d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f06d40097f0 tx=0x7f06d40327b0 comp rx=0 tx=0).stop 2024-09-19T04:37:20.627 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.960+0000 7f06e9000640 1 -- 172.21.15.123:0/2083095775 shutdown_connections 2024-09-19T04:37:20.627 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.960+0000 7f06e9000640 1 --2- 172.21.15.123:0/2083095775 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f06c8039260 0x7f06c803b720 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:20.627 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.960+0000 7f06e9000640 1 --2- 172.21.15.123:0/2083095775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e4107790 0x7f06e407e3d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:20.627 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.960+0000 7f06e9000640 1 -- 172.21.15.123:0/2083095775 >> 172.21.15.123:0/2083095775 conn(0x7f06e4102fa0 msgr2=0x7f06e4103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:20.627 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.961+0000 7f06e9000640 1 -- 172.21.15.123:0/2083095775 shutdown_connections 2024-09-19T04:37:20.627 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:19.961+0000 7f06e9000640 1 -- 172.21.15.123:0/2083095775 wait complete. 2024-09-19T04:37:20.639 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:37:20.639 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout disabled 2024-09-19T04:37:20.652 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:37:20.653 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-09-19T04:37:20.653 INFO:teuthology.orchestra.run.smithi123.stdout:firewalld.service is not enabled 2024-09-19T04:37:20.653 INFO:teuthology.orchestra.run.smithi123.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T04:37:20.655 INFO:teuthology.orchestra.run.smithi123.stdout:Ceph Dashboard is now available at: 2024-09-19T04:37:20.655 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:37:20.655 INFO:teuthology.orchestra.run.smithi123.stdout: URL: https://smithi123.front.sepia.ceph.com:8443/ 2024-09-19T04:37:20.655 INFO:teuthology.orchestra.run.smithi123.stdout: User: admin 2024-09-19T04:37:20.655 INFO:teuthology.orchestra.run.smithi123.stdout: Password: aow72x7zvl 2024-09-19T04:37:20.656 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:37:20.656 INFO:teuthology.orchestra.run.smithi123.stdout:Saving cluster configuration to /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config directory 2024-09-19T04:37:21.137 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:21 smithi123 ceph-mon[30511]: mgrmap e13: x(active, since 6s) 2024-09-19T04:37:22.041 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.123+0000 7fc878e00640 1 Processor -- start 2024-09-19T04:37:22.046 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.123+0000 7fc878e00640 1 -- start start 2024-09-19T04:37:22.046 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.123+0000 7fc878e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc874105620 0x7fc874105a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:22.046 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.123+0000 7fc878e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc874105ff0 con 0x7fc874105620 2024-09-19T04:37:22.046 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.124+0000 7fc871e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc874105620 0x7fc874105a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:22.046 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.124+0000 7fc871e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc874105620 0x7fc874105a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52918/0 (socket says 172.21.15.123:52918) 2024-09-19T04:37:22.046 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.124+0000 7fc871e00640 1 -- 172.21.15.123:0/3075804496 learned_addr learned my addr 172.21.15.123:0/3075804496 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:22.047 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.124+0000 7fc871e00640 1 -- 172.21.15.123:0/3075804496 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc874106780 con 0x7fc874105620 2024-09-19T04:37:22.047 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.125+0000 7fc871e00640 1 --2- 172.21.15.123:0/3075804496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc874105620 0x7fc874105a20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc864009820 tx=0x7fc86402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8ad48780e32392 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:22.047 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.125+0000 7fc870a00640 1 -- 172.21.15.123:0/3075804496 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc864037070 con 0x7fc874105620 2024-09-19T04:37:22.047 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.125+0000 7fc870a00640 1 -- 172.21.15.123:0/3075804496 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc864032920 con 0x7fc874105620 2024-09-19T04:37:22.047 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.125+0000 7fc870a00640 1 -- 172.21.15.123:0/3075804496 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc864030810 con 0x7fc874105620 2024-09-19T04:37:22.047 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.126+0000 7fc878e00640 1 -- 172.21.15.123:0/3075804496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc874105620 msgr2=0x7fc874105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:22.047 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.126+0000 7fc878e00640 1 --2- 172.21.15.123:0/3075804496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc874105620 0x7fc874105a20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc864009820 tx=0x7fc86402a270 comp rx=0 tx=0).stop 2024-09-19T04:37:22.047 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.126+0000 7fc878e00640 1 -- 172.21.15.123:0/3075804496 shutdown_connections 2024-09-19T04:37:22.047 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.126+0000 7fc878e00640 1 --2- 172.21.15.123:0/3075804496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc874105620 0x7fc874105a20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:22.047 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.126+0000 7fc878e00640 1 -- 172.21.15.123:0/3075804496 >> 172.21.15.123:0/3075804496 conn(0x7fc874101180 msgr2=0x7fc8741035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:22.047 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.126+0000 7fc878e00640 1 -- 172.21.15.123:0/3075804496 shutdown_connections 2024-09-19T04:37:22.047 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.126+0000 7fc878e00640 1 -- 172.21.15.123:0/3075804496 wait complete. 2024-09-19T04:37:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.127+0000 7fc878e00640 1 Processor -- start 2024-09-19T04:37:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.127+0000 7fc878e00640 1 -- start start 2024-09-19T04:37:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.127+0000 7fc878e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc874105620 0x7fc8741a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.127+0000 7fc878e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc874109b50 con 0x7fc874105620 2024-09-19T04:37:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.128+0000 7fc871e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc874105620 0x7fc8741a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.128+0000 7fc871e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc874105620 0x7fc8741a1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52926/0 (socket says 172.21.15.123:52926) 2024-09-19T04:37:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.128+0000 7fc871e00640 1 -- 172.21.15.123:0/1633467656 learned_addr learned my addr 172.21.15.123:0/1633467656 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.128+0000 7fc871e00640 1 -- 172.21.15.123:0/1633467656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8741a22d0 con 0x7fc874105620 2024-09-19T04:37:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.128+0000 7fc871e00640 1 --2- 172.21.15.123:0/1633467656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc874105620 0x7fc8741a1d90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc8640097f0 tx=0x7fc8640327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.128+0000 7fc86aa00640 1 -- 172.21.15.123:0/1633467656 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc864037040 con 0x7fc874105620 2024-09-19T04:37:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.128+0000 7fc86aa00640 1 -- 172.21.15.123:0/1633467656 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc864030cc0 con 0x7fc874105620 2024-09-19T04:37:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.129+0000 7fc878e00640 1 -- 172.21.15.123:0/1633467656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8741a2560 con 0x7fc874105620 2024-09-19T04:37:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.129+0000 7fc86aa00640 1 -- 172.21.15.123:0/1633467656 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc86403a6b0 con 0x7fc874105620 2024-09-19T04:37:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.129+0000 7fc878e00640 1 -- 172.21.15.123:0/1633467656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8741a5210 con 0x7fc874105620 2024-09-19T04:37:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.130+0000 7fc86aa00640 1 -- 172.21.15.123:0/1633467656 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fc86403b3f0 con 0x7fc874105620 2024-09-19T04:37:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.130+0000 7fc86aa00640 1 --2- 172.21.15.123:0/1633467656 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc8580392a0 0x7fc85803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.130+0000 7fc86aa00640 1 -- 172.21.15.123:0/1633467656 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc8640706b0 con 0x7fc874105620 2024-09-19T04:37:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.130+0000 7fc878e00640 1 -- 172.21.15.123:0/1633467656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc874105a20 con 0x7fc874105620 2024-09-19T04:37:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.131+0000 7fc871400640 1 --2- 172.21.15.123:0/1633467656 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc8580392a0 0x7fc85803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.131+0000 7fc871400640 1 --2- 172.21.15.123:0/1633467656 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc8580392a0 0x7fc85803b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc8600098c0 tx=0x7fc860006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.135+0000 7fc86aa00640 1 -- 172.21.15.123:0/1633467656 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc8640392f0 con 0x7fc874105620 2024-09-19T04:37:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.394+0000 7fc878e00640 1 -- 172.21.15.123:0/1633467656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fc8741a53e0 con 0x7fc874105620 2024-09-19T04:37:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.397+0000 7fc86aa00640 1 -- 172.21.15.123:0/1633467656 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fc86403b6e0 con 0x7fc874105620 2024-09-19T04:37:22.050 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T04:37:22.050 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.400+0000 7fc878e00640 1 -- 172.21.15.123:0/1633467656 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc8580392a0 msgr2=0x7fc85803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:22.050 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.400+0000 7fc878e00640 1 --2- 172.21.15.123:0/1633467656 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc8580392a0 0x7fc85803b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc8600098c0 tx=0x7fc860006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:22.050 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.400+0000 7fc878e00640 1 -- 172.21.15.123:0/1633467656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc874105620 msgr2=0x7fc8741a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:22.050 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.400+0000 7fc878e00640 1 --2- 172.21.15.123:0/1633467656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc874105620 0x7fc8741a1d90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc8640097f0 tx=0x7fc8640327b0 comp rx=0 tx=0).stop 2024-09-19T04:37:22.050 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.400+0000 7fc878e00640 1 -- 172.21.15.123:0/1633467656 shutdown_connections 2024-09-19T04:37:22.050 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.400+0000 7fc878e00640 1 --2- 172.21.15.123:0/1633467656 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc8580392a0 0x7fc85803b760 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:22.050 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.400+0000 7fc878e00640 1 --2- 172.21.15.123:0/1633467656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc874105620 0x7fc8741a1d90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:22.050 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.400+0000 7fc878e00640 1 -- 172.21.15.123:0/1633467656 >> 172.21.15.123:0/1633467656 conn(0x7fc874101180 msgr2=0x7fc874101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:22.050 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.400+0000 7fc878e00640 1 -- 172.21.15.123:0/1633467656 shutdown_connections 2024-09-19T04:37:22.050 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-09-19T04:37:21.400+0000 7fc878e00640 1 -- 172.21.15.123:0/1633467656 wait complete. 2024-09-19T04:37:22.050 INFO:teuthology.orchestra.run.smithi123.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T04:37:22.050 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:37:22.051 INFO:teuthology.orchestra.run.smithi123.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:37:22.051 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:37:22.051 INFO:teuthology.orchestra.run.smithi123.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T04:37:22.051 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:37:22.051 INFO:teuthology.orchestra.run.smithi123.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T04:37:22.051 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:37:22.051 INFO:teuthology.orchestra.run.smithi123.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T04:37:22.051 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:37:22.051 INFO:teuthology.orchestra.run.smithi123.stdout: ceph telemetry on 2024-09-19T04:37:22.051 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:37:22.051 INFO:teuthology.orchestra.run.smithi123.stdout:For more information see: 2024-09-19T04:37:22.051 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:37:22.051 INFO:teuthology.orchestra.run.smithi123.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T04:37:22.052 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:37:22.052 INFO:teuthology.orchestra.run.smithi123.stdout:Bootstrap complete. 2024-09-19T04:37:22.104 INFO:tasks.cephadm:Fetching config... 2024-09-19T04:37:22.105 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:37:22.105 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T04:37:22.142 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T04:37:22.142 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:37:22.143 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T04:37:22.221 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T04:37:22.221 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:37:22.221 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-19T04:37:22.319 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T04:37:22.319 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:37:22.319 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T04:37:22.394 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T04:37:22.394 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7OSSNG8zcJefpr+R9V/17bxDTyFPelwJTIF/flxiapM648SuBb+xUh2mOfl+Tl1/AxeADsRqNfoe5pI4zhNLoKOKraKBLfa1dWFMlJnRuMZQjRYELW49+e7ZyerHiguQp4odcwZ5ESEtzyOcKErK9B+WDRCVN62rXB/hwS/xfA60j/X8g7pm6HqWi6zG7E9iUk8DkICwbNRbrmWq4lwatZEaoRvuXMDH8rzhuhX/RDEsu3w5W6Up3VnDwSQuFWsTLSpdJASQFofzpLqGxosRTdvIzAARJVV3slgNDQF0a652Q62rQR0Nw1MVQ78VEfB5Xu5HwjogV2ZhXN/ePKOmhLVYURT3Y7cyQlo5c1AQyVCqAVsYk2cXMNB4ueAwowSHQrqW08otQc/aMwvwrzYKYRHpJ5CC+7zfjtoIN7Orq2u4g23uaCja0fEKvhZTtJwpKBQunKCF2grCeRg1dNMY0tB9OTKnFeMvt3xjIqguP1njKlJeIetwjgND+ErtMNcM= ceph-911fa052-7640-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T04:37:22.514 INFO:teuthology.orchestra.run.smithi123.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7OSSNG8zcJefpr+R9V/17bxDTyFPelwJTIF/flxiapM648SuBb+xUh2mOfl+Tl1/AxeADsRqNfoe5pI4zhNLoKOKraKBLfa1dWFMlJnRuMZQjRYELW49+e7ZyerHiguQp4odcwZ5ESEtzyOcKErK9B+WDRCVN62rXB/hwS/xfA60j/X8g7pm6HqWi6zG7E9iUk8DkICwbNRbrmWq4lwatZEaoRvuXMDH8rzhuhX/RDEsu3w5W6Up3VnDwSQuFWsTLSpdJASQFofzpLqGxosRTdvIzAARJVV3slgNDQF0a652Q62rQR0Nw1MVQ78VEfB5Xu5HwjogV2ZhXN/ePKOmhLVYURT3Y7cyQlo5c1AQyVCqAVsYk2cXMNB4ueAwowSHQrqW08otQc/aMwvwrzYKYRHpJ5CC+7zfjtoIN7Orq2u4g23uaCja0fEKvhZTtJwpKBQunKCF2grCeRg1dNMY0tB9OTKnFeMvt3xjIqguP1njKlJeIetwjgND+ErtMNcM= ceph-911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:37:22.539 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7OSSNG8zcJefpr+R9V/17bxDTyFPelwJTIF/flxiapM648SuBb+xUh2mOfl+Tl1/AxeADsRqNfoe5pI4zhNLoKOKraKBLfa1dWFMlJnRuMZQjRYELW49+e7ZyerHiguQp4odcwZ5ESEtzyOcKErK9B+WDRCVN62rXB/hwS/xfA60j/X8g7pm6HqWi6zG7E9iUk8DkICwbNRbrmWq4lwatZEaoRvuXMDH8rzhuhX/RDEsu3w5W6Up3VnDwSQuFWsTLSpdJASQFofzpLqGxosRTdvIzAARJVV3slgNDQF0a652Q62rQR0Nw1MVQ78VEfB5Xu5HwjogV2ZhXN/ePKOmhLVYURT3Y7cyQlo5c1AQyVCqAVsYk2cXMNB4ueAwowSHQrqW08otQc/aMwvwrzYKYRHpJ5CC+7zfjtoIN7Orq2u4g23uaCja0fEKvhZTtJwpKBQunKCF2grCeRg1dNMY0tB9OTKnFeMvt3xjIqguP1njKlJeIetwjgND+ErtMNcM= ceph-911fa052-7640-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T04:37:22.571 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:22 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1633467656' entity='client.admin' 2024-09-19T04:37:22.581 INFO:teuthology.orchestra.run.smithi183.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7OSSNG8zcJefpr+R9V/17bxDTyFPelwJTIF/flxiapM648SuBb+xUh2mOfl+Tl1/AxeADsRqNfoe5pI4zhNLoKOKraKBLfa1dWFMlJnRuMZQjRYELW49+e7ZyerHiguQp4odcwZ5ESEtzyOcKErK9B+WDRCVN62rXB/hwS/xfA60j/X8g7pm6HqWi6zG7E9iUk8DkICwbNRbrmWq4lwatZEaoRvuXMDH8rzhuhX/RDEsu3w5W6Up3VnDwSQuFWsTLSpdJASQFofzpLqGxosRTdvIzAARJVV3slgNDQF0a652Q62rQR0Nw1MVQ78VEfB5Xu5HwjogV2ZhXN/ePKOmhLVYURT3Y7cyQlo5c1AQyVCqAVsYk2cXMNB4ueAwowSHQrqW08otQc/aMwvwrzYKYRHpJ5CC+7zfjtoIN7Orq2u4g23uaCja0fEKvhZTtJwpKBQunKCF2grCeRg1dNMY0tB9OTKnFeMvt3xjIqguP1njKlJeIetwjgND+ErtMNcM= ceph-911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:37:22.593 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7OSSNG8zcJefpr+R9V/17bxDTyFPelwJTIF/flxiapM648SuBb+xUh2mOfl+Tl1/AxeADsRqNfoe5pI4zhNLoKOKraKBLfa1dWFMlJnRuMZQjRYELW49+e7ZyerHiguQp4odcwZ5ESEtzyOcKErK9B+WDRCVN62rXB/hwS/xfA60j/X8g7pm6HqWi6zG7E9iUk8DkICwbNRbrmWq4lwatZEaoRvuXMDH8rzhuhX/RDEsu3w5W6Up3VnDwSQuFWsTLSpdJASQFofzpLqGxosRTdvIzAARJVV3slgNDQF0a652Q62rQR0Nw1MVQ78VEfB5Xu5HwjogV2ZhXN/ePKOmhLVYURT3Y7cyQlo5c1AQyVCqAVsYk2cXMNB4ueAwowSHQrqW08otQc/aMwvwrzYKYRHpJ5CC+7zfjtoIN7Orq2u4g23uaCja0fEKvhZTtJwpKBQunKCF2grCeRg1dNMY0tB9OTKnFeMvt3xjIqguP1njKlJeIetwjgND+ErtMNcM= ceph-911fa052-7640-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T04:37:22.635 INFO:teuthology.orchestra.run.smithi193.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7OSSNG8zcJefpr+R9V/17bxDTyFPelwJTIF/flxiapM648SuBb+xUh2mOfl+Tl1/AxeADsRqNfoe5pI4zhNLoKOKraKBLfa1dWFMlJnRuMZQjRYELW49+e7ZyerHiguQp4odcwZ5ESEtzyOcKErK9B+WDRCVN62rXB/hwS/xfA60j/X8g7pm6HqWi6zG7E9iUk8DkICwbNRbrmWq4lwatZEaoRvuXMDH8rzhuhX/RDEsu3w5W6Up3VnDwSQuFWsTLSpdJASQFofzpLqGxosRTdvIzAARJVV3slgNDQF0a652Q62rQR0Nw1MVQ78VEfB5Xu5HwjogV2ZhXN/ePKOmhLVYURT3Y7cyQlo5c1AQyVCqAVsYk2cXMNB4ueAwowSHQrqW08otQc/aMwvwrzYKYRHpJ5CC+7zfjtoIN7Orq2u4g23uaCja0fEKvhZTtJwpKBQunKCF2grCeRg1dNMY0tB9OTKnFeMvt3xjIqguP1njKlJeIetwjgND+ErtMNcM= ceph-911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:37:22.648 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T04:37:23.027 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:37:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.566+0000 7f0c77e00640 1 -- 172.21.15.123:0/2387151008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c78101420 msgr2=0x7f0c78101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.566+0000 7f0c77e00640 1 --2- 172.21.15.123:0/2387151008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c78101420 0x7f0c78101820 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009880 tx=0x7f0c6402a5d0 comp rx=0 tx=0).stop 2024-09-19T04:37:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.566+0000 7f0c77e00640 1 -- 172.21.15.123:0/2387151008 shutdown_connections 2024-09-19T04:37:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.566+0000 7f0c77e00640 1 --2- 172.21.15.123:0/2387151008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c78101420 0x7f0c78101820 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.566+0000 7f0c77e00640 1 -- 172.21.15.123:0/2387151008 >> 172.21.15.123:0/2387151008 conn(0x7f0c780fcbd0 msgr2=0x7f0c780feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.566+0000 7f0c77e00640 1 -- 172.21.15.123:0/2387151008 shutdown_connections 2024-09-19T04:37:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.566+0000 7f0c77e00640 1 -- 172.21.15.123:0/2387151008 wait complete. 2024-09-19T04:37:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.567+0000 7f0c77e00640 1 Processor -- start 2024-09-19T04:37:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.568+0000 7f0c77e00640 1 -- start start 2024-09-19T04:37:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.568+0000 7f0c77e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c78101420 0x7f0c78075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.568+0000 7f0c77e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0c78107530 con 0x7f0c78101420 2024-09-19T04:37:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.568+0000 7f0c76a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c78101420 0x7f0c78075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.568+0000 7f0c76a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c78101420 0x7f0c78075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53790/0 (socket says 172.21.15.123:53790) 2024-09-19T04:37:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.568+0000 7f0c76a00640 1 -- 172.21.15.123:0/2183620857 learned_addr learned my addr 172.21.15.123:0/2183620857 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.569+0000 7f0c76a00640 1 -- 172.21.15.123:0/2183620857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c780763f0 con 0x7f0c78101420 2024-09-19T04:37:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.569+0000 7f0c76a00640 1 --2- 172.21.15.123:0/2183620857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c78101420 0x7f0c78075eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0c64004f60 tx=0x7f0c640048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.569+0000 7f0c6f400640 1 -- 172.21.15.123:0/2183620857 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c6403f070 con 0x7f0c78101420 2024-09-19T04:37:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.569+0000 7f0c77e00640 1 -- 172.21.15.123:0/2183620857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c780745e0 con 0x7f0c78101420 2024-09-19T04:37:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.569+0000 7f0c6f400640 1 -- 172.21.15.123:0/2183620857 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c64004cc0 con 0x7f0c78101420 2024-09-19T04:37:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.569+0000 7f0c6f400640 1 -- 172.21.15.123:0/2183620857 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c64035030 con 0x7f0c78101420 2024-09-19T04:37:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.570+0000 7f0c77e00640 1 -- 172.21.15.123:0/2183620857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c78074b20 con 0x7f0c78101420 2024-09-19T04:37:23.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.571+0000 7f0c77e00640 1 -- 172.21.15.123:0/2183620857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c78107cb0 con 0x7f0c78101420 2024-09-19T04:37:23.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.571+0000 7f0c6f400640 1 -- 172.21.15.123:0/2183620857 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f0c64042070 con 0x7f0c78101420 2024-09-19T04:37:23.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.572+0000 7f0c6f400640 1 --2- 172.21.15.123:0/2183620857 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0c5c039300 0x7f0c5c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:23.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.572+0000 7f0c6f400640 1 -- 172.21.15.123:0/2183620857 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0c6406d000 con 0x7f0c78101420 2024-09-19T04:37:23.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.572+0000 7f0c76000640 1 --2- 172.21.15.123:0/2183620857 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0c5c039300 0x7f0c5c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:23.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.573+0000 7f0c76000640 1 --2- 172.21.15.123:0/2183620857 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0c5c039300 0x7f0c5c03b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0c680098c0 tx=0x7f0c68006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:23.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.578+0000 7f0c6f400640 1 -- 172.21.15.123:0/2183620857 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0c64040260 con 0x7f0c78101420 2024-09-19T04:37:23.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.760+0000 7f0c77e00640 1 -- 172.21.15.123:0/2183620857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f0c7818ffd0 con 0x7f0c78101420 2024-09-19T04:37:23.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.763+0000 7f0c6f400640 1 -- 172.21.15.123:0/2183620857 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f0c64003dd0 con 0x7f0c78101420 2024-09-19T04:37:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.765+0000 7f0c77e00640 1 -- 172.21.15.123:0/2183620857 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0c5c039300 msgr2=0x7f0c5c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.765+0000 7f0c77e00640 1 --2- 172.21.15.123:0/2183620857 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0c5c039300 0x7f0c5c03b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0c680098c0 tx=0x7f0c68006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.765+0000 7f0c77e00640 1 -- 172.21.15.123:0/2183620857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c78101420 msgr2=0x7f0c78075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.765+0000 7f0c77e00640 1 --2- 172.21.15.123:0/2183620857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c78101420 0x7f0c78075eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0c64004f60 tx=0x7f0c640048f0 comp rx=0 tx=0).stop 2024-09-19T04:37:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.765+0000 7f0c77e00640 1 -- 172.21.15.123:0/2183620857 shutdown_connections 2024-09-19T04:37:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.765+0000 7f0c77e00640 1 --2- 172.21.15.123:0/2183620857 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0c5c039300 0x7f0c5c03b7c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.765+0000 7f0c77e00640 1 --2- 172.21.15.123:0/2183620857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c78101420 0x7f0c78075eb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.765+0000 7f0c77e00640 1 -- 172.21.15.123:0/2183620857 >> 172.21.15.123:0/2183620857 conn(0x7f0c780fcbd0 msgr2=0x7f0c780fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.765+0000 7f0c77e00640 1 -- 172.21.15.123:0/2183620857 shutdown_connections 2024-09-19T04:37:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:23.765+0000 7f0c77e00640 1 -- 172.21.15.123:0/2183620857 wait complete. 2024-09-19T04:37:24.375 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T04:37:24.376 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T04:37:24.942 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:37:25.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:24 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2183620857' entity='client.admin' 2024-09-19T04:37:25.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.608+0000 7fd16f800640 1 -- 172.21.15.123:0/1976968765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd168101620 msgr2=0x7fd168101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:25.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.608+0000 7fd16f800640 1 --2- 172.21.15.123:0/1976968765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd168101620 0x7fd168101a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009880 tx=0x7fd15c02a5d0 comp rx=0 tx=0).stop 2024-09-19T04:37:25.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.608+0000 7fd16f800640 1 -- 172.21.15.123:0/1976968765 shutdown_connections 2024-09-19T04:37:25.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.608+0000 7fd16f800640 1 --2- 172.21.15.123:0/1976968765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd168101620 0x7fd168101a20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:25.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.608+0000 7fd16f800640 1 -- 172.21.15.123:0/1976968765 >> 172.21.15.123:0/1976968765 conn(0x7fd1680fcdf0 msgr2=0x7fd1680ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:25.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.608+0000 7fd16f800640 1 -- 172.21.15.123:0/1976968765 shutdown_connections 2024-09-19T04:37:25.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.608+0000 7fd16f800640 1 -- 172.21.15.123:0/1976968765 wait complete. 2024-09-19T04:37:25.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.609+0000 7fd16f800640 1 Processor -- start 2024-09-19T04:37:25.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.610+0000 7fd16f800640 1 -- start start 2024-09-19T04:37:25.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.610+0000 7fd16f800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd168101620 0x7fd168198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:25.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.610+0000 7fd16f800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd168107730 con 0x7fd168101620 2024-09-19T04:37:25.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.610+0000 7fd16ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd168101620 0x7fd168198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:25.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.610+0000 7fd16ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd168101620 0x7fd168198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53808/0 (socket says 172.21.15.123:53808) 2024-09-19T04:37:25.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.610+0000 7fd16ce00640 1 -- 172.21.15.123:0/720874585 learned_addr learned my addr 172.21.15.123:0/720874585 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:25.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.611+0000 7fd16ce00640 1 -- 172.21.15.123:0/720874585 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1681994b0 con 0x7fd168101620 2024-09-19T04:37:25.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.611+0000 7fd16ce00640 1 --2- 172.21.15.123:0/720874585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd168101620 0x7fd168198f70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd15c004f60 tx=0x7fd15c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:25.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.611+0000 7fd165600640 1 -- 172.21.15.123:0/720874585 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd15c03f070 con 0x7fd168101620 2024-09-19T04:37:25.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.611+0000 7fd16f800640 1 -- 172.21.15.123:0/720874585 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd168199740 con 0x7fd168101620 2024-09-19T04:37:25.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.611+0000 7fd165600640 1 -- 172.21.15.123:0/720874585 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd15c004cc0 con 0x7fd168101620 2024-09-19T04:37:25.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.611+0000 7fd165600640 1 -- 172.21.15.123:0/720874585 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd15c035030 con 0x7fd168101620 2024-09-19T04:37:25.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.611+0000 7fd16f800640 1 -- 172.21.15.123:0/720874585 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd16819c430 con 0x7fd168101620 2024-09-19T04:37:25.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.612+0000 7fd165600640 1 -- 172.21.15.123:0/720874585 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fd15c042070 con 0x7fd168101620 2024-09-19T04:37:25.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.612+0000 7fd15aa00640 1 -- 172.21.15.123:0/720874585 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd144005290 con 0x7fd168101620 2024-09-19T04:37:25.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.613+0000 7fd165600640 1 --2- 172.21.15.123:0/720874585 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd1500392a0 0x7fd15003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:25.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.613+0000 7fd165600640 1 -- 172.21.15.123:0/720874585 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd15c06d4f0 con 0x7fd168101620 2024-09-19T04:37:25.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.613+0000 7fd167e00640 1 --2- 172.21.15.123:0/720874585 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd1500392a0 0x7fd15003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:25.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.614+0000 7fd167e00640 1 --2- 172.21.15.123:0/720874585 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd1500392a0 0x7fd15003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd1540098c0 tx=0x7fd154006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:25.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.618+0000 7fd165600640 1 -- 172.21.15.123:0/720874585 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd15c003dd0 con 0x7fd168101620 2024-09-19T04:37:25.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.804+0000 7fd15aa00640 1 -- 172.21.15.123:0/720874585 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fd144000bc0 con 0x7fd1500392a0 2024-09-19T04:37:25.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.808+0000 7fd165600640 1 -- 172.21.15.123:0/720874585 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd144000bc0 con 0x7fd1500392a0 2024-09-19T04:37:25.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.810+0000 7fd15aa00640 1 -- 172.21.15.123:0/720874585 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd1500392a0 msgr2=0x7fd15003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:25.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.810+0000 7fd15aa00640 1 --2- 172.21.15.123:0/720874585 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd1500392a0 0x7fd15003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd1540098c0 tx=0x7fd154006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:25.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.810+0000 7fd15aa00640 1 -- 172.21.15.123:0/720874585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd168101620 msgr2=0x7fd168198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:25.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.810+0000 7fd15aa00640 1 --2- 172.21.15.123:0/720874585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd168101620 0x7fd168198f70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd15c004f60 tx=0x7fd15c0048f0 comp rx=0 tx=0).stop 2024-09-19T04:37:25.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.810+0000 7fd15aa00640 1 -- 172.21.15.123:0/720874585 shutdown_connections 2024-09-19T04:37:25.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.810+0000 7fd15aa00640 1 --2- 172.21.15.123:0/720874585 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd1500392a0 0x7fd15003b760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:25.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.810+0000 7fd15aa00640 1 --2- 172.21.15.123:0/720874585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd168101620 0x7fd168198f70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:25.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.810+0000 7fd15aa00640 1 -- 172.21.15.123:0/720874585 >> 172.21.15.123:0/720874585 conn(0x7fd1680fcdf0 msgr2=0x7fd1680ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:25.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.811+0000 7fd15aa00640 1 -- 172.21.15.123:0/720874585 shutdown_connections 2024-09-19T04:37:25.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:25.811+0000 7fd15aa00640 1 -- 172.21.15.123:0/720874585 wait complete. 2024-09-19T04:37:26.556 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi183 2024-09-19T04:37:26.556 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-19T04:37:26.556 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.conf 2024-09-19T04:37:26.577 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-19T04:37:26.577 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:37:26.634 INFO:tasks.cephadm:Adding host smithi183 to orchestrator... 2024-09-19T04:37:26.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch host add smithi183 2024-09-19T04:37:27.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:26 smithi123 ceph-mon[30511]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:37:27.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:27.341 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:37:27.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.943+0000 7f00afe00640 1 -- 172.21.15.123:0/1829002753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b0101430 msgr2=0x7f00b0101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:27.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.943+0000 7f00afe00640 1 --2- 172.21.15.123:0/1829002753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b0101430 0x7f00b0101830 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009880 tx=0x7f00a002a5d0 comp rx=0 tx=0).stop 2024-09-19T04:37:27.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.943+0000 7f00afe00640 1 -- 172.21.15.123:0/1829002753 shutdown_connections 2024-09-19T04:37:27.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.943+0000 7f00afe00640 1 --2- 172.21.15.123:0/1829002753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b0101430 0x7f00b0101830 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:27.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.943+0000 7f00afe00640 1 -- 172.21.15.123:0/1829002753 >> 172.21.15.123:0/1829002753 conn(0x7f00b00fcbe0 msgr2=0x7f00b00ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:27.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.943+0000 7f00afe00640 1 -- 172.21.15.123:0/1829002753 shutdown_connections 2024-09-19T04:37:27.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.943+0000 7f00afe00640 1 -- 172.21.15.123:0/1829002753 wait complete. 2024-09-19T04:37:27.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.944+0000 7f00afe00640 1 Processor -- start 2024-09-19T04:37:27.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.944+0000 7f00afe00640 1 -- start start 2024-09-19T04:37:27.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.945+0000 7f00afe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b0101430 0x7f00b0198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:27.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.945+0000 7f00afe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f00b0107540 con 0x7f00b0101430 2024-09-19T04:37:27.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.945+0000 7f00aea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b0101430 0x7f00b0198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:27.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.945+0000 7f00aea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b0101430 0x7f00b0198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53820/0 (socket says 172.21.15.123:53820) 2024-09-19T04:37:27.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.945+0000 7f00aea00640 1 -- 172.21.15.123:0/353280885 learned_addr learned my addr 172.21.15.123:0/353280885 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:27.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.945+0000 7f00aea00640 1 -- 172.21.15.123:0/353280885 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b0199310 con 0x7f00b0101430 2024-09-19T04:37:27.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.946+0000 7f00aea00640 1 --2- 172.21.15.123:0/353280885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b0101430 0x7f00b0198dd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f00a0004f60 tx=0x7f00a00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:27.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.946+0000 7f00a7400640 1 -- 172.21.15.123:0/353280885 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00a003f070 con 0x7f00b0101430 2024-09-19T04:37:27.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.946+0000 7f00a7400640 1 -- 172.21.15.123:0/353280885 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a0004cc0 con 0x7f00b0101430 2024-09-19T04:37:27.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.946+0000 7f00afe00640 1 -- 172.21.15.123:0/353280885 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b01995a0 con 0x7f00b0101430 2024-09-19T04:37:27.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.946+0000 7f00a7400640 1 -- 172.21.15.123:0/353280885 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00a0035030 con 0x7f00b0101430 2024-09-19T04:37:27.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.946+0000 7f00afe00640 1 -- 172.21.15.123:0/353280885 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b019c290 con 0x7f00b0101430 2024-09-19T04:37:27.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.947+0000 7f00a7400640 1 -- 172.21.15.123:0/353280885 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f00a0042070 con 0x7f00b0101430 2024-09-19T04:37:27.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.947+0000 7f00afe00640 1 -- 172.21.15.123:0/353280885 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b0107cc0 con 0x7f00b0101430 2024-09-19T04:37:27.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.948+0000 7f00a7400640 1 --2- 172.21.15.123:0/353280885 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0094039300 0x7f009403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:27.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.948+0000 7f00a7400640 1 -- 172.21.15.123:0/353280885 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f00a006d010 con 0x7f00b0101430 2024-09-19T04:37:27.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.948+0000 7f00ae000640 1 --2- 172.21.15.123:0/353280885 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0094039300 0x7f009403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:27.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.949+0000 7f00ae000640 1 --2- 172.21.15.123:0/353280885 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0094039300 0x7f009403b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f009c0098c0 tx=0x7f009c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:27.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:27.953+0000 7f00a7400640 1 -- 172.21.15.123:0/353280885 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f00a0040260 con 0x7f00b0101430 2024-09-19T04:37:28.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:28.142+0000 7f00afe00640 1 -- 172.21.15.123:0/353280885 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}) -- 0x7f00b002d830 con 0x7f0094039300 2024-09-19T04:37:28.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:28 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:28.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:28 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:28.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:28 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:37:28.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:28 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:37:28.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:28 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:37:28.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:28 smithi123 ceph-mon[30511]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-19T04:37:29.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:29 smithi123 ceph-mon[30511]: Updating smithi123:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:37:29.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:29 smithi123 ceph-mon[30511]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:37:29.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:29 smithi123 ceph-mon[30511]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:37:29.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:29 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:29.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:29 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:29.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:29 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:30.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:30 smithi123 ceph-mon[30511]: Updating smithi123:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:37:30.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:30 smithi123 ceph-mon[30511]: Deploying cephadm binary to smithi183 2024-09-19T04:37:30.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:30 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:37:30.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:30 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:37:30.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:30 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:37:30.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:30 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:31.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:31.341+0000 7f00a7400640 1 -- 172.21.15.123:0/353280885 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f00b002d830 con 0x7f0094039300 2024-09-19T04:37:31.343 INFO:teuthology.orchestra.run.smithi123.stdout:Added host 'smithi183' with addr '172.21.15.183' 2024-09-19T04:37:31.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:31.344+0000 7f00afe00640 1 -- 172.21.15.123:0/353280885 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0094039300 msgr2=0x7f009403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:31.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:31.344+0000 7f00afe00640 1 --2- 172.21.15.123:0/353280885 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0094039300 0x7f009403b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f009c0098c0 tx=0x7f009c006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:31.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:31.344+0000 7f00afe00640 1 -- 172.21.15.123:0/353280885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b0101430 msgr2=0x7f00b0198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:31.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:31.344+0000 7f00afe00640 1 --2- 172.21.15.123:0/353280885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b0101430 0x7f00b0198dd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f00a0004f60 tx=0x7f00a00048f0 comp rx=0 tx=0).stop 2024-09-19T04:37:31.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:31.344+0000 7f00afe00640 1 -- 172.21.15.123:0/353280885 shutdown_connections 2024-09-19T04:37:31.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:31.344+0000 7f00afe00640 1 --2- 172.21.15.123:0/353280885 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0094039300 0x7f009403b7c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:31.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:31.344+0000 7f00afe00640 1 --2- 172.21.15.123:0/353280885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b0101430 0x7f00b0198dd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:31.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:31.344+0000 7f00afe00640 1 -- 172.21.15.123:0/353280885 >> 172.21.15.123:0/353280885 conn(0x7f00b00fcbe0 msgr2=0x7f00b00fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:31.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:31.345+0000 7f00afe00640 1 -- 172.21.15.123:0/353280885 shutdown_connections 2024-09-19T04:37:31.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:31.345+0000 7f00afe00640 1 -- 172.21.15.123:0/353280885 wait complete. 2024-09-19T04:37:32.000 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T04:37:32.370 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:37:32.644 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:32 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:32.644 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:32 smithi123 ceph-mon[30511]: Added host smithi183 2024-09-19T04:37:32.644 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:32 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:37:32.644 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:32 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:32.644 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:32 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:32.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.900+0000 7ff6c3e00640 1 -- 172.21.15.123:0/3734834944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6c4074420 msgr2=0x7ff6c4074800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:32.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.900+0000 7ff6c3e00640 1 --2- 172.21.15.123:0/3734834944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6c4074420 0x7ff6c4074800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009880 tx=0x7ff6b402a670 comp rx=0 tx=0).stop 2024-09-19T04:37:32.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.900+0000 7ff6c3e00640 1 -- 172.21.15.123:0/3734834944 shutdown_connections 2024-09-19T04:37:32.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.900+0000 7ff6c3e00640 1 --2- 172.21.15.123:0/3734834944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6c4074420 0x7ff6c4074800 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:32.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.900+0000 7ff6c3e00640 1 -- 172.21.15.123:0/3734834944 >> 172.21.15.123:0/3734834944 conn(0x7ff6c4078130 msgr2=0x7ff6c4078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:32.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.900+0000 7ff6c3e00640 1 -- 172.21.15.123:0/3734834944 shutdown_connections 2024-09-19T04:37:32.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.900+0000 7ff6c3e00640 1 -- 172.21.15.123:0/3734834944 wait complete. 2024-09-19T04:37:32.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.901+0000 7ff6c3e00640 1 Processor -- start 2024-09-19T04:37:32.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.901+0000 7ff6c3e00640 1 -- start start 2024-09-19T04:37:32.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.902+0000 7ff6c3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6c4074420 0x7ff6c4199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:32.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.902+0000 7ff6c3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c410dc50 con 0x7ff6c4074420 2024-09-19T04:37:32.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.902+0000 7ff6c2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6c4074420 0x7ff6c4199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:32.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.902+0000 7ff6c2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6c4074420 0x7ff6c4199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58056/0 (socket says 172.21.15.123:58056) 2024-09-19T04:37:32.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.902+0000 7ff6c2a00640 1 -- 172.21.15.123:0/4165201122 learned_addr learned my addr 172.21.15.123:0/4165201122 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:32.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.903+0000 7ff6c2a00640 1 -- 172.21.15.123:0/4165201122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c41996a0 con 0x7ff6c4074420 2024-09-19T04:37:32.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.903+0000 7ff6c2a00640 1 --2- 172.21.15.123:0/4165201122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6c4074420 0x7ff6c4199160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009230 tx=0x7ff6b4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:32.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.903+0000 7ff6bb400640 1 -- 172.21.15.123:0/4165201122 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6b403f070 con 0x7ff6c4074420 2024-09-19T04:37:32.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.903+0000 7ff6bb400640 1 -- 172.21.15.123:0/4165201122 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6b403bc80 con 0x7ff6c4074420 2024-09-19T04:37:32.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.903+0000 7ff6bb400640 1 -- 172.21.15.123:0/4165201122 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6b4035030 con 0x7ff6c4074420 2024-09-19T04:37:32.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.903+0000 7ff6c3e00640 1 -- 172.21.15.123:0/4165201122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c4199990 con 0x7ff6c4074420 2024-09-19T04:37:32.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.903+0000 7ff6c3e00640 1 -- 172.21.15.123:0/4165201122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c419da30 con 0x7ff6c4074420 2024-09-19T04:37:32.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.904+0000 7ff6bb400640 1 -- 172.21.15.123:0/4165201122 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7ff6b4042070 con 0x7ff6c4074420 2024-09-19T04:37:32.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.904+0000 7ff6c3e00640 1 -- 172.21.15.123:0/4165201122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c4075930 con 0x7ff6c4074420 2024-09-19T04:37:32.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.905+0000 7ff6bb400640 1 --2- 172.21.15.123:0/4165201122 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff6a8039300 0x7ff6a803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:32.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.905+0000 7ff6bb400640 1 -- 172.21.15.123:0/4165201122 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff6b406d010 con 0x7ff6c4074420 2024-09-19T04:37:32.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.905+0000 7ff6c2000640 1 --2- 172.21.15.123:0/4165201122 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff6a8039300 0x7ff6a803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:32.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.906+0000 7ff6c2000640 1 --2- 172.21.15.123:0/4165201122 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff6a8039300 0x7ff6a803b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff6b00098c0 tx=0x7ff6b0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:32.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:32.910+0000 7ff6bb400640 1 -- 172.21.15.123:0/4165201122 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff6b40372d0 con 0x7ff6c4074420 2024-09-19T04:37:33.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:33.126+0000 7ff6c3e00640 1 -- 172.21.15.123:0/4165201122 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff6c419a4c0 con 0x7ff6a8039300 2024-09-19T04:37:33.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:33.127+0000 7ff6bb400640 1 -- 172.21.15.123:0/4165201122 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7ff6c419a4c0 con 0x7ff6a8039300 2024-09-19T04:37:33.128 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:37:33.129 INFO:teuthology.orchestra.run.smithi123.stdout:[{"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}] 2024-09-19T04:37:33.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:33.130+0000 7ff6c3e00640 1 -- 172.21.15.123:0/4165201122 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff6a8039300 msgr2=0x7ff6a803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:33.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:33.130+0000 7ff6c3e00640 1 --2- 172.21.15.123:0/4165201122 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff6a8039300 0x7ff6a803b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff6b00098c0 tx=0x7ff6b0006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:33.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:33.130+0000 7ff6c3e00640 1 -- 172.21.15.123:0/4165201122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6c4074420 msgr2=0x7ff6c4199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:33.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:33.130+0000 7ff6c3e00640 1 --2- 172.21.15.123:0/4165201122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6c4074420 0x7ff6c4199160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009230 tx=0x7ff6b4003d40 comp rx=0 tx=0).stop 2024-09-19T04:37:33.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:33.130+0000 7ff6c3e00640 1 -- 172.21.15.123:0/4165201122 shutdown_connections 2024-09-19T04:37:33.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:33.130+0000 7ff6c3e00640 1 --2- 172.21.15.123:0/4165201122 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff6a8039300 0x7ff6a803b7c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:33.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:33.130+0000 7ff6c3e00640 1 --2- 172.21.15.123:0/4165201122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6c4074420 0x7ff6c4199160 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:33.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:33.130+0000 7ff6c3e00640 1 -- 172.21.15.123:0/4165201122 >> 172.21.15.123:0/4165201122 conn(0x7ff6c4078130 msgr2=0x7ff6c4107a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:33.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:33.130+0000 7ff6c3e00640 1 -- 172.21.15.123:0/4165201122 shutdown_connections 2024-09-19T04:37:33.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:33.130+0000 7ff6c3e00640 1 -- 172.21.15.123:0/4165201122 wait complete. 2024-09-19T04:37:33.714 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:33.714 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:33 smithi123 ceph-mon[30511]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:37:33.714 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:33 smithi123 ceph-mon[30511]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:33.716 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi193 2024-09-19T04:37:33.716 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-19T04:37:33.716 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.conf 2024-09-19T04:37:33.735 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-19T04:37:33.735 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:37:33.794 INFO:tasks.cephadm:Adding host smithi193 to orchestrator... 2024-09-19T04:37:33.794 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch host add smithi193 2024-09-19T04:37:34.149 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:37:34.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.622+0000 7fe03c800640 1 -- 172.21.15.123:0/1484405736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe034107660 msgr2=0x7fe034107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:34.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.622+0000 7fe03c800640 1 --2- 172.21.15.123:0/1484405736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe034107660 0x7fe034107a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe028009880 tx=0x7fe02802a5d0 comp rx=0 tx=0).stop 2024-09-19T04:37:34.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.622+0000 7fe03c800640 1 -- 172.21.15.123:0/1484405736 shutdown_connections 2024-09-19T04:37:34.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.622+0000 7fe03c800640 1 --2- 172.21.15.123:0/1484405736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe034107660 0x7fe034107a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:34.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.622+0000 7fe03c800640 1 -- 172.21.15.123:0/1484405736 >> 172.21.15.123:0/1484405736 conn(0x7fe0340fd2b0 msgr2=0x7fe0340ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:34.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.622+0000 7fe03c800640 1 -- 172.21.15.123:0/1484405736 shutdown_connections 2024-09-19T04:37:34.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.622+0000 7fe03c800640 1 -- 172.21.15.123:0/1484405736 wait complete. 2024-09-19T04:37:34.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.623+0000 7fe03c800640 1 Processor -- start 2024-09-19T04:37:34.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.623+0000 7fe03c800640 1 -- start start 2024-09-19T04:37:34.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.624+0000 7fe03c800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe034107660 0x7fe0341970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:34.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.624+0000 7fe03c800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe03410bc70 con 0x7fe034107660 2024-09-19T04:37:34.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.624+0000 7fe039e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe034107660 0x7fe0341970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:34.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.624+0000 7fe039e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe034107660 0x7fe0341970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58064/0 (socket says 172.21.15.123:58064) 2024-09-19T04:37:34.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.624+0000 7fe039e00640 1 -- 172.21.15.123:0/4047584705 learned_addr learned my addr 172.21.15.123:0/4047584705 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:34.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.624+0000 7fe039e00640 1 -- 172.21.15.123:0/4047584705 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0341975e0 con 0x7fe034107660 2024-09-19T04:37:34.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.625+0000 7fe039e00640 1 --2- 172.21.15.123:0/4047584705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe034107660 0x7fe0341970a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe028009230 tx=0x7fe028003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:34.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.625+0000 7fe02ea00640 1 -- 172.21.15.123:0/4047584705 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe02803f070 con 0x7fe034107660 2024-09-19T04:37:34.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.625+0000 7fe02ea00640 1 -- 172.21.15.123:0/4047584705 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe02803bd40 con 0x7fe034107660 2024-09-19T04:37:34.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.625+0000 7fe03c800640 1 -- 172.21.15.123:0/4047584705 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0341978d0 con 0x7fe034107660 2024-09-19T04:37:34.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.625+0000 7fe02ea00640 1 -- 172.21.15.123:0/4047584705 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe028035030 con 0x7fe034107660 2024-09-19T04:37:34.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.625+0000 7fe03c800640 1 -- 172.21.15.123:0/4047584705 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe03419b970 con 0x7fe034107660 2024-09-19T04:37:34.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.626+0000 7fe03c800640 1 -- 172.21.15.123:0/4047584705 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe034102d10 con 0x7fe034107660 2024-09-19T04:37:34.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.626+0000 7fe02ea00640 1 -- 172.21.15.123:0/4047584705 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fe028042070 con 0x7fe034107660 2024-09-19T04:37:34.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.627+0000 7fe02ea00640 1 --2- 172.21.15.123:0/4047584705 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe01c039300 0x7fe01c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:34.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.627+0000 7fe02ea00640 1 -- 172.21.15.123:0/4047584705 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe02806cea0 con 0x7fe034107660 2024-09-19T04:37:34.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.627+0000 7fe039400640 1 --2- 172.21.15.123:0/4047584705 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe01c039300 0x7fe01c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:34.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.628+0000 7fe039400640 1 --2- 172.21.15.123:0/4047584705 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe01c039300 0x7fe01c03b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe0240098c0 tx=0x7fe024006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:34.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.632+0000 7fe02ea00640 1 -- 172.21.15.123:0/4047584705 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe028030d60 con 0x7fe034107660 2024-09-19T04:37:34.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:34.810+0000 7fe03c800640 1 -- 172.21.15.123:0/4047584705 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}) -- 0x7fe034198400 con 0x7fe01c039300 2024-09-19T04:37:36.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:36 smithi123 ceph-mon[30511]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:37:36.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:36 smithi123 ceph-mon[30511]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:36.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:36 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:36.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:36 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:36.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:36 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:36.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:36 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:36.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:36 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:37:36.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:36 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:37:36.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:36 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:37:36.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:36 smithi123 ceph-mon[30511]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-19T04:37:36.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:36 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:36.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:36 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:36.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:36 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:37.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:37 smithi123 ceph-mon[30511]: Updating smithi183:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:37:37.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:37 smithi123 ceph-mon[30511]: Deploying cephadm binary to smithi193 2024-09-19T04:37:37.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:37 smithi123 ceph-mon[30511]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:37:37.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:37 smithi123 ceph-mon[30511]: Updating smithi183:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:37:38.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:38.094+0000 7fe02ea00640 1 -- 172.21.15.123:0/4047584705 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fe034198400 con 0x7fe01c039300 2024-09-19T04:37:38.095 INFO:teuthology.orchestra.run.smithi123.stdout:Added host 'smithi193' with addr '172.21.15.193' 2024-09-19T04:37:38.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:38.096+0000 7fe03c800640 1 -- 172.21.15.123:0/4047584705 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe01c039300 msgr2=0x7fe01c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:38.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:38.096+0000 7fe03c800640 1 --2- 172.21.15.123:0/4047584705 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe01c039300 0x7fe01c03b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe0240098c0 tx=0x7fe024006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:38.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:38.096+0000 7fe03c800640 1 -- 172.21.15.123:0/4047584705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe034107660 msgr2=0x7fe0341970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:38.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:38.096+0000 7fe03c800640 1 --2- 172.21.15.123:0/4047584705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe034107660 0x7fe0341970a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe028009230 tx=0x7fe028003d80 comp rx=0 tx=0).stop 2024-09-19T04:37:38.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:38.096+0000 7fe03c800640 1 -- 172.21.15.123:0/4047584705 shutdown_connections 2024-09-19T04:37:38.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:38.096+0000 7fe03c800640 1 --2- 172.21.15.123:0/4047584705 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe01c039300 0x7fe01c03b7c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:38.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:38.096+0000 7fe03c800640 1 --2- 172.21.15.123:0/4047584705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe034107660 0x7fe0341970a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:38.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:38.096+0000 7fe03c800640 1 -- 172.21.15.123:0/4047584705 >> 172.21.15.123:0/4047584705 conn(0x7fe0340fd2b0 msgr2=0x7fe0340feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:38.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:38.097+0000 7fe03c800640 1 -- 172.21.15.123:0/4047584705 shutdown_connections 2024-09-19T04:37:38.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:38.097+0000 7fe03c800640 1 -- 172.21.15.123:0/4047584705 wait complete. 2024-09-19T04:37:38.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:38 smithi123 ceph-mon[30511]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:38.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:38 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:38.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:38 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:37:38.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:38 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:38.755 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T04:37:39.111 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:37:39.549 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:39 smithi123 ceph-mon[30511]: Added host smithi193 2024-09-19T04:37:39.549 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:39 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:39.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.656+0000 7fc049e00640 1 -- 172.21.15.123:0/98440347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc044109670 msgr2=0x7fc044109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:39.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.656+0000 7fc049e00640 1 --2- 172.21.15.123:0/98440347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc044109670 0x7fc044109a50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc038009880 tx=0x7fc03802a5d0 comp rx=0 tx=0).stop 2024-09-19T04:37:39.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.656+0000 7fc049e00640 1 -- 172.21.15.123:0/98440347 shutdown_connections 2024-09-19T04:37:39.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.656+0000 7fc049e00640 1 --2- 172.21.15.123:0/98440347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc044109670 0x7fc044109a50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:39.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.656+0000 7fc049e00640 1 -- 172.21.15.123:0/98440347 >> 172.21.15.123:0/98440347 conn(0x7fc0440ff280 msgr2=0x7fc0441016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:39.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.656+0000 7fc049e00640 1 -- 172.21.15.123:0/98440347 shutdown_connections 2024-09-19T04:37:39.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.656+0000 7fc049e00640 1 -- 172.21.15.123:0/98440347 wait complete. 2024-09-19T04:37:39.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.657+0000 7fc049e00640 1 Processor -- start 2024-09-19T04:37:39.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.657+0000 7fc049e00640 1 -- start start 2024-09-19T04:37:39.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.657+0000 7fc049e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc044109670 0x7fc0441990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:39.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.657+0000 7fc049e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc04410dca0 con 0x7fc044109670 2024-09-19T04:37:39.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.658+0000 7fc048a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc044109670 0x7fc0441990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:39.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.658+0000 7fc048a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc044109670 0x7fc0441990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58084/0 (socket says 172.21.15.123:58084) 2024-09-19T04:37:39.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.658+0000 7fc048a00640 1 -- 172.21.15.123:0/563379452 learned_addr learned my addr 172.21.15.123:0/563379452 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:39.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.658+0000 7fc048a00640 1 -- 172.21.15.123:0/563379452 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc044199630 con 0x7fc044109670 2024-09-19T04:37:39.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.658+0000 7fc048a00640 1 --2- 172.21.15.123:0/563379452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc044109670 0x7fc0441990f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc038009230 tx=0x7fc038003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:39.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.658+0000 7fc041600640 1 -- 172.21.15.123:0/563379452 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc03803f070 con 0x7fc044109670 2024-09-19T04:37:39.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.658+0000 7fc041600640 1 -- 172.21.15.123:0/563379452 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc03803bd40 con 0x7fc044109670 2024-09-19T04:37:39.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.658+0000 7fc041600640 1 -- 172.21.15.123:0/563379452 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc038035030 con 0x7fc044109670 2024-09-19T04:37:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.658+0000 7fc049e00640 1 -- 172.21.15.123:0/563379452 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0441998c0 con 0x7fc044109670 2024-09-19T04:37:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.659+0000 7fc049e00640 1 -- 172.21.15.123:0/563379452 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04419d9c0 con 0x7fc044109670 2024-09-19T04:37:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.660+0000 7fc041600640 1 -- 172.21.15.123:0/563379452 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fc038042070 con 0x7fc044109670 2024-09-19T04:37:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.660+0000 7fc049e00640 1 -- 172.21.15.123:0/563379452 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc044104ce0 con 0x7fc044109670 2024-09-19T04:37:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.660+0000 7fc041600640 1 --2- 172.21.15.123:0/563379452 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc02c039300 0x7fc02c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.660+0000 7fc041600640 1 -- 172.21.15.123:0/563379452 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc03806cfc0 con 0x7fc044109670 2024-09-19T04:37:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.660+0000 7fc043e00640 1 --2- 172.21.15.123:0/563379452 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc02c039300 0x7fc02c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.661+0000 7fc043e00640 1 --2- 172.21.15.123:0/563379452 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc02c039300 0x7fc02c03b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc0300098c0 tx=0x7fc030006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:39.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.666+0000 7fc041600640 1 -- 172.21.15.123:0/563379452 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc0380412c0 con 0x7fc044109670 2024-09-19T04:37:39.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.843+0000 7fc049e00640 1 -- 172.21.15.123:0/563379452 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc04419a450 con 0x7fc02c039300 2024-09-19T04:37:39.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.844+0000 7fc041600640 1 -- 172.21.15.123:0/563379452 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7fc04419a450 con 0x7fc02c039300 2024-09-19T04:37:39.846 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:37:39.846 INFO:teuthology.orchestra.run.smithi123.stdout:[{"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}, {"addr": "172.21.15.193", "hostname": "smithi193", "labels": [], "status": ""}] 2024-09-19T04:37:39.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.846+0000 7fc049e00640 1 -- 172.21.15.123:0/563379452 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc02c039300 msgr2=0x7fc02c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:39.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.846+0000 7fc049e00640 1 --2- 172.21.15.123:0/563379452 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc02c039300 0x7fc02c03b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc0300098c0 tx=0x7fc030006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:39.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.846+0000 7fc049e00640 1 -- 172.21.15.123:0/563379452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc044109670 msgr2=0x7fc0441990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:39.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.846+0000 7fc049e00640 1 --2- 172.21.15.123:0/563379452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc044109670 0x7fc0441990f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc038009230 tx=0x7fc038003d80 comp rx=0 tx=0).stop 2024-09-19T04:37:39.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.847+0000 7fc049e00640 1 -- 172.21.15.123:0/563379452 shutdown_connections 2024-09-19T04:37:39.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.847+0000 7fc049e00640 1 --2- 172.21.15.123:0/563379452 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc02c039300 0x7fc02c03b7c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:39.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.847+0000 7fc049e00640 1 --2- 172.21.15.123:0/563379452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc044109670 0x7fc0441990f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:39.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.847+0000 7fc049e00640 1 -- 172.21.15.123:0/563379452 >> 172.21.15.123:0/563379452 conn(0x7fc0440ff280 msgr2=0x7fc044100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:39.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.847+0000 7fc049e00640 1 -- 172.21.15.123:0/563379452 shutdown_connections 2024-09-19T04:37:39.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:39.847+0000 7fc049e00640 1 -- 172.21.15.123:0/563379452 wait complete. 2024-09-19T04:37:40.485 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T04:37:40.485 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T04:37:40.838 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:37:40.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:40 smithi123 ceph-mon[30511]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:40.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:40 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:41.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.302+0000 7fbab0e00640 1 -- 172.21.15.123:0/3060934564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac109650 msgr2=0x7fbaac109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:41.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.302+0000 7fbab0e00640 1 --2- 172.21.15.123:0/3060934564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac109650 0x7fbaac109a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009880 tx=0x7fba9c02a5d0 comp rx=0 tx=0).stop 2024-09-19T04:37:41.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.302+0000 7fbab0e00640 1 -- 172.21.15.123:0/3060934564 shutdown_connections 2024-09-19T04:37:41.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.302+0000 7fbab0e00640 1 --2- 172.21.15.123:0/3060934564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac109650 0x7fbaac109a30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:41.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.302+0000 7fbab0e00640 1 -- 172.21.15.123:0/3060934564 >> 172.21.15.123:0/3060934564 conn(0x7fbaac0ff260 msgr2=0x7fbaac101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:41.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.302+0000 7fbab0e00640 1 -- 172.21.15.123:0/3060934564 shutdown_connections 2024-09-19T04:37:41.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.302+0000 7fbab0e00640 1 -- 172.21.15.123:0/3060934564 wait complete. 2024-09-19T04:37:41.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.303+0000 7fbab0e00640 1 Processor -- start 2024-09-19T04:37:41.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.303+0000 7fbab0e00640 1 -- start start 2024-09-19T04:37:41.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.303+0000 7fbab0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac109650 0x7fbaac199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:41.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.303+0000 7fbab0e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac10dc60 con 0x7fbaac109650 2024-09-19T04:37:41.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.304+0000 7fbaab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac109650 0x7fbaac199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:41.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.304+0000 7fbaab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac109650 0x7fbaac199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58100/0 (socket says 172.21.15.123:58100) 2024-09-19T04:37:41.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.304+0000 7fbaab400640 1 -- 172.21.15.123:0/391524049 learned_addr learned my addr 172.21.15.123:0/391524049 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:37:41.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.304+0000 7fbaab400640 1 -- 172.21.15.123:0/391524049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaac199640 con 0x7fbaac109650 2024-09-19T04:37:41.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.304+0000 7fbaab400640 1 --2- 172.21.15.123:0/391524049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac109650 0x7fbaac199100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009230 tx=0x7fba9c003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:41.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.305+0000 7fbaa3e00640 1 -- 172.21.15.123:0/391524049 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba9c03f070 con 0x7fbaac109650 2024-09-19T04:37:41.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.305+0000 7fbaa3e00640 1 -- 172.21.15.123:0/391524049 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba9c03bd40 con 0x7fbaac109650 2024-09-19T04:37:41.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.305+0000 7fbaa3e00640 1 -- 172.21.15.123:0/391524049 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba9c035030 con 0x7fbaac109650 2024-09-19T04:37:41.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.305+0000 7fbab0e00640 1 -- 172.21.15.123:0/391524049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaac199930 con 0x7fbaac109650 2024-09-19T04:37:41.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.305+0000 7fbab0e00640 1 -- 172.21.15.123:0/391524049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaac19d9d0 con 0x7fbaac109650 2024-09-19T04:37:41.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.306+0000 7fbaa3e00640 1 -- 172.21.15.123:0/391524049 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fba9c042070 con 0x7fbaac109650 2024-09-19T04:37:41.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.306+0000 7fbab0e00640 1 -- 172.21.15.123:0/391524049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaac104d00 con 0x7fbaac109650 2024-09-19T04:37:41.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.306+0000 7fbaa3e00640 1 --2- 172.21.15.123:0/391524049 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fba90039300 0x7fba9003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:41.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.306+0000 7fbaa3e00640 1 -- 172.21.15.123:0/391524049 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fba9c06cf60 con 0x7fbaac109650 2024-09-19T04:37:41.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.307+0000 7fbaaaa00640 1 --2- 172.21.15.123:0/391524049 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fba90039300 0x7fba9003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:41.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.307+0000 7fbaaaa00640 1 --2- 172.21.15.123:0/391524049 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fba90039300 0x7fba9003b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fba980098c0 tx=0x7fba98006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:41.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.311+0000 7fbaa3e00640 1 -- 172.21.15.123:0/391524049 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fba9c030d60 con 0x7fbaac109650 2024-09-19T04:37:41.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:41.489+0000 7fbab0e00640 1 -- 172.21.15.123:0/391524049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fbaac19a460 con 0x7fbaac109650 2024-09-19T04:37:41.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:41 smithi123 ceph-mon[30511]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:37:42.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:42.451+0000 7fbaa3e00640 1 -- 172.21.15.123:0/391524049 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fba9c0412c0 con 0x7fbaac109650 2024-09-19T04:37:42.453 INFO:teuthology.orchestra.run.smithi123.stderr:adjusted tunables profile to default 2024-09-19T04:37:42.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:42.454+0000 7fbab0e00640 1 -- 172.21.15.123:0/391524049 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fba90039300 msgr2=0x7fba9003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:42.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:42.454+0000 7fbab0e00640 1 --2- 172.21.15.123:0/391524049 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fba90039300 0x7fba9003b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fba980098c0 tx=0x7fba98006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:42.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:42.454+0000 7fbab0e00640 1 -- 172.21.15.123:0/391524049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac109650 msgr2=0x7fbaac199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:42.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:42.454+0000 7fbab0e00640 1 --2- 172.21.15.123:0/391524049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac109650 0x7fbaac199100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009230 tx=0x7fba9c003d80 comp rx=0 tx=0).stop 2024-09-19T04:37:42.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:42.454+0000 7fbab0e00640 1 -- 172.21.15.123:0/391524049 shutdown_connections 2024-09-19T04:37:42.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:42.454+0000 7fbab0e00640 1 --2- 172.21.15.123:0/391524049 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fba90039300 0x7fba9003b7c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:42.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:42.454+0000 7fbab0e00640 1 --2- 172.21.15.123:0/391524049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac109650 0x7fbaac199100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:42.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:42.454+0000 7fbab0e00640 1 -- 172.21.15.123:0/391524049 >> 172.21.15.123:0/391524049 conn(0x7fbaac0ff260 msgr2=0x7fbaac100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:42.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:42.454+0000 7fbab0e00640 1 -- 172.21.15.123:0/391524049 shutdown_connections 2024-09-19T04:37:42.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:37:42.454+0000 7fbab0e00640 1 -- 172.21.15.123:0/391524049 wait complete. 2024-09-19T04:37:42.762 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:42 smithi123 ceph-mon[30511]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:42.762 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:42 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/391524049' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T04:37:42.762 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:42.763 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:42.763 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:42.763 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:42.763 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:37:42.763 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:37:42.763 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:37:43.120 INFO:tasks.cephadm:Adding mon.a on smithi123 2024-09-19T04:37:43.120 INFO:tasks.cephadm:Adding mon.b on smithi183 2024-09-19T04:37:43.120 INFO:tasks.cephadm:Adding mon.c on smithi193 2024-09-19T04:37:43.120 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi123:172.21.15.123=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c' 2024-09-19T04:37:43.355 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:37:43.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.760+0000 7fd7bcfe7640 1 -- 172.21.15.193:0/3078584008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7b8100990 msgr2=0x7fd7b8100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:43.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.760+0000 7fd7bcfe7640 1 --2- 172.21.15.193:0/3078584008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7b8100990 0x7fd7b8100d70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009880 tx=0x7fd7a802a5d0 comp rx=0 tx=0).stop 2024-09-19T04:37:43.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.760+0000 7fd7bcfe7640 1 -- 172.21.15.193:0/3078584008 shutdown_connections 2024-09-19T04:37:43.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.760+0000 7fd7bcfe7640 1 --2- 172.21.15.193:0/3078584008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7b8100990 0x7fd7b8100d70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:43.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.760+0000 7fd7bcfe7640 1 -- 172.21.15.193:0/3078584008 >> 172.21.15.193:0/3078584008 conn(0x7fd7b8076250 msgr2=0x7fd7b8076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:43.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.760+0000 7fd7bcfe7640 1 -- 172.21.15.193:0/3078584008 shutdown_connections 2024-09-19T04:37:43.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.760+0000 7fd7bcfe7640 1 -- 172.21.15.193:0/3078584008 wait complete. 2024-09-19T04:37:43.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.760+0000 7fd7bcfe7640 1 Processor -- start 2024-09-19T04:37:43.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.760+0000 7fd7bcfe7640 1 -- start start 2024-09-19T04:37:43.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.761+0000 7fd7bcfe7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7b8100990 0x7fd7b8199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:43.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.761+0000 7fd7bcfe7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b8109a80 con 0x7fd7b8100990 2024-09-19T04:37:43.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.761+0000 7fd7b659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7b8100990 0x7fd7b8199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:43.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.761+0000 7fd7b659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7b8100990 0x7fd7b8199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.193:60496/0 (socket says 172.21.15.193:60496) 2024-09-19T04:37:43.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.761+0000 7fd7b659c640 1 -- 172.21.15.193:0/2533239567 learned_addr learned my addr 172.21.15.193:0/2533239567 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-19T04:37:43.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.761+0000 7fd7b659c640 1 -- 172.21.15.193:0/2533239567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b81998c0 con 0x7fd7b8100990 2024-09-19T04:37:43.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.762+0000 7fd7b659c640 1 --2- 172.21.15.193:0/2533239567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7b8100990 0x7fd7b8199380 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009230 tx=0x7fd7a8003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:43.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.762+0000 7fd7a77fe640 1 -- 172.21.15.193:0/2533239567 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7a803f070 con 0x7fd7b8100990 2024-09-19T04:37:43.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.762+0000 7fd7a77fe640 1 -- 172.21.15.193:0/2533239567 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7a803bd40 con 0x7fd7b8100990 2024-09-19T04:37:43.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.762+0000 7fd7bcfe7640 1 -- 172.21.15.193:0/2533239567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b8199bb0 con 0x7fd7b8100990 2024-09-19T04:37:43.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.762+0000 7fd7a77fe640 1 -- 172.21.15.193:0/2533239567 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7a8035030 con 0x7fd7b8100990 2024-09-19T04:37:43.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.762+0000 7fd7bcfe7640 1 -- 172.21.15.193:0/2533239567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b819dc50 con 0x7fd7b8100990 2024-09-19T04:37:43.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.763+0000 7fd7a77fe640 1 -- 172.21.15.193:0/2533239567 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fd7a8042070 con 0x7fd7b8100990 2024-09-19T04:37:43.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.763+0000 7fd7a77fe640 1 --2- 172.21.15.193:0/2533239567 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd7a00392a0 0x7fd7a003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:43.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.763+0000 7fd7a77fe640 1 -- 172.21.15.193:0/2533239567 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd7a80314c0 con 0x7fd7b8100990 2024-09-19T04:37:43.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.763+0000 7fd7bcfe7640 1 -- 172.21.15.193:0/2533239567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b8101c40 con 0x7fd7b8100990 2024-09-19T04:37:43.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.763+0000 7fd7b5d9b640 1 --2- 172.21.15.193:0/2533239567 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd7a00392a0 0x7fd7a003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:43.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.764+0000 7fd7b5d9b640 1 --2- 172.21.15.193:0/2533239567 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd7a00392a0 0x7fd7a003b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0098c0 tx=0x7fd7ac006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:43.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.768+0000 7fd7a77fe640 1 -- 172.21.15.193:0/2533239567 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd7a8041e50 con 0x7fd7b8100990 2024-09-19T04:37:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:43 smithi123 ceph-mon[30511]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-19T04:37:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:43 smithi123 ceph-mon[30511]: Updating smithi193:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:37:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:43 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/391524049' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T04:37:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:43 smithi123 ceph-mon[30511]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:37:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:43.955 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.953+0000 7fd7bcfe7640 1 -- 172.21.15.193:0/2533239567 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi123:172.21.15.123=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}) -- 0x7fd7b8100df0 con 0x7fd7a00392a0 2024-09-19T04:37:43.961 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.960+0000 7fd7a77fe640 1 -- 172.21.15.193:0/2533239567 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd7b8100df0 con 0x7fd7a00392a0 2024-09-19T04:37:43.961 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mon update... 2024-09-19T04:37:43.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.962+0000 7fd7bcfe7640 1 -- 172.21.15.193:0/2533239567 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd7a00392a0 msgr2=0x7fd7a003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:43.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.962+0000 7fd7bcfe7640 1 --2- 172.21.15.193:0/2533239567 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd7a00392a0 0x7fd7a003b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0098c0 tx=0x7fd7ac006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:43.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.962+0000 7fd7bcfe7640 1 -- 172.21.15.193:0/2533239567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7b8100990 msgr2=0x7fd7b8199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:43.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.962+0000 7fd7bcfe7640 1 --2- 172.21.15.193:0/2533239567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7b8100990 0x7fd7b8199380 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009230 tx=0x7fd7a8003d80 comp rx=0 tx=0).stop 2024-09-19T04:37:43.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.962+0000 7fd7bcfe7640 1 -- 172.21.15.193:0/2533239567 shutdown_connections 2024-09-19T04:37:43.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.962+0000 7fd7bcfe7640 1 --2- 172.21.15.193:0/2533239567 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd7a00392a0 0x7fd7a003b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:43.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.962+0000 7fd7bcfe7640 1 --2- 172.21.15.193:0/2533239567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7b8100990 0x7fd7b8199380 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:43.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.962+0000 7fd7bcfe7640 1 -- 172.21.15.193:0/2533239567 >> 172.21.15.193:0/2533239567 conn(0x7fd7b8076250 msgr2=0x7fd7b81020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:43.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.962+0000 7fd7bcfe7640 1 -- 172.21.15.193:0/2533239567 shutdown_connections 2024-09-19T04:37:43.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:43.962+0000 7fd7bcfe7640 1 -- 172.21.15.193:0/2533239567 wait complete. 2024-09-19T04:37:44.722 DEBUG:teuthology.orchestra.run.smithi183:mon.b> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mon.b.service 2024-09-19T04:37:44.725 DEBUG:teuthology.orchestra.run.smithi193:mon.c> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mon.c.service 2024-09-19T04:37:44.728 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:37:44.728 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:37:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:44 smithi123 ceph-mon[30511]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:37:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:44 smithi123 ceph-mon[30511]: Updating smithi193:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:37:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:44 smithi123 ceph-mon[30511]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:44 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:44 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:37:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:44 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:37:44.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:44 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:37:44.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:44 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:44.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:44 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:37:44.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:44 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:37:45.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:45 smithi123 ceph-mon[30511]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi123:172.21.15.123=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:37:45.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:45 smithi123 ceph-mon[30511]: Saving service mon spec with placement smithi123:172.21.15.123=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c;count:3 2024-09-19T04:37:45.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:45 smithi123 ceph-mon[30511]: Deploying daemon mon.c on smithi193 2024-09-19T04:37:46.852 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:37:46.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:46 smithi123 ceph-mon[30511]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:47.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.412+0000 7fb02817b640 1 -- 172.21.15.193:0/2006181485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb020107670 msgr2=0x7fb020107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:47.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.412+0000 7fb02817b640 1 --2- 172.21.15.193:0/2006181485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb020107670 0x7fb020107a50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009880 tx=0x7fb01c02a5d0 comp rx=0 tx=0).stop 2024-09-19T04:37:47.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.412+0000 7fb02817b640 1 -- 172.21.15.193:0/2006181485 shutdown_connections 2024-09-19T04:37:47.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.412+0000 7fb02817b640 1 --2- 172.21.15.193:0/2006181485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb020107670 0x7fb020107a50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:47.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.412+0000 7fb02817b640 1 -- 172.21.15.193:0/2006181485 >> 172.21.15.193:0/2006181485 conn(0x7fb0200fd330 msgr2=0x7fb0200ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:47.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.412+0000 7fb02817b640 1 -- 172.21.15.193:0/2006181485 shutdown_connections 2024-09-19T04:37:47.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.412+0000 7fb02817b640 1 -- 172.21.15.193:0/2006181485 wait complete. 2024-09-19T04:37:47.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.413+0000 7fb02817b640 1 Processor -- start 2024-09-19T04:37:47.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.413+0000 7fb02817b640 1 -- start start 2024-09-19T04:37:47.416 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.413+0000 7fb02817b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb020107670 0x7fb0201972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:47.416 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.413+0000 7fb02817b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb02010bd30 con 0x7fb020107670 2024-09-19T04:37:47.416 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.413+0000 7fb025f17640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb020107670 0x7fb0201972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:47.416 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.413+0000 7fb025f17640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb020107670 0x7fb0201972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.193:60520/0 (socket says 172.21.15.193:60520) 2024-09-19T04:37:47.416 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.413+0000 7fb025f17640 1 -- 172.21.15.193:0/1396838459 learned_addr learned my addr 172.21.15.193:0/1396838459 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-19T04:37:47.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.414+0000 7fb025f17640 1 -- 172.21.15.193:0/1396838459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb020197800 con 0x7fb020107670 2024-09-19T04:37:47.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.414+0000 7fb025f17640 1 --2- 172.21.15.193:0/1396838459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb020107670 0x7fb0201972c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009230 tx=0x7fb01c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:47.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.414+0000 7fb016ffd640 1 -- 172.21.15.193:0/1396838459 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb01c03f070 con 0x7fb020107670 2024-09-19T04:37:47.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.414+0000 7fb016ffd640 1 -- 172.21.15.193:0/1396838459 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb01c03bc80 con 0x7fb020107670 2024-09-19T04:37:47.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.414+0000 7fb02817b640 1 -- 172.21.15.193:0/1396838459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb020197af0 con 0x7fb020107670 2024-09-19T04:37:47.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.414+0000 7fb016ffd640 1 -- 172.21.15.193:0/1396838459 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb01c035030 con 0x7fb020107670 2024-09-19T04:37:47.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.414+0000 7fb02817b640 1 -- 172.21.15.193:0/1396838459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb02019bb90 con 0x7fb020107670 2024-09-19T04:37:47.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.415+0000 7fb02817b640 1 -- 172.21.15.193:0/1396838459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb020102db0 con 0x7fb020107670 2024-09-19T04:37:47.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.415+0000 7fb016ffd640 1 -- 172.21.15.193:0/1396838459 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fb01c042070 con 0x7fb020107670 2024-09-19T04:37:47.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.416+0000 7fb016ffd640 1 --2- 172.21.15.193:0/1396838459 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fb00c0392a0 0x7fb00c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:47.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.416+0000 7fb016ffd640 1 -- 172.21.15.193:0/1396838459 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb01c06d080 con 0x7fb020107670 2024-09-19T04:37:47.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.416+0000 7fb025716640 1 --2- 172.21.15.193:0/1396838459 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fb00c0392a0 0x7fb00c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:47.419 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.417+0000 7fb025716640 1 --2- 172.21.15.193:0/1396838459 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fb00c0392a0 0x7fb00c03b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb0100098c0 tx=0x7fb010006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:47.422 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.421+0000 7fb016ffd640 1 -- 172.21.15.193:0/1396838459 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb01c031e20 con 0x7fb020107670 2024-09-19T04:37:47.674 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.672+0000 7fb02817b640 1 -- 172.21.15.193:0/1396838459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb020198620 con 0x7fb020107670 2024-09-19T04:37:47.674 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.673+0000 7fb016ffd640 1 -- 172.21.15.193:0/1396838459 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7fb01c0372d0 con 0x7fb020107670 2024-09-19T04:37:47.675 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:37:47.675 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"911fa052-7640-11ef-bceb-c7b262605968","modified":"2024-09-19T04:36:17.413126Z","created":"2024-09-19T04:36:17.413126Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T04:37:47.675 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-09-19T04:37:47.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.675+0000 7fb02817b640 1 -- 172.21.15.193:0/1396838459 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fb00c0392a0 msgr2=0x7fb00c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:47.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.675+0000 7fb02817b640 1 --2- 172.21.15.193:0/1396838459 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fb00c0392a0 0x7fb00c03b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb0100098c0 tx=0x7fb010006db0 comp rx=0 tx=0).stop 2024-09-19T04:37:47.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.675+0000 7fb02817b640 1 -- 172.21.15.193:0/1396838459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb020107670 msgr2=0x7fb0201972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:47.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.675+0000 7fb02817b640 1 --2- 172.21.15.193:0/1396838459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb020107670 0x7fb0201972c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009230 tx=0x7fb01c003d40 comp rx=0 tx=0).stop 2024-09-19T04:37:47.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.675+0000 7fb02817b640 1 -- 172.21.15.193:0/1396838459 shutdown_connections 2024-09-19T04:37:47.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.675+0000 7fb02817b640 1 --2- 172.21.15.193:0/1396838459 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fb00c0392a0 0x7fb00c03b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:47.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.675+0000 7fb02817b640 1 --2- 172.21.15.193:0/1396838459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb020107670 0x7fb0201972c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:47.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.675+0000 7fb02817b640 1 -- 172.21.15.193:0/1396838459 >> 172.21.15.193:0/1396838459 conn(0x7fb0200fd330 msgr2=0x7fb020105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:47.678 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.675+0000 7fb02817b640 1 -- 172.21.15.193:0/1396838459 shutdown_connections 2024-09-19T04:37:47.678 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:47.675+0000 7fb02817b640 1 -- 172.21.15.193:0/1396838459 wait complete. 2024-09-19T04:37:48.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 podman[35573]: 2024-09-19 04:37:48.317661057 +0000 UTC m=+0.016450450 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:37:48.420 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 podman[35573]: 2024-09-19 04:37:48.418954685 +0000 UTC m=+0.117744060 container create 8e497ed1e57b31272a65c21f7282f0d4fcc100234cc9dd42d0186fa345ff6ecf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:37:48.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 podman[35573]: 2024-09-19 04:37:48.596778671 +0000 UTC m=+0.295568045 container init 8e497ed1e57b31272a65c21f7282f0d4fcc100234cc9dd42d0186fa345ff6ecf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T04:37:48.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 podman[35573]: 2024-09-19 04:37:48.599917986 +0000 UTC m=+0.298707364 container start 8e497ed1e57b31272a65c21f7282f0d4fcc100234cc9dd42d0186fa345ff6ecf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-19T04:37:48.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T04:37:48.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T04:37:48.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: pidfile_write: ignore empty --pid-file 2024-09-19T04:37:48.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: load: jerasure load: lrc 2024-09-19T04:37:48.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: RocksDB version: 7.9.2 2024-09-19T04:37:48.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Git sha 0 2024-09-19T04:37:48.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T04:37:48.674 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: DB SUMMARY 2024-09-19T04:37:48.674 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: DB Session ID: 6IQ8XJ48PM2ZU6C7MF6K 2024-09-19T04:37:48.674 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: CURRENT file: CURRENT 2024-09-19T04:37:48.674 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T04:37:48.674 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-19T04:37:48.674 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-19T04:37:48.674 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-19T04:37:48.674 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.error_if_exists: 0 2024-09-19T04:37:48.674 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.create_if_missing: 0 2024-09-19T04:37:48.674 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.paranoid_checks: 1 2024-09-19T04:37:48.674 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T04:37:48.675 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T04:37:48.675 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T04:37:48.675 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.env: 0x5645b4192940 2024-09-19T04:37:48.675 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T04:37:48.675 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.info_log: 0x5645b5edb6e0 2024-09-19T04:37:48.675 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T04:37:48.677 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.statistics: (nil) 2024-09-19T04:37:48.677 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.use_fsync: 0 2024-09-19T04:37:48.677 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_log_file_size: 0 2024-09-19T04:37:48.677 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T04:37:48.677 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T04:37:48.678 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T04:37:48.678 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T04:37:48.678 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.allow_fallocate: 1 2024-09-19T04:37:48.678 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T04:37:48.678 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T04:37:48.678 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.use_direct_reads: 0 2024-09-19T04:37:48.678 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T04:37:48.678 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T04:37:48.678 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.db_log_dir: 2024-09-19T04:37:48.678 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.wal_dir: 2024-09-19T04:37:48.678 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T04:37:48.678 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T04:37:48.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T04:37:48.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T04:37:48.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T04:37:48.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T04:37:48.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T04:37:48.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T04:37:48.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.write_buffer_manager: 0x5645b5eeadc0 2024-09-19T04:37:48.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T04:37:48.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T04:37:48.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T04:37:48.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T04:37:48.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T04:37:48.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T04:37:48.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T04:37:48.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T04:37:48.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.unordered_write: 0 2024-09-19T04:37:48.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T04:37:48.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T04:37:48.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T04:37:48.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T04:37:48.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.row_cache: None 2024-09-19T04:37:48.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.wal_filter: None 2024-09-19T04:37:48.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T04:37:48.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T04:37:48.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.two_write_queues: 0 2024-09-19T04:37:48.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T04:37:48.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.wal_compression: 0 2024-09-19T04:37:48.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.atomic_flush: 0 2024-09-19T04:37:48.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T04:37:48.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T04:37:48.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T04:37:48.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.log_readahead_size: 0 2024-09-19T04:37:48.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T04:37:48.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T04:37:48.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T04:37:48.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T04:37:48.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T04:37:48.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T04:37:48.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T04:37:48.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_background_jobs: 2 2024-09-19T04:37:48.683 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_background_compactions: -1 2024-09-19T04:37:48.683 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_subcompactions: 1 2024-09-19T04:37:48.683 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T04:37:48.683 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T04:37:48.683 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T04:37:48.683 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T04:37:48.683 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T04:37:48.683 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T04:37:48.683 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T04:37:48.683 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T04:37:48.684 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_open_files: -1 2024-09-19T04:37:48.684 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T04:37:48.684 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T04:37:48.684 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T04:37:48.684 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T04:37:48.684 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_background_flushes: -1 2024-09-19T04:37:48.684 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Compression algorithms supported: 2024-09-19T04:37:48.684 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: kZSTD supported: 0 2024-09-19T04:37:48.684 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: kXpressCompression supported: 0 2024-09-19T04:37:48.684 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: kBZip2Compression supported: 0 2024-09-19T04:37:48.684 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T04:37:48.685 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: kLZ4Compression supported: 1 2024-09-19T04:37:48.685 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: kZlibCompression supported: 1 2024-09-19T04:37:48.685 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T04:37:48.685 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: kSnappyCompression supported: 1 2024-09-19T04:37:48.685 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T04:37:48.685 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T04:37:48.685 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-19T04:37:48.685 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T04:37:48.685 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T04:37:48.685 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.merge_operator: 2024-09-19T04:37:48.686 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compaction_filter: None 2024-09-19T04:37:48.686 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T04:37:48.686 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T04:37:48.686 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T04:37:48.686 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T04:37:48.686 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5645b5edb320) 2024-09-19T04:37:48.686 INFO:journalctl@ceph.mon.c.smithi193.stdout: cache_index_and_filter_blocks: 1 2024-09-19T04:37:48.686 INFO:journalctl@ceph.mon.c.smithi193.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T04:37:48.686 INFO:journalctl@ceph.mon.c.smithi193.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T04:37:48.686 INFO:journalctl@ceph.mon.c.smithi193.stdout: pin_top_level_index_and_filter: 1 2024-09-19T04:37:48.687 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_type: 0 2024-09-19T04:37:48.687 INFO:journalctl@ceph.mon.c.smithi193.stdout: data_block_index_type: 0 2024-09-19T04:37:48.687 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_shortening: 1 2024-09-19T04:37:48.687 INFO:journalctl@ceph.mon.c.smithi193.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T04:37:48.687 INFO:journalctl@ceph.mon.c.smithi193.stdout: checksum: 4 2024-09-19T04:37:48.687 INFO:journalctl@ceph.mon.c.smithi193.stdout: no_block_cache: 0 2024-09-19T04:37:48.687 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache: 0x5645b5ecf1f0 2024-09-19T04:37:48.687 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_name: BinnedLRUCache 2024-09-19T04:37:48.687 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_options: 2024-09-19T04:37:48.687 INFO:journalctl@ceph.mon.c.smithi193.stdout: capacity : 536870912 2024-09-19T04:37:48.687 INFO:journalctl@ceph.mon.c.smithi193.stdout: num_shard_bits : 4 2024-09-19T04:37:48.687 INFO:journalctl@ceph.mon.c.smithi193.stdout: strict_capacity_limit : 0 2024-09-19T04:37:48.688 INFO:journalctl@ceph.mon.c.smithi193.stdout: high_pri_pool_ratio: 0.000 2024-09-19T04:37:48.688 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_compressed: (nil) 2024-09-19T04:37:48.688 INFO:journalctl@ceph.mon.c.smithi193.stdout: persistent_cache: (nil) 2024-09-19T04:37:48.688 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_size: 4096 2024-09-19T04:37:48.688 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_size_deviation: 10 2024-09-19T04:37:48.688 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_restart_interval: 16 2024-09-19T04:37:48.688 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_block_restart_interval: 1 2024-09-19T04:37:48.688 INFO:journalctl@ceph.mon.c.smithi193.stdout: metadata_block_size: 4096 2024-09-19T04:37:48.688 INFO:journalctl@ceph.mon.c.smithi193.stdout: partition_filters: 0 2024-09-19T04:37:48.688 INFO:journalctl@ceph.mon.c.smithi193.stdout: use_delta_encoding: 1 2024-09-19T04:37:48.688 INFO:journalctl@ceph.mon.c.smithi193.stdout: filter_policy: bloomfilter 2024-09-19T04:37:48.688 INFO:journalctl@ceph.mon.c.smithi193.stdout: whole_key_filtering: 1 2024-09-19T04:37:48.689 INFO:journalctl@ceph.mon.c.smithi193.stdout: verify_compression: 0 2024-09-19T04:37:48.689 INFO:journalctl@ceph.mon.c.smithi193.stdout: read_amp_bytes_per_bit: 0 2024-09-19T04:37:48.689 INFO:journalctl@ceph.mon.c.smithi193.stdout: format_version: 5 2024-09-19T04:37:48.689 INFO:journalctl@ceph.mon.c.smithi193.stdout: enable_index_compression: 1 2024-09-19T04:37:48.689 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_align: 0 2024-09-19T04:37:48.689 INFO:journalctl@ceph.mon.c.smithi193.stdout: max_auto_readahead_size: 262144 2024-09-19T04:37:48.689 INFO:journalctl@ceph.mon.c.smithi193.stdout: prepopulate_block_cache: 0 2024-09-19T04:37:48.689 INFO:journalctl@ceph.mon.c.smithi193.stdout: initial_auto_readahead_size: 8192 2024-09-19T04:37:48.689 INFO:journalctl@ceph.mon.c.smithi193.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T04:37:48.689 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T04:37:48.689 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T04:37:48.690 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compression: NoCompression 2024-09-19T04:37:48.690 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T04:37:48.690 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T04:37:48.690 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T04:37:48.690 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.num_levels: 7 2024-09-19T04:37:48.690 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T04:37:48.690 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T04:37:48.690 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T04:37:48.690 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T04:37:48.690 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T04:37:48.690 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T04:37:48.690 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T04:37:48.691 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:37:48.691 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T04:37:48.691 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T04:37:48.691 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:37:48.691 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:37:48.691 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T04:37:48.691 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T04:37:48.691 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T04:37:48.691 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T04:37:48.691 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:37:48.691 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:37:48.691 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T04:37:48.692 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T04:37:48.692 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:37:48.692 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T04:37:48.692 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T04:37:48.692 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T04:37:48.692 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T04:37:48.692 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T04:37:48.692 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T04:37:48.692 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T04:37:48.692 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T04:37:48.692 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T04:37:48.693 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T04:37:48.693 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T04:37:48.693 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T04:37:48.693 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T04:37:48.693 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T04:37:48.693 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T04:37:48.693 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T04:37:48.694 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T04:37:48.694 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T04:37:48.694 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T04:37:48.694 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T04:37:48.694 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T04:37:48.694 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T04:37:48.694 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T04:37:48.694 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T04:37:48.694 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T04:37:48.694 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T04:37:48.694 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T04:37:48.695 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T04:37:48.695 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T04:37:48.695 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T04:37:48.695 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T04:37:48.695 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T04:37:48.695 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T04:37:48.695 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.inplace_update_support: 0 2024-09-19T04:37:48.695 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T04:37:48.695 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T04:37:48.695 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T04:37:48.695 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T04:37:48.696 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.bloom_locality: 0 2024-09-19T04:37:48.696 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.max_successive_merges: 0 2024-09-19T04:37:48.696 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T04:37:48.696 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T04:37:48.696 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T04:37:48.696 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T04:37:48.696 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.ttl: 2592000 2024-09-19T04:37:48.696 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T04:37:48.696 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T04:37:48.696 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T04:37:48.696 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.enable_blob_files: false 2024-09-19T04:37:48.697 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.min_blob_size: 0 2024-09-19T04:37:48.697 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T04:37:48.697 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T04:37:48.697 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T04:37:48.697 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T04:37:48.697 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T04:37:48.697 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T04:37:48.697 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T04:37:48.697 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T04:37:48.697 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-19T04:37:48.697 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-19T04:37:48.697 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 32ab5fee-1328-436d-8780-0754a36588db 2024-09-19T04:37:48.698 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726720668626944, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-19T04:37:48.698 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-19T04:37:48.698 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726720668627363, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726720668, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "32ab5fee-1328-436d-8780-0754a36588db", "db_session_id": "6IQ8XJ48PM2ZU6C7MF6K", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-19T04:37:48.698 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726720668627456, "job": 1, "event": "recovery_finished"} 2024-09-19T04:37:48.698 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-19T04:37:48.698 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T04:37:48.698 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5645b5efce00 2024-09-19T04:37:48.698 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: DB pointer 0x5645b5ff8000 2024-09-19T04:37:48.698 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T04:37:48.698 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T04:37:48.699 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** DB Stats ** 2024-09-19T04:37:48.699 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:37:48.699 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T04:37:48.699 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:37:48.699 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:37:48.699 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T04:37:48.699 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:37:48.699 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:37:48.699 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-19T04:37:48.699 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2024-09-19T04:37:48.699 INFO:journalctl@ceph.mon.c.smithi193.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:37:48.700 INFO:journalctl@ceph.mon.c.smithi193.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T04:37:48.700 INFO:journalctl@ceph.mon.c.smithi193.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T04:37:48.700 INFO:journalctl@ceph.mon.c.smithi193.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T04:37:48.700 INFO:journalctl@ceph.mon.c.smithi193.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T04:37:48.700 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-19T04:37:48.700 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2024-09-19T04:37:48.700 INFO:journalctl@ceph.mon.c.smithi193.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:37:48.700 INFO:journalctl@ceph.mon.c.smithi193.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T04:37:48.700 INFO:journalctl@ceph.mon.c.smithi193.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T04:37:48.700 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-19T04:37:48.700 INFO:journalctl@ceph.mon.c.smithi193.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T04:37:48.700 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-19T04:37:48.701 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:37:48.701 INFO:journalctl@ceph.mon.c.smithi193.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T04:37:48.701 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T04:37:48.701 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T04:37:48.701 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T04:37:48.701 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T04:37:48.701 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:37:48.701 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:37:48.701 INFO:journalctl@ceph.mon.c.smithi193.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T04:37:48.701 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache BinnedLRUCache@0x5645b5ecf1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-19T04:37:48.701 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-19T04:37:48.702 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-19T04:37:48.702 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T04:37:48.702 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-19T04:37:48.702 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: using public_addr v2:172.21.15.193:0/0 -> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-09-19T04:37:48.702 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: starting mon.c rank -1 at public addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] at bind addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:37:48.702 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: mon.c@-1(???) e0 preinit fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:37:48.702 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: mon.c@-1(synchronizing).mds e1 new map 2024-09-19T04:37:48.702 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-19T04:37:48.702 INFO:journalctl@ceph.mon.c.smithi193.stdout: e1 2024-09-19T04:37:48.702 INFO:journalctl@ceph.mon.c.smithi193.stdout: btime 2024-09-19T04:36:21:083029+0000 2024-09-19T04:37:48.703 INFO:journalctl@ceph.mon.c.smithi193.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T04:37:48.703 INFO:journalctl@ceph.mon.c.smithi193.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:37:48.703 INFO:journalctl@ceph.mon.c.smithi193.stdout: legacy client fscid: -1 2024-09-19T04:37:48.703 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-19T04:37:48.703 INFO:journalctl@ceph.mon.c.smithi193.stdout: No filesystems configured 2024-09-19T04:37:48.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T04:37:48.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T04:37:48.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T04:37:48.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T04:37:48.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T04:37:48.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T04:37:48.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T04:37:48.704 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:37:48.704 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:37:48.704 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:37:48.704 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:37:48.704 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:48.704 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.704 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.704 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.704 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.704 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:37:48.704 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:37:48.705 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:37:48.705 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-19T04:37:48.705 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.705 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.705 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.705 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: Updating smithi183:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:37:48.705 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: Deploying cephadm binary to smithi193 2024-09-19T04:37:48.705 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:37:48.705 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: Updating smithi183:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:37:48.705 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:48.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:37:48.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: Added host smithi193 2024-09-19T04:37:48.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:48.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:37:48.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:48.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/391524049' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T04:37:48.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.707 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.707 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.707 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:37:48.707 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:37:48.707 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:37:48.707 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-19T04:37:48.707 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: Updating smithi193:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:37:48.707 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/391524049' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T04:37:48.707 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:37:48.707 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.708 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.708 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.708 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:37:48.708 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: Updating smithi193:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:37:48.708 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:48.708 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.708 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:37:48.708 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:37:48.708 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:37:48.708 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:48.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:37:48.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:37:48.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi123:172.21.15.123=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:37:48.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: Saving service mon spec with placement smithi123:172.21.15.123=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c;count:3 2024-09-19T04:37:48.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: Deploying daemon mon.c on smithi193 2024-09-19T04:37:48.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:48.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:48.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: from='client.? 172.21.15.193:0/1396838459' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:37:48.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 ceph-mon[35592]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T04:37:48.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 bash[35573]: 8e497ed1e57b31272a65c21f7282f0d4fcc100234cc9dd42d0186fa345ff6ecf 2024-09-19T04:37:48.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:48 smithi193 systemd[1]: Started Ceph mon.c for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:37:48.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:48 smithi123 ceph-mon[30511]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:48.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:48 smithi123 ceph-mon[30511]: from='client.? 172.21.15.193:0/1396838459' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:37:49.197 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:37:49.197 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:37:49.446 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:37:53.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:37:53.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:53.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: mon.a calling monitor election 2024-09-19T04:37:53.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:53.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:53.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:53.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: mon.c calling monitor election 2024-09-19T04:37:53.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:53.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:53.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:53.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:53.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:53.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:37:53.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: monmap epoch 2 2024-09-19T04:37:53.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:37:53.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: last_changed 2024-09-19T04:37:48.656309+0000 2024-09-19T04:37:53.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: created 2024-09-19T04:36:17.413126+0000 2024-09-19T04:37:53.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: min_mon_release 19 (squid) 2024-09-19T04:37:53.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: election_strategy: 1 2024-09-19T04:37:53.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-09-19T04:37:53.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-19T04:37:53.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: fsmap 2024-09-19T04:37:53.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:37:53.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: mgrmap e13: x(active, since 40s) 2024-09-19T04:37:53.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: overall HEALTH_OK 2024-09-19T04:37:53.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:53.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:53.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:53.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:37:53.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:37:54.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:37:54.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:54.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: mon.a calling monitor election 2024-09-19T04:37:54.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:54.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:54.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:54.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: mon.c calling monitor election 2024-09-19T04:37:54.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:54.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:54.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:54.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:54.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:54.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:37:54.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: monmap epoch 2 2024-09-19T04:37:54.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:37:54.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: last_changed 2024-09-19T04:37:48.656309+0000 2024-09-19T04:37:54.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: created 2024-09-19T04:36:17.413126+0000 2024-09-19T04:37:54.179 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: min_mon_release 19 (squid) 2024-09-19T04:37:54.179 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: election_strategy: 1 2024-09-19T04:37:54.179 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-09-19T04:37:54.179 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-19T04:37:54.179 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: fsmap 2024-09-19T04:37:54.179 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:37:54.179 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: mgrmap e13: x(active, since 40s) 2024-09-19T04:37:54.179 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: overall HEALTH_OK 2024-09-19T04:37:54.179 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:54.179 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:54.179 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:54.179 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:37:54.179 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:37:54.311 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.309+0000 7fc927372640 1 -- 172.21.15.193:0/264188276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc920106630 msgr2=0x7fc90c005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:54.312 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.309+0000 7fc927372640 1 --2- 172.21.15.193:0/264188276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc920106630 0x7fc90c005580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc91c012c60 tx=0x7fc91c02c620 comp rx=0 tx=0).stop 2024-09-19T04:37:54.312 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.309+0000 7fc927372640 1 -- 172.21.15.193:0/264188276 shutdown_connections 2024-09-19T04:37:54.312 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.309+0000 7fc927372640 1 --2- 172.21.15.193:0/264188276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc920106630 0x7fc90c005580 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:54.312 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.309+0000 7fc927372640 1 -- 172.21.15.193:0/264188276 >> 172.21.15.193:0/264188276 conn(0x7fc9200fc220 msgr2=0x7fc9200fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:54.312 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.309+0000 7fc927372640 1 -- 172.21.15.193:0/264188276 shutdown_connections 2024-09-19T04:37:54.312 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.309+0000 7fc927372640 1 -- 172.21.15.193:0/264188276 wait complete. 2024-09-19T04:37:54.313 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.310+0000 7fc927372640 1 Processor -- start 2024-09-19T04:37:54.313 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.311+0000 7fc927372640 1 -- start start 2024-09-19T04:37:54.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.311+0000 7fc927372640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc920106630 0x7fc920196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:54.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.311+0000 7fc927372640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc920196640 0x7fc92019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:54.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.311+0000 7fc927372640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc920108670 con 0x7fc920196640 2024-09-19T04:37:54.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.311+0000 7fc927372640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201084f0 con 0x7fc920106630 2024-09-19T04:37:54.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.311+0000 7fc92510e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc920106630 0x7fc920196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:54.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.311+0000 7fc92510e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc920106630 0x7fc920196100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:53320/0 (socket says 172.21.15.193:53320) 2024-09-19T04:37:54.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.311+0000 7fc92510e640 1 -- 172.21.15.193:0/1923972954 learned_addr learned my addr 172.21.15.193:0/1923972954 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-19T04:37:54.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.311+0000 7fc92510e640 1 -- 172.21.15.193:0/1923972954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc920106630 msgr2=0x7fc920196100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T04:37:54.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.311+0000 7fc92510e640 1 -- 172.21.15.193:0/1923972954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc920106630 msgr2=0x7fc920196100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:37:54.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.311+0000 7fc92510e640 1 --2- 172.21.15.193:0/1923972954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc920106630 0x7fc920196100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:37:54.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.311+0000 7fc92510e640 1 --2- 172.21.15.193:0/1923972954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc920106630 0x7fc920196100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:37:54.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.311+0000 7fc92490d640 1 --2- 172.21.15.193:0/1923972954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc920196640 0x7fc92019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:54.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.312+0000 7fc92490d640 1 -- 172.21.15.193:0/1923972954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc920106630 msgr2=0x7fc920196100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:37:54.315 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.312+0000 7fc92490d640 1 --2- 172.21.15.193:0/1923972954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc920106630 0x7fc920196100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:54.315 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.312+0000 7fc92490d640 1 -- 172.21.15.193:0/1923972954 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc920196c60 con 0x7fc920196640 2024-09-19T04:37:54.315 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.312+0000 7fc92490d640 1 --2- 172.21.15.193:0/1923972954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc920196640 0x7fc92019a9d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc91000b3d0 tx=0x7fc91000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:54.316 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.312+0000 7fc9167fc640 1 -- 172.21.15.193:0/1923972954 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc910011630 con 0x7fc920196640 2024-09-19T04:37:54.316 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.313+0000 7fc9167fc640 1 -- 172.21.15.193:0/1923972954 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc910011cb0 con 0x7fc920196640 2024-09-19T04:37:54.316 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.313+0000 7fc927372640 1 -- 172.21.15.193:0/1923972954 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92019af10 con 0x7fc920196640 2024-09-19T04:37:54.316 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.313+0000 7fc9167fc640 1 -- 172.21.15.193:0/1923972954 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc910010df0 con 0x7fc920196640 2024-09-19T04:37:54.316 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.313+0000 7fc927372640 1 -- 172.21.15.193:0/1923972954 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc92019b4d0 con 0x7fc920196640 2024-09-19T04:37:54.316 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.314+0000 7fc9167fc640 1 -- 172.21.15.193:0/1923972954 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fc910018020 con 0x7fc920196640 2024-09-19T04:37:54.316 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.314+0000 7fc927372640 1 -- 172.21.15.193:0/1923972954 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc92019b7e0 con 0x7fc920196640 2024-09-19T04:37:54.316 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.314+0000 7fc9167fc640 1 --2- 172.21.15.193:0/1923972954 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc904039040 0x7fc90403b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:54.316 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.314+0000 7fc9167fc640 1 -- 172.21.15.193:0/1923972954 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc91004bfb0 con 0x7fc920196640 2024-09-19T04:37:54.317 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.314+0000 7fc92510e640 1 --2- 172.21.15.193:0/1923972954 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc904039040 0x7fc90403b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:54.317 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.315+0000 7fc92510e640 1 --2- 172.21.15.193:0/1923972954 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc904039040 0x7fc90403b500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc91c035040 tx=0x7fc91c033660 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:54.321 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.320+0000 7fc9167fc640 1 -- 172.21.15.193:0/1923972954 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc91001f020 con 0x7fc920196640 2024-09-19T04:37:54.555 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.554+0000 7fc927372640 1 -- 172.21.15.193:0/1923972954 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc9201974c0 con 0x7fc920196640 2024-09-19T04:37:54.555 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.554+0000 7fc9167fc640 1 -- 172.21.15.193:0/1923972954 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fc910017500 con 0x7fc920196640 2024-09-19T04:37:54.556 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:37:54.556 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"911fa052-7640-11ef-bceb-c7b262605968","modified":"2024-09-19T04:37:48.656309Z","created":"2024-09-19T04:36:17.413126Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T04:37:54.556 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-09-19T04:37:54.558 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.556+0000 7fc927372640 1 -- 172.21.15.193:0/1923972954 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc904039040 msgr2=0x7fc90403b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:54.558 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.556+0000 7fc927372640 1 --2- 172.21.15.193:0/1923972954 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc904039040 0x7fc90403b500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc91c035040 tx=0x7fc91c033660 comp rx=0 tx=0).stop 2024-09-19T04:37:54.558 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.556+0000 7fc927372640 1 -- 172.21.15.193:0/1923972954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc920196640 msgr2=0x7fc92019a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:54.559 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.556+0000 7fc927372640 1 --2- 172.21.15.193:0/1923972954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc920196640 0x7fc92019a9d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc91000b3d0 tx=0x7fc91000b890 comp rx=0 tx=0).stop 2024-09-19T04:37:54.559 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.556+0000 7fc927372640 1 -- 172.21.15.193:0/1923972954 shutdown_connections 2024-09-19T04:37:54.559 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.556+0000 7fc927372640 1 --2- 172.21.15.193:0/1923972954 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc904039040 0x7fc90403b500 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:54.559 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.556+0000 7fc927372640 1 --2- 172.21.15.193:0/1923972954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc920196640 0x7fc92019a9d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:54.559 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.556+0000 7fc927372640 1 --2- 172.21.15.193:0/1923972954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc920106630 0x7fc920196100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:54.559 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.556+0000 7fc927372640 1 -- 172.21.15.193:0/1923972954 >> 172.21.15.193:0/1923972954 conn(0x7fc9200fc220 msgr2=0x7fc9200fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:54.559 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.556+0000 7fc927372640 1 -- 172.21.15.193:0/1923972954 shutdown_connections 2024-09-19T04:37:54.559 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:54.557+0000 7fc927372640 1 -- 172.21.15.193:0/1923972954 wait complete. 2024-09-19T04:37:54.886 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:54 smithi193 ceph-mon[35592]: Deploying daemon mon.b on smithi183 2024-09-19T04:37:54.886 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:54 smithi193 ceph-mon[35592]: from='client.? 172.21.15.193:0/1923972954' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:37:54.886 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:54 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:55.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:54 smithi123 ceph-mon[30511]: Deploying daemon mon.b on smithi183 2024-09-19T04:37:55.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:54 smithi123 ceph-mon[30511]: from='client.? 172.21.15.193:0/1923972954' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:37:55.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:54 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:55.832 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:55 smithi123 ceph-mon[30511]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:55.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:55 smithi193 ceph-mon[35592]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:56.160 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:37:56.160 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:37:56.412 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:37:56.847 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.841+0000 7fd2eee57640 1 -- 172.21.15.193:0/2179429194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e8107660 msgr2=0x7fd2e8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:56.847 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.841+0000 7fd2eee57640 1 --2- 172.21.15.193:0/2179429194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e8107660 0x7fd2e8107a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009880 tx=0x7fd2dc02a5d0 comp rx=0 tx=0).stop 2024-09-19T04:37:56.847 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.841+0000 7fd2eee57640 1 -- 172.21.15.193:0/2179429194 shutdown_connections 2024-09-19T04:37:56.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.841+0000 7fd2eee57640 1 --2- 172.21.15.193:0/2179429194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e8107660 0x7fd2e8107a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:56.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.841+0000 7fd2eee57640 1 -- 172.21.15.193:0/2179429194 >> 172.21.15.193:0/2179429194 conn(0x7fd2e80fd320 msgr2=0x7fd2e80ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:56.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.841+0000 7fd2eee57640 1 -- 172.21.15.193:0/2179429194 shutdown_connections 2024-09-19T04:37:56.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.841+0000 7fd2eee57640 1 -- 172.21.15.193:0/2179429194 wait complete. 2024-09-19T04:37:56.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.841+0000 7fd2eee57640 1 Processor -- start 2024-09-19T04:37:56.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.841+0000 7fd2eee57640 1 -- start start 2024-09-19T04:37:56.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.842+0000 7fd2eee57640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e8107660 0x7fd2e80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:56.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.842+0000 7fd2eee57640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e8074a30 0x7fd2e8074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:56.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.842+0000 7fd2eee57640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e810bd20 con 0x7fd2e8074a30 2024-09-19T04:37:56.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.842+0000 7fd2eee57640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e810bba0 con 0x7fd2e8107660 2024-09-19T04:37:56.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.842+0000 7fd2ecbf3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e8107660 0x7fd2e80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:56.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.842+0000 7fd2ecbf3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e8107660 0x7fd2e80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:53324/0 (socket says 172.21.15.193:53324) 2024-09-19T04:37:56.849 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.842+0000 7fd2ecbf3640 1 -- 172.21.15.193:0/1717986167 learned_addr learned my addr 172.21.15.193:0/1717986167 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-19T04:37:56.849 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.842+0000 7fd2e7fff640 1 --2- 172.21.15.193:0/1717986167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e8074a30 0x7fd2e8074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:56.849 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.842+0000 7fd2ecbf3640 1 -- 172.21.15.193:0/1717986167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e8107660 msgr2=0x7fd2e80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T04:37:56.849 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.842+0000 7fd2ecbf3640 1 -- 172.21.15.193:0/1717986167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e8107660 msgr2=0x7fd2e80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:37:56.849 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.842+0000 7fd2ecbf3640 1 --2- 172.21.15.193:0/1717986167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e8107660 0x7fd2e80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:37:56.849 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.842+0000 7fd2ecbf3640 1 --2- 172.21.15.193:0/1717986167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e8107660 0x7fd2e80744f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:37:56.849 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.842+0000 7fd2e7fff640 1 -- 172.21.15.193:0/1717986167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e8107660 msgr2=0x7fd2e80744f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:37:56.849 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.842+0000 7fd2e7fff640 1 --2- 172.21.15.193:0/1717986167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e8107660 0x7fd2e80744f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:56.849 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.842+0000 7fd2e7fff640 1 -- 172.21.15.193:0/1717986167 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e80763e0 con 0x7fd2e8074a30 2024-09-19T04:37:56.849 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.842+0000 7fd2e7fff640 1 --2- 172.21.15.193:0/1717986167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e8074a30 0x7fd2e8074e90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400b400 tx=0x7fd2d400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:56.849 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.843+0000 7fd2e5ffb640 1 -- 172.21.15.193:0/1717986167 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd2d4011630 con 0x7fd2e8074a30 2024-09-19T04:37:56.849 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.843+0000 7fd2eee57640 1 -- 172.21.15.193:0/1717986167 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e81a3650 con 0x7fd2e8074a30 2024-09-19T04:37:56.849 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.843+0000 7fd2e5ffb640 1 -- 172.21.15.193:0/1717986167 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d4011cb0 con 0x7fd2e8074a30 2024-09-19T04:37:56.850 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.843+0000 7fd2e5ffb640 1 -- 172.21.15.193:0/1717986167 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd2d4010d80 con 0x7fd2e8074a30 2024-09-19T04:37:56.850 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.843+0000 7fd2eee57640 1 -- 172.21.15.193:0/1717986167 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e81a3be0 con 0x7fd2e8074a30 2024-09-19T04:37:56.850 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.844+0000 7fd2e5ffb640 1 -- 172.21.15.193:0/1717986167 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fd2d4018020 con 0x7fd2e8074a30 2024-09-19T04:37:56.850 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.844+0000 7fd2eee57640 1 -- 172.21.15.193:0/1717986167 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e8102da0 con 0x7fd2e8074a30 2024-09-19T04:37:56.850 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.844+0000 7fd2e5ffb640 1 --2- 172.21.15.193:0/1717986167 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd2d8039030 0x7fd2d803b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:37:56.850 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.844+0000 7fd2e5ffb640 1 -- 172.21.15.193:0/1717986167 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd2d404c250 con 0x7fd2e8074a30 2024-09-19T04:37:56.850 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.844+0000 7fd2ecbf3640 1 --2- 172.21.15.193:0/1717986167 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd2d8039030 0x7fd2d803b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:37:56.850 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.845+0000 7fd2ecbf3640 1 --2- 172.21.15.193:0/1717986167 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd2d8039030 0x7fd2d803b4f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0096c0 tx=0x7fd2dc005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:37:56.851 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:56.849+0000 7fd2e5ffb640 1 -- 172.21.15.193:0/1717986167 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd2d401f020 con 0x7fd2e8074a30 2024-09-19T04:37:57.082 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:57.080+0000 7fd2eee57640 1 -- 172.21.15.193:0/1717986167 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd2e81a3ef0 con 0x7fd2e8074a30 2024-09-19T04:37:57.082 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:57.081+0000 7fd2e5ffb640 1 -- 172.21.15.193:0/1717986167 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fd2d4018310 con 0x7fd2e8074a30 2024-09-19T04:37:57.083 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:37:57.083 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"911fa052-7640-11ef-bceb-c7b262605968","modified":"2024-09-19T04:37:48.656309Z","created":"2024-09-19T04:36:17.413126Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T04:37:57.083 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-09-19T04:37:57.085 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:57.083+0000 7fd2eee57640 1 -- 172.21.15.193:0/1717986167 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd2d8039030 msgr2=0x7fd2d803b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:57.085 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:57.083+0000 7fd2eee57640 1 --2- 172.21.15.193:0/1717986167 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd2d8039030 0x7fd2d803b4f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0096c0 tx=0x7fd2dc005a40 comp rx=0 tx=0).stop 2024-09-19T04:37:57.085 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:57.083+0000 7fd2eee57640 1 -- 172.21.15.193:0/1717986167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e8074a30 msgr2=0x7fd2e8074e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:37:57.085 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:57.083+0000 7fd2eee57640 1 --2- 172.21.15.193:0/1717986167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e8074a30 0x7fd2e8074e90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400b400 tx=0x7fd2d400b8c0 comp rx=0 tx=0).stop 2024-09-19T04:37:57.085 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:57.083+0000 7fd2eee57640 1 -- 172.21.15.193:0/1717986167 shutdown_connections 2024-09-19T04:37:57.085 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:57.083+0000 7fd2eee57640 1 --2- 172.21.15.193:0/1717986167 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fd2d8039030 0x7fd2d803b4f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:57.085 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:57.083+0000 7fd2eee57640 1 --2- 172.21.15.193:0/1717986167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e8074a30 0x7fd2e8074e90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:57.085 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:57.083+0000 7fd2eee57640 1 --2- 172.21.15.193:0/1717986167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e8107660 0x7fd2e80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:37:57.085 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:57.083+0000 7fd2eee57640 1 -- 172.21.15.193:0/1717986167 >> 172.21.15.193:0/1717986167 conn(0x7fd2e80fd320 msgr2=0x7fd2e80fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:37:57.085 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:57.083+0000 7fd2eee57640 1 -- 172.21.15.193:0/1717986167 shutdown_connections 2024-09-19T04:37:57.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:37:57.083+0000 7fd2eee57640 1 -- 172.21.15.193:0/1717986167 wait complete. 2024-09-19T04:37:57.325 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 podman[35314]: 2024-09-19 04:37:57.298972609 +0000 UTC m=+0.087615455 container create cb785711cbc83b992774f0c9a5c964122c89785c86d6de7b9b783517597bdda5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T04:37:57.325 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 podman[35314]: 2024-09-19 04:37:57.227275111 +0000 UTC m=+0.015917922 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:37:57.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:37:57 smithi123 ceph-mon[30511]: from='client.? 172.21.15.193:0/1717986167' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:37:57.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:37:57 smithi193 ceph-mon[35592]: from='client.? 172.21.15.193:0/1717986167' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:37:57.577 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 podman[35314]: 2024-09-19 04:37:57.501228735 +0000 UTC m=+0.289871550 container init cb785711cbc83b992774f0c9a5c964122c89785c86d6de7b9b783517597bdda5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-19T04:37:57.577 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 podman[35314]: 2024-09-19 04:37:57.504411691 +0000 UTC m=+0.293054489 container start cb785711cbc83b992774f0c9a5c964122c89785c86d6de7b9b783517597bdda5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-b, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T04:37:57.577 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T04:37:57.577 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T04:37:57.578 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: pidfile_write: ignore empty --pid-file 2024-09-19T04:37:57.578 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: load: jerasure load: lrc 2024-09-19T04:37:57.578 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: RocksDB version: 7.9.2 2024-09-19T04:37:57.578 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Git sha 0 2024-09-19T04:37:57.578 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T04:37:57.578 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: DB SUMMARY 2024-09-19T04:37:57.578 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: DB Session ID: L76R81B16ABW7HWQ4ZFV 2024-09-19T04:37:57.578 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: CURRENT file: CURRENT 2024-09-19T04:37:57.578 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T04:37:57.578 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-19T04:37:57.579 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-19T04:37:57.579 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-19T04:37:57.579 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.error_if_exists: 0 2024-09-19T04:37:57.579 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.create_if_missing: 0 2024-09-19T04:37:57.579 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.paranoid_checks: 1 2024-09-19T04:37:57.579 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T04:37:57.579 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T04:37:57.579 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T04:37:57.579 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.env: 0x55616a7d7940 2024-09-19T04:37:57.579 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T04:37:57.580 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.info_log: 0x55616c29f6e0 2024-09-19T04:37:57.580 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T04:37:57.580 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.statistics: (nil) 2024-09-19T04:37:57.580 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.use_fsync: 0 2024-09-19T04:37:57.580 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_log_file_size: 0 2024-09-19T04:37:57.580 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T04:37:57.580 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T04:37:57.580 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T04:37:57.580 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T04:37:57.580 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.allow_fallocate: 1 2024-09-19T04:37:57.580 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T04:37:57.581 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T04:37:57.581 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.use_direct_reads: 0 2024-09-19T04:37:57.581 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T04:37:57.581 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T04:37:57.581 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.db_log_dir: 2024-09-19T04:37:57.581 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.wal_dir: 2024-09-19T04:37:57.581 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T04:37:57.581 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T04:37:57.581 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T04:37:57.581 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T04:37:57.582 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T04:37:57.582 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T04:37:57.582 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T04:37:57.582 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T04:37:57.582 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.write_buffer_manager: 0x55616c2aedc0 2024-09-19T04:37:57.582 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T04:37:57.582 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T04:37:57.582 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T04:37:57.582 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T04:37:57.582 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T04:37:57.582 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T04:37:57.582 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T04:37:57.582 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T04:37:57.582 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.unordered_write: 0 2024-09-19T04:37:57.583 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T04:37:57.583 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T04:37:57.583 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T04:37:57.583 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T04:37:57.583 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.row_cache: None 2024-09-19T04:37:57.583 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.wal_filter: None 2024-09-19T04:37:57.584 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.two_write_queues: 0 2024-09-19T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.wal_compression: 0 2024-09-19T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.atomic_flush: 0 2024-09-19T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.log_readahead_size: 0 2024-09-19T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T04:37:57.586 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T04:37:57.586 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T04:37:57.586 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_background_jobs: 2 2024-09-19T04:37:57.586 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_background_compactions: -1 2024-09-19T04:37:57.586 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_subcompactions: 1 2024-09-19T04:37:57.586 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T04:37:57.586 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T04:37:57.586 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T04:37:57.586 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T04:37:57.586 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T04:37:57.586 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T04:37:57.586 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T04:37:57.586 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T04:37:57.586 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_open_files: -1 2024-09-19T04:37:57.586 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T04:37:57.586 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T04:37:57.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T04:37:57.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T04:37:57.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_background_flushes: -1 2024-09-19T04:37:57.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Compression algorithms supported: 2024-09-19T04:37:57.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: kZSTD supported: 0 2024-09-19T04:37:57.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: kXpressCompression supported: 0 2024-09-19T04:37:57.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: kBZip2Compression supported: 0 2024-09-19T04:37:57.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T04:37:57.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: kLZ4Compression supported: 1 2024-09-19T04:37:57.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: kZlibCompression supported: 1 2024-09-19T04:37:57.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T04:37:57.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: kSnappyCompression supported: 1 2024-09-19T04:37:57.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T04:37:57.588 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T04:37:57.588 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-19T04:37:57.588 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T04:37:57.588 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T04:37:57.588 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.merge_operator: 2024-09-19T04:37:57.588 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compaction_filter: None 2024-09-19T04:37:57.588 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T04:37:57.588 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T04:37:57.588 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T04:37:57.588 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T04:37:57.588 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55616c29f320) 2024-09-19T04:37:57.588 INFO:journalctl@ceph.mon.b.smithi183.stdout: cache_index_and_filter_blocks: 1 2024-09-19T04:37:57.588 INFO:journalctl@ceph.mon.b.smithi183.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T04:37:57.588 INFO:journalctl@ceph.mon.b.smithi183.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T04:37:57.588 INFO:journalctl@ceph.mon.b.smithi183.stdout: pin_top_level_index_and_filter: 1 2024-09-19T04:37:57.589 INFO:journalctl@ceph.mon.b.smithi183.stdout: index_type: 0 2024-09-19T04:37:57.589 INFO:journalctl@ceph.mon.b.smithi183.stdout: data_block_index_type: 0 2024-09-19T04:37:57.589 INFO:journalctl@ceph.mon.b.smithi183.stdout: index_shortening: 1 2024-09-19T04:37:57.589 INFO:journalctl@ceph.mon.b.smithi183.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T04:37:57.589 INFO:journalctl@ceph.mon.b.smithi183.stdout: checksum: 4 2024-09-19T04:37:57.589 INFO:journalctl@ceph.mon.b.smithi183.stdout: no_block_cache: 0 2024-09-19T04:37:57.589 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_cache: 0x55616c2931f0 2024-09-19T04:37:57.589 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_cache_name: BinnedLRUCache 2024-09-19T04:37:57.589 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_cache_options: 2024-09-19T04:37:57.589 INFO:journalctl@ceph.mon.b.smithi183.stdout: capacity : 536870912 2024-09-19T04:37:57.589 INFO:journalctl@ceph.mon.b.smithi183.stdout: num_shard_bits : 4 2024-09-19T04:37:57.589 INFO:journalctl@ceph.mon.b.smithi183.stdout: strict_capacity_limit : 0 2024-09-19T04:37:57.589 INFO:journalctl@ceph.mon.b.smithi183.stdout: high_pri_pool_ratio: 0.000 2024-09-19T04:37:57.589 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_cache_compressed: (nil) 2024-09-19T04:37:57.589 INFO:journalctl@ceph.mon.b.smithi183.stdout: persistent_cache: (nil) 2024-09-19T04:37:57.590 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_size: 4096 2024-09-19T04:37:57.590 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_size_deviation: 10 2024-09-19T04:37:57.590 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_restart_interval: 16 2024-09-19T04:37:57.590 INFO:journalctl@ceph.mon.b.smithi183.stdout: index_block_restart_interval: 1 2024-09-19T04:37:57.590 INFO:journalctl@ceph.mon.b.smithi183.stdout: metadata_block_size: 4096 2024-09-19T04:37:57.590 INFO:journalctl@ceph.mon.b.smithi183.stdout: partition_filters: 0 2024-09-19T04:37:57.590 INFO:journalctl@ceph.mon.b.smithi183.stdout: use_delta_encoding: 1 2024-09-19T04:37:57.590 INFO:journalctl@ceph.mon.b.smithi183.stdout: filter_policy: bloomfilter 2024-09-19T04:37:57.590 INFO:journalctl@ceph.mon.b.smithi183.stdout: whole_key_filtering: 1 2024-09-19T04:37:57.590 INFO:journalctl@ceph.mon.b.smithi183.stdout: verify_compression: 0 2024-09-19T04:37:57.590 INFO:journalctl@ceph.mon.b.smithi183.stdout: read_amp_bytes_per_bit: 0 2024-09-19T04:37:57.590 INFO:journalctl@ceph.mon.b.smithi183.stdout: format_version: 5 2024-09-19T04:37:57.590 INFO:journalctl@ceph.mon.b.smithi183.stdout: enable_index_compression: 1 2024-09-19T04:37:57.590 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_align: 0 2024-09-19T04:37:57.590 INFO:journalctl@ceph.mon.b.smithi183.stdout: max_auto_readahead_size: 262144 2024-09-19T04:37:57.590 INFO:journalctl@ceph.mon.b.smithi183.stdout: prepopulate_block_cache: 0 2024-09-19T04:37:57.591 INFO:journalctl@ceph.mon.b.smithi183.stdout: initial_auto_readahead_size: 8192 2024-09-19T04:37:57.591 INFO:journalctl@ceph.mon.b.smithi183.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T04:37:57.591 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T04:37:57.591 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T04:37:57.591 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compression: NoCompression 2024-09-19T04:37:57.591 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T04:37:57.591 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T04:37:57.591 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T04:37:57.591 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.num_levels: 7 2024-09-19T04:37:57.591 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T04:37:57.591 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T04:37:57.591 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T04:37:57.591 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T04:37:57.591 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T04:37:57.591 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T04:37:57.591 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T04:37:57.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:37:57.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T04:37:57.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T04:37:57.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:37:57.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:37:57.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T04:37:57.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T04:37:57.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T04:37:57.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T04:37:57.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:37:57.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:37:57.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T04:37:57.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T04:37:57.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:37:57.593 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T04:37:57.593 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T04:37:57.593 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T04:37:57.593 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T04:37:57.593 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T04:37:57.593 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T04:37:57.594 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T04:37:57.594 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T04:37:57.594 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T04:37:57.594 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T04:37:57.594 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T04:37:57.594 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T04:37:57.594 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T04:37:57.594 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T04:37:57.594 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T04:37:57.594 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T04:37:57.594 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T04:37:57.595 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T04:37:57.595 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T04:37:57.595 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T04:37:57.595 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T04:37:57.595 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T04:37:57.595 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T04:37:57.595 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T04:37:57.595 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T04:37:57.595 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T04:37:57.595 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T04:37:57.595 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T04:37:57.595 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T04:37:57.595 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T04:37:57.595 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T04:37:57.595 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T04:37:57.595 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T04:37:57.596 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.inplace_update_support: 0 2024-09-19T04:37:57.596 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T04:37:57.596 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T04:37:57.596 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T04:37:57.596 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T04:37:57.596 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.bloom_locality: 0 2024-09-19T04:37:57.596 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.max_successive_merges: 0 2024-09-19T04:37:57.596 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T04:37:57.596 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T04:37:57.596 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T04:37:57.596 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T04:37:57.596 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.ttl: 2592000 2024-09-19T04:37:57.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T04:37:57.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T04:37:57.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T04:37:57.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.enable_blob_files: false 2024-09-19T04:37:57.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.min_blob_size: 0 2024-09-19T04:37:57.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T04:37:57.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T04:37:57.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T04:37:57.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T04:37:57.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T04:37:57.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T04:37:57.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T04:37:57.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T04:37:57.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-19T04:37:57.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-19T04:37:57.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f5501a51-684c-4a33-960a-126cd3275fed 2024-09-19T04:37:57.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726720677531484, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-19T04:37:57.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-19T04:37:57.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726720677531887, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726720677, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f5501a51-684c-4a33-960a-126cd3275fed", "db_session_id": "L76R81B16ABW7HWQ4ZFV", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-19T04:37:57.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726720677531981, "job": 1, "event": "recovery_finished"} 2024-09-19T04:37:57.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-19T04:37:57.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T04:37:57.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55616c2c0e00 2024-09-19T04:37:57.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: DB pointer 0x55616c3bc000 2024-09-19T04:37:57.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T04:37:57.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T04:37:57.598 INFO:journalctl@ceph.mon.b.smithi183.stdout: ** DB Stats ** 2024-09-19T04:37:57.598 INFO:journalctl@ceph.mon.b.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:37:57.598 INFO:journalctl@ceph.mon.b.smithi183.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T04:37:57.598 INFO:journalctl@ceph.mon.b.smithi183.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:37:57.598 INFO:journalctl@ceph.mon.b.smithi183.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:37:57.599 INFO:journalctl@ceph.mon.b.smithi183.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T04:37:57.599 INFO:journalctl@ceph.mon.b.smithi183.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:37:57.599 INFO:journalctl@ceph.mon.b.smithi183.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:37:57.599 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-09-19T04:37:57.599 INFO:journalctl@ceph.mon.b.smithi183.stdout: ** Compaction Stats [default] ** 2024-09-19T04:37:57.599 INFO:journalctl@ceph.mon.b.smithi183.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:37:57.599 INFO:journalctl@ceph.mon.b.smithi183.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T04:37:57.599 INFO:journalctl@ceph.mon.b.smithi183.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T04:37:57.599 INFO:journalctl@ceph.mon.b.smithi183.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T04:37:57.599 INFO:journalctl@ceph.mon.b.smithi183.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T04:37:57.599 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-09-19T04:37:57.599 INFO:journalctl@ceph.mon.b.smithi183.stdout: ** Compaction Stats [default] ** 2024-09-19T04:37:57.599 INFO:journalctl@ceph.mon.b.smithi183.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:37:57.599 INFO:journalctl@ceph.mon.b.smithi183.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T04:37:57.599 INFO:journalctl@ceph.mon.b.smithi183.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T04:37:57.600 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-09-19T04:37:57.600 INFO:journalctl@ceph.mon.b.smithi183.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T04:37:57.600 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-09-19T04:37:57.600 INFO:journalctl@ceph.mon.b.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:37:57.600 INFO:journalctl@ceph.mon.b.smithi183.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T04:37:57.600 INFO:journalctl@ceph.mon.b.smithi183.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T04:37:57.601 INFO:journalctl@ceph.mon.b.smithi183.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T04:37:57.601 INFO:journalctl@ceph.mon.b.smithi183.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T04:37:57.601 INFO:journalctl@ceph.mon.b.smithi183.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T04:37:57.601 INFO:journalctl@ceph.mon.b.smithi183.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:37:57.601 INFO:journalctl@ceph.mon.b.smithi183.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:37:57.601 INFO:journalctl@ceph.mon.b.smithi183.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T04:37:57.601 INFO:journalctl@ceph.mon.b.smithi183.stdout: Block cache BinnedLRUCache@0x55616c2931f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-19T04:37:57.601 INFO:journalctl@ceph.mon.b.smithi183.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-19T04:37:57.601 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-09-19T04:37:57.601 INFO:journalctl@ceph.mon.b.smithi183.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T04:37:57.602 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-19T04:37:57.602 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: using public_addr v2:172.21.15.183:0/0 -> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2024-09-19T04:37:57.602 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: starting mon.b rank -1 at public addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] at bind addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:37:57.602 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.b@-1(???) e0 preinit fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:37:57.602 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.b@-1(synchronizing).mds e1 new map 2024-09-19T04:37:57.602 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-19T04:37:57.602 INFO:journalctl@ceph.mon.b.smithi183.stdout: e1 2024-09-19T04:37:57.602 INFO:journalctl@ceph.mon.b.smithi183.stdout: btime 2024-09-19T04:36:21:083029+0000 2024-09-19T04:37:57.602 INFO:journalctl@ceph.mon.b.smithi183.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T04:37:57.602 INFO:journalctl@ceph.mon.b.smithi183.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:37:57.602 INFO:journalctl@ceph.mon.b.smithi183.stdout: legacy client fscid: -1 2024-09-19T04:37:57.602 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-09-19T04:37:57.602 INFO:journalctl@ceph.mon.b.smithi183.stdout: No filesystems configured 2024-09-19T04:37:57.603 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T04:37:57.603 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T04:37:57.603 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T04:37:57.603 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T04:37:57.603 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T04:37:57.603 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T04:37:57.603 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T04:37:57.603 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:37:57.603 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:37:57.603 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:37:57.603 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:37:57.603 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:57.603 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.603 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.604 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.604 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.604 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:37:57.604 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:37:57.604 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:37:57.604 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-19T04:37:57.604 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.604 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.604 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.604 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: Updating smithi183:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:37:57.604 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: Deploying cephadm binary to smithi193 2024-09-19T04:37:57.604 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:37:57.604 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: Updating smithi183:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:37:57.605 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:57.605 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.605 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:37:57.605 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.605 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: Added host smithi193 2024-09-19T04:37:57.605 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.605 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:57.605 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.605 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:37:57.605 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:57.605 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/391524049' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T04:37:57.606 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.606 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.606 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.606 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.606 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:37:57.606 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:37:57.606 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:37:57.606 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-19T04:37:57.606 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: Updating smithi193:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:37:57.606 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/391524049' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T04:37:57.606 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:37:57.606 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.606 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.606 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.607 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:37:57.607 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: Updating smithi193:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:37:57.607 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:57.607 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.607 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:37:57.607 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:37:57.607 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:37:57.607 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.607 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:37:57.607 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:37:57.607 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi123:172.21.15.123=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:37:57.607 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: Saving service mon spec with placement smithi123:172.21.15.123=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c;count:3 2024-09-19T04:37:57.607 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: Deploying daemon mon.c on smithi193 2024-09-19T04:37:57.607 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:57.608 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:57.608 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='client.? 172.21.15.193:0/1396838459' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:37:57.608 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:37:57.608 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:57.608 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.a calling monitor election 2024-09-19T04:37:57.608 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:57.608 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:57.608 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:57.608 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.c calling monitor election 2024-09-19T04:37:57.608 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:57.608 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:57.608 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:57.608 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:57.608 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:57.608 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:37:57.609 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: monmap epoch 2 2024-09-19T04:37:57.609 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:37:57.609 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: last_changed 2024-09-19T04:37:48.656309+0000 2024-09-19T04:37:57.609 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: created 2024-09-19T04:36:17.413126+0000 2024-09-19T04:37:57.609 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: min_mon_release 19 (squid) 2024-09-19T04:37:57.609 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: election_strategy: 1 2024-09-19T04:37:57.609 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-09-19T04:37:57.609 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-19T04:37:57.609 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: fsmap 2024-09-19T04:37:57.609 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:37:57.609 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mgrmap e13: x(active, since 40s) 2024-09-19T04:37:57.609 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: overall HEALTH_OK 2024-09-19T04:37:57.609 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.609 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.609 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:37:57.609 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:37:57.610 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:37:57.610 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: Deploying daemon mon.b on smithi183 2024-09-19T04:37:57.610 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='client.? 172.21.15.193:0/1923972954' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:37:57.610 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:37:57.610 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:37:57.610 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: from='client.? 172.21.15.193:0/1717986167' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:37:57.610 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 ceph-mon[35333]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T04:37:57.610 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 bash[35314]: cb785711cbc83b992774f0c9a5c964122c89785c86d6de7b9b783517597bdda5 2024-09-19T04:37:57.610 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:37:57 smithi183 systemd[1]: Started Ceph mon.b for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:37:58.740 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:37:58.741 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:37:58.991 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:38:02.585 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.584+0000 7f6c314e3640 1 -- 172.21.15.193:0/974841984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c18005740 msgr2=0x7f6c18007bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:02.586 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.584+0000 7f6c314e3640 1 --2- 172.21.15.193:0/974841984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c18005740 0x7f6c18007bd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c1016a0 tx=0x7f6c2402bd70 comp rx=0 tx=0).stop 2024-09-19T04:38:02.586 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.584+0000 7f6c314e3640 1 -- 172.21.15.193:0/974841984 shutdown_connections 2024-09-19T04:38:02.586 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.584+0000 7f6c314e3640 1 --2- 172.21.15.193:0/974841984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c18005740 0x7f6c18007bd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:02.586 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.584+0000 7f6c314e3640 1 -- 172.21.15.193:0/974841984 >> 172.21.15.193:0/974841984 conn(0x7f6c2c0fc220 msgr2=0x7f6c2c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:38:02.586 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.584+0000 7f6c314e3640 1 -- 172.21.15.193:0/974841984 shutdown_connections 2024-09-19T04:38:02.586 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.584+0000 7f6c314e3640 1 -- 172.21.15.193:0/974841984 wait complete. 2024-09-19T04:38:02.588 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.585+0000 7f6c314e3640 1 Processor -- start 2024-09-19T04:38:02.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.585+0000 7f6c314e3640 1 -- start start 2024-09-19T04:38:02.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.585+0000 7f6c314e3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c18005740 0x7f6c2c1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:02.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.585+0000 7f6c314e3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c2c199b30 0x7f6c2c19dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:02.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.585+0000 7f6c314e3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c2c19e400 0x7f6c2c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:02.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.585+0000 7f6c314e3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c107550 con 0x7f6c2c19e400 2024-09-19T04:38:02.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.585+0000 7f6c314e3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1073d0 con 0x7f6c2c199b30 2024-09-19T04:38:02.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.585+0000 7f6c314e3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1076d0 con 0x7f6c18005740 2024-09-19T04:38:02.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c18005740 0x7f6c2c1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:02.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c18005740 0x7f6c2c1995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:35838/0 (socket says 172.21.15.193:35838) 2024-09-19T04:38:02.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2affd640 1 -- 172.21.15.193:0/1929202205 learned_addr learned my addr 172.21.15.193:0/1929202205 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-19T04:38:02.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2affd640 1 -- 172.21.15.193:0/1929202205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c18005740 msgr2=0x7f6c2c1995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T04:38:02.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2affd640 1 -- 172.21.15.193:0/1929202205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c18005740 msgr2=0x7f6c2c1995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:38:02.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2affd640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c18005740 0x7f6c2c1995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:38:02.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2a7fc640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c2c199b30 0x7f6c2c19dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:02.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2affd640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c18005740 0x7f6c2c1995f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:38:02.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2b7fe640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c2c19e400 0x7f6c2c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:02.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2a7fc640 1 -- 172.21.15.193:0/1929202205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c2c199b30 msgr2=0x7f6c2c19dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T04:38:02.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2a7fc640 1 -- 172.21.15.193:0/1929202205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c2c199b30 msgr2=0x7f6c2c19dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:38:02.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2a7fc640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c2c199b30 0x7f6c2c19dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:38:02.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2a7fc640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c2c199b30 0x7f6c2c19dec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:38:02.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2b7fe640 1 -- 172.21.15.193:0/1929202205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c18005740 msgr2=0x7f6c2c1995f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:38:02.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2b7fe640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c18005740 0x7f6c2c1995f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:02.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2b7fe640 1 -- 172.21.15.193:0/1929202205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c2c199b30 msgr2=0x7f6c2c19dec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:38:02.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2b7fe640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c2c199b30 0x7f6c2c19dec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:02.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.586+0000 7f6c2b7fe640 1 -- 172.21.15.193:0/1929202205 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2c1a6170 con 0x7f6c2c19e400 2024-09-19T04:38:02.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.587+0000 7f6c2b7fe640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c2c19e400 0x7f6c2c19e8b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009fd0 tx=0x7f6c1c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:38:02.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.587+0000 7f6c23fff640 1 -- 172.21.15.193:0/1929202205 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c015020 con 0x7f6c2c19e400 2024-09-19T04:38:02.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.587+0000 7f6c23fff640 1 -- 172.21.15.193:0/1929202205 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c1c011d10 con 0x7f6c2c19e400 2024-09-19T04:38:02.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.587+0000 7f6c23fff640 1 -- 172.21.15.193:0/1929202205 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c00f2e0 con 0x7f6c2c19e400 2024-09-19T04:38:02.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.587+0000 7f6c314e3640 1 -- 172.21.15.193:0/1929202205 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2c1a63a0 con 0x7f6c2c19e400 2024-09-19T04:38:02.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.587+0000 7f6c314e3640 1 -- 172.21.15.193:0/1929202205 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2c1a68b0 con 0x7f6c2c19e400 2024-09-19T04:38:02.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.588+0000 7f6c314e3640 1 -- 172.21.15.193:0/1929202205 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c2c108710 con 0x7f6c2c19e400 2024-09-19T04:38:02.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.589+0000 7f6c23fff640 1 -- 172.21.15.193:0/1929202205 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f6c1c018020 con 0x7f6c2c19e400 2024-09-19T04:38:02.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.589+0000 7f6c23fff640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6c0c039730 0x7f6c0c03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:02.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.589+0000 7f6c23fff640 1 -- 172.21.15.193:0/1929202205 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6c1c04bf50 con 0x7f6c2c19e400 2024-09-19T04:38:02.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.589+0000 7f6c2affd640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6c0c039730 0x7f6c0c03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:02.593 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.590+0000 7f6c2affd640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6c0c039730 0x7f6c0c03bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009b10 tx=0x7f6c24011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:38:02.596 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.594+0000 7f6c23fff640 1 -- 172.21.15.193:0/1929202205 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6c1c01f020 con 0x7f6c2c19e400 2024-09-19T04:38:02.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.833+0000 7f6c314e3640 1 -- 172.21.15.193:0/1929202205 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6c2c104930 con 0x7f6c2c19e400 2024-09-19T04:38:02.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.834+0000 7f6c23fff640 1 -- 172.21.15.193:0/1929202205 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f6c1c017b00 con 0x7f6c2c19e400 2024-09-19T04:38:02.835 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:38:02.836 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":3,"fsid":"911fa052-7640-11ef-bceb-c7b262605968","modified":"2024-09-19T04:37:57.556731Z","created":"2024-09-19T04:36:17.413126Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T04:38:02.836 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 3 2024-09-19T04:38:02.836 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:38:02.836 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:02.836 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: mon.a calling monitor election 2024-09-19T04:38:02.836 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: mon.c calling monitor election 2024-09-19T04:38:02.836 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:38:02.836 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:02.837 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:02.837 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:02.837 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:02.837 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:02.837 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:02.837 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:02.837 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:38:02.837 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: monmap epoch 3 2024-09-19T04:38:02.837 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:38:02.837 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: last_changed 2024-09-19T04:37:57.556731+0000 2024-09-19T04:38:02.837 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: created 2024-09-19T04:36:17.413126+0000 2024-09-19T04:38:02.837 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: min_mon_release 19 (squid) 2024-09-19T04:38:02.837 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: election_strategy: 1 2024-09-19T04:38:02.838 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-09-19T04:38:02.838 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-19T04:38:02.838 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-19T04:38:02.838 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: fsmap 2024-09-19T04:38:02.838 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:38:02.838 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: mgrmap e13: x(active, since 49s) 2024-09-19T04:38:02.838 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: overall HEALTH_OK 2024-09-19T04:38:02.838 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:02.838 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:02.838 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:02.838 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:02.839 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.836+0000 7f6c314e3640 1 -- 172.21.15.193:0/1929202205 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6c0c039730 msgr2=0x7f6c0c03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:02.839 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.836+0000 7f6c314e3640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6c0c039730 0x7f6c0c03bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009b10 tx=0x7f6c24011040 comp rx=0 tx=0).stop 2024-09-19T04:38:02.839 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.836+0000 7f6c314e3640 1 -- 172.21.15.193:0/1929202205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c2c19e400 msgr2=0x7f6c2c19e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:02.839 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.836+0000 7f6c314e3640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c2c19e400 0x7f6c2c19e8b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009fd0 tx=0x7f6c1c00b870 comp rx=0 tx=0).stop 2024-09-19T04:38:02.839 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.836+0000 7f6c314e3640 1 -- 172.21.15.193:0/1929202205 shutdown_connections 2024-09-19T04:38:02.839 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.836+0000 7f6c314e3640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6c0c039730 0x7f6c0c03bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:02.839 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.836+0000 7f6c314e3640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c2c19e400 0x7f6c2c19e8b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:02.839 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.836+0000 7f6c314e3640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c2c199b30 0x7f6c2c19dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:02.839 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.836+0000 7f6c314e3640 1 --2- 172.21.15.193:0/1929202205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c18005740 0x7f6c2c1995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:02.839 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.836+0000 7f6c314e3640 1 -- 172.21.15.193:0/1929202205 >> 172.21.15.193:0/1929202205 conn(0x7f6c2c0fc220 msgr2=0x7f6c2c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:38:02.840 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.836+0000 7f6c314e3640 1 -- 172.21.15.193:0/1929202205 shutdown_connections 2024-09-19T04:38:02.840 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:02.836+0000 7f6c314e3640 1 -- 172.21.15.193:0/1929202205 wait complete. 2024-09-19T04:38:02.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:38:02.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:02.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: mon.a calling monitor election 2024-09-19T04:38:02.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: mon.c calling monitor election 2024-09-19T04:38:02.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:38:02.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:02.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:02.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:02.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:02.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:02.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:02.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:02.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:38:02.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: monmap epoch 3 2024-09-19T04:38:02.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:38:02.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: last_changed 2024-09-19T04:37:57.556731+0000 2024-09-19T04:38:02.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: created 2024-09-19T04:36:17.413126+0000 2024-09-19T04:38:02.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: min_mon_release 19 (squid) 2024-09-19T04:38:02.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: election_strategy: 1 2024-09-19T04:38:02.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-09-19T04:38:02.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-19T04:38:02.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-19T04:38:02.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: fsmap 2024-09-19T04:38:02.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:38:02.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: mgrmap e13: x(active, since 49s) 2024-09-19T04:38:02.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: overall HEALTH_OK 2024-09-19T04:38:02.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:02.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:02.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:02.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:03.501 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T04:38:03.501 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T04:38:03.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:03 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:03.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:03 smithi193 ceph-mon[35592]: from='client.? 172.21.15.193:0/1929202205' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:38:03.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:03 smithi193 ceph-mon[35592]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:03.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:03 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:03.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:03 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:03.871 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:38:03.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:03 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:03.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:03 smithi123 ceph-mon[30511]: from='client.? 172.21.15.193:0/1929202205' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:38:03.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:03 smithi123 ceph-mon[30511]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:03.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:03 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:03.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:03 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:04.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.344+0000 7f92c7200640 1 -- 172.21.15.123:0/1478776790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c8074590 msgr2=0x7f92c8074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:04.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.344+0000 7f92c7200640 1 --2- 172.21.15.123:0/1478776790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c8074590 0x7f92c8074970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009880 tx=0x7f92b802a670 comp rx=0 tx=0).stop 2024-09-19T04:38:04.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.344+0000 7f92c7200640 1 -- 172.21.15.123:0/1478776790 shutdown_connections 2024-09-19T04:38:04.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.344+0000 7f92c7200640 1 --2- 172.21.15.123:0/1478776790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c8074590 0x7f92c8074970 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:04.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.344+0000 7f92c7200640 1 -- 172.21.15.123:0/1478776790 >> 172.21.15.123:0/1478776790 conn(0x7f92c80fd0a0 msgr2=0x7f92c80ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:38:04.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.344+0000 7f92c7200640 1 -- 172.21.15.123:0/1478776790 shutdown_connections 2024-09-19T04:38:04.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.344+0000 7f92c7200640 1 -- 172.21.15.123:0/1478776790 wait complete. 2024-09-19T04:38:04.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.345+0000 7f92c7200640 1 Processor -- start 2024-09-19T04:38:04.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.345+0000 7f92c7200640 1 -- start start 2024-09-19T04:38:04.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.346+0000 7f92c7200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c8074590 0x7f92c81a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:04.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.346+0000 7f92c7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c81a3ae0 0x7f92c819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:04.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.346+0000 7f92c7200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92c819dbb0 0x7f92c819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:04.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.346+0000 7f92c7200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92c81120c0 con 0x7f92c81a3ae0 2024-09-19T04:38:04.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.346+0000 7f92c7200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92c8111f40 con 0x7f92c819dbb0 2024-09-19T04:38:04.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.346+0000 7f92c7200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f92c8112240 con 0x7f92c8074590 2024-09-19T04:38:04.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.346+0000 7f92c5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c81a3ae0 0x7f92c819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:04.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.346+0000 7f92c5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c8074590 0x7f92c81a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:04.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.346+0000 7f92c5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c8074590 0x7f92c81a35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:59744/0 (socket says 172.21.15.123:59744) 2024-09-19T04:38:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.346+0000 7f92c5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c81a3ae0 0x7f92c819d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37664/0 (socket says 172.21.15.123:37664) 2024-09-19T04:38:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.346+0000 7f92c5e00640 1 -- 172.21.15.123:0/3100350923 learned_addr learned my addr 172.21.15.123:0/3100350923 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:38:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.346+0000 7f92c6800640 1 --2- 172.21.15.123:0/3100350923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92c819dbb0 0x7f92c819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.347+0000 7f92c6800640 1 -- 172.21.15.123:0/3100350923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92c819dbb0 msgr2=0x7f92c819e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-19T04:38:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.347+0000 7f92c6800640 1 -- 172.21.15.123:0/3100350923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92c819dbb0 msgr2=0x7f92c819e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:38:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.347+0000 7f92c6800640 1 --2- 172.21.15.123:0/3100350923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92c819dbb0 0x7f92c819e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:38:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.347+0000 7f92c6800640 1 --2- 172.21.15.123:0/3100350923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92c819dbb0 0x7f92c819e060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:38:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.347+0000 7f92c5e00640 1 -- 172.21.15.123:0/3100350923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92c819dbb0 msgr2=0x7f92c819e060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:38:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.347+0000 7f92c5e00640 1 --2- 172.21.15.123:0/3100350923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92c819dbb0 0x7f92c819e060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.347+0000 7f92c5e00640 1 -- 172.21.15.123:0/3100350923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c81a3ae0 msgr2=0x7f92c819d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.347+0000 7f92c5e00640 1 --2- 172.21.15.123:0/3100350923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c81a3ae0 0x7f92c819d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.347+0000 7f92c5e00640 1 -- 172.21.15.123:0/3100350923 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c81aa350 con 0x7f92c8074590 2024-09-19T04:38:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.347+0000 7f92c5e00640 1 --2- 172.21.15.123:0/3100350923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c8074590 0x7f92c81a35a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f92b8006010 tx=0x7f92b8003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:38:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.347+0000 7f92bea00640 1 -- 172.21.15.123:0/3100350923 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b803f070 con 0x7f92c8074590 2024-09-19T04:38:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.348+0000 7f92bea00640 1 -- 172.21.15.123:0/3100350923 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92b803b460 con 0x7f92c8074590 2024-09-19T04:38:04.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.348+0000 7f92bea00640 1 -- 172.21.15.123:0/3100350923 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8035030 con 0x7f92c8074590 2024-09-19T04:38:04.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.348+0000 7f92c7200640 1 -- 172.21.15.123:0/3100350923 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c81aa580 con 0x7f92c8074590 2024-09-19T04:38:04.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.348+0000 7f92c7200640 1 -- 172.21.15.123:0/3100350923 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c81aa9e0 con 0x7f92c8074590 2024-09-19T04:38:04.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.349+0000 7f92c7200640 1 -- 172.21.15.123:0/3100350923 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92c8075cd0 con 0x7f92c8074590 2024-09-19T04:38:04.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.349+0000 7f92bea00640 1 -- 172.21.15.123:0/3100350923 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f92b8042070 con 0x7f92c8074590 2024-09-19T04:38:04.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.349+0000 7f92bea00640 1 --2- 172.21.15.123:0/3100350923 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f92ac039730 0x7f92ac03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:04.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.350+0000 7f92bea00640 1 -- 172.21.15.123:0/3100350923 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f92b806cff0 con 0x7f92c8074590 2024-09-19T04:38:04.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.350+0000 7f92c5400640 1 --2- 172.21.15.123:0/3100350923 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f92ac039730 0x7f92ac03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:04.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.350+0000 7f92c5400640 1 --2- 172.21.15.123:0/3100350923 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f92ac039730 0x7f92ac03bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f92b4009a30 tx=0x7f92b4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:38:04.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.354+0000 7f92bea00640 1 -- 172.21.15.123:0/3100350923 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f92b8040260 con 0x7f92c8074590 2024-09-19T04:38:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.540+0000 7f92c7200640 1 -- 172.21.15.123:0/3100350923 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f92c81aac70 con 0x7f92c8074590 2024-09-19T04:38:04.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.540+0000 7f92bea00640 1 -- 172.21.15.123:0/3100350923 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f92b803b9a0 con 0x7f92c8074590 2024-09-19T04:38:04.542 INFO:teuthology.orchestra.run.smithi123.stdout:# minimal ceph.conf for 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:38:04.542 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2024-09-19T04:38:04.542 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:38:04.542 INFO:teuthology.orchestra.run.smithi123.stdout: mon_host = [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-09-19T04:38:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.543+0000 7f92c7200640 1 -- 172.21.15.123:0/3100350923 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f92ac039730 msgr2=0x7f92ac03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.543+0000 7f92c7200640 1 --2- 172.21.15.123:0/3100350923 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f92ac039730 0x7f92ac03bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f92b4009a30 tx=0x7f92b4009190 comp rx=0 tx=0).stop 2024-09-19T04:38:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.543+0000 7f92c7200640 1 -- 172.21.15.123:0/3100350923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c8074590 msgr2=0x7f92c81a35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.543+0000 7f92c7200640 1 --2- 172.21.15.123:0/3100350923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c8074590 0x7f92c81a35a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f92b8006010 tx=0x7f92b8003c00 comp rx=0 tx=0).stop 2024-09-19T04:38:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.543+0000 7f92c7200640 1 -- 172.21.15.123:0/3100350923 shutdown_connections 2024-09-19T04:38:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.543+0000 7f92c7200640 1 --2- 172.21.15.123:0/3100350923 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f92ac039730 0x7f92ac03bbf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.543+0000 7f92c7200640 1 --2- 172.21.15.123:0/3100350923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92c819dbb0 0x7f92c819e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.543+0000 7f92c7200640 1 --2- 172.21.15.123:0/3100350923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c81a3ae0 0x7f92c819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.543+0000 7f92c7200640 1 --2- 172.21.15.123:0/3100350923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c8074590 0x7f92c81a35a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.543+0000 7f92c7200640 1 -- 172.21.15.123:0/3100350923 >> 172.21.15.123:0/3100350923 conn(0x7f92c80fd0a0 msgr2=0x7f92c80fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:38:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.544+0000 7f92c7200640 1 -- 172.21.15.123:0/3100350923 shutdown_connections 2024-09-19T04:38:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:04.544+0000 7f92c7200640 1 -- 172.21.15.123:0/3100350923 wait complete. 2024-09-19T04:38:04.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:38:04.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:04.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: mon.a calling monitor election 2024-09-19T04:38:04.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: mon.c calling monitor election 2024-09-19T04:38:04.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:38:04.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:04.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:04.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:04.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:04.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:04.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:04.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:04.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:38:04.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: monmap epoch 3 2024-09-19T04:38:04.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:38:04.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: last_changed 2024-09-19T04:37:57.556731+0000 2024-09-19T04:38:04.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: created 2024-09-19T04:36:17.413126+0000 2024-09-19T04:38:04.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: min_mon_release 19 (squid) 2024-09-19T04:38:04.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: election_strategy: 1 2024-09-19T04:38:04.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-09-19T04:38:04.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-19T04:38:04.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-19T04:38:04.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: fsmap 2024-09-19T04:38:04.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:38:04.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: mgrmap e13: x(active, since 49s) 2024-09-19T04:38:04.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: overall HEALTH_OK 2024-09-19T04:38:04.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:04.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:04.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:04.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:04.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:04.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: from='client.? 172.21.15.193:0/1929202205' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:38:04.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:04.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:04.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:05.277 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T04:38:05.277 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:38:05.277 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T04:38:05.335 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:38:05.335 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:38:05.431 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: mon.b calling monitor election 2024-09-19T04:38:05.431 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: mon.b calling monitor election 2024-09-19T04:38:05.431 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: mon.c calling monitor election 2024-09-19T04:38:05.431 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: mon.a calling monitor election 2024-09-19T04:38:05.431 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T04:38:05.431 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: monmap epoch 3 2024-09-19T04:38:05.432 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:38:05.432 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: last_changed 2024-09-19T04:37:57.556731+0000 2024-09-19T04:38:05.432 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: created 2024-09-19T04:36:17.413126+0000 2024-09-19T04:38:05.432 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: min_mon_release 19 (squid) 2024-09-19T04:38:05.432 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: election_strategy: 1 2024-09-19T04:38:05.432 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-09-19T04:38:05.432 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-19T04:38:05.432 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-19T04:38:05.432 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: fsmap 2024-09-19T04:38:05.432 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:38:05.432 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: mgrmap e13: x(active, since 51s) 2024-09-19T04:38:05.432 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: overall HEALTH_OK 2024-09-19T04:38:05.432 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.433 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.433 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.433 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.433 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.433 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.433 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:05 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.434 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-19T04:38:05.434 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T04:38:05.462 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-19T04:38:05.462 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:38:05.528 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-19T04:38:05.528 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T04:38:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: mon.b calling monitor election 2024-09-19T04:38:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: mon.b calling monitor election 2024-09-19T04:38:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: mon.c calling monitor election 2024-09-19T04:38:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: mon.a calling monitor election 2024-09-19T04:38:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T04:38:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: monmap epoch 3 2024-09-19T04:38:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:38:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: last_changed 2024-09-19T04:37:57.556731+0000 2024-09-19T04:38:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: created 2024-09-19T04:36:17.413126+0000 2024-09-19T04:38:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: min_mon_release 19 (squid) 2024-09-19T04:38:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: election_strategy: 1 2024-09-19T04:38:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-09-19T04:38:05.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-19T04:38:05.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-19T04:38:05.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: fsmap 2024-09-19T04:38:05.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:38:05.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: mgrmap e13: x(active, since 51s) 2024-09-19T04:38:05.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: overall HEALTH_OK 2024-09-19T04:38:05.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:05 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.558 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-19T04:38:05.558 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:38:05.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: mon.b calling monitor election 2024-09-19T04:38:05.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: mon.b calling monitor election 2024-09-19T04:38:05.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: mon.c calling monitor election 2024-09-19T04:38:05.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: mon.a calling monitor election 2024-09-19T04:38:05.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T04:38:05.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: monmap epoch 3 2024-09-19T04:38:05.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:38:05.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: last_changed 2024-09-19T04:37:57.556731+0000 2024-09-19T04:38:05.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: created 2024-09-19T04:36:17.413126+0000 2024-09-19T04:38:05.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: min_mon_release 19 (squid) 2024-09-19T04:38:05.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: election_strategy: 1 2024-09-19T04:38:05.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-09-19T04:38:05.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-19T04:38:05.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-19T04:38:05.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: fsmap 2024-09-19T04:38:05.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:38:05.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: mgrmap e13: x(active, since 51s) 2024-09-19T04:38:05.602 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: overall HEALTH_OK 2024-09-19T04:38:05.602 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.602 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.602 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.602 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.602 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.602 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.602 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:05 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:05.626 INFO:tasks.cephadm:Adding mgr.x on smithi123 2024-09-19T04:38:05.626 INFO:tasks.cephadm:Adding mgr.y on smithi183 2024-09-19T04:38:05.626 INFO:tasks.cephadm:Adding mgr.z on smithi193 2024-09-19T04:38:05.626 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi123=x;smithi183=y;smithi193=z' 2024-09-19T04:38:05.906 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:38:06.440 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.436+0000 7f04002bf640 1 -- 172.21.15.193:0/896999293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03f8102cc0 msgr2=0x7f03f81030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:06.440 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.436+0000 7f04002bf640 1 --2- 172.21.15.193:0/896999293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03f8102cc0 0x7f03f81030c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009880 tx=0x7f03f402a5d0 comp rx=0 tx=0).stop 2024-09-19T04:38:06.440 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.437+0000 7f04002bf640 1 -- 172.21.15.193:0/896999293 shutdown_connections 2024-09-19T04:38:06.440 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.437+0000 7f04002bf640 1 --2- 172.21.15.193:0/896999293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03f8102cc0 0x7f03f81030c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:06.441 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.437+0000 7f04002bf640 1 -- 172.21.15.193:0/896999293 >> 172.21.15.193:0/896999293 conn(0x7f03f80fb510 msgr2=0x7f03f80fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:38:06.441 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.437+0000 7f04002bf640 1 -- 172.21.15.193:0/896999293 shutdown_connections 2024-09-19T04:38:06.441 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.437+0000 7f04002bf640 1 -- 172.21.15.193:0/896999293 wait complete. 2024-09-19T04:38:06.441 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.437+0000 7f04002bf640 1 Processor -- start 2024-09-19T04:38:06.441 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.437+0000 7f04002bf640 1 -- start start 2024-09-19T04:38:06.441 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.437+0000 7f04002bf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03f8102cc0 0x7f03f81023c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:06.441 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.437+0000 7f04002bf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03f8100a10 0x7f03f8100e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:06.441 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.437+0000 7f04002bf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03f81013d0 0x7f03f81a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:06.441 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.437+0000 7f04002bf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03f81077a0 con 0x7f03f8100a10 2024-09-19T04:38:06.441 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.437+0000 7f04002bf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03f8107620 con 0x7f03f8102cc0 2024-09-19T04:38:06.441 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.437+0000 7f04002bf640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f03f8107920 con 0x7f03f81013d0 2024-09-19T04:38:06.442 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.438+0000 7f03fe85c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03f81013d0 0x7f03f81a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:06.442 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.438+0000 7f03fe85c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03f81013d0 0x7f03f81a1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:35846/0 (socket says 172.21.15.193:35846) 2024-09-19T04:38:06.442 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.438+0000 7f03fe85c640 1 -- 172.21.15.193:0/2137893234 learned_addr learned my addr 172.21.15.193:0/2137893234 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-19T04:38:06.442 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.438+0000 7f03fd85a640 1 --2- 172.21.15.193:0/2137893234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03f8100a10 0x7f03f8100e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:06.442 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.438+0000 7f03fe05b640 1 --2- 172.21.15.193:0/2137893234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03f8102cc0 0x7f03f81023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:06.442 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.438+0000 7f03fe85c640 1 -- 172.21.15.193:0/2137893234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03f8102cc0 msgr2=0x7f03f81023c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:06.442 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.438+0000 7f03fe85c640 1 --2- 172.21.15.193:0/2137893234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03f8102cc0 0x7f03f81023c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:06.442 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.438+0000 7f03fe85c640 1 -- 172.21.15.193:0/2137893234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03f8100a10 msgr2=0x7f03f8100e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:06.442 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.438+0000 7f03fe85c640 1 --2- 172.21.15.193:0/2137893234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03f8100a10 0x7f03f8100e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:06.442 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.438+0000 7f03fe85c640 1 -- 172.21.15.193:0/2137893234 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f81a15a0 con 0x7f03f81013d0 2024-09-19T04:38:06.443 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.438+0000 7f03fe85c640 1 --2- 172.21.15.193:0/2137893234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03f81013d0 0x7f03f81a1010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f03f000ba50 tx=0x7f03f000bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:38:06.443 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.438+0000 7f03ef7fe640 1 -- 172.21.15.193:0/2137893234 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0019070 con 0x7f03f81013d0 2024-09-19T04:38:06.443 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.438+0000 7f03ef7fe640 1 -- 172.21.15.193:0/2137893234 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f000cbe0 con 0x7f03f81013d0 2024-09-19T04:38:06.443 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.438+0000 7f04002bf640 1 -- 172.21.15.193:0/2137893234 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f81a1890 con 0x7f03f81013d0 2024-09-19T04:38:06.443 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.438+0000 7f03ef7fe640 1 -- 172.21.15.193:0/2137893234 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f00076a0 con 0x7f03f81013d0 2024-09-19T04:38:06.443 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.438+0000 7f04002bf640 1 -- 172.21.15.193:0/2137893234 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f81a1e00 con 0x7f03f81013d0 2024-09-19T04:38:06.444 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.439+0000 7f03ef7fe640 1 -- 172.21.15.193:0/2137893234 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f03f001c070 con 0x7f03f81013d0 2024-09-19T04:38:06.444 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.439+0000 7f03ef7fe640 1 --2- 172.21.15.193:0/2137893234 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f03e4039730 0x7f03e403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:06.444 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.440+0000 7f03ef7fe640 1 -- 172.21.15.193:0/2137893234 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f03f004bdc0 con 0x7f03f81013d0 2024-09-19T04:38:06.444 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.439+0000 7f04002bf640 1 -- 172.21.15.193:0/2137893234 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f8190560 con 0x7f03f81013d0 2024-09-19T04:38:06.444 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.440+0000 7f03fe05b640 1 --2- 172.21.15.193:0/2137893234 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f03e4039730 0x7f03e403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:06.445 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.440+0000 7f03fe05b640 1 --2- 172.21.15.193:0/2137893234 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f03e4039730 0x7f03e403bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009230 tx=0x7f03f40315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:38:06.445 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.444+0000 7f03ef7fe640 1 -- 172.21.15.193:0/2137893234 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f03f000e0a0 con 0x7f03f81013d0 2024-09-19T04:38:06.559 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:06 smithi123 ceph-mon[30511]: Updating smithi123:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:38:06.559 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:06 smithi123 ceph-mon[30511]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:06.559 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:06.559 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:06.559 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:06.559 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:06.560 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:06 smithi123 ceph-mon[30511]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T04:38:06.560 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:38:06.560 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:38:06.560 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:06.560 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:06 smithi123 ceph-mon[30511]: Reconfiguring daemon mon.a on smithi123 2024-09-19T04:38:06.560 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:06.624 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.622+0000 7f04002bf640 1 -- 172.21.15.193:0/2137893234 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi123=x;smithi183=y;smithi193=z", "target": ["mon-mgr", ""]}) -- 0x7f03f80626c0 con 0x7f03e4039730 2024-09-19T04:38:06.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:06 smithi193 ceph-mon[35592]: Updating smithi123:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:38:06.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:06 smithi193 ceph-mon[35592]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:06.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:06.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:06.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:06.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:06.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:06 smithi193 ceph-mon[35592]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T04:38:06.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:38:06.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:38:06.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:06.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:06 smithi193 ceph-mon[35592]: Reconfiguring daemon mon.a on smithi123 2024-09-19T04:38:06.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:06.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.628+0000 7f03ef7fe640 1 -- 172.21.15.193:0/2137893234 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f03f80626c0 con 0x7f03e4039730 2024-09-19T04:38:06.630 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mgr update... 2024-09-19T04:38:06.632 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.630+0000 7f04002bf640 1 -- 172.21.15.193:0/2137893234 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f03e4039730 msgr2=0x7f03e403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:06.632 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.630+0000 7f04002bf640 1 --2- 172.21.15.193:0/2137893234 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f03e4039730 0x7f03e403bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009230 tx=0x7f03f40315d0 comp rx=0 tx=0).stop 2024-09-19T04:38:06.632 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.630+0000 7f04002bf640 1 -- 172.21.15.193:0/2137893234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03f81013d0 msgr2=0x7f03f81a1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:06.632 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.630+0000 7f04002bf640 1 --2- 172.21.15.193:0/2137893234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03f81013d0 0x7f03f81a1010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f03f000ba50 tx=0x7f03f000bf10 comp rx=0 tx=0).stop 2024-09-19T04:38:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.631+0000 7f04002bf640 1 -- 172.21.15.193:0/2137893234 shutdown_connections 2024-09-19T04:38:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.631+0000 7f04002bf640 1 --2- 172.21.15.193:0/2137893234 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f03e4039730 0x7f03e403bbf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.631+0000 7f04002bf640 1 --2- 172.21.15.193:0/2137893234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03f81013d0 0x7f03f81a1010 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.631+0000 7f04002bf640 1 --2- 172.21.15.193:0/2137893234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03f8100a10 0x7f03f8100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.631+0000 7f04002bf640 1 --2- 172.21.15.193:0/2137893234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03f8102cc0 0x7f03f81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.631+0000 7f04002bf640 1 -- 172.21.15.193:0/2137893234 >> 172.21.15.193:0/2137893234 conn(0x7f03f80fb510 msgr2=0x7f03f80fc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:38:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.631+0000 7f04002bf640 1 -- 172.21.15.193:0/2137893234 shutdown_connections 2024-09-19T04:38:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:38:06.631+0000 7f04002bf640 1 -- 172.21.15.193:0/2137893234 wait complete. 2024-09-19T04:38:06.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:06 smithi183 ceph-mon[35333]: Updating smithi123:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:38:06.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:06 smithi183 ceph-mon[35333]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:06.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:06.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:06.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:06 smithi183 ceph-mon[35333]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T04:38:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:38:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:38:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:06 smithi183 ceph-mon[35333]: Reconfiguring daemon mon.a on smithi123 2024-09-19T04:38:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:38:06.926 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:38:06 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:38:06.556+0000 7f8da6a00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-19T04:38:07.213 DEBUG:teuthology.orchestra.run.smithi183:mgr.y> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mgr.y.service 2024-09-19T04:38:07.216 DEBUG:teuthology.orchestra.run.smithi193:mgr.z> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mgr.z.service 2024-09-19T04:38:07.219 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T04:38:07.219 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:38:07.219 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T04:38:07.256 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T04:38:07.256 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-09-19T04:38:07.332 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T04:38:07.332 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:38:07.332 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-09-19T04:38:07.332 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:38:07.332 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:38:07.332 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-19 04:37:26.411690913 +0000 2024-09-19T04:38:07.332 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-19 04:32:39.383652967 +0000 2024-09-19T04:38:07.332 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-19 04:32:39.383652967 +0000 2024-09-19T04:38:07.332 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-19 04:32:39.383652967 +0000 2024-09-19T04:38:07.333 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T04:38:07.430 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-19T04:38:07.430 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-19T04:38:07.430 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.00027914 s, 1.8 MB/s 2024-09-19T04:38:07.433 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T04:38:07.470 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-09-19T04:38:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T04:38:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:38:07.548 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-09-19T04:38:07.548 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:38:07.548 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:38:07.548 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-19 04:37:26.449689974 +0000 2024-09-19T04:38:07.548 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-19 04:32:39.837641704 +0000 2024-09-19T04:38:07.548 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-19 04:32:39.837641704 +0000 2024-09-19T04:38:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-19 04:32:39.837641704 +0000 2024-09-19T04:38:07.549 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T04:38:07.643 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-19T04:38:07.643 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-19T04:38:07.643 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000289051 s, 1.8 MB/s 2024-09-19T04:38:07.645 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T04:38:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:07 smithi123 ceph-mon[30511]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi123=x;smithi183=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:38:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:07 smithi123 ceph-mon[30511]: Saving service mgr spec with placement smithi123=x;smithi183=y;smithi193=z;count:3 2024-09-19T04:38:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:07 smithi123 ceph-mon[30511]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T04:38:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:38:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:38:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:07.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:07 smithi123 ceph-mon[30511]: Reconfiguring daemon mgr.x on smithi123 2024-09-19T04:38:07.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:07 smithi123 ceph-mon[30511]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:07.682 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-09-19T04:38:07.758 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T04:38:07.759 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:38:07.759 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-09-19T04:38:07.759 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:38:07.759 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:38:07.759 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-19 04:37:26.486689060 +0000 2024-09-19T04:38:07.759 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-19 04:32:40.299630242 +0000 2024-09-19T04:38:07.759 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-19 04:32:40.299630242 +0000 2024-09-19T04:38:07.759 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-19 04:32:40.299630242 +0000 2024-09-19T04:38:07.759 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T04:38:07.854 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-19T04:38:07.855 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-19T04:38:07.855 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000351542 s, 1.5 MB/s 2024-09-19T04:38:07.858 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T04:38:07.894 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-09-19T04:38:07.971 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T04:38:07.971 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:38:07.971 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-09-19T04:38:07.971 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:38:07.971 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:38:07.971 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-19 04:37:26.524688121 +0000 2024-09-19T04:38:07.971 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-19 04:32:40.768618606 +0000 2024-09-19T04:38:07.971 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-19 04:32:40.768618606 +0000 2024-09-19T04:38:07.971 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-19 04:32:40.768618606 +0000 2024-09-19T04:38:07.972 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T04:38:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:07 smithi193 ceph-mon[35592]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi123=x;smithi183=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:38:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:07 smithi193 ceph-mon[35592]: Saving service mgr spec with placement smithi123=x;smithi183=y;smithi193=z;count:3 2024-09-19T04:38:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:07 smithi193 ceph-mon[35592]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T04:38:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:38:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:38:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:07 smithi193 ceph-mon[35592]: Reconfiguring daemon mgr.x on smithi123 2024-09-19T04:38:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:07 smithi193 ceph-mon[35592]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:08.064 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-19T04:38:08.064 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-19T04:38:08.064 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000282829 s, 1.8 MB/s 2024-09-19T04:38:08.067 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T04:38:08.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:07 smithi183 ceph-mon[35333]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi123=x;smithi183=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:38:08.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:07 smithi183 ceph-mon[35333]: Saving service mgr spec with placement smithi123=x;smithi183=y;smithi193=z;count:3 2024-09-19T04:38:08.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:08.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:08.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:08.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:07 smithi183 ceph-mon[35333]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T04:38:08.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:38:08.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:38:08.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:08.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:07 smithi183 ceph-mon[35333]: Reconfiguring daemon mgr.x on smithi123 2024-09-19T04:38:08.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:07 smithi183 ceph-mon[35333]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:08.103 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-19T04:38:08.103 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T04:38:08.123 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T04:38:08.123 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-09-19T04:38:08.181 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T04:38:08.182 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:38:08.182 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T04:38:08.182 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:38:08.182 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:38:08.182 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-19 04:37:56.985200789 +0000 2024-09-19T04:38:08.183 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-19 04:32:12.556825280 +0000 2024-09-19T04:38:08.183 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-19 04:32:12.556825280 +0000 2024-09-19T04:38:08.183 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-19 04:32:12.556825280 +0000 2024-09-19T04:38:08.183 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T04:38:08.249 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-19T04:38:08.249 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-19T04:38:08.249 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000222253 s, 2.3 MB/s 2024-09-19T04:38:08.251 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T04:38:08.308 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-09-19T04:38:08.365 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T04:38:08.365 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:38:08.365 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-19T04:38:08.365 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:38:08.365 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:38:08.366 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-19 04:37:56.986200763 +0000 2024-09-19T04:38:08.366 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-19 04:32:12.867816956 +0000 2024-09-19T04:38:08.366 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-19 04:32:12.867816956 +0000 2024-09-19T04:38:08.366 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-19 04:32:12.867816956 +0000 2024-09-19T04:38:08.366 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T04:38:08.430 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-19T04:38:08.430 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-19T04:38:08.430 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000210385 s, 2.4 MB/s 2024-09-19T04:38:08.432 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T04:38:08.489 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-09-19T04:38:08.545 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T04:38:08.545 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:38:08.545 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-19T04:38:08.545 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:38:08.545 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:38:08.545 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-19 04:37:56.986200763 +0000 2024-09-19T04:38:08.545 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-19 04:32:13.194808203 +0000 2024-09-19T04:38:08.545 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-19 04:32:13.194808203 +0000 2024-09-19T04:38:08.546 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-19 04:32:13.194808203 +0000 2024-09-19T04:38:08.546 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T04:38:08.609 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-19T04:38:08.609 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-19T04:38:08.609 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000206588 s, 2.5 MB/s 2024-09-19T04:38:08.610 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T04:38:08.667 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-09-19T04:38:08.723 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T04:38:08.723 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:38:08.723 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-19T04:38:08.723 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:38:08.723 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:38:08.724 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-19 04:37:56.986200763 +0000 2024-09-19T04:38:08.724 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-19 04:32:13.514799637 +0000 2024-09-19T04:38:08.724 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-19 04:32:13.514799637 +0000 2024-09-19T04:38:08.724 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-19 04:32:13.514799637 +0000 2024-09-19T04:38:08.724 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T04:38:08.787 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-19T04:38:08.787 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-19T04:38:08.787 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000215861 s, 2.4 MB/s 2024-09-19T04:38:08.788 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T04:38:08.845 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-19T04:38:08.845 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T04:38:08.865 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T04:38:08.865 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2024-09-19T04:38:08.924 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T04:38:08.924 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:38:08.924 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T04:38:08.924 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:38:08.924 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:38:08.924 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-19 04:37:47.793722645 +0000 2024-09-19T04:38:08.924 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-19 04:32:15.373339990 +0000 2024-09-19T04:38:08.924 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-19 04:32:15.373339990 +0000 2024-09-19T04:38:08.924 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-19 04:32:15.373339990 +0000 2024-09-19T04:38:08.925 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T04:38:08.993 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-19T04:38:08.993 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-19T04:38:08.993 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.00029504 s, 1.7 MB/s 2024-09-19T04:38:08.994 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T04:38:09.053 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2024-09-19T04:38:09.111 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T04:38:09.111 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:38:09.111 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-19T04:38:09.111 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:38:09.112 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:38:09.112 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-19 04:37:47.794722630 +0000 2024-09-19T04:38:09.112 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-19 04:32:15.702334765 +0000 2024-09-19T04:38:09.112 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-19 04:32:15.702334765 +0000 2024-09-19T04:38:09.112 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-19 04:32:15.702334765 +0000 2024-09-19T04:38:09.112 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T04:38:09.180 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-19T04:38:09.180 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-19T04:38:09.180 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000238557 s, 2.1 MB/s 2024-09-19T04:38:09.181 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T04:38:09.240 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2024-09-19T04:38:09.299 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T04:38:09.299 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:38:09.299 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-19T04:38:09.299 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:38:09.300 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:38:09.300 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-19 04:37:47.794722630 +0000 2024-09-19T04:38:09.300 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-19 04:32:16.042329365 +0000 2024-09-19T04:38:09.300 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-19 04:32:16.042329365 +0000 2024-09-19T04:38:09.300 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-19 04:32:16.042329365 +0000 2024-09-19T04:38:09.300 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T04:38:09.366 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-19T04:38:09.366 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-19T04:38:09.367 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000232367 s, 2.2 MB/s 2024-09-19T04:38:09.368 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T04:38:09.426 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2024-09-19T04:38:09.482 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T04:38:09.482 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:38:09.483 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-19T04:38:09.483 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:38:09.483 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:38:09.483 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-19 04:37:47.794722630 +0000 2024-09-19T04:38:09.483 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-19 04:32:16.356324378 +0000 2024-09-19T04:38:09.483 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-19 04:32:16.356324378 +0000 2024-09-19T04:38:09.483 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-19 04:32:16.356324378 +0000 2024-09-19T04:38:09.483 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T04:38:09.548 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-19T04:38:09.548 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-19T04:38:09.549 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.00023697 s, 2.2 MB/s 2024-09-19T04:38:09.550 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T04:38:09.607 INFO:tasks.cephadm:Deploying osd.0 on smithi123 with /dev/vg_nvme/lv_4... 2024-09-19T04:38:09.608 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T04:38:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:09 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:09 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:09 smithi123 ceph-mon[30511]: Reconfiguring mon.b (monmap changed)... 2024-09-19T04:38:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:09 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:38:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:09 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:38:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:09 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:09 smithi123 ceph-mon[30511]: Reconfiguring daemon mon.b on smithi183 2024-09-19T04:38:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:09 smithi123 ceph-mon[30511]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:09.967 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:38:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:09 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:09 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:09 smithi183 ceph-mon[35333]: Reconfiguring mon.b (monmap changed)... 2024-09-19T04:38:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:09 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:38:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:09 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:38:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:09 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:09 smithi183 ceph-mon[35333]: Reconfiguring daemon mon.b on smithi183 2024-09-19T04:38:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:09 smithi183 ceph-mon[35333]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:10.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:09 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:10.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:09 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:10.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:09 smithi193 ceph-mon[35592]: Reconfiguring mon.b (monmap changed)... 2024-09-19T04:38:10.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:09 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:38:10.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:09 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:38:10.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:09 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:10.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:09 smithi193 ceph-mon[35592]: Reconfiguring daemon mon.b on smithi183 2024-09-19T04:38:10.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:09 smithi193 ceph-mon[35592]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:11.424 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:11 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:11.424 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:11 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:11.424 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:11 smithi123 ceph-mon[30511]: Reconfiguring mon.c (monmap changed)... 2024-09-19T04:38:11.424 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:11 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:38:11.424 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:11 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:38:11.424 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:11 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:11.424 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:11 smithi123 ceph-mon[30511]: Reconfiguring daemon mon.c on smithi193 2024-09-19T04:38:11.474 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:11 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:11.474 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:11 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:11.474 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:11 smithi193 ceph-mon[35592]: Reconfiguring mon.c (monmap changed)... 2024-09-19T04:38:11.474 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:11 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:38:11.474 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:11 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:38:11.474 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:11 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:11.474 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:11 smithi193 ceph-mon[35592]: Reconfiguring daemon mon.c on smithi193 2024-09-19T04:38:11.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:11 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:11.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:11 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:11.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:11 smithi183 ceph-mon[35333]: Reconfiguring mon.c (monmap changed)... 2024-09-19T04:38:11.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:11 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:38:11.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:11 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:38:11.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:11 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:11.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:11 smithi183 ceph-mon[35333]: Reconfiguring daemon mon.c on smithi193 2024-09-19T04:38:12.744 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:12 smithi123 ceph-mon[30511]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:12.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:12 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:12.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:12 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:12.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:12 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:12.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:12 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:12.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:12 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:38:12.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:12 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:12.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:12 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:38:12.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:12 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:38:12.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:12 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:38:12.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:12 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:12.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:12 smithi123 ceph-mon[30511]: Deploying daemon mgr.z on smithi193 2024-09-19T04:38:12.753 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:38:12.783 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_4 2024-09-19T04:38:12.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:12 smithi193 ceph-mon[35592]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:12.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:12 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:12.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:12 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:12.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:12 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:12.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:12 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:12.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:12 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:38:12.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:12 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:12.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:12 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:38:12.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:12 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:38:12.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:12 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:38:12.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:12 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:12.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:12 smithi193 ceph-mon[35592]: Deploying daemon mgr.z on smithi193 2024-09-19T04:38:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:12 smithi183 ceph-mon[35333]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:12 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:12 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:12 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:12 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:12 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:38:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:12 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:12.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:12 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:38:12.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:12 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:38:12.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:12 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:38:12.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:12 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:12.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:12 smithi183 ceph-mon[35333]: Deploying daemon mgr.z on smithi193 2024-09-19T04:38:13.142 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:38:13.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.633+0000 7fbd3d200640 1 -- 172.21.15.123:0/1128997928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd38076220 msgr2=0x7fbd38074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:13.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.633+0000 7fbd3d200640 1 --2- 172.21.15.123:0/1128997928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd38076220 0x7fbd38074680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009900 tx=0x7fbd2802a410 comp rx=0 tx=0).stop 2024-09-19T04:38:13.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.634+0000 7fbd3d200640 1 -- 172.21.15.123:0/1128997928 shutdown_connections 2024-09-19T04:38:13.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.634+0000 7fbd3d200640 1 --2- 172.21.15.123:0/1128997928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd38109a80 0x7fbd3810bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:13.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.634+0000 7fbd3d200640 1 --2- 172.21.15.123:0/1128997928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd38074c50 0x7fbd380750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:13.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.634+0000 7fbd3d200640 1 --2- 172.21.15.123:0/1128997928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd38076220 0x7fbd38074680 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:13.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.634+0000 7fbd3d200640 1 -- 172.21.15.123:0/1128997928 >> 172.21.15.123:0/1128997928 conn(0x7fbd380fd390 msgr2=0x7fbd380ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:38:13.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.634+0000 7fbd3d200640 1 -- 172.21.15.123:0/1128997928 shutdown_connections 2024-09-19T04:38:13.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.634+0000 7fbd3d200640 1 -- 172.21.15.123:0/1128997928 wait complete. 2024-09-19T04:38:13.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.635+0000 7fbd3d200640 1 Processor -- start 2024-09-19T04:38:13.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.635+0000 7fbd3d200640 1 -- start start 2024-09-19T04:38:13.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.635+0000 7fbd3d200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd38074c50 0x7fbd3819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:13.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.635+0000 7fbd3d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd38076220 0x7fbd3819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:13.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.635+0000 7fbd3d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd38109a80 0x7fbd381a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:13.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.635+0000 7fbd3d200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3810fa60 con 0x7fbd38109a80 2024-09-19T04:38:13.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.635+0000 7fbd3d200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3810f8e0 con 0x7fbd38076220 2024-09-19T04:38:13.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.635+0000 7fbd3d200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3810fbe0 con 0x7fbd38074c50 2024-09-19T04:38:13.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.635+0000 7fbd35400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd38076220 0x7fbd3819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:13.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.635+0000 7fbd36800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd38109a80 0x7fbd381a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:13.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.635+0000 7fbd35e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd38074c50 0x7fbd3819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:13.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.635+0000 7fbd35400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd38076220 0x7fbd3819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:54346/0 (socket says 172.21.15.123:54346) 2024-09-19T04:38:13.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.635+0000 7fbd35400640 1 -- 172.21.15.123:0/3886698655 learned_addr learned my addr 172.21.15.123:0/3886698655 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:38:13.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.636+0000 7fbd35400640 1 -- 172.21.15.123:0/3886698655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd38074c50 msgr2=0x7fbd3819a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:13.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.636+0000 7fbd35400640 1 --2- 172.21.15.123:0/3886698655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd38074c50 0x7fbd3819a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:13.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.636+0000 7fbd35400640 1 -- 172.21.15.123:0/3886698655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd38109a80 msgr2=0x7fbd381a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:13.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.636+0000 7fbd35400640 1 --2- 172.21.15.123:0/3886698655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd38109a80 0x7fbd381a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:13.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.636+0000 7fbd35400640 1 -- 172.21.15.123:0/3886698655 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd381a1d00 con 0x7fbd38076220 2024-09-19T04:38:13.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.636+0000 7fbd36800640 1 --2- 172.21.15.123:0/3886698655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd38109a80 0x7fbd381a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:38:13.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.636+0000 7fbd35400640 1 --2- 172.21.15.123:0/3886698655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd38076220 0x7fbd3819a610 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400e9f0 tx=0x7fbd2400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:38:13.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.636+0000 7fbd2ea00640 1 -- 172.21.15.123:0/3886698655 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2400cd80 con 0x7fbd38076220 2024-09-19T04:38:13.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.636+0000 7fbd2ea00640 1 -- 172.21.15.123:0/3886698655 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd24010910 con 0x7fbd38076220 2024-09-19T04:38:13.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.636+0000 7fbd3d200640 1 -- 172.21.15.123:0/3886698655 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd381a1ff0 con 0x7fbd38076220 2024-09-19T04:38:13.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.637+0000 7fbd2ea00640 1 -- 172.21.15.123:0/3886698655 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24018990 con 0x7fbd38076220 2024-09-19T04:38:13.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.637+0000 7fbd3d200640 1 -- 172.21.15.123:0/3886698655 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd381a24f0 con 0x7fbd38076220 2024-09-19T04:38:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.638+0000 7fbd3d200640 1 -- 172.21.15.123:0/3886698655 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd38108800 con 0x7fbd38076220 2024-09-19T04:38:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.638+0000 7fbd2ea00640 1 -- 172.21.15.123:0/3886698655 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fbd24018b30 con 0x7fbd38076220 2024-09-19T04:38:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.638+0000 7fbd2ea00640 1 --2- 172.21.15.123:0/3886698655 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbd1c039500 0x7fbd1c03b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.638+0000 7fbd2ea00640 1 -- 172.21.15.123:0/3886698655 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbd2404cb60 con 0x7fbd38076220 2024-09-19T04:38:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.639+0000 7fbd35e00640 1 --2- 172.21.15.123:0/3886698655 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbd1c039500 0x7fbd1c03b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.639+0000 7fbd35e00640 1 --2- 172.21.15.123:0/3886698655 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbd1c039500 0x7fbd1c03b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbd28005ab0 tx=0x7fbd28000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:38:13.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.644+0000 7fbd2ea00640 1 -- 172.21.15.123:0/3886698655 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbd2400b2c0 con 0x7fbd38076220 2024-09-19T04:38:13.736 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:13 smithi193 ceph-mgr[37408]: -- 172.21.15.193:0/221138173 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56225a5cb0e0 con 0x56225a5a8800 2024-09-19T04:38:13.736 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:13 smithi193 bash[37389]: e97a36cb7ef5d14853dc6b4e7abf74cc317c8885a14ace667c06164408c3e9d5 2024-09-19T04:38:13.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:13.833+0000 7fbd3d200640 1 -- 172.21.15.123:0/3886698655 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbd38062630 con 0x7fbd1c039500 2024-09-19T04:38:13.990 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:13 smithi193 systemd[1]: Started Ceph mgr.z for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:38:13.990 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:13 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:13.774+0000 7fa12ccc8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:38:13.990 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:13 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:13.833+0000 7fa12ccc8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:38:13.990 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:13 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:13.979+0000 7fa12ccc8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:38:14.330 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:14 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:14.046+0000 7fa12ccc8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:38:14.330 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:14 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:14.191+0000 7fa12ccc8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:38:14.330 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:14 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:14.328+0000 7fa12ccc8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:38:14.642 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:14 smithi193 ceph-mon[35592]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:14.643 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:14.643 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:14.643 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:14.643 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:38:14.643 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:38:14.643 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:38:14.643 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:14.643 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:38:14.643 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:38:14.643 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:14.667 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:14 smithi123 ceph-mon[30511]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:14.667 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:14.667 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:14.667 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:14.668 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:38:14.668 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:38:14.668 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:38:14.668 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:14.668 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:38:14.668 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:38:14.668 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:14 smithi183 ceph-mon[35333]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:38:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:38:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:38:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:14.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:38:14.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:38:14.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:14.990 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:14 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:14.641+0000 7fa12ccc8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:38:14.990 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:14 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:14.701+0000 7fa12ccc8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:38:14.990 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:14 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:14.886+0000 7fa12ccc8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:38:15.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:15 smithi183 ceph-mon[35333]: Deploying daemon mgr.y on smithi183 2024-09-19T04:38:15.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:15 smithi183 ceph-mon[35333]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:38:15.637 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:15 smithi123 ceph-mon[30511]: Deploying daemon mgr.y on smithi183 2024-09-19T04:38:15.637 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:15 smithi123 ceph-mon[30511]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:38:15.812 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:15 smithi193 ceph-mon[35592]: Deploying daemon mgr.y on smithi183 2024-09-19T04:38:15.812 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:15 smithi193 ceph-mon[35592]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:38:15.813 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:15 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:15.766+0000 7fa12ccc8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:38:16.002 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:15 smithi183 systemd[1]: Started Ceph mgr.y for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:38:16.067 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:15 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:15.835+0000 7fa12ccc8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:38:16.284 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:16 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:16.000+0000 7f0bcdb3a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:38:16.284 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:16 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:16.058+0000 7f0bcdb3a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:38:16.284 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:16 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:16.212+0000 7f0bcdb3a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:38:16.284 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:16 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:16.281+0000 7f0bcdb3a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:38:16.538 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:16 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:16.441+0000 7f0bcdb3a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:38:16.828 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:16 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:16.603+0000 7f0bcdb3a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:38:16.828 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:16 smithi183 ceph-mon[35333]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:16.828 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:16 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:16.829 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:16 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:16.829 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:16 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:16.829 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:16 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:16.829 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:16 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:16.846 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:16 smithi123 ceph-mon[30511]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:16.846 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:16 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:16.846 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:16 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:16.846 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:16 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:16.846 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:16 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:16.846 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:16 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:16.886 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:16 smithi193 ceph-mon[35592]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:16.886 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:16 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:16.886 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:16 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:16.886 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:16 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:16.886 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:16 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:16.886 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:16 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:16.887 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:16 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:16.623+0000 7fa12ccc8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:38:16.887 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:16 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:16.743+0000 7fa12ccc8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:38:17.100 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:16 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:16.930+0000 7f0bcdb3a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:38:17.100 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:16 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:16.991+0000 7f0bcdb3a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:38:17.186 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:16 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:16.897+0000 7fa12ccc8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:38:17.186 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:16 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:16.960+0000 7fa12ccc8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:38:17.186 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:17.116+0000 7fa12ccc8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:38:17.186 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:17.184+0000 7fa12ccc8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:38:17.490 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:17.255+0000 7fa12ccc8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:38:17.490 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:17.420+0000 7fa12ccc8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:38:17.490 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:17.488+0000 7fa12ccc8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:38:17.540 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:17 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:17.191+0000 7f0bcdb3a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:38:17.740 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:17.555+0000 7fa12ccc8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:38:17.740 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:17.637+0000 7fa12ccc8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:38:17.741 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:17.696+0000 7fa12ccc8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:38:17.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1193194361' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b"}]: dispatch 2024-09-19T04:38:17.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b"}]: dispatch 2024-09-19T04:38:17.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b"}]': finished 2024-09-19T04:38:17.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-mon[35592]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T04:38:17.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:38:17.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:17.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:17.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1255263153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:38:17.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:17.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:17.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:17.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:38:17.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:17.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:17.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:17 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1193194361' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b"}]: dispatch 2024-09-19T04:38:17.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:17 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b"}]: dispatch 2024-09-19T04:38:17.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:17 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b"}]': finished 2024-09-19T04:38:17.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:17 smithi183 ceph-mon[35333]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T04:38:17.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:38:17.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:17.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:17.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:17 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1255263153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:38:17.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:17.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:17.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:17.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:38:17.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:17.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:17.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:17 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1193194361' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b"}]: dispatch 2024-09-19T04:38:17.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:17 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b"}]: dispatch 2024-09-19T04:38:17.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:17 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b"}]': finished 2024-09-19T04:38:17.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:17 smithi123 ceph-mon[30511]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T04:38:17.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:38:17.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:17.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:17.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:17 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1255263153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:38:17.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:17.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:17.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:17.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:38:17.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:17.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:18.240 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:17 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:17.756+0000 7fa12ccc8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:38:18.350 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:18 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:18.076+0000 7f0bcdb3a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:38:18.350 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:18 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:18.143+0000 7f0bcdb3a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:38:18.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:18 smithi183 ceph-mon[35333]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:18.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:18 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:18.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:18 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:38:18.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:18 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:18.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:18 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:18.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:18 smithi123 ceph-mon[30511]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:18.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:18 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:18.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:18 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:38:18.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:18 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:18.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:18 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:18.990 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:38:18 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:38:18.549+0000 7fa12ccc8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:38:18.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:18 smithi193 ceph-mon[35592]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:18.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:18 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:18.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:18 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:38:18.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:18 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:18.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:18 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:19.124 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:18 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:18.863+0000 7f0bcdb3a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:38:19.124 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:18 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:18.979+0000 7f0bcdb3a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:38:19.124 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:19 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:19.122+0000 7f0bcdb3a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:38:19.398 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:19 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:19.182+0000 7f0bcdb3a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:38:19.399 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:19 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:19.329+0000 7f0bcdb3a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:38:19.399 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:19 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:19.396+0000 7f0bcdb3a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:38:19.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:19.549+0000 7fbd2ea00640 1 -- 172.21.15.123:0/3886698655 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mgrmap(e 14) ==== 92409+0+0 (secure 0 0 0) 0x7fbd24010ac0 con 0x7fbd38076220 2024-09-19T04:38:19.682 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:19 smithi183 ceph-mon[35333]: Standby manager daemon z started 2024-09-19T04:38:19.682 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:19 smithi183 ceph-mon[35333]: from='mgr.? 172.21.15.193:0/1987826981' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:38:19.682 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:19 smithi183 ceph-mon[35333]: from='mgr.? 172.21.15.193:0/1987826981' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:38:19.682 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:19 smithi183 ceph-mon[35333]: from='mgr.? 172.21.15.193:0/1987826981' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:38:19.682 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:19 smithi183 ceph-mon[35333]: from='mgr.? 172.21.15.193:0/1987826981' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:38:19.683 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:19 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:19.456+0000 7f0bcdb3a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:38:19.683 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:19 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:19.614+0000 7f0bcdb3a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:38:19.683 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:19 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:19.680+0000 7f0bcdb3a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:38:19.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:19 smithi123 ceph-mon[30511]: Standby manager daemon z started 2024-09-19T04:38:19.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:19 smithi123 ceph-mon[30511]: from='mgr.? 172.21.15.193:0/1987826981' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:38:19.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:19 smithi123 ceph-mon[30511]: from='mgr.? 172.21.15.193:0/1987826981' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:38:19.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:19 smithi123 ceph-mon[30511]: from='mgr.? 172.21.15.193:0/1987826981' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:38:19.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:19 smithi123 ceph-mon[30511]: from='mgr.? 172.21.15.193:0/1987826981' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:38:19.948 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:19 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:19.746+0000 7f0bcdb3a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:38:19.948 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:19 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:19.828+0000 7f0bcdb3a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:38:19.948 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:19 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:19.887+0000 7f0bcdb3a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:38:19.948 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:19 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:19.946+0000 7f0bcdb3a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:38:19.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:19 smithi193 ceph-mon[35592]: Standby manager daemon z started 2024-09-19T04:38:19.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:19 smithi193 ceph-mon[35592]: from='mgr.? 172.21.15.193:0/1987826981' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:38:19.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:19 smithi193 ceph-mon[35592]: from='mgr.? 172.21.15.193:0/1987826981' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:38:19.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:19 smithi193 ceph-mon[35592]: from='mgr.? 172.21.15.193:0/1987826981' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:38:19.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:19 smithi193 ceph-mon[35592]: from='mgr.? 172.21.15.193:0/1987826981' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:38:20.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:20 smithi183 ceph-mon[35333]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:20.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:20 smithi183 ceph-mon[35333]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-19T04:38:20.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:20 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:38:20.850 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:38:20 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:38:20.730+0000 7f0bcdb3a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:38:20.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:20 smithi123 ceph-mon[30511]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:20.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:20 smithi123 ceph-mon[30511]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-19T04:38:20.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:20 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:38:20.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:20 smithi193 ceph-mon[35592]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:20.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:20 smithi193 ceph-mon[35592]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-19T04:38:20.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:20 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:38:21.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:21.562+0000 7fbd2ea00640 1 -- 172.21.15.123:0/3886698655 <== mon.2 v2:172.21.15.183:3300/0 8 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7fbd2400f3c0 con 0x7fbd38076220 2024-09-19T04:38:21.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:21 smithi183 ceph-mon[35333]: Standby manager daemon y started 2024-09-19T04:38:21.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:21 smithi183 ceph-mon[35333]: from='mgr.? 172.21.15.183:0/1688158509' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:38:21.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:21 smithi183 ceph-mon[35333]: from='mgr.? 172.21.15.183:0/1688158509' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:38:21.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:21 smithi183 ceph-mon[35333]: from='mgr.? 172.21.15.183:0/1688158509' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:38:21.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:21 smithi183 ceph-mon[35333]: from='mgr.? 172.21.15.183:0/1688158509' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:38:21.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:21 smithi123 ceph-mon[30511]: Standby manager daemon y started 2024-09-19T04:38:21.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:21 smithi123 ceph-mon[30511]: from='mgr.? 172.21.15.183:0/1688158509' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:38:21.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:21 smithi123 ceph-mon[30511]: from='mgr.? 172.21.15.183:0/1688158509' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:38:21.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:21 smithi123 ceph-mon[30511]: from='mgr.? 172.21.15.183:0/1688158509' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:38:21.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:21 smithi123 ceph-mon[30511]: from='mgr.? 172.21.15.183:0/1688158509' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:38:21.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:21 smithi193 ceph-mon[35592]: Standby manager daemon y started 2024-09-19T04:38:21.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:21 smithi193 ceph-mon[35592]: from='mgr.? 172.21.15.183:0/1688158509' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:38:21.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:21 smithi193 ceph-mon[35592]: from='mgr.? 172.21.15.183:0/1688158509' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:38:21.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:21 smithi193 ceph-mon[35592]: from='mgr.? 172.21.15.183:0/1688158509' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:38:21.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:21 smithi193 ceph-mon[35592]: from='mgr.? 172.21.15.183:0/1688158509' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:38:22.840 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:22 smithi123 ceph-mon[30511]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:22.840 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:22 smithi123 ceph-mon[30511]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-19T04:38:22.840 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:38:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:22 smithi183 ceph-mon[35333]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:22 smithi183 ceph-mon[35333]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-19T04:38:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:38:22.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:22 smithi193 ceph-mon[35592]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:22.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:22 smithi193 ceph-mon[35592]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-19T04:38:22.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:38:23.842 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:23 smithi123 ceph-mon[30511]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:23.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:23 smithi193 ceph-mon[35592]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:24.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:23 smithi183 ceph-mon[35333]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:25.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:25 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T04:38:25.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:25 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:25.521 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:25 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T04:38:25.521 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:25 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:25.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:25 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T04:38:25.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:25 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:26.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:26 smithi123 ceph-mon[30511]: Deploying daemon osd.0 on smithi123 2024-09-19T04:38:26.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:26 smithi123 ceph-mon[30511]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:26.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:26 smithi193 ceph-mon[35592]: Deploying daemon osd.0 on smithi123 2024-09-19T04:38:26.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:26 smithi193 ceph-mon[35592]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:26.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:26 smithi183 ceph-mon[35333]: Deploying daemon osd.0 on smithi123 2024-09-19T04:38:26.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:26 smithi183 ceph-mon[35333]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:28.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:28 smithi123 ceph-mon[30511]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:28.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:28 smithi183 ceph-mon[35333]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:28.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:28 smithi193 ceph-mon[35592]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:30.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:30 smithi183 ceph-mon[35333]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:30.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:30 smithi123 ceph-mon[30511]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:30.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:30 smithi193 ceph-mon[35592]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:32.538 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:32 smithi123 ceph-mon[30511]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:32.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:32 smithi183 ceph-mon[35333]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:32.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:32 smithi193 ceph-mon[35592]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:33.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:33.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:33.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:33.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:33.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:33.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:33.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:33.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:33.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:34.428 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:34 smithi123 ceph-mon[30511]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:34.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:34 smithi183 ceph-mon[35333]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:34.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:34 smithi193 ceph-mon[35592]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:36 smithi183 ceph-mon[35333]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:36 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:36 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:36 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:36 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:38:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:36 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:36.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:36 smithi123 ceph-mon[30511]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:36.680 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:36 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:36.680 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:36 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:36.680 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:36 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:36.680 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:36 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:38:36.680 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:36 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:36.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:36 smithi193 ceph-mon[35592]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:36.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:36 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:36.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:36 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:36.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:36 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:36.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:36 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:38:36.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:36 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:37.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:37.492+0000 7fbd2ea00640 1 -- 172.21.15.123:0/3886698655 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbd38062630 con 0x7fbd1c039500 2024-09-19T04:38:37.495 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 0 on host 'smithi123' 2024-09-19T04:38:37.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:37.494+0000 7fbd3d200640 1 -- 172.21.15.123:0/3886698655 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbd1c039500 msgr2=0x7fbd1c03b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:37.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:37.494+0000 7fbd3d200640 1 --2- 172.21.15.123:0/3886698655 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbd1c039500 0x7fbd1c03b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbd28005ab0 tx=0x7fbd28000bc0 comp rx=0 tx=0).stop 2024-09-19T04:38:37.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:37.494+0000 7fbd3d200640 1 -- 172.21.15.123:0/3886698655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd38076220 msgr2=0x7fbd3819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:37.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:37.494+0000 7fbd3d200640 1 --2- 172.21.15.123:0/3886698655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd38076220 0x7fbd3819a610 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400e9f0 tx=0x7fbd2400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:38:37.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:37.495+0000 7fbd3d200640 1 -- 172.21.15.123:0/3886698655 shutdown_connections 2024-09-19T04:38:37.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:37.495+0000 7fbd3d200640 1 --2- 172.21.15.123:0/3886698655 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbd1c039500 0x7fbd1c03b9c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:37.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:37.495+0000 7fbd3d200640 1 --2- 172.21.15.123:0/3886698655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd38109a80 0x7fbd381a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:37.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:37.495+0000 7fbd3d200640 1 --2- 172.21.15.123:0/3886698655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd38076220 0x7fbd3819a610 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:37.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:37.495+0000 7fbd3d200640 1 --2- 172.21.15.123:0/3886698655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd38074c50 0x7fbd3819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:37.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:37.495+0000 7fbd3d200640 1 -- 172.21.15.123:0/3886698655 >> 172.21.15.123:0/3886698655 conn(0x7fbd380fd390 msgr2=0x7fbd381059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:38:37.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:37.495+0000 7fbd3d200640 1 -- 172.21.15.123:0/3886698655 shutdown_connections 2024-09-19T04:38:37.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:38:37.495+0000 7fbd3d200640 1 -- 172.21.15.123:0/3886698655 wait complete. 2024-09-19T04:38:38.229 DEBUG:teuthology.orchestra.run.smithi123:osd.0> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T04:38:38.234 INFO:tasks.cephadm:Deploying osd.1 on smithi183 with /dev/vg_nvme/lv_4... 2024-09-19T04:38:38.234 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T04:38:38.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:38 smithi123 ceph-mon[30511]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:38.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:38 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:38.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:38 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:38.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:38 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:38.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:38 smithi123 ceph-mon[30511]: from='osd.0 [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T04:38:38.487 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:38:38.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:38 smithi183 ceph-mon[35333]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:38.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:38 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:38.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:38 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:38.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:38 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:38.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:38 smithi183 ceph-mon[35333]: from='osd.0 [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T04:38:38.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:38 smithi193 ceph-mon[35592]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:38.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:38 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:38.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:38 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:38.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:38 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:38.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:38 smithi193 ceph-mon[35592]: from='osd.0 [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T04:38:39.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:39 smithi183 ceph-mon[35333]: from='osd.0 [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T04:38:39.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:39 smithi183 ceph-mon[35333]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T04:38:39.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:39 smithi183 ceph-mon[35333]: from='osd.0 [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-19T04:38:39.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:39 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:38:39.757 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:39 smithi123 ceph-mon[30511]: from='osd.0 [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T04:38:39.757 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:39 smithi123 ceph-mon[30511]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T04:38:39.757 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:39 smithi123 ceph-mon[30511]: from='osd.0 [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-19T04:38:39.757 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:39 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:38:39.757 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:38:39 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-0[44032]: 2024-09-19T04:38:39.519+0000 7f90e6c00640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T04:38:39.757 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:38:39 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-0[44032]: 2024-09-19T04:38:39.522+0000 7f90e1c00640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:38:39.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:39 smithi193 ceph-mon[35592]: from='osd.0 [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T04:38:39.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:39 smithi193 ceph-mon[35592]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T04:38:39.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:39 smithi193 ceph-mon[35592]: from='osd.0 [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-19T04:38:39.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:39 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:38:40.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:40 smithi183 ceph-mon[35333]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:40.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:40 smithi183 ceph-mon[35333]: from='osd.0 [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-19T04:38:40.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:40 smithi183 ceph-mon[35333]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T04:38:40.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:40 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:38:40.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:40 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:38:40.845 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:38:40.871 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_4 2024-09-19T04:38:40.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:40 smithi123 ceph-mon[30511]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:40.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:40 smithi123 ceph-mon[30511]: from='osd.0 [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-19T04:38:40.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:40 smithi123 ceph-mon[30511]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T04:38:40.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:40 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:38:40.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:40 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:38:40.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:40 smithi193 ceph-mon[35592]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:40.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:40 smithi193 ceph-mon[35592]: from='osd.0 [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-19T04:38:40.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:40 smithi193 ceph-mon[35592]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T04:38:40.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:40 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:38:40.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:40 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:38:41.111 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:38:41.561 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.553+0000 7f3a634a9640 1 -- 172.21.15.183:0/1570056591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a5c100b10 msgr2=0x7f3a5c100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:41.561 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.553+0000 7f3a634a9640 1 --2- 172.21.15.183:0/1570056591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a5c100b10 0x7f3a5c100f10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009900 tx=0x7f3a4c02a410 comp rx=0 tx=0).stop 2024-09-19T04:38:41.561 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.553+0000 7f3a634a9640 1 -- 172.21.15.183:0/1570056591 shutdown_connections 2024-09-19T04:38:41.561 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.553+0000 7f3a634a9640 1 --2- 172.21.15.183:0/1570056591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a5c1026d0 0x7f3a5c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:41.561 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.553+0000 7f3a634a9640 1 --2- 172.21.15.183:0/1570056591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a5c101d10 0x7f3a5c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:41.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.553+0000 7f3a634a9640 1 --2- 172.21.15.183:0/1570056591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a5c100b10 0x7f3a5c100f10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:41.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.553+0000 7f3a634a9640 1 -- 172.21.15.183:0/1570056591 >> 172.21.15.183:0/1570056591 conn(0x7f3a5c0fc2a0 msgr2=0x7f3a5c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:38:41.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.553+0000 7f3a634a9640 1 -- 172.21.15.183:0/1570056591 shutdown_connections 2024-09-19T04:38:41.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.553+0000 7f3a634a9640 1 -- 172.21.15.183:0/1570056591 wait complete. 2024-09-19T04:38:41.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.553+0000 7f3a634a9640 1 Processor -- start 2024-09-19T04:38:41.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a634a9640 1 -- start start 2024-09-19T04:38:41.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a634a9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a5c100b10 0x7f3a5c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:41.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a634a9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a5c101d10 0x7f3a5c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:41.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a634a9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a5c1026d0 0x7f3a5c19f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:41.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a634a9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c10c7f0 con 0x7f3a5c100b10 2024-09-19T04:38:41.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a634a9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c10c670 con 0x7f3a5c1026d0 2024-09-19T04:38:41.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a634a9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c10c970 con 0x7f3a5c101d10 2024-09-19T04:38:41.563 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a61a46640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a5c1026d0 0x7f3a5c19f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:41.563 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a61a46640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a5c1026d0 0x7f3a5c19f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:41770/0 (socket says 172.21.15.183:41770) 2024-09-19T04:38:41.563 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a61a46640 1 -- 172.21.15.183:0/1660774898 learned_addr learned my addr 172.21.15.183:0/1660774898 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-19T04:38:41.563 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a60a44640 1 --2- 172.21.15.183:0/1660774898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a5c101d10 0x7f3a5c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:41.563 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a61245640 1 --2- 172.21.15.183:0/1660774898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a5c100b10 0x7f3a5c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:41.563 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a61a46640 1 -- 172.21.15.183:0/1660774898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a5c101d10 msgr2=0x7f3a5c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:41.563 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a61a46640 1 --2- 172.21.15.183:0/1660774898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a5c101d10 0x7f3a5c19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:41.563 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a61a46640 1 -- 172.21.15.183:0/1660774898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a5c100b10 msgr2=0x7f3a5c06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:38:41.563 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a61a46640 1 --2- 172.21.15.183:0/1660774898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a5c100b10 0x7f3a5c06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:38:41.563 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a61a46640 1 -- 172.21.15.183:0/1660774898 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c19fb40 con 0x7f3a5c1026d0 2024-09-19T04:38:41.563 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a61a46640 1 --2- 172.21.15.183:0/1660774898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a5c1026d0 0x7f3a5c19f230 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400cad0 tx=0x7f3a5400ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:38:41.563 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.554+0000 7f3a527fc640 1 -- 172.21.15.183:0/1660774898 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5400ad60 con 0x7f3a5c1026d0 2024-09-19T04:38:41.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.555+0000 7f3a634a9640 1 -- 172.21.15.183:0/1660774898 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5c19fdd0 con 0x7f3a5c1026d0 2024-09-19T04:38:41.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.555+0000 7f3a527fc640 1 -- 172.21.15.183:0/1660774898 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a54005020 con 0x7f3a5c1026d0 2024-09-19T04:38:41.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.555+0000 7f3a527fc640 1 -- 172.21.15.183:0/1660774898 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5400f6b0 con 0x7f3a5c1026d0 2024-09-19T04:38:41.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.555+0000 7f3a634a9640 1 -- 172.21.15.183:0/1660774898 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5c1a02e0 con 0x7f3a5c1026d0 2024-09-19T04:38:41.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.555+0000 7f3a634a9640 1 -- 172.21.15.183:0/1660774898 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a5c107e40 con 0x7f3a5c1026d0 2024-09-19T04:38:41.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.557+0000 7f3a527fc640 1 -- 172.21.15.183:0/1660774898 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f3a54005560 con 0x7f3a5c1026d0 2024-09-19T04:38:41.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.557+0000 7f3a527fc640 1 --2- 172.21.15.183:0/1660774898 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3a480a31b0 0x7f3a480a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:38:41.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.557+0000 7f3a527fc640 1 -- 172.21.15.183:0/1660774898 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f3a540ce130 con 0x7f3a5c1026d0 2024-09-19T04:38:41.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.558+0000 7f3a61245640 1 --2- 172.21.15.183:0/1660774898 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3a480a31b0 0x7f3a480a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:38:41.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.558+0000 7f3a61245640 1 --2- 172.21.15.183:0/1660774898 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3a480a31b0 0x7f3a480a5670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c02aa00 tx=0x7f3a4c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:38:41.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.561+0000 7f3a527fc640 1 -- 172.21.15.183:0/1660774898 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3a54014030 con 0x7f3a5c1026d0 2024-09-19T04:38:41.673 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:41 smithi123 ceph-mon[30511]: purged_snaps scrub starts 2024-09-19T04:38:41.673 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:41 smithi123 ceph-mon[30511]: purged_snaps scrub ok 2024-09-19T04:38:41.673 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:41 smithi123 ceph-mon[30511]: osd.0 [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501] boot 2024-09-19T04:38:41.674 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:41 smithi123 ceph-mon[30511]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T04:38:41.674 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:41 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:38:41.742 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:38:41.741+0000 7f3a634a9640 1 -- 172.21.15.183:0/1660774898 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3a5c062630 con 0x7f3a480a31b0 2024-09-19T04:38:41.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:41 smithi183 ceph-mon[35333]: purged_snaps scrub starts 2024-09-19T04:38:41.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:41 smithi183 ceph-mon[35333]: purged_snaps scrub ok 2024-09-19T04:38:41.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:41 smithi183 ceph-mon[35333]: osd.0 [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501] boot 2024-09-19T04:38:41.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:41 smithi183 ceph-mon[35333]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T04:38:41.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:41 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:38:41.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:41 smithi193 ceph-mon[35592]: purged_snaps scrub starts 2024-09-19T04:38:41.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:41 smithi193 ceph-mon[35592]: purged_snaps scrub ok 2024-09-19T04:38:41.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:41 smithi193 ceph-mon[35592]: osd.0 [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501] boot 2024-09-19T04:38:41.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:41 smithi193 ceph-mon[35592]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T04:38:41.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:41 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:38:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:42 smithi183 ceph-mon[35333]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:42 smithi183 ceph-mon[35333]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T04:38:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:42 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:42 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:42 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:38:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:42 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:42 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:42 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:38:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:42 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:42 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:38:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:42 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:38:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:42 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:42.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:42 smithi123 ceph-mon[30511]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:42.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:42 smithi123 ceph-mon[30511]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T04:38:42.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:42.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:42.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:38:42.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:42.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:42.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:38:42.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:42.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:38:42.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:38:42.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:42.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:42 smithi193 ceph-mon[35592]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:38:42.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:42 smithi193 ceph-mon[35592]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T04:38:42.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:42 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:42.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:42 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:42.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:42 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:38:42.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:42 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:42.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:42 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:42.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:42 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:38:42.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:42 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:42.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:42 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:38:42.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:42 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:38:42.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:42 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:43 smithi183 ceph-mon[35333]: Detected new or changed devices on smithi123 2024-09-19T04:38:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:43 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi123 to 17089M 2024-09-19T04:38:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:43 smithi183 ceph-mon[35333]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:38:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:43 smithi123 ceph-mon[30511]: Detected new or changed devices on smithi123 2024-09-19T04:38:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:43 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi123 to 17089M 2024-09-19T04:38:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:43 smithi123 ceph-mon[30511]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:38:43.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:43 smithi193 ceph-mon[35592]: Detected new or changed devices on smithi123 2024-09-19T04:38:43.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:43 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi123 to 17089M 2024-09-19T04:38:43.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:43 smithi193 ceph-mon[35592]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:38:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:44 smithi183 ceph-mon[35333]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:44 smithi183 ceph-mon[35333]: from='client.? 172.21.15.183:0/505365297' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a526d4ce-586b-4d66-b720-e868df92fa24"}]: dispatch 2024-09-19T04:38:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:44 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a526d4ce-586b-4d66-b720-e868df92fa24"}]: dispatch 2024-09-19T04:38:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:44 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a526d4ce-586b-4d66-b720-e868df92fa24"}]': finished 2024-09-19T04:38:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:44 smithi183 ceph-mon[35333]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T04:38:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:44 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:38:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:44 smithi123 ceph-mon[30511]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:44 smithi123 ceph-mon[30511]: from='client.? 172.21.15.183:0/505365297' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a526d4ce-586b-4d66-b720-e868df92fa24"}]: dispatch 2024-09-19T04:38:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:44 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a526d4ce-586b-4d66-b720-e868df92fa24"}]: dispatch 2024-09-19T04:38:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:44 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a526d4ce-586b-4d66-b720-e868df92fa24"}]': finished 2024-09-19T04:38:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:44 smithi123 ceph-mon[30511]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T04:38:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:44 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:38:44.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:44 smithi193 ceph-mon[35592]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:44.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:44 smithi193 ceph-mon[35592]: from='client.? 172.21.15.183:0/505365297' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a526d4ce-586b-4d66-b720-e868df92fa24"}]: dispatch 2024-09-19T04:38:44.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:44 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a526d4ce-586b-4d66-b720-e868df92fa24"}]: dispatch 2024-09-19T04:38:44.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:44 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a526d4ce-586b-4d66-b720-e868df92fa24"}]': finished 2024-09-19T04:38:44.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:44 smithi193 ceph-mon[35592]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T04:38:44.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:44 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:38:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:45 smithi183 ceph-mon[35333]: from='client.? 172.21.15.183:0/2766094967' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:38:45.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:45 smithi123 ceph-mon[30511]: from='client.? 172.21.15.183:0/2766094967' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:38:45.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:45 smithi193 ceph-mon[35592]: from='client.? 172.21.15.183:0/2766094967' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:38:46.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:46 smithi183 ceph-mon[35333]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:46.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:46 smithi123 ceph-mon[30511]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:46.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:46 smithi193 ceph-mon[35592]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:48.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:48 smithi183 ceph-mon[35333]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:48.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:48 smithi123 ceph-mon[30511]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:48.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:48 smithi193 ceph-mon[35592]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:50.829 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:50 smithi183 ceph-mon[35333]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:50.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:50 smithi123 ceph-mon[30511]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:50.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:50 smithi193 ceph-mon[35592]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:52.838 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:52 smithi183 ceph-mon[35333]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:52.838 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:52 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T04:38:52.838 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:52 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:52.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:52 smithi123 ceph-mon[30511]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:52.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:52 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T04:38:52.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:52 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:52.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:52 smithi193 ceph-mon[35592]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:52.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:52 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T04:38:52.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:52 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:38:53.679 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:53 smithi183 ceph-mon[35333]: Deploying daemon osd.1 on smithi183 2024-09-19T04:38:53.892 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:53 smithi193 ceph-mon[35592]: Deploying daemon osd.1 on smithi183 2024-09-19T04:38:53.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:53 smithi123 ceph-mon[30511]: Deploying daemon osd.1 on smithi183 2024-09-19T04:38:54.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:54 smithi123 ceph-mon[30511]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:54 smithi183 ceph-mon[35333]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:54.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:54 smithi193 ceph-mon[35592]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:56.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:56 smithi183 ceph-mon[35333]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:56.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:56 smithi123 ceph-mon[30511]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:56.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:56 smithi193 ceph-mon[35592]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:58.802 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:58 smithi183 ceph-mon[35333]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:58.803 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:58 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:58.803 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:58 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:58.803 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:38:58 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:58.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:58 smithi123 ceph-mon[30511]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:58.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:58 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:58.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:58 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:58.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:38:58 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:58.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:58 smithi193 ceph-mon[35592]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:38:58.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:58 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:38:58.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:58 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:38:58.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:38:58 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:00.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:00 smithi183 ceph-mon[35333]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:39:00.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:00 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:00.664 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:00 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:00.664 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:00 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:00.664 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:00 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:39:00.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:00 smithi123 ceph-mon[30511]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:39:00.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:00 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:00.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:00 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:00.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:00 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:00.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:00 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:39:00.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:00 smithi193 ceph-mon[35592]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:39:00.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:00 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:00.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:00 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:00.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:00 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:00.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:00 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:39:01.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:01 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:01.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:01 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:01.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:01 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:02.427 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:39:02.425+0000 7f3a527fc640 1 -- 172.21.15.183:0/1660774898 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3a5c062630 con 0x7f3a480a31b0 2024-09-19T04:39:02.427 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 1 on host 'smithi183' 2024-09-19T04:39:02.429 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:39:02.427+0000 7f3a634a9640 1 -- 172.21.15.183:0/1660774898 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3a480a31b0 msgr2=0x7f3a480a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:39:02.429 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:39:02.427+0000 7f3a634a9640 1 --2- 172.21.15.183:0/1660774898 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3a480a31b0 0x7f3a480a5670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c02aa00 tx=0x7f3a4c0315b0 comp rx=0 tx=0).stop 2024-09-19T04:39:02.429 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:39:02.427+0000 7f3a634a9640 1 -- 172.21.15.183:0/1660774898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a5c1026d0 msgr2=0x7f3a5c19f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:39:02.429 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:39:02.427+0000 7f3a634a9640 1 --2- 172.21.15.183:0/1660774898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a5c1026d0 0x7f3a5c19f230 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400cad0 tx=0x7f3a5400ced0 comp rx=0 tx=0).stop 2024-09-19T04:39:02.429 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:39:02.428+0000 7f3a634a9640 1 -- 172.21.15.183:0/1660774898 shutdown_connections 2024-09-19T04:39:02.429 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:39:02.428+0000 7f3a634a9640 1 --2- 172.21.15.183:0/1660774898 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3a480a31b0 0x7f3a480a5670 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:02.429 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:39:02.428+0000 7f3a634a9640 1 --2- 172.21.15.183:0/1660774898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a5c1026d0 0x7f3a5c19f230 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:02.429 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:39:02.428+0000 7f3a634a9640 1 --2- 172.21.15.183:0/1660774898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a5c101d10 0x7f3a5c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:02.429 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:39:02.428+0000 7f3a634a9640 1 --2- 172.21.15.183:0/1660774898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a5c100b10 0x7f3a5c06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:02.429 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:39:02.428+0000 7f3a634a9640 1 -- 172.21.15.183:0/1660774898 >> 172.21.15.183:0/1660774898 conn(0x7f3a5c0fc2a0 msgr2=0x7f3a5c105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:39:02.430 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:39:02.428+0000 7f3a634a9640 1 -- 172.21.15.183:0/1660774898 shutdown_connections 2024-09-19T04:39:02.430 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:39:02.428+0000 7f3a634a9640 1 -- 172.21.15.183:0/1660774898 wait complete. 2024-09-19T04:39:02.728 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:02 smithi183 ceph-mon[35333]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:39:02.728 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:39:02.728 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:02.728 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:02.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:02 smithi123 ceph-mon[30511]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:39:02.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:39:02.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:02.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:02.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:02 smithi193 ceph-mon[35592]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:39:02.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:39:02.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:02.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:03.021 DEBUG:teuthology.orchestra.run.smithi183:osd.1> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T04:39:03.023 INFO:tasks.cephadm:Deploying osd.2 on smithi193 with /dev/vg_nvme/lv_4... 2024-09-19T04:39:03.023 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T04:39:03.277 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:39:03.687 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:03 smithi193 ceph-mon[35592]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:39:03.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:03 smithi183 ceph-mon[35333]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:39:03.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:03 smithi123 ceph-mon[30511]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:39:04.445 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:39:04 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-1[39806]: 2024-09-19T04:39:04.362+0000 7feabe315640 -1 osd.1 0 log_to_monitors true 2024-09-19T04:39:04.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:04 smithi193 ceph-mon[35592]: from='osd.1 [v2:172.21.15.183:6800/1259721614,v1:172.21.15.183:6801/1259721614]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:39:04.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:04 smithi193 ceph-mon[35592]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:39:04.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:04 smithi183 ceph-mon[35333]: from='osd.1 [v2:172.21.15.183:6800/1259721614,v1:172.21.15.183:6801/1259721614]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:39:04.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:04 smithi183 ceph-mon[35333]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:39:04.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:04 smithi123 ceph-mon[30511]: from='osd.1 [v2:172.21.15.183:6800/1259721614,v1:172.21.15.183:6801/1259721614]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:39:04.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:04 smithi123 ceph-mon[30511]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:39:05.666 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:05 smithi193 ceph-mon[35592]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T04:39:05.666 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:05 smithi193 ceph-mon[35592]: from='osd.1 [v2:172.21.15.183:6800/1259721614,v1:172.21.15.183:6801/1259721614]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:39:05.666 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:05 smithi193 ceph-mon[35592]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T04:39:05.666 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:05 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:39:05.666 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:05 smithi193 ceph-mon[35592]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:39:05.666 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:05 smithi193 ceph-mon[35592]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:39:05.666 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:05 smithi193 ceph-mon[35592]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-19T04:39:05.666 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:05 smithi193 ceph-mon[35592]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T04:39:05.666 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:05 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:39:05.668 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:39:05.695 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_4 2024-09-19T04:39:05.850 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:39:05 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-1[39806]: 2024-09-19T04:39:05.584+0000 7feaba298640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T04:39:05.850 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:39:05 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-1[39806]: 2024-09-19T04:39:05.587+0000 7feab60c0640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:39:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:05 smithi183 ceph-mon[35333]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T04:39:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:05 smithi183 ceph-mon[35333]: from='osd.1 [v2:172.21.15.183:6800/1259721614,v1:172.21.15.183:6801/1259721614]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:39:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:05 smithi183 ceph-mon[35333]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T04:39:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:05 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:39:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:05 smithi183 ceph-mon[35333]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:39:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:05 smithi183 ceph-mon[35333]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:39:05.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:05 smithi183 ceph-mon[35333]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-19T04:39:05.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:05 smithi183 ceph-mon[35333]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T04:39:05.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:05 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:39:05.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:05 smithi123 ceph-mon[30511]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T04:39:05.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:05 smithi123 ceph-mon[30511]: from='osd.1 [v2:172.21.15.183:6800/1259721614,v1:172.21.15.183:6801/1259721614]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:39:05.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:05 smithi123 ceph-mon[30511]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T04:39:05.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:05 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:39:05.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:05 smithi123 ceph-mon[30511]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:39:05.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:05 smithi123 ceph-mon[30511]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:39:05.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:05 smithi123 ceph-mon[30511]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-19T04:39:05.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:05 smithi123 ceph-mon[30511]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T04:39:05.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:05 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:39:05.946 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:39:06.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.376+0000 7f89828dc640 1 -- 172.21.15.193:0/432872992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f897c101c00 msgr2=0x7f897c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:39:06.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.376+0000 7f89828dc640 1 --2- 172.21.15.193:0/432872992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f897c101c00 0x7f897c102000 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f896c009900 tx=0x7f896c02a410 comp rx=0 tx=0).stop 2024-09-19T04:39:06.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.377+0000 7f89828dc640 1 -- 172.21.15.193:0/432872992 shutdown_connections 2024-09-19T04:39:06.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.377+0000 7f89828dc640 1 --2- 172.21.15.193:0/432872992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f897c1037c0 0x7f897c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:06.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.377+0000 7f89828dc640 1 --2- 172.21.15.193:0/432872992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f897c102e00 0x7f897c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:06.381 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.377+0000 7f89828dc640 1 --2- 172.21.15.193:0/432872992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f897c101c00 0x7f897c102000 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:06.381 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.377+0000 7f89828dc640 1 -- 172.21.15.193:0/432872992 >> 172.21.15.193:0/432872992 conn(0x7f897c0fd390 msgr2=0x7f897c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:39:06.381 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.377+0000 7f89828dc640 1 -- 172.21.15.193:0/432872992 shutdown_connections 2024-09-19T04:39:06.381 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.377+0000 7f89828dc640 1 -- 172.21.15.193:0/432872992 wait complete. 2024-09-19T04:39:06.381 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.377+0000 7f89828dc640 1 Processor -- start 2024-09-19T04:39:06.381 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.377+0000 7f89828dc640 1 -- start start 2024-09-19T04:39:06.381 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f89828dc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f897c101c00 0x7f897c197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:39:06.381 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f89828dc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f897c102e00 0x7f897c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:39:06.381 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f89828dc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f897c1037c0 0x7f897c19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:39:06.381 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f89828dc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f897c10d8e0 con 0x7f897c101c00 2024-09-19T04:39:06.381 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f89828dc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f897c10d760 con 0x7f897c102e00 2024-09-19T04:39:06.381 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f89828dc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f897c10da60 con 0x7f897c1037c0 2024-09-19T04:39:06.382 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f8980e79640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f897c1037c0 0x7f897c19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:39:06.382 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f8980e79640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f897c1037c0 0x7f897c19f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:39616/0 (socket says 172.21.15.193:39616) 2024-09-19T04:39:06.382 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f8980e79640 1 -- 172.21.15.193:0/1646730952 learned_addr learned my addr 172.21.15.193:0/1646730952 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-19T04:39:06.382 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f897bfff640 1 --2- 172.21.15.193:0/1646730952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f897c101c00 0x7f897c197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:39:06.382 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f897b7fe640 1 --2- 172.21.15.193:0/1646730952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f897c102e00 0x7f897c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:39:06.382 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f8980e79640 1 -- 172.21.15.193:0/1646730952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f897c102e00 msgr2=0x7f897c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:39:06.382 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f8980e79640 1 --2- 172.21.15.193:0/1646730952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f897c102e00 0x7f897c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:06.382 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f8980e79640 1 -- 172.21.15.193:0/1646730952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f897c101c00 msgr2=0x7f897c197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:39:06.382 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f8980e79640 1 --2- 172.21.15.193:0/1646730952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f897c101c00 0x7f897c197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:06.382 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f8980e79640 1 -- 172.21.15.193:0/1646730952 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c19fbe0 con 0x7f897c1037c0 2024-09-19T04:39:06.382 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f897b7fe640 1 --2- 172.21.15.193:0/1646730952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f897c102e00 0x7f897c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:39:06.382 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f8980e79640 1 --2- 172.21.15.193:0/1646730952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f897c1037c0 0x7f897c19f570 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f897400cad0 tx=0x7f897400cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:39:06.382 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f89797fa640 1 -- 172.21.15.193:0/1646730952 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897400ae20 con 0x7f897c1037c0 2024-09-19T04:39:06.383 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f89797fa640 1 -- 172.21.15.193:0/1646730952 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8974005020 con 0x7f897c1037c0 2024-09-19T04:39:06.383 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f89828dc640 1 -- 172.21.15.193:0/1646730952 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897c19fed0 con 0x7f897c1037c0 2024-09-19T04:39:06.383 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f89797fa640 1 -- 172.21.15.193:0/1646730952 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897400f6f0 con 0x7f897c1037c0 2024-09-19T04:39:06.383 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.378+0000 7f89828dc640 1 -- 172.21.15.193:0/1646730952 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f897c1a0400 con 0x7f897c1037c0 2024-09-19T04:39:06.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.379+0000 7f89828dc640 1 -- 172.21.15.193:0/1646730952 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f897c108ff0 con 0x7f897c1037c0 2024-09-19T04:39:06.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.380+0000 7f89797fa640 1 -- 172.21.15.193:0/1646730952 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f89740051d0 con 0x7f897c1037c0 2024-09-19T04:39:06.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.381+0000 7f89797fa640 1 --2- 172.21.15.193:0/1646730952 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f89640a31e0 0x7f89640a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:39:06.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.381+0000 7f89797fa640 1 -- 172.21.15.193:0/1646730952 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f89740ce400 con 0x7f897c1037c0 2024-09-19T04:39:06.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.381+0000 7f897bfff640 1 --2- 172.21.15.193:0/1646730952 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f89640a31e0 0x7f89640a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:39:06.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.382+0000 7f897bfff640 1 --2- 172.21.15.193:0/1646730952 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f89640a31e0 0x7f89640a56a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f896c006010 tx=0x7f896c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:39:06.386 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.384+0000 7f89797fa640 1 -- 172.21.15.193:0/1646730952 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8974014030 con 0x7f897c1037c0 2024-09-19T04:39:06.560 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:06.558+0000 7f89828dc640 1 -- 172.21.15.193:0/1646730952 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f897c062630 con 0x7f89640a31e0 2024-09-19T04:39:06.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:39:06.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: Detected new or changed devices on smithi183 2024-09-19T04:39:06.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:39:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-19T04:39:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:39:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:39:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:39:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:39:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:39:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: osd.1 [v2:172.21.15.183:6800/1259721614,v1:172.21.15.183:6801/1259721614] boot 2024-09-19T04:39:06.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T04:39:06.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:39:06.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:39:06.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: Detected new or changed devices on smithi183 2024-09-19T04:39:06.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:06.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:06.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:39:06.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-19T04:39:06.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:06.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:06.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:39:06.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:06.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:39:06.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:39:06.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:39:06.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:06.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:39:06.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: osd.1 [v2:172.21.15.183:6800/1259721614,v1:172.21.15.183:6801/1259721614] boot 2024-09-19T04:39:06.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T04:39:06.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:39:06.965 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:39:06.965 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: Detected new or changed devices on smithi183 2024-09-19T04:39:06.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:06.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:06.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:39:06.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-19T04:39:06.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:06.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:06.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:39:06.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:06.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:39:06.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:39:06.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:39:06.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:06.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:39:06.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: osd.1 [v2:172.21.15.183:6800/1259721614,v1:172.21.15.183:6801/1259721614] boot 2024-09-19T04:39:06.967 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T04:39:06.967 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:39:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:07 smithi123 ceph-mon[30511]: purged_snaps scrub starts 2024-09-19T04:39:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:07 smithi123 ceph-mon[30511]: purged_snaps scrub ok 2024-09-19T04:39:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:07 smithi123 ceph-mon[30511]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:39:07.882 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:07 smithi193 ceph-mon[35592]: purged_snaps scrub starts 2024-09-19T04:39:07.882 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:07 smithi193 ceph-mon[35592]: purged_snaps scrub ok 2024-09-19T04:39:07.882 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:07 smithi193 ceph-mon[35592]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:39:08.016 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:07 smithi183 ceph-mon[35333]: purged_snaps scrub starts 2024-09-19T04:39:08.016 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:07 smithi183 ceph-mon[35333]: purged_snaps scrub ok 2024-09-19T04:39:08.016 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:07 smithi183 ceph-mon[35333]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:39:08.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:08 smithi123 ceph-mon[30511]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T04:39:08.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:08 smithi193 ceph-mon[35592]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T04:39:09.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:08 smithi183 ceph-mon[35333]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T04:39:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:09 smithi123 ceph-mon[30511]: from='client.? 172.21.15.193:0/1445187120' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3650601a-6259-4b2a-97df-1d47b09b0127"}]: dispatch 2024-09-19T04:39:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:09 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3650601a-6259-4b2a-97df-1d47b09b0127"}]: dispatch 2024-09-19T04:39:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:09 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3650601a-6259-4b2a-97df-1d47b09b0127"}]': finished 2024-09-19T04:39:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:09 smithi123 ceph-mon[30511]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T04:39:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:09 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:09 smithi123 ceph-mon[30511]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:09 smithi123 ceph-mon[30511]: from='client.? 172.21.15.193:0/2229421977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:39:09.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:09 smithi193 ceph-mon[35592]: from='client.? 172.21.15.193:0/1445187120' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3650601a-6259-4b2a-97df-1d47b09b0127"}]: dispatch 2024-09-19T04:39:09.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:09 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3650601a-6259-4b2a-97df-1d47b09b0127"}]: dispatch 2024-09-19T04:39:09.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:09 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3650601a-6259-4b2a-97df-1d47b09b0127"}]': finished 2024-09-19T04:39:09.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:09 smithi193 ceph-mon[35592]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T04:39:09.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:09 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:09.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:09 smithi193 ceph-mon[35592]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:09.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:09 smithi193 ceph-mon[35592]: from='client.? 172.21.15.193:0/2229421977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:39:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:09 smithi183 ceph-mon[35333]: from='client.? 172.21.15.193:0/1445187120' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3650601a-6259-4b2a-97df-1d47b09b0127"}]: dispatch 2024-09-19T04:39:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:09 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3650601a-6259-4b2a-97df-1d47b09b0127"}]: dispatch 2024-09-19T04:39:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:09 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3650601a-6259-4b2a-97df-1d47b09b0127"}]': finished 2024-09-19T04:39:10.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:09 smithi183 ceph-mon[35333]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T04:39:10.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:09 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:10.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:09 smithi183 ceph-mon[35333]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:10.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:09 smithi183 ceph-mon[35333]: from='client.? 172.21.15.193:0/2229421977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:39:12.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:12 smithi183 ceph-mon[35333]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:12.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:12 smithi123 ceph-mon[30511]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:12.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:12 smithi193 ceph-mon[35592]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:14.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:14 smithi183 ceph-mon[35333]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:14.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:14 smithi123 ceph-mon[30511]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:14.690 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:14 smithi193 ceph-mon[35592]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:16.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:16 smithi193 ceph-mon[35592]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:16.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:16 smithi183 ceph-mon[35333]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:16.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:16 smithi123 ceph-mon[30511]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:17.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T04:39:17.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:17.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T04:39:17.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:17.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T04:39:17.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:18.655 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:18 smithi193 ceph-mon[35592]: Deploying daemon osd.2 on smithi193 2024-09-19T04:39:18.655 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:18 smithi193 ceph-mon[35592]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:18.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:18 smithi123 ceph-mon[30511]: Deploying daemon osd.2 on smithi193 2024-09-19T04:39:18.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:18 smithi123 ceph-mon[30511]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:18.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:18 smithi183 ceph-mon[35333]: Deploying daemon osd.2 on smithi193 2024-09-19T04:39:18.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:18 smithi183 ceph-mon[35333]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:20.580 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:20 smithi193 ceph-mon[35592]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:20 smithi123 ceph-mon[30511]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:20.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:20 smithi183 ceph-mon[35333]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:22.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:22 smithi123 ceph-mon[30511]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:22.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:22 smithi193 ceph-mon[35592]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:22 smithi183 ceph-mon[35333]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:24.557 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:24 smithi193 ceph-mon[35592]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:24.557 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:24 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:39:24.557 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:24 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:24.557 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:24 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:24.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:24 smithi123 ceph-mon[30511]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:24.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:24 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:39:24.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:24 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:24.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:24 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:24.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:24 smithi183 ceph-mon[35333]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:24.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:24 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:39:24.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:24 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:24.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:24 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:26.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:26 smithi183 ceph-mon[35333]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:26.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:26.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:26.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:26.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:39:26.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:26.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:26 smithi123 ceph-mon[30511]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:26.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:26.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:26.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:26.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:39:26.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:26.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:26 smithi193 ceph-mon[35592]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:26.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:26.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:26.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:26.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:39:26.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:27.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:27.696+0000 7f89797fa640 1 -- 172.21.15.193:0/1646730952 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f897c062630 con 0x7f89640a31e0 2024-09-19T04:39:27.698 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 2 on host 'smithi193' 2024-09-19T04:39:27.700 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:27.699+0000 7f89828dc640 1 -- 172.21.15.193:0/1646730952 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f89640a31e0 msgr2=0x7f89640a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:39:27.701 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:27.699+0000 7f89828dc640 1 --2- 172.21.15.193:0/1646730952 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f89640a31e0 0x7f89640a56a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f896c006010 tx=0x7f896c000bc0 comp rx=0 tx=0).stop 2024-09-19T04:39:27.701 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:27.699+0000 7f89828dc640 1 -- 172.21.15.193:0/1646730952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f897c1037c0 msgr2=0x7f897c19f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:39:27.701 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:27.699+0000 7f89828dc640 1 --2- 172.21.15.193:0/1646730952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f897c1037c0 0x7f897c19f570 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f897400cad0 tx=0x7f897400cf90 comp rx=0 tx=0).stop 2024-09-19T04:39:27.702 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:27.700+0000 7f89828dc640 1 -- 172.21.15.193:0/1646730952 shutdown_connections 2024-09-19T04:39:27.702 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:27.700+0000 7f89828dc640 1 --2- 172.21.15.193:0/1646730952 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f89640a31e0 0x7f89640a56a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:27.702 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:27.700+0000 7f89828dc640 1 --2- 172.21.15.193:0/1646730952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f897c1037c0 0x7f897c19f570 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:27.702 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:27.700+0000 7f89828dc640 1 --2- 172.21.15.193:0/1646730952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f897c102e00 0x7f897c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:27.702 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:27.700+0000 7f89828dc640 1 --2- 172.21.15.193:0/1646730952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f897c101c00 0x7f897c197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:27.702 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:27.700+0000 7f89828dc640 1 -- 172.21.15.193:0/1646730952 >> 172.21.15.193:0/1646730952 conn(0x7f897c0fd390 msgr2=0x7f897c1061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:39:27.702 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:27.700+0000 7f89828dc640 1 -- 172.21.15.193:0/1646730952 shutdown_connections 2024-09-19T04:39:27.702 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:39:27.700+0000 7f89828dc640 1 -- 172.21.15.193:0/1646730952 wait complete. 2024-09-19T04:39:28.292 DEBUG:teuthology.orchestra.run.smithi193:osd.2> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T04:39:28.295 INFO:tasks.cephadm:Deploying osd.3 on smithi123 with /dev/vg_nvme/lv_3... 2024-09-19T04:39:28.295 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T04:39:28.530 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:28 smithi193 ceph-mon[35592]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:28.530 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:28 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:39:28.531 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:28 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:28.531 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:28 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:28.666 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:39:28.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:28 smithi123 ceph-mon[30511]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:28.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:28 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:39:28.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:28 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:28.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:28 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:28.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:28 smithi183 ceph-mon[35333]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:28.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:28 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:39:28.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:28 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:28.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:28 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:29.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:29 smithi123 ceph-mon[30511]: from='osd.2 [v2:172.21.15.193:6800/2708151726,v1:172.21.15.193:6801/2708151726]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:39:29.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:29 smithi123 ceph-mon[30511]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:39:29.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:29 smithi193 ceph-mon[35592]: from='osd.2 [v2:172.21.15.193:6800/2708151726,v1:172.21.15.193:6801/2708151726]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:39:29.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:29 smithi193 ceph-mon[35592]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:39:29.617 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:39:29 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-2[40557]: 2024-09-19T04:39:29.346+0000 7f2c9e265640 -1 osd.2 0 log_to_monitors true 2024-09-19T04:39:29.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:29 smithi183 ceph-mon[35333]: from='osd.2 [v2:172.21.15.193:6800/2708151726,v1:172.21.15.193:6801/2708151726]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:39:29.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:29 smithi183 ceph-mon[35333]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:39:30.480 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:30 smithi123 ceph-mon[30511]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:30.480 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:30 smithi123 ceph-mon[30511]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T04:39:30.480 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:30 smithi123 ceph-mon[30511]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T04:39:30.480 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:30 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:30.480 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:30 smithi123 ceph-mon[30511]: from='osd.2 [v2:172.21.15.193:6800/2708151726,v1:172.21.15.193:6801/2708151726]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:39:30.481 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:30 smithi123 ceph-mon[30511]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:39:30.740 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:39:30 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-2[40557]: 2024-09-19T04:39:30.379+0000 7f2c9a1e8640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T04:39:30.741 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:39:30 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-2[40557]: 2024-09-19T04:39:30.382+0000 7f2c96010640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:39:30.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:30 smithi193 ceph-mon[35592]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:30.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:30 smithi193 ceph-mon[35592]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T04:39:30.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:30 smithi193 ceph-mon[35592]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T04:39:30.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:30 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:30.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:30 smithi193 ceph-mon[35592]: from='osd.2 [v2:172.21.15.193:6800/2708151726,v1:172.21.15.193:6801/2708151726]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:39:30.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:30 smithi193 ceph-mon[35592]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:39:30.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:30 smithi183 ceph-mon[35333]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:30.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:30 smithi183 ceph-mon[35333]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T04:39:30.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:30 smithi183 ceph-mon[35333]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T04:39:30.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:30 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:30.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:30 smithi183 ceph-mon[35333]: from='osd.2 [v2:172.21.15.193:6800/2708151726,v1:172.21.15.193:6801/2708151726]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:39:30.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:30 smithi183 ceph-mon[35333]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:39:31.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:31 smithi193 ceph-mon[35592]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-19T04:39:31.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:31 smithi193 ceph-mon[35592]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T04:39:31.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:31 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:31.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:31 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:31.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:31 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:31.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:31 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:31.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:31 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:39:31.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:31 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:31.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:31 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:31.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:31 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:39:31.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:31 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:31.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:31 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:31 smithi123 ceph-mon[30511]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-19T04:39:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:31 smithi123 ceph-mon[30511]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T04:39:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:31 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:31 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:31 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:31 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:31 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:39:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:31 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:31 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:31.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:31 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:39:31.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:31 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:31.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:31 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:31.750 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:39:31.783 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_3 2024-09-19T04:39:31.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:31 smithi183 ceph-mon[35333]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-19T04:39:31.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:31 smithi183 ceph-mon[35333]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T04:39:31.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:31 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:31.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:31 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:31.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:31 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:31.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:31 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:31.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:31 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:39:31.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:31 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:31.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:31 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:31.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:31 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:39:31.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:31 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:31.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:31 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:32.153 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:39:32.564 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:32 smithi123 ceph-mon[30511]: purged_snaps scrub starts 2024-09-19T04:39:32.564 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:32 smithi123 ceph-mon[30511]: purged_snaps scrub ok 2024-09-19T04:39:32.564 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:32 smithi123 ceph-mon[30511]: Detected new or changed devices on smithi193 2024-09-19T04:39:32.564 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:32 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-19T04:39:32.564 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:32 smithi123 ceph-mon[30511]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:32.564 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:32 smithi123 ceph-mon[30511]: osd.2 [v2:172.21.15.193:6800/2708151726,v1:172.21.15.193:6801/2708151726] boot 2024-09-19T04:39:32.564 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:32 smithi123 ceph-mon[30511]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T04:39:32.564 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:32 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:32.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.678+0000 7f25e9c00640 1 -- 172.21.15.123:0/1181267563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25e4103c50 msgr2=0x7f25e4104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:39:32.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.678+0000 7f25e9c00640 1 --2- 172.21.15.123:0/1181267563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25e4103c50 0x7f25e4104050 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f25d4009900 tx=0x7f25d402a410 comp rx=0 tx=0).stop 2024-09-19T04:39:32.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.678+0000 7f25e9c00640 1 -- 172.21.15.123:0/1181267563 shutdown_connections 2024-09-19T04:39:32.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.678+0000 7f25e9c00640 1 --2- 172.21.15.123:0/1181267563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25e4105810 0x7f25e410c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:32.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.678+0000 7f25e9c00640 1 --2- 172.21.15.123:0/1181267563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25e4104e50 0x7f25e41052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:32.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.678+0000 7f25e9c00640 1 --2- 172.21.15.123:0/1181267563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25e4103c50 0x7f25e4104050 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:32.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.678+0000 7f25e9c00640 1 -- 172.21.15.123:0/1181267563 >> 172.21.15.123:0/1181267563 conn(0x7f25e40ff3e0 msgr2=0x7f25e4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:39:32.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.678+0000 7f25e9c00640 1 -- 172.21.15.123:0/1181267563 shutdown_connections 2024-09-19T04:39:32.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.678+0000 7f25e9c00640 1 -- 172.21.15.123:0/1181267563 wait complete. 2024-09-19T04:39:32.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.679+0000 7f25e9c00640 1 Processor -- start 2024-09-19T04:39:32.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.679+0000 7f25e9c00640 1 -- start start 2024-09-19T04:39:32.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.680+0000 7f25e9c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25e4103c50 0x7f25e4199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:39:32.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.680+0000 7f25e9c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25e4104e50 0x7f25e419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:39:32.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.680+0000 7f25e9c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25e4105810 0x7f25e41a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:39:32.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.680+0000 7f25e9c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25e410f900 con 0x7f25e4105810 2024-09-19T04:39:32.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.680+0000 7f25e9c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f25e410f780 con 0x7f25e4104e50 2024-09-19T04:39:32.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.680+0000 7f25e9c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f25e410fa80 con 0x7f25e4103c50 2024-09-19T04:39:32.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.680+0000 7f25e3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25e4103c50 0x7f25e4199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:39:32.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.680+0000 7f25e3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25e4103c50 0x7f25e4199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:32784/0 (socket says 172.21.15.123:32784) 2024-09-19T04:39:32.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.680+0000 7f25e3e00640 1 -- 172.21.15.123:0/1113859042 learned_addr learned my addr 172.21.15.123:0/1113859042 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:39:32.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.680+0000 7f25e9200640 1 --2- 172.21.15.123:0/1113859042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25e4105810 0x7f25e41a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:39:32.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.680+0000 7f25e3400640 1 --2- 172.21.15.123:0/1113859042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25e4104e50 0x7f25e419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:39:32.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.680+0000 7f25e3e00640 1 -- 172.21.15.123:0/1113859042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25e4104e50 msgr2=0x7f25e419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:39:32.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.680+0000 7f25e3e00640 1 --2- 172.21.15.123:0/1113859042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25e4104e50 0x7f25e419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:32.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.680+0000 7f25e3e00640 1 -- 172.21.15.123:0/1113859042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25e4105810 msgr2=0x7f25e41a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:39:32.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.680+0000 7f25e3e00640 1 --2- 172.21.15.123:0/1113859042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25e4105810 0x7f25e41a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:32.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.681+0000 7f25e3e00640 1 -- 172.21.15.123:0/1113859042 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e41a1b20 con 0x7f25e4103c50 2024-09-19T04:39:32.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.681+0000 7f25e3400640 1 --2- 172.21.15.123:0/1113859042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25e4104e50 0x7f25e419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:39:32.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.681+0000 7f25e9200640 1 --2- 172.21.15.123:0/1113859042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25e4105810 0x7f25e41a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:39:32.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.681+0000 7f25e3e00640 1 --2- 172.21.15.123:0/1113859042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25e4103c50 0x7f25e4199ef0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f25d4031440 tx=0x7f25d4032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:39:32.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.681+0000 7f25e0c00640 1 -- 172.21.15.123:0/1113859042 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d403f030 con 0x7f25e4103c50 2024-09-19T04:39:32.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.681+0000 7f25e9c00640 1 -- 172.21.15.123:0/1113859042 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e41a1db0 con 0x7f25e4103c50 2024-09-19T04:39:32.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.681+0000 7f25e0c00640 1 -- 172.21.15.123:0/1113859042 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25d4003ce0 con 0x7f25e4103c50 2024-09-19T04:39:32.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.681+0000 7f25e0c00640 1 -- 172.21.15.123:0/1113859042 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4035070 con 0x7f25e4103c50 2024-09-19T04:39:32.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.681+0000 7f25e9c00640 1 -- 172.21.15.123:0/1113859042 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e41a2200 con 0x7f25e4103c50 2024-09-19T04:39:32.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.682+0000 7f25e9c00640 1 -- 172.21.15.123:0/1113859042 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e410afa0 con 0x7f25e4103c50 2024-09-19T04:39:32.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.684+0000 7f25e0c00640 1 -- 172.21.15.123:0/1113859042 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f25d4042030 con 0x7f25e4103c50 2024-09-19T04:39:32.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.686+0000 7f25e0c00640 1 --2- 172.21.15.123:0/1113859042 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f25c80a31e0 0x7f25c80a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:39:32.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.686+0000 7f25e0c00640 1 -- 172.21.15.123:0/1113859042 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7f25d40eec30 con 0x7f25e4103c50 2024-09-19T04:39:32.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.686+0000 7f25e3400640 1 --2- 172.21.15.123:0/1113859042 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f25c80a31e0 0x7f25c80a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:39:32.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.686+0000 7f25e3400640 1 --2- 172.21.15.123:0/1113859042 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f25c80a31e0 0x7f25c80a56a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f25e419b410 tx=0x7f25d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:39:32.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.688+0000 7f25e0c00640 1 -- 172.21.15.123:0/1113859042 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f25d40bc890 con 0x7f25e4103c50 2024-09-19T04:39:32.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:32 smithi193 ceph-mon[35592]: purged_snaps scrub starts 2024-09-19T04:39:32.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:32 smithi193 ceph-mon[35592]: purged_snaps scrub ok 2024-09-19T04:39:32.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:32 smithi193 ceph-mon[35592]: Detected new or changed devices on smithi193 2024-09-19T04:39:32.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:32 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-19T04:39:32.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:32 smithi193 ceph-mon[35592]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:32.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:32 smithi193 ceph-mon[35592]: osd.2 [v2:172.21.15.193:6800/2708151726,v1:172.21.15.193:6801/2708151726] boot 2024-09-19T04:39:32.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:32 smithi193 ceph-mon[35592]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T04:39:32.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:32 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:32.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:32 smithi183 ceph-mon[35333]: purged_snaps scrub starts 2024-09-19T04:39:32.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:32 smithi183 ceph-mon[35333]: purged_snaps scrub ok 2024-09-19T04:39:32.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:32 smithi183 ceph-mon[35333]: Detected new or changed devices on smithi193 2024-09-19T04:39:32.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:32 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-19T04:39:32.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:32 smithi183 ceph-mon[35333]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:39:32.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:32 smithi183 ceph-mon[35333]: osd.2 [v2:172.21.15.193:6800/2708151726,v1:172.21.15.193:6801/2708151726] boot 2024-09-19T04:39:32.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:32 smithi183 ceph-mon[35333]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T04:39:32.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:32 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:39:32.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:32.870+0000 7f25e9c00640 1 -- 172.21.15.123:0/1113859042 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f25e4062630 con 0x7f25c80a31e0 2024-09-19T04:39:34.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:39:34.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:39:34.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:34.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:33 smithi123 ceph-mon[30511]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T04:39:34.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:39:34.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:39:34.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:34.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:33 smithi193 ceph-mon[35592]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T04:39:34.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:39:34.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:39:34.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:34.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:33 smithi183 ceph-mon[35333]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T04:39:35.091 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:34 smithi123 ceph-mon[30511]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:39:35.091 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:34 smithi123 ceph-mon[30511]: pgmap v79: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:35.091 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:34 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:39:35.091 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:34 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:39:35.091 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:34 smithi123 ceph-mon[30511]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T04:39:35.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:34 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:39:35.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:34 smithi193 ceph-mon[35592]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:39:35.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:34 smithi193 ceph-mon[35592]: pgmap v79: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:35.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:34 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:39:35.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:34 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:39:35.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:34 smithi193 ceph-mon[35592]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T04:39:35.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:34 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:39:35.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:34 smithi183 ceph-mon[35333]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:39:35.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:34 smithi183 ceph-mon[35333]: pgmap v79: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:35.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:34 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:39:35.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:34 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:39:35.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:34 smithi183 ceph-mon[35333]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T04:39:35.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:34 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:39:36.177 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:39:35 smithi123 sudo[48571]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:39:36.177 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:39:35 smithi123 sudo[48571]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:39:36.177 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:39:35 smithi123 sudo[48571]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:39:36.177 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:39:35 smithi123 sudo[48571]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:39:36.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:36.219+0000 7f25e0c00640 1 -- 172.21.15.123:0/1113859042 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f25d403bd10 con 0x7f25e4103c50 2024-09-19T04:39:36.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:36 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:39:36.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:36 smithi193 ceph-mon[35592]: osdmap e21: 3 total, 3 up, 3 in 2024-09-19T04:39:36.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:36 smithi193 ceph-mon[35592]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:36.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:36 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2937671197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67c485ad-136e-4020-950d-9171a3c2de35"}]: dispatch 2024-09-19T04:39:36.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:36 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2937671197' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67c485ad-136e-4020-950d-9171a3c2de35"}]': finished 2024-09-19T04:39:36.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:36 smithi193 ceph-mon[35592]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T04:39:36.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:36 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:39:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:36 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:39:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:36 smithi183 ceph-mon[35333]: osdmap e21: 3 total, 3 up, 3 in 2024-09-19T04:39:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:36 smithi183 ceph-mon[35333]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:36 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2937671197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67c485ad-136e-4020-950d-9171a3c2de35"}]: dispatch 2024-09-19T04:39:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:36 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2937671197' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67c485ad-136e-4020-950d-9171a3c2de35"}]': finished 2024-09-19T04:39:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:36 smithi183 ceph-mon[35333]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T04:39:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:36 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:39:36.607 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:39:36 smithi123 sudo[48605]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:39:36.607 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:39:36 smithi123 sudo[48605]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:39:36.607 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:39:36 smithi123 sudo[48605]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:39:36.607 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:39:36 smithi123 sudo[48605]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:39:36.607 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:36 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:39:36.607 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:36 smithi123 ceph-mon[30511]: osdmap e21: 3 total, 3 up, 3 in 2024-09-19T04:39:36.607 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:36 smithi123 ceph-mon[30511]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:36.607 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:36 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2937671197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67c485ad-136e-4020-950d-9171a3c2de35"}]: dispatch 2024-09-19T04:39:36.608 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:36 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2937671197' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67c485ad-136e-4020-950d-9171a3c2de35"}]': finished 2024-09-19T04:39:36.608 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:36 smithi123 ceph-mon[30511]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T04:39:36.608 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:36 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:39:36.960 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:39:36 smithi183 sudo[43498]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:39:36.961 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:39:36 smithi183 sudo[43498]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:39:36.961 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:39:36 smithi183 sudo[43498]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:39:36.961 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:39:36 smithi183 sudo[43498]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:39:37.217 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:39:36 smithi183 sudo[43501]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:39:37.217 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:39:36 smithi183 sudo[43501]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:39:37.217 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:39:36 smithi183 sudo[43501]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:39:37.217 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:39:36 smithi183 sudo[43501]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:39:37.485 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:37 smithi193 ceph-mon[35592]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:39:37.486 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:37 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:37.486 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:37 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1431858450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:39:37.486 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:37 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:37.486 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:39:37 smithi193 sudo[44184]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:39:37.486 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:39:37 smithi193 sudo[44184]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:39:37.486 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:39:37 smithi193 sudo[44184]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:39:37.486 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:39:37 smithi193 sudo[44184]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:39:37.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:37 smithi183 ceph-mon[35333]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:39:37.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:37 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:37.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:37 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1431858450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:39:37.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:37 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:37.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:37 smithi123 ceph-mon[30511]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:39:37.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:37 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:37.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:37 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1431858450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:39:37.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:37 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:37.740 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:39:37 smithi193 sudo[44187]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:39:37.740 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:39:37 smithi193 sudo[44187]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:39:37.741 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:39:37 smithi193 sudo[44187]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:39:37.741 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:39:37 smithi193 sudo[44187]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:39:38.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:38 smithi123 sudo[49030]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:39:38.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:38 smithi123 sudo[49030]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:39:38.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:38 smithi123 sudo[49030]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:39:38.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:38 smithi123 sudo[49030]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:39:38.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:38 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:38.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:38 smithi183 ceph-mon[35333]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:39:38.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:38 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:38.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:38 smithi193 ceph-mon[35592]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:39:38.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:38 smithi193 sudo[44190]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:39:38.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:38 smithi193 sudo[44190]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:39:38.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:38 smithi193 sudo[44190]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:39:38.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:38 smithi193 sudo[44190]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:39:38.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:38 smithi123 sudo[49042]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:39:38.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:38 smithi123 sudo[49042]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:39:38.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:38 smithi123 sudo[49042]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:39:38.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:38 smithi123 sudo[49042]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:39:38.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:38 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:38.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:38 smithi123 ceph-mon[30511]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:39:39.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 sudo[44193]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:39:39.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 sudo[44193]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:39:39.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 sudo[44193]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:39:39.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 sudo[44193]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:39:39.593 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 sudo[43506]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:39:39.593 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 sudo[43506]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:39:39.593 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 sudo[43506]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:39:39.593 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 sudo[43506]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:39:39.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 sudo[43509]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:39:39.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 sudo[43509]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:39:39.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 sudo[43509]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:39:39.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 sudo[43509]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:39:39.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 ceph-mon[35333]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:39.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 ceph-mon[35333]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:39:39.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:39:39.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:39:39.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:39:39.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 ceph-mon[35333]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:39:39.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:39:39.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:39:39.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:39:39.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 ceph-mon[35333]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:39:39.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:39:39.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:39:39.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:39:39.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:39 smithi183 ceph-mon[35333]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:39.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:39 smithi123 ceph-mon[30511]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:39.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:39 smithi123 ceph-mon[30511]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:39:39.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:39 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:39:39.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:39 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:39:39.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:39 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:39:39.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:39 smithi123 ceph-mon[30511]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:39:39.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:39 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:39:39.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:39 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:39:39.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:39 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:39:39.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:39 smithi123 ceph-mon[30511]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:39:39.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:39 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:39:39.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:39 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:39:39.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:39 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:39:39.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:39 smithi123 ceph-mon[30511]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:39.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 ceph-mon[35592]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:39.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 ceph-mon[35592]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:39:39.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:39:39.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:39:39.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:39:39.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 ceph-mon[35592]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:39:39.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:39:39.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:39:39.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:39:39.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 ceph-mon[35592]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:39:39.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:39:39.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:39:39.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:39:39.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:39 smithi193 ceph-mon[35592]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:40.861 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:40 smithi123 ceph-mon[30511]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:39:40.861 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:40 smithi123 ceph-mon[30511]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:39:40.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:40 smithi193 ceph-mon[35592]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:39:40.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:40 smithi193 ceph-mon[35592]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:39:41.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:40 smithi183 ceph-mon[35333]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:39:41.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:40 smithi183 ceph-mon[35333]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:39:41.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:41 smithi123 ceph-mon[30511]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:41.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:41 smithi193 ceph-mon[35592]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:42.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:41 smithi183 ceph-mon[35333]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:44.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:44 smithi123 ceph-mon[30511]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T04:39:44.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:44 smithi183 ceph-mon[35333]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T04:39:44.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:44 smithi193 ceph-mon[35592]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T04:39:45.620 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:45 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:39:45.620 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:45 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:45.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:45 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:39:45.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:45 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:45.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:45 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:39:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:45 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:46.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:46 smithi183 ceph-mon[35333]: Deploying daemon osd.3 on smithi123 2024-09-19T04:39:46.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:46 smithi183 ceph-mon[35333]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T04:39:46.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:46 smithi123 ceph-mon[30511]: Deploying daemon osd.3 on smithi123 2024-09-19T04:39:46.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:46 smithi123 ceph-mon[30511]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T04:39:46.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:46 smithi193 ceph-mon[35592]: Deploying daemon osd.3 on smithi123 2024-09-19T04:39:46.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:46 smithi193 ceph-mon[35592]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T04:39:48.426 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:48 smithi123 ceph-mon[30511]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T04:39:48.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:48 smithi193 ceph-mon[35592]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T04:39:48.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:48 smithi183 ceph-mon[35333]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T04:39:50.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:50 smithi123 ceph-mon[30511]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T04:39:50.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:50 smithi193 ceph-mon[35592]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T04:39:50.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:50 smithi183 ceph-mon[35333]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T04:39:52.376 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:52 smithi123 ceph-mon[30511]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:39:52.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:52 smithi193 ceph-mon[35592]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:39:52.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:52 smithi183 ceph-mon[35333]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:39:53.471 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:39:53.471 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:53.472 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:53.642 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:39:53.642 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:53.642 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:53 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:39:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:53 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:53 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:54.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:54 smithi123 ceph-mon[30511]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:39:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:54 smithi183 ceph-mon[35333]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:39:54.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:54 smithi193 ceph-mon[35592]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:39:56.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:56 smithi183 ceph-mon[35333]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:56.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:56 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:56.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:56 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:56.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:56 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:56.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:56 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:39:56.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:56 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:56.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:56 smithi123 ceph-mon[30511]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:56.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:56 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:56.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:56 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:56.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:56 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:56.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:56 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:39:56.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:56 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:56.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:56 smithi193 ceph-mon[35592]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:56.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:56 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:56.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:56 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:56.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:56 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:39:56.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:56 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:39:56.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:56 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:57.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:57.270+0000 7f25e0c00640 1 -- 172.21.15.123:0/1113859042 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f25e4062630 con 0x7f25c80a31e0 2024-09-19T04:39:57.273 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 3 on host 'smithi123' 2024-09-19T04:39:57.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:57.273+0000 7f25e9c00640 1 -- 172.21.15.123:0/1113859042 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f25c80a31e0 msgr2=0x7f25c80a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:39:57.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:57.273+0000 7f25e9c00640 1 --2- 172.21.15.123:0/1113859042 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f25c80a31e0 0x7f25c80a56a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f25e419b410 tx=0x7f25d0005f90 comp rx=0 tx=0).stop 2024-09-19T04:39:57.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:57.273+0000 7f25e9c00640 1 -- 172.21.15.123:0/1113859042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25e4103c50 msgr2=0x7f25e4199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:39:57.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:57.273+0000 7f25e9c00640 1 --2- 172.21.15.123:0/1113859042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25e4103c50 0x7f25e4199ef0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f25d4031440 tx=0x7f25d4032ce0 comp rx=0 tx=0).stop 2024-09-19T04:39:57.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:57.273+0000 7f25e9c00640 1 -- 172.21.15.123:0/1113859042 shutdown_connections 2024-09-19T04:39:57.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:57.273+0000 7f25e9c00640 1 --2- 172.21.15.123:0/1113859042 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f25c80a31e0 0x7f25c80a56a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:57.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:57.273+0000 7f25e9c00640 1 --2- 172.21.15.123:0/1113859042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25e4105810 0x7f25e41a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:57.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:57.273+0000 7f25e9c00640 1 --2- 172.21.15.123:0/1113859042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25e4104e50 0x7f25e419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:57.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:57.273+0000 7f25e9c00640 1 --2- 172.21.15.123:0/1113859042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25e4103c50 0x7f25e4199ef0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:39:57.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:57.273+0000 7f25e9c00640 1 -- 172.21.15.123:0/1113859042 >> 172.21.15.123:0/1113859042 conn(0x7f25e40ff3e0 msgr2=0x7f25e4108170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:39:57.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:57.273+0000 7f25e9c00640 1 -- 172.21.15.123:0/1113859042 shutdown_connections 2024-09-19T04:39:57.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:39:57.273+0000 7f25e9c00640 1 -- 172.21.15.123:0/1113859042 wait complete. 2024-09-19T04:39:57.671 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:57 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:39:57.671 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:57 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:57.671 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:57 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:57.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:57 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:39:57.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:57 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:57.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:57 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:57.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:39:57.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:57.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:39:58.053 DEBUG:teuthology.orchestra.run.smithi123:osd.3> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T04:39:58.058 INFO:tasks.cephadm:Deploying osd.4 on smithi183 with /dev/vg_nvme/lv_3... 2024-09-19T04:39:58.059 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T04:39:58.313 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:39:58.535 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:58 smithi123 ceph-mon[30511]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:58.672 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:58 smithi183 ceph-mon[35333]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:58.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:58 smithi193 ceph-mon[35592]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:39:58.927 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:39:58 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3[50614]: 2024-09-19T04:39:58.601+0000 7ff88bc71640 -1 osd.3 0 log_to_monitors true 2024-09-19T04:39:59.615 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:39:59 smithi123 ceph-mon[30511]: from='osd.3 [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:39:59.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:39:59 smithi193 ceph-mon[35592]: from='osd.3 [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:39:59.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:39:59 smithi183 ceph-mon[35333]: from='osd.3 [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:40:00.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:00 smithi123 ceph-mon[30511]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:40:00.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:00 smithi123 ceph-mon[30511]: from='osd.3 [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:40:00.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:00 smithi123 ceph-mon[30511]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T04:40:00.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:00 smithi123 ceph-mon[30511]: from='osd.3 [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-19T04:40:00.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:00 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:40:00.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:00 smithi123 ceph-mon[30511]: overall HEALTH_OK 2024-09-19T04:40:00.678 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:40:00 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3[50614]: 2024-09-19T04:40:00.432+0000 7ff886800640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T04:40:00.678 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:40:00 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3[50614]: 2024-09-19T04:40:00.438+0000 7ff881800640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:40:00.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:00 smithi193 ceph-mon[35592]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:40:00.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:00 smithi193 ceph-mon[35592]: from='osd.3 [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:40:00.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:00 smithi193 ceph-mon[35592]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T04:40:00.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:00 smithi193 ceph-mon[35592]: from='osd.3 [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-19T04:40:00.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:00 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:40:00.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:00 smithi193 ceph-mon[35592]: overall HEALTH_OK 2024-09-19T04:40:00.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:00 smithi183 ceph-mon[35333]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:40:00.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:00 smithi183 ceph-mon[35333]: from='osd.3 [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:40:00.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:00 smithi183 ceph-mon[35333]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T04:40:00.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:00 smithi183 ceph-mon[35333]: from='osd.3 [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-19T04:40:00.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:00 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:40:00.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:00 smithi183 ceph-mon[35333]: overall HEALTH_OK 2024-09-19T04:40:00.970 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:40:00.995 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_3 2024-09-19T04:40:01.239 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:40:01.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:01 smithi123 ceph-mon[30511]: from='osd.3 [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-19T04:40:01.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:01 smithi123 ceph-mon[30511]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T04:40:01.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:01 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:40:01.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:01 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:40:01.681 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:01 smithi183 ceph-mon[35333]: from='osd.3 [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-19T04:40:01.681 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:01 smithi183 ceph-mon[35333]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T04:40:01.681 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:01 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:40:01.681 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:01 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:40:01.705 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.678+0000 7ffb10ce6640 1 -- 172.21.15.183:0/3878379661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb0c102e00 msgr2=0x7ffb0c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:40:01.705 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.678+0000 7ffb10ce6640 1 --2- 172.21.15.183:0/3878379661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb0c102e00 0x7ffb0c103280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009900 tx=0x7ffaf402a410 comp rx=0 tx=0).stop 2024-09-19T04:40:01.705 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.678+0000 7ffb10ce6640 1 -- 172.21.15.183:0/3878379661 shutdown_connections 2024-09-19T04:40:01.705 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.678+0000 7ffb10ce6640 1 --2- 172.21.15.183:0/3878379661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb0c1037c0 0x7ffb0c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:01.705 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.678+0000 7ffb10ce6640 1 --2- 172.21.15.183:0/3878379661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb0c102e00 0x7ffb0c103280 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:01.705 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.678+0000 7ffb10ce6640 1 --2- 172.21.15.183:0/3878379661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb0c101c00 0x7ffb0c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:01.705 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.678+0000 7ffb10ce6640 1 -- 172.21.15.183:0/3878379661 >> 172.21.15.183:0/3878379661 conn(0x7ffb0c0fd390 msgr2=0x7ffb0c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:40:01.705 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.678+0000 7ffb10ce6640 1 -- 172.21.15.183:0/3878379661 shutdown_connections 2024-09-19T04:40:01.705 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.678+0000 7ffb10ce6640 1 -- 172.21.15.183:0/3878379661 wait complete. 2024-09-19T04:40:01.705 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.679+0000 7ffb10ce6640 1 Processor -- start 2024-09-19T04:40:01.706 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.679+0000 7ffb10ce6640 1 -- start start 2024-09-19T04:40:01.706 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb10ce6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb0c101c00 0x7ffb0c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:40:01.706 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb10ce6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb0c102e00 0x7ffb0c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:40:01.706 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb10ce6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb0c1037c0 0x7ffb0c19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:40:01.706 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb10ce6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c10d880 con 0x7ffb0c101c00 2024-09-19T04:40:01.706 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb10ce6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c10d700 con 0x7ffb0c1037c0 2024-09-19T04:40:01.706 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb10ce6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c10da00 con 0x7ffb0c102e00 2024-09-19T04:40:01.706 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb0ad9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb0c1037c0 0x7ffb0c19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:40:01.706 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb0ad9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb0c1037c0 0x7ffb0c19f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:33450/0 (socket says 172.21.15.183:33450) 2024-09-19T04:40:01.706 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb0ad9d640 1 -- 172.21.15.183:0/2218157776 learned_addr learned my addr 172.21.15.183:0/2218157776 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-19T04:40:01.706 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb0a59c640 1 --2- 172.21.15.183:0/2218157776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb0c101c00 0x7ffb0c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:40:01.707 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb01d9b640 1 --2- 172.21.15.183:0/2218157776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb0c102e00 0x7ffb0c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:40:01.707 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb0ad9d640 1 -- 172.21.15.183:0/2218157776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb0c102e00 msgr2=0x7ffb0c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:40:01.707 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb0ad9d640 1 --2- 172.21.15.183:0/2218157776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb0c102e00 0x7ffb0c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:01.707 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb0ad9d640 1 -- 172.21.15.183:0/2218157776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb0c101c00 msgr2=0x7ffb0c197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:40:01.707 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb0ad9d640 1 --2- 172.21.15.183:0/2218157776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb0c101c00 0x7ffb0c197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:01.707 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb0ad9d640 1 -- 172.21.15.183:0/2218157776 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0c19fb60 con 0x7ffb0c1037c0 2024-09-19T04:40:01.707 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb0ad9d640 1 --2- 172.21.15.183:0/2218157776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb0c1037c0 0x7ffb0c19f4f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400eaa0 tx=0x7ffb0400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:40:01.707 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.680+0000 7ffb03fff640 1 -- 172.21.15.183:0/2218157776 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0400cdb0 con 0x7ffb0c1037c0 2024-09-19T04:40:01.707 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.681+0000 7ffb03fff640 1 -- 172.21.15.183:0/2218157776 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb04005020 con 0x7ffb0c1037c0 2024-09-19T04:40:01.707 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.681+0000 7ffb10ce6640 1 -- 172.21.15.183:0/2218157776 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb0c19fe50 con 0x7ffb0c1037c0 2024-09-19T04:40:01.707 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.681+0000 7ffb03fff640 1 -- 172.21.15.183:0/2218157776 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb040116b0 con 0x7ffb0c1037c0 2024-09-19T04:40:01.707 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.681+0000 7ffb10ce6640 1 -- 172.21.15.183:0/2218157776 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb0c1a02b0 con 0x7ffb0c1037c0 2024-09-19T04:40:01.707 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.682+0000 7ffb10ce6640 1 -- 172.21.15.183:0/2218157776 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb0c1090a0 con 0x7ffb0c1037c0 2024-09-19T04:40:01.708 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.683+0000 7ffb03fff640 1 -- 172.21.15.183:0/2218157776 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ffb04005560 con 0x7ffb0c1037c0 2024-09-19T04:40:01.708 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.683+0000 7ffb03fff640 1 --2- 172.21.15.183:0/2218157776 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ffaf80a3270 0x7ffaf80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:40:01.708 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.683+0000 7ffb03fff640 1 -- 172.21.15.183:0/2218157776 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3702+0+0 (secure 0 0 0) 0x7ffb040ce940 con 0x7ffb0c1037c0 2024-09-19T04:40:01.708 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.684+0000 7ffb0a59c640 1 --2- 172.21.15.183:0/2218157776 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ffaf80a3270 0x7ffaf80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:40:01.708 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.684+0000 7ffb0a59c640 1 --2- 172.21.15.183:0/2218157776 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ffaf80a3270 0x7ffaf80a5730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c102c60 tx=0x7ffafc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:40:01.708 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.689+0000 7ffb03fff640 1 -- 172.21.15.183:0/2218157776 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ffb04016030 con 0x7ffb0c1037c0 2024-09-19T04:40:01.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:01 smithi193 ceph-mon[35592]: from='osd.3 [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-19T04:40:01.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:01 smithi193 ceph-mon[35592]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T04:40:01.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:01 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:40:01.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:01 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:40:01.866 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:01.864+0000 7ffb10ce6640 1 -- 172.21.15.183:0/2218157776 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ffb0c062630 con 0x7ffaf80a3270 2024-09-19T04:40:02.548 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: purged_snaps scrub starts 2024-09-19T04:40:02.548 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: purged_snaps scrub ok 2024-09-19T04:40:02.549 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:40:02.549 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: osd.3 [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282] boot 2024-09-19T04:40:02.549 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T04:40:02.549 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:40:02.549 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: Detected new or changed devices on smithi123 2024-09-19T04:40:02.549 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:02.549 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:02.549 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:02.549 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:02.549 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi123 to 8544M 2024-09-19T04:40:02.549 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:02.549 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:02.549 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:40:02.549 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:02.550 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T04:40:02.550 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:40:02.550 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:40:02.550 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:02.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: purged_snaps scrub starts 2024-09-19T04:40:02.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: purged_snaps scrub ok 2024-09-19T04:40:02.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:40:02.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: osd.3 [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282] boot 2024-09-19T04:40:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T04:40:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:40:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: Detected new or changed devices on smithi123 2024-09-19T04:40:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi123 to 8544M 2024-09-19T04:40:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:40:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T04:40:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:40:02.679 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:40:02.679 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:02.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: purged_snaps scrub starts 2024-09-19T04:40:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: purged_snaps scrub ok 2024-09-19T04:40:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:40:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: osd.3 [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282] boot 2024-09-19T04:40:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T04:40:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:40:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: Detected new or changed devices on smithi123 2024-09-19T04:40:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi123 to 8544M 2024-09-19T04:40:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:02.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:02.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:40:02.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:02.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T04:40:02.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:40:02.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:40:02.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:03.573 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:03 smithi183 ceph-mon[35333]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:40:03.573 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:03 smithi183 ceph-mon[35333]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T04:40:03.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:03 smithi123 ceph-mon[30511]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:40:03.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:03 smithi123 ceph-mon[30511]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T04:40:03.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:03 smithi193 ceph-mon[35592]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:40:03.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:03 smithi193 ceph-mon[35592]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T04:40:04.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:04 smithi193 ceph-mon[35592]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:04.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:04 smithi193 ceph-mon[35592]: from='client.? 172.21.15.183:0/3826960300' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9098540-06ab-45a8-9cf9-f90519cc852e"}]: dispatch 2024-09-19T04:40:04.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:04 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9098540-06ab-45a8-9cf9-f90519cc852e"}]: dispatch 2024-09-19T04:40:04.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:04 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9098540-06ab-45a8-9cf9-f90519cc852e"}]': finished 2024-09-19T04:40:04.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:04 smithi193 ceph-mon[35592]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T04:40:04.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:04 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:04.848 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:04 smithi183 ceph-mon[35333]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:04.848 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:04 smithi183 ceph-mon[35333]: from='client.? 172.21.15.183:0/3826960300' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9098540-06ab-45a8-9cf9-f90519cc852e"}]: dispatch 2024-09-19T04:40:04.848 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:04 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9098540-06ab-45a8-9cf9-f90519cc852e"}]: dispatch 2024-09-19T04:40:04.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:04 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9098540-06ab-45a8-9cf9-f90519cc852e"}]': finished 2024-09-19T04:40:04.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:04 smithi183 ceph-mon[35333]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T04:40:04.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:04.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:04 smithi123 ceph-mon[30511]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:04.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:04 smithi123 ceph-mon[30511]: from='client.? 172.21.15.183:0/3826960300' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9098540-06ab-45a8-9cf9-f90519cc852e"}]: dispatch 2024-09-19T04:40:04.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:04 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9098540-06ab-45a8-9cf9-f90519cc852e"}]: dispatch 2024-09-19T04:40:04.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:04 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9098540-06ab-45a8-9cf9-f90519cc852e"}]': finished 2024-09-19T04:40:04.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:04 smithi123 ceph-mon[30511]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T04:40:04.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:04 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:05.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:05 smithi193 ceph-mon[35592]: from='client.? 172.21.15.183:0/1957265429' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:40:05.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:05 smithi193 ceph-mon[35592]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T04:40:05.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:05 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:05 smithi183 ceph-mon[35333]: from='client.? 172.21.15.183:0/1957265429' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:40:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:05 smithi183 ceph-mon[35333]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T04:40:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:05 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:05.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:05 smithi123 ceph-mon[30511]: from='client.? 172.21.15.183:0/1957265429' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:40:05.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:05 smithi123 ceph-mon[30511]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T04:40:05.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:05 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:06.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:06 smithi193 ceph-mon[35592]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:06.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:06 smithi183 ceph-mon[35333]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:06.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:06 smithi123 ceph-mon[30511]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:08.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:08 smithi193 ceph-mon[35592]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:08.834 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:08 smithi183 ceph-mon[35333]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:08.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:08 smithi123 ceph-mon[30511]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:10.513 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:10 smithi183 ceph-mon[35333]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T04:40:10.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:10 smithi193 ceph-mon[35592]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T04:40:10.827 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:10 smithi123 ceph-mon[30511]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T04:40:12.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:12 smithi193 ceph-mon[35592]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:40:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:12 smithi183 ceph-mon[35333]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:40:12.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:12 smithi123 ceph-mon[30511]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:40:13.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:13 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T04:40:13.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:13 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:13.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:13 smithi193 ceph-mon[35592]: Deploying daemon osd.4 on smithi183 2024-09-19T04:40:13.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:13 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T04:40:13.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:13 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:13.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:13 smithi183 ceph-mon[35333]: Deploying daemon osd.4 on smithi183 2024-09-19T04:40:13.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T04:40:13.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:13.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:13 smithi123 ceph-mon[30511]: Deploying daemon osd.4 on smithi183 2024-09-19T04:40:14.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:14 smithi193 ceph-mon[35592]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T04:40:14.836 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:14 smithi183 ceph-mon[35333]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T04:40:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:14 smithi123 ceph-mon[30511]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T04:40:16.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:16 smithi193 ceph-mon[35592]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:40:16.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:16 smithi183 ceph-mon[35333]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:40:16.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:16 smithi123 ceph-mon[30511]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:40:18.576 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:18 smithi183 ceph-mon[35333]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:40:18.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:18 smithi193 ceph-mon[35592]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:40:18.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:18 smithi123 ceph-mon[30511]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:40:19.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:19 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:40:19.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:19 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:19.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:19 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:19.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:19 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:40:19.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:19 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:19.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:19 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:19.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:19 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:40:19.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:19 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:19.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:19 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:20.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:20 smithi183 ceph-mon[35333]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:40:20.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:20 smithi193 ceph-mon[35592]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:40:20.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:20 smithi123 ceph-mon[30511]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:40:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:22 smithi183 ceph-mon[35333]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:40:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:22.897 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:22.896+0000 7ffb03fff640 1 -- 172.21.15.183:0/2218157776 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ffb0c062630 con 0x7ffaf80a3270 2024-09-19T04:40:22.898 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 4 on host 'smithi183' 2024-09-19T04:40:22.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:22.899+0000 7ffb10ce6640 1 -- 172.21.15.183:0/2218157776 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ffaf80a3270 msgr2=0x7ffaf80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:40:22.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:22.899+0000 7ffb10ce6640 1 --2- 172.21.15.183:0/2218157776 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ffaf80a3270 0x7ffaf80a5730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c102c60 tx=0x7ffafc005f90 comp rx=0 tx=0).stop 2024-09-19T04:40:22.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:22.899+0000 7ffb10ce6640 1 -- 172.21.15.183:0/2218157776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb0c1037c0 msgr2=0x7ffb0c19f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:40:22.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:22.899+0000 7ffb10ce6640 1 --2- 172.21.15.183:0/2218157776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb0c1037c0 0x7ffb0c19f4f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400eaa0 tx=0x7ffb0400ef60 comp rx=0 tx=0).stop 2024-09-19T04:40:22.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:22.899+0000 7ffb10ce6640 1 -- 172.21.15.183:0/2218157776 shutdown_connections 2024-09-19T04:40:22.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:22.899+0000 7ffb10ce6640 1 --2- 172.21.15.183:0/2218157776 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ffaf80a3270 0x7ffaf80a5730 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:22.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:22.899+0000 7ffb10ce6640 1 --2- 172.21.15.183:0/2218157776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb0c1037c0 0x7ffb0c19f4f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:22.901 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:22.899+0000 7ffb10ce6640 1 --2- 172.21.15.183:0/2218157776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb0c102e00 0x7ffb0c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:22.901 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:22.899+0000 7ffb10ce6640 1 --2- 172.21.15.183:0/2218157776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb0c101c00 0x7ffb0c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:22.901 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:22.899+0000 7ffb10ce6640 1 -- 172.21.15.183:0/2218157776 >> 172.21.15.183:0/2218157776 conn(0x7ffb0c0fd390 msgr2=0x7ffb0c106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:40:22.901 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:22.899+0000 7ffb10ce6640 1 -- 172.21.15.183:0/2218157776 shutdown_connections 2024-09-19T04:40:22.901 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:40:22.899+0000 7ffb10ce6640 1 -- 172.21.15.183:0/2218157776 wait complete. 2024-09-19T04:40:22.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:22 smithi123 ceph-mon[30511]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:22.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:22.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:22.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:22.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:40:22.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:22.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:22 smithi193 ceph-mon[35592]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:22.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:22.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:22.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:22.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:40:22.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:23.520 DEBUG:teuthology.orchestra.run.smithi183:osd.4> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T04:40:23.523 INFO:tasks.cephadm:Deploying osd.5 on smithi193 with /dev/vg_nvme/lv_3... 2024-09-19T04:40:23.523 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T04:40:23.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:23 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:40:23.978 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:23 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:23.978 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:23 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:23.979 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:23 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:40:23.979 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:23 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:23.979 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:23 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:23.979 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:40:23.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:23 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:40:23.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:23 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:23.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:23 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:24.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:24 smithi123 ceph-mon[30511]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:24.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:24 smithi183 ceph-mon[35333]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:24.907 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:24 smithi193 ceph-mon[35592]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:25.118 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:40:24 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4[46244]: 2024-09-19T04:40:24.867+0000 7f7b5e296640 -1 osd.4 0 log_to_monitors true 2024-09-19T04:40:25.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:25 smithi183 ceph-mon[35333]: from='osd.4 [v2:172.21.15.183:6808/667618665,v1:172.21.15.183:6809/667618665]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:40:25.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:25 smithi183 ceph-mon[35333]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:40:25.856 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:25 smithi123 ceph-mon[30511]: from='osd.4 [v2:172.21.15.183:6808/667618665,v1:172.21.15.183:6809/667618665]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:40:25.856 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:25 smithi123 ceph-mon[30511]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:40:25.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:25 smithi193 ceph-mon[35592]: from='osd.4 [v2:172.21.15.183:6808/667618665,v1:172.21.15.183:6809/667618665]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:40:25.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:25 smithi193 ceph-mon[35592]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:40:26.301 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:40:26.329 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_3 2024-09-19T04:40:26.577 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:40:26.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:26 smithi193 ceph-mon[35592]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:26.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:26 smithi193 ceph-mon[35592]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T04:40:26.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:26 smithi193 ceph-mon[35592]: from='osd.4 [v2:172.21.15.183:6808/667618665,v1:172.21.15.183:6809/667618665]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:40:26.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:26 smithi193 ceph-mon[35592]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T04:40:26.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:26.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:26 smithi193 ceph-mon[35592]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:40:26.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:26.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:26.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:26.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:26.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:26.834 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:26.834 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:40:26.834 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:26.850 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:40:26 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4[46244]: 2024-09-19T04:40:26.590+0000 7f7b5a219640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T04:40:26.850 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:40:26 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4[46244]: 2024-09-19T04:40:26.595+0000 7f7b56041640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:40:26.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:26 smithi183 ceph-mon[35333]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:26.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:26 smithi183 ceph-mon[35333]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T04:40:26.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:26 smithi183 ceph-mon[35333]: from='osd.4 [v2:172.21.15.183:6808/667618665,v1:172.21.15.183:6809/667618665]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:40:26.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:26 smithi183 ceph-mon[35333]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T04:40:26.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:26.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:26 smithi183 ceph-mon[35333]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:40:26.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:26.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:26.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:26.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:26.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:26.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:26.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:40:26.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:26.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:26 smithi123 ceph-mon[30511]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:26.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:26 smithi123 ceph-mon[30511]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T04:40:26.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:26 smithi123 ceph-mon[30511]: from='osd.4 [v2:172.21.15.183:6808/667618665,v1:172.21.15.183:6809/667618665]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:40:26.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:26 smithi123 ceph-mon[30511]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T04:40:26.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:26.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:26 smithi123 ceph-mon[30511]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:40:26.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:26.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:26.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:26.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:26.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:26.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:26.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:40:26.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:27.042 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.023+0000 7ff725acb640 1 -- 172.21.15.193:0/1980422680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff720103870 msgr2=0x7ff72010a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:40:27.042 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.023+0000 7ff725acb640 1 --2- 172.21.15.193:0/1980422680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff720103870 0x7ff72010a100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff718009cd0 tx=0x7ff71802a8e0 comp rx=0 tx=0).stop 2024-09-19T04:40:27.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.023+0000 7ff725acb640 1 -- 172.21.15.193:0/1980422680 shutdown_connections 2024-09-19T04:40:27.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.023+0000 7ff725acb640 1 --2- 172.21.15.193:0/1980422680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff720103870 0x7ff72010a100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:27.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.023+0000 7ff725acb640 1 --2- 172.21.15.193:0/1980422680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff720102eb0 0x7ff720103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:27.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.023+0000 7ff725acb640 1 --2- 172.21.15.193:0/1980422680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7200ffed0 0x7ff7201002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:27.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.023+0000 7ff725acb640 1 -- 172.21.15.193:0/1980422680 >> 172.21.15.193:0/1980422680 conn(0x7ff7200fba40 msgr2=0x7ff7200fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:40:27.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.023+0000 7ff725acb640 1 -- 172.21.15.193:0/1980422680 shutdown_connections 2024-09-19T04:40:27.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.023+0000 7ff725acb640 1 -- 172.21.15.193:0/1980422680 wait complete. 2024-09-19T04:40:27.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.024+0000 7ff725acb640 1 Processor -- start 2024-09-19T04:40:27.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.024+0000 7ff725acb640 1 -- start start 2024-09-19T04:40:27.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.024+0000 7ff725acb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7200ffed0 0x7ff720102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:40:27.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.024+0000 7ff725acb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff720102eb0 0x7ff720100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:40:27.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.024+0000 7ff725acb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff720103870 0x7ff7201013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:40:27.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.024+0000 7ff725acb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff72010d890 con 0x7ff720103870 2024-09-19T04:40:27.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.024+0000 7ff725acb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff72010d710 con 0x7ff720102eb0 2024-09-19T04:40:27.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.024+0000 7ff725acb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff72010da10 con 0x7ff7200ffed0 2024-09-19T04:40:27.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.024+0000 7ff71f7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7200ffed0 0x7ff720102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:40:27.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.024+0000 7ff71f7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7200ffed0 0x7ff720102810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:48928/0 (socket says 172.21.15.193:48928) 2024-09-19T04:40:27.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.025+0000 7ff71f7fe640 1 -- 172.21.15.193:0/1963903719 learned_addr learned my addr 172.21.15.193:0/1963903719 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-19T04:40:27.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.025+0000 7ff71ffff640 1 --2- 172.21.15.193:0/1963903719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff720103870 0x7ff7201013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:40:27.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.025+0000 7ff71effd640 1 --2- 172.21.15.193:0/1963903719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff720102eb0 0x7ff720100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:40:27.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.025+0000 7ff71f7fe640 1 -- 172.21.15.193:0/1963903719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff720102eb0 msgr2=0x7ff720100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:40:27.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.025+0000 7ff71f7fe640 1 --2- 172.21.15.193:0/1963903719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff720102eb0 0x7ff720100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:27.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.025+0000 7ff71f7fe640 1 -- 172.21.15.193:0/1963903719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff720103870 msgr2=0x7ff7201013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:40:27.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.025+0000 7ff71f7fe640 1 --2- 172.21.15.193:0/1963903719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff720103870 0x7ff7201013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:27.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.025+0000 7ff71f7fe640 1 -- 172.21.15.193:0/1963903719 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff720101c60 con 0x7ff7200ffed0 2024-09-19T04:40:27.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.025+0000 7ff71effd640 1 --2- 172.21.15.193:0/1963903719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff720102eb0 0x7ff720100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:40:27.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.025+0000 7ff71ffff640 1 --2- 172.21.15.193:0/1963903719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff720103870 0x7ff7201013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:40:27.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.025+0000 7ff71f7fe640 1 --2- 172.21.15.193:0/1963903719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7200ffed0 0x7ff720102810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff71000d950 tx=0x7ff71000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:40:27.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.025+0000 7ff71cff9640 1 -- 172.21.15.193:0/1963903719 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710017020 con 0x7ff7200ffed0 2024-09-19T04:40:27.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.025+0000 7ff725acb640 1 -- 172.21.15.193:0/1963903719 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7201a20a0 con 0x7ff7200ffed0 2024-09-19T04:40:27.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.025+0000 7ff71cff9640 1 -- 172.21.15.193:0/1963903719 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff710009c30 con 0x7ff7200ffed0 2024-09-19T04:40:27.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.025+0000 7ff71cff9640 1 -- 172.21.15.193:0/1963903719 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71000f6b0 con 0x7ff7200ffed0 2024-09-19T04:40:27.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.025+0000 7ff725acb640 1 -- 172.21.15.193:0/1963903719 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7201a2600 con 0x7ff7200ffed0 2024-09-19T04:40:27.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.026+0000 7ff725acb640 1 -- 172.21.15.193:0/1963903719 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7201002d0 con 0x7ff7200ffed0 2024-09-19T04:40:27.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.027+0000 7ff71cff9640 1 -- 172.21.15.193:0/1963903719 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff71001a020 con 0x7ff7200ffed0 2024-09-19T04:40:27.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.028+0000 7ff71cff9640 1 --2- 172.21.15.193:0/1963903719 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff7080a3240 0x7ff7080a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:40:27.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.028+0000 7ff71cff9640 1 -- 172.21.15.193:0/1963903719 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7ff7100ce830 con 0x7ff7200ffed0 2024-09-19T04:40:27.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.028+0000 7ff71effd640 1 --2- 172.21.15.193:0/1963903719 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff7080a3240 0x7ff7080a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:40:27.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.028+0000 7ff71effd640 1 --2- 172.21.15.193:0/1963903719 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff7080a3240 0x7ff7080a5700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff714005fd0 tx=0x7ff7140094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:40:27.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.032+0000 7ff71cff9640 1 -- 172.21.15.193:0/1963903719 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff710021020 con 0x7ff7200ffed0 2024-09-19T04:40:27.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:27.207+0000 7ff725acb640 1 -- 172.21.15.193:0/1963903719 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff720062630 con 0x7ff7080a3240 2024-09-19T04:40:27.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:27 smithi183 ceph-mon[35333]: Detected new or changed devices on smithi183 2024-09-19T04:40:27.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:27 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-19T04:40:27.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:27 smithi183 ceph-mon[35333]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-19T04:40:27.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:27 smithi183 ceph-mon[35333]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T04:40:27.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:27 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:27.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:27 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:27.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:27 smithi183 ceph-mon[35333]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:40:27.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:27 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:40:27.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:27 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:40:27.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:27 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:27.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:27 smithi183 ceph-mon[35333]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:27.862 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:27 smithi193 ceph-mon[35592]: Detected new or changed devices on smithi183 2024-09-19T04:40:27.862 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:27 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-19T04:40:27.862 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:27 smithi193 ceph-mon[35592]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-19T04:40:27.862 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:27 smithi193 ceph-mon[35592]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T04:40:27.862 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:27 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:27.862 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:27 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:27.862 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:27 smithi193 ceph-mon[35592]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:40:27.862 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:27 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:40:27.863 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:27 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:40:27.863 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:27 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:27.863 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:27 smithi193 ceph-mon[35592]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:27.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:27 smithi123 ceph-mon[30511]: Detected new or changed devices on smithi183 2024-09-19T04:40:27.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:27 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-19T04:40:27.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:27 smithi123 ceph-mon[30511]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-19T04:40:27.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:27 smithi123 ceph-mon[30511]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T04:40:27.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:27 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:27.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:27 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:27.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:27 smithi123 ceph-mon[30511]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:40:27.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:27 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:40:27.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:27 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:40:27.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:27 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:27.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:27 smithi123 ceph-mon[30511]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:40:28.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:28 smithi193 ceph-mon[35592]: purged_snaps scrub starts 2024-09-19T04:40:28.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:28 smithi193 ceph-mon[35592]: purged_snaps scrub ok 2024-09-19T04:40:28.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:28 smithi193 ceph-mon[35592]: osd.4 [v2:172.21.15.183:6808/667618665,v1:172.21.15.183:6809/667618665] boot 2024-09-19T04:40:28.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:28 smithi193 ceph-mon[35592]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T04:40:28.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:28 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:28.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:28 smithi183 ceph-mon[35333]: purged_snaps scrub starts 2024-09-19T04:40:28.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:28 smithi183 ceph-mon[35333]: purged_snaps scrub ok 2024-09-19T04:40:28.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:28 smithi183 ceph-mon[35333]: osd.4 [v2:172.21.15.183:6808/667618665,v1:172.21.15.183:6809/667618665] boot 2024-09-19T04:40:28.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:28 smithi183 ceph-mon[35333]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T04:40:28.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:28 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:28.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:28 smithi123 ceph-mon[30511]: purged_snaps scrub starts 2024-09-19T04:40:28.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:28 smithi123 ceph-mon[30511]: purged_snaps scrub ok 2024-09-19T04:40:28.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:28 smithi123 ceph-mon[30511]: osd.4 [v2:172.21.15.183:6808/667618665,v1:172.21.15.183:6809/667618665] boot 2024-09-19T04:40:28.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:28 smithi123 ceph-mon[30511]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T04:40:28.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:28 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:40:29.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:29 smithi123 ceph-mon[30511]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T04:40:29.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:29 smithi123 ceph-mon[30511]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:29.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:29 smithi123 ceph-mon[30511]: from='client.? 172.21.15.193:0/364167009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "278a8dbd-6492-42d4-b489-1f832350bc6e"}]: dispatch 2024-09-19T04:40:29.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:29 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "278a8dbd-6492-42d4-b489-1f832350bc6e"}]: dispatch 2024-09-19T04:40:29.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:29 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "278a8dbd-6492-42d4-b489-1f832350bc6e"}]': finished 2024-09-19T04:40:29.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:29 smithi123 ceph-mon[30511]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T04:40:29.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:29 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:29.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:29 smithi193 ceph-mon[35592]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T04:40:29.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:29 smithi193 ceph-mon[35592]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:29.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:29 smithi193 ceph-mon[35592]: from='client.? 172.21.15.193:0/364167009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "278a8dbd-6492-42d4-b489-1f832350bc6e"}]: dispatch 2024-09-19T04:40:29.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:29 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "278a8dbd-6492-42d4-b489-1f832350bc6e"}]: dispatch 2024-09-19T04:40:29.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:29 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "278a8dbd-6492-42d4-b489-1f832350bc6e"}]': finished 2024-09-19T04:40:29.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:29 smithi193 ceph-mon[35592]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T04:40:29.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:29 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:30.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:29 smithi183 ceph-mon[35333]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T04:40:30.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:29 smithi183 ceph-mon[35333]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:30.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:29 smithi183 ceph-mon[35333]: from='client.? 172.21.15.193:0/364167009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "278a8dbd-6492-42d4-b489-1f832350bc6e"}]: dispatch 2024-09-19T04:40:30.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:29 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "278a8dbd-6492-42d4-b489-1f832350bc6e"}]: dispatch 2024-09-19T04:40:30.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:29 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "278a8dbd-6492-42d4-b489-1f832350bc6e"}]': finished 2024-09-19T04:40:30.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:29 smithi183 ceph-mon[35333]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T04:40:30.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:29 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:30.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:30 smithi123 ceph-mon[30511]: from='client.? 172.21.15.193:0/3505389158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:40:30.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:30 smithi193 ceph-mon[35592]: from='client.? 172.21.15.193:0/3505389158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:40:31.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:30 smithi183 ceph-mon[35333]: from='client.? 172.21.15.193:0/3505389158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:40:31.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:31 smithi123 ceph-mon[30511]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:31.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:31 smithi193 ceph-mon[35592]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:32.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:31 smithi183 ceph-mon[35333]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:34.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:34 smithi123 ceph-mon[30511]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:34.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:34 smithi193 ceph-mon[35592]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:34.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:34 smithi183 ceph-mon[35333]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:36.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:36 smithi193 ceph-mon[35592]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:36.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:36 smithi123 ceph-mon[30511]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:36.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:36 smithi183 ceph-mon[35333]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:38.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:38 smithi193 ceph-mon[35592]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:38.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:38 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T04:40:38.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:38 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:38.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:38 smithi193 ceph-mon[35592]: Deploying daemon osd.5 on smithi193 2024-09-19T04:40:38.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:38 smithi183 ceph-mon[35333]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:38.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:38 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T04:40:38.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:38 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:38.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:38 smithi183 ceph-mon[35333]: Deploying daemon osd.5 on smithi193 2024-09-19T04:40:38.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:38 smithi123 ceph-mon[30511]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:38.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:38 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T04:40:38.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:38 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:38.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:38 smithi123 ceph-mon[30511]: Deploying daemon osd.5 on smithi193 2024-09-19T04:40:40.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:40 smithi193 ceph-mon[35592]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:40.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:40 smithi123 ceph-mon[30511]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:40.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:40 smithi183 ceph-mon[35333]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:42.487 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:42 smithi193 ceph-mon[35592]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:42.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:42 smithi123 ceph-mon[30511]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:42 smithi183 ceph-mon[35333]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:44.455 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:44 smithi193 ceph-mon[35592]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:44.456 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:44 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:40:44.456 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:44 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:44.456 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:44 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:44.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:44 smithi123 ceph-mon[30511]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:44.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:44 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:40:44.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:44 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:44.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:44 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:44 smithi183 ceph-mon[35333]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:44 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:40:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:44 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:44 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:46.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:46 smithi123 ceph-mon[30511]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:46.716 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:46 smithi193 ceph-mon[35592]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:46.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:46 smithi183 ceph-mon[35333]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:47.471 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:47.469+0000 7ff71cff9640 1 -- 172.21.15.193:0/1963903719 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff720062630 con 0x7ff7080a3240 2024-09-19T04:40:47.471 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 5 on host 'smithi193' 2024-09-19T04:40:47.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:47.471+0000 7ff725acb640 1 -- 172.21.15.193:0/1963903719 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff7080a3240 msgr2=0x7ff7080a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:40:47.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:47.471+0000 7ff725acb640 1 --2- 172.21.15.193:0/1963903719 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff7080a3240 0x7ff7080a5700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff714005fd0 tx=0x7ff7140094d0 comp rx=0 tx=0).stop 2024-09-19T04:40:47.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:47.471+0000 7ff725acb640 1 -- 172.21.15.193:0/1963903719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7200ffed0 msgr2=0x7ff720102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:40:47.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:47.471+0000 7ff725acb640 1 --2- 172.21.15.193:0/1963903719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7200ffed0 0x7ff720102810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff71000d950 tx=0x7ff71000de10 comp rx=0 tx=0).stop 2024-09-19T04:40:47.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:47.472+0000 7ff725acb640 1 -- 172.21.15.193:0/1963903719 shutdown_connections 2024-09-19T04:40:47.474 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:47.472+0000 7ff725acb640 1 --2- 172.21.15.193:0/1963903719 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff7080a3240 0x7ff7080a5700 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:47.474 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:47.472+0000 7ff725acb640 1 --2- 172.21.15.193:0/1963903719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff720103870 0x7ff7201013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:47.474 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:47.472+0000 7ff725acb640 1 --2- 172.21.15.193:0/1963903719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff720102eb0 0x7ff720100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:47.474 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:47.472+0000 7ff725acb640 1 --2- 172.21.15.193:0/1963903719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7200ffed0 0x7ff720102810 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:47.474 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:47.472+0000 7ff725acb640 1 -- 172.21.15.193:0/1963903719 >> 172.21.15.193:0/1963903719 conn(0x7ff7200fba40 msgr2=0x7ff7201062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:40:47.474 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:47.472+0000 7ff725acb640 1 -- 172.21.15.193:0/1963903719 shutdown_connections 2024-09-19T04:40:47.474 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:40:47.472+0000 7ff725acb640 1 -- 172.21.15.193:0/1963903719 wait complete. 2024-09-19T04:40:47.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:47 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:47.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:47 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:47.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:47 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:47.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:47 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:40:47.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:47 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:47.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:47 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:40:47.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:47 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:47.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:47 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:47 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:47 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:47 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:47 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:40:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:47 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:47 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:40:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:47 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:47 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:47.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:47 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:47.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:47 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:47.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:47 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:47.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:47 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:40:47.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:47 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:47.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:47 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:40:47.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:47 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:47.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:47 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:48.178 DEBUG:teuthology.orchestra.run.smithi193:osd.5> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T04:40:48.180 INFO:tasks.cephadm:Deploying osd.6 on smithi123 with /dev/vg_nvme/lv_2... 2024-09-19T04:40:48.180 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T04:40:48.550 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:40:48.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:48 smithi193 ceph-mon[35592]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:48.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:48 smithi183 ceph-mon[35333]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:48.880 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:48 smithi123 ceph-mon[30511]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:49.842 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:40:49 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:40:49.731+0000 7f2df8e21640 -1 osd.5 0 log_to_monitors true 2024-09-19T04:40:50.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:50 smithi193 ceph-mon[35592]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:50.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:50 smithi193 ceph-mon[35592]: from='osd.5 [v2:172.21.15.193:6808/2292251639,v1:172.21.15.193:6809/2292251639]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:40:50.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:50 smithi193 ceph-mon[35592]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:40:50.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:50 smithi123 ceph-mon[30511]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:50.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:50 smithi123 ceph-mon[30511]: from='osd.5 [v2:172.21.15.193:6808/2292251639,v1:172.21.15.193:6809/2292251639]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:40:50.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:50 smithi123 ceph-mon[30511]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:40:50.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:50 smithi183 ceph-mon[35333]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:50.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:50 smithi183 ceph-mon[35333]: from='osd.5 [v2:172.21.15.193:6808/2292251639,v1:172.21.15.193:6809/2292251639]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:40:50.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:50 smithi183 ceph-mon[35333]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:40:51.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:51 smithi123 ceph-mon[30511]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T04:40:51.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:51 smithi123 ceph-mon[30511]: from='osd.5 [v2:172.21.15.193:6808/2292251639,v1:172.21.15.193:6809/2292251639]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:40:51.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:51 smithi123 ceph-mon[30511]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T04:40:51.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:51 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:51.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:51 smithi123 ceph-mon[30511]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:40:51.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:51 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:51.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:51 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:51.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:51 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:51.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:51 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:51.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:51 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:51.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:51 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:51.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:51 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:40:51.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:51 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:51.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:51 smithi123 ceph-mon[30511]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-19T04:40:51.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:51 smithi123 ceph-mon[30511]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T04:40:51.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:51 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:51.761 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:40:51.794 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_2 2024-09-19T04:40:51.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:51 smithi183 ceph-mon[35333]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T04:40:51.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:51 smithi183 ceph-mon[35333]: from='osd.5 [v2:172.21.15.193:6808/2292251639,v1:172.21.15.193:6809/2292251639]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:40:51.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:51 smithi183 ceph-mon[35333]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T04:40:51.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:51 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:51.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:51 smithi183 ceph-mon[35333]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:40:51.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:51 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:51.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:51 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:51.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:51 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:51.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:51 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:51.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:51 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:51.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:51 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:51.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:51 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:40:51.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:51 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:51.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:51 smithi183 ceph-mon[35333]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-19T04:40:51.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:51 smithi183 ceph-mon[35333]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T04:40:51.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:51 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:51.896 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-mon[35592]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T04:40:51.896 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-mon[35592]: from='osd.5 [v2:172.21.15.193:6808/2292251639,v1:172.21.15.193:6809/2292251639]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:40:51.896 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-mon[35592]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T04:40:51.896 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:51.896 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-mon[35592]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:40:51.896 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:51.896 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:51.896 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:51.896 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:40:51.897 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:51.897 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:51.897 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:40:51.897 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:40:51.897 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-mon[35592]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-19T04:40:51.897 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-mon[35592]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T04:40:51.897 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:52.176 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:40:52.240 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:40:51.893+0000 7f2df4da4640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T04:40:52.241 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:40:51 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:40:51.899+0000 7f2decbc4640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:40:52.584 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:52 smithi123 ceph-mon[30511]: Detected new or changed devices on smithi193 2024-09-19T04:40:52.584 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:52 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi193 to 8544M 2024-09-19T04:40:52.584 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:52 smithi123 ceph-mon[30511]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:52.584 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:52 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:52.585 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:52 smithi123 ceph-mon[30511]: osdmap e37: 6 total, 5 up, 6 in 2024-09-19T04:40:52.585 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:52 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:52.585 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:52 smithi123 ceph-mon[30511]: from='osd.5 ' entity='osd.5' 2024-09-19T04:40:52.585 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:52 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:52.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.802+0000 7f8ee9200640 1 -- 172.21.15.123:0/3839761831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ee4076ba0 msgr2=0x7f8ee4076fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:40:52.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.802+0000 7f8ee9200640 1 --2- 172.21.15.123:0/3839761831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ee4076ba0 0x7f8ee4076fa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4009900 tx=0x7f8ed402a410 comp rx=0 tx=0).stop 2024-09-19T04:40:52.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.802+0000 7f8ee9200640 1 -- 172.21.15.123:0/3839761831 shutdown_connections 2024-09-19T04:40:52.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.802+0000 7f8ee9200640 1 --2- 172.21.15.123:0/3839761831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ee4078790 0x7f8ee4079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:52.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.802+0000 7f8ee9200640 1 --2- 172.21.15.123:0/3839761831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ee4077df0 0x7f8ee4078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:52.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.802+0000 7f8ee9200640 1 --2- 172.21.15.123:0/3839761831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ee4076ba0 0x7f8ee4076fa0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:52.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.802+0000 7f8ee9200640 1 -- 172.21.15.123:0/3839761831 >> 172.21.15.123:0/3839761831 conn(0x7f8ee40ff5b0 msgr2=0x7f8ee41019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:40:52.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.802+0000 7f8ee9200640 1 -- 172.21.15.123:0/3839761831 shutdown_connections 2024-09-19T04:40:52.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.802+0000 7f8ee9200640 1 -- 172.21.15.123:0/3839761831 wait complete. 2024-09-19T04:40:52.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.803+0000 7f8ee9200640 1 Processor -- start 2024-09-19T04:40:52.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.803+0000 7f8ee9200640 1 -- start start 2024-09-19T04:40:52.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.803+0000 7f8ee9200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ee4076ba0 0x7f8ee419e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:40:52.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.803+0000 7f8ee9200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ee4077df0 0x7f8ee419e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:40:52.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.803+0000 7f8ee9200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ee4078790 0x7f8ee41a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:40:52.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.803+0000 7f8ee9200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee4113e60 con 0x7f8ee4076ba0 2024-09-19T04:40:52.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.803+0000 7f8ee9200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee4113ce0 con 0x7f8ee4078790 2024-09-19T04:40:52.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.803+0000 7f8ee9200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee4113fe0 con 0x7f8ee4077df0 2024-09-19T04:40:52.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.804+0000 7f8ee1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ee4076ba0 0x7f8ee419e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:40:52.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.804+0000 7f8edbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ee4077df0 0x7f8ee419e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:40:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.804+0000 7f8edbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ee4077df0 0x7f8ee419e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:43550/0 (socket says 172.21.15.123:43550) 2024-09-19T04:40:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.804+0000 7f8edbe00640 1 -- 172.21.15.123:0/3780483235 learned_addr learned my addr 172.21.15.123:0/3780483235 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:40:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.804+0000 7f8ee1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ee4076ba0 0x7f8ee419e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35522/0 (socket says 172.21.15.123:35522) 2024-09-19T04:40:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.804+0000 7f8ee2800640 1 --2- 172.21.15.123:0/3780483235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ee4078790 0x7f8ee41a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:40:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.804+0000 7f8ee1e00640 1 -- 172.21.15.123:0/3780483235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ee4077df0 msgr2=0x7f8ee419e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:40:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.804+0000 7f8ee1e00640 1 --2- 172.21.15.123:0/3780483235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ee4077df0 0x7f8ee419e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.804+0000 7f8ee1e00640 1 -- 172.21.15.123:0/3780483235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ee4078790 msgr2=0x7f8ee41a5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:40:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.804+0000 7f8ee1e00640 1 --2- 172.21.15.123:0/3780483235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ee4078790 0x7f8ee41a5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:40:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.804+0000 7f8ee1e00640 1 -- 172.21.15.123:0/3780483235 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee41a60e0 con 0x7f8ee4076ba0 2024-09-19T04:40:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.804+0000 7f8ee2800640 1 --2- 172.21.15.123:0/3780483235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ee4078790 0x7f8ee41a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:40:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.804+0000 7f8edbe00640 1 --2- 172.21.15.123:0/3780483235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ee4077df0 0x7f8ee419e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:40:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.805+0000 7f8ee1e00640 1 --2- 172.21.15.123:0/3780483235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ee4076ba0 0x7f8ee419e4b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4031440 tx=0x7f8ed40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:40:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.805+0000 7f8edaa00640 1 -- 172.21.15.123:0/3780483235 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed403f070 con 0x7f8ee4076ba0 2024-09-19T04:40:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.805+0000 7f8ee9200640 1 -- 172.21.15.123:0/3780483235 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee41a6370 con 0x7f8ee4076ba0 2024-09-19T04:40:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.805+0000 7f8edaa00640 1 -- 172.21.15.123:0/3780483235 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ed403b460 con 0x7f8ee4076ba0 2024-09-19T04:40:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.805+0000 7f8edaa00640 1 -- 172.21.15.123:0/3780483235 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4035070 con 0x7f8ee4076ba0 2024-09-19T04:40:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.805+0000 7f8ee9200640 1 -- 172.21.15.123:0/3780483235 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee41a6850 con 0x7f8ee4076ba0 2024-09-19T04:40:52.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.806+0000 7f8ee9200640 1 -- 172.21.15.123:0/3780483235 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee4110780 con 0x7f8ee4076ba0 2024-09-19T04:40:52.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.808+0000 7f8edaa00640 1 -- 172.21.15.123:0/3780483235 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f8ed403b980 con 0x7f8ee4076ba0 2024-09-19T04:40:52.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.809+0000 7f8edaa00640 1 --2- 172.21.15.123:0/3780483235 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8ec80a3270 0x7f8ec80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:40:52.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.809+0000 7f8edaa00640 1 -- 172.21.15.123:0/3780483235 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4286+0+0 (secure 0 0 0) 0x7f8ed40ef090 con 0x7f8ee4076ba0 2024-09-19T04:40:52.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.809+0000 7f8edbe00640 1 --2- 172.21.15.123:0/3780483235 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8ec80a3270 0x7f8ec80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:40:52.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.810+0000 7f8edbe00640 1 --2- 172.21.15.123:0/3780483235 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8ec80a3270 0x7f8ec80a5730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8ee419f9d0 tx=0x7f8ecc00b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:40:52.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:52.814+0000 7f8edaa00640 1 -- 172.21.15.123:0/3780483235 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8ed40bccd0 con 0x7f8ee4076ba0 2024-09-19T04:40:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:52 smithi183 ceph-mon[35333]: Detected new or changed devices on smithi193 2024-09-19T04:40:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:52 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi193 to 8544M 2024-09-19T04:40:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:52 smithi183 ceph-mon[35333]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:52 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:52 smithi183 ceph-mon[35333]: osdmap e37: 6 total, 5 up, 6 in 2024-09-19T04:40:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:52 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:52 smithi183 ceph-mon[35333]: from='osd.5 ' entity='osd.5' 2024-09-19T04:40:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:52 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:52.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:52 smithi193 ceph-mon[35592]: Detected new or changed devices on smithi193 2024-09-19T04:40:52.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:52 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi193 to 8544M 2024-09-19T04:40:52.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:52 smithi193 ceph-mon[35592]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:52.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:52 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:52.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:52 smithi193 ceph-mon[35592]: osdmap e37: 6 total, 5 up, 6 in 2024-09-19T04:40:52.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:52 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:52.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:52 smithi193 ceph-mon[35592]: from='osd.5 ' entity='osd.5' 2024-09-19T04:40:52.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:52 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:53.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:40:53.003+0000 7f8ee9200640 1 -- 172.21.15.123:0/3780483235 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8ee4062630 con 0x7f8ec80a3270 2024-09-19T04:40:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:53 smithi183 ceph-mon[35333]: purged_snaps scrub starts 2024-09-19T04:40:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:53 smithi183 ceph-mon[35333]: purged_snaps scrub ok 2024-09-19T04:40:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:53 smithi183 ceph-mon[35333]: osd.5 [v2:172.21.15.193:6808/2292251639,v1:172.21.15.193:6809/2292251639] boot 2024-09-19T04:40:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:53 smithi183 ceph-mon[35333]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T04:40:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:53 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:53 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:40:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:53 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:40:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:53 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:53.892 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:53 smithi193 ceph-mon[35592]: purged_snaps scrub starts 2024-09-19T04:40:53.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:53 smithi193 ceph-mon[35592]: purged_snaps scrub ok 2024-09-19T04:40:53.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:53 smithi193 ceph-mon[35592]: osd.5 [v2:172.21.15.193:6808/2292251639,v1:172.21.15.193:6809/2292251639] boot 2024-09-19T04:40:53.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:53 smithi193 ceph-mon[35592]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T04:40:53.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:53.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:40:53.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:40:53.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:53.909 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:53 smithi123 ceph-mon[30511]: purged_snaps scrub starts 2024-09-19T04:40:53.909 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:53 smithi123 ceph-mon[30511]: purged_snaps scrub ok 2024-09-19T04:40:53.909 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:53 smithi123 ceph-mon[30511]: osd.5 [v2:172.21.15.193:6808/2292251639,v1:172.21.15.193:6809/2292251639] boot 2024-09-19T04:40:53.909 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:53 smithi123 ceph-mon[30511]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T04:40:53.909 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:40:53.909 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:40:53.909 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:40:53.910 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:40:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:54 smithi183 ceph-mon[35333]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:40:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:54 smithi183 ceph-mon[35333]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:54 smithi183 ceph-mon[35333]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T04:40:54.905 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:54 smithi193 ceph-mon[35592]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:40:54.905 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:54 smithi193 ceph-mon[35592]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:54.905 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:54 smithi193 ceph-mon[35592]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T04:40:54.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:54 smithi123 ceph-mon[30511]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:40:54.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:54 smithi123 ceph-mon[30511]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:40:54.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:54 smithi123 ceph-mon[30511]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T04:40:55.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:55 smithi123 ceph-mon[30511]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T04:40:55.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:55 smithi123 ceph-mon[30511]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:40:56.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:55 smithi193 ceph-mon[35592]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T04:40:56.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:55 smithi193 ceph-mon[35592]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:40:56.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:55 smithi183 ceph-mon[35333]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T04:40:56.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:55 smithi183 ceph-mon[35333]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:40:57.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:56 smithi123 ceph-mon[30511]: osdmap e41: 6 total, 6 up, 6 in 2024-09-19T04:40:57.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:56 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1749091702' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fda4e39-6adb-4941-a39b-a9a15019088c"}]: dispatch 2024-09-19T04:40:57.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:56 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fda4e39-6adb-4941-a39b-a9a15019088c"}]: dispatch 2024-09-19T04:40:57.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:56 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fda4e39-6adb-4941-a39b-a9a15019088c"}]': finished 2024-09-19T04:40:57.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:56 smithi123 ceph-mon[30511]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T04:40:57.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:56 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:40:57.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:56 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/108758855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:40:57.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:56 smithi193 ceph-mon[35592]: osdmap e41: 6 total, 6 up, 6 in 2024-09-19T04:40:57.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:56 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1749091702' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fda4e39-6adb-4941-a39b-a9a15019088c"}]: dispatch 2024-09-19T04:40:57.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:56 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fda4e39-6adb-4941-a39b-a9a15019088c"}]: dispatch 2024-09-19T04:40:57.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:56 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fda4e39-6adb-4941-a39b-a9a15019088c"}]': finished 2024-09-19T04:40:57.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:56 smithi193 ceph-mon[35592]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T04:40:57.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:56 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:40:57.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:56 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/108758855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:40:57.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:56 smithi183 ceph-mon[35333]: osdmap e41: 6 total, 6 up, 6 in 2024-09-19T04:40:57.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:56 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1749091702' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fda4e39-6adb-4941-a39b-a9a15019088c"}]: dispatch 2024-09-19T04:40:57.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:56 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fda4e39-6adb-4941-a39b-a9a15019088c"}]: dispatch 2024-09-19T04:40:57.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:56 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fda4e39-6adb-4941-a39b-a9a15019088c"}]': finished 2024-09-19T04:40:57.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:56 smithi183 ceph-mon[35333]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T04:40:57.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:56 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:40:57.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:56 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/108758855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:40:58.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:40:57 smithi123 ceph-mon[30511]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:40:58.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:40:57 smithi193 ceph-mon[35592]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:40:58.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:40:57 smithi183 ceph-mon[35333]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:00.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:00 smithi123 ceph-mon[30511]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-19T04:41:00.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:00 smithi193 ceph-mon[35592]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-19T04:41:00.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:00 smithi183 ceph-mon[35333]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-19T04:41:02.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:02 smithi123 ceph-mon[30511]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T04:41:02.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:02 smithi193 ceph-mon[35592]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T04:41:02.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:02 smithi183 ceph-mon[35333]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T04:41:04.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:04 smithi123 ceph-mon[30511]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:41:04.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:04 smithi193 ceph-mon[35592]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:41:04.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:04 smithi183 ceph-mon[35333]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:41:05.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:05 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T04:41:05.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:05 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:05.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:05 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T04:41:05.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:05 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:05 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T04:41:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:05 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:06.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:06 smithi193 ceph-mon[35592]: Deploying daemon osd.6 on smithi123 2024-09-19T04:41:06.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:06 smithi193 ceph-mon[35592]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T04:41:06.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:06 smithi123 ceph-mon[30511]: Deploying daemon osd.6 on smithi123 2024-09-19T04:41:06.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:06 smithi123 ceph-mon[30511]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T04:41:06.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:06 smithi183 ceph-mon[35333]: Deploying daemon osd.6 on smithi123 2024-09-19T04:41:06.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:06 smithi183 ceph-mon[35333]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T04:41:08.669 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:08 smithi123 ceph-mon[30511]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T04:41:08.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:08 smithi193 ceph-mon[35592]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T04:41:08.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:08 smithi183 ceph-mon[35333]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T04:41:10.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:10 smithi123 ceph-mon[30511]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:41:10.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:10 smithi193 ceph-mon[35592]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:41:10.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:10 smithi183 ceph-mon[35333]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:41:12.445 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:12 smithi123 ceph-mon[30511]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:12.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:12 smithi193 ceph-mon[35592]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:12.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:12 smithi183 ceph-mon[35333]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:13.510 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:41:13.510 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:13.511 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:13.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:13 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:41:13.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:13 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:13.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:13 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:13.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:13 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:41:13.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:13 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:13.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:13 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:14.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:14 smithi123 ceph-mon[30511]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:14.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:14 smithi193 ceph-mon[35592]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:14 smithi183 ceph-mon[35333]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:16.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:16 smithi123 ceph-mon[30511]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:16.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:16 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:16.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:16 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:16.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:16 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:16.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:16 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:41:16.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:16 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:16.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:16 smithi193 ceph-mon[35592]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:16.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:16 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:16.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:16 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:16.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:16 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:16.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:16 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:41:16.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:16 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:16.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:16 smithi183 ceph-mon[35333]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:16.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:16 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:16.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:16 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:16.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:16 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:16.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:16 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:41:16.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:16 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:17.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:41:17.259+0000 7f8edaa00640 1 -- 172.21.15.123:0/3780483235 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8ee4062630 con 0x7f8ec80a3270 2024-09-19T04:41:17.261 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 6 on host 'smithi123' 2024-09-19T04:41:17.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:41:17.262+0000 7f8ee9200640 1 -- 172.21.15.123:0/3780483235 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8ec80a3270 msgr2=0x7f8ec80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:41:17.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:41:17.262+0000 7f8ee9200640 1 --2- 172.21.15.123:0/3780483235 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8ec80a3270 0x7f8ec80a5730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8ee419f9d0 tx=0x7f8ecc00b480 comp rx=0 tx=0).stop 2024-09-19T04:41:17.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:41:17.262+0000 7f8ee9200640 1 -- 172.21.15.123:0/3780483235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ee4076ba0 msgr2=0x7f8ee419e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:41:17.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:41:17.262+0000 7f8ee9200640 1 --2- 172.21.15.123:0/3780483235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ee4076ba0 0x7f8ee419e4b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4031440 tx=0x7f8ed40043b0 comp rx=0 tx=0).stop 2024-09-19T04:41:17.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:41:17.262+0000 7f8ee9200640 1 -- 172.21.15.123:0/3780483235 shutdown_connections 2024-09-19T04:41:17.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:41:17.262+0000 7f8ee9200640 1 --2- 172.21.15.123:0/3780483235 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8ec80a3270 0x7f8ec80a5730 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:17.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:41:17.262+0000 7f8ee9200640 1 --2- 172.21.15.123:0/3780483235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ee4078790 0x7f8ee41a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:17.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:41:17.262+0000 7f8ee9200640 1 --2- 172.21.15.123:0/3780483235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ee4077df0 0x7f8ee419e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:17.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:41:17.262+0000 7f8ee9200640 1 --2- 172.21.15.123:0/3780483235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ee4076ba0 0x7f8ee419e4b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:17.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:41:17.262+0000 7f8ee9200640 1 -- 172.21.15.123:0/3780483235 >> 172.21.15.123:0/3780483235 conn(0x7f8ee40ff5b0 msgr2=0x7f8ee4101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:41:17.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:41:17.263+0000 7f8ee9200640 1 -- 172.21.15.123:0/3780483235 shutdown_connections 2024-09-19T04:41:17.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:41:17.264+0000 7f8ee9200640 1 -- 172.21.15.123:0/3780483235 wait complete. 2024-09-19T04:41:17.644 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:41:17.645 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:17.645 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:17.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:41:17.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:17.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:17.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:41:17.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:17.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:18.107 DEBUG:teuthology.orchestra.run.smithi123:osd.6> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T04:41:18.111 INFO:tasks.cephadm:Deploying osd.7 on smithi183 with /dev/vg_nvme/lv_2... 2024-09-19T04:41:18.111 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T04:41:18.366 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:41:18.600 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:18 smithi123 ceph-mon[30511]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:18.600 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:18 smithi123 ceph-mon[30511]: from='osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T04:41:18.704 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:18 smithi183 ceph-mon[35333]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:18.704 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:18 smithi183 ceph-mon[35333]: from='osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T04:41:18.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:18 smithi193 ceph-mon[35592]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:18.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:18 smithi193 ceph-mon[35592]: from='osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T04:41:19.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:19 smithi123 ceph-mon[30511]: from='osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T04:41:19.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:19 smithi123 ceph-mon[30511]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T04:41:19.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:19 smithi123 ceph-mon[30511]: from='osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-19T04:41:19.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:19 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:41:19.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:19 smithi123 ceph-mon[30511]: from='osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-19T04:41:19.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:19 smithi123 ceph-mon[30511]: osdmap e44: 7 total, 6 up, 7 in 2024-09-19T04:41:19.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:19 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:41:19.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:19 smithi183 ceph-mon[35333]: from='osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T04:41:19.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:19 smithi183 ceph-mon[35333]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T04:41:19.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:19 smithi183 ceph-mon[35333]: from='osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-19T04:41:19.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:19 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:41:19.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:19 smithi183 ceph-mon[35333]: from='osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-19T04:41:19.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:19 smithi183 ceph-mon[35333]: osdmap e44: 7 total, 6 up, 7 in 2024-09-19T04:41:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:19 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:41:19.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:19 smithi193 ceph-mon[35592]: from='osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T04:41:19.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:19 smithi193 ceph-mon[35592]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T04:41:19.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:19 smithi193 ceph-mon[35592]: from='osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-19T04:41:19.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:19 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:41:19.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:19 smithi193 ceph-mon[35592]: from='osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-19T04:41:19.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:19 smithi193 ceph-mon[35592]: osdmap e44: 7 total, 6 up, 7 in 2024-09-19T04:41:19.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:19 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:41:20.109 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:41:19 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:41:19.904+0000 7f8365400640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T04:41:20.109 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:41:19 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:41:19.930+0000 7f835aa00640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:41:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:20 smithi123 ceph-mon[30511]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:20 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:41:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:20 smithi123 ceph-mon[30511]: from='osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499]' entity='osd.6' 2024-09-19T04:41:20.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:20 smithi193 ceph-mon[35592]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:20.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:20 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:41:20.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:20 smithi193 ceph-mon[35592]: from='osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499]' entity='osd.6' 2024-09-19T04:41:20.816 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:20 smithi183 ceph-mon[35333]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:20.816 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:20 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:41:20.816 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:20 smithi183 ceph-mon[35333]: from='osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499]' entity='osd.6' 2024-09-19T04:41:20.818 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:41:20.845 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_2 2024-09-19T04:41:21.096 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:41:21.528 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:21 smithi183 ceph-mon[35333]: purged_snaps scrub starts 2024-09-19T04:41:21.528 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:21 smithi183 ceph-mon[35333]: purged_snaps scrub ok 2024-09-19T04:41:21.528 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:21 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:41:21.528 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:21 smithi183 ceph-mon[35333]: osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499] boot 2024-09-19T04:41:21.528 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:21 smithi183 ceph-mon[35333]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T04:41:21.529 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:21 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:41:21.537 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.535+0000 7f8bbf483640 1 -- 172.21.15.183:0/2284579336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bb8101d00 msgr2=0x7f8bb8102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:41:21.537 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.535+0000 7f8bbf483640 1 --2- 172.21.15.183:0/2284579336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bb8101d00 0x7f8bb8102180 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009900 tx=0x7f8ba802a410 comp rx=0 tx=0).stop 2024-09-19T04:41:21.537 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.535+0000 7f8bbf483640 1 -- 172.21.15.183:0/2284579336 shutdown_connections 2024-09-19T04:41:21.537 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.535+0000 7f8bbf483640 1 --2- 172.21.15.183:0/2284579336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb81026c0 0x7f8bb8108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:21.537 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.535+0000 7f8bbf483640 1 --2- 172.21.15.183:0/2284579336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bb8101d00 0x7f8bb8102180 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:21.538 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.535+0000 7f8bbf483640 1 --2- 172.21.15.183:0/2284579336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bb8100b00 0x7f8bb8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:21.538 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.535+0000 7f8bbf483640 1 -- 172.21.15.183:0/2284579336 >> 172.21.15.183:0/2284579336 conn(0x7f8bb80fc290 msgr2=0x7f8bb80fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:41:21.538 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.535+0000 7f8bbf483640 1 -- 172.21.15.183:0/2284579336 shutdown_connections 2024-09-19T04:41:21.538 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.535+0000 7f8bbf483640 1 -- 172.21.15.183:0/2284579336 wait complete. 2024-09-19T04:41:21.538 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.535+0000 7f8bbf483640 1 Processor -- start 2024-09-19T04:41:21.538 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.535+0000 7f8bbf483640 1 -- start start 2024-09-19T04:41:21.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbf483640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bb8100b00 0x7f8bb8196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:41:21.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbf483640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb8101d00 0x7f8bb81972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:41:21.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbf483640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bb81026c0 0x7f8bb819e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:41:21.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbf483640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb810c780 con 0x7f8bb8101d00 2024-09-19T04:41:21.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbf483640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb810c600 con 0x7f8bb8100b00 2024-09-19T04:41:21.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbf483640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb810c900 con 0x7f8bb81026c0 2024-09-19T04:41:21.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbd21f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bb8100b00 0x7f8bb8196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:41:21.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbd21f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bb8100b00 0x7f8bb8196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:33310/0 (socket says 172.21.15.183:33310) 2024-09-19T04:41:21.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbd21f640 1 -- 172.21.15.183:0/2033101215 learned_addr learned my addr 172.21.15.183:0/2033101215 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-19T04:41:21.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbda20640 1 --2- 172.21.15.183:0/2033101215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bb81026c0 0x7f8bb819e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:41:21.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbca1e640 1 --2- 172.21.15.183:0/2033101215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb8101d00 0x7f8bb81972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:41:21.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbd21f640 1 -- 172.21.15.183:0/2033101215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bb81026c0 msgr2=0x7f8bb819e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:41:21.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbd21f640 1 --2- 172.21.15.183:0/2033101215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bb81026c0 0x7f8bb819e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:21.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbd21f640 1 -- 172.21.15.183:0/2033101215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb8101d00 msgr2=0x7f8bb81972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:41:21.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbd21f640 1 --2- 172.21.15.183:0/2033101215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb8101d00 0x7f8bb81972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:21.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbd21f640 1 -- 172.21.15.183:0/2033101215 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb819e9e0 con 0x7f8bb8100b00 2024-09-19T04:41:21.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbda20640 1 --2- 172.21.15.183:0/2033101215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bb81026c0 0x7f8bb819e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:41:21.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbd21f640 1 --2- 172.21.15.183:0/2033101215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bb8100b00 0x7f8bb8196db0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400d950 tx=0x7f8bb400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:41:21.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bae7fc640 1 -- 172.21.15.183:0/2033101215 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4017070 con 0x7f8bb8100b00 2024-09-19T04:41:21.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bae7fc640 1 -- 172.21.15.183:0/2033101215 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb4009c30 con 0x7f8bb8100b00 2024-09-19T04:41:21.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bae7fc640 1 -- 172.21.15.183:0/2033101215 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb400f600 con 0x7f8bb8100b00 2024-09-19T04:41:21.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.536+0000 7f8bbf483640 1 -- 172.21.15.183:0/2033101215 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb819ecd0 con 0x7f8bb8100b00 2024-09-19T04:41:21.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.537+0000 7f8bbf483640 1 -- 172.21.15.183:0/2033101215 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb819f190 con 0x7f8bb8100b00 2024-09-19T04:41:21.542 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.538+0000 7f8bbf483640 1 -- 172.21.15.183:0/2033101215 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb819f360 con 0x7f8bb8100b00 2024-09-19T04:41:21.542 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.539+0000 7f8bae7fc640 1 -- 172.21.15.183:0/2033101215 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f8bb401a070 con 0x7f8bb8100b00 2024-09-19T04:41:21.542 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.539+0000 7f8bae7fc640 1 --2- 172.21.15.183:0/2033101215 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8ba40a32f0 0x7f8ba40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:41:21.542 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.539+0000 7f8bae7fc640 1 -- 172.21.15.183:0/2033101215 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4965+0+0 (secure 0 0 0) 0x7f8bb40ce690 con 0x7f8bb8100b00 2024-09-19T04:41:21.543 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.540+0000 7f8bbca1e640 1 --2- 172.21.15.183:0/2033101215 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8ba40a32f0 0x7f8ba40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:41:21.543 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.540+0000 7f8bbca1e640 1 --2- 172.21.15.183:0/2033101215 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8ba40a32f0 0x7f8ba40a57b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8000c00 tx=0x7f8ba80315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:41:21.545 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.544+0000 7f8bae7fc640 1 -- 172.21.15.183:0/2033101215 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8bb4009de0 con 0x7f8bb8100b00 2024-09-19T04:41:21.623 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:21 smithi123 ceph-mon[30511]: purged_snaps scrub starts 2024-09-19T04:41:21.623 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:21 smithi123 ceph-mon[30511]: purged_snaps scrub ok 2024-09-19T04:41:21.623 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:21 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:41:21.624 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:21 smithi123 ceph-mon[30511]: osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499] boot 2024-09-19T04:41:21.624 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:21 smithi123 ceph-mon[30511]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T04:41:21.624 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:21 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:41:21.719 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:21.716+0000 7f8bbf483640 1 -- 172.21.15.183:0/2033101215 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8bb8062630 con 0x7f8ba40a32f0 2024-09-19T04:41:21.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:21 smithi193 ceph-mon[35592]: purged_snaps scrub starts 2024-09-19T04:41:21.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:21 smithi193 ceph-mon[35592]: purged_snaps scrub ok 2024-09-19T04:41:21.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:21 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:41:21.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:21 smithi193 ceph-mon[35592]: osd.6 [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499] boot 2024-09-19T04:41:21.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:21 smithi193 ceph-mon[35592]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T04:41:21.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:21 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:41:22.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:22 smithi123 ceph-mon[30511]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:22.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:41:22.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:41:22.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:22.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:22 smithi123 ceph-mon[30511]: osdmap e46: 7 total, 7 up, 7 in 2024-09-19T04:41:22.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:22.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:22.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:22.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:22.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:22.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:22.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:22.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:41:22.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:22.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:22 smithi193 ceph-mon[35592]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:22.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:41:22.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:41:22.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:22.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:22 smithi193 ceph-mon[35592]: osdmap e46: 7 total, 7 up, 7 in 2024-09-19T04:41:22.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:22.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:22.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:22.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:22.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:22.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:22.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:22.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:41:22.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:22 smithi183 ceph-mon[35333]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:41:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:41:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:41:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:22 smithi183 ceph-mon[35333]: osdmap e46: 7 total, 7 up, 7 in 2024-09-19T04:41:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:22.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:22.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:22.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:22.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:22.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:41:22.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:23.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:23 smithi123 ceph-mon[30511]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:41:23.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:23 smithi123 ceph-mon[30511]: Detected new or changed devices on smithi123 2024-09-19T04:41:23.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:23 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi123 to 5696M 2024-09-19T04:41:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:23 smithi183 ceph-mon[35333]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:41:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:23 smithi183 ceph-mon[35333]: Detected new or changed devices on smithi123 2024-09-19T04:41:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:23 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi123 to 5696M 2024-09-19T04:41:23.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:23 smithi193 ceph-mon[35592]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:41:23.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:23 smithi193 ceph-mon[35592]: Detected new or changed devices on smithi123 2024-09-19T04:41:23.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:23 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi123 to 5696M 2024-09-19T04:41:24.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:24 smithi183 ceph-mon[35333]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:24.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:24 smithi183 ceph-mon[35333]: from='client.? 172.21.15.183:0/2224862224' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c0a021e-9f3b-4766-9686-0ef24b68cf25"}]: dispatch 2024-09-19T04:41:24.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:24 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c0a021e-9f3b-4766-9686-0ef24b68cf25"}]: dispatch 2024-09-19T04:41:24.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:24 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c0a021e-9f3b-4766-9686-0ef24b68cf25"}]': finished 2024-09-19T04:41:24.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:24 smithi183 ceph-mon[35333]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T04:41:24.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:24 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:24.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:24 smithi123 ceph-mon[30511]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:24.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:24 smithi123 ceph-mon[30511]: from='client.? 172.21.15.183:0/2224862224' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c0a021e-9f3b-4766-9686-0ef24b68cf25"}]: dispatch 2024-09-19T04:41:24.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:24 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c0a021e-9f3b-4766-9686-0ef24b68cf25"}]: dispatch 2024-09-19T04:41:24.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:24 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c0a021e-9f3b-4766-9686-0ef24b68cf25"}]': finished 2024-09-19T04:41:24.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:24 smithi123 ceph-mon[30511]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T04:41:24.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:24 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:24.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:24 smithi193 ceph-mon[35592]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:24.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:24 smithi193 ceph-mon[35592]: from='client.? 172.21.15.183:0/2224862224' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c0a021e-9f3b-4766-9686-0ef24b68cf25"}]: dispatch 2024-09-19T04:41:24.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:24 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c0a021e-9f3b-4766-9686-0ef24b68cf25"}]: dispatch 2024-09-19T04:41:24.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:24 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c0a021e-9f3b-4766-9686-0ef24b68cf25"}]': finished 2024-09-19T04:41:24.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:24 smithi193 ceph-mon[35592]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T04:41:24.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:24 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:25.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:25 smithi123 ceph-mon[30511]: from='client.? 172.21.15.183:0/40313938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:41:25.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:25 smithi193 ceph-mon[35592]: from='client.? 172.21.15.183:0/40313938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:41:25.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:25 smithi183 ceph-mon[35333]: from='client.? 172.21.15.183:0/40313938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:41:26.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:26 smithi123 ceph-mon[30511]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:26.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:26 smithi193 ceph-mon[35592]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:26.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:26 smithi183 ceph-mon[35333]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:28.682 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:28 smithi183 ceph-mon[35333]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:28.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:28 smithi193 ceph-mon[35592]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:28.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:28 smithi123 ceph-mon[30511]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:30.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:30 smithi183 ceph-mon[35333]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:30.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:30 smithi193 ceph-mon[35592]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:30.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:30 smithi123 ceph-mon[30511]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:32.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:32 smithi183 ceph-mon[35333]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:32.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:32 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T04:41:32.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:32 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:32.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:32 smithi193 ceph-mon[35592]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:32.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:32 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T04:41:32.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:32 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:32.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:32 smithi123 ceph-mon[30511]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:32.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:32 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T04:41:32.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:32 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:33.721 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:33 smithi183 ceph-mon[35333]: Deploying daemon osd.7 on smithi183 2024-09-19T04:41:33.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:33 smithi193 ceph-mon[35592]: Deploying daemon osd.7 on smithi183 2024-09-19T04:41:33.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:33 smithi123 ceph-mon[30511]: Deploying daemon osd.7 on smithi183 2024-09-19T04:41:34.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:34 smithi183 ceph-mon[35333]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:34.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:34 smithi193 ceph-mon[35592]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:34.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:34 smithi123 ceph-mon[30511]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:36.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:36 smithi193 ceph-mon[35592]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:36.789 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:36 smithi183 ceph-mon[35333]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:36.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:36 smithi123 ceph-mon[30511]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:38.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:38 smithi193 ceph-mon[35592]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:38.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:38 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:41:38.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:38 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:38.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:38 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:38.773 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:38 smithi183 ceph-mon[35333]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:38.773 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:38 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:41:38.773 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:38 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:38.773 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:38 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:38.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:38 smithi123 ceph-mon[30511]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:38.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:38 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:41:38.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:38 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:38.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:38 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:40.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:40 smithi193 ceph-mon[35592]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:40.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:40 smithi183 ceph-mon[35333]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:40.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:40 smithi123 ceph-mon[30511]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:41.699 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:41.698+0000 7f8bae7fc640 1 -- 172.21.15.183:0/2033101215 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8bb8062630 con 0x7f8ba40a32f0 2024-09-19T04:41:41.699 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 7 on host 'smithi183' 2024-09-19T04:41:41.702 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:41.701+0000 7f8bbf483640 1 -- 172.21.15.183:0/2033101215 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8ba40a32f0 msgr2=0x7f8ba40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:41:41.702 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:41.701+0000 7f8bbf483640 1 --2- 172.21.15.183:0/2033101215 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8ba40a32f0 0x7f8ba40a57b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8000c00 tx=0x7f8ba80315d0 comp rx=0 tx=0).stop 2024-09-19T04:41:41.702 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:41.701+0000 7f8bbf483640 1 -- 172.21.15.183:0/2033101215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bb8100b00 msgr2=0x7f8bb8196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:41:41.702 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:41.701+0000 7f8bbf483640 1 --2- 172.21.15.183:0/2033101215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bb8100b00 0x7f8bb8196db0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400d950 tx=0x7f8bb400de10 comp rx=0 tx=0).stop 2024-09-19T04:41:41.703 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:41.701+0000 7f8bbf483640 1 -- 172.21.15.183:0/2033101215 shutdown_connections 2024-09-19T04:41:41.703 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:41.701+0000 7f8bbf483640 1 --2- 172.21.15.183:0/2033101215 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8ba40a32f0 0x7f8ba40a57b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:41.703 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:41.701+0000 7f8bbf483640 1 --2- 172.21.15.183:0/2033101215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bb81026c0 0x7f8bb819e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:41.703 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:41.701+0000 7f8bbf483640 1 --2- 172.21.15.183:0/2033101215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb8101d00 0x7f8bb81972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:41.703 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:41.701+0000 7f8bbf483640 1 --2- 172.21.15.183:0/2033101215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bb8100b00 0x7f8bb8196db0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:41.703 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:41.701+0000 7f8bbf483640 1 -- 172.21.15.183:0/2033101215 >> 172.21.15.183:0/2033101215 conn(0x7f8bb80fc290 msgr2=0x7f8bb80fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:41:41.703 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:41.701+0000 7f8bbf483640 1 -- 172.21.15.183:0/2033101215 shutdown_connections 2024-09-19T04:41:41.703 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:41:41.701+0000 7f8bbf483640 1 -- 172.21.15.183:0/2033101215 wait complete. 2024-09-19T04:41:42.415 DEBUG:teuthology.orchestra.run.smithi183:osd.7> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T04:41:42.418 INFO:tasks.cephadm:Deploying osd.8 on smithi193 with /dev/vg_nvme/lv_2... 2024-09-19T04:41:42.418 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T04:41:42.672 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:41:42.721 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:42 smithi183 ceph-mon[35333]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:42.721 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:42 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:42.721 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:42 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:42.721 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:42 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:42.721 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:42 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:42.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:42 smithi193 ceph-mon[35592]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:42.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:42 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:42.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:42 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:42.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:42 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:42.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:42 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:42.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:42 smithi123 ceph-mon[30511]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:42.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:42.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:42.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:42.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:44.641 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:44 smithi183 ceph-mon[35333]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:44.641 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:44 smithi183 ceph-mon[35333]: from='osd.7 [v2:172.21.15.183:6816/3718518226,v1:172.21.15.183:6817/3718518226]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:41:44.641 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:44 smithi183 ceph-mon[35333]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:41:44.642 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:41:44 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:41:44.374+0000 7f5603491640 -1 osd.7 0 log_to_monitors true 2024-09-19T04:41:44.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:44 smithi193 ceph-mon[35592]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:44.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:44 smithi193 ceph-mon[35592]: from='osd.7 [v2:172.21.15.183:6816/3718518226,v1:172.21.15.183:6817/3718518226]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:41:44.731 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:44 smithi193 ceph-mon[35592]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:41:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:44 smithi123 ceph-mon[30511]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:44 smithi123 ceph-mon[30511]: from='osd.7 [v2:172.21.15.183:6816/3718518226,v1:172.21.15.183:6817/3718518226]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:41:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:44 smithi123 ceph-mon[30511]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:41:45.426 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:41:45.453 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_2 2024-09-19T04:41:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:45 smithi183 ceph-mon[35333]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T04:41:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:45 smithi183 ceph-mon[35333]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T04:41:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:45 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:45 smithi183 ceph-mon[35333]: from='osd.7 [v2:172.21.15.183:6816/3718518226,v1:172.21.15.183:6817/3718518226]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:41:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:45 smithi183 ceph-mon[35333]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:41:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:45 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:45 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:45 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:45.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:45 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:45.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:45 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:45.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:45 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:45.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:45 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:45.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:45 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:41:45.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:45 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:45.709 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:41:45.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:45 smithi193 ceph-mon[35592]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T04:41:45.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:45 smithi193 ceph-mon[35592]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T04:41:45.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:45 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:45.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:45 smithi193 ceph-mon[35592]: from='osd.7 [v2:172.21.15.183:6816/3718518226,v1:172.21.15.183:6817/3718518226]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:41:45.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:45 smithi193 ceph-mon[35592]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:41:45.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:45 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:45.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:45 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:45.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:45 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:45.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:45 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:45.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:45 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:45.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:45 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:45.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:45 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:45.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:45 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:41:45.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:45 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:45.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:45 smithi123 ceph-mon[30511]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T04:41:45.931 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:45 smithi123 ceph-mon[30511]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T04:41:45.931 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:45 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:45.931 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:45 smithi123 ceph-mon[30511]: from='osd.7 [v2:172.21.15.183:6816/3718518226,v1:172.21.15.183:6817/3718518226]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:41:45.931 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:45 smithi123 ceph-mon[30511]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:41:45.931 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:45 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:45.932 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:45 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:45.932 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:45 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:45.932 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:45 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:45.932 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:45 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:41:45.932 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:45 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:45.932 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:45 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:45.932 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:45 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:41:45.932 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:45 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:46.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.201+0000 7fca10b47640 1 -- 172.21.15.193:0/4258376609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca0c105750 msgr2=0x7fca0c105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:41:46.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.201+0000 7fca10b47640 1 --2- 172.21.15.193:0/4258376609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca0c105750 0x7fca0c105c00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fca04009ca0 tx=0x7fca0402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:41:46.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.202+0000 7fca10b47640 1 -- 172.21.15.193:0/4258376609 shutdown_connections 2024-09-19T04:41:46.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.202+0000 7fca10b47640 1 --2- 172.21.15.193:0/4258376609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca0c105750 0x7fca0c105c00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:46.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.202+0000 7fca10b47640 1 --2- 172.21.15.193:0/4258376609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca0c102da0 0x7fca0c105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:46.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.202+0000 7fca10b47640 1 --2- 172.21.15.193:0/4258376609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca0c100450 0x7fca0c102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:46.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.202+0000 7fca10b47640 1 -- 172.21.15.193:0/4258376609 >> 172.21.15.193:0/4258376609 conn(0x7fca0c0fc280 msgr2=0x7fca0c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:41:46.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.202+0000 7fca10b47640 1 -- 172.21.15.193:0/4258376609 shutdown_connections 2024-09-19T04:41:46.207 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.202+0000 7fca10b47640 1 -- 172.21.15.193:0/4258376609 wait complete. 2024-09-19T04:41:46.207 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.202+0000 7fca10b47640 1 Processor -- start 2024-09-19T04:41:46.207 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.202+0000 7fca10b47640 1 -- start start 2024-09-19T04:41:46.207 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca10b47640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca0c100450 0x7fca0c19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:41:46.207 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca10b47640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca0c102da0 0x7fca0c19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:41:46.207 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca10b47640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca0c105750 0x7fca0c1a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:41:46.207 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca10b47640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c113e60 con 0x7fca0c102da0 2024-09-19T04:41:46.207 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca10b47640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c113ce0 con 0x7fca0c100450 2024-09-19T04:41:46.207 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca10b47640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c113fe0 con 0x7fca0c105750 2024-09-19T04:41:46.207 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca0ad9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca0c105750 0x7fca0c1a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:41:46.207 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca0ad9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca0c105750 0x7fca0c1a5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:44804/0 (socket says 172.21.15.193:44804) 2024-09-19T04:41:46.207 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca0ad9d640 1 -- 172.21.15.193:0/270617295 learned_addr learned my addr 172.21.15.193:0/270617295 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-19T04:41:46.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca09d9b640 1 --2- 172.21.15.193:0/270617295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca0c102da0 0x7fca0c19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:41:46.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca0a59c640 1 --2- 172.21.15.193:0/270617295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca0c100450 0x7fca0c19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:41:46.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca0ad9d640 1 -- 172.21.15.193:0/270617295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca0c100450 msgr2=0x7fca0c19e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:41:46.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca0ad9d640 1 --2- 172.21.15.193:0/270617295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca0c100450 0x7fca0c19e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:46.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca0ad9d640 1 -- 172.21.15.193:0/270617295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca0c102da0 msgr2=0x7fca0c19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:41:46.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca0ad9d640 1 --2- 172.21.15.193:0/270617295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca0c102da0 0x7fca0c19e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:41:46.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca0ad9d640 1 -- 172.21.15.193:0/270617295 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0c1a6090 con 0x7fca0c105750 2024-09-19T04:41:46.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca0ad9d640 1 --2- 172.21.15.193:0/270617295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca0c105750 0x7fca0c1a5a20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fca04005e30 tx=0x7fca04031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:41:46.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fc9fb7fe640 1 -- 172.21.15.193:0/270617295 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca04003c30 con 0x7fca0c105750 2024-09-19T04:41:46.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fc9fb7fe640 1 -- 172.21.15.193:0/270617295 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca04003dd0 con 0x7fca0c105750 2024-09-19T04:41:46.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fca10b47640 1 -- 172.21.15.193:0/270617295 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0c1a6320 con 0x7fca0c105750 2024-09-19T04:41:46.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.203+0000 7fc9fb7fe640 1 -- 172.21.15.193:0/270617295 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0402fde0 con 0x7fca0c105750 2024-09-19T04:41:46.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.204+0000 7fca10b47640 1 -- 172.21.15.193:0/270617295 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca0c1a6860 con 0x7fca0c105750 2024-09-19T04:41:46.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.204+0000 7fca10b47640 1 -- 172.21.15.193:0/270617295 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca0c0fdd70 con 0x7fca0c105750 2024-09-19T04:41:46.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.206+0000 7fc9fb7fe640 1 -- 172.21.15.193:0/270617295 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fca04004050 con 0x7fca0c105750 2024-09-19T04:41:46.210 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.207+0000 7fc9fb7fe640 1 --2- 172.21.15.193:0/270617295 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc9f40a3270 0x7fc9f40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:41:46.210 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.207+0000 7fc9fb7fe640 1 -- 172.21.15.193:0/270617295 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5117+0+0 (secure 0 0 0) 0x7fca040f0150 con 0x7fca0c105750 2024-09-19T04:41:46.210 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.207+0000 7fca0a59c640 1 --2- 172.21.15.193:0/270617295 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc9f40a3270 0x7fc9f40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:41:46.210 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.208+0000 7fca0a59c640 1 --2- 172.21.15.193:0/270617295 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc9f40a3270 0x7fc9f40a5730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc005fd0 tx=0x7fc9fc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:41:46.213 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.210+0000 7fc9fb7fe640 1 -- 172.21.15.193:0/270617295 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fca040bdaf0 con 0x7fca0c105750 2024-09-19T04:41:46.349 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:41:45 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:41:45.879+0000 7f55ff414640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T04:41:46.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:41:45 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:41:45.889+0000 7f55f7234640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:41:46.398 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:41:46.396+0000 7fca10b47640 1 -- 172.21.15.193:0/270617295 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fca0c062630 con 0x7fc9f40a3270 2024-09-19T04:41:46.735 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:46 smithi193 ceph-mon[35592]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:46.735 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:46 smithi193 ceph-mon[35592]: Detected new or changed devices on smithi183 2024-09-19T04:41:46.735 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:46 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-19T04:41:46.735 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:46 smithi193 ceph-mon[35592]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-19T04:41:46.735 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:46 smithi193 ceph-mon[35592]: osdmap e49: 8 total, 7 up, 8 in 2024-09-19T04:41:46.735 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:46 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:46.736 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:46 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:46.736 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:46 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:41:46.736 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:46 smithi193 ceph-mon[35592]: from='osd.7 ' entity='osd.7' 2024-09-19T04:41:46.736 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:46 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:46.736 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:46 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:41:46.736 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:46 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:46.736 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:46 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:41:46.736 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:46 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:41:46.736 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:46 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:46.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:46 smithi183 ceph-mon[35333]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:46.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:46 smithi183 ceph-mon[35333]: Detected new or changed devices on smithi183 2024-09-19T04:41:46.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:46 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-19T04:41:46.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:46 smithi183 ceph-mon[35333]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-19T04:41:46.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:46 smithi183 ceph-mon[35333]: osdmap e49: 8 total, 7 up, 8 in 2024-09-19T04:41:46.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:46 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:46.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:46 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:46.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:46 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:41:46.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:46 smithi183 ceph-mon[35333]: from='osd.7 ' entity='osd.7' 2024-09-19T04:41:46.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:46 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:46.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:46 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:41:46.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:46 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:46.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:46 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:41:46.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:46 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:41:46.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:46 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:46.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:46 smithi123 ceph-mon[30511]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:46.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:46 smithi123 ceph-mon[30511]: Detected new or changed devices on smithi183 2024-09-19T04:41:46.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:46 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-19T04:41:46.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:46 smithi123 ceph-mon[30511]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-19T04:41:46.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:46 smithi123 ceph-mon[30511]: osdmap e49: 8 total, 7 up, 8 in 2024-09-19T04:41:46.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:46 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:46.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:46 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:46.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:46 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:41:46.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:46 smithi123 ceph-mon[30511]: from='osd.7 ' entity='osd.7' 2024-09-19T04:41:46.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:46 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:46.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:46 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:41:46.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:46 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:41:46.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:46 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:41:46.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:46 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:41:46.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:46 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:47.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:47 smithi193 ceph-mon[35592]: purged_snaps scrub starts 2024-09-19T04:41:47.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:47 smithi193 ceph-mon[35592]: purged_snaps scrub ok 2024-09-19T04:41:47.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:47 smithi193 ceph-mon[35592]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:41:47.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:47 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:47.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:47 smithi193 ceph-mon[35592]: osd.7 [v2:172.21.15.183:6816/3718518226,v1:172.21.15.183:6817/3718518226] boot 2024-09-19T04:41:47.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:47 smithi193 ceph-mon[35592]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T04:41:47.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:47 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:47.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:47 smithi193 ceph-mon[35592]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T04:41:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:47 smithi183 ceph-mon[35333]: purged_snaps scrub starts 2024-09-19T04:41:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:47 smithi183 ceph-mon[35333]: purged_snaps scrub ok 2024-09-19T04:41:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:47 smithi183 ceph-mon[35333]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:41:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:47 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:47 smithi183 ceph-mon[35333]: osd.7 [v2:172.21.15.183:6816/3718518226,v1:172.21.15.183:6817/3718518226] boot 2024-09-19T04:41:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:47 smithi183 ceph-mon[35333]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T04:41:47.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:47 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:47.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:47 smithi183 ceph-mon[35333]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T04:41:47.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:47 smithi123 ceph-mon[30511]: purged_snaps scrub starts 2024-09-19T04:41:47.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:47 smithi123 ceph-mon[30511]: purged_snaps scrub ok 2024-09-19T04:41:47.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:47 smithi123 ceph-mon[30511]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:41:47.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:47 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:47.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:47 smithi123 ceph-mon[30511]: osd.7 [v2:172.21.15.183:6816/3718518226,v1:172.21.15.183:6817/3718518226] boot 2024-09-19T04:41:47.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:47 smithi123 ceph-mon[30511]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T04:41:47.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:47 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:41:47.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:47 smithi123 ceph-mon[30511]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T04:41:48.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:48 smithi193 ceph-mon[35592]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:48.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:48 smithi193 ceph-mon[35592]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T04:41:48.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:48 smithi183 ceph-mon[35333]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:48.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:48 smithi183 ceph-mon[35333]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T04:41:48.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:48 smithi123 ceph-mon[30511]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:41:48.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:48 smithi123 ceph-mon[30511]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T04:41:50.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:49 smithi123 ceph-mon[30511]: osdmap e53: 8 total, 8 up, 8 in 2024-09-19T04:41:50.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:49 smithi123 ceph-mon[30511]: from='client.? 172.21.15.193:0/3567901864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0e67e0-6305-4f81-9211-d41960099a8b"}]: dispatch 2024-09-19T04:41:50.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:49 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0e67e0-6305-4f81-9211-d41960099a8b"}]: dispatch 2024-09-19T04:41:50.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:49 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b0e67e0-6305-4f81-9211-d41960099a8b"}]': finished 2024-09-19T04:41:50.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:49 smithi123 ceph-mon[30511]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T04:41:50.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:49 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:41:50.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:49 smithi123 ceph-mon[30511]: pgmap v182: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:41:50.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:49 smithi123 ceph-mon[30511]: from='client.? 172.21.15.193:0/56195051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:41:50.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:49 smithi193 ceph-mon[35592]: osdmap e53: 8 total, 8 up, 8 in 2024-09-19T04:41:50.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:49 smithi193 ceph-mon[35592]: from='client.? 172.21.15.193:0/3567901864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0e67e0-6305-4f81-9211-d41960099a8b"}]: dispatch 2024-09-19T04:41:50.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:49 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0e67e0-6305-4f81-9211-d41960099a8b"}]: dispatch 2024-09-19T04:41:50.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:49 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b0e67e0-6305-4f81-9211-d41960099a8b"}]': finished 2024-09-19T04:41:50.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:49 smithi193 ceph-mon[35592]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T04:41:50.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:49 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:41:50.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:49 smithi193 ceph-mon[35592]: pgmap v182: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:41:50.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:49 smithi193 ceph-mon[35592]: from='client.? 172.21.15.193:0/56195051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:41:50.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:49 smithi183 ceph-mon[35333]: osdmap e53: 8 total, 8 up, 8 in 2024-09-19T04:41:50.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:49 smithi183 ceph-mon[35333]: from='client.? 172.21.15.193:0/3567901864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0e67e0-6305-4f81-9211-d41960099a8b"}]: dispatch 2024-09-19T04:41:50.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:49 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0e67e0-6305-4f81-9211-d41960099a8b"}]: dispatch 2024-09-19T04:41:50.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:49 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b0e67e0-6305-4f81-9211-d41960099a8b"}]': finished 2024-09-19T04:41:50.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:49 smithi183 ceph-mon[35333]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T04:41:50.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:49 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:41:50.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:49 smithi183 ceph-mon[35333]: pgmap v182: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:41:50.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:49 smithi183 ceph-mon[35333]: from='client.? 172.21.15.193:0/56195051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:41:52.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:52 smithi123 ceph-mon[30511]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:41:52.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:52 smithi193 ceph-mon[35592]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:41:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:52 smithi183 ceph-mon[35333]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:41:54.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:54 smithi193 ceph-mon[35592]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:41:54.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:54 smithi123 ceph-mon[30511]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:41:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:54 smithi183 ceph-mon[35333]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:41:56.650 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:56 smithi193 ceph-mon[35592]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:41:56.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:56 smithi183 ceph-mon[35333]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:41:56.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:56 smithi123 ceph-mon[30511]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:41:58.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:58 smithi123 ceph-mon[30511]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:41:58.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:58 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T04:41:58.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:58 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:58.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:41:58 smithi123 ceph-mon[30511]: Deploying daemon osd.8 on smithi193 2024-09-19T04:41:58.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:58 smithi193 ceph-mon[35592]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:41:58.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:58 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T04:41:58.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:58 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:58.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:41:58 smithi193 ceph-mon[35592]: Deploying daemon osd.8 on smithi193 2024-09-19T04:41:58.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:58 smithi183 ceph-mon[35333]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:41:58.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:58 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T04:41:58.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:58 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:41:58.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:41:58 smithi183 ceph-mon[35333]: Deploying daemon osd.8 on smithi193 2024-09-19T04:42:00.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:00 smithi123 ceph-mon[30511]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:00.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:00 smithi193 ceph-mon[35592]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:00.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:00 smithi183 ceph-mon[35333]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:02.658 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:02 smithi193 ceph-mon[35592]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:02.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:02 smithi123 ceph-mon[30511]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:02.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:02 smithi183 ceph-mon[35333]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:04.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:04 smithi123 ceph-mon[30511]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:04.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:04 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:42:04.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:04 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:04.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:04 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:04.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:04 smithi193 ceph-mon[35592]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:04.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:04 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:42:04.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:04 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:04.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:04 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:04.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:04 smithi183 ceph-mon[35333]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:04.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:42:04.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:04.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:06.679 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:06 smithi123 ceph-mon[30511]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:06.696 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:06 smithi193 ceph-mon[35592]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:06.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:06 smithi183 ceph-mon[35333]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:07.426 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:42:07.425+0000 7fc9fb7fe640 1 -- 172.21.15.193:0/270617295 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fca0c062630 con 0x7fc9f40a3270 2024-09-19T04:42:07.427 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 8 on host 'smithi193' 2024-09-19T04:42:07.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:42:07.427+0000 7fca10b47640 1 -- 172.21.15.193:0/270617295 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc9f40a3270 msgr2=0x7fc9f40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:42:07.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:42:07.427+0000 7fca10b47640 1 --2- 172.21.15.193:0/270617295 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc9f40a3270 0x7fc9f40a5730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc005fd0 tx=0x7fc9fc005850 comp rx=0 tx=0).stop 2024-09-19T04:42:07.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:42:07.428+0000 7fca10b47640 1 -- 172.21.15.193:0/270617295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca0c105750 msgr2=0x7fca0c1a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:42:07.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:42:07.428+0000 7fca10b47640 1 --2- 172.21.15.193:0/270617295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca0c105750 0x7fca0c1a5a20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fca04005e30 tx=0x7fca04031ee0 comp rx=0 tx=0).stop 2024-09-19T04:42:07.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:42:07.428+0000 7fca10b47640 1 -- 172.21.15.193:0/270617295 shutdown_connections 2024-09-19T04:42:07.430 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:42:07.428+0000 7fca10b47640 1 --2- 172.21.15.193:0/270617295 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc9f40a3270 0x7fc9f40a5730 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:07.430 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:42:07.428+0000 7fca10b47640 1 --2- 172.21.15.193:0/270617295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca0c105750 0x7fca0c1a5a20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:07.430 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:42:07.428+0000 7fca10b47640 1 --2- 172.21.15.193:0/270617295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca0c102da0 0x7fca0c19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:07.430 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:42:07.428+0000 7fca10b47640 1 --2- 172.21.15.193:0/270617295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca0c100450 0x7fca0c19e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:07.430 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:42:07.428+0000 7fca10b47640 1 -- 172.21.15.193:0/270617295 >> 172.21.15.193:0/270617295 conn(0x7fca0c0fc280 msgr2=0x7fca0c100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:42:07.430 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:42:07.428+0000 7fca10b47640 1 -- 172.21.15.193:0/270617295 shutdown_connections 2024-09-19T04:42:07.430 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:42:07.428+0000 7fca10b47640 1 -- 172.21.15.193:0/270617295 wait complete. 2024-09-19T04:42:07.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:07.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:07.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:07.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:42:07.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:07.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:42:07.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:07.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:42:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:42:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:07.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:07.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:07.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:07.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:07.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:42:07.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:07.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:42:07.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:07.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:08.188 DEBUG:teuthology.orchestra.run.smithi193:osd.8> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.8.service 2024-09-19T04:42:08.190 INFO:tasks.cephadm:Deploying osd.9 on smithi123 with /dev/vg_nvme/lv_1... 2024-09-19T04:42:08.190 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T04:42:08.561 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:42:08.845 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:08 smithi123 ceph-mon[30511]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:08.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:08 smithi183 ceph-mon[35333]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:08.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:08 smithi193 ceph-mon[35592]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:09.737 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:42:09 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:42:09.574+0000 7f2560951640 -1 osd.8 0 log_to_monitors true 2024-09-19T04:42:10.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:10 smithi193 ceph-mon[35592]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:10.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:10 smithi193 ceph-mon[35592]: from='osd.8 [v2:172.21.15.193:6816/3551404118,v1:172.21.15.193:6817/3551404118]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:42:10.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:10 smithi193 ceph-mon[35592]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:42:10.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:10 smithi123 ceph-mon[30511]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:10.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:10 smithi123 ceph-mon[30511]: from='osd.8 [v2:172.21.15.193:6816/3551404118,v1:172.21.15.193:6817/3551404118]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:42:10.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:10 smithi123 ceph-mon[30511]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:42:10.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:10 smithi183 ceph-mon[35333]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:10.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:10 smithi183 ceph-mon[35333]: from='osd.8 [v2:172.21.15.193:6816/3551404118,v1:172.21.15.193:6817/3551404118]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:42:10.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:10 smithi183 ceph-mon[35333]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:42:11.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:11 smithi123 ceph-mon[30511]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T04:42:11.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:11 smithi123 ceph-mon[30511]: from='osd.8 [v2:172.21.15.193:6816/3551404118,v1:172.21.15.193:6817/3551404118]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:42:11.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:11 smithi123 ceph-mon[30511]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T04:42:11.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:11 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:42:11.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:11 smithi123 ceph-mon[30511]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:42:11.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:11 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:11.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:11 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:11.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:11 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:11.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:11 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:11.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:11 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:11.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:11 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:11.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:11 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:11.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:11 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:42:11.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:11 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:11.685 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:42:11.717 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_1 2024-09-19T04:42:11.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:11 smithi183 ceph-mon[35333]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T04:42:11.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:11 smithi183 ceph-mon[35333]: from='osd.8 [v2:172.21.15.193:6816/3551404118,v1:172.21.15.193:6817/3551404118]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:42:11.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:11 smithi183 ceph-mon[35333]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T04:42:11.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:11 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:42:11.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:11 smithi183 ceph-mon[35333]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:42:11.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:11 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:11.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:11 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:11.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:11 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:11.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:11 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:11.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:11 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:11.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:11 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:11.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:11 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:11.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:11 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:42:11.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:11 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:11.990 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:42:11 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:42:11.532+0000 7f255c8d4640 -1 osd.8 0 waiting for initial osdmap 2024-09-19T04:42:11.991 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:42:11 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:42:11.543+0000 7f25586fc640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:42:11.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:11 smithi193 ceph-mon[35592]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T04:42:11.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:11 smithi193 ceph-mon[35592]: from='osd.8 [v2:172.21.15.193:6816/3551404118,v1:172.21.15.193:6817/3551404118]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:42:11.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:11 smithi193 ceph-mon[35592]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T04:42:11.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:11 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:42:11.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:11 smithi193 ceph-mon[35592]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:42:11.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:11 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:11.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:11 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:11.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:11 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:11.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:11 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:11.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:11 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:11.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:11 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:11.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:11 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:11.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:11 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:42:11.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:11 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:12.081 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:42:12.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.644+0000 7f1af6400640 1 -- 172.21.15.123:0/3563964068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0103e00 msgr2=0x7f1af0104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:42:12.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.644+0000 7f1af6400640 1 --2- 172.21.15.123:0/3563964068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0103e00 0x7f1af0104200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009900 tx=0x7f1ae002a410 comp rx=0 tx=0).stop 2024-09-19T04:42:12.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.644+0000 7f1af6400640 1 -- 172.21.15.123:0/3563964068 shutdown_connections 2024-09-19T04:42:12.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.644+0000 7f1af6400640 1 --2- 172.21.15.123:0/3563964068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1af0105920 0x7f1af010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:12.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.644+0000 7f1af6400640 1 --2- 172.21.15.123:0/3563964068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1af0104f60 0x7f1af01053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:12.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.644+0000 7f1af6400640 1 --2- 172.21.15.123:0/3563964068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0103e00 0x7f1af0104200 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:12.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.644+0000 7f1af6400640 1 -- 172.21.15.123:0/3563964068 >> 172.21.15.123:0/3563964068 conn(0x7f1af00ff5b0 msgr2=0x7f1af01019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:42:12.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.644+0000 7f1af6400640 1 -- 172.21.15.123:0/3563964068 shutdown_connections 2024-09-19T04:42:12.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.644+0000 7f1af6400640 1 -- 172.21.15.123:0/3563964068 wait complete. 2024-09-19T04:42:12.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.645+0000 7f1af6400640 1 Processor -- start 2024-09-19T04:42:12.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.645+0000 7f1af6400640 1 -- start start 2024-09-19T04:42:12.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.645+0000 7f1af6400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1af0103e00 0x7f1af019a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:42:12.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.645+0000 7f1af6400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0104f60 0x7f1af019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:42:12.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.645+0000 7f1af6400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1af0105920 0x7f1af01a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:42:12.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.645+0000 7f1af6400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1af010fa90 con 0x7f1af0104f60 2024-09-19T04:42:12.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.645+0000 7f1af6400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1af010f910 con 0x7f1af0105920 2024-09-19T04:42:12.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.645+0000 7f1af6400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1af010fc10 con 0x7f1af0103e00 2024-09-19T04:42:12.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.646+0000 7f1ae6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0104f60 0x7f1af019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:42:12.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.646+0000 7f1ae6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0104f60 0x7f1af019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38478/0 (socket says 172.21.15.123:38478) 2024-09-19T04:42:12.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.646+0000 7f1ae6a00640 1 -- 172.21.15.123:0/1820676866 learned_addr learned my addr 172.21.15.123:0/1820676866 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:42:12.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.646+0000 7f1aef400640 1 --2- 172.21.15.123:0/1820676866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1af0103e00 0x7f1af019a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:42:12.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.646+0000 7f1aefe00640 1 --2- 172.21.15.123:0/1820676866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1af0105920 0x7f1af01a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:42:12.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.646+0000 7f1ae6a00640 1 -- 172.21.15.123:0/1820676866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1af0103e00 msgr2=0x7f1af019a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:42:12.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.646+0000 7f1ae6a00640 1 --2- 172.21.15.123:0/1820676866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1af0103e00 0x7f1af019a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:12.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.646+0000 7f1ae6a00640 1 -- 172.21.15.123:0/1820676866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1af0105920 msgr2=0x7f1af01a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:42:12.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.646+0000 7f1ae6a00640 1 --2- 172.21.15.123:0/1820676866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1af0105920 0x7f1af01a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:12.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.646+0000 7f1ae6a00640 1 -- 172.21.15.123:0/1820676866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af01a1d40 con 0x7f1af0104f60 2024-09-19T04:42:12.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.646+0000 7f1ae6a00640 1 --2- 172.21.15.123:0/1820676866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0104f60 0x7f1af019a650 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800e9c0 tx=0x7f1ad800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:42:12.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.647+0000 7f1aecc00640 1 -- 172.21.15.123:0/1820676866 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad800ccc0 con 0x7f1af0104f60 2024-09-19T04:42:12.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.647+0000 7f1aecc00640 1 -- 172.21.15.123:0/1820676866 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad80035f0 con 0x7f1af0104f60 2024-09-19T04:42:12.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.647+0000 7f1af6400640 1 -- 172.21.15.123:0/1820676866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af01a1fd0 con 0x7f1af0104f60 2024-09-19T04:42:12.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.647+0000 7f1aecc00640 1 -- 172.21.15.123:0/1820676866 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad800ce60 con 0x7f1af0104f60 2024-09-19T04:42:12.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.647+0000 7f1af6400640 1 -- 172.21.15.123:0/1820676866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af01a24e0 con 0x7f1af0104f60 2024-09-19T04:42:12.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.648+0000 7f1af6400640 1 -- 172.21.15.123:0/1820676866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af010b1c0 con 0x7f1af0104f60 2024-09-19T04:42:12.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.650+0000 7f1aecc00640 1 -- 172.21.15.123:0/1820676866 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f1ad80037a0 con 0x7f1af0104f60 2024-09-19T04:42:12.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.651+0000 7f1aecc00640 1 --2- 172.21.15.123:0/1820676866 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f1ad40a3270 0x7f1ad40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:42:12.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.651+0000 7f1aecc00640 1 -- 172.21.15.123:0/1820676866 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5862+0+0 (secure 0 0 0) 0x7f1ad80cf400 con 0x7f1af0104f60 2024-09-19T04:42:12.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.652+0000 7f1aef400640 1 --2- 172.21.15.123:0/1820676866 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f1ad40a3270 0x7f1ad40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:42:12.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.652+0000 7f1aef400640 1 --2- 172.21.15.123:0/1820676866 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f1ad40a3270 0x7f1ad40a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00314e0 tx=0x7f1ae0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:42:12.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.654+0000 7f1aecc00640 1 -- 172.21.15.123:0/1820676866 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1ad809caa0 con 0x7f1af0104f60 2024-09-19T04:42:12.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:12.839+0000 7f1af6400640 1 -- 172.21.15.123:0/1820676866 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1af0062630 con 0x7f1ad40a3270 2024-09-19T04:42:12.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:12 smithi123 ceph-mon[30511]: Detected new or changed devices on smithi193 2024-09-19T04:42:12.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:12 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi193 to 5696M 2024-09-19T04:42:12.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:12 smithi123 ceph-mon[30511]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:12.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:12 smithi123 ceph-mon[30511]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-19T04:42:12.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:12 smithi123 ceph-mon[30511]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T04:42:12.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:12 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:42:12.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:12 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:42:12.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:12 smithi123 ceph-mon[30511]: osd.8 [v2:172.21.15.193:6816/3551404118,v1:172.21.15.193:6817/3551404118] boot 2024-09-19T04:42:12.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:12 smithi123 ceph-mon[30511]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T04:42:12.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:12 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:42:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:12 smithi183 ceph-mon[35333]: Detected new or changed devices on smithi193 2024-09-19T04:42:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:12 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi193 to 5696M 2024-09-19T04:42:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:12 smithi183 ceph-mon[35333]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:12 smithi183 ceph-mon[35333]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-19T04:42:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:12 smithi183 ceph-mon[35333]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T04:42:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:12 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:42:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:12 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:42:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:12 smithi183 ceph-mon[35333]: osd.8 [v2:172.21.15.193:6816/3551404118,v1:172.21.15.193:6817/3551404118] boot 2024-09-19T04:42:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:12 smithi183 ceph-mon[35333]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T04:42:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:12 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:42:12.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:12 smithi193 ceph-mon[35592]: Detected new or changed devices on smithi193 2024-09-19T04:42:12.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:12 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi193 to 5696M 2024-09-19T04:42:12.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:12 smithi193 ceph-mon[35592]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:42:12.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:12 smithi193 ceph-mon[35592]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-19T04:42:12.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:12 smithi193 ceph-mon[35592]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T04:42:12.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:12 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:42:12.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:12 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:42:12.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:12 smithi193 ceph-mon[35592]: osd.8 [v2:172.21.15.193:6816/3551404118,v1:172.21.15.193:6817/3551404118] boot 2024-09-19T04:42:12.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:12 smithi193 ceph-mon[35592]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T04:42:12.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:12 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:42:13.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:13 smithi123 ceph-mon[30511]: purged_snaps scrub starts 2024-09-19T04:42:13.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:13 smithi123 ceph-mon[30511]: purged_snaps scrub ok 2024-09-19T04:42:13.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:42:13.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:42:13.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:13.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:13 smithi123 ceph-mon[30511]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T04:42:13.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:13 smithi183 ceph-mon[35333]: purged_snaps scrub starts 2024-09-19T04:42:13.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:13 smithi183 ceph-mon[35333]: purged_snaps scrub ok 2024-09-19T04:42:13.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:13 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:42:13.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:13 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:42:13.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:13 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:13.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:13 smithi183 ceph-mon[35333]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T04:42:13.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:13 smithi193 ceph-mon[35592]: purged_snaps scrub starts 2024-09-19T04:42:13.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:13 smithi193 ceph-mon[35592]: purged_snaps scrub ok 2024-09-19T04:42:13.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:13 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:42:13.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:13 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:42:13.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:13 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:13.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:13 smithi193 ceph-mon[35592]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T04:42:14.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:14 smithi123 ceph-mon[30511]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:42:14.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:14 smithi123 ceph-mon[30511]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:14.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:14 smithi123 ceph-mon[30511]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T04:42:14.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:14 smithi183 ceph-mon[35333]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:42:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:14 smithi183 ceph-mon[35333]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:14 smithi183 ceph-mon[35333]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T04:42:14.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:14 smithi193 ceph-mon[35592]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:42:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:14 smithi193 ceph-mon[35592]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:14 smithi193 ceph-mon[35592]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T04:42:16.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:15 smithi123 ceph-mon[30511]: osdmap e60: 9 total, 9 up, 9 in 2024-09-19T04:42:16.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:15 smithi123 ceph-mon[30511]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:16.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:15 smithi193 ceph-mon[35592]: osdmap e60: 9 total, 9 up, 9 in 2024-09-19T04:42:16.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:15 smithi193 ceph-mon[35592]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:16.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:15 smithi183 ceph-mon[35333]: osdmap e60: 9 total, 9 up, 9 in 2024-09-19T04:42:16.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:15 smithi183 ceph-mon[35333]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:17.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:16 smithi123 ceph-mon[30511]: osdmap e61: 9 total, 9 up, 9 in 2024-09-19T04:42:17.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:16 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/438863703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "978ef3fa-c6ec-421f-b194-56d44813be5b"}]: dispatch 2024-09-19T04:42:17.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:16 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/438863703' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "978ef3fa-c6ec-421f-b194-56d44813be5b"}]': finished 2024-09-19T04:42:17.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:16 smithi123 ceph-mon[30511]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T04:42:17.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:16 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:17.178 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:16 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3885453698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:42:17.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:16 smithi193 ceph-mon[35592]: osdmap e61: 9 total, 9 up, 9 in 2024-09-19T04:42:17.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:16 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/438863703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "978ef3fa-c6ec-421f-b194-56d44813be5b"}]: dispatch 2024-09-19T04:42:17.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:16 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/438863703' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "978ef3fa-c6ec-421f-b194-56d44813be5b"}]': finished 2024-09-19T04:42:17.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:16 smithi193 ceph-mon[35592]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T04:42:17.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:16 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:17.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:16 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3885453698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:42:17.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:16 smithi183 ceph-mon[35333]: osdmap e61: 9 total, 9 up, 9 in 2024-09-19T04:42:17.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:16 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/438863703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "978ef3fa-c6ec-421f-b194-56d44813be5b"}]: dispatch 2024-09-19T04:42:17.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:16 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/438863703' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "978ef3fa-c6ec-421f-b194-56d44813be5b"}]': finished 2024-09-19T04:42:17.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:16 smithi183 ceph-mon[35333]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T04:42:17.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:16 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:17.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:16 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3885453698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:42:18.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:17 smithi123 ceph-mon[30511]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:18.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:17 smithi193 ceph-mon[35592]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:18.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:17 smithi183 ceph-mon[35333]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:20 smithi123 ceph-mon[30511]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-19T04:42:20.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:20 smithi193 ceph-mon[35592]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-19T04:42:20.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:20 smithi183 ceph-mon[35333]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-19T04:42:22.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:22 smithi123 ceph-mon[30511]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T04:42:22.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:22 smithi193 ceph-mon[35592]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T04:42:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:22 smithi183 ceph-mon[35333]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T04:42:24.595 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:24 smithi123 ceph-mon[30511]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:42:24.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:24 smithi193 ceph-mon[35592]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:42:24.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:24 smithi183 ceph-mon[35333]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:42:26.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:26 smithi183 ceph-mon[35333]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T04:42:26.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T04:42:26.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:26.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:26 smithi123 ceph-mon[30511]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T04:42:26.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T04:42:26.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:26.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:26 smithi193 ceph-mon[35592]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T04:42:26.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T04:42:26.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:27.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:27 smithi123 ceph-mon[30511]: Deploying daemon osd.9 on smithi123 2024-09-19T04:42:27.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:27 smithi193 ceph-mon[35592]: Deploying daemon osd.9 on smithi123 2024-09-19T04:42:27.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:27 smithi183 ceph-mon[35333]: Deploying daemon osd.9 on smithi123 2024-09-19T04:42:28.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:28 smithi123 ceph-mon[30511]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T04:42:28.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:28 smithi193 ceph-mon[35592]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T04:42:28.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:28 smithi183 ceph-mon[35333]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T04:42:30.541 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:30 smithi123 ceph-mon[30511]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:42:30.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:30 smithi193 ceph-mon[35592]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:42:30.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:30 smithi183 ceph-mon[35333]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:42:32.645 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:32 smithi123 ceph-mon[30511]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:32.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:32 smithi193 ceph-mon[35592]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:32.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:32 smithi183 ceph-mon[35333]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:33.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:42:33.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:33.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:33.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:42:33.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:33.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:33.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:42:33.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:33.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:34.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:34 smithi123 ceph-mon[30511]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:34.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:34 smithi193 ceph-mon[35592]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:34.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:34 smithi183 ceph-mon[35333]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:36.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:36 smithi123 ceph-mon[30511]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:36.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:36 smithi193 ceph-mon[35592]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:36.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:36 smithi183 ceph-mon[35333]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:37.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:37.670+0000 7f1aecc00640 1 -- 172.21.15.123:0/1820676866 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1af0062630 con 0x7f1ad40a3270 2024-09-19T04:42:37.672 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 9 on host 'smithi123' 2024-09-19T04:42:37.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:37.672+0000 7f1af6400640 1 -- 172.21.15.123:0/1820676866 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f1ad40a3270 msgr2=0x7f1ad40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:42:37.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:37.672+0000 7f1af6400640 1 --2- 172.21.15.123:0/1820676866 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f1ad40a3270 0x7f1ad40a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00314e0 tx=0x7f1ae0000bc0 comp rx=0 tx=0).stop 2024-09-19T04:42:37.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:37.672+0000 7f1af6400640 1 -- 172.21.15.123:0/1820676866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0104f60 msgr2=0x7f1af019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:42:37.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:37.672+0000 7f1af6400640 1 --2- 172.21.15.123:0/1820676866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0104f60 0x7f1af019a650 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800e9c0 tx=0x7f1ad800edc0 comp rx=0 tx=0).stop 2024-09-19T04:42:37.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:37.673+0000 7f1af6400640 1 -- 172.21.15.123:0/1820676866 shutdown_connections 2024-09-19T04:42:37.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:37.673+0000 7f1af6400640 1 --2- 172.21.15.123:0/1820676866 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f1ad40a3270 0x7f1ad40a5730 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:37.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:37.673+0000 7f1af6400640 1 --2- 172.21.15.123:0/1820676866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1af0105920 0x7f1af01a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:37.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:37.673+0000 7f1af6400640 1 --2- 172.21.15.123:0/1820676866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0104f60 0x7f1af019a650 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:37.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:37.673+0000 7f1af6400640 1 --2- 172.21.15.123:0/1820676866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1af0103e00 0x7f1af019a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:37.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:37.673+0000 7f1af6400640 1 -- 172.21.15.123:0/1820676866 >> 172.21.15.123:0/1820676866 conn(0x7f1af00ff5b0 msgr2=0x7f1af0108390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:42:37.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:37.673+0000 7f1af6400640 1 -- 172.21.15.123:0/1820676866 shutdown_connections 2024-09-19T04:42:37.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:42:37.673+0000 7f1af6400640 1 -- 172.21.15.123:0/1820676866 wait complete. 2024-09-19T04:42:37.870 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:37 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:37.870 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:37 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:37.871 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:37 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:37.871 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:37 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:42:37.871 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:37 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:37.871 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:37 smithi123 ceph-mon[30511]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:37.871 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:37 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:42:37.871 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:37 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:37.871 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:37 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:38.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:37 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:38.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:37 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:38.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:37 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:38.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:37 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:42:38.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:37 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:38.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:37 smithi193 ceph-mon[35592]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:38.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:37 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:42:38.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:37 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:38.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:37 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:38.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:37 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:38.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:37 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:38.351 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:37 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:38.351 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:37 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:42:38.351 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:37 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:38.351 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:37 smithi183 ceph-mon[35333]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:38.351 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:37 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:42:38.351 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:37 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:38.351 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:37 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:38.558 DEBUG:teuthology.orchestra.run.smithi123:osd.9> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.9.service 2024-09-19T04:42:38.562 INFO:tasks.cephadm:Deploying osd.10 on smithi183 with /dev/vg_nvme/lv_1... 2024-09-19T04:42:38.562 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T04:42:38.817 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:42:39.415 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:42:39 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:42:39.348+0000 7fb56aa7a640 -1 osd.9 0 log_to_monitors true 2024-09-19T04:42:39.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:39 smithi183 ceph-mon[35333]: from='osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T04:42:39.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:39 smithi123 ceph-mon[30511]: from='osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T04:42:39.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:39 smithi193 ceph-mon[35592]: from='osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T04:42:40.593 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:40 smithi183 ceph-mon[35333]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:40.593 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:40 smithi183 ceph-mon[35333]: from='osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T04:42:40.593 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:40 smithi183 ceph-mon[35333]: osdmap e63: 10 total, 9 up, 10 in 2024-09-19T04:42:40.593 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:40 smithi183 ceph-mon[35333]: from='osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-19T04:42:40.593 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:40 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:40.594 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:40 smithi183 ceph-mon[35333]: from='osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-19T04:42:40.594 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:40 smithi183 ceph-mon[35333]: osdmap e64: 10 total, 9 up, 10 in 2024-09-19T04:42:40.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:40 smithi123 ceph-mon[30511]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:40.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:40 smithi123 ceph-mon[30511]: from='osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T04:42:40.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:40 smithi123 ceph-mon[30511]: osdmap e63: 10 total, 9 up, 10 in 2024-09-19T04:42:40.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:40 smithi123 ceph-mon[30511]: from='osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-19T04:42:40.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:40 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:40.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:40 smithi123 ceph-mon[30511]: from='osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-19T04:42:40.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:40 smithi123 ceph-mon[30511]: osdmap e64: 10 total, 9 up, 10 in 2024-09-19T04:42:40.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:40 smithi193 ceph-mon[35592]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:40.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:40 smithi193 ceph-mon[35592]: from='osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T04:42:40.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:40 smithi193 ceph-mon[35592]: osdmap e63: 10 total, 9 up, 10 in 2024-09-19T04:42:40.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:40 smithi193 ceph-mon[35592]: from='osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-19T04:42:40.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:40 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:40.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:40 smithi193 ceph-mon[35592]: from='osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-19T04:42:40.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:40 smithi193 ceph-mon[35592]: osdmap e64: 10 total, 9 up, 10 in 2024-09-19T04:42:41.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:42:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:42:40.898+0000 7fb566400640 -1 osd.9 0 waiting for initial osdmap 2024-09-19T04:42:41.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:42:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:42:40.935+0000 7fb55ba00640 -1 osd.9 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:42:41.590 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:41 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:41.590 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:41 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:41.590 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:41 smithi183 ceph-mon[35333]: from='osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149]' entity='osd.9' 2024-09-19T04:42:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-19T04:42:41.621 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_1 2024-09-19T04:42:41.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:41 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:41.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:41 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:41.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:41 smithi123 ceph-mon[30511]: from='osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149]' entity='osd.9' 2024-09-19T04:42:41.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:41 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:41.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:41 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:41.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:41 smithi193 ceph-mon[35592]: from='osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149]' entity='osd.9' 2024-09-19T04:42:41.880 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:42:42.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.412+0000 7f9e3594f640 1 -- 172.21.15.183:0/3477504913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e301037b0 msgr2=0x7f9e3010a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:42:42.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.412+0000 7f9e3594f640 1 --2- 172.21.15.183:0/3477504913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e301037b0 0x7f9e3010a040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009ca0 tx=0x7f9e2802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:42:42.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.412+0000 7f9e3594f640 1 -- 172.21.15.183:0/3477504913 shutdown_connections 2024-09-19T04:42:42.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.412+0000 7f9e3594f640 1 --2- 172.21.15.183:0/3477504913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e301037b0 0x7f9e3010a040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:42.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.412+0000 7f9e3594f640 1 --2- 172.21.15.183:0/3477504913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e30102df0 0x7f9e30103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:42.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.412+0000 7f9e3594f640 1 --2- 172.21.15.183:0/3477504913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e30101bf0 0x7f9e30101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:42.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.412+0000 7f9e3594f640 1 -- 172.21.15.183:0/3477504913 >> 172.21.15.183:0/3477504913 conn(0x7f9e300fd3c0 msgr2=0x7f9e300ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:42:42.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.412+0000 7f9e3594f640 1 -- 172.21.15.183:0/3477504913 shutdown_connections 2024-09-19T04:42:42.447 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.412+0000 7f9e3594f640 1 -- 172.21.15.183:0/3477504913 wait complete. 2024-09-19T04:42:42.447 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.413+0000 7f9e3594f640 1 Processor -- start 2024-09-19T04:42:42.447 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.413+0000 7f9e3594f640 1 -- start start 2024-09-19T04:42:42.447 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.413+0000 7f9e3594f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e30101bf0 0x7f9e30197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:42:42.447 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.413+0000 7f9e3594f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e30102df0 0x7f9e301983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:42:42.447 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.413+0000 7f9e3594f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e301037b0 0x7f9e3019f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:42:42.447 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.413+0000 7f9e3594f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3010d870 con 0x7f9e30102df0 2024-09-19T04:42:42.447 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.413+0000 7f9e3594f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3010d6f0 con 0x7f9e301037b0 2024-09-19T04:42:42.447 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.413+0000 7f9e3594f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3010d9f0 con 0x7f9e30101bf0 2024-09-19T04:42:42.447 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.413+0000 7f9e2f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e301037b0 0x7f9e3019f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:42:42.447 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.413+0000 7f9e2f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e301037b0 0x7f9e3019f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:48712/0 (socket says 172.21.15.183:48712) 2024-09-19T04:42:42.448 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.413+0000 7f9e2f7fe640 1 -- 172.21.15.183:0/2432005555 learned_addr learned my addr 172.21.15.183:0/2432005555 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-19T04:42:42.448 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.413+0000 7f9e2effd640 1 --2- 172.21.15.183:0/2432005555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e30101bf0 0x7f9e30197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:42:42.448 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.413+0000 7f9e2e7fc640 1 --2- 172.21.15.183:0/2432005555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e30102df0 0x7f9e301983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:42:42.448 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.414+0000 7f9e2f7fe640 1 -- 172.21.15.183:0/2432005555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e30101bf0 msgr2=0x7f9e30197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:42:42.448 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.414+0000 7f9e2f7fe640 1 --2- 172.21.15.183:0/2432005555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e30101bf0 0x7f9e30197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:42.448 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.414+0000 7f9e2f7fe640 1 -- 172.21.15.183:0/2432005555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e30102df0 msgr2=0x7f9e301983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:42:42.448 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.414+0000 7f9e2f7fe640 1 --2- 172.21.15.183:0/2432005555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e30102df0 0x7f9e301983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:42:42.448 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.414+0000 7f9e2f7fe640 1 -- 172.21.15.183:0/2432005555 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3019fad0 con 0x7f9e301037b0 2024-09-19T04:42:42.448 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.414+0000 7f9e2f7fe640 1 --2- 172.21.15.183:0/2432005555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e301037b0 0x7f9e3019f460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9e28031ae0 tx=0x7f9e28031ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:42:42.448 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.414+0000 7f9e3494d640 1 -- 172.21.15.183:0/2432005555 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e28003c30 con 0x7f9e301037b0 2024-09-19T04:42:42.448 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.414+0000 7f9e3494d640 1 -- 172.21.15.183:0/2432005555 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e28003dd0 con 0x7f9e301037b0 2024-09-19T04:42:42.449 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.414+0000 7f9e3594f640 1 -- 172.21.15.183:0/2432005555 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e3019fd60 con 0x7f9e301037b0 2024-09-19T04:42:42.449 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.414+0000 7f9e3494d640 1 -- 172.21.15.183:0/2432005555 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2802fcb0 con 0x7f9e301037b0 2024-09-19T04:42:42.449 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.414+0000 7f9e3594f640 1 -- 172.21.15.183:0/2432005555 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e301a0240 con 0x7f9e301037b0 2024-09-19T04:42:42.449 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.415+0000 7f9e3594f640 1 -- 172.21.15.183:0/2432005555 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e30108f20 con 0x7f9e301037b0 2024-09-19T04:42:42.449 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.416+0000 7f9e3494d640 1 -- 172.21.15.183:0/2432005555 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9e28004050 con 0x7f9e301037b0 2024-09-19T04:42:42.449 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.417+0000 7f9e3494d640 1 --2- 172.21.15.183:0/2432005555 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9e180a3270 0x7f9e180a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:42:42.449 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.417+0000 7f9e3494d640 1 -- 172.21.15.183:0/2432005555 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6261+0+0 (secure 0 0 0) 0x7f9e280f0310 con 0x7f9e301037b0 2024-09-19T04:42:42.449 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.417+0000 7f9e2effd640 1 --2- 172.21.15.183:0/2432005555 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9e180a3270 0x7f9e180a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:42:42.449 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.417+0000 7f9e2effd640 1 --2- 172.21.15.183:0/2432005555 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9e180a3270 0x7f9e180a5730 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9e30102c50 tx=0x7f9e2000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:42:42.449 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.421+0000 7f9e3494d640 1 -- 172.21.15.183:0/2432005555 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9e280bd950 con 0x7f9e301037b0 2024-09-19T04:42:42.606 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:42:42.604+0000 7f9e3594f640 1 -- 172.21.15.183:0/2432005555 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9e30062630 con 0x7f9e180a3270 2024-09-19T04:42:42.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:42 smithi123 ceph-mon[30511]: purged_snaps scrub starts 2024-09-19T04:42:42.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:42 smithi123 ceph-mon[30511]: purged_snaps scrub ok 2024-09-19T04:42:42.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:42 smithi123 ceph-mon[30511]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:42.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:42.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:42 smithi123 ceph-mon[30511]: osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149] boot 2024-09-19T04:42:42.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:42 smithi123 ceph-mon[30511]: osdmap e65: 10 total, 10 up, 10 in 2024-09-19T04:42:42.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:42 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:42.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:42 smithi193 ceph-mon[35592]: purged_snaps scrub starts 2024-09-19T04:42:42.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:42 smithi193 ceph-mon[35592]: purged_snaps scrub ok 2024-09-19T04:42:42.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:42 smithi193 ceph-mon[35592]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:42.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:42 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:42.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:42 smithi193 ceph-mon[35592]: osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149] boot 2024-09-19T04:42:42.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:42 smithi193 ceph-mon[35592]: osdmap e65: 10 total, 10 up, 10 in 2024-09-19T04:42:42.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:42 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:42 smithi183 ceph-mon[35333]: purged_snaps scrub starts 2024-09-19T04:42:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:42 smithi183 ceph-mon[35333]: purged_snaps scrub ok 2024-09-19T04:42:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:42 smithi183 ceph-mon[35333]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:42:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:42 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:42 smithi183 ceph-mon[35333]: osd.9 [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149] boot 2024-09-19T04:42:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:42 smithi183 ceph-mon[35333]: osdmap e65: 10 total, 10 up, 10 in 2024-09-19T04:42:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:42 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:42:43.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:43 smithi183 ceph-mon[35333]: Detected new or changed devices on smithi123 2024-09-19T04:42:43.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:43.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:43.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:43.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:43.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:43.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:43.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:43 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi123 to 4272M 2024-09-19T04:42:43.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:43.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:43.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:42:43.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:43.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:42:43.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:42:43.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:43.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:43 smithi183 ceph-mon[35333]: osdmap e66: 10 total, 10 up, 10 in 2024-09-19T04:42:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:43 smithi123 ceph-mon[30511]: Detected new or changed devices on smithi123 2024-09-19T04:42:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:43.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:43.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:43.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:43.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:43.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:43.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:43 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi123 to 4272M 2024-09-19T04:42:43.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:43.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:43.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:42:43.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:43.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:42:43.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:42:43.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:43.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:43 smithi123 ceph-mon[30511]: osdmap e66: 10 total, 10 up, 10 in 2024-09-19T04:42:43.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:43 smithi193 ceph-mon[35592]: Detected new or changed devices on smithi123 2024-09-19T04:42:43.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:43.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:43.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:43.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:43.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:43.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:42:43.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:43 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi123 to 4272M 2024-09-19T04:42:43.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:43.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:43.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:42:43.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:43.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:42:43.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:42:43.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:43.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:43 smithi193 ceph-mon[35592]: osdmap e66: 10 total, 10 up, 10 in 2024-09-19T04:42:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:44 smithi183 ceph-mon[35333]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:42:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:44 smithi183 ceph-mon[35333]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:44 smithi123 ceph-mon[30511]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:42:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:44 smithi123 ceph-mon[30511]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:44.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:44 smithi193 ceph-mon[35592]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:42:44.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:44 smithi193 ceph-mon[35592]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:45 smithi183 ceph-mon[35333]: from='client.? 172.21.15.183:0/1547462040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5495e09b-7809-47b0-b7e0-b56c452f9f22"}]: dispatch 2024-09-19T04:42:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:45 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5495e09b-7809-47b0-b7e0-b56c452f9f22"}]: dispatch 2024-09-19T04:42:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:45 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5495e09b-7809-47b0-b7e0-b56c452f9f22"}]': finished 2024-09-19T04:42:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:45 smithi183 ceph-mon[35333]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T04:42:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:45 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:42:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:45 smithi183 ceph-mon[35333]: from='client.? 172.21.15.183:0/3829915788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:42:45.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:45 smithi123 ceph-mon[30511]: from='client.? 172.21.15.183:0/1547462040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5495e09b-7809-47b0-b7e0-b56c452f9f22"}]: dispatch 2024-09-19T04:42:45.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:45 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5495e09b-7809-47b0-b7e0-b56c452f9f22"}]: dispatch 2024-09-19T04:42:45.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:45 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5495e09b-7809-47b0-b7e0-b56c452f9f22"}]': finished 2024-09-19T04:42:45.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:45 smithi123 ceph-mon[30511]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T04:42:45.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:45 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:42:45.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:45 smithi123 ceph-mon[30511]: from='client.? 172.21.15.183:0/3829915788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:42:45.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:45 smithi193 ceph-mon[35592]: from='client.? 172.21.15.183:0/1547462040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5495e09b-7809-47b0-b7e0-b56c452f9f22"}]: dispatch 2024-09-19T04:42:45.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:45 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5495e09b-7809-47b0-b7e0-b56c452f9f22"}]: dispatch 2024-09-19T04:42:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:45 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5495e09b-7809-47b0-b7e0-b56c452f9f22"}]': finished 2024-09-19T04:42:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:45 smithi193 ceph-mon[35592]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T04:42:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:45 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:42:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:45 smithi193 ceph-mon[35592]: from='client.? 172.21.15.183:0/3829915788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:42:46.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:46 smithi183 ceph-mon[35333]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:46.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:46 smithi123 ceph-mon[30511]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:46.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:46 smithi193 ceph-mon[35592]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:48.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:48 smithi183 ceph-mon[35333]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:48.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:48 smithi123 ceph-mon[30511]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:48.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:48 smithi193 ceph-mon[35592]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:50.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:50 smithi183 ceph-mon[35333]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:50.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:50 smithi123 ceph-mon[30511]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:50.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:50 smithi193 ceph-mon[35592]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:52.795 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:52 smithi183 ceph-mon[35333]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:52.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:52 smithi123 ceph-mon[30511]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:52.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:52 smithi193 ceph-mon[35592]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:53 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T04:42:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:53 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:53.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T04:42:53.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:53.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T04:42:53.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:42:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:54 smithi183 ceph-mon[35333]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:54 smithi183 ceph-mon[35333]: Deploying daemon osd.10 on smithi183 2024-09-19T04:42:54.892 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:54 smithi193 ceph-mon[35592]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:54.892 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:54 smithi193 ceph-mon[35592]: Deploying daemon osd.10 on smithi183 2024-09-19T04:42:54.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:54 smithi123 ceph-mon[30511]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:54.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:54 smithi123 ceph-mon[30511]: Deploying daemon osd.10 on smithi183 2024-09-19T04:42:56.725 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:56 smithi183 ceph-mon[35333]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:56.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:56 smithi123 ceph-mon[30511]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:56.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:56 smithi193 ceph-mon[35592]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:58.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:58 smithi183 ceph-mon[35333]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:58.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:58 smithi123 ceph-mon[30511]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:58.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:58 smithi193 ceph-mon[35592]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:42:59.923 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:59 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:42:59.923 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:59 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:59.924 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:42:59 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:59.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:59 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:42:59.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:59 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:59.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:42:59 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:59.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:59 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:42:59.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:59 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:42:59.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:42:59 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:00.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:00 smithi183 ceph-mon[35333]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:43:00.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:00 smithi123 ceph-mon[30511]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:43:00.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:00 smithi193 ceph-mon[35592]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:43:01.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:01 smithi183 ceph-mon[35333]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:43:01.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:01 smithi123 ceph-mon[30511]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:43:01.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:01 smithi193 ceph-mon[35592]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:43:02.929 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:43:02.928+0000 7f9e3494d640 1 -- 172.21.15.183:0/2432005555 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9e30062630 con 0x7f9e180a3270 2024-09-19T04:43:02.929 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 10 on host 'smithi183' 2024-09-19T04:43:02.931 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:43:02.930+0000 7f9e3594f640 1 -- 172.21.15.183:0/2432005555 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9e180a3270 msgr2=0x7f9e180a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:02.931 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:43:02.930+0000 7f9e3594f640 1 --2- 172.21.15.183:0/2432005555 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9e180a3270 0x7f9e180a5730 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9e30102c50 tx=0x7f9e2000b390 comp rx=0 tx=0).stop 2024-09-19T04:43:02.931 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:43:02.930+0000 7f9e3594f640 1 -- 172.21.15.183:0/2432005555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e301037b0 msgr2=0x7f9e3019f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:02.931 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:43:02.930+0000 7f9e3594f640 1 --2- 172.21.15.183:0/2432005555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e301037b0 0x7f9e3019f460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9e28031ae0 tx=0x7f9e28031ee0 comp rx=0 tx=0).stop 2024-09-19T04:43:02.931 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:43:02.930+0000 7f9e3594f640 1 -- 172.21.15.183:0/2432005555 shutdown_connections 2024-09-19T04:43:02.931 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:43:02.930+0000 7f9e3594f640 1 --2- 172.21.15.183:0/2432005555 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9e180a3270 0x7f9e180a5730 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:02.931 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:43:02.930+0000 7f9e3594f640 1 --2- 172.21.15.183:0/2432005555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e301037b0 0x7f9e3019f460 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:02.931 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:43:02.930+0000 7f9e3594f640 1 --2- 172.21.15.183:0/2432005555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e30102df0 0x7f9e301983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:02.932 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:43:02.930+0000 7f9e3594f640 1 --2- 172.21.15.183:0/2432005555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e30101bf0 0x7f9e30197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:02.932 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:43:02.930+0000 7f9e3594f640 1 -- 172.21.15.183:0/2432005555 >> 172.21.15.183:0/2432005555 conn(0x7f9e300fd3c0 msgr2=0x7f9e301060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:02.932 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:43:02.930+0000 7f9e3594f640 1 -- 172.21.15.183:0/2432005555 shutdown_connections 2024-09-19T04:43:02.932 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-19T04:43:02.930+0000 7f9e3594f640 1 -- 172.21.15.183:0/2432005555 wait complete. 2024-09-19T04:43:03.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:03 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:03.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:03 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:03.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:03 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:03.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:03 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:43:03.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:03 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:03.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:03 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:43:03.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:03 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:03.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:03 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:03.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:03 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:03.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:03 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:03.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:03 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:03.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:03 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:43:03.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:03 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:03.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:03 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:43:03.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:03 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:03.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:03 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:03.528 DEBUG:teuthology.orchestra.run.smithi183:osd.10> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.10.service 2024-09-19T04:43:03.530 INFO:tasks.cephadm:Deploying osd.11 on smithi193 with /dev/vg_nvme/lv_1... 2024-09-19T04:43:03.531 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T04:43:03.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:03 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:03.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:03 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:03.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:03 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:03.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:03 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:43:03.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:03 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:03.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:03 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:43:03.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:03 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:03.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:03 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:03.786 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:43:04.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:04 smithi183 ceph-mon[35333]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:43:04.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:04 smithi193 ceph-mon[35592]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:43:04.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:04 smithi123 ceph-mon[30511]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:43:05.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:05 smithi183 ceph-mon[35333]: from='osd.10 [v2:172.21.15.183:6824/3355349245,v1:172.21.15.183:6825/3355349245]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:43:05.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:05 smithi183 ceph-mon[35333]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:43:05.600 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:43:05 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:43:05.292+0000 7f8be6adc640 -1 osd.10 0 log_to_monitors true 2024-09-19T04:43:05.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:05 smithi123 ceph-mon[30511]: from='osd.10 [v2:172.21.15.183:6824/3355349245,v1:172.21.15.183:6825/3355349245]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:43:05.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:05 smithi123 ceph-mon[30511]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:43:05.727 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:05 smithi193 ceph-mon[35592]: from='osd.10 [v2:172.21.15.183:6824/3355349245,v1:172.21.15.183:6825/3355349245]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:43:05.727 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:05 smithi193 ceph-mon[35592]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:43:06.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:06 smithi193 ceph-mon[35592]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:43:06.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:06 smithi193 ceph-mon[35592]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T04:43:06.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:06 smithi193 ceph-mon[35592]: from='osd.10 [v2:172.21.15.183:6824/3355349245,v1:172.21.15.183:6825/3355349245]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:43:06.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:06 smithi193 ceph-mon[35592]: osdmap e68: 11 total, 10 up, 11 in 2024-09-19T04:43:06.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:06 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:43:06.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:06 smithi193 ceph-mon[35592]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:43:06.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:06 smithi193 ceph-mon[35592]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-19T04:43:06.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:06 smithi193 ceph-mon[35592]: osdmap e69: 11 total, 10 up, 11 in 2024-09-19T04:43:06.600 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:43:06 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:43:06.357+0000 7f8be2a5f640 -1 osd.10 0 waiting for initial osdmap 2024-09-19T04:43:06.600 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:43:06 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:43:06.370+0000 7f8bde887640 -1 osd.10 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:43:06.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:06 smithi183 ceph-mon[35333]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:43:06.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:06 smithi183 ceph-mon[35333]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T04:43:06.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:06 smithi183 ceph-mon[35333]: from='osd.10 [v2:172.21.15.183:6824/3355349245,v1:172.21.15.183:6825/3355349245]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:43:06.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:06 smithi183 ceph-mon[35333]: osdmap e68: 11 total, 10 up, 11 in 2024-09-19T04:43:06.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:06 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:43:06.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:06 smithi183 ceph-mon[35333]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:43:06.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:06 smithi183 ceph-mon[35333]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-19T04:43:06.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:06 smithi183 ceph-mon[35333]: osdmap e69: 11 total, 10 up, 11 in 2024-09-19T04:43:06.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:06 smithi123 ceph-mon[30511]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:43:06.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:06 smithi123 ceph-mon[30511]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T04:43:06.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:06 smithi123 ceph-mon[30511]: from='osd.10 [v2:172.21.15.183:6824/3355349245,v1:172.21.15.183:6825/3355349245]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:43:06.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:06 smithi123 ceph-mon[30511]: osdmap e68: 11 total, 10 up, 11 in 2024-09-19T04:43:06.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:06 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:43:06.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:06 smithi123 ceph-mon[30511]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-19T04:43:06.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:06 smithi123 ceph-mon[30511]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-19T04:43:06.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:06 smithi123 ceph-mon[30511]: osdmap e69: 11 total, 10 up, 11 in 2024-09-19T04:43:06.692 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-19T04:43:06.727 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_1 2024-09-19T04:43:06.985 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:43:07.527 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.503+0000 7f55ce678640 1 -- 172.21.15.193:0/2046367069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55c8100b00 msgr2=0x7f55c8100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:07.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.503+0000 7f55ce678640 1 --2- 172.21.15.193:0/2046367069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55c8100b00 0x7f55c8100f00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009900 tx=0x7f55b802a410 comp rx=0 tx=0).stop 2024-09-19T04:43:07.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.503+0000 7f55ce678640 1 -- 172.21.15.193:0/2046367069 shutdown_connections 2024-09-19T04:43:07.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.503+0000 7f55ce678640 1 --2- 172.21.15.193:0/2046367069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55c81026c0 0x7f55c8108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:07.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.503+0000 7f55ce678640 1 --2- 172.21.15.193:0/2046367069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55c8101d00 0x7f55c8102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:07.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.503+0000 7f55ce678640 1 --2- 172.21.15.193:0/2046367069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55c8100b00 0x7f55c8100f00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:07.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.503+0000 7f55ce678640 1 -- 172.21.15.193:0/2046367069 >> 172.21.15.193:0/2046367069 conn(0x7f55c80fc270 msgr2=0x7f55c80fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:07.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.503+0000 7f55ce678640 1 -- 172.21.15.193:0/2046367069 shutdown_connections 2024-09-19T04:43:07.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.503+0000 7f55ce678640 1 -- 172.21.15.193:0/2046367069 wait complete. 2024-09-19T04:43:07.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.504+0000 7f55ce678640 1 Processor -- start 2024-09-19T04:43:07.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.504+0000 7f55ce678640 1 -- start start 2024-09-19T04:43:07.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.504+0000 7f55ce678640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55c8100b00 0x7f55c8198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:07.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.504+0000 7f55ce678640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55c8101d00 0x7f55c8198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:07.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.504+0000 7f55ce678640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55c81026c0 0x7f55c819f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:07.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.504+0000 7f55ce678640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55c810c7b0 con 0x7f55c8100b00 2024-09-19T04:43:07.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.504+0000 7f55ce678640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f55c810c630 con 0x7f55c81026c0 2024-09-19T04:43:07.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.504+0000 7f55ce678640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55c810c930 con 0x7f55c8101d00 2024-09-19T04:43:07.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.504+0000 7f55c77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55c8101d00 0x7f55c8198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:07.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.505+0000 7f55c77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55c8101d00 0x7f55c8198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:60868/0 (socket says 172.21.15.193:60868) 2024-09-19T04:43:07.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.505+0000 7f55c77fe640 1 -- 172.21.15.193:0/1504533264 learned_addr learned my addr 172.21.15.193:0/1504533264 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-19T04:43:07.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.505+0000 7f55ccc15640 1 --2- 172.21.15.193:0/1504533264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55c81026c0 0x7f55c819f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:07.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.505+0000 7f55c7fff640 1 --2- 172.21.15.193:0/1504533264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55c8100b00 0x7f55c8198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:07.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.505+0000 7f55c77fe640 1 -- 172.21.15.193:0/1504533264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55c81026c0 msgr2=0x7f55c819f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:07.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.505+0000 7f55c77fe640 1 --2- 172.21.15.193:0/1504533264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55c81026c0 0x7f55c819f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:07.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.505+0000 7f55c77fe640 1 -- 172.21.15.193:0/1504533264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55c8100b00 msgr2=0x7f55c8198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:07.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.505+0000 7f55c77fe640 1 --2- 172.21.15.193:0/1504533264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55c8100b00 0x7f55c8198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:07.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.505+0000 7f55c77fe640 1 -- 172.21.15.193:0/1504533264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c819fc30 con 0x7f55c8101d00 2024-09-19T04:43:07.530 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.505+0000 7f55ccc15640 1 --2- 172.21.15.193:0/1504533264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55c81026c0 0x7f55c819f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:43:07.530 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.505+0000 7f55c77fe640 1 --2- 172.21.15.193:0/1504533264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55c8101d00 0x7f55c8198540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00e9f0 tx=0x7f55bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:07.530 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.505+0000 7f55c57fa640 1 -- 172.21.15.193:0/1504533264 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00cd80 con 0x7f55c8101d00 2024-09-19T04:43:07.530 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.505+0000 7f55ce678640 1 -- 172.21.15.193:0/1504533264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c819ff20 con 0x7f55c8101d00 2024-09-19T04:43:07.530 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.505+0000 7f55c57fa640 1 -- 172.21.15.193:0/1504533264 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55bc010910 con 0x7f55c8101d00 2024-09-19T04:43:07.530 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.505+0000 7f55c57fa640 1 -- 172.21.15.193:0/1504533264 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0188f0 con 0x7f55c8101d00 2024-09-19T04:43:07.530 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.505+0000 7f55ce678640 1 -- 172.21.15.193:0/1504533264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c81a03d0 con 0x7f55c8101d00 2024-09-19T04:43:07.530 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.506+0000 7f55ce678640 1 -- 172.21.15.193:0/1504533264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c81a0600 con 0x7f55c8101d00 2024-09-19T04:43:07.530 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.507+0000 7f55c57fa640 1 -- 172.21.15.193:0/1504533264 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f55bc010ac0 con 0x7f55c8101d00 2024-09-19T04:43:07.530 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.508+0000 7f55c57fa640 1 --2- 172.21.15.193:0/1504533264 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f55b00a3270 0x7f55b00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:07.530 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.508+0000 7f55c57fa640 1 -- 172.21.15.193:0/1504533264 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6727+0+0 (secure 0 0 0) 0x7f55bc0cf7a0 con 0x7f55c8101d00 2024-09-19T04:43:07.530 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.509+0000 7f55c7fff640 1 --2- 172.21.15.193:0/1504533264 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f55b00a3270 0x7f55b00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:07.530 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.509+0000 7f55c7fff640 1 --2- 172.21.15.193:0/1504533264 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f55b00a3270 0x7f55b00a5730 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f55b8006010 tx=0x7f55b80315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:07.530 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.512+0000 7f55c57fa640 1 -- 172.21.15.193:0/1504533264 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f55bc09caf0 con 0x7f55c8101d00 2024-09-19T04:43:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:43:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:43:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:07 smithi123 ceph-mon[30511]: Detected new or changed devices on smithi183 2024-09-19T04:43:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:07.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:07.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:07.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:07.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:07 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-19T04:43:07.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:07.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:07.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:43:07.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:07.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:07 smithi123 ceph-mon[30511]: osd.10 [v2:172.21.15.183:6824/3355349245,v1:172.21.15.183:6825/3355349245] boot 2024-09-19T04:43:07.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:07 smithi123 ceph-mon[30511]: osdmap e70: 11 total, 11 up, 11 in 2024-09-19T04:43:07.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:43:07.702 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:07.700+0000 7f55ce678640 1 -- 172.21.15.193:0/1504533264 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f55c806af50 con 0x7f55b00a3270 2024-09-19T04:43:07.702 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:43:07.702 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:43:07.702 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:07 smithi193 ceph-mon[35592]: Detected new or changed devices on smithi183 2024-09-19T04:43:07.702 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:07.702 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:07.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:07.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:07.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:07.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:07.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:07 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-19T04:43:07.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:07.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:07.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:43:07.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:07.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:07 smithi193 ceph-mon[35592]: osd.10 [v2:172.21.15.183:6824/3355349245,v1:172.21.15.183:6825/3355349245] boot 2024-09-19T04:43:07.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:07 smithi193 ceph-mon[35592]: osdmap e70: 11 total, 11 up, 11 in 2024-09-19T04:43:07.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:43:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:43:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:43:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:07 smithi183 ceph-mon[35333]: Detected new or changed devices on smithi183 2024-09-19T04:43:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:07 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-19T04:43:07.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:07.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:07.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:43:07.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:07.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:07 smithi183 ceph-mon[35333]: osd.10 [v2:172.21.15.183:6824/3355349245,v1:172.21.15.183:6825/3355349245] boot 2024-09-19T04:43:07.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:07 smithi183 ceph-mon[35333]: osdmap e70: 11 total, 11 up, 11 in 2024-09-19T04:43:07.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:43:08.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:08 smithi193 ceph-mon[35592]: purged_snaps scrub starts 2024-09-19T04:43:08.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:08 smithi193 ceph-mon[35592]: purged_snaps scrub ok 2024-09-19T04:43:08.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:08 smithi193 ceph-mon[35592]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:43:08.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:08 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:43:08.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:08 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:43:08.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:08 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:08.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:08 smithi193 ceph-mon[35592]: osdmap e71: 11 total, 11 up, 11 in 2024-09-19T04:43:08.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:08 smithi123 ceph-mon[30511]: purged_snaps scrub starts 2024-09-19T04:43:08.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:08 smithi123 ceph-mon[30511]: purged_snaps scrub ok 2024-09-19T04:43:08.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:08 smithi123 ceph-mon[30511]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:43:08.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:08 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:43:08.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:08 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:43:08.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:08 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:08.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:08 smithi123 ceph-mon[30511]: osdmap e71: 11 total, 11 up, 11 in 2024-09-19T04:43:08.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:08 smithi183 ceph-mon[35333]: purged_snaps scrub starts 2024-09-19T04:43:08.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:08 smithi183 ceph-mon[35333]: purged_snaps scrub ok 2024-09-19T04:43:08.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:08 smithi183 ceph-mon[35333]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:43:08.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:08 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:43:08.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:08 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:43:08.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:08 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:08.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:08 smithi183 ceph-mon[35333]: osdmap e71: 11 total, 11 up, 11 in 2024-09-19T04:43:09.444 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:09 smithi193 ceph-mon[35592]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:43:09.444 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:09 smithi193 ceph-mon[35592]: osdmap e72: 11 total, 11 up, 11 in 2024-09-19T04:43:09.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:09 smithi123 ceph-mon[30511]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:43:09.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:09 smithi123 ceph-mon[30511]: osdmap e72: 11 total, 11 up, 11 in 2024-09-19T04:43:09.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:09 smithi183 ceph-mon[35333]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:43:09.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:09 smithi183 ceph-mon[35333]: osdmap e72: 11 total, 11 up, 11 in 2024-09-19T04:43:10.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:10 smithi123 ceph-mon[30511]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:10.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:10 smithi123 ceph-mon[30511]: osdmap e73: 11 total, 11 up, 11 in 2024-09-19T04:43:10.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:10 smithi123 ceph-mon[30511]: from='client.? 172.21.15.193:0/1203847494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03046fef-b16b-42f8-9a19-6c0b78cbfe7b"}]: dispatch 2024-09-19T04:43:10.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:10 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03046fef-b16b-42f8-9a19-6c0b78cbfe7b"}]: dispatch 2024-09-19T04:43:10.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:10 smithi123 ceph-mon[30511]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03046fef-b16b-42f8-9a19-6c0b78cbfe7b"}]': finished 2024-09-19T04:43:10.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:10 smithi123 ceph-mon[30511]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T04:43:10.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:10 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:10.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:10 smithi193 ceph-mon[35592]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:10.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:10 smithi193 ceph-mon[35592]: osdmap e73: 11 total, 11 up, 11 in 2024-09-19T04:43:10.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:10 smithi193 ceph-mon[35592]: from='client.? 172.21.15.193:0/1203847494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03046fef-b16b-42f8-9a19-6c0b78cbfe7b"}]: dispatch 2024-09-19T04:43:10.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:10 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03046fef-b16b-42f8-9a19-6c0b78cbfe7b"}]: dispatch 2024-09-19T04:43:10.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:10 smithi193 ceph-mon[35592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03046fef-b16b-42f8-9a19-6c0b78cbfe7b"}]': finished 2024-09-19T04:43:10.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:10 smithi193 ceph-mon[35592]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T04:43:10.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:10 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:10.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:10 smithi183 ceph-mon[35333]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:10.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:10 smithi183 ceph-mon[35333]: osdmap e73: 11 total, 11 up, 11 in 2024-09-19T04:43:10.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:10 smithi183 ceph-mon[35333]: from='client.? 172.21.15.193:0/1203847494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03046fef-b16b-42f8-9a19-6c0b78cbfe7b"}]: dispatch 2024-09-19T04:43:10.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:10 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03046fef-b16b-42f8-9a19-6c0b78cbfe7b"}]: dispatch 2024-09-19T04:43:10.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:10 smithi183 ceph-mon[35333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03046fef-b16b-42f8-9a19-6c0b78cbfe7b"}]': finished 2024-09-19T04:43:10.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:10 smithi183 ceph-mon[35333]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T04:43:10.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:10 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:11.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:11 smithi123 ceph-mon[30511]: from='client.? 172.21.15.193:0/745596122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:43:11.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:11 smithi193 ceph-mon[35592]: from='client.? 172.21.15.193:0/745596122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:43:11.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:11 smithi183 ceph-mon[35333]: from='client.? 172.21.15.193:0/745596122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:43:12.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:12 smithi123 ceph-mon[30511]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:12.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:12 smithi193 ceph-mon[35592]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:12 smithi183 ceph-mon[35333]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:14.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:14 smithi123 ceph-mon[30511]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-19T04:43:14.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:14 smithi193 ceph-mon[35592]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-19T04:43:14.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:14 smithi183 ceph-mon[35333]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-19T04:43:16.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:16 smithi193 ceph-mon[35592]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T04:43:16.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:16 smithi123 ceph-mon[30511]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T04:43:16.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:16 smithi183 ceph-mon[35333]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T04:43:18.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:18 smithi123 ceph-mon[30511]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:43:18.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:18 smithi193 ceph-mon[35592]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:43:18.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:18 smithi183 ceph-mon[35333]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:43:19.456 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:19 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T04:43:19.457 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:19 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:19.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:19 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T04:43:19.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:19 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:19.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:19 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T04:43:19.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:19 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:20.578 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:20 smithi193 ceph-mon[35592]: Deploying daemon osd.11 on smithi193 2024-09-19T04:43:20.578 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:20 smithi193 ceph-mon[35592]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T04:43:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:20 smithi123 ceph-mon[30511]: Deploying daemon osd.11 on smithi193 2024-09-19T04:43:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:20 smithi123 ceph-mon[30511]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T04:43:20.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:20 smithi183 ceph-mon[35333]: Deploying daemon osd.11 on smithi193 2024-09-19T04:43:20.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:20 smithi183 ceph-mon[35333]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T04:43:22.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:22 smithi123 ceph-mon[30511]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T04:43:22.708 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:22 smithi193 ceph-mon[35592]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T04:43:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:22 smithi183 ceph-mon[35333]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T04:43:24.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:24 smithi123 ceph-mon[30511]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:43:24.707 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:24 smithi193 ceph-mon[35592]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:43:24.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:24 smithi183 ceph-mon[35333]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:43:25.576 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:25 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:43:25.576 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:25 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:25.576 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:25 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:25.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:25 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:43:25.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:25 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:25.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:25 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:25.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:25 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:43:25.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:25 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:25.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:25 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:26.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:26 smithi123 ceph-mon[30511]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:26.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:26 smithi193 ceph-mon[35592]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:26.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:26 smithi183 ceph-mon[35333]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:28.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:28 smithi123 ceph-mon[30511]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:28.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:28 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:28.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:28 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:28.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:28 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:28.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:28 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:43:28.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:28 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:28.711 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:28 smithi193 ceph-mon[35592]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:28.711 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:28 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:28.712 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:28 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:28.712 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:28 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:28.712 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:28 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:43:28.712 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:28 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:28.751 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:28.749+0000 7f55c57fa640 1 -- 172.21.15.193:0/1504533264 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f55c806af50 con 0x7f55b00a3270 2024-09-19T04:43:28.751 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 11 on host 'smithi193' 2024-09-19T04:43:28.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:28.751+0000 7f55ce678640 1 -- 172.21.15.193:0/1504533264 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f55b00a3270 msgr2=0x7f55b00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:28.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:28.751+0000 7f55ce678640 1 --2- 172.21.15.193:0/1504533264 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f55b00a3270 0x7f55b00a5730 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f55b8006010 tx=0x7f55b80315b0 comp rx=0 tx=0).stop 2024-09-19T04:43:28.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:28.751+0000 7f55ce678640 1 -- 172.21.15.193:0/1504533264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55c8101d00 msgr2=0x7f55c8198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:28.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:28.751+0000 7f55ce678640 1 --2- 172.21.15.193:0/1504533264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55c8101d00 0x7f55c8198540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00e9f0 tx=0x7f55bc00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:43:28.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:28.751+0000 7f55ce678640 1 -- 172.21.15.193:0/1504533264 shutdown_connections 2024-09-19T04:43:28.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:28.751+0000 7f55ce678640 1 --2- 172.21.15.193:0/1504533264 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f55b00a3270 0x7f55b00a5730 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:28.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:28.751+0000 7f55ce678640 1 --2- 172.21.15.193:0/1504533264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55c81026c0 0x7f55c819f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:28.754 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:28.751+0000 7f55ce678640 1 --2- 172.21.15.193:0/1504533264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55c8101d00 0x7f55c8198540 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:28.754 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:28.752+0000 7f55ce678640 1 --2- 172.21.15.193:0/1504533264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55c8100b00 0x7f55c8198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:28.754 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:28.752+0000 7f55ce678640 1 -- 172.21.15.193:0/1504533264 >> 172.21.15.193:0/1504533264 conn(0x7f55c80fc270 msgr2=0x7f55c806a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:28.754 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:28.752+0000 7f55ce678640 1 -- 172.21.15.193:0/1504533264 shutdown_connections 2024-09-19T04:43:28.754 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:28.752+0000 7f55ce678640 1 -- 172.21.15.193:0/1504533264 wait complete. 2024-09-19T04:43:28.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:28 smithi183 ceph-mon[35333]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:28.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:28 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:28.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:28 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:28.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:28 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:28.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:28 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:43:28.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:28 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:29.466 DEBUG:teuthology.orchestra.run.smithi193:osd.11> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.11.service 2024-09-19T04:43:29.468 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-19T04:43:29.468 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T04:43:29.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:29 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:43:29.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:29 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:29.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:29 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:29.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:29 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:43:29.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:29 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:29.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:29 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:29.841 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:43:29.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:29 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:43:29.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:29 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:29.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:29 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:30.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.413+0000 7f885d000640 1 -- 172.21.15.123:0/1619671468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8858076220 msgr2=0x7f8858074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:30.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.413+0000 7f885d000640 1 --2- 172.21.15.123:0/1619671468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8858076220 0x7f8858074680 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8848009900 tx=0x7f884802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:43:30.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.413+0000 7f885d000640 1 -- 172.21.15.123:0/1619671468 shutdown_connections 2024-09-19T04:43:30.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.413+0000 7f885d000640 1 --2- 172.21.15.123:0/1619671468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8858109a80 0x7f885810bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:30.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.413+0000 7f885d000640 1 --2- 172.21.15.123:0/1619671468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8858074c50 0x7f88580750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:30.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.413+0000 7f885d000640 1 --2- 172.21.15.123:0/1619671468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8858076220 0x7f8858074680 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:30.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.413+0000 7f885d000640 1 -- 172.21.15.123:0/1619671468 >> 172.21.15.123:0/1619671468 conn(0x7f88580fd390 msgr2=0x7f88580ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:30.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.413+0000 7f885d000640 1 -- 172.21.15.123:0/1619671468 shutdown_connections 2024-09-19T04:43:30.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.413+0000 7f885d000640 1 -- 172.21.15.123:0/1619671468 wait complete. 2024-09-19T04:43:30.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.414+0000 7f885d000640 1 Processor -- start 2024-09-19T04:43:30.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.414+0000 7f885d000640 1 -- start start 2024-09-19T04:43:30.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.414+0000 7f885d000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8858074c50 0x7f885819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:30.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.414+0000 7f885d000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8858076220 0x7f885819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:30.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.414+0000 7f885d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8858109a80 0x7f88581a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:30.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.414+0000 7f885d000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f885810fad0 con 0x7f8858109a80 2024-09-19T04:43:30.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.414+0000 7f885d000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f885810f950 con 0x7f8858074c50 2024-09-19T04:43:30.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.414+0000 7f885d000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f885810fc50 con 0x7f8858076220 2024-09-19T04:43:30.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.415+0000 7f8855400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8858076220 0x7f885819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.415+0000 7f8855400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8858076220 0x7f885819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:59776/0 (socket says 172.21.15.123:59776) 2024-09-19T04:43:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.415+0000 7f8856800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8858109a80 0x7f88581a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.415+0000 7f8855400640 1 -- 172.21.15.123:0/346660746 learned_addr learned my addr 172.21.15.123:0/346660746 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:43:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.415+0000 7f8855e00640 1 --2- 172.21.15.123:0/346660746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8858074c50 0x7f885819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.415+0000 7f8855400640 1 -- 172.21.15.123:0/346660746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8858074c50 msgr2=0x7f885819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.415+0000 7f8855400640 1 --2- 172.21.15.123:0/346660746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8858074c50 0x7f885819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.415+0000 7f8855400640 1 -- 172.21.15.123:0/346660746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8858109a80 msgr2=0x7f88581a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.415+0000 7f8855400640 1 --2- 172.21.15.123:0/346660746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8858109a80 0x7f88581a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.415+0000 7f8855400640 1 -- 172.21.15.123:0/346660746 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88581a1d20 con 0x7f8858076220 2024-09-19T04:43:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.415+0000 7f8855400640 1 --2- 172.21.15.123:0/346660746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8858076220 0x7f885819a630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f884400ea60 tx=0x7f884400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:30.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.415+0000 7f884ea00640 1 -- 172.21.15.123:0/346660746 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884400cde0 con 0x7f8858076220 2024-09-19T04:43:30.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.416+0000 7f884ea00640 1 -- 172.21.15.123:0/346660746 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88440035f0 con 0x7f8858076220 2024-09-19T04:43:30.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.416+0000 7f885d000640 1 -- 172.21.15.123:0/346660746 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88581a1fb0 con 0x7f8858076220 2024-09-19T04:43:30.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.416+0000 7f884ea00640 1 -- 172.21.15.123:0/346660746 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8844010670 con 0x7f8858076220 2024-09-19T04:43:30.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.416+0000 7f885d000640 1 -- 172.21.15.123:0/346660746 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88581a24c0 con 0x7f8858076220 2024-09-19T04:43:30.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.417+0000 7f885d000640 1 -- 172.21.15.123:0/346660746 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8858108800 con 0x7f8858076220 2024-09-19T04:43:30.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.418+0000 7f884ea00640 1 -- 172.21.15.123:0/346660746 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f88440037a0 con 0x7f8858076220 2024-09-19T04:43:30.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.420+0000 7f884ea00640 1 --2- 172.21.15.123:0/346660746 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f883c0a3270 0x7f883c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:30.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.420+0000 7f884ea00640 1 -- 172.21.15.123:0/346660746 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6809+0+0 (secure 0 0 0) 0x7f88440cf6a0 con 0x7f8858076220 2024-09-19T04:43:30.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.420+0000 7f8855e00640 1 --2- 172.21.15.123:0/346660746 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f883c0a3270 0x7f883c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:30.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.421+0000 7f8855e00640 1 --2- 172.21.15.123:0/346660746 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f883c0a3270 0x7f883c0a5730 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f884802a9e0 tx=0x7f88480315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:30.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.423+0000 7f884ea00640 1 -- 172.21.15.123:0/346660746 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f884409c900 con 0x7f8858076220 2024-09-19T04:43:30.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.598+0000 7f885d000640 1 -- 172.21.15.123:0/346660746 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8858062e90 con 0x7f8858076220 2024-09-19T04:43:30.600 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:30 smithi123 ceph-mon[30511]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:30.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.598+0000 7f884ea00640 1 -- 172.21.15.123:0/346660746 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f884409c200 con 0x7f8858076220 2024-09-19T04:43:30.600 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:43:30.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.601+0000 7f885d000640 1 -- 172.21.15.123:0/346660746 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f883c0a3270 msgr2=0x7f883c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:30.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.601+0000 7f885d000640 1 --2- 172.21.15.123:0/346660746 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f883c0a3270 0x7f883c0a5730 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f884802a9e0 tx=0x7f88480315b0 comp rx=0 tx=0).stop 2024-09-19T04:43:30.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.601+0000 7f885d000640 1 -- 172.21.15.123:0/346660746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8858076220 msgr2=0x7f885819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:30.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.601+0000 7f885d000640 1 --2- 172.21.15.123:0/346660746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8858076220 0x7f885819a630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f884400ea60 tx=0x7f884400ef20 comp rx=0 tx=0).stop 2024-09-19T04:43:30.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.601+0000 7f885d000640 1 -- 172.21.15.123:0/346660746 shutdown_connections 2024-09-19T04:43:30.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.601+0000 7f885d000640 1 --2- 172.21.15.123:0/346660746 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f883c0a3270 0x7f883c0a5730 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:30.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.601+0000 7f885d000640 1 --2- 172.21.15.123:0/346660746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8858109a80 0x7f88581a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:30.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.601+0000 7f885d000640 1 --2- 172.21.15.123:0/346660746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8858076220 0x7f885819a630 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:30.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.601+0000 7f885d000640 1 --2- 172.21.15.123:0/346660746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8858074c50 0x7f885819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:30.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.601+0000 7f885d000640 1 -- 172.21.15.123:0/346660746 >> 172.21.15.123:0/346660746 conn(0x7f88580fd390 msgr2=0x7f88581059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:30.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.602+0000 7f885d000640 1 -- 172.21.15.123:0/346660746 shutdown_connections 2024-09-19T04:43:30.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:30.602+0000 7f885d000640 1 -- 172.21.15.123:0/346660746 wait complete. 2024-09-19T04:43:30.642 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:30 smithi193 ceph-mon[35592]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:30.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:30 smithi183 ceph-mon[35333]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:31.441 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1726720986,"num_in_osds":12,"osd_in_since":1726720990,"num_remapped_pgs":0} 2024-09-19T04:43:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:31 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/346660746' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:43:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:31 smithi123 ceph-mon[30511]: from='osd.11 [v2:172.21.15.193:6824/4291617038,v1:172.21.15.193:6825/4291617038]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:43:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:31 smithi123 ceph-mon[30511]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:43:31.688 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:31 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/346660746' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:43:31.688 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:31 smithi193 ceph-mon[35592]: from='osd.11 [v2:172.21.15.193:6824/4291617038,v1:172.21.15.193:6825/4291617038]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:43:31.688 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:31 smithi193 ceph-mon[35592]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:43:31.688 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:43:31 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:43:31.306+0000 7f89ac6d8640 -1 osd.11 0 log_to_monitors true 2024-09-19T04:43:31.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:31 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/346660746' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:43:31.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:31 smithi183 ceph-mon[35333]: from='osd.11 [v2:172.21.15.193:6824/4291617038,v1:172.21.15.193:6825/4291617038]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:43:31.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:31 smithi183 ceph-mon[35333]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:43:32.442 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T04:43:32.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:32 smithi123 ceph-mon[30511]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:32.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:32 smithi123 ceph-mon[30511]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T04:43:32.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:32 smithi123 ceph-mon[30511]: from='osd.11 [v2:172.21.15.193:6824/4291617038,v1:172.21.15.193:6825/4291617038]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:43:32.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:32 smithi123 ceph-mon[30511]: osdmap e75: 12 total, 11 up, 12 in 2024-09-19T04:43:32.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:32 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:32.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:32 smithi123 ceph-mon[30511]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:43:32.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:32 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:32.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:32 smithi193 ceph-mon[35592]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:32.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:32 smithi193 ceph-mon[35592]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T04:43:32.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:32 smithi193 ceph-mon[35592]: from='osd.11 [v2:172.21.15.193:6824/4291617038,v1:172.21.15.193:6825/4291617038]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:43:32.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:32 smithi193 ceph-mon[35592]: osdmap e75: 12 total, 11 up, 12 in 2024-09-19T04:43:32.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:32 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:32.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:32 smithi193 ceph-mon[35592]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:43:32.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:32 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:32.811 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:43:32.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:32 smithi183 ceph-mon[35333]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:32.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:32 smithi183 ceph-mon[35333]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T04:43:32.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:32 smithi183 ceph-mon[35333]: from='osd.11 [v2:172.21.15.193:6824/4291617038,v1:172.21.15.193:6825/4291617038]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:43:32.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:32 smithi183 ceph-mon[35333]: osdmap e75: 12 total, 11 up, 12 in 2024-09-19T04:43:32.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:32 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:32.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:32 smithi183 ceph-mon[35333]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-19T04:43:32.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:32 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:33.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:43:32 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:43:32.815+0000 7f89a865b640 -1 osd.11 0 waiting for initial osdmap 2024-09-19T04:43:33.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:43:32 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:43:32.828+0000 7f89a047b640 -1 osd.11 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:43:33.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.354+0000 7facbae00640 1 -- 172.21.15.123:0/3607768462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb4078790 msgr2=0x7facb4079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:33.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.354+0000 7facbae00640 1 --2- 172.21.15.123:0/3607768462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb4078790 0x7facb4079030 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7facac009ca0 tx=0x7facac02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:43:33.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.354+0000 7facbae00640 1 -- 172.21.15.123:0/3607768462 shutdown_connections 2024-09-19T04:43:33.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.354+0000 7facbae00640 1 --2- 172.21.15.123:0/3607768462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb4078790 0x7facb4079030 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:33.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.354+0000 7facbae00640 1 --2- 172.21.15.123:0/3607768462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facb4077df0 0x7facb4078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:33.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.354+0000 7facbae00640 1 --2- 172.21.15.123:0/3607768462 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb4076ba0 0x7facb4076fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:33.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.354+0000 7facbae00640 1 -- 172.21.15.123:0/3607768462 >> 172.21.15.123:0/3607768462 conn(0x7facb40ff5b0 msgr2=0x7facb41019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:33.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.354+0000 7facbae00640 1 -- 172.21.15.123:0/3607768462 shutdown_connections 2024-09-19T04:43:33.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.354+0000 7facbae00640 1 -- 172.21.15.123:0/3607768462 wait complete. 2024-09-19T04:43:33.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.355+0000 7facbae00640 1 Processor -- start 2024-09-19T04:43:33.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.355+0000 7facbae00640 1 -- start start 2024-09-19T04:43:33.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.355+0000 7facbae00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facb4076ba0 0x7facb419e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:33.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.355+0000 7facbae00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb4077df0 0x7facb419e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:33.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.355+0000 7facbae00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb4078790 0x7facb41a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:33.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.355+0000 7facbae00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7facb4113f10 con 0x7facb4078790 2024-09-19T04:43:33.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.355+0000 7facbae00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7facb4113d90 con 0x7facb4076ba0 2024-09-19T04:43:33.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.356+0000 7facbae00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7facb4114090 con 0x7facb4077df0 2024-09-19T04:43:33.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.356+0000 7facb8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb4078790 0x7facb41a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:33.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.356+0000 7facb3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facb4076ba0 0x7facb419e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:33.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.356+0000 7facb3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb4077df0 0x7facb419e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:33.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.356+0000 7facb3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facb4076ba0 0x7facb419e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:43890/0 (socket says 172.21.15.123:43890) 2024-09-19T04:43:33.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.356+0000 7facb3e00640 1 -- 172.21.15.123:0/1815380106 learned_addr learned my addr 172.21.15.123:0/1815380106 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:43:33.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.356+0000 7facb3400640 1 -- 172.21.15.123:0/1815380106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facb4076ba0 msgr2=0x7facb419e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:33.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.356+0000 7facb3400640 1 --2- 172.21.15.123:0/1815380106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facb4076ba0 0x7facb419e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:33.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.356+0000 7facb3400640 1 -- 172.21.15.123:0/1815380106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb4078790 msgr2=0x7facb41a5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:33.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.356+0000 7facb3400640 1 --2- 172.21.15.123:0/1815380106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb4078790 0x7facb41a5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:33.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.356+0000 7facb3400640 1 -- 172.21.15.123:0/1815380106 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb41a60b0 con 0x7facb4077df0 2024-09-19T04:43:33.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.357+0000 7facb3400640 1 --2- 172.21.15.123:0/1815380106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb4077df0 0x7facb419e9c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faca000e9f0 tx=0x7faca000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:33.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.357+0000 7facb0c00640 1 -- 172.21.15.123:0/1815380106 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca000ccc0 con 0x7facb4077df0 2024-09-19T04:43:33.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.357+0000 7facbae00640 1 -- 172.21.15.123:0/1815380106 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb41a63a0 con 0x7facb4077df0 2024-09-19T04:43:33.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.357+0000 7facb0c00640 1 -- 172.21.15.123:0/1815380106 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faca0010910 con 0x7facb4077df0 2024-09-19T04:43:33.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.357+0000 7facb0c00640 1 -- 172.21.15.123:0/1815380106 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0018960 con 0x7facb4077df0 2024-09-19T04:43:33.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.357+0000 7facbae00640 1 -- 172.21.15.123:0/1815380106 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb41a68e0 con 0x7facb4077df0 2024-09-19T04:43:33.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.358+0000 7facbae00640 1 -- 172.21.15.123:0/1815380106 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb4110780 con 0x7facb4077df0 2024-09-19T04:43:33.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.361+0000 7facb0c00640 1 -- 172.21.15.123:0/1815380106 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7faca0010ac0 con 0x7facb4077df0 2024-09-19T04:43:33.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.362+0000 7facb0c00640 1 --2- 172.21.15.123:0/1815380106 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fac980a3270 0x7fac980a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:33.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.362+0000 7facb0c00640 1 -- 172.21.15.123:0/1815380106 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6847+0+0 (secure 0 0 0) 0x7faca00cf750 con 0x7facb4077df0 2024-09-19T04:43:33.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.362+0000 7facb3e00640 1 --2- 172.21.15.123:0/1815380106 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fac980a3270 0x7fac980a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:33.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.363+0000 7facb3e00640 1 --2- 172.21.15.123:0/1815380106 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fac980a3270 0x7fac980a5730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faca4009cd0 tx=0x7faca4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:33.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.366+0000 7facb0c00640 1 -- 172.21.15.123:0/1815380106 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faca009c340 con 0x7facb4077df0 2024-09-19T04:43:33.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.548+0000 7facbae00640 1 -- 172.21.15.123:0/1815380106 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7facb4062df0 con 0x7facb4077df0 2024-09-19T04:43:33.550 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:33 smithi123 ceph-mon[30511]: Detected new or changed devices on smithi193 2024-09-19T04:43:33.551 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:33.551 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:33.551 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:33.551 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:33.551 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:33.551 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:33 smithi123 ceph-mon[30511]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-19T04:43:33.551 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:33 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi193 to 4272M 2024-09-19T04:43:33.551 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:33 smithi123 ceph-mon[30511]: osdmap e76: 12 total, 11 up, 12 in 2024-09-19T04:43:33.551 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:33.551 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:33.551 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:33.551 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:33.551 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:43:33.552 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:33.552 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:33 smithi123 ceph-mon[30511]: from='osd.11 ' entity='osd.11' 2024-09-19T04:43:33.552 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:33 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:33.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.549+0000 7facb0c00640 1 -- 172.21.15.123:0/1815380106 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7faca00a0640 con 0x7facb4077df0 2024-09-19T04:43:33.552 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:43:33.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.552+0000 7facbae00640 1 -- 172.21.15.123:0/1815380106 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fac980a3270 msgr2=0x7fac980a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:33.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.552+0000 7facbae00640 1 --2- 172.21.15.123:0/1815380106 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fac980a3270 0x7fac980a5730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faca4009cd0 tx=0x7faca4005f90 comp rx=0 tx=0).stop 2024-09-19T04:43:33.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.552+0000 7facbae00640 1 -- 172.21.15.123:0/1815380106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb4077df0 msgr2=0x7facb419e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:33.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.552+0000 7facbae00640 1 --2- 172.21.15.123:0/1815380106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb4077df0 0x7facb419e9c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faca000e9f0 tx=0x7faca000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:43:33.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.552+0000 7facbae00640 1 -- 172.21.15.123:0/1815380106 shutdown_connections 2024-09-19T04:43:33.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.552+0000 7facbae00640 1 --2- 172.21.15.123:0/1815380106 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fac980a3270 0x7fac980a5730 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:33.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.552+0000 7facbae00640 1 --2- 172.21.15.123:0/1815380106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb4078790 0x7facb41a5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:33.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.552+0000 7facbae00640 1 --2- 172.21.15.123:0/1815380106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb4077df0 0x7facb419e9c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:33.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.552+0000 7facbae00640 1 --2- 172.21.15.123:0/1815380106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facb4076ba0 0x7facb419e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:33.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.552+0000 7facbae00640 1 -- 172.21.15.123:0/1815380106 >> 172.21.15.123:0/1815380106 conn(0x7facb40ff5b0 msgr2=0x7facb4101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:33.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.552+0000 7facbae00640 1 -- 172.21.15.123:0/1815380106 shutdown_connections 2024-09-19T04:43:33.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:33.552+0000 7facbae00640 1 -- 172.21.15.123:0/1815380106 wait complete. 2024-09-19T04:43:33.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:33 smithi193 ceph-mon[35592]: Detected new or changed devices on smithi193 2024-09-19T04:43:33.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:33.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:33.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:33.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:33.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:33.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:33 smithi193 ceph-mon[35592]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-19T04:43:33.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:33 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi193 to 4272M 2024-09-19T04:43:33.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:33 smithi193 ceph-mon[35592]: osdmap e76: 12 total, 11 up, 12 in 2024-09-19T04:43:33.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:33.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:33.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:33.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:33.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:43:33.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:33.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:33 smithi193 ceph-mon[35592]: from='osd.11 ' entity='osd.11' 2024-09-19T04:43:33.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:33 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:33.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:33 smithi183 ceph-mon[35333]: Detected new or changed devices on smithi193 2024-09-19T04:43:33.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:33.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:33.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:33.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:33.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:43:33.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:33 smithi183 ceph-mon[35333]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-19T04:43:33.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:33 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi193 to 4272M 2024-09-19T04:43:33.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:33 smithi183 ceph-mon[35333]: osdmap e76: 12 total, 11 up, 12 in 2024-09-19T04:43:33.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:33.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:33.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:33.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:33.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:43:33.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:33.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:33 smithi183 ceph-mon[35333]: from='osd.11 ' entity='osd.11' 2024-09-19T04:43:33.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:33 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:34.355 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1726720986,"num_in_osds":12,"osd_in_since":1726720990,"num_remapped_pgs":0} 2024-09-19T04:43:34.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:34 smithi123 ceph-mon[30511]: purged_snaps scrub starts 2024-09-19T04:43:34.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:34 smithi123 ceph-mon[30511]: purged_snaps scrub ok 2024-09-19T04:43:34.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:34 smithi123 ceph-mon[30511]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:34.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:34 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1815380106' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:43:34.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:34 smithi123 ceph-mon[30511]: osd.11 [v2:172.21.15.193:6824/4291617038,v1:172.21.15.193:6825/4291617038] boot 2024-09-19T04:43:34.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:34 smithi123 ceph-mon[30511]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T04:43:34.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:34 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:34.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:34 smithi193 ceph-mon[35592]: purged_snaps scrub starts 2024-09-19T04:43:34.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:34 smithi193 ceph-mon[35592]: purged_snaps scrub ok 2024-09-19T04:43:34.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:34 smithi193 ceph-mon[35592]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:34.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:34 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1815380106' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:43:34.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:34 smithi193 ceph-mon[35592]: osd.11 [v2:172.21.15.193:6824/4291617038,v1:172.21.15.193:6825/4291617038] boot 2024-09-19T04:43:34.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:34 smithi193 ceph-mon[35592]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T04:43:34.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:34 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:34.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:34 smithi183 ceph-mon[35333]: purged_snaps scrub starts 2024-09-19T04:43:34.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:34 smithi183 ceph-mon[35333]: purged_snaps scrub ok 2024-09-19T04:43:34.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:34 smithi183 ceph-mon[35333]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:43:34.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:34 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1815380106' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:43:34.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:34 smithi183 ceph-mon[35333]: osd.11 [v2:172.21.15.193:6824/4291617038,v1:172.21.15.193:6825/4291617038] boot 2024-09-19T04:43:34.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:34 smithi183 ceph-mon[35333]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T04:43:34.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:34 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:43:35.356 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T04:43:35.719 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:43:36.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:35 smithi183 ceph-mon[35333]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T04:43:36.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:35 smithi183 ceph-mon[35333]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:36.148 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:35 smithi123 ceph-mon[30511]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T04:43:36.148 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:35 smithi123 ceph-mon[30511]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:36.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:35 smithi193 ceph-mon[35592]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T04:43:36.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:35 smithi193 ceph-mon[35592]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:36.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.253+0000 7f3459e00640 1 -- 172.21.15.123:0/962146660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3454076ba0 msgr2=0x7f3454076fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:36.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.253+0000 7f3459e00640 1 --2- 172.21.15.123:0/962146660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3454076ba0 0x7f3454076fa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3448009900 tx=0x7f344802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:43:36.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.253+0000 7f3459e00640 1 -- 172.21.15.123:0/962146660 shutdown_connections 2024-09-19T04:43:36.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.253+0000 7f3459e00640 1 --2- 172.21.15.123:0/962146660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3454078790 0x7f3454079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:36.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.253+0000 7f3459e00640 1 --2- 172.21.15.123:0/962146660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3454077df0 0x7f3454078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:36.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.253+0000 7f3459e00640 1 --2- 172.21.15.123:0/962146660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3454076ba0 0x7f3454076fa0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:36.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.253+0000 7f3459e00640 1 -- 172.21.15.123:0/962146660 >> 172.21.15.123:0/962146660 conn(0x7f34540ff3e0 msgr2=0x7f3454101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:36.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.253+0000 7f3459e00640 1 -- 172.21.15.123:0/962146660 shutdown_connections 2024-09-19T04:43:36.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.253+0000 7f3459e00640 1 -- 172.21.15.123:0/962146660 wait complete. 2024-09-19T04:43:36.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.254+0000 7f3459e00640 1 Processor -- start 2024-09-19T04:43:36.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.254+0000 7f3459e00640 1 -- start start 2024-09-19T04:43:36.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3459e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3454076ba0 0x7f345419e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:36.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3459e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3454077df0 0x7f345419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:36.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3459e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3454078790 0x7f34541a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:36.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3459e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3454113d60 con 0x7f3454078790 2024-09-19T04:43:36.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3459e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3454113be0 con 0x7f3454076ba0 2024-09-19T04:43:36.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3459e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3454113ee0 con 0x7f3454077df0 2024-09-19T04:43:36.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3453e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3454077df0 0x7f345419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:36.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3458a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3454076ba0 0x7f345419e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:36.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3458a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3454076ba0 0x7f345419e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:43912/0 (socket says 172.21.15.123:43912) 2024-09-19T04:43:36.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3459400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3454078790 0x7f34541a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:36.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3453e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3454077df0 0x7f345419e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42806/0 (socket says 172.21.15.123:42806) 2024-09-19T04:43:36.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3458a00640 1 -- 172.21.15.123:0/3944803421 learned_addr learned my addr 172.21.15.123:0/3944803421 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:43:36.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3458a00640 1 -- 172.21.15.123:0/3944803421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3454077df0 msgr2=0x7f345419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3458a00640 1 --2- 172.21.15.123:0/3944803421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3454077df0 0x7f345419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3458a00640 1 -- 172.21.15.123:0/3944803421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3454078790 msgr2=0x7f34541a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3458a00640 1 --2- 172.21.15.123:0/3944803421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3454078790 0x7f34541a58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3458a00640 1 -- 172.21.15.123:0/3944803421 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34541a5f10 con 0x7f3454076ba0 2024-09-19T04:43:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3453e00640 1 --2- 172.21.15.123:0/3944803421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3454077df0 0x7f345419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:43:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.255+0000 7f3459400640 1 --2- 172.21.15.123:0/3944803421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3454078790 0x7f34541a58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:43:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.256+0000 7f3458a00640 1 --2- 172.21.15.123:0/3944803421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3454076ba0 0x7f345419e2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3448031440 tx=0x7f3448032e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.256+0000 7f3451600640 1 -- 172.21.15.123:0/3944803421 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344803f030 con 0x7f3454076ba0 2024-09-19T04:43:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.256+0000 7f3459e00640 1 -- 172.21.15.123:0/3944803421 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34541a61a0 con 0x7f3454076ba0 2024-09-19T04:43:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.256+0000 7f3451600640 1 -- 172.21.15.123:0/3944803421 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3448003ce0 con 0x7f3454076ba0 2024-09-19T04:43:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.256+0000 7f3451600640 1 -- 172.21.15.123:0/3944803421 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448035070 con 0x7f3454076ba0 2024-09-19T04:43:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.256+0000 7f3459e00640 1 -- 172.21.15.123:0/3944803421 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34541a6680 con 0x7f3454076ba0 2024-09-19T04:43:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.257+0000 7f3459e00640 1 -- 172.21.15.123:0/3944803421 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34541105d0 con 0x7f3454076ba0 2024-09-19T04:43:36.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.259+0000 7f3451600640 1 -- 172.21.15.123:0/3944803421 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f3448042030 con 0x7f3454076ba0 2024-09-19T04:43:36.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.260+0000 7f3451600640 1 --2- 172.21.15.123:0/3944803421 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f343c0a3270 0x7f343c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:36.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.260+0000 7f3451600640 1 -- 172.21.15.123:0/3944803421 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f34480efa70 con 0x7f3454076ba0 2024-09-19T04:43:36.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.261+0000 7f3453e00640 1 --2- 172.21.15.123:0/3944803421 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f343c0a3270 0x7f343c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:36.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.261+0000 7f3453e00640 1 --2- 172.21.15.123:0/3944803421 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f343c0a3270 0x7f343c0a5730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f345419f800 tx=0x7f344000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:36.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.264+0000 7f3451600640 1 -- 172.21.15.123:0/3944803421 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f34480bc540 con 0x7f3454076ba0 2024-09-19T04:43:36.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.443+0000 7f3459e00640 1 -- 172.21.15.123:0/3944803421 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3454062df0 con 0x7f3454076ba0 2024-09-19T04:43:36.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.444+0000 7f3451600640 1 -- 172.21.15.123:0/3944803421 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) ==== 74+0+133 (secure 0 0 0) 0x7f34480c0840 con 0x7f3454076ba0 2024-09-19T04:43:36.445 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:43:36.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.446+0000 7f3459e00640 1 -- 172.21.15.123:0/3944803421 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f343c0a3270 msgr2=0x7f343c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:36.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.446+0000 7f3459e00640 1 --2- 172.21.15.123:0/3944803421 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f343c0a3270 0x7f343c0a5730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f345419f800 tx=0x7f344000b410 comp rx=0 tx=0).stop 2024-09-19T04:43:36.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.446+0000 7f3459e00640 1 -- 172.21.15.123:0/3944803421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3454076ba0 msgr2=0x7f345419e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:36.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.446+0000 7f3459e00640 1 --2- 172.21.15.123:0/3944803421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3454076ba0 0x7f345419e2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3448031440 tx=0x7f3448032e20 comp rx=0 tx=0).stop 2024-09-19T04:43:36.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.446+0000 7f3459e00640 1 -- 172.21.15.123:0/3944803421 shutdown_connections 2024-09-19T04:43:36.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.446+0000 7f3459e00640 1 --2- 172.21.15.123:0/3944803421 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f343c0a3270 0x7f343c0a5730 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:36.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.446+0000 7f3459e00640 1 --2- 172.21.15.123:0/3944803421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3454078790 0x7f34541a58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:36.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.446+0000 7f3459e00640 1 --2- 172.21.15.123:0/3944803421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3454077df0 0x7f345419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:36.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.446+0000 7f3459e00640 1 --2- 172.21.15.123:0/3944803421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3454076ba0 0x7f345419e2e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:36.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.446+0000 7f3459e00640 1 -- 172.21.15.123:0/3944803421 >> 172.21.15.123:0/3944803421 conn(0x7f34540ff3e0 msgr2=0x7f3454100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:36.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.446+0000 7f3459e00640 1 -- 172.21.15.123:0/3944803421 shutdown_connections 2024-09-19T04:43:36.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:36.446+0000 7f3459e00640 1 -- 172.21.15.123:0/3944803421 wait complete. 2024-09-19T04:43:37.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:36 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3944803421' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:43:37.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:36 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3944803421' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:43:37.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:36 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3944803421' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:43:37.329 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1726721013,"num_in_osds":12,"osd_in_since":1726720990,"num_remapped_pgs":0} 2024-09-19T04:43:37.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:43:37.695 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:43:38.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:37 smithi183 ceph-mon[35333]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:38.102 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:37 smithi123 ceph-mon[30511]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:38.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:37 smithi193 ceph-mon[35592]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:38.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.314+0000 7f82eaa00640 1 -- 172.21.15.123:0/2479258534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82e4103e30 msgr2=0x7f82e4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:38.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.314+0000 7f82eaa00640 1 --2- 172.21.15.123:0/2479258534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82e4103e30 0x7f82e4104210 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f82d8009900 tx=0x7f82d802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:43:38.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.314+0000 7f82eaa00640 1 -- 172.21.15.123:0/2479258534 shutdown_connections 2024-09-19T04:43:38.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.314+0000 7f82eaa00640 1 --2- 172.21.15.123:0/2479258534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82e4108f60 0x7f82e4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:38.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.314+0000 7f82eaa00640 1 --2- 172.21.15.123:0/2479258534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82e41047e0 0x7f82e4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:38.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.314+0000 7f82eaa00640 1 --2- 172.21.15.123:0/2479258534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82e4103e30 0x7f82e4104210 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:38.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.314+0000 7f82eaa00640 1 -- 172.21.15.123:0/2479258534 >> 172.21.15.123:0/2479258534 conn(0x7f82e40ff8d0 msgr2=0x7f82e4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:38.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.315+0000 7f82eaa00640 1 -- 172.21.15.123:0/2479258534 shutdown_connections 2024-09-19T04:43:38.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.315+0000 7f82eaa00640 1 -- 172.21.15.123:0/2479258534 wait complete. 2024-09-19T04:43:38.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.316+0000 7f82eaa00640 1 Processor -- start 2024-09-19T04:43:38.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.316+0000 7f82eaa00640 1 -- start start 2024-09-19T04:43:38.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.316+0000 7f82eaa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82e4103e30 0x7f82e419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:38.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.316+0000 7f82eaa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82e41047e0 0x7f82e419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:38.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.316+0000 7f82eaa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82e4108f60 0x7f82e419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:38.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.316+0000 7f82eaa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f82e4114010 con 0x7f82e4108f60 2024-09-19T04:43:38.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.316+0000 7f82eaa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f82e4113e90 con 0x7f82e4103e30 2024-09-19T04:43:38.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.316+0000 7f82eaa00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f82e4114190 con 0x7f82e41047e0 2024-09-19T04:43:38.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.317+0000 7f82e9600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82e4103e30 0x7f82e419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:38.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.317+0000 7f82e9600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82e4103e30 0x7f82e419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:43924/0 (socket says 172.21.15.123:43924) 2024-09-19T04:43:38.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.317+0000 7f82e9600640 1 -- 172.21.15.123:0/4030064672 learned_addr learned my addr 172.21.15.123:0/4030064672 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:43:38.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.317+0000 7f82ea000640 1 --2- 172.21.15.123:0/4030064672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82e4108f60 0x7f82e419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:38.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.317+0000 7f82e8c00640 1 --2- 172.21.15.123:0/4030064672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82e41047e0 0x7f82e419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:38.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.317+0000 7f82e9600640 1 -- 172.21.15.123:0/4030064672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82e41047e0 msgr2=0x7f82e419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:38.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.317+0000 7f82e9600640 1 --2- 172.21.15.123:0/4030064672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82e41047e0 0x7f82e419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:38.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.317+0000 7f82e9600640 1 -- 172.21.15.123:0/4030064672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82e4108f60 msgr2=0x7f82e419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:38.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.317+0000 7f82e9600640 1 --2- 172.21.15.123:0/4030064672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82e4108f60 0x7f82e419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:38.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.317+0000 7f82e9600640 1 -- 172.21.15.123:0/4030064672 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e419f400 con 0x7f82e4103e30 2024-09-19T04:43:38.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.317+0000 7f82e8c00640 1 --2- 172.21.15.123:0/4030064672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82e41047e0 0x7f82e419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:43:38.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.317+0000 7f82ea000640 1 --2- 172.21.15.123:0/4030064672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82e4108f60 0x7f82e419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:43:38.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.317+0000 7f82e9600640 1 --2- 172.21.15.123:0/4030064672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82e4103e30 0x7f82e419a450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f82d8004770 tx=0x7f82d8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:38.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.318+0000 7f82e2000640 1 -- 172.21.15.123:0/4030064672 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d803f070 con 0x7f82e4103e30 2024-09-19T04:43:38.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.318+0000 7f82e2000640 1 -- 172.21.15.123:0/4030064672 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d8031ca0 con 0x7f82e4103e30 2024-09-19T04:43:38.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.318+0000 7f82eaa00640 1 -- 172.21.15.123:0/4030064672 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e419f690 con 0x7f82e4103e30 2024-09-19T04:43:38.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.318+0000 7f82e2000640 1 -- 172.21.15.123:0/4030064672 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d8035030 con 0x7f82e4103e30 2024-09-19T04:43:38.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.318+0000 7f82eaa00640 1 -- 172.21.15.123:0/4030064672 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e41a6fd0 con 0x7f82e4103e30 2024-09-19T04:43:38.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.319+0000 7f82eaa00640 1 -- 172.21.15.123:0/4030064672 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e4105310 con 0x7f82e4103e30 2024-09-19T04:43:38.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.321+0000 7f82e2000640 1 -- 172.21.15.123:0/4030064672 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f82d8042070 con 0x7f82e4103e30 2024-09-19T04:43:38.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.322+0000 7f82e2000640 1 --2- 172.21.15.123:0/4030064672 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f82cc0a3240 0x7f82cc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:38.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.322+0000 7f82e2000640 1 -- 172.21.15.123:0/4030064672 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f82d80efe50 con 0x7f82e4103e30 2024-09-19T04:43:38.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.322+0000 7f82e8c00640 1 --2- 172.21.15.123:0/4030064672 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f82cc0a3240 0x7f82cc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:38.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.323+0000 7f82e8c00640 1 --2- 172.21.15.123:0/4030064672 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f82cc0a3240 0x7f82cc0a5700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f82e40ffcf0 tx=0x7f82d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:38.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.325+0000 7f82e2000640 1 -- 172.21.15.123:0/4030064672 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f82d80bc920 con 0x7f82e4103e30 2024-09-19T04:43:38.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.501+0000 7f82eaa00640 1 -- 172.21.15.123:0/4030064672 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82e410c140 con 0x7f82e4103e30 2024-09-19T04:43:38.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.503+0000 7f82e2000640 1 -- 172.21.15.123:0/4030064672 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+18551 (secure 0 0 0) 0x7f82d80c0c20 con 0x7f82e4103e30 2024-09-19T04:43:38.505 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:43:38.505 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":78,"fsid":"911fa052-7640-11ef-bceb-c7b262605968","created":"2024-09-19T04:36:21.083333+0000","modified":"2024-09-19T04:43:34.826971+0000","last_up_change":"2024-09-19T04:43:33.818439+0000","last_in_change":"2024-09-19T04:43:10.148272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:39:33.856981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6803","nonce":3631596501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6805","nonce":3631596501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6809","nonce":3631596501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6807","nonce":3631596501}]},"public_addr":"172.21.15.123:6803/3631596501","cluster_addr":"172.21.15.123:6805/3631596501","heartbeat_back_addr":"172.21.15.123:6809/3631596501","heartbeat_front_addr":"172.21.15.123:6807/3631596501","state":["exists","up"]},{"osd":1,"uuid":"a526d4ce-586b-4d66-b720-e868df92fa24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6801","nonce":1259721614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6803","nonce":1259721614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6807","nonce":1259721614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6805","nonce":1259721614}]},"public_addr":"172.21.15.183:6801/1259721614","cluster_addr":"172.21.15.183:6803/1259721614","heartbeat_back_addr":"172.21.15.183:6807/1259721614","heartbeat_front_addr":"172.21.15.183:6805/1259721614","state":["exists","up"]},{"osd":2,"uuid":"3650601a-6259-4b2a-97df-1d47b09b0127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6801","nonce":2708151726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6803","nonce":2708151726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6807","nonce":2708151726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6805","nonce":2708151726}]},"public_addr":"172.21.15.193:6801/2708151726","cluster_addr":"172.21.15.193:6803/2708151726","heartbeat_back_addr":"172.21.15.193:6807/2708151726","heartbeat_front_addr":"172.21.15.193:6805/2708151726","state":["exists","up"]},{"osd":3,"uuid":"67c485ad-136e-4020-950d-9171a3c2de35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6811","nonce":3214806282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6813","nonce":3214806282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6817","nonce":3214806282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6815","nonce":3214806282}]},"public_addr":"172.21.15.123:6811/3214806282","cluster_addr":"172.21.15.123:6813/3214806282","heartbeat_back_addr":"172.21.15.123:6817/3214806282","heartbeat_front_addr":"172.21.15.123:6815/3214806282","state":["exists","up"]},{"osd":4,"uuid":"c9098540-06ab-45a8-9cf9-f90519cc852e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6809","nonce":667618665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6811","nonce":667618665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6815","nonce":667618665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6813","nonce":667618665}]},"public_addr":"172.21.15.183:6809/667618665","cluster_addr":"172.21.15.183:6811/667618665","heartbeat_back_addr":"172.21.15.183:6815/667618665","heartbeat_front_addr":"172.21.15.183:6813/667618665","state":["exists","up"]},{"osd":5,"uuid":"278a8dbd-6492-42d4-b489-1f832350bc6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6809","nonce":2292251639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6811","nonce":2292251639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6815","nonce":2292251639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6813","nonce":2292251639}]},"public_addr":"172.21.15.193:6809/2292251639","cluster_addr":"172.21.15.193:6811/2292251639","heartbeat_back_addr":"172.21.15.193:6815/2292251639","heartbeat_front_addr":"172.21.15.193:6813/2292251639","state":["exists","up"]},{"osd":6,"uuid":"0fda4e39-6adb-4941-a39b-a9a15019088c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6819","nonce":226007499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6821","nonce":226007499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6825","nonce":226007499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6823","nonce":226007499}]},"public_addr":"172.21.15.123:6819/226007499","cluster_addr":"172.21.15.123:6821/226007499","heartbeat_back_addr":"172.21.15.123:6825/226007499","heartbeat_front_addr":"172.21.15.123:6823/226007499","state":["exists","up"]},{"osd":7,"uuid":"5c0a021e-9f3b-4766-9686-0ef24b68cf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6817","nonce":3718518226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6819","nonce":3718518226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6823","nonce":3718518226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6821","nonce":3718518226}]},"public_addr":"172.21.15.183:6817/3718518226","cluster_addr":"172.21.15.183:6819/3718518226","heartbeat_back_addr":"172.21.15.183:6823/3718518226","heartbeat_front_addr":"172.21.15.183:6821/3718518226","state":["exists","up"]},{"osd":8,"uuid":"9b0e67e0-6305-4f81-9211-d41960099a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6817","nonce":3551404118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6819","nonce":3551404118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6823","nonce":3551404118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6821","nonce":3551404118}]},"public_addr":"172.21.15.193:6817/3551404118","cluster_addr":"172.21.15.193:6819/3551404118","heartbeat_back_addr":"172.21.15.193:6823/3551404118","heartbeat_front_addr":"172.21.15.193:6821/3551404118","state":["exists","up"]},{"osd":9,"uuid":"978ef3fa-c6ec-421f-b194-56d44813be5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6827","nonce":1439901149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6829","nonce":1439901149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6833","nonce":1439901149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6831","nonce":1439901149}]},"public_addr":"172.21.15.123:6827/1439901149","cluster_addr":"172.21.15.123:6829/1439901149","heartbeat_back_addr":"172.21.15.123:6833/1439901149","heartbeat_front_addr":"172.21.15.123:6831/1439901149","state":["exists","up"]},{"osd":10,"uuid":"5495e09b-7809-47b0-b7e0-b56c452f9f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6825","nonce":3355349245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6827","nonce":3355349245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6831","nonce":3355349245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6829","nonce":3355349245}]},"public_addr":"172.21.15.183:6825/3355349245","cluster_addr":"172.21.15.183:6827/3355349245","heartbeat_back_addr":"172.21.15.183:6831/3355349245","heartbeat_front_addr":"172.21.15.183:6829/3355349245","state":["exists","up"]},{"osd":11,"uuid":"03046fef-b16b-42f8-9a19-6c0b78cbfe7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6825","nonce":4291617038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6827","nonce":4291617038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6831","nonce":4291617038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6829","nonce":4291617038}]},"public_addr":"172.21.15.193:6825/4291617038","cluster_addr":"172.21.15.193:6827/4291617038","heartbeat_back_addr":"172.21.15.193:6831/4291617038","heartbeat_front_addr":"172.21.15.193:6829/4291617038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:38:39.142529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:05.336911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:30.310967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:59.655973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:25.890576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:50.778689+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:19.063731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:45.424269+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:10.562149+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:40.382368+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:06.297731+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:32.352172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/4040401634":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/3498196650":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6800/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:6800/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/471028951":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/1009496020":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/2000593686":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/2580747029":"2024-09-20T04:36:45.913060+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:43:38.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.505+0000 7f82eaa00640 1 -- 172.21.15.123:0/4030064672 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f82cc0a3240 msgr2=0x7f82cc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:38.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.505+0000 7f82eaa00640 1 --2- 172.21.15.123:0/4030064672 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f82cc0a3240 0x7f82cc0a5700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f82e40ffcf0 tx=0x7f82d0005f90 comp rx=0 tx=0).stop 2024-09-19T04:43:38.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.505+0000 7f82eaa00640 1 -- 172.21.15.123:0/4030064672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82e4103e30 msgr2=0x7f82e419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:38.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.505+0000 7f82eaa00640 1 --2- 172.21.15.123:0/4030064672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82e4103e30 0x7f82e419a450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f82d8004770 tx=0x7f82d8003c10 comp rx=0 tx=0).stop 2024-09-19T04:43:38.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.506+0000 7f82eaa00640 1 -- 172.21.15.123:0/4030064672 shutdown_connections 2024-09-19T04:43:38.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.506+0000 7f82eaa00640 1 --2- 172.21.15.123:0/4030064672 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f82cc0a3240 0x7f82cc0a5700 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:38.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.506+0000 7f82eaa00640 1 --2- 172.21.15.123:0/4030064672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82e4108f60 0x7f82e419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:38.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.506+0000 7f82eaa00640 1 --2- 172.21.15.123:0/4030064672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82e41047e0 0x7f82e419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:38.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.506+0000 7f82eaa00640 1 --2- 172.21.15.123:0/4030064672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82e4103e30 0x7f82e419a450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:38.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.506+0000 7f82eaa00640 1 -- 172.21.15.123:0/4030064672 >> 172.21.15.123:0/4030064672 conn(0x7f82e40ff8d0 msgr2=0x7f82e4100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:38.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.506+0000 7f82eaa00640 1 -- 172.21.15.123:0/4030064672 shutdown_connections 2024-09-19T04:43:38.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:38.506+0000 7f82eaa00640 1 -- 172.21.15.123:0/4030064672 wait complete. 2024-09-19T04:43:39.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:38 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/4030064672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:43:39.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:38 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/4030064672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:43:39.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:38 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/4030064672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:43:39.338 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:39:33.856981+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T04:43:39.338 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T04:43:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:43:40.115 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:39 smithi123 ceph-mon[30511]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.225+0000 7f2899e00640 1 -- 172.21.15.123:0/2496653241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2894101de0 msgr2=0x7f28941021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.225+0000 7f2899e00640 1 --2- 172.21.15.123:0/2496653241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2894101de0 0x7f28941021c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2884009900 tx=0x7f288402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:43:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.225+0000 7f2899e00640 1 -- 172.21.15.123:0/2496653241 shutdown_connections 2024-09-19T04:43:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.225+0000 7f2899e00640 1 --2- 172.21.15.123:0/2496653241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2894106f10 0x7f289410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.225+0000 7f2899e00640 1 --2- 172.21.15.123:0/2496653241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2894102790 0x7f28941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.225+0000 7f2899e00640 1 --2- 172.21.15.123:0/2496653241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2894101de0 0x7f28941021c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.225+0000 7f2899e00640 1 -- 172.21.15.123:0/2496653241 >> 172.21.15.123:0/2496653241 conn(0x7f28940fd8c0 msgr2=0x7f28940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.225+0000 7f2899e00640 1 -- 172.21.15.123:0/2496653241 shutdown_connections 2024-09-19T04:43:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.225+0000 7f2899e00640 1 -- 172.21.15.123:0/2496653241 wait complete. 2024-09-19T04:43:40.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.227+0000 7f2899e00640 1 Processor -- start 2024-09-19T04:43:40.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.227+0000 7f2899e00640 1 -- start start 2024-09-19T04:43:40.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.227+0000 7f2899e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2894101de0 0x7f289419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:40.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.227+0000 7f2899e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2894102790 0x7f289419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:40.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.227+0000 7f2899e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2894106f10 0x7f289419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:40.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.227+0000 7f2899e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2894111f70 con 0x7f2894102790 2024-09-19T04:43:40.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.227+0000 7f2899e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2894111df0 con 0x7f2894101de0 2024-09-19T04:43:40.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.227+0000 7f2899e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f28941120f0 con 0x7f2894106f10 2024-09-19T04:43:40.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.228+0000 7f2892000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2894102790 0x7f289419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:40.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.228+0000 7f2892000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2894102790 0x7f289419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60568/0 (socket says 172.21.15.123:60568) 2024-09-19T04:43:40.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.228+0000 7f2892000640 1 -- 172.21.15.123:0/613840351 learned_addr learned my addr 172.21.15.123:0/613840351 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:43:40.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.228+0000 7f2892a00640 1 --2- 172.21.15.123:0/613840351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2894101de0 0x7f289419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:40.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.228+0000 7f2893400640 1 --2- 172.21.15.123:0/613840351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2894106f10 0x7f289419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:40.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.228+0000 7f2892000640 1 -- 172.21.15.123:0/613840351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2894106f10 msgr2=0x7f289419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:40.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.228+0000 7f2892000640 1 --2- 172.21.15.123:0/613840351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2894106f10 0x7f289419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:40.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.228+0000 7f2892000640 1 -- 172.21.15.123:0/613840351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2894101de0 msgr2=0x7f289419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:40.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.228+0000 7f2892000640 1 --2- 172.21.15.123:0/613840351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2894101de0 0x7f289419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:40.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.228+0000 7f2892000640 1 -- 172.21.15.123:0/613840351 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289419f5e0 con 0x7f2894102790 2024-09-19T04:43:40.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.228+0000 7f2892000640 1 --2- 172.21.15.123:0/613840351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2894102790 0x7f289419ab70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f288000d950 tx=0x7f288000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:40.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.229+0000 7f2898a00640 1 -- 172.21.15.123:0/613840351 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880017030 con 0x7f2894102790 2024-09-19T04:43:40.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.229+0000 7f2898a00640 1 -- 172.21.15.123:0/613840351 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2880004020 con 0x7f2894102790 2024-09-19T04:43:40.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.229+0000 7f2898a00640 1 -- 172.21.15.123:0/613840351 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28800047a0 con 0x7f2894102790 2024-09-19T04:43:40.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.229+0000 7f2899e00640 1 -- 172.21.15.123:0/613840351 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289419f8d0 con 0x7f2894102790 2024-09-19T04:43:40.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.229+0000 7f2899e00640 1 -- 172.21.15.123:0/613840351 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28941a71b0 con 0x7f2894102790 2024-09-19T04:43:40.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.230+0000 7f2899e00640 1 -- 172.21.15.123:0/613840351 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28941021c0 con 0x7f2894102790 2024-09-19T04:43:40.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.232+0000 7f2898a00640 1 -- 172.21.15.123:0/613840351 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f28800041d0 con 0x7f2894102790 2024-09-19T04:43:40.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.234+0000 7f2898a00640 1 --2- 172.21.15.123:0/613840351 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f28780a32c0 0x7f28780a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:40.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.234+0000 7f2898a00640 1 -- 172.21.15.123:0/613840351 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f28800cedd0 con 0x7f2894102790 2024-09-19T04:43:40.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.234+0000 7f2892a00640 1 --2- 172.21.15.123:0/613840351 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f28780a32c0 0x7f28780a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:40.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.235+0000 7f2892a00640 1 --2- 172.21.15.123:0/613840351 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f28780a32c0 0x7f28780a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2884009230 tx=0x7f28840058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:40.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.238+0000 7f2898a00640 1 -- 172.21.15.123:0/613840351 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f288009bef0 con 0x7f2894102790 2024-09-19T04:43:40.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:39 smithi193 ceph-mon[35592]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:40.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:39 smithi183 ceph-mon[35333]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:40.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.412+0000 7f2899e00640 1 -- 172.21.15.123:0/613840351 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f28941a7470 con 0x7f2894102790 2024-09-19T04:43:40.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.413+0000 7f2898a00640 1 -- 172.21.15.123:0/613840351 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f288009b7f0 con 0x7f2894102790 2024-09-19T04:43:40.414 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-09-19T04:43:40.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.416+0000 7f2899e00640 1 -- 172.21.15.123:0/613840351 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f28780a32c0 msgr2=0x7f28780a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:40.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.416+0000 7f2899e00640 1 --2- 172.21.15.123:0/613840351 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f28780a32c0 0x7f28780a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2884009230 tx=0x7f28840058d0 comp rx=0 tx=0).stop 2024-09-19T04:43:40.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.416+0000 7f2899e00640 1 -- 172.21.15.123:0/613840351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2894102790 msgr2=0x7f289419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:40.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.416+0000 7f2899e00640 1 --2- 172.21.15.123:0/613840351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2894102790 0x7f289419ab70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f288000d950 tx=0x7f288000de10 comp rx=0 tx=0).stop 2024-09-19T04:43:40.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.416+0000 7f2899e00640 1 -- 172.21.15.123:0/613840351 shutdown_connections 2024-09-19T04:43:40.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.416+0000 7f2899e00640 1 --2- 172.21.15.123:0/613840351 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f28780a32c0 0x7f28780a5780 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:40.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.416+0000 7f2899e00640 1 --2- 172.21.15.123:0/613840351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2894106f10 0x7f289419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:40.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.416+0000 7f2899e00640 1 --2- 172.21.15.123:0/613840351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2894102790 0x7f289419ab70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:40.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.416+0000 7f2899e00640 1 --2- 172.21.15.123:0/613840351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2894101de0 0x7f289419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:40.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.416+0000 7f2899e00640 1 -- 172.21.15.123:0/613840351 >> 172.21.15.123:0/613840351 conn(0x7f28940fd8c0 msgr2=0x7f28940feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:40.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.416+0000 7f2899e00640 1 -- 172.21.15.123:0/613840351 shutdown_connections 2024-09-19T04:43:40.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:40.416+0000 7f2899e00640 1 -- 172.21.15.123:0/613840351 wait complete. 2024-09-19T04:43:41.153 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:40 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/613840351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:43:41.191 INFO:tasks.cephadm:Adding mds.a on smithi123 2024-09-19T04:43:41.191 INFO:tasks.cephadm:Adding mds.d on smithi123 2024-09-19T04:43:41.191 INFO:tasks.cephadm:Adding mds.g on smithi123 2024-09-19T04:43:41.191 INFO:tasks.cephadm:Adding mds.j on smithi123 2024-09-19T04:43:41.191 INFO:tasks.cephadm:Adding mds.b on smithi183 2024-09-19T04:43:41.191 INFO:tasks.cephadm:Adding mds.e on smithi183 2024-09-19T04:43:41.191 INFO:tasks.cephadm:Adding mds.h on smithi183 2024-09-19T04:43:41.192 INFO:tasks.cephadm:Adding mds.k on smithi183 2024-09-19T04:43:41.192 INFO:tasks.cephadm:Adding mds.c on smithi193 2024-09-19T04:43:41.192 INFO:tasks.cephadm:Adding mds.f on smithi193 2024-09-19T04:43:41.192 INFO:tasks.cephadm:Adding mds.i on smithi193 2024-09-19T04:43:41.192 INFO:tasks.cephadm:Adding mds.l on smithi193 2024-09-19T04:43:41.192 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l' 2024-09-19T04:43:41.221 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:40 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/613840351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:43:41.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:40 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/613840351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:43:41.448 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:43:42.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.984+0000 7f4d16d98640 1 -- 172.21.15.193:0/3505633209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d10105010 msgr2=0x7f4d10107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:42.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.984+0000 7f4d16d98640 1 --2- 172.21.15.193:0/3505633209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d10105010 0x7f4d10107400 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009900 tx=0x7f4cfc02a410 comp rx=0 tx=0).stop 2024-09-19T04:43:42.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.984+0000 7f4d16d98640 1 -- 172.21.15.193:0/3505633209 shutdown_connections 2024-09-19T04:43:42.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.984+0000 7f4d16d98640 1 --2- 172.21.15.193:0/3505633209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d10107940 0x7f4d10109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:42.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.984+0000 7f4d16d98640 1 --2- 172.21.15.193:0/3505633209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d10105010 0x7f4d10107400 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:42.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.984+0000 7f4d16d98640 1 --2- 172.21.15.193:0/3505633209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d101026e0 0x7f4d10104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:42.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.984+0000 7f4d16d98640 1 -- 172.21.15.193:0/3505633209 >> 172.21.15.193:0/3505633209 conn(0x7f4d100fc280 msgr2=0x7f4d100fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:42.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.984+0000 7f4d16d98640 1 -- 172.21.15.193:0/3505633209 shutdown_connections 2024-09-19T04:43:42.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.984+0000 7f4d16d98640 1 -- 172.21.15.193:0/3505633209 wait complete. 2024-09-19T04:43:42.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.984+0000 7f4d16d98640 1 Processor -- start 2024-09-19T04:43:42.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.984+0000 7f4d16d98640 1 -- start start 2024-09-19T04:43:42.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d16d98640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d101026e0 0x7f4d10197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:42.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d16d98640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d10105010 0x7f4d10198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:42.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d16d98640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d10107940 0x7f4d1019f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:42.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d16d98640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1010d8b0 con 0x7f4d10105010 2024-09-19T04:43:42.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d16d98640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1010d730 con 0x7f4d101026e0 2024-09-19T04:43:42.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d16d98640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1010da30 con 0x7f4d10107940 2024-09-19T04:43:42.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d15335640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d10107940 0x7f4d1019f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:42.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d15335640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d10107940 0x7f4d1019f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:56454/0 (socket says 172.21.15.193:56454) 2024-09-19T04:43:42.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d15335640 1 -- 172.21.15.193:0/2399248811 learned_addr learned my addr 172.21.15.193:0/2399248811 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-19T04:43:42.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d0ffff640 1 --2- 172.21.15.193:0/2399248811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d10105010 0x7f4d10198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:42.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d15335640 1 -- 172.21.15.193:0/2399248811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d101026e0 msgr2=0x7f4d10197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:42.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d14b34640 1 --2- 172.21.15.193:0/2399248811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d101026e0 0x7f4d10197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:42.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d15335640 1 --2- 172.21.15.193:0/2399248811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d101026e0 0x7f4d10197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:42.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d15335640 1 -- 172.21.15.193:0/2399248811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d10105010 msgr2=0x7f4d10198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:42.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d15335640 1 --2- 172.21.15.193:0/2399248811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d10105010 0x7f4d10198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:42.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d15335640 1 -- 172.21.15.193:0/2399248811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1019fbf0 con 0x7f4d10107940 2024-09-19T04:43:42.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d15335640 1 --2- 172.21.15.193:0/2399248811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d10107940 0x7f4d1019f580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800eaa0 tx=0x7f4d0800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:42.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d0dffb640 1 -- 172.21.15.193:0/2399248811 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0800cdb0 con 0x7f4d10107940 2024-09-19T04:43:42.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d0dffb640 1 -- 172.21.15.193:0/2399248811 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d08005020 con 0x7f4d10107940 2024-09-19T04:43:42.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d16d98640 1 -- 172.21.15.193:0/2399248811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1019fee0 con 0x7f4d10107940 2024-09-19T04:43:42.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.985+0000 7f4d0dffb640 1 -- 172.21.15.193:0/2399248811 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d080116b0 con 0x7f4d10107940 2024-09-19T04:43:42.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.986+0000 7f4d16d98640 1 -- 172.21.15.193:0/2399248811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d101a03c0 con 0x7f4d10107940 2024-09-19T04:43:42.012 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.987+0000 7f4d16d98640 1 -- 172.21.15.193:0/2399248811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d100fdd70 con 0x7f4d10107940 2024-09-19T04:43:42.012 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.987+0000 7f4d0dffb640 1 -- 172.21.15.193:0/2399248811 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f4d08005560 con 0x7f4d10107940 2024-09-19T04:43:42.012 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.988+0000 7f4d0dffb640 1 --2- 172.21.15.193:0/2399248811 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4d000a3270 0x7f4d000a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:42.012 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.988+0000 7f4d0dffb640 1 -- 172.21.15.193:0/2399248811 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f4d080cf750 con 0x7f4d10107940 2024-09-19T04:43:42.012 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.988+0000 7f4d14b34640 1 --2- 172.21.15.193:0/2399248811 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4d000a3270 0x7f4d000a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:42.012 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.989+0000 7f4d14b34640 1 --2- 172.21.15.193:0/2399248811 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4d000a3270 0x7f4d000a5730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400a810 tx=0x7f4d04008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:42.012 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:41.992+0000 7f4d0dffb640 1 -- 172.21.15.193:0/2399248811 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4d08016030 con 0x7f4d10107940 2024-09-19T04:43:42.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:41 smithi193 ceph-mon[35592]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:42.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:41 smithi123 ceph-mon[30511]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:42.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:42.177+0000 7f4d16d98640 1 -- 172.21.15.193:0/2399248811 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}) -- 0x7f4d10062730 con 0x7f4d000a3270 2024-09-19T04:43:42.186 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:42.185+0000 7f4d0dffb640 1 -- 172.21.15.193:0/2399248811 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f4d10062730 con 0x7f4d000a3270 2024-09-19T04:43:42.187 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mds.all update... 2024-09-19T04:43:42.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:42.187+0000 7f4d16d98640 1 -- 172.21.15.193:0/2399248811 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4d000a3270 msgr2=0x7f4d000a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:42.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:42.187+0000 7f4d16d98640 1 --2- 172.21.15.193:0/2399248811 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4d000a3270 0x7f4d000a5730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400a810 tx=0x7f4d04008040 comp rx=0 tx=0).stop 2024-09-19T04:43:42.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:42.187+0000 7f4d16d98640 1 -- 172.21.15.193:0/2399248811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d10107940 msgr2=0x7f4d1019f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:42.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:42.187+0000 7f4d16d98640 1 --2- 172.21.15.193:0/2399248811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d10107940 0x7f4d1019f580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800eaa0 tx=0x7f4d0800ef60 comp rx=0 tx=0).stop 2024-09-19T04:43:42.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:42.187+0000 7f4d16d98640 1 -- 172.21.15.193:0/2399248811 shutdown_connections 2024-09-19T04:43:42.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:42.187+0000 7f4d16d98640 1 --2- 172.21.15.193:0/2399248811 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4d000a3270 0x7f4d000a5730 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:42.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:42.187+0000 7f4d16d98640 1 --2- 172.21.15.193:0/2399248811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d10107940 0x7f4d1019f580 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:42.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:42.187+0000 7f4d16d98640 1 --2- 172.21.15.193:0/2399248811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d10105010 0x7f4d10198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:42.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:42.187+0000 7f4d16d98640 1 --2- 172.21.15.193:0/2399248811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d101026e0 0x7f4d10197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:42.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:42.187+0000 7f4d16d98640 1 -- 172.21.15.193:0/2399248811 >> 172.21.15.193:0/2399248811 conn(0x7f4d100fc280 msgr2=0x7f4d10103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:42.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:42.188+0000 7f4d16d98640 1 -- 172.21.15.193:0/2399248811 shutdown_connections 2024-09-19T04:43:42.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-19T04:43:42.188+0000 7f4d16d98640 1 -- 172.21.15.193:0/2399248811 wait complete. 2024-09-19T04:43:42.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:41 smithi183 ceph-mon[35333]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:42.907 DEBUG:teuthology.orchestra.run.smithi123:mds.a> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.a.service 2024-09-19T04:43:42.912 DEBUG:teuthology.orchestra.run.smithi123:mds.d> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.d.service 2024-09-19T04:43:42.956 DEBUG:teuthology.orchestra.run.smithi123:mds.g> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.g.service 2024-09-19T04:43:43.001 DEBUG:teuthology.orchestra.run.smithi123:mds.j> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.j.service 2024-09-19T04:43:43.045 DEBUG:teuthology.orchestra.run.smithi183:mds.b> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.b.service 2024-09-19T04:43:43.048 DEBUG:teuthology.orchestra.run.smithi183:mds.e> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.e.service 2024-09-19T04:43:43.091 DEBUG:teuthology.orchestra.run.smithi183:mds.h> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.h.service 2024-09-19T04:43:43.133 DEBUG:teuthology.orchestra.run.smithi183:mds.k> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.k.service 2024-09-19T04:43:43.176 DEBUG:teuthology.orchestra.run.smithi193:mds.c> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.c.service 2024-09-19T04:43:43.179 DEBUG:teuthology.orchestra.run.smithi193:mds.f> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.f.service 2024-09-19T04:43:43.222 DEBUG:teuthology.orchestra.run.smithi193:mds.i> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.i.service 2024-09-19T04:43:43.264 DEBUG:teuthology.orchestra.run.smithi193:mds.l> sudo journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.l.service 2024-09-19T04:43:43.308 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-19T04:43:43.308 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:43:43.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:43 smithi183 ceph-mon[35333]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:43:43.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:43 smithi183 ceph-mon[35333]: Saving service mds.all spec with placement smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-09-19T04:43:43.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:43.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:43:43.351 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:43.351 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:43:43.351 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:43.351 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:43:43.351 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:43:43.351 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:43 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:43.351 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:43 smithi183 ceph-mon[35333]: Deploying daemon mds.l on smithi193 2024-09-19T04:43:43.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:43 smithi123 ceph-mon[30511]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:43:43.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:43 smithi123 ceph-mon[30511]: Saving service mds.all spec with placement smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-09-19T04:43:43.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:43.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:43:43.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:43.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:43:43.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:43.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:43:43.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:43:43.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:43 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:43.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:43 smithi123 ceph-mon[30511]: Deploying daemon mds.l on smithi193 2024-09-19T04:43:43.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:43 smithi193 ceph-mon[35592]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:43:43.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:43 smithi193 ceph-mon[35592]: Saving service mds.all spec with placement smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-09-19T04:43:43.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:43.485 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:43:43.485 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:43.485 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:43:43.485 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:43.485 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:43:43.485 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:43:43.485 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:43 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:43.485 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:43 smithi193 ceph-mon[35592]: Deploying daemon mds.l on smithi193 2024-09-19T04:43:43.671 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:43:44.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.215+0000 7f769aa00640 1 -- 172.21.15.123:0/893559803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76941019c0 msgr2=0x7f7694101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:44.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.215+0000 7f769aa00640 1 --2- 172.21.15.123:0/893559803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76941019c0 0x7f7694101e20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7680009930 tx=0x7f768002a420 comp rx=0 tx=0).stop 2024-09-19T04:43:44.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.215+0000 7f769aa00640 1 -- 172.21.15.123:0/893559803 shutdown_connections 2024-09-19T04:43:44.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.215+0000 7f769aa00640 1 --2- 172.21.15.123:0/893559803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7694102360 0x7f769410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:44.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.215+0000 7f769aa00640 1 --2- 172.21.15.123:0/893559803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76941019c0 0x7f7694101e20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:44.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.215+0000 7f769aa00640 1 --2- 172.21.15.123:0/893559803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76941079c0 0x7f7694107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:44.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.215+0000 7f769aa00640 1 -- 172.21.15.123:0/893559803 >> 172.21.15.123:0/893559803 conn(0x7f76940fd740 msgr2=0x7f76940ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:44.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.215+0000 7f769aa00640 1 -- 172.21.15.123:0/893559803 shutdown_connections 2024-09-19T04:43:44.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.215+0000 7f769aa00640 1 -- 172.21.15.123:0/893559803 wait complete. 2024-09-19T04:43:44.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.216+0000 7f769aa00640 1 Processor -- start 2024-09-19T04:43:44.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.216+0000 7f769aa00640 1 -- start start 2024-09-19T04:43:44.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.217+0000 7f769aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76941019c0 0x7f76941982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:44.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.217+0000 7f769aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7694102360 0x7f7694198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:44.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.217+0000 7f769aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76941079c0 0x7f769419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:44.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.217+0000 7f769aa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7694111dd0 con 0x7f7694102360 2024-09-19T04:43:44.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.217+0000 7f769aa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7694111c50 con 0x7f76941079c0 2024-09-19T04:43:44.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.217+0000 7f769aa00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7694111f50 con 0x7f76941019c0 2024-09-19T04:43:44.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.217+0000 7f7699600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76941019c0 0x7f76941982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:44.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.217+0000 7f7699600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76941019c0 0x7f76941982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:40532/0 (socket says 172.21.15.123:40532) 2024-09-19T04:43:44.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.217+0000 7f7699600640 1 -- 172.21.15.123:0/1100081976 learned_addr learned my addr 172.21.15.123:0/1100081976 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:43:44.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.217+0000 7f7698c00640 1 --2- 172.21.15.123:0/1100081976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7694102360 0x7f7694198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:44.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.217+0000 7f769a000640 1 --2- 172.21.15.123:0/1100081976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76941079c0 0x7f769419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:44.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.217+0000 7f7699600640 1 -- 172.21.15.123:0/1100081976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76941079c0 msgr2=0x7f769419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:44.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.217+0000 7f7699600640 1 --2- 172.21.15.123:0/1100081976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76941079c0 0x7f769419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:44.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.217+0000 7f7699600640 1 -- 172.21.15.123:0/1100081976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7694102360 msgr2=0x7f7694198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:44.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.217+0000 7f7699600640 1 --2- 172.21.15.123:0/1100081976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7694102360 0x7f7694198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:44.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.217+0000 7f7699600640 1 -- 172.21.15.123:0/1100081976 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769419d270 con 0x7f76941019c0 2024-09-19T04:43:44.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.217+0000 7f769a000640 1 --2- 172.21.15.123:0/1100081976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76941079c0 0x7f769419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:43:44.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.218+0000 7f7699600640 1 --2- 172.21.15.123:0/1100081976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76941019c0 0x7f76941982c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7688009de0 tx=0x7f768800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:44.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.218+0000 7f7692000640 1 -- 172.21.15.123:0/1100081976 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768801f070 con 0x7f76941019c0 2024-09-19T04:43:44.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.218+0000 7f769aa00640 1 -- 172.21.15.123:0/1100081976 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769419d560 con 0x7f76941019c0 2024-09-19T04:43:44.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.218+0000 7f7692000640 1 -- 172.21.15.123:0/1100081976 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7688011750 con 0x7f76941019c0 2024-09-19T04:43:44.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.218+0000 7f7692000640 1 -- 172.21.15.123:0/1100081976 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768801a850 con 0x7f76941019c0 2024-09-19T04:43:44.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.218+0000 7f769aa00640 1 -- 172.21.15.123:0/1100081976 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7694074420 con 0x7f76941019c0 2024-09-19T04:43:44.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.219+0000 7f769aa00640 1 -- 172.21.15.123:0/1100081976 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7694101e20 con 0x7f76941019c0 2024-09-19T04:43:44.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.221+0000 7f7692000640 1 -- 172.21.15.123:0/1100081976 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f7688022070 con 0x7f76941019c0 2024-09-19T04:43:44.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.222+0000 7f7692000640 1 --2- 172.21.15.123:0/1100081976 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f767c0a32c0 0x7f767c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:44.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.222+0000 7f7692000640 1 -- 172.21.15.123:0/1100081976 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f76880d3170 con 0x7f76941019c0 2024-09-19T04:43:44.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.223+0000 7f7698c00640 1 --2- 172.21.15.123:0/1100081976 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f767c0a32c0 0x7f767c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:44.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.223+0000 7f7698c00640 1 --2- 172.21.15.123:0/1100081976 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f767c0a32c0 0x7f767c0a5780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7680006010 tx=0x7f7680000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:44.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.230+0000 7f7692000640 1 -- 172.21.15.123:0/1100081976 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f768809fc40 con 0x7f76941019c0 2024-09-19T04:43:44.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:44 smithi193 ceph-mon[35592]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:44.426 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:44 smithi123 ceph-mon[30511]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:44.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.442+0000 7f769aa00640 1 -- 172.21.15.123:0/1100081976 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7694199620 con 0x7f76941019c0 2024-09-19T04:43:44.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.442+0000 7f7692000640 1 -- 172.21.15.123:0/1100081976 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f76880a3f40 con 0x7f76941019c0 2024-09-19T04:43:44.444 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:43:44.444 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"btime":"2024-09-19T04:36:21:083029+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-19T04:43:44.444 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 1 2024-09-19T04:43:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.444+0000 7f769aa00640 1 -- 172.21.15.123:0/1100081976 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f767c0a32c0 msgr2=0x7f767c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.444+0000 7f769aa00640 1 --2- 172.21.15.123:0/1100081976 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f767c0a32c0 0x7f767c0a5780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7680006010 tx=0x7f7680000bc0 comp rx=0 tx=0).stop 2024-09-19T04:43:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.444+0000 7f769aa00640 1 -- 172.21.15.123:0/1100081976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76941019c0 msgr2=0x7f76941982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.444+0000 7f769aa00640 1 --2- 172.21.15.123:0/1100081976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76941019c0 0x7f76941982c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7688009de0 tx=0x7f768800c6a0 comp rx=0 tx=0).stop 2024-09-19T04:43:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.445+0000 7f769aa00640 1 -- 172.21.15.123:0/1100081976 shutdown_connections 2024-09-19T04:43:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.445+0000 7f769aa00640 1 --2- 172.21.15.123:0/1100081976 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f767c0a32c0 0x7f767c0a5780 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.445+0000 7f769aa00640 1 --2- 172.21.15.123:0/1100081976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76941079c0 0x7f769419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:44.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.445+0000 7f769aa00640 1 --2- 172.21.15.123:0/1100081976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7694102360 0x7f7694198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:44.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.445+0000 7f769aa00640 1 --2- 172.21.15.123:0/1100081976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76941019c0 0x7f76941982c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:44.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.445+0000 7f769aa00640 1 -- 172.21.15.123:0/1100081976 >> 172.21.15.123:0/1100081976 conn(0x7f76940fd740 msgr2=0x7f76940fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:44.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.445+0000 7f769aa00640 1 -- 172.21.15.123:0/1100081976 shutdown_connections 2024-09-19T04:43:44.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:44.445+0000 7f769aa00640 1 -- 172.21.15.123:0/1100081976 wait complete. 2024-09-19T04:43:44.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:44 smithi183 ceph-mon[35333]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:44.741 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 04:43:44 smithi193 systemd[1]: Starting Ceph mds.l for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:43:45.102 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 04:43:44 smithi193 podman[65049]: 2024-09-19 04:43:44.709654401 +0000 UTC m=+0.015679670 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:43:45.103 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 04:43:44 smithi193 podman[65049]: 2024-09-19 04:43:44.832592109 +0000 UTC m=+0.138617371 container create c44754603a223b4e9c2e38c6a5a0800a9c26a0fbc61b3ae888f0eddbbb767ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T04:43:45.103 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 04:43:45 smithi193 podman[65049]: 2024-09-19 04:43:45.102108923 +0000 UTC m=+0.408134190 container init c44754603a223b4e9c2e38c6a5a0800a9c26a0fbc61b3ae888f0eddbbb767ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4) 2024-09-19T04:43:45.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:45 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1100081976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:43:45.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:45 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.193:6832/964700250,v1:172.21.15.193:6833/964700250] up:boot 2024-09-19T04:43:45.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:45 smithi193 ceph-mon[35592]: fsmap 1 up:standby 2024-09-19T04:43:45.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:45 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:43:45.491 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 04:43:45 smithi193 podman[65049]: 2024-09-19 04:43:45.105061245 +0000 UTC m=+0.411086509 container start c44754603a223b4e9c2e38c6a5a0800a9c26a0fbc61b3ae888f0eddbbb767ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T04:43:45.491 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 04:43:45 smithi193 ceph-mds[65068]: -- 172.21.15.193:0/233171469 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5567b4b9f860 con 0x5567b4b7c800 2024-09-19T04:43:45.491 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 04:43:45 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-l[65064]: starting mds.l at 2024-09-19T04:43:45.491 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 04:43:45 smithi193 bash[65049]: c44754603a223b4e9c2e38c6a5a0800a9c26a0fbc61b3ae888f0eddbbb767ead 2024-09-19T04:43:45.491 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 04:43:45 smithi193 systemd[1]: Started Ceph mds.l for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:43:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:45 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1100081976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:43:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:45 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.193:6832/964700250,v1:172.21.15.193:6833/964700250] up:boot 2024-09-19T04:43:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:45 smithi183 ceph-mon[35333]: fsmap 1 up:standby 2024-09-19T04:43:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:45 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:43:45.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:45 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1100081976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:43:45.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:45 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.193:6832/964700250,v1:172.21.15.193:6833/964700250] up:boot 2024-09-19T04:43:45.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:45 smithi123 ceph-mon[30511]: fsmap 1 up:standby 2024-09-19T04:43:45.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:45 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:43:46.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:46 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:46.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:46 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:46.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:46 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:46.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:46 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:43:46.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:46 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:43:46.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:46 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:46.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:46 smithi183 ceph-mon[35333]: Deploying daemon mds.k on smithi183 2024-09-19T04:43:46.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:46 smithi183 ceph-mon[35333]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:46.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:46 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:46.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:46 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:46.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:46 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:46.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:46 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:43:46.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:46 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:43:46.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:46 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:46.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:46 smithi123 ceph-mon[30511]: Deploying daemon mds.k on smithi183 2024-09-19T04:43:46.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:46 smithi123 ceph-mon[30511]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:46.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:46 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:46.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:46 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:46.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:46 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:46.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:46 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:43:46.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:46 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:43:46.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:46 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:46.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:46 smithi193 ceph-mon[35592]: Deploying daemon mds.k on smithi183 2024-09-19T04:43:46.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:46 smithi193 ceph-mon[35592]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:47.299 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:43:47.489 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:43:47 smithi183 systemd[1]: Starting Ceph mds.k for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:43:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:43:47.812 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:43:47 smithi183 podman[64272]: 2024-09-19 04:43:47.390909491 +0000 UTC m=+0.015772472 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:43:47.813 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:43:47 smithi183 podman[64272]: 2024-09-19 04:43:47.498381007 +0000 UTC m=+0.123243984 container create a5492af672c9e3906481622d003c142b60ca68889d0979b85f6548b34ba0c0b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T04:43:48.100 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:43:47 smithi183 podman[64272]: 2024-09-19 04:43:47.809382686 +0000 UTC m=+0.434245663 container init a5492af672c9e3906481622d003c142b60ca68889d0979b85f6548b34ba0c0b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-19T04:43:48.100 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:43:47 smithi183 podman[64272]: 2024-09-19 04:43:47.812597659 +0000 UTC m=+0.437460646 container start a5492af672c9e3906481622d003c142b60ca68889d0979b85f6548b34ba0c0b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-k, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-19T04:43:48.100 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:43:47 smithi183 ceph-mds[64292]: -- 172.21.15.183:0/4147318795 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a983299860 con 0x55a983277400 2024-09-19T04:43:48.100 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:43:47 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-k[64288]: starting mds.k at 2024-09-19T04:43:48.100 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:43:47 smithi183 bash[64272]: a5492af672c9e3906481622d003c142b60ca68889d0979b85f6548b34ba0c0b8 2024-09-19T04:43:48.100 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:43:47 smithi183 systemd[1]: Started Ceph mds.k for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:43:48.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.303+0000 7f3c7d600640 1 -- 172.21.15.123:0/637544588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c78102360 msgr2=0x7f3c7810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:48.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.303+0000 7f3c7d600640 1 --2- 172.21.15.123:0/637544588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c78102360 0x7f3c7810e890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3c70009be0 tx=0x7f3c7002a6c0 comp rx=0 tx=0).stop 2024-09-19T04:43:48.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.303+0000 7f3c7d600640 1 -- 172.21.15.123:0/637544588 shutdown_connections 2024-09-19T04:43:48.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.303+0000 7f3c7d600640 1 --2- 172.21.15.123:0/637544588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c78102360 0x7f3c7810e890 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:48.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.303+0000 7f3c7d600640 1 --2- 172.21.15.123:0/637544588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c781019c0 0x7f3c78101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:48.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.303+0000 7f3c7d600640 1 --2- 172.21.15.123:0/637544588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c781079c0 0x7f3c78107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:48.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.303+0000 7f3c7d600640 1 -- 172.21.15.123:0/637544588 >> 172.21.15.123:0/637544588 conn(0x7f3c780fd740 msgr2=0x7f3c780ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:48.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.303+0000 7f3c7d600640 1 -- 172.21.15.123:0/637544588 shutdown_connections 2024-09-19T04:43:48.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.303+0000 7f3c7d600640 1 -- 172.21.15.123:0/637544588 wait complete. 2024-09-19T04:43:48.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.304+0000 7f3c7d600640 1 Processor -- start 2024-09-19T04:43:48.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.304+0000 7f3c7d600640 1 -- start start 2024-09-19T04:43:48.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.305+0000 7f3c7d600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c781019c0 0x7f3c780744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:48.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.305+0000 7f3c7d600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c78102360 0x7f3c78074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:48.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.305+0000 7f3c7d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c781079c0 0x7f3c78074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:48.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.305+0000 7f3c7d600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3c78111de0 con 0x7f3c781079c0 2024-09-19T04:43:48.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.305+0000 7f3c7d600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3c78111c60 con 0x7f3c78102360 2024-09-19T04:43:48.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.305+0000 7f3c7d600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3c78111f60 con 0x7f3c781019c0 2024-09-19T04:43:48.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.305+0000 7f3c77400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c78102360 0x7f3c78074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:48.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.305+0000 7f3c7cc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c781079c0 0x7f3c78074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:48.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.305+0000 7f3c77400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c78102360 0x7f3c78074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:57630/0 (socket says 172.21.15.123:57630) 2024-09-19T04:43:48.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.305+0000 7f3c77400640 1 -- 172.21.15.123:0/2830875385 learned_addr learned my addr 172.21.15.123:0/2830875385 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:43:48.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.305+0000 7f3c77e00640 1 --2- 172.21.15.123:0/2830875385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c781019c0 0x7f3c780744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:48.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.305+0000 7f3c77400640 1 -- 172.21.15.123:0/2830875385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c781019c0 msgr2=0x7f3c780744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:48.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.305+0000 7f3c77400640 1 --2- 172.21.15.123:0/2830875385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c781019c0 0x7f3c780744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:48.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.305+0000 7f3c77400640 1 -- 172.21.15.123:0/2830875385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c781079c0 msgr2=0x7f3c78074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:48.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.305+0000 7f3c77400640 1 --2- 172.21.15.123:0/2830875385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c781079c0 0x7f3c78074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:48.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.305+0000 7f3c77400640 1 -- 172.21.15.123:0/2830875385 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c781aac30 con 0x7f3c78102360 2024-09-19T04:43:48.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.306+0000 7f3c77400640 1 --2- 172.21.15.123:0/2830875385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c78102360 0x7f3c78074a30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400b4a0 tx=0x7f3c6400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:48.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.306+0000 7f3c74c00640 1 -- 172.21.15.123:0/2830875385 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c64014070 con 0x7f3c78102360 2024-09-19T04:43:48.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.306+0000 7f3c7d600640 1 -- 172.21.15.123:0/2830875385 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c781aae60 con 0x7f3c78102360 2024-09-19T04:43:48.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.306+0000 7f3c74c00640 1 -- 172.21.15.123:0/2830875385 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c64010af0 con 0x7f3c78102360 2024-09-19T04:43:48.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.306+0000 7f3c74c00640 1 -- 172.21.15.123:0/2830875385 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c64017d10 con 0x7f3c78102360 2024-09-19T04:43:48.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.306+0000 7f3c7d600640 1 -- 172.21.15.123:0/2830875385 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c781ab310 con 0x7f3c78102360 2024-09-19T04:43:48.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.307+0000 7f3c7d600640 1 -- 172.21.15.123:0/2830875385 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c78103100 con 0x7f3c78102360 2024-09-19T04:43:48.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.309+0000 7f3c74c00640 1 -- 172.21.15.123:0/2830875385 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f3c64017070 con 0x7f3c78102360 2024-09-19T04:43:48.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.310+0000 7f3c74c00640 1 --2- 172.21.15.123:0/2830875385 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3c5c0a32c0 0x7f3c5c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:48.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.311+0000 7f3c74c00640 1 -- 172.21.15.123:0/2830875385 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f3c640ce760 con 0x7f3c78102360 2024-09-19T04:43:48.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.311+0000 7f3c77e00640 1 --2- 172.21.15.123:0/2830875385 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3c5c0a32c0 0x7f3c5c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:48.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.311+0000 7f3c77e00640 1 --2- 172.21.15.123:0/2830875385 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3c5c0a32c0 0x7f3c5c0a5780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009530 tx=0x7f3c68009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:48.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.315+0000 7f3c74c00640 1 -- 172.21.15.123:0/2830875385 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3c6409b230 con 0x7f3c78102360 2024-09-19T04:43:48.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.551+0000 7f3c7d600640 1 -- 172.21.15.123:0/2830875385 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c781ab600 con 0x7f3c78102360 2024-09-19T04:43:48.552 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:48 smithi123 ceph-mon[30511]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:48.553 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:48 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:48.553 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:48 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:48.553 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:48 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:48.553 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:48 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:43:48.553 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:48 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:43:48.553 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:48 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:48.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.551+0000 7f3c74c00640 1 -- 172.21.15.123:0/2830875385 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2263 (secure 0 0 0) 0x7f3c6409f530 con 0x7f3c78102360 2024-09-19T04:43:48.553 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:43:48.554 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":3,"btime":"2024-09-19T04:43:48:392374+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T04:43:48.554 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 3 2024-09-19T04:43:48.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.553+0000 7f3c7d600640 1 -- 172.21.15.123:0/2830875385 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3c5c0a32c0 msgr2=0x7f3c5c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:48.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.553+0000 7f3c7d600640 1 --2- 172.21.15.123:0/2830875385 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3c5c0a32c0 0x7f3c5c0a5780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009530 tx=0x7f3c68009260 comp rx=0 tx=0).stop 2024-09-19T04:43:48.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.553+0000 7f3c7d600640 1 -- 172.21.15.123:0/2830875385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c78102360 msgr2=0x7f3c78074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:48.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.553+0000 7f3c7d600640 1 --2- 172.21.15.123:0/2830875385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c78102360 0x7f3c78074a30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400b4a0 tx=0x7f3c6400b960 comp rx=0 tx=0).stop 2024-09-19T04:43:48.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.554+0000 7f3c7d600640 1 -- 172.21.15.123:0/2830875385 shutdown_connections 2024-09-19T04:43:48.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.554+0000 7f3c7d600640 1 --2- 172.21.15.123:0/2830875385 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3c5c0a32c0 0x7f3c5c0a5780 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:48.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.554+0000 7f3c7d600640 1 --2- 172.21.15.123:0/2830875385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c781079c0 0x7f3c78074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:48.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.554+0000 7f3c7d600640 1 --2- 172.21.15.123:0/2830875385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c78102360 0x7f3c78074a30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:48.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.554+0000 7f3c7d600640 1 --2- 172.21.15.123:0/2830875385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c781019c0 0x7f3c780744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:48.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.554+0000 7f3c7d600640 1 -- 172.21.15.123:0/2830875385 >> 172.21.15.123:0/2830875385 conn(0x7f3c780fd740 msgr2=0x7f3c780fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:48.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.554+0000 7f3c7d600640 1 -- 172.21.15.123:0/2830875385 shutdown_connections 2024-09-19T04:43:48.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:48.554+0000 7f3c7d600640 1 -- 172.21.15.123:0/2830875385 wait complete. 2024-09-19T04:43:48.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:48 smithi193 ceph-mon[35592]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:48.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:48.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:48.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:48.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:43:48.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:43:48.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:48 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:48.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:48 smithi183 ceph-mon[35333]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:48.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:48 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:48.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:48 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:48.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:48 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:48.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:48 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:43:48.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:48 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:43:48.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:48 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:49.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:49 smithi123 ceph-mon[30511]: Deploying daemon mds.j on smithi123 2024-09-19T04:43:49.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:49 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.183:6832/2688234655,v1:172.21.15.183:6833/2688234655] up:boot 2024-09-19T04:43:49.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:49 smithi123 ceph-mon[30511]: fsmap 2 up:standby 2024-09-19T04:43:49.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:49 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:43:49.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:49 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2830875385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:43:49.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:49 smithi193 ceph-mon[35592]: Deploying daemon mds.j on smithi123 2024-09-19T04:43:49.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:49 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.183:6832/2688234655,v1:172.21.15.183:6833/2688234655] up:boot 2024-09-19T04:43:49.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:49 smithi193 ceph-mon[35592]: fsmap 2 up:standby 2024-09-19T04:43:49.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:49 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:43:49.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:49 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2830875385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:43:49.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:49 smithi183 ceph-mon[35333]: Deploying daemon mds.j on smithi123 2024-09-19T04:43:49.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:49 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.183:6832/2688234655,v1:172.21.15.183:6833/2688234655] up:boot 2024-09-19T04:43:49.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:49 smithi183 ceph-mon[35333]: fsmap 2 up:standby 2024-09-19T04:43:49.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:49 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:43:49.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:49 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2830875385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:43:50.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:50 smithi183 ceph-mon[35333]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:50.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:50 smithi123 ceph-mon[30511]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:50.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:50 smithi193 ceph-mon[35592]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:51.427 INFO:journalctl@ceph.mds.j.smithi123.stdout:Sep 19 04:43:51 smithi123 systemd[1]: Starting Ceph mds.j for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:43:51.471 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:43:51.713 INFO:journalctl@ceph.mds.j.smithi123.stdout:Sep 19 04:43:51 smithi123 podman[69269]: 2024-09-19 04:43:51.333121846 +0000 UTC m=+0.027258864 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:43:51.713 INFO:journalctl@ceph.mds.j.smithi123.stdout:Sep 19 04:43:51 smithi123 podman[69269]: 2024-09-19 04:43:51.466576338 +0000 UTC m=+0.160713360 container create 984c514baed29c8198763ac0f40337ebb2008f83dbf070e49b2c956c30e1649e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-j, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T04:43:51.713 INFO:journalctl@ceph.mds.j.smithi123.stdout:Sep 19 04:43:51 smithi123 podman[69269]: 2024-09-19 04:43:51.711569687 +0000 UTC m=+0.405706704 container init 984c514baed29c8198763ac0f40337ebb2008f83dbf070e49b2c956c30e1649e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-j, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T04:43:51.933 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:43:52.074 INFO:journalctl@ceph.mds.j.smithi123.stdout:Sep 19 04:43:51 smithi123 podman[69269]: 2024-09-19 04:43:51.718299615 +0000 UTC m=+0.412436633 container start 984c514baed29c8198763ac0f40337ebb2008f83dbf070e49b2c956c30e1649e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-j, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-19T04:43:52.074 INFO:journalctl@ceph.mds.j.smithi123.stdout:Sep 19 04:43:51 smithi123 ceph-mds[69313]: -- 172.21.15.123:0/3645874769 <== mon.0 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b30a9d860 con 0x564b30a7b400 2024-09-19T04:43:52.074 INFO:journalctl@ceph.mds.j.smithi123.stdout:Sep 19 04:43:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-j[69302]: starting mds.j at 2024-09-19T04:43:52.074 INFO:journalctl@ceph.mds.j.smithi123.stdout:Sep 19 04:43:51 smithi123 bash[69269]: 984c514baed29c8198763ac0f40337ebb2008f83dbf070e49b2c956c30e1649e 2024-09-19T04:43:52.074 INFO:journalctl@ceph.mds.j.smithi123.stdout:Sep 19 04:43:51 smithi123 systemd[1]: Started Ceph mds.j for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:43:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.504+0000 7f28a1000640 1 -- 172.21.15.123:0/788385370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f289c104010 msgr2=0x7f289c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.504+0000 7f28a1000640 1 --2- 172.21.15.123:0/788385370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f289c104010 0x7f289c1043f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f288c009900 tx=0x7f288c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:43:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.505+0000 7f28a1000640 1 -- 172.21.15.123:0/788385370 shutdown_connections 2024-09-19T04:43:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.505+0000 7f28a1000640 1 --2- 172.21.15.123:0/788385370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f289c109140 0x7f289c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.505+0000 7f28a1000640 1 --2- 172.21.15.123:0/788385370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f289c1049c0 0x7f289c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.505+0000 7f28a1000640 1 --2- 172.21.15.123:0/788385370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f289c104010 0x7f289c1043f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.505+0000 7f28a1000640 1 -- 172.21.15.123:0/788385370 >> 172.21.15.123:0/788385370 conn(0x7f289c0ffa50 msgr2=0x7f289c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.505+0000 7f28a1000640 1 -- 172.21.15.123:0/788385370 shutdown_connections 2024-09-19T04:43:52.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.505+0000 7f28a1000640 1 -- 172.21.15.123:0/788385370 wait complete. 2024-09-19T04:43:52.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.506+0000 7f28a1000640 1 Processor -- start 2024-09-19T04:43:52.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.506+0000 7f28a1000640 1 -- start start 2024-09-19T04:43:52.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.506+0000 7f28a1000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f289c104010 0x7f289c110ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:52.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.506+0000 7f28a1000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f289c1049c0 0x7f289c10db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:52.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.506+0000 7f28a1000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f289c109140 0x7f289c10e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:52.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.506+0000 7f28a1000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f289c114140 con 0x7f289c104010 2024-09-19T04:43:52.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.506+0000 7f28a1000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f289c113fc0 con 0x7f289c109140 2024-09-19T04:43:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.506+0000 7f28a1000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f289c1142c0 con 0x7f289c1049c0 2024-09-19T04:43:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.506+0000 7f2899e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f289c104010 0x7f289c110ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.506+0000 7f2899e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f289c104010 0x7f289c110ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46850/0 (socket says 172.21.15.123:46850) 2024-09-19T04:43:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.506+0000 7f2899e00640 1 -- 172.21.15.123:0/3773333138 learned_addr learned my addr 172.21.15.123:0/3773333138 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:43:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.506+0000 7f289a800640 1 --2- 172.21.15.123:0/3773333138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f289c109140 0x7f289c10e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.507+0000 7f2893e00640 1 --2- 172.21.15.123:0/3773333138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f289c1049c0 0x7f289c10db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.507+0000 7f289a800640 1 -- 172.21.15.123:0/3773333138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f289c1049c0 msgr2=0x7f289c10db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.507+0000 7f289a800640 1 --2- 172.21.15.123:0/3773333138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f289c1049c0 0x7f289c10db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.507+0000 7f289a800640 1 -- 172.21.15.123:0/3773333138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f289c104010 msgr2=0x7f289c110ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.507+0000 7f289a800640 1 --2- 172.21.15.123:0/3773333138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f289c104010 0x7f289c110ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.507+0000 7f289a800640 1 -- 172.21.15.123:0/3773333138 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289c10e940 con 0x7f289c109140 2024-09-19T04:43:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.507+0000 7f2899e00640 1 --2- 172.21.15.123:0/3773333138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f289c104010 0x7f289c110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:43:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.507+0000 7f289a800640 1 --2- 172.21.15.123:0/3773333138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f289c109140 0x7f289c10e0b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2894009fd0 tx=0x7f289400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:52.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.508+0000 7f2892a00640 1 -- 172.21.15.123:0/3773333138 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289400cd60 con 0x7f289c109140 2024-09-19T04:43:52.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.508+0000 7f2892a00640 1 -- 172.21.15.123:0/3773333138 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2894005020 con 0x7f289c109140 2024-09-19T04:43:52.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.508+0000 7f28a1000640 1 -- 172.21.15.123:0/3773333138 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289c1aaee0 con 0x7f289c109140 2024-09-19T04:43:52.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.508+0000 7f2892a00640 1 -- 172.21.15.123:0/3773333138 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894010690 con 0x7f289c109140 2024-09-19T04:43:52.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.508+0000 7f28a1000640 1 -- 172.21.15.123:0/3773333138 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f289c1ab2e0 con 0x7f289c109140 2024-09-19T04:43:52.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.509+0000 7f28a1000640 1 -- 172.21.15.123:0/3773333138 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f289c1054f0 con 0x7f289c109140 2024-09-19T04:43:52.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.511+0000 7f2892a00640 1 -- 172.21.15.123:0/3773333138 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2894005540 con 0x7f289c109140 2024-09-19T04:43:52.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.513+0000 7f2892a00640 1 --2- 172.21.15.123:0/3773333138 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f28800a3240 0x7f28800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:52.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.513+0000 7f2892a00640 1 -- 172.21.15.123:0/3773333138 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f28940cfb20 con 0x7f289c109140 2024-09-19T04:43:52.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.513+0000 7f2899e00640 1 --2- 172.21.15.123:0/3773333138 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f28800a3240 0x7f28800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:52.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.514+0000 7f2899e00640 1 --2- 172.21.15.123:0/3773333138 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f28800a3240 0x7f28800a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f288c0096c0 tx=0x7f288c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:52.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.516+0000 7f2892a00640 1 -- 172.21.15.123:0/3773333138 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f289409cc40 con 0x7f289c109140 2024-09-19T04:43:52.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:52 smithi123 ceph-mon[30511]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:52.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:52 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:52.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:52 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:52.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:52 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:52.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:52 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:43:52.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:52 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:43:52.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:52 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:52.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:52 smithi193 ceph-mon[35592]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:52.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:52 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:52.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:52 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:52.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:52 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:52.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:52 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:43:52.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:52 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:43:52.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:52 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:52.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.741+0000 7f28a1000640 1 -- 172.21.15.123:0/3773333138 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f289c02d7c0 con 0x7f289c109140 2024-09-19T04:43:52.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.742+0000 7f2892a00640 1 -- 172.21.15.123:0/3773333138 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7f289409c540 con 0x7f289c109140 2024-09-19T04:43:52.744 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:43:52.744 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":4,"btime":"2024-09-19T04:43:52:408111+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T04:43:52.744 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 4 2024-09-19T04:43:52.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.745+0000 7f28a1000640 1 -- 172.21.15.123:0/3773333138 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f28800a3240 msgr2=0x7f28800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:52.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.745+0000 7f28a1000640 1 --2- 172.21.15.123:0/3773333138 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f28800a3240 0x7f28800a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f288c0096c0 tx=0x7f288c005a20 comp rx=0 tx=0).stop 2024-09-19T04:43:52.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.745+0000 7f28a1000640 1 -- 172.21.15.123:0/3773333138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f289c109140 msgr2=0x7f289c10e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:52.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.745+0000 7f28a1000640 1 --2- 172.21.15.123:0/3773333138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f289c109140 0x7f289c10e0b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2894009fd0 tx=0x7f289400ee50 comp rx=0 tx=0).stop 2024-09-19T04:43:52.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.745+0000 7f28a1000640 1 -- 172.21.15.123:0/3773333138 shutdown_connections 2024-09-19T04:43:52.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.745+0000 7f28a1000640 1 --2- 172.21.15.123:0/3773333138 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f28800a3240 0x7f28800a5700 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:52.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.745+0000 7f28a1000640 1 --2- 172.21.15.123:0/3773333138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f289c109140 0x7f289c10e0b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:52.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.745+0000 7f28a1000640 1 --2- 172.21.15.123:0/3773333138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f289c1049c0 0x7f289c10db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:52.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.745+0000 7f28a1000640 1 --2- 172.21.15.123:0/3773333138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f289c104010 0x7f289c110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:52.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.745+0000 7f28a1000640 1 -- 172.21.15.123:0/3773333138 >> 172.21.15.123:0/3773333138 conn(0x7f289c0ffa50 msgr2=0x7f289c100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:52.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.745+0000 7f28a1000640 1 -- 172.21.15.123:0/3773333138 shutdown_connections 2024-09-19T04:43:52.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:52.745+0000 7f28a1000640 1 -- 172.21.15.123:0/3773333138 wait complete. 2024-09-19T04:43:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:52 smithi183 ceph-mon[35333]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:52 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:52 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:52 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:52 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:43:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:52 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:43:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:52 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:53.471 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:53 smithi123 ceph-mon[30511]: Deploying daemon mds.d on smithi123 2024-09-19T04:43:53.471 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:53 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.123:6834/1654466507,v1:172.21.15.123:6835/1654466507] up:boot 2024-09-19T04:43:53.471 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:53 smithi123 ceph-mon[30511]: fsmap 3 up:standby 2024-09-19T04:43:53.471 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:53 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:43:53.471 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:53 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3773333138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:43:53.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:53 smithi193 ceph-mon[35592]: Deploying daemon mds.d on smithi123 2024-09-19T04:43:53.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:53 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.123:6834/1654466507,v1:172.21.15.123:6835/1654466507] up:boot 2024-09-19T04:43:53.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:53 smithi193 ceph-mon[35592]: fsmap 3 up:standby 2024-09-19T04:43:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:53 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:43:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:53 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3773333138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:43:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:53 smithi183 ceph-mon[35333]: Deploying daemon mds.d on smithi123 2024-09-19T04:43:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:53 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.123:6834/1654466507,v1:172.21.15.123:6835/1654466507] up:boot 2024-09-19T04:43:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:53 smithi183 ceph-mon[35333]: fsmap 3 up:standby 2024-09-19T04:43:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:53 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:43:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:53 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3773333138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:43:54.462 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:54 smithi123 ceph-mon[30511]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:54.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:54 smithi193 ceph-mon[35592]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:54 smithi183 ceph-mon[35333]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:55.428 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:43:55 smithi123 systemd[1]: Starting Ceph mds.d for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:43:55.751 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:43:55 smithi123 podman[69680]: 2024-09-19 04:43:55.355817449 +0000 UTC m=+0.027614787 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:43:55.751 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:43:55 smithi123 podman[69680]: 2024-09-19 04:43:55.472087895 +0000 UTC m=+0.143885233 container create fd9818a3f941a28b655f57661934bdc74216d33d3e053c13a8085959bcdde0f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T04:43:55.752 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:43:55 smithi123 podman[69680]: 2024-09-19 04:43:55.750321855 +0000 UTC m=+0.422119205 container init fd9818a3f941a28b655f57661934bdc74216d33d3e053c13a8085959bcdde0f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-d, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T04:43:55.956 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:43:56.007 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:43:55 smithi123 podman[69680]: 2024-09-19 04:43:55.757079116 +0000 UTC m=+0.428876458 container start fd9818a3f941a28b655f57661934bdc74216d33d3e053c13a8085959bcdde0f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-19T04:43:56.007 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:43:55 smithi123 ceph-mds[69707]: -- 172.21.15.123:0/3702521288 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bdad445860 con 0x55bdad423000 2024-09-19T04:43:56.007 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:43:55 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-d[69697]: starting mds.d at 2024-09-19T04:43:56.008 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:43:55 smithi123 bash[69680]: fd9818a3f941a28b655f57661934bdc74216d33d3e053c13a8085959bcdde0f6 2024-09-19T04:43:56.008 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:43:55 smithi123 systemd[1]: Started Ceph mds.d for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:43:56.336 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:43:56.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:56 smithi183 ceph-mon[35333]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:56.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:56 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:56.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:56 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:56.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:56 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:56.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:56 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:43:56.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:56 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:43:56.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:56 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:56.689 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:56 smithi123 ceph-mon[30511]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:56.689 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:56 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:56.689 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:56 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:56.690 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:56 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:56.690 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:56 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:43:56.690 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:56 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:43:56.690 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:56 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:56.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:56 smithi193 ceph-mon[35592]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:56.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:56 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:56.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:56 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:56.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:56 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:56.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:56 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:43:56.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:56 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:43:56.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:56 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:56.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.923+0000 7f9cfe400640 1 -- 172.21.15.123:0/2251981258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cf8112b90 msgr2=0x7f9cf8114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:56.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.923+0000 7f9cfe400640 1 --2- 172.21.15.123:0/2251981258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cf8112b90 0x7f9cf8114f80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009ca0 tx=0x7f9cf002a950 comp rx=0 tx=0).stop 2024-09-19T04:43:56.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.923+0000 7f9cfe400640 1 -- 172.21.15.123:0/2251981258 shutdown_connections 2024-09-19T04:43:56.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.923+0000 7f9cfe400640 1 --2- 172.21.15.123:0/2251981258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cf8112b90 0x7f9cf8114f80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:56.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.923+0000 7f9cfe400640 1 --2- 172.21.15.123:0/2251981258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cf8074eb0 0x7f9cf8112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:56.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.923+0000 7f9cfe400640 1 --2- 172.21.15.123:0/2251981258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cf8074590 0x7f9cf8074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:56.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.923+0000 7f9cfe400640 1 -- 172.21.15.123:0/2251981258 >> 172.21.15.123:0/2251981258 conn(0x7f9cf80fd8c0 msgr2=0x7f9cf80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:56.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.923+0000 7f9cfe400640 1 -- 172.21.15.123:0/2251981258 shutdown_connections 2024-09-19T04:43:56.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.923+0000 7f9cfe400640 1 -- 172.21.15.123:0/2251981258 wait complete. 2024-09-19T04:43:56.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.924+0000 7f9cfe400640 1 Processor -- start 2024-09-19T04:43:56.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.924+0000 7f9cfe400640 1 -- start start 2024-09-19T04:43:56.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.924+0000 7f9cfe400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cf8074590 0x7f9cf819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:56.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.924+0000 7f9cfe400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cf8074eb0 0x7f9cf819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:56.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.924+0000 7f9cfe400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cf8112b90 0x7f9cf81a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:56.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.924+0000 7f9cfe400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf81185c0 con 0x7f9cf8074590 2024-09-19T04:43:56.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.924+0000 7f9cfe400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf8118440 con 0x7f9cf8112b90 2024-09-19T04:43:56.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.924+0000 7f9cfe400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf8118740 con 0x7f9cf8074eb0 2024-09-19T04:43:56.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.924+0000 7f9cf7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cf8074590 0x7f9cf819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:56.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.924+0000 7f9cf7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cf8112b90 0x7f9cf81a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:56.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.924+0000 7f9cf7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cf8112b90 0x7f9cf81a3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:58704/0 (socket says 172.21.15.123:58704) 2024-09-19T04:43:56.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.924+0000 7f9cf7e00640 1 -- 172.21.15.123:0/3209702761 learned_addr learned my addr 172.21.15.123:0/3209702761 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:43:56.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.925+0000 7f9cf6a00640 1 --2- 172.21.15.123:0/3209702761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cf8074eb0 0x7f9cf819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:56.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.925+0000 7f9cf7e00640 1 -- 172.21.15.123:0/3209702761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cf8074eb0 msgr2=0x7f9cf819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:56.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.925+0000 7f9cf7e00640 1 --2- 172.21.15.123:0/3209702761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cf8074eb0 0x7f9cf819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:56.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.925+0000 7f9cf7e00640 1 -- 172.21.15.123:0/3209702761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cf8074590 msgr2=0x7f9cf819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:56.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.925+0000 7f9cf7e00640 1 --2- 172.21.15.123:0/3209702761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cf8074590 0x7f9cf819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:56.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.925+0000 7f9cf7e00640 1 -- 172.21.15.123:0/3209702761 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf81a3b40 con 0x7f9cf8112b90 2024-09-19T04:43:56.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.925+0000 7f9cf7400640 1 --2- 172.21.15.123:0/3209702761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cf8074590 0x7f9cf819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:43:56.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.925+0000 7f9cf7e00640 1 --2- 172.21.15.123:0/3209702761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cf8112b90 0x7f9cf81a3410 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0005e30 tx=0x7f9cf0003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:56.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.926+0000 7f9cefe00640 1 -- 172.21.15.123:0/3209702761 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0034040 con 0x7f9cf8112b90 2024-09-19T04:43:56.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.926+0000 7f9cfe400640 1 -- 172.21.15.123:0/3209702761 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf81a3dd0 con 0x7f9cf8112b90 2024-09-19T04:43:56.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.926+0000 7f9cefe00640 1 -- 172.21.15.123:0/3209702761 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cf002f940 con 0x7f9cf8112b90 2024-09-19T04:43:56.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.926+0000 7f9cefe00640 1 -- 172.21.15.123:0/3209702761 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00045f0 con 0x7f9cf8112b90 2024-09-19T04:43:56.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.926+0000 7f9cfe400640 1 -- 172.21.15.123:0/3209702761 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf81ab6c0 con 0x7f9cf8112b90 2024-09-19T04:43:56.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.927+0000 7f9cfe400640 1 -- 172.21.15.123:0/3209702761 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf8074970 con 0x7f9cf8112b90 2024-09-19T04:43:56.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.929+0000 7f9cefe00640 1 -- 172.21.15.123:0/3209702761 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9cf0037020 con 0x7f9cf8112b90 2024-09-19T04:43:56.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.930+0000 7f9cefe00640 1 --2- 172.21.15.123:0/3209702761 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9cdc0a3240 0x7f9cdc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:43:56.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.930+0000 7f9cefe00640 1 -- 172.21.15.123:0/3209702761 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f9cf00f03a0 con 0x7f9cf8112b90 2024-09-19T04:43:56.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.930+0000 7f9cf7400640 1 --2- 172.21.15.123:0/3209702761 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9cdc0a3240 0x7f9cdc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:43:56.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.931+0000 7f9cf7400640 1 --2- 172.21.15.123:0/3209702761 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9cdc0a3240 0x7f9cdc0a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009fc0 tx=0x7f9ce8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:43:56.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:56.932+0000 7f9cefe00640 1 -- 172.21.15.123:0/3209702761 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9cf0081210 con 0x7f9cf8112b90 2024-09-19T04:43:57.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:57.145+0000 7f9cfe400640 1 -- 172.21.15.123:0/3209702761 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cf8104fd0 con 0x7f9cf8112b90 2024-09-19T04:43:57.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:57.146+0000 7f9cefe00640 1 -- 172.21.15.123:0/3209702761 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3919 (secure 0 0 0) 0x7f9cf00bd4c0 con 0x7f9cf8112b90 2024-09-19T04:43:57.147 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:43:57.147 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":5,"btime":"2024-09-19T04:43:56:427789+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-09-19T04:43:57.148 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 5 2024-09-19T04:43:57.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:57.148+0000 7f9cfe400640 1 -- 172.21.15.123:0/3209702761 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9cdc0a3240 msgr2=0x7f9cdc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:57.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:57.148+0000 7f9cfe400640 1 --2- 172.21.15.123:0/3209702761 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9cdc0a3240 0x7f9cdc0a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009fc0 tx=0x7f9ce8005f90 comp rx=0 tx=0).stop 2024-09-19T04:43:57.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:57.148+0000 7f9cfe400640 1 -- 172.21.15.123:0/3209702761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cf8112b90 msgr2=0x7f9cf81a3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:43:57.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:57.148+0000 7f9cfe400640 1 --2- 172.21.15.123:0/3209702761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cf8112b90 0x7f9cf81a3410 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0005e30 tx=0x7f9cf0003b80 comp rx=0 tx=0).stop 2024-09-19T04:43:57.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:57.148+0000 7f9cfe400640 1 -- 172.21.15.123:0/3209702761 shutdown_connections 2024-09-19T04:43:57.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:57.148+0000 7f9cfe400640 1 --2- 172.21.15.123:0/3209702761 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9cdc0a3240 0x7f9cdc0a5700 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:57.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:57.148+0000 7f9cfe400640 1 --2- 172.21.15.123:0/3209702761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cf8112b90 0x7f9cf81a3410 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:57.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:57.148+0000 7f9cfe400640 1 --2- 172.21.15.123:0/3209702761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cf8074eb0 0x7f9cf819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:57.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:57.148+0000 7f9cfe400640 1 --2- 172.21.15.123:0/3209702761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cf8074590 0x7f9cf819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:43:57.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:57.148+0000 7f9cfe400640 1 -- 172.21.15.123:0/3209702761 >> 172.21.15.123:0/3209702761 conn(0x7f9cf80fd8c0 msgr2=0x7f9cf80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:43:57.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:57.148+0000 7f9cfe400640 1 -- 172.21.15.123:0/3209702761 shutdown_connections 2024-09-19T04:43:57.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:43:57.148+0000 7f9cfe400640 1 -- 172.21.15.123:0/3209702761 wait complete. 2024-09-19T04:43:57.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:57 smithi193 ceph-mon[35592]: Deploying daemon mds.e on smithi183 2024-09-19T04:43:57.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:57 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.123:6836/3620540498,v1:172.21.15.123:6837/3620540498] up:boot 2024-09-19T04:43:57.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:57 smithi193 ceph-mon[35592]: fsmap 4 up:standby 2024-09-19T04:43:57.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:57 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:43:57.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:57 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3209702761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:43:57.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:57 smithi183 ceph-mon[35333]: Deploying daemon mds.e on smithi183 2024-09-19T04:43:57.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:57 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.123:6836/3620540498,v1:172.21.15.123:6837/3620540498] up:boot 2024-09-19T04:43:57.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:57 smithi183 ceph-mon[35333]: fsmap 4 up:standby 2024-09-19T04:43:57.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:57 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:43:57.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:57 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3209702761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:43:57.873 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:57 smithi123 ceph-mon[30511]: Deploying daemon mds.e on smithi183 2024-09-19T04:43:57.873 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:57 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.123:6836/3620540498,v1:172.21.15.123:6837/3620540498] up:boot 2024-09-19T04:43:57.873 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:57 smithi123 ceph-mon[30511]: fsmap 4 up:standby 2024-09-19T04:43:57.873 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:57 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:43:57.873 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:57 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3209702761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:43:58.100 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:43:57 smithi183 systemd[1]: Starting Ceph mds.e for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:43:58.445 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:43:58 smithi183 podman[64586]: 2024-09-19 04:43:58.046905578 +0000 UTC m=+0.015609345 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:43:58.445 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:43:58 smithi183 podman[64586]: 2024-09-19 04:43:58.178088232 +0000 UTC m=+0.146792042 container create c61564e5ddbaefbc1effa6bce2c3e4fef5c84476f4484e40b6134db7ea19d41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T04:43:58.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:58 smithi193 ceph-mon[35592]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:58.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:58 smithi183 ceph-mon[35333]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:58.850 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:43:58 smithi183 podman[64586]: 2024-09-19 04:43:58.56374986 +0000 UTC m=+0.532453630 container init c61564e5ddbaefbc1effa6bce2c3e4fef5c84476f4484e40b6134db7ea19d41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T04:43:58.850 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:43:58 smithi183 podman[64586]: 2024-09-19 04:43:58.566860698 +0000 UTC m=+0.535564462 container start c61564e5ddbaefbc1effa6bce2c3e4fef5c84476f4484e40b6134db7ea19d41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T04:43:58.850 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:43:58 smithi183 ceph-mds[64605]: -- 172.21.15.183:0/305939305 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56517f20b860 con 0x56517f1e8800 2024-09-19T04:43:58.850 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:43:58 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-e[64601]: starting mds.e at 2024-09-19T04:43:58.850 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:43:58 smithi183 bash[64586]: c61564e5ddbaefbc1effa6bce2c3e4fef5c84476f4484e40b6134db7ea19d41b 2024-09-19T04:43:58.850 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:43:58 smithi183 systemd[1]: Started Ceph mds.e for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:43:58.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:58 smithi123 ceph-mon[30511]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:59.911 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:43:59.966 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:59 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:59.966 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:59 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:59.966 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:59 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:59.966 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:59 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:43:59.966 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:59 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:43:59.966 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:59 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:59.966 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:59 smithi123 ceph-mon[30511]: Deploying daemon mds.f on smithi193 2024-09-19T04:43:59.966 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:59 smithi123 ceph-mon[30511]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:59.966 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:59 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.183:6834/309335638,v1:172.21.15.183:6835/309335638] up:boot 2024-09-19T04:43:59.966 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:59 smithi123 ceph-mon[30511]: fsmap 5 up:standby 2024-09-19T04:43:59.966 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:43:59 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:43:59.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:59 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:59.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:59 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:59.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:59 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:43:59.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:59 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:43:59.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:59 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:43:59.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:59 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:43:59.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:59 smithi193 ceph-mon[35592]: Deploying daemon mds.f on smithi193 2024-09-19T04:43:59.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:59 smithi193 ceph-mon[35592]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:43:59.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:59 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.183:6834/309335638,v1:172.21.15.183:6835/309335638] up:boot 2024-09-19T04:43:59.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:59 smithi193 ceph-mon[35592]: fsmap 5 up:standby 2024-09-19T04:43:59.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:43:59 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:44:00.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:59 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:00.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:59 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:00.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:59 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:00.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:59 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:00.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:59 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:00.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:59 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:00.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:59 smithi183 ceph-mon[35333]: Deploying daemon mds.f on smithi193 2024-09-19T04:44:00.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:59 smithi183 ceph-mon[35333]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:00.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:59 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.183:6834/309335638,v1:172.21.15.183:6835/309335638] up:boot 2024-09-19T04:44:00.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:59 smithi183 ceph-mon[35333]: fsmap 5 up:standby 2024-09-19T04:44:00.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:43:59 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:44:00.285 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:00.741 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 04:44:00 smithi193 systemd[1]: Starting Ceph mds.f for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:44:00.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.821+0000 7f02bd200640 1 -- 172.21.15.123:0/1309983035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b8101de0 msgr2=0x7f02b81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:00.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.821+0000 7f02bd200640 1 --2- 172.21.15.123:0/1309983035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b8101de0 0x7f02b81021c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009900 tx=0x7f02a802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:44:00.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.821+0000 7f02bd200640 1 -- 172.21.15.123:0/1309983035 shutdown_connections 2024-09-19T04:44:00.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.821+0000 7f02bd200640 1 --2- 172.21.15.123:0/1309983035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02b8106f10 0x7f02b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:00.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.821+0000 7f02bd200640 1 --2- 172.21.15.123:0/1309983035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02b8102790 0x7f02b81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:00.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.821+0000 7f02bd200640 1 --2- 172.21.15.123:0/1309983035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b8101de0 0x7f02b81021c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:00.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.821+0000 7f02bd200640 1 -- 172.21.15.123:0/1309983035 >> 172.21.15.123:0/1309983035 conn(0x7f02b80fd8c0 msgr2=0x7f02b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:00.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.821+0000 7f02bd200640 1 -- 172.21.15.123:0/1309983035 shutdown_connections 2024-09-19T04:44:00.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.821+0000 7f02bd200640 1 -- 172.21.15.123:0/1309983035 wait complete. 2024-09-19T04:44:00.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.822+0000 7f02bd200640 1 Processor -- start 2024-09-19T04:44:00.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.822+0000 7f02bd200640 1 -- start start 2024-09-19T04:44:00.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.822+0000 7f02bd200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02b8101de0 0x7f02b81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:00.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.822+0000 7f02bd200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02b8102790 0x7f02b8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:00.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.822+0000 7f02bd200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b8106f10 0x7f02b819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:00.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.822+0000 7f02bd200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f02b8111f70 con 0x7f02b8106f10 2024-09-19T04:44:00.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.822+0000 7f02bd200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02b8111df0 con 0x7f02b8102790 2024-09-19T04:44:00.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.822+0000 7f02bd200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f02b81120f0 con 0x7f02b8101de0 2024-09-19T04:44:00.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.822+0000 7f02b5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02b8101de0 0x7f02b81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:00.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.822+0000 7f02b5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02b8101de0 0x7f02b81983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:35278/0 (socket says 172.21.15.123:35278) 2024-09-19T04:44:00.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.822+0000 7f02b5e00640 1 -- 172.21.15.123:0/4170338597 learned_addr learned my addr 172.21.15.123:0/4170338597 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:00.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.822+0000 7f02b6800640 1 --2- 172.21.15.123:0/4170338597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b8106f10 0x7f02b819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:00.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.822+0000 7f02ad400640 1 --2- 172.21.15.123:0/4170338597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02b8102790 0x7f02b8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:00.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.823+0000 7f02b5e00640 1 -- 172.21.15.123:0/4170338597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02b8102790 msgr2=0x7f02b8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:00.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.823+0000 7f02b5e00640 1 --2- 172.21.15.123:0/4170338597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02b8102790 0x7f02b8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:00.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.823+0000 7f02b5e00640 1 -- 172.21.15.123:0/4170338597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b8106f10 msgr2=0x7f02b819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:00.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.823+0000 7f02b5e00640 1 --2- 172.21.15.123:0/4170338597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b8106f10 0x7f02b819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:00.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.823+0000 7f02b5e00640 1 -- 172.21.15.123:0/4170338597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b819d3a0 con 0x7f02b8101de0 2024-09-19T04:44:00.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.823+0000 7f02b5e00640 1 --2- 172.21.15.123:0/4170338597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02b8101de0 0x7f02b81983f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f02a8006010 tx=0x7f02a80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:00.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.823+0000 7f02af400640 1 -- 172.21.15.123:0/4170338597 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a803f030 con 0x7f02b8101de0 2024-09-19T04:44:00.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.823+0000 7f02af400640 1 -- 172.21.15.123:0/4170338597 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a8003ce0 con 0x7f02b8101de0 2024-09-19T04:44:00.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.823+0000 7f02bd200640 1 -- 172.21.15.123:0/4170338597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b819d630 con 0x7f02b8101de0 2024-09-19T04:44:00.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.823+0000 7f02af400640 1 -- 172.21.15.123:0/4170338597 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a8035070 con 0x7f02b8101de0 2024-09-19T04:44:00.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.824+0000 7f02bd200640 1 -- 172.21.15.123:0/4170338597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b81a4f70 con 0x7f02b8101de0 2024-09-19T04:44:00.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.824+0000 7f02bd200640 1 -- 172.21.15.123:0/4170338597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b81032c0 con 0x7f02b8101de0 2024-09-19T04:44:00.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.826+0000 7f02af400640 1 -- 172.21.15.123:0/4170338597 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f02a8042030 con 0x7f02b8101de0 2024-09-19T04:44:00.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.827+0000 7f02af400640 1 --2- 172.21.15.123:0/4170338597 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f029c0a3240 0x7f029c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:00.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.828+0000 7f02af400640 1 -- 172.21.15.123:0/4170338597 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f02a80efdb0 con 0x7f02b8101de0 2024-09-19T04:44:00.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.828+0000 7f02ad400640 1 --2- 172.21.15.123:0/4170338597 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f029c0a3240 0x7f029c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:00.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.828+0000 7f02ad400640 1 --2- 172.21.15.123:0/4170338597 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f029c0a3240 0x7f029c0a5700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f02b80fdd50 tx=0x7f02a000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:00.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:00.830+0000 7f02af400640 1 -- 172.21.15.123:0/4170338597 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f02a80bc940 con 0x7f02b8101de0 2024-09-19T04:44:01.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:01.038+0000 7f02bd200640 1 -- 172.21.15.123:0/4170338597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02b810a0f0 con 0x7f02b8101de0 2024-09-19T04:44:01.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:01.039+0000 7f02af400640 1 -- 172.21.15.123:0/4170338597 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4744 (secure 0 0 0) 0x7f02a80c0c40 con 0x7f02b8101de0 2024-09-19T04:44:01.041 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:44:01.041 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":6,"btime":"2024-09-19T04:43:59:443743+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-19T04:44:01.042 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 6 2024-09-19T04:44:01.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:01.041+0000 7f02bd200640 1 -- 172.21.15.123:0/4170338597 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f029c0a3240 msgr2=0x7f029c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:01.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:01.041+0000 7f02bd200640 1 --2- 172.21.15.123:0/4170338597 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f029c0a3240 0x7f029c0a5700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f02b80fdd50 tx=0x7f02a000b410 comp rx=0 tx=0).stop 2024-09-19T04:44:01.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:01.041+0000 7f02bd200640 1 -- 172.21.15.123:0/4170338597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02b8101de0 msgr2=0x7f02b81983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:01.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:01.041+0000 7f02bd200640 1 --2- 172.21.15.123:0/4170338597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02b8101de0 0x7f02b81983f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f02a8006010 tx=0x7f02a80042f0 comp rx=0 tx=0).stop 2024-09-19T04:44:01.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:01.041+0000 7f02bd200640 1 -- 172.21.15.123:0/4170338597 shutdown_connections 2024-09-19T04:44:01.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:01.041+0000 7f02bd200640 1 --2- 172.21.15.123:0/4170338597 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f029c0a3240 0x7f029c0a5700 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:01.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:01.041+0000 7f02bd200640 1 --2- 172.21.15.123:0/4170338597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b8106f10 0x7f02b819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:01.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:01.041+0000 7f02bd200640 1 --2- 172.21.15.123:0/4170338597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02b8102790 0x7f02b8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:01.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:01.041+0000 7f02bd200640 1 --2- 172.21.15.123:0/4170338597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02b8101de0 0x7f02b81983f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:01.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:01.041+0000 7f02bd200640 1 -- 172.21.15.123:0/4170338597 >> 172.21.15.123:0/4170338597 conn(0x7f02b80fd8c0 msgr2=0x7f02b80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:01.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:01.041+0000 7f02bd200640 1 -- 172.21.15.123:0/4170338597 shutdown_connections 2024-09-19T04:44:01.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:01.041+0000 7f02bd200640 1 -- 172.21.15.123:0/4170338597 wait complete. 2024-09-19T04:44:01.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:01 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/4170338597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:01.134 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 04:44:00 smithi193 podman[65366]: 2024-09-19 04:44:00.763424621 +0000 UTC m=+0.016155627 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:44:01.134 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 04:44:00 smithi193 podman[65366]: 2024-09-19 04:44:00.889357225 +0000 UTC m=+0.142088220 container create 0f3bca754953689d7fae833f8e81f711a64503ca4a6bfe6104811f59c01a5fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-19T04:44:01.134 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 04:44:01 smithi193 podman[65366]: 2024-09-19 04:44:01.133862756 +0000 UTC m=+0.386593751 container init 0f3bca754953689d7fae833f8e81f711a64503ca4a6bfe6104811f59c01a5fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:44:01.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:01 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/4170338597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:01.391 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:01 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/4170338597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:01.490 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 04:44:01 smithi193 podman[65366]: 2024-09-19 04:44:01.136787042 +0000 UTC m=+0.389518038 container start 0f3bca754953689d7fae833f8e81f711a64503ca4a6bfe6104811f59c01a5fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:44:01.490 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 04:44:01 smithi193 ceph-mds[65386]: -- 172.21.15.193:0/3901571380 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558c67c43860 con 0x558c67c20800 2024-09-19T04:44:01.491 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 04:44:01 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-f[65382]: starting mds.f at 2024-09-19T04:44:01.491 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 04:44:01 smithi193 bash[65366]: 0f3bca754953689d7fae833f8e81f711a64503ca4a6bfe6104811f59c01a5fac 2024-09-19T04:44:01.491 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 04:44:01 smithi193 systemd[1]: Started Ceph mds.f for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:44:02.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:02.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:02.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:02.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:02.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:02.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:02.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:02 smithi183 ceph-mon[35333]: Deploying daemon mds.c on smithi193 2024-09-19T04:44:02.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:02 smithi183 ceph-mon[35333]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:02.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:02 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] up:boot 2024-09-19T04:44:02.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:02 smithi183 ceph-mon[35333]: fsmap 6 up:standby 2024-09-19T04:44:02.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:02 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:44:02.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:02.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:02.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:02.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:02.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:02.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:02 smithi123 ceph-mon[30511]: Deploying daemon mds.c on smithi193 2024-09-19T04:44:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:02 smithi123 ceph-mon[30511]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:02 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] up:boot 2024-09-19T04:44:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:02 smithi123 ceph-mon[30511]: fsmap 6 up:standby 2024-09-19T04:44:02.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:02 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:44:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:02 smithi193 ceph-mon[35592]: Deploying daemon mds.c on smithi193 2024-09-19T04:44:02.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:02 smithi193 ceph-mon[35592]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:02.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:02 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] up:boot 2024-09-19T04:44:02.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:02 smithi193 ceph-mon[35592]: fsmap 6 up:standby 2024-09-19T04:44:02.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:02 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:44:03.489 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 04:44:03 smithi193 systemd[1]: Starting Ceph mds.c for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:44:03.741 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 04:44:03 smithi193 podman[65668]: 2024-09-19 04:44:03.391057276 +0000 UTC m=+0.015978536 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:44:03.741 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 04:44:03 smithi193 podman[65668]: 2024-09-19 04:44:03.52050642 +0000 UTC m=+0.145427713 container create dafb2eeaedcb7a6fc35275f6ee9e349d225d0d1bd39d6d5aec97ca6a546c05bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-09-19T04:44:03.756 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:44:04.139 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:04.241 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 04:44:03 smithi193 podman[65668]: 2024-09-19 04:44:03.790516494 +0000 UTC m=+0.415437745 container init dafb2eeaedcb7a6fc35275f6ee9e349d225d0d1bd39d6d5aec97ca6a546c05bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:44:04.241 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 04:44:03 smithi193 podman[65668]: 2024-09-19 04:44:03.793335973 +0000 UTC m=+0.418257232 container start dafb2eeaedcb7a6fc35275f6ee9e349d225d0d1bd39d6d5aec97ca6a546c05bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-c, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-19T04:44:04.241 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 04:44:03 smithi193 ceph-mds[65687]: -- 172.21.15.193:0/4270276585 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55774cec9860 con 0x55774cea6800 2024-09-19T04:44:04.241 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 04:44:03 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-c[65683]: starting mds.c at 2024-09-19T04:44:04.241 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 04:44:03 smithi193 bash[65668]: dafb2eeaedcb7a6fc35275f6ee9e349d225d0d1bd39d6d5aec97ca6a546c05bb 2024-09-19T04:44:04.241 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 04:44:03 smithi193 systemd[1]: Started Ceph mds.c for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:44:04.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:04 smithi123 ceph-mon[30511]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:04.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:04 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:04.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:04 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:04.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:04 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:04.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:04 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:04.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:04 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:04.581 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:04 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:04.644 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:04 smithi183 ceph-mon[35333]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:04.644 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:04.644 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:04.644 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:04.644 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:04.644 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:04.644 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:04 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:04.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.688+0000 7f137a000640 1 -- 172.21.15.123:0/426739838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1374112a30 msgr2=0x7f1374114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:04.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.688+0000 7f137a000640 1 --2- 172.21.15.123:0/426739838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1374112a30 0x7f1374114e20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f136c009ca0 tx=0x7f136c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:44:04.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.688+0000 7f137a000640 1 -- 172.21.15.123:0/426739838 shutdown_connections 2024-09-19T04:44:04.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.688+0000 7f137a000640 1 --2- 172.21.15.123:0/426739838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1374112a30 0x7f1374114e20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:04.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.688+0000 7f137a000640 1 --2- 172.21.15.123:0/426739838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1374077040 0x7f13741124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:04.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.688+0000 7f137a000640 1 --2- 172.21.15.123:0/426739838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1374076720 0x7f1374076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.688+0000 7f137a000640 1 -- 172.21.15.123:0/426739838 >> 172.21.15.123:0/426739838 conn(0x7f13740ffa90 msgr2=0x7f1374101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.688+0000 7f137a000640 1 -- 172.21.15.123:0/426739838 shutdown_connections 2024-09-19T04:44:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.688+0000 7f137a000640 1 -- 172.21.15.123:0/426739838 wait complete. 2024-09-19T04:44:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.689+0000 7f137a000640 1 Processor -- start 2024-09-19T04:44:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.689+0000 7f137a000640 1 -- start start 2024-09-19T04:44:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.689+0000 7f137a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1374076720 0x7f13741a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.690+0000 7f137a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1374077040 0x7f13741a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.690+0000 7f137a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1374112a30 0x7f137419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.690+0000 7f137a000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f13741185e0 con 0x7f1374077040 2024-09-19T04:44:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.690+0000 7f137a000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1374118460 con 0x7f1374076720 2024-09-19T04:44:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.690+0000 7f137a000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1374118760 con 0x7f1374112a30 2024-09-19T04:44:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.690+0000 7f1372a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1374077040 0x7f13741a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.690+0000 7f1373400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1374076720 0x7f13741a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.690+0000 7f1372a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1374077040 0x7f13741a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53268/0 (socket says 172.21.15.123:53268) 2024-09-19T04:44:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.690+0000 7f1372a00640 1 -- 172.21.15.123:0/4125617694 learned_addr learned my addr 172.21.15.123:0/4125617694 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.690+0000 7f1373e00640 1 --2- 172.21.15.123:0/4125617694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1374112a30 0x7f137419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.690+0000 7f1372a00640 1 -- 172.21.15.123:0/4125617694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1374112a30 msgr2=0x7f137419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.690+0000 7f1372a00640 1 --2- 172.21.15.123:0/4125617694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1374112a30 0x7f137419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.690+0000 7f1372a00640 1 -- 172.21.15.123:0/4125617694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1374076720 msgr2=0x7f13741a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.690+0000 7f1372a00640 1 --2- 172.21.15.123:0/4125617694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1374076720 0x7f13741a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.690+0000 7f1372a00640 1 -- 172.21.15.123:0/4125617694 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137419efc0 con 0x7f1374077040 2024-09-19T04:44:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.690+0000 7f1373400640 1 --2- 172.21.15.123:0/4125617694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1374076720 0x7f13741a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:44:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.690+0000 7f1372a00640 1 --2- 172.21.15.123:0/4125617694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1374077040 0x7f13741a4c10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f136000e9f0 tx=0x7f136000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.691+0000 7f136be00640 1 -- 172.21.15.123:0/4125617694 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136000ccc0 con 0x7f1374077040 2024-09-19T04:44:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.691+0000 7f136be00640 1 -- 172.21.15.123:0/4125617694 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f136000ce60 con 0x7f1374077040 2024-09-19T04:44:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.691+0000 7f137a000640 1 -- 172.21.15.123:0/4125617694 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137419f2b0 con 0x7f1374077040 2024-09-19T04:44:04.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.691+0000 7f136be00640 1 -- 172.21.15.123:0/4125617694 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360010690 con 0x7f1374077040 2024-09-19T04:44:04.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.691+0000 7f137a000640 1 -- 172.21.15.123:0/4125617694 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13741ab5a0 con 0x7f1374077040 2024-09-19T04:44:04.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.692+0000 7f137a000640 1 -- 172.21.15.123:0/4125617694 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1374076b00 con 0x7f1374077040 2024-09-19T04:44:04.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.694+0000 7f136be00640 1 -- 172.21.15.123:0/4125617694 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f13600046c0 con 0x7f1374077040 2024-09-19T04:44:04.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.695+0000 7f136be00640 1 --2- 172.21.15.123:0/4125617694 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f13580a3240 0x7f13580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:04.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.696+0000 7f136be00640 1 -- 172.21.15.123:0/4125617694 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f13600cf870 con 0x7f1374077040 2024-09-19T04:44:04.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.696+0000 7f1373400640 1 --2- 172.21.15.123:0/4125617694 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f13580a3240 0x7f13580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:04.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.696+0000 7f1373400640 1 --2- 172.21.15.123:0/4125617694 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f13580a3240 0x7f13580a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f137419f9d0 tx=0x7f136400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:04.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.700+0000 7f136be00640 1 -- 172.21.15.123:0/4125617694 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f136009c990 con 0x7f1374077040 2024-09-19T04:44:04.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:04 smithi193 ceph-mon[35592]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:04.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:04 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:04.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:04 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:04.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:04 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:04.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:04 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:04.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:04 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:04.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:04 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:04.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.910+0000 7f137a000640 1 -- 172.21.15.123:0/4125617694 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f137419fdc0 con 0x7f1374077040 2024-09-19T04:44:04.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.911+0000 7f136be00640 1 -- 172.21.15.123:0/4125617694 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6400 (secure 0 0 0) 0x7f137419fdc0 con 0x7f1374077040 2024-09-19T04:44:04.913 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:44:04.913 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":8,"btime":"2024-09-19T04:44:04:396218+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T04:44:04.913 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 8 2024-09-19T04:44:04.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.914+0000 7f137a000640 1 -- 172.21.15.123:0/4125617694 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f13580a3240 msgr2=0x7f13580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:04.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.914+0000 7f137a000640 1 --2- 172.21.15.123:0/4125617694 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f13580a3240 0x7f13580a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f137419f9d0 tx=0x7f136400a2d0 comp rx=0 tx=0).stop 2024-09-19T04:44:04.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.914+0000 7f137a000640 1 -- 172.21.15.123:0/4125617694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1374077040 msgr2=0x7f13741a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:04.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.914+0000 7f137a000640 1 --2- 172.21.15.123:0/4125617694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1374077040 0x7f13741a4c10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f136000e9f0 tx=0x7f136000edf0 comp rx=0 tx=0).stop 2024-09-19T04:44:04.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.914+0000 7f137a000640 1 -- 172.21.15.123:0/4125617694 shutdown_connections 2024-09-19T04:44:04.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.914+0000 7f137a000640 1 --2- 172.21.15.123:0/4125617694 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f13580a3240 0x7f13580a5700 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:04.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.914+0000 7f137a000640 1 --2- 172.21.15.123:0/4125617694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1374112a30 0x7f137419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:04.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.914+0000 7f137a000640 1 --2- 172.21.15.123:0/4125617694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1374077040 0x7f13741a4c10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:04.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.914+0000 7f137a000640 1 --2- 172.21.15.123:0/4125617694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1374076720 0x7f13741a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:04.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.914+0000 7f137a000640 1 -- 172.21.15.123:0/4125617694 >> 172.21.15.123:0/4125617694 conn(0x7f13740ffa90 msgr2=0x7f1374101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:04.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.914+0000 7f137a000640 1 -- 172.21.15.123:0/4125617694 shutdown_connections 2024-09-19T04:44:04.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:04.914+0000 7f137a000640 1 -- 172.21.15.123:0/4125617694 wait complete. 2024-09-19T04:44:05.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:05 smithi123 ceph-mon[30511]: Deploying daemon mds.h on smithi183 2024-09-19T04:44:05.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:05 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] up:boot 2024-09-19T04:44:05.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:05 smithi123 ceph-mon[30511]: fsmap 7 up:standby 2024-09-19T04:44:05.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:05 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:44:05.596 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:05 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/4125617694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:05.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:05 smithi193 ceph-mon[35592]: Deploying daemon mds.h on smithi183 2024-09-19T04:44:05.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:05 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] up:boot 2024-09-19T04:44:05.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:05 smithi193 ceph-mon[35592]: fsmap 7 up:standby 2024-09-19T04:44:05.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:05 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:44:05.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:05 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/4125617694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:05.774 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:05 smithi183 ceph-mon[35333]: Deploying daemon mds.h on smithi183 2024-09-19T04:44:05.775 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:05 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] up:boot 2024-09-19T04:44:05.775 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:05 smithi183 ceph-mon[35333]: fsmap 7 up:standby 2024-09-19T04:44:05.775 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:05 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:44:05.775 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:05 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/4125617694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:06.100 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:44:05 smithi183 systemd[1]: Starting Ceph mds.h for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:44:06.445 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:06 smithi183 ceph-mon[35333]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:06.445 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:44:06 smithi183 podman[64885]: 2024-09-19 04:44:06.067913871 +0000 UTC m=+0.015594433 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:44:06.445 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:44:06 smithi183 podman[64885]: 2024-09-19 04:44:06.19198239 +0000 UTC m=+0.139662942 container create 7bb03127b5ddfbff49f30e57e6d7c2ebd267d7afaccb8b2bdc5e30664d7eb198 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T04:44:06.445 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:44:06 smithi183 podman[64885]: 2024-09-19 04:44:06.444871783 +0000 UTC m=+0.392552335 container init 7bb03127b5ddfbff49f30e57e6d7c2ebd267d7afaccb8b2bdc5e30664d7eb198 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-h, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T04:44:06.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:06 smithi123 ceph-mon[30511]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:06.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:06 smithi193 ceph-mon[35592]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:06.850 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:44:06 smithi183 podman[64885]: 2024-09-19 04:44:06.447931553 +0000 UTC m=+0.395612114 container start 7bb03127b5ddfbff49f30e57e6d7c2ebd267d7afaccb8b2bdc5e30664d7eb198 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-h, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:44:06.850 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:44:06 smithi183 ceph-mds[64905]: -- 172.21.15.183:0/1307288799 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dda2845860 con 0x55dda2822800 2024-09-19T04:44:06.850 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:44:06 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-h[64901]: starting mds.h at 2024-09-19T04:44:06.850 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:44:06 smithi183 bash[64885]: 7bb03127b5ddfbff49f30e57e6d7c2ebd267d7afaccb8b2bdc5e30664d7eb198 2024-09-19T04:44:06.850 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:44:06 smithi183 systemd[1]: Started Ceph mds.h for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:44:07.632 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:44:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:07 smithi183 ceph-mon[35333]: Deploying daemon mds.i on smithi193 2024-09-19T04:44:07.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:07 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] up:boot 2024-09-19T04:44:07.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:07 smithi183 ceph-mon[35333]: fsmap 8 up:standby 2024-09-19T04:44:07.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:07 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:44:07.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:07.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:07.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:07.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:07.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:07.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:07.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:07 smithi123 ceph-mon[30511]: Deploying daemon mds.i on smithi193 2024-09-19T04:44:07.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:07 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] up:boot 2024-09-19T04:44:07.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:07 smithi123 ceph-mon[30511]: fsmap 8 up:standby 2024-09-19T04:44:07.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:07 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:44:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:07 smithi193 ceph-mon[35592]: Deploying daemon mds.i on smithi193 2024-09-19T04:44:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:07 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] up:boot 2024-09-19T04:44:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:07 smithi193 ceph-mon[35592]: fsmap 8 up:standby 2024-09-19T04:44:07.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:07 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:44:08.006 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.549+0000 7f69f1a00640 1 -- 172.21.15.123:0/3366751167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ec102790 msgr2=0x7f69ec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.549+0000 7f69f1a00640 1 --2- 172.21.15.123:0/3366751167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ec102790 0x7f69ec1067e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009900 tx=0x7f69d802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:44:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.550+0000 7f69f1a00640 1 -- 172.21.15.123:0/3366751167 shutdown_connections 2024-09-19T04:44:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.550+0000 7f69f1a00640 1 --2- 172.21.15.123:0/3366751167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69ec106f10 0x7f69ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.550+0000 7f69f1a00640 1 --2- 172.21.15.123:0/3366751167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ec102790 0x7f69ec1067e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.550+0000 7f69f1a00640 1 --2- 172.21.15.123:0/3366751167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69ec101de0 0x7f69ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.550+0000 7f69f1a00640 1 -- 172.21.15.123:0/3366751167 >> 172.21.15.123:0/3366751167 conn(0x7f69ec0fd8c0 msgr2=0x7f69ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.550+0000 7f69f1a00640 1 -- 172.21.15.123:0/3366751167 shutdown_connections 2024-09-19T04:44:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.550+0000 7f69f1a00640 1 -- 172.21.15.123:0/3366751167 wait complete. 2024-09-19T04:44:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.551+0000 7f69f1a00640 1 Processor -- start 2024-09-19T04:44:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.551+0000 7f69f1a00640 1 -- start start 2024-09-19T04:44:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.551+0000 7f69f1a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ec101de0 0x7f69ec19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.551+0000 7f69f1a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69ec102790 0x7f69ec19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.551+0000 7f69f1a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69ec106f10 0x7f69ec19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.551+0000 7f69f1a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69ec111f80 con 0x7f69ec101de0 2024-09-19T04:44:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.551+0000 7f69f1a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69ec111e00 con 0x7f69ec106f10 2024-09-19T04:44:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.551+0000 7f69f1a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f69ec112100 con 0x7f69ec102790 2024-09-19T04:44:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.551+0000 7f69eaa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ec101de0 0x7f69ec19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.551+0000 7f69eaa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ec101de0 0x7f69ec19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53276/0 (socket says 172.21.15.123:53276) 2024-09-19T04:44:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.551+0000 7f69eaa00640 1 -- 172.21.15.123:0/4203611819 learned_addr learned my addr 172.21.15.123:0/4203611819 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.551+0000 7f69eb400640 1 --2- 172.21.15.123:0/4203611819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69ec106f10 0x7f69ec19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.551+0000 7f69ea000640 1 --2- 172.21.15.123:0/4203611819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69ec102790 0x7f69ec19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.552+0000 7f69eaa00640 1 -- 172.21.15.123:0/4203611819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69ec102790 msgr2=0x7f69ec19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.552+0000 7f69eaa00640 1 --2- 172.21.15.123:0/4203611819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69ec102790 0x7f69ec19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.552+0000 7f69eaa00640 1 -- 172.21.15.123:0/4203611819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69ec106f10 msgr2=0x7f69ec19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.552+0000 7f69eaa00640 1 --2- 172.21.15.123:0/4203611819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69ec106f10 0x7f69ec19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.552+0000 7f69eaa00640 1 -- 172.21.15.123:0/4203611819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69ec19f5f0 con 0x7f69ec101de0 2024-09-19T04:44:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.552+0000 7f69eb400640 1 --2- 172.21.15.123:0/4203611819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69ec106f10 0x7f69ec19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:44:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.552+0000 7f69eaa00640 1 --2- 172.21.15.123:0/4203611819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ec101de0 0x7f69ec19a640 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00d950 tx=0x7f69dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.552+0000 7f69e3400640 1 -- 172.21.15.123:0/4203611819 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc017030 con 0x7f69ec101de0 2024-09-19T04:44:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.552+0000 7f69e3400640 1 -- 172.21.15.123:0/4203611819 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69dc004020 con 0x7f69ec101de0 2024-09-19T04:44:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.552+0000 7f69f1a00640 1 -- 172.21.15.123:0/4203611819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69ec19f8e0 con 0x7f69ec101de0 2024-09-19T04:44:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.552+0000 7f69e3400640 1 -- 172.21.15.123:0/4203611819 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc0047a0 con 0x7f69ec101de0 2024-09-19T04:44:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.552+0000 7f69f1a00640 1 -- 172.21.15.123:0/4203611819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69ec1a71c0 con 0x7f69ec101de0 2024-09-19T04:44:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.554+0000 7f69f1a00640 1 -- 172.21.15.123:0/4203611819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69ec1021c0 con 0x7f69ec101de0 2024-09-19T04:44:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.556+0000 7f69e3400640 1 -- 172.21.15.123:0/4203611819 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f69dc0041d0 con 0x7f69ec101de0 2024-09-19T04:44:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.557+0000 7f69e3400640 1 --2- 172.21.15.123:0/4203611819 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f69d00a32c0 0x7f69d00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.557+0000 7f69e3400640 1 -- 172.21.15.123:0/4203611819 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f69dc0cf450 con 0x7f69ec101de0 2024-09-19T04:44:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.557+0000 7f69ea000640 1 --2- 172.21.15.123:0/4203611819 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f69d00a32c0 0x7f69d00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:08.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.558+0000 7f69ea000640 1 --2- 172.21.15.123:0/4203611819 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f69d00a32c0 0x7f69d00a5780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f69d8000c00 tx=0x7f69d8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:08.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.559+0000 7f69e3400640 1 -- 172.21.15.123:0/4203611819 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f69dc09c690 con 0x7f69ec101de0 2024-09-19T04:44:08.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.768+0000 7f69f1a00640 1 -- 172.21.15.123:0/4203611819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69ec1a7420 con 0x7f69ec101de0 2024-09-19T04:44:08.769 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:08 smithi123 ceph-mon[30511]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:08.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.768+0000 7f69e3400640 1 -- 172.21.15.123:0/4203611819 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7228 (secure 0 0 0) 0x7f69dc09bf90 con 0x7f69ec101de0 2024-09-19T04:44:08.770 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:44:08.770 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"btime":"2024-09-19T04:44:07:412136+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T04:44:08.770 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-09-19T04:44:08.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.771+0000 7f69f1a00640 1 -- 172.21.15.123:0/4203611819 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f69d00a32c0 msgr2=0x7f69d00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:08.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.771+0000 7f69f1a00640 1 --2- 172.21.15.123:0/4203611819 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f69d00a32c0 0x7f69d00a5780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f69d8000c00 tx=0x7f69d8005a20 comp rx=0 tx=0).stop 2024-09-19T04:44:08.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.771+0000 7f69f1a00640 1 -- 172.21.15.123:0/4203611819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ec101de0 msgr2=0x7f69ec19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:08.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.771+0000 7f69f1a00640 1 --2- 172.21.15.123:0/4203611819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ec101de0 0x7f69ec19a640 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00d950 tx=0x7f69dc00de10 comp rx=0 tx=0).stop 2024-09-19T04:44:08.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.771+0000 7f69f1a00640 1 -- 172.21.15.123:0/4203611819 shutdown_connections 2024-09-19T04:44:08.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.771+0000 7f69f1a00640 1 --2- 172.21.15.123:0/4203611819 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f69d00a32c0 0x7f69d00a5780 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:08.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.771+0000 7f69f1a00640 1 --2- 172.21.15.123:0/4203611819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69ec106f10 0x7f69ec19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:08.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.771+0000 7f69f1a00640 1 --2- 172.21.15.123:0/4203611819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69ec102790 0x7f69ec19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:08.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.771+0000 7f69f1a00640 1 --2- 172.21.15.123:0/4203611819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ec101de0 0x7f69ec19a640 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:08.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.771+0000 7f69f1a00640 1 -- 172.21.15.123:0/4203611819 >> 172.21.15.123:0/4203611819 conn(0x7f69ec0fd8c0 msgr2=0x7f69ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:08.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.772+0000 7f69f1a00640 1 -- 172.21.15.123:0/4203611819 shutdown_connections 2024-09-19T04:44:08.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:08.772+0000 7f69f1a00640 1 -- 172.21.15.123:0/4203611819 wait complete. 2024-09-19T04:44:08.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:08 smithi193 ceph-mon[35592]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:08.911 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 04:44:08 smithi193 systemd[1]: Starting Ceph mds.i for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:44:09.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:08 smithi183 ceph-mon[35333]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:09.197 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 04:44:08 smithi193 podman[65971]: 2024-09-19 04:44:08.81249673 +0000 UTC m=+0.015624768 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:44:09.197 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 04:44:08 smithi193 podman[65971]: 2024-09-19 04:44:08.943497272 +0000 UTC m=+0.146625353 container create 00b1c8207448b65c608ee3be07da182ced6fa2d63f1614a61114af18a74789ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-i, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:44:09.490 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 04:44:09 smithi193 podman[65971]: 2024-09-19 04:44:09.19663529 +0000 UTC m=+0.399763343 container init 00b1c8207448b65c608ee3be07da182ced6fa2d63f1614a61114af18a74789ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-i, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-19T04:44:09.491 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 04:44:09 smithi193 podman[65971]: 2024-09-19 04:44:09.19966926 +0000 UTC m=+0.402797300 container start 00b1c8207448b65c608ee3be07da182ced6fa2d63f1614a61114af18a74789ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-19T04:44:09.491 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 04:44:09 smithi193 ceph-mds[65992]: -- 172.21.15.193:0/331221851 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5618ebefb860 con 0x5618ebed9000 2024-09-19T04:44:09.491 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 04:44:09 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-i[65988]: starting mds.i at 2024-09-19T04:44:09.491 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 04:44:09 smithi193 bash[65971]: 00b1c8207448b65c608ee3be07da182ced6fa2d63f1614a61114af18a74789ad 2024-09-19T04:44:09.491 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 04:44:09 smithi193 systemd[1]: Started Ceph mds.i for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:44:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:09 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/4203611819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:09 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:09 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:09 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:09 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:09 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:09 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:09.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:09 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/4203611819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:09.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:09 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:09.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:09 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:09.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:09 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:09.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:09 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:09.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:09 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:09.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:09 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:09 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/4203611819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:09 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:09 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:09 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:09 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:09 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:09 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:10.914 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:10 smithi123 ceph-mon[30511]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:10.915 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:10 smithi123 ceph-mon[30511]: Deploying daemon mds.g on smithi123 2024-09-19T04:44:10.915 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:10 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] up:boot 2024-09-19T04:44:10.915 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:10 smithi123 ceph-mon[30511]: fsmap 9 up:standby 2024-09-19T04:44:10.915 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:10 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:44:10.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:10 smithi193 ceph-mon[35592]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:10.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:10 smithi193 ceph-mon[35592]: Deploying daemon mds.g on smithi123 2024-09-19T04:44:10.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:10 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] up:boot 2024-09-19T04:44:10.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:10 smithi193 ceph-mon[35592]: fsmap 9 up:standby 2024-09-19T04:44:10.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:10 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:44:11.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:10 smithi183 ceph-mon[35333]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:11.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:10 smithi183 ceph-mon[35333]: Deploying daemon mds.g on smithi123 2024-09-19T04:44:11.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:10 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] up:boot 2024-09-19T04:44:11.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:10 smithi183 ceph-mon[35333]: fsmap 9 up:standby 2024-09-19T04:44:11.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:10 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:44:11.474 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:44:11.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:11 smithi123 ceph-mon[30511]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:11.965 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:11.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:11 smithi193 ceph-mon[35592]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:12.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:11 smithi183 ceph-mon[35333]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:12.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.567+0000 7fcff7000640 1 -- 172.21.15.123:0/3848667049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff01047e0 msgr2=0x7fcff0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:12.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.567+0000 7fcff7000640 1 --2- 172.21.15.123:0/3848667049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff01047e0 0x7fcff0108830 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009900 tx=0x7fcfdc02a410 comp rx=0 tx=0).stop 2024-09-19T04:44:12.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.567+0000 7fcff7000640 1 -- 172.21.15.123:0/3848667049 shutdown_connections 2024-09-19T04:44:12.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.567+0000 7fcff7000640 1 --2- 172.21.15.123:0/3848667049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcff0108f60 0x7fcff0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:12.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.567+0000 7fcff7000640 1 --2- 172.21.15.123:0/3848667049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff01047e0 0x7fcff0108830 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:12.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.567+0000 7fcff7000640 1 --2- 172.21.15.123:0/3848667049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcff0103e30 0x7fcff0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:12.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.567+0000 7fcff7000640 1 -- 172.21.15.123:0/3848667049 >> 172.21.15.123:0/3848667049 conn(0x7fcff00ff8d0 msgr2=0x7fcff0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:12.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.567+0000 7fcff7000640 1 -- 172.21.15.123:0/3848667049 shutdown_connections 2024-09-19T04:44:12.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.567+0000 7fcff7000640 1 -- 172.21.15.123:0/3848667049 wait complete. 2024-09-19T04:44:12.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.568+0000 7fcff7000640 1 Processor -- start 2024-09-19T04:44:12.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.568+0000 7fcff7000640 1 -- start start 2024-09-19T04:44:12.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.569+0000 7fcff7000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcff0103e30 0x7fcff019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:12.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.569+0000 7fcff7000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcff01047e0 0x7fcff019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:12.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.569+0000 7fcff7000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff0108f60 0x7fcff019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:12.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.569+0000 7fcff7000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcff0114010 con 0x7fcff0108f60 2024-09-19T04:44:12.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.569+0000 7fcff7000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcff0113e90 con 0x7fcff0103e30 2024-09-19T04:44:12.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.569+0000 7fcff7000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcff0114190 con 0x7fcff01047e0 2024-09-19T04:44:12.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.569+0000 7fcff5c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcff0103e30 0x7fcff019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:12.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.569+0000 7fcff6600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff0108f60 0x7fcff019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:12.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.569+0000 7fcff5c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcff0103e30 0x7fcff019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:41500/0 (socket says 172.21.15.123:41500) 2024-09-19T04:44:12.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.569+0000 7fcff5c00640 1 -- 172.21.15.123:0/2677853921 learned_addr learned my addr 172.21.15.123:0/2677853921 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:12.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.569+0000 7fcff5200640 1 --2- 172.21.15.123:0/2677853921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcff01047e0 0x7fcff019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:12.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.570+0000 7fcff6600640 1 -- 172.21.15.123:0/2677853921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcff01047e0 msgr2=0x7fcff019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:12.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.570+0000 7fcff6600640 1 --2- 172.21.15.123:0/2677853921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcff01047e0 0x7fcff019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:12.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.570+0000 7fcff6600640 1 -- 172.21.15.123:0/2677853921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcff0103e30 msgr2=0x7fcff019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:12.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.570+0000 7fcff6600640 1 --2- 172.21.15.123:0/2677853921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcff0103e30 0x7fcff019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:12.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.570+0000 7fcff6600640 1 -- 172.21.15.123:0/2677853921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff019f460 con 0x7fcff0108f60 2024-09-19T04:44:12.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.570+0000 7fcff5c00640 1 --2- 172.21.15.123:0/2677853921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcff0103e30 0x7fcff019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:44:12.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.570+0000 7fcff6600640 1 --2- 172.21.15.123:0/2677853921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff0108f60 0x7fcff019ed80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800da10 tx=0x7fcfe800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:12.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.570+0000 7fcfee000640 1 -- 172.21.15.123:0/2677853921 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800bd70 con 0x7fcff0108f60 2024-09-19T04:44:12.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.570+0000 7fcfee000640 1 -- 172.21.15.123:0/2677853921 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfe8003e50 con 0x7fcff0108f60 2024-09-19T04:44:12.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.570+0000 7fcff7000640 1 -- 172.21.15.123:0/2677853921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff019f750 con 0x7fcff0108f60 2024-09-19T04:44:12.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.570+0000 7fcfee000640 1 -- 172.21.15.123:0/2677853921 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe80106a0 con 0x7fcff0108f60 2024-09-19T04:44:12.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.570+0000 7fcff7000640 1 -- 172.21.15.123:0/2677853921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff01a7030 con 0x7fcff0108f60 2024-09-19T04:44:12.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.571+0000 7fcff7000640 1 -- 172.21.15.123:0/2677853921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff0104210 con 0x7fcff0108f60 2024-09-19T04:44:12.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.573+0000 7fcfee000640 1 -- 172.21.15.123:0/2677853921 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fcfe8003970 con 0x7fcff0108f60 2024-09-19T04:44:12.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.574+0000 7fcfee000640 1 --2- 172.21.15.123:0/2677853921 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcfd80a3240 0x7fcfd80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:12.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.575+0000 7fcfee000640 1 -- 172.21.15.123:0/2677853921 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fcfe80cf850 con 0x7fcff0108f60 2024-09-19T04:44:12.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.575+0000 7fcff5c00640 1 --2- 172.21.15.123:0/2677853921 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcfd80a3240 0x7fcfd80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:12.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.575+0000 7fcff5c00640 1 --2- 172.21.15.123:0/2677853921 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcfd80a3240 0x7fcfd80a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4005fd0 tx=0x7fcfe400b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:12.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.579+0000 7fcfee000640 1 -- 172.21.15.123:0/2677853921 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcfe809c970 con 0x7fcff0108f60 2024-09-19T04:44:12.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.807+0000 7fcff7000640 1 -- 172.21.15.123:0/2677853921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcff010bbb0 con 0x7fcff0108f60 2024-09-19T04:44:12.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.807+0000 7fcfee000640 1 -- 172.21.15.123:0/2677853921 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8058 (secure 0 0 0) 0x7fcfe809c270 con 0x7fcff0108f60 2024-09-19T04:44:12.809 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:44:12.809 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":10,"btime":"2024-09-19T04:44:09:612460+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T04:44:12.809 INFO:journalctl@ceph.mds.g.smithi123.stdout:Sep 19 04:44:12 smithi123 podman[70539]: 2024-09-19 04:44:12.757968705 +0000 UTC m=+0.145805488 container create bfae55e6fe727ee0b2fa72c6d6b30be7332ef7dd252d3f06c6bdc88ffa847f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-g, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:44:12.810 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 10 2024-09-19T04:44:12.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.810+0000 7fcff7000640 1 -- 172.21.15.123:0/2677853921 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcfd80a3240 msgr2=0x7fcfd80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:12.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.810+0000 7fcff7000640 1 --2- 172.21.15.123:0/2677853921 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcfd80a3240 0x7fcfd80a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4005fd0 tx=0x7fcfe400b4a0 comp rx=0 tx=0).stop 2024-09-19T04:44:12.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.810+0000 7fcff7000640 1 -- 172.21.15.123:0/2677853921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff0108f60 msgr2=0x7fcff019ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:12.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.810+0000 7fcff7000640 1 --2- 172.21.15.123:0/2677853921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff0108f60 0x7fcff019ed80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800da10 tx=0x7fcfe800ded0 comp rx=0 tx=0).stop 2024-09-19T04:44:12.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.810+0000 7fcff7000640 1 -- 172.21.15.123:0/2677853921 shutdown_connections 2024-09-19T04:44:12.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.810+0000 7fcff7000640 1 --2- 172.21.15.123:0/2677853921 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcfd80a3240 0x7fcfd80a5700 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:12.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.810+0000 7fcff7000640 1 --2- 172.21.15.123:0/2677853921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff0108f60 0x7fcff019ed80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:12.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.810+0000 7fcff7000640 1 --2- 172.21.15.123:0/2677853921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcff01047e0 0x7fcff019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:12.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.810+0000 7fcff7000640 1 --2- 172.21.15.123:0/2677853921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcff0103e30 0x7fcff019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:12.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.810+0000 7fcff7000640 1 -- 172.21.15.123:0/2677853921 >> 172.21.15.123:0/2677853921 conn(0x7fcff00ff8d0 msgr2=0x7fcff0100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:12.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.810+0000 7fcff7000640 1 -- 172.21.15.123:0/2677853921 shutdown_connections 2024-09-19T04:44:12.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:12.810+0000 7fcff7000640 1 -- 172.21.15.123:0/2677853921 wait complete. 2024-09-19T04:44:13.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:12 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2677853921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:13.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:12 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2677853921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:13.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:12 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2677853921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:13.677 INFO:journalctl@ceph.mds.g.smithi123.stdout:Sep 19 04:44:13 smithi123 podman[70539]: 2024-09-19 04:44:13.363446436 +0000 UTC m=+0.751283220 container init bfae55e6fe727ee0b2fa72c6d6b30be7332ef7dd252d3f06c6bdc88ffa847f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-g, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T04:44:13.677 INFO:journalctl@ceph.mds.g.smithi123.stdout:Sep 19 04:44:13 smithi123 podman[70539]: 2024-09-19 04:44:13.3698146 +0000 UTC m=+0.757651381 container start bfae55e6fe727ee0b2fa72c6d6b30be7332ef7dd252d3f06c6bdc88ffa847f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-g, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T04:44:13.677 INFO:journalctl@ceph.mds.g.smithi123.stdout:Sep 19 04:44:13 smithi123 ceph-mds[70571]: -- 172.21.15.123:0/3424043806 <== mon.0 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608a98ef860 con 0x5608a98cd000 2024-09-19T04:44:13.677 INFO:journalctl@ceph.mds.g.smithi123.stdout:Sep 19 04:44:13 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-g[70566]: starting mds.g at 2024-09-19T04:44:13.677 INFO:journalctl@ceph.mds.g.smithi123.stdout:Sep 19 04:44:13 smithi123 bash[70539]: bfae55e6fe727ee0b2fa72c6d6b30be7332ef7dd252d3f06c6bdc88ffa847f08 2024-09-19T04:44:13.677 INFO:journalctl@ceph.mds.g.smithi123.stdout:Sep 19 04:44:13 smithi123 systemd[1]: Started Ceph mds.g for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:44:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:14 smithi183 ceph-mon[35333]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:14 smithi183 ceph-mon[35333]: Deploying daemon mds.a on smithi123 2024-09-19T04:44:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:14 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.123:6838/4146531731,v1:172.21.15.123:6839/4146531731] up:boot 2024-09-19T04:44:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:14 smithi183 ceph-mon[35333]: fsmap 10 up:standby 2024-09-19T04:44:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:44:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:14 smithi123 ceph-mon[30511]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:14.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:14 smithi123 ceph-mon[30511]: Deploying daemon mds.a on smithi123 2024-09-19T04:44:14.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:14 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.123:6838/4146531731,v1:172.21.15.123:6839/4146531731] up:boot 2024-09-19T04:44:14.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:14 smithi123 ceph-mon[30511]: fsmap 10 up:standby 2024-09-19T04:44:14.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:44:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:14 smithi193 ceph-mon[35592]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:14 smithi193 ceph-mon[35592]: Deploying daemon mds.a on smithi123 2024-09-19T04:44:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:14 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.123:6838/4146531731,v1:172.21.15.123:6839/4146531731] up:boot 2024-09-19T04:44:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:14 smithi193 ceph-mon[35592]: fsmap 10 up:standby 2024-09-19T04:44:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:44:15.823 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:44:16.318 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:44:15 smithi123 systemd[1]: Starting Ceph mds.a for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:44:16.672 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:16 smithi123 ceph-mon[30511]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:16.673 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:44:16 smithi123 podman[70876]: 2024-09-19 04:44:16.221231613 +0000 UTC m=+0.034088354 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:44:16.673 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:44:16 smithi123 podman[70876]: 2024-09-19 04:44:16.410798674 +0000 UTC m=+0.223655404 container create 87f469624551f4ec3964683d8d13d595ff02a215d16ee6957ed9f9730a4ff126 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-19T04:44:16.673 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:44:16 smithi123 podman[70876]: 2024-09-19 04:44:16.672075776 +0000 UTC m=+0.484932537 container init 87f469624551f4ec3964683d8d13d595ff02a215d16ee6957ed9f9730a4ff126 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T04:44:16.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:16 smithi183 ceph-mon[35333]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:16.927 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:44:16 smithi123 podman[70876]: 2024-09-19 04:44:16.678374236 +0000 UTC m=+0.491230968 container start 87f469624551f4ec3964683d8d13d595ff02a215d16ee6957ed9f9730a4ff126 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-a, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:44:16.927 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:44:16 smithi123 ceph-mds[70911]: -- 172.21.15.123:0/592981547 <== mon.0 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56393e935c20 con 0x56393e912800 2024-09-19T04:44:16.927 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:44:16 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-a[70907]: starting mds.a at 2024-09-19T04:44:16.927 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:44:16 smithi123 bash[70876]: 87f469624551f4ec3964683d8d13d595ff02a215d16ee6957ed9f9730a4ff126 2024-09-19T04:44:16.927 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:44:16 smithi123 systemd[1]: Started Ceph mds.a for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:44:16.941 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:16.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:16 smithi193 ceph-mon[35592]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:17.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.512+0000 7fb043800640 1 -- 172.21.15.123:0/2160268609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb03c074f40 msgr2=0x7fb03c10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:17.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.512+0000 7fb043800640 1 --2- 172.21.15.123:0/2160268609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb03c074f40 0x7fb03c10de80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb024009900 tx=0x7fb02402a480 comp rx=0 tx=0).stop 2024-09-19T04:44:17.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.513+0000 7fb043800640 1 -- 172.21.15.123:0/2160268609 shutdown_connections 2024-09-19T04:44:17.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.513+0000 7fb043800640 1 --2- 172.21.15.123:0/2160268609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb03c10e3c0 0x7fb03c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:17.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.513+0000 7fb043800640 1 --2- 172.21.15.123:0/2160268609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb03c074f40 0x7fb03c10de80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:17.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.513+0000 7fb043800640 1 --2- 172.21.15.123:0/2160268609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb03c074620 0x7fb03c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:17.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.513+0000 7fb043800640 1 -- 172.21.15.123:0/2160268609 >> 172.21.15.123:0/2160268609 conn(0x7fb03c0fd8c0 msgr2=0x7fb03c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:17.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.513+0000 7fb043800640 1 -- 172.21.15.123:0/2160268609 shutdown_connections 2024-09-19T04:44:17.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.513+0000 7fb043800640 1 -- 172.21.15.123:0/2160268609 wait complete. 2024-09-19T04:44:17.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.514+0000 7fb043800640 1 Processor -- start 2024-09-19T04:44:17.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.514+0000 7fb043800640 1 -- start start 2024-09-19T04:44:17.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.514+0000 7fb043800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb03c074620 0x7fb03c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:17.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.514+0000 7fb043800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb03c074f40 0x7fb03c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:17.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.514+0000 7fb043800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb03c10e3c0 0x7fb03c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:17.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.514+0000 7fb043800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c114140 con 0x7fb03c10e3c0 2024-09-19T04:44:17.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.514+0000 7fb043800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c113fc0 con 0x7fb03c074f40 2024-09-19T04:44:17.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.514+0000 7fb043800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c1142c0 con 0x7fb03c074620 2024-09-19T04:44:17.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.514+0000 7fb033e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb03c074f40 0x7fb03c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:17.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.514+0000 7fb040e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb03c074620 0x7fb03c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:17.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.514+0000 7fb033e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb03c074f40 0x7fb03c19aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:41534/0 (socket says 172.21.15.123:41534) 2024-09-19T04:44:17.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.514+0000 7fb033e00640 1 -- 172.21.15.123:0/1761045286 learned_addr learned my addr 172.21.15.123:0/1761045286 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:17.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.514+0000 7fb041800640 1 --2- 172.21.15.123:0/1761045286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb03c10e3c0 0x7fb03c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:17.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.515+0000 7fb040e00640 1 -- 172.21.15.123:0/1761045286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb03c074f40 msgr2=0x7fb03c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:17.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.515+0000 7fb040e00640 1 --2- 172.21.15.123:0/1761045286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb03c074f40 0x7fb03c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:17.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.515+0000 7fb040e00640 1 -- 172.21.15.123:0/1761045286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb03c10e3c0 msgr2=0x7fb03c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:17.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.515+0000 7fb040e00640 1 --2- 172.21.15.123:0/1761045286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb03c10e3c0 0x7fb03c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:17.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.515+0000 7fb040e00640 1 -- 172.21.15.123:0/1761045286 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03c19f610 con 0x7fb03c074620 2024-09-19T04:44:17.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.515+0000 7fb033e00640 1 --2- 172.21.15.123:0/1761045286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb03c074f40 0x7fb03c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:44:17.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.515+0000 7fb040e00640 1 --2- 172.21.15.123:0/1761045286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb03c074620 0x7fb03c19a660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb02c005fd0 tx=0x7fb02c00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:17.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.515+0000 7fb03a000640 1 -- 172.21.15.123:0/1761045286 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c009c30 con 0x7fb03c074620 2024-09-19T04:44:17.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.516+0000 7fb043800640 1 -- 172.21.15.123:0/1761045286 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03c19f900 con 0x7fb03c074620 2024-09-19T04:44:17.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.516+0000 7fb03a000640 1 -- 172.21.15.123:0/1761045286 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb02c009dd0 con 0x7fb03c074620 2024-09-19T04:44:17.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.516+0000 7fb03a000640 1 -- 172.21.15.123:0/1761045286 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c010460 con 0x7fb03c074620 2024-09-19T04:44:17.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.516+0000 7fb043800640 1 -- 172.21.15.123:0/1761045286 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03c1a71e0 con 0x7fb03c074620 2024-09-19T04:44:17.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.517+0000 7fb043800640 1 -- 172.21.15.123:0/1761045286 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb03c074a00 con 0x7fb03c074620 2024-09-19T04:44:17.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.519+0000 7fb03a000640 1 -- 172.21.15.123:0/1761045286 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb02c003f50 con 0x7fb03c074620 2024-09-19T04:44:17.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.520+0000 7fb03a000640 1 --2- 172.21.15.123:0/1761045286 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fb0200a3240 0x7fb0200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:17.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.520+0000 7fb03a000640 1 -- 172.21.15.123:0/1761045286 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fb02c0cf400 con 0x7fb03c074620 2024-09-19T04:44:17.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.520+0000 7fb033e00640 1 --2- 172.21.15.123:0/1761045286 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fb0200a3240 0x7fb0200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:17.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.521+0000 7fb033e00640 1 --2- 172.21.15.123:0/1761045286 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fb0200a3240 0x7fb0200a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb024031550 tx=0x7fb024005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:17.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.522+0000 7fb03a000640 1 -- 172.21.15.123:0/1761045286 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb02c017020 con 0x7fb03c074620 2024-09-19T04:44:17.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.735+0000 7fb043800640 1 -- 172.21.15.123:0/1761045286 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb03c19b9c0 con 0x7fb03c074620 2024-09-19T04:44:17.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.736+0000 7fb03a000640 1 -- 172.21.15.123:0/1761045286 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9713 (secure 0 0 0) 0x7fb03c19b9c0 con 0x7fb03c074620 2024-09-19T04:44:17.738 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:44:17.738 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":12,"btime":"2024-09-19T04:44:17:532961+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T04:44:17.739 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 12 2024-09-19T04:44:17.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.739+0000 7fb043800640 1 -- 172.21.15.123:0/1761045286 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fb0200a3240 msgr2=0x7fb0200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:17.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.739+0000 7fb043800640 1 --2- 172.21.15.123:0/1761045286 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fb0200a3240 0x7fb0200a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb024031550 tx=0x7fb024005a20 comp rx=0 tx=0).stop 2024-09-19T04:44:17.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.739+0000 7fb043800640 1 -- 172.21.15.123:0/1761045286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb03c074620 msgr2=0x7fb03c19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:17.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.739+0000 7fb043800640 1 --2- 172.21.15.123:0/1761045286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb03c074620 0x7fb03c19a660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb02c005fd0 tx=0x7fb02c00dde0 comp rx=0 tx=0).stop 2024-09-19T04:44:17.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.739+0000 7fb043800640 1 -- 172.21.15.123:0/1761045286 shutdown_connections 2024-09-19T04:44:17.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.739+0000 7fb043800640 1 --2- 172.21.15.123:0/1761045286 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fb0200a3240 0x7fb0200a5700 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:17.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.739+0000 7fb043800640 1 --2- 172.21.15.123:0/1761045286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb03c10e3c0 0x7fb03c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:17.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.739+0000 7fb043800640 1 --2- 172.21.15.123:0/1761045286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb03c074f40 0x7fb03c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:17.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.739+0000 7fb043800640 1 --2- 172.21.15.123:0/1761045286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb03c074620 0x7fb03c19a660 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:17.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.739+0000 7fb043800640 1 -- 172.21.15.123:0/1761045286 >> 172.21.15.123:0/1761045286 conn(0x7fb03c0fd8c0 msgr2=0x7fb03c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:17.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.739+0000 7fb043800640 1 -- 172.21.15.123:0/1761045286 shutdown_connections 2024-09-19T04:44:17.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:17.739+0000 7fb043800640 1 -- 172.21.15.123:0/1761045286 wait complete. 2024-09-19T04:44:17.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:17.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:17.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:17.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:17.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:17.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:17.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:17 smithi183 ceph-mon[35333]: Deploying daemon mds.b on smithi183 2024-09-19T04:44:17.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:17 smithi183 ceph-mon[35333]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:17.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:17 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.123:6840/784762674,v1:172.21.15.123:6841/784762674] up:boot 2024-09-19T04:44:17.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:17 smithi183 ceph-mon[35333]: fsmap 11 up:standby 2024-09-19T04:44:17.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:17 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:44:17.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:17 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1761045286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:18.090 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:18.090 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:18.090 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:18.090 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:18.090 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:18.091 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:18.091 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:17 smithi123 ceph-mon[30511]: Deploying daemon mds.b on smithi183 2024-09-19T04:44:18.091 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:17 smithi123 ceph-mon[30511]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:18.091 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:17 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.123:6840/784762674,v1:172.21.15.123:6841/784762674] up:boot 2024-09-19T04:44:18.091 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:17 smithi123 ceph-mon[30511]: fsmap 11 up:standby 2024-09-19T04:44:18.091 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:17 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:44:18.091 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:17 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1761045286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:18.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:18.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:18.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:18.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:44:18.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:44:18.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:18.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:17 smithi193 ceph-mon[35592]: Deploying daemon mds.b on smithi183 2024-09-19T04:44:18.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:17 smithi193 ceph-mon[35592]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:18.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:17 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.123:6840/784762674,v1:172.21.15.123:6841/784762674] up:boot 2024-09-19T04:44:18.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:17 smithi193 ceph-mon[35592]: fsmap 11 up:standby 2024-09-19T04:44:18.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:17 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:44:18.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:17 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1761045286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:18.850 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:44:18 smithi183 systemd[1]: Starting Ceph mds.b for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:44:19.224 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:44:18 smithi183 podman[65188]: 2024-09-19 04:44:18.860632303 +0000 UTC m=+0.016758764 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:44:19.224 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:44:18 smithi183 podman[65188]: 2024-09-19 04:44:18.979225739 +0000 UTC m=+0.135352193 container create 711b223d35df9c7e8dc222a60d43d305984c48f35f2156e7de797729fe0d4f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-b, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T04:44:19.224 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:44:19 smithi183 podman[65188]: 2024-09-19 04:44:19.223641256 +0000 UTC m=+0.379767723 container init 711b223d35df9c7e8dc222a60d43d305984c48f35f2156e7de797729fe0d4f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-b, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:44:19.600 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:44:19 smithi183 podman[65188]: 2024-09-19 04:44:19.226999769 +0000 UTC m=+0.383126225 container start 711b223d35df9c7e8dc222a60d43d305984c48f35f2156e7de797729fe0d4f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-b, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-19T04:44:19.600 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:44:19 smithi183 ceph-mds[65207]: -- 172.21.15.183:0/3287981407 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5609076fd860 con 0x5609076db000 2024-09-19T04:44:19.600 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:44:19 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-b[65203]: starting mds.b at 2024-09-19T04:44:19.600 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:44:19 smithi183 bash[65188]: 711b223d35df9c7e8dc222a60d43d305984c48f35f2156e7de797729fe0d4f04 2024-09-19T04:44:19.600 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:44:19 smithi183 systemd[1]: Started Ceph mds.b for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:44:20.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:20 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] up:boot 2024-09-19T04:44:20.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:20 smithi193 ceph-mon[35592]: fsmap 12 up:standby 2024-09-19T04:44:20.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:20 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:44:20.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:20 smithi193 ceph-mon[35592]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:20.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:20 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:20.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:20 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:20.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:20 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:20.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:20 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:20.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:20 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:20.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:20 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:44:20.528 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:44:20.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:20 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] up:boot 2024-09-19T04:44:20.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:20 smithi123 ceph-mon[30511]: fsmap 12 up:standby 2024-09-19T04:44:20.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:20 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:44:20.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:20 smithi123 ceph-mon[30511]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:20.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:20 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:20.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:20 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:20.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:20 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:20.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:20 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:20.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:20 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:20.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:20 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:44:20.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:20 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] up:boot 2024-09-19T04:44:20.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:20 smithi183 ceph-mon[35333]: fsmap 12 up:standby 2024-09-19T04:44:20.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:20 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:44:20.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:20 smithi183 ceph-mon[35333]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:20.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:20 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:20.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:20 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:20.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:20 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:20.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:20 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:20.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:20 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:20.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:20 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:44:20.968 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:21.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.590+0000 7f7df2400640 1 -- 172.21.15.123:0/3931630058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dec1049b0 msgr2=0x7f7dec108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:21.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.590+0000 7f7df2400640 1 --2- 172.21.15.123:0/3931630058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dec1049b0 0x7f7dec108a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8009860 tx=0x7f7dd802a340 comp rx=0 tx=0).stop 2024-09-19T04:44:21.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.590+0000 7f7df2400640 1 -- 172.21.15.123:0/3931630058 shutdown_connections 2024-09-19T04:44:21.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.590+0000 7f7df2400640 1 --2- 172.21.15.123:0/3931630058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dec109130 0x7f7dec110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:21.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.590+0000 7f7df2400640 1 --2- 172.21.15.123:0/3931630058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dec1049b0 0x7f7dec108a00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:21.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.590+0000 7f7df2400640 1 --2- 172.21.15.123:0/3931630058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7dec104000 0x7f7dec1043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:21.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.590+0000 7f7df2400640 1 -- 172.21.15.123:0/3931630058 >> 172.21.15.123:0/3931630058 conn(0x7f7dec0ffa20 msgr2=0x7f7dec101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:21.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.590+0000 7f7df2400640 1 -- 172.21.15.123:0/3931630058 shutdown_connections 2024-09-19T04:44:21.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.591+0000 7f7df2400640 1 -- 172.21.15.123:0/3931630058 wait complete. 2024-09-19T04:44:21.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.591+0000 7f7df2400640 1 Processor -- start 2024-09-19T04:44:21.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.592+0000 7f7df2400640 1 -- start start 2024-09-19T04:44:21.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.592+0000 7f7df2400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7dec104000 0x7f7dec19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:21.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.592+0000 7f7df2400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dec1049b0 0x7f7dec19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:21.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.592+0000 7f7df2400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dec109130 0x7f7dec19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:21.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.592+0000 7f7df2400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec114130 con 0x7f7dec109130 2024-09-19T04:44:21.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.592+0000 7f7df2400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec113fb0 con 0x7f7dec1049b0 2024-09-19T04:44:21.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.592+0000 7f7df2400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec1142b0 con 0x7f7dec104000 2024-09-19T04:44:21.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.592+0000 7f7deaa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dec1049b0 0x7f7dec19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:21.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.592+0000 7f7deaa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dec1049b0 0x7f7dec19ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:41552/0 (socket says 172.21.15.123:41552) 2024-09-19T04:44:21.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.592+0000 7f7deaa00640 1 -- 172.21.15.123:0/462904130 learned_addr learned my addr 172.21.15.123:0/462904130 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:21.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.592+0000 7f7deb400640 1 --2- 172.21.15.123:0/462904130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7dec104000 0x7f7dec19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:21.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.592+0000 7f7debe00640 1 --2- 172.21.15.123:0/462904130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dec109130 0x7f7dec19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:21.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.593+0000 7f7deaa00640 1 -- 172.21.15.123:0/462904130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7dec104000 msgr2=0x7f7dec19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:21.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.593+0000 7f7deaa00640 1 --2- 172.21.15.123:0/462904130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7dec104000 0x7f7dec19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:21.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.593+0000 7f7deaa00640 1 -- 172.21.15.123:0/462904130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dec109130 msgr2=0x7f7dec19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:21.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.593+0000 7f7deaa00640 1 --2- 172.21.15.123:0/462904130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dec109130 0x7f7dec19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:21.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.593+0000 7f7deaa00640 1 -- 172.21.15.123:0/462904130 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dec19f6e0 con 0x7f7dec1049b0 2024-09-19T04:44:21.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.593+0000 7f7debe00640 1 --2- 172.21.15.123:0/462904130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dec109130 0x7f7dec19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:44:21.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.593+0000 7f7deaa00640 1 --2- 172.21.15.123:0/462904130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dec1049b0 0x7f7dec19ac70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8035040 tx=0x7f7dd8033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:21.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.593+0000 7f7de3e00640 1 -- 172.21.15.123:0/462904130 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8037040 con 0x7f7dec1049b0 2024-09-19T04:44:21.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.593+0000 7f7de3e00640 1 -- 172.21.15.123:0/462904130 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dd8040070 con 0x7f7dec1049b0 2024-09-19T04:44:21.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.593+0000 7f7de3e00640 1 -- 172.21.15.123:0/462904130 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80039b0 con 0x7f7dec1049b0 2024-09-19T04:44:21.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.593+0000 7f7df2400640 1 -- 172.21.15.123:0/462904130 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dec19f970 con 0x7f7dec1049b0 2024-09-19T04:44:21.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.593+0000 7f7df2400640 1 -- 172.21.15.123:0/462904130 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dec1a72b0 con 0x7f7dec1049b0 2024-09-19T04:44:21.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.594+0000 7f7df2400640 1 -- 172.21.15.123:0/462904130 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dec1043e0 con 0x7f7dec1049b0 2024-09-19T04:44:21.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.597+0000 7f7de3e00640 1 -- 172.21.15.123:0/462904130 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f7dd80305e0 con 0x7f7dec1049b0 2024-09-19T04:44:21.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.598+0000 7f7de3e00640 1 --2- 172.21.15.123:0/462904130 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7dd00a32c0 0x7f7dd00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:21.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.598+0000 7f7de3e00640 1 -- 172.21.15.123:0/462904130 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f7dd80ef4c0 con 0x7f7dec1049b0 2024-09-19T04:44:21.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.598+0000 7f7deb400640 1 --2- 172.21.15.123:0/462904130 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7dd00a32c0 0x7f7dd00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:21.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.599+0000 7f7deb400640 1 --2- 172.21.15.123:0/462904130 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7dd00a32c0 0x7f7dd00a5780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009bd0 tx=0x7f7ddc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:21.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.602+0000 7f7de3e00640 1 -- 172.21.15.123:0/462904130 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7dd80bc5e0 con 0x7f7dec1049b0 2024-09-19T04:44:21.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.827+0000 7f7df2400640 1 -- 172.21.15.123:0/462904130 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dec10c370 con 0x7f7dec1049b0 2024-09-19T04:44:21.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.827+0000 7f7de3e00640 1 -- 172.21.15.123:0/462904130 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10539 (secure 0 0 0) 0x7f7dd80bbee0 con 0x7f7dec1049b0 2024-09-19T04:44:21.829 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:44:21.829 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":13,"btime":"2024-09-19T04:44:19:305488+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24539,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-19T04:44:21.830 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 13 2024-09-19T04:44:21.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.830+0000 7f7df2400640 1 -- 172.21.15.123:0/462904130 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7dd00a32c0 msgr2=0x7f7dd00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:21.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.830+0000 7f7df2400640 1 --2- 172.21.15.123:0/462904130 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7dd00a32c0 0x7f7dd00a5780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009bd0 tx=0x7f7ddc005f90 comp rx=0 tx=0).stop 2024-09-19T04:44:21.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.830+0000 7f7df2400640 1 -- 172.21.15.123:0/462904130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dec1049b0 msgr2=0x7f7dec19ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:21.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.830+0000 7f7df2400640 1 --2- 172.21.15.123:0/462904130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dec1049b0 0x7f7dec19ac70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8035040 tx=0x7f7dd8033b60 comp rx=0 tx=0).stop 2024-09-19T04:44:21.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.831+0000 7f7df2400640 1 -- 172.21.15.123:0/462904130 shutdown_connections 2024-09-19T04:44:21.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.831+0000 7f7df2400640 1 --2- 172.21.15.123:0/462904130 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7dd00a32c0 0x7f7dd00a5780 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:21.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.831+0000 7f7df2400640 1 --2- 172.21.15.123:0/462904130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dec109130 0x7f7dec19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:21.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.831+0000 7f7df2400640 1 --2- 172.21.15.123:0/462904130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dec1049b0 0x7f7dec19ac70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:21.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.831+0000 7f7df2400640 1 --2- 172.21.15.123:0/462904130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7dec104000 0x7f7dec19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:21.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.831+0000 7f7df2400640 1 -- 172.21.15.123:0/462904130 >> 172.21.15.123:0/462904130 conn(0x7f7dec0ffa20 msgr2=0x7f7dec100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:21.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.831+0000 7f7df2400640 1 -- 172.21.15.123:0/462904130 shutdown_connections 2024-09-19T04:44:21.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:21.831+0000 7f7df2400640 1 -- 172.21.15.123:0/462904130 wait complete. 2024-09-19T04:44:22.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:22.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:22.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:22.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:22 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:22.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:22 smithi183 ceph-mon[35333]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:22.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:22 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/462904130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:22.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:22.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:22.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:22.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:22 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:22.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:22 smithi123 ceph-mon[30511]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:22.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:22 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/462904130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:22.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:22.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:22.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:22.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:22 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:22.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:22 smithi193 ceph-mon[35592]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:22.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:22 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/462904130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:22.581 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-19T04:44:22.974 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:23.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.549+0000 7f2f8b200640 1 -- 172.21.15.123:0/1853038502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f8c103e30 msgr2=0x7f2f8c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:23.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.549+0000 7f2f8b200640 1 --2- 172.21.15.123:0/1853038502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f8c103e30 0x7f2f8c104210 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009900 tx=0x7f2f7c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:44:23.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.549+0000 7f2f8b200640 1 -- 172.21.15.123:0/1853038502 shutdown_connections 2024-09-19T04:44:23.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.549+0000 7f2f8b200640 1 --2- 172.21.15.123:0/1853038502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f8c108f60 0x7f2f8c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:23.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.549+0000 7f2f8b200640 1 --2- 172.21.15.123:0/1853038502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f8c1047e0 0x7f2f8c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:23.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.549+0000 7f2f8b200640 1 --2- 172.21.15.123:0/1853038502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f8c103e30 0x7f2f8c104210 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:23.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.549+0000 7f2f8b200640 1 -- 172.21.15.123:0/1853038502 >> 172.21.15.123:0/1853038502 conn(0x7f2f8c0ff910 msgr2=0x7f2f8c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:23.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.549+0000 7f2f8b200640 1 -- 172.21.15.123:0/1853038502 shutdown_connections 2024-09-19T04:44:23.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.549+0000 7f2f8b200640 1 -- 172.21.15.123:0/1853038502 wait complete. 2024-09-19T04:44:23.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.550+0000 7f2f8b200640 1 Processor -- start 2024-09-19T04:44:23.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.550+0000 7f2f8b200640 1 -- start start 2024-09-19T04:44:23.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.550+0000 7f2f8b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f8c103e30 0x7f2f8c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:23.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.550+0000 7f2f8b200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f8c1047e0 0x7f2f8c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:23.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.550+0000 7f2f8b200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f8c108f60 0x7f2f8c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:23.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.550+0000 7f2f8b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c114010 con 0x7f2f8c103e30 2024-09-19T04:44:23.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.550+0000 7f2f8b200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c113e90 con 0x7f2f8c108f60 2024-09-19T04:44:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.550+0000 7f2f8b200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c114190 con 0x7f2f8c1047e0 2024-09-19T04:44:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.551+0000 7f2f89e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f8c103e30 0x7f2f8c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.551+0000 7f2f89e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f8c103e30 0x7f2f8c19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49076/0 (socket says 172.21.15.123:49076) 2024-09-19T04:44:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.551+0000 7f2f89e00640 1 -- 172.21.15.123:0/3622397155 learned_addr learned my addr 172.21.15.123:0/3622397155 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.551+0000 7f2f8a800640 1 --2- 172.21.15.123:0/3622397155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f8c108f60 0x7f2f8c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.551+0000 7f2f89400640 1 --2- 172.21.15.123:0/3622397155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f8c1047e0 0x7f2f8c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.551+0000 7f2f89e00640 1 -- 172.21.15.123:0/3622397155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f8c1047e0 msgr2=0x7f2f8c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.551+0000 7f2f89e00640 1 --2- 172.21.15.123:0/3622397155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f8c1047e0 0x7f2f8c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.551+0000 7f2f89e00640 1 -- 172.21.15.123:0/3622397155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f8c108f60 msgr2=0x7f2f8c19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.551+0000 7f2f89e00640 1 --2- 172.21.15.123:0/3622397155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f8c108f60 0x7f2f8c19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.551+0000 7f2f89e00640 1 -- 172.21.15.123:0/3622397155 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8c19f4a0 con 0x7f2f8c103e30 2024-09-19T04:44:23.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.551+0000 7f2f89400640 1 --2- 172.21.15.123:0/3622397155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f8c1047e0 0x7f2f8c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:44:23.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.552+0000 7f2f89e00640 1 --2- 172.21.15.123:0/3622397155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f8c103e30 0x7f2f8c19a4f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c004770 tx=0x7f2f7c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:23.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.552+0000 7f2f82a00640 1 -- 172.21.15.123:0/3622397155 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c03f070 con 0x7f2f8c103e30 2024-09-19T04:44:23.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.552+0000 7f2f82a00640 1 -- 172.21.15.123:0/3622397155 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f7c031ca0 con 0x7f2f8c103e30 2024-09-19T04:44:23.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.552+0000 7f2f82a00640 1 -- 172.21.15.123:0/3622397155 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c035030 con 0x7f2f8c103e30 2024-09-19T04:44:23.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.552+0000 7f2f8b200640 1 -- 172.21.15.123:0/3622397155 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f8c19f730 con 0x7f2f8c103e30 2024-09-19T04:44:23.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.552+0000 7f2f8b200640 1 -- 172.21.15.123:0/3622397155 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f8c19fbc0 con 0x7f2f8c103e30 2024-09-19T04:44:23.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.553+0000 7f2f8b200640 1 -- 172.21.15.123:0/3622397155 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f8c105310 con 0x7f2f8c103e30 2024-09-19T04:44:23.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.555+0000 7f2f82a00640 1 -- 172.21.15.123:0/3622397155 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2f7c042070 con 0x7f2f8c103e30 2024-09-19T04:44:23.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.557+0000 7f2f82a00640 1 --2- 172.21.15.123:0/3622397155 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f2f700a32c0 0x7f2f700a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:23.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.557+0000 7f2f82a00640 1 -- 172.21.15.123:0/3622397155 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f2f7c0efa50 con 0x7f2f8c103e30 2024-09-19T04:44:23.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.557+0000 7f2f89400640 1 --2- 172.21.15.123:0/3622397155 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f2f700a32c0 0x7f2f700a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:23.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.558+0000 7f2f89400640 1 --2- 172.21.15.123:0/3622397155 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f2f700a32c0 0x7f2f700a5780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c0ffd50 tx=0x7f2f78005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:23.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.561+0000 7f2f82a00640 1 -- 172.21.15.123:0/3622397155 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2f7c0bc5e0 con 0x7f2f8c103e30 2024-09-19T04:44:23.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.772+0000 7f2f8b200640 1 -- 172.21.15.123:0/3622397155 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2f8c104210 con 0x7f2f8c103e30 2024-09-19T04:44:23.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.773+0000 7f2f82a00640 1 -- 172.21.15.123:0/3622397155 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) ==== 84+0+65 (secure 0 0 0) 0x7f2f7c0c08e0 con 0x7f2f8c103e30 2024-09-19T04:44:23.774 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:44:23.774 INFO:teuthology.orchestra.run.smithi123.stdout:[ 2024-09-19T04:44:23.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:44:23.774 INFO:teuthology.orchestra.run.smithi123.stdout: "poolnum": 1, 2024-09-19T04:44:23.775 INFO:teuthology.orchestra.run.smithi123.stdout: "poolname": ".mgr" 2024-09-19T04:44:23.775 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:44:23.775 INFO:teuthology.orchestra.run.smithi123.stdout:] 2024-09-19T04:44:23.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.776+0000 7f2f8b200640 1 -- 172.21.15.123:0/3622397155 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f2f700a32c0 msgr2=0x7f2f700a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:23.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.776+0000 7f2f8b200640 1 --2- 172.21.15.123:0/3622397155 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f2f700a32c0 0x7f2f700a5780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c0ffd50 tx=0x7f2f78005f90 comp rx=0 tx=0).stop 2024-09-19T04:44:23.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.776+0000 7f2f8b200640 1 -- 172.21.15.123:0/3622397155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f8c103e30 msgr2=0x7f2f8c19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:23.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.776+0000 7f2f8b200640 1 --2- 172.21.15.123:0/3622397155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f8c103e30 0x7f2f8c19a4f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c004770 tx=0x7f2f7c003c10 comp rx=0 tx=0).stop 2024-09-19T04:44:23.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.776+0000 7f2f8b200640 1 -- 172.21.15.123:0/3622397155 shutdown_connections 2024-09-19T04:44:23.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.776+0000 7f2f8b200640 1 --2- 172.21.15.123:0/3622397155 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f2f700a32c0 0x7f2f700a5780 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:23.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.776+0000 7f2f8b200640 1 --2- 172.21.15.123:0/3622397155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f8c108f60 0x7f2f8c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:23.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.776+0000 7f2f8b200640 1 --2- 172.21.15.123:0/3622397155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f8c1047e0 0x7f2f8c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:23.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.776+0000 7f2f8b200640 1 --2- 172.21.15.123:0/3622397155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f8c103e30 0x7f2f8c19a4f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:23.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.776+0000 7f2f8b200640 1 -- 172.21.15.123:0/3622397155 >> 172.21.15.123:0/3622397155 conn(0x7f2f8c0ff910 msgr2=0x7f2f8c100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:23.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.776+0000 7f2f8b200640 1 -- 172.21.15.123:0/3622397155 shutdown_connections 2024-09-19T04:44:23.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:23.776+0000 7f2f8b200640 1 -- 172.21.15.123:0/3622397155 wait complete. 2024-09-19T04:44:24.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:23 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:24.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:23 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:24.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:23 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:24.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:23 smithi183 ceph-mon[35333]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:24.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:23 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:24.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:23 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:44:24.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:23 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:24.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:23 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3622397155' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T04:44:24.159 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:23 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:24.159 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:23 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:24.159 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:23 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:24.159 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:23 smithi123 ceph-mon[30511]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:24.159 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:23 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:24.159 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:23 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:44:24.159 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:23 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:24.159 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:23 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3622397155' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T04:44:24.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:23 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:24.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:23 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:24.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:23 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:24.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:23 smithi193 ceph-mon[35592]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:24.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:23 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:24.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:23 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:44:24.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:23 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:24.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:23 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3622397155' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T04:44:24.566 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T04:44:24.566 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T04:44:25.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:24 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:44:25.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:24 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:44:25.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:24 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:44:25.614 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:26.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.212+0000 7f033f000640 1 -- 172.21.15.123:0/863004741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0338102b90 msgr2=0x7f0338102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:26.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.212+0000 7f033f000640 1 --2- 172.21.15.123:0/863004741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0338102b90 0x7f0338102f70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f032c009860 tx=0x7f032c02a3d0 comp rx=0 tx=0).stop 2024-09-19T04:44:26.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.212+0000 7f033f000640 1 -- 172.21.15.123:0/863004741 shutdown_connections 2024-09-19T04:44:26.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.212+0000 7f033f000640 1 --2- 172.21.15.123:0/863004741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f033810bfc0 0x7f033810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:26.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.212+0000 7f033f000640 1 --2- 172.21.15.123:0/863004741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03381034b0 0x7f033810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:26.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.212+0000 7f033f000640 1 --2- 172.21.15.123:0/863004741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0338102b90 0x7f0338102f70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:26.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.212+0000 7f033f000640 1 -- 172.21.15.123:0/863004741 >> 172.21.15.123:0/863004741 conn(0x7f03380fc610 msgr2=0x7f03380fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:26.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.212+0000 7f033f000640 1 -- 172.21.15.123:0/863004741 shutdown_connections 2024-09-19T04:44:26.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.212+0000 7f033f000640 1 -- 172.21.15.123:0/863004741 wait complete. 2024-09-19T04:44:26.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.213+0000 7f033f000640 1 Processor -- start 2024-09-19T04:44:26.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.213+0000 7f033f000640 1 -- start start 2024-09-19T04:44:26.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.214+0000 7f033f000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0338102b90 0x7f03381007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:26.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.214+0000 7f033f000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03381034b0 0x7f0338100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:26.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.214+0000 7f033f000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f033810bfc0 0x7f0338101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:26.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.214+0000 7f033f000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0338111e10 con 0x7f033810bfc0 2024-09-19T04:44:26.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.214+0000 7f033f000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0338111c90 con 0x7f03381034b0 2024-09-19T04:44:26.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.214+0000 7f033f000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0338111f90 con 0x7f0338102b90 2024-09-19T04:44:26.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.214+0000 7f033d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03381034b0 0x7f0338100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:26.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.214+0000 7f033dc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0338102b90 0x7f03381007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:26.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.214+0000 7f033d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03381034b0 0x7f0338100d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:54000/0 (socket says 172.21.15.123:54000) 2024-09-19T04:44:26.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.214+0000 7f033d200640 1 -- 172.21.15.123:0/1739892374 learned_addr learned my addr 172.21.15.123:0/1739892374 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:26.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.214+0000 7f033e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f033810bfc0 0x7f0338101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:26.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.215+0000 7f033d200640 1 -- 172.21.15.123:0/1739892374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0338102b90 msgr2=0x7f03381007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:26.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.215+0000 7f033d200640 1 --2- 172.21.15.123:0/1739892374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0338102b90 0x7f03381007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:26.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.215+0000 7f033d200640 1 -- 172.21.15.123:0/1739892374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f033810bfc0 msgr2=0x7f0338101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:26.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.215+0000 7f033d200640 1 --2- 172.21.15.123:0/1739892374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f033810bfc0 0x7f0338101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:26.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.215+0000 7f033d200640 1 -- 172.21.15.123:0/1739892374 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03381a6740 con 0x7f03381034b0 2024-09-19T04:44:26.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.215+0000 7f033d200640 1 --2- 172.21.15.123:0/1739892374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03381034b0 0x7f0338100d30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f032400e9f0 tx=0x7f032400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:26.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.215+0000 7f0336000640 1 -- 172.21.15.123:0/1739892374 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032400ccc0 con 0x7f03381034b0 2024-09-19T04:44:26.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.215+0000 7f0336000640 1 -- 172.21.15.123:0/1739892374 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f032400ce60 con 0x7f03381034b0 2024-09-19T04:44:26.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.215+0000 7f033f000640 1 -- 172.21.15.123:0/1739892374 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03381a6a30 con 0x7f03381034b0 2024-09-19T04:44:26.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.216+0000 7f0336000640 1 -- 172.21.15.123:0/1739892374 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324010690 con 0x7f03381034b0 2024-09-19T04:44:26.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.216+0000 7f033f000640 1 -- 172.21.15.123:0/1739892374 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03381a6f20 con 0x7f03381034b0 2024-09-19T04:44:26.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.217+0000 7f033f000640 1 -- 172.21.15.123:0/1739892374 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0338102f70 con 0x7f03381034b0 2024-09-19T04:44:26.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.219+0000 7f0336000640 1 -- 172.21.15.123:0/1739892374 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f03240046c0 con 0x7f03381034b0 2024-09-19T04:44:26.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.220+0000 7f0336000640 1 --2- 172.21.15.123:0/1739892374 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f03200a3240 0x7f03200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:26.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.220+0000 7f0336000640 1 -- 172.21.15.123:0/1739892374 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f03240cf870 con 0x7f03381034b0 2024-09-19T04:44:26.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.220+0000 7f033dc00640 1 --2- 172.21.15.123:0/1739892374 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f03200a3240 0x7f03200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:26.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.221+0000 7f033dc00640 1 --2- 172.21.15.123:0/1739892374 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f03200a3240 0x7f03200a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f032c02a9c0 tx=0x7f032c005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:26.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.224+0000 7f0336000640 1 -- 172.21.15.123:0/1739892374 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f032409c990 con 0x7f03381034b0 2024-09-19T04:44:26.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:26.414+0000 7f033f000640 1 -- 172.21.15.123:0/1739892374 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0338107060 con 0x7f03381034b0 2024-09-19T04:44:26.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:26.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:26.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:26.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:26 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:26.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:26 smithi183 ceph-mon[35333]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:26.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:26.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:26.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:26.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:26 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:26.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:26 smithi123 ceph-mon[30511]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:26.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:26.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:26.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:26.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:26 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:26.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:26 smithi193 ceph-mon[35592]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:27.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:27.318+0000 7f0336000640 1 -- 172.21.15.123:0/1739892374 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) ==== 138+0+0 (secure 0 0 0) 0x7f032409c290 con 0x7f03381034b0 2024-09-19T04:44:27.320 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' created 2024-09-19T04:44:27.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:27.321+0000 7f033f000640 1 -- 172.21.15.123:0/1739892374 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f03200a3240 msgr2=0x7f03200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:27.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:27.321+0000 7f033f000640 1 --2- 172.21.15.123:0/1739892374 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f03200a3240 0x7f03200a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f032c02a9c0 tx=0x7f032c005c90 comp rx=0 tx=0).stop 2024-09-19T04:44:27.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:27.321+0000 7f033f000640 1 -- 172.21.15.123:0/1739892374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03381034b0 msgr2=0x7f0338100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:27.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:27.321+0000 7f033f000640 1 --2- 172.21.15.123:0/1739892374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03381034b0 0x7f0338100d30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f032400e9f0 tx=0x7f032400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:44:27.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:27.321+0000 7f033f000640 1 -- 172.21.15.123:0/1739892374 shutdown_connections 2024-09-19T04:44:27.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:27.321+0000 7f033f000640 1 --2- 172.21.15.123:0/1739892374 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f03200a3240 0x7f03200a5700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:27.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:27.321+0000 7f033f000640 1 --2- 172.21.15.123:0/1739892374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f033810bfc0 0x7f0338101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:27.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:27.321+0000 7f033f000640 1 --2- 172.21.15.123:0/1739892374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03381034b0 0x7f0338100d30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:27.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:27.321+0000 7f033f000640 1 --2- 172.21.15.123:0/1739892374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0338102b90 0x7f03381007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:27.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:27.321+0000 7f033f000640 1 -- 172.21.15.123:0/1739892374 >> 172.21.15.123:0/1739892374 conn(0x7f03380fc610 msgr2=0x7f03380fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:27.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:27.321+0000 7f033f000640 1 -- 172.21.15.123:0/1739892374 shutdown_connections 2024-09-19T04:44:27.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:27.321+0000 7f033f000640 1 -- 172.21.15.123:0/1739892374 wait complete. 2024-09-19T04:44:27.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:27 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1739892374' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:44:27.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:27 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:44:27.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:27 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:27.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:27 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:27.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:27 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:27.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:27 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:44:27.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:27 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:27.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:27 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1739892374' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:44:27.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:27 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:44:27.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:27 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:27.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:27 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:27.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:27 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:27.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:27 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:44:27.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:27 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:27.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:27 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1739892374' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:44:27.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:27 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:44:27.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:27 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:27.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:27 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:27.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:27 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:44:27.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:27 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:44:27.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:27 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:44:28.081 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T04:44:28.459 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:28.598 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:28 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T04:44:28.598 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:28 smithi123 ceph-mon[30511]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T04:44:28.599 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:28 smithi123 ceph-mon[30511]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:28.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:28 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T04:44:28.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:28 smithi183 ceph-mon[35333]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T04:44:28.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:28 smithi183 ceph-mon[35333]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:28.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:28 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T04:44:28.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:28 smithi193 ceph-mon[35592]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T04:44:28.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:28 smithi193 ceph-mon[35592]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:29.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.028+0000 7fc905e00640 1 -- 172.21.15.123:0/3150021397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc900106f10 msgr2=0x7fc90010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:29.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.028+0000 7fc905e00640 1 --2- 172.21.15.123:0/3150021397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc900106f10 0x7fc90010ea90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009ca0 tx=0x7fc8f802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:44:29.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.028+0000 7fc905e00640 1 -- 172.21.15.123:0/3150021397 shutdown_connections 2024-09-19T04:44:29.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.028+0000 7fc905e00640 1 --2- 172.21.15.123:0/3150021397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc900106f10 0x7fc90010ea90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:29.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.028+0000 7fc905e00640 1 --2- 172.21.15.123:0/3150021397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc900102790 0x7fc9001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:29.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.028+0000 7fc905e00640 1 --2- 172.21.15.123:0/3150021397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc900101de0 0x7fc9001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:29.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.028+0000 7fc905e00640 1 -- 172.21.15.123:0/3150021397 >> 172.21.15.123:0/3150021397 conn(0x7fc9000fd8c0 msgr2=0x7fc9000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:29.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.028+0000 7fc905e00640 1 -- 172.21.15.123:0/3150021397 shutdown_connections 2024-09-19T04:44:29.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.028+0000 7fc905e00640 1 -- 172.21.15.123:0/3150021397 wait complete. 2024-09-19T04:44:29.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.029+0000 7fc905e00640 1 Processor -- start 2024-09-19T04:44:29.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.029+0000 7fc905e00640 1 -- start start 2024-09-19T04:44:29.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.030+0000 7fc905e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc900101de0 0x7fc90019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:29.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.030+0000 7fc905e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc900102790 0x7fc90019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:29.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.030+0000 7fc905e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc900106f10 0x7fc90019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:29.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.030+0000 7fc905e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc900111fc0 con 0x7fc900102790 2024-09-19T04:44:29.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.030+0000 7fc905e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc900111e40 con 0x7fc900106f10 2024-09-19T04:44:29.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.030+0000 7fc905e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc900112140 con 0x7fc900101de0 2024-09-19T04:44:29.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.030+0000 7fc8fea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc900101de0 0x7fc90019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:29.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.030+0000 7fc8fea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc900101de0 0x7fc90019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58628/0 (socket says 172.21.15.123:58628) 2024-09-19T04:44:29.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.030+0000 7fc8fea00640 1 -- 172.21.15.123:0/718586057 learned_addr learned my addr 172.21.15.123:0/718586057 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:29.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.030+0000 7fc8fe000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc900102790 0x7fc90019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:29.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.030+0000 7fc8ff400640 1 --2- 172.21.15.123:0/718586057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc900106f10 0x7fc90019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:29.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.030+0000 7fc8fea00640 1 -- 172.21.15.123:0/718586057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc900106f10 msgr2=0x7fc90019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:29.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.030+0000 7fc8fea00640 1 --2- 172.21.15.123:0/718586057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc900106f10 0x7fc90019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:29.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.030+0000 7fc8fea00640 1 -- 172.21.15.123:0/718586057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc900102790 msgr2=0x7fc90019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:29.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.030+0000 7fc8fea00640 1 --2- 172.21.15.123:0/718586057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc900102790 0x7fc90019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:29.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.030+0000 7fc8fea00640 1 -- 172.21.15.123:0/718586057 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90019f610 con 0x7fc900101de0 2024-09-19T04:44:29.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.030+0000 7fc8ff400640 1 --2- 172.21.15.123:0/718586057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc900106f10 0x7fc90019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:44:29.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.031+0000 7fc8fea00640 1 --2- 172.21.15.123:0/718586057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc900101de0 0x7fc90019a660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000d950 tx=0x7fc8f000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:29.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.031+0000 7fc904a00640 1 -- 172.21.15.123:0/718586057 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0017030 con 0x7fc900101de0 2024-09-19T04:44:29.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.031+0000 7fc905e00640 1 -- 172.21.15.123:0/718586057 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90019f900 con 0x7fc900101de0 2024-09-19T04:44:29.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.031+0000 7fc904a00640 1 -- 172.21.15.123:0/718586057 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f0004020 con 0x7fc900101de0 2024-09-19T04:44:29.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.031+0000 7fc904a00640 1 -- 172.21.15.123:0/718586057 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f00047a0 con 0x7fc900101de0 2024-09-19T04:44:29.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.031+0000 7fc905e00640 1 -- 172.21.15.123:0/718586057 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9001a71e0 con 0x7fc900101de0 2024-09-19T04:44:29.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.032+0000 7fc905e00640 1 -- 172.21.15.123:0/718586057 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9001021c0 con 0x7fc900101de0 2024-09-19T04:44:29.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.034+0000 7fc904a00640 1 -- 172.21.15.123:0/718586057 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc8f00041d0 con 0x7fc900101de0 2024-09-19T04:44:29.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.035+0000 7fc904a00640 1 --2- 172.21.15.123:0/718586057 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc8e40a3240 0x7fc8e40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:29.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.035+0000 7fc904a00640 1 -- 172.21.15.123:0/718586057 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7497+0+0 (secure 0 0 0) 0x7fc8f00cf700 con 0x7fc900101de0 2024-09-19T04:44:29.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.036+0000 7fc8fe000640 1 --2- 172.21.15.123:0/718586057 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc8e40a3240 0x7fc8e40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:29.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.036+0000 7fc8fe000640 1 --2- 172.21.15.123:0/718586057 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc8e40a3240 0x7fc8e40a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc9001087e0 tx=0x7fc8ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:29.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.038+0000 7fc904a00640 1 -- 172.21.15.123:0/718586057 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc8f009c6b0 con 0x7fc900101de0 2024-09-19T04:44:29.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.215+0000 7fc905e00640 1 -- 172.21.15.123:0/718586057 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc9001a7410 con 0x7fc900101de0 2024-09-19T04:44:29.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.734+0000 7fc904a00640 1 -- 172.21.15.123:0/718586057 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) ==== 170+0+0 (secure 0 0 0) 0x7fc8f009bfb0 con 0x7fc900101de0 2024-09-19T04:44:29.737 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' created 2024-09-19T04:44:29.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.741+0000 7fc905e00640 1 -- 172.21.15.123:0/718586057 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc8e40a3240 msgr2=0x7fc8e40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:29.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.741+0000 7fc905e00640 1 --2- 172.21.15.123:0/718586057 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc8e40a3240 0x7fc8e40a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc9001087e0 tx=0x7fc8ec005f90 comp rx=0 tx=0).stop 2024-09-19T04:44:29.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.741+0000 7fc905e00640 1 -- 172.21.15.123:0/718586057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc900101de0 msgr2=0x7fc90019a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:29.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.741+0000 7fc905e00640 1 --2- 172.21.15.123:0/718586057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc900101de0 0x7fc90019a660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000d950 tx=0x7fc8f000de10 comp rx=0 tx=0).stop 2024-09-19T04:44:29.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.741+0000 7fc905e00640 1 -- 172.21.15.123:0/718586057 shutdown_connections 2024-09-19T04:44:29.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.741+0000 7fc905e00640 1 --2- 172.21.15.123:0/718586057 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc8e40a3240 0x7fc8e40a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:29.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.741+0000 7fc905e00640 1 --2- 172.21.15.123:0/718586057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc900106f10 0x7fc90019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:29.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.741+0000 7fc905e00640 1 --2- 172.21.15.123:0/718586057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc900102790 0x7fc90019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:29.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.741+0000 7fc905e00640 1 --2- 172.21.15.123:0/718586057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc900101de0 0x7fc90019a660 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:29.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.741+0000 7fc905e00640 1 -- 172.21.15.123:0/718586057 >> 172.21.15.123:0/718586057 conn(0x7fc9000fd8c0 msgr2=0x7fc9000ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:29.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.742+0000 7fc905e00640 1 -- 172.21.15.123:0/718586057 shutdown_connections 2024-09-19T04:44:29.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:29.742+0000 7fc905e00640 1 -- 172.21.15.123:0/718586057 wait complete. 2024-09-19T04:44:29.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:29 smithi193 ceph-mon[35592]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T04:44:29.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:29 smithi193 ceph-mon[35592]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T04:44:30.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:29 smithi183 ceph-mon[35333]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T04:44:30.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:29 smithi183 ceph-mon[35333]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T04:44:30.141 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:29 smithi123 ceph-mon[30511]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T04:44:30.141 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:29 smithi123 ceph-mon[30511]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T04:44:30.639 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T04:44:30.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:30 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/718586057' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:44:30.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:30 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:44:30.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:30 smithi123 ceph-mon[30511]: pgmap v288: 33 pgs: 1 creating+activating, 22 creating+peering, 5 active+clean, 5 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:30.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:30 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T04:44:30.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:30 smithi123 ceph-mon[30511]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T04:44:30.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:30 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/718586057' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:44:30.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:30 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:44:30.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:30 smithi193 ceph-mon[35592]: pgmap v288: 33 pgs: 1 creating+activating, 22 creating+peering, 5 active+clean, 5 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:30.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:30 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T04:44:30.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:30 smithi193 ceph-mon[35592]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T04:44:31.025 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:31.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:30 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/718586057' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:44:31.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:30 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:44:31.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:30 smithi183 ceph-mon[35333]: pgmap v288: 33 pgs: 1 creating+activating, 22 creating+peering, 5 active+clean, 5 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:31.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:30 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T04:44:31.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:30 smithi183 ceph-mon[35333]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T04:44:31.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.541+0000 7fdf4b200640 1 -- 172.21.15.123:0/3328064029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf4c1047e0 msgr2=0x7fdf4c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:31.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.541+0000 7fdf4b200640 1 --2- 172.21.15.123:0/3328064029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf4c1047e0 0x7fdf4c108830 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009900 tx=0x7fdf3402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:44:31.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.541+0000 7fdf4b200640 1 -- 172.21.15.123:0/3328064029 shutdown_connections 2024-09-19T04:44:31.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.541+0000 7fdf4b200640 1 --2- 172.21.15.123:0/3328064029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf4c108f60 0x7fdf4c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:31.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.541+0000 7fdf4b200640 1 --2- 172.21.15.123:0/3328064029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf4c1047e0 0x7fdf4c108830 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:31.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.541+0000 7fdf4b200640 1 --2- 172.21.15.123:0/3328064029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf4c103e30 0x7fdf4c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:31.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.541+0000 7fdf4b200640 1 -- 172.21.15.123:0/3328064029 >> 172.21.15.123:0/3328064029 conn(0x7fdf4c0ff8d0 msgr2=0x7fdf4c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:31.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.541+0000 7fdf4b200640 1 -- 172.21.15.123:0/3328064029 shutdown_connections 2024-09-19T04:44:31.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.541+0000 7fdf4b200640 1 -- 172.21.15.123:0/3328064029 wait complete. 2024-09-19T04:44:31.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.543+0000 7fdf4b200640 1 Processor -- start 2024-09-19T04:44:31.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.543+0000 7fdf4b200640 1 -- start start 2024-09-19T04:44:31.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.543+0000 7fdf4b200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf4c103e30 0x7fdf4c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:31.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.543+0000 7fdf4b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf4c1047e0 0x7fdf4c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:31.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.543+0000 7fdf4b200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf4c108f60 0x7fdf4c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:31.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.543+0000 7fdf4b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c113fc0 con 0x7fdf4c1047e0 2024-09-19T04:44:31.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.543+0000 7fdf4b200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c113e40 con 0x7fdf4c103e30 2024-09-19T04:44:31.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.543+0000 7fdf4b200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c114140 con 0x7fdf4c108f60 2024-09-19T04:44:31.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.543+0000 7fdf49e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf4c103e30 0x7fdf4c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:31.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.543+0000 7fdf41400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf4c1047e0 0x7fdf4c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:31.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.543+0000 7fdf49e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf4c103e30 0x7fdf4c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:54046/0 (socket says 172.21.15.123:54046) 2024-09-19T04:44:31.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.543+0000 7fdf49e00640 1 -- 172.21.15.123:0/3401969665 learned_addr learned my addr 172.21.15.123:0/3401969665 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:31.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.544+0000 7fdf4a800640 1 --2- 172.21.15.123:0/3401969665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf4c108f60 0x7fdf4c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:31.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.544+0000 7fdf49e00640 1 -- 172.21.15.123:0/3401969665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf4c108f60 msgr2=0x7fdf4c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:31.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.544+0000 7fdf49e00640 1 --2- 172.21.15.123:0/3401969665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf4c108f60 0x7fdf4c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:31.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.544+0000 7fdf49e00640 1 -- 172.21.15.123:0/3401969665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf4c1047e0 msgr2=0x7fdf4c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:31.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.544+0000 7fdf49e00640 1 --2- 172.21.15.123:0/3401969665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf4c1047e0 0x7fdf4c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:31.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.544+0000 7fdf49e00640 1 -- 172.21.15.123:0/3401969665 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4c19f410 con 0x7fdf4c103e30 2024-09-19T04:44:31.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.544+0000 7fdf41400640 1 --2- 172.21.15.123:0/3401969665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf4c1047e0 0x7fdf4c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:44:31.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.544+0000 7fdf49e00640 1 --2- 172.21.15.123:0/3401969665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf4c103e30 0x7fdf4c19a460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00d950 tx=0x7fdf3c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:31.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.544+0000 7fdf43400640 1 -- 172.21.15.123:0/3401969665 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c017030 con 0x7fdf4c103e30 2024-09-19T04:44:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.544+0000 7fdf4b200640 1 -- 172.21.15.123:0/3401969665 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4c19f700 con 0x7fdf4c103e30 2024-09-19T04:44:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.544+0000 7fdf43400640 1 -- 172.21.15.123:0/3401969665 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf3c004020 con 0x7fdf4c103e30 2024-09-19T04:44:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.544+0000 7fdf43400640 1 -- 172.21.15.123:0/3401969665 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0047a0 con 0x7fdf4c103e30 2024-09-19T04:44:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.545+0000 7fdf4b200640 1 -- 172.21.15.123:0/3401969665 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf4c1a6fe0 con 0x7fdf4c103e30 2024-09-19T04:44:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.546+0000 7fdf4b200640 1 -- 172.21.15.123:0/3401969665 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf4c104210 con 0x7fdf4c103e30 2024-09-19T04:44:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.548+0000 7fdf43400640 1 -- 172.21.15.123:0/3401969665 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fdf3c0041d0 con 0x7fdf4c103e30 2024-09-19T04:44:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.549+0000 7fdf43400640 1 --2- 172.21.15.123:0/3401969665 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fdf300a3240 0x7fdf300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.549+0000 7fdf43400640 1 -- 172.21.15.123:0/3401969665 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7879+0+0 (secure 0 0 0) 0x7fdf3c0cf7d0 con 0x7fdf4c103e30 2024-09-19T04:44:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.549+0000 7fdf41400640 1 --2- 172.21.15.123:0/3401969665 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fdf300a3240 0x7fdf300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.550+0000 7fdf41400640 1 --2- 172.21.15.123:0/3401969665 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fdf300a3240 0x7fdf300a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009230 tx=0x7fdf340058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.551+0000 7fdf43400640 1 -- 172.21.15.123:0/3401969665 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdf3c09c600 con 0x7fdf4c103e30 2024-09-19T04:44:31.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:31.764+0000 7fdf4b200640 1 -- 172.21.15.123:0/3401969665 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fdf4c10bc10 con 0x7fdf4c103e30 2024-09-19T04:44:32.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:31 smithi183 ceph-mon[35333]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T04:44:32.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:31 smithi183 ceph-mon[35333]: pgmap v291: 97 pgs: 1 creating+activating, 22 creating+peering, 5 active+clean, 69 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:32.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:31 smithi123 ceph-mon[30511]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T04:44:32.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:31 smithi123 ceph-mon[30511]: pgmap v291: 97 pgs: 1 creating+activating, 22 creating+peering, 5 active+clean, 69 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:32.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:31 smithi193 ceph-mon[35592]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T04:44:32.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:31 smithi193 ceph-mon[35592]: pgmap v291: 97 pgs: 1 creating+activating, 22 creating+peering, 5 active+clean, 69 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:32.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:32.748+0000 7fdf43400640 1 -- 172.21.15.123:0/3401969665 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fdf3c09bf00 con 0x7fdf4c103e30 2024-09-19T04:44:32.750 INFO:teuthology.orchestra.run.smithi123.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T04:44:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:32.752+0000 7fdf4b200640 1 -- 172.21.15.123:0/3401969665 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fdf300a3240 msgr2=0x7fdf300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:32.752+0000 7fdf4b200640 1 --2- 172.21.15.123:0/3401969665 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fdf300a3240 0x7fdf300a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009230 tx=0x7fdf340058d0 comp rx=0 tx=0).stop 2024-09-19T04:44:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:32.752+0000 7fdf4b200640 1 -- 172.21.15.123:0/3401969665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf4c103e30 msgr2=0x7fdf4c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:32.752+0000 7fdf4b200640 1 --2- 172.21.15.123:0/3401969665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf4c103e30 0x7fdf4c19a460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00d950 tx=0x7fdf3c00de10 comp rx=0 tx=0).stop 2024-09-19T04:44:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:32.753+0000 7fdf4b200640 1 -- 172.21.15.123:0/3401969665 shutdown_connections 2024-09-19T04:44:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:32.753+0000 7fdf4b200640 1 --2- 172.21.15.123:0/3401969665 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fdf300a3240 0x7fdf300a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:32.753+0000 7fdf4b200640 1 --2- 172.21.15.123:0/3401969665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf4c108f60 0x7fdf4c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:32.753+0000 7fdf4b200640 1 --2- 172.21.15.123:0/3401969665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf4c1047e0 0x7fdf4c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:32.753+0000 7fdf4b200640 1 --2- 172.21.15.123:0/3401969665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf4c103e30 0x7fdf4c19a460 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:32.753+0000 7fdf4b200640 1 -- 172.21.15.123:0/3401969665 >> 172.21.15.123:0/3401969665 conn(0x7fdf4c0ff8d0 msgr2=0x7fdf4c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:32.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:32.754+0000 7fdf4b200640 1 -- 172.21.15.123:0/3401969665 shutdown_connections 2024-09-19T04:44:32.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:32.754+0000 7fdf4b200640 1 -- 172.21.15.123:0/3401969665 wait complete. 2024-09-19T04:44:33.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:32 smithi183 ceph-mon[35333]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T04:44:33.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:32 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3401969665' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T04:44:33.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:32 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T04:44:33.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:32 smithi183 ceph-mon[35333]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T04:44:33.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:32 smithi183 ceph-mon[35333]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T04:44:33.118 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:32 smithi123 ceph-mon[30511]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T04:44:33.118 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:32 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3401969665' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T04:44:33.118 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:32 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T04:44:33.118 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:32 smithi123 ceph-mon[30511]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T04:44:33.118 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:32 smithi123 ceph-mon[30511]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T04:44:33.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:32 smithi193 ceph-mon[35592]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T04:44:33.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:32 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3401969665' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T04:44:33.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:32 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T04:44:33.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:32 smithi193 ceph-mon[35592]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T04:44:33.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:32 smithi193 ceph-mon[35592]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T04:44:33.600 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:44:33.969 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:34.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:33 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T04:44:34.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:33 smithi183 ceph-mon[35333]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T04:44:34.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:33 smithi183 ceph-mon[35333]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T04:44:34.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:33 smithi183 ceph-mon[35333]: pgmap v294: 97 pgs: 1 creating+activating, 84 active+clean, 12 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:34.107 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:33 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T04:44:34.108 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:33 smithi123 ceph-mon[30511]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T04:44:34.108 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:33 smithi123 ceph-mon[30511]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T04:44:34.108 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:33 smithi123 ceph-mon[30511]: pgmap v294: 97 pgs: 1 creating+activating, 84 active+clean, 12 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:34.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:33 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T04:44:34.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:33 smithi193 ceph-mon[35592]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T04:44:34.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:33 smithi193 ceph-mon[35592]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T04:44:34.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:33 smithi193 ceph-mon[35592]: pgmap v294: 97 pgs: 1 creating+activating, 84 active+clean, 12 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:44:34.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.505+0000 7f7149e00640 1 -- 172.21.15.123:0/381380856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7144108f60 msgr2=0x7f7144110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:34.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.505+0000 7f7149e00640 1 --2- 172.21.15.123:0/381380856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7144108f60 0x7f7144110ae0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f713c009ca0 tx=0x7f713c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:44:34.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.506+0000 7f7149e00640 1 -- 172.21.15.123:0/381380856 shutdown_connections 2024-09-19T04:44:34.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.506+0000 7f7149e00640 1 --2- 172.21.15.123:0/381380856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7144108f60 0x7f7144110ae0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:34.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.506+0000 7f7149e00640 1 --2- 172.21.15.123:0/381380856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71441047e0 0x7f7144108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:34.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.506+0000 7f7149e00640 1 --2- 172.21.15.123:0/381380856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7144103e30 0x7f7144104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:34.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.506+0000 7f7149e00640 1 -- 172.21.15.123:0/381380856 >> 172.21.15.123:0/381380856 conn(0x7f71440ff8d0 msgr2=0x7f7144101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:34.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.506+0000 7f7149e00640 1 -- 172.21.15.123:0/381380856 shutdown_connections 2024-09-19T04:44:34.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.506+0000 7f7149e00640 1 -- 172.21.15.123:0/381380856 wait complete. 2024-09-19T04:44:34.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.507+0000 7f7149e00640 1 Processor -- start 2024-09-19T04:44:34.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.507+0000 7f7149e00640 1 -- start start 2024-09-19T04:44:34.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.507+0000 7f7149e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7144103e30 0x7f714419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:34.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.507+0000 7f7149e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71441047e0 0x7f714419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:34.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.507+0000 7f7149e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7144108f60 0x7f714419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:34.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.507+0000 7f7149e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7144114010 con 0x7f7144103e30 2024-09-19T04:44:34.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.507+0000 7f7149e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7144113e90 con 0x7f7144108f60 2024-09-19T04:44:34.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.507+0000 7f7149e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7144114190 con 0x7f71441047e0 2024-09-19T04:44:34.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.507+0000 7f7148a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7144103e30 0x7f714419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:34.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.507+0000 7f7148a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7144103e30 0x7f714419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43838/0 (socket says 172.21.15.123:43838) 2024-09-19T04:44:34.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.507+0000 7f7148a00640 1 -- 172.21.15.123:0/3705130750 learned_addr learned my addr 172.21.15.123:0/3705130750 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:34.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.508+0000 7f7143e00640 1 --2- 172.21.15.123:0/3705130750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71441047e0 0x7f714419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:34.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.508+0000 7f7149400640 1 --2- 172.21.15.123:0/3705130750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7144108f60 0x7f714419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:34.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.508+0000 7f7148a00640 1 -- 172.21.15.123:0/3705130750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71441047e0 msgr2=0x7f714419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:34.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.508+0000 7f7148a00640 1 --2- 172.21.15.123:0/3705130750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71441047e0 0x7f714419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:34.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.508+0000 7f7148a00640 1 -- 172.21.15.123:0/3705130750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7144108f60 msgr2=0x7f714419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:34.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.508+0000 7f7148a00640 1 --2- 172.21.15.123:0/3705130750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7144108f60 0x7f714419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:34.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.508+0000 7f7148a00640 1 -- 172.21.15.123:0/3705130750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714419f3c0 con 0x7f7144103e30 2024-09-19T04:44:34.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.508+0000 7f7148a00640 1 --2- 172.21.15.123:0/3705130750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7144103e30 0x7f714419a410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f713400e9f0 tx=0x7f713400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:34.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.509+0000 7f7141600640 1 -- 172.21.15.123:0/3705130750 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713400cd80 con 0x7f7144103e30 2024-09-19T04:44:34.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.509+0000 7f7141600640 1 -- 172.21.15.123:0/3705130750 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7134010910 con 0x7f7144103e30 2024-09-19T04:44:34.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.509+0000 7f7149e00640 1 -- 172.21.15.123:0/3705130750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714419f6b0 con 0x7f7144103e30 2024-09-19T04:44:34.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.509+0000 7f7141600640 1 -- 172.21.15.123:0/3705130750 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7134018990 con 0x7f7144103e30 2024-09-19T04:44:34.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.509+0000 7f7149e00640 1 -- 172.21.15.123:0/3705130750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71441a6f90 con 0x7f7144103e30 2024-09-19T04:44:34.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.510+0000 7f7149e00640 1 -- 172.21.15.123:0/3705130750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7144104210 con 0x7f7144103e30 2024-09-19T04:44:34.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.513+0000 7f7141600640 1 -- 172.21.15.123:0/3705130750 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f7134010ac0 con 0x7f7144103e30 2024-09-19T04:44:34.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.514+0000 7f7141600640 1 --2- 172.21.15.123:0/3705130750 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f71280a3240 0x7f71280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:34.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.515+0000 7f7141600640 1 -- 172.21.15.123:0/3705130750 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f71340cff40 con 0x7f7144103e30 2024-09-19T04:44:34.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.515+0000 7f7143e00640 1 --2- 172.21.15.123:0/3705130750 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f71280a3240 0x7f71280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:34.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.515+0000 7f7143e00640 1 --2- 172.21.15.123:0/3705130750 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f71280a3240 0x7f71280a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7144100140 tx=0x7f7130005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:34.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.517+0000 7f7141600640 1 -- 172.21.15.123:0/3705130750 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f713409c6c0 con 0x7f7144103e30 2024-09-19T04:44:34.599 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:44:34 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-b[65203]: 2024-09-19T04:44:34.313+0000 7fbe924d0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T04:44:34.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.698+0000 7f7149e00640 1 -- 172.21.15.123:0/3705130750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71441a7240 con 0x7f7144103e30 2024-09-19T04:44:34.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.699+0000 7f7141600640 1 -- 172.21.15.123:0/3705130750 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22362 (secure 0 0 0) 0x7f71340a09c0 con 0x7f7144103e30 2024-09-19T04:44:34.701 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:44:34.701 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":84,"fsid":"911fa052-7640-11ef-bceb-c7b262605968","created":"2024-09-19T04:36:21.083333+0000","modified":"2024-09-19T04:44:31.766762+0000","last_up_change":"2024-09-19T04:43:33.818439+0000","last_in_change":"2024-09-19T04:43:10.148272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:39:33.856981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:44:26.956483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:44:29.718625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6803","nonce":3631596501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6805","nonce":3631596501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6809","nonce":3631596501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6807","nonce":3631596501}]},"public_addr":"172.21.15.123:6803/3631596501","cluster_addr":"172.21.15.123:6805/3631596501","heartbeat_back_addr":"172.21.15.123:6809/3631596501","heartbeat_front_addr":"172.21.15.123:6807/3631596501","state":["exists","up"]},{"osd":1,"uuid":"a526d4ce-586b-4d66-b720-e868df92fa24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6801","nonce":1259721614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6803","nonce":1259721614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6807","nonce":1259721614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6805","nonce":1259721614}]},"public_addr":"172.21.15.183:6801/1259721614","cluster_addr":"172.21.15.183:6803/1259721614","heartbeat_back_addr":"172.21.15.183:6807/1259721614","heartbeat_front_addr":"172.21.15.183:6805/1259721614","state":["exists","up"]},{"osd":2,"uuid":"3650601a-6259-4b2a-97df-1d47b09b0127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6801","nonce":2708151726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6803","nonce":2708151726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6807","nonce":2708151726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6805","nonce":2708151726}]},"public_addr":"172.21.15.193:6801/2708151726","cluster_addr":"172.21.15.193:6803/2708151726","heartbeat_back_addr":"172.21.15.193:6807/2708151726","heartbeat_front_addr":"172.21.15.193:6805/2708151726","state":["exists","up"]},{"osd":3,"uuid":"67c485ad-136e-4020-950d-9171a3c2de35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6811","nonce":3214806282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6813","nonce":3214806282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6817","nonce":3214806282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6815","nonce":3214806282}]},"public_addr":"172.21.15.123:6811/3214806282","cluster_addr":"172.21.15.123:6813/3214806282","heartbeat_back_addr":"172.21.15.123:6817/3214806282","heartbeat_front_addr":"172.21.15.123:6815/3214806282","state":["exists","up"]},{"osd":4,"uuid":"c9098540-06ab-45a8-9cf9-f90519cc852e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6809","nonce":667618665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6811","nonce":667618665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6815","nonce":667618665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6813","nonce":667618665}]},"public_addr":"172.21.15.183:6809/667618665","cluster_addr":"172.21.15.183:6811/667618665","heartbeat_back_addr":"172.21.15.183:6815/667618665","heartbeat_front_addr":"172.21.15.183:6813/667618665","state":["exists","up"]},{"osd":5,"uuid":"278a8dbd-6492-42d4-b489-1f832350bc6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6809","nonce":2292251639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6811","nonce":2292251639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6815","nonce":2292251639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6813","nonce":2292251639}]},"public_addr":"172.21.15.193:6809/2292251639","cluster_addr":"172.21.15.193:6811/2292251639","heartbeat_back_addr":"172.21.15.193:6815/2292251639","heartbeat_front_addr":"172.21.15.193:6813/2292251639","state":["exists","up"]},{"osd":6,"uuid":"0fda4e39-6adb-4941-a39b-a9a15019088c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6819","nonce":226007499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6821","nonce":226007499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6825","nonce":226007499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6823","nonce":226007499}]},"public_addr":"172.21.15.123:6819/226007499","cluster_addr":"172.21.15.123:6821/226007499","heartbeat_back_addr":"172.21.15.123:6825/226007499","heartbeat_front_addr":"172.21.15.123:6823/226007499","state":["exists","up"]},{"osd":7,"uuid":"5c0a021e-9f3b-4766-9686-0ef24b68cf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6817","nonce":3718518226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6819","nonce":3718518226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6823","nonce":3718518226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6821","nonce":3718518226}]},"public_addr":"172.21.15.183:6817/3718518226","cluster_addr":"172.21.15.183:6819/3718518226","heartbeat_back_addr":"172.21.15.183:6823/3718518226","heartbeat_front_addr":"172.21.15.183:6821/3718518226","state":["exists","up"]},{"osd":8,"uuid":"9b0e67e0-6305-4f81-9211-d41960099a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6817","nonce":3551404118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6819","nonce":3551404118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6823","nonce":3551404118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6821","nonce":3551404118}]},"public_addr":"172.21.15.193:6817/3551404118","cluster_addr":"172.21.15.193:6819/3551404118","heartbeat_back_addr":"172.21.15.193:6823/3551404118","heartbeat_front_addr":"172.21.15.193:6821/3551404118","state":["exists","up"]},{"osd":9,"uuid":"978ef3fa-c6ec-421f-b194-56d44813be5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6827","nonce":1439901149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6829","nonce":1439901149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6833","nonce":1439901149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6831","nonce":1439901149}]},"public_addr":"172.21.15.123:6827/1439901149","cluster_addr":"172.21.15.123:6829/1439901149","heartbeat_back_addr":"172.21.15.123:6833/1439901149","heartbeat_front_addr":"172.21.15.123:6831/1439901149","state":["exists","up"]},{"osd":10,"uuid":"5495e09b-7809-47b0-b7e0-b56c452f9f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6825","nonce":3355349245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6827","nonce":3355349245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6831","nonce":3355349245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6829","nonce":3355349245}]},"public_addr":"172.21.15.183:6825/3355349245","cluster_addr":"172.21.15.183:6827/3355349245","heartbeat_back_addr":"172.21.15.183:6831/3355349245","heartbeat_front_addr":"172.21.15.183:6829/3355349245","state":["exists","up"]},{"osd":11,"uuid":"03046fef-b16b-42f8-9a19-6c0b78cbfe7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6825","nonce":4291617038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6827","nonce":4291617038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6831","nonce":4291617038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6829","nonce":4291617038}]},"public_addr":"172.21.15.193:6825/4291617038","cluster_addr":"172.21.15.193:6827/4291617038","heartbeat_back_addr":"172.21.15.193:6831/4291617038","heartbeat_front_addr":"172.21.15.193:6829/4291617038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:38:39.142529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:05.336911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:30.310967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:59.655973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:25.890576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:50.778689+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:19.063731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:45.424269+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:10.562149+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:40.382368+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:06.297731+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:32.352172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/4040401634":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/3498196650":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6800/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:6800/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/471028951":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/1009496020":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/2000593686":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/2580747029":"2024-09-20T04:36:45.913060+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:44:34.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.702+0000 7f7149e00640 1 -- 172.21.15.123:0/3705130750 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f71280a3240 msgr2=0x7f71280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:34.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.702+0000 7f7149e00640 1 --2- 172.21.15.123:0/3705130750 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f71280a3240 0x7f71280a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7144100140 tx=0x7f7130005f90 comp rx=0 tx=0).stop 2024-09-19T04:44:34.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.702+0000 7f7149e00640 1 -- 172.21.15.123:0/3705130750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7144103e30 msgr2=0x7f714419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:34.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.702+0000 7f7149e00640 1 --2- 172.21.15.123:0/3705130750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7144103e30 0x7f714419a410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f713400e9f0 tx=0x7f713400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:44:34.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.702+0000 7f7149e00640 1 -- 172.21.15.123:0/3705130750 shutdown_connections 2024-09-19T04:44:34.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.702+0000 7f7149e00640 1 --2- 172.21.15.123:0/3705130750 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f71280a3240 0x7f71280a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:34.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.702+0000 7f7149e00640 1 --2- 172.21.15.123:0/3705130750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7144108f60 0x7f714419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:34.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.702+0000 7f7149e00640 1 --2- 172.21.15.123:0/3705130750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71441047e0 0x7f714419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:34.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.702+0000 7f7149e00640 1 --2- 172.21.15.123:0/3705130750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7144103e30 0x7f714419a410 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:34.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.702+0000 7f7149e00640 1 -- 172.21.15.123:0/3705130750 >> 172.21.15.123:0/3705130750 conn(0x7f71440ff8d0 msgr2=0x7f7144100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:34.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.702+0000 7f7149e00640 1 -- 172.21.15.123:0/3705130750 shutdown_connections 2024-09-19T04:44:34.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:34.702+0000 7f7149e00640 1 -- 172.21.15.123:0/3705130750 wait complete. 2024-09-19T04:44:35.051 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:34 smithi123 ceph-mon[30511]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T04:44:35.052 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:34 smithi123 ceph-mon[30511]: Cluster is now healthy 2024-09-19T04:44:35.052 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:34 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] up:active 2024-09-19T04:44:35.052 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:34 smithi123 ceph-mon[30511]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:44:35.052 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:34 smithi123 ceph-mon[30511]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:44:35.052 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:34 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3705130750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:44:35.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:34 smithi183 ceph-mon[35333]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T04:44:35.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:34 smithi183 ceph-mon[35333]: Cluster is now healthy 2024-09-19T04:44:35.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:34 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] up:active 2024-09-19T04:44:35.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:34 smithi183 ceph-mon[35333]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:44:35.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:34 smithi183 ceph-mon[35333]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:44:35.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:34 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3705130750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:44:35.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:34 smithi193 ceph-mon[35592]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T04:44:35.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:34 smithi193 ceph-mon[35592]: Cluster is now healthy 2024-09-19T04:44:35.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:34 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] up:active 2024-09-19T04:44:35.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:34 smithi193 ceph-mon[35592]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:44:35.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:34 smithi193 ceph-mon[35592]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:44:35.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:34 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3705130750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:44:35.399 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:44:35.768 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:35.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:35 smithi193 ceph-mon[35592]: pgmap v295: 97 pgs: 97 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 364 B/s wr, 0 op/s 2024-09-19T04:44:36.066 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:35 smithi123 ceph-mon[30511]: pgmap v295: 97 pgs: 97 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 364 B/s wr, 0 op/s 2024-09-19T04:44:36.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:35 smithi183 ceph-mon[35333]: pgmap v295: 97 pgs: 97 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 364 B/s wr, 0 op/s 2024-09-19T04:44:36.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.330+0000 7f2121c00640 1 -- 172.21.15.123:0/1397722962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f211c1049c0 msgr2=0x7f211c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:36.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.330+0000 7f2121c00640 1 --2- 172.21.15.123:0/1397722962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f211c1049c0 0x7f211c108a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2104009900 tx=0x7f210402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:44:36.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.330+0000 7f2121c00640 1 -- 172.21.15.123:0/1397722962 shutdown_connections 2024-09-19T04:44:36.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.330+0000 7f2121c00640 1 --2- 172.21.15.123:0/1397722962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f211c109140 0x7f211c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:36.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.330+0000 7f2121c00640 1 --2- 172.21.15.123:0/1397722962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f211c1049c0 0x7f211c108a10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:36.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.330+0000 7f2121c00640 1 --2- 172.21.15.123:0/1397722962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f211c104010 0x7f211c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:36.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.330+0000 7f2121c00640 1 -- 172.21.15.123:0/1397722962 >> 172.21.15.123:0/1397722962 conn(0x7f211c0ffa50 msgr2=0x7f211c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:36.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.330+0000 7f2121c00640 1 -- 172.21.15.123:0/1397722962 shutdown_connections 2024-09-19T04:44:36.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.330+0000 7f2121c00640 1 -- 172.21.15.123:0/1397722962 wait complete. 2024-09-19T04:44:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.331+0000 7f2121c00640 1 Processor -- start 2024-09-19T04:44:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.331+0000 7f2121c00640 1 -- start start 2024-09-19T04:44:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.332+0000 7f2121c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f211c104010 0x7f211c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.332+0000 7f2121c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f211c1049c0 0x7f211c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.332+0000 7f2121c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f211c109140 0x7f211c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.332+0000 7f2121c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f211c114190 con 0x7f211c1049c0 2024-09-19T04:44:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.332+0000 7f2121c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f211c114010 con 0x7f211c109140 2024-09-19T04:44:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.332+0000 7f2121c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f211c114310 con 0x7f211c104010 2024-09-19T04:44:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.332+0000 7f2113e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f211c1049c0 0x7f211c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.332+0000 7f211aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f211c104010 0x7f211c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.332+0000 7f2113e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f211c1049c0 0x7f211c19ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43860/0 (socket says 172.21.15.123:43860) 2024-09-19T04:44:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.332+0000 7f2113e00640 1 -- 172.21.15.123:0/1567121138 learned_addr learned my addr 172.21.15.123:0/1567121138 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.332+0000 7f211b400640 1 --2- 172.21.15.123:0/1567121138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f211c109140 0x7f211c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.332+0000 7f2113e00640 1 -- 172.21.15.123:0/1567121138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f211c104010 msgr2=0x7f211c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.332+0000 7f2113e00640 1 --2- 172.21.15.123:0/1567121138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f211c104010 0x7f211c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.332+0000 7f2113e00640 1 -- 172.21.15.123:0/1567121138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f211c109140 msgr2=0x7f211c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.332+0000 7f2113e00640 1 --2- 172.21.15.123:0/1567121138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f211c109140 0x7f211c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.332+0000 7f2113e00640 1 -- 172.21.15.123:0/1567121138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211c19f6a0 con 0x7f211c1049c0 2024-09-19T04:44:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.332+0000 7f211aa00640 1 --2- 172.21.15.123:0/1567121138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f211c104010 0x7f211c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:44:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.333+0000 7f2113e00640 1 --2- 172.21.15.123:0/1567121138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f211c1049c0 0x7f211c19ac30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2104004770 tx=0x7f2104003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.333+0000 7f2113400640 1 -- 172.21.15.123:0/1567121138 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210403f070 con 0x7f211c1049c0 2024-09-19T04:44:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.333+0000 7f2113400640 1 -- 172.21.15.123:0/1567121138 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2104031ca0 con 0x7f211c1049c0 2024-09-19T04:44:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.333+0000 7f2121c00640 1 -- 172.21.15.123:0/1567121138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211c19f930 con 0x7f211c1049c0 2024-09-19T04:44:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.333+0000 7f2113400640 1 -- 172.21.15.123:0/1567121138 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2104035030 con 0x7f211c1049c0 2024-09-19T04:44:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.333+0000 7f2121c00640 1 -- 172.21.15.123:0/1567121138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f211c1a7270 con 0x7f211c1049c0 2024-09-19T04:44:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.335+0000 7f2121c00640 1 -- 172.21.15.123:0/1567121138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f211c1054f0 con 0x7f211c1049c0 2024-09-19T04:44:36.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.337+0000 7f2113400640 1 -- 172.21.15.123:0/1567121138 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2104042070 con 0x7f211c1049c0 2024-09-19T04:44:36.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.338+0000 7f2113400640 1 --2- 172.21.15.123:0/1567121138 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f21000a3240 0x7f21000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:36.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.338+0000 7f2113400640 1 -- 172.21.15.123:0/1567121138 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f21040efb50 con 0x7f211c1049c0 2024-09-19T04:44:36.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.339+0000 7f211aa00640 1 --2- 172.21.15.123:0/1567121138 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f21000a3240 0x7f21000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:36.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.339+0000 7f211aa00640 1 --2- 172.21.15.123:0/1567121138 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f21000a3240 0x7f21000a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f210c0095f0 tx=0x7f210c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:36.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.342+0000 7f2113400640 1 -- 172.21.15.123:0/1567121138 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f21040bc2d0 con 0x7f211c1049c0 2024-09-19T04:44:36.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.520+0000 7f2121c00640 1 -- 172.21.15.123:0/1567121138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f211c10c380 con 0x7f211c1049c0 2024-09-19T04:44:36.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.522+0000 7f2113400640 1 -- 172.21.15.123:0/1567121138 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22362 (secure 0 0 0) 0x7f21040c05d0 con 0x7f211c1049c0 2024-09-19T04:44:36.524 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:44:36.524 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":84,"fsid":"911fa052-7640-11ef-bceb-c7b262605968","created":"2024-09-19T04:36:21.083333+0000","modified":"2024-09-19T04:44:31.766762+0000","last_up_change":"2024-09-19T04:43:33.818439+0000","last_in_change":"2024-09-19T04:43:10.148272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:39:33.856981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:44:26.956483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:44:29.718625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6803","nonce":3631596501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6805","nonce":3631596501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6809","nonce":3631596501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6807","nonce":3631596501}]},"public_addr":"172.21.15.123:6803/3631596501","cluster_addr":"172.21.15.123:6805/3631596501","heartbeat_back_addr":"172.21.15.123:6809/3631596501","heartbeat_front_addr":"172.21.15.123:6807/3631596501","state":["exists","up"]},{"osd":1,"uuid":"a526d4ce-586b-4d66-b720-e868df92fa24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6801","nonce":1259721614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6803","nonce":1259721614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6807","nonce":1259721614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6805","nonce":1259721614}]},"public_addr":"172.21.15.183:6801/1259721614","cluster_addr":"172.21.15.183:6803/1259721614","heartbeat_back_addr":"172.21.15.183:6807/1259721614","heartbeat_front_addr":"172.21.15.183:6805/1259721614","state":["exists","up"]},{"osd":2,"uuid":"3650601a-6259-4b2a-97df-1d47b09b0127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6801","nonce":2708151726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6803","nonce":2708151726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6807","nonce":2708151726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6805","nonce":2708151726}]},"public_addr":"172.21.15.193:6801/2708151726","cluster_addr":"172.21.15.193:6803/2708151726","heartbeat_back_addr":"172.21.15.193:6807/2708151726","heartbeat_front_addr":"172.21.15.193:6805/2708151726","state":["exists","up"]},{"osd":3,"uuid":"67c485ad-136e-4020-950d-9171a3c2de35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6811","nonce":3214806282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6813","nonce":3214806282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6817","nonce":3214806282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6815","nonce":3214806282}]},"public_addr":"172.21.15.123:6811/3214806282","cluster_addr":"172.21.15.123:6813/3214806282","heartbeat_back_addr":"172.21.15.123:6817/3214806282","heartbeat_front_addr":"172.21.15.123:6815/3214806282","state":["exists","up"]},{"osd":4,"uuid":"c9098540-06ab-45a8-9cf9-f90519cc852e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6809","nonce":667618665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6811","nonce":667618665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6815","nonce":667618665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6813","nonce":667618665}]},"public_addr":"172.21.15.183:6809/667618665","cluster_addr":"172.21.15.183:6811/667618665","heartbeat_back_addr":"172.21.15.183:6815/667618665","heartbeat_front_addr":"172.21.15.183:6813/667618665","state":["exists","up"]},{"osd":5,"uuid":"278a8dbd-6492-42d4-b489-1f832350bc6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6809","nonce":2292251639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6811","nonce":2292251639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6815","nonce":2292251639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6813","nonce":2292251639}]},"public_addr":"172.21.15.193:6809/2292251639","cluster_addr":"172.21.15.193:6811/2292251639","heartbeat_back_addr":"172.21.15.193:6815/2292251639","heartbeat_front_addr":"172.21.15.193:6813/2292251639","state":["exists","up"]},{"osd":6,"uuid":"0fda4e39-6adb-4941-a39b-a9a15019088c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6819","nonce":226007499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6821","nonce":226007499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6825","nonce":226007499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6823","nonce":226007499}]},"public_addr":"172.21.15.123:6819/226007499","cluster_addr":"172.21.15.123:6821/226007499","heartbeat_back_addr":"172.21.15.123:6825/226007499","heartbeat_front_addr":"172.21.15.123:6823/226007499","state":["exists","up"]},{"osd":7,"uuid":"5c0a021e-9f3b-4766-9686-0ef24b68cf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6817","nonce":3718518226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6819","nonce":3718518226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6823","nonce":3718518226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6821","nonce":3718518226}]},"public_addr":"172.21.15.183:6817/3718518226","cluster_addr":"172.21.15.183:6819/3718518226","heartbeat_back_addr":"172.21.15.183:6823/3718518226","heartbeat_front_addr":"172.21.15.183:6821/3718518226","state":["exists","up"]},{"osd":8,"uuid":"9b0e67e0-6305-4f81-9211-d41960099a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6817","nonce":3551404118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6819","nonce":3551404118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6823","nonce":3551404118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6821","nonce":3551404118}]},"public_addr":"172.21.15.193:6817/3551404118","cluster_addr":"172.21.15.193:6819/3551404118","heartbeat_back_addr":"172.21.15.193:6823/3551404118","heartbeat_front_addr":"172.21.15.193:6821/3551404118","state":["exists","up"]},{"osd":9,"uuid":"978ef3fa-c6ec-421f-b194-56d44813be5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6827","nonce":1439901149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6829","nonce":1439901149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6833","nonce":1439901149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6831","nonce":1439901149}]},"public_addr":"172.21.15.123:6827/1439901149","cluster_addr":"172.21.15.123:6829/1439901149","heartbeat_back_addr":"172.21.15.123:6833/1439901149","heartbeat_front_addr":"172.21.15.123:6831/1439901149","state":["exists","up"]},{"osd":10,"uuid":"5495e09b-7809-47b0-b7e0-b56c452f9f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6825","nonce":3355349245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6827","nonce":3355349245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6831","nonce":3355349245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6829","nonce":3355349245}]},"public_addr":"172.21.15.183:6825/3355349245","cluster_addr":"172.21.15.183:6827/3355349245","heartbeat_back_addr":"172.21.15.183:6831/3355349245","heartbeat_front_addr":"172.21.15.183:6829/3355349245","state":["exists","up"]},{"osd":11,"uuid":"03046fef-b16b-42f8-9a19-6c0b78cbfe7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6825","nonce":4291617038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6827","nonce":4291617038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6831","nonce":4291617038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6829","nonce":4291617038}]},"public_addr":"172.21.15.193:6825/4291617038","cluster_addr":"172.21.15.193:6827/4291617038","heartbeat_back_addr":"172.21.15.193:6831/4291617038","heartbeat_front_addr":"172.21.15.193:6829/4291617038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:38:39.142529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:05.336911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:30.310967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:59.655973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:25.890576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:50.778689+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:19.063731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:45.424269+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:10.562149+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:40.382368+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:06.297731+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:32.352172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/4040401634":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/3498196650":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6800/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:6800/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/471028951":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/1009496020":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/2000593686":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/2580747029":"2024-09-20T04:36:45.913060+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:44:36.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.524+0000 7f2121c00640 1 -- 172.21.15.123:0/1567121138 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f21000a3240 msgr2=0x7f21000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:36.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.524+0000 7f2121c00640 1 --2- 172.21.15.123:0/1567121138 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f21000a3240 0x7f21000a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f210c0095f0 tx=0x7f210c005f90 comp rx=0 tx=0).stop 2024-09-19T04:44:36.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.524+0000 7f2121c00640 1 -- 172.21.15.123:0/1567121138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f211c1049c0 msgr2=0x7f211c19ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:36.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.524+0000 7f2121c00640 1 --2- 172.21.15.123:0/1567121138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f211c1049c0 0x7f211c19ac30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2104004770 tx=0x7f2104003c10 comp rx=0 tx=0).stop 2024-09-19T04:44:36.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.525+0000 7f2121c00640 1 -- 172.21.15.123:0/1567121138 shutdown_connections 2024-09-19T04:44:36.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.525+0000 7f2121c00640 1 --2- 172.21.15.123:0/1567121138 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f21000a3240 0x7f21000a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:36.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.525+0000 7f2121c00640 1 --2- 172.21.15.123:0/1567121138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f211c109140 0x7f211c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:36.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.525+0000 7f2121c00640 1 --2- 172.21.15.123:0/1567121138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f211c1049c0 0x7f211c19ac30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:36.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.525+0000 7f2121c00640 1 --2- 172.21.15.123:0/1567121138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f211c104010 0x7f211c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:36.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.525+0000 7f2121c00640 1 -- 172.21.15.123:0/1567121138 >> 172.21.15.123:0/1567121138 conn(0x7f211c0ffa50 msgr2=0x7f211c100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:36.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.525+0000 7f2121c00640 1 -- 172.21.15.123:0/1567121138 shutdown_connections 2024-09-19T04:44:36.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:36.525+0000 7f2121c00640 1 -- 172.21.15.123:0/1567121138 wait complete. 2024-09-19T04:44:37.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:36 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1567121138' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:44:37.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:36 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1567121138' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:44:37.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:36 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1567121138' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:44:37.276 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-19T04:44:37.645 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:38.057 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:37 smithi123 ceph-mon[30511]: pgmap v296: 97 pgs: 97 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 309 B/s wr, 1 op/s 2024-09-19T04:44:38.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:37 smithi183 ceph-mon[35333]: pgmap v296: 97 pgs: 97 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 309 B/s wr, 1 op/s 2024-09-19T04:44:38.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.167+0000 7f9d4be00640 1 -- 172.21.15.123:0/4205884820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d441049a0 msgr2=0x7f9d441089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:38.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.167+0000 7f9d4be00640 1 --2- 172.21.15.123:0/4205884820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d441049a0 0x7f9d441089f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009900 tx=0x7f9d3002a410 comp rx=0 tx=0).stop 2024-09-19T04:44:38.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.167+0000 7f9d4be00640 1 -- 172.21.15.123:0/4205884820 shutdown_connections 2024-09-19T04:44:38.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.167+0000 7f9d4be00640 1 --2- 172.21.15.123:0/4205884820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d44109120 0x7f9d44110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:38.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.167+0000 7f9d4be00640 1 --2- 172.21.15.123:0/4205884820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d441049a0 0x7f9d441089f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:38.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.167+0000 7f9d4be00640 1 --2- 172.21.15.123:0/4205884820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d44103ff0 0x7f9d441043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:38.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.167+0000 7f9d4be00640 1 -- 172.21.15.123:0/4205884820 >> 172.21.15.123:0/4205884820 conn(0x7f9d440ffa90 msgr2=0x7f9d44101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:38.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.167+0000 7f9d4be00640 1 -- 172.21.15.123:0/4205884820 shutdown_connections 2024-09-19T04:44:38.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.167+0000 7f9d4be00640 1 -- 172.21.15.123:0/4205884820 wait complete. 2024-09-19T04:44:38.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.168+0000 7f9d4be00640 1 Processor -- start 2024-09-19T04:44:38.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.168+0000 7f9d4be00640 1 -- start start 2024-09-19T04:44:38.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.168+0000 7f9d4be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d44103ff0 0x7f9d4419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:38.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.168+0000 7f9d4be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d441049a0 0x7f9d4419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:38.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.169+0000 7f9d4be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d44109120 0x7f9d4419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:38.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.169+0000 7f9d4be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d44114140 con 0x7f9d441049a0 2024-09-19T04:44:38.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.169+0000 7f9d4be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9d44113fc0 con 0x7f9d44109120 2024-09-19T04:44:38.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.169+0000 7f9d4be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9d441142c0 con 0x7f9d44103ff0 2024-09-19T04:44:38.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.169+0000 7f9d48a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d441049a0 0x7f9d4419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:38.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.169+0000 7f9d49400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d44103ff0 0x7f9d4419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:38.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.169+0000 7f9d49400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d44103ff0 0x7f9d4419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:57386/0 (socket says 172.21.15.123:57386) 2024-09-19T04:44:38.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.169+0000 7f9d49400640 1 -- 172.21.15.123:0/1092807277 learned_addr learned my addr 172.21.15.123:0/1092807277 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:38.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.169+0000 7f9d49e00640 1 --2- 172.21.15.123:0/1092807277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d44109120 0x7f9d4419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:38.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.169+0000 7f9d49400640 1 -- 172.21.15.123:0/1092807277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d44109120 msgr2=0x7f9d4419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:38.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.169+0000 7f9d49400640 1 --2- 172.21.15.123:0/1092807277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d44109120 0x7f9d4419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:38.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.169+0000 7f9d49400640 1 -- 172.21.15.123:0/1092807277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d441049a0 msgr2=0x7f9d4419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:38.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.169+0000 7f9d49400640 1 --2- 172.21.15.123:0/1092807277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d441049a0 0x7f9d4419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:38.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.169+0000 7f9d49400640 1 -- 172.21.15.123:0/1092807277 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4419f650 con 0x7f9d44103ff0 2024-09-19T04:44:38.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.169+0000 7f9d48a00640 1 --2- 172.21.15.123:0/1092807277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d441049a0 0x7f9d4419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:44:38.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.170+0000 7f9d49400640 1 --2- 172.21.15.123:0/1092807277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d44103ff0 0x7f9d4419a6a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800e9f0 tx=0x7f9d3800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:38.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.170+0000 7f9d42000640 1 -- 172.21.15.123:0/1092807277 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3800cd80 con 0x7f9d44103ff0 2024-09-19T04:44:38.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.170+0000 7f9d4be00640 1 -- 172.21.15.123:0/1092807277 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d4419f940 con 0x7f9d44103ff0 2024-09-19T04:44:38.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.170+0000 7f9d42000640 1 -- 172.21.15.123:0/1092807277 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d38010910 con 0x7f9d44103ff0 2024-09-19T04:44:38.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.170+0000 7f9d42000640 1 -- 172.21.15.123:0/1092807277 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d380188f0 con 0x7f9d44103ff0 2024-09-19T04:44:38.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.170+0000 7f9d4be00640 1 -- 172.21.15.123:0/1092807277 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d441a7220 con 0x7f9d44103ff0 2024-09-19T04:44:38.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.171+0000 7f9d4be00640 1 -- 172.21.15.123:0/1092807277 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d441043d0 con 0x7f9d44103ff0 2024-09-19T04:44:38.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.173+0000 7f9d42000640 1 -- 172.21.15.123:0/1092807277 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9d38010ac0 con 0x7f9d44103ff0 2024-09-19T04:44:38.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.174+0000 7f9d42000640 1 --2- 172.21.15.123:0/1092807277 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9d2c0a32c0 0x7f9d2c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:38.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.174+0000 7f9d42000640 1 -- 172.21.15.123:0/1092807277 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f9d380cfba0 con 0x7f9d44103ff0 2024-09-19T04:44:38.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.175+0000 7f9d48a00640 1 --2- 172.21.15.123:0/1092807277 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9d2c0a32c0 0x7f9d2c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:38.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.175+0000 7f9d48a00640 1 --2- 172.21.15.123:0/1092807277 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9d2c0a32c0 0x7f9d2c0a5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009230 tx=0x7f9d300058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:38.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.178+0000 7f9d42000640 1 -- 172.21.15.123:0/1092807277 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9d3809c320 con 0x7f9d44103ff0 2024-09-19T04:44:38.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:37 smithi193 ceph-mon[35592]: pgmap v296: 97 pgs: 97 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 309 B/s wr, 1 op/s 2024-09-19T04:44:38.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.388+0000 7f9d4be00640 1 -- 172.21.15.123:0/1092807277 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9d4410c360 con 0x7f9d44103ff0 2024-09-19T04:44:38.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.782+0000 7f9d42000640 1 -- 172.21.15.123:0/1092807277 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f9d380a0620 con 0x7f9d44103ff0 2024-09-19T04:44:38.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.785+0000 7f9d4be00640 1 -- 172.21.15.123:0/1092807277 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9d2c0a32c0 msgr2=0x7f9d2c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:38.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.785+0000 7f9d4be00640 1 --2- 172.21.15.123:0/1092807277 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9d2c0a32c0 0x7f9d2c0a5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009230 tx=0x7f9d300058d0 comp rx=0 tx=0).stop 2024-09-19T04:44:38.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.785+0000 7f9d4be00640 1 -- 172.21.15.123:0/1092807277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d44103ff0 msgr2=0x7f9d4419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:38.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.785+0000 7f9d4be00640 1 --2- 172.21.15.123:0/1092807277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d44103ff0 0x7f9d4419a6a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800e9f0 tx=0x7f9d3800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:44:38.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.785+0000 7f9d4be00640 1 -- 172.21.15.123:0/1092807277 shutdown_connections 2024-09-19T04:44:38.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.785+0000 7f9d4be00640 1 --2- 172.21.15.123:0/1092807277 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9d2c0a32c0 0x7f9d2c0a5780 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:38.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.785+0000 7f9d4be00640 1 --2- 172.21.15.123:0/1092807277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d44109120 0x7f9d4419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:38.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.785+0000 7f9d4be00640 1 --2- 172.21.15.123:0/1092807277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d441049a0 0x7f9d4419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:38.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.785+0000 7f9d4be00640 1 --2- 172.21.15.123:0/1092807277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d44103ff0 0x7f9d4419a6a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:38.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.785+0000 7f9d4be00640 1 -- 172.21.15.123:0/1092807277 >> 172.21.15.123:0/1092807277 conn(0x7f9d440ffa90 msgr2=0x7f9d44100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:38.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.785+0000 7f9d4be00640 1 -- 172.21.15.123:0/1092807277 shutdown_connections 2024-09-19T04:44:38.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:38.785+0000 7f9d4be00640 1 -- 172.21.15.123:0/1092807277 wait complete. 2024-09-19T04:44:39.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:38 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1092807277' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T04:44:39.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:38 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T04:44:39.137 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:38 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1092807277' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T04:44:39.137 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:38 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T04:44:39.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:38 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1092807277' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T04:44:39.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:38 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T04:44:39.544 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-19T04:44:39.544 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-19T04:44:39.917 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:40.056 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:39 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T04:44:40.056 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:39 smithi123 ceph-mon[30511]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:44:40.056 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:39 smithi123 ceph-mon[30511]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T04:44:40.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:39 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T04:44:40.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:39 smithi183 ceph-mon[35333]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:44:40.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:39 smithi183 ceph-mon[35333]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T04:44:40.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:39 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T04:44:40.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:39 smithi193 ceph-mon[35592]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:44:40.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:39 smithi193 ceph-mon[35592]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T04:44:40.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.442+0000 7ff63a200640 1 -- 172.21.15.123:0/596276479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff634102790 msgr2=0x7ff6341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:40.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.442+0000 7ff63a200640 1 --2- 172.21.15.123:0/596276479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff634102790 0x7ff6341067e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009900 tx=0x7ff61c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:44:40.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.443+0000 7ff63a200640 1 -- 172.21.15.123:0/596276479 shutdown_connections 2024-09-19T04:44:40.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.443+0000 7ff63a200640 1 --2- 172.21.15.123:0/596276479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff634106f10 0x7ff63410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:40.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.443+0000 7ff63a200640 1 --2- 172.21.15.123:0/596276479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff634102790 0x7ff6341067e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:40.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.443+0000 7ff63a200640 1 --2- 172.21.15.123:0/596276479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff634101de0 0x7ff6341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:40.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.443+0000 7ff63a200640 1 -- 172.21.15.123:0/596276479 >> 172.21.15.123:0/596276479 conn(0x7ff6340fd8c0 msgr2=0x7ff6340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:40.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.443+0000 7ff63a200640 1 -- 172.21.15.123:0/596276479 shutdown_connections 2024-09-19T04:44:40.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.443+0000 7ff63a200640 1 -- 172.21.15.123:0/596276479 wait complete. 2024-09-19T04:44:40.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.444+0000 7ff63a200640 1 Processor -- start 2024-09-19T04:44:40.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.444+0000 7ff63a200640 1 -- start start 2024-09-19T04:44:40.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.444+0000 7ff63a200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff634101de0 0x7ff63419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:40.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.444+0000 7ff63a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff634102790 0x7ff63419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:40.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.444+0000 7ff63a200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff634106f10 0x7ff63419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:40.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.444+0000 7ff63a200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff634111fc0 con 0x7ff634102790 2024-09-19T04:44:40.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.444+0000 7ff63a200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff634111e40 con 0x7ff634101de0 2024-09-19T04:44:40.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.444+0000 7ff63a200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff634112140 con 0x7ff634106f10 2024-09-19T04:44:40.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.444+0000 7ff62aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff634102790 0x7ff63419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:40.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.444+0000 7ff62aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff634102790 0x7ff63419aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43888/0 (socket says 172.21.15.123:43888) 2024-09-19T04:44:40.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.444+0000 7ff62aa00640 1 -- 172.21.15.123:0/2693120401 learned_addr learned my addr 172.21.15.123:0/2693120401 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:40.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.444+0000 7ff633400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff634101de0 0x7ff63419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:40.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.445+0000 7ff633e00640 1 --2- 172.21.15.123:0/2693120401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff634106f10 0x7ff63419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:40.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.445+0000 7ff62aa00640 1 -- 172.21.15.123:0/2693120401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff634106f10 msgr2=0x7ff63419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:40.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.445+0000 7ff62aa00640 1 --2- 172.21.15.123:0/2693120401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff634106f10 0x7ff63419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:40.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.445+0000 7ff62aa00640 1 -- 172.21.15.123:0/2693120401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff634101de0 msgr2=0x7ff63419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:40.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.445+0000 7ff62aa00640 1 --2- 172.21.15.123:0/2693120401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff634101de0 0x7ff63419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:40.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.445+0000 7ff62aa00640 1 -- 172.21.15.123:0/2693120401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63419f610 con 0x7ff634102790 2024-09-19T04:44:40.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.445+0000 7ff62aa00640 1 --2- 172.21.15.123:0/2693120401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff634102790 0x7ff63419aba0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff61c004770 tx=0x7ff61c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:40.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.445+0000 7ff630c00640 1 -- 172.21.15.123:0/2693120401 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c03f070 con 0x7ff634102790 2024-09-19T04:44:40.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.445+0000 7ff630c00640 1 -- 172.21.15.123:0/2693120401 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff61c031ca0 con 0x7ff634102790 2024-09-19T04:44:40.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.445+0000 7ff630c00640 1 -- 172.21.15.123:0/2693120401 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c035030 con 0x7ff634102790 2024-09-19T04:44:40.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.446+0000 7ff63a200640 1 -- 172.21.15.123:0/2693120401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63419f8a0 con 0x7ff634102790 2024-09-19T04:44:40.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.446+0000 7ff63a200640 1 -- 172.21.15.123:0/2693120401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63419fd30 con 0x7ff634102790 2024-09-19T04:44:40.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.447+0000 7ff63a200640 1 -- 172.21.15.123:0/2693120401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6341021c0 con 0x7ff634102790 2024-09-19T04:44:40.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.449+0000 7ff630c00640 1 -- 172.21.15.123:0/2693120401 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff61c042070 con 0x7ff634102790 2024-09-19T04:44:40.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.450+0000 7ff630c00640 1 --2- 172.21.15.123:0/2693120401 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff6180a32c0 0x7ff6180a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:40.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.450+0000 7ff630c00640 1 -- 172.21.15.123:0/2693120401 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7ff61c0efe30 con 0x7ff634102790 2024-09-19T04:44:40.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.450+0000 7ff633400640 1 --2- 172.21.15.123:0/2693120401 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff6180a32c0 0x7ff6180a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:40.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.451+0000 7ff633400640 1 --2- 172.21.15.123:0/2693120401 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff6180a32c0 0x7ff6180a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff624005d00 tx=0x7ff62400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:40.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.454+0000 7ff630c00640 1 -- 172.21.15.123:0/2693120401 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff61c0bc5b0 con 0x7ff634102790 2024-09-19T04:44:40.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.661+0000 7ff63a200640 1 -- 172.21.15.123:0/2693120401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7ff6341a73a0 con 0x7ff634102790 2024-09-19T04:44:40.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.791+0000 7ff630c00640 1 -- 172.21.15.123:0/2693120401 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7ff61c0c08b0 con 0x7ff634102790 2024-09-19T04:44:40.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.793+0000 7ff63a200640 1 -- 172.21.15.123:0/2693120401 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff6180a32c0 msgr2=0x7ff6180a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:40.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.793+0000 7ff63a200640 1 --2- 172.21.15.123:0/2693120401 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff6180a32c0 0x7ff6180a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff624005d00 tx=0x7ff62400b410 comp rx=0 tx=0).stop 2024-09-19T04:44:40.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.793+0000 7ff63a200640 1 -- 172.21.15.123:0/2693120401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff634102790 msgr2=0x7ff63419aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:40.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.793+0000 7ff63a200640 1 --2- 172.21.15.123:0/2693120401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff634102790 0x7ff63419aba0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff61c004770 tx=0x7ff61c003c10 comp rx=0 tx=0).stop 2024-09-19T04:44:40.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.793+0000 7ff63a200640 1 -- 172.21.15.123:0/2693120401 shutdown_connections 2024-09-19T04:44:40.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.793+0000 7ff63a200640 1 --2- 172.21.15.123:0/2693120401 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff6180a32c0 0x7ff6180a5780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:40.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.793+0000 7ff63a200640 1 --2- 172.21.15.123:0/2693120401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff634106f10 0x7ff63419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:40.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.793+0000 7ff63a200640 1 --2- 172.21.15.123:0/2693120401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff634102790 0x7ff63419aba0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:40.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.793+0000 7ff63a200640 1 --2- 172.21.15.123:0/2693120401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff634101de0 0x7ff63419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:40.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.793+0000 7ff63a200640 1 -- 172.21.15.123:0/2693120401 >> 172.21.15.123:0/2693120401 conn(0x7ff6340fd8c0 msgr2=0x7ff6340feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:40.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.793+0000 7ff63a200640 1 -- 172.21.15.123:0/2693120401 shutdown_connections 2024-09-19T04:44:40.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:40.794+0000 7ff63a200640 1 -- 172.21.15.123:0/2693120401 wait complete. 2024-09-19T04:44:40.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:40 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2693120401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T04:44:41.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:40 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2693120401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T04:44:41.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:40 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2693120401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T04:44:41.600 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-19T04:44:41.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:41 smithi123 ceph-mon[30511]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T04:44:41.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:41 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2693120401' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T04:44:41.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:41 smithi123 ceph-mon[30511]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T04:44:41.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:41 smithi123 ceph-mon[30511]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:44:41.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:41 smithi123 ceph-mon[30511]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T04:44:41.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:41 smithi123 ceph-mon[30511]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T04:44:41.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:41 smithi123 ceph-mon[30511]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-19T04:44:41.974 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:42.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:41 smithi183 ceph-mon[35333]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T04:44:42.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:41 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2693120401' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T04:44:42.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:41 smithi183 ceph-mon[35333]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T04:44:42.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:41 smithi183 ceph-mon[35333]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:44:42.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:41 smithi183 ceph-mon[35333]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T04:44:42.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:41 smithi183 ceph-mon[35333]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T04:44:42.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:41 smithi183 ceph-mon[35333]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-19T04:44:42.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:41 smithi193 ceph-mon[35592]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T04:44:42.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:41 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2693120401' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T04:44:42.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:41 smithi193 ceph-mon[35592]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T04:44:42.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:41 smithi193 ceph-mon[35592]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:44:42.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:41 smithi193 ceph-mon[35592]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T04:44:42.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:41 smithi193 ceph-mon[35592]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T04:44:42.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:41 smithi193 ceph-mon[35592]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-19T04:44:42.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.542+0000 7f1853600640 1 -- 172.21.15.123:0/3425237968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184c106f10 msgr2=0x7f184c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:42.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.542+0000 7f1853600640 1 --2- 172.21.15.123:0/3425237968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184c106f10 0x7f184c10ea90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1844009ca0 tx=0x7f184402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:44:42.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.542+0000 7f1853600640 1 -- 172.21.15.123:0/3425237968 shutdown_connections 2024-09-19T04:44:42.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.542+0000 7f1853600640 1 --2- 172.21.15.123:0/3425237968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184c106f10 0x7f184c10ea90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:42.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.542+0000 7f1853600640 1 --2- 172.21.15.123:0/3425237968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c102790 0x7f184c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:42.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.542+0000 7f1853600640 1 --2- 172.21.15.123:0/3425237968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f184c101de0 0x7f184c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:42.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.542+0000 7f1853600640 1 -- 172.21.15.123:0/3425237968 >> 172.21.15.123:0/3425237968 conn(0x7f184c0fd8c0 msgr2=0x7f184c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:42.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.542+0000 7f1853600640 1 -- 172.21.15.123:0/3425237968 shutdown_connections 2024-09-19T04:44:42.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.542+0000 7f1853600640 1 -- 172.21.15.123:0/3425237968 wait complete. 2024-09-19T04:44:42.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.544+0000 7f1853600640 1 Processor -- start 2024-09-19T04:44:42.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.544+0000 7f1853600640 1 -- start start 2024-09-19T04:44:42.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.544+0000 7f1853600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f184c101de0 0x7f184c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:42.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.544+0000 7f1853600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c102790 0x7f184c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:42.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.544+0000 7f1853600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184c106f10 0x7f184c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:42.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.544+0000 7f1853600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f184c111f70 con 0x7f184c106f10 2024-09-19T04:44:42.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.544+0000 7f1853600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f184c111df0 con 0x7f184c102790 2024-09-19T04:44:42.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.544+0000 7f1853600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f184c1120f0 con 0x7f184c101de0 2024-09-19T04:44:42.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.545+0000 7f1850c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f184c101de0 0x7f184c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:42.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.545+0000 7f184be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c102790 0x7f184c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:42.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.545+0000 7f1850c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f184c101de0 0x7f184c1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:51056/0 (socket says 172.21.15.123:51056) 2024-09-19T04:44:42.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.545+0000 7f1850c00640 1 -- 172.21.15.123:0/774078377 learned_addr learned my addr 172.21.15.123:0/774078377 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:42.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.545+0000 7f1851600640 1 --2- 172.21.15.123:0/774078377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184c106f10 0x7f184c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:42.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.545+0000 7f184be00640 1 -- 172.21.15.123:0/774078377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f184c101de0 msgr2=0x7f184c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:42.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.545+0000 7f184be00640 1 --2- 172.21.15.123:0/774078377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f184c101de0 0x7f184c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:42.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.545+0000 7f184be00640 1 -- 172.21.15.123:0/774078377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184c106f10 msgr2=0x7f184c19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:42.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.545+0000 7f184be00640 1 --2- 172.21.15.123:0/774078377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184c106f10 0x7f184c19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:42.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.545+0000 7f184be00640 1 -- 172.21.15.123:0/774078377 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184c19d370 con 0x7f184c102790 2024-09-19T04:44:42.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.545+0000 7f184be00640 1 --2- 172.21.15.123:0/774078377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c102790 0x7f184c198900 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f183800e9f0 tx=0x7f183800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:42.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.546+0000 7f1849600640 1 -- 172.21.15.123:0/774078377 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183800ccc0 con 0x7f184c102790 2024-09-19T04:44:42.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.546+0000 7f1853600640 1 -- 172.21.15.123:0/774078377 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184c19d660 con 0x7f184c102790 2024-09-19T04:44:42.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.546+0000 7f1849600640 1 -- 172.21.15.123:0/774078377 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1838010910 con 0x7f184c102790 2024-09-19T04:44:42.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.546+0000 7f1849600640 1 -- 172.21.15.123:0/774078377 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838018960 con 0x7f184c102790 2024-09-19T04:44:42.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.546+0000 7f1853600640 1 -- 172.21.15.123:0/774078377 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184c1a4f40 con 0x7f184c102790 2024-09-19T04:44:42.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.547+0000 7f1853600640 1 -- 172.21.15.123:0/774078377 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f184c1021c0 con 0x7f184c102790 2024-09-19T04:44:42.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.549+0000 7f1849600640 1 -- 172.21.15.123:0/774078377 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f1838010ac0 con 0x7f184c102790 2024-09-19T04:44:42.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.551+0000 7f1849600640 1 --2- 172.21.15.123:0/774078377 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f18340a3240 0x7f18340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:42.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.551+0000 7f1849600640 1 -- 172.21.15.123:0/774078377 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f18380cfff0 con 0x7f184c102790 2024-09-19T04:44:42.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.551+0000 7f1850c00640 1 --2- 172.21.15.123:0/774078377 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f18340a3240 0x7f18340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:42.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.551+0000 7f1850c00640 1 --2- 172.21.15.123:0/774078377 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f18340a3240 0x7f18340a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1840009cd0 tx=0x7f1840005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:42.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.552+0000 7f1849600640 1 -- 172.21.15.123:0/774078377 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f183809c790 con 0x7f184c102790 2024-09-19T04:44:42.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.768+0000 7f1853600640 1 -- 172.21.15.123:0/774078377 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f184c10a1b0 con 0x7f184c102790 2024-09-19T04:44:42.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.815+0000 7f1849600640 1 -- 172.21.15.123:0/774078377 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f18380a0a90 con 0x7f184c102790 2024-09-19T04:44:42.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.817+0000 7f1853600640 1 -- 172.21.15.123:0/774078377 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f18340a3240 msgr2=0x7f18340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:42.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.817+0000 7f1853600640 1 --2- 172.21.15.123:0/774078377 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f18340a3240 0x7f18340a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1840009cd0 tx=0x7f1840005f90 comp rx=0 tx=0).stop 2024-09-19T04:44:42.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.817+0000 7f1853600640 1 -- 172.21.15.123:0/774078377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c102790 msgr2=0x7f184c198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:42.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.817+0000 7f1853600640 1 --2- 172.21.15.123:0/774078377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c102790 0x7f184c198900 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f183800e9f0 tx=0x7f183800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:44:42.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.817+0000 7f1853600640 1 -- 172.21.15.123:0/774078377 shutdown_connections 2024-09-19T04:44:42.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.817+0000 7f1853600640 1 --2- 172.21.15.123:0/774078377 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f18340a3240 0x7f18340a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:42.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.817+0000 7f1853600640 1 --2- 172.21.15.123:0/774078377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184c106f10 0x7f184c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:42.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.817+0000 7f1853600640 1 --2- 172.21.15.123:0/774078377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c102790 0x7f184c198900 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:42.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.817+0000 7f1853600640 1 --2- 172.21.15.123:0/774078377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f184c101de0 0x7f184c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:42.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.817+0000 7f1853600640 1 -- 172.21.15.123:0/774078377 >> 172.21.15.123:0/774078377 conn(0x7f184c0fd8c0 msgr2=0x7f184c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:42.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.818+0000 7f1853600640 1 -- 172.21.15.123:0/774078377 shutdown_connections 2024-09-19T04:44:42.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:42.818+0000 7f1853600640 1 -- 172.21.15.123:0/774078377 wait complete. 2024-09-19T04:44:43.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:42 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] up:active 2024-09-19T04:44:43.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:42 smithi183 ceph-mon[35333]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T04:44:43.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:42 smithi183 ceph-mon[35333]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T04:44:43.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:42 smithi183 ceph-mon[35333]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-19T04:44:43.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:42 smithi183 ceph-mon[35333]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T04:44:43.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:42 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/774078377' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T04:44:43.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:42 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T04:44:43.155 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:42 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] up:active 2024-09-19T04:44:43.155 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:42 smithi123 ceph-mon[30511]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T04:44:43.155 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:42 smithi123 ceph-mon[30511]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T04:44:43.155 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:42 smithi123 ceph-mon[30511]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-19T04:44:43.155 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:42 smithi123 ceph-mon[30511]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T04:44:43.155 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:42 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/774078377' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T04:44:43.155 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:42 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T04:44:43.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:42 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] up:active 2024-09-19T04:44:43.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:42 smithi193 ceph-mon[35592]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T04:44:43.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:42 smithi193 ceph-mon[35592]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T04:44:43.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:42 smithi193 ceph-mon[35592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-19T04:44:43.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:42 smithi193 ceph-mon[35592]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T04:44:43.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:42 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/774078377' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T04:44:43.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:42 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T04:44:43.618 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-19T04:44:43.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:43 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T04:44:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:43 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] up:active 2024-09-19T04:44:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:43 smithi123 ceph-mon[30511]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T04:44:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:43 smithi123 ceph-mon[30511]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-19T04:44:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:43 smithi123 ceph-mon[30511]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-19T04:44:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:43 smithi123 ceph-mon[30511]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-19T04:44:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:43 smithi123 ceph-mon[30511]: pgmap v299: 97 pgs: 97 active+clean; 580 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-09-19T04:44:43.990 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:44.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:43 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T04:44:44.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:43 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] up:active 2024-09-19T04:44:44.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:43 smithi183 ceph-mon[35333]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T04:44:44.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:43 smithi183 ceph-mon[35333]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-19T04:44:44.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:43 smithi183 ceph-mon[35333]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-19T04:44:44.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:43 smithi183 ceph-mon[35333]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-19T04:44:44.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:43 smithi183 ceph-mon[35333]: pgmap v299: 97 pgs: 97 active+clean; 580 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-09-19T04:44:44.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:43 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T04:44:44.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:43 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] up:active 2024-09-19T04:44:44.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:43 smithi193 ceph-mon[35592]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T04:44:44.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:43 smithi193 ceph-mon[35592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-19T04:44:44.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:43 smithi193 ceph-mon[35592]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-19T04:44:44.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:43 smithi193 ceph-mon[35592]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-19T04:44:44.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:43 smithi193 ceph-mon[35592]: pgmap v299: 97 pgs: 97 active+clean; 580 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-09-19T04:44:44.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.580+0000 7f6cdc200640 1 -- 172.21.15.123:0/2331552825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cd4106f10 msgr2=0x7f6cd410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:44.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.580+0000 7f6cdc200640 1 --2- 172.21.15.123:0/2331552825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cd4106f10 0x7f6cd410ea90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009900 tx=0x7f6ccc02a400 comp rx=0 tx=0).stop 2024-09-19T04:44:44.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.580+0000 7f6cdc200640 1 -- 172.21.15.123:0/2331552825 shutdown_connections 2024-09-19T04:44:44.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.580+0000 7f6cdc200640 1 --2- 172.21.15.123:0/2331552825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cd4106f10 0x7f6cd410ea90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:44.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.580+0000 7f6cdc200640 1 --2- 172.21.15.123:0/2331552825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cd4102790 0x7f6cd41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:44.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.580+0000 7f6cdc200640 1 --2- 172.21.15.123:0/2331552825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cd4101de0 0x7f6cd41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:44.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.580+0000 7f6cdc200640 1 -- 172.21.15.123:0/2331552825 >> 172.21.15.123:0/2331552825 conn(0x7f6cd40fd8c0 msgr2=0x7f6cd40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:44.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.580+0000 7f6cdc200640 1 -- 172.21.15.123:0/2331552825 shutdown_connections 2024-09-19T04:44:44.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.580+0000 7f6cdc200640 1 -- 172.21.15.123:0/2331552825 wait complete. 2024-09-19T04:44:44.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.581+0000 7f6cdc200640 1 Processor -- start 2024-09-19T04:44:44.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.581+0000 7f6cdc200640 1 -- start start 2024-09-19T04:44:44.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.581+0000 7f6cdc200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cd4101de0 0x7f6cd419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.581+0000 7f6cdc200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cd4102790 0x7f6cd419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.581+0000 7f6cdc200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cd4106f10 0x7f6cd419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.581+0000 7f6cdc200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd4111f70 con 0x7f6cd4102790 2024-09-19T04:44:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.581+0000 7f6cdc200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd4111df0 con 0x7f6cd4106f10 2024-09-19T04:44:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.581+0000 7f6cdc200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd41120f0 con 0x7f6cd4101de0 2024-09-19T04:44:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.582+0000 7f6cd8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cd4102790 0x7f6cd419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.582+0000 7f6cd9800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cd4101de0 0x7f6cd419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.582+0000 7f6cda200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cd4106f10 0x7f6cd419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.582+0000 7f6cd9800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cd4101de0 0x7f6cd419a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:51086/0 (socket says 172.21.15.123:51086) 2024-09-19T04:44:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.582+0000 7f6cd9800640 1 -- 172.21.15.123:0/2608731129 learned_addr learned my addr 172.21.15.123:0/2608731129 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.582+0000 7f6cda200640 1 -- 172.21.15.123:0/2608731129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cd4101de0 msgr2=0x7f6cd419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.582+0000 7f6cda200640 1 --2- 172.21.15.123:0/2608731129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cd4101de0 0x7f6cd419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:44.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.582+0000 7f6cda200640 1 -- 172.21.15.123:0/2608731129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cd4102790 msgr2=0x7f6cd419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:44.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.582+0000 7f6cda200640 1 --2- 172.21.15.123:0/2608731129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cd4102790 0x7f6cd419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:44.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.582+0000 7f6cda200640 1 -- 172.21.15.123:0/2608731129 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd419f580 con 0x7f6cd4106f10 2024-09-19T04:44:44.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.583+0000 7f6cda200640 1 --2- 172.21.15.123:0/2608731129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cd4106f10 0x7f6cd419eea0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009230 tx=0x7f6ccc005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:44.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.583+0000 7f6cd2000640 1 -- 172.21.15.123:0/2608731129 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc03f030 con 0x7f6cd4106f10 2024-09-19T04:44:44.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.583+0000 7f6cdc200640 1 -- 172.21.15.123:0/2608731129 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd419f810 con 0x7f6cd4106f10 2024-09-19T04:44:44.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.583+0000 7f6cd2000640 1 -- 172.21.15.123:0/2608731129 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ccc004350 con 0x7f6cd4106f10 2024-09-19T04:44:44.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.583+0000 7f6cdc200640 1 -- 172.21.15.123:0/2608731129 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd41a7150 con 0x7f6cd4106f10 2024-09-19T04:44:44.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.583+0000 7f6cd2000640 1 -- 172.21.15.123:0/2608731129 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc049060 con 0x7f6cd4106f10 2024-09-19T04:44:44.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.584+0000 7f6cdc200640 1 -- 172.21.15.123:0/2608731129 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd41021c0 con 0x7f6cd4106f10 2024-09-19T04:44:44.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.586+0000 7f6cd2000640 1 -- 172.21.15.123:0/2608731129 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f6ccc042030 con 0x7f6cd4106f10 2024-09-19T04:44:44.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.587+0000 7f6cd2000640 1 --2- 172.21.15.123:0/2608731129 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6cbc0a32c0 0x7f6cbc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:44.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.587+0000 7f6cd2000640 1 -- 172.21.15.123:0/2608731129 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f6ccc0efa40 con 0x7f6cd4106f10 2024-09-19T04:44:44.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.588+0000 7f6cd9800640 1 --2- 172.21.15.123:0/2608731129 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6cbc0a32c0 0x7f6cbc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:44.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.588+0000 7f6cd9800640 1 --2- 172.21.15.123:0/2608731129 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6cbc0a32c0 0x7f6cbc0a5780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8009fd0 tx=0x7f6cc8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:44.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.589+0000 7f6cd2000640 1 -- 172.21.15.123:0/2608731129 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6ccc0bc1c0 con 0x7f6cd4106f10 2024-09-19T04:44:44.600 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:44:44 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-b[65203]: 2024-09-19T04:44:44.314+0000 7fbe924d0640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-19T04:44:44.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.802+0000 7f6cdc200640 1 -- 172.21.15.123:0/2608731129 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f6cd41a7380 con 0x7f6cd4106f10 2024-09-19T04:44:44.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.834+0000 7f6cd2000640 1 -- 172.21.15.123:0/2608731129 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f6ccc0c04c0 con 0x7f6cd4106f10 2024-09-19T04:44:44.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.836+0000 7f6cdc200640 1 -- 172.21.15.123:0/2608731129 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6cbc0a32c0 msgr2=0x7f6cbc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:44.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.836+0000 7f6cdc200640 1 --2- 172.21.15.123:0/2608731129 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6cbc0a32c0 0x7f6cbc0a5780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8009fd0 tx=0x7f6cc8009500 comp rx=0 tx=0).stop 2024-09-19T04:44:44.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.836+0000 7f6cdc200640 1 -- 172.21.15.123:0/2608731129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cd4106f10 msgr2=0x7f6cd419eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:44.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.836+0000 7f6cdc200640 1 --2- 172.21.15.123:0/2608731129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cd4106f10 0x7f6cd419eea0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009230 tx=0x7f6ccc005e70 comp rx=0 tx=0).stop 2024-09-19T04:44:44.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.837+0000 7f6cdc200640 1 -- 172.21.15.123:0/2608731129 shutdown_connections 2024-09-19T04:44:44.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.837+0000 7f6cdc200640 1 --2- 172.21.15.123:0/2608731129 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6cbc0a32c0 0x7f6cbc0a5780 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:44.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.837+0000 7f6cdc200640 1 --2- 172.21.15.123:0/2608731129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cd4106f10 0x7f6cd419eea0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:44.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.837+0000 7f6cdc200640 1 --2- 172.21.15.123:0/2608731129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cd4102790 0x7f6cd419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:44.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.837+0000 7f6cdc200640 1 --2- 172.21.15.123:0/2608731129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cd4101de0 0x7f6cd419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:44.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.837+0000 7f6cdc200640 1 -- 172.21.15.123:0/2608731129 >> 172.21.15.123:0/2608731129 conn(0x7f6cd40fd8c0 msgr2=0x7f6cd40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:44.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.837+0000 7f6cdc200640 1 -- 172.21.15.123:0/2608731129 shutdown_connections 2024-09-19T04:44:44.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:44.837+0000 7f6cdc200640 1 -- 172.21.15.123:0/2608731129 wait complete. 2024-09-19T04:44:45.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:44 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] up:active 2024-09-19T04:44:45.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:44 smithi183 ceph-mon[35333]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T04:44:45.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:44 smithi183 ceph-mon[35333]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T04:44:45.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:44 smithi183 ceph-mon[35333]: Cluster is now healthy 2024-09-19T04:44:45.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:44 smithi183 ceph-mon[35333]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-09-19T04:44:45.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:44 smithi183 ceph-mon[35333]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T04:44:45.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:44 smithi183 ceph-mon[35333]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-19T04:44:45.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:44 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2608731129' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T04:44:45.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:44 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T04:44:45.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:44 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] up:active 2024-09-19T04:44:45.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:44 smithi123 ceph-mon[30511]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T04:44:45.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:44 smithi123 ceph-mon[30511]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T04:44:45.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:44 smithi123 ceph-mon[30511]: Cluster is now healthy 2024-09-19T04:44:45.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:44 smithi123 ceph-mon[30511]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-09-19T04:44:45.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:44 smithi123 ceph-mon[30511]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T04:44:45.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:44 smithi123 ceph-mon[30511]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-19T04:44:45.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:44 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2608731129' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T04:44:45.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:44 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T04:44:45.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:44 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] up:active 2024-09-19T04:44:45.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:44 smithi193 ceph-mon[35592]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T04:44:45.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:44 smithi193 ceph-mon[35592]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T04:44:45.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:44 smithi193 ceph-mon[35592]: Cluster is now healthy 2024-09-19T04:44:45.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:44 smithi193 ceph-mon[35592]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-09-19T04:44:45.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:44 smithi193 ceph-mon[35592]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T04:44:45.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:44 smithi193 ceph-mon[35592]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-19T04:44:45.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:44 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2608731129' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T04:44:45.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:44 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T04:44:45.610 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-09-19T04:44:45.610 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-19T04:44:45.976 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:46.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:45 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T04:44:46.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:45 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] up:active 2024-09-19T04:44:46.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:45 smithi183 ceph-mon[35333]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T04:44:46.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:45 smithi183 ceph-mon[35333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:44:46.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:45 smithi183 ceph-mon[35333]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 9 op/s 2024-09-19T04:44:46.115 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:45 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T04:44:46.115 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:45 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] up:active 2024-09-19T04:44:46.115 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:45 smithi123 ceph-mon[30511]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T04:44:46.115 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:45 smithi123 ceph-mon[30511]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:44:46.115 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:45 smithi123 ceph-mon[30511]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 9 op/s 2024-09-19T04:44:46.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:45 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T04:44:46.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:45 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] up:active 2024-09-19T04:44:46.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:45 smithi193 ceph-mon[35592]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T04:44:46.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:45 smithi193 ceph-mon[35592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:44:46.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:45 smithi193 ceph-mon[35592]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 9 op/s 2024-09-19T04:44:46.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.572+0000 7fe35f400640 1 -- 172.21.15.123:0/3450249442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe358104010 msgr2=0x7fe3581043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:46.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.572+0000 7fe35f400640 1 --2- 172.21.15.123:0/3450249442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe358104010 0x7fe3581043f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe348009900 tx=0x7fe34802a410 comp rx=0 tx=0).stop 2024-09-19T04:44:46.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.572+0000 7fe35f400640 1 -- 172.21.15.123:0/3450249442 shutdown_connections 2024-09-19T04:44:46.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.572+0000 7fe35f400640 1 --2- 172.21.15.123:0/3450249442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe358109140 0x7fe358110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:46.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.572+0000 7fe35f400640 1 --2- 172.21.15.123:0/3450249442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3581049c0 0x7fe358108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:46.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.572+0000 7fe35f400640 1 --2- 172.21.15.123:0/3450249442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe358104010 0x7fe3581043f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:46.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.572+0000 7fe35f400640 1 -- 172.21.15.123:0/3450249442 >> 172.21.15.123:0/3450249442 conn(0x7fe3580ffa50 msgr2=0x7fe358101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:46.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.572+0000 7fe35f400640 1 -- 172.21.15.123:0/3450249442 shutdown_connections 2024-09-19T04:44:46.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.572+0000 7fe35f400640 1 -- 172.21.15.123:0/3450249442 wait complete. 2024-09-19T04:44:46.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.573+0000 7fe35f400640 1 Processor -- start 2024-09-19T04:44:46.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.573+0000 7fe35f400640 1 -- start start 2024-09-19T04:44:46.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.574+0000 7fe35f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe358104010 0x7fe35819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:46.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.574+0000 7fe35f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3581049c0 0x7fe35819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:46.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.574+0000 7fe35f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe358109140 0x7fe35819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:46.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.574+0000 7fe35f400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3581141b0 con 0x7fe3581049c0 2024-09-19T04:44:46.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.574+0000 7fe35f400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe358114030 con 0x7fe358104010 2024-09-19T04:44:46.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.574+0000 7fe35f400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe358114330 con 0x7fe358109140 2024-09-19T04:44:46.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.574+0000 7fe35ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe358104010 0x7fe35819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:46.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.574+0000 7fe34fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3581049c0 0x7fe35819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:46.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.574+0000 7fe35ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe358104010 0x7fe35819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:48178/0 (socket says 172.21.15.123:48178) 2024-09-19T04:44:46.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.574+0000 7fe35ca00640 1 -- 172.21.15.123:0/2546826397 learned_addr learned my addr 172.21.15.123:0/2546826397 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:46.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.574+0000 7fe35d400640 1 --2- 172.21.15.123:0/2546826397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe358109140 0x7fe35819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:46.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.574+0000 7fe35ca00640 1 -- 172.21.15.123:0/2546826397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe358109140 msgr2=0x7fe35819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:46.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.574+0000 7fe35ca00640 1 --2- 172.21.15.123:0/2546826397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe358109140 0x7fe35819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:46.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.575+0000 7fe35ca00640 1 -- 172.21.15.123:0/2546826397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3581049c0 msgr2=0x7fe35819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:46.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.575+0000 7fe35ca00640 1 --2- 172.21.15.123:0/2546826397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3581049c0 0x7fe35819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:46.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.575+0000 7fe35ca00640 1 -- 172.21.15.123:0/2546826397 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35819f600 con 0x7fe358104010 2024-09-19T04:44:46.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.575+0000 7fe35d400640 1 --2- 172.21.15.123:0/2546826397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe358109140 0x7fe35819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:44:46.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.575+0000 7fe34fe00640 1 --2- 172.21.15.123:0/2546826397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3581049c0 0x7fe35819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:44:46.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.575+0000 7fe35ca00640 1 --2- 172.21.15.123:0/2546826397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe358104010 0x7fe35819a650 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe348004770 tx=0x7fe348003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:46.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.575+0000 7fe356000640 1 -- 172.21.15.123:0/2546826397 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34803f070 con 0x7fe358104010 2024-09-19T04:44:46.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.575+0000 7fe35f400640 1 -- 172.21.15.123:0/2546826397 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe35819f890 con 0x7fe358104010 2024-09-19T04:44:46.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.575+0000 7fe356000640 1 -- 172.21.15.123:0/2546826397 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe348031ca0 con 0x7fe358104010 2024-09-19T04:44:46.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.575+0000 7fe356000640 1 -- 172.21.15.123:0/2546826397 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe348035030 con 0x7fe358104010 2024-09-19T04:44:46.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.575+0000 7fe35f400640 1 -- 172.21.15.123:0/2546826397 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3581a71d0 con 0x7fe358104010 2024-09-19T04:44:46.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.576+0000 7fe35f400640 1 -- 172.21.15.123:0/2546826397 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3581054f0 con 0x7fe358104010 2024-09-19T04:44:46.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.578+0000 7fe356000640 1 -- 172.21.15.123:0/2546826397 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fe348042070 con 0x7fe358104010 2024-09-19T04:44:46.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.580+0000 7fe356000640 1 --2- 172.21.15.123:0/2546826397 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe33c0a3240 0x7fe33c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:46.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.580+0000 7fe356000640 1 -- 172.21.15.123:0/2546826397 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fe3480f0170 con 0x7fe358104010 2024-09-19T04:44:46.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.580+0000 7fe34fe00640 1 --2- 172.21.15.123:0/2546826397 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe33c0a3240 0x7fe33c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:46.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.580+0000 7fe34fe00640 1 --2- 172.21.15.123:0/2546826397 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe33c0a3240 0x7fe33c0a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3580ffe70 tx=0x7fe340005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:46.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.582+0000 7fe356000640 1 -- 172.21.15.123:0/2546826397 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe3480bd000 con 0x7fe358104010 2024-09-19T04:44:46.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.785+0000 7fe35f400640 1 -- 172.21.15.123:0/2546826397 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe358062630 con 0x7fe33c0a3240 2024-09-19T04:44:46.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.812+0000 7fe356000640 1 -- 172.21.15.123:0/2546826397 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe358062630 con 0x7fe33c0a3240 2024-09-19T04:44:46.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.815+0000 7fe35f400640 1 -- 172.21.15.123:0/2546826397 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe33c0a3240 msgr2=0x7fe33c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:46.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.815+0000 7fe35f400640 1 --2- 172.21.15.123:0/2546826397 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe33c0a3240 0x7fe33c0a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3580ffe70 tx=0x7fe340005f90 comp rx=0 tx=0).stop 2024-09-19T04:44:46.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.815+0000 7fe35f400640 1 -- 172.21.15.123:0/2546826397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe358104010 msgr2=0x7fe35819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:46.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.815+0000 7fe35f400640 1 --2- 172.21.15.123:0/2546826397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe358104010 0x7fe35819a650 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe348004770 tx=0x7fe348003c10 comp rx=0 tx=0).stop 2024-09-19T04:44:46.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.815+0000 7fe35f400640 1 -- 172.21.15.123:0/2546826397 shutdown_connections 2024-09-19T04:44:46.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.815+0000 7fe35f400640 1 --2- 172.21.15.123:0/2546826397 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe33c0a3240 0x7fe33c0a5700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:46.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.815+0000 7fe35f400640 1 --2- 172.21.15.123:0/2546826397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe358109140 0x7fe35819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:46.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.815+0000 7fe35f400640 1 --2- 172.21.15.123:0/2546826397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3581049c0 0x7fe35819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:46.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.815+0000 7fe35f400640 1 --2- 172.21.15.123:0/2546826397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe358104010 0x7fe35819a650 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:46.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.815+0000 7fe35f400640 1 -- 172.21.15.123:0/2546826397 >> 172.21.15.123:0/2546826397 conn(0x7fe3580ffa50 msgr2=0x7fe358100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:46.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.815+0000 7fe35f400640 1 -- 172.21.15.123:0/2546826397 shutdown_connections 2024-09-19T04:44:46.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:46.815+0000 7fe35f400640 1 -- 172.21.15.123:0/2546826397 wait complete. 2024-09-19T04:44:47.177 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:44:46 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:44:46.792+0000 7f8d66e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:44:47.177 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:44:46 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:44:46.792+0000 7f8d66e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:44:47.177 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:44:46 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:44:46.792+0000 7f8d66e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:44:47.178 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:44:46 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:44:46.792+0000 7f8d66e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:44:47.178 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:44:46 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:44:46.792+0000 7f8d66e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:44:47.722 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-19T04:44:48.094 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:48.486 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:48 smithi123 ceph-mon[30511]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:44:48.487 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:48 smithi123 ceph-mon[30511]: pgmap v301: 97 pgs: 97 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-19T04:44:48.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.600+0000 7f66f7e00640 1 -- 172.21.15.123:0/2295792102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f8074e30 msgr2=0x7f66f810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:48.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.600+0000 7f66f7e00640 1 --2- 172.21.15.123:0/2295792102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f8074e30 0x7f66f810dd40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009900 tx=0x7f66e002a480 comp rx=0 tx=0).stop 2024-09-19T04:44:48.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.600+0000 7f66f7e00640 1 -- 172.21.15.123:0/2295792102 shutdown_connections 2024-09-19T04:44:48.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.600+0000 7f66f7e00640 1 --2- 172.21.15.123:0/2295792102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66f810e280 0x7f66f8110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:48.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.600+0000 7f66f7e00640 1 --2- 172.21.15.123:0/2295792102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f8074e30 0x7f66f810dd40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:48.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.600+0000 7f66f7e00640 1 --2- 172.21.15.123:0/2295792102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66f8074510 0x7f66f80748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:48.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.600+0000 7f66f7e00640 1 -- 172.21.15.123:0/2295792102 >> 172.21.15.123:0/2295792102 conn(0x7f66f80fd740 msgr2=0x7f66f80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:48.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.600+0000 7f66f7e00640 1 -- 172.21.15.123:0/2295792102 shutdown_connections 2024-09-19T04:44:48.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.600+0000 7f66f7e00640 1 -- 172.21.15.123:0/2295792102 wait complete. 2024-09-19T04:44:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.601+0000 7f66f7e00640 1 Processor -- start 2024-09-19T04:44:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.601+0000 7f66f7e00640 1 -- start start 2024-09-19T04:44:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.601+0000 7f66f7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f8074510 0x7f66f819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.601+0000 7f66f7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66f8074e30 0x7f66f819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.601+0000 7f66f7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66f810e280 0x7f66f819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.601+0000 7f66f7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66f8114000 con 0x7f66f8074510 2024-09-19T04:44:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.601+0000 7f66f7e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f66f8113e80 con 0x7f66f8074e30 2024-09-19T04:44:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.601+0000 7f66f7e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f66f8114180 con 0x7f66f810e280 2024-09-19T04:44:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.602+0000 7f66f6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f8074510 0x7f66f819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.602+0000 7f66f6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f8074510 0x7f66f819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43256/0 (socket says 172.21.15.123:43256) 2024-09-19T04:44:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.602+0000 7f66f6a00640 1 -- 172.21.15.123:0/791281941 learned_addr learned my addr 172.21.15.123:0/791281941 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.602+0000 7f66f7400640 1 --2- 172.21.15.123:0/791281941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66f810e280 0x7f66f819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.602+0000 7f66efe00640 1 --2- 172.21.15.123:0/791281941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66f8074e30 0x7f66f819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.602+0000 7f66f6a00640 1 -- 172.21.15.123:0/791281941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66f810e280 msgr2=0x7f66f819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.602+0000 7f66f6a00640 1 --2- 172.21.15.123:0/791281941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66f810e280 0x7f66f819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.602+0000 7f66f6a00640 1 -- 172.21.15.123:0/791281941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66f8074e30 msgr2=0x7f66f819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.602+0000 7f66f6a00640 1 --2- 172.21.15.123:0/791281941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66f8074e30 0x7f66f819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.602+0000 7f66f6a00640 1 -- 172.21.15.123:0/791281941 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f819f490 con 0x7f66f8074510 2024-09-19T04:44:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.602+0000 7f66f6a00640 1 --2- 172.21.15.123:0/791281941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f8074510 0x7f66f819a4e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f66e800e9f0 tx=0x7f66e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.603+0000 7f66ef400640 1 -- 172.21.15.123:0/791281941 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e800cd80 con 0x7f66f8074510 2024-09-19T04:44:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.603+0000 7f66ef400640 1 -- 172.21.15.123:0/791281941 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66e8010910 con 0x7f66f8074510 2024-09-19T04:44:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.603+0000 7f66f7e00640 1 -- 172.21.15.123:0/791281941 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f819f780 con 0x7f66f8074510 2024-09-19T04:44:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.603+0000 7f66ef400640 1 -- 172.21.15.123:0/791281941 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8018990 con 0x7f66f8074510 2024-09-19T04:44:48.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.603+0000 7f66f7e00640 1 -- 172.21.15.123:0/791281941 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f81a7060 con 0x7f66f8074510 2024-09-19T04:44:48.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.604+0000 7f66f7e00640 1 -- 172.21.15.123:0/791281941 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f80748f0 con 0x7f66f8074510 2024-09-19T04:44:48.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.606+0000 7f66ef400640 1 -- 172.21.15.123:0/791281941 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f66e8010ac0 con 0x7f66f8074510 2024-09-19T04:44:48.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.607+0000 7f66ef400640 1 --2- 172.21.15.123:0/791281941 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f66dc0a32d0 0x7f66dc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:48.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.607+0000 7f66ef400640 1 -- 172.21.15.123:0/791281941 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f66e80cfd00 con 0x7f66f8074510 2024-09-19T04:44:48.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.607+0000 7f66efe00640 1 --2- 172.21.15.123:0/791281941 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f66dc0a32d0 0x7f66dc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:48.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.608+0000 7f66efe00640 1 --2- 172.21.15.123:0/791281941 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f66dc0a32d0 0x7f66dc0a5790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f66e0000c00 tx=0x7f66e0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:48.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.611+0000 7f66ef400640 1 -- 172.21.15.123:0/791281941 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f66e809c320 con 0x7f66f8074510 2024-09-19T04:44:48.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:48 smithi193 ceph-mon[35592]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:44:48.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:48 smithi193 ceph-mon[35592]: pgmap v301: 97 pgs: 97 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-19T04:44:48.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.815+0000 7f66f7e00640 1 -- 172.21.15.123:0/791281941 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f66f8062630 con 0x7f66dc0a32d0 2024-09-19T04:44:48.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.840+0000 7f66ef400640 1 -- 172.21.15.123:0/791281941 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f66f8062630 con 0x7f66dc0a32d0 2024-09-19T04:44:48.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.842+0000 7f66f7e00640 1 -- 172.21.15.123:0/791281941 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f66dc0a32d0 msgr2=0x7f66dc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:48.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.842+0000 7f66f7e00640 1 --2- 172.21.15.123:0/791281941 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f66dc0a32d0 0x7f66dc0a5790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f66e0000c00 tx=0x7f66e0005b50 comp rx=0 tx=0).stop 2024-09-19T04:44:48.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.842+0000 7f66f7e00640 1 -- 172.21.15.123:0/791281941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f8074510 msgr2=0x7f66f819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:48.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.842+0000 7f66f7e00640 1 --2- 172.21.15.123:0/791281941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f8074510 0x7f66f819a4e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f66e800e9f0 tx=0x7f66e800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:44:48.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.842+0000 7f66f7e00640 1 -- 172.21.15.123:0/791281941 shutdown_connections 2024-09-19T04:44:48.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.842+0000 7f66f7e00640 1 --2- 172.21.15.123:0/791281941 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f66dc0a32d0 0x7f66dc0a5790 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:48.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.842+0000 7f66f7e00640 1 --2- 172.21.15.123:0/791281941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66f810e280 0x7f66f819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:48.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.842+0000 7f66f7e00640 1 --2- 172.21.15.123:0/791281941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66f8074e30 0x7f66f819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:48.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.842+0000 7f66f7e00640 1 --2- 172.21.15.123:0/791281941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f8074510 0x7f66f819a4e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:48.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.842+0000 7f66f7e00640 1 -- 172.21.15.123:0/791281941 >> 172.21.15.123:0/791281941 conn(0x7f66f80fd740 msgr2=0x7f66f80fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:48.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.843+0000 7f66f7e00640 1 -- 172.21.15.123:0/791281941 shutdown_connections 2024-09-19T04:44:48.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:48.843+0000 7f66f7e00640 1 -- 172.21.15.123:0/791281941 wait complete. 2024-09-19T04:44:48.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:48 smithi183 ceph-mon[35333]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:44:48.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:48 smithi183 ceph-mon[35333]: pgmap v301: 97 pgs: 97 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-19T04:44:49.608 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:49 smithi123 ceph-mon[30511]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T04:44:49.644 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-19T04:44:49.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:49 smithi193 ceph-mon[35592]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T04:44:49.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:49 smithi183 ceph-mon[35333]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T04:44:50.014 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:50.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.583+0000 7fcf41e00640 1 -- 172.21.15.123:0/1332321960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3c106f10 msgr2=0x7fcf3c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:50.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.583+0000 7fcf41e00640 1 --2- 172.21.15.123:0/1332321960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3c106f10 0x7fcf3c10ea90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009900 tx=0x7fcf3402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:44:50.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.583+0000 7fcf41e00640 1 -- 172.21.15.123:0/1332321960 shutdown_connections 2024-09-19T04:44:50.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.583+0000 7fcf41e00640 1 --2- 172.21.15.123:0/1332321960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3c106f10 0x7fcf3c10ea90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:50.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.583+0000 7fcf41e00640 1 --2- 172.21.15.123:0/1332321960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf3c102790 0x7fcf3c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:50.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.584+0000 7fcf41e00640 1 --2- 172.21.15.123:0/1332321960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf3c101de0 0x7fcf3c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:50.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.584+0000 7fcf41e00640 1 -- 172.21.15.123:0/1332321960 >> 172.21.15.123:0/1332321960 conn(0x7fcf3c0fd8c0 msgr2=0x7fcf3c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:50.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.584+0000 7fcf41e00640 1 -- 172.21.15.123:0/1332321960 shutdown_connections 2024-09-19T04:44:50.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.584+0000 7fcf41e00640 1 -- 172.21.15.123:0/1332321960 wait complete. 2024-09-19T04:44:50.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.585+0000 7fcf41e00640 1 Processor -- start 2024-09-19T04:44:50.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.585+0000 7fcf41e00640 1 -- start start 2024-09-19T04:44:50.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.585+0000 7fcf41e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf3c101de0 0x7fcf3c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:50.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.585+0000 7fcf41e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf3c102790 0x7fcf3c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:50.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.585+0000 7fcf41e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3c106f10 0x7fcf3c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:50.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.585+0000 7fcf41e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c111f70 con 0x7fcf3c106f10 2024-09-19T04:44:50.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.585+0000 7fcf41e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c111df0 con 0x7fcf3c102790 2024-09-19T04:44:50.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.585+0000 7fcf41e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c1120f0 con 0x7fcf3c101de0 2024-09-19T04:44:50.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.585+0000 7fcf3a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf3c102790 0x7fcf3c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:50.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.585+0000 7fcf3b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3c106f10 0x7fcf3c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:50.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.585+0000 7fcf3a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf3c102790 0x7fcf3c19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:48222/0 (socket says 172.21.15.123:48222) 2024-09-19T04:44:50.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.585+0000 7fcf3aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf3c101de0 0x7fcf3c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:50.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.585+0000 7fcf3a000640 1 -- 172.21.15.123:0/590222280 learned_addr learned my addr 172.21.15.123:0/590222280 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:50.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.586+0000 7fcf3a000640 1 -- 172.21.15.123:0/590222280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf3c101de0 msgr2=0x7fcf3c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:50.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.586+0000 7fcf3a000640 1 --2- 172.21.15.123:0/590222280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf3c101de0 0x7fcf3c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:50.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.586+0000 7fcf3a000640 1 -- 172.21.15.123:0/590222280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3c106f10 msgr2=0x7fcf3c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:50.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.586+0000 7fcf3a000640 1 --2- 172.21.15.123:0/590222280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3c106f10 0x7fcf3c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:50.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.586+0000 7fcf3a000640 1 -- 172.21.15.123:0/590222280 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3c19f620 con 0x7fcf3c102790 2024-09-19T04:44:50.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.586+0000 7fcf3b400640 1 --2- 172.21.15.123:0/590222280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3c106f10 0x7fcf3c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:44:50.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.586+0000 7fcf3a000640 1 --2- 172.21.15.123:0/590222280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf3c102790 0x7fcf3c19abb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcf2800d770 tx=0x7fcf2800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:50.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.586+0000 7fcf40a00640 1 -- 172.21.15.123:0/590222280 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28017070 con 0x7fcf3c102790 2024-09-19T04:44:50.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.587+0000 7fcf40a00640 1 -- 172.21.15.123:0/590222280 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf28005500 con 0x7fcf3c102790 2024-09-19T04:44:50.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.587+0000 7fcf40a00640 1 -- 172.21.15.123:0/590222280 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28004e00 con 0x7fcf3c102790 2024-09-19T04:44:50.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.587+0000 7fcf41e00640 1 -- 172.21.15.123:0/590222280 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3c19f910 con 0x7fcf3c102790 2024-09-19T04:44:50.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.587+0000 7fcf41e00640 1 -- 172.21.15.123:0/590222280 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf3c1a71f0 con 0x7fcf3c102790 2024-09-19T04:44:50.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.588+0000 7fcf41e00640 1 -- 172.21.15.123:0/590222280 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf3c1021c0 con 0x7fcf3c102790 2024-09-19T04:44:50.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.590+0000 7fcf40a00640 1 -- 172.21.15.123:0/590222280 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fcf28004fa0 con 0x7fcf3c102790 2024-09-19T04:44:50.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.591+0000 7fcf40a00640 1 --2- 172.21.15.123:0/590222280 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcf200a3350 0x7fcf200a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:50.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.591+0000 7fcf40a00640 1 -- 172.21.15.123:0/590222280 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fcf280cf360 con 0x7fcf3c102790 2024-09-19T04:44:50.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.591+0000 7fcf3aa00640 1 --2- 172.21.15.123:0/590222280 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcf200a3350 0x7fcf200a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:50.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.592+0000 7fcf3aa00640 1 --2- 172.21.15.123:0/590222280 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcf200a3350 0x7fcf200a5810 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c005fd0 tx=0x7fcf2c009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:50.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.594+0000 7fcf40a00640 1 -- 172.21.15.123:0/590222280 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcf2809ba00 con 0x7fcf3c102790 2024-09-19T04:44:50.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:50 smithi193 ceph-mon[35592]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:44:50.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:50 smithi193 ceph-mon[35592]: pgmap v302: 97 pgs: 97 active+clean; 591 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.7 KiB/s wr, 22 op/s 2024-09-19T04:44:50.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.794+0000 7fcf41e00640 1 -- 172.21.15.123:0/590222280 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fcf3c075430 con 0x7fcf200a3350 2024-09-19T04:44:50.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:50 smithi123 ceph-mon[30511]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:44:50.796 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:50 smithi123 ceph-mon[30511]: pgmap v302: 97 pgs: 97 active+clean; 591 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.7 KiB/s wr, 22 op/s 2024-09-19T04:44:50.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.799+0000 7fcf40a00640 1 -- 172.21.15.123:0/590222280 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fcf3c075430 con 0x7fcf200a3350 2024-09-19T04:44:50.800 INFO:teuthology.orchestra.run.smithi123.stdout:/volumes/qa/sv_0/9061b899-57f4-43e2-96f5-9d6ecc5d4407 2024-09-19T04:44:50.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.801+0000 7fcf41e00640 1 -- 172.21.15.123:0/590222280 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcf200a3350 msgr2=0x7fcf200a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:50.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.801+0000 7fcf41e00640 1 --2- 172.21.15.123:0/590222280 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcf200a3350 0x7fcf200a5810 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c005fd0 tx=0x7fcf2c009760 comp rx=0 tx=0).stop 2024-09-19T04:44:50.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.801+0000 7fcf41e00640 1 -- 172.21.15.123:0/590222280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf3c102790 msgr2=0x7fcf3c19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:50.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.801+0000 7fcf41e00640 1 --2- 172.21.15.123:0/590222280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf3c102790 0x7fcf3c19abb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcf2800d770 tx=0x7fcf2800dc30 comp rx=0 tx=0).stop 2024-09-19T04:44:50.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.801+0000 7fcf41e00640 1 -- 172.21.15.123:0/590222280 shutdown_connections 2024-09-19T04:44:50.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.801+0000 7fcf41e00640 1 --2- 172.21.15.123:0/590222280 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcf200a3350 0x7fcf200a5810 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:50.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.801+0000 7fcf41e00640 1 --2- 172.21.15.123:0/590222280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3c106f10 0x7fcf3c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:50.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.801+0000 7fcf41e00640 1 --2- 172.21.15.123:0/590222280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf3c102790 0x7fcf3c19abb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:50.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.801+0000 7fcf41e00640 1 --2- 172.21.15.123:0/590222280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf3c101de0 0x7fcf3c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:50.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.801+0000 7fcf41e00640 1 -- 172.21.15.123:0/590222280 >> 172.21.15.123:0/590222280 conn(0x7fcf3c0fd8c0 msgr2=0x7fcf3c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:50.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.801+0000 7fcf41e00640 1 -- 172.21.15.123:0/590222280 shutdown_connections 2024-09-19T04:44:50.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:50.801+0000 7fcf41e00640 1 -- 172.21.15.123:0/590222280 wait complete. 2024-09-19T04:44:50.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:50 smithi183 ceph-mon[35333]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:44:50.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:50 smithi183 ceph-mon[35333]: pgmap v302: 97 pgs: 97 active+clean; 591 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.7 KiB/s wr, 22 op/s 2024-09-19T04:44:51.623 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-19T04:44:51.995 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:52.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.535+0000 7f673f600640 1 -- 172.21.15.123:0/2473487864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6738102790 msgr2=0x7f67381067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:52.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.535+0000 7f673f600640 1 --2- 172.21.15.123:0/2473487864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6738102790 0x7f67381067e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6720009900 tx=0x7f672002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:44:52.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.536+0000 7f673f600640 1 -- 172.21.15.123:0/2473487864 shutdown_connections 2024-09-19T04:44:52.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.536+0000 7f673f600640 1 --2- 172.21.15.123:0/2473487864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6738106f10 0x7f673810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:52.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.536+0000 7f673f600640 1 --2- 172.21.15.123:0/2473487864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6738102790 0x7f67381067e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:52.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.536+0000 7f673f600640 1 --2- 172.21.15.123:0/2473487864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6738101de0 0x7f67381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:52.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.536+0000 7f673f600640 1 -- 172.21.15.123:0/2473487864 >> 172.21.15.123:0/2473487864 conn(0x7f67380fd8c0 msgr2=0x7f67380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:52.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.536+0000 7f673f600640 1 -- 172.21.15.123:0/2473487864 shutdown_connections 2024-09-19T04:44:52.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.536+0000 7f673f600640 1 -- 172.21.15.123:0/2473487864 wait complete. 2024-09-19T04:44:52.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.537+0000 7f673f600640 1 Processor -- start 2024-09-19T04:44:52.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.537+0000 7f673f600640 1 -- start start 2024-09-19T04:44:52.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.537+0000 7f673f600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6738101de0 0x7f6738198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:52.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.537+0000 7f673f600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6738102790 0x7f6738198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:52.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.537+0000 7f673f600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6738106f10 0x7f673819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:52.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.537+0000 7f673f600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6738111f30 con 0x7f6738101de0 2024-09-19T04:44:52.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.537+0000 7f673f600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6738111db0 con 0x7f6738106f10 2024-09-19T04:44:52.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.537+0000 7f673f600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f67381120b0 con 0x7f6738102790 2024-09-19T04:44:52.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.537+0000 7f673cc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6738101de0 0x7f6738198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:52.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.537+0000 7f673d600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6738106f10 0x7f673819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:52.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.537+0000 7f673cc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6738101de0 0x7f6738198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57088/0 (socket says 172.21.15.123:57088) 2024-09-19T04:44:52.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.537+0000 7f673cc00640 1 -- 172.21.15.123:0/1958534904 learned_addr learned my addr 172.21.15.123:0/1958534904 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:52.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.538+0000 7f672fe00640 1 --2- 172.21.15.123:0/1958534904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6738102790 0x7f6738198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:52.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.538+0000 7f673cc00640 1 -- 172.21.15.123:0/1958534904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6738102790 msgr2=0x7f6738198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:52.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.538+0000 7f673cc00640 1 --2- 172.21.15.123:0/1958534904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6738102790 0x7f6738198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:52.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.538+0000 7f673cc00640 1 -- 172.21.15.123:0/1958534904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6738106f10 msgr2=0x7f673819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:52.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.538+0000 7f673cc00640 1 --2- 172.21.15.123:0/1958534904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6738106f10 0x7f673819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:52.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.538+0000 7f673cc00640 1 -- 172.21.15.123:0/1958534904 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673819d400 con 0x7f6738101de0 2024-09-19T04:44:52.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.538+0000 7f673d600640 1 --2- 172.21.15.123:0/1958534904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6738106f10 0x7f673819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:44:52.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.538+0000 7f673cc00640 1 --2- 172.21.15.123:0/1958534904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6738101de0 0x7f6738198450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f672800e9f0 tx=0x7f672800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:52.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.538+0000 7f6736000640 1 -- 172.21.15.123:0/1958534904 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672800cd80 con 0x7f6738101de0 2024-09-19T04:44:52.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.538+0000 7f6736000640 1 -- 172.21.15.123:0/1958534904 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6728010910 con 0x7f6738101de0 2024-09-19T04:44:52.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.538+0000 7f673f600640 1 -- 172.21.15.123:0/1958534904 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673819d6f0 con 0x7f6738101de0 2024-09-19T04:44:52.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.538+0000 7f6736000640 1 -- 172.21.15.123:0/1958534904 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728018990 con 0x7f6738101de0 2024-09-19T04:44:52.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.539+0000 7f673f600640 1 -- 172.21.15.123:0/1958534904 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67381a4fd0 con 0x7f6738101de0 2024-09-19T04:44:52.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.540+0000 7f673f600640 1 -- 172.21.15.123:0/1958534904 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67381021c0 con 0x7f6738101de0 2024-09-19T04:44:52.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.541+0000 7f6736000640 1 -- 172.21.15.123:0/1958534904 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6728010ac0 con 0x7f6738101de0 2024-09-19T04:44:52.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.543+0000 7f6736000640 1 --2- 172.21.15.123:0/1958534904 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f671c0a32d0 0x7f671c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:52.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.543+0000 7f6736000640 1 -- 172.21.15.123:0/1958534904 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f67280cfce0 con 0x7f6738101de0 2024-09-19T04:44:52.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.543+0000 7f672fe00640 1 --2- 172.21.15.123:0/1958534904 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f671c0a32d0 0x7f671c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:52.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.544+0000 7f672fe00640 1 --2- 172.21.15.123:0/1958534904 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f671c0a32d0 0x7f671c0a5790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6720009230 tx=0x7f67200058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:52.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.547+0000 7f6736000640 1 -- 172.21.15.123:0/1958534904 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f672809c300 con 0x7f6738101de0 2024-09-19T04:44:52.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:52 smithi123 ceph-mon[30511]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:44:52.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:52 smithi123 ceph-mon[30511]: pgmap v303: 97 pgs: 97 active+clean; 591 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.7 KiB/s wr, 18 op/s 2024-09-19T04:44:52.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:52 smithi193 ceph-mon[35592]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:44:52.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:52 smithi193 ceph-mon[35592]: pgmap v303: 97 pgs: 97 active+clean; 591 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.7 KiB/s wr, 18 op/s 2024-09-19T04:44:52.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.751+0000 7f673f600640 1 -- 172.21.15.123:0/1958534904 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f673810a160 con 0x7f671c0a32d0 2024-09-19T04:44:52.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.778+0000 7f6736000640 1 -- 172.21.15.123:0/1958534904 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6720031720 con 0x7f671c0a32d0 2024-09-19T04:44:52.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.780+0000 7f673f600640 1 -- 172.21.15.123:0/1958534904 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f671c0a32d0 msgr2=0x7f671c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:52.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.780+0000 7f673f600640 1 --2- 172.21.15.123:0/1958534904 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f671c0a32d0 0x7f671c0a5790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6720009230 tx=0x7f67200058d0 comp rx=0 tx=0).stop 2024-09-19T04:44:52.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.780+0000 7f673f600640 1 -- 172.21.15.123:0/1958534904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6738101de0 msgr2=0x7f6738198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:52.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.780+0000 7f673f600640 1 --2- 172.21.15.123:0/1958534904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6738101de0 0x7f6738198450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f672800e9f0 tx=0x7f672800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:44:52.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.781+0000 7f673f600640 1 -- 172.21.15.123:0/1958534904 shutdown_connections 2024-09-19T04:44:52.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.781+0000 7f673f600640 1 --2- 172.21.15.123:0/1958534904 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f671c0a32d0 0x7f671c0a5790 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:52.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.781+0000 7f673f600640 1 --2- 172.21.15.123:0/1958534904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6738106f10 0x7f673819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:52.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.781+0000 7f673f600640 1 --2- 172.21.15.123:0/1958534904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6738102790 0x7f6738198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:52.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.781+0000 7f673f600640 1 --2- 172.21.15.123:0/1958534904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6738101de0 0x7f6738198450 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:52.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.781+0000 7f673f600640 1 -- 172.21.15.123:0/1958534904 >> 172.21.15.123:0/1958534904 conn(0x7f67380fd8c0 msgr2=0x7f67380fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:52.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.781+0000 7f673f600640 1 -- 172.21.15.123:0/1958534904 shutdown_connections 2024-09-19T04:44:52.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:52.781+0000 7f673f600640 1 -- 172.21.15.123:0/1958534904 wait complete. 2024-09-19T04:44:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:52 smithi183 ceph-mon[35333]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:44:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:52 smithi183 ceph-mon[35333]: pgmap v303: 97 pgs: 97 active+clean; 591 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.7 KiB/s wr, 18 op/s 2024-09-19T04:44:53.649 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-19T04:44:54.017 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:54.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.515+0000 7f5dfe000640 1 -- 172.21.15.123:0/4037720346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5df8101de0 msgr2=0x7f5df81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:54.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.515+0000 7f5dfe000640 1 --2- 172.21.15.123:0/4037720346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5df8101de0 0x7f5df81021c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009900 tx=0x7f5de802a410 comp rx=0 tx=0).stop 2024-09-19T04:44:54.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.515+0000 7f5dfe000640 1 -- 172.21.15.123:0/4037720346 shutdown_connections 2024-09-19T04:44:54.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.515+0000 7f5dfe000640 1 --2- 172.21.15.123:0/4037720346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5df8106f10 0x7f5df810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:54.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.515+0000 7f5dfe000640 1 --2- 172.21.15.123:0/4037720346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5df8102790 0x7f5df81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:54.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.515+0000 7f5dfe000640 1 --2- 172.21.15.123:0/4037720346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5df8101de0 0x7f5df81021c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:54.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.515+0000 7f5dfe000640 1 -- 172.21.15.123:0/4037720346 >> 172.21.15.123:0/4037720346 conn(0x7f5df80fd8c0 msgr2=0x7f5df80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:54.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.515+0000 7f5dfe000640 1 -- 172.21.15.123:0/4037720346 shutdown_connections 2024-09-19T04:44:54.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.515+0000 7f5dfe000640 1 -- 172.21.15.123:0/4037720346 wait complete. 2024-09-19T04:44:54.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.516+0000 7f5dfe000640 1 Processor -- start 2024-09-19T04:44:54.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.516+0000 7f5dfe000640 1 -- start start 2024-09-19T04:44:54.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.517+0000 7f5dfe000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5df8101de0 0x7f5df81984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:54.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.517+0000 7f5dfe000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5df8102790 0x7f5df81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:54.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.517+0000 7f5dfe000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5df8106f10 0x7f5df819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:54.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.517+0000 7f5dfe000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5df8111fc0 con 0x7f5df8102790 2024-09-19T04:44:54.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.517+0000 7f5dfe000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5df8111e40 con 0x7f5df8106f10 2024-09-19T04:44:54.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.517+0000 7f5dfe000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5df8112140 con 0x7f5df8101de0 2024-09-19T04:44:54.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.517+0000 7f5deea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5df8102790 0x7f5df81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:54.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.517+0000 7f5deea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5df8102790 0x7f5df81989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57102/0 (socket says 172.21.15.123:57102) 2024-09-19T04:44:54.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.517+0000 7f5deea00640 1 -- 172.21.15.123:0/4121429480 learned_addr learned my addr 172.21.15.123:0/4121429480 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:54.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.517+0000 7f5df7e00640 1 --2- 172.21.15.123:0/4121429480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5df8106f10 0x7f5df819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:54.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.517+0000 7f5df7400640 1 --2- 172.21.15.123:0/4121429480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5df8101de0 0x7f5df81984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:54.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.517+0000 7f5deea00640 1 -- 172.21.15.123:0/4121429480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5df8101de0 msgr2=0x7f5df81984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:54.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.517+0000 7f5deea00640 1 --2- 172.21.15.123:0/4121429480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5df8101de0 0x7f5df81984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:54.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.517+0000 7f5deea00640 1 -- 172.21.15.123:0/4121429480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5df8106f10 msgr2=0x7f5df819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:54.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.517+0000 7f5deea00640 1 --2- 172.21.15.123:0/4121429480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5df8106f10 0x7f5df819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:54.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.517+0000 7f5deea00640 1 -- 172.21.15.123:0/4121429480 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df819d450 con 0x7f5df8102790 2024-09-19T04:44:54.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.518+0000 7f5deea00640 1 --2- 172.21.15.123:0/4121429480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5df8102790 0x7f5df81989e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5de000e9f0 tx=0x7f5de000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:54.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.518+0000 7f5df4c00640 1 -- 172.21.15.123:0/4121429480 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de000cd80 con 0x7f5df8102790 2024-09-19T04:44:54.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.518+0000 7f5df4c00640 1 -- 172.21.15.123:0/4121429480 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de0005500 con 0x7f5df8102790 2024-09-19T04:44:54.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.518+0000 7f5df4c00640 1 -- 172.21.15.123:0/4121429480 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0010690 con 0x7f5df8102790 2024-09-19T04:44:54.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.518+0000 7f5dfe000640 1 -- 172.21.15.123:0/4121429480 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df819d740 con 0x7f5df8102790 2024-09-19T04:44:54.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.518+0000 7f5dfe000640 1 -- 172.21.15.123:0/4121429480 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df81a5020 con 0x7f5df8102790 2024-09-19T04:44:54.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.519+0000 7f5dfe000640 1 -- 172.21.15.123:0/4121429480 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df81021c0 con 0x7f5df8102790 2024-09-19T04:44:54.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.521+0000 7f5df4c00640 1 -- 172.21.15.123:0/4121429480 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5de00046c0 con 0x7f5df8102790 2024-09-19T04:44:54.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.522+0000 7f5df4c00640 1 --2- 172.21.15.123:0/4121429480 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5ddc0a3350 0x7f5ddc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:54.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.522+0000 7f5df4c00640 1 -- 172.21.15.123:0/4121429480 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f5de00cfd70 con 0x7f5df8102790 2024-09-19T04:44:54.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.522+0000 7f5df7400640 1 --2- 172.21.15.123:0/4121429480 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5ddc0a3350 0x7f5ddc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:54.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.523+0000 7f5df7400640 1 --2- 172.21.15.123:0/4121429480 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5ddc0a3350 0x7f5ddc0a5810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5de8000c00 tx=0x7f5de8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:54.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.526+0000 7f5df4c00640 1 -- 172.21.15.123:0/4121429480 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5de009c9e0 con 0x7f5df8102790 2024-09-19T04:44:54.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.730+0000 7f5dfe000640 1 -- 172.21.15.123:0/4121429480 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f5df8062630 con 0x7f5ddc0a3350 2024-09-19T04:44:54.732 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:54 smithi123 ceph-mon[30511]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:44:54.732 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:54 smithi123 ceph-mon[30511]: pgmap v304: 97 pgs: 97 active+clean; 610 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 23 op/s 2024-09-19T04:44:54.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.737+0000 7f5df4c00640 1 -- 172.21.15.123:0/4121429480 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5df8062630 con 0x7f5ddc0a3350 2024-09-19T04:44:54.738 INFO:teuthology.orchestra.run.smithi123.stdout:/volumes/qa/sv_1/403e363f-2f5b-43da-b341-82f288e03744 2024-09-19T04:44:54.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:54 smithi193 ceph-mon[35592]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:44:54.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:54 smithi193 ceph-mon[35592]: pgmap v304: 97 pgs: 97 active+clean; 610 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 23 op/s 2024-09-19T04:44:54.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.739+0000 7f5dfe000640 1 -- 172.21.15.123:0/4121429480 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5ddc0a3350 msgr2=0x7f5ddc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:54.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.739+0000 7f5dfe000640 1 --2- 172.21.15.123:0/4121429480 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5ddc0a3350 0x7f5ddc0a5810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5de8000c00 tx=0x7f5de8005b50 comp rx=0 tx=0).stop 2024-09-19T04:44:54.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.739+0000 7f5dfe000640 1 -- 172.21.15.123:0/4121429480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5df8102790 msgr2=0x7f5df81989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:54.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.739+0000 7f5dfe000640 1 --2- 172.21.15.123:0/4121429480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5df8102790 0x7f5df81989e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5de000e9f0 tx=0x7f5de000edf0 comp rx=0 tx=0).stop 2024-09-19T04:44:54.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.739+0000 7f5dfe000640 1 -- 172.21.15.123:0/4121429480 shutdown_connections 2024-09-19T04:44:54.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.739+0000 7f5dfe000640 1 --2- 172.21.15.123:0/4121429480 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5ddc0a3350 0x7f5ddc0a5810 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:54.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.739+0000 7f5dfe000640 1 --2- 172.21.15.123:0/4121429480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5df8106f10 0x7f5df819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:54.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.739+0000 7f5dfe000640 1 --2- 172.21.15.123:0/4121429480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5df8102790 0x7f5df81989e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:54.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.739+0000 7f5dfe000640 1 --2- 172.21.15.123:0/4121429480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5df8101de0 0x7f5df81984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:54.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.739+0000 7f5dfe000640 1 -- 172.21.15.123:0/4121429480 >> 172.21.15.123:0/4121429480 conn(0x7f5df80fd8c0 msgr2=0x7f5df80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:54.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.739+0000 7f5dfe000640 1 -- 172.21.15.123:0/4121429480 shutdown_connections 2024-09-19T04:44:54.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:54.739+0000 7f5dfe000640 1 -- 172.21.15.123:0/4121429480 wait complete. 2024-09-19T04:44:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:54 smithi183 ceph-mon[35333]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:44:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:54 smithi183 ceph-mon[35333]: pgmap v304: 97 pgs: 97 active+clean; 610 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 23 op/s 2024-09-19T04:44:55.501 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:44:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:56.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.448+0000 7fc351000640 1 -- 172.21.15.123:0/4182153904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc34c1049a0 msgr2=0x7fc34c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:56.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.448+0000 7fc351000640 1 --2- 172.21.15.123:0/4182153904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc34c1049a0 0x7fc34c1089f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc334009900 tx=0x7fc33402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:44:56.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.448+0000 7fc351000640 1 -- 172.21.15.123:0/4182153904 shutdown_connections 2024-09-19T04:44:56.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.448+0000 7fc351000640 1 --2- 172.21.15.123:0/4182153904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc34c109120 0x7fc34c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:56.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.448+0000 7fc351000640 1 --2- 172.21.15.123:0/4182153904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc34c1049a0 0x7fc34c1089f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:56.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.448+0000 7fc351000640 1 --2- 172.21.15.123:0/4182153904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc34c103ff0 0x7fc34c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:56.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.448+0000 7fc351000640 1 -- 172.21.15.123:0/4182153904 >> 172.21.15.123:0/4182153904 conn(0x7fc34c0ffa90 msgr2=0x7fc34c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:56.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.448+0000 7fc351000640 1 -- 172.21.15.123:0/4182153904 shutdown_connections 2024-09-19T04:44:56.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.448+0000 7fc351000640 1 -- 172.21.15.123:0/4182153904 wait complete. 2024-09-19T04:44:56.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.449+0000 7fc351000640 1 Processor -- start 2024-09-19T04:44:56.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.449+0000 7fc351000640 1 -- start start 2024-09-19T04:44:56.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.449+0000 7fc351000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc34c103ff0 0x7fc34c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:56.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.449+0000 7fc351000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc34c1049a0 0x7fc34c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:56.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.449+0000 7fc351000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc34c109120 0x7fc34c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:56.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.449+0000 7fc351000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c114190 con 0x7fc34c1049a0 2024-09-19T04:44:56.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.449+0000 7fc351000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c114010 con 0x7fc34c109120 2024-09-19T04:44:56.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.449+0000 7fc351000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c114310 con 0x7fc34c103ff0 2024-09-19T04:44:56.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.449+0000 7fc343e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc34c1049a0 0x7fc34c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:56.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.450+0000 7fc343e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc34c1049a0 0x7fc34c19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57124/0 (socket says 172.21.15.123:57124) 2024-09-19T04:44:56.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.450+0000 7fc343e00640 1 -- 172.21.15.123:0/1879241050 learned_addr learned my addr 172.21.15.123:0/1879241050 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:56.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.450+0000 7fc34a800640 1 --2- 172.21.15.123:0/1879241050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc34c109120 0x7fc34c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:56.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.450+0000 7fc349e00640 1 --2- 172.21.15.123:0/1879241050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc34c103ff0 0x7fc34c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:56.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.450+0000 7fc343e00640 1 -- 172.21.15.123:0/1879241050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc34c103ff0 msgr2=0x7fc34c19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:56.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.450+0000 7fc343e00640 1 --2- 172.21.15.123:0/1879241050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc34c103ff0 0x7fc34c19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:56.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.450+0000 7fc343e00640 1 -- 172.21.15.123:0/1879241050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc34c109120 msgr2=0x7fc34c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:56.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.450+0000 7fc343e00640 1 --2- 172.21.15.123:0/1879241050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc34c109120 0x7fc34c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:56.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.450+0000 7fc343e00640 1 -- 172.21.15.123:0/1879241050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc34c19f5e0 con 0x7fc34c1049a0 2024-09-19T04:44:56.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.450+0000 7fc34a800640 1 --2- 172.21.15.123:0/1879241050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc34c109120 0x7fc34c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:44:56.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.450+0000 7fc343e00640 1 --2- 172.21.15.123:0/1879241050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc34c1049a0 0x7fc34c19ab70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc334004770 tx=0x7fc334003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:56.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.451+0000 7fc342a00640 1 -- 172.21.15.123:0/1879241050 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33403f070 con 0x7fc34c1049a0 2024-09-19T04:44:56.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.451+0000 7fc342a00640 1 -- 172.21.15.123:0/1879241050 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc334031ca0 con 0x7fc34c1049a0 2024-09-19T04:44:56.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.451+0000 7fc351000640 1 -- 172.21.15.123:0/1879241050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc34c19f870 con 0x7fc34c1049a0 2024-09-19T04:44:56.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.451+0000 7fc342a00640 1 -- 172.21.15.123:0/1879241050 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334035030 con 0x7fc34c1049a0 2024-09-19T04:44:56.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.451+0000 7fc351000640 1 -- 172.21.15.123:0/1879241050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc34c1a71b0 con 0x7fc34c1049a0 2024-09-19T04:44:56.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.452+0000 7fc351000640 1 -- 172.21.15.123:0/1879241050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc34c1054d0 con 0x7fc34c1049a0 2024-09-19T04:44:56.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.454+0000 7fc342a00640 1 -- 172.21.15.123:0/1879241050 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc334042070 con 0x7fc34c1049a0 2024-09-19T04:44:56.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.455+0000 7fc342a00640 1 --2- 172.21.15.123:0/1879241050 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc3300a32d0 0x7fc3300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:56.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.455+0000 7fc342a00640 1 -- 172.21.15.123:0/1879241050 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fc3340effb0 con 0x7fc34c1049a0 2024-09-19T04:44:56.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.455+0000 7fc349e00640 1 --2- 172.21.15.123:0/1879241050 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc3300a32d0 0x7fc3300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:56.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.456+0000 7fc349e00640 1 --2- 172.21.15.123:0/1879241050 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc3300a32d0 0x7fc3300a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009bd0 tx=0x7fc33c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:56.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.459+0000 7fc342a00640 1 -- 172.21.15.123:0/1879241050 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc3340bcc20 con 0x7fc34c1049a0 2024-09-19T04:44:56.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.669+0000 7fc351000640 1 -- 172.21.15.123:0/1879241050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc34c10c360 con 0x7fc34c1049a0 2024-09-19T04:44:56.671 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:56 smithi123 ceph-mon[30511]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:44:56.671 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:56 smithi123 ceph-mon[30511]: pgmap v305: 97 pgs: 97 active+clean; 610 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 18 op/s 2024-09-19T04:44:56.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.670+0000 7fc342a00640 1 -- 172.21.15.123:0/1879241050 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11990 (secure 0 0 0) 0x7fc3340bc520 con 0x7fc34c1049a0 2024-09-19T04:44:56.672 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:44:56.672 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"btime":"2024-09-19T04:44:44:834422+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:44:44.834407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24529,24505,24539,24500,24499]},"id":1}]} 2024-09-19T04:44:56.672 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 27 2024-09-19T04:44:56.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.672+0000 7fc351000640 1 -- 172.21.15.123:0/1879241050 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc3300a32d0 msgr2=0x7fc3300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:56.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.672+0000 7fc351000640 1 --2- 172.21.15.123:0/1879241050 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc3300a32d0 0x7fc3300a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009bd0 tx=0x7fc33c005f90 comp rx=0 tx=0).stop 2024-09-19T04:44:56.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.672+0000 7fc351000640 1 -- 172.21.15.123:0/1879241050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc34c1049a0 msgr2=0x7fc34c19ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:56.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.672+0000 7fc351000640 1 --2- 172.21.15.123:0/1879241050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc34c1049a0 0x7fc34c19ab70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc334004770 tx=0x7fc334003c10 comp rx=0 tx=0).stop 2024-09-19T04:44:56.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.672+0000 7fc351000640 1 -- 172.21.15.123:0/1879241050 shutdown_connections 2024-09-19T04:44:56.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.672+0000 7fc351000640 1 --2- 172.21.15.123:0/1879241050 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc3300a32d0 0x7fc3300a5790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:56.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.672+0000 7fc351000640 1 --2- 172.21.15.123:0/1879241050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc34c109120 0x7fc34c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:56.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.672+0000 7fc351000640 1 --2- 172.21.15.123:0/1879241050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc34c1049a0 0x7fc34c19ab70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:56.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.672+0000 7fc351000640 1 --2- 172.21.15.123:0/1879241050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc34c103ff0 0x7fc34c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:56.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.672+0000 7fc351000640 1 -- 172.21.15.123:0/1879241050 >> 172.21.15.123:0/1879241050 conn(0x7fc34c0ffa90 msgr2=0x7fc34c100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:56.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.673+0000 7fc351000640 1 -- 172.21.15.123:0/1879241050 shutdown_connections 2024-09-19T04:44:56.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:56.673+0000 7fc351000640 1 -- 172.21.15.123:0/1879241050 wait complete. 2024-09-19T04:44:56.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:56 smithi193 ceph-mon[35592]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:44:56.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:56 smithi193 ceph-mon[35592]: pgmap v305: 97 pgs: 97 active+clean; 610 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 18 op/s 2024-09-19T04:44:56.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:56 smithi183 ceph-mon[35333]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:44:56.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:56 smithi183 ceph-mon[35333]: pgmap v305: 97 pgs: 97 active+clean; 610 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 18 op/s 2024-09-19T04:44:57.550 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:44:57.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:57 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1879241050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:57.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:57 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1879241050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:57.921 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:57.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:57 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1879241050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:44:58.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.443+0000 7f4d4d800640 1 -- 172.21.15.123:0/1136205245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d48103ff0 msgr2=0x7f4d481043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:58.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.443+0000 7f4d4d800640 1 --2- 172.21.15.123:0/1136205245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d48103ff0 0x7f4d481043d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009900 tx=0x7f4d3802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:44:58.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.443+0000 7f4d4d800640 1 -- 172.21.15.123:0/1136205245 shutdown_connections 2024-09-19T04:44:58.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.443+0000 7f4d4d800640 1 --2- 172.21.15.123:0/1136205245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d48109120 0x7f4d48110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:58.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.443+0000 7f4d4d800640 1 --2- 172.21.15.123:0/1136205245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d481049a0 0x7f4d481089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:58.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.443+0000 7f4d4d800640 1 --2- 172.21.15.123:0/1136205245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d48103ff0 0x7f4d481043d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:58.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.443+0000 7f4d4d800640 1 -- 172.21.15.123:0/1136205245 >> 172.21.15.123:0/1136205245 conn(0x7f4d480ffa90 msgr2=0x7f4d48101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:58.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.443+0000 7f4d4d800640 1 -- 172.21.15.123:0/1136205245 shutdown_connections 2024-09-19T04:44:58.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.443+0000 7f4d4d800640 1 -- 172.21.15.123:0/1136205245 wait complete. 2024-09-19T04:44:58.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.444+0000 7f4d4d800640 1 Processor -- start 2024-09-19T04:44:58.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.444+0000 7f4d4d800640 1 -- start start 2024-09-19T04:44:58.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.445+0000 7f4d4d800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d48103ff0 0x7f4d4819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:58.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.445+0000 7f4d4d800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d481049a0 0x7f4d4819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:58.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.445+0000 7f4d4d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d48109120 0x7f4d4819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:58.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.445+0000 7f4d4d800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d48114150 con 0x7f4d48109120 2024-09-19T04:44:58.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.445+0000 7f4d4d800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4d48113fd0 con 0x7f4d48103ff0 2024-09-19T04:44:58.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.445+0000 7f4d4d800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481142d0 con 0x7f4d481049a0 2024-09-19T04:44:58.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.445+0000 7f4d46000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d481049a0 0x7f4d4819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:58.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.445+0000 7f4d46000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d481049a0 0x7f4d4819ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:59172/0 (socket says 172.21.15.123:59172) 2024-09-19T04:44:58.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.445+0000 7f4d46a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d48103ff0 0x7f4d4819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:58.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.445+0000 7f4d46000640 1 -- 172.21.15.123:0/1539103736 learned_addr learned my addr 172.21.15.123:0/1539103736 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:44:58.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.445+0000 7f4d47400640 1 --2- 172.21.15.123:0/1539103736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d48109120 0x7f4d4819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:58.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.445+0000 7f4d46a00640 1 -- 172.21.15.123:0/1539103736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d481049a0 msgr2=0x7f4d4819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:58.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.445+0000 7f4d46a00640 1 --2- 172.21.15.123:0/1539103736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d481049a0 0x7f4d4819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:58.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.445+0000 7f4d46a00640 1 -- 172.21.15.123:0/1539103736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d48109120 msgr2=0x7f4d4819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:58.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.445+0000 7f4d46a00640 1 --2- 172.21.15.123:0/1539103736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d48109120 0x7f4d4819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:58.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.445+0000 7f4d46a00640 1 -- 172.21.15.123:0/1539103736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4819f5a0 con 0x7f4d48103ff0 2024-09-19T04:44:58.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.445+0000 7f4d46000640 1 --2- 172.21.15.123:0/1539103736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d481049a0 0x7f4d4819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:44:58.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.446+0000 7f4d46a00640 1 --2- 172.21.15.123:0/1539103736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d48103ff0 0x7f4d4819a5f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d38005940 tx=0x7f4d38004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:58.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.447+0000 7f4d3f400640 1 -- 172.21.15.123:0/1539103736 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3803f030 con 0x7f4d48103ff0 2024-09-19T04:44:58.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.447+0000 7f4d3f400640 1 -- 172.21.15.123:0/1539103736 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d38003ad0 con 0x7f4d48103ff0 2024-09-19T04:44:58.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.447+0000 7f4d4d800640 1 -- 172.21.15.123:0/1539103736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4819f830 con 0x7f4d48103ff0 2024-09-19T04:44:58.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.447+0000 7f4d3f400640 1 -- 172.21.15.123:0/1539103736 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38035070 con 0x7f4d48103ff0 2024-09-19T04:44:58.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.447+0000 7f4d4d800640 1 -- 172.21.15.123:0/1539103736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d481a7170 con 0x7f4d48103ff0 2024-09-19T04:44:58.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.448+0000 7f4d4d800640 1 -- 172.21.15.123:0/1539103736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d481043d0 con 0x7f4d48103ff0 2024-09-19T04:44:58.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.449+0000 7f4d3f400640 1 -- 172.21.15.123:0/1539103736 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4d38042030 con 0x7f4d48103ff0 2024-09-19T04:44:58.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.451+0000 7f4d3f400640 1 --2- 172.21.15.123:0/1539103736 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4d2c0a3350 0x7f4d2c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:44:58.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.451+0000 7f4d3f400640 1 -- 172.21.15.123:0/1539103736 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f4d380eff40 con 0x7f4d48103ff0 2024-09-19T04:44:58.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.451+0000 7f4d46000640 1 --2- 172.21.15.123:0/1539103736 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4d2c0a3350 0x7f4d2c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:44:58.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.452+0000 7f4d46000640 1 --2- 172.21.15.123:0/1539103736 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4d2c0a3350 0x7f4d2c0a5810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4d480ffeb0 tx=0x7f4d3400b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:44:58.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.455+0000 7f4d3f400640 1 -- 172.21.15.123:0/1539103736 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4d380bc560 con 0x7f4d48103ff0 2024-09-19T04:44:58.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.632+0000 7f4d4d800640 1 -- 172.21.15.123:0/1539103736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d4810c300 con 0x7f4d48103ff0 2024-09-19T04:44:58.633 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:58 smithi123 ceph-mon[30511]: pgmap v306: 97 pgs: 97 active+clean; 610 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-09-19T04:44:58.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.633+0000 7f4d3f400640 1 -- 172.21.15.123:0/1539103736 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22362 (secure 0 0 0) 0x7f4d380c0860 con 0x7f4d48103ff0 2024-09-19T04:44:58.636 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:44:58.636 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":84,"fsid":"911fa052-7640-11ef-bceb-c7b262605968","created":"2024-09-19T04:36:21.083333+0000","modified":"2024-09-19T04:44:31.766762+0000","last_up_change":"2024-09-19T04:43:33.818439+0000","last_in_change":"2024-09-19T04:43:10.148272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:39:33.856981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:44:26.956483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:44:29.718625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6803","nonce":3631596501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6805","nonce":3631596501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6809","nonce":3631596501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6807","nonce":3631596501}]},"public_addr":"172.21.15.123:6803/3631596501","cluster_addr":"172.21.15.123:6805/3631596501","heartbeat_back_addr":"172.21.15.123:6809/3631596501","heartbeat_front_addr":"172.21.15.123:6807/3631596501","state":["exists","up"]},{"osd":1,"uuid":"a526d4ce-586b-4d66-b720-e868df92fa24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6801","nonce":1259721614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6803","nonce":1259721614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6807","nonce":1259721614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6805","nonce":1259721614}]},"public_addr":"172.21.15.183:6801/1259721614","cluster_addr":"172.21.15.183:6803/1259721614","heartbeat_back_addr":"172.21.15.183:6807/1259721614","heartbeat_front_addr":"172.21.15.183:6805/1259721614","state":["exists","up"]},{"osd":2,"uuid":"3650601a-6259-4b2a-97df-1d47b09b0127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6801","nonce":2708151726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6803","nonce":2708151726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6807","nonce":2708151726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6805","nonce":2708151726}]},"public_addr":"172.21.15.193:6801/2708151726","cluster_addr":"172.21.15.193:6803/2708151726","heartbeat_back_addr":"172.21.15.193:6807/2708151726","heartbeat_front_addr":"172.21.15.193:6805/2708151726","state":["exists","up"]},{"osd":3,"uuid":"67c485ad-136e-4020-950d-9171a3c2de35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6811","nonce":3214806282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6813","nonce":3214806282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6817","nonce":3214806282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6815","nonce":3214806282}]},"public_addr":"172.21.15.123:6811/3214806282","cluster_addr":"172.21.15.123:6813/3214806282","heartbeat_back_addr":"172.21.15.123:6817/3214806282","heartbeat_front_addr":"172.21.15.123:6815/3214806282","state":["exists","up"]},{"osd":4,"uuid":"c9098540-06ab-45a8-9cf9-f90519cc852e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6809","nonce":667618665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6811","nonce":667618665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6815","nonce":667618665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6813","nonce":667618665}]},"public_addr":"172.21.15.183:6809/667618665","cluster_addr":"172.21.15.183:6811/667618665","heartbeat_back_addr":"172.21.15.183:6815/667618665","heartbeat_front_addr":"172.21.15.183:6813/667618665","state":["exists","up"]},{"osd":5,"uuid":"278a8dbd-6492-42d4-b489-1f832350bc6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6809","nonce":2292251639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6811","nonce":2292251639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6815","nonce":2292251639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6813","nonce":2292251639}]},"public_addr":"172.21.15.193:6809/2292251639","cluster_addr":"172.21.15.193:6811/2292251639","heartbeat_back_addr":"172.21.15.193:6815/2292251639","heartbeat_front_addr":"172.21.15.193:6813/2292251639","state":["exists","up"]},{"osd":6,"uuid":"0fda4e39-6adb-4941-a39b-a9a15019088c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6819","nonce":226007499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6821","nonce":226007499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6825","nonce":226007499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6823","nonce":226007499}]},"public_addr":"172.21.15.123:6819/226007499","cluster_addr":"172.21.15.123:6821/226007499","heartbeat_back_addr":"172.21.15.123:6825/226007499","heartbeat_front_addr":"172.21.15.123:6823/226007499","state":["exists","up"]},{"osd":7,"uuid":"5c0a021e-9f3b-4766-9686-0ef24b68cf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6817","nonce":3718518226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6819","nonce":3718518226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6823","nonce":3718518226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6821","nonce":3718518226}]},"public_addr":"172.21.15.183:6817/3718518226","cluster_addr":"172.21.15.183:6819/3718518226","heartbeat_back_addr":"172.21.15.183:6823/3718518226","heartbeat_front_addr":"172.21.15.183:6821/3718518226","state":["exists","up"]},{"osd":8,"uuid":"9b0e67e0-6305-4f81-9211-d41960099a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6817","nonce":3551404118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6819","nonce":3551404118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6823","nonce":3551404118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6821","nonce":3551404118}]},"public_addr":"172.21.15.193:6817/3551404118","cluster_addr":"172.21.15.193:6819/3551404118","heartbeat_back_addr":"172.21.15.193:6823/3551404118","heartbeat_front_addr":"172.21.15.193:6821/3551404118","state":["exists","up"]},{"osd":9,"uuid":"978ef3fa-c6ec-421f-b194-56d44813be5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6827","nonce":1439901149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6829","nonce":1439901149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6833","nonce":1439901149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6831","nonce":1439901149}]},"public_addr":"172.21.15.123:6827/1439901149","cluster_addr":"172.21.15.123:6829/1439901149","heartbeat_back_addr":"172.21.15.123:6833/1439901149","heartbeat_front_addr":"172.21.15.123:6831/1439901149","state":["exists","up"]},{"osd":10,"uuid":"5495e09b-7809-47b0-b7e0-b56c452f9f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6825","nonce":3355349245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6827","nonce":3355349245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6831","nonce":3355349245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6829","nonce":3355349245}]},"public_addr":"172.21.15.183:6825/3355349245","cluster_addr":"172.21.15.183:6827/3355349245","heartbeat_back_addr":"172.21.15.183:6831/3355349245","heartbeat_front_addr":"172.21.15.183:6829/3355349245","state":["exists","up"]},{"osd":11,"uuid":"03046fef-b16b-42f8-9a19-6c0b78cbfe7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6825","nonce":4291617038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6827","nonce":4291617038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6831","nonce":4291617038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6829","nonce":4291617038}]},"public_addr":"172.21.15.193:6825/4291617038","cluster_addr":"172.21.15.193:6827/4291617038","heartbeat_back_addr":"172.21.15.193:6831/4291617038","heartbeat_front_addr":"172.21.15.193:6829/4291617038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:38:39.142529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:05.336911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:30.310967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:59.655973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:25.890576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:50.778689+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:19.063731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:45.424269+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:10.562149+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:40.382368+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:06.297731+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:32.352172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/4040401634":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/3498196650":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6800/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:6800/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/471028951":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/1009496020":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/2000593686":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/2580747029":"2024-09-20T04:36:45.913060+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:44:58.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.637+0000 7f4d4d800640 1 -- 172.21.15.123:0/1539103736 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4d2c0a3350 msgr2=0x7f4d2c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:58.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.637+0000 7f4d4d800640 1 --2- 172.21.15.123:0/1539103736 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4d2c0a3350 0x7f4d2c0a5810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4d480ffeb0 tx=0x7f4d3400b4a0 comp rx=0 tx=0).stop 2024-09-19T04:44:58.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.637+0000 7f4d4d800640 1 -- 172.21.15.123:0/1539103736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d48103ff0 msgr2=0x7f4d4819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:44:58.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.637+0000 7f4d4d800640 1 --2- 172.21.15.123:0/1539103736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d48103ff0 0x7f4d4819a5f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d38005940 tx=0x7f4d38004270 comp rx=0 tx=0).stop 2024-09-19T04:44:58.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.637+0000 7f4d4d800640 1 -- 172.21.15.123:0/1539103736 shutdown_connections 2024-09-19T04:44:58.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.637+0000 7f4d4d800640 1 --2- 172.21.15.123:0/1539103736 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4d2c0a3350 0x7f4d2c0a5810 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:58.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.637+0000 7f4d4d800640 1 --2- 172.21.15.123:0/1539103736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d48109120 0x7f4d4819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:58.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.637+0000 7f4d4d800640 1 --2- 172.21.15.123:0/1539103736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d481049a0 0x7f4d4819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:58.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.637+0000 7f4d4d800640 1 --2- 172.21.15.123:0/1539103736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d48103ff0 0x7f4d4819a5f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:44:58.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.637+0000 7f4d4d800640 1 -- 172.21.15.123:0/1539103736 >> 172.21.15.123:0/1539103736 conn(0x7f4d480ffa90 msgr2=0x7f4d48100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:44:58.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.637+0000 7f4d4d800640 1 -- 172.21.15.123:0/1539103736 shutdown_connections 2024-09-19T04:44:58.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:44:58.637+0000 7f4d4d800640 1 -- 172.21.15.123:0/1539103736 wait complete. 2024-09-19T04:44:58.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:58 smithi193 ceph-mon[35592]: pgmap v306: 97 pgs: 97 active+clean; 610 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-09-19T04:44:58.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:58 smithi183 ceph-mon[35333]: pgmap v306: 97 pgs: 97 active+clean; 610 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-09-19T04:44:59.489 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T04:44:59.489 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:44:59.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:44:59 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1539103736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:44:59.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:44:59 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1539103736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:44:59.860 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:44:59.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:44:59 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1539103736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:45:00.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.399+0000 7fcc28400640 1 -- 172.21.15.123:0/3011823388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc20106f10 msgr2=0x7fcc2010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:00.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.399+0000 7fcc28400640 1 --2- 172.21.15.123:0/3011823388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc20106f10 0x7fcc2010ea90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009ca0 tx=0x7fcc1802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:45:00.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.399+0000 7fcc28400640 1 -- 172.21.15.123:0/3011823388 shutdown_connections 2024-09-19T04:45:00.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.399+0000 7fcc28400640 1 --2- 172.21.15.123:0/3011823388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc20106f10 0x7fcc2010ea90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:00.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.399+0000 7fcc28400640 1 --2- 172.21.15.123:0/3011823388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc20102790 0x7fcc201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:00.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.399+0000 7fcc28400640 1 --2- 172.21.15.123:0/3011823388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc20101de0 0x7fcc201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:00.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.399+0000 7fcc28400640 1 -- 172.21.15.123:0/3011823388 >> 172.21.15.123:0/3011823388 conn(0x7fcc200fd8c0 msgr2=0x7fcc200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:00.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.399+0000 7fcc28400640 1 -- 172.21.15.123:0/3011823388 shutdown_connections 2024-09-19T04:45:00.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.399+0000 7fcc28400640 1 -- 172.21.15.123:0/3011823388 wait complete. 2024-09-19T04:45:00.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.400+0000 7fcc28400640 1 Processor -- start 2024-09-19T04:45:00.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.401+0000 7fcc28400640 1 -- start start 2024-09-19T04:45:00.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.401+0000 7fcc28400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc20101de0 0x7fcc2019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:00.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.401+0000 7fcc28400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc20102790 0x7fcc2019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:00.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.401+0000 7fcc28400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc20106f10 0x7fcc2019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:00.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.401+0000 7fcc28400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcc20111f70 con 0x7fcc20101de0 2024-09-19T04:45:00.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.401+0000 7fcc28400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc20111df0 con 0x7fcc20106f10 2024-09-19T04:45:00.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.401+0000 7fcc28400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcc201120f0 con 0x7fcc20102790 2024-09-19T04:45:00.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.401+0000 7fcc25a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc20101de0 0x7fcc2019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:00.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.401+0000 7fcc25a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc20101de0 0x7fcc2019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57174/0 (socket says 172.21.15.123:57174) 2024-09-19T04:45:00.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.401+0000 7fcc25a00640 1 -- 172.21.15.123:0/3091160968 learned_addr learned my addr 172.21.15.123:0/3091160968 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:00.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.401+0000 7fcc25000640 1 --2- 172.21.15.123:0/3091160968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc20102790 0x7fcc2019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:00.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.401+0000 7fcc26400640 1 --2- 172.21.15.123:0/3091160968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc20106f10 0x7fcc2019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:00.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.401+0000 7fcc25a00640 1 -- 172.21.15.123:0/3091160968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc20102790 msgr2=0x7fcc2019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:00.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.402+0000 7fcc25a00640 1 --2- 172.21.15.123:0/3091160968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc20102790 0x7fcc2019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:00.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.402+0000 7fcc25a00640 1 -- 172.21.15.123:0/3091160968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc20106f10 msgr2=0x7fcc2019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:00.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.402+0000 7fcc25a00640 1 --2- 172.21.15.123:0/3091160968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc20106f10 0x7fcc2019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:00.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.402+0000 7fcc25a00640 1 -- 172.21.15.123:0/3091160968 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2019f590 con 0x7fcc20101de0 2024-09-19T04:45:00.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.402+0000 7fcc25000640 1 --2- 172.21.15.123:0/3091160968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc20102790 0x7fcc2019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:45:00.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.402+0000 7fcc25a00640 1 --2- 172.21.15.123:0/3091160968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc20101de0 0x7fcc2019a5e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400e9f0 tx=0x7fcc1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:00.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.402+0000 7fcc1e000640 1 -- 172.21.15.123:0/3091160968 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1400cd80 con 0x7fcc20101de0 2024-09-19T04:45:00.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.402+0000 7fcc1e000640 1 -- 172.21.15.123:0/3091160968 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc14010910 con 0x7fcc20101de0 2024-09-19T04:45:00.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.402+0000 7fcc28400640 1 -- 172.21.15.123:0/3091160968 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2019f880 con 0x7fcc20101de0 2024-09-19T04:45:00.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.402+0000 7fcc1e000640 1 -- 172.21.15.123:0/3091160968 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14018990 con 0x7fcc20101de0 2024-09-19T04:45:00.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.402+0000 7fcc28400640 1 -- 172.21.15.123:0/3091160968 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc201a7160 con 0x7fcc20101de0 2024-09-19T04:45:00.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.403+0000 7fcc28400640 1 -- 172.21.15.123:0/3091160968 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc201021c0 con 0x7fcc20101de0 2024-09-19T04:45:00.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.405+0000 7fcc1e000640 1 -- 172.21.15.123:0/3091160968 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fcc14010ac0 con 0x7fcc20101de0 2024-09-19T04:45:00.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.407+0000 7fcc1e000640 1 --2- 172.21.15.123:0/3091160968 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcc080a32d0 0x7fcc080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:00.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.407+0000 7fcc1e000640 1 -- 172.21.15.123:0/3091160968 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fcc140cfe60 con 0x7fcc20101de0 2024-09-19T04:45:00.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.407+0000 7fcc25000640 1 --2- 172.21.15.123:0/3091160968 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcc080a32d0 0x7fcc080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:00.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.408+0000 7fcc25000640 1 --2- 172.21.15.123:0/3091160968 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcc080a32d0 0x7fcc080a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcc2010ef40 tx=0x7fcc0c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:00.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.411+0000 7fcc1e000640 1 -- 172.21.15.123:0/3091160968 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcc1409cad0 con 0x7fcc20101de0 2024-09-19T04:45:00.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.584+0000 7fcc28400640 1 -- 172.21.15.123:0/3091160968 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcc200febb0 con 0x7fcc080a32d0 2024-09-19T04:45:00.586 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:00 smithi123 ceph-mon[30511]: pgmap v307: 97 pgs: 97 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 21 op/s 2024-09-19T04:45:00.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.596+0000 7fcc1e000640 1 -- 172.21.15.123:0/3091160968 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258894 (secure 0 0 0) 0x7fcc200febb0 con 0x7fcc080a32d0 2024-09-19T04:45:00.601 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:45:00.602 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-09-19T04:45:00.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.600+0000 7fcc28400640 1 -- 172.21.15.123:0/3091160968 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcc080a32d0 msgr2=0x7fcc080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:00.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.600+0000 7fcc28400640 1 --2- 172.21.15.123:0/3091160968 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcc080a32d0 0x7fcc080a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcc2010ef40 tx=0x7fcc0c005f90 comp rx=0 tx=0).stop 2024-09-19T04:45:00.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.600+0000 7fcc28400640 1 -- 172.21.15.123:0/3091160968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc20101de0 msgr2=0x7fcc2019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:00.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.600+0000 7fcc28400640 1 --2- 172.21.15.123:0/3091160968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc20101de0 0x7fcc2019a5e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400e9f0 tx=0x7fcc1400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:45:00.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.600+0000 7fcc28400640 1 -- 172.21.15.123:0/3091160968 shutdown_connections 2024-09-19T04:45:00.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.600+0000 7fcc28400640 1 --2- 172.21.15.123:0/3091160968 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fcc080a32d0 0x7fcc080a5790 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:00.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.600+0000 7fcc28400640 1 --2- 172.21.15.123:0/3091160968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc20106f10 0x7fcc2019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:00.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.600+0000 7fcc28400640 1 --2- 172.21.15.123:0/3091160968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc20102790 0x7fcc2019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:00.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.600+0000 7fcc28400640 1 --2- 172.21.15.123:0/3091160968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc20101de0 0x7fcc2019a5e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:00.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.600+0000 7fcc28400640 1 -- 172.21.15.123:0/3091160968 >> 172.21.15.123:0/3091160968 conn(0x7fcc200fd8c0 msgr2=0x7fcc200ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:00.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.600+0000 7fcc28400640 1 -- 172.21.15.123:0/3091160968 shutdown_connections 2024-09-19T04:45:00.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:00.600+0000 7fcc28400640 1 -- 172.21.15.123:0/3091160968 wait complete. 2024-09-19T04:45:00.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:00 smithi193 ceph-mon[35592]: pgmap v307: 97 pgs: 97 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 21 op/s 2024-09-19T04:45:00.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:00 smithi183 ceph-mon[35333]: pgmap v307: 97 pgs: 97 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 21 op/s 2024-09-19T04:45:01.471 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-19T04:44:59.355853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647529,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":304,"num_read_kb":265,"num_write":416,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356392,"kb_used_data":11304,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585432,"statfs":{"total":1151940427776,"available":1151575482368,"internally_reserved":0,"allocated":11575296,"data_stored":7194243,"data_compressed":108024,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51245,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":165,"num_read_kb":143,"num_write":96,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003116"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.763716+0000","last_change":"2024-09-19T04:44:30.754400+0000","last_active":"2024-09-19T04:44:32.763716+0000","last_peered":"2024-09-19T04:44:32.763716+0000","last_clean":"2024-09-19T04:44:32.763716+0000","last_became_active":"2024-09-19T04:44:30.754028+0000","last_became_peered":"2024-09-19T04:44:30.754028+0000","last_unstale":"2024-09-19T04:44:32.763716+0000","last_undegraded":"2024-09-19T04:44:32.763716+0000","last_fullsized":"2024-09-19T04:44:32.763716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:25.668478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.756971+0000","last_change":"2024-09-19T04:44:30.758072+0000","last_active":"2024-09-19T04:44:32.756971+0000","last_peered":"2024-09-19T04:44:32.756971+0000","last_clean":"2024-09-19T04:44:32.756971+0000","last_became_active":"2024-09-19T04:44:30.756734+0000","last_became_peered":"2024-09-19T04:44:30.756734+0000","last_unstale":"2024-09-19T04:44:32.756971+0000","last_undegraded":"2024-09-19T04:44:32.756971+0000","last_fullsized":"2024-09-19T04:44:32.756971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:56:47.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.757241+0000","last_change":"2024-09-19T04:44:30.760208+0000","last_active":"2024-09-19T04:44:32.757241+0000","last_peered":"2024-09-19T04:44:32.757241+0000","last_clean":"2024-09-19T04:44:32.757241+0000","last_became_active":"2024-09-19T04:44:30.759719+0000","last_became_peered":"2024-09-19T04:44:30.759719+0000","last_unstale":"2024-09-19T04:44:32.757241+0000","last_undegraded":"2024-09-19T04:44:32.757241+0000","last_fullsized":"2024-09-19T04:44:32.757241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:14.041853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.760379+0000","last_change":"2024-09-19T04:44:30.747813+0000","last_active":"2024-09-19T04:44:32.760379+0000","last_peered":"2024-09-19T04:44:32.760379+0000","last_clean":"2024-09-19T04:44:32.760379+0000","last_became_active":"2024-09-19T04:44:30.747486+0000","last_became_peered":"2024-09-19T04:44:30.747486+0000","last_unstale":"2024-09-19T04:44:32.760379+0000","last_undegraded":"2024-09-19T04:44:32.760379+0000","last_fullsized":"2024-09-19T04:44:32.760379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:11.315273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.754512+0000","last_change":"2024-09-19T04:44:30.755800+0000","last_active":"2024-09-19T04:44:32.754512+0000","last_peered":"2024-09-19T04:44:32.754512+0000","last_clean":"2024-09-19T04:44:32.754512+0000","last_became_active":"2024-09-19T04:44:30.755448+0000","last_became_peered":"2024-09-19T04:44:30.755448+0000","last_unstale":"2024-09-19T04:44:32.754512+0000","last_undegraded":"2024-09-19T04:44:32.754512+0000","last_fullsized":"2024-09-19T04:44:32.754512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:39.308614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.966128+0000","last_change":"2024-09-19T04:44:30.746230+0000","last_active":"2024-09-19T04:44:31.966128+0000","last_peered":"2024-09-19T04:44:31.966128+0000","last_clean":"2024-09-19T04:44:31.966128+0000","last_became_active":"2024-09-19T04:44:30.745884+0000","last_became_peered":"2024-09-19T04:44:30.745884+0000","last_unstale":"2024-09-19T04:44:31.966128+0000","last_undegraded":"2024-09-19T04:44:31.966128+0000","last_fullsized":"2024-09-19T04:44:31.966128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:51:58.550322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.306935+0000","last_change":"2024-09-19T04:44:30.747870+0000","last_active":"2024-09-19T04:44:32.306935+0000","last_peered":"2024-09-19T04:44:32.306935+0000","last_clean":"2024-09-19T04:44:32.306935+0000","last_became_active":"2024-09-19T04:44:30.747536+0000","last_became_peered":"2024-09-19T04:44:30.747536+0000","last_unstale":"2024-09-19T04:44:32.306935+0000","last_undegraded":"2024-09-19T04:44:32.306935+0000","last_fullsized":"2024-09-19T04:44:32.306935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:20.074032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.349053+0000","last_change":"2024-09-19T04:44:30.750425+0000","last_active":"2024-09-19T04:44:32.349053+0000","last_peered":"2024-09-19T04:44:32.349053+0000","last_clean":"2024-09-19T04:44:32.349053+0000","last_became_active":"2024-09-19T04:44:30.749841+0000","last_became_peered":"2024-09-19T04:44:30.749841+0000","last_unstale":"2024-09-19T04:44:32.349053+0000","last_undegraded":"2024-09-19T04:44:32.349053+0000","last_fullsized":"2024-09-19T04:44:32.349053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:12:08.420727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.754381+0000","last_change":"2024-09-19T04:44:30.750624+0000","last_active":"2024-09-19T04:44:32.754381+0000","last_peered":"2024-09-19T04:44:32.754381+0000","last_clean":"2024-09-19T04:44:32.754381+0000","last_became_active":"2024-09-19T04:44:30.748370+0000","last_became_peered":"2024-09-19T04:44:30.748370+0000","last_unstale":"2024-09-19T04:44:32.754381+0000","last_undegraded":"2024-09-19T04:44:32.754381+0000","last_fullsized":"2024-09-19T04:44:32.754381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:36.131634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.967177+0000","last_change":"2024-09-19T04:44:30.750035+0000","last_active":"2024-09-19T04:44:31.967177+0000","last_peered":"2024-09-19T04:44:31.967177+0000","last_clean":"2024-09-19T04:44:31.967177+0000","last_became_active":"2024-09-19T04:44:30.749699+0000","last_became_peered":"2024-09-19T04:44:30.749699+0000","last_unstale":"2024-09-19T04:44:31.967177+0000","last_undegraded":"2024-09-19T04:44:31.967177+0000","last_fullsized":"2024-09-19T04:44:31.967177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:22.637193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.307412+0000","last_change":"2024-09-19T04:44:30.760258+0000","last_active":"2024-09-19T04:44:32.307412+0000","last_peered":"2024-09-19T04:44:32.307412+0000","last_clean":"2024-09-19T04:44:32.307412+0000","last_became_active":"2024-09-19T04:44:30.759925+0000","last_became_peered":"2024-09-19T04:44:30.759925+0000","last_unstale":"2024-09-19T04:44:32.307412+0000","last_undegraded":"2024-09-19T04:44:32.307412+0000","last_fullsized":"2024-09-19T04:44:32.307412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:58.144691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.760614+0000","last_change":"2024-09-19T04:44:30.747703+0000","last_active":"2024-09-19T04:44:32.760614+0000","last_peered":"2024-09-19T04:44:32.760614+0000","last_clean":"2024-09-19T04:44:32.760614+0000","last_became_active":"2024-09-19T04:44:30.747369+0000","last_became_peered":"2024-09-19T04:44:30.747369+0000","last_unstale":"2024-09-19T04:44:32.760614+0000","last_undegraded":"2024-09-19T04:44:32.760614+0000","last_fullsized":"2024-09-19T04:44:32.760614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:06.920357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.967545+0000","last_change":"2024-09-19T04:44:30.748844+0000","last_active":"2024-09-19T04:44:31.967545+0000","last_peered":"2024-09-19T04:44:31.967545+0000","last_clean":"2024-09-19T04:44:31.967545+0000","last_became_active":"2024-09-19T04:44:30.748513+0000","last_became_peered":"2024-09-19T04:44:30.748513+0000","last_unstale":"2024-09-19T04:44:31.967545+0000","last_undegraded":"2024-09-19T04:44:31.967545+0000","last_fullsized":"2024-09-19T04:44:31.967545+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:45.107718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.756835+0000","last_change":"2024-09-19T04:44:30.750526+0000","last_active":"2024-09-19T04:44:32.756835+0000","last_peered":"2024-09-19T04:44:32.756835+0000","last_clean":"2024-09-19T04:44:32.756835+0000","last_became_active":"2024-09-19T04:44:30.749693+0000","last_became_peered":"2024-09-19T04:44:30.749693+0000","last_unstale":"2024-09-19T04:44:32.756835+0000","last_undegraded":"2024-09-19T04:44:32.756835+0000","last_fullsized":"2024-09-19T04:44:32.756835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:43:04.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.757106+0000","last_change":"2024-09-19T04:44:30.754310+0000","last_active":"2024-09-19T04:44:32.757106+0000","last_peered":"2024-09-19T04:44:32.757106+0000","last_clean":"2024-09-19T04:44:32.757106+0000","last_became_active":"2024-09-19T04:44:30.753933+0000","last_became_peered":"2024-09-19T04:44:30.753933+0000","last_unstale":"2024-09-19T04:44:32.757106+0000","last_undegraded":"2024-09-19T04:44:32.757106+0000","last_fullsized":"2024-09-19T04:44:32.757106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:40:44.985693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.753494+0000","last_change":"2024-09-19T04:44:30.757458+0000","last_active":"2024-09-19T04:44:32.753494+0000","last_peered":"2024-09-19T04:44:32.753494+0000","last_clean":"2024-09-19T04:44:32.753494+0000","last_became_active":"2024-09-19T04:44:30.757127+0000","last_became_peered":"2024-09-19T04:44:30.757127+0000","last_unstale":"2024-09-19T04:44:32.753494+0000","last_undegraded":"2024-09-19T04:44:32.753494+0000","last_fullsized":"2024-09-19T04:44:32.753494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:59.325865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.763759+0000","last_change":"2024-09-19T04:44:30.754685+0000","last_active":"2024-09-19T04:44:32.763759+0000","last_peered":"2024-09-19T04:44:32.763759+0000","last_clean":"2024-09-19T04:44:32.763759+0000","last_became_active":"2024-09-19T04:44:30.753625+0000","last_became_peered":"2024-09-19T04:44:30.753625+0000","last_unstale":"2024-09-19T04:44:32.763759+0000","last_undegraded":"2024-09-19T04:44:32.763759+0000","last_fullsized":"2024-09-19T04:44:32.763759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:46:01.164693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.758520+0000","last_change":"2024-09-19T04:44:30.745823+0000","last_active":"2024-09-19T04:44:32.758520+0000","last_peered":"2024-09-19T04:44:32.758520+0000","last_clean":"2024-09-19T04:44:32.758520+0000","last_became_active":"2024-09-19T04:44:30.745224+0000","last_became_peered":"2024-09-19T04:44:30.745224+0000","last_unstale":"2024-09-19T04:44:32.758520+0000","last_undegraded":"2024-09-19T04:44:32.758520+0000","last_fullsized":"2024-09-19T04:44:32.758520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:23.682118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.755558+0000","last_change":"2024-09-19T04:44:30.752138+0000","last_active":"2024-09-19T04:44:32.755558+0000","last_peered":"2024-09-19T04:44:32.755558+0000","last_clean":"2024-09-19T04:44:32.755558+0000","last_became_active":"2024-09-19T04:44:30.751737+0000","last_became_peered":"2024-09-19T04:44:30.751737+0000","last_unstale":"2024-09-19T04:44:32.755558+0000","last_undegraded":"2024-09-19T04:44:32.755558+0000","last_fullsized":"2024-09-19T04:44:32.755558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:08.204601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.763612+0000","last_change":"2024-09-19T04:44:30.748870+0000","last_active":"2024-09-19T04:44:32.763612+0000","last_peered":"2024-09-19T04:44:32.763612+0000","last_clean":"2024-09-19T04:44:32.763612+0000","last_became_active":"2024-09-19T04:44:30.748549+0000","last_became_peered":"2024-09-19T04:44:30.748549+0000","last_unstale":"2024-09-19T04:44:32.763612+0000","last_undegraded":"2024-09-19T04:44:32.763612+0000","last_fullsized":"2024-09-19T04:44:32.763612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:12.951564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.967611+0000","last_change":"2024-09-19T04:44:30.759937+0000","last_active":"2024-09-19T04:44:31.967611+0000","last_peered":"2024-09-19T04:44:31.967611+0000","last_clean":"2024-09-19T04:44:31.967611+0000","last_became_active":"2024-09-19T04:44:30.759659+0000","last_became_peered":"2024-09-19T04:44:30.759659+0000","last_unstale":"2024-09-19T04:44:31.967611+0000","last_undegraded":"2024-09-19T04:44:31.967611+0000","last_fullsized":"2024-09-19T04:44:31.967611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:07.608994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.753749+0000","last_change":"2024-09-19T04:44:30.756216+0000","last_active":"2024-09-19T04:44:32.753749+0000","last_peered":"2024-09-19T04:44:32.753749+0000","last_clean":"2024-09-19T04:44:32.753749+0000","last_became_active":"2024-09-19T04:44:30.755903+0000","last_became_peered":"2024-09-19T04:44:30.755903+0000","last_unstale":"2024-09-19T04:44:32.753749+0000","last_undegraded":"2024-09-19T04:44:32.753749+0000","last_fullsized":"2024-09-19T04:44:32.753749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:16.966305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.349064+0000","last_change":"2024-09-19T04:44:30.746297+0000","last_active":"2024-09-19T04:44:32.349064+0000","last_peered":"2024-09-19T04:44:32.349064+0000","last_clean":"2024-09-19T04:44:32.349064+0000","last_became_active":"2024-09-19T04:44:30.746013+0000","last_became_peered":"2024-09-19T04:44:30.746013+0000","last_unstale":"2024-09-19T04:44:32.349064+0000","last_undegraded":"2024-09-19T04:44:32.349064+0000","last_fullsized":"2024-09-19T04:44:32.349064+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:17.086180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.966963+0000","last_change":"2024-09-19T04:44:30.756317+0000","last_active":"2024-09-19T04:44:31.966963+0000","last_peered":"2024-09-19T04:44:31.966963+0000","last_clean":"2024-09-19T04:44:31.966963+0000","last_became_active":"2024-09-19T04:44:30.755998+0000","last_became_peered":"2024-09-19T04:44:30.755998+0000","last_unstale":"2024-09-19T04:44:31.966963+0000","last_undegraded":"2024-09-19T04:44:31.966963+0000","last_fullsized":"2024-09-19T04:44:31.966963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:06:06.725565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.349533+0000","last_change":"2024-09-19T04:44:30.753423+0000","last_active":"2024-09-19T04:44:32.349533+0000","last_peered":"2024-09-19T04:44:32.349533+0000","last_clean":"2024-09-19T04:44:32.349533+0000","last_became_active":"2024-09-19T04:44:30.752139+0000","last_became_peered":"2024-09-19T04:44:30.752139+0000","last_unstale":"2024-09-19T04:44:32.349533+0000","last_undegraded":"2024-09-19T04:44:32.349533+0000","last_fullsized":"2024-09-19T04:44:32.349533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:18.046893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.755643+0000","last_change":"2024-09-19T04:44:30.751114+0000","last_active":"2024-09-19T04:44:32.755643+0000","last_peered":"2024-09-19T04:44:32.755643+0000","last_clean":"2024-09-19T04:44:32.755643+0000","last_became_active":"2024-09-19T04:44:30.750784+0000","last_became_peered":"2024-09-19T04:44:30.750784+0000","last_unstale":"2024-09-19T04:44:32.755643+0000","last_undegraded":"2024-09-19T04:44:32.755643+0000","last_fullsized":"2024-09-19T04:44:32.755643+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:22.040253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.763630+0000","last_change":"2024-09-19T04:44:30.747191+0000","last_active":"2024-09-19T04:44:32.763630+0000","last_peered":"2024-09-19T04:44:32.763630+0000","last_clean":"2024-09-19T04:44:32.763630+0000","last_became_active":"2024-09-19T04:44:30.746830+0000","last_became_peered":"2024-09-19T04:44:30.746830+0000","last_unstale":"2024-09-19T04:44:32.763630+0000","last_undegraded":"2024-09-19T04:44:32.763630+0000","last_fullsized":"2024-09-19T04:44:32.763630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:24.634920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.967278+0000","last_change":"2024-09-19T04:44:30.755174+0000","last_active":"2024-09-19T04:44:31.967278+0000","last_peered":"2024-09-19T04:44:31.967278+0000","last_clean":"2024-09-19T04:44:31.967278+0000","last_became_active":"2024-09-19T04:44:30.754885+0000","last_became_peered":"2024-09-19T04:44:30.754885+0000","last_unstale":"2024-09-19T04:44:31.967278+0000","last_undegraded":"2024-09-19T04:44:31.967278+0000","last_fullsized":"2024-09-19T04:44:31.967278+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:11:32.428033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.757351+0000","last_change":"2024-09-19T04:44:30.764992+0000","last_active":"2024-09-19T04:44:32.757351+0000","last_peered":"2024-09-19T04:44:32.757351+0000","last_clean":"2024-09-19T04:44:32.757351+0000","last_became_active":"2024-09-19T04:44:30.764575+0000","last_became_peered":"2024-09-19T04:44:30.764575+0000","last_unstale":"2024-09-19T04:44:32.757351+0000","last_undegraded":"2024-09-19T04:44:32.757351+0000","last_fullsized":"2024-09-19T04:44:32.757351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:54.562710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.755504+0000","last_change":"2024-09-19T04:44:30.757343+0000","last_active":"2024-09-19T04:44:32.755504+0000","last_peered":"2024-09-19T04:44:32.755504+0000","last_clean":"2024-09-19T04:44:32.755504+0000","last_became_active":"2024-09-19T04:44:30.756981+0000","last_became_peered":"2024-09-19T04:44:30.756981+0000","last_unstale":"2024-09-19T04:44:32.755504+0000","last_undegraded":"2024-09-19T04:44:32.755504+0000","last_fullsized":"2024-09-19T04:44:32.755504+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:07.388384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.755339+0000","last_change":"2024-09-19T04:44:30.749358+0000","last_active":"2024-09-19T04:44:32.755339+0000","last_peered":"2024-09-19T04:44:32.755339+0000","last_clean":"2024-09-19T04:44:32.755339+0000","last_became_active":"2024-09-19T04:44:30.749028+0000","last_became_peered":"2024-09-19T04:44:30.749028+0000","last_unstale":"2024-09-19T04:44:32.755339+0000","last_undegraded":"2024-09-19T04:44:32.755339+0000","last_fullsized":"2024-09-19T04:44:32.755339+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:32.352823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.763660+0000","last_change":"2024-09-19T04:44:30.748408+0000","last_active":"2024-09-19T04:44:32.763660+0000","last_peered":"2024-09-19T04:44:32.763660+0000","last_clean":"2024-09-19T04:44:32.763660+0000","last_became_active":"2024-09-19T04:44:30.747960+0000","last_became_peered":"2024-09-19T04:44:30.747960+0000","last_unstale":"2024-09-19T04:44:32.763660+0000","last_undegraded":"2024-09-19T04:44:32.763660+0000","last_fullsized":"2024-09-19T04:44:32.763660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:51.043791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.755216+0000","last_change":"2024-09-19T04:44:30.747194+0000","last_active":"2024-09-19T04:44:32.755216+0000","last_peered":"2024-09-19T04:44:32.755216+0000","last_clean":"2024-09-19T04:44:32.755216+0000","last_became_active":"2024-09-19T04:44:30.746871+0000","last_became_peered":"2024-09-19T04:44:30.746871+0000","last_unstale":"2024-09-19T04:44:32.755216+0000","last_undegraded":"2024-09-19T04:44:32.755216+0000","last_fullsized":"2024-09-19T04:44:32.755216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:52.340341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:56.863918+0000","last_change":"2024-09-19T04:44:28.337746+0000","last_active":"2024-09-19T04:44:56.863918+0000","last_peered":"2024-09-19T04:44:56.863918+0000","last_clean":"2024-09-19T04:44:56.863918+0000","last_became_active":"2024-09-19T04:44:28.337410+0000","last_became_peered":"2024-09-19T04:44:28.337410+0000","last_unstale":"2024-09-19T04:44:56.863918+0000","last_undegraded":"2024-09-19T04:44:56.863918+0000","last_fullsized":"2024-09-19T04:44:56.863918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:41:27.436939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.966538+0000","last_change":"2024-09-19T04:44:30.755512+0000","last_active":"2024-09-19T04:44:31.966538+0000","last_peered":"2024-09-19T04:44:31.966538+0000","last_clean":"2024-09-19T04:44:31.966538+0000","last_became_active":"2024-09-19T04:44:30.755086+0000","last_became_peered":"2024-09-19T04:44:30.755086+0000","last_unstale":"2024-09-19T04:44:31.966538+0000","last_undegraded":"2024-09-19T04:44:31.966538+0000","last_fullsized":"2024-09-19T04:44:31.966538+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:33.870376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:42.831284+0000","last_change":"2024-09-19T04:44:28.334492+0000","last_active":"2024-09-19T04:44:42.831284+0000","last_peered":"2024-09-19T04:44:42.831284+0000","last_clean":"2024-09-19T04:44:42.831284+0000","last_became_active":"2024-09-19T04:44:28.334098+0000","last_became_peered":"2024-09-19T04:44:28.334098+0000","last_unstale":"2024-09-19T04:44:42.831284+0000","last_undegraded":"2024-09-19T04:44:42.831284+0000","last_fullsized":"2024-09-19T04:44:42.831284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:48.156590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:41.818139+0000","last_change":"2024-09-19T04:44:28.335593+0000","last_active":"2024-09-19T04:44:41.818139+0000","last_peered":"2024-09-19T04:44:41.818139+0000","last_clean":"2024-09-19T04:44:41.818139+0000","last_became_active":"2024-09-19T04:44:28.335060+0000","last_became_peered":"2024-09-19T04:44:28.335060+0000","last_unstale":"2024-09-19T04:44:41.818139+0000","last_undegraded":"2024-09-19T04:44:41.818139+0000","last_fullsized":"2024-09-19T04:44:41.818139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:27.859960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.349338+0000","last_change":"2024-09-19T04:44:30.748235+0000","last_active":"2024-09-19T04:44:32.349338+0000","last_peered":"2024-09-19T04:44:32.349338+0000","last_clean":"2024-09-19T04:44:32.349338+0000","last_became_active":"2024-09-19T04:44:30.747881+0000","last_became_peered":"2024-09-19T04:44:30.747881+0000","last_unstale":"2024-09-19T04:44:32.349338+0000","last_undegraded":"2024-09-19T04:44:32.349338+0000","last_fullsized":"2024-09-19T04:44:32.349338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:40:39.183897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.839216+0000","last_change":"2024-09-19T04:44:28.337762+0000","last_active":"2024-09-19T04:44:43.839216+0000","last_peered":"2024-09-19T04:44:43.839216+0000","last_clean":"2024-09-19T04:44:43.839216+0000","last_became_active":"2024-09-19T04:44:28.337383+0000","last_became_peered":"2024-09-19T04:44:28.337383+0000","last_unstale":"2024-09-19T04:44:43.839216+0000","last_undegraded":"2024-09-19T04:44:43.839216+0000","last_fullsized":"2024-09-19T04:44:43.839216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:07:55.268958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.757712+0000","last_change":"2024-09-19T04:44:30.754318+0000","last_active":"2024-09-19T04:44:32.757712+0000","last_peered":"2024-09-19T04:44:32.757712+0000","last_clean":"2024-09-19T04:44:32.757712+0000","last_became_active":"2024-09-19T04:44:30.754025+0000","last_became_peered":"2024-09-19T04:44:30.754025+0000","last_unstale":"2024-09-19T04:44:32.757712+0000","last_undegraded":"2024-09-19T04:44:32.757712+0000","last_fullsized":"2024-09-19T04:44:32.757712+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:58.190504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:56.862941+0000","last_change":"2024-09-19T04:44:28.335947+0000","last_active":"2024-09-19T04:44:56.862941+0000","last_peered":"2024-09-19T04:44:56.862941+0000","last_clean":"2024-09-19T04:44:56.862941+0000","last_became_active":"2024-09-19T04:44:28.335588+0000","last_became_peered":"2024-09-19T04:44:28.335588+0000","last_unstale":"2024-09-19T04:44:56.862941+0000","last_undegraded":"2024-09-19T04:44:56.862941+0000","last_fullsized":"2024-09-19T04:44:56.862941+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:44.024915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.754044+0000","last_change":"2024-09-19T04:44:30.748272+0000","last_active":"2024-09-19T04:44:32.754044+0000","last_peered":"2024-09-19T04:44:32.754044+0000","last_clean":"2024-09-19T04:44:32.754044+0000","last_became_active":"2024-09-19T04:44:30.747147+0000","last_became_peered":"2024-09-19T04:44:30.747147+0000","last_unstale":"2024-09-19T04:44:32.754044+0000","last_undegraded":"2024-09-19T04:44:32.754044+0000","last_fullsized":"2024-09-19T04:44:32.754044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:11.653693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.837670+0000","last_change":"2024-09-19T04:44:28.337120+0000","last_active":"2024-09-19T04:44:43.837670+0000","last_peered":"2024-09-19T04:44:43.837670+0000","last_clean":"2024-09-19T04:44:43.837670+0000","last_became_active":"2024-09-19T04:44:28.336053+0000","last_became_peered":"2024-09-19T04:44:28.336053+0000","last_unstale":"2024-09-19T04:44:43.837670+0000","last_undegraded":"2024-09-19T04:44:43.837670+0000","last_fullsized":"2024-09-19T04:44:43.837670+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:56:57.514931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.757478+0000","last_change":"2024-09-19T04:44:30.755238+0000","last_active":"2024-09-19T04:44:32.757478+0000","last_peered":"2024-09-19T04:44:32.757478+0000","last_clean":"2024-09-19T04:44:32.757478+0000","last_became_active":"2024-09-19T04:44:30.754748+0000","last_became_peered":"2024-09-19T04:44:30.754748+0000","last_unstale":"2024-09-19T04:44:32.757478+0000","last_undegraded":"2024-09-19T04:44:32.757478+0000","last_fullsized":"2024-09-19T04:44:32.757478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:38.954491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:42.828735+0000","last_change":"2024-09-19T04:44:28.596147+0000","last_active":"2024-09-19T04:44:42.828735+0000","last_peered":"2024-09-19T04:44:42.828735+0000","last_clean":"2024-09-19T04:44:42.828735+0000","last_became_active":"2024-09-19T04:44:28.595785+0000","last_became_peered":"2024-09-19T04:44:28.595785+0000","last_unstale":"2024-09-19T04:44:42.828735+0000","last_undegraded":"2024-09-19T04:44:42.828735+0000","last_fullsized":"2024-09-19T04:44:42.828735+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:39.138724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.754127+0000","last_change":"2024-09-19T04:44:30.753817+0000","last_active":"2024-09-19T04:44:32.754127+0000","last_peered":"2024-09-19T04:44:32.754127+0000","last_clean":"2024-09-19T04:44:32.754127+0000","last_became_active":"2024-09-19T04:44:30.752541+0000","last_became_peered":"2024-09-19T04:44:30.752541+0000","last_unstale":"2024-09-19T04:44:32.754127+0000","last_undegraded":"2024-09-19T04:44:32.754127+0000","last_fullsized":"2024-09-19T04:44:32.754127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:16.121822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'27","reported_seq":79,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:56.863320+0000","last_change":"2024-09-19T04:44:28.335449+0000","last_active":"2024-09-19T04:44:56.863320+0000","last_peered":"2024-09-19T04:44:56.863320+0000","last_clean":"2024-09-19T04:44:56.863320+0000","last_became_active":"2024-09-19T04:44:28.335080+0000","last_became_peered":"2024-09-19T04:44:28.335080+0000","last_unstale":"2024-09-19T04:44:56.863320+0000","last_undegraded":"2024-09-19T04:44:56.863320+0000","last_fullsized":"2024-09-19T04:44:56.863320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:28.436290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50112,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":33,"num_read_kb":64,"num_write":29,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.757132+0000","last_change":"2024-09-19T04:44:30.749624+0000","last_active":"2024-09-19T04:44:32.757132+0000","last_peered":"2024-09-19T04:44:32.757132+0000","last_clean":"2024-09-19T04:44:32.757132+0000","last_became_active":"2024-09-19T04:44:30.749295+0000","last_became_peered":"2024-09-19T04:44:30.749295+0000","last_unstale":"2024-09-19T04:44:32.757132+0000","last_undegraded":"2024-09-19T04:44:32.757132+0000","last_fullsized":"2024-09-19T04:44:32.757132+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:48:46.942237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:44.843058+0000","last_change":"2024-09-19T04:44:28.337863+0000","last_active":"2024-09-19T04:44:44.843058+0000","last_peered":"2024-09-19T04:44:44.843058+0000","last_clean":"2024-09-19T04:44:44.843058+0000","last_became_active":"2024-09-19T04:44:28.337522+0000","last_became_peered":"2024-09-19T04:44:28.337522+0000","last_unstale":"2024-09-19T04:44:44.843058+0000","last_undegraded":"2024-09-19T04:44:44.843058+0000","last_fullsized":"2024-09-19T04:44:44.843058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:02.488622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.966914+0000","last_change":"2024-09-19T04:44:30.759916+0000","last_active":"2024-09-19T04:44:31.966914+0000","last_peered":"2024-09-19T04:44:31.966914+0000","last_clean":"2024-09-19T04:44:31.966914+0000","last_became_active":"2024-09-19T04:44:30.758835+0000","last_became_peered":"2024-09-19T04:44:30.758835+0000","last_unstale":"2024-09-19T04:44:31.966914+0000","last_undegraded":"2024-09-19T04:44:31.966914+0000","last_fullsized":"2024-09-19T04:44:31.966914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:38.614343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.837422+0000","last_change":"2024-09-19T04:44:28.337574+0000","last_active":"2024-09-19T04:44:43.837422+0000","last_peered":"2024-09-19T04:44:43.837422+0000","last_clean":"2024-09-19T04:44:43.837422+0000","last_became_active":"2024-09-19T04:44:28.337224+0000","last_became_peered":"2024-09-19T04:44:28.337224+0000","last_unstale":"2024-09-19T04:44:43.837422+0000","last_undegraded":"2024-09-19T04:44:43.837422+0000","last_fullsized":"2024-09-19T04:44:43.837422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:20.538181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.307164+0000","last_change":"2024-09-19T04:44:30.758152+0000","last_active":"2024-09-19T04:44:32.307164+0000","last_peered":"2024-09-19T04:44:32.307164+0000","last_clean":"2024-09-19T04:44:32.307164+0000","last_became_active":"2024-09-19T04:44:30.757801+0000","last_became_peered":"2024-09-19T04:44:30.757801+0000","last_unstale":"2024-09-19T04:44:32.307164+0000","last_undegraded":"2024-09-19T04:44:32.307164+0000","last_fullsized":"2024-09-19T04:44:32.307164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:05.212471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:55.863188+0000","last_change":"2024-09-19T04:44:28.339340+0000","last_active":"2024-09-19T04:44:55.863188+0000","last_peered":"2024-09-19T04:44:55.863188+0000","last_clean":"2024-09-19T04:44:55.863188+0000","last_became_active":"2024-09-19T04:44:28.339019+0000","last_became_peered":"2024-09-19T04:44:28.339019+0000","last_unstale":"2024-09-19T04:44:55.863188+0000","last_undegraded":"2024-09-19T04:44:55.863188+0000","last_fullsized":"2024-09-19T04:44:55.863188+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:29:59.356748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.306894+0000","last_change":"2024-09-19T04:44:30.751217+0000","last_active":"2024-09-19T04:44:32.306894+0000","last_peered":"2024-09-19T04:44:32.306894+0000","last_clean":"2024-09-19T04:44:32.306894+0000","last_became_active":"2024-09-19T04:44:30.750863+0000","last_became_peered":"2024-09-19T04:44:30.750863+0000","last_unstale":"2024-09-19T04:44:32.306894+0000","last_undegraded":"2024-09-19T04:44:32.306894+0000","last_fullsized":"2024-09-19T04:44:32.306894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:09.957572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.838396+0000","last_change":"2024-09-19T04:44:28.596585+0000","last_active":"2024-09-19T04:44:43.838396+0000","last_peered":"2024-09-19T04:44:43.838396+0000","last_clean":"2024-09-19T04:44:43.838396+0000","last_became_active":"2024-09-19T04:44:28.596218+0000","last_became_peered":"2024-09-19T04:44:28.596218+0000","last_unstale":"2024-09-19T04:44:43.838396+0000","last_undegraded":"2024-09-19T04:44:43.838396+0000","last_fullsized":"2024-09-19T04:44:43.838396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:29:35.395610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.966362+0000","last_change":"2024-09-19T04:44:30.746590+0000","last_active":"2024-09-19T04:44:31.966362+0000","last_peered":"2024-09-19T04:44:31.966362+0000","last_clean":"2024-09-19T04:44:31.966362+0000","last_became_active":"2024-09-19T04:44:30.746263+0000","last_became_peered":"2024-09-19T04:44:30.746263+0000","last_unstale":"2024-09-19T04:44:31.966362+0000","last_undegraded":"2024-09-19T04:44:31.966362+0000","last_fullsized":"2024-09-19T04:44:31.966362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:38.955613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.811752+0000","last_change":"2024-09-19T04:44:28.335826+0000","last_active":"2024-09-19T04:44:43.811752+0000","last_peered":"2024-09-19T04:44:43.811752+0000","last_clean":"2024-09-19T04:44:43.811752+0000","last_became_active":"2024-09-19T04:44:28.335481+0000","last_became_peered":"2024-09-19T04:44:28.335481+0000","last_unstale":"2024-09-19T04:44:43.811752+0000","last_undegraded":"2024-09-19T04:44:43.811752+0000","last_fullsized":"2024-09-19T04:44:43.811752+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:20.989500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.763607+0000","last_change":"2024-09-19T04:44:30.744130+0000","last_active":"2024-09-19T04:44:32.763607+0000","last_peered":"2024-09-19T04:44:32.763607+0000","last_clean":"2024-09-19T04:44:32.763607+0000","last_became_active":"2024-09-19T04:44:30.743772+0000","last_became_peered":"2024-09-19T04:44:30.743772+0000","last_unstale":"2024-09-19T04:44:32.763607+0000","last_undegraded":"2024-09-19T04:44:32.763607+0000","last_fullsized":"2024-09-19T04:44:32.763607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:34.956670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.837451+0000","last_change":"2024-09-19T04:44:28.337543+0000","last_active":"2024-09-19T04:44:43.837451+0000","last_peered":"2024-09-19T04:44:43.837451+0000","last_clean":"2024-09-19T04:44:43.837451+0000","last_became_active":"2024-09-19T04:44:28.336128+0000","last_became_peered":"2024-09-19T04:44:28.336128+0000","last_unstale":"2024-09-19T04:44:43.837451+0000","last_undegraded":"2024-09-19T04:44:43.837451+0000","last_fullsized":"2024-09-19T04:44:43.837451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:31.907777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.307438+0000","last_change":"2024-09-19T04:44:30.757489+0000","last_active":"2024-09-19T04:44:32.307438+0000","last_peered":"2024-09-19T04:44:32.307438+0000","last_clean":"2024-09-19T04:44:32.307438+0000","last_became_active":"2024-09-19T04:44:30.757041+0000","last_became_peered":"2024-09-19T04:44:30.757041+0000","last_unstale":"2024-09-19T04:44:32.307438+0000","last_undegraded":"2024-09-19T04:44:32.307438+0000","last_fullsized":"2024-09-19T04:44:32.307438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:03.395913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.837733+0000","last_change":"2024-09-19T04:44:28.327393+0000","last_active":"2024-09-19T04:44:43.837733+0000","last_peered":"2024-09-19T04:44:43.837733+0000","last_clean":"2024-09-19T04:44:43.837733+0000","last_became_active":"2024-09-19T04:44:28.327116+0000","last_became_peered":"2024-09-19T04:44:28.327116+0000","last_unstale":"2024-09-19T04:44:43.837733+0000","last_undegraded":"2024-09-19T04:44:43.837733+0000","last_fullsized":"2024-09-19T04:44:43.837733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:42.360980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.492611+0000","last_change":"2024-09-19T04:44:30.754239+0000","last_active":"2024-09-19T04:44:32.492611+0000","last_peered":"2024-09-19T04:44:32.492611+0000","last_clean":"2024-09-19T04:44:32.492611+0000","last_became_active":"2024-09-19T04:44:30.753885+0000","last_became_peered":"2024-09-19T04:44:30.753885+0000","last_unstale":"2024-09-19T04:44:32.492611+0000","last_undegraded":"2024-09-19T04:44:32.492611+0000","last_fullsized":"2024-09-19T04:44:32.492611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:27.355600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'5","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:55.862076+0000","last_change":"2024-09-19T04:44:28.596214+0000","last_active":"2024-09-19T04:44:55.862076+0000","last_peered":"2024-09-19T04:44:55.862076+0000","last_clean":"2024-09-19T04:44:55.862076+0000","last_became_active":"2024-09-19T04:44:28.595841+0000","last_became_peered":"2024-09-19T04:44:28.595841+0000","last_unstale":"2024-09-19T04:44:55.862076+0000","last_undegraded":"2024-09-19T04:44:55.862076+0000","last_fullsized":"2024-09-19T04:44:55.862076+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:43.864702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.307461+0000","last_change":"2024-09-19T04:44:30.756192+0000","last_active":"2024-09-19T04:44:32.307461+0000","last_peered":"2024-09-19T04:44:32.307461+0000","last_clean":"2024-09-19T04:44:32.307461+0000","last_became_active":"2024-09-19T04:44:30.755853+0000","last_became_peered":"2024-09-19T04:44:30.755853+0000","last_unstale":"2024-09-19T04:44:32.307461+0000","last_undegraded":"2024-09-19T04:44:32.307461+0000","last_fullsized":"2024-09-19T04:44:32.307461+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:25:49.206978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":345,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.306983+0000","last_change":"2024-09-19T04:43:10.156816+0000","last_active":"2024-09-19T04:44:32.306983+0000","last_peered":"2024-09-19T04:44:32.306983+0000","last_clean":"2024-09-19T04:44:32.306983+0000","last_became_active":"2024-09-19T04:43:10.156359+0000","last_became_peered":"2024-09-19T04:43:10.156359+0000","last_unstale":"2024-09-19T04:44:32.306983+0000","last_undegraded":"2024-09-19T04:44:32.306983+0000","last_fullsized":"2024-09-19T04:44:32.306983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:39:34.200793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:39:34.200793+0000","last_clean_scrub_stamp":"2024-09-19T04:39:34.200793+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:13.453818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:44.845247+0000","last_change":"2024-09-19T04:44:28.337637+0000","last_active":"2024-09-19T04:44:44.845247+0000","last_peered":"2024-09-19T04:44:44.845247+0000","last_clean":"2024-09-19T04:44:44.845247+0000","last_became_active":"2024-09-19T04:44:28.337235+0000","last_became_peered":"2024-09-19T04:44:28.337235+0000","last_unstale":"2024-09-19T04:44:44.845247+0000","last_undegraded":"2024-09-19T04:44:44.845247+0000","last_fullsized":"2024-09-19T04:44:44.845247+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:42.104972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.349528+0000","last_change":"2024-09-19T04:44:30.754697+0000","last_active":"2024-09-19T04:44:32.349528+0000","last_peered":"2024-09-19T04:44:32.349528+0000","last_clean":"2024-09-19T04:44:32.349528+0000","last_became_active":"2024-09-19T04:44:30.753214+0000","last_became_peered":"2024-09-19T04:44:30.753214+0000","last_unstale":"2024-09-19T04:44:32.349528+0000","last_undegraded":"2024-09-19T04:44:32.349528+0000","last_fullsized":"2024-09-19T04:44:32.349528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:37.455470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:41.816042+0000","last_change":"2024-09-19T04:44:28.336124+0000","last_active":"2024-09-19T04:44:41.816042+0000","last_peered":"2024-09-19T04:44:41.816042+0000","last_clean":"2024-09-19T04:44:41.816042+0000","last_became_active":"2024-09-19T04:44:28.335754+0000","last_became_peered":"2024-09-19T04:44:28.335754+0000","last_unstale":"2024-09-19T04:44:41.816042+0000","last_undegraded":"2024-09-19T04:44:41.816042+0000","last_fullsized":"2024-09-19T04:44:41.816042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:41.604367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.966871+0000","last_change":"2024-09-19T04:44:30.749939+0000","last_active":"2024-09-19T04:44:31.966871+0000","last_peered":"2024-09-19T04:44:31.966871+0000","last_clean":"2024-09-19T04:44:31.966871+0000","last_became_active":"2024-09-19T04:44:30.747122+0000","last_became_peered":"2024-09-19T04:44:30.747122+0000","last_unstale":"2024-09-19T04:44:31.966871+0000","last_undegraded":"2024-09-19T04:44:31.966871+0000","last_fullsized":"2024-09-19T04:44:31.966871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:46:08.371493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:54.861625+0000","last_change":"2024-09-19T04:44:28.594768+0000","last_active":"2024-09-19T04:44:54.861625+0000","last_peered":"2024-09-19T04:44:54.861625+0000","last_clean":"2024-09-19T04:44:54.861625+0000","last_became_active":"2024-09-19T04:44:28.594320+0000","last_became_peered":"2024-09-19T04:44:28.594320+0000","last_unstale":"2024-09-19T04:44:54.861625+0000","last_undegraded":"2024-09-19T04:44:54.861625+0000","last_fullsized":"2024-09-19T04:44:54.861625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:32:17.736874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.967034+0000","last_change":"2024-09-19T04:44:30.746666+0000","last_active":"2024-09-19T04:44:31.967034+0000","last_peered":"2024-09-19T04:44:31.967034+0000","last_clean":"2024-09-19T04:44:31.967034+0000","last_became_active":"2024-09-19T04:44:30.746312+0000","last_became_peered":"2024-09-19T04:44:30.746312+0000","last_unstale":"2024-09-19T04:44:31.967034+0000","last_undegraded":"2024-09-19T04:44:31.967034+0000","last_fullsized":"2024-09-19T04:44:31.967034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:13.562810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'5","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:55.862089+0000","last_change":"2024-09-19T04:44:28.336492+0000","last_active":"2024-09-19T04:44:55.862089+0000","last_peered":"2024-09-19T04:44:55.862089+0000","last_clean":"2024-09-19T04:44:55.862089+0000","last_became_active":"2024-09-19T04:44:28.336044+0000","last_became_peered":"2024-09-19T04:44:28.336044+0000","last_unstale":"2024-09-19T04:44:55.862089+0000","last_undegraded":"2024-09-19T04:44:55.862089+0000","last_fullsized":"2024-09-19T04:44:55.862089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:18.760473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.966467+0000","last_change":"2024-09-19T04:44:30.756340+0000","last_active":"2024-09-19T04:44:31.966467+0000","last_peered":"2024-09-19T04:44:31.966467+0000","last_clean":"2024-09-19T04:44:31.966467+0000","last_became_active":"2024-09-19T04:44:30.755989+0000","last_became_peered":"2024-09-19T04:44:30.755989+0000","last_unstale":"2024-09-19T04:44:31.966467+0000","last_undegraded":"2024-09-19T04:44:31.966467+0000","last_fullsized":"2024-09-19T04:44:31.966467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:13:11.792019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.758057+0000","last_change":"2024-09-19T04:44:28.596322+0000","last_active":"2024-09-19T04:44:32.758057+0000","last_peered":"2024-09-19T04:44:32.758057+0000","last_clean":"2024-09-19T04:44:32.758057+0000","last_became_active":"2024-09-19T04:44:28.595566+0000","last_became_peered":"2024-09-19T04:44:28.595566+0000","last_unstale":"2024-09-19T04:44:32.758057+0000","last_undegraded":"2024-09-19T04:44:32.758057+0000","last_fullsized":"2024-09-19T04:44:32.758057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:23.878145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.366685+0000","last_change":"2024-09-19T04:44:30.752321+0000","last_active":"2024-09-19T04:44:32.366685+0000","last_peered":"2024-09-19T04:44:32.366685+0000","last_clean":"2024-09-19T04:44:32.366685+0000","last_became_active":"2024-09-19T04:44:30.751980+0000","last_became_peered":"2024-09-19T04:44:30.751980+0000","last_unstale":"2024-09-19T04:44:32.366685+0000","last_undegraded":"2024-09-19T04:44:32.366685+0000","last_fullsized":"2024-09-19T04:44:32.366685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:41.863034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.838509+0000","last_change":"2024-09-19T04:44:28.596547+0000","last_active":"2024-09-19T04:44:43.838509+0000","last_peered":"2024-09-19T04:44:43.838509+0000","last_clean":"2024-09-19T04:44:43.838509+0000","last_became_active":"2024-09-19T04:44:28.596181+0000","last_became_peered":"2024-09-19T04:44:28.596181+0000","last_unstale":"2024-09-19T04:44:43.838509+0000","last_undegraded":"2024-09-19T04:44:43.838509+0000","last_fullsized":"2024-09-19T04:44:43.838509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:53.251737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.967409+0000","last_change":"2024-09-19T04:44:30.751533+0000","last_active":"2024-09-19T04:44:31.967409+0000","last_peered":"2024-09-19T04:44:31.967409+0000","last_clean":"2024-09-19T04:44:31.967409+0000","last_became_active":"2024-09-19T04:44:30.748012+0000","last_became_peered":"2024-09-19T04:44:30.748012+0000","last_unstale":"2024-09-19T04:44:31.967409+0000","last_undegraded":"2024-09-19T04:44:31.967409+0000","last_fullsized":"2024-09-19T04:44:31.967409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:32.360147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'12","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:54.755433+0000","last_change":"2024-09-19T04:44:28.335884+0000","last_active":"2024-09-19T04:44:54.755433+0000","last_peered":"2024-09-19T04:44:54.755433+0000","last_clean":"2024-09-19T04:44:54.755433+0000","last_became_active":"2024-09-19T04:44:28.335521+0000","last_became_peered":"2024-09-19T04:44:28.335521+0000","last_unstale":"2024-09-19T04:44:54.755433+0000","last_undegraded":"2024-09-19T04:44:54.755433+0000","last_fullsized":"2024-09-19T04:44:54.755433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:32:49.764893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.349718+0000","last_change":"2024-09-19T04:44:30.759578+0000","last_active":"2024-09-19T04:44:32.349718+0000","last_peered":"2024-09-19T04:44:32.349718+0000","last_clean":"2024-09-19T04:44:32.349718+0000","last_became_active":"2024-09-19T04:44:30.759245+0000","last_became_peered":"2024-09-19T04:44:30.759245+0000","last_unstale":"2024-09-19T04:44:32.349718+0000","last_undegraded":"2024-09-19T04:44:32.349718+0000","last_fullsized":"2024-09-19T04:44:32.349718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:37.121770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.839671+0000","last_change":"2024-09-19T04:44:28.593404+0000","last_active":"2024-09-19T04:44:43.839671+0000","last_peered":"2024-09-19T04:44:43.839671+0000","last_clean":"2024-09-19T04:44:43.839671+0000","last_became_active":"2024-09-19T04:44:28.593037+0000","last_became_peered":"2024-09-19T04:44:28.593037+0000","last_unstale":"2024-09-19T04:44:43.839671+0000","last_undegraded":"2024-09-19T04:44:43.839671+0000","last_fullsized":"2024-09-19T04:44:43.839671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:08:29.218543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.757281+0000","last_change":"2024-09-19T04:44:30.763656+0000","last_active":"2024-09-19T04:44:32.757281+0000","last_peered":"2024-09-19T04:44:32.757281+0000","last_clean":"2024-09-19T04:44:32.757281+0000","last_became_active":"2024-09-19T04:44:30.761653+0000","last_became_peered":"2024-09-19T04:44:30.761653+0000","last_unstale":"2024-09-19T04:44:32.757281+0000","last_undegraded":"2024-09-19T04:44:32.757281+0000","last_fullsized":"2024-09-19T04:44:32.757281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:22.077931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:40.808459+0000","last_change":"2024-09-19T04:44:28.594541+0000","last_active":"2024-09-19T04:44:40.808459+0000","last_peered":"2024-09-19T04:44:40.808459+0000","last_clean":"2024-09-19T04:44:40.808459+0000","last_became_active":"2024-09-19T04:44:28.594262+0000","last_became_peered":"2024-09-19T04:44:28.594262+0000","last_unstale":"2024-09-19T04:44:40.808459+0000","last_undegraded":"2024-09-19T04:44:40.808459+0000","last_fullsized":"2024-09-19T04:44:40.808459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:45.685034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.754243+0000","last_change":"2024-09-19T04:44:30.757861+0000","last_active":"2024-09-19T04:44:32.754243+0000","last_peered":"2024-09-19T04:44:32.754243+0000","last_clean":"2024-09-19T04:44:32.754243+0000","last_became_active":"2024-09-19T04:44:30.757509+0000","last_became_peered":"2024-09-19T04:44:30.757509+0000","last_unstale":"2024-09-19T04:44:32.754243+0000","last_undegraded":"2024-09-19T04:44:32.754243+0000","last_fullsized":"2024-09-19T04:44:32.754243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:17.737055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.837765+0000","last_change":"2024-09-19T04:44:28.335225+0000","last_active":"2024-09-19T04:44:43.837765+0000","last_peered":"2024-09-19T04:44:43.837765+0000","last_clean":"2024-09-19T04:44:43.837765+0000","last_became_active":"2024-09-19T04:44:28.334804+0000","last_became_peered":"2024-09-19T04:44:28.334804+0000","last_unstale":"2024-09-19T04:44:43.837765+0000","last_undegraded":"2024-09-19T04:44:43.837765+0000","last_fullsized":"2024-09-19T04:44:43.837765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:15.639418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.763615+0000","last_change":"2024-09-19T04:44:30.754018+0000","last_active":"2024-09-19T04:44:32.763615+0000","last_peered":"2024-09-19T04:44:32.763615+0000","last_clean":"2024-09-19T04:44:32.763615+0000","last_became_active":"2024-09-19T04:44:30.753639+0000","last_became_peered":"2024-09-19T04:44:30.753639+0000","last_unstale":"2024-09-19T04:44:32.763615+0000","last_undegraded":"2024-09-19T04:44:32.763615+0000","last_fullsized":"2024-09-19T04:44:32.763615+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:05.012847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:42.829548+0000","last_change":"2024-09-19T04:44:28.335884+0000","last_active":"2024-09-19T04:44:42.829548+0000","last_peered":"2024-09-19T04:44:42.829548+0000","last_clean":"2024-09-19T04:44:42.829548+0000","last_became_active":"2024-09-19T04:44:28.335501+0000","last_became_peered":"2024-09-19T04:44:28.335501+0000","last_unstale":"2024-09-19T04:44:42.829548+0000","last_undegraded":"2024-09-19T04:44:42.829548+0000","last_fullsized":"2024-09-19T04:44:42.829548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:23.541939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.349482+0000","last_change":"2024-09-19T04:44:30.758543+0000","last_active":"2024-09-19T04:44:32.349482+0000","last_peered":"2024-09-19T04:44:32.349482+0000","last_clean":"2024-09-19T04:44:32.349482+0000","last_became_active":"2024-09-19T04:44:30.757228+0000","last_became_peered":"2024-09-19T04:44:30.757228+0000","last_unstale":"2024-09-19T04:44:32.349482+0000","last_undegraded":"2024-09-19T04:44:32.349482+0000","last_fullsized":"2024-09-19T04:44:32.349482+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:13.877742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:40.808803+0000","last_change":"2024-09-19T04:44:28.339152+0000","last_active":"2024-09-19T04:44:40.808803+0000","last_peered":"2024-09-19T04:44:40.808803+0000","last_clean":"2024-09-19T04:44:40.808803+0000","last_became_active":"2024-09-19T04:44:28.338778+0000","last_became_peered":"2024-09-19T04:44:28.338778+0000","last_unstale":"2024-09-19T04:44:40.808803+0000","last_undegraded":"2024-09-19T04:44:40.808803+0000","last_fullsized":"2024-09-19T04:44:40.808803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:53.906755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.307146+0000","last_change":"2024-09-19T04:44:30.759330+0000","last_active":"2024-09-19T04:44:32.307146+0000","last_peered":"2024-09-19T04:44:32.307146+0000","last_clean":"2024-09-19T04:44:32.307146+0000","last_became_active":"2024-09-19T04:44:30.758987+0000","last_became_peered":"2024-09-19T04:44:30.758987+0000","last_unstale":"2024-09-19T04:44:32.307146+0000","last_undegraded":"2024-09-19T04:44:32.307146+0000","last_fullsized":"2024-09-19T04:44:32.307146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:10.105051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.966540+0000","last_change":"2024-09-19T04:44:30.748089+0000","last_active":"2024-09-19T04:44:31.966540+0000","last_peered":"2024-09-19T04:44:31.966540+0000","last_clean":"2024-09-19T04:44:31.966540+0000","last_became_active":"2024-09-19T04:44:30.747566+0000","last_became_peered":"2024-09-19T04:44:30.747566+0000","last_unstale":"2024-09-19T04:44:31.966540+0000","last_undegraded":"2024-09-19T04:44:31.966540+0000","last_fullsized":"2024-09-19T04:44:31.966540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:32.406745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.839563+0000","last_change":"2024-09-19T04:44:28.334217+0000","last_active":"2024-09-19T04:44:43.839563+0000","last_peered":"2024-09-19T04:44:43.839563+0000","last_clean":"2024-09-19T04:44:43.839563+0000","last_became_active":"2024-09-19T04:44:28.333768+0000","last_became_peered":"2024-09-19T04:44:28.333768+0000","last_unstale":"2024-09-19T04:44:43.839563+0000","last_undegraded":"2024-09-19T04:44:43.839563+0000","last_fullsized":"2024-09-19T04:44:43.839563+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:52:56.626635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.754179+0000","last_change":"2024-09-19T04:44:30.747498+0000","last_active":"2024-09-19T04:44:32.754179+0000","last_peered":"2024-09-19T04:44:32.754179+0000","last_clean":"2024-09-19T04:44:32.754179+0000","last_became_active":"2024-09-19T04:44:30.746822+0000","last_became_peered":"2024-09-19T04:44:30.746822+0000","last_unstale":"2024-09-19T04:44:32.754179+0000","last_undegraded":"2024-09-19T04:44:32.754179+0000","last_fullsized":"2024-09-19T04:44:32.754179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:09.460263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:44.842877+0000","last_change":"2024-09-19T04:44:28.336755+0000","last_active":"2024-09-19T04:44:44.842877+0000","last_peered":"2024-09-19T04:44:44.842877+0000","last_clean":"2024-09-19T04:44:44.842877+0000","last_became_active":"2024-09-19T04:44:28.336405+0000","last_became_peered":"2024-09-19T04:44:28.336405+0000","last_unstale":"2024-09-19T04:44:44.842877+0000","last_undegraded":"2024-09-19T04:44:44.842877+0000","last_fullsized":"2024-09-19T04:44:44.842877+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:58:13.725792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.757300+0000","last_change":"2024-09-19T04:44:30.762776+0000","last_active":"2024-09-19T04:44:32.757300+0000","last_peered":"2024-09-19T04:44:32.757300+0000","last_clean":"2024-09-19T04:44:32.757300+0000","last_became_active":"2024-09-19T04:44:30.761308+0000","last_became_peered":"2024-09-19T04:44:30.761308+0000","last_unstale":"2024-09-19T04:44:32.757300+0000","last_undegraded":"2024-09-19T04:44:32.757300+0000","last_fullsized":"2024-09-19T04:44:32.757300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:19.960125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.755814+0000","last_change":"2024-09-19T04:44:28.594858+0000","last_active":"2024-09-19T04:44:32.755814+0000","last_peered":"2024-09-19T04:44:32.755814+0000","last_clean":"2024-09-19T04:44:32.755814+0000","last_became_active":"2024-09-19T04:44:28.594515+0000","last_became_peered":"2024-09-19T04:44:28.594515+0000","last_unstale":"2024-09-19T04:44:32.755814+0000","last_undegraded":"2024-09-19T04:44:32.755814+0000","last_fullsized":"2024-09-19T04:44:32.755814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:53.728393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.349385+0000","last_change":"2024-09-19T04:44:30.753604+0000","last_active":"2024-09-19T04:44:32.349385+0000","last_peered":"2024-09-19T04:44:32.349385+0000","last_clean":"2024-09-19T04:44:32.349385+0000","last_became_active":"2024-09-19T04:44:30.751961+0000","last_became_peered":"2024-09-19T04:44:30.751961+0000","last_unstale":"2024-09-19T04:44:32.349385+0000","last_undegraded":"2024-09-19T04:44:32.349385+0000","last_fullsized":"2024-09-19T04:44:32.349385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:46:58.759650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:55.861648+0000","last_change":"2024-09-19T04:44:28.339499+0000","last_active":"2024-09-19T04:44:55.861648+0000","last_peered":"2024-09-19T04:44:55.861648+0000","last_clean":"2024-09-19T04:44:55.861648+0000","last_became_active":"2024-09-19T04:44:28.338774+0000","last_became_peered":"2024-09-19T04:44:28.338774+0000","last_unstale":"2024-09-19T04:44:55.861648+0000","last_undegraded":"2024-09-19T04:44:55.861648+0000","last_fullsized":"2024-09-19T04:44:55.861648+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:12.261237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56927,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":168,"num_read_kb":146,"num_write":233,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":170781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247103,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481811,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":438830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710744,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":1080364,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874269,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":438760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":439138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364839,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1031658,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528366,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":438477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757299,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":1030943,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953527,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":438731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":438302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":489722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":489502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":439816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34357,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34357,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34357,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":51864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":52241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:45:01.476 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:45:01.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:01 smithi193 ceph-mon[35592]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:45:01.846 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:01.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:01 smithi183 ceph-mon[35333]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:45:01.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:01 smithi123 ceph-mon[30511]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:45:02.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.434+0000 7f8853c00640 1 -- 172.21.15.123:0/2905305921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c10d530 msgr2=0x7f884c1150b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:02.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.434+0000 7f8853c00640 1 --2- 172.21.15.123:0/2905305921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c10d530 0x7f884c1150b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8844009ca0 tx=0x7f884402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:45:02.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.434+0000 7f8853c00640 1 -- 172.21.15.123:0/2905305921 shutdown_connections 2024-09-19T04:45:02.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.434+0000 7f8853c00640 1 --2- 172.21.15.123:0/2905305921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c10d530 0x7f884c1150b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:02.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.434+0000 7f8853c00640 1 --2- 172.21.15.123:0/2905305921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f884c108e00 0x7f884c10ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:02.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.434+0000 7f8853c00640 1 --2- 172.21.15.123:0/2905305921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f884c108450 0x7f884c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:02.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.434+0000 7f8853c00640 1 -- 172.21.15.123:0/2905305921 >> 172.21.15.123:0/2905305921 conn(0x7f884c077ba0 msgr2=0x7f884c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:02.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.434+0000 7f8853c00640 1 -- 172.21.15.123:0/2905305921 shutdown_connections 2024-09-19T04:45:02.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.434+0000 7f8853c00640 1 -- 172.21.15.123:0/2905305921 wait complete. 2024-09-19T04:45:02.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.435+0000 7f8853c00640 1 Processor -- start 2024-09-19T04:45:02.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.435+0000 7f8853c00640 1 -- start start 2024-09-19T04:45:02.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.436+0000 7f8853c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c108450 0x7f884c1a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:02.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.436+0000 7f8853c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f884c108e00 0x7f884c1a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:02.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.436+0000 7f8853c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f884c10d530 0x7f884c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:02.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.436+0000 7f8853c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f884c118590 con 0x7f884c108450 2024-09-19T04:45:02.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.436+0000 7f8853c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f884c118410 con 0x7f884c108e00 2024-09-19T04:45:02.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.436+0000 7f8853c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f884c118710 con 0x7f884c10d530 2024-09-19T04:45:02.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.436+0000 7f8851200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c108450 0x7f884c1a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:02.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.436+0000 7f8851200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c108450 0x7f884c1a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55944/0 (socket says 172.21.15.123:55944) 2024-09-19T04:45:02.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.436+0000 7f8851200640 1 -- 172.21.15.123:0/2149914832 learned_addr learned my addr 172.21.15.123:0/2149914832 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:02.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.436+0000 7f884be00640 1 --2- 172.21.15.123:0/2149914832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f884c108e00 0x7f884c1a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:02.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.436+0000 7f8851c00640 1 --2- 172.21.15.123:0/2149914832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f884c10d530 0x7f884c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:02.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.436+0000 7f8851200640 1 -- 172.21.15.123:0/2149914832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f884c10d530 msgr2=0x7f884c19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:02.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.436+0000 7f8851200640 1 --2- 172.21.15.123:0/2149914832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f884c10d530 0x7f884c19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:02.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.436+0000 7f8851200640 1 -- 172.21.15.123:0/2149914832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f884c108e00 msgr2=0x7f884c1a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:02.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.436+0000 7f8851200640 1 --2- 172.21.15.123:0/2149914832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f884c108e00 0x7f884c1a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:02.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.436+0000 7f8851200640 1 -- 172.21.15.123:0/2149914832 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884c19ef80 con 0x7f884c108450 2024-09-19T04:45:02.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.437+0000 7f8851200640 1 --2- 172.21.15.123:0/2149914832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c108450 0x7f884c1a4660 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f884000e9f0 tx=0x7f884000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:02.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.437+0000 7f8849600640 1 -- 172.21.15.123:0/2149914832 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884000cd80 con 0x7f884c108450 2024-09-19T04:45:02.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.437+0000 7f8849600640 1 -- 172.21.15.123:0/2149914832 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8840010910 con 0x7f884c108450 2024-09-19T04:45:02.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.437+0000 7f8853c00640 1 -- 172.21.15.123:0/2149914832 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884c19f270 con 0x7f884c108450 2024-09-19T04:45:02.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.437+0000 7f8853c00640 1 -- 172.21.15.123:0/2149914832 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f884c1ab4c0 con 0x7f884c108450 2024-09-19T04:45:02.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.437+0000 7f8849600640 1 -- 172.21.15.123:0/2149914832 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8840018990 con 0x7f884c108450 2024-09-19T04:45:02.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.438+0000 7f8853c00640 1 -- 172.21.15.123:0/2149914832 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f884c1098e0 con 0x7f884c108450 2024-09-19T04:45:02.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.441+0000 7f8849600640 1 -- 172.21.15.123:0/2149914832 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8840010ac0 con 0x7f884c108450 2024-09-19T04:45:02.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.442+0000 7f8849600640 1 --2- 172.21.15.123:0/2149914832 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f88340a32d0 0x7f88340a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:02.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.442+0000 7f8849600640 1 -- 172.21.15.123:0/2149914832 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f88400cfce0 con 0x7f884c108450 2024-09-19T04:45:02.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.442+0000 7f884be00640 1 --2- 172.21.15.123:0/2149914832 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f88340a32d0 0x7f88340a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:02.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.443+0000 7f884be00640 1 --2- 172.21.15.123:0/2149914832 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f88340a32d0 0x7f88340a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f884c078410 tx=0x7f883800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:02.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.446+0000 7f8849600640 1 -- 172.21.15.123:0/2149914832 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f884009c300 con 0x7f884c108450 2024-09-19T04:45:02.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.623+0000 7f8853c00640 1 -- 172.21.15.123:0/2149914832 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f884c1107e0 con 0x7f88340a32d0 2024-09-19T04:45:02.625 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:02 smithi123 ceph-mon[30511]: pgmap v308: 97 pgs: 97 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T04:45:02.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.631+0000 7f8849600640 1 -- 172.21.15.123:0/2149914832 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258888 (secure 0 0 0) 0x7f8838010b40 con 0x7f88340a32d0 2024-09-19T04:45:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:45:02.635 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-09-19T04:45:02.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.636+0000 7f8853c00640 1 -- 172.21.15.123:0/2149914832 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f88340a32d0 msgr2=0x7f88340a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:02.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.636+0000 7f8853c00640 1 --2- 172.21.15.123:0/2149914832 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f88340a32d0 0x7f88340a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f884c078410 tx=0x7f883800b390 comp rx=0 tx=0).stop 2024-09-19T04:45:02.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.636+0000 7f8853c00640 1 -- 172.21.15.123:0/2149914832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c108450 msgr2=0x7f884c1a4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:02.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.636+0000 7f8853c00640 1 --2- 172.21.15.123:0/2149914832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c108450 0x7f884c1a4660 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f884000e9f0 tx=0x7f884000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:45:02.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.636+0000 7f8853c00640 1 -- 172.21.15.123:0/2149914832 shutdown_connections 2024-09-19T04:45:02.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.636+0000 7f8853c00640 1 --2- 172.21.15.123:0/2149914832 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f88340a32d0 0x7f88340a5790 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:02.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.636+0000 7f8853c00640 1 --2- 172.21.15.123:0/2149914832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f884c10d530 0x7f884c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:02.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.636+0000 7f8853c00640 1 --2- 172.21.15.123:0/2149914832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f884c108e00 0x7f884c1a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:02.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.636+0000 7f8853c00640 1 --2- 172.21.15.123:0/2149914832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c108450 0x7f884c1a4660 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:02.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.636+0000 7f8853c00640 1 -- 172.21.15.123:0/2149914832 >> 172.21.15.123:0/2149914832 conn(0x7f884c077ba0 msgr2=0x7f884c078e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:02.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.636+0000 7f8853c00640 1 -- 172.21.15.123:0/2149914832 shutdown_connections 2024-09-19T04:45:02.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:02.636+0000 7f8853c00640 1 -- 172.21.15.123:0/2149914832 wait complete. 2024-09-19T04:45:02.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:02 smithi193 ceph-mon[35592]: pgmap v308: 97 pgs: 97 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T04:45:02.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:02 smithi183 ceph-mon[35333]: pgmap v308: 97 pgs: 97 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T04:45:03.373 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-19T04:45:01.356515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647529,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":304,"num_read_kb":265,"num_write":416,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356392,"kb_used_data":11304,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585432,"statfs":{"total":1151940427776,"available":1151575482368,"internally_reserved":0,"allocated":11575296,"data_stored":7194243,"data_compressed":108024,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42132,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":26,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003270"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.763716+0000","last_change":"2024-09-19T04:44:30.754400+0000","last_active":"2024-09-19T04:44:32.763716+0000","last_peered":"2024-09-19T04:44:32.763716+0000","last_clean":"2024-09-19T04:44:32.763716+0000","last_became_active":"2024-09-19T04:44:30.754028+0000","last_became_peered":"2024-09-19T04:44:30.754028+0000","last_unstale":"2024-09-19T04:44:32.763716+0000","last_undegraded":"2024-09-19T04:44:32.763716+0000","last_fullsized":"2024-09-19T04:44:32.763716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:25.668478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.756971+0000","last_change":"2024-09-19T04:44:30.758072+0000","last_active":"2024-09-19T04:44:32.756971+0000","last_peered":"2024-09-19T04:44:32.756971+0000","last_clean":"2024-09-19T04:44:32.756971+0000","last_became_active":"2024-09-19T04:44:30.756734+0000","last_became_peered":"2024-09-19T04:44:30.756734+0000","last_unstale":"2024-09-19T04:44:32.756971+0000","last_undegraded":"2024-09-19T04:44:32.756971+0000","last_fullsized":"2024-09-19T04:44:32.756971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:56:47.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.757241+0000","last_change":"2024-09-19T04:44:30.760208+0000","last_active":"2024-09-19T04:44:32.757241+0000","last_peered":"2024-09-19T04:44:32.757241+0000","last_clean":"2024-09-19T04:44:32.757241+0000","last_became_active":"2024-09-19T04:44:30.759719+0000","last_became_peered":"2024-09-19T04:44:30.759719+0000","last_unstale":"2024-09-19T04:44:32.757241+0000","last_undegraded":"2024-09-19T04:44:32.757241+0000","last_fullsized":"2024-09-19T04:44:32.757241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:14.041853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.760379+0000","last_change":"2024-09-19T04:44:30.747813+0000","last_active":"2024-09-19T04:44:32.760379+0000","last_peered":"2024-09-19T04:44:32.760379+0000","last_clean":"2024-09-19T04:44:32.760379+0000","last_became_active":"2024-09-19T04:44:30.747486+0000","last_became_peered":"2024-09-19T04:44:30.747486+0000","last_unstale":"2024-09-19T04:44:32.760379+0000","last_undegraded":"2024-09-19T04:44:32.760379+0000","last_fullsized":"2024-09-19T04:44:32.760379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:11.315273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.754512+0000","last_change":"2024-09-19T04:44:30.755800+0000","last_active":"2024-09-19T04:44:32.754512+0000","last_peered":"2024-09-19T04:44:32.754512+0000","last_clean":"2024-09-19T04:44:32.754512+0000","last_became_active":"2024-09-19T04:44:30.755448+0000","last_became_peered":"2024-09-19T04:44:30.755448+0000","last_unstale":"2024-09-19T04:44:32.754512+0000","last_undegraded":"2024-09-19T04:44:32.754512+0000","last_fullsized":"2024-09-19T04:44:32.754512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:39.308614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.966128+0000","last_change":"2024-09-19T04:44:30.746230+0000","last_active":"2024-09-19T04:44:31.966128+0000","last_peered":"2024-09-19T04:44:31.966128+0000","last_clean":"2024-09-19T04:44:31.966128+0000","last_became_active":"2024-09-19T04:44:30.745884+0000","last_became_peered":"2024-09-19T04:44:30.745884+0000","last_unstale":"2024-09-19T04:44:31.966128+0000","last_undegraded":"2024-09-19T04:44:31.966128+0000","last_fullsized":"2024-09-19T04:44:31.966128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:51:58.550322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.306935+0000","last_change":"2024-09-19T04:44:30.747870+0000","last_active":"2024-09-19T04:44:32.306935+0000","last_peered":"2024-09-19T04:44:32.306935+0000","last_clean":"2024-09-19T04:44:32.306935+0000","last_became_active":"2024-09-19T04:44:30.747536+0000","last_became_peered":"2024-09-19T04:44:30.747536+0000","last_unstale":"2024-09-19T04:44:32.306935+0000","last_undegraded":"2024-09-19T04:44:32.306935+0000","last_fullsized":"2024-09-19T04:44:32.306935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:20.074032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.349053+0000","last_change":"2024-09-19T04:44:30.750425+0000","last_active":"2024-09-19T04:44:32.349053+0000","last_peered":"2024-09-19T04:44:32.349053+0000","last_clean":"2024-09-19T04:44:32.349053+0000","last_became_active":"2024-09-19T04:44:30.749841+0000","last_became_peered":"2024-09-19T04:44:30.749841+0000","last_unstale":"2024-09-19T04:44:32.349053+0000","last_undegraded":"2024-09-19T04:44:32.349053+0000","last_fullsized":"2024-09-19T04:44:32.349053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:12:08.420727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.754381+0000","last_change":"2024-09-19T04:44:30.750624+0000","last_active":"2024-09-19T04:44:32.754381+0000","last_peered":"2024-09-19T04:44:32.754381+0000","last_clean":"2024-09-19T04:44:32.754381+0000","last_became_active":"2024-09-19T04:44:30.748370+0000","last_became_peered":"2024-09-19T04:44:30.748370+0000","last_unstale":"2024-09-19T04:44:32.754381+0000","last_undegraded":"2024-09-19T04:44:32.754381+0000","last_fullsized":"2024-09-19T04:44:32.754381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:36.131634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.967177+0000","last_change":"2024-09-19T04:44:30.750035+0000","last_active":"2024-09-19T04:44:31.967177+0000","last_peered":"2024-09-19T04:44:31.967177+0000","last_clean":"2024-09-19T04:44:31.967177+0000","last_became_active":"2024-09-19T04:44:30.749699+0000","last_became_peered":"2024-09-19T04:44:30.749699+0000","last_unstale":"2024-09-19T04:44:31.967177+0000","last_undegraded":"2024-09-19T04:44:31.967177+0000","last_fullsized":"2024-09-19T04:44:31.967177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:22.637193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.307412+0000","last_change":"2024-09-19T04:44:30.760258+0000","last_active":"2024-09-19T04:44:32.307412+0000","last_peered":"2024-09-19T04:44:32.307412+0000","last_clean":"2024-09-19T04:44:32.307412+0000","last_became_active":"2024-09-19T04:44:30.759925+0000","last_became_peered":"2024-09-19T04:44:30.759925+0000","last_unstale":"2024-09-19T04:44:32.307412+0000","last_undegraded":"2024-09-19T04:44:32.307412+0000","last_fullsized":"2024-09-19T04:44:32.307412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:58.144691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.760614+0000","last_change":"2024-09-19T04:44:30.747703+0000","last_active":"2024-09-19T04:44:32.760614+0000","last_peered":"2024-09-19T04:44:32.760614+0000","last_clean":"2024-09-19T04:44:32.760614+0000","last_became_active":"2024-09-19T04:44:30.747369+0000","last_became_peered":"2024-09-19T04:44:30.747369+0000","last_unstale":"2024-09-19T04:44:32.760614+0000","last_undegraded":"2024-09-19T04:44:32.760614+0000","last_fullsized":"2024-09-19T04:44:32.760614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:06.920357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.967545+0000","last_change":"2024-09-19T04:44:30.748844+0000","last_active":"2024-09-19T04:44:31.967545+0000","last_peered":"2024-09-19T04:44:31.967545+0000","last_clean":"2024-09-19T04:44:31.967545+0000","last_became_active":"2024-09-19T04:44:30.748513+0000","last_became_peered":"2024-09-19T04:44:30.748513+0000","last_unstale":"2024-09-19T04:44:31.967545+0000","last_undegraded":"2024-09-19T04:44:31.967545+0000","last_fullsized":"2024-09-19T04:44:31.967545+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:45.107718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.756835+0000","last_change":"2024-09-19T04:44:30.750526+0000","last_active":"2024-09-19T04:44:32.756835+0000","last_peered":"2024-09-19T04:44:32.756835+0000","last_clean":"2024-09-19T04:44:32.756835+0000","last_became_active":"2024-09-19T04:44:30.749693+0000","last_became_peered":"2024-09-19T04:44:30.749693+0000","last_unstale":"2024-09-19T04:44:32.756835+0000","last_undegraded":"2024-09-19T04:44:32.756835+0000","last_fullsized":"2024-09-19T04:44:32.756835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:43:04.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.757106+0000","last_change":"2024-09-19T04:44:30.754310+0000","last_active":"2024-09-19T04:44:32.757106+0000","last_peered":"2024-09-19T04:44:32.757106+0000","last_clean":"2024-09-19T04:44:32.757106+0000","last_became_active":"2024-09-19T04:44:30.753933+0000","last_became_peered":"2024-09-19T04:44:30.753933+0000","last_unstale":"2024-09-19T04:44:32.757106+0000","last_undegraded":"2024-09-19T04:44:32.757106+0000","last_fullsized":"2024-09-19T04:44:32.757106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:40:44.985693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.753494+0000","last_change":"2024-09-19T04:44:30.757458+0000","last_active":"2024-09-19T04:44:32.753494+0000","last_peered":"2024-09-19T04:44:32.753494+0000","last_clean":"2024-09-19T04:44:32.753494+0000","last_became_active":"2024-09-19T04:44:30.757127+0000","last_became_peered":"2024-09-19T04:44:30.757127+0000","last_unstale":"2024-09-19T04:44:32.753494+0000","last_undegraded":"2024-09-19T04:44:32.753494+0000","last_fullsized":"2024-09-19T04:44:32.753494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:59.325865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.763759+0000","last_change":"2024-09-19T04:44:30.754685+0000","last_active":"2024-09-19T04:44:32.763759+0000","last_peered":"2024-09-19T04:44:32.763759+0000","last_clean":"2024-09-19T04:44:32.763759+0000","last_became_active":"2024-09-19T04:44:30.753625+0000","last_became_peered":"2024-09-19T04:44:30.753625+0000","last_unstale":"2024-09-19T04:44:32.763759+0000","last_undegraded":"2024-09-19T04:44:32.763759+0000","last_fullsized":"2024-09-19T04:44:32.763759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:46:01.164693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.758520+0000","last_change":"2024-09-19T04:44:30.745823+0000","last_active":"2024-09-19T04:44:32.758520+0000","last_peered":"2024-09-19T04:44:32.758520+0000","last_clean":"2024-09-19T04:44:32.758520+0000","last_became_active":"2024-09-19T04:44:30.745224+0000","last_became_peered":"2024-09-19T04:44:30.745224+0000","last_unstale":"2024-09-19T04:44:32.758520+0000","last_undegraded":"2024-09-19T04:44:32.758520+0000","last_fullsized":"2024-09-19T04:44:32.758520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:23.682118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.755558+0000","last_change":"2024-09-19T04:44:30.752138+0000","last_active":"2024-09-19T04:44:32.755558+0000","last_peered":"2024-09-19T04:44:32.755558+0000","last_clean":"2024-09-19T04:44:32.755558+0000","last_became_active":"2024-09-19T04:44:30.751737+0000","last_became_peered":"2024-09-19T04:44:30.751737+0000","last_unstale":"2024-09-19T04:44:32.755558+0000","last_undegraded":"2024-09-19T04:44:32.755558+0000","last_fullsized":"2024-09-19T04:44:32.755558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:08.204601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.763612+0000","last_change":"2024-09-19T04:44:30.748870+0000","last_active":"2024-09-19T04:44:32.763612+0000","last_peered":"2024-09-19T04:44:32.763612+0000","last_clean":"2024-09-19T04:44:32.763612+0000","last_became_active":"2024-09-19T04:44:30.748549+0000","last_became_peered":"2024-09-19T04:44:30.748549+0000","last_unstale":"2024-09-19T04:44:32.763612+0000","last_undegraded":"2024-09-19T04:44:32.763612+0000","last_fullsized":"2024-09-19T04:44:32.763612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:12.951564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.967611+0000","last_change":"2024-09-19T04:44:30.759937+0000","last_active":"2024-09-19T04:44:31.967611+0000","last_peered":"2024-09-19T04:44:31.967611+0000","last_clean":"2024-09-19T04:44:31.967611+0000","last_became_active":"2024-09-19T04:44:30.759659+0000","last_became_peered":"2024-09-19T04:44:30.759659+0000","last_unstale":"2024-09-19T04:44:31.967611+0000","last_undegraded":"2024-09-19T04:44:31.967611+0000","last_fullsized":"2024-09-19T04:44:31.967611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:07.608994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.753749+0000","last_change":"2024-09-19T04:44:30.756216+0000","last_active":"2024-09-19T04:44:32.753749+0000","last_peered":"2024-09-19T04:44:32.753749+0000","last_clean":"2024-09-19T04:44:32.753749+0000","last_became_active":"2024-09-19T04:44:30.755903+0000","last_became_peered":"2024-09-19T04:44:30.755903+0000","last_unstale":"2024-09-19T04:44:32.753749+0000","last_undegraded":"2024-09-19T04:44:32.753749+0000","last_fullsized":"2024-09-19T04:44:32.753749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:16.966305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.349064+0000","last_change":"2024-09-19T04:44:30.746297+0000","last_active":"2024-09-19T04:44:32.349064+0000","last_peered":"2024-09-19T04:44:32.349064+0000","last_clean":"2024-09-19T04:44:32.349064+0000","last_became_active":"2024-09-19T04:44:30.746013+0000","last_became_peered":"2024-09-19T04:44:30.746013+0000","last_unstale":"2024-09-19T04:44:32.349064+0000","last_undegraded":"2024-09-19T04:44:32.349064+0000","last_fullsized":"2024-09-19T04:44:32.349064+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:17.086180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.966963+0000","last_change":"2024-09-19T04:44:30.756317+0000","last_active":"2024-09-19T04:44:31.966963+0000","last_peered":"2024-09-19T04:44:31.966963+0000","last_clean":"2024-09-19T04:44:31.966963+0000","last_became_active":"2024-09-19T04:44:30.755998+0000","last_became_peered":"2024-09-19T04:44:30.755998+0000","last_unstale":"2024-09-19T04:44:31.966963+0000","last_undegraded":"2024-09-19T04:44:31.966963+0000","last_fullsized":"2024-09-19T04:44:31.966963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:06:06.725565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.349533+0000","last_change":"2024-09-19T04:44:30.753423+0000","last_active":"2024-09-19T04:44:32.349533+0000","last_peered":"2024-09-19T04:44:32.349533+0000","last_clean":"2024-09-19T04:44:32.349533+0000","last_became_active":"2024-09-19T04:44:30.752139+0000","last_became_peered":"2024-09-19T04:44:30.752139+0000","last_unstale":"2024-09-19T04:44:32.349533+0000","last_undegraded":"2024-09-19T04:44:32.349533+0000","last_fullsized":"2024-09-19T04:44:32.349533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:18.046893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.755643+0000","last_change":"2024-09-19T04:44:30.751114+0000","last_active":"2024-09-19T04:44:32.755643+0000","last_peered":"2024-09-19T04:44:32.755643+0000","last_clean":"2024-09-19T04:44:32.755643+0000","last_became_active":"2024-09-19T04:44:30.750784+0000","last_became_peered":"2024-09-19T04:44:30.750784+0000","last_unstale":"2024-09-19T04:44:32.755643+0000","last_undegraded":"2024-09-19T04:44:32.755643+0000","last_fullsized":"2024-09-19T04:44:32.755643+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:22.040253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.763630+0000","last_change":"2024-09-19T04:44:30.747191+0000","last_active":"2024-09-19T04:44:32.763630+0000","last_peered":"2024-09-19T04:44:32.763630+0000","last_clean":"2024-09-19T04:44:32.763630+0000","last_became_active":"2024-09-19T04:44:30.746830+0000","last_became_peered":"2024-09-19T04:44:30.746830+0000","last_unstale":"2024-09-19T04:44:32.763630+0000","last_undegraded":"2024-09-19T04:44:32.763630+0000","last_fullsized":"2024-09-19T04:44:32.763630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:24.634920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.967278+0000","last_change":"2024-09-19T04:44:30.755174+0000","last_active":"2024-09-19T04:44:31.967278+0000","last_peered":"2024-09-19T04:44:31.967278+0000","last_clean":"2024-09-19T04:44:31.967278+0000","last_became_active":"2024-09-19T04:44:30.754885+0000","last_became_peered":"2024-09-19T04:44:30.754885+0000","last_unstale":"2024-09-19T04:44:31.967278+0000","last_undegraded":"2024-09-19T04:44:31.967278+0000","last_fullsized":"2024-09-19T04:44:31.967278+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:11:32.428033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.757351+0000","last_change":"2024-09-19T04:44:30.764992+0000","last_active":"2024-09-19T04:44:32.757351+0000","last_peered":"2024-09-19T04:44:32.757351+0000","last_clean":"2024-09-19T04:44:32.757351+0000","last_became_active":"2024-09-19T04:44:30.764575+0000","last_became_peered":"2024-09-19T04:44:30.764575+0000","last_unstale":"2024-09-19T04:44:32.757351+0000","last_undegraded":"2024-09-19T04:44:32.757351+0000","last_fullsized":"2024-09-19T04:44:32.757351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:54.562710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.755504+0000","last_change":"2024-09-19T04:44:30.757343+0000","last_active":"2024-09-19T04:44:32.755504+0000","last_peered":"2024-09-19T04:44:32.755504+0000","last_clean":"2024-09-19T04:44:32.755504+0000","last_became_active":"2024-09-19T04:44:30.756981+0000","last_became_peered":"2024-09-19T04:44:30.756981+0000","last_unstale":"2024-09-19T04:44:32.755504+0000","last_undegraded":"2024-09-19T04:44:32.755504+0000","last_fullsized":"2024-09-19T04:44:32.755504+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:07.388384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.755339+0000","last_change":"2024-09-19T04:44:30.749358+0000","last_active":"2024-09-19T04:44:32.755339+0000","last_peered":"2024-09-19T04:44:32.755339+0000","last_clean":"2024-09-19T04:44:32.755339+0000","last_became_active":"2024-09-19T04:44:30.749028+0000","last_became_peered":"2024-09-19T04:44:30.749028+0000","last_unstale":"2024-09-19T04:44:32.755339+0000","last_undegraded":"2024-09-19T04:44:32.755339+0000","last_fullsized":"2024-09-19T04:44:32.755339+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:32.352823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.763660+0000","last_change":"2024-09-19T04:44:30.748408+0000","last_active":"2024-09-19T04:44:32.763660+0000","last_peered":"2024-09-19T04:44:32.763660+0000","last_clean":"2024-09-19T04:44:32.763660+0000","last_became_active":"2024-09-19T04:44:30.747960+0000","last_became_peered":"2024-09-19T04:44:30.747960+0000","last_unstale":"2024-09-19T04:44:32.763660+0000","last_undegraded":"2024-09-19T04:44:32.763660+0000","last_fullsized":"2024-09-19T04:44:32.763660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:51.043791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.755216+0000","last_change":"2024-09-19T04:44:30.747194+0000","last_active":"2024-09-19T04:44:32.755216+0000","last_peered":"2024-09-19T04:44:32.755216+0000","last_clean":"2024-09-19T04:44:32.755216+0000","last_became_active":"2024-09-19T04:44:30.746871+0000","last_became_peered":"2024-09-19T04:44:30.746871+0000","last_unstale":"2024-09-19T04:44:32.755216+0000","last_undegraded":"2024-09-19T04:44:32.755216+0000","last_fullsized":"2024-09-19T04:44:32.755216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:52.340341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:56.863918+0000","last_change":"2024-09-19T04:44:28.337746+0000","last_active":"2024-09-19T04:44:56.863918+0000","last_peered":"2024-09-19T04:44:56.863918+0000","last_clean":"2024-09-19T04:44:56.863918+0000","last_became_active":"2024-09-19T04:44:28.337410+0000","last_became_peered":"2024-09-19T04:44:28.337410+0000","last_unstale":"2024-09-19T04:44:56.863918+0000","last_undegraded":"2024-09-19T04:44:56.863918+0000","last_fullsized":"2024-09-19T04:44:56.863918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:41:27.436939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.966538+0000","last_change":"2024-09-19T04:44:30.755512+0000","last_active":"2024-09-19T04:44:31.966538+0000","last_peered":"2024-09-19T04:44:31.966538+0000","last_clean":"2024-09-19T04:44:31.966538+0000","last_became_active":"2024-09-19T04:44:30.755086+0000","last_became_peered":"2024-09-19T04:44:30.755086+0000","last_unstale":"2024-09-19T04:44:31.966538+0000","last_undegraded":"2024-09-19T04:44:31.966538+0000","last_fullsized":"2024-09-19T04:44:31.966538+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:33.870376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:42.831284+0000","last_change":"2024-09-19T04:44:28.334492+0000","last_active":"2024-09-19T04:44:42.831284+0000","last_peered":"2024-09-19T04:44:42.831284+0000","last_clean":"2024-09-19T04:44:42.831284+0000","last_became_active":"2024-09-19T04:44:28.334098+0000","last_became_peered":"2024-09-19T04:44:28.334098+0000","last_unstale":"2024-09-19T04:44:42.831284+0000","last_undegraded":"2024-09-19T04:44:42.831284+0000","last_fullsized":"2024-09-19T04:44:42.831284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:48.156590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:41.818139+0000","last_change":"2024-09-19T04:44:28.335593+0000","last_active":"2024-09-19T04:44:41.818139+0000","last_peered":"2024-09-19T04:44:41.818139+0000","last_clean":"2024-09-19T04:44:41.818139+0000","last_became_active":"2024-09-19T04:44:28.335060+0000","last_became_peered":"2024-09-19T04:44:28.335060+0000","last_unstale":"2024-09-19T04:44:41.818139+0000","last_undegraded":"2024-09-19T04:44:41.818139+0000","last_fullsized":"2024-09-19T04:44:41.818139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:27.859960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.349338+0000","last_change":"2024-09-19T04:44:30.748235+0000","last_active":"2024-09-19T04:44:32.349338+0000","last_peered":"2024-09-19T04:44:32.349338+0000","last_clean":"2024-09-19T04:44:32.349338+0000","last_became_active":"2024-09-19T04:44:30.747881+0000","last_became_peered":"2024-09-19T04:44:30.747881+0000","last_unstale":"2024-09-19T04:44:32.349338+0000","last_undegraded":"2024-09-19T04:44:32.349338+0000","last_fullsized":"2024-09-19T04:44:32.349338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:40:39.183897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.839216+0000","last_change":"2024-09-19T04:44:28.337762+0000","last_active":"2024-09-19T04:44:43.839216+0000","last_peered":"2024-09-19T04:44:43.839216+0000","last_clean":"2024-09-19T04:44:43.839216+0000","last_became_active":"2024-09-19T04:44:28.337383+0000","last_became_peered":"2024-09-19T04:44:28.337383+0000","last_unstale":"2024-09-19T04:44:43.839216+0000","last_undegraded":"2024-09-19T04:44:43.839216+0000","last_fullsized":"2024-09-19T04:44:43.839216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:07:55.268958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.757712+0000","last_change":"2024-09-19T04:44:30.754318+0000","last_active":"2024-09-19T04:44:32.757712+0000","last_peered":"2024-09-19T04:44:32.757712+0000","last_clean":"2024-09-19T04:44:32.757712+0000","last_became_active":"2024-09-19T04:44:30.754025+0000","last_became_peered":"2024-09-19T04:44:30.754025+0000","last_unstale":"2024-09-19T04:44:32.757712+0000","last_undegraded":"2024-09-19T04:44:32.757712+0000","last_fullsized":"2024-09-19T04:44:32.757712+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:58.190504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:56.862941+0000","last_change":"2024-09-19T04:44:28.335947+0000","last_active":"2024-09-19T04:44:56.862941+0000","last_peered":"2024-09-19T04:44:56.862941+0000","last_clean":"2024-09-19T04:44:56.862941+0000","last_became_active":"2024-09-19T04:44:28.335588+0000","last_became_peered":"2024-09-19T04:44:28.335588+0000","last_unstale":"2024-09-19T04:44:56.862941+0000","last_undegraded":"2024-09-19T04:44:56.862941+0000","last_fullsized":"2024-09-19T04:44:56.862941+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:44.024915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.754044+0000","last_change":"2024-09-19T04:44:30.748272+0000","last_active":"2024-09-19T04:44:32.754044+0000","last_peered":"2024-09-19T04:44:32.754044+0000","last_clean":"2024-09-19T04:44:32.754044+0000","last_became_active":"2024-09-19T04:44:30.747147+0000","last_became_peered":"2024-09-19T04:44:30.747147+0000","last_unstale":"2024-09-19T04:44:32.754044+0000","last_undegraded":"2024-09-19T04:44:32.754044+0000","last_fullsized":"2024-09-19T04:44:32.754044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:11.653693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.837670+0000","last_change":"2024-09-19T04:44:28.337120+0000","last_active":"2024-09-19T04:44:43.837670+0000","last_peered":"2024-09-19T04:44:43.837670+0000","last_clean":"2024-09-19T04:44:43.837670+0000","last_became_active":"2024-09-19T04:44:28.336053+0000","last_became_peered":"2024-09-19T04:44:28.336053+0000","last_unstale":"2024-09-19T04:44:43.837670+0000","last_undegraded":"2024-09-19T04:44:43.837670+0000","last_fullsized":"2024-09-19T04:44:43.837670+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:56:57.514931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.757478+0000","last_change":"2024-09-19T04:44:30.755238+0000","last_active":"2024-09-19T04:44:32.757478+0000","last_peered":"2024-09-19T04:44:32.757478+0000","last_clean":"2024-09-19T04:44:32.757478+0000","last_became_active":"2024-09-19T04:44:30.754748+0000","last_became_peered":"2024-09-19T04:44:30.754748+0000","last_unstale":"2024-09-19T04:44:32.757478+0000","last_undegraded":"2024-09-19T04:44:32.757478+0000","last_fullsized":"2024-09-19T04:44:32.757478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:38.954491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:42.828735+0000","last_change":"2024-09-19T04:44:28.596147+0000","last_active":"2024-09-19T04:44:42.828735+0000","last_peered":"2024-09-19T04:44:42.828735+0000","last_clean":"2024-09-19T04:44:42.828735+0000","last_became_active":"2024-09-19T04:44:28.595785+0000","last_became_peered":"2024-09-19T04:44:28.595785+0000","last_unstale":"2024-09-19T04:44:42.828735+0000","last_undegraded":"2024-09-19T04:44:42.828735+0000","last_fullsized":"2024-09-19T04:44:42.828735+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:39.138724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.754127+0000","last_change":"2024-09-19T04:44:30.753817+0000","last_active":"2024-09-19T04:44:32.754127+0000","last_peered":"2024-09-19T04:44:32.754127+0000","last_clean":"2024-09-19T04:44:32.754127+0000","last_became_active":"2024-09-19T04:44:30.752541+0000","last_became_peered":"2024-09-19T04:44:30.752541+0000","last_unstale":"2024-09-19T04:44:32.754127+0000","last_undegraded":"2024-09-19T04:44:32.754127+0000","last_fullsized":"2024-09-19T04:44:32.754127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:16.121822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'27","reported_seq":79,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:56.863320+0000","last_change":"2024-09-19T04:44:28.335449+0000","last_active":"2024-09-19T04:44:56.863320+0000","last_peered":"2024-09-19T04:44:56.863320+0000","last_clean":"2024-09-19T04:44:56.863320+0000","last_became_active":"2024-09-19T04:44:28.335080+0000","last_became_peered":"2024-09-19T04:44:28.335080+0000","last_unstale":"2024-09-19T04:44:56.863320+0000","last_undegraded":"2024-09-19T04:44:56.863320+0000","last_fullsized":"2024-09-19T04:44:56.863320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:28.436290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50112,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":33,"num_read_kb":64,"num_write":29,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.757132+0000","last_change":"2024-09-19T04:44:30.749624+0000","last_active":"2024-09-19T04:44:32.757132+0000","last_peered":"2024-09-19T04:44:32.757132+0000","last_clean":"2024-09-19T04:44:32.757132+0000","last_became_active":"2024-09-19T04:44:30.749295+0000","last_became_peered":"2024-09-19T04:44:30.749295+0000","last_unstale":"2024-09-19T04:44:32.757132+0000","last_undegraded":"2024-09-19T04:44:32.757132+0000","last_fullsized":"2024-09-19T04:44:32.757132+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:48:46.942237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:44.843058+0000","last_change":"2024-09-19T04:44:28.337863+0000","last_active":"2024-09-19T04:44:44.843058+0000","last_peered":"2024-09-19T04:44:44.843058+0000","last_clean":"2024-09-19T04:44:44.843058+0000","last_became_active":"2024-09-19T04:44:28.337522+0000","last_became_peered":"2024-09-19T04:44:28.337522+0000","last_unstale":"2024-09-19T04:44:44.843058+0000","last_undegraded":"2024-09-19T04:44:44.843058+0000","last_fullsized":"2024-09-19T04:44:44.843058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:02.488622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.966914+0000","last_change":"2024-09-19T04:44:30.759916+0000","last_active":"2024-09-19T04:44:31.966914+0000","last_peered":"2024-09-19T04:44:31.966914+0000","last_clean":"2024-09-19T04:44:31.966914+0000","last_became_active":"2024-09-19T04:44:30.758835+0000","last_became_peered":"2024-09-19T04:44:30.758835+0000","last_unstale":"2024-09-19T04:44:31.966914+0000","last_undegraded":"2024-09-19T04:44:31.966914+0000","last_fullsized":"2024-09-19T04:44:31.966914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:38.614343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.837422+0000","last_change":"2024-09-19T04:44:28.337574+0000","last_active":"2024-09-19T04:44:43.837422+0000","last_peered":"2024-09-19T04:44:43.837422+0000","last_clean":"2024-09-19T04:44:43.837422+0000","last_became_active":"2024-09-19T04:44:28.337224+0000","last_became_peered":"2024-09-19T04:44:28.337224+0000","last_unstale":"2024-09-19T04:44:43.837422+0000","last_undegraded":"2024-09-19T04:44:43.837422+0000","last_fullsized":"2024-09-19T04:44:43.837422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:20.538181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.307164+0000","last_change":"2024-09-19T04:44:30.758152+0000","last_active":"2024-09-19T04:44:32.307164+0000","last_peered":"2024-09-19T04:44:32.307164+0000","last_clean":"2024-09-19T04:44:32.307164+0000","last_became_active":"2024-09-19T04:44:30.757801+0000","last_became_peered":"2024-09-19T04:44:30.757801+0000","last_unstale":"2024-09-19T04:44:32.307164+0000","last_undegraded":"2024-09-19T04:44:32.307164+0000","last_fullsized":"2024-09-19T04:44:32.307164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:05.212471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:55.863188+0000","last_change":"2024-09-19T04:44:28.339340+0000","last_active":"2024-09-19T04:44:55.863188+0000","last_peered":"2024-09-19T04:44:55.863188+0000","last_clean":"2024-09-19T04:44:55.863188+0000","last_became_active":"2024-09-19T04:44:28.339019+0000","last_became_peered":"2024-09-19T04:44:28.339019+0000","last_unstale":"2024-09-19T04:44:55.863188+0000","last_undegraded":"2024-09-19T04:44:55.863188+0000","last_fullsized":"2024-09-19T04:44:55.863188+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:29:59.356748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.306894+0000","last_change":"2024-09-19T04:44:30.751217+0000","last_active":"2024-09-19T04:44:32.306894+0000","last_peered":"2024-09-19T04:44:32.306894+0000","last_clean":"2024-09-19T04:44:32.306894+0000","last_became_active":"2024-09-19T04:44:30.750863+0000","last_became_peered":"2024-09-19T04:44:30.750863+0000","last_unstale":"2024-09-19T04:44:32.306894+0000","last_undegraded":"2024-09-19T04:44:32.306894+0000","last_fullsized":"2024-09-19T04:44:32.306894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:09.957572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.838396+0000","last_change":"2024-09-19T04:44:28.596585+0000","last_active":"2024-09-19T04:44:43.838396+0000","last_peered":"2024-09-19T04:44:43.838396+0000","last_clean":"2024-09-19T04:44:43.838396+0000","last_became_active":"2024-09-19T04:44:28.596218+0000","last_became_peered":"2024-09-19T04:44:28.596218+0000","last_unstale":"2024-09-19T04:44:43.838396+0000","last_undegraded":"2024-09-19T04:44:43.838396+0000","last_fullsized":"2024-09-19T04:44:43.838396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:29:35.395610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.966362+0000","last_change":"2024-09-19T04:44:30.746590+0000","last_active":"2024-09-19T04:44:31.966362+0000","last_peered":"2024-09-19T04:44:31.966362+0000","last_clean":"2024-09-19T04:44:31.966362+0000","last_became_active":"2024-09-19T04:44:30.746263+0000","last_became_peered":"2024-09-19T04:44:30.746263+0000","last_unstale":"2024-09-19T04:44:31.966362+0000","last_undegraded":"2024-09-19T04:44:31.966362+0000","last_fullsized":"2024-09-19T04:44:31.966362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:38.955613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.811752+0000","last_change":"2024-09-19T04:44:28.335826+0000","last_active":"2024-09-19T04:44:43.811752+0000","last_peered":"2024-09-19T04:44:43.811752+0000","last_clean":"2024-09-19T04:44:43.811752+0000","last_became_active":"2024-09-19T04:44:28.335481+0000","last_became_peered":"2024-09-19T04:44:28.335481+0000","last_unstale":"2024-09-19T04:44:43.811752+0000","last_undegraded":"2024-09-19T04:44:43.811752+0000","last_fullsized":"2024-09-19T04:44:43.811752+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:20.989500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.763607+0000","last_change":"2024-09-19T04:44:30.744130+0000","last_active":"2024-09-19T04:44:32.763607+0000","last_peered":"2024-09-19T04:44:32.763607+0000","last_clean":"2024-09-19T04:44:32.763607+0000","last_became_active":"2024-09-19T04:44:30.743772+0000","last_became_peered":"2024-09-19T04:44:30.743772+0000","last_unstale":"2024-09-19T04:44:32.763607+0000","last_undegraded":"2024-09-19T04:44:32.763607+0000","last_fullsized":"2024-09-19T04:44:32.763607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:34.956670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.837451+0000","last_change":"2024-09-19T04:44:28.337543+0000","last_active":"2024-09-19T04:44:43.837451+0000","last_peered":"2024-09-19T04:44:43.837451+0000","last_clean":"2024-09-19T04:44:43.837451+0000","last_became_active":"2024-09-19T04:44:28.336128+0000","last_became_peered":"2024-09-19T04:44:28.336128+0000","last_unstale":"2024-09-19T04:44:43.837451+0000","last_undegraded":"2024-09-19T04:44:43.837451+0000","last_fullsized":"2024-09-19T04:44:43.837451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:31.907777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.307438+0000","last_change":"2024-09-19T04:44:30.757489+0000","last_active":"2024-09-19T04:44:32.307438+0000","last_peered":"2024-09-19T04:44:32.307438+0000","last_clean":"2024-09-19T04:44:32.307438+0000","last_became_active":"2024-09-19T04:44:30.757041+0000","last_became_peered":"2024-09-19T04:44:30.757041+0000","last_unstale":"2024-09-19T04:44:32.307438+0000","last_undegraded":"2024-09-19T04:44:32.307438+0000","last_fullsized":"2024-09-19T04:44:32.307438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:03.395913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.837733+0000","last_change":"2024-09-19T04:44:28.327393+0000","last_active":"2024-09-19T04:44:43.837733+0000","last_peered":"2024-09-19T04:44:43.837733+0000","last_clean":"2024-09-19T04:44:43.837733+0000","last_became_active":"2024-09-19T04:44:28.327116+0000","last_became_peered":"2024-09-19T04:44:28.327116+0000","last_unstale":"2024-09-19T04:44:43.837733+0000","last_undegraded":"2024-09-19T04:44:43.837733+0000","last_fullsized":"2024-09-19T04:44:43.837733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:42.360980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.492611+0000","last_change":"2024-09-19T04:44:30.754239+0000","last_active":"2024-09-19T04:44:32.492611+0000","last_peered":"2024-09-19T04:44:32.492611+0000","last_clean":"2024-09-19T04:44:32.492611+0000","last_became_active":"2024-09-19T04:44:30.753885+0000","last_became_peered":"2024-09-19T04:44:30.753885+0000","last_unstale":"2024-09-19T04:44:32.492611+0000","last_undegraded":"2024-09-19T04:44:32.492611+0000","last_fullsized":"2024-09-19T04:44:32.492611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:27.355600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'5","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:55.862076+0000","last_change":"2024-09-19T04:44:28.596214+0000","last_active":"2024-09-19T04:44:55.862076+0000","last_peered":"2024-09-19T04:44:55.862076+0000","last_clean":"2024-09-19T04:44:55.862076+0000","last_became_active":"2024-09-19T04:44:28.595841+0000","last_became_peered":"2024-09-19T04:44:28.595841+0000","last_unstale":"2024-09-19T04:44:55.862076+0000","last_undegraded":"2024-09-19T04:44:55.862076+0000","last_fullsized":"2024-09-19T04:44:55.862076+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:43.864702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.307461+0000","last_change":"2024-09-19T04:44:30.756192+0000","last_active":"2024-09-19T04:44:32.307461+0000","last_peered":"2024-09-19T04:44:32.307461+0000","last_clean":"2024-09-19T04:44:32.307461+0000","last_became_active":"2024-09-19T04:44:30.755853+0000","last_became_peered":"2024-09-19T04:44:30.755853+0000","last_unstale":"2024-09-19T04:44:32.307461+0000","last_undegraded":"2024-09-19T04:44:32.307461+0000","last_fullsized":"2024-09-19T04:44:32.307461+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:25:49.206978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":345,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.306983+0000","last_change":"2024-09-19T04:43:10.156816+0000","last_active":"2024-09-19T04:44:32.306983+0000","last_peered":"2024-09-19T04:44:32.306983+0000","last_clean":"2024-09-19T04:44:32.306983+0000","last_became_active":"2024-09-19T04:43:10.156359+0000","last_became_peered":"2024-09-19T04:43:10.156359+0000","last_unstale":"2024-09-19T04:44:32.306983+0000","last_undegraded":"2024-09-19T04:44:32.306983+0000","last_fullsized":"2024-09-19T04:44:32.306983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:39:34.200793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:39:34.200793+0000","last_clean_scrub_stamp":"2024-09-19T04:39:34.200793+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:13.453818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:44.845247+0000","last_change":"2024-09-19T04:44:28.337637+0000","last_active":"2024-09-19T04:44:44.845247+0000","last_peered":"2024-09-19T04:44:44.845247+0000","last_clean":"2024-09-19T04:44:44.845247+0000","last_became_active":"2024-09-19T04:44:28.337235+0000","last_became_peered":"2024-09-19T04:44:28.337235+0000","last_unstale":"2024-09-19T04:44:44.845247+0000","last_undegraded":"2024-09-19T04:44:44.845247+0000","last_fullsized":"2024-09-19T04:44:44.845247+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:42.104972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.349528+0000","last_change":"2024-09-19T04:44:30.754697+0000","last_active":"2024-09-19T04:44:32.349528+0000","last_peered":"2024-09-19T04:44:32.349528+0000","last_clean":"2024-09-19T04:44:32.349528+0000","last_became_active":"2024-09-19T04:44:30.753214+0000","last_became_peered":"2024-09-19T04:44:30.753214+0000","last_unstale":"2024-09-19T04:44:32.349528+0000","last_undegraded":"2024-09-19T04:44:32.349528+0000","last_fullsized":"2024-09-19T04:44:32.349528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:37.455470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:41.816042+0000","last_change":"2024-09-19T04:44:28.336124+0000","last_active":"2024-09-19T04:44:41.816042+0000","last_peered":"2024-09-19T04:44:41.816042+0000","last_clean":"2024-09-19T04:44:41.816042+0000","last_became_active":"2024-09-19T04:44:28.335754+0000","last_became_peered":"2024-09-19T04:44:28.335754+0000","last_unstale":"2024-09-19T04:44:41.816042+0000","last_undegraded":"2024-09-19T04:44:41.816042+0000","last_fullsized":"2024-09-19T04:44:41.816042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:41.604367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.966871+0000","last_change":"2024-09-19T04:44:30.749939+0000","last_active":"2024-09-19T04:44:31.966871+0000","last_peered":"2024-09-19T04:44:31.966871+0000","last_clean":"2024-09-19T04:44:31.966871+0000","last_became_active":"2024-09-19T04:44:30.747122+0000","last_became_peered":"2024-09-19T04:44:30.747122+0000","last_unstale":"2024-09-19T04:44:31.966871+0000","last_undegraded":"2024-09-19T04:44:31.966871+0000","last_fullsized":"2024-09-19T04:44:31.966871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:46:08.371493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:54.861625+0000","last_change":"2024-09-19T04:44:28.594768+0000","last_active":"2024-09-19T04:44:54.861625+0000","last_peered":"2024-09-19T04:44:54.861625+0000","last_clean":"2024-09-19T04:44:54.861625+0000","last_became_active":"2024-09-19T04:44:28.594320+0000","last_became_peered":"2024-09-19T04:44:28.594320+0000","last_unstale":"2024-09-19T04:44:54.861625+0000","last_undegraded":"2024-09-19T04:44:54.861625+0000","last_fullsized":"2024-09-19T04:44:54.861625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:32:17.736874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.967034+0000","last_change":"2024-09-19T04:44:30.746666+0000","last_active":"2024-09-19T04:44:31.967034+0000","last_peered":"2024-09-19T04:44:31.967034+0000","last_clean":"2024-09-19T04:44:31.967034+0000","last_became_active":"2024-09-19T04:44:30.746312+0000","last_became_peered":"2024-09-19T04:44:30.746312+0000","last_unstale":"2024-09-19T04:44:31.967034+0000","last_undegraded":"2024-09-19T04:44:31.967034+0000","last_fullsized":"2024-09-19T04:44:31.967034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:13.562810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'5","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:55.862089+0000","last_change":"2024-09-19T04:44:28.336492+0000","last_active":"2024-09-19T04:44:55.862089+0000","last_peered":"2024-09-19T04:44:55.862089+0000","last_clean":"2024-09-19T04:44:55.862089+0000","last_became_active":"2024-09-19T04:44:28.336044+0000","last_became_peered":"2024-09-19T04:44:28.336044+0000","last_unstale":"2024-09-19T04:44:55.862089+0000","last_undegraded":"2024-09-19T04:44:55.862089+0000","last_fullsized":"2024-09-19T04:44:55.862089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:18.760473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.966467+0000","last_change":"2024-09-19T04:44:30.756340+0000","last_active":"2024-09-19T04:44:31.966467+0000","last_peered":"2024-09-19T04:44:31.966467+0000","last_clean":"2024-09-19T04:44:31.966467+0000","last_became_active":"2024-09-19T04:44:30.755989+0000","last_became_peered":"2024-09-19T04:44:30.755989+0000","last_unstale":"2024-09-19T04:44:31.966467+0000","last_undegraded":"2024-09-19T04:44:31.966467+0000","last_fullsized":"2024-09-19T04:44:31.966467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:13:11.792019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.758057+0000","last_change":"2024-09-19T04:44:28.596322+0000","last_active":"2024-09-19T04:44:32.758057+0000","last_peered":"2024-09-19T04:44:32.758057+0000","last_clean":"2024-09-19T04:44:32.758057+0000","last_became_active":"2024-09-19T04:44:28.595566+0000","last_became_peered":"2024-09-19T04:44:28.595566+0000","last_unstale":"2024-09-19T04:44:32.758057+0000","last_undegraded":"2024-09-19T04:44:32.758057+0000","last_fullsized":"2024-09-19T04:44:32.758057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:23.878145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.366685+0000","last_change":"2024-09-19T04:44:30.752321+0000","last_active":"2024-09-19T04:44:32.366685+0000","last_peered":"2024-09-19T04:44:32.366685+0000","last_clean":"2024-09-19T04:44:32.366685+0000","last_became_active":"2024-09-19T04:44:30.751980+0000","last_became_peered":"2024-09-19T04:44:30.751980+0000","last_unstale":"2024-09-19T04:44:32.366685+0000","last_undegraded":"2024-09-19T04:44:32.366685+0000","last_fullsized":"2024-09-19T04:44:32.366685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:41.863034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.838509+0000","last_change":"2024-09-19T04:44:28.596547+0000","last_active":"2024-09-19T04:44:43.838509+0000","last_peered":"2024-09-19T04:44:43.838509+0000","last_clean":"2024-09-19T04:44:43.838509+0000","last_became_active":"2024-09-19T04:44:28.596181+0000","last_became_peered":"2024-09-19T04:44:28.596181+0000","last_unstale":"2024-09-19T04:44:43.838509+0000","last_undegraded":"2024-09-19T04:44:43.838509+0000","last_fullsized":"2024-09-19T04:44:43.838509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:53.251737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.967409+0000","last_change":"2024-09-19T04:44:30.751533+0000","last_active":"2024-09-19T04:44:31.967409+0000","last_peered":"2024-09-19T04:44:31.967409+0000","last_clean":"2024-09-19T04:44:31.967409+0000","last_became_active":"2024-09-19T04:44:30.748012+0000","last_became_peered":"2024-09-19T04:44:30.748012+0000","last_unstale":"2024-09-19T04:44:31.967409+0000","last_undegraded":"2024-09-19T04:44:31.967409+0000","last_fullsized":"2024-09-19T04:44:31.967409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:32.360147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'12","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:54.755433+0000","last_change":"2024-09-19T04:44:28.335884+0000","last_active":"2024-09-19T04:44:54.755433+0000","last_peered":"2024-09-19T04:44:54.755433+0000","last_clean":"2024-09-19T04:44:54.755433+0000","last_became_active":"2024-09-19T04:44:28.335521+0000","last_became_peered":"2024-09-19T04:44:28.335521+0000","last_unstale":"2024-09-19T04:44:54.755433+0000","last_undegraded":"2024-09-19T04:44:54.755433+0000","last_fullsized":"2024-09-19T04:44:54.755433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:32:49.764893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.349718+0000","last_change":"2024-09-19T04:44:30.759578+0000","last_active":"2024-09-19T04:44:32.349718+0000","last_peered":"2024-09-19T04:44:32.349718+0000","last_clean":"2024-09-19T04:44:32.349718+0000","last_became_active":"2024-09-19T04:44:30.759245+0000","last_became_peered":"2024-09-19T04:44:30.759245+0000","last_unstale":"2024-09-19T04:44:32.349718+0000","last_undegraded":"2024-09-19T04:44:32.349718+0000","last_fullsized":"2024-09-19T04:44:32.349718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:37.121770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.839671+0000","last_change":"2024-09-19T04:44:28.593404+0000","last_active":"2024-09-19T04:44:43.839671+0000","last_peered":"2024-09-19T04:44:43.839671+0000","last_clean":"2024-09-19T04:44:43.839671+0000","last_became_active":"2024-09-19T04:44:28.593037+0000","last_became_peered":"2024-09-19T04:44:28.593037+0000","last_unstale":"2024-09-19T04:44:43.839671+0000","last_undegraded":"2024-09-19T04:44:43.839671+0000","last_fullsized":"2024-09-19T04:44:43.839671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:08:29.218543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.757281+0000","last_change":"2024-09-19T04:44:30.763656+0000","last_active":"2024-09-19T04:44:32.757281+0000","last_peered":"2024-09-19T04:44:32.757281+0000","last_clean":"2024-09-19T04:44:32.757281+0000","last_became_active":"2024-09-19T04:44:30.761653+0000","last_became_peered":"2024-09-19T04:44:30.761653+0000","last_unstale":"2024-09-19T04:44:32.757281+0000","last_undegraded":"2024-09-19T04:44:32.757281+0000","last_fullsized":"2024-09-19T04:44:32.757281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:22.077931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:40.808459+0000","last_change":"2024-09-19T04:44:28.594541+0000","last_active":"2024-09-19T04:44:40.808459+0000","last_peered":"2024-09-19T04:44:40.808459+0000","last_clean":"2024-09-19T04:44:40.808459+0000","last_became_active":"2024-09-19T04:44:28.594262+0000","last_became_peered":"2024-09-19T04:44:28.594262+0000","last_unstale":"2024-09-19T04:44:40.808459+0000","last_undegraded":"2024-09-19T04:44:40.808459+0000","last_fullsized":"2024-09-19T04:44:40.808459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:45.685034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.754243+0000","last_change":"2024-09-19T04:44:30.757861+0000","last_active":"2024-09-19T04:44:32.754243+0000","last_peered":"2024-09-19T04:44:32.754243+0000","last_clean":"2024-09-19T04:44:32.754243+0000","last_became_active":"2024-09-19T04:44:30.757509+0000","last_became_peered":"2024-09-19T04:44:30.757509+0000","last_unstale":"2024-09-19T04:44:32.754243+0000","last_undegraded":"2024-09-19T04:44:32.754243+0000","last_fullsized":"2024-09-19T04:44:32.754243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:17.737055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.837765+0000","last_change":"2024-09-19T04:44:28.335225+0000","last_active":"2024-09-19T04:44:43.837765+0000","last_peered":"2024-09-19T04:44:43.837765+0000","last_clean":"2024-09-19T04:44:43.837765+0000","last_became_active":"2024-09-19T04:44:28.334804+0000","last_became_peered":"2024-09-19T04:44:28.334804+0000","last_unstale":"2024-09-19T04:44:43.837765+0000","last_undegraded":"2024-09-19T04:44:43.837765+0000","last_fullsized":"2024-09-19T04:44:43.837765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:15.639418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.763615+0000","last_change":"2024-09-19T04:44:30.754018+0000","last_active":"2024-09-19T04:44:32.763615+0000","last_peered":"2024-09-19T04:44:32.763615+0000","last_clean":"2024-09-19T04:44:32.763615+0000","last_became_active":"2024-09-19T04:44:30.753639+0000","last_became_peered":"2024-09-19T04:44:30.753639+0000","last_unstale":"2024-09-19T04:44:32.763615+0000","last_undegraded":"2024-09-19T04:44:32.763615+0000","last_fullsized":"2024-09-19T04:44:32.763615+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:05.012847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:42.829548+0000","last_change":"2024-09-19T04:44:28.335884+0000","last_active":"2024-09-19T04:44:42.829548+0000","last_peered":"2024-09-19T04:44:42.829548+0000","last_clean":"2024-09-19T04:44:42.829548+0000","last_became_active":"2024-09-19T04:44:28.335501+0000","last_became_peered":"2024-09-19T04:44:28.335501+0000","last_unstale":"2024-09-19T04:44:42.829548+0000","last_undegraded":"2024-09-19T04:44:42.829548+0000","last_fullsized":"2024-09-19T04:44:42.829548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:23.541939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.349482+0000","last_change":"2024-09-19T04:44:30.758543+0000","last_active":"2024-09-19T04:44:32.349482+0000","last_peered":"2024-09-19T04:44:32.349482+0000","last_clean":"2024-09-19T04:44:32.349482+0000","last_became_active":"2024-09-19T04:44:30.757228+0000","last_became_peered":"2024-09-19T04:44:30.757228+0000","last_unstale":"2024-09-19T04:44:32.349482+0000","last_undegraded":"2024-09-19T04:44:32.349482+0000","last_fullsized":"2024-09-19T04:44:32.349482+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:13.877742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:40.808803+0000","last_change":"2024-09-19T04:44:28.339152+0000","last_active":"2024-09-19T04:44:40.808803+0000","last_peered":"2024-09-19T04:44:40.808803+0000","last_clean":"2024-09-19T04:44:40.808803+0000","last_became_active":"2024-09-19T04:44:28.338778+0000","last_became_peered":"2024-09-19T04:44:28.338778+0000","last_unstale":"2024-09-19T04:44:40.808803+0000","last_undegraded":"2024-09-19T04:44:40.808803+0000","last_fullsized":"2024-09-19T04:44:40.808803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:53.906755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.307146+0000","last_change":"2024-09-19T04:44:30.759330+0000","last_active":"2024-09-19T04:44:32.307146+0000","last_peered":"2024-09-19T04:44:32.307146+0000","last_clean":"2024-09-19T04:44:32.307146+0000","last_became_active":"2024-09-19T04:44:30.758987+0000","last_became_peered":"2024-09-19T04:44:30.758987+0000","last_unstale":"2024-09-19T04:44:32.307146+0000","last_undegraded":"2024-09-19T04:44:32.307146+0000","last_fullsized":"2024-09-19T04:44:32.307146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:10.105051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:31.966540+0000","last_change":"2024-09-19T04:44:30.748089+0000","last_active":"2024-09-19T04:44:31.966540+0000","last_peered":"2024-09-19T04:44:31.966540+0000","last_clean":"2024-09-19T04:44:31.966540+0000","last_became_active":"2024-09-19T04:44:30.747566+0000","last_became_peered":"2024-09-19T04:44:30.747566+0000","last_unstale":"2024-09-19T04:44:31.966540+0000","last_undegraded":"2024-09-19T04:44:31.966540+0000","last_fullsized":"2024-09-19T04:44:31.966540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:32.406745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:43.839563+0000","last_change":"2024-09-19T04:44:28.334217+0000","last_active":"2024-09-19T04:44:43.839563+0000","last_peered":"2024-09-19T04:44:43.839563+0000","last_clean":"2024-09-19T04:44:43.839563+0000","last_became_active":"2024-09-19T04:44:28.333768+0000","last_became_peered":"2024-09-19T04:44:28.333768+0000","last_unstale":"2024-09-19T04:44:43.839563+0000","last_undegraded":"2024-09-19T04:44:43.839563+0000","last_fullsized":"2024-09-19T04:44:43.839563+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:52:56.626635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.754179+0000","last_change":"2024-09-19T04:44:30.747498+0000","last_active":"2024-09-19T04:44:32.754179+0000","last_peered":"2024-09-19T04:44:32.754179+0000","last_clean":"2024-09-19T04:44:32.754179+0000","last_became_active":"2024-09-19T04:44:30.746822+0000","last_became_peered":"2024-09-19T04:44:30.746822+0000","last_unstale":"2024-09-19T04:44:32.754179+0000","last_undegraded":"2024-09-19T04:44:32.754179+0000","last_fullsized":"2024-09-19T04:44:32.754179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:09.460263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:44.842877+0000","last_change":"2024-09-19T04:44:28.336755+0000","last_active":"2024-09-19T04:44:44.842877+0000","last_peered":"2024-09-19T04:44:44.842877+0000","last_clean":"2024-09-19T04:44:44.842877+0000","last_became_active":"2024-09-19T04:44:28.336405+0000","last_became_peered":"2024-09-19T04:44:28.336405+0000","last_unstale":"2024-09-19T04:44:44.842877+0000","last_undegraded":"2024-09-19T04:44:44.842877+0000","last_fullsized":"2024-09-19T04:44:44.842877+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:58:13.725792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.757300+0000","last_change":"2024-09-19T04:44:30.762776+0000","last_active":"2024-09-19T04:44:32.757300+0000","last_peered":"2024-09-19T04:44:32.757300+0000","last_clean":"2024-09-19T04:44:32.757300+0000","last_became_active":"2024-09-19T04:44:30.761308+0000","last_became_peered":"2024-09-19T04:44:30.761308+0000","last_unstale":"2024-09-19T04:44:32.757300+0000","last_undegraded":"2024-09-19T04:44:32.757300+0000","last_fullsized":"2024-09-19T04:44:32.757300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:19.960125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.755814+0000","last_change":"2024-09-19T04:44:28.594858+0000","last_active":"2024-09-19T04:44:32.755814+0000","last_peered":"2024-09-19T04:44:32.755814+0000","last_clean":"2024-09-19T04:44:32.755814+0000","last_became_active":"2024-09-19T04:44:28.594515+0000","last_became_peered":"2024-09-19T04:44:28.594515+0000","last_unstale":"2024-09-19T04:44:32.755814+0000","last_undegraded":"2024-09-19T04:44:32.755814+0000","last_fullsized":"2024-09-19T04:44:32.755814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:53.728393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:32.349385+0000","last_change":"2024-09-19T04:44:30.753604+0000","last_active":"2024-09-19T04:44:32.349385+0000","last_peered":"2024-09-19T04:44:32.349385+0000","last_clean":"2024-09-19T04:44:32.349385+0000","last_became_active":"2024-09-19T04:44:30.751961+0000","last_became_peered":"2024-09-19T04:44:30.751961+0000","last_unstale":"2024-09-19T04:44:32.349385+0000","last_undegraded":"2024-09-19T04:44:32.349385+0000","last_fullsized":"2024-09-19T04:44:32.349385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:46:58.759650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:44:55.861648+0000","last_change":"2024-09-19T04:44:28.339499+0000","last_active":"2024-09-19T04:44:55.861648+0000","last_peered":"2024-09-19T04:44:55.861648+0000","last_clean":"2024-09-19T04:44:55.861648+0000","last_became_active":"2024-09-19T04:44:28.338774+0000","last_became_peered":"2024-09-19T04:44:28.338774+0000","last_unstale":"2024-09-19T04:44:55.861648+0000","last_undegraded":"2024-09-19T04:44:55.861648+0000","last_fullsized":"2024-09-19T04:44:55.861648+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:12.261237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56927,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":168,"num_read_kb":146,"num_write":233,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":170781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247103,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481812,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":438830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710744,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":1080364,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874269,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":438760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":439138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364839,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1031658,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528366,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":438477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757299,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":1030943,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953527,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":438731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":438302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":489722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":489502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":439816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34357,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34357,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34357,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":51864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":52241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:45:03.377 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T04:45:03.378 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T04:45:03.378 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T04:45:03.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:03 smithi123 ceph-mon[30511]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:45:03.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:03 smithi193 ceph-mon[35592]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:45:03.785 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:03.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:03 smithi183 ceph-mon[35333]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:45:04.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.341+0000 7fc93ca00640 1 -- 172.21.15.123:0/224418533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9381002d0 msgr2=0x7fc9381026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.341+0000 7fc93ca00640 1 --2- 172.21.15.123:0/224418533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9381002d0 0x7fc9381026e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc928009ca0 tx=0x7fc92802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:45:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.341+0000 7fc93ca00640 1 -- 172.21.15.123:0/224418533 shutdown_connections 2024-09-19T04:45:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.341+0000 7fc93ca00640 1 --2- 172.21.15.123:0/224418533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9381055d0 0x7fc938105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.341+0000 7fc93ca00640 1 --2- 172.21.15.123:0/224418533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc938102c20 0x7fc938105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.341+0000 7fc93ca00640 1 --2- 172.21.15.123:0/224418533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9381002d0 0x7fc9381026e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.341+0000 7fc93ca00640 1 -- 172.21.15.123:0/224418533 >> 172.21.15.123:0/224418533 conn(0x7fc9380fc0e0 msgr2=0x7fc9380fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.341+0000 7fc93ca00640 1 -- 172.21.15.123:0/224418533 shutdown_connections 2024-09-19T04:45:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.341+0000 7fc93ca00640 1 -- 172.21.15.123:0/224418533 wait complete. 2024-09-19T04:45:04.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.342+0000 7fc93ca00640 1 Processor -- start 2024-09-19T04:45:04.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.342+0000 7fc93ca00640 1 -- start start 2024-09-19T04:45:04.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.342+0000 7fc93ca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9381002d0 0x7fc93819e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:04.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.342+0000 7fc93ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc938102c20 0x7fc93819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:04.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.342+0000 7fc93ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9381055d0 0x7fc9381a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:04.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.342+0000 7fc93ca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc938113d30 con 0x7fc938102c20 2024-09-19T04:45:04.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.342+0000 7fc93ca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc938113bb0 con 0x7fc9381055d0 2024-09-19T04:45:04.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.342+0000 7fc93ca00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc938113eb0 con 0x7fc9381002d0 2024-09-19T04:45:04.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.342+0000 7fc92ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc938102c20 0x7fc93819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:04.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.343+0000 7fc937e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9381055d0 0x7fc9381a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:04.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.343+0000 7fc92ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc938102c20 0x7fc93819e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55964/0 (socket says 172.21.15.123:55964) 2024-09-19T04:45:04.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.343+0000 7fc937400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9381002d0 0x7fc93819e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:04.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.343+0000 7fc92ea00640 1 -- 172.21.15.123:0/1432770455 learned_addr learned my addr 172.21.15.123:0/1432770455 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:04.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.343+0000 7fc937e00640 1 -- 172.21.15.123:0/1432770455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9381002d0 msgr2=0x7fc93819e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:04.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.343+0000 7fc937e00640 1 --2- 172.21.15.123:0/1432770455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9381002d0 0x7fc93819e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:04.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.343+0000 7fc937e00640 1 -- 172.21.15.123:0/1432770455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc938102c20 msgr2=0x7fc93819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:04.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.343+0000 7fc937e00640 1 --2- 172.21.15.123:0/1432770455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc938102c20 0x7fc93819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:04.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.343+0000 7fc937e00640 1 -- 172.21.15.123:0/1432770455 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9381a5f60 con 0x7fc9381055d0 2024-09-19T04:45:04.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.343+0000 7fc937e00640 1 --2- 172.21.15.123:0/1432770455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9381055d0 0x7fc9381a58f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc93000e970 tx=0x7fc93000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:04.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.343+0000 7fc934c00640 1 -- 172.21.15.123:0/1432770455 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93000ccc0 con 0x7fc9381055d0 2024-09-19T04:45:04.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.344+0000 7fc93ca00640 1 -- 172.21.15.123:0/1432770455 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9381a6250 con 0x7fc9381055d0 2024-09-19T04:45:04.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.344+0000 7fc934c00640 1 -- 172.21.15.123:0/1432770455 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc930005020 con 0x7fc9381055d0 2024-09-19T04:45:04.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.344+0000 7fc934c00640 1 -- 172.21.15.123:0/1432770455 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc930004a00 con 0x7fc9381055d0 2024-09-19T04:45:04.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.344+0000 7fc93ca00640 1 -- 172.21.15.123:0/1432770455 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9381a6740 con 0x7fc9381055d0 2024-09-19T04:45:04.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.345+0000 7fc93ca00640 1 -- 172.21.15.123:0/1432770455 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9380fdbd0 con 0x7fc9381055d0 2024-09-19T04:45:04.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.347+0000 7fc934c00640 1 -- 172.21.15.123:0/1432770455 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc9300051d0 con 0x7fc9381055d0 2024-09-19T04:45:04.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.348+0000 7fc934c00640 1 --2- 172.21.15.123:0/1432770455 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc91c0a3300 0x7fc91c0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:04.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.348+0000 7fc934c00640 1 -- 172.21.15.123:0/1432770455 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fc9300cf9e0 con 0x7fc9381055d0 2024-09-19T04:45:04.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.348+0000 7fc937400640 1 --2- 172.21.15.123:0/1432770455 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc91c0a3300 0x7fc91c0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.349+0000 7fc937400640 1 --2- 172.21.15.123:0/1432770455 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc91c0a3300 0x7fc91c0a57c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc928006010 tx=0x7fc928005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:04.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.350+0000 7fc934c00640 1 -- 172.21.15.123:0/1432770455 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc93009c6e0 con 0x7fc9381055d0 2024-09-19T04:45:04.592 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:04 smithi123 ceph-mon[30511]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 15 op/s 2024-09-19T04:45:04.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.591+0000 7fc93ca00640 1 -- 172.21.15.123:0/1432770455 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fc93802d7c0 con 0x7fc9381055d0 2024-09-19T04:45:04.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.598+0000 7fc934c00640 1 -- 172.21.15.123:0/1432770455 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fc93009bfe0 con 0x7fc9381055d0 2024-09-19T04:45:04.599 INFO:teuthology.orchestra.run.smithi123.stdout:[client.0] 2024-09-19T04:45:04.599 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBQrOtmbyFWIxAAkiYIjbNf4YRBOogTw6cwLg== 2024-09-19T04:45:04.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.600+0000 7fc93ca00640 1 -- 172.21.15.123:0/1432770455 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc91c0a3300 msgr2=0x7fc91c0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:04.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.600+0000 7fc93ca00640 1 --2- 172.21.15.123:0/1432770455 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc91c0a3300 0x7fc91c0a57c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc928006010 tx=0x7fc928005f50 comp rx=0 tx=0).stop 2024-09-19T04:45:04.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.600+0000 7fc93ca00640 1 -- 172.21.15.123:0/1432770455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9381055d0 msgr2=0x7fc9381a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:04.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.600+0000 7fc93ca00640 1 --2- 172.21.15.123:0/1432770455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9381055d0 0x7fc9381a58f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc93000e970 tx=0x7fc93000ee30 comp rx=0 tx=0).stop 2024-09-19T04:45:04.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.600+0000 7fc93ca00640 1 -- 172.21.15.123:0/1432770455 shutdown_connections 2024-09-19T04:45:04.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.600+0000 7fc93ca00640 1 --2- 172.21.15.123:0/1432770455 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc91c0a3300 0x7fc91c0a57c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:04.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.600+0000 7fc93ca00640 1 --2- 172.21.15.123:0/1432770455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9381055d0 0x7fc9381a58f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:04.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.600+0000 7fc93ca00640 1 --2- 172.21.15.123:0/1432770455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc938102c20 0x7fc93819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:04.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.600+0000 7fc93ca00640 1 --2- 172.21.15.123:0/1432770455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9381002d0 0x7fc93819e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:04.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.600+0000 7fc93ca00640 1 -- 172.21.15.123:0/1432770455 >> 172.21.15.123:0/1432770455 conn(0x7fc9380fc0e0 msgr2=0x7fc938103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:04.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.601+0000 7fc93ca00640 1 -- 172.21.15.123:0/1432770455 shutdown_connections 2024-09-19T04:45:04.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:04.601+0000 7fc93ca00640 1 -- 172.21.15.123:0/1432770455 wait complete. 2024-09-19T04:45:04.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:04 smithi193 ceph-mon[35592]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 15 op/s 2024-09-19T04:45:04.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:04 smithi183 ceph-mon[35333]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 15 op/s 2024-09-19T04:45:05.251 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:45:05.252 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T04:45:05.252 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T04:45:05.331 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T04:45:05.332 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T04:45:05.332 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T04:45:05.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:05 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1432770455' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:45:05.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:05 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:45:05.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:05 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T04:45:05.741 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:05 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1432770455' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:45:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:05 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:45:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:05 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T04:45:05.880 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:05 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1432770455' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:45:05.880 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:05 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:45:05.880 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:05 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T04:45:06.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.257+0000 7f2c82e00640 1 -- 172.21.15.123:0/216416324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c7c101de0 msgr2=0x7f2c7c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:06.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.257+0000 7f2c82e00640 1 --2- 172.21.15.123:0/216416324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c7c101de0 0x7f2c7c1021c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009900 tx=0x7f2c6c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:45:06.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.257+0000 7f2c82e00640 1 -- 172.21.15.123:0/216416324 shutdown_connections 2024-09-19T04:45:06.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.257+0000 7f2c82e00640 1 --2- 172.21.15.123:0/216416324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c7c106f10 0x7f2c7c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:06.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.257+0000 7f2c82e00640 1 --2- 172.21.15.123:0/216416324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c7c102790 0x7f2c7c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:06.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.257+0000 7f2c82e00640 1 --2- 172.21.15.123:0/216416324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c7c101de0 0x7f2c7c1021c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:06.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.257+0000 7f2c82e00640 1 -- 172.21.15.123:0/216416324 >> 172.21.15.123:0/216416324 conn(0x7f2c7c0fd8c0 msgr2=0x7f2c7c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:06.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.257+0000 7f2c82e00640 1 -- 172.21.15.123:0/216416324 shutdown_connections 2024-09-19T04:45:06.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.257+0000 7f2c82e00640 1 -- 172.21.15.123:0/216416324 wait complete. 2024-09-19T04:45:06.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.258+0000 7f2c82e00640 1 Processor -- start 2024-09-19T04:45:06.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.259+0000 7f2c82e00640 1 -- start start 2024-09-19T04:45:06.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.259+0000 7f2c82e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c7c101de0 0x7f2c7c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:06.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.259+0000 7f2c82e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c7c102790 0x7f2c7c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:06.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.259+0000 7f2c82e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c7c106f10 0x7f2c7c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:06.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.259+0000 7f2c82e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c111fc0 con 0x7f2c7c106f10 2024-09-19T04:45:06.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.259+0000 7f2c82e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c111e40 con 0x7f2c7c101de0 2024-09-19T04:45:06.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.259+0000 7f2c82e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c112140 con 0x7f2c7c102790 2024-09-19T04:45:06.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.259+0000 7f2c80e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c7c106f10 0x7f2c7c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:06.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.259+0000 7f2c7be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c7c101de0 0x7f2c7c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:06.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.259+0000 7f2c7be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c7c101de0 0x7f2c7c19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:60486/0 (socket says 172.21.15.123:60486) 2024-09-19T04:45:06.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.259+0000 7f2c7be00640 1 -- 172.21.15.123:0/1435016186 learned_addr learned my addr 172.21.15.123:0/1435016186 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:06.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.259+0000 7f2c7b400640 1 --2- 172.21.15.123:0/1435016186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c7c102790 0x7f2c7c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:06.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.260+0000 7f2c80e00640 1 -- 172.21.15.123:0/1435016186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c7c102790 msgr2=0x7f2c7c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:06.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.260+0000 7f2c80e00640 1 --2- 172.21.15.123:0/1435016186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c7c102790 0x7f2c7c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:06.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.260+0000 7f2c80e00640 1 -- 172.21.15.123:0/1435016186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c7c101de0 msgr2=0x7f2c7c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:06.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.260+0000 7f2c80e00640 1 --2- 172.21.15.123:0/1435016186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c7c101de0 0x7f2c7c19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:06.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.260+0000 7f2c80e00640 1 -- 172.21.15.123:0/1435016186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7c19f600 con 0x7f2c7c106f10 2024-09-19T04:45:06.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.260+0000 7f2c7be00640 1 --2- 172.21.15.123:0/1435016186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c7c101de0 0x7f2c7c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:06.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.260+0000 7f2c80e00640 1 --2- 172.21.15.123:0/1435016186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c7c106f10 0x7f2c7c19ef20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400e940 tx=0x7f2c7400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:06.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.260+0000 7f2c78c00640 1 -- 172.21.15.123:0/1435016186 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74004d20 con 0x7f2c7c106f10 2024-09-19T04:45:06.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.260+0000 7f2c78c00640 1 -- 172.21.15.123:0/1435016186 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c74005020 con 0x7f2c7c106f10 2024-09-19T04:45:06.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.260+0000 7f2c82e00640 1 -- 172.21.15.123:0/1435016186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7c19f8f0 con 0x7f2c7c106f10 2024-09-19T04:45:06.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.260+0000 7f2c78c00640 1 -- 172.21.15.123:0/1435016186 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c740116b0 con 0x7f2c7c106f10 2024-09-19T04:45:06.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.260+0000 7f2c82e00640 1 -- 172.21.15.123:0/1435016186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c7c1a71d0 con 0x7f2c7c106f10 2024-09-19T04:45:06.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.261+0000 7f2c82e00640 1 -- 172.21.15.123:0/1435016186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c7c1021c0 con 0x7f2c7c106f10 2024-09-19T04:45:06.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.263+0000 7f2c78c00640 1 -- 172.21.15.123:0/1435016186 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f2c740054d0 con 0x7f2c7c106f10 2024-09-19T04:45:06.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.264+0000 7f2c78c00640 1 --2- 172.21.15.123:0/1435016186 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f2c600a3350 0x7f2c600a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:06.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.265+0000 7f2c78c00640 1 -- 172.21.15.123:0/1435016186 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f2c740cfd30 con 0x7f2c7c106f10 2024-09-19T04:45:06.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.265+0000 7f2c7be00640 1 --2- 172.21.15.123:0/1435016186 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f2c600a3350 0x7f2c600a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:06.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.265+0000 7f2c7be00640 1 --2- 172.21.15.123:0/1435016186 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f2c600a3350 0x7f2c600a5810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c0096c0 tx=0x7f2c6c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:06.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.267+0000 7f2c78c00640 1 -- 172.21.15.123:0/1435016186 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2c74016030 con 0x7f2c7c106f10 2024-09-19T04:45:06.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.491+0000 7f2c82e00640 1 -- 172.21.15.123:0/1435016186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f2c7c19b9b0 con 0x7f2c7c106f10 2024-09-19T04:45:06.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.496+0000 7f2c78c00640 1 -- 172.21.15.123:0/1435016186 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264059 (secure 0 0 0) 0x7f2c7c19b9b0 con 0x7f2c7c106f10 2024-09-19T04:45:06.501 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:45:06.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.500+0000 7f2c82e00640 1 -- 172.21.15.123:0/1435016186 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f2c600a3350 msgr2=0x7f2c600a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:06.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.500+0000 7f2c82e00640 1 --2- 172.21.15.123:0/1435016186 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f2c600a3350 0x7f2c600a5810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c0096c0 tx=0x7f2c6c0046c0 comp rx=0 tx=0).stop 2024-09-19T04:45:06.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.500+0000 7f2c82e00640 1 -- 172.21.15.123:0/1435016186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c7c106f10 msgr2=0x7f2c7c19ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:06.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.500+0000 7f2c82e00640 1 --2- 172.21.15.123:0/1435016186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c7c106f10 0x7f2c7c19ef20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400e940 tx=0x7f2c7400ee00 comp rx=0 tx=0).stop 2024-09-19T04:45:06.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.500+0000 7f2c82e00640 1 -- 172.21.15.123:0/1435016186 shutdown_connections 2024-09-19T04:45:06.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.500+0000 7f2c82e00640 1 --2- 172.21.15.123:0/1435016186 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f2c600a3350 0x7f2c600a5810 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:06.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.500+0000 7f2c82e00640 1 --2- 172.21.15.123:0/1435016186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c7c106f10 0x7f2c7c19ef20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:06.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.500+0000 7f2c82e00640 1 --2- 172.21.15.123:0/1435016186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c7c102790 0x7f2c7c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:06.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.500+0000 7f2c82e00640 1 --2- 172.21.15.123:0/1435016186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c7c101de0 0x7f2c7c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:06.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.500+0000 7f2c82e00640 1 -- 172.21.15.123:0/1435016186 >> 172.21.15.123:0/1435016186 conn(0x7f2c7c0fd8c0 msgr2=0x7f2c7c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:06.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.500+0000 7f2c82e00640 1 -- 172.21.15.123:0/1435016186 shutdown_connections 2024-09-19T04:45:06.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:06.501+0000 7f2c82e00640 1 -- 172.21.15.123:0/1435016186 wait complete. 2024-09-19T04:45:06.857 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:06 smithi123 ceph-mon[30511]: pgmap v310: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-19T04:45:06.857 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:06 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1435016186' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T04:45:06.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:06 smithi193 ceph-mon[35592]: pgmap v310: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-19T04:45:06.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:06 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1435016186' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T04:45:07.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:06 smithi183 ceph-mon[35333]: pgmap v310: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-19T04:45:07.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:06 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1435016186' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T04:45:07.269 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1532622003},{"type":"v1","addr":"172.21.15.123:6801","nonce":1532622003}]},"active_addr":"172.21.15.123:6801/1532622003","active_change":"2024-09-19T04:37:13.257954+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24115,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.123:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":2833919666}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":2764863023}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":3244738342}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":3339335756}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":3620716937}]}]} 2024-09-19T04:45:07.274 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T04:45:07.274 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T04:45:07.275 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:45:07.642 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:08.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.202+0000 7f9e23600640 1 -- 172.21.15.123:0/659170687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1c074620 msgr2=0x7f9e1c074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:08.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.202+0000 7f9e23600640 1 --2- 172.21.15.123:0/659170687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1c074620 0x7f9e1c074a00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009900 tx=0x7f9e0c02a480 comp rx=0 tx=0).stop 2024-09-19T04:45:08.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.203+0000 7f9e23600640 1 -- 172.21.15.123:0/659170687 shutdown_connections 2024-09-19T04:45:08.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.203+0000 7f9e23600640 1 --2- 172.21.15.123:0/659170687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e1c10e3c0 0x7f9e1c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:08.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.203+0000 7f9e23600640 1 --2- 172.21.15.123:0/659170687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e1c074f40 0x7f9e1c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:08.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.203+0000 7f9e23600640 1 --2- 172.21.15.123:0/659170687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1c074620 0x7f9e1c074a00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:08.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.203+0000 7f9e23600640 1 -- 172.21.15.123:0/659170687 >> 172.21.15.123:0/659170687 conn(0x7f9e1c0fd8c0 msgr2=0x7f9e1c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:08.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.203+0000 7f9e23600640 1 -- 172.21.15.123:0/659170687 shutdown_connections 2024-09-19T04:45:08.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.203+0000 7f9e23600640 1 -- 172.21.15.123:0/659170687 wait complete. 2024-09-19T04:45:08.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.204+0000 7f9e23600640 1 Processor -- start 2024-09-19T04:45:08.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.204+0000 7f9e23600640 1 -- start start 2024-09-19T04:45:08.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.204+0000 7f9e23600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e1c074620 0x7f9e1c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:08.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.204+0000 7f9e23600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1c074f40 0x7f9e1c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:08.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.204+0000 7f9e23600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e1c10e3c0 0x7f9e1c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:08.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.204+0000 7f9e23600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c1141c0 con 0x7f9e1c074f40 2024-09-19T04:45:08.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.204+0000 7f9e23600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c114040 con 0x7f9e1c10e3c0 2024-09-19T04:45:08.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.204+0000 7f9e23600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c114340 con 0x7f9e1c074620 2024-09-19T04:45:08.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.204+0000 7f9e1be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1c074f40 0x7f9e1c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:08.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.204+0000 7f9e1be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1c074f40 0x7f9e1c19abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56006/0 (socket says 172.21.15.123:56006) 2024-09-19T04:45:08.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.204+0000 7f9e1be00640 1 -- 172.21.15.123:0/3766003183 learned_addr learned my addr 172.21.15.123:0/3766003183 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:08.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.205+0000 7f9e20c00640 1 --2- 172.21.15.123:0/3766003183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e1c074620 0x7f9e1c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:08.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.205+0000 7f9e21600640 1 --2- 172.21.15.123:0/3766003183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e1c10e3c0 0x7f9e1c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:08.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.205+0000 7f9e1be00640 1 -- 172.21.15.123:0/3766003183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e1c074620 msgr2=0x7f9e1c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:08.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.205+0000 7f9e1be00640 1 --2- 172.21.15.123:0/3766003183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e1c074620 0x7f9e1c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:08.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.205+0000 7f9e1be00640 1 -- 172.21.15.123:0/3766003183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e1c10e3c0 msgr2=0x7f9e1c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:08.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.205+0000 7f9e1be00640 1 --2- 172.21.15.123:0/3766003183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e1c10e3c0 0x7f9e1c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:08.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.205+0000 7f9e1be00640 1 -- 172.21.15.123:0/3766003183 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c19f650 con 0x7f9e1c074f40 2024-09-19T04:45:08.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.205+0000 7f9e21600640 1 --2- 172.21.15.123:0/3766003183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e1c10e3c0 0x7f9e1c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:08.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.205+0000 7f9e1be00640 1 --2- 172.21.15.123:0/3766003183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1c074f40 0x7f9e1c19abe0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800e9f0 tx=0x7f9e0800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:08.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.206+0000 7f9e19600640 1 -- 172.21.15.123:0/3766003183 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0800cd80 con 0x7f9e1c074f40 2024-09-19T04:45:08.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.206+0000 7f9e19600640 1 -- 172.21.15.123:0/3766003183 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e08005500 con 0x7f9e1c074f40 2024-09-19T04:45:08.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.206+0000 7f9e23600640 1 -- 172.21.15.123:0/3766003183 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1c19f940 con 0x7f9e1c074f40 2024-09-19T04:45:08.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.206+0000 7f9e19600640 1 -- 172.21.15.123:0/3766003183 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e08010690 con 0x7f9e1c074f40 2024-09-19T04:45:08.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.206+0000 7f9e23600640 1 -- 172.21.15.123:0/3766003183 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e1c1a7220 con 0x7f9e1c074f40 2024-09-19T04:45:08.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.207+0000 7f9e23600640 1 -- 172.21.15.123:0/3766003183 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e1c074a00 con 0x7f9e1c074f40 2024-09-19T04:45:08.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.209+0000 7f9e19600640 1 -- 172.21.15.123:0/3766003183 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9e080046c0 con 0x7f9e1c074f40 2024-09-19T04:45:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.210+0000 7f9e19600640 1 --2- 172.21.15.123:0/3766003183 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9e000a32d0 0x7f9e000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.210+0000 7f9e19600640 1 -- 172.21.15.123:0/3766003183 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f9e080cfdd0 con 0x7f9e1c074f40 2024-09-19T04:45:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.210+0000 7f9e20c00640 1 --2- 172.21.15.123:0/3766003183 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9e000a32d0 0x7f9e000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.211+0000 7f9e20c00640 1 --2- 172.21.15.123:0/3766003183 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9e000a32d0 0x7f9e000a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c000c00 tx=0x7f9e0c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:08.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.214+0000 7f9e19600640 1 -- 172.21.15.123:0/3766003183 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9e0809ca40 con 0x7f9e1c074f40 2024-09-19T04:45:08.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.392+0000 7f9e23600640 1 -- 172.21.15.123:0/3766003183 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e1c02d7c0 con 0x7f9e1c074f40 2024-09-19T04:45:08.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.394+0000 7f9e19600640 1 -- 172.21.15.123:0/3766003183 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22362 (secure 0 0 0) 0x7f9e0809c340 con 0x7f9e1c074f40 2024-09-19T04:45:08.396 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:45:08.396 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":84,"fsid":"911fa052-7640-11ef-bceb-c7b262605968","created":"2024-09-19T04:36:21.083333+0000","modified":"2024-09-19T04:44:31.766762+0000","last_up_change":"2024-09-19T04:43:33.818439+0000","last_in_change":"2024-09-19T04:43:10.148272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:39:33.856981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:44:26.956483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:44:29.718625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6803","nonce":3631596501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6805","nonce":3631596501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6809","nonce":3631596501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6807","nonce":3631596501}]},"public_addr":"172.21.15.123:6803/3631596501","cluster_addr":"172.21.15.123:6805/3631596501","heartbeat_back_addr":"172.21.15.123:6809/3631596501","heartbeat_front_addr":"172.21.15.123:6807/3631596501","state":["exists","up"]},{"osd":1,"uuid":"a526d4ce-586b-4d66-b720-e868df92fa24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6801","nonce":1259721614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6803","nonce":1259721614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6807","nonce":1259721614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6805","nonce":1259721614}]},"public_addr":"172.21.15.183:6801/1259721614","cluster_addr":"172.21.15.183:6803/1259721614","heartbeat_back_addr":"172.21.15.183:6807/1259721614","heartbeat_front_addr":"172.21.15.183:6805/1259721614","state":["exists","up"]},{"osd":2,"uuid":"3650601a-6259-4b2a-97df-1d47b09b0127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6801","nonce":2708151726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6803","nonce":2708151726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6807","nonce":2708151726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6805","nonce":2708151726}]},"public_addr":"172.21.15.193:6801/2708151726","cluster_addr":"172.21.15.193:6803/2708151726","heartbeat_back_addr":"172.21.15.193:6807/2708151726","heartbeat_front_addr":"172.21.15.193:6805/2708151726","state":["exists","up"]},{"osd":3,"uuid":"67c485ad-136e-4020-950d-9171a3c2de35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6811","nonce":3214806282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6813","nonce":3214806282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6817","nonce":3214806282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6815","nonce":3214806282}]},"public_addr":"172.21.15.123:6811/3214806282","cluster_addr":"172.21.15.123:6813/3214806282","heartbeat_back_addr":"172.21.15.123:6817/3214806282","heartbeat_front_addr":"172.21.15.123:6815/3214806282","state":["exists","up"]},{"osd":4,"uuid":"c9098540-06ab-45a8-9cf9-f90519cc852e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6809","nonce":667618665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6811","nonce":667618665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6815","nonce":667618665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6813","nonce":667618665}]},"public_addr":"172.21.15.183:6809/667618665","cluster_addr":"172.21.15.183:6811/667618665","heartbeat_back_addr":"172.21.15.183:6815/667618665","heartbeat_front_addr":"172.21.15.183:6813/667618665","state":["exists","up"]},{"osd":5,"uuid":"278a8dbd-6492-42d4-b489-1f832350bc6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6809","nonce":2292251639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6811","nonce":2292251639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6815","nonce":2292251639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6813","nonce":2292251639}]},"public_addr":"172.21.15.193:6809/2292251639","cluster_addr":"172.21.15.193:6811/2292251639","heartbeat_back_addr":"172.21.15.193:6815/2292251639","heartbeat_front_addr":"172.21.15.193:6813/2292251639","state":["exists","up"]},{"osd":6,"uuid":"0fda4e39-6adb-4941-a39b-a9a15019088c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6819","nonce":226007499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6821","nonce":226007499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6825","nonce":226007499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6823","nonce":226007499}]},"public_addr":"172.21.15.123:6819/226007499","cluster_addr":"172.21.15.123:6821/226007499","heartbeat_back_addr":"172.21.15.123:6825/226007499","heartbeat_front_addr":"172.21.15.123:6823/226007499","state":["exists","up"]},{"osd":7,"uuid":"5c0a021e-9f3b-4766-9686-0ef24b68cf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6817","nonce":3718518226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6819","nonce":3718518226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6823","nonce":3718518226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6821","nonce":3718518226}]},"public_addr":"172.21.15.183:6817/3718518226","cluster_addr":"172.21.15.183:6819/3718518226","heartbeat_back_addr":"172.21.15.183:6823/3718518226","heartbeat_front_addr":"172.21.15.183:6821/3718518226","state":["exists","up"]},{"osd":8,"uuid":"9b0e67e0-6305-4f81-9211-d41960099a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6817","nonce":3551404118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6819","nonce":3551404118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6823","nonce":3551404118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6821","nonce":3551404118}]},"public_addr":"172.21.15.193:6817/3551404118","cluster_addr":"172.21.15.193:6819/3551404118","heartbeat_back_addr":"172.21.15.193:6823/3551404118","heartbeat_front_addr":"172.21.15.193:6821/3551404118","state":["exists","up"]},{"osd":9,"uuid":"978ef3fa-c6ec-421f-b194-56d44813be5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6827","nonce":1439901149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6829","nonce":1439901149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6833","nonce":1439901149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6831","nonce":1439901149}]},"public_addr":"172.21.15.123:6827/1439901149","cluster_addr":"172.21.15.123:6829/1439901149","heartbeat_back_addr":"172.21.15.123:6833/1439901149","heartbeat_front_addr":"172.21.15.123:6831/1439901149","state":["exists","up"]},{"osd":10,"uuid":"5495e09b-7809-47b0-b7e0-b56c452f9f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6825","nonce":3355349245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6827","nonce":3355349245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6831","nonce":3355349245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6829","nonce":3355349245}]},"public_addr":"172.21.15.183:6825/3355349245","cluster_addr":"172.21.15.183:6827/3355349245","heartbeat_back_addr":"172.21.15.183:6831/3355349245","heartbeat_front_addr":"172.21.15.183:6829/3355349245","state":["exists","up"]},{"osd":11,"uuid":"03046fef-b16b-42f8-9a19-6c0b78cbfe7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6825","nonce":4291617038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6827","nonce":4291617038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6831","nonce":4291617038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6829","nonce":4291617038}]},"public_addr":"172.21.15.193:6825/4291617038","cluster_addr":"172.21.15.193:6827/4291617038","heartbeat_back_addr":"172.21.15.193:6831/4291617038","heartbeat_front_addr":"172.21.15.193:6829/4291617038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:38:39.142529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:05.336911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:30.310967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:59.655973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:25.890576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:50.778689+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:19.063731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:45.424269+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:10.562149+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:40.382368+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:06.297731+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:32.352172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/4040401634":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/3498196650":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6800/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:6800/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/471028951":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/1009496020":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/2000593686":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/2580747029":"2024-09-20T04:36:45.913060+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:45:08.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.396+0000 7f9e23600640 1 -- 172.21.15.123:0/3766003183 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9e000a32d0 msgr2=0x7f9e000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:08.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.396+0000 7f9e23600640 1 --2- 172.21.15.123:0/3766003183 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9e000a32d0 0x7f9e000a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c000c00 tx=0x7f9e0c005b50 comp rx=0 tx=0).stop 2024-09-19T04:45:08.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.396+0000 7f9e23600640 1 -- 172.21.15.123:0/3766003183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1c074f40 msgr2=0x7f9e1c19abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:08.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.396+0000 7f9e23600640 1 --2- 172.21.15.123:0/3766003183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1c074f40 0x7f9e1c19abe0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800e9f0 tx=0x7f9e0800edf0 comp rx=0 tx=0).stop 2024-09-19T04:45:08.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.397+0000 7f9e23600640 1 -- 172.21.15.123:0/3766003183 shutdown_connections 2024-09-19T04:45:08.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.397+0000 7f9e23600640 1 --2- 172.21.15.123:0/3766003183 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9e000a32d0 0x7f9e000a5790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:08.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.397+0000 7f9e23600640 1 --2- 172.21.15.123:0/3766003183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e1c10e3c0 0x7f9e1c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:08.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.397+0000 7f9e23600640 1 --2- 172.21.15.123:0/3766003183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1c074f40 0x7f9e1c19abe0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:08.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.397+0000 7f9e23600640 1 --2- 172.21.15.123:0/3766003183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e1c074620 0x7f9e1c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:08.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.397+0000 7f9e23600640 1 -- 172.21.15.123:0/3766003183 >> 172.21.15.123:0/3766003183 conn(0x7f9e1c0fd8c0 msgr2=0x7f9e1c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:08.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.397+0000 7f9e23600640 1 -- 172.21.15.123:0/3766003183 shutdown_connections 2024-09-19T04:45:08.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:08.397+0000 7f9e23600640 1 -- 172.21.15.123:0/3766003183 wait complete. 2024-09-19T04:45:08.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:08 smithi123 ceph-mon[30511]: pgmap v311: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-19T04:45:08.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:08 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3766003183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:45:08.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:08 smithi193 ceph-mon[35592]: pgmap v311: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-19T04:45:08.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:08 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3766003183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:45:09.095 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T04:45:09.095 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:45:09.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:08 smithi183 ceph-mon[35333]: pgmap v311: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-19T04:45:09.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:08 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3766003183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:45:09.464 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:10.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.009+0000 7f8b7be00640 1 -- 172.21.15.123:0/2301719708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b7410e3c0 msgr2=0x7f8b74110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:10.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.009+0000 7f8b7be00640 1 --2- 172.21.15.123:0/2301719708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b7410e3c0 0x7f8b74110810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c009ca0 tx=0x7f8b6c02a950 comp rx=0 tx=0).stop 2024-09-19T04:45:10.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.009+0000 7f8b7be00640 1 -- 172.21.15.123:0/2301719708 shutdown_connections 2024-09-19T04:45:10.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.009+0000 7f8b7be00640 1 --2- 172.21.15.123:0/2301719708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b7410e3c0 0x7f8b74110810 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:10.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.009+0000 7f8b7be00640 1 --2- 172.21.15.123:0/2301719708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b74074f40 0x7f8b7410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:10.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.009+0000 7f8b7be00640 1 --2- 172.21.15.123:0/2301719708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b74074620 0x7f8b74074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:10.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.010+0000 7f8b7be00640 1 -- 172.21.15.123:0/2301719708 >> 172.21.15.123:0/2301719708 conn(0x7f8b740fd8c0 msgr2=0x7f8b740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:10.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.010+0000 7f8b7be00640 1 -- 172.21.15.123:0/2301719708 shutdown_connections 2024-09-19T04:45:10.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.010+0000 7f8b7be00640 1 -- 172.21.15.123:0/2301719708 wait complete. 2024-09-19T04:45:10.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.010+0000 7f8b7be00640 1 Processor -- start 2024-09-19T04:45:10.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.011+0000 7f8b7be00640 1 -- start start 2024-09-19T04:45:10.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.011+0000 7f8b7be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b74074620 0x7f8b7419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:10.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.011+0000 7f8b7be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b74074f40 0x7f8b7419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:10.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.011+0000 7f8b7be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b7410e3c0 0x7f8b7419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:10.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.011+0000 7f8b7be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b74114170 con 0x7f8b7410e3c0 2024-09-19T04:45:10.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.011+0000 7f8b7be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8b74113ff0 con 0x7f8b74074f40 2024-09-19T04:45:10.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.011+0000 7f8b7be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741142f0 con 0x7f8b74074620 2024-09-19T04:45:10.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.011+0000 7f8b79e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b7410e3c0 0x7f8b7419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:10.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.011+0000 7f8b79e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b7410e3c0 0x7f8b7419ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56018/0 (socket says 172.21.15.123:56018) 2024-09-19T04:45:10.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.011+0000 7f8b79e00640 1 -- 172.21.15.123:0/594338755 learned_addr learned my addr 172.21.15.123:0/594338755 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:10.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.011+0000 7f8b78a00640 1 --2- 172.21.15.123:0/594338755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b74074f40 0x7f8b7419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:10.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.012+0000 7f8b79400640 1 --2- 172.21.15.123:0/594338755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b74074620 0x7f8b7419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:10.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.012+0000 7f8b78a00640 1 -- 172.21.15.123:0/594338755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b74074620 msgr2=0x7f8b7419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:10.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.012+0000 7f8b78a00640 1 --2- 172.21.15.123:0/594338755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b74074620 0x7f8b7419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:10.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.012+0000 7f8b78a00640 1 -- 172.21.15.123:0/594338755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b7410e3c0 msgr2=0x7f8b7419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:10.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.012+0000 7f8b78a00640 1 --2- 172.21.15.123:0/594338755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b7410e3c0 0x7f8b7419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:10.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.012+0000 7f8b78a00640 1 -- 172.21.15.123:0/594338755 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7419f620 con 0x7f8b74074f40 2024-09-19T04:45:10.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.012+0000 7f8b79e00640 1 --2- 172.21.15.123:0/594338755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b7410e3c0 0x7f8b7419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:10.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.012+0000 7f8b78a00640 1 --2- 172.21.15.123:0/594338755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b74074f40 0x7f8b7419abb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000e9f0 tx=0x7f8b6000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:10.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.012+0000 7f8b72000640 1 -- 172.21.15.123:0/594338755 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6000ccc0 con 0x7f8b74074f40 2024-09-19T04:45:10.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.012+0000 7f8b72000640 1 -- 172.21.15.123:0/594338755 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b60005020 con 0x7f8b74074f40 2024-09-19T04:45:10.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.012+0000 7f8b7be00640 1 -- 172.21.15.123:0/594338755 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7419f910 con 0x7f8b74074f40 2024-09-19T04:45:10.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.013+0000 7f8b72000640 1 -- 172.21.15.123:0/594338755 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b600049e0 con 0x7f8b74074f40 2024-09-19T04:45:10.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.013+0000 7f8b7be00640 1 -- 172.21.15.123:0/594338755 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b741a71f0 con 0x7f8b74074f40 2024-09-19T04:45:10.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.014+0000 7f8b7be00640 1 -- 172.21.15.123:0/594338755 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b74074a00 con 0x7f8b74074f40 2024-09-19T04:45:10.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.016+0000 7f8b72000640 1 -- 172.21.15.123:0/594338755 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8b60005560 con 0x7f8b74074f40 2024-09-19T04:45:10.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.017+0000 7f8b72000640 1 --2- 172.21.15.123:0/594338755 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8b5c0a32d0 0x7f8b5c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:10.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.018+0000 7f8b72000640 1 -- 172.21.15.123:0/594338755 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f8b600cfbe0 con 0x7f8b74074f40 2024-09-19T04:45:10.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.018+0000 7f8b79400640 1 --2- 172.21.15.123:0/594338755 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8b5c0a32d0 0x7f8b5c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:10.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.018+0000 7f8b79400640 1 --2- 172.21.15.123:0/594338755 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8b5c0a32d0 0x7f8b5c0a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009bd0 tx=0x7f8b68005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:10.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.019+0000 7f8b72000640 1 -- 172.21.15.123:0/594338755 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8b600051d0 con 0x7f8b74074f40 2024-09-19T04:45:10.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.197+0000 7f8b7be00640 1 -- 172.21.15.123:0/594338755 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b7419b9d0 con 0x7f8b74074f40 2024-09-19T04:45:10.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.199+0000 7f8b72000640 1 -- 172.21.15.123:0/594338755 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22362 (secure 0 0 0) 0x7f8b7419b9d0 con 0x7f8b74074f40 2024-09-19T04:45:10.201 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:45:10.201 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":84,"fsid":"911fa052-7640-11ef-bceb-c7b262605968","created":"2024-09-19T04:36:21.083333+0000","modified":"2024-09-19T04:44:31.766762+0000","last_up_change":"2024-09-19T04:43:33.818439+0000","last_in_change":"2024-09-19T04:43:10.148272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:39:33.856981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:44:26.956483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:44:29.718625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6803","nonce":3631596501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6805","nonce":3631596501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6809","nonce":3631596501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6807","nonce":3631596501}]},"public_addr":"172.21.15.123:6803/3631596501","cluster_addr":"172.21.15.123:6805/3631596501","heartbeat_back_addr":"172.21.15.123:6809/3631596501","heartbeat_front_addr":"172.21.15.123:6807/3631596501","state":["exists","up"]},{"osd":1,"uuid":"a526d4ce-586b-4d66-b720-e868df92fa24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6801","nonce":1259721614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6803","nonce":1259721614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6807","nonce":1259721614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6805","nonce":1259721614}]},"public_addr":"172.21.15.183:6801/1259721614","cluster_addr":"172.21.15.183:6803/1259721614","heartbeat_back_addr":"172.21.15.183:6807/1259721614","heartbeat_front_addr":"172.21.15.183:6805/1259721614","state":["exists","up"]},{"osd":2,"uuid":"3650601a-6259-4b2a-97df-1d47b09b0127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6801","nonce":2708151726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6803","nonce":2708151726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6807","nonce":2708151726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6805","nonce":2708151726}]},"public_addr":"172.21.15.193:6801/2708151726","cluster_addr":"172.21.15.193:6803/2708151726","heartbeat_back_addr":"172.21.15.193:6807/2708151726","heartbeat_front_addr":"172.21.15.193:6805/2708151726","state":["exists","up"]},{"osd":3,"uuid":"67c485ad-136e-4020-950d-9171a3c2de35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6811","nonce":3214806282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6813","nonce":3214806282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6817","nonce":3214806282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6815","nonce":3214806282}]},"public_addr":"172.21.15.123:6811/3214806282","cluster_addr":"172.21.15.123:6813/3214806282","heartbeat_back_addr":"172.21.15.123:6817/3214806282","heartbeat_front_addr":"172.21.15.123:6815/3214806282","state":["exists","up"]},{"osd":4,"uuid":"c9098540-06ab-45a8-9cf9-f90519cc852e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6809","nonce":667618665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6811","nonce":667618665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6815","nonce":667618665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6813","nonce":667618665}]},"public_addr":"172.21.15.183:6809/667618665","cluster_addr":"172.21.15.183:6811/667618665","heartbeat_back_addr":"172.21.15.183:6815/667618665","heartbeat_front_addr":"172.21.15.183:6813/667618665","state":["exists","up"]},{"osd":5,"uuid":"278a8dbd-6492-42d4-b489-1f832350bc6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6809","nonce":2292251639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6811","nonce":2292251639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6815","nonce":2292251639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6813","nonce":2292251639}]},"public_addr":"172.21.15.193:6809/2292251639","cluster_addr":"172.21.15.193:6811/2292251639","heartbeat_back_addr":"172.21.15.193:6815/2292251639","heartbeat_front_addr":"172.21.15.193:6813/2292251639","state":["exists","up"]},{"osd":6,"uuid":"0fda4e39-6adb-4941-a39b-a9a15019088c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6819","nonce":226007499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6821","nonce":226007499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6825","nonce":226007499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6823","nonce":226007499}]},"public_addr":"172.21.15.123:6819/226007499","cluster_addr":"172.21.15.123:6821/226007499","heartbeat_back_addr":"172.21.15.123:6825/226007499","heartbeat_front_addr":"172.21.15.123:6823/226007499","state":["exists","up"]},{"osd":7,"uuid":"5c0a021e-9f3b-4766-9686-0ef24b68cf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6817","nonce":3718518226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6819","nonce":3718518226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6823","nonce":3718518226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6821","nonce":3718518226}]},"public_addr":"172.21.15.183:6817/3718518226","cluster_addr":"172.21.15.183:6819/3718518226","heartbeat_back_addr":"172.21.15.183:6823/3718518226","heartbeat_front_addr":"172.21.15.183:6821/3718518226","state":["exists","up"]},{"osd":8,"uuid":"9b0e67e0-6305-4f81-9211-d41960099a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6817","nonce":3551404118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6819","nonce":3551404118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6823","nonce":3551404118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6821","nonce":3551404118}]},"public_addr":"172.21.15.193:6817/3551404118","cluster_addr":"172.21.15.193:6819/3551404118","heartbeat_back_addr":"172.21.15.193:6823/3551404118","heartbeat_front_addr":"172.21.15.193:6821/3551404118","state":["exists","up"]},{"osd":9,"uuid":"978ef3fa-c6ec-421f-b194-56d44813be5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6827","nonce":1439901149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6829","nonce":1439901149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6833","nonce":1439901149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6831","nonce":1439901149}]},"public_addr":"172.21.15.123:6827/1439901149","cluster_addr":"172.21.15.123:6829/1439901149","heartbeat_back_addr":"172.21.15.123:6833/1439901149","heartbeat_front_addr":"172.21.15.123:6831/1439901149","state":["exists","up"]},{"osd":10,"uuid":"5495e09b-7809-47b0-b7e0-b56c452f9f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6825","nonce":3355349245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6827","nonce":3355349245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6831","nonce":3355349245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6829","nonce":3355349245}]},"public_addr":"172.21.15.183:6825/3355349245","cluster_addr":"172.21.15.183:6827/3355349245","heartbeat_back_addr":"172.21.15.183:6831/3355349245","heartbeat_front_addr":"172.21.15.183:6829/3355349245","state":["exists","up"]},{"osd":11,"uuid":"03046fef-b16b-42f8-9a19-6c0b78cbfe7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6825","nonce":4291617038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6827","nonce":4291617038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6831","nonce":4291617038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6829","nonce":4291617038}]},"public_addr":"172.21.15.193:6825/4291617038","cluster_addr":"172.21.15.193:6827/4291617038","heartbeat_back_addr":"172.21.15.193:6831/4291617038","heartbeat_front_addr":"172.21.15.193:6829/4291617038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:38:39.142529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:05.336911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:30.310967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:59.655973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:25.890576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:50.778689+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:19.063731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:45.424269+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:10.562149+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:40.382368+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:06.297731+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:32.352172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/4040401634":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/3498196650":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6800/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:6800/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/471028951":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/1009496020":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/2000593686":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/2580747029":"2024-09-20T04:36:45.913060+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:45:10.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.202+0000 7f8b7be00640 1 -- 172.21.15.123:0/594338755 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8b5c0a32d0 msgr2=0x7f8b5c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:10.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.202+0000 7f8b7be00640 1 --2- 172.21.15.123:0/594338755 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8b5c0a32d0 0x7f8b5c0a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009bd0 tx=0x7f8b68005f90 comp rx=0 tx=0).stop 2024-09-19T04:45:10.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.202+0000 7f8b7be00640 1 -- 172.21.15.123:0/594338755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b74074f40 msgr2=0x7f8b7419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:10.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.202+0000 7f8b7be00640 1 --2- 172.21.15.123:0/594338755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b74074f40 0x7f8b7419abb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000e9f0 tx=0x7f8b6000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:45:10.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.202+0000 7f8b7be00640 1 -- 172.21.15.123:0/594338755 shutdown_connections 2024-09-19T04:45:10.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.202+0000 7f8b7be00640 1 --2- 172.21.15.123:0/594338755 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8b5c0a32d0 0x7f8b5c0a5790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:10.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.202+0000 7f8b7be00640 1 --2- 172.21.15.123:0/594338755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b7410e3c0 0x7f8b7419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:10.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.202+0000 7f8b7be00640 1 --2- 172.21.15.123:0/594338755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b74074f40 0x7f8b7419abb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:10.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.202+0000 7f8b7be00640 1 --2- 172.21.15.123:0/594338755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b74074620 0x7f8b7419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:10.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.202+0000 7f8b7be00640 1 -- 172.21.15.123:0/594338755 >> 172.21.15.123:0/594338755 conn(0x7f8b740fd8c0 msgr2=0x7f8b740fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:10.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.202+0000 7f8b7be00640 1 -- 172.21.15.123:0/594338755 shutdown_connections 2024-09-19T04:45:10.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:10.202+0000 7f8b7be00640 1 -- 172.21.15.123:0/594338755 wait complete. 2024-09-19T04:45:10.867 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:10 smithi123 ceph-mon[30511]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-09-19T04:45:10.867 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:10 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/594338755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:45:10.908 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T04:45:10.909 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T04:45:10.909 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T04:45:10.909 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T04:45:10.910 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T04:45:10.910 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T04:45:10.910 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T04:45:10.911 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T04:45:10.912 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-19T04:45:10.912 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-19T04:45:10.912 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-19T04:45:10.913 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-19T04:45:10.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:10 smithi193 ceph-mon[35592]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-09-19T04:45:10.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:10 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/594338755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:45:11.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:10 smithi183 ceph-mon[35333]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-09-19T04:45:11.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:10 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/594338755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:45:11.880 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:11.897 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:11.932 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:11.978 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:12.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.624+0000 7fb796e00640 1 -- 172.21.15.123:0/154746423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7901079c0 msgr2=0x7fb790107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:12.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.624+0000 7fb796e00640 1 --2- 172.21.15.123:0/154746423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7901079c0 0x7fb790107da0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb784009860 tx=0x7fb78402a340 comp rx=0 tx=0).stop 2024-09-19T04:45:12.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.624+0000 7fb796e00640 1 -- 172.21.15.123:0/154746423 shutdown_connections 2024-09-19T04:45:12.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.624+0000 7fb796e00640 1 --2- 172.21.15.123:0/154746423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb790102360 0x7fb79010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:12.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.624+0000 7fb796e00640 1 --2- 172.21.15.123:0/154746423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7901019c0 0x7fb790101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:12.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.624+0000 7fb796e00640 1 --2- 172.21.15.123:0/154746423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7901079c0 0x7fb790107da0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:12.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.624+0000 7fb796e00640 1 -- 172.21.15.123:0/154746423 >> 172.21.15.123:0/154746423 conn(0x7fb7900fd740 msgr2=0x7fb7900ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:12.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.624+0000 7fb796e00640 1 -- 172.21.15.123:0/154746423 shutdown_connections 2024-09-19T04:45:12.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.624+0000 7fb796e00640 1 -- 172.21.15.123:0/154746423 wait complete. 2024-09-19T04:45:12.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.625+0000 7fb796e00640 1 Processor -- start 2024-09-19T04:45:12.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.625+0000 7fb796e00640 1 -- start start 2024-09-19T04:45:12.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.626+0000 7fb796e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7901019c0 0x7fb79019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:12.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.626+0000 7fb796e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb790102360 0x7fb79019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:12.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.626+0000 7fb796e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7901079c0 0x7fb79019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:12.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.626+0000 7fb796e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb790111dc0 con 0x7fb7901019c0 2024-09-19T04:45:12.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.626+0000 7fb796e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb790111c40 con 0x7fb790102360 2024-09-19T04:45:12.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.626+0000 7fb796e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb790111f40 con 0x7fb7901079c0 2024-09-19T04:45:12.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.626+0000 7fb795a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7901019c0 0x7fb79019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:12.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.626+0000 7fb795a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7901019c0 0x7fb79019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56298/0 (socket says 172.21.15.123:56298) 2024-09-19T04:45:12.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.626+0000 7fb795a00640 1 -- 172.21.15.123:0/592301704 learned_addr learned my addr 172.21.15.123:0/592301704 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:12.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.626+0000 7fb796400640 1 --2- 172.21.15.123:0/592301704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7901079c0 0x7fb79019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:12.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.626+0000 7fb795000640 1 --2- 172.21.15.123:0/592301704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb790102360 0x7fb79019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:12.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.626+0000 7fb795a00640 1 -- 172.21.15.123:0/592301704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7901079c0 msgr2=0x7fb79019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:12.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.626+0000 7fb795a00640 1 --2- 172.21.15.123:0/592301704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7901079c0 0x7fb79019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:12.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.626+0000 7fb795a00640 1 -- 172.21.15.123:0/592301704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb790102360 msgr2=0x7fb79019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:12.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.626+0000 7fb795a00640 1 --2- 172.21.15.123:0/592301704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb790102360 0x7fb79019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:12.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.626+0000 7fb795a00640 1 -- 172.21.15.123:0/592301704 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79019f470 con 0x7fb7901019c0 2024-09-19T04:45:12.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.627+0000 7fb795a00640 1 --2- 172.21.15.123:0/592301704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7901019c0 0x7fb79019a4c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb784035040 tx=0x7fb784033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:12.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.627+0000 7fb78e000640 1 -- 172.21.15.123:0/592301704 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78403f020 con 0x7fb7901019c0 2024-09-19T04:45:12.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.627+0000 7fb796e00640 1 -- 172.21.15.123:0/592301704 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79019f700 con 0x7fb7901019c0 2024-09-19T04:45:12.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.627+0000 7fb78e000640 1 -- 172.21.15.123:0/592301704 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb784004040 con 0x7fb7901019c0 2024-09-19T04:45:12.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.627+0000 7fb78e000640 1 -- 172.21.15.123:0/592301704 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784037040 con 0x7fb7901019c0 2024-09-19T04:45:12.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.627+0000 7fb796e00640 1 -- 172.21.15.123:0/592301704 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7901a7040 con 0x7fb7901019c0 2024-09-19T04:45:12.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.628+0000 7fb796e00640 1 -- 172.21.15.123:0/592301704 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb7900ff8a0 con 0x7fb7901019c0 2024-09-19T04:45:12.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.630+0000 7fb78e000640 1 -- 172.21.15.123:0/592301704 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb784042020 con 0x7fb7901019c0 2024-09-19T04:45:12.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.631+0000 7fb78e000640 1 --2- 172.21.15.123:0/592301704 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fb7780a3350 0x7fb7780a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:12.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.631+0000 7fb78e000640 1 -- 172.21.15.123:0/592301704 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fb7840efa00 con 0x7fb7901019c0 2024-09-19T04:45:12.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.632+0000 7fb795000640 1 --2- 172.21.15.123:0/592301704 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fb7780a3350 0x7fb7780a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:12.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.632+0000 7fb78e000640 1 --2- 172.21.15.123:0/592301704 >> [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501] conn(0x7fb7780acba0 0x7fb7780aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:12.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.632+0000 7fb78e000640 1 -- 172.21.15.123:0/592301704 --> [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb78400b0b0 con 0x7fb7780acba0 2024-09-19T04:45:12.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.632+0000 7fb78e000640 1 -- 172.21.15.123:0/592301704 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fb7840f6050 con 0x7fb7901019c0 2024-09-19T04:45:12.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.632+0000 7fb796400640 1 --2- 172.21.15.123:0/592301704 >> [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501] conn(0x7fb7780acba0 0x7fb7780aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:12.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.632+0000 7fb795000640 1 --2- 172.21.15.123:0/592301704 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fb7780a3350 0x7fb7780a5810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb79010ec90 tx=0x7fb77c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:12.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.633+0000 7fb796400640 1 --2- 172.21.15.123:0/592301704 >> [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501] conn(0x7fb7780acba0 0x7fb7780aefc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:12.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.634+0000 7fb78e000640 1 -- 172.21.15.123:0/592301704 <== osd.0 v2:172.21.15.123:6802/3631596501 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb78400b0b0 con 0x7fb7780acba0 2024-09-19T04:45:12.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.648+0000 7fb796e00640 1 -- 172.21.15.123:0/592301704 --> [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb790103100 con 0x7fb7780acba0 2024-09-19T04:45:12.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.648+0000 7fb78e000640 1 -- 172.21.15.123:0/592301704 <== osd.0 v2:172.21.15.123:6802/3631596501 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb790103100 con 0x7fb7780acba0 2024-09-19T04:45:12.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.649+0000 7fb796e00640 1 -- 172.21.15.123:0/592301704 >> [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501] conn(0x7fb7780acba0 msgr2=0x7fb7780aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:12.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.649+0000 7fb796e00640 1 --2- 172.21.15.123:0/592301704 >> [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501] conn(0x7fb7780acba0 0x7fb7780aefc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:12.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.649+0000 7fb796e00640 1 -- 172.21.15.123:0/592301704 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fb7780a3350 msgr2=0x7fb7780a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:12.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.649+0000 7fb796e00640 1 --2- 172.21.15.123:0/592301704 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fb7780a3350 0x7fb7780a5810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb79010ec90 tx=0x7fb77c005f90 comp rx=0 tx=0).stop 2024-09-19T04:45:12.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.649+0000 7fb796e00640 1 -- 172.21.15.123:0/592301704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7901019c0 msgr2=0x7fb79019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:12.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.649+0000 7fb796e00640 1 --2- 172.21.15.123:0/592301704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7901019c0 0x7fb79019a4c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb784035040 tx=0x7fb784033b60 comp rx=0 tx=0).stop 2024-09-19T04:45:12.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.649+0000 7fb796400640 1 -- 172.21.15.123:0/592301704 reap_dead start 2024-09-19T04:45:12.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.649+0000 7fb796e00640 1 -- 172.21.15.123:0/592301704 shutdown_connections 2024-09-19T04:45:12.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.649+0000 7fb796e00640 1 -- 172.21.15.123:0/592301704 >> 172.21.15.123:0/592301704 conn(0x7fb7900fd740 msgr2=0x7fb7900fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:12.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.649+0000 7fb796e00640 1 -- 172.21.15.123:0/592301704 shutdown_connections 2024-09-19T04:45:12.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:12.649+0000 7fb796e00640 1 -- 172.21.15.123:0/592301704 wait complete. 2024-09-19T04:45:12.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:12 smithi123 ceph-mon[30511]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T04:45:12.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:12 smithi193 ceph-mon[35592]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T04:45:13.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:12 smithi183 ceph-mon[35333]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T04:45:13.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.178+0000 7f32fda00640 1 -- 172.21.15.123:0/1216126770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32f8101de0 msgr2=0x7f32f81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:13.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.178+0000 7f32fda00640 1 --2- 172.21.15.123:0/1216126770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32f8101de0 0x7f32f81021c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f32e8009900 tx=0x7f32e802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:45:13.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.178+0000 7f32fda00640 1 -- 172.21.15.123:0/1216126770 shutdown_connections 2024-09-19T04:45:13.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.178+0000 7f32fda00640 1 --2- 172.21.15.123:0/1216126770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32f8106f10 0x7f32f810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:13.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.178+0000 7f32fda00640 1 --2- 172.21.15.123:0/1216126770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32f8102790 0x7f32f81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:13.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.178+0000 7f32fda00640 1 --2- 172.21.15.123:0/1216126770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32f8101de0 0x7f32f81021c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:13.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.178+0000 7f32fda00640 1 -- 172.21.15.123:0/1216126770 >> 172.21.15.123:0/1216126770 conn(0x7f32f80fd8c0 msgr2=0x7f32f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:13.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.178+0000 7f32fda00640 1 -- 172.21.15.123:0/1216126770 shutdown_connections 2024-09-19T04:45:13.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.178+0000 7f32fda00640 1 -- 172.21.15.123:0/1216126770 wait complete. 2024-09-19T04:45:13.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.179+0000 7f32fda00640 1 Processor -- start 2024-09-19T04:45:13.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.179+0000 7f32fda00640 1 -- start start 2024-09-19T04:45:13.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.179+0000 7f32fda00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32f8101de0 0x7f32f819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:13.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.179+0000 7f32fda00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32f8102790 0x7f32f819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:13.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.179+0000 7f32fda00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32f8106f10 0x7f32f819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:13.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.179+0000 7f32fda00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8111fc0 con 0x7f32f8106f10 2024-09-19T04:45:13.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.179+0000 7f32fda00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8111e40 con 0x7f32f8101de0 2024-09-19T04:45:13.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.179+0000 7f32fda00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8112140 con 0x7f32f8102790 2024-09-19T04:45:13.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.179+0000 7f32f7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32f8106f10 0x7f32f819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:13.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.179+0000 7f32f6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32f8101de0 0x7f32f819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:13.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.179+0000 7f32f7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32f8106f10 0x7f32f819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56314/0 (socket says 172.21.15.123:56314) 2024-09-19T04:45:13.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.179+0000 7f32f7400640 1 -- 172.21.15.123:0/3359086460 learned_addr learned my addr 172.21.15.123:0/3359086460 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:13.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.180+0000 7f32ee000640 1 --2- 172.21.15.123:0/3359086460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32f8102790 0x7f32f819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:13.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.180+0000 7f32f7400640 1 -- 172.21.15.123:0/3359086460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32f8102790 msgr2=0x7f32f819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:13.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.180+0000 7f32f7400640 1 --2- 172.21.15.123:0/3359086460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32f8102790 0x7f32f819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:13.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.180+0000 7f32f7400640 1 -- 172.21.15.123:0/3359086460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32f8101de0 msgr2=0x7f32f819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:13.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.180+0000 7f32f7400640 1 --2- 172.21.15.123:0/3359086460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32f8101de0 0x7f32f819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:13.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.180+0000 7f32f7400640 1 -- 172.21.15.123:0/3359086460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f819f600 con 0x7f32f8106f10 2024-09-19T04:45:13.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.180+0000 7f32f7400640 1 --2- 172.21.15.123:0/3359086460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32f8106f10 0x7f32f819ef20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f32f000e970 tx=0x7f32f000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:13.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.180+0000 7f32efe00640 1 -- 172.21.15.123:0/3359086460 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f000ccc0 con 0x7f32f8106f10 2024-09-19T04:45:13.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.180+0000 7f32efe00640 1 -- 172.21.15.123:0/3359086460 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f000ce60 con 0x7f32f8106f10 2024-09-19T04:45:13.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.180+0000 7f32efe00640 1 -- 172.21.15.123:0/3359086460 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0010490 con 0x7f32f8106f10 2024-09-19T04:45:13.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.180+0000 7f32fda00640 1 -- 172.21.15.123:0/3359086460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f819f8f0 con 0x7f32f8106f10 2024-09-19T04:45:13.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.181+0000 7f32fda00640 1 -- 172.21.15.123:0/3359086460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f81a71d0 con 0x7f32f8106f10 2024-09-19T04:45:13.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.181+0000 7f32ecc00640 1 -- 172.21.15.123:0/3359086460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f32f8062980 con 0x7f32f8106f10 2024-09-19T04:45:13.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.183+0000 7f32efe00640 1 -- 172.21.15.123:0/3359086460 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f32f0005000 con 0x7f32f8106f10 2024-09-19T04:45:13.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.185+0000 7f32efe00640 1 --2- 172.21.15.123:0/3359086460 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f32dc0a32d0 0x7f32dc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:13.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.185+0000 7f32efe00640 1 -- 172.21.15.123:0/3359086460 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f32f00cff60 con 0x7f32f8106f10 2024-09-19T04:45:13.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.185+0000 7f32f6a00640 1 --2- 172.21.15.123:0/3359086460 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f32dc0a32d0 0x7f32dc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:13.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.185+0000 7f32efe00640 1 --2- 172.21.15.123:0/3359086460 >> [v2:172.21.15.183:6824/3355349245,v1:172.21.15.183:6825/3355349245] conn(0x7f32dc0acb20 0x7f32dc0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:13.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.185+0000 7f32efe00640 1 -- 172.21.15.123:0/3359086460 --> [v2:172.21.15.183:6824/3355349245,v1:172.21.15.183:6825/3355349245] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32f0004470 con 0x7f32dc0acb20 2024-09-19T04:45:13.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.185+0000 7f32efe00640 1 -- 172.21.15.123:0/3359086460 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f32f00d0310 con 0x7f32f8106f10 2024-09-19T04:45:13.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.185+0000 7f32f6a00640 1 --2- 172.21.15.123:0/3359086460 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f32dc0a32d0 0x7f32dc0a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f32e8000c00 tx=0x7f32e8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:13.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.185+0000 7f32ee000640 1 --2- 172.21.15.123:0/3359086460 >> [v2:172.21.15.183:6824/3355349245,v1:172.21.15.183:6825/3355349245] conn(0x7f32dc0acb20 0x7f32dc0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:13.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.186+0000 7f32ee000640 1 --2- 172.21.15.123:0/3359086460 >> [v2:172.21.15.183:6824/3355349245,v1:172.21.15.183:6825/3355349245] conn(0x7f32dc0acb20 0x7f32dc0aef40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:13.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.187+0000 7f32efe00640 1 -- 172.21.15.123:0/3359086460 <== osd.10 v2:172.21.15.183:6824/3355349245 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32f0004470 con 0x7f32dc0acb20 2024-09-19T04:45:13.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.203+0000 7f32ecc00640 1 -- 172.21.15.123:0/3359086460 --> [v2:172.21.15.183:6824/3355349245,v1:172.21.15.183:6825/3355349245] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f32f8062e00 con 0x7f32dc0acb20 2024-09-19T04:45:13.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.204+0000 7f32efe00640 1 -- 172.21.15.123:0/3359086460 <== osd.10 v2:172.21.15.183:6824/3355349245 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f32f8062e00 con 0x7f32dc0acb20 2024-09-19T04:45:13.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.205+0000 7f32ecc00640 1 -- 172.21.15.123:0/3359086460 >> [v2:172.21.15.183:6824/3355349245,v1:172.21.15.183:6825/3355349245] conn(0x7f32dc0acb20 msgr2=0x7f32dc0aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:13.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.205+0000 7f32ecc00640 1 --2- 172.21.15.123:0/3359086460 >> [v2:172.21.15.183:6824/3355349245,v1:172.21.15.183:6825/3355349245] conn(0x7f32dc0acb20 0x7f32dc0aef40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:13.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.205+0000 7f32ecc00640 1 -- 172.21.15.123:0/3359086460 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f32dc0a32d0 msgr2=0x7f32dc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:13.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.205+0000 7f32ecc00640 1 --2- 172.21.15.123:0/3359086460 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f32dc0a32d0 0x7f32dc0a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f32e8000c00 tx=0x7f32e8005b50 comp rx=0 tx=0).stop 2024-09-19T04:45:13.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.205+0000 7f32ecc00640 1 -- 172.21.15.123:0/3359086460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32f8106f10 msgr2=0x7f32f819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:13.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.205+0000 7f32ecc00640 1 --2- 172.21.15.123:0/3359086460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32f8106f10 0x7f32f819ef20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f32f000e970 tx=0x7f32f000ed70 comp rx=0 tx=0).stop 2024-09-19T04:45:13.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.205+0000 7f32f7400640 1 -- 172.21.15.123:0/3359086460 reap_dead start 2024-09-19T04:45:13.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.205+0000 7f32ecc00640 1 -- 172.21.15.123:0/3359086460 shutdown_connections 2024-09-19T04:45:13.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.205+0000 7f32ecc00640 1 -- 172.21.15.123:0/3359086460 >> 172.21.15.123:0/3359086460 conn(0x7f32f80fd8c0 msgr2=0x7f32f80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:13.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.205+0000 7f32ecc00640 1 -- 172.21.15.123:0/3359086460 shutdown_connections 2024-09-19T04:45:13.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:13.205+0000 7f32ecc00640 1 -- 172.21.15.123:0/3359086460 wait complete. 2024-09-19T04:45:13.231 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:13.231 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:13.231 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:13.233 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:13.240 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:13.240 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:13.248 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:13.248 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:13.817 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-19T04:45:13.817 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-19T04:45:13.818 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-09-19T04:45:13.818 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-19T04:45:13.818 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-09-19T04:45:13.818 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-09-19T04:45:13.818 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-09-19T04:45:13.818 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:13 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-09-19T04:45:13.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:13 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-19T04:45:13.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:13 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-19T04:45:13.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:13 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-09-19T04:45:13.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:13 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-19T04:45:13.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:13 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-09-19T04:45:13.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:13 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-09-19T04:45:13.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:13 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-09-19T04:45:13.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:13 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-09-19T04:45:14.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:13 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-19T04:45:14.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:13 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-19T04:45:14.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:13 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-09-19T04:45:14.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:13 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-19T04:45:14.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:13 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-09-19T04:45:14.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:13 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-09-19T04:45:14.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:13 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-09-19T04:45:14.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:13 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-09-19T04:45:14.755 INFO:teuthology.orchestra.run.smithi123.stdout:300647710748 2024-09-19T04:45:14.756 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-19T04:45:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:14 smithi123 ceph-mon[30511]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-19T04:45:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-19T04:45:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-19T04:45:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-19T04:45:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-19T04:45:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-19T04:45:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-19T04:45:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-09-19T04:45:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:14 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-09-19T04:45:14.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:14 smithi123 ceph-mon[30511]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T04:45:14.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:14 smithi193 ceph-mon[35592]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-19T04:45:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-19T04:45:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-19T04:45:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-19T04:45:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-19T04:45:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-19T04:45:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-19T04:45:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-09-19T04:45:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:14 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-09-19T04:45:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:14 smithi193 ceph-mon[35592]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T04:45:15.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:14 smithi183 ceph-mon[35333]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-19T04:45:15.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-19T04:45:15.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-19T04:45:15.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-19T04:45:15.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-19T04:45:15.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-19T04:45:15.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-19T04:45:15.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-09-19T04:45:15.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:14 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-09-19T04:45:15.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:14 smithi183 ceph-mon[35333]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T04:45:15.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.337+0000 7efd0f600640 1 -- 172.21.15.123:0/1768113694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd08101de0 msgr2=0x7efd081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:15.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.337+0000 7efd0f600640 1 --2- 172.21.15.123:0/1768113694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd08101de0 0x7efd081021c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009900 tx=0x7efcfc02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:45:15.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.337+0000 7efd0f600640 1 -- 172.21.15.123:0/1768113694 shutdown_connections 2024-09-19T04:45:15.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.337+0000 7efd0f600640 1 --2- 172.21.15.123:0/1768113694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd08106f10 0x7efd0810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:15.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.337+0000 7efd0f600640 1 --2- 172.21.15.123:0/1768113694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd08102790 0x7efd081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:15.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.337+0000 7efd0f600640 1 --2- 172.21.15.123:0/1768113694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd08101de0 0x7efd081021c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:15.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.337+0000 7efd0f600640 1 -- 172.21.15.123:0/1768113694 >> 172.21.15.123:0/1768113694 conn(0x7efd080fd8c0 msgr2=0x7efd080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:15.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.337+0000 7efd0f600640 1 -- 172.21.15.123:0/1768113694 shutdown_connections 2024-09-19T04:45:15.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.337+0000 7efd0f600640 1 -- 172.21.15.123:0/1768113694 wait complete. 2024-09-19T04:45:15.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.338+0000 7efd0f600640 1 Processor -- start 2024-09-19T04:45:15.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.338+0000 7efd0f600640 1 -- start start 2024-09-19T04:45:15.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.339+0000 7efd0f600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd08101de0 0x7efd080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:15.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.339+0000 7efd0f600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd08102790 0x7efd08074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:15.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.339+0000 7efd0f600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd08106f10 0x7efd08074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:15.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.339+0000 7efd0f600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efd08111fc0 con 0x7efd08106f10 2024-09-19T04:45:15.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.339+0000 7efd0f600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efd08111e40 con 0x7efd08102790 2024-09-19T04:45:15.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.339+0000 7efd0f600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efd08112140 con 0x7efd08101de0 2024-09-19T04:45:15.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.339+0000 7efd0d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd08106f10 0x7efd08074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:15.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.339+0000 7efd0d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd08106f10 0x7efd08074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56326/0 (socket says 172.21.15.123:56326) 2024-09-19T04:45:15.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.339+0000 7efd0d600640 1 -- 172.21.15.123:0/3981103719 learned_addr learned my addr 172.21.15.123:0/3981103719 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:15.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.339+0000 7efd0cc00640 1 --2- 172.21.15.123:0/3981103719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd08101de0 0x7efd080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:15.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.339+0000 7efd07e00640 1 --2- 172.21.15.123:0/3981103719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd08102790 0x7efd08074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:15.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.339+0000 7efd0d600640 1 -- 172.21.15.123:0/3981103719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd08101de0 msgr2=0x7efd080744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:15.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.339+0000 7efd0d600640 1 --2- 172.21.15.123:0/3981103719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd08101de0 0x7efd080744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:15.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.339+0000 7efd0d600640 1 -- 172.21.15.123:0/3981103719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd08102790 msgr2=0x7efd08074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:15.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.339+0000 7efd0d600640 1 --2- 172.21.15.123:0/3981103719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd08102790 0x7efd08074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:15.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.339+0000 7efd0d600640 1 -- 172.21.15.123:0/3981103719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd081a6900 con 0x7efd08106f10 2024-09-19T04:45:15.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.340+0000 7efd0d600640 1 --2- 172.21.15.123:0/3981103719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd08106f10 0x7efd08074f70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efd0000eaa0 tx=0x7efd0000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:15.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.340+0000 7efd05600640 1 -- 172.21.15.123:0/3981103719 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0000cdb0 con 0x7efd08106f10 2024-09-19T04:45:15.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.340+0000 7efd05600640 1 -- 172.21.15.123:0/3981103719 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd00005020 con 0x7efd08106f10 2024-09-19T04:45:15.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.340+0000 7efd0f600640 1 -- 172.21.15.123:0/3981103719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd081a6bf0 con 0x7efd08106f10 2024-09-19T04:45:15.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.340+0000 7efd05600640 1 -- 172.21.15.123:0/3981103719 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd000116b0 con 0x7efd08106f10 2024-09-19T04:45:15.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.340+0000 7efd0f600640 1 -- 172.21.15.123:0/3981103719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd081a70e0 con 0x7efd08106f10 2024-09-19T04:45:15.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.341+0000 7efd0f600640 1 -- 172.21.15.123:0/3981103719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efd080628c0 con 0x7efd08106f10 2024-09-19T04:45:15.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.343+0000 7efd05600640 1 -- 172.21.15.123:0/3981103719 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7efd00005560 con 0x7efd08106f10 2024-09-19T04:45:15.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.344+0000 7efd05600640 1 --2- 172.21.15.123:0/3981103719 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7efcf00a32d0 0x7efcf00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:15.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.344+0000 7efd05600640 1 -- 172.21.15.123:0/3981103719 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7efd000cfff0 con 0x7efd08106f10 2024-09-19T04:45:15.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.345+0000 7efd0cc00640 1 --2- 172.21.15.123:0/3981103719 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7efcf00a32d0 0x7efcf00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:15.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.345+0000 7efd05600640 1 --2- 172.21.15.123:0/3981103719 >> [v2:172.21.15.183:6816/3718518226,v1:172.21.15.183:6817/3718518226] conn(0x7efcf00ace80 0x7efcf00af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:15.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.345+0000 7efd05600640 1 -- 172.21.15.123:0/3981103719 --> [v2:172.21.15.183:6816/3718518226,v1:172.21.15.183:6817/3718518226] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd00014ec0 con 0x7efcf00ace80 2024-09-19T04:45:15.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.345+0000 7efd05600640 1 -- 172.21.15.123:0/3981103719 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7efd000d0440 con 0x7efd08106f10 2024-09-19T04:45:15.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.345+0000 7efd07e00640 1 --2- 172.21.15.123:0/3981103719 >> [v2:172.21.15.183:6816/3718518226,v1:172.21.15.183:6817/3718518226] conn(0x7efcf00ace80 0x7efcf00af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:15.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.345+0000 7efd0cc00640 1 --2- 172.21.15.123:0/3981103719 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7efcf00a32d0 0x7efcf00a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efcfc000c00 tx=0x7efcfc005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:15.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.346+0000 7efd07e00640 1 --2- 172.21.15.123:0/3981103719 >> [v2:172.21.15.183:6816/3718518226,v1:172.21.15.183:6817/3718518226] conn(0x7efcf00ace80 0x7efcf00af320 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:15.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.347+0000 7efd05600640 1 -- 172.21.15.123:0/3981103719 <== osd.7 v2:172.21.15.183:6816/3718518226 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efd00014ec0 con 0x7efcf00ace80 2024-09-19T04:45:15.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.361+0000 7efd0f600640 1 -- 172.21.15.123:0/3981103719 --> [v2:172.21.15.183:6816/3718518226,v1:172.21.15.183:6817/3718518226] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efd08062d40 con 0x7efcf00ace80 2024-09-19T04:45:15.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.361+0000 7efd05600640 1 -- 172.21.15.123:0/3981103719 <== osd.7 v2:172.21.15.183:6816/3718518226 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efd08062d40 con 0x7efcf00ace80 2024-09-19T04:45:15.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.362+0000 7efd0f600640 1 -- 172.21.15.123:0/3981103719 >> [v2:172.21.15.183:6816/3718518226,v1:172.21.15.183:6817/3718518226] conn(0x7efcf00ace80 msgr2=0x7efcf00af320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:15.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.362+0000 7efd0f600640 1 --2- 172.21.15.123:0/3981103719 >> [v2:172.21.15.183:6816/3718518226,v1:172.21.15.183:6817/3718518226] conn(0x7efcf00ace80 0x7efcf00af320 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:15.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.362+0000 7efd0f600640 1 -- 172.21.15.123:0/3981103719 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7efcf00a32d0 msgr2=0x7efcf00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:15.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.362+0000 7efd0f600640 1 --2- 172.21.15.123:0/3981103719 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7efcf00a32d0 0x7efcf00a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efcfc000c00 tx=0x7efcfc005b50 comp rx=0 tx=0).stop 2024-09-19T04:45:15.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.362+0000 7efd0f600640 1 -- 172.21.15.123:0/3981103719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd08106f10 msgr2=0x7efd08074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:15.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.362+0000 7efd0f600640 1 --2- 172.21.15.123:0/3981103719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd08106f10 0x7efd08074f70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efd0000eaa0 tx=0x7efd0000ef60 comp rx=0 tx=0).stop 2024-09-19T04:45:15.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.362+0000 7efd0d600640 1 -- 172.21.15.123:0/3981103719 reap_dead start 2024-09-19T04:45:15.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.362+0000 7efd0f600640 1 -- 172.21.15.123:0/3981103719 shutdown_connections 2024-09-19T04:45:15.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.362+0000 7efd0f600640 1 -- 172.21.15.123:0/3981103719 >> 172.21.15.123:0/3981103719 conn(0x7efd080fd8c0 msgr2=0x7efd080febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:15.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.362+0000 7efd0f600640 1 -- 172.21.15.123:0/3981103719 shutdown_connections 2024-09-19T04:45:15.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.362+0000 7efd0f600640 1 -- 172.21.15.123:0/3981103719 wait complete. 2024-09-19T04:45:15.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.613+0000 7fc779200640 1 -- 172.21.15.123:0/3080089435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7741049a0 msgr2=0x7fc7741089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:15.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.613+0000 7fc779200640 1 --2- 172.21.15.123:0/3080089435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7741049a0 0x7fc7741089f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc760009900 tx=0x7fc76002a410 comp rx=0 tx=0).stop 2024-09-19T04:45:15.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.613+0000 7fc779200640 1 -- 172.21.15.123:0/3080089435 shutdown_connections 2024-09-19T04:45:15.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.613+0000 7fc779200640 1 --2- 172.21.15.123:0/3080089435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc774109120 0x7fc774110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:15.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.613+0000 7fc779200640 1 --2- 172.21.15.123:0/3080089435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7741049a0 0x7fc7741089f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:15.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.613+0000 7fc779200640 1 --2- 172.21.15.123:0/3080089435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc774103ff0 0x7fc7741043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:15.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.613+0000 7fc779200640 1 -- 172.21.15.123:0/3080089435 >> 172.21.15.123:0/3080089435 conn(0x7fc7740ffa90 msgr2=0x7fc774101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:15.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.613+0000 7fc779200640 1 -- 172.21.15.123:0/3080089435 shutdown_connections 2024-09-19T04:45:15.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.613+0000 7fc779200640 1 -- 172.21.15.123:0/3080089435 wait complete. 2024-09-19T04:45:15.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.614+0000 7fc779200640 1 Processor -- start 2024-09-19T04:45:15.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.614+0000 7fc779200640 1 -- start start 2024-09-19T04:45:15.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.614+0000 7fc779200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc774103ff0 0x7fc77419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:15.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.614+0000 7fc779200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7741049a0 0x7fc77419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:15.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.614+0000 7fc779200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc774109120 0x7fc77419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:15.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.614+0000 7fc779200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc774114140 con 0x7fc774109120 2024-09-19T04:45:15.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.614+0000 7fc779200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc774113fc0 con 0x7fc774103ff0 2024-09-19T04:45:15.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.614+0000 7fc779200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc7741142c0 con 0x7fc7741049a0 2024-09-19T04:45:15.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.615+0000 7fc772800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc774109120 0x7fc77419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:15.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.615+0000 7fc772800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc774109120 0x7fc77419eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56350/0 (socket says 172.21.15.123:56350) 2024-09-19T04:45:15.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.615+0000 7fc772800640 1 -- 172.21.15.123:0/1264036847 learned_addr learned my addr 172.21.15.123:0/1264036847 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:15.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.615+0000 7fc771e00640 1 --2- 172.21.15.123:0/1264036847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc774103ff0 0x7fc77419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:15.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.615+0000 7fc771400640 1 --2- 172.21.15.123:0/1264036847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7741049a0 0x7fc77419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:15.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.615+0000 7fc772800640 1 -- 172.21.15.123:0/1264036847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7741049a0 msgr2=0x7fc77419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:15.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.615+0000 7fc772800640 1 --2- 172.21.15.123:0/1264036847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7741049a0 0x7fc77419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:15.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.615+0000 7fc772800640 1 -- 172.21.15.123:0/1264036847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc774103ff0 msgr2=0x7fc77419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:15.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.615+0000 7fc772800640 1 --2- 172.21.15.123:0/1264036847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc774103ff0 0x7fc77419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:15.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.615+0000 7fc772800640 1 -- 172.21.15.123:0/1264036847 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77419f5d0 con 0x7fc774109120 2024-09-19T04:45:15.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.615+0000 7fc772800640 1 --2- 172.21.15.123:0/1264036847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc774109120 0x7fc77419eef0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00e970 tx=0x7fc76c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:15.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.616+0000 7fc76aa00640 1 -- 172.21.15.123:0/1264036847 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c00ccc0 con 0x7fc774109120 2024-09-19T04:45:15.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.616+0000 7fc76aa00640 1 -- 172.21.15.123:0/1264036847 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc76c00ce60 con 0x7fc774109120 2024-09-19T04:45:15.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.616+0000 7fc779200640 1 -- 172.21.15.123:0/1264036847 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77419f8c0 con 0x7fc774109120 2024-09-19T04:45:15.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.616+0000 7fc76aa00640 1 -- 172.21.15.123:0/1264036847 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c010490 con 0x7fc774109120 2024-09-19T04:45:15.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.616+0000 7fc779200640 1 -- 172.21.15.123:0/1264036847 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7741a71a0 con 0x7fc774109120 2024-09-19T04:45:15.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.617+0000 7fc779200640 1 -- 172.21.15.123:0/1264036847 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc774062980 con 0x7fc774109120 2024-09-19T04:45:15.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.619+0000 7fc76aa00640 1 -- 172.21.15.123:0/1264036847 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc76c005000 con 0x7fc774109120 2024-09-19T04:45:15.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.620+0000 7fc76aa00640 1 --2- 172.21.15.123:0/1264036847 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc7580a3350 0x7fc7580a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:15.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.620+0000 7fc76aa00640 1 -- 172.21.15.123:0/1264036847 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fc76c0d0260 con 0x7fc774109120 2024-09-19T04:45:15.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.620+0000 7fc771e00640 1 --2- 172.21.15.123:0/1264036847 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc7580a3350 0x7fc7580a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:15.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.620+0000 7fc76aa00640 1 --2- 172.21.15.123:0/1264036847 >> [v2:172.21.15.193:6808/2292251639,v1:172.21.15.193:6809/2292251639] conn(0x7fc7580acf00 0x7fc7580af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:15.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.620+0000 7fc76aa00640 1 -- 172.21.15.123:0/1264036847 --> [v2:172.21.15.193:6808/2292251639,v1:172.21.15.193:6809/2292251639] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc7580afa70 con 0x7fc7580acf00 2024-09-19T04:45:15.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.620+0000 7fc76aa00640 1 -- 172.21.15.123:0/1264036847 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fc76c0d0610 con 0x7fc774109120 2024-09-19T04:45:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.621+0000 7fc771400640 1 --2- 172.21.15.123:0/1264036847 >> [v2:172.21.15.193:6808/2292251639,v1:172.21.15.193:6809/2292251639] conn(0x7fc7580acf00 0x7fc7580af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.621+0000 7fc771e00640 1 --2- 172.21.15.123:0/1264036847 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc7580a3350 0x7fc7580a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc764005fd0 tx=0x7fc764009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.621+0000 7fc771400640 1 --2- 172.21.15.123:0/1264036847 >> [v2:172.21.15.193:6808/2292251639,v1:172.21.15.193:6809/2292251639] conn(0x7fc7580acf00 0x7fc7580af3a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.622+0000 7fc76aa00640 1 -- 172.21.15.123:0/1264036847 <== osd.5 v2:172.21.15.193:6808/2292251639 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc7580afa70 con 0x7fc7580acf00 2024-09-19T04:45:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.636+0000 7fc779200640 1 -- 172.21.15.123:0/1264036847 --> [v2:172.21.15.193:6808/2292251639,v1:172.21.15.193:6809/2292251639] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc774062e00 con 0x7fc7580acf00 2024-09-19T04:45:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.637+0000 7fc76aa00640 1 -- 172.21.15.123:0/1264036847 <== osd.5 v2:172.21.15.193:6808/2292251639 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc774062e00 con 0x7fc7580acf00 2024-09-19T04:45:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.637+0000 7fc779200640 1 -- 172.21.15.123:0/1264036847 >> [v2:172.21.15.193:6808/2292251639,v1:172.21.15.193:6809/2292251639] conn(0x7fc7580acf00 msgr2=0x7fc7580af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.637+0000 7fc779200640 1 --2- 172.21.15.123:0/1264036847 >> [v2:172.21.15.193:6808/2292251639,v1:172.21.15.193:6809/2292251639] conn(0x7fc7580acf00 0x7fc7580af3a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.637+0000 7fc779200640 1 -- 172.21.15.123:0/1264036847 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc7580a3350 msgr2=0x7fc7580a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.637+0000 7fc779200640 1 --2- 172.21.15.123:0/1264036847 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc7580a3350 0x7fc7580a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc764005fd0 tx=0x7fc764009500 comp rx=0 tx=0).stop 2024-09-19T04:45:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.637+0000 7fc779200640 1 -- 172.21.15.123:0/1264036847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc774109120 msgr2=0x7fc77419eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.637+0000 7fc779200640 1 --2- 172.21.15.123:0/1264036847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc774109120 0x7fc77419eef0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00e970 tx=0x7fc76c00ed70 comp rx=0 tx=0).stop 2024-09-19T04:45:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.637+0000 7fc772800640 1 -- 172.21.15.123:0/1264036847 reap_dead start 2024-09-19T04:45:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.638+0000 7fc779200640 1 -- 172.21.15.123:0/1264036847 shutdown_connections 2024-09-19T04:45:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.638+0000 7fc779200640 1 -- 172.21.15.123:0/1264036847 >> 172.21.15.123:0/1264036847 conn(0x7fc7740ffa90 msgr2=0x7fc774100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.638+0000 7fc779200640 1 -- 172.21.15.123:0/1264036847 shutdown_connections 2024-09-19T04:45:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:15.638+0000 7fc779200640 1 -- 172.21.15.123:0/1264036847 wait complete. 2024-09-19T04:45:15.845 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:15 smithi123 ceph-mon[30511]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T04:45:15.846 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:15 smithi123 ceph-mon[30511]: pgmap v317: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-19T04:45:15.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:15 smithi193 ceph-mon[35592]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T04:45:16.083 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:15 smithi193 ceph-mon[35592]: pgmap v317: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-19T04:45:16.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:15 smithi183 ceph-mon[35333]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T04:45:16.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:15 smithi183 ceph-mon[35333]: pgmap v317: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-19T04:45:16.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.471+0000 7f7f7ea00640 1 -- 172.21.15.123:0/1481859424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f78101de0 msgr2=0x7f7f781021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:16.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.471+0000 7f7f7ea00640 1 --2- 172.21.15.123:0/1481859424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f78101de0 0x7f7f781021c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009900 tx=0x7f7f6802a410 comp rx=0 tx=0).stop 2024-09-19T04:45:16.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.471+0000 7f7f7ea00640 1 -- 172.21.15.123:0/1481859424 shutdown_connections 2024-09-19T04:45:16.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.471+0000 7f7f7ea00640 1 --2- 172.21.15.123:0/1481859424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f78106f10 0x7f7f7810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:16.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.471+0000 7f7f7ea00640 1 --2- 172.21.15.123:0/1481859424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f78102790 0x7f7f781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:16.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.471+0000 7f7f7ea00640 1 --2- 172.21.15.123:0/1481859424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f78101de0 0x7f7f781021c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:16.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.471+0000 7f7f7ea00640 1 -- 172.21.15.123:0/1481859424 >> 172.21.15.123:0/1481859424 conn(0x7f7f780fd8c0 msgr2=0x7f7f780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:16.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.471+0000 7f7f7ea00640 1 -- 172.21.15.123:0/1481859424 shutdown_connections 2024-09-19T04:45:16.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.471+0000 7f7f7ea00640 1 -- 172.21.15.123:0/1481859424 wait complete. 2024-09-19T04:45:16.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.472+0000 7f7f7ea00640 1 Processor -- start 2024-09-19T04:45:16.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.473+0000 7f7f7ea00640 1 -- start start 2024-09-19T04:45:16.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.473+0000 7f7f7ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f78101de0 0x7f7f7819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:16.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.473+0000 7f7f7ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f78102790 0x7f7f7819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:16.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.473+0000 7f7f7ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f78106f10 0x7f7f7819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:16.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.473+0000 7f7f7ea00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f78111f80 con 0x7f7f78106f10 2024-09-19T04:45:16.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.473+0000 7f7f7ea00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7f78111e00 con 0x7f7f78102790 2024-09-19T04:45:16.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.473+0000 7f7f7ea00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7f78112100 con 0x7f7f78101de0 2024-09-19T04:45:16.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.473+0000 7f7f77400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f78102790 0x7f7f7819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:16.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.473+0000 7f7f77e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f78101de0 0x7f7f7819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:16.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.473+0000 7f7f7ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f78106f10 0x7f7f7819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:16.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.473+0000 7f7f77400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f78102790 0x7f7f7819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:54994/0 (socket says 172.21.15.123:54994) 2024-09-19T04:45:16.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.473+0000 7f7f77400640 1 -- 172.21.15.123:0/442493070 learned_addr learned my addr 172.21.15.123:0/442493070 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:16.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.473+0000 7f7f77e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f78101de0 0x7f7f7819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:41972/0 (socket says 172.21.15.123:41972) 2024-09-19T04:45:16.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.474+0000 7f7f7ca00640 1 -- 172.21.15.123:0/442493070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f78101de0 msgr2=0x7f7f7819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:16.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.474+0000 7f7f7ca00640 1 --2- 172.21.15.123:0/442493070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f78101de0 0x7f7f7819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:16.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.474+0000 7f7f7ca00640 1 -- 172.21.15.123:0/442493070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f78102790 msgr2=0x7f7f7819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:16.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.474+0000 7f7f7ca00640 1 --2- 172.21.15.123:0/442493070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f78102790 0x7f7f7819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:16.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.474+0000 7f7f7ca00640 1 -- 172.21.15.123:0/442493070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7819f620 con 0x7f7f78106f10 2024-09-19T04:45:16.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.474+0000 7f7f77e00640 1 --2- 172.21.15.123:0/442493070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f78101de0 0x7f7f7819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:16.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.474+0000 7f7f7ca00640 1 --2- 172.21.15.123:0/442493070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f78106f10 0x7f7f7819ef40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009fd0 tx=0x7f7f7000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:16.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.474+0000 7f7f74c00640 1 -- 172.21.15.123:0/442493070 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7000ce20 con 0x7f7f78106f10 2024-09-19T04:45:16.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.474+0000 7f7f7ea00640 1 -- 172.21.15.123:0/442493070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7819f910 con 0x7f7f78106f10 2024-09-19T04:45:16.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.474+0000 7f7f74c00640 1 -- 172.21.15.123:0/442493070 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f70005020 con 0x7f7f78106f10 2024-09-19T04:45:16.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.475+0000 7f7f74c00640 1 -- 172.21.15.123:0/442493070 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70010700 con 0x7f7f78106f10 2024-09-19T04:45:16.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.475+0000 7f7f7ea00640 1 -- 172.21.15.123:0/442493070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f781a71f0 con 0x7f7f78106f10 2024-09-19T04:45:16.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.476+0000 7f7f7ea00640 1 -- 172.21.15.123:0/442493070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7f78075740 con 0x7f7f78106f10 2024-09-19T04:45:16.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.478+0000 7f7f74c00640 1 -- 172.21.15.123:0/442493070 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7f70005540 con 0x7f7f78106f10 2024-09-19T04:45:16.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.479+0000 7f7f74c00640 1 --2- 172.21.15.123:0/442493070 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7f5c0a32d0 0x7f7f5c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:16.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.479+0000 7f7f74c00640 1 -- 172.21.15.123:0/442493070 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f7f700d00d0 con 0x7f7f78106f10 2024-09-19T04:45:16.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.479+0000 7f7f77e00640 1 --2- 172.21.15.123:0/442493070 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7f5c0a32d0 0x7f7f5c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:16.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.479+0000 7f7f74c00640 1 --2- 172.21.15.123:0/442493070 >> [v2:172.21.15.193:6816/3551404118,v1:172.21.15.193:6817/3551404118] conn(0x7f7f5c0ace80 0x7f7f5c0af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:16.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.479+0000 7f7f74c00640 1 -- 172.21.15.123:0/442493070 --> [v2:172.21.15.193:6816/3551404118,v1:172.21.15.193:6817/3551404118] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7f5c0af9f0 con 0x7f7f5c0ace80 2024-09-19T04:45:16.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.479+0000 7f7f74c00640 1 -- 172.21.15.123:0/442493070 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f7f700d7050 con 0x7f7f78106f10 2024-09-19T04:45:16.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.480+0000 7f7f77400640 1 --2- 172.21.15.123:0/442493070 >> [v2:172.21.15.193:6816/3551404118,v1:172.21.15.193:6817/3551404118] conn(0x7f7f5c0ace80 0x7f7f5c0af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:16.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.480+0000 7f7f77e00640 1 --2- 172.21.15.123:0/442493070 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7f5c0a32d0 0x7f7f5c0a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7f68000c00 tx=0x7f7f68031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:16.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.480+0000 7f7f77400640 1 --2- 172.21.15.123:0/442493070 >> [v2:172.21.15.193:6816/3551404118,v1:172.21.15.193:6817/3551404118] conn(0x7f7f5c0ace80 0x7f7f5c0af320 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:16.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.481+0000 7f7f74c00640 1 -- 172.21.15.123:0/442493070 <== osd.8 v2:172.21.15.193:6816/3551404118 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f5c0af9f0 con 0x7f7f5c0ace80 2024-09-19T04:45:16.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.495+0000 7f7f7ea00640 1 -- 172.21.15.123:0/442493070 --> [v2:172.21.15.193:6816/3551404118,v1:172.21.15.193:6817/3551404118] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7f781032c0 con 0x7f7f5c0ace80 2024-09-19T04:45:16.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.496+0000 7f7f74c00640 1 -- 172.21.15.123:0/442493070 <== osd.8 v2:172.21.15.193:6816/3551404118 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7f781032c0 con 0x7f7f5c0ace80 2024-09-19T04:45:16.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.496+0000 7f7f7ea00640 1 -- 172.21.15.123:0/442493070 >> [v2:172.21.15.193:6816/3551404118,v1:172.21.15.193:6817/3551404118] conn(0x7f7f5c0ace80 msgr2=0x7f7f5c0af320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:16.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.496+0000 7f7f7ea00640 1 --2- 172.21.15.123:0/442493070 >> [v2:172.21.15.193:6816/3551404118,v1:172.21.15.193:6817/3551404118] conn(0x7f7f5c0ace80 0x7f7f5c0af320 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:16.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.497+0000 7f7f7ea00640 1 -- 172.21.15.123:0/442493070 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7f5c0a32d0 msgr2=0x7f7f5c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:16.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.497+0000 7f7f7ea00640 1 --2- 172.21.15.123:0/442493070 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7f5c0a32d0 0x7f7f5c0a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7f68000c00 tx=0x7f7f68031510 comp rx=0 tx=0).stop 2024-09-19T04:45:16.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.497+0000 7f7f7ea00640 1 -- 172.21.15.123:0/442493070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f78106f10 msgr2=0x7f7f7819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:16.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.497+0000 7f7f7ea00640 1 --2- 172.21.15.123:0/442493070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f78106f10 0x7f7f7819ef40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009fd0 tx=0x7f7f7000ef10 comp rx=0 tx=0).stop 2024-09-19T04:45:16.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.497+0000 7f7f7ca00640 1 -- 172.21.15.123:0/442493070 reap_dead start 2024-09-19T04:45:16.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.497+0000 7f7f7ea00640 1 -- 172.21.15.123:0/442493070 shutdown_connections 2024-09-19T04:45:16.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.497+0000 7f7f7ea00640 1 -- 172.21.15.123:0/442493070 >> 172.21.15.123:0/442493070 conn(0x7f7f780fd8c0 msgr2=0x7f7f780fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:16.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.497+0000 7f7f7ea00640 1 -- 172.21.15.123:0/442493070 shutdown_connections 2024-09-19T04:45:16.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:16.497+0000 7f7f7ea00640 1 -- 172.21.15.123:0/442493070 wait complete. 2024-09-19T04:45:17.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.409+0000 7fe44e600640 1 -- 172.21.15.123:0/3144107354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe448102790 msgr2=0x7fe4481067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:17.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.409+0000 7fe44e600640 1 --2- 172.21.15.123:0/3144107354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe448102790 0x7fe4481067e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe430009900 tx=0x7fe43002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:45:17.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.409+0000 7fe44e600640 1 -- 172.21.15.123:0/3144107354 shutdown_connections 2024-09-19T04:45:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.409+0000 7fe44e600640 1 --2- 172.21.15.123:0/3144107354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe448106f10 0x7fe44810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.409+0000 7fe44e600640 1 --2- 172.21.15.123:0/3144107354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe448102790 0x7fe4481067e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.409+0000 7fe44e600640 1 --2- 172.21.15.123:0/3144107354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe448101de0 0x7fe4481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.409+0000 7fe44e600640 1 -- 172.21.15.123:0/3144107354 >> 172.21.15.123:0/3144107354 conn(0x7fe4480fd8c0 msgr2=0x7fe4480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.409+0000 7fe44e600640 1 -- 172.21.15.123:0/3144107354 shutdown_connections 2024-09-19T04:45:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.409+0000 7fe44e600640 1 -- 172.21.15.123:0/3144107354 wait complete. 2024-09-19T04:45:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.410+0000 7fe44e600640 1 Processor -- start 2024-09-19T04:45:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.410+0000 7fe44e600640 1 -- start start 2024-09-19T04:45:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.411+0000 7fe44e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe448101de0 0x7fe448198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.411+0000 7fe44e600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe448102790 0x7fe4481989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.411+0000 7fe44e600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe448106f10 0x7fe44819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:17.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.411+0000 7fe44e600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe448111fc0 con 0x7fe448101de0 2024-09-19T04:45:17.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.411+0000 7fe44e600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe448111e40 con 0x7fe448102790 2024-09-19T04:45:17.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.411+0000 7fe44e600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe448112140 con 0x7fe448106f10 2024-09-19T04:45:17.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.411+0000 7fe447400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe448101de0 0x7fe448198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:17.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.411+0000 7fe447400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe448101de0 0x7fe448198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56392/0 (socket says 172.21.15.123:56392) 2024-09-19T04:45:17.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.411+0000 7fe447400640 1 -- 172.21.15.123:0/3557239844 learned_addr learned my addr 172.21.15.123:0/3557239844 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:17.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.411+0000 7fe447e00640 1 --2- 172.21.15.123:0/3557239844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe448106f10 0x7fe44819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:17.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.411+0000 7fe43ea00640 1 --2- 172.21.15.123:0/3557239844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe448102790 0x7fe4481989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:17.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.412+0000 7fe447400640 1 -- 172.21.15.123:0/3557239844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe448106f10 msgr2=0x7fe44819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:17.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.412+0000 7fe447400640 1 --2- 172.21.15.123:0/3557239844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe448106f10 0x7fe44819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:17.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.412+0000 7fe447400640 1 -- 172.21.15.123:0/3557239844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe448102790 msgr2=0x7fe4481989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:17.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.412+0000 7fe447400640 1 --2- 172.21.15.123:0/3557239844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe448102790 0x7fe4481989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:17.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.412+0000 7fe447400640 1 -- 172.21.15.123:0/3557239844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44819d410 con 0x7fe448101de0 2024-09-19T04:45:17.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.412+0000 7fe447400640 1 --2- 172.21.15.123:0/3557239844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe448101de0 0x7fe448198460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe43800d950 tx=0x7fe43800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.412+0000 7fe444c00640 1 -- 172.21.15.123:0/3557239844 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438017030 con 0x7fe448101de0 2024-09-19T04:45:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.412+0000 7fe444c00640 1 -- 172.21.15.123:0/3557239844 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe438004020 con 0x7fe448101de0 2024-09-19T04:45:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.412+0000 7fe44e600640 1 -- 172.21.15.123:0/3557239844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44819d700 con 0x7fe448101de0 2024-09-19T04:45:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.412+0000 7fe444c00640 1 -- 172.21.15.123:0/3557239844 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4380047a0 con 0x7fe448101de0 2024-09-19T04:45:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.412+0000 7fe44e600640 1 -- 172.21.15.123:0/3557239844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4481a4fe0 con 0x7fe448101de0 2024-09-19T04:45:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.413+0000 7fe44e600640 1 -- 172.21.15.123:0/3557239844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe448062900 con 0x7fe448101de0 2024-09-19T04:45:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.417+0000 7fe444c00640 1 -- 172.21.15.123:0/3557239844 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe4380041d0 con 0x7fe448101de0 2024-09-19T04:45:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.419+0000 7fe444c00640 1 --2- 172.21.15.123:0/3557239844 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe42c0a32d0 0x7fe42c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.419+0000 7fe444c00640 1 -- 172.21.15.123:0/3557239844 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fe4380cfaf0 con 0x7fe448101de0 2024-09-19T04:45:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.419+0000 7fe43ea00640 1 --2- 172.21.15.123:0/3557239844 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe42c0a32d0 0x7fe42c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.419+0000 7fe444c00640 1 --2- 172.21.15.123:0/3557239844 >> [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499] conn(0x7fe42c0ace80 0x7fe42c0af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.419+0000 7fe444c00640 1 -- 172.21.15.123:0/3557239844 --> [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe42c0af9f0 con 0x7fe42c0ace80 2024-09-19T04:45:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.419+0000 7fe444c00640 1 -- 172.21.15.123:0/3557239844 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fe4380cfea0 con 0x7fe448101de0 2024-09-19T04:45:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.419+0000 7fe447e00640 1 --2- 172.21.15.123:0/3557239844 >> [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499] conn(0x7fe42c0ace80 0x7fe42c0af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.420+0000 7fe43ea00640 1 --2- 172.21.15.123:0/3557239844 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe42c0a32d0 0x7fe42c0a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe430000c00 tx=0x7fe430005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:17.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.420+0000 7fe447e00640 1 --2- 172.21.15.123:0/3557239844 >> [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499] conn(0x7fe42c0ace80 0x7fe42c0af320 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:17.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.421+0000 7fe444c00640 1 -- 172.21.15.123:0/3557239844 <== osd.6 v2:172.21.15.123:6818/226007499 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe42c0af9f0 con 0x7fe42c0ace80 2024-09-19T04:45:17.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.435+0000 7fe44e600640 1 -- 172.21.15.123:0/3557239844 --> [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe448062d80 con 0x7fe42c0ace80 2024-09-19T04:45:17.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.436+0000 7fe444c00640 1 -- 172.21.15.123:0/3557239844 <== osd.6 v2:172.21.15.123:6818/226007499 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe448062d80 con 0x7fe42c0ace80 2024-09-19T04:45:17.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.436+0000 7fe44e600640 1 -- 172.21.15.123:0/3557239844 >> [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499] conn(0x7fe42c0ace80 msgr2=0x7fe42c0af320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:17.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.436+0000 7fe44e600640 1 --2- 172.21.15.123:0/3557239844 >> [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499] conn(0x7fe42c0ace80 0x7fe42c0af320 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:17.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.436+0000 7fe44e600640 1 -- 172.21.15.123:0/3557239844 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe42c0a32d0 msgr2=0x7fe42c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:17.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.436+0000 7fe44e600640 1 --2- 172.21.15.123:0/3557239844 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe42c0a32d0 0x7fe42c0a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe430000c00 tx=0x7fe430005b50 comp rx=0 tx=0).stop 2024-09-19T04:45:17.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.436+0000 7fe44e600640 1 -- 172.21.15.123:0/3557239844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe448101de0 msgr2=0x7fe448198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:17.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.436+0000 7fe44e600640 1 --2- 172.21.15.123:0/3557239844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe448101de0 0x7fe448198460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe43800d950 tx=0x7fe43800de10 comp rx=0 tx=0).stop 2024-09-19T04:45:17.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.436+0000 7fe447e00640 1 -- 172.21.15.123:0/3557239844 reap_dead start 2024-09-19T04:45:17.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.436+0000 7fe44e600640 1 -- 172.21.15.123:0/3557239844 shutdown_connections 2024-09-19T04:45:17.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.436+0000 7fe44e600640 1 -- 172.21.15.123:0/3557239844 >> 172.21.15.123:0/3557239844 conn(0x7fe4480fd8c0 msgr2=0x7fe4480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:17.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.437+0000 7fe44e600640 1 -- 172.21.15.123:0/3557239844 shutdown_connections 2024-09-19T04:45:17.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:17.437+0000 7fe44e600640 1 -- 172.21.15.123:0/3557239844 wait complete. 2024-09-19T04:45:18.523 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:18 smithi123 ceph-mon[30511]: pgmap v318: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-19T04:45:18.559 INFO:teuthology.orchestra.run.smithi123.stdout:244813135911 2024-09-19T04:45:18.560 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-19T04:45:18.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:18 smithi193 ceph-mon[35592]: pgmap v318: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-19T04:45:18.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:18 smithi183 ceph-mon[35333]: pgmap v318: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-19T04:45:19.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.274+0000 7f78ce400640 1 -- 172.21.15.123:0/3787097276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78c8104010 msgr2=0x7f78c81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:19.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.274+0000 7f78ce400640 1 --2- 172.21.15.123:0/3787097276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78c8104010 0x7f78c81043f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009860 tx=0x7f78b802a340 comp rx=0 tx=0).stop 2024-09-19T04:45:19.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.274+0000 7f78ce400640 1 -- 172.21.15.123:0/3787097276 shutdown_connections 2024-09-19T04:45:19.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.274+0000 7f78ce400640 1 --2- 172.21.15.123:0/3787097276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78c8109140 0x7f78c8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:19.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.274+0000 7f78ce400640 1 --2- 172.21.15.123:0/3787097276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78c81049c0 0x7f78c8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:19.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.274+0000 7f78ce400640 1 --2- 172.21.15.123:0/3787097276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78c8104010 0x7f78c81043f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:19.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.274+0000 7f78ce400640 1 -- 172.21.15.123:0/3787097276 >> 172.21.15.123:0/3787097276 conn(0x7f78c80ffa50 msgr2=0x7f78c8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:19.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.274+0000 7f78ce400640 1 -- 172.21.15.123:0/3787097276 shutdown_connections 2024-09-19T04:45:19.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.274+0000 7f78ce400640 1 -- 172.21.15.123:0/3787097276 wait complete. 2024-09-19T04:45:19.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.275+0000 7f78ce400640 1 Processor -- start 2024-09-19T04:45:19.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.275+0000 7f78ce400640 1 -- start start 2024-09-19T04:45:19.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.275+0000 7f78ce400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78c8104010 0x7f78c819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:19.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.275+0000 7f78ce400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78c81049c0 0x7f78c819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:19.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.275+0000 7f78ce400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78c8109140 0x7f78c819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:19.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.275+0000 7f78ce400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8114160 con 0x7f78c81049c0 2024-09-19T04:45:19.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.275+0000 7f78ce400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8113fe0 con 0x7f78c8104010 2024-09-19T04:45:19.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.275+0000 7f78ce400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78c81142e0 con 0x7f78c8109140 2024-09-19T04:45:19.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.276+0000 7f78c7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78c8104010 0x7f78c819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:19.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.276+0000 7f78bfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78c81049c0 0x7f78c819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:19.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.276+0000 7f78c7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78c8104010 0x7f78c819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:55048/0 (socket says 172.21.15.123:55048) 2024-09-19T04:45:19.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.276+0000 7f78c7400640 1 -- 172.21.15.123:0/2323218346 learned_addr learned my addr 172.21.15.123:0/2323218346 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:19.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.276+0000 7f78c7e00640 1 --2- 172.21.15.123:0/2323218346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78c8109140 0x7f78c819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:19.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.276+0000 7f78bfe00640 1 -- 172.21.15.123:0/2323218346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78c8109140 msgr2=0x7f78c819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:19.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.276+0000 7f78bfe00640 1 --2- 172.21.15.123:0/2323218346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78c8109140 0x7f78c819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:19.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.276+0000 7f78bfe00640 1 -- 172.21.15.123:0/2323218346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78c8104010 msgr2=0x7f78c819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:19.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.276+0000 7f78bfe00640 1 --2- 172.21.15.123:0/2323218346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78c8104010 0x7f78c819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:19.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.276+0000 7f78bfe00640 1 -- 172.21.15.123:0/2323218346 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c819f5d0 con 0x7f78c81049c0 2024-09-19T04:45:19.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.276+0000 7f78c7e00640 1 --2- 172.21.15.123:0/2323218346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78c8109140 0x7f78c819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:45:19.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.276+0000 7f78c7400640 1 --2- 172.21.15.123:0/2323218346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78c8104010 0x7f78c819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:19.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.276+0000 7f78bfe00640 1 --2- 172.21.15.123:0/2323218346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78c81049c0 0x7f78c819ab60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f78b000e9f0 tx=0x7f78b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:19.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.277+0000 7f78c4c00640 1 -- 172.21.15.123:0/2323218346 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b000ccc0 con 0x7f78c81049c0 2024-09-19T04:45:19.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.277+0000 7f78c4c00640 1 -- 172.21.15.123:0/2323218346 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b0005020 con 0x7f78c81049c0 2024-09-19T04:45:19.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.277+0000 7f78ce400640 1 -- 172.21.15.123:0/2323218346 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c819f8c0 con 0x7f78c81049c0 2024-09-19T04:45:19.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.277+0000 7f78c4c00640 1 -- 172.21.15.123:0/2323218346 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b00049e0 con 0x7f78c81049c0 2024-09-19T04:45:19.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.277+0000 7f78ce400640 1 -- 172.21.15.123:0/2323218346 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c81a71a0 con 0x7f78c81049c0 2024-09-19T04:45:19.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.278+0000 7f78ce400640 1 -- 172.21.15.123:0/2323218346 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f78c8062980 con 0x7f78c81049c0 2024-09-19T04:45:19.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.280+0000 7f78c4c00640 1 -- 172.21.15.123:0/2323218346 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f78b0005560 con 0x7f78c81049c0 2024-09-19T04:45:19.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.281+0000 7f78c4c00640 1 --2- 172.21.15.123:0/2323218346 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f78ac0a3350 0x7f78ac0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:19.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.281+0000 7f78c4c00640 1 -- 172.21.15.123:0/2323218346 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f78b00cfd10 con 0x7f78c81049c0 2024-09-19T04:45:19.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.281+0000 7f78c7400640 1 --2- 172.21.15.123:0/2323218346 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f78ac0a3350 0x7f78ac0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:19.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.282+0000 7f78c4c00640 1 --2- 172.21.15.123:0/2323218346 >> [v2:172.21.15.193:6800/2708151726,v1:172.21.15.193:6801/2708151726] conn(0x7f78ac0acf00 0x7f78ac0af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:19.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.282+0000 7f78c4c00640 1 -- 172.21.15.123:0/2323218346 --> [v2:172.21.15.193:6800/2708151726,v1:172.21.15.193:6801/2708151726] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78b0022e50 con 0x7f78ac0acf00 2024-09-19T04:45:19.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.282+0000 7f78c4c00640 1 -- 172.21.15.123:0/2323218346 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f78b00d00c0 con 0x7f78c81049c0 2024-09-19T04:45:19.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.282+0000 7f78c7e00640 1 --2- 172.21.15.123:0/2323218346 >> [v2:172.21.15.193:6800/2708151726,v1:172.21.15.193:6801/2708151726] conn(0x7f78ac0acf00 0x7f78ac0af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:19.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.282+0000 7f78c7400640 1 --2- 172.21.15.123:0/2323218346 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f78ac0a3350 0x7f78ac0a5810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f78b80057c0 tx=0x7f78b80056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:19.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.283+0000 7f78c7e00640 1 --2- 172.21.15.123:0/2323218346 >> [v2:172.21.15.193:6800/2708151726,v1:172.21.15.193:6801/2708151726] conn(0x7f78ac0acf00 0x7f78ac0af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:19.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.283+0000 7f78c4c00640 1 -- 172.21.15.123:0/2323218346 <== osd.2 v2:172.21.15.193:6800/2708151726 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f78b0022e50 con 0x7f78ac0acf00 2024-09-19T04:45:19.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.298+0000 7f78ce400640 1 -- 172.21.15.123:0/2323218346 --> [v2:172.21.15.193:6800/2708151726,v1:172.21.15.193:6801/2708151726] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f78c8062e00 con 0x7f78ac0acf00 2024-09-19T04:45:19.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.299+0000 7f78c4c00640 1 -- 172.21.15.123:0/2323218346 <== osd.2 v2:172.21.15.193:6800/2708151726 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f78c8062e00 con 0x7f78ac0acf00 2024-09-19T04:45:19.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.299+0000 7f78ce400640 1 -- 172.21.15.123:0/2323218346 >> [v2:172.21.15.193:6800/2708151726,v1:172.21.15.193:6801/2708151726] conn(0x7f78ac0acf00 msgr2=0x7f78ac0af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:19.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.299+0000 7f78ce400640 1 --2- 172.21.15.123:0/2323218346 >> [v2:172.21.15.193:6800/2708151726,v1:172.21.15.193:6801/2708151726] conn(0x7f78ac0acf00 0x7f78ac0af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:19.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.299+0000 7f78ce400640 1 -- 172.21.15.123:0/2323218346 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f78ac0a3350 msgr2=0x7f78ac0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:19.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.299+0000 7f78ce400640 1 --2- 172.21.15.123:0/2323218346 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f78ac0a3350 0x7f78ac0a5810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f78b80057c0 tx=0x7f78b80056d0 comp rx=0 tx=0).stop 2024-09-19T04:45:19.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.300+0000 7f78ce400640 1 -- 172.21.15.123:0/2323218346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78c81049c0 msgr2=0x7f78c819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:19.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.300+0000 7f78ce400640 1 --2- 172.21.15.123:0/2323218346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78c81049c0 0x7f78c819ab60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f78b000e9f0 tx=0x7f78b000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:45:19.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.300+0000 7f78c7e00640 1 -- 172.21.15.123:0/2323218346 reap_dead start 2024-09-19T04:45:19.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.300+0000 7f78ce400640 1 -- 172.21.15.123:0/2323218346 shutdown_connections 2024-09-19T04:45:19.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.300+0000 7f78ce400640 1 -- 172.21.15.123:0/2323218346 >> 172.21.15.123:0/2323218346 conn(0x7f78c80ffa50 msgr2=0x7f78c8100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:19.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.300+0000 7f78ce400640 1 -- 172.21.15.123:0/2323218346 shutdown_connections 2024-09-19T04:45:19.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.300+0000 7f78ce400640 1 -- 172.21.15.123:0/2323218346 wait complete. 2024-09-19T04:45:20.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.902+0000 7fc415c00640 1 -- 172.21.15.123:0/1379775856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc410103e30 msgr2=0x7fc410104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.902+0000 7fc415c00640 1 --2- 172.21.15.123:0/1379775856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc410103e30 0x7fc410104210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc400009900 tx=0x7fc40002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.902+0000 7fc415c00640 1 -- 172.21.15.123:0/1379775856 shutdown_connections 2024-09-19T04:45:20.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.902+0000 7fc415c00640 1 --2- 172.21.15.123:0/1379775856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc410108f60 0x7fc410110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.902+0000 7fc415c00640 1 --2- 172.21.15.123:0/1379775856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4101047e0 0x7fc410108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.902+0000 7fc415c00640 1 --2- 172.21.15.123:0/1379775856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc410103e30 0x7fc410104210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.902+0000 7fc415c00640 1 -- 172.21.15.123:0/1379775856 >> 172.21.15.123:0/1379775856 conn(0x7fc4100ff8d0 msgr2=0x7fc410101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:20.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.902+0000 7fc415c00640 1 -- 172.21.15.123:0/1379775856 shutdown_connections 2024-09-19T04:45:20.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.902+0000 7fc415c00640 1 -- 172.21.15.123:0/1379775856 wait complete. 2024-09-19T04:45:20.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.903+0000 7fc415c00640 1 Processor -- start 2024-09-19T04:45:20.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.903+0000 7fc415c00640 1 -- start start 2024-09-19T04:45:20.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.903+0000 7fc415c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc410103e30 0x7fc41019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:20.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.903+0000 7fc415c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4101047e0 0x7fc41019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:20.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.903+0000 7fc415c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc410108f60 0x7fc41019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:20.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.903+0000 7fc415c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc410114010 con 0x7fc410108f60 2024-09-19T04:45:20.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.903+0000 7fc415c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc410113e90 con 0x7fc4101047e0 2024-09-19T04:45:20.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.903+0000 7fc415c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc410114190 con 0x7fc410103e30 2024-09-19T04:45:20.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.904+0000 7fc415200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc410108f60 0x7fc41019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:20.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.904+0000 7fc415200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc410108f60 0x7fc41019eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56428/0 (socket says 172.21.15.123:56428) 2024-09-19T04:45:20.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.904+0000 7fc415200640 1 -- 172.21.15.123:0/2068009164 learned_addr learned my addr 172.21.15.123:0/2068009164 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:20.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.904+0000 7fc40fe00640 1 --2- 172.21.15.123:0/2068009164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc410103e30 0x7fc41019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:20.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.904+0000 7fc407e00640 1 --2- 172.21.15.123:0/2068009164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4101047e0 0x7fc41019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:20.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.904+0000 7fc415200640 1 -- 172.21.15.123:0/2068009164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc410103e30 msgr2=0x7fc41019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.904+0000 7fc415200640 1 --2- 172.21.15.123:0/2068009164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc410103e30 0x7fc41019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.904+0000 7fc415200640 1 -- 172.21.15.123:0/2068009164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4101047e0 msgr2=0x7fc41019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.904+0000 7fc415200640 1 --2- 172.21.15.123:0/2068009164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4101047e0 0x7fc41019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.904+0000 7fc415200640 1 -- 172.21.15.123:0/2068009164 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41019f480 con 0x7fc410108f60 2024-09-19T04:45:20.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.904+0000 7fc415200640 1 --2- 172.21.15.123:0/2068009164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc410108f60 0x7fc41019eda0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc40800eb10 tx=0x7fc40800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:20.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.905+0000 7fc40d600640 1 -- 172.21.15.123:0/2068009164 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408004b20 con 0x7fc410108f60 2024-09-19T04:45:20.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.905+0000 7fc40d600640 1 -- 172.21.15.123:0/2068009164 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc408004cc0 con 0x7fc410108f60 2024-09-19T04:45:20.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.905+0000 7fc415c00640 1 -- 172.21.15.123:0/2068009164 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41019f770 con 0x7fc410108f60 2024-09-19T04:45:20.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.905+0000 7fc40d600640 1 -- 172.21.15.123:0/2068009164 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408010760 con 0x7fc410108f60 2024-09-19T04:45:20.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.905+0000 7fc415c00640 1 -- 172.21.15.123:0/2068009164 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4101a7050 con 0x7fc410108f60 2024-09-19T04:45:20.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.906+0000 7fc415c00640 1 -- 172.21.15.123:0/2068009164 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc410062980 con 0x7fc410108f60 2024-09-19T04:45:20.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.908+0000 7fc40d600640 1 -- 172.21.15.123:0/2068009164 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc408005020 con 0x7fc410108f60 2024-09-19T04:45:20.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.909+0000 7fc40d600640 1 --2- 172.21.15.123:0/2068009164 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc3f40a32d0 0x7fc3f40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:20.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.909+0000 7fc40d600640 1 -- 172.21.15.123:0/2068009164 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fc4080d0410 con 0x7fc410108f60 2024-09-19T04:45:20.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.909+0000 7fc40fe00640 1 --2- 172.21.15.123:0/2068009164 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc3f40a32d0 0x7fc3f40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:20.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.909+0000 7fc40d600640 1 --2- 172.21.15.123:0/2068009164 >> [v2:172.21.15.183:6800/1259721614,v1:172.21.15.183:6801/1259721614] conn(0x7fc3f40ace80 0x7fc3f40af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:20.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.909+0000 7fc40d600640 1 -- 172.21.15.123:0/2068009164 --> [v2:172.21.15.183:6800/1259721614,v1:172.21.15.183:6801/1259721614] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc3f40af9f0 con 0x7fc3f40ace80 2024-09-19T04:45:20.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.909+0000 7fc40d600640 1 -- 172.21.15.123:0/2068009164 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fc4080d07c0 con 0x7fc410108f60 2024-09-19T04:45:20.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.910+0000 7fc407e00640 1 --2- 172.21.15.123:0/2068009164 >> [v2:172.21.15.183:6800/1259721614,v1:172.21.15.183:6801/1259721614] conn(0x7fc3f40ace80 0x7fc3f40af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:20.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.910+0000 7fc40fe00640 1 --2- 172.21.15.123:0/2068009164 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc3f40a32d0 0x7fc3f40a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc400000c00 tx=0x7fc400005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:20.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.910+0000 7fc407e00640 1 --2- 172.21.15.123:0/2068009164 >> [v2:172.21.15.183:6800/1259721614,v1:172.21.15.183:6801/1259721614] conn(0x7fc3f40ace80 0x7fc3f40af320 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:20.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.911+0000 7fc40d600640 1 -- 172.21.15.123:0/2068009164 <== osd.1 v2:172.21.15.183:6800/1259721614 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3f40af9f0 con 0x7fc3f40ace80 2024-09-19T04:45:20.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.927+0000 7fc415c00640 1 -- 172.21.15.123:0/2068009164 --> [v2:172.21.15.183:6800/1259721614,v1:172.21.15.183:6801/1259721614] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc410062e00 con 0x7fc3f40ace80 2024-09-19T04:45:20.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.927+0000 7fc40d600640 1 -- 172.21.15.123:0/2068009164 <== osd.1 v2:172.21.15.183:6800/1259721614 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc410062e00 con 0x7fc3f40ace80 2024-09-19T04:45:20.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.928+0000 7fc415c00640 1 -- 172.21.15.123:0/2068009164 >> [v2:172.21.15.183:6800/1259721614,v1:172.21.15.183:6801/1259721614] conn(0x7fc3f40ace80 msgr2=0x7fc3f40af320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.928+0000 7fc415c00640 1 --2- 172.21.15.123:0/2068009164 >> [v2:172.21.15.183:6800/1259721614,v1:172.21.15.183:6801/1259721614] conn(0x7fc3f40ace80 0x7fc3f40af320 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.928+0000 7fc415c00640 1 -- 172.21.15.123:0/2068009164 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc3f40a32d0 msgr2=0x7fc3f40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.928+0000 7fc415c00640 1 --2- 172.21.15.123:0/2068009164 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fc3f40a32d0 0x7fc3f40a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc400000c00 tx=0x7fc400005b50 comp rx=0 tx=0).stop 2024-09-19T04:45:20.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.928+0000 7fc415c00640 1 -- 172.21.15.123:0/2068009164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc410108f60 msgr2=0x7fc41019eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.928+0000 7fc415c00640 1 --2- 172.21.15.123:0/2068009164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc410108f60 0x7fc41019eda0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc40800eb10 tx=0x7fc40800c2d0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.928+0000 7fc415200640 1 -- 172.21.15.123:0/2068009164 reap_dead start 2024-09-19T04:45:20.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.928+0000 7fc415c00640 1 -- 172.21.15.123:0/2068009164 shutdown_connections 2024-09-19T04:45:20.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.928+0000 7fc415c00640 1 -- 172.21.15.123:0/2068009164 >> 172.21.15.123:0/2068009164 conn(0x7fc4100ff8d0 msgr2=0x7fc410100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:20.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.928+0000 7fc415c00640 1 -- 172.21.15.123:0/2068009164 shutdown_connections 2024-09-19T04:45:20.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.928+0000 7fc415c00640 1 -- 172.21.15.123:0/2068009164 wait complete. 2024-09-19T04:45:20.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.042+0000 7f8d31800640 1 -- 172.21.15.123:0/675106451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d2c103e30 msgr2=0x7f8d2c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.042+0000 7f8d31800640 1 --2- 172.21.15.123:0/675106451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d2c103e30 0x7f8d2c104210 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009900 tx=0x7f8d1c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.042+0000 7f8d31800640 1 -- 172.21.15.123:0/675106451 shutdown_connections 2024-09-19T04:45:20.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.042+0000 7f8d31800640 1 --2- 172.21.15.123:0/675106451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d2c108f60 0x7f8d2c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.042+0000 7f8d31800640 1 --2- 172.21.15.123:0/675106451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d2c1047e0 0x7f8d2c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.042+0000 7f8d31800640 1 --2- 172.21.15.123:0/675106451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d2c103e30 0x7f8d2c104210 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.042+0000 7f8d31800640 1 -- 172.21.15.123:0/675106451 >> 172.21.15.123:0/675106451 conn(0x7f8d2c0ff8d0 msgr2=0x7f8d2c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:20.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.042+0000 7f8d31800640 1 -- 172.21.15.123:0/675106451 shutdown_connections 2024-09-19T04:45:20.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.042+0000 7f8d31800640 1 -- 172.21.15.123:0/675106451 wait complete. 2024-09-19T04:45:20.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.043+0000 7f8d31800640 1 Processor -- start 2024-09-19T04:45:20.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.043+0000 7f8d31800640 1 -- start start 2024-09-19T04:45:20.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.043+0000 7f8d31800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d2c103e30 0x7f8d2c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:20.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.043+0000 7f8d31800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d2c1047e0 0x7f8d2c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:20.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.043+0000 7f8d31800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d2c108f60 0x7f8d2c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:20.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.043+0000 7f8d31800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c113fc0 con 0x7f8d2c1047e0 2024-09-19T04:45:20.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.043+0000 7f8d31800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c113e40 con 0x7f8d2c103e30 2024-09-19T04:45:20.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.043+0000 7f8d31800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c114140 con 0x7f8d2c108f60 2024-09-19T04:45:20.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.043+0000 7f8d23e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d2c1047e0 0x7f8d2c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:20.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.044+0000 7f8d23e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d2c1047e0 0x7f8d2c19a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56454/0 (socket says 172.21.15.123:56454) 2024-09-19T04:45:20.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.044+0000 7f8d23e00640 1 -- 172.21.15.123:0/2340157126 learned_addr learned my addr 172.21.15.123:0/2340157126 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:20.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.044+0000 7f8d2be00640 1 --2- 172.21.15.123:0/2340157126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d2c103e30 0x7f8d2c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:20.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.044+0000 7f8d30e00640 1 --2- 172.21.15.123:0/2340157126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d2c108f60 0x7f8d2c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:20.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.044+0000 7f8d23e00640 1 -- 172.21.15.123:0/2340157126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d2c108f60 msgr2=0x7f8d2c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.044+0000 7f8d23e00640 1 --2- 172.21.15.123:0/2340157126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d2c108f60 0x7f8d2c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.044+0000 7f8d23e00640 1 -- 172.21.15.123:0/2340157126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d2c103e30 msgr2=0x7f8d2c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.044+0000 7f8d23e00640 1 --2- 172.21.15.123:0/2340157126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d2c103e30 0x7f8d2c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.044+0000 7f8d23e00640 1 -- 172.21.15.123:0/2340157126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2c19f410 con 0x7f8d2c1047e0 2024-09-19T04:45:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.044+0000 7f8d30e00640 1 --2- 172.21.15.123:0/2340157126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d2c108f60 0x7f8d2c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:45:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.044+0000 7f8d23e00640 1 --2- 172.21.15.123:0/2340157126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d2c1047e0 0x7f8d2c19a9a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400e9f0 tx=0x7f8d1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.044+0000 7f8d29600640 1 -- 172.21.15.123:0/2340157126 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400cd80 con 0x7f8d2c1047e0 2024-09-19T04:45:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.045+0000 7f8d31800640 1 -- 172.21.15.123:0/2340157126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2c19f700 con 0x7f8d2c1047e0 2024-09-19T04:45:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.045+0000 7f8d29600640 1 -- 172.21.15.123:0/2340157126 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d14010910 con 0x7f8d2c1047e0 2024-09-19T04:45:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.045+0000 7f8d29600640 1 -- 172.21.15.123:0/2340157126 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d140188f0 con 0x7f8d2c1047e0 2024-09-19T04:45:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.045+0000 7f8d31800640 1 -- 172.21.15.123:0/2340157126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d2c1a6fe0 con 0x7f8d2c1047e0 2024-09-19T04:45:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.046+0000 7f8d31800640 1 -- 172.21.15.123:0/2340157126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8d2c062940 con 0x7f8d2c1047e0 2024-09-19T04:45:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.047+0000 7f8d29600640 1 -- 172.21.15.123:0/2340157126 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8d14010ac0 con 0x7f8d2c1047e0 2024-09-19T04:45:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.049+0000 7f8d29600640 1 --2- 172.21.15.123:0/2340157126 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8d100a32d0 0x7f8d100a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.049+0000 7f8d29600640 1 -- 172.21.15.123:0/2340157126 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f8d140cff30 con 0x7f8d2c1047e0 2024-09-19T04:45:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.049+0000 7f8d2be00640 1 --2- 172.21.15.123:0/2340157126 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8d100a32d0 0x7f8d100a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.049+0000 7f8d29600640 1 --2- 172.21.15.123:0/2340157126 >> [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282] conn(0x7f8d100ace80 0x7f8d100af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.049+0000 7f8d29600640 1 -- 172.21.15.123:0/2340157126 --> [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d14012ec0 con 0x7f8d100ace80 2024-09-19T04:45:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.049+0000 7f8d29600640 1 -- 172.21.15.123:0/2340157126 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f8d140d2550 con 0x7f8d2c1047e0 2024-09-19T04:45:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.050+0000 7f8d30e00640 1 --2- 172.21.15.123:0/2340157126 >> [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282] conn(0x7f8d100ace80 0x7f8d100af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.050+0000 7f8d2be00640 1 --2- 172.21.15.123:0/2340157126 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8d100a32d0 0x7f8d100a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009230 tx=0x7f8d1c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.050+0000 7f8d30e00640 1 --2- 172.21.15.123:0/2340157126 >> [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282] conn(0x7f8d100ace80 0x7f8d100af320 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.051+0000 7f8d29600640 1 -- 172.21.15.123:0/2340157126 <== osd.3 v2:172.21.15.123:6810/3214806282 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d14012ec0 con 0x7f8d100ace80 2024-09-19T04:45:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.065+0000 7f8d31800640 1 -- 172.21.15.123:0/2340157126 --> [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8d2c062d70 con 0x7f8d100ace80 2024-09-19T04:45:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.066+0000 7f8d29600640 1 -- 172.21.15.123:0/2340157126 <== osd.3 v2:172.21.15.123:6810/3214806282 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8d2c062d70 con 0x7f8d100ace80 2024-09-19T04:45:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.066+0000 7f8d31800640 1 -- 172.21.15.123:0/2340157126 >> [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282] conn(0x7f8d100ace80 msgr2=0x7f8d100af320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.066+0000 7f8d31800640 1 --2- 172.21.15.123:0/2340157126 >> [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282] conn(0x7f8d100ace80 0x7f8d100af320 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.066+0000 7f8d31800640 1 -- 172.21.15.123:0/2340157126 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8d100a32d0 msgr2=0x7f8d100a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.066+0000 7f8d31800640 1 --2- 172.21.15.123:0/2340157126 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8d100a32d0 0x7f8d100a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009230 tx=0x7f8d1c0058d0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.066+0000 7f8d31800640 1 -- 172.21.15.123:0/2340157126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d2c1047e0 msgr2=0x7f8d2c19a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.066+0000 7f8d31800640 1 --2- 172.21.15.123:0/2340157126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d2c1047e0 0x7f8d2c19a9a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400e9f0 tx=0x7f8d1400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.066+0000 7f8d30e00640 1 -- 172.21.15.123:0/2340157126 reap_dead start 2024-09-19T04:45:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.067+0000 7f8d31800640 1 -- 172.21.15.123:0/2340157126 shutdown_connections 2024-09-19T04:45:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.067+0000 7f8d31800640 1 -- 172.21.15.123:0/2340157126 >> 172.21.15.123:0/2340157126 conn(0x7f8d2c0ff8d0 msgr2=0x7f8d2c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.067+0000 7f8d31800640 1 -- 172.21.15.123:0/2340157126 shutdown_connections 2024-09-19T04:45:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:20.067+0000 7f8d31800640 1 -- 172.21.15.123:0/2340157126 wait complete. 2024-09-19T04:45:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:20 smithi123 ceph-mon[30511]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-19T04:45:20.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.171+0000 7fe452200640 1 -- 172.21.15.123:0/1729266679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe44c10d530 msgr2=0x7fe44c1150b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.171+0000 7fe452200640 1 --2- 172.21.15.123:0/1729266679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe44c10d530 0x7fe44c1150b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe444009ca0 tx=0x7fe44402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.171+0000 7fe452200640 1 -- 172.21.15.123:0/1729266679 shutdown_connections 2024-09-19T04:45:20.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.171+0000 7fe452200640 1 --2- 172.21.15.123:0/1729266679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe44c10d530 0x7fe44c1150b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.171+0000 7fe452200640 1 --2- 172.21.15.123:0/1729266679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe44c108e00 0x7fe44c10ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.171+0000 7fe452200640 1 --2- 172.21.15.123:0/1729266679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe44c108450 0x7fe44c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.171+0000 7fe452200640 1 -- 172.21.15.123:0/1729266679 >> 172.21.15.123:0/1729266679 conn(0x7fe44c077ba0 msgr2=0x7fe44c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:20.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.171+0000 7fe452200640 1 -- 172.21.15.123:0/1729266679 shutdown_connections 2024-09-19T04:45:20.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.171+0000 7fe452200640 1 -- 172.21.15.123:0/1729266679 wait complete. 2024-09-19T04:45:20.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.172+0000 7fe452200640 1 Processor -- start 2024-09-19T04:45:20.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.172+0000 7fe452200640 1 -- start start 2024-09-19T04:45:20.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.172+0000 7fe452200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe44c108450 0x7fe44c1a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:20.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.172+0000 7fe452200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe44c108e00 0x7fe44c1a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:20.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.172+0000 7fe452200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe44c10d530 0x7fe44c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:20.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.172+0000 7fe452200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c118590 con 0x7fe44c108e00 2024-09-19T04:45:20.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.172+0000 7fe452200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c118410 con 0x7fe44c108450 2024-09-19T04:45:20.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.172+0000 7fe452200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c118710 con 0x7fe44c10d530 2024-09-19T04:45:20.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.173+0000 7fe44aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe44c108e00 0x7fe44c1a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:20.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.173+0000 7fe44b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe44c108450 0x7fe44c1a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:20.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.173+0000 7fe44aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe44c108e00 0x7fe44c1a4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56398/0 (socket says 172.21.15.123:56398) 2024-09-19T04:45:20.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.173+0000 7fe44b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe44c108450 0x7fe44c1a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:55036/0 (socket says 172.21.15.123:55036) 2024-09-19T04:45:20.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.173+0000 7fe44aa00640 1 -- 172.21.15.123:0/3512680784 learned_addr learned my addr 172.21.15.123:0/3512680784 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:20.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.173+0000 7fe44be00640 1 --2- 172.21.15.123:0/3512680784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe44c10d530 0x7fe44c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:20.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.173+0000 7fe44b400640 1 -- 172.21.15.123:0/3512680784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe44c10d530 msgr2=0x7fe44c19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.173+0000 7fe44b400640 1 --2- 172.21.15.123:0/3512680784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe44c10d530 0x7fe44c19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.173+0000 7fe44b400640 1 -- 172.21.15.123:0/3512680784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe44c108e00 msgr2=0x7fe44c1a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.173+0000 7fe44b400640 1 --2- 172.21.15.123:0/3512680784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe44c108e00 0x7fe44c1a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.173+0000 7fe44b400640 1 -- 172.21.15.123:0/3512680784 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c19efa0 con 0x7fe44c108450 2024-09-19T04:45:20.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.173+0000 7fe44aa00640 1 --2- 172.21.15.123:0/3512680784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe44c108e00 0x7fe44c1a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:20.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.173+0000 7fe44b400640 1 --2- 172.21.15.123:0/3512680784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe44c108450 0x7fe44c1a4680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe43c005fd0 tx=0x7fe43c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:20.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.174+0000 7fe443e00640 1 -- 172.21.15.123:0/3512680784 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c009c30 con 0x7fe44c108450 2024-09-19T04:45:20.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.174+0000 7fe452200640 1 -- 172.21.15.123:0/3512680784 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44c19f290 con 0x7fe44c108450 2024-09-19T04:45:20.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.174+0000 7fe443e00640 1 -- 172.21.15.123:0/3512680784 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe43c009dd0 con 0x7fe44c108450 2024-09-19T04:45:20.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.174+0000 7fe443e00640 1 -- 172.21.15.123:0/3512680784 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c010460 con 0x7fe44c108450 2024-09-19T04:45:20.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.174+0000 7fe452200640 1 -- 172.21.15.123:0/3512680784 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44c1ab4e0 con 0x7fe44c108450 2024-09-19T04:45:20.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.175+0000 7fe452200640 1 -- 172.21.15.123:0/3512680784 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe44c079180 con 0x7fe44c108450 2024-09-19T04:45:20.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.177+0000 7fe443e00640 1 -- 172.21.15.123:0/3512680784 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe43c003ef0 con 0x7fe44c108450 2024-09-19T04:45:20.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.178+0000 7fe443e00640 1 --2- 172.21.15.123:0/3512680784 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe4300a32d0 0x7fe4300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:20.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.178+0000 7fe443e00640 1 -- 172.21.15.123:0/3512680784 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fe43c0cf9a0 con 0x7fe44c108450 2024-09-19T04:45:20.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.178+0000 7fe44aa00640 1 --2- 172.21.15.123:0/3512680784 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe4300a32d0 0x7fe4300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:20.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.179+0000 7fe443e00640 1 --2- 172.21.15.123:0/3512680784 >> [v2:172.21.15.183:6808/667618665,v1:172.21.15.183:6809/667618665] conn(0x7fe4300ace80 0x7fe4300af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:20.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.179+0000 7fe443e00640 1 -- 172.21.15.123:0/3512680784 --> [v2:172.21.15.183:6808/667618665,v1:172.21.15.183:6809/667618665] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe43c00d130 con 0x7fe4300ace80 2024-09-19T04:45:20.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.179+0000 7fe443e00640 1 -- 172.21.15.123:0/3512680784 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fe43c0cfd50 con 0x7fe44c108450 2024-09-19T04:45:20.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.179+0000 7fe44be00640 1 --2- 172.21.15.123:0/3512680784 >> [v2:172.21.15.183:6808/667618665,v1:172.21.15.183:6809/667618665] conn(0x7fe4300ace80 0x7fe4300af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:20.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.179+0000 7fe44aa00640 1 --2- 172.21.15.123:0/3512680784 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe4300a32d0 0x7fe4300a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe44c107a40 tx=0x7fe438005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:20.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.180+0000 7fe44be00640 1 --2- 172.21.15.123:0/3512680784 >> [v2:172.21.15.183:6808/667618665,v1:172.21.15.183:6809/667618665] conn(0x7fe4300ace80 0x7fe4300af320 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:20.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.181+0000 7fe443e00640 1 -- 172.21.15.123:0/3512680784 <== osd.4 v2:172.21.15.183:6808/667618665 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe43c00d130 con 0x7fe4300ace80 2024-09-19T04:45:20.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.195+0000 7fe452200640 1 -- 172.21.15.123:0/3512680784 --> [v2:172.21.15.183:6808/667618665,v1:172.21.15.183:6809/667618665] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe44c1098e0 con 0x7fe4300ace80 2024-09-19T04:45:20.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.196+0000 7fe443e00640 1 -- 172.21.15.123:0/3512680784 <== osd.4 v2:172.21.15.183:6808/667618665 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe44c1098e0 con 0x7fe4300ace80 2024-09-19T04:45:20.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.196+0000 7fe452200640 1 -- 172.21.15.123:0/3512680784 >> [v2:172.21.15.183:6808/667618665,v1:172.21.15.183:6809/667618665] conn(0x7fe4300ace80 msgr2=0x7fe4300af320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.196+0000 7fe452200640 1 --2- 172.21.15.123:0/3512680784 >> [v2:172.21.15.183:6808/667618665,v1:172.21.15.183:6809/667618665] conn(0x7fe4300ace80 0x7fe4300af320 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.196+0000 7fe452200640 1 -- 172.21.15.123:0/3512680784 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe4300a32d0 msgr2=0x7fe4300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.196+0000 7fe452200640 1 --2- 172.21.15.123:0/3512680784 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe4300a32d0 0x7fe4300a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe44c107a40 tx=0x7fe438005f90 comp rx=0 tx=0).stop 2024-09-19T04:45:20.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.196+0000 7fe452200640 1 -- 172.21.15.123:0/3512680784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe44c108450 msgr2=0x7fe44c1a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:20.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.196+0000 7fe452200640 1 --2- 172.21.15.123:0/3512680784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe44c108450 0x7fe44c1a4680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe43c005fd0 tx=0x7fe43c00dde0 comp rx=0 tx=0).stop 2024-09-19T04:45:20.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.196+0000 7fe44be00640 1 -- 172.21.15.123:0/3512680784 reap_dead start 2024-09-19T04:45:20.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.197+0000 7fe452200640 1 -- 172.21.15.123:0/3512680784 shutdown_connections 2024-09-19T04:45:20.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.197+0000 7fe452200640 1 -- 172.21.15.123:0/3512680784 >> 172.21.15.123:0/3512680784 conn(0x7fe44c077ba0 msgr2=0x7fe44c078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:20.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.197+0000 7fe452200640 1 -- 172.21.15.123:0/3512680784 shutdown_connections 2024-09-19T04:45:20.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:19.197+0000 7fe452200640 1 -- 172.21.15.123:0/3512680784 wait complete. 2024-09-19T04:45:20.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:20 smithi193 ceph-mon[35592]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-19T04:45:20.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:20 smithi183 ceph-mon[35333]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-19T04:45:21.162 INFO:teuthology.orchestra.run.smithi123.stdout:214748364843 2024-09-19T04:45:21.163 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T04:45:21.536 INFO:teuthology.orchestra.run.smithi123.stdout:137438953532 2024-09-19T04:45:21.537 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T04:45:22.267 INFO:teuthology.orchestra.run.smithi123.stdout:34359738449 2024-09-19T04:45:22.268 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T04:45:22.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:22 smithi123 ceph-mon[30511]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T04:45:22.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:22 smithi193 ceph-mon[35592]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T04:45:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:22 smithi183 ceph-mon[35333]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T04:45:22.959 INFO:teuthology.orchestra.run.smithi123.stdout:193273528371 2024-09-19T04:45:22.960 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T04:45:23.333 INFO:teuthology.orchestra.run.smithi123.stdout:163208757303 2024-09-19T04:45:23.334 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T04:45:24.032 INFO:teuthology.orchestra.run.smithi123.stdout:55834574924 2024-09-19T04:45:24.033 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T04:45:24.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.233+0000 7f9a7bc00640 1 -- 172.21.15.123:0/2543978784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a74103ff0 msgr2=0x7f9a741043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:24.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.233+0000 7f9a7bc00640 1 --2- 172.21.15.123:0/2543978784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a74103ff0 0x7f9a741043d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009900 tx=0x7f9a6802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:45:24.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.234+0000 7f9a7bc00640 1 -- 172.21.15.123:0/2543978784 shutdown_connections 2024-09-19T04:45:24.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.234+0000 7f9a7bc00640 1 --2- 172.21.15.123:0/2543978784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a74109120 0x7f9a74110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:24.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.234+0000 7f9a7bc00640 1 --2- 172.21.15.123:0/2543978784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a741049a0 0x7f9a741089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:24.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.234+0000 7f9a7bc00640 1 --2- 172.21.15.123:0/2543978784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a74103ff0 0x7f9a741043d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:24.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.234+0000 7f9a7bc00640 1 -- 172.21.15.123:0/2543978784 >> 172.21.15.123:0/2543978784 conn(0x7f9a740ffa90 msgr2=0x7f9a74101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:24.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.234+0000 7f9a7bc00640 1 -- 172.21.15.123:0/2543978784 shutdown_connections 2024-09-19T04:45:24.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.234+0000 7f9a7bc00640 1 -- 172.21.15.123:0/2543978784 wait complete. 2024-09-19T04:45:24.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.235+0000 7f9a7bc00640 1 Processor -- start 2024-09-19T04:45:24.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.235+0000 7f9a7bc00640 1 -- start start 2024-09-19T04:45:24.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.236+0000 7f9a7bc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a74103ff0 0x7f9a7419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:24.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.236+0000 7f9a7bc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a741049a0 0x7f9a7419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:24.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.236+0000 7f9a7bc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a74109120 0x7f9a7419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:24.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.236+0000 7f9a7bc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9a74114170 con 0x7f9a741049a0 2024-09-19T04:45:24.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.236+0000 7f9a7bc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9a74113ff0 con 0x7f9a74109120 2024-09-19T04:45:24.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.236+0000 7f9a7bc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a741142f0 con 0x7f9a74103ff0 2024-09-19T04:45:24.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.236+0000 7f9a79200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a74103ff0 0x7f9a7419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:24.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.236+0000 7f9a73e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a741049a0 0x7f9a7419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:24.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.236+0000 7f9a79200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a74103ff0 0x7f9a7419a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42772/0 (socket says 172.21.15.123:42772) 2024-09-19T04:45:24.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.236+0000 7f9a79200640 1 -- 172.21.15.123:0/2552387074 learned_addr learned my addr 172.21.15.123:0/2552387074 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:24.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.236+0000 7f9a79c00640 1 --2- 172.21.15.123:0/2552387074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a74109120 0x7f9a7419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:24.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.237+0000 7f9a79200640 1 -- 172.21.15.123:0/2552387074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a74109120 msgr2=0x7f9a7419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:24.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.237+0000 7f9a79200640 1 --2- 172.21.15.123:0/2552387074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a74109120 0x7f9a7419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:24.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.237+0000 7f9a79200640 1 -- 172.21.15.123:0/2552387074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a741049a0 msgr2=0x7f9a7419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:24.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.237+0000 7f9a79200640 1 --2- 172.21.15.123:0/2552387074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a741049a0 0x7f9a7419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:24.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.237+0000 7f9a79200640 1 -- 172.21.15.123:0/2552387074 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7419f5b0 con 0x7f9a74103ff0 2024-09-19T04:45:24.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.237+0000 7f9a79200640 1 --2- 172.21.15.123:0/2552387074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a74103ff0 0x7f9a7419a600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9a68004770 tx=0x7f9a68003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:24.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.237+0000 7f9a71600640 1 -- 172.21.15.123:0/2552387074 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6803f070 con 0x7f9a74103ff0 2024-09-19T04:45:24.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.238+0000 7f9a7bc00640 1 -- 172.21.15.123:0/2552387074 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7419f840 con 0x7f9a74103ff0 2024-09-19T04:45:24.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.238+0000 7f9a71600640 1 -- 172.21.15.123:0/2552387074 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a68031ca0 con 0x7f9a74103ff0 2024-09-19T04:45:24.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.238+0000 7f9a71600640 1 -- 172.21.15.123:0/2552387074 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a68035030 con 0x7f9a74103ff0 2024-09-19T04:45:24.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.238+0000 7f9a7bc00640 1 -- 172.21.15.123:0/2552387074 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a741a7180 con 0x7f9a74103ff0 2024-09-19T04:45:24.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.240+0000 7f9a7bc00640 1 -- 172.21.15.123:0/2552387074 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9a74062980 con 0x7f9a74103ff0 2024-09-19T04:45:24.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.242+0000 7f9a71600640 1 -- 172.21.15.123:0/2552387074 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9a68042070 con 0x7f9a74103ff0 2024-09-19T04:45:24.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.243+0000 7f9a71600640 1 --2- 172.21.15.123:0/2552387074 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9a5c0a32d0 0x7f9a5c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:24.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.244+0000 7f9a71600640 1 -- 172.21.15.123:0/2552387074 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f9a680f0750 con 0x7f9a74103ff0 2024-09-19T04:45:24.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.244+0000 7f9a71600640 1 --2- 172.21.15.123:0/2552387074 >> [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149] conn(0x7f9a5c0ace80 0x7f9a5c0af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:24.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.244+0000 7f9a71600640 1 -- 172.21.15.123:0/2552387074 --> [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9a68038e90 con 0x7f9a5c0ace80 2024-09-19T04:45:24.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.244+0000 7f9a71600640 1 -- 172.21.15.123:0/2552387074 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f9a680f0b00 con 0x7f9a74103ff0 2024-09-19T04:45:24.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.244+0000 7f9a79c00640 1 --2- 172.21.15.123:0/2552387074 >> [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149] conn(0x7f9a5c0ace80 0x7f9a5c0af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:24.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.244+0000 7f9a73e00640 1 --2- 172.21.15.123:0/2552387074 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9a5c0a32d0 0x7f9a5c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:24.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.244+0000 7f9a79c00640 1 --2- 172.21.15.123:0/2552387074 >> [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149] conn(0x7f9a5c0ace80 0x7f9a5c0af320 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:24.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.246+0000 7f9a73e00640 1 --2- 172.21.15.123:0/2552387074 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9a5c0a32d0 0x7f9a5c0a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9a740fff90 tx=0x7f9a6400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:24.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.246+0000 7f9a71600640 1 -- 172.21.15.123:0/2552387074 <== osd.9 v2:172.21.15.123:6826/1439901149 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9a68038e90 con 0x7f9a5c0ace80 2024-09-19T04:45:24.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.261+0000 7f9a7bc00640 1 -- 172.21.15.123:0/2552387074 --> [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9a74062d80 con 0x7f9a5c0ace80 2024-09-19T04:45:24.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.263+0000 7f9a71600640 1 -- 172.21.15.123:0/2552387074 <== osd.9 v2:172.21.15.123:6826/1439901149 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9a74062d80 con 0x7f9a5c0ace80 2024-09-19T04:45:24.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.263+0000 7f9a7bc00640 1 -- 172.21.15.123:0/2552387074 >> [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149] conn(0x7f9a5c0ace80 msgr2=0x7f9a5c0af320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:24.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.263+0000 7f9a7bc00640 1 --2- 172.21.15.123:0/2552387074 >> [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149] conn(0x7f9a5c0ace80 0x7f9a5c0af320 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:24.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.263+0000 7f9a7bc00640 1 -- 172.21.15.123:0/2552387074 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9a5c0a32d0 msgr2=0x7f9a5c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:24.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.263+0000 7f9a7bc00640 1 --2- 172.21.15.123:0/2552387074 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9a5c0a32d0 0x7f9a5c0a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9a740fff90 tx=0x7f9a6400b410 comp rx=0 tx=0).stop 2024-09-19T04:45:24.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.263+0000 7f9a7bc00640 1 -- 172.21.15.123:0/2552387074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a74103ff0 msgr2=0x7f9a7419a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:24.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.263+0000 7f9a7bc00640 1 --2- 172.21.15.123:0/2552387074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a74103ff0 0x7f9a7419a600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9a68004770 tx=0x7f9a68003c10 comp rx=0 tx=0).stop 2024-09-19T04:45:24.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.263+0000 7f9a79c00640 1 -- 172.21.15.123:0/2552387074 reap_dead start 2024-09-19T04:45:24.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.263+0000 7f9a7bc00640 1 -- 172.21.15.123:0/2552387074 shutdown_connections 2024-09-19T04:45:24.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.263+0000 7f9a7bc00640 1 -- 172.21.15.123:0/2552387074 >> 172.21.15.123:0/2552387074 conn(0x7f9a740ffa90 msgr2=0x7f9a74100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:24.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.264+0000 7f9a7bc00640 1 -- 172.21.15.123:0/2552387074 shutdown_connections 2024-09-19T04:45:24.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.264+0000 7f9a7bc00640 1 -- 172.21.15.123:0/2552387074 wait complete. 2024-09-19T04:45:24.547 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:24 smithi123 ceph-mon[30511]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T04:45:24.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.553+0000 7ff120400640 1 -- 172.21.15.123:0/381485088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff118106f20 msgr2=0x7ff11810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:24.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.553+0000 7ff120400640 1 --2- 172.21.15.123:0/381485088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff118106f20 0x7ff11810eaa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff110009ca0 tx=0x7ff11002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:45:24.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.553+0000 7ff120400640 1 -- 172.21.15.123:0/381485088 shutdown_connections 2024-09-19T04:45:24.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.553+0000 7ff120400640 1 --2- 172.21.15.123:0/381485088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff118106f20 0x7ff11810eaa0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:24.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.553+0000 7ff120400640 1 --2- 172.21.15.123:0/381485088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1181027a0 0x7ff1181067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:24.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.553+0000 7ff120400640 1 --2- 172.21.15.123:0/381485088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff118101df0 0x7ff1181021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:24.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.553+0000 7ff120400640 1 -- 172.21.15.123:0/381485088 >> 172.21.15.123:0/381485088 conn(0x7ff1180fd8b0 msgr2=0x7ff1180ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:24.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.554+0000 7ff120400640 1 -- 172.21.15.123:0/381485088 shutdown_connections 2024-09-19T04:45:24.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.554+0000 7ff120400640 1 -- 172.21.15.123:0/381485088 wait complete. 2024-09-19T04:45:24.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.554+0000 7ff120400640 1 Processor -- start 2024-09-19T04:45:24.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.555+0000 7ff120400640 1 -- start start 2024-09-19T04:45:24.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.556+0000 7ff120400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff118101df0 0x7ff11819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:24.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.556+0000 7ff120400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1181027a0 0x7ff11819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:24.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.556+0000 7ff120400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff118106f20 0x7ff11819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:24.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.556+0000 7ff120400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff118111f80 con 0x7ff118101df0 2024-09-19T04:45:24.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.556+0000 7ff120400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff118111e00 con 0x7ff1181027a0 2024-09-19T04:45:24.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.556+0000 7ff120400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff118112100 con 0x7ff118106f20 2024-09-19T04:45:24.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.556+0000 7ff11da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff118101df0 0x7ff11819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:24.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.556+0000 7ff11d000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1181027a0 0x7ff11819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:24.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.556+0000 7ff11da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff118101df0 0x7ff11819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56616/0 (socket says 172.21.15.123:56616) 2024-09-19T04:45:24.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.556+0000 7ff11da00640 1 -- 172.21.15.123:0/2737585062 learned_addr learned my addr 172.21.15.123:0/2737585062 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:24.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.556+0000 7ff11d000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1181027a0 0x7ff11819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:59760/0 (socket says 172.21.15.123:59760) 2024-09-19T04:45:24.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.556+0000 7ff11e400640 1 --2- 172.21.15.123:0/2737585062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff118106f20 0x7ff11819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:24.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.557+0000 7ff11d000640 1 -- 172.21.15.123:0/2737585062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff118106f20 msgr2=0x7ff11819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:24.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.557+0000 7ff11d000640 1 --2- 172.21.15.123:0/2737585062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff118106f20 0x7ff11819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:24.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.557+0000 7ff11d000640 1 -- 172.21.15.123:0/2737585062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff118101df0 msgr2=0x7ff11819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:24.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.557+0000 7ff11d000640 1 --2- 172.21.15.123:0/2737585062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff118101df0 0x7ff11819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:24.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.557+0000 7ff11d000640 1 -- 172.21.15.123:0/2737585062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11819f560 con 0x7ff1181027a0 2024-09-19T04:45:24.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.557+0000 7ff11d000640 1 --2- 172.21.15.123:0/2737585062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1181027a0 0x7ff11819aaf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff10400e9f0 tx=0x7ff10400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:24.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.557+0000 7ff116000640 1 -- 172.21.15.123:0/2737585062 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10400ccc0 con 0x7ff1181027a0 2024-09-19T04:45:24.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.557+0000 7ff120400640 1 -- 172.21.15.123:0/2737585062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11819f850 con 0x7ff1181027a0 2024-09-19T04:45:24.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.558+0000 7ff116000640 1 -- 172.21.15.123:0/2737585062 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff104005020 con 0x7ff1181027a0 2024-09-19T04:45:24.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.558+0000 7ff116000640 1 -- 172.21.15.123:0/2737585062 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1040049e0 con 0x7ff1181027a0 2024-09-19T04:45:24.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.558+0000 7ff120400640 1 -- 172.21.15.123:0/2737585062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1181a7130 con 0x7ff1181027a0 2024-09-19T04:45:24.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.559+0000 7ff120400640 1 -- 172.21.15.123:0/2737585062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff0f8001030 con 0x7ff1181027a0 2024-09-19T04:45:24.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.560+0000 7ff116000640 1 -- 172.21.15.123:0/2737585062 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff104004cc0 con 0x7ff1181027a0 2024-09-19T04:45:24.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.562+0000 7ff116000640 1 --2- 172.21.15.123:0/2737585062 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff1000a3350 0x7ff1000a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:24.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.562+0000 7ff116000640 1 -- 172.21.15.123:0/2737585062 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7ff1040cf820 con 0x7ff1181027a0 2024-09-19T04:45:24.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.562+0000 7ff11da00640 1 --2- 172.21.15.123:0/2737585062 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff1000a3350 0x7ff1000a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:24.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.562+0000 7ff116000640 1 --2- 172.21.15.123:0/2737585062 >> [v2:172.21.15.193:6824/4291617038,v1:172.21.15.193:6825/4291617038] conn(0x7ff1000acf00 0x7ff1000af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:24.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.562+0000 7ff116000640 1 -- 172.21.15.123:0/2737585062 --> [v2:172.21.15.193:6824/4291617038,v1:172.21.15.193:6825/4291617038] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1000afa70 con 0x7ff1000acf00 2024-09-19T04:45:24.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.562+0000 7ff116000640 1 -- 172.21.15.123:0/2737585062 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7ff1040d6050 con 0x7ff1181027a0 2024-09-19T04:45:24.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.563+0000 7ff11da00640 1 --2- 172.21.15.123:0/2737585062 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff1000a3350 0x7ff1000a5810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009c90 tx=0x7ff10c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:24.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.563+0000 7ff11e400640 1 --2- 172.21.15.123:0/2737585062 >> [v2:172.21.15.193:6824/4291617038,v1:172.21.15.193:6825/4291617038] conn(0x7ff1000acf00 0x7ff1000af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:24.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.564+0000 7ff11e400640 1 --2- 172.21.15.123:0/2737585062 >> [v2:172.21.15.193:6824/4291617038,v1:172.21.15.193:6825/4291617038] conn(0x7ff1000acf00 0x7ff1000af3a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:24.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.565+0000 7ff116000640 1 -- 172.21.15.123:0/2737585062 <== osd.11 v2:172.21.15.193:6824/4291617038 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1000afa70 con 0x7ff1000acf00 2024-09-19T04:45:24.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.580+0000 7ff120400640 1 -- 172.21.15.123:0/2737585062 --> [v2:172.21.15.193:6824/4291617038,v1:172.21.15.193:6825/4291617038] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff0f8002dd0 con 0x7ff1000acf00 2024-09-19T04:45:24.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.580+0000 7ff116000640 1 -- 172.21.15.123:0/2737585062 <== osd.11 v2:172.21.15.193:6824/4291617038 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff0f8002dd0 con 0x7ff1000acf00 2024-09-19T04:45:24.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.581+0000 7ff120400640 1 -- 172.21.15.123:0/2737585062 >> [v2:172.21.15.193:6824/4291617038,v1:172.21.15.193:6825/4291617038] conn(0x7ff1000acf00 msgr2=0x7ff1000af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:24.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.581+0000 7ff120400640 1 --2- 172.21.15.123:0/2737585062 >> [v2:172.21.15.193:6824/4291617038,v1:172.21.15.193:6825/4291617038] conn(0x7ff1000acf00 0x7ff1000af3a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:24.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.581+0000 7ff120400640 1 -- 172.21.15.123:0/2737585062 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff1000a3350 msgr2=0x7ff1000a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:24.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.581+0000 7ff120400640 1 --2- 172.21.15.123:0/2737585062 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff1000a3350 0x7ff1000a5810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009c90 tx=0x7ff10c005f90 comp rx=0 tx=0).stop 2024-09-19T04:45:24.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.581+0000 7ff120400640 1 -- 172.21.15.123:0/2737585062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1181027a0 msgr2=0x7ff11819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:24.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.581+0000 7ff120400640 1 --2- 172.21.15.123:0/2737585062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1181027a0 0x7ff11819aaf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff10400e9f0 tx=0x7ff10400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:45:24.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.581+0000 7ff11e400640 1 -- 172.21.15.123:0/2737585062 reap_dead start 2024-09-19T04:45:24.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.581+0000 7ff120400640 1 -- 172.21.15.123:0/2737585062 shutdown_connections 2024-09-19T04:45:24.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.581+0000 7ff120400640 1 -- 172.21.15.123:0/2737585062 >> 172.21.15.123:0/2737585062 conn(0x7ff1180fd8b0 msgr2=0x7ff1180ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:24.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.581+0000 7ff120400640 1 -- 172.21.15.123:0/2737585062 shutdown_connections 2024-09-19T04:45:24.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:24.582+0000 7ff120400640 1 -- 172.21.15.123:0/2737585062 wait complete. 2024-09-19T04:45:24.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:24 smithi193 ceph-mon[35592]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T04:45:24.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:24 smithi183 ceph-mon[35333]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T04:45:25.212 INFO:teuthology.orchestra.run.smithi123.stdout:77309411399 2024-09-19T04:45:25.212 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T04:45:25.725 INFO:teuthology.orchestra.run.smithi123.stdout:107374182466 2024-09-19T04:45:25.725 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T04:45:26.552 INFO:teuthology.orchestra.run.smithi123.stdout:279172874275 2024-09-19T04:45:26.553 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-19T04:45:26.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:26 smithi193 ceph-mon[35592]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:45:26.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:26 smithi183 ceph-mon[35333]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:45:26.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:26 smithi123 ceph-mon[30511]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:45:27.058 INFO:teuthology.orchestra.run.smithi123.stdout:330712481818 2024-09-19T04:45:27.058 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-19T04:45:27.104 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:27.105 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:27.112 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:27.114 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:27.198 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:27.216 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:27.217 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:27.263 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:27.266 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:27.606 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:27 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:45:27.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:27 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:45:27.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:27 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:45:28.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.916+0000 7f4b41600640 1 -- 172.21.15.123:0/2277646981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b3c101df0 msgr2=0x7f4b3c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:28.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.916+0000 7f4b41600640 1 --2- 172.21.15.123:0/2277646981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b3c101df0 0x7f4b3c1021d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009900 tx=0x7f4b2c02a400 comp rx=0 tx=0).stop 2024-09-19T04:45:28.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.916+0000 7f4b41600640 1 -- 172.21.15.123:0/2277646981 shutdown_connections 2024-09-19T04:45:28.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.916+0000 7f4b41600640 1 --2- 172.21.15.123:0/2277646981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b3c106f20 0x7f4b3c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.916+0000 7f4b41600640 1 --2- 172.21.15.123:0/2277646981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b3c1027a0 0x7f4b3c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.916+0000 7f4b41600640 1 --2- 172.21.15.123:0/2277646981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b3c101df0 0x7f4b3c1021d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.916+0000 7f4b41600640 1 -- 172.21.15.123:0/2277646981 >> 172.21.15.123:0/2277646981 conn(0x7f4b3c0fd8b0 msgr2=0x7f4b3c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:28.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.916+0000 7f4b41600640 1 -- 172.21.15.123:0/2277646981 shutdown_connections 2024-09-19T04:45:28.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.916+0000 7f4b41600640 1 -- 172.21.15.123:0/2277646981 wait complete. 2024-09-19T04:45:28.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.917+0000 7f4b41600640 1 Processor -- start 2024-09-19T04:45:28.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.918+0000 7f4b41600640 1 -- start start 2024-09-19T04:45:28.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.918+0000 7f4b41600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b3c101df0 0x7f4b3c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:28.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.918+0000 7f4b41600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b3c1027a0 0x7f4b3c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:28.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.918+0000 7f4b41600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b3c106f20 0x7f4b3c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:28.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.918+0000 7f4b41600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c111f80 con 0x7f4b3c1027a0 2024-09-19T04:45:28.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.918+0000 7f4b41600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c111e00 con 0x7f4b3c101df0 2024-09-19T04:45:28.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.918+0000 7f4b41600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c112100 con 0x7f4b3c106f20 2024-09-19T04:45:28.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.918+0000 7f4b3a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b3c1027a0 0x7f4b3c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:28.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.918+0000 7f4b3aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b3c101df0 0x7f4b3c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:28.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.918+0000 7f4b3a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b3c1027a0 0x7f4b3c19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56636/0 (socket says 172.21.15.123:56636) 2024-09-19T04:45:28.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.918+0000 7f4b3a000640 1 -- 172.21.15.123:0/1269098184 learned_addr learned my addr 172.21.15.123:0/1269098184 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:28.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.918+0000 7f4b3b400640 1 --2- 172.21.15.123:0/1269098184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b3c106f20 0x7f4b3c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:28.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.919+0000 7f4b3aa00640 1 -- 172.21.15.123:0/1269098184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b3c106f20 msgr2=0x7f4b3c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:28.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.919+0000 7f4b3aa00640 1 --2- 172.21.15.123:0/1269098184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b3c106f20 0x7f4b3c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.919+0000 7f4b3aa00640 1 -- 172.21.15.123:0/1269098184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b3c1027a0 msgr2=0x7f4b3c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:28.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.919+0000 7f4b3aa00640 1 --2- 172.21.15.123:0/1269098184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b3c1027a0 0x7f4b3c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.919+0000 7f4b3aa00640 1 -- 172.21.15.123:0/1269098184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c19f5a0 con 0x7f4b3c101df0 2024-09-19T04:45:28.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.919+0000 7f4b3a000640 1 --2- 172.21.15.123:0/1269098184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b3c1027a0 0x7f4b3c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:28.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.919+0000 7f4b3aa00640 1 --2- 172.21.15.123:0/1269098184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b3c101df0 0x7f4b3c19a5f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c005940 tx=0x7f4b2c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:28.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.919+0000 7f4b33400640 1 -- 172.21.15.123:0/1269098184 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c03f030 con 0x7f4b3c101df0 2024-09-19T04:45:28.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.919+0000 7f4b41600640 1 -- 172.21.15.123:0/1269098184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3c19f830 con 0x7f4b3c101df0 2024-09-19T04:45:28.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.919+0000 7f4b33400640 1 -- 172.21.15.123:0/1269098184 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c0044c0 con 0x7f4b3c101df0 2024-09-19T04:45:28.245 INFO:teuthology.orchestra.run.smithi123.stdout:163208757305 2024-09-19T04:45:28.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.919+0000 7f4b33400640 1 -- 172.21.15.123:0/1269098184 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c035070 con 0x7f4b3c101df0 2024-09-19T04:45:28.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.920+0000 7f4b41600640 1 -- 172.21.15.123:0/1269098184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3c1a7170 con 0x7f4b3c101df0 2024-09-19T04:45:28.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.921+0000 7f4b41600640 1 -- 172.21.15.123:0/1269098184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b3c1021d0 con 0x7f4b3c101df0 2024-09-19T04:45:28.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.923+0000 7f4b33400640 1 -- 172.21.15.123:0/1269098184 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4b2c042030 con 0x7f4b3c101df0 2024-09-19T04:45:28.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.924+0000 7f4b33400640 1 --2- 172.21.15.123:0/1269098184 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4b200a3350 0x7f4b200a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:28.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.924+0000 7f4b33400640 1 -- 172.21.15.123:0/1269098184 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f4b2c0efff0 con 0x7f4b3c101df0 2024-09-19T04:45:28.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.925+0000 7f4b3a000640 1 --2- 172.21.15.123:0/1269098184 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4b200a3350 0x7f4b200a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:28.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.926+0000 7f4b33400640 1 -- 172.21.15.123:0/1269098184 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4b2c0bc510 con 0x7f4b3c101df0 2024-09-19T04:45:28.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:27.926+0000 7f4b3a000640 1 --2- 172.21.15.123:0/1269098184 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4b200a3350 0x7f4b200a5810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c076480 tx=0x7f4b28005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:28.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.114+0000 7f4b41600640 1 -- 172.21.15.123:0/1269098184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4b3c1a7340 con 0x7f4b3c101df0 2024-09-19T04:45:28.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.114+0000 7f4b33400640 1 -- 172.21.15.123:0/1269098184 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4b2c0c0810 con 0x7f4b3c101df0 2024-09-19T04:45:28.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.117+0000 7f4b41600640 1 -- 172.21.15.123:0/1269098184 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4b200a3350 msgr2=0x7f4b200a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:28.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.117+0000 7f4b41600640 1 --2- 172.21.15.123:0/1269098184 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4b200a3350 0x7f4b200a5810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c076480 tx=0x7f4b28005f90 comp rx=0 tx=0).stop 2024-09-19T04:45:28.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.117+0000 7f4b41600640 1 -- 172.21.15.123:0/1269098184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b3c101df0 msgr2=0x7f4b3c19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:28.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.117+0000 7f4b41600640 1 --2- 172.21.15.123:0/1269098184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b3c101df0 0x7f4b3c19a5f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c005940 tx=0x7f4b2c0040c0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.117+0000 7f4b41600640 1 -- 172.21.15.123:0/1269098184 shutdown_connections 2024-09-19T04:45:28.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.117+0000 7f4b41600640 1 --2- 172.21.15.123:0/1269098184 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4b200a3350 0x7f4b200a5810 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.117+0000 7f4b41600640 1 --2- 172.21.15.123:0/1269098184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b3c106f20 0x7f4b3c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.117+0000 7f4b41600640 1 --2- 172.21.15.123:0/1269098184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b3c1027a0 0x7f4b3c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.117+0000 7f4b41600640 1 --2- 172.21.15.123:0/1269098184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b3c101df0 0x7f4b3c19a5f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.117+0000 7f4b41600640 1 -- 172.21.15.123:0/1269098184 >> 172.21.15.123:0/1269098184 conn(0x7f4b3c0fd8b0 msgr2=0x7f4b3c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:28.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.117+0000 7f4b41600640 1 -- 172.21.15.123:0/1269098184 shutdown_connections 2024-09-19T04:45:28.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.117+0000 7f4b41600640 1 -- 172.21.15.123:0/1269098184 wait complete. 2024-09-19T04:45:28.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.700+0000 7f2289400640 1 -- 172.21.15.123:0/2938777395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2284074f40 msgr2=0x7f228410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:28.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.700+0000 7f2289400640 1 --2- 172.21.15.123:0/2938777395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2284074f40 0x7f228410de80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2270009900 tx=0x7f227002a480 comp rx=0 tx=0).stop 2024-09-19T04:45:28.702 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:28 smithi123 ceph-mon[30511]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:45:28.702 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:28 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1269098184' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T04:45:28.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.700+0000 7f2289400640 1 -- 172.21.15.123:0/2938777395 shutdown_connections 2024-09-19T04:45:28.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.700+0000 7f2289400640 1 --2- 172.21.15.123:0/2938777395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f228410e3c0 0x7f2284110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.700+0000 7f2289400640 1 --2- 172.21.15.123:0/2938777395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2284074f40 0x7f228410de80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.700+0000 7f2289400640 1 --2- 172.21.15.123:0/2938777395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2284074620 0x7f2284074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.700+0000 7f2289400640 1 -- 172.21.15.123:0/2938777395 >> 172.21.15.123:0/2938777395 conn(0x7f22840fd8c0 msgr2=0x7f22840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:28.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.700+0000 7f2289400640 1 -- 172.21.15.123:0/2938777395 shutdown_connections 2024-09-19T04:45:28.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.700+0000 7f2289400640 1 -- 172.21.15.123:0/2938777395 wait complete. 2024-09-19T04:45:28.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.702+0000 7f2289400640 1 Processor -- start 2024-09-19T04:45:28.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.702+0000 7f2289400640 1 -- start start 2024-09-19T04:45:28.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.702+0000 7f2289400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2284074620 0x7f228419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:28.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.702+0000 7f2289400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2284074f40 0x7f228419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:28.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.702+0000 7f2289400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f228410e3c0 0x7f228419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:28.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.702+0000 7f2289400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f22841141b0 con 0x7f2284074f40 2024-09-19T04:45:28.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.702+0000 7f2289400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2284114030 con 0x7f2284074620 2024-09-19T04:45:28.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.702+0000 7f2289400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2284114330 con 0x7f228410e3c0 2024-09-19T04:45:28.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.703+0000 7f2282800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2284074620 0x7f228419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:28.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.703+0000 7f2281e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2284074f40 0x7f228419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.703+0000 7f2282800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2284074620 0x7f228419a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:59792/0 (socket says 172.21.15.123:59792) 2024-09-19T04:45:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.703+0000 7f2282800640 1 -- 172.21.15.123:0/829002005 learned_addr learned my addr 172.21.15.123:0/829002005 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.703+0000 7f2283200640 1 --2- 172.21.15.123:0/829002005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f228410e3c0 0x7f228419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.703+0000 7f2282800640 1 -- 172.21.15.123:0/829002005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f228410e3c0 msgr2=0x7f228419df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.703+0000 7f2282800640 1 --2- 172.21.15.123:0/829002005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f228410e3c0 0x7f228419df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.703+0000 7f2282800640 1 -- 172.21.15.123:0/829002005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2284074f40 msgr2=0x7f228419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.703+0000 7f2282800640 1 --2- 172.21.15.123:0/829002005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2284074f40 0x7f228419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.703+0000 7f2282800640 1 -- 172.21.15.123:0/829002005 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228419e6c0 con 0x7f2284074620 2024-09-19T04:45:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.703+0000 7f2282800640 1 --2- 172.21.15.123:0/829002005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2284074620 0x7f228419a770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f227400ea60 tx=0x7f227400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.703+0000 7f227b400640 1 -- 172.21.15.123:0/829002005 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227400cde0 con 0x7f2284074620 2024-09-19T04:45:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.704+0000 7f227b400640 1 -- 172.21.15.123:0/829002005 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22740035f0 con 0x7f2284074620 2024-09-19T04:45:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.704+0000 7f2289400640 1 -- 172.21.15.123:0/829002005 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228419e9b0 con 0x7f2284074620 2024-09-19T04:45:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.704+0000 7f227b400640 1 -- 172.21.15.123:0/829002005 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274010670 con 0x7f2284074620 2024-09-19T04:45:28.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.704+0000 7f2289400640 1 -- 172.21.15.123:0/829002005 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22841a7320 con 0x7f2284074620 2024-09-19T04:45:28.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.705+0000 7f2289400640 1 -- 172.21.15.123:0/829002005 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2284074a00 con 0x7f2284074620 2024-09-19T04:45:28.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.707+0000 7f227b400640 1 -- 172.21.15.123:0/829002005 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f22740037a0 con 0x7f2284074620 2024-09-19T04:45:28.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.709+0000 7f227b400640 1 --2- 172.21.15.123:0/829002005 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f22680a32d0 0x7f22680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:28.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.709+0000 7f227b400640 1 -- 172.21.15.123:0/829002005 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f22740cfd40 con 0x7f2284074620 2024-09-19T04:45:28.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.709+0000 7f2281e00640 1 --2- 172.21.15.123:0/829002005 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f22680a32d0 0x7f22680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:28.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.710+0000 7f2281e00640 1 --2- 172.21.15.123:0/829002005 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f22680a32d0 0x7f22680a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2270000c00 tx=0x7f2270005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:28.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.712+0000 7f227b400640 1 -- 172.21.15.123:0/829002005 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f227409c910 con 0x7f2284074620 2024-09-19T04:45:28.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:28 smithi193 ceph-mon[35592]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:45:28.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:28 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1269098184' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T04:45:28.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:28 smithi183 ceph-mon[35333]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:45:28.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:28 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1269098184' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T04:45:28.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.892+0000 7f2289400640 1 -- 172.21.15.123:0/829002005 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f228419bad0 con 0x7f2284074620 2024-09-19T04:45:28.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.892+0000 7f227b400640 1 -- 172.21.15.123:0/829002005 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f227409c210 con 0x7f2284074620 2024-09-19T04:45:28.894 INFO:teuthology.orchestra.run.smithi123.stdout:193273528373 2024-09-19T04:45:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.894+0000 7f2289400640 1 -- 172.21.15.123:0/829002005 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f22680a32d0 msgr2=0x7f22680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.894+0000 7f2289400640 1 --2- 172.21.15.123:0/829002005 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f22680a32d0 0x7f22680a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2270000c00 tx=0x7f2270005b50 comp rx=0 tx=0).stop 2024-09-19T04:45:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.894+0000 7f2289400640 1 -- 172.21.15.123:0/829002005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2284074620 msgr2=0x7f228419a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.894+0000 7f2289400640 1 --2- 172.21.15.123:0/829002005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2284074620 0x7f228419a770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f227400ea60 tx=0x7f227400ef20 comp rx=0 tx=0).stop 2024-09-19T04:45:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.894+0000 7f2289400640 1 -- 172.21.15.123:0/829002005 shutdown_connections 2024-09-19T04:45:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.894+0000 7f2289400640 1 --2- 172.21.15.123:0/829002005 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f22680a32d0 0x7f22680a5790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.894+0000 7f2289400640 1 --2- 172.21.15.123:0/829002005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f228410e3c0 0x7f228419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.894+0000 7f2289400640 1 --2- 172.21.15.123:0/829002005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2284074f40 0x7f228419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.894+0000 7f2289400640 1 --2- 172.21.15.123:0/829002005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2284074620 0x7f228419a770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:28.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.894+0000 7f2289400640 1 -- 172.21.15.123:0/829002005 >> 172.21.15.123:0/829002005 conn(0x7f22840fd8c0 msgr2=0x7f22840fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:28.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.895+0000 7f2289400640 1 -- 172.21.15.123:0/829002005 shutdown_connections 2024-09-19T04:45:28.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:28.895+0000 7f2289400640 1 -- 172.21.15.123:0/829002005 wait complete. 2024-09-19T04:45:29.854 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:29 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:29.854 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:29 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:29.855 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:29 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:29.855 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:29 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:29.855 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:29 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/829002005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T04:45:29.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:29 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:29.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:29 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:29.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:29 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:29.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:29 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:29.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:29 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/829002005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T04:45:30.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:29 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:30.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:29 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:30.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:29 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:30.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:29 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:30.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:29 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/829002005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T04:45:30.215 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528371 got 193273528373 for osd.6 2024-09-19T04:45:30.215 DEBUG:teuthology.parallel:result is None 2024-09-19T04:45:30.746 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:30 smithi123 ceph-mon[30511]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:45:30.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.754+0000 7f9440e00640 1 -- 172.21.15.123:0/2237720235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f943c102370 msgr2=0x7f943c10e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:30.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.754+0000 7f9440e00640 1 --2- 172.21.15.123:0/2237720235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f943c102370 0x7f943c10e8a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9434009cd0 tx=0x7f943402a8e0 comp rx=0 tx=0).stop 2024-09-19T04:45:30.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.754+0000 7f9440e00640 1 -- 172.21.15.123:0/2237720235 shutdown_connections 2024-09-19T04:45:30.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.754+0000 7f9440e00640 1 --2- 172.21.15.123:0/2237720235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f943c102370 0x7f943c10e8a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:30.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.754+0000 7f9440e00640 1 --2- 172.21.15.123:0/2237720235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f943c1019d0 0x7f943c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:30.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.754+0000 7f9440e00640 1 --2- 172.21.15.123:0/2237720235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f943c1079d0 0x7f943c107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:30.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.754+0000 7f9440e00640 1 -- 172.21.15.123:0/2237720235 >> 172.21.15.123:0/2237720235 conn(0x7f943c0fd710 msgr2=0x7f943c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:30.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.754+0000 7f9440e00640 1 -- 172.21.15.123:0/2237720235 shutdown_connections 2024-09-19T04:45:30.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.754+0000 7f9440e00640 1 -- 172.21.15.123:0/2237720235 wait complete. 2024-09-19T04:45:30.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.755+0000 7f9440e00640 1 Processor -- start 2024-09-19T04:45:30.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.755+0000 7f9440e00640 1 -- start start 2024-09-19T04:45:30.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.756+0000 7f9440e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f943c1019d0 0x7f943c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:30.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.756+0000 7f9440e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f943c102370 0x7f943c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:30.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.756+0000 7f9440e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f943c1079d0 0x7f943c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:30.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.756+0000 7f9440e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f943c111dd0 con 0x7f943c1079d0 2024-09-19T04:45:30.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.756+0000 7f9440e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f943c111c50 con 0x7f943c102370 2024-09-19T04:45:30.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.756+0000 7f9440e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f943c111f50 con 0x7f943c1019d0 2024-09-19T04:45:30.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.756+0000 7f943b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f943c1019d0 0x7f943c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:30.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.756+0000 7f943b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f943c1019d0 0x7f943c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42892/0 (socket says 172.21.15.123:42892) 2024-09-19T04:45:30.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.756+0000 7f943b400640 1 -- 172.21.15.123:0/4043077143 learned_addr learned my addr 172.21.15.123:0/4043077143 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:30.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.756+0000 7f943be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f943c1079d0 0x7f943c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:30.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.756+0000 7f943aa00640 1 --2- 172.21.15.123:0/4043077143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f943c102370 0x7f943c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:30.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.756+0000 7f943be00640 1 -- 172.21.15.123:0/4043077143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f943c1019d0 msgr2=0x7f943c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:30.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.756+0000 7f943be00640 1 --2- 172.21.15.123:0/4043077143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f943c1019d0 0x7f943c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:30.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.756+0000 7f943be00640 1 -- 172.21.15.123:0/4043077143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f943c102370 msgr2=0x7f943c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:30.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.756+0000 7f943be00640 1 --2- 172.21.15.123:0/4043077143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f943c102370 0x7f943c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:30.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.756+0000 7f943be00640 1 -- 172.21.15.123:0/4043077143 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943c19f410 con 0x7f943c1079d0 2024-09-19T04:45:30.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.757+0000 7f943be00640 1 --2- 172.21.15.123:0/4043077143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f943c1079d0 0x7f943c19ed30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f943402ae00 tx=0x7f9434031df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:30.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.757+0000 7f9433e00640 1 -- 172.21.15.123:0/4043077143 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434003d80 con 0x7f943c1079d0 2024-09-19T04:45:30.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.757+0000 7f9433e00640 1 -- 172.21.15.123:0/4043077143 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9434004520 con 0x7f943c1079d0 2024-09-19T04:45:30.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.757+0000 7f9433e00640 1 -- 172.21.15.123:0/4043077143 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434030d00 con 0x7f943c1079d0 2024-09-19T04:45:30.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.757+0000 7f9440e00640 1 -- 172.21.15.123:0/4043077143 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f943c19f6a0 con 0x7f943c1079d0 2024-09-19T04:45:30.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.757+0000 7f9440e00640 1 -- 172.21.15.123:0/4043077143 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f943c1a6fe0 con 0x7f943c1079d0 2024-09-19T04:45:30.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.758+0000 7f9440e00640 1 -- 172.21.15.123:0/4043077143 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f943c101e30 con 0x7f943c1079d0 2024-09-19T04:45:30.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.760+0000 7f9433e00640 1 -- 172.21.15.123:0/4043077143 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f94340046d0 con 0x7f943c1079d0 2024-09-19T04:45:30.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.761+0000 7f9433e00640 1 --2- 172.21.15.123:0/4043077143 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f94200a3350 0x7f94200a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:30.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.762+0000 7f9433e00640 1 -- 172.21.15.123:0/4043077143 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f943402e030 con 0x7f943c1079d0 2024-09-19T04:45:30.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.762+0000 7f943b400640 1 --2- 172.21.15.123:0/4043077143 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f94200a3350 0x7f94200a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:30.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.762+0000 7f943b400640 1 --2- 172.21.15.123:0/4043077143 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f94200a3350 0x7f94200a5810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f942c009c50 tx=0x7f942c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:30.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.766+0000 7f9433e00640 1 -- 172.21.15.123:0/4043077143 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f943404b060 con 0x7f943c1079d0 2024-09-19T04:45:30.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.962+0000 7f9440e00640 1 -- 172.21.15.123:0/4043077143 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f943c1a71b0 con 0x7f943c1079d0 2024-09-19T04:45:30.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.962+0000 7f9433e00640 1 -- 172.21.15.123:0/4043077143 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f94340bd9a0 con 0x7f943c1079d0 2024-09-19T04:45:30.964 INFO:teuthology.orchestra.run.smithi123.stdout:137438953534 2024-09-19T04:45:30.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.964+0000 7f9440e00640 1 -- 172.21.15.123:0/4043077143 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f94200a3350 msgr2=0x7f94200a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:30.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.964+0000 7f9440e00640 1 --2- 172.21.15.123:0/4043077143 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f94200a3350 0x7f94200a5810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f942c009c50 tx=0x7f942c005f90 comp rx=0 tx=0).stop 2024-09-19T04:45:30.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.964+0000 7f9440e00640 1 -- 172.21.15.123:0/4043077143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f943c1079d0 msgr2=0x7f943c19ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:30.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.964+0000 7f9440e00640 1 --2- 172.21.15.123:0/4043077143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f943c1079d0 0x7f943c19ed30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f943402ae00 tx=0x7f9434031df0 comp rx=0 tx=0).stop 2024-09-19T04:45:30.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.964+0000 7f9440e00640 1 -- 172.21.15.123:0/4043077143 shutdown_connections 2024-09-19T04:45:30.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.964+0000 7f9440e00640 1 --2- 172.21.15.123:0/4043077143 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f94200a3350 0x7f94200a5810 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:30.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.964+0000 7f9440e00640 1 --2- 172.21.15.123:0/4043077143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f943c1079d0 0x7f943c19ed30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:30.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.964+0000 7f9440e00640 1 --2- 172.21.15.123:0/4043077143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f943c102370 0x7f943c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:30.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.964+0000 7f9440e00640 1 --2- 172.21.15.123:0/4043077143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f943c1019d0 0x7f943c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:30.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.964+0000 7f9440e00640 1 -- 172.21.15.123:0/4043077143 >> 172.21.15.123:0/4043077143 conn(0x7f943c0fd710 msgr2=0x7f943c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:30.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.964+0000 7f9440e00640 1 -- 172.21.15.123:0/4043077143 shutdown_connections 2024-09-19T04:45:30.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.965+0000 7f9440e00640 1 -- 172.21.15.123:0/4043077143 wait complete. 2024-09-19T04:45:30.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:30 smithi193 ceph-mon[35592]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:45:31.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:30 smithi183 ceph-mon[35333]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:45:31.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.219+0000 7f8408a00640 1 -- 172.21.15.123:0/1161257107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8404109120 msgr2=0x7f8404110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:31.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.219+0000 7f8408a00640 1 --2- 172.21.15.123:0/1161257107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8404109120 0x7f8404110ca0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009ca0 tx=0x7f83fc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:45:31.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.219+0000 7f8408a00640 1 -- 172.21.15.123:0/1161257107 shutdown_connections 2024-09-19T04:45:31.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.220+0000 7f8408a00640 1 --2- 172.21.15.123:0/1161257107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8404109120 0x7f8404110ca0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:31.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.220+0000 7f8408a00640 1 --2- 172.21.15.123:0/1161257107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84041049a0 0x7f84041089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:31.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.220+0000 7f8408a00640 1 --2- 172.21.15.123:0/1161257107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8404103ff0 0x7f84041043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:31.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.220+0000 7f8408a00640 1 -- 172.21.15.123:0/1161257107 >> 172.21.15.123:0/1161257107 conn(0x7f84040ffa90 msgr2=0x7f8404101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:31.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.220+0000 7f8408a00640 1 -- 172.21.15.123:0/1161257107 shutdown_connections 2024-09-19T04:45:31.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.220+0000 7f8408a00640 1 -- 172.21.15.123:0/1161257107 wait complete. 2024-09-19T04:45:31.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.220+0000 7f8408a00640 1 Processor -- start 2024-09-19T04:45:31.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.221+0000 7f8408a00640 1 -- start start 2024-09-19T04:45:31.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.221+0000 7f8408a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8404103ff0 0x7f840419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:31.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.221+0000 7f8408a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84041049a0 0x7f840419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:31.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.221+0000 7f8408a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8404109120 0x7f840419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:31.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.221+0000 7f8408a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8404114180 con 0x7f8404103ff0 2024-09-19T04:45:31.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.221+0000 7f8408a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8404114000 con 0x7f84041049a0 2024-09-19T04:45:31.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.221+0000 7f8408a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8404114300 con 0x7f8404109120 2024-09-19T04:45:31.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.221+0000 7f8401e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8404103ff0 0x7f840419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:31.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.221+0000 7f8401e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8404103ff0 0x7f840419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56702/0 (socket says 172.21.15.123:56702) 2024-09-19T04:45:31.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.221+0000 7f8401e00640 1 -- 172.21.15.123:0/1948911571 learned_addr learned my addr 172.21.15.123:0/1948911571 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:31.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.221+0000 7f83f9400640 1 --2- 172.21.15.123:0/1948911571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84041049a0 0x7f840419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:31.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.222+0000 7f8402800640 1 --2- 172.21.15.123:0/1948911571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8404109120 0x7f840419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:31.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.222+0000 7f8401e00640 1 -- 172.21.15.123:0/1948911571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8404109120 msgr2=0x7f840419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:31.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.222+0000 7f8401e00640 1 --2- 172.21.15.123:0/1948911571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8404109120 0x7f840419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:31.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.222+0000 7f8401e00640 1 -- 172.21.15.123:0/1948911571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84041049a0 msgr2=0x7f840419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:31.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.222+0000 7f8401e00640 1 --2- 172.21.15.123:0/1948911571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84041049a0 0x7f840419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:31.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.222+0000 7f8401e00640 1 -- 172.21.15.123:0/1948911571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840419f570 con 0x7f8404103ff0 2024-09-19T04:45:31.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.222+0000 7f83f9400640 1 --2- 172.21.15.123:0/1948911571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84041049a0 0x7f840419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:31.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.222+0000 7f8401e00640 1 --2- 172.21.15.123:0/1948911571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8404103ff0 0x7f840419a5c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f83f400e9f0 tx=0x7f83f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:31.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.222+0000 7f83fb400640 1 -- 172.21.15.123:0/1948911571 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f400cd80 con 0x7f8404103ff0 2024-09-19T04:45:31.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.222+0000 7f83fb400640 1 -- 172.21.15.123:0/1948911571 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83f4010910 con 0x7f8404103ff0 2024-09-19T04:45:31.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.222+0000 7f83fb400640 1 -- 172.21.15.123:0/1948911571 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4018990 con 0x7f8404103ff0 2024-09-19T04:45:31.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.222+0000 7f8408a00640 1 -- 172.21.15.123:0/1948911571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840419f860 con 0x7f8404103ff0 2024-09-19T04:45:31.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.223+0000 7f8408a00640 1 -- 172.21.15.123:0/1948911571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84041a7140 con 0x7f8404103ff0 2024-09-19T04:45:31.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.224+0000 7f8408a00640 1 -- 172.21.15.123:0/1948911571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84041043d0 con 0x7f8404103ff0 2024-09-19T04:45:31.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.225+0000 7f83fb400640 1 -- 172.21.15.123:0/1948911571 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f83f4010ac0 con 0x7f8404103ff0 2024-09-19T04:45:31.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.227+0000 7f83fb400640 1 --2- 172.21.15.123:0/1948911571 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f83e80a3350 0x7f83e80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:31.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.227+0000 7f83fb400640 1 -- 172.21.15.123:0/1948911571 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f83f40cfe10 con 0x7f8404103ff0 2024-09-19T04:45:31.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.227+0000 7f83f9400640 1 --2- 172.21.15.123:0/1948911571 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f83e80a3350 0x7f83e80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:31.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.227+0000 7f83f9400640 1 --2- 172.21.15.123:0/1948911571 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f83e80a3350 0x7f83e80a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f840410ab00 tx=0x7f83ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:31.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.230+0000 7f83fb400640 1 -- 172.21.15.123:0/1948911571 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f83f40d3030 con 0x7f8404103ff0 2024-09-19T04:45:31.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.411+0000 7f8408a00640 1 -- 172.21.15.123:0/1948911571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f84041a74a0 con 0x7f8404103ff0 2024-09-19T04:45:31.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.412+0000 7f83fb400640 1 -- 172.21.15.123:0/1948911571 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f83f409c980 con 0x7f8404103ff0 2024-09-19T04:45:31.413 INFO:teuthology.orchestra.run.smithi123.stdout:107374182468 2024-09-19T04:45:31.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.414+0000 7f8408a00640 1 -- 172.21.15.123:0/1948911571 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f83e80a3350 msgr2=0x7f83e80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:31.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.414+0000 7f8408a00640 1 --2- 172.21.15.123:0/1948911571 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f83e80a3350 0x7f83e80a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f840410ab00 tx=0x7f83ec005f90 comp rx=0 tx=0).stop 2024-09-19T04:45:31.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.414+0000 7f8408a00640 1 -- 172.21.15.123:0/1948911571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8404103ff0 msgr2=0x7f840419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:31.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.414+0000 7f8408a00640 1 --2- 172.21.15.123:0/1948911571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8404103ff0 0x7f840419a5c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f83f400e9f0 tx=0x7f83f400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:45:31.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.414+0000 7f8408a00640 1 -- 172.21.15.123:0/1948911571 shutdown_connections 2024-09-19T04:45:31.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.414+0000 7f8408a00640 1 --2- 172.21.15.123:0/1948911571 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f83e80a3350 0x7f83e80a5810 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:31.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.414+0000 7f8408a00640 1 --2- 172.21.15.123:0/1948911571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8404109120 0x7f840419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:31.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.414+0000 7f8408a00640 1 --2- 172.21.15.123:0/1948911571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84041049a0 0x7f840419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:31.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.414+0000 7f8408a00640 1 --2- 172.21.15.123:0/1948911571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8404103ff0 0x7f840419a5c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:31.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.414+0000 7f8408a00640 1 -- 172.21.15.123:0/1948911571 >> 172.21.15.123:0/1948911571 conn(0x7f84040ffa90 msgr2=0x7f84041005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:31.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.415+0000 7f8408a00640 1 -- 172.21.15.123:0/1948911571 shutdown_connections 2024-09-19T04:45:31.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.415+0000 7f8408a00640 1 -- 172.21.15.123:0/1948911571 wait complete. 2024-09-19T04:45:31.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:31 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1962396788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T04:45:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:31 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/4043077143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T04:45:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:31 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1948911571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T04:45:31.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.890+0000 7f3d5ee00640 1 -- 172.21.15.123:0/2520484985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d58101df0 msgr2=0x7f3d581021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:31.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.890+0000 7f3d5ee00640 1 --2- 172.21.15.123:0/2520484985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d58101df0 0x7f3d581021d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3d48009ca0 tx=0x7f3d4802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:45:31.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.890+0000 7f3d5ee00640 1 -- 172.21.15.123:0/2520484985 shutdown_connections 2024-09-19T04:45:31.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.890+0000 7f3d5ee00640 1 --2- 172.21.15.123:0/2520484985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d58106f20 0x7f3d5810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:31.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.890+0000 7f3d5ee00640 1 --2- 172.21.15.123:0/2520484985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d581027a0 0x7f3d581067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:31.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.890+0000 7f3d5ee00640 1 --2- 172.21.15.123:0/2520484985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d58101df0 0x7f3d581021d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:31.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.890+0000 7f3d5ee00640 1 -- 172.21.15.123:0/2520484985 >> 172.21.15.123:0/2520484985 conn(0x7f3d580fd8b0 msgr2=0x7f3d580ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:31.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.891+0000 7f3d5ee00640 1 -- 172.21.15.123:0/2520484985 shutdown_connections 2024-09-19T04:45:31.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.891+0000 7f3d5ee00640 1 -- 172.21.15.123:0/2520484985 wait complete. 2024-09-19T04:45:31.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.891+0000 7f3d5ee00640 1 Processor -- start 2024-09-19T04:45:31.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.892+0000 7f3d5ee00640 1 -- start start 2024-09-19T04:45:31.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.892+0000 7f3d5ee00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d58101df0 0x7f3d5819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:31.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.892+0000 7f3d5ee00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d581027a0 0x7f3d5819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:31.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.892+0000 7f3d5ee00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d58106f20 0x7f3d5819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:31.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.892+0000 7f3d5ee00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3d58111f80 con 0x7f3d581027a0 2024-09-19T04:45:31.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.892+0000 7f3d5ee00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3d58111e00 con 0x7f3d58106f20 2024-09-19T04:45:31.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.892+0000 7f3d5ee00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3d58112100 con 0x7f3d58101df0 2024-09-19T04:45:31.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.892+0000 7f3d57400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d581027a0 0x7f3d5819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:31.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.892+0000 7f3d57400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d581027a0 0x7f3d5819abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56726/0 (socket says 172.21.15.123:56726) 2024-09-19T04:45:31.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.892+0000 7f3d57400640 1 -- 172.21.15.123:0/3961223990 learned_addr learned my addr 172.21.15.123:0/3961223990 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:31.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.892+0000 7f3d5ce00640 1 --2- 172.21.15.123:0/3961223990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d58106f20 0x7f3d5819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:31.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.892+0000 7f3d57e00640 1 --2- 172.21.15.123:0/3961223990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d58101df0 0x7f3d5819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:31.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.893+0000 7f3d57400640 1 -- 172.21.15.123:0/3961223990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d58101df0 msgr2=0x7f3d5819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:31.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.893+0000 7f3d57400640 1 --2- 172.21.15.123:0/3961223990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d58101df0 0x7f3d5819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:31.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.893+0000 7f3d57400640 1 -- 172.21.15.123:0/3961223990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d58106f20 msgr2=0x7f3d5819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:31.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.893+0000 7f3d57400640 1 --2- 172.21.15.123:0/3961223990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d58106f20 0x7f3d5819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:31.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.893+0000 7f3d57400640 1 -- 172.21.15.123:0/3961223990 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5819f630 con 0x7f3d581027a0 2024-09-19T04:45:31.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.893+0000 7f3d5ce00640 1 --2- 172.21.15.123:0/3961223990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d58106f20 0x7f3d5819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:31.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.893+0000 7f3d57400640 1 --2- 172.21.15.123:0/3961223990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d581027a0 0x7f3d5819abc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400e9f0 tx=0x7f3d4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:31.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.893+0000 7f3d54c00640 1 -- 172.21.15.123:0/3961223990 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4400cd80 con 0x7f3d581027a0 2024-09-19T04:45:31.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.893+0000 7f3d54c00640 1 -- 172.21.15.123:0/3961223990 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d44005500 con 0x7f3d581027a0 2024-09-19T04:45:31.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.893+0000 7f3d5ee00640 1 -- 172.21.15.123:0/3961223990 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d5819f920 con 0x7f3d581027a0 2024-09-19T04:45:31.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.894+0000 7f3d5ee00640 1 -- 172.21.15.123:0/3961223990 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d581a7200 con 0x7f3d581027a0 2024-09-19T04:45:31.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.894+0000 7f3d54c00640 1 -- 172.21.15.123:0/3961223990 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d44010690 con 0x7f3d581027a0 2024-09-19T04:45:31.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.895+0000 7f3d5ee00640 1 -- 172.21.15.123:0/3961223990 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d38005290 con 0x7f3d581027a0 2024-09-19T04:45:31.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.898+0000 7f3d54c00640 1 -- 172.21.15.123:0/3961223990 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3d440046c0 con 0x7f3d581027a0 2024-09-19T04:45:31.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.899+0000 7f3d54c00640 1 --2- 172.21.15.123:0/3961223990 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3d3c0a3350 0x7f3d3c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:31.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.899+0000 7f3d54c00640 1 -- 172.21.15.123:0/3961223990 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f3d440cff20 con 0x7f3d581027a0 2024-09-19T04:45:31.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.899+0000 7f3d57e00640 1 --2- 172.21.15.123:0/3961223990 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3d3c0a3350 0x7f3d3c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:31.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.900+0000 7f3d57e00640 1 --2- 172.21.15.123:0/3961223990 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3d3c0a3350 0x7f3d3c0a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3d48006010 tx=0x7f3d48009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:31.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:31.901+0000 7f3d54c00640 1 -- 172.21.15.123:0/3961223990 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3d44054210 con 0x7f3d581027a0 2024-09-19T04:45:31.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:31 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1962396788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T04:45:31.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:31 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/4043077143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T04:45:31.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:31 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1948911571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T04:45:32.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.084+0000 7f3d5ee00640 1 -- 172.21.15.123:0/3961223990 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3d38005850 con 0x7f3d581027a0 2024-09-19T04:45:32.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.085+0000 7f3d54c00640 1 -- 172.21.15.123:0/3961223990 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3d4409ca90 con 0x7f3d581027a0 2024-09-19T04:45:32.086 INFO:teuthology.orchestra.run.smithi123.stdout:55834574926 2024-09-19T04:45:32.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.087+0000 7f3d5ee00640 1 -- 172.21.15.123:0/3961223990 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3d3c0a3350 msgr2=0x7f3d3c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:32.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.087+0000 7f3d5ee00640 1 --2- 172.21.15.123:0/3961223990 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3d3c0a3350 0x7f3d3c0a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3d48006010 tx=0x7f3d48009360 comp rx=0 tx=0).stop 2024-09-19T04:45:32.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.087+0000 7f3d5ee00640 1 -- 172.21.15.123:0/3961223990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d581027a0 msgr2=0x7f3d5819abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:32.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.087+0000 7f3d5ee00640 1 --2- 172.21.15.123:0/3961223990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d581027a0 0x7f3d5819abc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400e9f0 tx=0x7f3d4400edf0 comp rx=0 tx=0).stop 2024-09-19T04:45:32.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.087+0000 7f3d5ee00640 1 -- 172.21.15.123:0/3961223990 shutdown_connections 2024-09-19T04:45:32.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.087+0000 7f3d5ee00640 1 --2- 172.21.15.123:0/3961223990 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f3d3c0a3350 0x7f3d3c0a5810 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:32.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.087+0000 7f3d5ee00640 1 --2- 172.21.15.123:0/3961223990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d58106f20 0x7f3d5819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:32.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.087+0000 7f3d5ee00640 1 --2- 172.21.15.123:0/3961223990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d581027a0 0x7f3d5819abc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:32.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.087+0000 7f3d5ee00640 1 --2- 172.21.15.123:0/3961223990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d58101df0 0x7f3d5819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:32.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.087+0000 7f3d5ee00640 1 -- 172.21.15.123:0/3961223990 >> 172.21.15.123:0/3961223990 conn(0x7f3d580fd8b0 msgr2=0x7f3d580fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:32.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.087+0000 7f3d5ee00640 1 -- 172.21.15.123:0/3961223990 shutdown_connections 2024-09-19T04:45:32.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.087+0000 7f3d5ee00640 1 -- 172.21.15.123:0/3961223990 wait complete. 2024-09-19T04:45:32.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:31 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1962396788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T04:45:32.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:31 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/4043077143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T04:45:32.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:31 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1948911571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T04:45:32.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.647+0000 7f4588400640 1 -- 172.21.15.123:0/1217574296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4580109140 msgr2=0x7f4580110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:32.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.647+0000 7f4588400640 1 --2- 172.21.15.123:0/1217574296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4580109140 0x7f4580110cc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4578009ca0 tx=0x7f457802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:45:32.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.647+0000 7f4588400640 1 -- 172.21.15.123:0/1217574296 shutdown_connections 2024-09-19T04:45:32.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.647+0000 7f4588400640 1 --2- 172.21.15.123:0/1217574296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4580109140 0x7f4580110cc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:32.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.647+0000 7f4588400640 1 --2- 172.21.15.123:0/1217574296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45801049c0 0x7f4580108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:32.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.647+0000 7f4588400640 1 --2- 172.21.15.123:0/1217574296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4580104010 0x7f45801043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:32.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.647+0000 7f4588400640 1 -- 172.21.15.123:0/1217574296 >> 172.21.15.123:0/1217574296 conn(0x7f45800ffa50 msgr2=0x7f4580101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:32.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.647+0000 7f4588400640 1 -- 172.21.15.123:0/1217574296 shutdown_connections 2024-09-19T04:45:32.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.647+0000 7f4588400640 1 -- 172.21.15.123:0/1217574296 wait complete. 2024-09-19T04:45:32.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.649+0000 7f4588400640 1 Processor -- start 2024-09-19T04:45:32.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.649+0000 7f4588400640 1 -- start start 2024-09-19T04:45:32.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.649+0000 7f4588400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4580104010 0x7f458019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:32.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.649+0000 7f4588400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45801049c0 0x7f458019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:32.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.649+0000 7f4588400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4580109140 0x7f458019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:32.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.649+0000 7f4588400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4580114170 con 0x7f4580109140 2024-09-19T04:45:32.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.649+0000 7f4588400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4580113ff0 con 0x7f4580104010 2024-09-19T04:45:32.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.649+0000 7f4588400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45801142f0 con 0x7f45801049c0 2024-09-19T04:45:32.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.650+0000 7f4585a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4580104010 0x7f458019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:32.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.650+0000 7f4586400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4580109140 0x7f458019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:32.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.650+0000 7f4585a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4580104010 0x7f458019a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:59802/0 (socket says 172.21.15.123:59802) 2024-09-19T04:45:32.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.650+0000 7f4585a00640 1 -- 172.21.15.123:0/1962396788 learned_addr learned my addr 172.21.15.123:0/1962396788 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:32.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.650+0000 7f4585000640 1 --2- 172.21.15.123:0/1962396788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45801049c0 0x7f458019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:32.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.650+0000 7f4585a00640 1 -- 172.21.15.123:0/1962396788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45801049c0 msgr2=0x7f458019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:32.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.650+0000 7f4585a00640 1 --2- 172.21.15.123:0/1962396788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45801049c0 0x7f458019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:32.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.650+0000 7f4585a00640 1 -- 172.21.15.123:0/1962396788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4580109140 msgr2=0x7f458019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:32.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.650+0000 7f4585a00640 1 --2- 172.21.15.123:0/1962396788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4580109140 0x7f458019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:32.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.650+0000 7f4585a00640 1 -- 172.21.15.123:0/1962396788 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458019f680 con 0x7f4580104010 2024-09-19T04:45:32.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.650+0000 7f4585a00640 1 --2- 172.21.15.123:0/1962396788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4580104010 0x7f458019a6d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f457400e9f0 tx=0x7f457400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:32.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.651+0000 7f457e000640 1 -- 172.21.15.123:0/1962396788 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457400cd80 con 0x7f4580104010 2024-09-19T04:45:32.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.651+0000 7f457e000640 1 -- 172.21.15.123:0/1962396788 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4574005500 con 0x7f4580104010 2024-09-19T04:45:32.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.651+0000 7f4588400640 1 -- 172.21.15.123:0/1962396788 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458019f970 con 0x7f4580104010 2024-09-19T04:45:32.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.651+0000 7f457e000640 1 -- 172.21.15.123:0/1962396788 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4574010690 con 0x7f4580104010 2024-09-19T04:45:32.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.651+0000 7f4588400640 1 -- 172.21.15.123:0/1962396788 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45801a7250 con 0x7f4580104010 2024-09-19T04:45:32.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.652+0000 7f4588400640 1 -- 172.21.15.123:0/1962396788 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45801043f0 con 0x7f4580104010 2024-09-19T04:45:32.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.655+0000 7f457e000640 1 -- 172.21.15.123:0/1962396788 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f45740046c0 con 0x7f4580104010 2024-09-19T04:45:32.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.656+0000 7f457e000640 1 --2- 172.21.15.123:0/1962396788 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f45680a32d0 0x7f45680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:32.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.656+0000 7f457e000640 1 -- 172.21.15.123:0/1962396788 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f45740d0190 con 0x7f4580104010 2024-09-19T04:45:32.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.656+0000 7f4585000640 1 --2- 172.21.15.123:0/1962396788 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f45680a32d0 0x7f45680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:32.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.657+0000 7f4585000640 1 --2- 172.21.15.123:0/1962396788 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f45680a32d0 0x7f45680a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f45800ffe70 tx=0x7f456c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:32.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.657+0000 7f457e000640 1 -- 172.21.15.123:0/1962396788 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f457409cd00 con 0x7f4580104010 2024-09-19T04:45:32.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.844+0000 7f4588400640 1 -- 172.21.15.123:0/1962396788 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f458010c3b0 con 0x7f4580104010 2024-09-19T04:45:32.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.845+0000 7f457e000640 1 -- 172.21.15.123:0/1962396788 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f457409c600 con 0x7f4580104010 2024-09-19T04:45:32.397 INFO:teuthology.orchestra.run.smithi123.stdout:244813135914 2024-09-19T04:45:32.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.847+0000 7f4588400640 1 -- 172.21.15.123:0/1962396788 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f45680a32d0 msgr2=0x7f45680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:32.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.847+0000 7f4588400640 1 --2- 172.21.15.123:0/1962396788 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f45680a32d0 0x7f45680a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f45800ffe70 tx=0x7f456c005f90 comp rx=0 tx=0).stop 2024-09-19T04:45:32.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.847+0000 7f4588400640 1 -- 172.21.15.123:0/1962396788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4580104010 msgr2=0x7f458019a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:32.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.847+0000 7f4588400640 1 --2- 172.21.15.123:0/1962396788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4580104010 0x7f458019a6d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f457400e9f0 tx=0x7f457400edf0 comp rx=0 tx=0).stop 2024-09-19T04:45:32.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.847+0000 7f4588400640 1 -- 172.21.15.123:0/1962396788 shutdown_connections 2024-09-19T04:45:32.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.847+0000 7f4588400640 1 --2- 172.21.15.123:0/1962396788 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f45680a32d0 0x7f45680a5790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:32.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.847+0000 7f4588400640 1 --2- 172.21.15.123:0/1962396788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4580109140 0x7f458019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:32.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.847+0000 7f4588400640 1 --2- 172.21.15.123:0/1962396788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45801049c0 0x7f458019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:32.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.847+0000 7f4588400640 1 --2- 172.21.15.123:0/1962396788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4580104010 0x7f458019a6d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:32.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.847+0000 7f4588400640 1 -- 172.21.15.123:0/1962396788 >> 172.21.15.123:0/1962396788 conn(0x7f45800ffa50 msgr2=0x7f4580100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:32.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.847+0000 7f4588400640 1 -- 172.21.15.123:0/1962396788 shutdown_connections 2024-09-19T04:45:32.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:30.847+0000 7f4588400640 1 -- 172.21.15.123:0/1962396788 wait complete. 2024-09-19T04:45:32.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:32 smithi123 ceph-mon[30511]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:45:32.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:32 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3961223990' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T04:45:32.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:32 smithi193 ceph-mon[35592]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:45:32.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:32 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3961223990' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T04:45:33.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:32 smithi183 ceph-mon[35333]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:45:33.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:32 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3961223990' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T04:45:33.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.361+0000 7f4bc5000640 1 -- 172.21.15.123:0/253791213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bc01047e0 msgr2=0x7f4bc0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:33.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.361+0000 7f4bc5000640 1 --2- 172.21.15.123:0/253791213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bc01047e0 0x7f4bc0108830 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009900 tx=0x7f4bac02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:45:33.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.362+0000 7f4bc5000640 1 -- 172.21.15.123:0/253791213 shutdown_connections 2024-09-19T04:45:33.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.362+0000 7f4bc5000640 1 --2- 172.21.15.123:0/253791213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bc0108f60 0x7f4bc0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:33.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.362+0000 7f4bc5000640 1 --2- 172.21.15.123:0/253791213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bc01047e0 0x7f4bc0108830 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:33.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.362+0000 7f4bc5000640 1 --2- 172.21.15.123:0/253791213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bc0103e30 0x7f4bc0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:33.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.362+0000 7f4bc5000640 1 -- 172.21.15.123:0/253791213 >> 172.21.15.123:0/253791213 conn(0x7f4bc00ff8d0 msgr2=0x7f4bc0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:33.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.362+0000 7f4bc5000640 1 -- 172.21.15.123:0/253791213 shutdown_connections 2024-09-19T04:45:33.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.362+0000 7f4bc5000640 1 -- 172.21.15.123:0/253791213 wait complete. 2024-09-19T04:45:33.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.363+0000 7f4bc5000640 1 Processor -- start 2024-09-19T04:45:33.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.363+0000 7f4bc5000640 1 -- start start 2024-09-19T04:45:33.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.363+0000 7f4bc5000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bc0103e30 0x7f4bc019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:33.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.363+0000 7f4bc5000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bc01047e0 0x7f4bc019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:33.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.363+0000 7f4bc5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bc0108f60 0x7f4bc019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:33.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.363+0000 7f4bc5000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc0113fc0 con 0x7f4bc0108f60 2024-09-19T04:45:33.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.363+0000 7f4bc5000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc0113e40 con 0x7f4bc0103e30 2024-09-19T04:45:33.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.363+0000 7f4bc5000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc0114140 con 0x7f4bc01047e0 2024-09-19T04:45:33.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.363+0000 7f4bbfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bc0108f60 0x7f4bc019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:33.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.363+0000 7f4bbfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bc0108f60 0x7f4bc019ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60520/0 (socket says 172.21.15.123:60520) 2024-09-19T04:45:33.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.363+0000 7f4bbfe00640 1 -- 172.21.15.123:0/2929079511 learned_addr learned my addr 172.21.15.123:0/2929079511 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:33.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.363+0000 7f4bbf400640 1 --2- 172.21.15.123:0/2929079511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bc0103e30 0x7f4bc019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:33.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.364+0000 7f4bbea00640 1 --2- 172.21.15.123:0/2929079511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bc01047e0 0x7f4bc019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:33.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.364+0000 7f4bbfe00640 1 -- 172.21.15.123:0/2929079511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bc01047e0 msgr2=0x7f4bc019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:33.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.364+0000 7f4bbfe00640 1 --2- 172.21.15.123:0/2929079511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bc01047e0 0x7f4bc019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:33.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.364+0000 7f4bbfe00640 1 -- 172.21.15.123:0/2929079511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bc0103e30 msgr2=0x7f4bc019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:33.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.364+0000 7f4bbfe00640 1 --2- 172.21.15.123:0/2929079511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bc0103e30 0x7f4bc019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:33.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.364+0000 7f4bbfe00640 1 -- 172.21.15.123:0/2929079511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc019f430 con 0x7f4bc0108f60 2024-09-19T04:45:33.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.364+0000 7f4bbfe00640 1 --2- 172.21.15.123:0/2929079511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bc0108f60 0x7f4bc019ed50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800eb10 tx=0x7f4bb800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:33.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.365+0000 7f4bb7e00640 1 -- 172.21.15.123:0/2929079511 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8004b20 con 0x7f4bc0108f60 2024-09-19T04:45:33.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.365+0000 7f4bc5000640 1 -- 172.21.15.123:0/2929079511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc019f720 con 0x7f4bc0108f60 2024-09-19T04:45:33.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.365+0000 7f4bb7e00640 1 -- 172.21.15.123:0/2929079511 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb8004cc0 con 0x7f4bc0108f60 2024-09-19T04:45:33.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.365+0000 7f4bb7e00640 1 -- 172.21.15.123:0/2929079511 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8010760 con 0x7f4bc0108f60 2024-09-19T04:45:33.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.365+0000 7f4bc5000640 1 -- 172.21.15.123:0/2929079511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc01a7000 con 0x7f4bc0108f60 2024-09-19T04:45:33.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.366+0000 7f4bc5000640 1 -- 172.21.15.123:0/2929079511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc0104210 con 0x7f4bc0108f60 2024-09-19T04:45:33.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.368+0000 7f4bb7e00640 1 -- 172.21.15.123:0/2929079511 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4bb8005020 con 0x7f4bc0108f60 2024-09-19T04:45:33.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.369+0000 7f4bb7e00640 1 --2- 172.21.15.123:0/2929079511 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4ba40a32d0 0x7f4ba40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:33.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.369+0000 7f4bb7e00640 1 -- 172.21.15.123:0/2929079511 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f4bb80cffe0 con 0x7f4bc0108f60 2024-09-19T04:45:33.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.369+0000 7f4bbf400640 1 --2- 172.21.15.123:0/2929079511 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4ba40a32d0 0x7f4ba40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:33.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.370+0000 7f4bbf400640 1 --2- 172.21.15.123:0/2929079511 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4ba40a32d0 0x7f4ba40a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0005d00 tx=0x7f4bb000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:33.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.373+0000 7f4bb7e00640 1 -- 172.21.15.123:0/2929079511 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4bb809cb50 con 0x7f4bc0108f60 2024-09-19T04:45:33.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.553+0000 7f4bc5000640 1 -- 172.21.15.123:0/2929079511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f4bc010bc10 con 0x7f4bc0108f60 2024-09-19T04:45:33.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.553+0000 7f4bb7e00640 1 -- 172.21.15.123:0/2929079511 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4bb809c450 con 0x7f4bc0108f60 2024-09-19T04:45:33.555 INFO:teuthology.orchestra.run.smithi123.stdout:214748364846 2024-09-19T04:45:33.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.555+0000 7f4bc5000640 1 -- 172.21.15.123:0/2929079511 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4ba40a32d0 msgr2=0x7f4ba40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:33.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.555+0000 7f4bc5000640 1 --2- 172.21.15.123:0/2929079511 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4ba40a32d0 0x7f4ba40a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0005d00 tx=0x7f4bb000b410 comp rx=0 tx=0).stop 2024-09-19T04:45:33.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.555+0000 7f4bc5000640 1 -- 172.21.15.123:0/2929079511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bc0108f60 msgr2=0x7f4bc019ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:33.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.555+0000 7f4bc5000640 1 --2- 172.21.15.123:0/2929079511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bc0108f60 0x7f4bc019ed50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800eb10 tx=0x7f4bb800c2d0 comp rx=0 tx=0).stop 2024-09-19T04:45:33.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.556+0000 7f4bc5000640 1 -- 172.21.15.123:0/2929079511 shutdown_connections 2024-09-19T04:45:33.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.556+0000 7f4bc5000640 1 --2- 172.21.15.123:0/2929079511 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f4ba40a32d0 0x7f4ba40a5790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:33.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.556+0000 7f4bc5000640 1 --2- 172.21.15.123:0/2929079511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bc0108f60 0x7f4bc019ed50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:33.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.556+0000 7f4bc5000640 1 --2- 172.21.15.123:0/2929079511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bc01047e0 0x7f4bc019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:33.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.556+0000 7f4bc5000640 1 --2- 172.21.15.123:0/2929079511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bc0103e30 0x7f4bc019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:33.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.556+0000 7f4bc5000640 1 -- 172.21.15.123:0/2929079511 >> 172.21.15.123:0/2929079511 conn(0x7f4bc00ff8d0 msgr2=0x7f4bc0100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:33.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.556+0000 7f4bc5000640 1 -- 172.21.15.123:0/2929079511 shutdown_connections 2024-09-19T04:45:33.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.556+0000 7f4bc5000640 1 -- 172.21.15.123:0/2929079511 wait complete. 2024-09-19T04:45:33.781 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574924 got 55834574926 for osd.1 2024-09-19T04:45:33.782 DEBUG:teuthology.parallel:result is None 2024-09-19T04:45:33.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:33 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1292414532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T04:45:33.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:33 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2929079511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T04:45:33.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:33 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1292414532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T04:45:33.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:33 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2929079511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T04:45:34.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.941+0000 7f0dd3e00640 1 -- 172.21.15.123:0/677410548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dd41047e0 msgr2=0x7f0dd4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:34.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.941+0000 7f0dd3e00640 1 --2- 172.21.15.123:0/677410548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dd41047e0 0x7f0dd4108830 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009900 tx=0x7f0dc002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:45:34.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.941+0000 7f0dd3e00640 1 -- 172.21.15.123:0/677410548 shutdown_connections 2024-09-19T04:45:34.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.941+0000 7f0dd3e00640 1 --2- 172.21.15.123:0/677410548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dd4108f60 0x7f0dd4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:34.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.941+0000 7f0dd3e00640 1 --2- 172.21.15.123:0/677410548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dd41047e0 0x7f0dd4108830 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:34.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.941+0000 7f0dd3e00640 1 --2- 172.21.15.123:0/677410548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dd4103e30 0x7f0dd4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:34.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.941+0000 7f0dd3e00640 1 -- 172.21.15.123:0/677410548 >> 172.21.15.123:0/677410548 conn(0x7f0dd40ff8d0 msgr2=0x7f0dd4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:34.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.941+0000 7f0dd3e00640 1 -- 172.21.15.123:0/677410548 shutdown_connections 2024-09-19T04:45:34.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.941+0000 7f0dd3e00640 1 -- 172.21.15.123:0/677410548 wait complete. 2024-09-19T04:45:34.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.942+0000 7f0dd3e00640 1 Processor -- start 2024-09-19T04:45:34.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.942+0000 7f0dd3e00640 1 -- start start 2024-09-19T04:45:34.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.942+0000 7f0dd3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dd4103e30 0x7f0dd419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:34.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.943+0000 7f0dd3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dd41047e0 0x7f0dd419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:34.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.943+0000 7f0dd3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dd4108f60 0x7f0dd419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:34.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.943+0000 7f0dd3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd4114010 con 0x7f0dd4103e30 2024-09-19T04:45:34.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.943+0000 7f0dd3e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd4113e90 con 0x7f0dd41047e0 2024-09-19T04:45:34.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.943+0000 7f0dd3e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd4114190 con 0x7f0dd4108f60 2024-09-19T04:45:34.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.943+0000 7f0dd2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dd4103e30 0x7f0dd419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:34.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.943+0000 7f0dd2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dd4103e30 0x7f0dd419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60506/0 (socket says 172.21.15.123:60506) 2024-09-19T04:45:34.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.943+0000 7f0dd2a00640 1 -- 172.21.15.123:0/1292414532 learned_addr learned my addr 172.21.15.123:0/1292414532 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:34.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.943+0000 7f0dd3400640 1 --2- 172.21.15.123:0/1292414532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dd4108f60 0x7f0dd419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:34.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.943+0000 7f0dd2000640 1 --2- 172.21.15.123:0/1292414532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dd41047e0 0x7f0dd419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:34.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.943+0000 7f0dd2a00640 1 -- 172.21.15.123:0/1292414532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dd4108f60 msgr2=0x7f0dd419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:34.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.943+0000 7f0dd2a00640 1 --2- 172.21.15.123:0/1292414532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dd4108f60 0x7f0dd419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:34.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.943+0000 7f0dd2a00640 1 -- 172.21.15.123:0/1292414532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dd41047e0 msgr2=0x7f0dd419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:34.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.943+0000 7f0dd2a00640 1 --2- 172.21.15.123:0/1292414532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dd41047e0 0x7f0dd419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:34.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.943+0000 7f0dd2a00640 1 -- 172.21.15.123:0/1292414532 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd419f440 con 0x7f0dd4103e30 2024-09-19T04:45:34.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.943+0000 7f0dd2000640 1 --2- 172.21.15.123:0/1292414532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dd41047e0 0x7f0dd419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:45:34.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.944+0000 7f0dd2a00640 1 --2- 172.21.15.123:0/1292414532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dd4103e30 0x7f0dd419a490 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400e9f0 tx=0x7f0dc400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:34.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.944+0000 7f0dcb400640 1 -- 172.21.15.123:0/1292414532 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc400cd80 con 0x7f0dd4103e30 2024-09-19T04:45:34.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.944+0000 7f0dcb400640 1 -- 172.21.15.123:0/1292414532 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc4005500 con 0x7f0dd4103e30 2024-09-19T04:45:34.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.944+0000 7f0dcb400640 1 -- 172.21.15.123:0/1292414532 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4010690 con 0x7f0dd4103e30 2024-09-19T04:45:34.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.944+0000 7f0dd3e00640 1 -- 172.21.15.123:0/1292414532 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd419f730 con 0x7f0dd4103e30 2024-09-19T04:45:34.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.944+0000 7f0dd3e00640 1 -- 172.21.15.123:0/1292414532 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd41a7010 con 0x7f0dd4103e30 2024-09-19T04:45:34.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.945+0000 7f0dd3e00640 1 -- 172.21.15.123:0/1292414532 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd4104210 con 0x7f0dd4103e30 2024-09-19T04:45:34.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.947+0000 7f0dcb400640 1 -- 172.21.15.123:0/1292414532 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0dc40046c0 con 0x7f0dd4103e30 2024-09-19T04:45:34.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.948+0000 7f0dcb400640 1 --2- 172.21.15.123:0/1292414532 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0db80a32d0 0x7f0db80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:34.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.948+0000 7f0dcb400640 1 -- 172.21.15.123:0/1292414532 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f0dc40cff80 con 0x7f0dd4103e30 2024-09-19T04:45:34.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.948+0000 7f0dd2000640 1 --2- 172.21.15.123:0/1292414532 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0db80a32d0 0x7f0db80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:34.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.949+0000 7f0dd2000640 1 --2- 172.21.15.123:0/1292414532 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0db80a32d0 0x7f0db80a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0000c00 tx=0x7f0dc0005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:34.072 INFO:teuthology.orchestra.run.smithi123.stdout:34359738452 2024-09-19T04:45:34.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:32.952+0000 7f0dcb400640 1 -- 172.21.15.123:0/1292414532 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0dc409caf0 con 0x7f0dd4103e30 2024-09-19T04:45:34.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.135+0000 7f0dd3e00640 1 -- 172.21.15.123:0/1292414532 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0dd410c1a0 con 0x7f0dd4103e30 2024-09-19T04:45:34.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.136+0000 7f0dcb400640 1 -- 172.21.15.123:0/1292414532 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0dc409c3f0 con 0x7f0dd4103e30 2024-09-19T04:45:34.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.138+0000 7f0dd3e00640 1 -- 172.21.15.123:0/1292414532 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0db80a32d0 msgr2=0x7f0db80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:34.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.138+0000 7f0dd3e00640 1 --2- 172.21.15.123:0/1292414532 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0db80a32d0 0x7f0db80a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0000c00 tx=0x7f0dc0005a40 comp rx=0 tx=0).stop 2024-09-19T04:45:34.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.138+0000 7f0dd3e00640 1 -- 172.21.15.123:0/1292414532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dd4103e30 msgr2=0x7f0dd419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:34.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.138+0000 7f0dd3e00640 1 --2- 172.21.15.123:0/1292414532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dd4103e30 0x7f0dd419a490 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400e9f0 tx=0x7f0dc400edf0 comp rx=0 tx=0).stop 2024-09-19T04:45:34.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.138+0000 7f0dd3e00640 1 -- 172.21.15.123:0/1292414532 shutdown_connections 2024-09-19T04:45:34.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.138+0000 7f0dd3e00640 1 --2- 172.21.15.123:0/1292414532 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0db80a32d0 0x7f0db80a5790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:34.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.138+0000 7f0dd3e00640 1 --2- 172.21.15.123:0/1292414532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dd4108f60 0x7f0dd419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:34.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.138+0000 7f0dd3e00640 1 --2- 172.21.15.123:0/1292414532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dd41047e0 0x7f0dd419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:34.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.138+0000 7f0dd3e00640 1 --2- 172.21.15.123:0/1292414532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dd4103e30 0x7f0dd419a490 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:34.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.138+0000 7f0dd3e00640 1 -- 172.21.15.123:0/1292414532 >> 172.21.15.123:0/1292414532 conn(0x7f0dd40ff8d0 msgr2=0x7f0dd4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:34.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.138+0000 7f0dd3e00640 1 -- 172.21.15.123:0/1292414532 shutdown_connections 2024-09-19T04:45:34.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:33.138+0000 7f0dd3e00640 1 -- 172.21.15.123:0/1292414532 wait complete. 2024-09-19T04:45:34.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:33 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1292414532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T04:45:34.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:33 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2929079511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T04:45:34.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:34 smithi123 ceph-mon[30511]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T04:45:34.729 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:34.781 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182466 got 107374182468 for osd.3 2024-09-19T04:45:34.781 DEBUG:teuthology.parallel:result is None 2024-09-19T04:45:34.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:34 smithi193 ceph-mon[35592]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T04:45:35.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:34 smithi183 ceph-mon[35333]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T04:45:35.126 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738449 got 34359738452 for osd.0 2024-09-19T04:45:35.126 DEBUG:teuthology.parallel:result is None 2024-09-19T04:45:35.737 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:35 smithi123 ceph-mon[30511]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:45:35.775 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135911 got 244813135914 for osd.8 2024-09-19T04:45:35.775 DEBUG:teuthology.parallel:result is None 2024-09-19T04:45:35.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:35 smithi193 ceph-mon[35592]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:45:36.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:35 smithi183 ceph-mon[35333]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:45:36.103 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364843 got 214748364846 for osd.7 2024-09-19T04:45:36.103 DEBUG:teuthology.parallel:result is None 2024-09-19T04:45:36.423 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757303 got 163208757305 for osd.5 2024-09-19T04:45:36.423 DEBUG:teuthology.parallel:result is None 2024-09-19T04:45:36.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.888+0000 7f64b3200640 1 -- 172.21.15.123:0/563989923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac102790 msgr2=0x7f64ac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:36.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.888+0000 7f64b3200640 1 --2- 172.21.15.123:0/563989923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac102790 0x7f64ac1067e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6498009900 tx=0x7f649802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:45:36.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.888+0000 7f64b3200640 1 -- 172.21.15.123:0/563989923 shutdown_connections 2024-09-19T04:45:36.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.888+0000 7f64b3200640 1 --2- 172.21.15.123:0/563989923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64ac106f10 0x7f64ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:36.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.888+0000 7f64b3200640 1 --2- 172.21.15.123:0/563989923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac102790 0x7f64ac1067e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:36.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.888+0000 7f64b3200640 1 --2- 172.21.15.123:0/563989923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64ac101de0 0x7f64ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:36.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.888+0000 7f64b3200640 1 -- 172.21.15.123:0/563989923 >> 172.21.15.123:0/563989923 conn(0x7f64ac0fd8c0 msgr2=0x7f64ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:36.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.888+0000 7f64b3200640 1 -- 172.21.15.123:0/563989923 shutdown_connections 2024-09-19T04:45:36.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.889+0000 7f64b3200640 1 -- 172.21.15.123:0/563989923 wait complete. 2024-09-19T04:45:36.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.889+0000 7f64b3200640 1 Processor -- start 2024-09-19T04:45:36.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.890+0000 7f64b3200640 1 -- start start 2024-09-19T04:45:36.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.890+0000 7f64b3200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64ac101de0 0x7f64ac198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:36.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.890+0000 7f64b3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac102790 0x7f64ac198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:36.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.890+0000 7f64b3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64ac106f10 0x7f64ac19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:36.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.890+0000 7f64b3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac111f70 con 0x7f64ac102790 2024-09-19T04:45:36.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.890+0000 7f64b3200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac111df0 con 0x7f64ac106f10 2024-09-19T04:45:36.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.890+0000 7f64b3200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac1120f0 con 0x7f64ac101de0 2024-09-19T04:45:36.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.890+0000 7f64abe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64ac101de0 0x7f64ac198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:36.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.890+0000 7f64ab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac102790 0x7f64ac198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:36.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.890+0000 7f64abe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64ac101de0 0x7f64ac198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:52740/0 (socket says 172.21.15.123:52740) 2024-09-19T04:45:36.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.890+0000 7f64b1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64ac106f10 0x7f64ac19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:36.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.890+0000 7f64abe00640 1 -- 172.21.15.123:0/1942812287 learned_addr learned my addr 172.21.15.123:0/1942812287 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:36.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.891+0000 7f64abe00640 1 -- 172.21.15.123:0/1942812287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64ac106f10 msgr2=0x7f64ac19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:36.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.891+0000 7f64abe00640 1 --2- 172.21.15.123:0/1942812287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64ac106f10 0x7f64ac19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:36.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.891+0000 7f64abe00640 1 -- 172.21.15.123:0/1942812287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac102790 msgr2=0x7f64ac198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:36.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.891+0000 7f64abe00640 1 --2- 172.21.15.123:0/1942812287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac102790 0x7f64ac198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:36.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.891+0000 7f64abe00640 1 -- 172.21.15.123:0/1942812287 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac19d3c0 con 0x7f64ac101de0 2024-09-19T04:45:36.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.891+0000 7f64b1200640 1 --2- 172.21.15.123:0/1942812287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64ac106f10 0x7f64ac19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:36.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.891+0000 7f64abe00640 1 --2- 172.21.15.123:0/1942812287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64ac101de0 0x7f64ac198410 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f649c00e9f0 tx=0x7f649c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:36.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.891+0000 7f64a8c00640 1 -- 172.21.15.123:0/1942812287 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c00cd80 con 0x7f64ac101de0 2024-09-19T04:45:36.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.891+0000 7f64b3200640 1 -- 172.21.15.123:0/1942812287 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ac19d6b0 con 0x7f64ac101de0 2024-09-19T04:45:36.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.892+0000 7f64a8c00640 1 -- 172.21.15.123:0/1942812287 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f649c010910 con 0x7f64ac101de0 2024-09-19T04:45:36.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.892+0000 7f64a8c00640 1 -- 172.21.15.123:0/1942812287 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c0188f0 con 0x7f64ac101de0 2024-09-19T04:45:36.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.892+0000 7f64b3200640 1 -- 172.21.15.123:0/1942812287 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ac074420 con 0x7f64ac101de0 2024-09-19T04:45:36.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.893+0000 7f64b3200640 1 -- 172.21.15.123:0/1942812287 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ac1021c0 con 0x7f64ac101de0 2024-09-19T04:45:36.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.895+0000 7f64a8c00640 1 -- 172.21.15.123:0/1942812287 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f649c010ac0 con 0x7f64ac101de0 2024-09-19T04:45:36.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.896+0000 7f64a8c00640 1 --2- 172.21.15.123:0/1942812287 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f64900a3350 0x7f64900a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:36.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.896+0000 7f64a8c00640 1 -- 172.21.15.123:0/1942812287 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f649c0d0090 con 0x7f64ac101de0 2024-09-19T04:45:36.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.896+0000 7f64ab400640 1 --2- 172.21.15.123:0/1942812287 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f64900a3350 0x7f64900a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:36.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.897+0000 7f64ab400640 1 --2- 172.21.15.123:0/1942812287 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f64900a3350 0x7f64900a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6498009230 tx=0x7f649803c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:36.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:36.898+0000 7f64a8c00640 1 -- 172.21.15.123:0/1942812287 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f649c09c610 con 0x7f64ac101de0 2024-09-19T04:45:37.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.083+0000 7f64b3200640 1 -- 172.21.15.123:0/1942812287 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f64ac10a0f0 con 0x7f64ac101de0 2024-09-19T04:45:37.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.084+0000 7f64a8c00640 1 -- 172.21.15.123:0/1942812287 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f649c0a0910 con 0x7f64ac101de0 2024-09-19T04:45:37.085 INFO:teuthology.orchestra.run.smithi123.stdout:300647710752 2024-09-19T04:45:37.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.086+0000 7f64b3200640 1 -- 172.21.15.123:0/1942812287 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f64900a3350 msgr2=0x7f64900a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:37.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.086+0000 7f64b3200640 1 --2- 172.21.15.123:0/1942812287 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f64900a3350 0x7f64900a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6498009230 tx=0x7f649803c040 comp rx=0 tx=0).stop 2024-09-19T04:45:37.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.086+0000 7f64b3200640 1 -- 172.21.15.123:0/1942812287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64ac101de0 msgr2=0x7f64ac198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:37.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.086+0000 7f64b3200640 1 --2- 172.21.15.123:0/1942812287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64ac101de0 0x7f64ac198410 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f649c00e9f0 tx=0x7f649c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:45:37.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.086+0000 7f64b3200640 1 -- 172.21.15.123:0/1942812287 shutdown_connections 2024-09-19T04:45:37.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.086+0000 7f64b3200640 1 --2- 172.21.15.123:0/1942812287 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f64900a3350 0x7f64900a5810 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:37.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.086+0000 7f64b3200640 1 --2- 172.21.15.123:0/1942812287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64ac106f10 0x7f64ac19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:37.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.086+0000 7f64b3200640 1 --2- 172.21.15.123:0/1942812287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac102790 0x7f64ac198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:37.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.086+0000 7f64b3200640 1 --2- 172.21.15.123:0/1942812287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64ac101de0 0x7f64ac198410 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:37.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.086+0000 7f64b3200640 1 -- 172.21.15.123:0/1942812287 >> 172.21.15.123:0/1942812287 conn(0x7f64ac0fd8c0 msgr2=0x7f64ac0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:37.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.086+0000 7f64b3200640 1 -- 172.21.15.123:0/1942812287 shutdown_connections 2024-09-19T04:45:37.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.086+0000 7f64b3200640 1 -- 172.21.15.123:0/1942812287 wait complete. 2024-09-19T04:45:37.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.116+0000 7eff57e00640 1 -- 172.21.15.123:0/3814464450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50106f00 msgr2=0x7eff5010ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:37.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.116+0000 7eff57e00640 1 --2- 172.21.15.123:0/3814464450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50106f00 0x7eff5010ea80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7eff48009ca0 tx=0x7eff4802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:45:37.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.116+0000 7eff57e00640 1 -- 172.21.15.123:0/3814464450 shutdown_connections 2024-09-19T04:45:37.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.116+0000 7eff57e00640 1 --2- 172.21.15.123:0/3814464450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50106f00 0x7eff5010ea80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:37.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.116+0000 7eff57e00640 1 --2- 172.21.15.123:0/3814464450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff50102780 0x7eff501067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:37.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.116+0000 7eff57e00640 1 --2- 172.21.15.123:0/3814464450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff50101dd0 0x7eff501021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:37.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.116+0000 7eff57e00640 1 -- 172.21.15.123:0/3814464450 >> 172.21.15.123:0/3814464450 conn(0x7eff500fd8b0 msgr2=0x7eff500ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:37.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.116+0000 7eff57e00640 1 -- 172.21.15.123:0/3814464450 shutdown_connections 2024-09-19T04:45:37.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.116+0000 7eff57e00640 1 -- 172.21.15.123:0/3814464450 wait complete. 2024-09-19T04:45:37.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.117+0000 7eff57e00640 1 Processor -- start 2024-09-19T04:45:37.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.118+0000 7eff57e00640 1 -- start start 2024-09-19T04:45:37.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.118+0000 7eff57e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50101dd0 0x7eff5019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:37.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.118+0000 7eff57e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff50102780 0x7eff5019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:37.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.118+0000 7eff57e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff50106f00 0x7eff5019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:37.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.118+0000 7eff57e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff50111fb0 con 0x7eff50101dd0 2024-09-19T04:45:37.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.118+0000 7eff57e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff50111e30 con 0x7eff50106f00 2024-09-19T04:45:37.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.118+0000 7eff57e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7eff50112130 con 0x7eff50102780 2024-09-19T04:45:37.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.118+0000 7eff55400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50101dd0 0x7eff5019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:37.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.118+0000 7eff55400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50101dd0 0x7eff5019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60574/0 (socket says 172.21.15.123:60574) 2024-09-19T04:45:37.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.118+0000 7eff55400640 1 -- 172.21.15.123:0/2646858573 learned_addr learned my addr 172.21.15.123:0/2646858573 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:37.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.118+0000 7eff55e00640 1 --2- 172.21.15.123:0/2646858573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff50106f00 0x7eff5019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:37.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.118+0000 7eff54a00640 1 --2- 172.21.15.123:0/2646858573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff50102780 0x7eff5019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:37.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.119+0000 7eff55400640 1 -- 172.21.15.123:0/2646858573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff50102780 msgr2=0x7eff5019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:37.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.119+0000 7eff55400640 1 --2- 172.21.15.123:0/2646858573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff50102780 0x7eff5019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:37.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.119+0000 7eff55400640 1 -- 172.21.15.123:0/2646858573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff50106f00 msgr2=0x7eff5019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:37.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.119+0000 7eff55400640 1 --2- 172.21.15.123:0/2646858573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff50106f00 0x7eff5019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:37.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.119+0000 7eff55400640 1 -- 172.21.15.123:0/2646858573 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5019f5c0 con 0x7eff50101dd0 2024-09-19T04:45:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.119+0000 7eff55400640 1 --2- 172.21.15.123:0/2646858573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50101dd0 0x7eff5019a610 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff4400e9f0 tx=0x7eff4400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.119+0000 7eff4e000640 1 -- 172.21.15.123:0/2646858573 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4400cd80 con 0x7eff50101dd0 2024-09-19T04:45:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.119+0000 7eff57e00640 1 -- 172.21.15.123:0/2646858573 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5019f8b0 con 0x7eff50101dd0 2024-09-19T04:45:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.119+0000 7eff4e000640 1 -- 172.21.15.123:0/2646858573 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff44010910 con 0x7eff50101dd0 2024-09-19T04:45:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.119+0000 7eff4e000640 1 -- 172.21.15.123:0/2646858573 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff440188f0 con 0x7eff50101dd0 2024-09-19T04:45:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.119+0000 7eff57e00640 1 -- 172.21.15.123:0/2646858573 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff501a7190 con 0x7eff50101dd0 2024-09-19T04:45:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.120+0000 7eff57e00640 1 -- 172.21.15.123:0/2646858573 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff501021b0 con 0x7eff50101dd0 2024-09-19T04:45:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.122+0000 7eff4e000640 1 -- 172.21.15.123:0/2646858573 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7eff44010ac0 con 0x7eff50101dd0 2024-09-19T04:45:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.124+0000 7eff4e000640 1 --2- 172.21.15.123:0/2646858573 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7eff380a3350 0x7eff380a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.124+0000 7eff4e000640 1 -- 172.21.15.123:0/2646858573 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7eff440d0180 con 0x7eff50101dd0 2024-09-19T04:45:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.124+0000 7eff54a00640 1 --2- 172.21.15.123:0/2646858573 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7eff380a3350 0x7eff380a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.125+0000 7eff54a00640 1 --2- 172.21.15.123:0/2646858573 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7eff380a3350 0x7eff380a5810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7eff50076480 tx=0x7eff3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:37.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.126+0000 7eff4e000640 1 -- 172.21.15.123:0/2646858573 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7eff4409c700 con 0x7eff50101dd0 2024-09-19T04:45:37.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.306+0000 7eff57e00640 1 -- 172.21.15.123:0/2646858573 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7eff501a7430 con 0x7eff50101dd0 2024-09-19T04:45:37.308 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:37 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1942812287' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T04:45:37.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.306+0000 7eff4e000640 1 -- 172.21.15.123:0/2646858573 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7eff440a0a00 con 0x7eff50101dd0 2024-09-19T04:45:37.308 INFO:teuthology.orchestra.run.smithi123.stdout:279172874277 2024-09-19T04:45:37.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.309+0000 7eff57e00640 1 -- 172.21.15.123:0/2646858573 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7eff380a3350 msgr2=0x7eff380a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:37.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.309+0000 7eff57e00640 1 --2- 172.21.15.123:0/2646858573 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7eff380a3350 0x7eff380a5810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7eff50076480 tx=0x7eff3c005f90 comp rx=0 tx=0).stop 2024-09-19T04:45:37.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.309+0000 7eff57e00640 1 -- 172.21.15.123:0/2646858573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50101dd0 msgr2=0x7eff5019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:37.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.309+0000 7eff57e00640 1 --2- 172.21.15.123:0/2646858573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50101dd0 0x7eff5019a610 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff4400e9f0 tx=0x7eff4400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:45:37.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.309+0000 7eff57e00640 1 -- 172.21.15.123:0/2646858573 shutdown_connections 2024-09-19T04:45:37.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.309+0000 7eff57e00640 1 --2- 172.21.15.123:0/2646858573 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7eff380a3350 0x7eff380a5810 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:37.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.309+0000 7eff57e00640 1 --2- 172.21.15.123:0/2646858573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff50106f00 0x7eff5019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:37.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.309+0000 7eff57e00640 1 --2- 172.21.15.123:0/2646858573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff50102780 0x7eff5019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:37.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.309+0000 7eff57e00640 1 --2- 172.21.15.123:0/2646858573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50101dd0 0x7eff5019a610 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:37.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.309+0000 7eff57e00640 1 -- 172.21.15.123:0/2646858573 >> 172.21.15.123:0/2646858573 conn(0x7eff500fd8b0 msgr2=0x7eff500febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:37.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.309+0000 7eff57e00640 1 -- 172.21.15.123:0/2646858573 shutdown_connections 2024-09-19T04:45:37.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:37.309+0000 7eff57e00640 1 -- 172.21.15.123:0/2646858573 wait complete. 2024-09-19T04:45:37.395 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:37.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:37 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1942812287' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T04:45:37.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:37 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1942812287' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T04:45:38.126 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874275 got 279172874277 for osd.9 2024-09-19T04:45:38.126 DEBUG:teuthology.parallel:result is None 2024-09-19T04:45:38.365 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:38 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2646858573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T04:45:38.365 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:38 smithi123 ceph-mon[30511]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-19T04:45:38.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:38 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2646858573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T04:45:38.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:38 smithi193 ceph-mon[35592]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-19T04:45:38.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:38 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2646858573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T04:45:38.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:38 smithi183 ceph-mon[35333]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-19T04:45:38.668 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953532 got 137438953534 for osd.4 2024-09-19T04:45:38.668 DEBUG:teuthology.parallel:result is None 2024-09-19T04:45:38.684 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:39.698 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710748 got 300647710752 for osd.10 2024-09-19T04:45:39.698 DEBUG:teuthology.parallel:result is None 2024-09-19T04:45:40.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.530+0000 7fbfb9c00640 1 -- 172.21.15.123:0/943189320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb40745c0 msgr2=0x7fbfb40749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:40.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.530+0000 7fbfb9c00640 1 --2- 172.21.15.123:0/943189320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb40745c0 0x7fbfb40749a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009900 tx=0x7fbfa402a480 comp rx=0 tx=0).stop 2024-09-19T04:45:40.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.530+0000 7fbfb9c00640 1 -- 172.21.15.123:0/943189320 shutdown_connections 2024-09-19T04:45:40.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.530+0000 7fbfb9c00640 1 --2- 172.21.15.123:0/943189320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb410e430 0x7fbfb4110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.530+0000 7fbfb9c00640 1 --2- 172.21.15.123:0/943189320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfb4074ee0 0x7fbfb410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.530+0000 7fbfb9c00640 1 --2- 172.21.15.123:0/943189320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb40745c0 0x7fbfb40749a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.530+0000 7fbfb9c00640 1 -- 172.21.15.123:0/943189320 >> 172.21.15.123:0/943189320 conn(0x7fbfb40fd8b0 msgr2=0x7fbfb40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:40.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.530+0000 7fbfb9c00640 1 -- 172.21.15.123:0/943189320 shutdown_connections 2024-09-19T04:45:40.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.530+0000 7fbfb9c00640 1 -- 172.21.15.123:0/943189320 wait complete. 2024-09-19T04:45:40.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.531+0000 7fbfb9c00640 1 Processor -- start 2024-09-19T04:45:40.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.531+0000 7fbfb9c00640 1 -- start start 2024-09-19T04:45:40.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.532+0000 7fbfb9c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb40745c0 0x7fbfb419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:40.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.532+0000 7fbfb9c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb4074ee0 0x7fbfb419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:40.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.532+0000 7fbfb9c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfb410e430 0x7fbfb419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:40.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.532+0000 7fbfb9c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41141f0 con 0x7fbfb40745c0 2024-09-19T04:45:40.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.532+0000 7fbfb9c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb4114070 con 0x7fbfb4074ee0 2024-09-19T04:45:40.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.532+0000 7fbfb9c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb4114370 con 0x7fbfb410e430 2024-09-19T04:45:40.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.532+0000 7fbfb2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb40745c0 0x7fbfb419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:40.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.532+0000 7fbfb2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb40745c0 0x7fbfb419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60598/0 (socket says 172.21.15.123:60598) 2024-09-19T04:45:40.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.532+0000 7fbfb2a00640 1 -- 172.21.15.123:0/3116084222 learned_addr learned my addr 172.21.15.123:0/3116084222 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:40.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.532+0000 7fbfb3400640 1 --2- 172.21.15.123:0/3116084222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfb410e430 0x7fbfb419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:40.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.532+0000 7fbfb2000640 1 --2- 172.21.15.123:0/3116084222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb4074ee0 0x7fbfb419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:40.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.532+0000 7fbfb2a00640 1 -- 172.21.15.123:0/3116084222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfb410e430 msgr2=0x7fbfb419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:40.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.532+0000 7fbfb2a00640 1 --2- 172.21.15.123:0/3116084222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfb410e430 0x7fbfb419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.532+0000 7fbfb2a00640 1 -- 172.21.15.123:0/3116084222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb4074ee0 msgr2=0x7fbfb419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:40.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.532+0000 7fbfb2a00640 1 --2- 172.21.15.123:0/3116084222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb4074ee0 0x7fbfb419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.532+0000 7fbfb2a00640 1 -- 172.21.15.123:0/3116084222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb419f620 con 0x7fbfb40745c0 2024-09-19T04:45:40.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.533+0000 7fbfb2a00640 1 --2- 172.21.15.123:0/3116084222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb40745c0 0x7fbfb419a670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4004770 tx=0x7fbfa4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:40.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.533+0000 7fbfab400640 1 -- 172.21.15.123:0/3116084222 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa403f070 con 0x7fbfb40745c0 2024-09-19T04:45:40.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.533+0000 7fbfab400640 1 -- 172.21.15.123:0/3116084222 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfa4031ca0 con 0x7fbfb40745c0 2024-09-19T04:45:40.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.533+0000 7fbfab400640 1 -- 172.21.15.123:0/3116084222 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4035030 con 0x7fbfb40745c0 2024-09-19T04:45:40.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.533+0000 7fbfb9c00640 1 -- 172.21.15.123:0/3116084222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb419f8b0 con 0x7fbfb40745c0 2024-09-19T04:45:40.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.533+0000 7fbfb9c00640 1 -- 172.21.15.123:0/3116084222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb419fd40 con 0x7fbfb40745c0 2024-09-19T04:45:40.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.534+0000 7fbfb9c00640 1 -- 172.21.15.123:0/3116084222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb40749a0 con 0x7fbfb40745c0 2024-09-19T04:45:40.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.536+0000 7fbfab400640 1 -- 172.21.15.123:0/3116084222 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fbfa4042070 con 0x7fbfb40745c0 2024-09-19T04:45:40.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.538+0000 7fbfab400640 1 --2- 172.21.15.123:0/3116084222 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbf980a3350 0x7fbf980a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:40.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.538+0000 7fbfab400640 1 -- 172.21.15.123:0/3116084222 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fbfa40efff0 con 0x7fbfb40745c0 2024-09-19T04:45:40.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.538+0000 7fbfb2000640 1 --2- 172.21.15.123:0/3116084222 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbf980a3350 0x7fbf980a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:40.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.539+0000 7fbfb2000640 1 --2- 172.21.15.123:0/3116084222 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbf980a3350 0x7fbf980a5810 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbfb410f750 tx=0x7fbfa0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:40.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.542+0000 7fbfab400640 1 -- 172.21.15.123:0/3116084222 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbfa40bc610 con 0x7fbfb40745c0 2024-09-19T04:45:40.673 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:40 smithi123 ceph-mon[30511]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-19T04:45:40.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.725+0000 7fbfb9c00640 1 -- 172.21.15.123:0/3116084222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fbfb41a7360 con 0x7fbfb40745c0 2024-09-19T04:45:40.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.726+0000 7fbfab400640 1 -- 172.21.15.123:0/3116084222 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fbfa40c0910 con 0x7fbfb40745c0 2024-09-19T04:45:40.727 INFO:teuthology.orchestra.run.smithi123.stdout:330712481820 2024-09-19T04:45:40.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.728+0000 7fbfb9c00640 1 -- 172.21.15.123:0/3116084222 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbf980a3350 msgr2=0x7fbf980a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:40.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.728+0000 7fbfb9c00640 1 --2- 172.21.15.123:0/3116084222 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbf980a3350 0x7fbf980a5810 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbfb410f750 tx=0x7fbfa0005f90 comp rx=0 tx=0).stop 2024-09-19T04:45:40.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.728+0000 7fbfb9c00640 1 -- 172.21.15.123:0/3116084222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb40745c0 msgr2=0x7fbfb419a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:40.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.728+0000 7fbfb9c00640 1 --2- 172.21.15.123:0/3116084222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb40745c0 0x7fbfb419a670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4004770 tx=0x7fbfa4003c10 comp rx=0 tx=0).stop 2024-09-19T04:45:40.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.728+0000 7fbfb9c00640 1 -- 172.21.15.123:0/3116084222 shutdown_connections 2024-09-19T04:45:40.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.728+0000 7fbfb9c00640 1 --2- 172.21.15.123:0/3116084222 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbf980a3350 0x7fbf980a5810 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.728+0000 7fbfb9c00640 1 --2- 172.21.15.123:0/3116084222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfb410e430 0x7fbfb419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.728+0000 7fbfb9c00640 1 --2- 172.21.15.123:0/3116084222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb4074ee0 0x7fbfb419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.728+0000 7fbfb9c00640 1 --2- 172.21.15.123:0/3116084222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb40745c0 0x7fbfb419a670 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.728+0000 7fbfb9c00640 1 -- 172.21.15.123:0/3116084222 >> 172.21.15.123:0/3116084222 conn(0x7fbfb40fd8b0 msgr2=0x7fbfb40fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:40.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.728+0000 7fbfb9c00640 1 -- 172.21.15.123:0/3116084222 shutdown_connections 2024-09-19T04:45:40.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.728+0000 7fbfb9c00640 1 -- 172.21.15.123:0/3116084222 wait complete. 2024-09-19T04:45:40.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:40 smithi193 ceph-mon[35592]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-19T04:45:40.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.780+0000 7f499f800640 1 -- 172.21.15.123:0/4178912644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4998074ee0 msgr2=0x7f499810dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:40.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.780+0000 7f499f800640 1 --2- 172.21.15.123:0/4178912644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4998074ee0 0x7f499810dee0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4984009900 tx=0x7f498402a480 comp rx=0 tx=0).stop 2024-09-19T04:45:40.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.780+0000 7f499f800640 1 -- 172.21.15.123:0/4178912644 shutdown_connections 2024-09-19T04:45:40.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.780+0000 7f499f800640 1 --2- 172.21.15.123:0/4178912644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f499810e420 0x7f4998110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.780+0000 7f499f800640 1 --2- 172.21.15.123:0/4178912644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4998074ee0 0x7f499810dee0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.780+0000 7f499f800640 1 --2- 172.21.15.123:0/4178912644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49980745c0 0x7f49980749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.780+0000 7f499f800640 1 -- 172.21.15.123:0/4178912644 >> 172.21.15.123:0/4178912644 conn(0x7f49980fd8c0 msgr2=0x7f49980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:40.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.780+0000 7f499f800640 1 -- 172.21.15.123:0/4178912644 shutdown_connections 2024-09-19T04:45:40.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.780+0000 7f499f800640 1 -- 172.21.15.123:0/4178912644 wait complete. 2024-09-19T04:45:40.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.781+0000 7f499f800640 1 Processor -- start 2024-09-19T04:45:40.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.781+0000 7f499f800640 1 -- start start 2024-09-19T04:45:40.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.781+0000 7f499f800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49980745c0 0x7f499819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:40.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.781+0000 7f499f800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4998074ee0 0x7f499819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:40.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.781+0000 7f499f800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f499810e420 0x7f499819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:40.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.782+0000 7f499f800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f49981141b0 con 0x7f49980745c0 2024-09-19T04:45:40.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.782+0000 7f499f800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4998114030 con 0x7f4998074ee0 2024-09-19T04:45:40.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.782+0000 7f499f800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4998114330 con 0x7f499810e420 2024-09-19T04:45:40.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.782+0000 7f499ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49980745c0 0x7f499819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:40.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.782+0000 7f499ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49980745c0 0x7f499819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60616/0 (socket says 172.21.15.123:60616) 2024-09-19T04:45:40.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.782+0000 7f499ce00640 1 -- 172.21.15.123:0/1609916123 learned_addr learned my addr 172.21.15.123:0/1609916123 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:40.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.782+0000 7f499d800640 1 --2- 172.21.15.123:0/1609916123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f499810e420 0x7f499819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:40.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.782+0000 7f4997e00640 1 --2- 172.21.15.123:0/1609916123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4998074ee0 0x7f499819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:40.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.782+0000 7f499ce00640 1 -- 172.21.15.123:0/1609916123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f499810e420 msgr2=0x7f499819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:40.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.782+0000 7f499ce00640 1 --2- 172.21.15.123:0/1609916123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f499810e420 0x7f499819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.782+0000 7f499ce00640 1 -- 172.21.15.123:0/1609916123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4998074ee0 msgr2=0x7f499819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:40.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.782+0000 7f499ce00640 1 --2- 172.21.15.123:0/1609916123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4998074ee0 0x7f499819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.782+0000 7f499ce00640 1 -- 172.21.15.123:0/1609916123 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499819f620 con 0x7f49980745c0 2024-09-19T04:45:40.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.782+0000 7f499d800640 1 --2- 172.21.15.123:0/1609916123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f499810e420 0x7f499819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:40.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.783+0000 7f499ce00640 1 --2- 172.21.15.123:0/1609916123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49980745c0 0x7f499819a670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f498c00ea60 tx=0x7f498c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:40.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.783+0000 7f4995600640 1 -- 172.21.15.123:0/1609916123 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c00cde0 con 0x7f49980745c0 2024-09-19T04:45:40.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.783+0000 7f4995600640 1 -- 172.21.15.123:0/1609916123 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f498c0035f0 con 0x7f49980745c0 2024-09-19T04:45:40.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.783+0000 7f499f800640 1 -- 172.21.15.123:0/1609916123 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499819f910 con 0x7f49980745c0 2024-09-19T04:45:40.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.783+0000 7f4995600640 1 -- 172.21.15.123:0/1609916123 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c010670 con 0x7f49980745c0 2024-09-19T04:45:40.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.783+0000 7f499f800640 1 -- 172.21.15.123:0/1609916123 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49981a71f0 con 0x7f49980745c0 2024-09-19T04:45:40.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.784+0000 7f499f800640 1 -- 172.21.15.123:0/1609916123 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49980749a0 con 0x7f49980745c0 2024-09-19T04:45:40.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.786+0000 7f4995600640 1 -- 172.21.15.123:0/1609916123 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f498c0037a0 con 0x7f49980745c0 2024-09-19T04:45:40.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.787+0000 7f4995600640 1 --2- 172.21.15.123:0/1609916123 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f49800a32d0 0x7f49800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:40.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.787+0000 7f4995600640 1 -- 172.21.15.123:0/1609916123 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f498c0cfde0 con 0x7f49980745c0 2024-09-19T04:45:40.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.787+0000 7f4997e00640 1 --2- 172.21.15.123:0/1609916123 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f49800a32d0 0x7f49800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:40.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.788+0000 7f4997e00640 1 --2- 172.21.15.123:0/1609916123 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f49800a32d0 0x7f49800a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4984000c00 tx=0x7f4984005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:40.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.791+0000 7f4995600640 1 -- 172.21.15.123:0/1609916123 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f498c09c950 con 0x7f49980745c0 2024-09-19T04:45:40.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:40 smithi183 ceph-mon[35333]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-19T04:45:40.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.974+0000 7f499f800640 1 -- 172.21.15.123:0/1609916123 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4998109010 con 0x7f49980745c0 2024-09-19T04:45:40.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.974+0000 7f4995600640 1 -- 172.21.15.123:0/1609916123 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f498c09c250 con 0x7f49980745c0 2024-09-19T04:45:40.976 INFO:teuthology.orchestra.run.smithi123.stdout:77309411403 2024-09-19T04:45:40.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.976+0000 7f499f800640 1 -- 172.21.15.123:0/1609916123 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f49800a32d0 msgr2=0x7f49800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:40.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.976+0000 7f499f800640 1 --2- 172.21.15.123:0/1609916123 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f49800a32d0 0x7f49800a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4984000c00 tx=0x7f4984005b50 comp rx=0 tx=0).stop 2024-09-19T04:45:40.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.976+0000 7f499f800640 1 -- 172.21.15.123:0/1609916123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49980745c0 msgr2=0x7f499819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:40.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.976+0000 7f499f800640 1 --2- 172.21.15.123:0/1609916123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49980745c0 0x7f499819a670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f498c00ea60 tx=0x7f498c00ef20 comp rx=0 tx=0).stop 2024-09-19T04:45:40.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.976+0000 7f499f800640 1 -- 172.21.15.123:0/1609916123 shutdown_connections 2024-09-19T04:45:40.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.976+0000 7f499f800640 1 --2- 172.21.15.123:0/1609916123 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f49800a32d0 0x7f49800a5790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.976+0000 7f499f800640 1 --2- 172.21.15.123:0/1609916123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f499810e420 0x7f499819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.976+0000 7f499f800640 1 --2- 172.21.15.123:0/1609916123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4998074ee0 0x7f499819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.976+0000 7f499f800640 1 --2- 172.21.15.123:0/1609916123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49980745c0 0x7f499819a670 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:40.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.976+0000 7f499f800640 1 -- 172.21.15.123:0/1609916123 >> 172.21.15.123:0/1609916123 conn(0x7f49980fd8c0 msgr2=0x7f49980fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:40.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.977+0000 7f499f800640 1 -- 172.21.15.123:0/1609916123 shutdown_connections 2024-09-19T04:45:40.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:40.977+0000 7f499f800640 1 -- 172.21.15.123:0/1609916123 wait complete. 2024-09-19T04:45:41.588 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:41 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3116084222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T04:45:41.675 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:41 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1609916123' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T04:45:41.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:41 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3116084222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T04:45:41.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:41 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1609916123' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T04:45:41.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:41 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3116084222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T04:45:41.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:41 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1609916123' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T04:45:41.911 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481818 got 330712481820 for osd.11 2024-09-19T04:45:41.911 DEBUG:teuthology.parallel:result is None 2024-09-19T04:45:42.331 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411399 got 77309411403 for osd.2 2024-09-19T04:45:42.331 DEBUG:teuthology.parallel:result is None 2024-09-19T04:45:42.332 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T04:45:42.332 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:45:42.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:42 smithi123 ceph-mon[30511]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:45:42.739 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:42.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:42 smithi193 ceph-mon[35592]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:45:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:42 smithi183 ceph-mon[35333]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:45:43.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.316+0000 7ff463600640 1 -- 172.21.15.123:0/3856173087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff45c10e3c0 msgr2=0x7ff45c110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:43.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.316+0000 7ff463600640 1 --2- 172.21.15.123:0/3856173087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff45c10e3c0 0x7ff45c110810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff454009ca0 tx=0x7ff45402a950 comp rx=0 tx=0).stop 2024-09-19T04:45:43.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.316+0000 7ff463600640 1 -- 172.21.15.123:0/3856173087 shutdown_connections 2024-09-19T04:45:43.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.316+0000 7ff463600640 1 --2- 172.21.15.123:0/3856173087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff45c10e3c0 0x7ff45c110810 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:43.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.316+0000 7ff463600640 1 --2- 172.21.15.123:0/3856173087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff45c074f40 0x7ff45c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:43.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.316+0000 7ff463600640 1 --2- 172.21.15.123:0/3856173087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff45c074620 0x7ff45c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:43.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.316+0000 7ff463600640 1 -- 172.21.15.123:0/3856173087 >> 172.21.15.123:0/3856173087 conn(0x7ff45c0fd8c0 msgr2=0x7ff45c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:43.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.316+0000 7ff463600640 1 -- 172.21.15.123:0/3856173087 shutdown_connections 2024-09-19T04:45:43.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.316+0000 7ff463600640 1 -- 172.21.15.123:0/3856173087 wait complete. 2024-09-19T04:45:43.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.318+0000 7ff463600640 1 Processor -- start 2024-09-19T04:45:43.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.318+0000 7ff463600640 1 -- start start 2024-09-19T04:45:43.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.318+0000 7ff463600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff45c074620 0x7ff45c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:43.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.318+0000 7ff463600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff45c074f40 0x7ff45c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:43.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.318+0000 7ff463600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff45c10e3c0 0x7ff45c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:43.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.318+0000 7ff463600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c1141b0 con 0x7ff45c074620 2024-09-19T04:45:43.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.318+0000 7ff463600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c114030 con 0x7ff45c074f40 2024-09-19T04:45:43.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.318+0000 7ff463600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c114330 con 0x7ff45c10e3c0 2024-09-19T04:45:43.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.318+0000 7ff460c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff45c074620 0x7ff45c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:43.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.319+0000 7ff460c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff45c074620 0x7ff45c19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46536/0 (socket says 172.21.15.123:46536) 2024-09-19T04:45:43.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.319+0000 7ff45be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff45c074f40 0x7ff45c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:43.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.319+0000 7ff460c00640 1 -- 172.21.15.123:0/1761000178 learned_addr learned my addr 172.21.15.123:0/1761000178 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:43.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.319+0000 7ff461600640 1 --2- 172.21.15.123:0/1761000178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff45c10e3c0 0x7ff45c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:43.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.319+0000 7ff460c00640 1 -- 172.21.15.123:0/1761000178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff45c10e3c0 msgr2=0x7ff45c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:43.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.319+0000 7ff460c00640 1 --2- 172.21.15.123:0/1761000178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff45c10e3c0 0x7ff45c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:43.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.319+0000 7ff460c00640 1 -- 172.21.15.123:0/1761000178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff45c074f40 msgr2=0x7ff45c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:43.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.319+0000 7ff460c00640 1 --2- 172.21.15.123:0/1761000178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff45c074f40 0x7ff45c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:43.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.319+0000 7ff460c00640 1 -- 172.21.15.123:0/1761000178 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c19f5e0 con 0x7ff45c074620 2024-09-19T04:45:43.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.319+0000 7ff45be00640 1 --2- 172.21.15.123:0/1761000178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff45c074f40 0x7ff45c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:43.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.319+0000 7ff460c00640 1 --2- 172.21.15.123:0/1761000178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff45c074620 0x7ff45c19a630 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00e9f0 tx=0x7ff44c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:43.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.319+0000 7ff459600640 1 -- 172.21.15.123:0/1761000178 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c00cd80 con 0x7ff45c074620 2024-09-19T04:45:43.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.320+0000 7ff459600640 1 -- 172.21.15.123:0/1761000178 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff44c005500 con 0x7ff45c074620 2024-09-19T04:45:43.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.320+0000 7ff463600640 1 -- 172.21.15.123:0/1761000178 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45c19f8d0 con 0x7ff45c074620 2024-09-19T04:45:43.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.320+0000 7ff459600640 1 -- 172.21.15.123:0/1761000178 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c010690 con 0x7ff45c074620 2024-09-19T04:45:43.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.320+0000 7ff463600640 1 -- 172.21.15.123:0/1761000178 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45c1a71b0 con 0x7ff45c074620 2024-09-19T04:45:43.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.321+0000 7ff463600640 1 -- 172.21.15.123:0/1761000178 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45c074a00 con 0x7ff45c074620 2024-09-19T04:45:43.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.323+0000 7ff459600640 1 -- 172.21.15.123:0/1761000178 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff44c0046c0 con 0x7ff45c074620 2024-09-19T04:45:43.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.324+0000 7ff459600640 1 --2- 172.21.15.123:0/1761000178 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff4400a32d0 0x7ff4400a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:43.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.324+0000 7ff459600640 1 -- 172.21.15.123:0/1761000178 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7ff44c0cfed0 con 0x7ff45c074620 2024-09-19T04:45:43.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.324+0000 7ff45be00640 1 --2- 172.21.15.123:0/1761000178 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff4400a32d0 0x7ff4400a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:43.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.325+0000 7ff45be00640 1 --2- 172.21.15.123:0/1761000178 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff4400a32d0 0x7ff4400a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff45c110210 tx=0x7ff448005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:43.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.328+0000 7ff459600640 1 -- 172.21.15.123:0/1761000178 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff44c09ca40 con 0x7ff45c074620 2024-09-19T04:45:43.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.523+0000 7ff463600640 1 -- 172.21.15.123:0/1761000178 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff45c0ffbe0 con 0x7ff4400a32d0 2024-09-19T04:45:43.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.530+0000 7ff459600640 1 -- 172.21.15.123:0/1761000178 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258908 (secure 0 0 0) 0x7ff45c0ffbe0 con 0x7ff4400a32d0 2024-09-19T04:45:43.536 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:45:43.537 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-09-19T04:45:43.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.534+0000 7ff463600640 1 -- 172.21.15.123:0/1761000178 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff4400a32d0 msgr2=0x7ff4400a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:43.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.534+0000 7ff463600640 1 --2- 172.21.15.123:0/1761000178 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff4400a32d0 0x7ff4400a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff45c110210 tx=0x7ff448005f90 comp rx=0 tx=0).stop 2024-09-19T04:45:43.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.534+0000 7ff463600640 1 -- 172.21.15.123:0/1761000178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff45c074620 msgr2=0x7ff45c19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:43.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.534+0000 7ff463600640 1 --2- 172.21.15.123:0/1761000178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff45c074620 0x7ff45c19a630 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00e9f0 tx=0x7ff44c00edf0 comp rx=0 tx=0).stop 2024-09-19T04:45:43.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.534+0000 7ff463600640 1 -- 172.21.15.123:0/1761000178 shutdown_connections 2024-09-19T04:45:43.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.534+0000 7ff463600640 1 --2- 172.21.15.123:0/1761000178 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7ff4400a32d0 0x7ff4400a5790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:43.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.534+0000 7ff463600640 1 --2- 172.21.15.123:0/1761000178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff45c10e3c0 0x7ff45c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:43.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.534+0000 7ff463600640 1 --2- 172.21.15.123:0/1761000178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff45c074f40 0x7ff45c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:43.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.534+0000 7ff463600640 1 --2- 172.21.15.123:0/1761000178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff45c074620 0x7ff45c19a630 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:43.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.534+0000 7ff463600640 1 -- 172.21.15.123:0/1761000178 >> 172.21.15.123:0/1761000178 conn(0x7ff45c0fd8c0 msgr2=0x7ff45c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:43.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.535+0000 7ff463600640 1 -- 172.21.15.123:0/1761000178 shutdown_connections 2024-09-19T04:45:43.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:43.535+0000 7ff463600640 1 -- 172.21.15.123:0/1761000178 wait complete. 2024-09-19T04:45:44.325 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-19T04:45:43.367746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657181,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":749,"num_read_kb":499,"num_write":426,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356768,"kb_used_data":11616,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585056,"statfs":{"total":1151940427776,"available":1151575097344,"internally_reserved":0,"allocated":11894784,"data_stored":7428423,"data_compressed":108024,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":145,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003307"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628752+0000","last_change":"2024-09-19T04:44:30.754400+0000","last_active":"2024-09-19T04:45:14.628752+0000","last_peered":"2024-09-19T04:45:14.628752+0000","last_clean":"2024-09-19T04:45:14.628752+0000","last_became_active":"2024-09-19T04:44:30.754028+0000","last_became_peered":"2024-09-19T04:44:30.754028+0000","last_unstale":"2024-09-19T04:45:14.628752+0000","last_undegraded":"2024-09-19T04:45:14.628752+0000","last_fullsized":"2024-09-19T04:45:14.628752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:25.668478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628375+0000","last_change":"2024-09-19T04:44:30.758072+0000","last_active":"2024-09-19T04:45:14.628375+0000","last_peered":"2024-09-19T04:45:14.628375+0000","last_clean":"2024-09-19T04:45:14.628375+0000","last_became_active":"2024-09-19T04:44:30.756734+0000","last_became_peered":"2024-09-19T04:44:30.756734+0000","last_unstale":"2024-09-19T04:45:14.628375+0000","last_undegraded":"2024-09-19T04:45:14.628375+0000","last_fullsized":"2024-09-19T04:45:14.628375+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:56:47.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628259+0000","last_change":"2024-09-19T04:44:30.760208+0000","last_active":"2024-09-19T04:45:14.628259+0000","last_peered":"2024-09-19T04:45:14.628259+0000","last_clean":"2024-09-19T04:45:14.628259+0000","last_became_active":"2024-09-19T04:44:30.759719+0000","last_became_peered":"2024-09-19T04:44:30.759719+0000","last_unstale":"2024-09-19T04:45:14.628259+0000","last_undegraded":"2024-09-19T04:45:14.628259+0000","last_fullsized":"2024-09-19T04:45:14.628259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:14.041853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624883+0000","last_change":"2024-09-19T04:44:30.747813+0000","last_active":"2024-09-19T04:45:14.624883+0000","last_peered":"2024-09-19T04:45:14.624883+0000","last_clean":"2024-09-19T04:45:14.624883+0000","last_became_active":"2024-09-19T04:44:30.747486+0000","last_became_peered":"2024-09-19T04:44:30.747486+0000","last_unstale":"2024-09-19T04:45:14.624883+0000","last_undegraded":"2024-09-19T04:45:14.624883+0000","last_fullsized":"2024-09-19T04:45:14.624883+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:11.315273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626200+0000","last_change":"2024-09-19T04:44:30.755800+0000","last_active":"2024-09-19T04:45:14.626200+0000","last_peered":"2024-09-19T04:45:14.626200+0000","last_clean":"2024-09-19T04:45:14.626200+0000","last_became_active":"2024-09-19T04:44:30.755448+0000","last_became_peered":"2024-09-19T04:44:30.755448+0000","last_unstale":"2024-09-19T04:45:14.626200+0000","last_undegraded":"2024-09-19T04:45:14.626200+0000","last_fullsized":"2024-09-19T04:45:14.626200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:39.308614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626315+0000","last_change":"2024-09-19T04:44:30.746230+0000","last_active":"2024-09-19T04:45:14.626315+0000","last_peered":"2024-09-19T04:45:14.626315+0000","last_clean":"2024-09-19T04:45:14.626315+0000","last_became_active":"2024-09-19T04:44:30.745884+0000","last_became_peered":"2024-09-19T04:44:30.745884+0000","last_unstale":"2024-09-19T04:45:14.626315+0000","last_undegraded":"2024-09-19T04:45:14.626315+0000","last_fullsized":"2024-09-19T04:45:14.626315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:51:58.550322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.630633+0000","last_change":"2024-09-19T04:44:30.747870+0000","last_active":"2024-09-19T04:45:14.630633+0000","last_peered":"2024-09-19T04:45:14.630633+0000","last_clean":"2024-09-19T04:45:14.630633+0000","last_became_active":"2024-09-19T04:44:30.747536+0000","last_became_peered":"2024-09-19T04:44:30.747536+0000","last_unstale":"2024-09-19T04:45:14.630633+0000","last_undegraded":"2024-09-19T04:45:14.630633+0000","last_fullsized":"2024-09-19T04:45:14.630633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:20.074032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.625535+0000","last_change":"2024-09-19T04:44:30.750425+0000","last_active":"2024-09-19T04:45:14.625535+0000","last_peered":"2024-09-19T04:45:14.625535+0000","last_clean":"2024-09-19T04:45:14.625535+0000","last_became_active":"2024-09-19T04:44:30.749841+0000","last_became_peered":"2024-09-19T04:44:30.749841+0000","last_unstale":"2024-09-19T04:45:14.625535+0000","last_undegraded":"2024-09-19T04:45:14.625535+0000","last_fullsized":"2024-09-19T04:45:14.625535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:12:08.420727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:15.280290+0000","last_change":"2024-09-19T04:44:30.750624+0000","last_active":"2024-09-19T04:45:15.280290+0000","last_peered":"2024-09-19T04:45:15.280290+0000","last_clean":"2024-09-19T04:45:15.280290+0000","last_became_active":"2024-09-19T04:44:30.748370+0000","last_became_peered":"2024-09-19T04:44:30.748370+0000","last_unstale":"2024-09-19T04:45:15.280290+0000","last_undegraded":"2024-09-19T04:45:15.280290+0000","last_fullsized":"2024-09-19T04:45:15.280290+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:36.131634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626597+0000","last_change":"2024-09-19T04:44:30.750035+0000","last_active":"2024-09-19T04:45:14.626597+0000","last_peered":"2024-09-19T04:45:14.626597+0000","last_clean":"2024-09-19T04:45:14.626597+0000","last_became_active":"2024-09-19T04:44:30.749699+0000","last_became_peered":"2024-09-19T04:44:30.749699+0000","last_unstale":"2024-09-19T04:45:14.626597+0000","last_undegraded":"2024-09-19T04:45:14.626597+0000","last_fullsized":"2024-09-19T04:45:14.626597+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:22.637193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.631031+0000","last_change":"2024-09-19T04:44:30.760258+0000","last_active":"2024-09-19T04:45:14.631031+0000","last_peered":"2024-09-19T04:45:14.631031+0000","last_clean":"2024-09-19T04:45:14.631031+0000","last_became_active":"2024-09-19T04:44:30.759925+0000","last_became_peered":"2024-09-19T04:44:30.759925+0000","last_unstale":"2024-09-19T04:45:14.631031+0000","last_undegraded":"2024-09-19T04:45:14.631031+0000","last_fullsized":"2024-09-19T04:45:14.631031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:58.144691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.625003+0000","last_change":"2024-09-19T04:44:30.747703+0000","last_active":"2024-09-19T04:45:14.625003+0000","last_peered":"2024-09-19T04:45:14.625003+0000","last_clean":"2024-09-19T04:45:14.625003+0000","last_became_active":"2024-09-19T04:44:30.747369+0000","last_became_peered":"2024-09-19T04:44:30.747369+0000","last_unstale":"2024-09-19T04:45:14.625003+0000","last_undegraded":"2024-09-19T04:45:14.625003+0000","last_fullsized":"2024-09-19T04:45:14.625003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:06.920357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.890281+0000","last_change":"2024-09-19T04:44:30.748844+0000","last_active":"2024-09-19T04:45:14.890281+0000","last_peered":"2024-09-19T04:45:14.890281+0000","last_clean":"2024-09-19T04:45:14.890281+0000","last_became_active":"2024-09-19T04:44:30.748513+0000","last_became_peered":"2024-09-19T04:44:30.748513+0000","last_unstale":"2024-09-19T04:45:14.890281+0000","last_undegraded":"2024-09-19T04:45:14.890281+0000","last_fullsized":"2024-09-19T04:45:14.890281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:45.107718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626968+0000","last_change":"2024-09-19T04:44:30.750526+0000","last_active":"2024-09-19T04:45:14.626968+0000","last_peered":"2024-09-19T04:45:14.626968+0000","last_clean":"2024-09-19T04:45:14.626968+0000","last_became_active":"2024-09-19T04:44:30.749693+0000","last_became_peered":"2024-09-19T04:44:30.749693+0000","last_unstale":"2024-09-19T04:45:14.626968+0000","last_undegraded":"2024-09-19T04:45:14.626968+0000","last_fullsized":"2024-09-19T04:45:14.626968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:43:04.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626303+0000","last_change":"2024-09-19T04:44:30.754310+0000","last_active":"2024-09-19T04:45:14.626303+0000","last_peered":"2024-09-19T04:45:14.626303+0000","last_clean":"2024-09-19T04:45:14.626303+0000","last_became_active":"2024-09-19T04:44:30.753933+0000","last_became_peered":"2024-09-19T04:44:30.753933+0000","last_unstale":"2024-09-19T04:45:14.626303+0000","last_undegraded":"2024-09-19T04:45:14.626303+0000","last_fullsized":"2024-09-19T04:45:14.626303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:40:44.985693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:15.279784+0000","last_change":"2024-09-19T04:44:30.757458+0000","last_active":"2024-09-19T04:45:15.279784+0000","last_peered":"2024-09-19T04:45:15.279784+0000","last_clean":"2024-09-19T04:45:15.279784+0000","last_became_active":"2024-09-19T04:44:30.757127+0000","last_became_peered":"2024-09-19T04:44:30.757127+0000","last_unstale":"2024-09-19T04:45:15.279784+0000","last_undegraded":"2024-09-19T04:45:15.279784+0000","last_fullsized":"2024-09-19T04:45:15.279784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:59.325865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628783+0000","last_change":"2024-09-19T04:44:30.754685+0000","last_active":"2024-09-19T04:45:14.628783+0000","last_peered":"2024-09-19T04:45:14.628783+0000","last_clean":"2024-09-19T04:45:14.628783+0000","last_became_active":"2024-09-19T04:44:30.753625+0000","last_became_peered":"2024-09-19T04:44:30.753625+0000","last_unstale":"2024-09-19T04:45:14.628783+0000","last_undegraded":"2024-09-19T04:45:14.628783+0000","last_fullsized":"2024-09-19T04:45:14.628783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:46:01.164693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628803+0000","last_change":"2024-09-19T04:44:30.745823+0000","last_active":"2024-09-19T04:45:14.628803+0000","last_peered":"2024-09-19T04:45:14.628803+0000","last_clean":"2024-09-19T04:45:14.628803+0000","last_became_active":"2024-09-19T04:44:30.745224+0000","last_became_peered":"2024-09-19T04:44:30.745224+0000","last_unstale":"2024-09-19T04:45:14.628803+0000","last_undegraded":"2024-09-19T04:45:14.628803+0000","last_fullsized":"2024-09-19T04:45:14.628803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:23.682118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.627051+0000","last_change":"2024-09-19T04:44:30.752138+0000","last_active":"2024-09-19T04:45:14.627051+0000","last_peered":"2024-09-19T04:45:14.627051+0000","last_clean":"2024-09-19T04:45:14.627051+0000","last_became_active":"2024-09-19T04:44:30.751737+0000","last_became_peered":"2024-09-19T04:44:30.751737+0000","last_unstale":"2024-09-19T04:45:14.627051+0000","last_undegraded":"2024-09-19T04:45:14.627051+0000","last_fullsized":"2024-09-19T04:45:14.627051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:08.204601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628311+0000","last_change":"2024-09-19T04:44:30.748870+0000","last_active":"2024-09-19T04:45:14.628311+0000","last_peered":"2024-09-19T04:45:14.628311+0000","last_clean":"2024-09-19T04:45:14.628311+0000","last_became_active":"2024-09-19T04:44:30.748549+0000","last_became_peered":"2024-09-19T04:44:30.748549+0000","last_unstale":"2024-09-19T04:45:14.628311+0000","last_undegraded":"2024-09-19T04:45:14.628311+0000","last_fullsized":"2024-09-19T04:45:14.628311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:12.951564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.890313+0000","last_change":"2024-09-19T04:44:30.759937+0000","last_active":"2024-09-19T04:45:14.890313+0000","last_peered":"2024-09-19T04:45:14.890313+0000","last_clean":"2024-09-19T04:45:14.890313+0000","last_became_active":"2024-09-19T04:44:30.759659+0000","last_became_peered":"2024-09-19T04:44:30.759659+0000","last_unstale":"2024-09-19T04:45:14.890313+0000","last_undegraded":"2024-09-19T04:45:14.890313+0000","last_fullsized":"2024-09-19T04:45:14.890313+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:07.608994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:15.280226+0000","last_change":"2024-09-19T04:44:30.756216+0000","last_active":"2024-09-19T04:45:15.280226+0000","last_peered":"2024-09-19T04:45:15.280226+0000","last_clean":"2024-09-19T04:45:15.280226+0000","last_became_active":"2024-09-19T04:44:30.755903+0000","last_became_peered":"2024-09-19T04:44:30.755903+0000","last_unstale":"2024-09-19T04:45:15.280226+0000","last_undegraded":"2024-09-19T04:45:15.280226+0000","last_fullsized":"2024-09-19T04:45:15.280226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:16.966305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.625684+0000","last_change":"2024-09-19T04:44:30.746297+0000","last_active":"2024-09-19T04:45:14.625684+0000","last_peered":"2024-09-19T04:45:14.625684+0000","last_clean":"2024-09-19T04:45:14.625684+0000","last_became_active":"2024-09-19T04:44:30.746013+0000","last_became_peered":"2024-09-19T04:44:30.746013+0000","last_unstale":"2024-09-19T04:45:14.625684+0000","last_undegraded":"2024-09-19T04:45:14.625684+0000","last_fullsized":"2024-09-19T04:45:14.625684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:17.086180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.631318+0000","last_change":"2024-09-19T04:45:14.630690+0000","last_active":"2024-09-19T04:45:14.631318+0000","last_peered":"2024-09-19T04:45:14.631318+0000","last_clean":"2024-09-19T04:45:14.631318+0000","last_became_active":"2024-09-19T04:45:14.630228+0000","last_became_peered":"2024-09-19T04:45:14.630228+0000","last_unstale":"2024-09-19T04:45:14.631318+0000","last_undegraded":"2024-09-19T04:45:14.631318+0000","last_fullsized":"2024-09-19T04:45:14.631318+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:39.901088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626571+0000","last_change":"2024-09-19T04:44:30.753423+0000","last_active":"2024-09-19T04:45:14.626571+0000","last_peered":"2024-09-19T04:45:14.626571+0000","last_clean":"2024-09-19T04:45:14.626571+0000","last_became_active":"2024-09-19T04:44:30.752139+0000","last_became_peered":"2024-09-19T04:44:30.752139+0000","last_unstale":"2024-09-19T04:45:14.626571+0000","last_undegraded":"2024-09-19T04:45:14.626571+0000","last_fullsized":"2024-09-19T04:45:14.626571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:18.046893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626952+0000","last_change":"2024-09-19T04:44:30.751114+0000","last_active":"2024-09-19T04:45:14.626952+0000","last_peered":"2024-09-19T04:45:14.626952+0000","last_clean":"2024-09-19T04:45:14.626952+0000","last_became_active":"2024-09-19T04:44:30.750784+0000","last_became_peered":"2024-09-19T04:44:30.750784+0000","last_unstale":"2024-09-19T04:45:14.626952+0000","last_undegraded":"2024-09-19T04:45:14.626952+0000","last_fullsized":"2024-09-19T04:45:14.626952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:22.040253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628904+0000","last_change":"2024-09-19T04:44:30.747191+0000","last_active":"2024-09-19T04:45:14.628904+0000","last_peered":"2024-09-19T04:45:14.628904+0000","last_clean":"2024-09-19T04:45:14.628904+0000","last_became_active":"2024-09-19T04:44:30.746830+0000","last_became_peered":"2024-09-19T04:44:30.746830+0000","last_unstale":"2024-09-19T04:45:14.628904+0000","last_undegraded":"2024-09-19T04:45:14.628904+0000","last_fullsized":"2024-09-19T04:45:14.628904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:24.634920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.890044+0000","last_change":"2024-09-19T04:44:30.755174+0000","last_active":"2024-09-19T04:45:14.890044+0000","last_peered":"2024-09-19T04:45:14.890044+0000","last_clean":"2024-09-19T04:45:14.890044+0000","last_became_active":"2024-09-19T04:44:30.754885+0000","last_became_peered":"2024-09-19T04:44:30.754885+0000","last_unstale":"2024-09-19T04:45:14.890044+0000","last_undegraded":"2024-09-19T04:45:14.890044+0000","last_fullsized":"2024-09-19T04:45:14.890044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:11:32.428033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.891484+0000","last_change":"2024-09-19T04:45:14.891484+0000","last_active":"2024-09-19T04:45:14.891484+0000","last_peered":"2024-09-19T04:45:14.891484+0000","last_clean":"2024-09-19T04:45:14.891484+0000","last_became_active":"2024-09-19T04:45:14.891120+0000","last_became_peered":"2024-09-19T04:45:14.891120+0000","last_unstale":"2024-09-19T04:45:14.891484+0000","last_undegraded":"2024-09-19T04:45:14.891484+0000","last_fullsized":"2024-09-19T04:45:14.891484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:20.959571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626410+0000","last_change":"2024-09-19T04:44:30.757343+0000","last_active":"2024-09-19T04:45:14.626410+0000","last_peered":"2024-09-19T04:45:14.626410+0000","last_clean":"2024-09-19T04:45:14.626410+0000","last_became_active":"2024-09-19T04:44:30.756981+0000","last_became_peered":"2024-09-19T04:44:30.756981+0000","last_unstale":"2024-09-19T04:45:14.626410+0000","last_undegraded":"2024-09-19T04:45:14.626410+0000","last_fullsized":"2024-09-19T04:45:14.626410+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:07.388384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.631104+0000","last_change":"2024-09-19T04:45:14.631104+0000","last_active":"2024-09-19T04:45:14.631104+0000","last_peered":"2024-09-19T04:45:14.631104+0000","last_clean":"2024-09-19T04:45:14.631104+0000","last_became_active":"2024-09-19T04:45:14.630654+0000","last_became_peered":"2024-09-19T04:45:14.630654+0000","last_unstale":"2024-09-19T04:45:14.631104+0000","last_undegraded":"2024-09-19T04:45:14.631104+0000","last_fullsized":"2024-09-19T04:45:14.631104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:37.659295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628174+0000","last_change":"2024-09-19T04:44:30.748408+0000","last_active":"2024-09-19T04:45:14.628174+0000","last_peered":"2024-09-19T04:45:14.628174+0000","last_clean":"2024-09-19T04:45:14.628174+0000","last_became_active":"2024-09-19T04:44:30.747960+0000","last_became_peered":"2024-09-19T04:44:30.747960+0000","last_unstale":"2024-09-19T04:45:14.628174+0000","last_undegraded":"2024-09-19T04:45:14.628174+0000","last_fullsized":"2024-09-19T04:45:14.628174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:51.043791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626572+0000","last_change":"2024-09-19T04:44:30.747194+0000","last_active":"2024-09-19T04:45:14.626572+0000","last_peered":"2024-09-19T04:45:14.626572+0000","last_clean":"2024-09-19T04:45:14.626572+0000","last_became_active":"2024-09-19T04:44:30.746871+0000","last_became_peered":"2024-09-19T04:44:30.746871+0000","last_unstale":"2024-09-19T04:45:14.626572+0000","last_undegraded":"2024-09-19T04:45:14.626572+0000","last_fullsized":"2024-09-19T04:45:14.626572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:52.340341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":84,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:36.919999+0000","last_change":"2024-09-19T04:44:28.337746+0000","last_active":"2024-09-19T04:45:36.919999+0000","last_peered":"2024-09-19T04:45:36.919999+0000","last_clean":"2024-09-19T04:45:36.919999+0000","last_became_active":"2024-09-19T04:44:28.337410+0000","last_became_peered":"2024-09-19T04:44:28.337410+0000","last_unstale":"2024-09-19T04:45:36.919999+0000","last_undegraded":"2024-09-19T04:45:36.919999+0000","last_fullsized":"2024-09-19T04:45:36.919999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:41:27.436939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624245+0000","last_change":"2024-09-19T04:44:30.755512+0000","last_active":"2024-09-19T04:45:14.624245+0000","last_peered":"2024-09-19T04:45:14.624245+0000","last_clean":"2024-09-19T04:45:14.624245+0000","last_became_active":"2024-09-19T04:44:30.755086+0000","last_became_peered":"2024-09-19T04:44:30.755086+0000","last_unstale":"2024-09-19T04:45:14.624245+0000","last_undegraded":"2024-09-19T04:45:14.624245+0000","last_fullsized":"2024-09-19T04:45:14.624245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:33.870376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626364+0000","last_change":"2024-09-19T04:44:28.334492+0000","last_active":"2024-09-19T04:45:14.626364+0000","last_peered":"2024-09-19T04:45:14.626364+0000","last_clean":"2024-09-19T04:45:14.626364+0000","last_became_active":"2024-09-19T04:44:28.334098+0000","last_became_peered":"2024-09-19T04:44:28.334098+0000","last_unstale":"2024-09-19T04:45:14.626364+0000","last_undegraded":"2024-09-19T04:45:14.626364+0000","last_fullsized":"2024-09-19T04:45:14.626364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:48.156590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.889858+0000","last_change":"2024-09-19T04:44:28.335593+0000","last_active":"2024-09-19T04:45:14.889858+0000","last_peered":"2024-09-19T04:45:14.889858+0000","last_clean":"2024-09-19T04:45:14.889858+0000","last_became_active":"2024-09-19T04:44:28.335060+0000","last_became_peered":"2024-09-19T04:44:28.335060+0000","last_unstale":"2024-09-19T04:45:14.889858+0000","last_undegraded":"2024-09-19T04:45:14.889858+0000","last_fullsized":"2024-09-19T04:45:14.889858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:27.859960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.625727+0000","last_change":"2024-09-19T04:44:30.748235+0000","last_active":"2024-09-19T04:45:14.625727+0000","last_peered":"2024-09-19T04:45:14.625727+0000","last_clean":"2024-09-19T04:45:14.625727+0000","last_became_active":"2024-09-19T04:44:30.747881+0000","last_became_peered":"2024-09-19T04:44:30.747881+0000","last_unstale":"2024-09-19T04:45:14.625727+0000","last_undegraded":"2024-09-19T04:45:14.625727+0000","last_fullsized":"2024-09-19T04:45:14.625727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:40:39.183897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624461+0000","last_change":"2024-09-19T04:44:28.337762+0000","last_active":"2024-09-19T04:45:14.624461+0000","last_peered":"2024-09-19T04:45:14.624461+0000","last_clean":"2024-09-19T04:45:14.624461+0000","last_became_active":"2024-09-19T04:44:28.337383+0000","last_became_peered":"2024-09-19T04:44:28.337383+0000","last_unstale":"2024-09-19T04:45:14.624461+0000","last_undegraded":"2024-09-19T04:45:14.624461+0000","last_fullsized":"2024-09-19T04:45:14.624461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:07:55.268958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.627805+0000","last_change":"2024-09-19T04:44:30.754318+0000","last_active":"2024-09-19T04:45:14.627805+0000","last_peered":"2024-09-19T04:45:14.627805+0000","last_clean":"2024-09-19T04:45:14.627805+0000","last_became_active":"2024-09-19T04:44:30.754025+0000","last_became_peered":"2024-09-19T04:44:30.754025+0000","last_unstale":"2024-09-19T04:45:14.627805+0000","last_undegraded":"2024-09-19T04:45:14.627805+0000","last_fullsized":"2024-09-19T04:45:14.627805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:58.190504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":143,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:41.926127+0000","last_change":"2024-09-19T04:44:28.335947+0000","last_active":"2024-09-19T04:45:41.926127+0000","last_peered":"2024-09-19T04:45:41.926127+0000","last_clean":"2024-09-19T04:45:41.926127+0000","last_became_active":"2024-09-19T04:44:28.335588+0000","last_became_peered":"2024-09-19T04:44:28.335588+0000","last_unstale":"2024-09-19T04:45:41.926127+0000","last_undegraded":"2024-09-19T04:45:41.926127+0000","last_fullsized":"2024-09-19T04:45:41.926127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:44.024915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:15.280281+0000","last_change":"2024-09-19T04:44:30.748272+0000","last_active":"2024-09-19T04:45:15.280281+0000","last_peered":"2024-09-19T04:45:15.280281+0000","last_clean":"2024-09-19T04:45:15.280281+0000","last_became_active":"2024-09-19T04:44:30.747147+0000","last_became_peered":"2024-09-19T04:44:30.747147+0000","last_unstale":"2024-09-19T04:45:15.280281+0000","last_undegraded":"2024-09-19T04:45:15.280281+0000","last_fullsized":"2024-09-19T04:45:15.280281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:11.653693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.631249+0000","last_change":"2024-09-19T04:44:28.337120+0000","last_active":"2024-09-19T04:45:14.631249+0000","last_peered":"2024-09-19T04:45:14.631249+0000","last_clean":"2024-09-19T04:45:14.631249+0000","last_became_active":"2024-09-19T04:44:28.336053+0000","last_became_peered":"2024-09-19T04:44:28.336053+0000","last_unstale":"2024-09-19T04:45:14.631249+0000","last_undegraded":"2024-09-19T04:45:14.631249+0000","last_fullsized":"2024-09-19T04:45:14.631249+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:56:57.514931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628345+0000","last_change":"2024-09-19T04:44:30.755238+0000","last_active":"2024-09-19T04:45:14.628345+0000","last_peered":"2024-09-19T04:45:14.628345+0000","last_clean":"2024-09-19T04:45:14.628345+0000","last_became_active":"2024-09-19T04:44:30.754748+0000","last_became_peered":"2024-09-19T04:44:30.754748+0000","last_unstale":"2024-09-19T04:45:14.628345+0000","last_undegraded":"2024-09-19T04:45:14.628345+0000","last_fullsized":"2024-09-19T04:45:14.628345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:38.954491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624886+0000","last_change":"2024-09-19T04:44:28.596147+0000","last_active":"2024-09-19T04:45:14.624886+0000","last_peered":"2024-09-19T04:45:14.624886+0000","last_clean":"2024-09-19T04:45:14.624886+0000","last_became_active":"2024-09-19T04:44:28.595785+0000","last_became_peered":"2024-09-19T04:44:28.595785+0000","last_unstale":"2024-09-19T04:45:14.624886+0000","last_undegraded":"2024-09-19T04:45:14.624886+0000","last_fullsized":"2024-09-19T04:45:14.624886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:39.138724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:15.279971+0000","last_change":"2024-09-19T04:44:30.753817+0000","last_active":"2024-09-19T04:45:15.279971+0000","last_peered":"2024-09-19T04:45:15.279971+0000","last_clean":"2024-09-19T04:45:15.279971+0000","last_became_active":"2024-09-19T04:44:30.752541+0000","last_became_peered":"2024-09-19T04:44:30.752541+0000","last_unstale":"2024-09-19T04:45:15.279971+0000","last_undegraded":"2024-09-19T04:45:15.279971+0000","last_fullsized":"2024-09-19T04:45:15.279971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:16.121822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'31","reported_seq":179,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:41.926153+0000","last_change":"2024-09-19T04:44:28.335449+0000","last_active":"2024-09-19T04:45:41.926153+0000","last_peered":"2024-09-19T04:45:41.926153+0000","last_clean":"2024-09-19T04:45:41.926153+0000","last_became_active":"2024-09-19T04:44:28.335080+0000","last_became_peered":"2024-09-19T04:44:28.335080+0000","last_unstale":"2024-09-19T04:45:41.926153+0000","last_undegraded":"2024-09-19T04:45:41.926153+0000","last_fullsized":"2024-09-19T04:45:41.926153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:28.436290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":125,"num_read_kb":120,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626580+0000","last_change":"2024-09-19T04:44:30.749624+0000","last_active":"2024-09-19T04:45:14.626580+0000","last_peered":"2024-09-19T04:45:14.626580+0000","last_clean":"2024-09-19T04:45:14.626580+0000","last_became_active":"2024-09-19T04:44:30.749295+0000","last_became_peered":"2024-09-19T04:44:30.749295+0000","last_unstale":"2024-09-19T04:45:14.626580+0000","last_undegraded":"2024-09-19T04:45:14.626580+0000","last_fullsized":"2024-09-19T04:45:14.626580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:48:46.942237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628160+0000","last_change":"2024-09-19T04:44:28.337863+0000","last_active":"2024-09-19T04:45:14.628160+0000","last_peered":"2024-09-19T04:45:14.628160+0000","last_clean":"2024-09-19T04:45:14.628160+0000","last_became_active":"2024-09-19T04:44:28.337522+0000","last_became_peered":"2024-09-19T04:44:28.337522+0000","last_unstale":"2024-09-19T04:45:14.628160+0000","last_undegraded":"2024-09-19T04:45:14.628160+0000","last_fullsized":"2024-09-19T04:45:14.628160+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:02.488622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.889972+0000","last_change":"2024-09-19T04:44:30.759916+0000","last_active":"2024-09-19T04:45:14.889972+0000","last_peered":"2024-09-19T04:45:14.889972+0000","last_clean":"2024-09-19T04:45:14.889972+0000","last_became_active":"2024-09-19T04:44:30.758835+0000","last_became_peered":"2024-09-19T04:44:30.758835+0000","last_unstale":"2024-09-19T04:45:14.889972+0000","last_undegraded":"2024-09-19T04:45:14.889972+0000","last_fullsized":"2024-09-19T04:45:14.889972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:38.614343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624654+0000","last_change":"2024-09-19T04:44:28.337574+0000","last_active":"2024-09-19T04:45:14.624654+0000","last_peered":"2024-09-19T04:45:14.624654+0000","last_clean":"2024-09-19T04:45:14.624654+0000","last_became_active":"2024-09-19T04:44:28.337224+0000","last_became_peered":"2024-09-19T04:44:28.337224+0000","last_unstale":"2024-09-19T04:45:14.624654+0000","last_undegraded":"2024-09-19T04:45:14.624654+0000","last_fullsized":"2024-09-19T04:45:14.624654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:20.538181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.630949+0000","last_change":"2024-09-19T04:44:30.758152+0000","last_active":"2024-09-19T04:45:14.630949+0000","last_peered":"2024-09-19T04:45:14.630949+0000","last_clean":"2024-09-19T04:45:14.630949+0000","last_became_active":"2024-09-19T04:44:30.757801+0000","last_became_peered":"2024-09-19T04:44:30.757801+0000","last_unstale":"2024-09-19T04:45:14.630949+0000","last_undegraded":"2024-09-19T04:45:14.630949+0000","last_fullsized":"2024-09-19T04:45:14.630949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:05.212471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":90,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:40.931838+0000","last_change":"2024-09-19T04:44:28.339340+0000","last_active":"2024-09-19T04:45:40.931838+0000","last_peered":"2024-09-19T04:45:40.931838+0000","last_clean":"2024-09-19T04:45:40.931838+0000","last_became_active":"2024-09-19T04:44:28.339019+0000","last_became_peered":"2024-09-19T04:44:28.339019+0000","last_unstale":"2024-09-19T04:45:40.931838+0000","last_undegraded":"2024-09-19T04:45:40.931838+0000","last_fullsized":"2024-09-19T04:45:40.931838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:29:59.356748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":62,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.630567+0000","last_change":"2024-09-19T04:44:30.751217+0000","last_active":"2024-09-19T04:45:14.630567+0000","last_peered":"2024-09-19T04:45:14.630567+0000","last_clean":"2024-09-19T04:45:14.630567+0000","last_became_active":"2024-09-19T04:44:30.750863+0000","last_became_peered":"2024-09-19T04:44:30.750863+0000","last_unstale":"2024-09-19T04:45:14.630567+0000","last_undegraded":"2024-09-19T04:45:14.630567+0000","last_fullsized":"2024-09-19T04:45:14.630567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:09.957572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626711+0000","last_change":"2024-09-19T04:44:28.596585+0000","last_active":"2024-09-19T04:45:14.626711+0000","last_peered":"2024-09-19T04:45:14.626711+0000","last_clean":"2024-09-19T04:45:14.626711+0000","last_became_active":"2024-09-19T04:44:28.596218+0000","last_became_peered":"2024-09-19T04:44:28.596218+0000","last_unstale":"2024-09-19T04:45:14.626711+0000","last_undegraded":"2024-09-19T04:45:14.626711+0000","last_fullsized":"2024-09-19T04:45:14.626711+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:29:35.395610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624525+0000","last_change":"2024-09-19T04:44:30.746590+0000","last_active":"2024-09-19T04:45:14.624525+0000","last_peered":"2024-09-19T04:45:14.624525+0000","last_clean":"2024-09-19T04:45:14.624525+0000","last_became_active":"2024-09-19T04:44:30.746263+0000","last_became_peered":"2024-09-19T04:44:30.746263+0000","last_unstale":"2024-09-19T04:45:14.624525+0000","last_undegraded":"2024-09-19T04:45:14.624525+0000","last_fullsized":"2024-09-19T04:45:14.624525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:38.955613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626982+0000","last_change":"2024-09-19T04:44:28.335826+0000","last_active":"2024-09-19T04:45:14.626982+0000","last_peered":"2024-09-19T04:45:14.626982+0000","last_clean":"2024-09-19T04:45:14.626982+0000","last_became_active":"2024-09-19T04:44:28.335481+0000","last_became_peered":"2024-09-19T04:44:28.335481+0000","last_unstale":"2024-09-19T04:45:14.626982+0000","last_undegraded":"2024-09-19T04:45:14.626982+0000","last_fullsized":"2024-09-19T04:45:14.626982+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:20.989500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628587+0000","last_change":"2024-09-19T04:44:30.744130+0000","last_active":"2024-09-19T04:45:14.628587+0000","last_peered":"2024-09-19T04:45:14.628587+0000","last_clean":"2024-09-19T04:45:14.628587+0000","last_became_active":"2024-09-19T04:44:30.743772+0000","last_became_peered":"2024-09-19T04:44:30.743772+0000","last_unstale":"2024-09-19T04:45:14.628587+0000","last_undegraded":"2024-09-19T04:45:14.628587+0000","last_fullsized":"2024-09-19T04:45:14.628587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:34.956670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.890508+0000","last_change":"2024-09-19T04:44:28.337543+0000","last_active":"2024-09-19T04:45:14.890508+0000","last_peered":"2024-09-19T04:45:14.890508+0000","last_clean":"2024-09-19T04:45:14.890508+0000","last_became_active":"2024-09-19T04:44:28.336128+0000","last_became_peered":"2024-09-19T04:44:28.336128+0000","last_unstale":"2024-09-19T04:45:14.890508+0000","last_undegraded":"2024-09-19T04:45:14.890508+0000","last_fullsized":"2024-09-19T04:45:14.890508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:31.907777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.631436+0000","last_change":"2024-09-19T04:44:30.757489+0000","last_active":"2024-09-19T04:45:14.631436+0000","last_peered":"2024-09-19T04:45:14.631436+0000","last_clean":"2024-09-19T04:45:14.631436+0000","last_became_active":"2024-09-19T04:44:30.757041+0000","last_became_peered":"2024-09-19T04:44:30.757041+0000","last_unstale":"2024-09-19T04:45:14.631436+0000","last_undegraded":"2024-09-19T04:45:14.631436+0000","last_fullsized":"2024-09-19T04:45:14.631436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:03.395913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624980+0000","last_change":"2024-09-19T04:44:28.327393+0000","last_active":"2024-09-19T04:45:14.624980+0000","last_peered":"2024-09-19T04:45:14.624980+0000","last_clean":"2024-09-19T04:45:14.624980+0000","last_became_active":"2024-09-19T04:44:28.327116+0000","last_became_peered":"2024-09-19T04:44:28.327116+0000","last_unstale":"2024-09-19T04:45:14.624980+0000","last_undegraded":"2024-09-19T04:45:14.624980+0000","last_fullsized":"2024-09-19T04:45:14.624980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:42.360980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.630775+0000","last_change":"2024-09-19T04:44:30.754239+0000","last_active":"2024-09-19T04:45:14.630775+0000","last_peered":"2024-09-19T04:45:14.630775+0000","last_clean":"2024-09-19T04:45:14.630775+0000","last_became_active":"2024-09-19T04:44:30.753885+0000","last_became_peered":"2024-09-19T04:44:30.753885+0000","last_unstale":"2024-09-19T04:45:14.630775+0000","last_undegraded":"2024-09-19T04:45:14.630775+0000","last_fullsized":"2024-09-19T04:45:14.630775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:27.355600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'5","reported_seq":89,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:40.925377+0000","last_change":"2024-09-19T04:44:28.596214+0000","last_active":"2024-09-19T04:45:40.925377+0000","last_peered":"2024-09-19T04:45:40.925377+0000","last_clean":"2024-09-19T04:45:40.925377+0000","last_became_active":"2024-09-19T04:44:28.595841+0000","last_became_peered":"2024-09-19T04:44:28.595841+0000","last_unstale":"2024-09-19T04:45:40.925377+0000","last_undegraded":"2024-09-19T04:45:40.925377+0000","last_fullsized":"2024-09-19T04:45:40.925377+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:43.864702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":60,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.631119+0000","last_change":"2024-09-19T04:44:30.756192+0000","last_active":"2024-09-19T04:45:14.631119+0000","last_peered":"2024-09-19T04:45:14.631119+0000","last_clean":"2024-09-19T04:45:14.631119+0000","last_became_active":"2024-09-19T04:44:30.755853+0000","last_became_peered":"2024-09-19T04:44:30.755853+0000","last_unstale":"2024-09-19T04:45:14.631119+0000","last_undegraded":"2024-09-19T04:45:14.631119+0000","last_fullsized":"2024-09-19T04:45:14.631119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:25:49.206978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.630819+0000","last_change":"2024-09-19T04:43:10.156816+0000","last_active":"2024-09-19T04:45:14.630819+0000","last_peered":"2024-09-19T04:45:14.630819+0000","last_clean":"2024-09-19T04:45:14.630819+0000","last_became_active":"2024-09-19T04:43:10.156359+0000","last_became_peered":"2024-09-19T04:43:10.156359+0000","last_unstale":"2024-09-19T04:45:14.630819+0000","last_undegraded":"2024-09-19T04:45:14.630819+0000","last_fullsized":"2024-09-19T04:45:14.630819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:39:34.200793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:39:34.200793+0000","last_clean_scrub_stamp":"2024-09-19T04:39:34.200793+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:13.453818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624794+0000","last_change":"2024-09-19T04:44:28.337637+0000","last_active":"2024-09-19T04:45:14.624794+0000","last_peered":"2024-09-19T04:45:14.624794+0000","last_clean":"2024-09-19T04:45:14.624794+0000","last_became_active":"2024-09-19T04:44:28.337235+0000","last_became_peered":"2024-09-19T04:44:28.337235+0000","last_unstale":"2024-09-19T04:45:14.624794+0000","last_undegraded":"2024-09-19T04:45:14.624794+0000","last_fullsized":"2024-09-19T04:45:14.624794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:42.104972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:15.281849+0000","last_change":"2024-09-19T04:45:15.281422+0000","last_active":"2024-09-19T04:45:15.281849+0000","last_peered":"2024-09-19T04:45:15.281849+0000","last_clean":"2024-09-19T04:45:15.281849+0000","last_became_active":"2024-09-19T04:45:15.281031+0000","last_became_peered":"2024-09-19T04:45:15.281031+0000","last_unstale":"2024-09-19T04:45:15.281849+0000","last_undegraded":"2024-09-19T04:45:15.281849+0000","last_fullsized":"2024-09-19T04:45:15.281849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:05.058469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.630698+0000","last_change":"2024-09-19T04:44:28.336124+0000","last_active":"2024-09-19T04:45:14.630698+0000","last_peered":"2024-09-19T04:45:14.630698+0000","last_clean":"2024-09-19T04:45:14.630698+0000","last_became_active":"2024-09-19T04:44:28.335754+0000","last_became_peered":"2024-09-19T04:44:28.335754+0000","last_unstale":"2024-09-19T04:45:14.630698+0000","last_undegraded":"2024-09-19T04:45:14.630698+0000","last_fullsized":"2024-09-19T04:45:14.630698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:41.604367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.890383+0000","last_change":"2024-09-19T04:44:30.749939+0000","last_active":"2024-09-19T04:45:14.890383+0000","last_peered":"2024-09-19T04:45:14.890383+0000","last_clean":"2024-09-19T04:45:14.890383+0000","last_became_active":"2024-09-19T04:44:30.747122+0000","last_became_peered":"2024-09-19T04:44:30.747122+0000","last_unstale":"2024-09-19T04:45:14.890383+0000","last_undegraded":"2024-09-19T04:45:14.890383+0000","last_fullsized":"2024-09-19T04:45:14.890383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:46:08.371493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":86,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:39.929630+0000","last_change":"2024-09-19T04:44:28.594768+0000","last_active":"2024-09-19T04:45:39.929630+0000","last_peered":"2024-09-19T04:45:39.929630+0000","last_clean":"2024-09-19T04:45:39.929630+0000","last_became_active":"2024-09-19T04:44:28.594320+0000","last_became_peered":"2024-09-19T04:44:28.594320+0000","last_unstale":"2024-09-19T04:45:39.929630+0000","last_undegraded":"2024-09-19T04:45:39.929630+0000","last_fullsized":"2024-09-19T04:45:39.929630+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:32:17.736874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624241+0000","last_change":"2024-09-19T04:44:30.746666+0000","last_active":"2024-09-19T04:45:14.624241+0000","last_peered":"2024-09-19T04:45:14.624241+0000","last_clean":"2024-09-19T04:45:14.624241+0000","last_became_active":"2024-09-19T04:44:30.746312+0000","last_became_peered":"2024-09-19T04:44:30.746312+0000","last_unstale":"2024-09-19T04:45:14.624241+0000","last_undegraded":"2024-09-19T04:45:14.624241+0000","last_fullsized":"2024-09-19T04:45:14.624241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:13.562810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'5","reported_seq":88,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:40.924725+0000","last_change":"2024-09-19T04:44:28.336492+0000","last_active":"2024-09-19T04:45:40.924725+0000","last_peered":"2024-09-19T04:45:40.924725+0000","last_clean":"2024-09-19T04:45:40.924725+0000","last_became_active":"2024-09-19T04:44:28.336044+0000","last_became_peered":"2024-09-19T04:44:28.336044+0000","last_unstale":"2024-09-19T04:45:40.924725+0000","last_undegraded":"2024-09-19T04:45:40.924725+0000","last_fullsized":"2024-09-19T04:45:40.924725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:18.760473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.630959+0000","last_change":"2024-09-19T04:45:14.630959+0000","last_active":"2024-09-19T04:45:14.630959+0000","last_peered":"2024-09-19T04:45:14.630959+0000","last_clean":"2024-09-19T04:45:14.630959+0000","last_became_active":"2024-09-19T04:45:14.630528+0000","last_became_peered":"2024-09-19T04:45:14.630528+0000","last_unstale":"2024-09-19T04:45:14.630959+0000","last_undegraded":"2024-09-19T04:45:14.630959+0000","last_fullsized":"2024-09-19T04:45:14.630959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:29.754355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.627368+0000","last_change":"2024-09-19T04:44:28.596322+0000","last_active":"2024-09-19T04:45:14.627368+0000","last_peered":"2024-09-19T04:45:14.627368+0000","last_clean":"2024-09-19T04:45:14.627368+0000","last_became_active":"2024-09-19T04:44:28.595566+0000","last_became_peered":"2024-09-19T04:44:28.595566+0000","last_unstale":"2024-09-19T04:45:14.627368+0000","last_undegraded":"2024-09-19T04:45:14.627368+0000","last_fullsized":"2024-09-19T04:45:14.627368+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:23.878145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626282+0000","last_change":"2024-09-19T04:44:30.752321+0000","last_active":"2024-09-19T04:45:14.626282+0000","last_peered":"2024-09-19T04:45:14.626282+0000","last_clean":"2024-09-19T04:45:14.626282+0000","last_became_active":"2024-09-19T04:44:30.751980+0000","last_became_peered":"2024-09-19T04:44:30.751980+0000","last_unstale":"2024-09-19T04:45:14.626282+0000","last_undegraded":"2024-09-19T04:45:14.626282+0000","last_fullsized":"2024-09-19T04:45:14.626282+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:41.863034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.627183+0000","last_change":"2024-09-19T04:44:28.596547+0000","last_active":"2024-09-19T04:45:14.627183+0000","last_peered":"2024-09-19T04:45:14.627183+0000","last_clean":"2024-09-19T04:45:14.627183+0000","last_became_active":"2024-09-19T04:44:28.596181+0000","last_became_peered":"2024-09-19T04:44:28.596181+0000","last_unstale":"2024-09-19T04:45:14.627183+0000","last_undegraded":"2024-09-19T04:45:14.627183+0000","last_fullsized":"2024-09-19T04:45:14.627183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:53.251737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.890110+0000","last_change":"2024-09-19T04:44:30.751533+0000","last_active":"2024-09-19T04:45:14.890110+0000","last_peered":"2024-09-19T04:45:14.890110+0000","last_clean":"2024-09-19T04:45:14.890110+0000","last_became_active":"2024-09-19T04:44:30.748012+0000","last_became_peered":"2024-09-19T04:44:30.748012+0000","last_unstale":"2024-09-19T04:45:14.890110+0000","last_undegraded":"2024-09-19T04:45:14.890110+0000","last_fullsized":"2024-09-19T04:45:14.890110+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:32.360147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'15","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628907+0000","last_change":"2024-09-19T04:44:28.335884+0000","last_active":"2024-09-19T04:45:14.628907+0000","last_peered":"2024-09-19T04:45:14.628907+0000","last_clean":"2024-09-19T04:45:14.628907+0000","last_became_active":"2024-09-19T04:44:28.335521+0000","last_became_peered":"2024-09-19T04:44:28.335521+0000","last_unstale":"2024-09-19T04:45:14.628907+0000","last_undegraded":"2024-09-19T04:45:14.628907+0000","last_fullsized":"2024-09-19T04:45:14.628907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:32:49.764893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626658+0000","last_change":"2024-09-19T04:44:30.759578+0000","last_active":"2024-09-19T04:45:14.626658+0000","last_peered":"2024-09-19T04:45:14.626658+0000","last_clean":"2024-09-19T04:45:14.626658+0000","last_became_active":"2024-09-19T04:44:30.759245+0000","last_became_peered":"2024-09-19T04:44:30.759245+0000","last_unstale":"2024-09-19T04:45:14.626658+0000","last_undegraded":"2024-09-19T04:45:14.626658+0000","last_fullsized":"2024-09-19T04:45:14.626658+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:37.121770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.625164+0000","last_change":"2024-09-19T04:44:28.593404+0000","last_active":"2024-09-19T04:45:14.625164+0000","last_peered":"2024-09-19T04:45:14.625164+0000","last_clean":"2024-09-19T04:45:14.625164+0000","last_became_active":"2024-09-19T04:44:28.593037+0000","last_became_peered":"2024-09-19T04:44:28.593037+0000","last_unstale":"2024-09-19T04:45:14.625164+0000","last_undegraded":"2024-09-19T04:45:14.625164+0000","last_fullsized":"2024-09-19T04:45:14.625164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:08:29.218543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.891783+0000","last_change":"2024-09-19T04:45:14.891783+0000","last_active":"2024-09-19T04:45:14.891783+0000","last_peered":"2024-09-19T04:45:14.891783+0000","last_clean":"2024-09-19T04:45:14.891783+0000","last_became_active":"2024-09-19T04:45:14.891396+0000","last_became_peered":"2024-09-19T04:45:14.891396+0000","last_unstale":"2024-09-19T04:45:14.891783+0000","last_undegraded":"2024-09-19T04:45:14.891783+0000","last_fullsized":"2024-09-19T04:45:14.891783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:56.443270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:15.279950+0000","last_change":"2024-09-19T04:44:28.594541+0000","last_active":"2024-09-19T04:45:15.279950+0000","last_peered":"2024-09-19T04:45:15.279950+0000","last_clean":"2024-09-19T04:45:15.279950+0000","last_became_active":"2024-09-19T04:44:28.594262+0000","last_became_peered":"2024-09-19T04:44:28.594262+0000","last_unstale":"2024-09-19T04:45:15.279950+0000","last_undegraded":"2024-09-19T04:45:15.279950+0000","last_fullsized":"2024-09-19T04:45:15.279950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:45.685034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626342+0000","last_change":"2024-09-19T04:44:30.757861+0000","last_active":"2024-09-19T04:45:14.626342+0000","last_peered":"2024-09-19T04:45:14.626342+0000","last_clean":"2024-09-19T04:45:14.626342+0000","last_became_active":"2024-09-19T04:44:30.757509+0000","last_became_peered":"2024-09-19T04:44:30.757509+0000","last_unstale":"2024-09-19T04:45:14.626342+0000","last_undegraded":"2024-09-19T04:45:14.626342+0000","last_fullsized":"2024-09-19T04:45:14.626342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:17.737055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.625793+0000","last_change":"2024-09-19T04:44:28.335225+0000","last_active":"2024-09-19T04:45:14.625793+0000","last_peered":"2024-09-19T04:45:14.625793+0000","last_clean":"2024-09-19T04:45:14.625793+0000","last_became_active":"2024-09-19T04:44:28.334804+0000","last_became_peered":"2024-09-19T04:44:28.334804+0000","last_unstale":"2024-09-19T04:45:14.625793+0000","last_undegraded":"2024-09-19T04:45:14.625793+0000","last_fullsized":"2024-09-19T04:45:14.625793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:15.639418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628208+0000","last_change":"2024-09-19T04:44:30.754018+0000","last_active":"2024-09-19T04:45:14.628208+0000","last_peered":"2024-09-19T04:45:14.628208+0000","last_clean":"2024-09-19T04:45:14.628208+0000","last_became_active":"2024-09-19T04:44:30.753639+0000","last_became_peered":"2024-09-19T04:44:30.753639+0000","last_unstale":"2024-09-19T04:45:14.628208+0000","last_undegraded":"2024-09-19T04:45:14.628208+0000","last_fullsized":"2024-09-19T04:45:14.628208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:05.012847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.636102+0000","last_change":"2024-09-19T04:45:14.636102+0000","last_active":"2024-09-19T04:45:14.636102+0000","last_peered":"2024-09-19T04:45:14.636102+0000","last_clean":"2024-09-19T04:45:14.636102+0000","last_became_active":"2024-09-19T04:45:14.631201+0000","last_became_peered":"2024-09-19T04:45:14.631201+0000","last_unstale":"2024-09-19T04:45:14.636102+0000","last_undegraded":"2024-09-19T04:45:14.636102+0000","last_fullsized":"2024-09-19T04:45:14.636102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:04.331207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626033+0000","last_change":"2024-09-19T04:44:30.758543+0000","last_active":"2024-09-19T04:45:14.626033+0000","last_peered":"2024-09-19T04:45:14.626033+0000","last_clean":"2024-09-19T04:45:14.626033+0000","last_became_active":"2024-09-19T04:44:30.757228+0000","last_became_peered":"2024-09-19T04:44:30.757228+0000","last_unstale":"2024-09-19T04:45:14.626033+0000","last_undegraded":"2024-09-19T04:45:14.626033+0000","last_fullsized":"2024-09-19T04:45:14.626033+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:13.877742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626322+0000","last_change":"2024-09-19T04:44:28.339152+0000","last_active":"2024-09-19T04:45:14.626322+0000","last_peered":"2024-09-19T04:45:14.626322+0000","last_clean":"2024-09-19T04:45:14.626322+0000","last_became_active":"2024-09-19T04:44:28.338778+0000","last_became_peered":"2024-09-19T04:44:28.338778+0000","last_unstale":"2024-09-19T04:45:14.626322+0000","last_undegraded":"2024-09-19T04:45:14.626322+0000","last_fullsized":"2024-09-19T04:45:14.626322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:53.906755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.630965+0000","last_change":"2024-09-19T04:44:30.759330+0000","last_active":"2024-09-19T04:45:14.630965+0000","last_peered":"2024-09-19T04:45:14.630965+0000","last_clean":"2024-09-19T04:45:14.630965+0000","last_became_active":"2024-09-19T04:44:30.758987+0000","last_became_peered":"2024-09-19T04:44:30.758987+0000","last_unstale":"2024-09-19T04:45:14.630965+0000","last_undegraded":"2024-09-19T04:45:14.630965+0000","last_fullsized":"2024-09-19T04:45:14.630965+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:10.105051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624874+0000","last_change":"2024-09-19T04:44:30.748089+0000","last_active":"2024-09-19T04:45:14.624874+0000","last_peered":"2024-09-19T04:45:14.624874+0000","last_clean":"2024-09-19T04:45:14.624874+0000","last_became_active":"2024-09-19T04:44:30.747566+0000","last_became_peered":"2024-09-19T04:44:30.747566+0000","last_unstale":"2024-09-19T04:45:14.624874+0000","last_undegraded":"2024-09-19T04:45:14.624874+0000","last_fullsized":"2024-09-19T04:45:14.624874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:32.406745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.627290+0000","last_change":"2024-09-19T04:44:28.334217+0000","last_active":"2024-09-19T04:45:14.627290+0000","last_peered":"2024-09-19T04:45:14.627290+0000","last_clean":"2024-09-19T04:45:14.627290+0000","last_became_active":"2024-09-19T04:44:28.333768+0000","last_became_peered":"2024-09-19T04:44:28.333768+0000","last_unstale":"2024-09-19T04:45:14.627290+0000","last_undegraded":"2024-09-19T04:45:14.627290+0000","last_fullsized":"2024-09-19T04:45:14.627290+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:52:56.626635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:15.279903+0000","last_change":"2024-09-19T04:44:30.747498+0000","last_active":"2024-09-19T04:45:15.279903+0000","last_peered":"2024-09-19T04:45:15.279903+0000","last_clean":"2024-09-19T04:45:15.279903+0000","last_became_active":"2024-09-19T04:44:30.746822+0000","last_became_peered":"2024-09-19T04:44:30.746822+0000","last_unstale":"2024-09-19T04:45:15.279903+0000","last_undegraded":"2024-09-19T04:45:15.279903+0000","last_fullsized":"2024-09-19T04:45:15.279903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:09.460263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.645843+0000","last_change":"2024-09-19T04:45:14.645843+0000","last_active":"2024-09-19T04:45:14.645843+0000","last_peered":"2024-09-19T04:45:14.645843+0000","last_clean":"2024-09-19T04:45:14.645843+0000","last_became_active":"2024-09-19T04:45:14.632156+0000","last_became_peered":"2024-09-19T04:45:14.632156+0000","last_unstale":"2024-09-19T04:45:14.645843+0000","last_undegraded":"2024-09-19T04:45:14.645843+0000","last_fullsized":"2024-09-19T04:45:14.645843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:34.406160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628341+0000","last_change":"2024-09-19T04:44:30.762776+0000","last_active":"2024-09-19T04:45:14.628341+0000","last_peered":"2024-09-19T04:45:14.628341+0000","last_clean":"2024-09-19T04:45:14.628341+0000","last_became_active":"2024-09-19T04:44:30.761308+0000","last_became_peered":"2024-09-19T04:44:30.761308+0000","last_unstale":"2024-09-19T04:45:14.628341+0000","last_undegraded":"2024-09-19T04:45:14.628341+0000","last_fullsized":"2024-09-19T04:45:14.628341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:19.960125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626776+0000","last_change":"2024-09-19T04:44:28.594858+0000","last_active":"2024-09-19T04:45:14.626776+0000","last_peered":"2024-09-19T04:45:14.626776+0000","last_clean":"2024-09-19T04:45:14.626776+0000","last_became_active":"2024-09-19T04:44:28.594515+0000","last_became_peered":"2024-09-19T04:44:28.594515+0000","last_unstale":"2024-09-19T04:45:14.626776+0000","last_undegraded":"2024-09-19T04:45:14.626776+0000","last_fullsized":"2024-09-19T04:45:14.626776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:53.728393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626685+0000","last_change":"2024-09-19T04:44:30.753604+0000","last_active":"2024-09-19T04:45:14.626685+0000","last_peered":"2024-09-19T04:45:14.626685+0000","last_clean":"2024-09-19T04:45:14.626685+0000","last_became_active":"2024-09-19T04:44:30.751961+0000","last_became_peered":"2024-09-19T04:44:30.751961+0000","last_unstale":"2024-09-19T04:45:14.626685+0000","last_undegraded":"2024-09-19T04:45:14.626685+0000","last_fullsized":"2024-09-19T04:45:14.626685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:46:58.759650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":96,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:40.923879+0000","last_change":"2024-09-19T04:44:28.339499+0000","last_active":"2024-09-19T04:45:40.923879+0000","last_peered":"2024-09-19T04:45:40.923879+0000","last_clean":"2024-09-19T04:45:40.923879+0000","last_became_active":"2024-09-19T04:44:28.338774+0000","last_became_peered":"2024-09-19T04:44:28.338774+0000","last_unstale":"2024-09-19T04:45:40.923879+0000","last_undegraded":"2024-09-19T04:45:40.923879+0000","last_fullsized":"2024-09-19T04:45:40.923879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:12.261237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":59,"num_read_kb":59,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66579,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":613,"num_read_kb":380,"num_write":243,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":199737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247103,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481821,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":942080,"data_stored":455932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710754,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1097728,"data_stored":1107118,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874279,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":455862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":962560,"data_stored":456240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364849,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1015808,"data_stored":1048760,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528376,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":950272,"data_stored":455579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1028096,"data_stored":1048045,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953537,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":455833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":455404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":516476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1011712,"data_stored":516256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":456918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34357,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34357,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34357,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:45:44.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:45:44.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:44 smithi123 ceph-mon[30511]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:45:44.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:44 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:44.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:44 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:44.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:44 smithi123 ceph-mon[30511]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:45:44.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:44 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:45:44.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:44 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:45:44.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:44 smithi123 ceph-mon[30511]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:44.698 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:44.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:44 smithi193 ceph-mon[35592]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:45:44.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:44 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:44.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:44 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:44.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:44 smithi193 ceph-mon[35592]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:45:44.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:44 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:45:44.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:44 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:45:44.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:44 smithi193 ceph-mon[35592]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:44 smithi183 ceph-mon[35333]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:45:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:44 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:44 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:44 smithi183 ceph-mon[35333]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:45:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:44 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:45:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:44 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:45:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:44 smithi183 ceph-mon[35333]: from='mgr.14150 172.21.15.123:0/1056385417' entity='mgr.x' 2024-09-19T04:45:45.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.326+0000 7f5df2c00640 1 -- 172.21.15.123:0/717175935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dec102d50 msgr2=0x7f5dec103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:45.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.326+0000 7f5df2c00640 1 --2- 172.21.15.123:0/717175935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dec102d50 0x7f5dec103130 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009900 tx=0x7f5ddc02a480 comp rx=0 tx=0).stop 2024-09-19T04:45:45.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.326+0000 7f5df2c00640 1 -- 172.21.15.123:0/717175935 shutdown_connections 2024-09-19T04:45:45.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.326+0000 7f5df2c00640 1 --2- 172.21.15.123:0/717175935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec10c180 0x7f5dec10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:45.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.326+0000 7f5df2c00640 1 --2- 172.21.15.123:0/717175935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dec103670 0x7f5dec10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:45.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.326+0000 7f5df2c00640 1 --2- 172.21.15.123:0/717175935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dec102d50 0x7f5dec103130 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:45.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.326+0000 7f5df2c00640 1 -- 172.21.15.123:0/717175935 >> 172.21.15.123:0/717175935 conn(0x7f5dec0fc7d0 msgr2=0x7f5dec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:45.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.326+0000 7f5df2c00640 1 -- 172.21.15.123:0/717175935 shutdown_connections 2024-09-19T04:45:45.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.326+0000 7f5df2c00640 1 -- 172.21.15.123:0/717175935 wait complete. 2024-09-19T04:45:45.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.327+0000 7f5df2c00640 1 Processor -- start 2024-09-19T04:45:45.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.327+0000 7f5df2c00640 1 -- start start 2024-09-19T04:45:45.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.328+0000 7f5df2c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec102d50 0x7f5dec19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:45.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.328+0000 7f5df2c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dec103670 0x7f5dec19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:45.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.328+0000 7f5df2c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dec10c180 0x7f5dec19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:45.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.328+0000 7f5df2c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec111fd0 con 0x7f5dec103670 2024-09-19T04:45:45.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.328+0000 7f5df2c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec111e50 con 0x7f5dec102d50 2024-09-19T04:45:45.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.328+0000 7f5df2c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec112150 con 0x7f5dec10c180 2024-09-19T04:45:45.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.328+0000 7f5deb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dec103670 0x7f5dec19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:45.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.328+0000 7f5debe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec102d50 0x7f5dec19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:45.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.328+0000 7f5debe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec102d50 0x7f5dec19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:43666/0 (socket says 172.21.15.123:43666) 2024-09-19T04:45:45.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.328+0000 7f5deb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dec103670 0x7f5dec19ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46554/0 (socket says 172.21.15.123:46554) 2024-09-19T04:45:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.328+0000 7f5debe00640 1 -- 172.21.15.123:0/1841949769 learned_addr learned my addr 172.21.15.123:0/1841949769 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.328+0000 7f5df0c00640 1 --2- 172.21.15.123:0/1841949769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dec10c180 0x7f5dec19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.329+0000 7f5debe00640 1 -- 172.21.15.123:0/1841949769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dec10c180 msgr2=0x7f5dec19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.329+0000 7f5debe00640 1 --2- 172.21.15.123:0/1841949769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dec10c180 0x7f5dec19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.329+0000 7f5debe00640 1 -- 172.21.15.123:0/1841949769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dec103670 msgr2=0x7f5dec19ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.329+0000 7f5debe00640 1 --2- 172.21.15.123:0/1841949769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dec103670 0x7f5dec19ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.329+0000 7f5debe00640 1 -- 172.21.15.123:0/1841949769 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec19f670 con 0x7f5dec102d50 2024-09-19T04:45:45.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.329+0000 7f5debe00640 1 --2- 172.21.15.123:0/1841949769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec102d50 0x7f5dec19a6c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc004770 tx=0x7f5ddc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:45.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.329+0000 7f5de8c00640 1 -- 172.21.15.123:0/1841949769 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc03f070 con 0x7f5dec102d50 2024-09-19T04:45:45.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.329+0000 7f5df2c00640 1 -- 172.21.15.123:0/1841949769 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec19f900 con 0x7f5dec102d50 2024-09-19T04:45:45.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.329+0000 7f5de8c00640 1 -- 172.21.15.123:0/1841949769 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ddc031ca0 con 0x7f5dec102d50 2024-09-19T04:45:45.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.329+0000 7f5de8c00640 1 -- 172.21.15.123:0/1841949769 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc035030 con 0x7f5dec102d50 2024-09-19T04:45:45.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.329+0000 7f5df2c00640 1 -- 172.21.15.123:0/1841949769 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec1a7240 con 0x7f5dec102d50 2024-09-19T04:45:45.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.330+0000 7f5df2c00640 1 -- 172.21.15.123:0/1841949769 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dec103130 con 0x7f5dec102d50 2024-09-19T04:45:45.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.332+0000 7f5de8c00640 1 -- 172.21.15.123:0/1841949769 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5ddc042070 con 0x7f5dec102d50 2024-09-19T04:45:45.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.334+0000 7f5de8c00640 1 --2- 172.21.15.123:0/1841949769 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5dd00a32d0 0x7f5dd00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:45.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.334+0000 7f5de8c00640 1 -- 172.21.15.123:0/1841949769 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f5ddc0f08f0 con 0x7f5dec102d50 2024-09-19T04:45:45.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.334+0000 7f5deb400640 1 --2- 172.21.15.123:0/1841949769 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5dd00a32d0 0x7f5dd00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:45.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.334+0000 7f5deb400640 1 --2- 172.21.15.123:0/1841949769 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5dd00a32d0 0x7f5dd00a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0fec10 tx=0x7f5dd800b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:45.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.336+0000 7f5de8c00640 1 -- 172.21.15.123:0/1841949769 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5ddc0bd460 con 0x7f5dec102d50 2024-09-19T04:45:45.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.511+0000 7f5df2c00640 1 -- 172.21.15.123:0/1841949769 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5dec102110 con 0x7f5dd00a32d0 2024-09-19T04:45:45.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.517+0000 7f5de8c00640 1 -- 172.21.15.123:0/1841949769 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258908 (secure 0 0 0) 0x7f5dec102110 con 0x7f5dd00a32d0 2024-09-19T04:45:45.521 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:45:45.522 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-09-19T04:45:45.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.521+0000 7f5df2c00640 1 -- 172.21.15.123:0/1841949769 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5dd00a32d0 msgr2=0x7f5dd00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:45.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.521+0000 7f5df2c00640 1 --2- 172.21.15.123:0/1841949769 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5dd00a32d0 0x7f5dd00a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0fec10 tx=0x7f5dd800b2e0 comp rx=0 tx=0).stop 2024-09-19T04:45:45.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.521+0000 7f5df2c00640 1 -- 172.21.15.123:0/1841949769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec102d50 msgr2=0x7f5dec19a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:45.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.521+0000 7f5df2c00640 1 --2- 172.21.15.123:0/1841949769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec102d50 0x7f5dec19a6c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc004770 tx=0x7f5ddc003c10 comp rx=0 tx=0).stop 2024-09-19T04:45:45.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.521+0000 7f5df2c00640 1 -- 172.21.15.123:0/1841949769 shutdown_connections 2024-09-19T04:45:45.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.521+0000 7f5df2c00640 1 --2- 172.21.15.123:0/1841949769 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5dd00a32d0 0x7f5dd00a5790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:45.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.521+0000 7f5df2c00640 1 --2- 172.21.15.123:0/1841949769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dec10c180 0x7f5dec19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:45.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.521+0000 7f5df2c00640 1 --2- 172.21.15.123:0/1841949769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dec103670 0x7f5dec19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:45.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.521+0000 7f5df2c00640 1 --2- 172.21.15.123:0/1841949769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec102d50 0x7f5dec19a6c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:45.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.521+0000 7f5df2c00640 1 -- 172.21.15.123:0/1841949769 >> 172.21.15.123:0/1841949769 conn(0x7f5dec0fc7d0 msgr2=0x7f5dec10d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:45.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.522+0000 7f5df2c00640 1 -- 172.21.15.123:0/1841949769 shutdown_connections 2024-09-19T04:45:45.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:45.522+0000 7f5df2c00640 1 -- 172.21.15.123:0/1841949769 wait complete. 2024-09-19T04:45:46.237 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-19T04:45:45.368263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657181,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":754,"num_read_kb":499,"num_write":426,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356768,"kb_used_data":11616,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585056,"statfs":{"total":1151940427776,"available":1151575097344,"internally_reserved":0,"allocated":11894784,"data_stored":7428423,"data_compressed":108024,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003411"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628752+0000","last_change":"2024-09-19T04:44:30.754400+0000","last_active":"2024-09-19T04:45:14.628752+0000","last_peered":"2024-09-19T04:45:14.628752+0000","last_clean":"2024-09-19T04:45:14.628752+0000","last_became_active":"2024-09-19T04:44:30.754028+0000","last_became_peered":"2024-09-19T04:44:30.754028+0000","last_unstale":"2024-09-19T04:45:14.628752+0000","last_undegraded":"2024-09-19T04:45:14.628752+0000","last_fullsized":"2024-09-19T04:45:14.628752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:25.668478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628375+0000","last_change":"2024-09-19T04:44:30.758072+0000","last_active":"2024-09-19T04:45:14.628375+0000","last_peered":"2024-09-19T04:45:14.628375+0000","last_clean":"2024-09-19T04:45:14.628375+0000","last_became_active":"2024-09-19T04:44:30.756734+0000","last_became_peered":"2024-09-19T04:44:30.756734+0000","last_unstale":"2024-09-19T04:45:14.628375+0000","last_undegraded":"2024-09-19T04:45:14.628375+0000","last_fullsized":"2024-09-19T04:45:14.628375+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:56:47.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628259+0000","last_change":"2024-09-19T04:44:30.760208+0000","last_active":"2024-09-19T04:45:14.628259+0000","last_peered":"2024-09-19T04:45:14.628259+0000","last_clean":"2024-09-19T04:45:14.628259+0000","last_became_active":"2024-09-19T04:44:30.759719+0000","last_became_peered":"2024-09-19T04:44:30.759719+0000","last_unstale":"2024-09-19T04:45:14.628259+0000","last_undegraded":"2024-09-19T04:45:14.628259+0000","last_fullsized":"2024-09-19T04:45:14.628259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:14.041853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624883+0000","last_change":"2024-09-19T04:44:30.747813+0000","last_active":"2024-09-19T04:45:14.624883+0000","last_peered":"2024-09-19T04:45:14.624883+0000","last_clean":"2024-09-19T04:45:14.624883+0000","last_became_active":"2024-09-19T04:44:30.747486+0000","last_became_peered":"2024-09-19T04:44:30.747486+0000","last_unstale":"2024-09-19T04:45:14.624883+0000","last_undegraded":"2024-09-19T04:45:14.624883+0000","last_fullsized":"2024-09-19T04:45:14.624883+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:11.315273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626200+0000","last_change":"2024-09-19T04:44:30.755800+0000","last_active":"2024-09-19T04:45:14.626200+0000","last_peered":"2024-09-19T04:45:14.626200+0000","last_clean":"2024-09-19T04:45:14.626200+0000","last_became_active":"2024-09-19T04:44:30.755448+0000","last_became_peered":"2024-09-19T04:44:30.755448+0000","last_unstale":"2024-09-19T04:45:14.626200+0000","last_undegraded":"2024-09-19T04:45:14.626200+0000","last_fullsized":"2024-09-19T04:45:14.626200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:39.308614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626315+0000","last_change":"2024-09-19T04:44:30.746230+0000","last_active":"2024-09-19T04:45:14.626315+0000","last_peered":"2024-09-19T04:45:14.626315+0000","last_clean":"2024-09-19T04:45:14.626315+0000","last_became_active":"2024-09-19T04:44:30.745884+0000","last_became_peered":"2024-09-19T04:44:30.745884+0000","last_unstale":"2024-09-19T04:45:14.626315+0000","last_undegraded":"2024-09-19T04:45:14.626315+0000","last_fullsized":"2024-09-19T04:45:14.626315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:51:58.550322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.630633+0000","last_change":"2024-09-19T04:44:30.747870+0000","last_active":"2024-09-19T04:45:14.630633+0000","last_peered":"2024-09-19T04:45:14.630633+0000","last_clean":"2024-09-19T04:45:14.630633+0000","last_became_active":"2024-09-19T04:44:30.747536+0000","last_became_peered":"2024-09-19T04:44:30.747536+0000","last_unstale":"2024-09-19T04:45:14.630633+0000","last_undegraded":"2024-09-19T04:45:14.630633+0000","last_fullsized":"2024-09-19T04:45:14.630633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:20.074032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.625535+0000","last_change":"2024-09-19T04:44:30.750425+0000","last_active":"2024-09-19T04:45:14.625535+0000","last_peered":"2024-09-19T04:45:14.625535+0000","last_clean":"2024-09-19T04:45:14.625535+0000","last_became_active":"2024-09-19T04:44:30.749841+0000","last_became_peered":"2024-09-19T04:44:30.749841+0000","last_unstale":"2024-09-19T04:45:14.625535+0000","last_undegraded":"2024-09-19T04:45:14.625535+0000","last_fullsized":"2024-09-19T04:45:14.625535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:12:08.420727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:15.280290+0000","last_change":"2024-09-19T04:44:30.750624+0000","last_active":"2024-09-19T04:45:15.280290+0000","last_peered":"2024-09-19T04:45:15.280290+0000","last_clean":"2024-09-19T04:45:15.280290+0000","last_became_active":"2024-09-19T04:44:30.748370+0000","last_became_peered":"2024-09-19T04:44:30.748370+0000","last_unstale":"2024-09-19T04:45:15.280290+0000","last_undegraded":"2024-09-19T04:45:15.280290+0000","last_fullsized":"2024-09-19T04:45:15.280290+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:36.131634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626597+0000","last_change":"2024-09-19T04:44:30.750035+0000","last_active":"2024-09-19T04:45:14.626597+0000","last_peered":"2024-09-19T04:45:14.626597+0000","last_clean":"2024-09-19T04:45:14.626597+0000","last_became_active":"2024-09-19T04:44:30.749699+0000","last_became_peered":"2024-09-19T04:44:30.749699+0000","last_unstale":"2024-09-19T04:45:14.626597+0000","last_undegraded":"2024-09-19T04:45:14.626597+0000","last_fullsized":"2024-09-19T04:45:14.626597+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:22.637193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.631031+0000","last_change":"2024-09-19T04:44:30.760258+0000","last_active":"2024-09-19T04:45:14.631031+0000","last_peered":"2024-09-19T04:45:14.631031+0000","last_clean":"2024-09-19T04:45:14.631031+0000","last_became_active":"2024-09-19T04:44:30.759925+0000","last_became_peered":"2024-09-19T04:44:30.759925+0000","last_unstale":"2024-09-19T04:45:14.631031+0000","last_undegraded":"2024-09-19T04:45:14.631031+0000","last_fullsized":"2024-09-19T04:45:14.631031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:58.144691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.625003+0000","last_change":"2024-09-19T04:44:30.747703+0000","last_active":"2024-09-19T04:45:14.625003+0000","last_peered":"2024-09-19T04:45:14.625003+0000","last_clean":"2024-09-19T04:45:14.625003+0000","last_became_active":"2024-09-19T04:44:30.747369+0000","last_became_peered":"2024-09-19T04:44:30.747369+0000","last_unstale":"2024-09-19T04:45:14.625003+0000","last_undegraded":"2024-09-19T04:45:14.625003+0000","last_fullsized":"2024-09-19T04:45:14.625003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:06.920357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.890281+0000","last_change":"2024-09-19T04:44:30.748844+0000","last_active":"2024-09-19T04:45:14.890281+0000","last_peered":"2024-09-19T04:45:14.890281+0000","last_clean":"2024-09-19T04:45:14.890281+0000","last_became_active":"2024-09-19T04:44:30.748513+0000","last_became_peered":"2024-09-19T04:44:30.748513+0000","last_unstale":"2024-09-19T04:45:14.890281+0000","last_undegraded":"2024-09-19T04:45:14.890281+0000","last_fullsized":"2024-09-19T04:45:14.890281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:45.107718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626968+0000","last_change":"2024-09-19T04:44:30.750526+0000","last_active":"2024-09-19T04:45:14.626968+0000","last_peered":"2024-09-19T04:45:14.626968+0000","last_clean":"2024-09-19T04:45:14.626968+0000","last_became_active":"2024-09-19T04:44:30.749693+0000","last_became_peered":"2024-09-19T04:44:30.749693+0000","last_unstale":"2024-09-19T04:45:14.626968+0000","last_undegraded":"2024-09-19T04:45:14.626968+0000","last_fullsized":"2024-09-19T04:45:14.626968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:43:04.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626303+0000","last_change":"2024-09-19T04:44:30.754310+0000","last_active":"2024-09-19T04:45:14.626303+0000","last_peered":"2024-09-19T04:45:14.626303+0000","last_clean":"2024-09-19T04:45:14.626303+0000","last_became_active":"2024-09-19T04:44:30.753933+0000","last_became_peered":"2024-09-19T04:44:30.753933+0000","last_unstale":"2024-09-19T04:45:14.626303+0000","last_undegraded":"2024-09-19T04:45:14.626303+0000","last_fullsized":"2024-09-19T04:45:14.626303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:40:44.985693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:15.279784+0000","last_change":"2024-09-19T04:44:30.757458+0000","last_active":"2024-09-19T04:45:15.279784+0000","last_peered":"2024-09-19T04:45:15.279784+0000","last_clean":"2024-09-19T04:45:15.279784+0000","last_became_active":"2024-09-19T04:44:30.757127+0000","last_became_peered":"2024-09-19T04:44:30.757127+0000","last_unstale":"2024-09-19T04:45:15.279784+0000","last_undegraded":"2024-09-19T04:45:15.279784+0000","last_fullsized":"2024-09-19T04:45:15.279784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:59.325865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628783+0000","last_change":"2024-09-19T04:44:30.754685+0000","last_active":"2024-09-19T04:45:14.628783+0000","last_peered":"2024-09-19T04:45:14.628783+0000","last_clean":"2024-09-19T04:45:14.628783+0000","last_became_active":"2024-09-19T04:44:30.753625+0000","last_became_peered":"2024-09-19T04:44:30.753625+0000","last_unstale":"2024-09-19T04:45:14.628783+0000","last_undegraded":"2024-09-19T04:45:14.628783+0000","last_fullsized":"2024-09-19T04:45:14.628783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:46:01.164693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628803+0000","last_change":"2024-09-19T04:44:30.745823+0000","last_active":"2024-09-19T04:45:14.628803+0000","last_peered":"2024-09-19T04:45:14.628803+0000","last_clean":"2024-09-19T04:45:14.628803+0000","last_became_active":"2024-09-19T04:44:30.745224+0000","last_became_peered":"2024-09-19T04:44:30.745224+0000","last_unstale":"2024-09-19T04:45:14.628803+0000","last_undegraded":"2024-09-19T04:45:14.628803+0000","last_fullsized":"2024-09-19T04:45:14.628803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:23.682118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.627051+0000","last_change":"2024-09-19T04:44:30.752138+0000","last_active":"2024-09-19T04:45:14.627051+0000","last_peered":"2024-09-19T04:45:14.627051+0000","last_clean":"2024-09-19T04:45:14.627051+0000","last_became_active":"2024-09-19T04:44:30.751737+0000","last_became_peered":"2024-09-19T04:44:30.751737+0000","last_unstale":"2024-09-19T04:45:14.627051+0000","last_undegraded":"2024-09-19T04:45:14.627051+0000","last_fullsized":"2024-09-19T04:45:14.627051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:08.204601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628311+0000","last_change":"2024-09-19T04:44:30.748870+0000","last_active":"2024-09-19T04:45:14.628311+0000","last_peered":"2024-09-19T04:45:14.628311+0000","last_clean":"2024-09-19T04:45:14.628311+0000","last_became_active":"2024-09-19T04:44:30.748549+0000","last_became_peered":"2024-09-19T04:44:30.748549+0000","last_unstale":"2024-09-19T04:45:14.628311+0000","last_undegraded":"2024-09-19T04:45:14.628311+0000","last_fullsized":"2024-09-19T04:45:14.628311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:12.951564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.890313+0000","last_change":"2024-09-19T04:44:30.759937+0000","last_active":"2024-09-19T04:45:14.890313+0000","last_peered":"2024-09-19T04:45:14.890313+0000","last_clean":"2024-09-19T04:45:14.890313+0000","last_became_active":"2024-09-19T04:44:30.759659+0000","last_became_peered":"2024-09-19T04:44:30.759659+0000","last_unstale":"2024-09-19T04:45:14.890313+0000","last_undegraded":"2024-09-19T04:45:14.890313+0000","last_fullsized":"2024-09-19T04:45:14.890313+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:07.608994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:15.280226+0000","last_change":"2024-09-19T04:44:30.756216+0000","last_active":"2024-09-19T04:45:15.280226+0000","last_peered":"2024-09-19T04:45:15.280226+0000","last_clean":"2024-09-19T04:45:15.280226+0000","last_became_active":"2024-09-19T04:44:30.755903+0000","last_became_peered":"2024-09-19T04:44:30.755903+0000","last_unstale":"2024-09-19T04:45:15.280226+0000","last_undegraded":"2024-09-19T04:45:15.280226+0000","last_fullsized":"2024-09-19T04:45:15.280226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:16.966305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.625684+0000","last_change":"2024-09-19T04:44:30.746297+0000","last_active":"2024-09-19T04:45:14.625684+0000","last_peered":"2024-09-19T04:45:14.625684+0000","last_clean":"2024-09-19T04:45:14.625684+0000","last_became_active":"2024-09-19T04:44:30.746013+0000","last_became_peered":"2024-09-19T04:44:30.746013+0000","last_unstale":"2024-09-19T04:45:14.625684+0000","last_undegraded":"2024-09-19T04:45:14.625684+0000","last_fullsized":"2024-09-19T04:45:14.625684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:17.086180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.631318+0000","last_change":"2024-09-19T04:45:14.630690+0000","last_active":"2024-09-19T04:45:14.631318+0000","last_peered":"2024-09-19T04:45:14.631318+0000","last_clean":"2024-09-19T04:45:14.631318+0000","last_became_active":"2024-09-19T04:45:14.630228+0000","last_became_peered":"2024-09-19T04:45:14.630228+0000","last_unstale":"2024-09-19T04:45:14.631318+0000","last_undegraded":"2024-09-19T04:45:14.631318+0000","last_fullsized":"2024-09-19T04:45:14.631318+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:39.901088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626571+0000","last_change":"2024-09-19T04:44:30.753423+0000","last_active":"2024-09-19T04:45:14.626571+0000","last_peered":"2024-09-19T04:45:14.626571+0000","last_clean":"2024-09-19T04:45:14.626571+0000","last_became_active":"2024-09-19T04:44:30.752139+0000","last_became_peered":"2024-09-19T04:44:30.752139+0000","last_unstale":"2024-09-19T04:45:14.626571+0000","last_undegraded":"2024-09-19T04:45:14.626571+0000","last_fullsized":"2024-09-19T04:45:14.626571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:18.046893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626952+0000","last_change":"2024-09-19T04:44:30.751114+0000","last_active":"2024-09-19T04:45:14.626952+0000","last_peered":"2024-09-19T04:45:14.626952+0000","last_clean":"2024-09-19T04:45:14.626952+0000","last_became_active":"2024-09-19T04:44:30.750784+0000","last_became_peered":"2024-09-19T04:44:30.750784+0000","last_unstale":"2024-09-19T04:45:14.626952+0000","last_undegraded":"2024-09-19T04:45:14.626952+0000","last_fullsized":"2024-09-19T04:45:14.626952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:22.040253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628904+0000","last_change":"2024-09-19T04:44:30.747191+0000","last_active":"2024-09-19T04:45:14.628904+0000","last_peered":"2024-09-19T04:45:14.628904+0000","last_clean":"2024-09-19T04:45:14.628904+0000","last_became_active":"2024-09-19T04:44:30.746830+0000","last_became_peered":"2024-09-19T04:44:30.746830+0000","last_unstale":"2024-09-19T04:45:14.628904+0000","last_undegraded":"2024-09-19T04:45:14.628904+0000","last_fullsized":"2024-09-19T04:45:14.628904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:24.634920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.890044+0000","last_change":"2024-09-19T04:44:30.755174+0000","last_active":"2024-09-19T04:45:14.890044+0000","last_peered":"2024-09-19T04:45:14.890044+0000","last_clean":"2024-09-19T04:45:14.890044+0000","last_became_active":"2024-09-19T04:44:30.754885+0000","last_became_peered":"2024-09-19T04:44:30.754885+0000","last_unstale":"2024-09-19T04:45:14.890044+0000","last_undegraded":"2024-09-19T04:45:14.890044+0000","last_fullsized":"2024-09-19T04:45:14.890044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:11:32.428033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.891484+0000","last_change":"2024-09-19T04:45:14.891484+0000","last_active":"2024-09-19T04:45:14.891484+0000","last_peered":"2024-09-19T04:45:14.891484+0000","last_clean":"2024-09-19T04:45:14.891484+0000","last_became_active":"2024-09-19T04:45:14.891120+0000","last_became_peered":"2024-09-19T04:45:14.891120+0000","last_unstale":"2024-09-19T04:45:14.891484+0000","last_undegraded":"2024-09-19T04:45:14.891484+0000","last_fullsized":"2024-09-19T04:45:14.891484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:20.959571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626410+0000","last_change":"2024-09-19T04:44:30.757343+0000","last_active":"2024-09-19T04:45:14.626410+0000","last_peered":"2024-09-19T04:45:14.626410+0000","last_clean":"2024-09-19T04:45:14.626410+0000","last_became_active":"2024-09-19T04:44:30.756981+0000","last_became_peered":"2024-09-19T04:44:30.756981+0000","last_unstale":"2024-09-19T04:45:14.626410+0000","last_undegraded":"2024-09-19T04:45:14.626410+0000","last_fullsized":"2024-09-19T04:45:14.626410+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:07.388384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.631104+0000","last_change":"2024-09-19T04:45:14.631104+0000","last_active":"2024-09-19T04:45:14.631104+0000","last_peered":"2024-09-19T04:45:14.631104+0000","last_clean":"2024-09-19T04:45:14.631104+0000","last_became_active":"2024-09-19T04:45:14.630654+0000","last_became_peered":"2024-09-19T04:45:14.630654+0000","last_unstale":"2024-09-19T04:45:14.631104+0000","last_undegraded":"2024-09-19T04:45:14.631104+0000","last_fullsized":"2024-09-19T04:45:14.631104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:37.659295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628174+0000","last_change":"2024-09-19T04:44:30.748408+0000","last_active":"2024-09-19T04:45:14.628174+0000","last_peered":"2024-09-19T04:45:14.628174+0000","last_clean":"2024-09-19T04:45:14.628174+0000","last_became_active":"2024-09-19T04:44:30.747960+0000","last_became_peered":"2024-09-19T04:44:30.747960+0000","last_unstale":"2024-09-19T04:45:14.628174+0000","last_undegraded":"2024-09-19T04:45:14.628174+0000","last_fullsized":"2024-09-19T04:45:14.628174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:51.043791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626572+0000","last_change":"2024-09-19T04:44:30.747194+0000","last_active":"2024-09-19T04:45:14.626572+0000","last_peered":"2024-09-19T04:45:14.626572+0000","last_clean":"2024-09-19T04:45:14.626572+0000","last_became_active":"2024-09-19T04:44:30.746871+0000","last_became_peered":"2024-09-19T04:44:30.746871+0000","last_unstale":"2024-09-19T04:45:14.626572+0000","last_undegraded":"2024-09-19T04:45:14.626572+0000","last_fullsized":"2024-09-19T04:45:14.626572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:52.340341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":89,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:41.926907+0000","last_change":"2024-09-19T04:44:28.337746+0000","last_active":"2024-09-19T04:45:41.926907+0000","last_peered":"2024-09-19T04:45:41.926907+0000","last_clean":"2024-09-19T04:45:41.926907+0000","last_became_active":"2024-09-19T04:44:28.337410+0000","last_became_peered":"2024-09-19T04:44:28.337410+0000","last_unstale":"2024-09-19T04:45:41.926907+0000","last_undegraded":"2024-09-19T04:45:41.926907+0000","last_fullsized":"2024-09-19T04:45:41.926907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:41:27.436939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624245+0000","last_change":"2024-09-19T04:44:30.755512+0000","last_active":"2024-09-19T04:45:14.624245+0000","last_peered":"2024-09-19T04:45:14.624245+0000","last_clean":"2024-09-19T04:45:14.624245+0000","last_became_active":"2024-09-19T04:44:30.755086+0000","last_became_peered":"2024-09-19T04:44:30.755086+0000","last_unstale":"2024-09-19T04:45:14.624245+0000","last_undegraded":"2024-09-19T04:45:14.624245+0000","last_fullsized":"2024-09-19T04:45:14.624245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:33.870376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626364+0000","last_change":"2024-09-19T04:44:28.334492+0000","last_active":"2024-09-19T04:45:14.626364+0000","last_peered":"2024-09-19T04:45:14.626364+0000","last_clean":"2024-09-19T04:45:14.626364+0000","last_became_active":"2024-09-19T04:44:28.334098+0000","last_became_peered":"2024-09-19T04:44:28.334098+0000","last_unstale":"2024-09-19T04:45:14.626364+0000","last_undegraded":"2024-09-19T04:45:14.626364+0000","last_fullsized":"2024-09-19T04:45:14.626364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:48.156590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.889858+0000","last_change":"2024-09-19T04:44:28.335593+0000","last_active":"2024-09-19T04:45:14.889858+0000","last_peered":"2024-09-19T04:45:14.889858+0000","last_clean":"2024-09-19T04:45:14.889858+0000","last_became_active":"2024-09-19T04:44:28.335060+0000","last_became_peered":"2024-09-19T04:44:28.335060+0000","last_unstale":"2024-09-19T04:45:14.889858+0000","last_undegraded":"2024-09-19T04:45:14.889858+0000","last_fullsized":"2024-09-19T04:45:14.889858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:27.859960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.625727+0000","last_change":"2024-09-19T04:44:30.748235+0000","last_active":"2024-09-19T04:45:14.625727+0000","last_peered":"2024-09-19T04:45:14.625727+0000","last_clean":"2024-09-19T04:45:14.625727+0000","last_became_active":"2024-09-19T04:44:30.747881+0000","last_became_peered":"2024-09-19T04:44:30.747881+0000","last_unstale":"2024-09-19T04:45:14.625727+0000","last_undegraded":"2024-09-19T04:45:14.625727+0000","last_fullsized":"2024-09-19T04:45:14.625727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:40:39.183897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624461+0000","last_change":"2024-09-19T04:44:28.337762+0000","last_active":"2024-09-19T04:45:14.624461+0000","last_peered":"2024-09-19T04:45:14.624461+0000","last_clean":"2024-09-19T04:45:14.624461+0000","last_became_active":"2024-09-19T04:44:28.337383+0000","last_became_peered":"2024-09-19T04:44:28.337383+0000","last_unstale":"2024-09-19T04:45:14.624461+0000","last_undegraded":"2024-09-19T04:45:14.624461+0000","last_fullsized":"2024-09-19T04:45:14.624461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:07:55.268958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.627805+0000","last_change":"2024-09-19T04:44:30.754318+0000","last_active":"2024-09-19T04:45:14.627805+0000","last_peered":"2024-09-19T04:45:14.627805+0000","last_clean":"2024-09-19T04:45:14.627805+0000","last_became_active":"2024-09-19T04:44:30.754025+0000","last_became_peered":"2024-09-19T04:44:30.754025+0000","last_unstale":"2024-09-19T04:45:14.627805+0000","last_undegraded":"2024-09-19T04:45:14.627805+0000","last_fullsized":"2024-09-19T04:45:14.627805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:58.190504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":143,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:41.926127+0000","last_change":"2024-09-19T04:44:28.335947+0000","last_active":"2024-09-19T04:45:41.926127+0000","last_peered":"2024-09-19T04:45:41.926127+0000","last_clean":"2024-09-19T04:45:41.926127+0000","last_became_active":"2024-09-19T04:44:28.335588+0000","last_became_peered":"2024-09-19T04:44:28.335588+0000","last_unstale":"2024-09-19T04:45:41.926127+0000","last_undegraded":"2024-09-19T04:45:41.926127+0000","last_fullsized":"2024-09-19T04:45:41.926127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:44.024915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:15.280281+0000","last_change":"2024-09-19T04:44:30.748272+0000","last_active":"2024-09-19T04:45:15.280281+0000","last_peered":"2024-09-19T04:45:15.280281+0000","last_clean":"2024-09-19T04:45:15.280281+0000","last_became_active":"2024-09-19T04:44:30.747147+0000","last_became_peered":"2024-09-19T04:44:30.747147+0000","last_unstale":"2024-09-19T04:45:15.280281+0000","last_undegraded":"2024-09-19T04:45:15.280281+0000","last_fullsized":"2024-09-19T04:45:15.280281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:11.653693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.631249+0000","last_change":"2024-09-19T04:44:28.337120+0000","last_active":"2024-09-19T04:45:14.631249+0000","last_peered":"2024-09-19T04:45:14.631249+0000","last_clean":"2024-09-19T04:45:14.631249+0000","last_became_active":"2024-09-19T04:44:28.336053+0000","last_became_peered":"2024-09-19T04:44:28.336053+0000","last_unstale":"2024-09-19T04:45:14.631249+0000","last_undegraded":"2024-09-19T04:45:14.631249+0000","last_fullsized":"2024-09-19T04:45:14.631249+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:56:57.514931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628345+0000","last_change":"2024-09-19T04:44:30.755238+0000","last_active":"2024-09-19T04:45:14.628345+0000","last_peered":"2024-09-19T04:45:14.628345+0000","last_clean":"2024-09-19T04:45:14.628345+0000","last_became_active":"2024-09-19T04:44:30.754748+0000","last_became_peered":"2024-09-19T04:44:30.754748+0000","last_unstale":"2024-09-19T04:45:14.628345+0000","last_undegraded":"2024-09-19T04:45:14.628345+0000","last_fullsized":"2024-09-19T04:45:14.628345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:38.954491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624886+0000","last_change":"2024-09-19T04:44:28.596147+0000","last_active":"2024-09-19T04:45:14.624886+0000","last_peered":"2024-09-19T04:45:14.624886+0000","last_clean":"2024-09-19T04:45:14.624886+0000","last_became_active":"2024-09-19T04:44:28.595785+0000","last_became_peered":"2024-09-19T04:44:28.595785+0000","last_unstale":"2024-09-19T04:45:14.624886+0000","last_undegraded":"2024-09-19T04:45:14.624886+0000","last_fullsized":"2024-09-19T04:45:14.624886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:39.138724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:15.279971+0000","last_change":"2024-09-19T04:44:30.753817+0000","last_active":"2024-09-19T04:45:15.279971+0000","last_peered":"2024-09-19T04:45:15.279971+0000","last_clean":"2024-09-19T04:45:15.279971+0000","last_became_active":"2024-09-19T04:44:30.752541+0000","last_became_peered":"2024-09-19T04:44:30.752541+0000","last_unstale":"2024-09-19T04:45:15.279971+0000","last_undegraded":"2024-09-19T04:45:15.279971+0000","last_fullsized":"2024-09-19T04:45:15.279971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:16.121822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'31","reported_seq":179,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:41.926153+0000","last_change":"2024-09-19T04:44:28.335449+0000","last_active":"2024-09-19T04:45:41.926153+0000","last_peered":"2024-09-19T04:45:41.926153+0000","last_clean":"2024-09-19T04:45:41.926153+0000","last_became_active":"2024-09-19T04:44:28.335080+0000","last_became_peered":"2024-09-19T04:44:28.335080+0000","last_unstale":"2024-09-19T04:45:41.926153+0000","last_undegraded":"2024-09-19T04:45:41.926153+0000","last_fullsized":"2024-09-19T04:45:41.926153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:28.436290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":125,"num_read_kb":120,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626580+0000","last_change":"2024-09-19T04:44:30.749624+0000","last_active":"2024-09-19T04:45:14.626580+0000","last_peered":"2024-09-19T04:45:14.626580+0000","last_clean":"2024-09-19T04:45:14.626580+0000","last_became_active":"2024-09-19T04:44:30.749295+0000","last_became_peered":"2024-09-19T04:44:30.749295+0000","last_unstale":"2024-09-19T04:45:14.626580+0000","last_undegraded":"2024-09-19T04:45:14.626580+0000","last_fullsized":"2024-09-19T04:45:14.626580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:48:46.942237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628160+0000","last_change":"2024-09-19T04:44:28.337863+0000","last_active":"2024-09-19T04:45:14.628160+0000","last_peered":"2024-09-19T04:45:14.628160+0000","last_clean":"2024-09-19T04:45:14.628160+0000","last_became_active":"2024-09-19T04:44:28.337522+0000","last_became_peered":"2024-09-19T04:44:28.337522+0000","last_unstale":"2024-09-19T04:45:14.628160+0000","last_undegraded":"2024-09-19T04:45:14.628160+0000","last_fullsized":"2024-09-19T04:45:14.628160+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:02.488622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.889972+0000","last_change":"2024-09-19T04:44:30.759916+0000","last_active":"2024-09-19T04:45:14.889972+0000","last_peered":"2024-09-19T04:45:14.889972+0000","last_clean":"2024-09-19T04:45:14.889972+0000","last_became_active":"2024-09-19T04:44:30.758835+0000","last_became_peered":"2024-09-19T04:44:30.758835+0000","last_unstale":"2024-09-19T04:45:14.889972+0000","last_undegraded":"2024-09-19T04:45:14.889972+0000","last_fullsized":"2024-09-19T04:45:14.889972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:38.614343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624654+0000","last_change":"2024-09-19T04:44:28.337574+0000","last_active":"2024-09-19T04:45:14.624654+0000","last_peered":"2024-09-19T04:45:14.624654+0000","last_clean":"2024-09-19T04:45:14.624654+0000","last_became_active":"2024-09-19T04:44:28.337224+0000","last_became_peered":"2024-09-19T04:44:28.337224+0000","last_unstale":"2024-09-19T04:45:14.624654+0000","last_undegraded":"2024-09-19T04:45:14.624654+0000","last_fullsized":"2024-09-19T04:45:14.624654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:20.538181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.630949+0000","last_change":"2024-09-19T04:44:30.758152+0000","last_active":"2024-09-19T04:45:14.630949+0000","last_peered":"2024-09-19T04:45:14.630949+0000","last_clean":"2024-09-19T04:45:14.630949+0000","last_became_active":"2024-09-19T04:44:30.757801+0000","last_became_peered":"2024-09-19T04:44:30.757801+0000","last_unstale":"2024-09-19T04:45:14.630949+0000","last_undegraded":"2024-09-19T04:45:14.630949+0000","last_fullsized":"2024-09-19T04:45:14.630949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:05.212471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":90,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:40.931838+0000","last_change":"2024-09-19T04:44:28.339340+0000","last_active":"2024-09-19T04:45:40.931838+0000","last_peered":"2024-09-19T04:45:40.931838+0000","last_clean":"2024-09-19T04:45:40.931838+0000","last_became_active":"2024-09-19T04:44:28.339019+0000","last_became_peered":"2024-09-19T04:44:28.339019+0000","last_unstale":"2024-09-19T04:45:40.931838+0000","last_undegraded":"2024-09-19T04:45:40.931838+0000","last_fullsized":"2024-09-19T04:45:40.931838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:29:59.356748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":62,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.630567+0000","last_change":"2024-09-19T04:44:30.751217+0000","last_active":"2024-09-19T04:45:14.630567+0000","last_peered":"2024-09-19T04:45:14.630567+0000","last_clean":"2024-09-19T04:45:14.630567+0000","last_became_active":"2024-09-19T04:44:30.750863+0000","last_became_peered":"2024-09-19T04:44:30.750863+0000","last_unstale":"2024-09-19T04:45:14.630567+0000","last_undegraded":"2024-09-19T04:45:14.630567+0000","last_fullsized":"2024-09-19T04:45:14.630567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:09.957572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626711+0000","last_change":"2024-09-19T04:44:28.596585+0000","last_active":"2024-09-19T04:45:14.626711+0000","last_peered":"2024-09-19T04:45:14.626711+0000","last_clean":"2024-09-19T04:45:14.626711+0000","last_became_active":"2024-09-19T04:44:28.596218+0000","last_became_peered":"2024-09-19T04:44:28.596218+0000","last_unstale":"2024-09-19T04:45:14.626711+0000","last_undegraded":"2024-09-19T04:45:14.626711+0000","last_fullsized":"2024-09-19T04:45:14.626711+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:29:35.395610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624525+0000","last_change":"2024-09-19T04:44:30.746590+0000","last_active":"2024-09-19T04:45:14.624525+0000","last_peered":"2024-09-19T04:45:14.624525+0000","last_clean":"2024-09-19T04:45:14.624525+0000","last_became_active":"2024-09-19T04:44:30.746263+0000","last_became_peered":"2024-09-19T04:44:30.746263+0000","last_unstale":"2024-09-19T04:45:14.624525+0000","last_undegraded":"2024-09-19T04:45:14.624525+0000","last_fullsized":"2024-09-19T04:45:14.624525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:38.955613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626982+0000","last_change":"2024-09-19T04:44:28.335826+0000","last_active":"2024-09-19T04:45:14.626982+0000","last_peered":"2024-09-19T04:45:14.626982+0000","last_clean":"2024-09-19T04:45:14.626982+0000","last_became_active":"2024-09-19T04:44:28.335481+0000","last_became_peered":"2024-09-19T04:44:28.335481+0000","last_unstale":"2024-09-19T04:45:14.626982+0000","last_undegraded":"2024-09-19T04:45:14.626982+0000","last_fullsized":"2024-09-19T04:45:14.626982+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:20.989500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628587+0000","last_change":"2024-09-19T04:44:30.744130+0000","last_active":"2024-09-19T04:45:14.628587+0000","last_peered":"2024-09-19T04:45:14.628587+0000","last_clean":"2024-09-19T04:45:14.628587+0000","last_became_active":"2024-09-19T04:44:30.743772+0000","last_became_peered":"2024-09-19T04:44:30.743772+0000","last_unstale":"2024-09-19T04:45:14.628587+0000","last_undegraded":"2024-09-19T04:45:14.628587+0000","last_fullsized":"2024-09-19T04:45:14.628587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:34.956670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.890508+0000","last_change":"2024-09-19T04:44:28.337543+0000","last_active":"2024-09-19T04:45:14.890508+0000","last_peered":"2024-09-19T04:45:14.890508+0000","last_clean":"2024-09-19T04:45:14.890508+0000","last_became_active":"2024-09-19T04:44:28.336128+0000","last_became_peered":"2024-09-19T04:44:28.336128+0000","last_unstale":"2024-09-19T04:45:14.890508+0000","last_undegraded":"2024-09-19T04:45:14.890508+0000","last_fullsized":"2024-09-19T04:45:14.890508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:31.907777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.631436+0000","last_change":"2024-09-19T04:44:30.757489+0000","last_active":"2024-09-19T04:45:14.631436+0000","last_peered":"2024-09-19T04:45:14.631436+0000","last_clean":"2024-09-19T04:45:14.631436+0000","last_became_active":"2024-09-19T04:44:30.757041+0000","last_became_peered":"2024-09-19T04:44:30.757041+0000","last_unstale":"2024-09-19T04:45:14.631436+0000","last_undegraded":"2024-09-19T04:45:14.631436+0000","last_fullsized":"2024-09-19T04:45:14.631436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:03.395913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624980+0000","last_change":"2024-09-19T04:44:28.327393+0000","last_active":"2024-09-19T04:45:14.624980+0000","last_peered":"2024-09-19T04:45:14.624980+0000","last_clean":"2024-09-19T04:45:14.624980+0000","last_became_active":"2024-09-19T04:44:28.327116+0000","last_became_peered":"2024-09-19T04:44:28.327116+0000","last_unstale":"2024-09-19T04:45:14.624980+0000","last_undegraded":"2024-09-19T04:45:14.624980+0000","last_fullsized":"2024-09-19T04:45:14.624980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:42.360980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.630775+0000","last_change":"2024-09-19T04:44:30.754239+0000","last_active":"2024-09-19T04:45:14.630775+0000","last_peered":"2024-09-19T04:45:14.630775+0000","last_clean":"2024-09-19T04:45:14.630775+0000","last_became_active":"2024-09-19T04:44:30.753885+0000","last_became_peered":"2024-09-19T04:44:30.753885+0000","last_unstale":"2024-09-19T04:45:14.630775+0000","last_undegraded":"2024-09-19T04:45:14.630775+0000","last_fullsized":"2024-09-19T04:45:14.630775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:27.355600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'5","reported_seq":89,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:40.925377+0000","last_change":"2024-09-19T04:44:28.596214+0000","last_active":"2024-09-19T04:45:40.925377+0000","last_peered":"2024-09-19T04:45:40.925377+0000","last_clean":"2024-09-19T04:45:40.925377+0000","last_became_active":"2024-09-19T04:44:28.595841+0000","last_became_peered":"2024-09-19T04:44:28.595841+0000","last_unstale":"2024-09-19T04:45:40.925377+0000","last_undegraded":"2024-09-19T04:45:40.925377+0000","last_fullsized":"2024-09-19T04:45:40.925377+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:43.864702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":60,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.631119+0000","last_change":"2024-09-19T04:44:30.756192+0000","last_active":"2024-09-19T04:45:14.631119+0000","last_peered":"2024-09-19T04:45:14.631119+0000","last_clean":"2024-09-19T04:45:14.631119+0000","last_became_active":"2024-09-19T04:44:30.755853+0000","last_became_peered":"2024-09-19T04:44:30.755853+0000","last_unstale":"2024-09-19T04:45:14.631119+0000","last_undegraded":"2024-09-19T04:45:14.631119+0000","last_fullsized":"2024-09-19T04:45:14.631119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:25:49.206978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.630819+0000","last_change":"2024-09-19T04:43:10.156816+0000","last_active":"2024-09-19T04:45:14.630819+0000","last_peered":"2024-09-19T04:45:14.630819+0000","last_clean":"2024-09-19T04:45:14.630819+0000","last_became_active":"2024-09-19T04:43:10.156359+0000","last_became_peered":"2024-09-19T04:43:10.156359+0000","last_unstale":"2024-09-19T04:45:14.630819+0000","last_undegraded":"2024-09-19T04:45:14.630819+0000","last_fullsized":"2024-09-19T04:45:14.630819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:39:34.200793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:39:34.200793+0000","last_clean_scrub_stamp":"2024-09-19T04:39:34.200793+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:13.453818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624794+0000","last_change":"2024-09-19T04:44:28.337637+0000","last_active":"2024-09-19T04:45:14.624794+0000","last_peered":"2024-09-19T04:45:14.624794+0000","last_clean":"2024-09-19T04:45:14.624794+0000","last_became_active":"2024-09-19T04:44:28.337235+0000","last_became_peered":"2024-09-19T04:44:28.337235+0000","last_unstale":"2024-09-19T04:45:14.624794+0000","last_undegraded":"2024-09-19T04:45:14.624794+0000","last_fullsized":"2024-09-19T04:45:14.624794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:42.104972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:15.281849+0000","last_change":"2024-09-19T04:45:15.281422+0000","last_active":"2024-09-19T04:45:15.281849+0000","last_peered":"2024-09-19T04:45:15.281849+0000","last_clean":"2024-09-19T04:45:15.281849+0000","last_became_active":"2024-09-19T04:45:15.281031+0000","last_became_peered":"2024-09-19T04:45:15.281031+0000","last_unstale":"2024-09-19T04:45:15.281849+0000","last_undegraded":"2024-09-19T04:45:15.281849+0000","last_fullsized":"2024-09-19T04:45:15.281849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:05.058469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.630698+0000","last_change":"2024-09-19T04:44:28.336124+0000","last_active":"2024-09-19T04:45:14.630698+0000","last_peered":"2024-09-19T04:45:14.630698+0000","last_clean":"2024-09-19T04:45:14.630698+0000","last_became_active":"2024-09-19T04:44:28.335754+0000","last_became_peered":"2024-09-19T04:44:28.335754+0000","last_unstale":"2024-09-19T04:45:14.630698+0000","last_undegraded":"2024-09-19T04:45:14.630698+0000","last_fullsized":"2024-09-19T04:45:14.630698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:41.604367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.890383+0000","last_change":"2024-09-19T04:44:30.749939+0000","last_active":"2024-09-19T04:45:14.890383+0000","last_peered":"2024-09-19T04:45:14.890383+0000","last_clean":"2024-09-19T04:45:14.890383+0000","last_became_active":"2024-09-19T04:44:30.747122+0000","last_became_peered":"2024-09-19T04:44:30.747122+0000","last_unstale":"2024-09-19T04:45:14.890383+0000","last_undegraded":"2024-09-19T04:45:14.890383+0000","last_fullsized":"2024-09-19T04:45:14.890383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:46:08.371493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":86,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:39.929630+0000","last_change":"2024-09-19T04:44:28.594768+0000","last_active":"2024-09-19T04:45:39.929630+0000","last_peered":"2024-09-19T04:45:39.929630+0000","last_clean":"2024-09-19T04:45:39.929630+0000","last_became_active":"2024-09-19T04:44:28.594320+0000","last_became_peered":"2024-09-19T04:44:28.594320+0000","last_unstale":"2024-09-19T04:45:39.929630+0000","last_undegraded":"2024-09-19T04:45:39.929630+0000","last_fullsized":"2024-09-19T04:45:39.929630+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:32:17.736874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624241+0000","last_change":"2024-09-19T04:44:30.746666+0000","last_active":"2024-09-19T04:45:14.624241+0000","last_peered":"2024-09-19T04:45:14.624241+0000","last_clean":"2024-09-19T04:45:14.624241+0000","last_became_active":"2024-09-19T04:44:30.746312+0000","last_became_peered":"2024-09-19T04:44:30.746312+0000","last_unstale":"2024-09-19T04:45:14.624241+0000","last_undegraded":"2024-09-19T04:45:14.624241+0000","last_fullsized":"2024-09-19T04:45:14.624241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:13.562810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'5","reported_seq":88,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:40.924725+0000","last_change":"2024-09-19T04:44:28.336492+0000","last_active":"2024-09-19T04:45:40.924725+0000","last_peered":"2024-09-19T04:45:40.924725+0000","last_clean":"2024-09-19T04:45:40.924725+0000","last_became_active":"2024-09-19T04:44:28.336044+0000","last_became_peered":"2024-09-19T04:44:28.336044+0000","last_unstale":"2024-09-19T04:45:40.924725+0000","last_undegraded":"2024-09-19T04:45:40.924725+0000","last_fullsized":"2024-09-19T04:45:40.924725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:18.760473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.630959+0000","last_change":"2024-09-19T04:45:14.630959+0000","last_active":"2024-09-19T04:45:14.630959+0000","last_peered":"2024-09-19T04:45:14.630959+0000","last_clean":"2024-09-19T04:45:14.630959+0000","last_became_active":"2024-09-19T04:45:14.630528+0000","last_became_peered":"2024-09-19T04:45:14.630528+0000","last_unstale":"2024-09-19T04:45:14.630959+0000","last_undegraded":"2024-09-19T04:45:14.630959+0000","last_fullsized":"2024-09-19T04:45:14.630959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:29.754355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.627368+0000","last_change":"2024-09-19T04:44:28.596322+0000","last_active":"2024-09-19T04:45:14.627368+0000","last_peered":"2024-09-19T04:45:14.627368+0000","last_clean":"2024-09-19T04:45:14.627368+0000","last_became_active":"2024-09-19T04:44:28.595566+0000","last_became_peered":"2024-09-19T04:44:28.595566+0000","last_unstale":"2024-09-19T04:45:14.627368+0000","last_undegraded":"2024-09-19T04:45:14.627368+0000","last_fullsized":"2024-09-19T04:45:14.627368+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:23.878145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626282+0000","last_change":"2024-09-19T04:44:30.752321+0000","last_active":"2024-09-19T04:45:14.626282+0000","last_peered":"2024-09-19T04:45:14.626282+0000","last_clean":"2024-09-19T04:45:14.626282+0000","last_became_active":"2024-09-19T04:44:30.751980+0000","last_became_peered":"2024-09-19T04:44:30.751980+0000","last_unstale":"2024-09-19T04:45:14.626282+0000","last_undegraded":"2024-09-19T04:45:14.626282+0000","last_fullsized":"2024-09-19T04:45:14.626282+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:41.863034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.627183+0000","last_change":"2024-09-19T04:44:28.596547+0000","last_active":"2024-09-19T04:45:14.627183+0000","last_peered":"2024-09-19T04:45:14.627183+0000","last_clean":"2024-09-19T04:45:14.627183+0000","last_became_active":"2024-09-19T04:44:28.596181+0000","last_became_peered":"2024-09-19T04:44:28.596181+0000","last_unstale":"2024-09-19T04:45:14.627183+0000","last_undegraded":"2024-09-19T04:45:14.627183+0000","last_fullsized":"2024-09-19T04:45:14.627183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:53.251737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.890110+0000","last_change":"2024-09-19T04:44:30.751533+0000","last_active":"2024-09-19T04:45:14.890110+0000","last_peered":"2024-09-19T04:45:14.890110+0000","last_clean":"2024-09-19T04:45:14.890110+0000","last_became_active":"2024-09-19T04:44:30.748012+0000","last_became_peered":"2024-09-19T04:44:30.748012+0000","last_unstale":"2024-09-19T04:45:14.890110+0000","last_undegraded":"2024-09-19T04:45:14.890110+0000","last_fullsized":"2024-09-19T04:45:14.890110+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:32.360147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'15","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628907+0000","last_change":"2024-09-19T04:44:28.335884+0000","last_active":"2024-09-19T04:45:14.628907+0000","last_peered":"2024-09-19T04:45:14.628907+0000","last_clean":"2024-09-19T04:45:14.628907+0000","last_became_active":"2024-09-19T04:44:28.335521+0000","last_became_peered":"2024-09-19T04:44:28.335521+0000","last_unstale":"2024-09-19T04:45:14.628907+0000","last_undegraded":"2024-09-19T04:45:14.628907+0000","last_fullsized":"2024-09-19T04:45:14.628907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:32:49.764893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626658+0000","last_change":"2024-09-19T04:44:30.759578+0000","last_active":"2024-09-19T04:45:14.626658+0000","last_peered":"2024-09-19T04:45:14.626658+0000","last_clean":"2024-09-19T04:45:14.626658+0000","last_became_active":"2024-09-19T04:44:30.759245+0000","last_became_peered":"2024-09-19T04:44:30.759245+0000","last_unstale":"2024-09-19T04:45:14.626658+0000","last_undegraded":"2024-09-19T04:45:14.626658+0000","last_fullsized":"2024-09-19T04:45:14.626658+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:37.121770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.625164+0000","last_change":"2024-09-19T04:44:28.593404+0000","last_active":"2024-09-19T04:45:14.625164+0000","last_peered":"2024-09-19T04:45:14.625164+0000","last_clean":"2024-09-19T04:45:14.625164+0000","last_became_active":"2024-09-19T04:44:28.593037+0000","last_became_peered":"2024-09-19T04:44:28.593037+0000","last_unstale":"2024-09-19T04:45:14.625164+0000","last_undegraded":"2024-09-19T04:45:14.625164+0000","last_fullsized":"2024-09-19T04:45:14.625164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:08:29.218543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.891783+0000","last_change":"2024-09-19T04:45:14.891783+0000","last_active":"2024-09-19T04:45:14.891783+0000","last_peered":"2024-09-19T04:45:14.891783+0000","last_clean":"2024-09-19T04:45:14.891783+0000","last_became_active":"2024-09-19T04:45:14.891396+0000","last_became_peered":"2024-09-19T04:45:14.891396+0000","last_unstale":"2024-09-19T04:45:14.891783+0000","last_undegraded":"2024-09-19T04:45:14.891783+0000","last_fullsized":"2024-09-19T04:45:14.891783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:56.443270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:15.279950+0000","last_change":"2024-09-19T04:44:28.594541+0000","last_active":"2024-09-19T04:45:15.279950+0000","last_peered":"2024-09-19T04:45:15.279950+0000","last_clean":"2024-09-19T04:45:15.279950+0000","last_became_active":"2024-09-19T04:44:28.594262+0000","last_became_peered":"2024-09-19T04:44:28.594262+0000","last_unstale":"2024-09-19T04:45:15.279950+0000","last_undegraded":"2024-09-19T04:45:15.279950+0000","last_fullsized":"2024-09-19T04:45:15.279950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:45.685034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626342+0000","last_change":"2024-09-19T04:44:30.757861+0000","last_active":"2024-09-19T04:45:14.626342+0000","last_peered":"2024-09-19T04:45:14.626342+0000","last_clean":"2024-09-19T04:45:14.626342+0000","last_became_active":"2024-09-19T04:44:30.757509+0000","last_became_peered":"2024-09-19T04:44:30.757509+0000","last_unstale":"2024-09-19T04:45:14.626342+0000","last_undegraded":"2024-09-19T04:45:14.626342+0000","last_fullsized":"2024-09-19T04:45:14.626342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:17.737055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.625793+0000","last_change":"2024-09-19T04:44:28.335225+0000","last_active":"2024-09-19T04:45:14.625793+0000","last_peered":"2024-09-19T04:45:14.625793+0000","last_clean":"2024-09-19T04:45:14.625793+0000","last_became_active":"2024-09-19T04:44:28.334804+0000","last_became_peered":"2024-09-19T04:44:28.334804+0000","last_unstale":"2024-09-19T04:45:14.625793+0000","last_undegraded":"2024-09-19T04:45:14.625793+0000","last_fullsized":"2024-09-19T04:45:14.625793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:15.639418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628208+0000","last_change":"2024-09-19T04:44:30.754018+0000","last_active":"2024-09-19T04:45:14.628208+0000","last_peered":"2024-09-19T04:45:14.628208+0000","last_clean":"2024-09-19T04:45:14.628208+0000","last_became_active":"2024-09-19T04:44:30.753639+0000","last_became_peered":"2024-09-19T04:44:30.753639+0000","last_unstale":"2024-09-19T04:45:14.628208+0000","last_undegraded":"2024-09-19T04:45:14.628208+0000","last_fullsized":"2024-09-19T04:45:14.628208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:05.012847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.636102+0000","last_change":"2024-09-19T04:45:14.636102+0000","last_active":"2024-09-19T04:45:14.636102+0000","last_peered":"2024-09-19T04:45:14.636102+0000","last_clean":"2024-09-19T04:45:14.636102+0000","last_became_active":"2024-09-19T04:45:14.631201+0000","last_became_peered":"2024-09-19T04:45:14.631201+0000","last_unstale":"2024-09-19T04:45:14.636102+0000","last_undegraded":"2024-09-19T04:45:14.636102+0000","last_fullsized":"2024-09-19T04:45:14.636102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:04.331207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626033+0000","last_change":"2024-09-19T04:44:30.758543+0000","last_active":"2024-09-19T04:45:14.626033+0000","last_peered":"2024-09-19T04:45:14.626033+0000","last_clean":"2024-09-19T04:45:14.626033+0000","last_became_active":"2024-09-19T04:44:30.757228+0000","last_became_peered":"2024-09-19T04:44:30.757228+0000","last_unstale":"2024-09-19T04:45:14.626033+0000","last_undegraded":"2024-09-19T04:45:14.626033+0000","last_fullsized":"2024-09-19T04:45:14.626033+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:13.877742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626322+0000","last_change":"2024-09-19T04:44:28.339152+0000","last_active":"2024-09-19T04:45:14.626322+0000","last_peered":"2024-09-19T04:45:14.626322+0000","last_clean":"2024-09-19T04:45:14.626322+0000","last_became_active":"2024-09-19T04:44:28.338778+0000","last_became_peered":"2024-09-19T04:44:28.338778+0000","last_unstale":"2024-09-19T04:45:14.626322+0000","last_undegraded":"2024-09-19T04:45:14.626322+0000","last_fullsized":"2024-09-19T04:45:14.626322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:53.906755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.630965+0000","last_change":"2024-09-19T04:44:30.759330+0000","last_active":"2024-09-19T04:45:14.630965+0000","last_peered":"2024-09-19T04:45:14.630965+0000","last_clean":"2024-09-19T04:45:14.630965+0000","last_became_active":"2024-09-19T04:44:30.758987+0000","last_became_peered":"2024-09-19T04:44:30.758987+0000","last_unstale":"2024-09-19T04:45:14.630965+0000","last_undegraded":"2024-09-19T04:45:14.630965+0000","last_fullsized":"2024-09-19T04:45:14.630965+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:10.105051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.624874+0000","last_change":"2024-09-19T04:44:30.748089+0000","last_active":"2024-09-19T04:45:14.624874+0000","last_peered":"2024-09-19T04:45:14.624874+0000","last_clean":"2024-09-19T04:45:14.624874+0000","last_became_active":"2024-09-19T04:44:30.747566+0000","last_became_peered":"2024-09-19T04:44:30.747566+0000","last_unstale":"2024-09-19T04:45:14.624874+0000","last_undegraded":"2024-09-19T04:45:14.624874+0000","last_fullsized":"2024-09-19T04:45:14.624874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:32.406745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.627290+0000","last_change":"2024-09-19T04:44:28.334217+0000","last_active":"2024-09-19T04:45:14.627290+0000","last_peered":"2024-09-19T04:45:14.627290+0000","last_clean":"2024-09-19T04:45:14.627290+0000","last_became_active":"2024-09-19T04:44:28.333768+0000","last_became_peered":"2024-09-19T04:44:28.333768+0000","last_unstale":"2024-09-19T04:45:14.627290+0000","last_undegraded":"2024-09-19T04:45:14.627290+0000","last_fullsized":"2024-09-19T04:45:14.627290+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:52:56.626635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:15.279903+0000","last_change":"2024-09-19T04:44:30.747498+0000","last_active":"2024-09-19T04:45:15.279903+0000","last_peered":"2024-09-19T04:45:15.279903+0000","last_clean":"2024-09-19T04:45:15.279903+0000","last_became_active":"2024-09-19T04:44:30.746822+0000","last_became_peered":"2024-09-19T04:44:30.746822+0000","last_unstale":"2024-09-19T04:45:15.279903+0000","last_undegraded":"2024-09-19T04:45:15.279903+0000","last_fullsized":"2024-09-19T04:45:15.279903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:09.460263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.645843+0000","last_change":"2024-09-19T04:45:14.645843+0000","last_active":"2024-09-19T04:45:14.645843+0000","last_peered":"2024-09-19T04:45:14.645843+0000","last_clean":"2024-09-19T04:45:14.645843+0000","last_became_active":"2024-09-19T04:45:14.632156+0000","last_became_peered":"2024-09-19T04:45:14.632156+0000","last_unstale":"2024-09-19T04:45:14.645843+0000","last_undegraded":"2024-09-19T04:45:14.645843+0000","last_fullsized":"2024-09-19T04:45:14.645843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:34.406160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.628341+0000","last_change":"2024-09-19T04:44:30.762776+0000","last_active":"2024-09-19T04:45:14.628341+0000","last_peered":"2024-09-19T04:45:14.628341+0000","last_clean":"2024-09-19T04:45:14.628341+0000","last_became_active":"2024-09-19T04:44:30.761308+0000","last_became_peered":"2024-09-19T04:44:30.761308+0000","last_unstale":"2024-09-19T04:45:14.628341+0000","last_undegraded":"2024-09-19T04:45:14.628341+0000","last_fullsized":"2024-09-19T04:45:14.628341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:19.960125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626776+0000","last_change":"2024-09-19T04:44:28.594858+0000","last_active":"2024-09-19T04:45:14.626776+0000","last_peered":"2024-09-19T04:45:14.626776+0000","last_clean":"2024-09-19T04:45:14.626776+0000","last_became_active":"2024-09-19T04:44:28.594515+0000","last_became_peered":"2024-09-19T04:44:28.594515+0000","last_unstale":"2024-09-19T04:45:14.626776+0000","last_undegraded":"2024-09-19T04:45:14.626776+0000","last_fullsized":"2024-09-19T04:45:14.626776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:53.728393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:14.626685+0000","last_change":"2024-09-19T04:44:30.753604+0000","last_active":"2024-09-19T04:45:14.626685+0000","last_peered":"2024-09-19T04:45:14.626685+0000","last_clean":"2024-09-19T04:45:14.626685+0000","last_became_active":"2024-09-19T04:44:30.751961+0000","last_became_peered":"2024-09-19T04:44:30.751961+0000","last_unstale":"2024-09-19T04:45:14.626685+0000","last_undegraded":"2024-09-19T04:45:14.626685+0000","last_fullsized":"2024-09-19T04:45:14.626685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:46:58.759650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":96,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T04:45:40.923879+0000","last_change":"2024-09-19T04:44:28.339499+0000","last_active":"2024-09-19T04:45:40.923879+0000","last_peered":"2024-09-19T04:45:40.923879+0000","last_clean":"2024-09-19T04:45:40.923879+0000","last_became_active":"2024-09-19T04:44:28.338774+0000","last_became_peered":"2024-09-19T04:44:28.338774+0000","last_unstale":"2024-09-19T04:45:40.923879+0000","last_undegraded":"2024-09-19T04:45:40.923879+0000","last_fullsized":"2024-09-19T04:45:40.923879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:12.261237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":59,"num_read_kb":59,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66579,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":618,"num_read_kb":380,"num_write":243,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":199737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247103,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481822,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":942080,"data_stored":455932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710754,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1097728,"data_stored":1107118,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874279,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":455862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":962560,"data_stored":456240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364849,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1015808,"data_stored":1048760,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528376,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":950272,"data_stored":455579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1028096,"data_stored":1048045,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953537,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":455833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":455404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":516476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1011712,"data_stored":516256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":456918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34357,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34357,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34357,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:45:46.241 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T04:45:46.241 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T04:45:46.241 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T04:45:46.241 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T04:45:46.609 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:46.749 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:46 smithi123 ceph-mon[30511]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:45:46.749 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:46 smithi123 ceph-mon[30511]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:45:46.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:46 smithi183 ceph-mon[35333]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:45:46.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:46 smithi183 ceph-mon[35333]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:45:46.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:46 smithi193 ceph-mon[35592]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:45:46.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:46 smithi193 ceph-mon[35592]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:45:47.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.185+0000 7fbf03200640 1 -- 172.21.15.123:0/3307164138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf04103e30 msgr2=0x7fbf04104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:47.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.185+0000 7fbf03200640 1 --2- 172.21.15.123:0/3307164138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf04103e30 0x7fbf04104210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009900 tx=0x7fbef402a400 comp rx=0 tx=0).stop 2024-09-19T04:45:47.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.185+0000 7fbf03200640 1 -- 172.21.15.123:0/3307164138 shutdown_connections 2024-09-19T04:45:47.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.185+0000 7fbf03200640 1 --2- 172.21.15.123:0/3307164138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf04108f60 0x7fbf04110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:47.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.185+0000 7fbf03200640 1 --2- 172.21.15.123:0/3307164138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf041047e0 0x7fbf04108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:47.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.185+0000 7fbf03200640 1 --2- 172.21.15.123:0/3307164138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf04103e30 0x7fbf04104210 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:47.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.185+0000 7fbf03200640 1 -- 172.21.15.123:0/3307164138 >> 172.21.15.123:0/3307164138 conn(0x7fbf040ff8d0 msgr2=0x7fbf04101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:47.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.186+0000 7fbf03200640 1 -- 172.21.15.123:0/3307164138 shutdown_connections 2024-09-19T04:45:47.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.186+0000 7fbf03200640 1 -- 172.21.15.123:0/3307164138 wait complete. 2024-09-19T04:45:47.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.186+0000 7fbf03200640 1 Processor -- start 2024-09-19T04:45:47.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.187+0000 7fbf03200640 1 -- start start 2024-09-19T04:45:47.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.187+0000 7fbf03200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04103e30 0x7fbf0419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:47.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.187+0000 7fbf03200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf041047e0 0x7fbf0419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:47.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.187+0000 7fbf03200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf04108f60 0x7fbf0419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:47.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.187+0000 7fbf03200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04114010 con 0x7fbf04108f60 2024-09-19T04:45:47.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.187+0000 7fbf03200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04113e90 con 0x7fbf04103e30 2024-09-19T04:45:47.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.187+0000 7fbf03200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04114190 con 0x7fbf041047e0 2024-09-19T04:45:47.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.187+0000 7fbf01e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04103e30 0x7fbf0419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:47.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.187+0000 7fbf01e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04103e30 0x7fbf0419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:43680/0 (socket says 172.21.15.123:43680) 2024-09-19T04:45:47.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.187+0000 7fbf01e00640 1 -- 172.21.15.123:0/2736440811 learned_addr learned my addr 172.21.15.123:0/2736440811 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:47.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.187+0000 7fbef9400640 1 --2- 172.21.15.123:0/2736440811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf041047e0 0x7fbf0419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:47.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.188+0000 7fbf02800640 1 --2- 172.21.15.123:0/2736440811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf04108f60 0x7fbf0419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:47.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.188+0000 7fbf02800640 1 -- 172.21.15.123:0/2736440811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf041047e0 msgr2=0x7fbf0419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:47.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.188+0000 7fbf02800640 1 --2- 172.21.15.123:0/2736440811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf041047e0 0x7fbf0419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:47.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.188+0000 7fbf02800640 1 -- 172.21.15.123:0/2736440811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04103e30 msgr2=0x7fbf0419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:47.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.188+0000 7fbf02800640 1 --2- 172.21.15.123:0/2736440811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04103e30 0x7fbf0419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:47.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.188+0000 7fbf02800640 1 -- 172.21.15.123:0/2736440811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0419f3e0 con 0x7fbf04108f60 2024-09-19T04:45:47.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.188+0000 7fbf02800640 1 --2- 172.21.15.123:0/2736440811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf04108f60 0x7fbf0419ed00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00e940 tx=0x7fbefc00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:47.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.189+0000 7fbefb400640 1 -- 172.21.15.123:0/2736440811 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc004d20 con 0x7fbf04108f60 2024-09-19T04:45:47.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.189+0000 7fbefb400640 1 -- 172.21.15.123:0/2736440811 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbefc005020 con 0x7fbf04108f60 2024-09-19T04:45:47.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.189+0000 7fbefb400640 1 -- 172.21.15.123:0/2736440811 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc0116b0 con 0x7fbf04108f60 2024-09-19T04:45:47.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.189+0000 7fbf03200640 1 -- 172.21.15.123:0/2736440811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0419f6d0 con 0x7fbf04108f60 2024-09-19T04:45:47.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.189+0000 7fbf03200640 1 -- 172.21.15.123:0/2736440811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf041a6fb0 con 0x7fbf04108f60 2024-09-19T04:45:47.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.190+0000 7fbf03200640 1 -- 172.21.15.123:0/2736440811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf04104210 con 0x7fbf04108f60 2024-09-19T04:45:47.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.192+0000 7fbefb400640 1 -- 172.21.15.123:0/2736440811 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fbefc0054d0 con 0x7fbf04108f60 2024-09-19T04:45:47.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.193+0000 7fbefb400640 1 --2- 172.21.15.123:0/2736440811 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbee80a3350 0x7fbee80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:47.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.193+0000 7fbefb400640 1 -- 172.21.15.123:0/2736440811 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fbefc0cfaa0 con 0x7fbf04108f60 2024-09-19T04:45:47.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.193+0000 7fbf01e00640 1 --2- 172.21.15.123:0/2736440811 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbee80a3350 0x7fbee80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:47.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.194+0000 7fbf01e00640 1 --2- 172.21.15.123:0/2736440811 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbee80a3350 0x7fbee80a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbef40096c0 tx=0x7fbef40046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:47.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.197+0000 7fbefb400640 1 -- 172.21.15.123:0/2736440811 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbefc016030 con 0x7fbf04108f60 2024-09-19T04:45:47.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.408+0000 7fbf03200640 1 -- 172.21.15.123:0/2736440811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbf0402d7c0 con 0x7fbf04108f60 2024-09-19T04:45:47.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.409+0000 7fbefb400640 1 -- 172.21.15.123:0/2736440811 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fbefc09bfc0 con 0x7fbf04108f60 2024-09-19T04:45:47.410 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:45:47.410 INFO:teuthology.orchestra.run.smithi123.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T04:45:47.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.411+0000 7fbf03200640 1 -- 172.21.15.123:0/2736440811 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbee80a3350 msgr2=0x7fbee80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:47.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.411+0000 7fbf03200640 1 --2- 172.21.15.123:0/2736440811 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbee80a3350 0x7fbee80a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbef40096c0 tx=0x7fbef40046c0 comp rx=0 tx=0).stop 2024-09-19T04:45:47.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.411+0000 7fbf03200640 1 -- 172.21.15.123:0/2736440811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf04108f60 msgr2=0x7fbf0419ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:47.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.411+0000 7fbf03200640 1 --2- 172.21.15.123:0/2736440811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf04108f60 0x7fbf0419ed00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00e940 tx=0x7fbefc00ee00 comp rx=0 tx=0).stop 2024-09-19T04:45:47.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.411+0000 7fbf03200640 1 -- 172.21.15.123:0/2736440811 shutdown_connections 2024-09-19T04:45:47.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.411+0000 7fbf03200640 1 --2- 172.21.15.123:0/2736440811 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fbee80a3350 0x7fbee80a5810 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:47.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.411+0000 7fbf03200640 1 --2- 172.21.15.123:0/2736440811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf04108f60 0x7fbf0419ed00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:47.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.411+0000 7fbf03200640 1 --2- 172.21.15.123:0/2736440811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf041047e0 0x7fbf0419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:47.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.411+0000 7fbf03200640 1 --2- 172.21.15.123:0/2736440811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04103e30 0x7fbf0419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:47.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.411+0000 7fbf03200640 1 -- 172.21.15.123:0/2736440811 >> 172.21.15.123:0/2736440811 conn(0x7fbf040ff8d0 msgr2=0x7fbf04100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:47.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.411+0000 7fbf03200640 1 -- 172.21.15.123:0/2736440811 shutdown_connections 2024-09-19T04:45:47.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:47.412+0000 7fbf03200640 1 -- 172.21.15.123:0/2736440811 wait complete. 2024-09-19T04:45:47.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:47 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2736440811' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T04:45:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:47 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2736440811' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T04:45:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:47 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2736440811' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T04:45:48.131 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T04:45:48.131 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:45:48.499 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:48.787 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:48 smithi123 ceph-mon[30511]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-19T04:45:48.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:48 smithi183 ceph-mon[35333]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-19T04:45:48.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:48 smithi193 ceph-mon[35592]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-19T04:45:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.056+0000 7fe9c4800640 1 -- 172.21.15.123:0/379897831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9bc0745c0 msgr2=0x7fe9bc0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.056+0000 7fe9c4800640 1 --2- 172.21.15.123:0/379897831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9bc0745c0 0x7fe9bc0749a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009900 tx=0x7fe9b002a480 comp rx=0 tx=0).stop 2024-09-19T04:45:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.056+0000 7fe9c4800640 1 -- 172.21.15.123:0/379897831 shutdown_connections 2024-09-19T04:45:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.056+0000 7fe9c4800640 1 --2- 172.21.15.123:0/379897831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9bc10e430 0x7fe9bc110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.056+0000 7fe9c4800640 1 --2- 172.21.15.123:0/379897831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9bc074ee0 0x7fe9bc10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:49.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.056+0000 7fe9c4800640 1 --2- 172.21.15.123:0/379897831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9bc0745c0 0x7fe9bc0749a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:49.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.056+0000 7fe9c4800640 1 -- 172.21.15.123:0/379897831 >> 172.21.15.123:0/379897831 conn(0x7fe9bc0fd8b0 msgr2=0x7fe9bc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:49.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.056+0000 7fe9c4800640 1 -- 172.21.15.123:0/379897831 shutdown_connections 2024-09-19T04:45:49.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.056+0000 7fe9c4800640 1 -- 172.21.15.123:0/379897831 wait complete. 2024-09-19T04:45:49.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.057+0000 7fe9c4800640 1 Processor -- start 2024-09-19T04:45:49.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.057+0000 7fe9c4800640 1 -- start start 2024-09-19T04:45:49.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.058+0000 7fe9c4800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9bc0745c0 0x7fe9bc19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:49.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.058+0000 7fe9c4800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9bc074ee0 0x7fe9bc19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:49.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.058+0000 7fe9c4800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9bc10e430 0x7fe9bc19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:49.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.058+0000 7fe9c4800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc1141f0 con 0x7fe9bc10e430 2024-09-19T04:45:49.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.058+0000 7fe9c4800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc114070 con 0x7fe9bc074ee0 2024-09-19T04:45:49.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.058+0000 7fe9c4800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc114370 con 0x7fe9bc0745c0 2024-09-19T04:45:49.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.058+0000 7fe9c2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9bc10e430 0x7fe9bc19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:49.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.058+0000 7fe9c2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9bc10e430 0x7fe9bc19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46576/0 (socket says 172.21.15.123:46576) 2024-09-19T04:45:49.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.058+0000 7fe9c2800640 1 -- 172.21.15.123:0/2150675000 learned_addr learned my addr 172.21.15.123:0/2150675000 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:49.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.058+0000 7fe9c1e00640 1 --2- 172.21.15.123:0/2150675000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9bc0745c0 0x7fe9bc19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:49.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.058+0000 7fe9c1400640 1 --2- 172.21.15.123:0/2150675000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9bc074ee0 0x7fe9bc19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:49.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.058+0000 7fe9c2800640 1 -- 172.21.15.123:0/2150675000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9bc0745c0 msgr2=0x7fe9bc19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:49.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.058+0000 7fe9c2800640 1 --2- 172.21.15.123:0/2150675000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9bc0745c0 0x7fe9bc19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:49.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.058+0000 7fe9c2800640 1 -- 172.21.15.123:0/2150675000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9bc074ee0 msgr2=0x7fe9bc19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:49.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.058+0000 7fe9c2800640 1 --2- 172.21.15.123:0/2150675000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9bc074ee0 0x7fe9bc19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:49.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.059+0000 7fe9c2800640 1 -- 172.21.15.123:0/2150675000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9bc19f640 con 0x7fe9bc10e430 2024-09-19T04:45:49.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.059+0000 7fe9c2800640 1 --2- 172.21.15.123:0/2150675000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9bc10e430 0x7fe9bc19ef60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800dac0 tx=0x7fe9b800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:49.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.059+0000 7fe9b6a00640 1 -- 172.21.15.123:0/2150675000 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8017070 con 0x7fe9bc10e430 2024-09-19T04:45:49.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.059+0000 7fe9b6a00640 1 -- 172.21.15.123:0/2150675000 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9b8003e50 con 0x7fe9bc10e430 2024-09-19T04:45:49.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.059+0000 7fe9b6a00640 1 -- 172.21.15.123:0/2150675000 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80106c0 con 0x7fe9bc10e430 2024-09-19T04:45:49.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.059+0000 7fe9c4800640 1 -- 172.21.15.123:0/2150675000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9bc19f930 con 0x7fe9bc10e430 2024-09-19T04:45:49.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.059+0000 7fe9c4800640 1 -- 172.21.15.123:0/2150675000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9bc1a7210 con 0x7fe9bc10e430 2024-09-19T04:45:49.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.060+0000 7fe9c4800640 1 -- 172.21.15.123:0/2150675000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9bc0749a0 con 0x7fe9bc10e430 2024-09-19T04:45:49.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.062+0000 7fe9b6a00640 1 -- 172.21.15.123:0/2150675000 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe9b8003970 con 0x7fe9bc10e430 2024-09-19T04:45:49.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.064+0000 7fe9b6a00640 1 --2- 172.21.15.123:0/2150675000 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe9a40a3350 0x7fe9a40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:49.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.065+0000 7fe9b6a00640 1 -- 172.21.15.123:0/2150675000 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fe9b80cf650 con 0x7fe9bc10e430 2024-09-19T04:45:49.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.065+0000 7fe9c1e00640 1 --2- 172.21.15.123:0/2150675000 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe9a40a3350 0x7fe9a40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:49.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.065+0000 7fe9c1e00640 1 --2- 172.21.15.123:0/2150675000 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe9a40a3350 0x7fe9a40a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0000c00 tx=0x7fe9b0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:49.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.068+0000 7fe9b6a00640 1 -- 172.21.15.123:0/2150675000 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe9b809bb70 con 0x7fe9bc10e430 2024-09-19T04:45:49.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.280+0000 7fe9c4800640 1 -- 172.21.15.123:0/2150675000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9bc109020 con 0x7fe9bc10e430 2024-09-19T04:45:49.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.281+0000 7fe9b6a00640 1 -- 172.21.15.123:0/2150675000 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11990 (secure 0 0 0) 0x7fe9b809fe70 con 0x7fe9bc10e430 2024-09-19T04:45:49.283 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:45:49.284 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"btime":"2024-09-19T04:44:44:834422+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:44:44.834407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24529,24505,24539,24500,24499]},"id":1}]} 2024-09-19T04:45:49.284 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 27 2024-09-19T04:45:49.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.283+0000 7fe9c4800640 1 -- 172.21.15.123:0/2150675000 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe9a40a3350 msgr2=0x7fe9a40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:49.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.284+0000 7fe9c4800640 1 --2- 172.21.15.123:0/2150675000 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe9a40a3350 0x7fe9a40a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0000c00 tx=0x7fe9b0005b50 comp rx=0 tx=0).stop 2024-09-19T04:45:49.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.284+0000 7fe9c4800640 1 -- 172.21.15.123:0/2150675000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9bc10e430 msgr2=0x7fe9bc19ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:49.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.284+0000 7fe9c4800640 1 --2- 172.21.15.123:0/2150675000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9bc10e430 0x7fe9bc19ef60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800dac0 tx=0x7fe9b800df80 comp rx=0 tx=0).stop 2024-09-19T04:45:49.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.284+0000 7fe9c4800640 1 -- 172.21.15.123:0/2150675000 shutdown_connections 2024-09-19T04:45:49.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.284+0000 7fe9c4800640 1 --2- 172.21.15.123:0/2150675000 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe9a40a3350 0x7fe9a40a5810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:49.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.284+0000 7fe9c4800640 1 --2- 172.21.15.123:0/2150675000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9bc10e430 0x7fe9bc19ef60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:49.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.284+0000 7fe9c4800640 1 --2- 172.21.15.123:0/2150675000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9bc074ee0 0x7fe9bc19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:49.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.284+0000 7fe9c4800640 1 --2- 172.21.15.123:0/2150675000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9bc0745c0 0x7fe9bc19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:49.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.284+0000 7fe9c4800640 1 -- 172.21.15.123:0/2150675000 >> 172.21.15.123:0/2150675000 conn(0x7fe9bc0fd8b0 msgr2=0x7fe9bc0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:49.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.284+0000 7fe9c4800640 1 -- 172.21.15.123:0/2150675000 shutdown_connections 2024-09-19T04:45:49.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:49.284+0000 7fe9c4800640 1 -- 172.21.15.123:0/2150675000 wait complete. 2024-09-19T04:45:49.803 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:49 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2150675000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:45:49.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:49 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2150675000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:45:49.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:49 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2150675000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:45:50.260 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:45:50.625 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:50.764 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:50 smithi123 ceph-mon[30511]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-19T04:45:50.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:50 smithi183 ceph-mon[35333]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-19T04:45:50.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:50 smithi193 ceph-mon[35592]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-19T04:45:51.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.226+0000 7f37f1800640 1 -- 172.21.15.123:0/3276258045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ec1047e0 msgr2=0x7f37ec108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:51.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.226+0000 7f37f1800640 1 --2- 172.21.15.123:0/3276258045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ec1047e0 0x7f37ec108830 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009900 tx=0x7f37d802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:45:51.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.226+0000 7f37f1800640 1 -- 172.21.15.123:0/3276258045 shutdown_connections 2024-09-19T04:45:51.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.226+0000 7f37f1800640 1 --2- 172.21.15.123:0/3276258045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37ec108f60 0x7f37ec110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:51.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.226+0000 7f37f1800640 1 --2- 172.21.15.123:0/3276258045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ec1047e0 0x7f37ec108830 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:51.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.226+0000 7f37f1800640 1 --2- 172.21.15.123:0/3276258045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec103e30 0x7f37ec104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:51.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.226+0000 7f37f1800640 1 -- 172.21.15.123:0/3276258045 >> 172.21.15.123:0/3276258045 conn(0x7f37ec0ff910 msgr2=0x7f37ec101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:51.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.226+0000 7f37f1800640 1 -- 172.21.15.123:0/3276258045 shutdown_connections 2024-09-19T04:45:51.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.226+0000 7f37f1800640 1 -- 172.21.15.123:0/3276258045 wait complete. 2024-09-19T04:45:51.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.227+0000 7f37f1800640 1 Processor -- start 2024-09-19T04:45:51.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.227+0000 7f37f1800640 1 -- start start 2024-09-19T04:45:51.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.227+0000 7f37f1800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37ec103e30 0x7f37ec19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:51.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.227+0000 7f37f1800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec1047e0 0x7f37ec19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:51.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.227+0000 7f37f1800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ec108f60 0x7f37ec19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:51.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.227+0000 7f37f1800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec114010 con 0x7f37ec108f60 2024-09-19T04:45:51.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.227+0000 7f37f1800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec113e90 con 0x7f37ec1047e0 2024-09-19T04:45:51.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.227+0000 7f37f1800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec114190 con 0x7f37ec103e30 2024-09-19T04:45:51.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.228+0000 7f37eb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec1047e0 0x7f37ec19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:51.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.228+0000 7f37f0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ec108f60 0x7f37ec19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:51.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.228+0000 7f37eb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec1047e0 0x7f37ec19a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:43708/0 (socket says 172.21.15.123:43708) 2024-09-19T04:45:51.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.228+0000 7f37eb400640 1 -- 172.21.15.123:0/1928084483 learned_addr learned my addr 172.21.15.123:0/1928084483 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:51.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.228+0000 7f37ebe00640 1 --2- 172.21.15.123:0/1928084483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37ec103e30 0x7f37ec19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:51.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.228+0000 7f37f0e00640 1 -- 172.21.15.123:0/1928084483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37ec103e30 msgr2=0x7f37ec19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:51.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.228+0000 7f37f0e00640 1 --2- 172.21.15.123:0/1928084483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37ec103e30 0x7f37ec19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:51.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.228+0000 7f37f0e00640 1 -- 172.21.15.123:0/1928084483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec1047e0 msgr2=0x7f37ec19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:51.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.228+0000 7f37f0e00640 1 --2- 172.21.15.123:0/1928084483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec1047e0 0x7f37ec19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:51.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.228+0000 7f37f0e00640 1 -- 172.21.15.123:0/1928084483 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec19f420 con 0x7f37ec108f60 2024-09-19T04:45:51.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.228+0000 7f37eb400640 1 --2- 172.21.15.123:0/1928084483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec1047e0 0x7f37ec19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:51.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.228+0000 7f37f0e00640 1 --2- 172.21.15.123:0/1928084483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ec108f60 0x7f37ec19ed40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f37e400d680 tx=0x7f37e400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:51.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.229+0000 7f37e8c00640 1 -- 172.21.15.123:0/1928084483 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4017070 con 0x7f37ec108f60 2024-09-19T04:45:51.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.229+0000 7f37e8c00640 1 -- 172.21.15.123:0/1928084483 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e4009820 con 0x7f37ec108f60 2024-09-19T04:45:51.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.229+0000 7f37e8c00640 1 -- 172.21.15.123:0/1928084483 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4004e00 con 0x7f37ec108f60 2024-09-19T04:45:51.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.229+0000 7f37f1800640 1 -- 172.21.15.123:0/1928084483 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ec19f710 con 0x7f37ec108f60 2024-09-19T04:45:51.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.229+0000 7f37f1800640 1 -- 172.21.15.123:0/1928084483 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ec1a6ff0 con 0x7f37ec108f60 2024-09-19T04:45:51.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.230+0000 7f37f1800640 1 -- 172.21.15.123:0/1928084483 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ec105310 con 0x7f37ec108f60 2024-09-19T04:45:51.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.232+0000 7f37e8c00640 1 -- 172.21.15.123:0/1928084483 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f37e40099d0 con 0x7f37ec108f60 2024-09-19T04:45:51.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.233+0000 7f37e8c00640 1 --2- 172.21.15.123:0/1928084483 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f37d00a32d0 0x7f37d00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:51.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.233+0000 7f37e8c00640 1 -- 172.21.15.123:0/1928084483 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f37e40cf290 con 0x7f37ec108f60 2024-09-19T04:45:51.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.233+0000 7f37ebe00640 1 --2- 172.21.15.123:0/1928084483 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f37d00a32d0 0x7f37d00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:51.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.234+0000 7f37ebe00640 1 --2- 172.21.15.123:0/1928084483 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f37d00a32d0 0x7f37d00a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f37dc005fd0 tx=0x7f37dc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:51.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.237+0000 7f37e8c00640 1 -- 172.21.15.123:0/1928084483 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f37e409b7b0 con 0x7f37ec108f60 2024-09-19T04:45:51.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.449+0000 7f37f1800640 1 -- 172.21.15.123:0/1928084483 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37ec104210 con 0x7f37ec108f60 2024-09-19T04:45:51.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.450+0000 7f37e8c00640 1 -- 172.21.15.123:0/1928084483 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11990 (secure 0 0 0) 0x7f37ec104210 con 0x7f37ec108f60 2024-09-19T04:45:51.452 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:45:51.452 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"btime":"2024-09-19T04:44:44:834422+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:44:44.834407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24529,24505,24539,24500,24499]},"id":1}]} 2024-09-19T04:45:51.452 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 27 2024-09-19T04:45:51.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.452+0000 7f37f1800640 1 -- 172.21.15.123:0/1928084483 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f37d00a32d0 msgr2=0x7f37d00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:51.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.452+0000 7f37f1800640 1 --2- 172.21.15.123:0/1928084483 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f37d00a32d0 0x7f37d00a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f37dc005fd0 tx=0x7f37dc009500 comp rx=0 tx=0).stop 2024-09-19T04:45:51.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.452+0000 7f37f1800640 1 -- 172.21.15.123:0/1928084483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ec108f60 msgr2=0x7f37ec19ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:51.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.452+0000 7f37f1800640 1 --2- 172.21.15.123:0/1928084483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ec108f60 0x7f37ec19ed40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f37e400d680 tx=0x7f37e400db40 comp rx=0 tx=0).stop 2024-09-19T04:45:51.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.452+0000 7f37f1800640 1 -- 172.21.15.123:0/1928084483 shutdown_connections 2024-09-19T04:45:51.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.452+0000 7f37f1800640 1 --2- 172.21.15.123:0/1928084483 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f37d00a32d0 0x7f37d00a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:51.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.452+0000 7f37f1800640 1 --2- 172.21.15.123:0/1928084483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ec108f60 0x7f37ec19ed40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:51.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.452+0000 7f37f1800640 1 --2- 172.21.15.123:0/1928084483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec1047e0 0x7f37ec19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:51.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.452+0000 7f37f1800640 1 --2- 172.21.15.123:0/1928084483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37ec103e30 0x7f37ec19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:51.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.453+0000 7f37f1800640 1 -- 172.21.15.123:0/1928084483 >> 172.21.15.123:0/1928084483 conn(0x7f37ec0ff910 msgr2=0x7f37ec100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:51.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.453+0000 7f37f1800640 1 -- 172.21.15.123:0/1928084483 shutdown_connections 2024-09-19T04:45:51.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:51.453+0000 7f37f1800640 1 -- 172.21.15.123:0/1928084483 wait complete. 2024-09-19T04:45:51.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:51 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1928084483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:45:51.863 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:51 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1928084483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:45:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:51 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1928084483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:45:52.294 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:45:52.661 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:52.799 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:52 smithi123 ceph-mon[30511]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:45:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:52 smithi183 ceph-mon[35333]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:45:52.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:52 smithi193 ceph-mon[35592]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:45:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.209+0000 7fe7e9c00640 1 -- 172.21.15.123:0/3442234520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7e4102360 msgr2=0x7fe7e410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.209+0000 7fe7e9c00640 1 --2- 172.21.15.123:0/3442234520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7e4102360 0x7fe7e410e890 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009cd0 tx=0x7fe7dc02a8e0 comp rx=0 tx=0).stop 2024-09-19T04:45:53.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.210+0000 7fe7e9c00640 1 -- 172.21.15.123:0/3442234520 shutdown_connections 2024-09-19T04:45:53.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.210+0000 7fe7e9c00640 1 --2- 172.21.15.123:0/3442234520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7e4102360 0x7fe7e410e890 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:53.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.210+0000 7fe7e9c00640 1 --2- 172.21.15.123:0/3442234520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7e41019c0 0x7fe7e4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:53.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.210+0000 7fe7e9c00640 1 --2- 172.21.15.123:0/3442234520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7e41079c0 0x7fe7e4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:53.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.210+0000 7fe7e9c00640 1 -- 172.21.15.123:0/3442234520 >> 172.21.15.123:0/3442234520 conn(0x7fe7e40fd6e0 msgr2=0x7fe7e40ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:53.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.210+0000 7fe7e9c00640 1 -- 172.21.15.123:0/3442234520 shutdown_connections 2024-09-19T04:45:53.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.210+0000 7fe7e9c00640 1 -- 172.21.15.123:0/3442234520 wait complete. 2024-09-19T04:45:53.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.211+0000 7fe7e9c00640 1 Processor -- start 2024-09-19T04:45:53.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.211+0000 7fe7e9c00640 1 -- start start 2024-09-19T04:45:53.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.212+0000 7fe7e9c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7e41019c0 0x7fe7e419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:53.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.212+0000 7fe7e9c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7e4102360 0x7fe7e419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:53.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.212+0000 7fe7e9c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7e41079c0 0x7fe7e419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:53.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.212+0000 7fe7e9c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e4111e10 con 0x7fe7e4102360 2024-09-19T04:45:53.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.212+0000 7fe7e9c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e4111c90 con 0x7fe7e41019c0 2024-09-19T04:45:53.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.212+0000 7fe7e9c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e4111f90 con 0x7fe7e41079c0 2024-09-19T04:45:53.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.212+0000 7fe7db400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7e4102360 0x7fe7e419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:53.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.212+0000 7fe7db400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7e4102360 0x7fe7e419a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37106/0 (socket says 172.21.15.123:37106) 2024-09-19T04:45:53.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.212+0000 7fe7db400640 1 -- 172.21.15.123:0/619870260 learned_addr learned my addr 172.21.15.123:0/619870260 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:53.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.212+0000 7fe7e3e00640 1 --2- 172.21.15.123:0/619870260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7e41019c0 0x7fe7e419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:53.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.212+0000 7fe7e9200640 1 --2- 172.21.15.123:0/619870260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7e41079c0 0x7fe7e419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:53.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.212+0000 7fe7db400640 1 -- 172.21.15.123:0/619870260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7e41079c0 msgr2=0x7fe7e419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:53.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.212+0000 7fe7db400640 1 --2- 172.21.15.123:0/619870260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7e41079c0 0x7fe7e419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:53.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.212+0000 7fe7db400640 1 -- 172.21.15.123:0/619870260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7e41019c0 msgr2=0x7fe7e419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:53.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.212+0000 7fe7db400640 1 --2- 172.21.15.123:0/619870260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7e41019c0 0x7fe7e419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:53.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.212+0000 7fe7db400640 1 -- 172.21.15.123:0/619870260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e419f3d0 con 0x7fe7e4102360 2024-09-19T04:45:53.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.213+0000 7fe7db400640 1 --2- 172.21.15.123:0/619870260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7e4102360 0x7fe7e419a960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00e9f0 tx=0x7fe7cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:53.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.213+0000 7fe7e1600640 1 -- 172.21.15.123:0/619870260 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc00cd80 con 0x7fe7e4102360 2024-09-19T04:45:53.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.213+0000 7fe7e9c00640 1 -- 172.21.15.123:0/619870260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e419f6c0 con 0x7fe7e4102360 2024-09-19T04:45:53.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.213+0000 7fe7e1600640 1 -- 172.21.15.123:0/619870260 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7cc005500 con 0x7fe7e4102360 2024-09-19T04:45:53.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.213+0000 7fe7e9c00640 1 -- 172.21.15.123:0/619870260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e41a6fa0 con 0x7fe7e4102360 2024-09-19T04:45:53.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.213+0000 7fe7e1600640 1 -- 172.21.15.123:0/619870260 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc018630 con 0x7fe7e4102360 2024-09-19T04:45:53.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.214+0000 7fe7e9c00640 1 -- 172.21.15.123:0/619870260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e4101e20 con 0x7fe7e4102360 2024-09-19T04:45:53.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.216+0000 7fe7e1600640 1 -- 172.21.15.123:0/619870260 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe7cc0046c0 con 0x7fe7e4102360 2024-09-19T04:45:53.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.217+0000 7fe7e1600640 1 --2- 172.21.15.123:0/619870260 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe7c80a32d0 0x7fe7c80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:53.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.217+0000 7fe7e1600640 1 -- 172.21.15.123:0/619870260 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fe7cc0cffd0 con 0x7fe7e4102360 2024-09-19T04:45:53.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.217+0000 7fe7e3e00640 1 --2- 172.21.15.123:0/619870260 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe7c80a32d0 0x7fe7c80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:53.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.218+0000 7fe7e3e00640 1 --2- 172.21.15.123:0/619870260 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe7c80a32d0 0x7fe7c80a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009b50 tx=0x7fe7d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:53.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.221+0000 7fe7e1600640 1 -- 172.21.15.123:0/619870260 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe7cc09cb40 con 0x7fe7e4102360 2024-09-19T04:45:53.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.398+0000 7fe7e9c00640 1 -- 172.21.15.123:0/619870260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7e41a72d0 con 0x7fe7e4102360 2024-09-19T04:45:53.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.400+0000 7fe7e1600640 1 -- 172.21.15.123:0/619870260 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22794 (secure 0 0 0) 0x7fe7cc09c440 con 0x7fe7e4102360 2024-09-19T04:45:53.402 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:45:53.402 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":86,"fsid":"911fa052-7640-11ef-bceb-c7b262605968","created":"2024-09-19T04:36:21.083333+0000","modified":"2024-09-19T04:45:14.619563+0000","last_up_change":"2024-09-19T04:43:33.818439+0000","last_in_change":"2024-09-19T04:43:10.148272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:39:33.856981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:44:26.956483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:44:29.718625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6803","nonce":3631596501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6805","nonce":3631596501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6809","nonce":3631596501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6807","nonce":3631596501}]},"public_addr":"172.21.15.123:6803/3631596501","cluster_addr":"172.21.15.123:6805/3631596501","heartbeat_back_addr":"172.21.15.123:6809/3631596501","heartbeat_front_addr":"172.21.15.123:6807/3631596501","state":["exists","up"]},{"osd":1,"uuid":"a526d4ce-586b-4d66-b720-e868df92fa24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6801","nonce":1259721614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6803","nonce":1259721614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6807","nonce":1259721614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6805","nonce":1259721614}]},"public_addr":"172.21.15.183:6801/1259721614","cluster_addr":"172.21.15.183:6803/1259721614","heartbeat_back_addr":"172.21.15.183:6807/1259721614","heartbeat_front_addr":"172.21.15.183:6805/1259721614","state":["exists","up"]},{"osd":2,"uuid":"3650601a-6259-4b2a-97df-1d47b09b0127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6801","nonce":2708151726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6803","nonce":2708151726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6807","nonce":2708151726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6805","nonce":2708151726}]},"public_addr":"172.21.15.193:6801/2708151726","cluster_addr":"172.21.15.193:6803/2708151726","heartbeat_back_addr":"172.21.15.193:6807/2708151726","heartbeat_front_addr":"172.21.15.193:6805/2708151726","state":["exists","up"]},{"osd":3,"uuid":"67c485ad-136e-4020-950d-9171a3c2de35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6811","nonce":3214806282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6813","nonce":3214806282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6817","nonce":3214806282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6815","nonce":3214806282}]},"public_addr":"172.21.15.123:6811/3214806282","cluster_addr":"172.21.15.123:6813/3214806282","heartbeat_back_addr":"172.21.15.123:6817/3214806282","heartbeat_front_addr":"172.21.15.123:6815/3214806282","state":["exists","up"]},{"osd":4,"uuid":"c9098540-06ab-45a8-9cf9-f90519cc852e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6809","nonce":667618665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6811","nonce":667618665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6815","nonce":667618665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6813","nonce":667618665}]},"public_addr":"172.21.15.183:6809/667618665","cluster_addr":"172.21.15.183:6811/667618665","heartbeat_back_addr":"172.21.15.183:6815/667618665","heartbeat_front_addr":"172.21.15.183:6813/667618665","state":["exists","up"]},{"osd":5,"uuid":"278a8dbd-6492-42d4-b489-1f832350bc6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6809","nonce":2292251639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6811","nonce":2292251639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6815","nonce":2292251639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6813","nonce":2292251639}]},"public_addr":"172.21.15.193:6809/2292251639","cluster_addr":"172.21.15.193:6811/2292251639","heartbeat_back_addr":"172.21.15.193:6815/2292251639","heartbeat_front_addr":"172.21.15.193:6813/2292251639","state":["exists","up"]},{"osd":6,"uuid":"0fda4e39-6adb-4941-a39b-a9a15019088c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6819","nonce":226007499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6821","nonce":226007499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6825","nonce":226007499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6823","nonce":226007499}]},"public_addr":"172.21.15.123:6819/226007499","cluster_addr":"172.21.15.123:6821/226007499","heartbeat_back_addr":"172.21.15.123:6825/226007499","heartbeat_front_addr":"172.21.15.123:6823/226007499","state":["exists","up"]},{"osd":7,"uuid":"5c0a021e-9f3b-4766-9686-0ef24b68cf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6817","nonce":3718518226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6819","nonce":3718518226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6823","nonce":3718518226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6821","nonce":3718518226}]},"public_addr":"172.21.15.183:6817/3718518226","cluster_addr":"172.21.15.183:6819/3718518226","heartbeat_back_addr":"172.21.15.183:6823/3718518226","heartbeat_front_addr":"172.21.15.183:6821/3718518226","state":["exists","up"]},{"osd":8,"uuid":"9b0e67e0-6305-4f81-9211-d41960099a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6817","nonce":3551404118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6819","nonce":3551404118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6823","nonce":3551404118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6821","nonce":3551404118}]},"public_addr":"172.21.15.193:6817/3551404118","cluster_addr":"172.21.15.193:6819/3551404118","heartbeat_back_addr":"172.21.15.193:6823/3551404118","heartbeat_front_addr":"172.21.15.193:6821/3551404118","state":["exists","up"]},{"osd":9,"uuid":"978ef3fa-c6ec-421f-b194-56d44813be5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6827","nonce":1439901149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6829","nonce":1439901149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6833","nonce":1439901149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6831","nonce":1439901149}]},"public_addr":"172.21.15.123:6827/1439901149","cluster_addr":"172.21.15.123:6829/1439901149","heartbeat_back_addr":"172.21.15.123:6833/1439901149","heartbeat_front_addr":"172.21.15.123:6831/1439901149","state":["exists","up"]},{"osd":10,"uuid":"5495e09b-7809-47b0-b7e0-b56c452f9f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6825","nonce":3355349245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6827","nonce":3355349245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6831","nonce":3355349245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6829","nonce":3355349245}]},"public_addr":"172.21.15.183:6825/3355349245","cluster_addr":"172.21.15.183:6827/3355349245","heartbeat_back_addr":"172.21.15.183:6831/3355349245","heartbeat_front_addr":"172.21.15.183:6829/3355349245","state":["exists","up"]},{"osd":11,"uuid":"03046fef-b16b-42f8-9a19-6c0b78cbfe7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6825","nonce":4291617038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6827","nonce":4291617038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6831","nonce":4291617038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6829","nonce":4291617038}]},"public_addr":"172.21.15.193:6825/4291617038","cluster_addr":"172.21.15.193:6827/4291617038","heartbeat_back_addr":"172.21.15.193:6831/4291617038","heartbeat_front_addr":"172.21.15.193:6829/4291617038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:38:39.142529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:05.336911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:30.310967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:59.655973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:25.890576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:50.778689+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:19.063731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:45.424269+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:10.562149+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:40.382368+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:06.297731+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:32.352172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/4040401634":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/3498196650":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6800/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:6800/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/471028951":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/1009496020":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/2000593686":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/2580747029":"2024-09-20T04:36:45.913060+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:45:53.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.402+0000 7fe7e9c00640 1 -- 172.21.15.123:0/619870260 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe7c80a32d0 msgr2=0x7fe7c80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:53.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.402+0000 7fe7e9c00640 1 --2- 172.21.15.123:0/619870260 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe7c80a32d0 0x7fe7c80a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009b50 tx=0x7fe7d4005f90 comp rx=0 tx=0).stop 2024-09-19T04:45:53.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.402+0000 7fe7e9c00640 1 -- 172.21.15.123:0/619870260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7e4102360 msgr2=0x7fe7e419a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:53.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.402+0000 7fe7e9c00640 1 --2- 172.21.15.123:0/619870260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7e4102360 0x7fe7e419a960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00e9f0 tx=0x7fe7cc00edf0 comp rx=0 tx=0).stop 2024-09-19T04:45:53.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.402+0000 7fe7e9c00640 1 -- 172.21.15.123:0/619870260 shutdown_connections 2024-09-19T04:45:53.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.402+0000 7fe7e9c00640 1 --2- 172.21.15.123:0/619870260 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe7c80a32d0 0x7fe7c80a5790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:53.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.402+0000 7fe7e9c00640 1 --2- 172.21.15.123:0/619870260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7e41079c0 0x7fe7e419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:53.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.402+0000 7fe7e9c00640 1 --2- 172.21.15.123:0/619870260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7e4102360 0x7fe7e419a960 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:53.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.402+0000 7fe7e9c00640 1 --2- 172.21.15.123:0/619870260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7e41019c0 0x7fe7e419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:53.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.402+0000 7fe7e9c00640 1 -- 172.21.15.123:0/619870260 >> 172.21.15.123:0/619870260 conn(0x7fe7e40fd6e0 msgr2=0x7fe7e410cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:53.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.403+0000 7fe7e9c00640 1 -- 172.21.15.123:0/619870260 shutdown_connections 2024-09-19T04:45:53.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:53.403+0000 7fe7e9c00640 1 -- 172.21.15.123:0/619870260 wait complete. 2024-09-19T04:45:53.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:53 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/619870260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:45:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:53 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/619870260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:45:53.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:53 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/619870260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:45:54.203 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:45:54.571 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:54 smithi183 ceph-mon[35333]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:45:54.867 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:54 smithi123 ceph-mon[30511]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:45:54.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:54 smithi193 ceph-mon[35592]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:45:55.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.145+0000 7f0be8600640 1 -- 172.21.15.123:0/1661901718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be0103ff0 msgr2=0x7f0be01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:55.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.145+0000 7f0be8600640 1 --2- 172.21.15.123:0/1661901718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be0103ff0 0x7f0be01043d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009900 tx=0x7f0bd402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:45:55.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.145+0000 7f0be8600640 1 -- 172.21.15.123:0/1661901718 shutdown_connections 2024-09-19T04:45:55.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.145+0000 7f0be8600640 1 --2- 172.21.15.123:0/1661901718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0be0109120 0x7f0be0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:55.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.145+0000 7f0be8600640 1 --2- 172.21.15.123:0/1661901718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be01049a0 0x7f0be01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:55.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.145+0000 7f0be8600640 1 --2- 172.21.15.123:0/1661901718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be0103ff0 0x7f0be01043d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:55.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.145+0000 7f0be8600640 1 -- 172.21.15.123:0/1661901718 >> 172.21.15.123:0/1661901718 conn(0x7f0be00ffa90 msgr2=0x7f0be0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:55.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.145+0000 7f0be8600640 1 -- 172.21.15.123:0/1661901718 shutdown_connections 2024-09-19T04:45:55.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.145+0000 7f0be8600640 1 -- 172.21.15.123:0/1661901718 wait complete. 2024-09-19T04:45:55.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.146+0000 7f0be8600640 1 Processor -- start 2024-09-19T04:45:55.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.146+0000 7f0be8600640 1 -- start start 2024-09-19T04:45:55.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.147+0000 7f0be8600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be0103ff0 0x7f0be019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:55.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.147+0000 7f0be8600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be01049a0 0x7f0be019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:55.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.147+0000 7f0be8600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0be0109120 0x7f0be019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:55.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.147+0000 7f0be8600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0114170 con 0x7f0be0103ff0 2024-09-19T04:45:55.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.147+0000 7f0be8600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0113ff0 con 0x7f0be01049a0 2024-09-19T04:45:55.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.147+0000 7f0be8600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0be01142f0 con 0x7f0be0109120 2024-09-19T04:45:55.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.147+0000 7f0be5200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be01049a0 0x7f0be019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:55.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.147+0000 7f0be5200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be01049a0 0x7f0be019ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:52706/0 (socket says 172.21.15.123:52706) 2024-09-19T04:45:55.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.147+0000 7f0be5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be0103ff0 0x7f0be019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:55.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.147+0000 7f0be5200640 1 -- 172.21.15.123:0/791071358 learned_addr learned my addr 172.21.15.123:0/791071358 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:55.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.147+0000 7f0be6600640 1 --2- 172.21.15.123:0/791071358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0be0109120 0x7f0be019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:55.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.147+0000 7f0be5200640 1 -- 172.21.15.123:0/791071358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0be0109120 msgr2=0x7f0be019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:55.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.147+0000 7f0be5200640 1 --2- 172.21.15.123:0/791071358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0be0109120 0x7f0be019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:55.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.147+0000 7f0be5200640 1 -- 172.21.15.123:0/791071358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be0103ff0 msgr2=0x7f0be019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:55.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.147+0000 7f0be5200640 1 --2- 172.21.15.123:0/791071358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be0103ff0 0x7f0be019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:55.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.147+0000 7f0be5200640 1 -- 172.21.15.123:0/791071358 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be019f5a0 con 0x7f0be01049a0 2024-09-19T04:45:55.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.148+0000 7f0be5c00640 1 --2- 172.21.15.123:0/791071358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be0103ff0 0x7f0be019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:55.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.148+0000 7f0be5200640 1 --2- 172.21.15.123:0/791071358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be01049a0 0x7f0be019ab30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00ea60 tx=0x7f0bcc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:55.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.148+0000 7f0bde000640 1 -- 172.21.15.123:0/791071358 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc00cde0 con 0x7f0be01049a0 2024-09-19T04:45:55.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.148+0000 7f0bde000640 1 -- 172.21.15.123:0/791071358 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bcc0035f0 con 0x7f0be01049a0 2024-09-19T04:45:55.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.148+0000 7f0be8600640 1 -- 172.21.15.123:0/791071358 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be019f890 con 0x7f0be01049a0 2024-09-19T04:45:55.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.148+0000 7f0bde000640 1 -- 172.21.15.123:0/791071358 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc010670 con 0x7f0be01049a0 2024-09-19T04:45:55.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.148+0000 7f0be8600640 1 -- 172.21.15.123:0/791071358 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be01a7170 con 0x7f0be01049a0 2024-09-19T04:45:55.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.149+0000 7f0be8600640 1 -- 172.21.15.123:0/791071358 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be01043d0 con 0x7f0be01049a0 2024-09-19T04:45:55.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.151+0000 7f0bde000640 1 -- 172.21.15.123:0/791071358 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0bcc0037a0 con 0x7f0be01049a0 2024-09-19T04:45:55.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.153+0000 7f0bde000640 1 --2- 172.21.15.123:0/791071358 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0bc80a3350 0x7f0bc80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:55.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.153+0000 7f0bde000640 1 -- 172.21.15.123:0/791071358 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f0bcc0cfe00 con 0x7f0be01049a0 2024-09-19T04:45:55.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.153+0000 7f0be5c00640 1 --2- 172.21.15.123:0/791071358 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0bc80a3350 0x7f0bc80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:55.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.154+0000 7f0be5c00640 1 --2- 172.21.15.123:0/791071358 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0bc80a3350 0x7f0bc80a5810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4000c00 tx=0x7f0bd4031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:55.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.155+0000 7f0bde000640 1 -- 172.21.15.123:0/791071358 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0bcc09ca00 con 0x7f0be01049a0 2024-09-19T04:45:55.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.366+0000 7f0be8600640 1 -- 172.21.15.123:0/791071358 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0be019b950 con 0x7f0be01049a0 2024-09-19T04:45:55.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.368+0000 7f0bde000640 1 -- 172.21.15.123:0/791071358 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11990 (secure 0 0 0) 0x7f0be019b950 con 0x7f0be01049a0 2024-09-19T04:45:55.369 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:45:55.369 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"btime":"2024-09-19T04:44:44:834422+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:44:44.834407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24529,24505,24539,24500,24499]},"id":1}]} 2024-09-19T04:45:55.370 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 27 2024-09-19T04:45:55.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.370+0000 7f0be8600640 1 -- 172.21.15.123:0/791071358 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0bc80a3350 msgr2=0x7f0bc80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.370+0000 7f0be8600640 1 --2- 172.21.15.123:0/791071358 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0bc80a3350 0x7f0bc80a5810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4000c00 tx=0x7f0bd4031510 comp rx=0 tx=0).stop 2024-09-19T04:45:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.370+0000 7f0be8600640 1 -- 172.21.15.123:0/791071358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be01049a0 msgr2=0x7f0be019ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.370+0000 7f0be8600640 1 --2- 172.21.15.123:0/791071358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be01049a0 0x7f0be019ab30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00ea60 tx=0x7f0bcc00ef20 comp rx=0 tx=0).stop 2024-09-19T04:45:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.370+0000 7f0be8600640 1 -- 172.21.15.123:0/791071358 shutdown_connections 2024-09-19T04:45:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.370+0000 7f0be8600640 1 --2- 172.21.15.123:0/791071358 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f0bc80a3350 0x7f0bc80a5810 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.370+0000 7f0be8600640 1 --2- 172.21.15.123:0/791071358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0be0109120 0x7f0be019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.370+0000 7f0be8600640 1 --2- 172.21.15.123:0/791071358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be01049a0 0x7f0be019ab30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.370+0000 7f0be8600640 1 --2- 172.21.15.123:0/791071358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be0103ff0 0x7f0be019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.370+0000 7f0be8600640 1 -- 172.21.15.123:0/791071358 >> 172.21.15.123:0/791071358 conn(0x7f0be00ffa90 msgr2=0x7f0be0100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.370+0000 7f0be8600640 1 -- 172.21.15.123:0/791071358 shutdown_connections 2024-09-19T04:45:55.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:55.370+0000 7f0be8600640 1 -- 172.21.15.123:0/791071358 wait complete. 2024-09-19T04:45:55.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:55 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/791071358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:45:55.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:55 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/791071358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:45:55.904 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:55 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/791071358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:45:56.213 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T04:44:31.766928+0000', 'modified': '2024-09-19T04:44:44.834407+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24539, 'mds_1': 24529, 'mds_2': 24505, 'mds_3': 24500, 'mds_4': 24499}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14514': {'gid': 14514, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6841/784762674', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6840', 'nonce': 784762674}, {'type': 'v1', 'addr': '172.21.15.123:6841', 'nonce': 784762674}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6833/2688234655', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6832', 'nonce': 2688234655}, {'type': 'v1', 'addr': '172.21.15.183:6833', 'nonce': 2688234655}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6833/964700250', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6832', 'nonce': 964700250}, {'type': 'v1', 'addr': '172.21.15.193:6833', 'nonce': 964700250}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'd', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6837/3620540498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 3620540498}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 3620540498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/309335638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 309335638}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 309335638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.193:6835/1351624350', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 1351624350}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 1351624350}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.183:6837/3418533099', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 3418533099}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 3418533099}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.193:6837/3444596480', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 3444596480}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 3444596480}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24529': {'gid': 24529, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.193:6839/4148250455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 4148250455}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 4148250455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.183:6839/157697996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 157697996}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 157697996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24539, 'qdb_cluster': [24529, 24505, 24539, 24500, 24499]} 2024-09-19T04:45:56.214 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T04:45:56.214 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T04:45:56.601 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:56.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:56 smithi183 ceph-mon[35333]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:45:56.883 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:56 smithi123 ceph-mon[30511]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:45:56.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:56 smithi193 ceph-mon[35592]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:45:57.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.176+0000 7fefc9c00640 1 -- 172.21.15.123:0/1457672727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc4108f60 msgr2=0x7fefc4110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:57.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.176+0000 7fefc9c00640 1 --2- 172.21.15.123:0/1457672727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc4108f60 0x7fefc4110ae0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009ca0 tx=0x7fefbc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.176+0000 7fefc9c00640 1 -- 172.21.15.123:0/1457672727 shutdown_connections 2024-09-19T04:45:57.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.176+0000 7fefc9c00640 1 --2- 172.21.15.123:0/1457672727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc4108f60 0x7fefc4110ae0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.176+0000 7fefc9c00640 1 --2- 172.21.15.123:0/1457672727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc41047e0 0x7fefc4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.176+0000 7fefc9c00640 1 --2- 172.21.15.123:0/1457672727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc4103e30 0x7fefc4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.176+0000 7fefc9c00640 1 -- 172.21.15.123:0/1457672727 >> 172.21.15.123:0/1457672727 conn(0x7fefc40ff8d0 msgr2=0x7fefc4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:57.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.176+0000 7fefc9c00640 1 -- 172.21.15.123:0/1457672727 shutdown_connections 2024-09-19T04:45:57.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.176+0000 7fefc9c00640 1 -- 172.21.15.123:0/1457672727 wait complete. 2024-09-19T04:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.177+0000 7fefc9c00640 1 Processor -- start 2024-09-19T04:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.178+0000 7fefc9c00640 1 -- start start 2024-09-19T04:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.178+0000 7fefc9c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc4103e30 0x7fefc419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.178+0000 7fefc9c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc41047e0 0x7fefc419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.178+0000 7fefc9c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc4108f60 0x7fefc419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.178+0000 7fefc9c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fefc4114010 con 0x7fefc4103e30 2024-09-19T04:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.178+0000 7fefc9c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fefc4113e90 con 0x7fefc4108f60 2024-09-19T04:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.178+0000 7fefc9c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fefc4114190 con 0x7fefc41047e0 2024-09-19T04:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.178+0000 7fefc3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc4103e30 0x7fefc419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.178+0000 7fefc3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc4103e30 0x7fefc419a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37142/0 (socket says 172.21.15.123:37142) 2024-09-19T04:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.178+0000 7fefc3e00640 1 -- 172.21.15.123:0/537967115 learned_addr learned my addr 172.21.15.123:0/537967115 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.178+0000 7fefc3400640 1 --2- 172.21.15.123:0/537967115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc41047e0 0x7fefc419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.179+0000 7fefc9200640 1 --2- 172.21.15.123:0/537967115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc4108f60 0x7fefc419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.179+0000 7fefc3400640 1 -- 172.21.15.123:0/537967115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc4108f60 msgr2=0x7fefc419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:57.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.179+0000 7fefc3400640 1 --2- 172.21.15.123:0/537967115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc4108f60 0x7fefc419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.179+0000 7fefc3400640 1 -- 172.21.15.123:0/537967115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc4103e30 msgr2=0x7fefc419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:57.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.179+0000 7fefc3400640 1 --2- 172.21.15.123:0/537967115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc4103e30 0x7fefc419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.179+0000 7fefc3400640 1 -- 172.21.15.123:0/537967115 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc419f3a0 con 0x7fefc41047e0 2024-09-19T04:45:57.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.179+0000 7fefc3e00640 1 --2- 172.21.15.123:0/537967115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc4103e30 0x7fefc419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:57.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.179+0000 7fefc3400640 1 --2- 172.21.15.123:0/537967115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc41047e0 0x7fefc419a930 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fefb000e9f0 tx=0x7fefb000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:57.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.179+0000 7fefc0c00640 1 -- 172.21.15.123:0/537967115 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb000ccc0 con 0x7fefc41047e0 2024-09-19T04:45:57.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.179+0000 7fefc0c00640 1 -- 172.21.15.123:0/537967115 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb0010910 con 0x7fefc41047e0 2024-09-19T04:45:57.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.179+0000 7fefc9c00640 1 -- 172.21.15.123:0/537967115 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc419f690 con 0x7fefc41047e0 2024-09-19T04:45:57.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.180+0000 7fefc0c00640 1 -- 172.21.15.123:0/537967115 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb0018960 con 0x7fefc41047e0 2024-09-19T04:45:57.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.180+0000 7fefc9c00640 1 -- 172.21.15.123:0/537967115 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc41a6f70 con 0x7fefc41047e0 2024-09-19T04:45:57.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.183+0000 7fefc0c00640 1 -- 172.21.15.123:0/537967115 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fefb0010ac0 con 0x7fefc41047e0 2024-09-19T04:45:57.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.183+0000 7fefcdc23740 1 Processor -- start 2024-09-19T04:45:57.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.183+0000 7fefcdc23740 1 -- start start 2024-09-19T04:45:57.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.183+0000 7fefcdc23740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e206154e0 0x556e206158c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.183+0000 7fefcdc23740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20615e00 0x556e20620180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.183+0000 7fefcdc23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e206206c0 0x556e20622ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.183+0000 7fefcdc23740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556e2057b0b0 con 0x556e20615e00 2024-09-19T04:45:57.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.183+0000 7fefcdc23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556e20522ab0 con 0x556e206206c0 2024-09-19T04:45:57.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.183+0000 7fefcdc23740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556e20577070 con 0x556e206154e0 2024-09-19T04:45:57.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.183+0000 7fefc3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20615e00 0x556e20620180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.184+0000 7fefc3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e206206c0 0x556e20622ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.184+0000 7fefc3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20615e00 0x556e20620180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37152/0 (socket says 172.21.15.123:37152) 2024-09-19T04:45:57.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.184+0000 7fefc3e00640 1 -- 172.21.15.123:0/883306495 learned_addr learned my addr 172.21.15.123:0/883306495 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:57.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.184+0000 7fefc9200640 1 --2- 172.21.15.123:0/883306495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e206154e0 0x556e206158c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.184+0000 7fefc3e00640 1 -- 172.21.15.123:0/883306495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e206154e0 msgr2=0x556e206158c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:45:57.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.184+0000 7fefc3e00640 1 --2- 172.21.15.123:0/883306495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e206154e0 0x556e206158c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.184+0000 7fefc3e00640 1 -- 172.21.15.123:0/883306495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e206206c0 msgr2=0x556e20622ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:45:57.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.184+0000 7fefc3e00640 1 --2- 172.21.15.123:0/883306495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e206206c0 0x556e20622ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.184+0000 7fefc3e00640 1 -- 172.21.15.123:0/883306495 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e2052f4a0 con 0x556e20615e00 2024-09-19T04:45:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.184+0000 7fefc3e00640 1 --2- 172.21.15.123:0/883306495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20615e00 0x556e20620180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fefb4011630 tx=0x7fefb4011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8828cb88bbf4062a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.184+0000 7fefc0c00640 1 --2- 172.21.15.123:0/537967115 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fefa80a3350 0x7fefa80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.185+0000 7fefb9600640 1 -- 172.21.15.123:0/883306495 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb401e070 con 0x556e20615e00 2024-09-19T04:45:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.185+0000 7fefc0c00640 1 -- 172.21.15.123:0/537967115 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fefb00cfda0 con 0x7fefc41047e0 2024-09-19T04:45:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.185+0000 7fefb9600640 1 -- 172.21.15.123:0/883306495 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb400fbd0 con 0x556e20615e00 2024-09-19T04:45:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.185+0000 7fefb9600640 1 -- 172.21.15.123:0/883306495 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4019d00 con 0x556e20615e00 2024-09-19T04:45:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.185+0000 7fefc9200640 1 --2- 172.21.15.123:0/537967115 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fefa80a3350 0x7fefa80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.185+0000 7fefc9200640 1 --2- 172.21.15.123:0/537967115 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fefa80a3350 0x7fefa80a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fefbc006010 tx=0x7fefbc005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.185+0000 7fefcdc23740 1 -- 172.21.15.123:0/883306495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20615e00 msgr2=0x556e20620180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.185+0000 7fefcdc23740 1 --2- 172.21.15.123:0/883306495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20615e00 0x556e20620180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fefb4011630 tx=0x7fefb4011af0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.186+0000 7fefcdc23740 1 -- 172.21.15.123:0/883306495 shutdown_connections 2024-09-19T04:45:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.186+0000 7fefcdc23740 1 --2- 172.21.15.123:0/883306495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e206206c0 0x556e20622ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.186+0000 7fefcdc23740 1 --2- 172.21.15.123:0/883306495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20615e00 0x556e20620180 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.186+0000 7fefcdc23740 1 --2- 172.21.15.123:0/883306495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e206154e0 0x556e206158c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.186+0000 7fefcdc23740 1 -- 172.21.15.123:0/883306495 >> 172.21.15.123:0/883306495 conn(0x556e20611a70 msgr2=0x556e20611e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.186+0000 7fefcdc23740 1 -- 172.21.15.123:0/883306495 shutdown_connections 2024-09-19T04:45:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.186+0000 7fefcdc23740 1 -- 172.21.15.123:0/883306495 wait complete. 2024-09-19T04:45:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.186+0000 7fefcdc23740 1 -- start start 2024-09-19T04:45:57.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.186+0000 7fefcdc23740 1 Processor -- start 2024-09-19T04:45:57.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.187+0000 7fefcdc23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e206154e0 0x556e2061d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.187+0000 7fefcdc23740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e20615e00 0x556e2061daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.187+0000 7fefcdc23740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e206206c0 0x556e2061e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.187+0000 7fefcdc23740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556e20623330 con 0x556e206206c0 2024-09-19T04:45:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.187+0000 7fefcdc23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556e20503920 con 0x556e206154e0 2024-09-19T04:45:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.187+0000 7fefcdc23740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556e206234b0 con 0x556e20615e00 2024-09-19T04:45:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.187+0000 7fefcdc23740 10 client.0 _ll_register_callbacks cb 0x556e205278a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:45:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.187+0000 7fefc9200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e206206c0 0x556e2061e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.187+0000 7fefc3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e206154e0 0x556e2061d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.187+0000 7fefc3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e206154e0 0x556e2061d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:52760/0 (socket says 172.21.15.123:52760) 2024-09-19T04:45:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.187+0000 7fefc3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e20615e00 0x556e2061daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.187+0000 7fefc3e00640 1 -- 172.21.15.123:0/3188206936 learned_addr learned my addr 172.21.15.123:0/3188206936 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.187+0000 7fefc3e00640 1 -- 172.21.15.123:0/3188206936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e20615e00 msgr2=0x556e2061daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:45:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.187+0000 7fefc3e00640 1 --2- 172.21.15.123:0/3188206936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e20615e00 0x556e2061daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.188+0000 7fefc3e00640 1 -- 172.21.15.123:0/3188206936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e206206c0 msgr2=0x556e2061e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:45:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.188+0000 7fefc3e00640 1 --2- 172.21.15.123:0/3188206936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e206206c0 0x556e2061e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.188+0000 7fefc3e00640 1 -- 172.21.15.123:0/3188206936 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e202ec2b0 con 0x556e206154e0 2024-09-19T04:45:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.188+0000 7fefc3e00640 1 --2- 172.21.15.123:0/3188206936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e206154e0 0x556e2061d5b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fefb401c040 tx=0x7fefb400ffb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ced47f3a86f20 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.188+0000 7fefae000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:45:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.188+0000 7fefae000640 1 -- 172.21.15.123:0/3188206936 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4026020 con 0x556e206154e0 2024-09-19T04:45:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.188+0000 7fefcdc23740 1 -- 172.21.15.123:0/3188206936 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefb400fbd0 con 0x556e206154e0 2024-09-19T04:45:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.188+0000 7fefae000640 1 -- 172.21.15.123:0/3188206936 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fefb4009ce0 con 0x556e206154e0 2024-09-19T04:45:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.188+0000 7fefae000640 1 -- 172.21.15.123:0/3188206936 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fefb401e040 con 0x556e206154e0 2024-09-19T04:45:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.188+0000 7fefcdc23740 10 client.24887 fetch_fsmap learned FSMap version 27 2024-09-19T04:45:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.188+0000 7fefcdc23740 1 -- 172.21.15.123:0/3188206936 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e2062efe0 con 0x556e206154e0 2024-09-19T04:45:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.189+0000 7fefae000640 1 -- 172.21.15.123:0/3188206936 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb400fbd0 con 0x556e206154e0 2024-09-19T04:45:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.189+0000 7fefae000640 1 -- 172.21.15.123:0/3188206936 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fefb4021050 con 0x556e206154e0 2024-09-19T04:45:57.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.189+0000 7fefcdc23740 10 client.24887 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:45:57.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.189+0000 7fefcdc23740 10 client.24887 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T04:45:57.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.189+0000 7fefcdc23740 20 client.24887 populate_metadata read hostname 'smithi123' 2024-09-19T04:45:57.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.189+0000 7fefcdc23740 1 --2- 172.21.15.123:0/3188206936 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556e20631390 0x556e20632800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.189+0000 7fefcdc23740 4 client.24887 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:45:57.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.189+0000 7fefcdc23740 1 -- 172.21.15.123:0/3188206936 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e20588620 con 0x556e20631390 2024-09-19T04:45:57.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.189+0000 7fefc3400640 1 --2- 172.21.15.123:0/3188206936 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556e20631390 0x556e20632800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.190+0000 7fefc3400640 1 --2- 172.21.15.123:0/3188206936 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556e20631390 0x556e20632800 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=219a6396b79772c5 server_cookie=edf7c01e90f0ea8c in_seq=0 out_seq=0 2024-09-19T04:45:57.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.190+0000 7fefae000640 10 client.24887 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:45:57.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.191+0000 7fefae000640 1 -- 172.21.15.123:0/3188206936 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e20588620 con 0x556e20631390 2024-09-19T04:45:57.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.191+0000 7fefae000640 10 client.24887 handle_command_reply: tid=0 2024-09-19T04:45:57.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.191+0000 7fefcdc23740 1 client.24887 shutdown 2024-09-19T04:45:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.191+0000 7fefcdc23740 1 -- 172.21.15.123:0/3188206936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e206154e0 msgr2=0x556e2061d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.191+0000 7fefcdc23740 1 --2- 172.21.15.123:0/3188206936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e206154e0 0x556e2061d5b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fefb401c040 tx=0x7fefb400ffb0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.191+0000 7fefcdc23740 1 -- 172.21.15.123:0/3188206936 shutdown_connections 2024-09-19T04:45:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.191+0000 7fefcdc23740 1 --2- 172.21.15.123:0/3188206936 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556e20631390 0x556e20632800 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.191+0000 7fefcdc23740 1 --2- 172.21.15.123:0/3188206936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e206206c0 0x556e2061e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.191+0000 7fefcdc23740 1 --2- 172.21.15.123:0/3188206936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e20615e00 0x556e2061daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.191+0000 7fefae000640 0 client.24887 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:45:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.191+0000 7fefcdc23740 1 --2- 172.21.15.123:0/3188206936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e206154e0 0x556e2061d5b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.191+0000 7fefcdc23740 1 -- 172.21.15.123:0/3188206936 >> 172.21.15.123:0/3188206936 conn(0x556e20611a70 msgr2=0x556e20617750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.191+0000 7fefcdc23740 1 -- 172.21.15.123:0/3188206936 shutdown_connections 2024-09-19T04:45:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.191+0000 7fefcdc23740 1 -- 172.21.15.123:0/3188206936 wait complete. 2024-09-19T04:45:57.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.191+0000 7fefcdc23740 20 client.24887 trim_cache size 0 max 16384 2024-09-19T04:45:57.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.209+0000 7fefcdc23740 1 Processor -- start 2024-09-19T04:45:57.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.210+0000 7fefcdc23740 1 -- start start 2024-09-19T04:45:57.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.210+0000 7fefcdc23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e20611a70 0x556e20617640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.210+0000 7fefcdc23740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e206154e0 0x556e2061db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.210+0000 7fefcdc23740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20615e00 0x556e2061e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.210+0000 7fefcdc23740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556e20630b40 con 0x556e20615e00 2024-09-19T04:45:57.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.210+0000 7fefcdc23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556e20631210 con 0x556e20611a70 2024-09-19T04:45:57.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.210+0000 7fefcdc23740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556e206318e0 con 0x556e206154e0 2024-09-19T04:45:57.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.210+0000 7fefc9200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20615e00 0x556e2061e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.210+0000 7fefc9200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20615e00 0x556e2061e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37166/0 (socket says 172.21.15.123:37166) 2024-09-19T04:45:57.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.210+0000 7fefc9200640 1 -- 172.21.15.123:0/1287476660 learned_addr learned my addr 172.21.15.123:0/1287476660 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:57.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.210+0000 7fefc3e00640 1 --2- 172.21.15.123:0/1287476660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e20611a70 0x556e20617640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.210+0000 7fefc3400640 1 --2- 172.21.15.123:0/1287476660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e206154e0 0x556e2061db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.211+0000 7fefc3e00640 1 -- 172.21.15.123:0/1287476660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e206154e0 msgr2=0x556e2061db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:45:57.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.211+0000 7fefc3e00640 1 --2- 172.21.15.123:0/1287476660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e206154e0 0x556e2061db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.211+0000 7fefc3e00640 1 -- 172.21.15.123:0/1287476660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20615e00 msgr2=0x556e2061e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:45:57.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.211+0000 7fefc3e00640 1 --2- 172.21.15.123:0/1287476660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20615e00 0x556e2061e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.211+0000 7fefc3e00640 1 -- 172.21.15.123:0/1287476660 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e2061bb00 con 0x556e20611a70 2024-09-19T04:45:57.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.211+0000 7fefc9200640 1 --2- 172.21.15.123:0/1287476660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20615e00 0x556e2061e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:57.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.211+0000 7fefc3e00640 1 --2- 172.21.15.123:0/1287476660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e20611a70 0x556e20617640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fefb400f3c0 tx=0x7fefb4009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ede9883f463d6562 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:57.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.211+0000 7fefb8c00640 1 -- 172.21.15.123:0/1287476660 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4026040 con 0x556e20611a70 2024-09-19T04:45:57.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.211+0000 7fefb8c00640 1 -- 172.21.15.123:0/1287476660 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb4004680 con 0x556e20611a70 2024-09-19T04:45:57.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.211+0000 7fefb8c00640 1 -- 172.21.15.123:0/1287476660 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4004b60 con 0x556e20611a70 2024-09-19T04:45:57.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.212+0000 7fefcdc23740 1 -- 172.21.15.123:0/1287476660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e20611a70 msgr2=0x556e20617640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:57.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.212+0000 7fefcdc23740 1 --2- 172.21.15.123:0/1287476660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e20611a70 0x556e20617640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fefb400f3c0 tx=0x7fefb4009fd0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.212+0000 7fefcdc23740 1 -- 172.21.15.123:0/1287476660 shutdown_connections 2024-09-19T04:45:57.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.212+0000 7fefcdc23740 1 --2- 172.21.15.123:0/1287476660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20615e00 0x556e2061e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.212+0000 7fefcdc23740 1 --2- 172.21.15.123:0/1287476660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e206154e0 0x556e2061db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.212+0000 7fefcdc23740 1 --2- 172.21.15.123:0/1287476660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e20611a70 0x556e20617640 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.212+0000 7fefcdc23740 1 -- 172.21.15.123:0/1287476660 >> 172.21.15.123:0/1287476660 conn(0x556e20630760 msgr2=0x556e20616f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:57.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.212+0000 7fefcdc23740 1 -- 172.21.15.123:0/1287476660 shutdown_connections 2024-09-19T04:45:57.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.212+0000 7fefcdc23740 1 -- 172.21.15.123:0/1287476660 wait complete. 2024-09-19T04:45:57.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.212+0000 7fefcdc23740 1 -- start start 2024-09-19T04:45:57.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.212+0000 7fefcdc23740 1 Processor -- start 2024-09-19T04:45:57.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.213+0000 7fefcdc23740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20611a70 0x556e20635080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.213+0000 7fefcdc23740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e206154e0 0x556e206355c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.213+0000 7fefcdc23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e20615e00 0x556e20639700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.213+0000 7fefcdc23740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556e20632680 con 0x556e20611a70 2024-09-19T04:45:57.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.213+0000 7fefcdc23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556e20631fb0 con 0x556e20615e00 2024-09-19T04:45:57.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.213+0000 7fefcdc23740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556e20632d50 con 0x556e206154e0 2024-09-19T04:45:57.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.213+0000 7fefcdc23740 10 client.0 _ll_register_callbacks cb 0x7fefb4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:45:57.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.213+0000 7fefc3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20611a70 0x556e20635080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.213+0000 7fefc3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20611a70 0x556e20635080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37170/0 (socket says 172.21.15.123:37170) 2024-09-19T04:45:57.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.213+0000 7fefc3e00640 1 -- 172.21.15.123:0/3741517659 learned_addr learned my addr 172.21.15.123:0/3741517659 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:57.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.213+0000 7fefc9200640 1 --2- 172.21.15.123:0/3741517659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e20615e00 0x556e20639700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.213+0000 7fefc3400640 1 --2- 172.21.15.123:0/3741517659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e206154e0 0x556e206355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.214+0000 7fefc3e00640 1 -- 172.21.15.123:0/3741517659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e206154e0 msgr2=0x556e206355c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:45:57.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.214+0000 7fefc3e00640 1 --2- 172.21.15.123:0/3741517659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e206154e0 0x556e206355c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.214+0000 7fefc3e00640 1 -- 172.21.15.123:0/3741517659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e20615e00 msgr2=0x556e20639700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:45:57.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.214+0000 7fefc3e00640 1 --2- 172.21.15.123:0/3741517659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e20615e00 0x556e20639700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.214+0000 7fefc3e00640 1 -- 172.21.15.123:0/3741517659 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e20639da0 con 0x556e20611a70 2024-09-19T04:45:57.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.214+0000 7fefc3e00640 1 --2- 172.21.15.123:0/3741517659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20611a70 0x556e20635080 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fefb402f040 tx=0x7fefb4004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202f9dea2b9307f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:57.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.214+0000 7fefae000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:45:57.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.214+0000 7fefae000640 1 -- 172.21.15.123:0/3741517659 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4026030 con 0x556e20611a70 2024-09-19T04:45:57.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.214+0000 7fefcdc23740 1 -- 172.21.15.123:0/3741517659 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefb4004680 con 0x556e20611a70 2024-09-19T04:45:57.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.214+0000 7fefae000640 1 -- 172.21.15.123:0/3741517659 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fefb4005250 con 0x556e20611a70 2024-09-19T04:45:57.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.214+0000 7fefae000640 1 -- 172.21.15.123:0/3741517659 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fefb401ae60 con 0x556e20611a70 2024-09-19T04:45:57.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.214+0000 7fefcdc23740 10 client.14874 fetch_fsmap learned FSMap version 27 2024-09-19T04:45:57.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.214+0000 7fefcdc23740 1 -- 172.21.15.123:0/3741517659 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e2069c490 con 0x556e20611a70 2024-09-19T04:45:57.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.215+0000 7fefae000640 1 -- 172.21.15.123:0/3741517659 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb4004680 con 0x556e20611a70 2024-09-19T04:45:57.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.215+0000 7fefae000640 1 -- 172.21.15.123:0/3741517659 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fefb4021040 con 0x556e20611a70 2024-09-19T04:45:57.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.215+0000 7fefcdc23740 10 client.14874 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:45:57.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.215+0000 7fefcdc23740 10 client.14874 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T04:45:57.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.215+0000 7fefcdc23740 20 client.14874 populate_metadata read hostname 'smithi123' 2024-09-19T04:45:57.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.215+0000 7fefcdc23740 1 --2- 172.21.15.123:0/3741517659 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556e206206c0 0x556e2069f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.215+0000 7fefcdc23740 4 client.14874 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:45:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.215+0000 7fefcdc23740 1 -- 172.21.15.123:0/3741517659 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "status"}) -- 0x556e20582b30 con 0x556e206206c0 2024-09-19T04:45:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.215+0000 7fefc3400640 1 --2- 172.21.15.123:0/3741517659 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556e206206c0 0x556e2069f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.216+0000 7fefc3400640 1 --2- 172.21.15.123:0/3741517659 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556e206206c0 0x556e2069f9b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6852f38e9a9e05f2 server_cookie=b9a55827c5eabf2e in_seq=0 out_seq=0 2024-09-19T04:45:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.216+0000 7fefae000640 10 client.14874 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:45:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.216+0000 7fefae000640 1 -- 172.21.15.123:0/3741517659 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x556e20582b30 con 0x556e206206c0 2024-09-19T04:45:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.216+0000 7fefae000640 10 client.14874 handle_command_reply: tid=0 2024-09-19T04:45:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.216+0000 7fefcdc23740 1 client.14874 shutdown 2024-09-19T04:45:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.216+0000 7fefcdc23740 1 -- 172.21.15.123:0/3741517659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20611a70 msgr2=0x556e20635080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.216+0000 7fefcdc23740 1 --2- 172.21.15.123:0/3741517659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20611a70 0x556e20635080 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fefb402f040 tx=0x7fefb4004df0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.216+0000 7fefcdc23740 1 -- 172.21.15.123:0/3741517659 shutdown_connections 2024-09-19T04:45:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.216+0000 7fefcdc23740 1 --2- 172.21.15.123:0/3741517659 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556e206206c0 0x556e2069f9b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.216+0000 7fefcdc23740 1 --2- 172.21.15.123:0/3741517659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e20615e00 0x556e20639700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.216+0000 7fefcdc23740 1 --2- 172.21.15.123:0/3741517659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e206154e0 0x556e206355c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.216+0000 7fefcdc23740 1 --2- 172.21.15.123:0/3741517659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e20611a70 0x556e20635080 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.216+0000 7fefcdc23740 1 -- 172.21.15.123:0/3741517659 >> 172.21.15.123:0/3741517659 conn(0x556e20630760 msgr2=0x556e20617750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.216+0000 7fefae000640 0 client.14874 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:45:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.217+0000 7fefcdc23740 1 -- 172.21.15.123:0/3741517659 shutdown_connections 2024-09-19T04:45:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.217+0000 7fefcdc23740 1 -- 172.21.15.123:0/3741517659 wait complete. 2024-09-19T04:45:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.217+0000 7fefcdc23740 20 client.14874 trim_cache size 0 max 16384 2024-09-19T04:45:57.222 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:45:57.222 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "911fa052-7640-11ef-bceb-c7b262605968", 2024-09-19T04:45:57.222 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 4, 2024-09-19T04:45:57.222 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24499, 2024-09-19T04:45:57.222 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-09-19T04:45:57.223 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-09-19T04:45:57.223 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-09-19T04:45:57.223 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 73.389081722, 2024-09-19T04:45:57.223 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 27, 2024-09-19T04:45:57.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 86, 2024-09-19T04:45:57.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T04:45:57.223 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 116.054553614 2024-09-19T04:45:57.223 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:45:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.217+0000 7fefc9c00640 1 -- 172.21.15.123:0/537967115 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fefa80a3350 msgr2=0x7fefa80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.217+0000 7fefc9c00640 1 --2- 172.21.15.123:0/537967115 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fefa80a3350 0x7fefa80a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fefbc006010 tx=0x7fefbc005d80 comp rx=0 tx=0).stop 2024-09-19T04:45:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.217+0000 7fefc9c00640 1 -- 172.21.15.123:0/537967115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc41047e0 msgr2=0x7fefc419a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.217+0000 7fefc9c00640 1 --2- 172.21.15.123:0/537967115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc41047e0 0x7fefc419a930 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fefb000e9f0 tx=0x7fefb000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.217+0000 7fefc9c00640 1 -- 172.21.15.123:0/537967115 shutdown_connections 2024-09-19T04:45:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.217+0000 7fefc9c00640 1 --2- 172.21.15.123:0/537967115 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fefa80a3350 0x7fefa80a5810 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.217+0000 7fefc9c00640 1 --2- 172.21.15.123:0/537967115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc4108f60 0x7fefc419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.217+0000 7fefc9c00640 1 --2- 172.21.15.123:0/537967115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc41047e0 0x7fefc419a930 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.217+0000 7fefc9c00640 1 --2- 172.21.15.123:0/537967115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc4103e30 0x7fefc419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.217+0000 7fefc9c00640 1 -- 172.21.15.123:0/537967115 >> 172.21.15.123:0/537967115 conn(0x7fefc40ff8d0 msgr2=0x7fefc4100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.218+0000 7fefc9c00640 1 -- 172.21.15.123:0/537967115 shutdown_connections 2024-09-19T04:45:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:57.218+0000 7fefc9c00640 1 -- 172.21.15.123:0/537967115 wait complete. 2024-09-19T04:45:57.951 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T04:45:58.320 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:45:58.774 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:45:58 smithi123 ceph-mon[30511]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-19T04:45:58.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:45:58 smithi183 ceph-mon[35333]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-19T04:45:58.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.884+0000 7f6c4d600640 1 -- 172.21.15.123:0/2444463678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c48106f10 msgr2=0x7f6c4810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.884+0000 7f6c4d600640 1 --2- 172.21.15.123:0/2444463678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c48106f10 0x7f6c4810ea90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009ca0 tx=0x7f6c4002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.884+0000 7f6c4d600640 1 -- 172.21.15.123:0/2444463678 shutdown_connections 2024-09-19T04:45:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.884+0000 7f6c4d600640 1 --2- 172.21.15.123:0/2444463678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c48106f10 0x7f6c4810ea90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.884+0000 7f6c4d600640 1 --2- 172.21.15.123:0/2444463678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c48102790 0x7f6c481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.884+0000 7f6c4d600640 1 --2- 172.21.15.123:0/2444463678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c48101de0 0x7f6c481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.884+0000 7f6c4d600640 1 -- 172.21.15.123:0/2444463678 >> 172.21.15.123:0/2444463678 conn(0x7f6c480fd8c0 msgr2=0x7f6c480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:58.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.885+0000 7f6c4d600640 1 -- 172.21.15.123:0/2444463678 shutdown_connections 2024-09-19T04:45:58.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.885+0000 7f6c4d600640 1 -- 172.21.15.123:0/2444463678 wait complete. 2024-09-19T04:45:58.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.886+0000 7f6c4d600640 1 Processor -- start 2024-09-19T04:45:58.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.886+0000 7f6c4d600640 1 -- start start 2024-09-19T04:45:58.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.886+0000 7f6c4d600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c48101de0 0x7f6c4819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.886+0000 7f6c4d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c48102790 0x7f6c4819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.886+0000 7f6c4d600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c48106f10 0x7f6c4819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.886+0000 7f6c4d600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c48111fc0 con 0x7f6c48102790 2024-09-19T04:45:58.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.886+0000 7f6c4d600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6c48111e40 con 0x7f6c48101de0 2024-09-19T04:45:58.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.886+0000 7f6c4d600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6c48112140 con 0x7f6c48106f10 2024-09-19T04:45:58.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.886+0000 7f6c46000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c48102790 0x7f6c4819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.886+0000 7f6c46000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c48102790 0x7f6c4819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37184/0 (socket says 172.21.15.123:37184) 2024-09-19T04:45:58.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.886+0000 7f6c46000640 1 -- 172.21.15.123:0/3168856987 learned_addr learned my addr 172.21.15.123:0/3168856987 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:58.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.887+0000 7f6c46a00640 1 --2- 172.21.15.123:0/3168856987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c48101de0 0x7f6c4819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.887+0000 7f6c47400640 1 --2- 172.21.15.123:0/3168856987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c48106f10 0x7f6c4819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.887+0000 7f6c46000640 1 -- 172.21.15.123:0/3168856987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c48106f10 msgr2=0x7f6c4819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:58.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.887+0000 7f6c46000640 1 --2- 172.21.15.123:0/3168856987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c48106f10 0x7f6c4819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.887+0000 7f6c46000640 1 -- 172.21.15.123:0/3168856987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c48101de0 msgr2=0x7f6c4819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:58.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.887+0000 7f6c46000640 1 --2- 172.21.15.123:0/3168856987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c48101de0 0x7f6c4819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.887+0000 7f6c46000640 1 -- 172.21.15.123:0/3168856987 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4819f590 con 0x7f6c48102790 2024-09-19T04:45:58.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.887+0000 7f6c46a00640 1 --2- 172.21.15.123:0/3168856987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c48101de0 0x7f6c4819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:58.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.887+0000 7f6c46000640 1 --2- 172.21.15.123:0/3168856987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c48102790 0x7f6c4819ab20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400d950 tx=0x7f6c3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:58.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.887+0000 7f6c3f400640 1 -- 172.21.15.123:0/3168856987 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34017030 con 0x7f6c48102790 2024-09-19T04:45:58.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.888+0000 7f6c3f400640 1 -- 172.21.15.123:0/3168856987 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c34004020 con 0x7f6c48102790 2024-09-19T04:45:58.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.888+0000 7f6c4d600640 1 -- 172.21.15.123:0/3168856987 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4819f880 con 0x7f6c48102790 2024-09-19T04:45:58.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.888+0000 7f6c3f400640 1 -- 172.21.15.123:0/3168856987 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c340047a0 con 0x7f6c48102790 2024-09-19T04:45:58.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.888+0000 7f6c4d600640 1 -- 172.21.15.123:0/3168856987 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c481a7160 con 0x7f6c48102790 2024-09-19T04:45:58.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.891+0000 7f6c500d9740 1 Processor -- start 2024-09-19T04:45:58.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.891+0000 7f6c500d9740 1 -- start start 2024-09-19T04:45:58.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.891+0000 7f6c3f400640 1 -- 172.21.15.123:0/3168856987 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6c340041d0 con 0x7f6c48102790 2024-09-19T04:45:58.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.891+0000 7f6c500d9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988330 0x55ad95988710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.892+0000 7f6c500d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95988c50 0x55ad95992fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.892+0000 7f6c500d9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 0x55ad95995900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.892+0000 7f6c500d9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ad958ee0b0 con 0x55ad95993510 2024-09-19T04:45:58.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.892+0000 7f6c500d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ad95895ab0 con 0x55ad95988c50 2024-09-19T04:45:58.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.892+0000 7f6c500d9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ad958ea070 con 0x55ad95988330 2024-09-19T04:45:58.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.892+0000 7f6c46000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 0x55ad95995900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.892+0000 7f6c46a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95988c50 0x55ad95992fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.892+0000 7f6c47400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988330 0x55ad95988710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.892+0000 7f6c46000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 0x55ad95995900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37190/0 (socket says 172.21.15.123:37190) 2024-09-19T04:45:58.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.892+0000 7f6c46000640 1 -- 172.21.15.123:0/2612746417 learned_addr learned my addr 172.21.15.123:0/2612746417 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:58.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.892+0000 7f6c47400640 1 -- 172.21.15.123:0/2612746417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95988c50 msgr2=0x55ad95992fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:45:58.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.892+0000 7f6c47400640 1 --2- 172.21.15.123:0/2612746417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95988c50 0x55ad95992fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.892+0000 7f6c47400640 1 -- 172.21.15.123:0/2612746417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 msgr2=0x55ad95995900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:45:58.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.892+0000 7f6c47400640 1 --2- 172.21.15.123:0/2612746417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 0x55ad95995900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.892+0000 7f6c47400640 1 -- 172.21.15.123:0/2612746417 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad958a24a0 con 0x55ad95988330 2024-09-19T04:45:58.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.893+0000 7f6c47400640 1 --2- 172.21.15.123:0/2612746417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988330 0x55ad95988710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6c40005e30 tx=0x7f6c4002f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de9d9f6e82291647 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:58.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.893+0000 7f6c33400640 1 -- 172.21.15.123:0/2612746417 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40003970 con 0x55ad95988330 2024-09-19T04:45:58.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.893+0000 7f6c33400640 1 -- 172.21.15.123:0/2612746417 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c40036030 con 0x55ad95988330 2024-09-19T04:45:58.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.893+0000 7f6c33400640 1 -- 172.21.15.123:0/2612746417 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40004460 con 0x55ad95988330 2024-09-19T04:45:58.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.893+0000 7f6c3f400640 1 --2- 172.21.15.123:0/3168856987 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6c2c0a32d0 0x7f6c2c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.893+0000 7f6c3f400640 1 -- 172.21.15.123:0/3168856987 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f6c340cf3e0 con 0x7f6c48102790 2024-09-19T04:45:58.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.893+0000 7f6c46a00640 1 --2- 172.21.15.123:0/3168856987 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6c2c0a32d0 0x7f6c2c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.894+0000 7f6c500d9740 1 -- 172.21.15.123:0/2612746417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988330 msgr2=0x55ad95988710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:58.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.894+0000 7f6c500d9740 1 --2- 172.21.15.123:0/2612746417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988330 0x55ad95988710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6c40005e30 tx=0x7f6c4002f3c0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.894+0000 7f6c500d9740 1 -- 172.21.15.123:0/2612746417 shutdown_connections 2024-09-19T04:45:58.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.894+0000 7f6c500d9740 1 --2- 172.21.15.123:0/2612746417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 0x55ad95995900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.894+0000 7f6c500d9740 1 --2- 172.21.15.123:0/2612746417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95988c50 0x55ad95992fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.894+0000 7f6c500d9740 1 --2- 172.21.15.123:0/2612746417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988330 0x55ad95988710 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.894+0000 7f6c500d9740 1 -- 172.21.15.123:0/2612746417 >> 172.21.15.123:0/2612746417 conn(0x55ad959848c0 msgr2=0x55ad95984ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:58.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.894+0000 7f6c46a00640 1 --2- 172.21.15.123:0/3168856987 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6c2c0a32d0 0x7f6c2c0a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55ad95985720 tx=0x7f6c3800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:58.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.894+0000 7f6c500d9740 1 -- 172.21.15.123:0/2612746417 shutdown_connections 2024-09-19T04:45:58.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.894+0000 7f6c500d9740 1 -- 172.21.15.123:0/2612746417 wait complete. 2024-09-19T04:45:58.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.895+0000 7f6c500d9740 1 -- start start 2024-09-19T04:45:58.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.895+0000 7f6c500d9740 1 Processor -- start 2024-09-19T04:45:58.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c500d9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988330 0x55ad95990440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c500d9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95988c50 0x55ad95990980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c500d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95993510 0x55ad95990ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c500d9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ad95996000 con 0x55ad95988c50 2024-09-19T04:45:58.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c500d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ad95876920 con 0x55ad95993510 2024-09-19T04:45:58.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c500d9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ad95996180 con 0x55ad95988330 2024-09-19T04:45:58.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c500d9740 10 client.0 _ll_register_callbacks cb 0x55ad9589a8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:45:58.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c46000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95988c50 0x55ad95990980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c46000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95988c50 0x55ad95990980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37206/0 (socket says 172.21.15.123:37206) 2024-09-19T04:45:58.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c46000640 1 -- 172.21.15.123:0/303908645 learned_addr learned my addr 172.21.15.123:0/303908645 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:58.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c46a00640 1 --2- 172.21.15.123:0/303908645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988330 0x55ad95990440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c47400640 1 --2- 172.21.15.123:0/303908645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95993510 0x55ad95990ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c46000640 1 -- 172.21.15.123:0/303908645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988330 msgr2=0x55ad95990440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:45:58.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c46000640 1 --2- 172.21.15.123:0/303908645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988330 0x55ad95990440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c46000640 1 -- 172.21.15.123:0/303908645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95993510 msgr2=0x55ad95990ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:45:58.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c46000640 1 --2- 172.21.15.123:0/303908645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95993510 0x55ad95990ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c46000640 1 -- 172.21.15.123:0/303908645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad9565f2b0 con 0x55ad95988c50 2024-09-19T04:45:58.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.896+0000 7f6c47400640 1 --2- 172.21.15.123:0/303908645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95993510 0x55ad95990ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:45:58.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.897+0000 7f6c46000640 1 --2- 172.21.15.123:0/303908645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95988c50 0x55ad95990980 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6c340cf680 tx=0x7f6c340974b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704f579110c03e5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:58.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.897+0000 7f6c27e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:45:58.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.897+0000 7f6c27e00640 1 -- 172.21.15.123:0/303908645 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3409b620 con 0x55ad95988c50 2024-09-19T04:45:58.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.897+0000 7f6c27e00640 1 -- 172.21.15.123:0/303908645 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f6c34084ba0 con 0x55ad95988c50 2024-09-19T04:45:58.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.897+0000 7f6c500d9740 1 -- 172.21.15.123:0/303908645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c40036030 con 0x55ad95988c50 2024-09-19T04:45:58.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.897+0000 7f6c27e00640 1 -- 172.21.15.123:0/303908645 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c3409b1b0 con 0x55ad95988c50 2024-09-19T04:45:58.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.897+0000 7f6c500d9740 10 client.14886 fetch_fsmap learned FSMap version 27 2024-09-19T04:45:58.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.897+0000 7f6c500d9740 1 -- 172.21.15.123:0/303908645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ad959a1c50 con 0x55ad95988c50 2024-09-19T04:45:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.897+0000 7f6c27e00640 1 -- 172.21.15.123:0/303908645 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c40036030 con 0x55ad95988c50 2024-09-19T04:45:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.897+0000 7f6c27e00640 1 -- 172.21.15.123:0/303908645 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6c3409cb80 con 0x55ad95988c50 2024-09-19T04:45:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.898+0000 7f6c500d9740 10 client.14886 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:45:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.898+0000 7f6c500d9740 10 client.14886 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T04:45:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.898+0000 7f6c500d9740 20 client.14886 populate_metadata read hostname 'smithi123' 2024-09-19T04:45:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.898+0000 7f6c500d9740 1 --2- 172.21.15.123:0/303908645 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55ad959a4220 0x55ad959a5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.898+0000 7f6c500d9740 4 client.14886 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:45:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.898+0000 7f6c500d9740 1 -- 172.21.15.123:0/303908645 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad958fb620 con 0x55ad959a4220 2024-09-19T04:45:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.898+0000 7f6c46a00640 1 --2- 172.21.15.123:0/303908645 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55ad959a4220 0x55ad959a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.899+0000 7f6c46a00640 1 --2- 172.21.15.123:0/303908645 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55ad959a4220 0x55ad959a5690 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bda610e04ca6921e server_cookie=82ea0abf1b292a0a in_seq=0 out_seq=0 2024-09-19T04:45:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.899+0000 7f6c27e00640 10 client.14886 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:45:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.899+0000 7f6c27e00640 1 -- 172.21.15.123:0/303908645 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad958fb620 con 0x55ad959a4220 2024-09-19T04:45:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.899+0000 7f6c27e00640 10 client.14886 handle_command_reply: tid=0 2024-09-19T04:45:58.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.899+0000 7f6c500d9740 1 client.14886 shutdown 2024-09-19T04:45:58.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.900+0000 7f6c500d9740 1 -- 172.21.15.123:0/303908645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95988c50 msgr2=0x55ad95990980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:58.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.900+0000 7f6c500d9740 1 --2- 172.21.15.123:0/303908645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95988c50 0x55ad95990980 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6c340cf680 tx=0x7f6c340974b0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.900+0000 7f6c500d9740 1 -- 172.21.15.123:0/303908645 shutdown_connections 2024-09-19T04:45:58.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.900+0000 7f6c500d9740 1 --2- 172.21.15.123:0/303908645 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55ad959a4220 0x55ad959a5690 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.900+0000 7f6c500d9740 1 --2- 172.21.15.123:0/303908645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95993510 0x55ad95990ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.900+0000 7f6c27e00640 0 client.14886 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:45:58.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.900+0000 7f6c500d9740 1 --2- 172.21.15.123:0/303908645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95988c50 0x55ad95990980 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.900+0000 7f6c500d9740 1 --2- 172.21.15.123:0/303908645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988330 0x55ad95990440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.900+0000 7f6c500d9740 1 -- 172.21.15.123:0/303908645 >> 172.21.15.123:0/303908645 conn(0x55ad959848c0 msgr2=0x55ad9598a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:58.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.900+0000 7f6c500d9740 1 -- 172.21.15.123:0/303908645 shutdown_connections 2024-09-19T04:45:58.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.900+0000 7f6c500d9740 1 -- 172.21.15.123:0/303908645 wait complete. 2024-09-19T04:45:58.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.900+0000 7f6c500d9740 20 client.14886 trim_cache size 0 max 16384 2024-09-19T04:45:58.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.918+0000 7f6c500d9740 1 Processor -- start 2024-09-19T04:45:58.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.919+0000 7f6c500d9740 1 -- start start 2024-09-19T04:45:58.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.919+0000 7f6c500d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95988330 0x55ad9598fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.919+0000 7f6c500d9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988c50 0x55ad9598ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.919+0000 7f6c500d9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 0x55ad9598f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.919+0000 7f6c500d9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ad959a39d0 con 0x55ad95993510 2024-09-19T04:45:58.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.919+0000 7f6c500d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ad959a40a0 con 0x55ad95988330 2024-09-19T04:45:58.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.919+0000 7f6c500d9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ad959a4770 con 0x55ad95988c50 2024-09-19T04:45:58.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.919+0000 7f6c47400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 0x55ad9598f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.919+0000 7f6c47400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 0x55ad9598f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37210/0 (socket says 172.21.15.123:37210) 2024-09-19T04:45:58.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.919+0000 7f6c47400640 1 -- 172.21.15.123:0/95791659 learned_addr learned my addr 172.21.15.123:0/95791659 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:58.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.919+0000 7f6c46a00640 1 --2- 172.21.15.123:0/95791659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95988330 0x55ad9598fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.919+0000 7f6c46000640 1 --2- 172.21.15.123:0/95791659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988c50 0x55ad9598ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.920+0000 7f6c47400640 1 -- 172.21.15.123:0/95791659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988c50 msgr2=0x55ad9598ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:45:58.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.920+0000 7f6c47400640 1 --2- 172.21.15.123:0/95791659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988c50 0x55ad9598ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.920+0000 7f6c47400640 1 -- 172.21.15.123:0/95791659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95988330 msgr2=0x55ad9598fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:45:58.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.920+0000 7f6c47400640 1 --2- 172.21.15.123:0/95791659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95988330 0x55ad9598fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.920+0000 7f6c47400640 1 -- 172.21.15.123:0/95791659 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad9598e9b0 con 0x55ad95993510 2024-09-19T04:45:58.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.920+0000 7f6c46a00640 1 --2- 172.21.15.123:0/95791659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95988330 0x55ad9598fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:45:58.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.920+0000 7f6c47400640 1 --2- 172.21.15.123:0/95791659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 0x55ad9598f270 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6c40005e30 tx=0x7f6c40000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46053fbeed5010e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:58.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.920+0000 7f6c32a00640 1 -- 172.21.15.123:0/95791659 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40031cf0 con 0x55ad95993510 2024-09-19T04:45:58.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.920+0000 7f6c32a00640 1 -- 172.21.15.123:0/95791659 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c40036040 con 0x55ad95993510 2024-09-19T04:45:58.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.920+0000 7f6c32a00640 1 -- 172.21.15.123:0/95791659 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c400042a0 con 0x55ad95993510 2024-09-19T04:45:58.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.921+0000 7f6c500d9740 1 -- 172.21.15.123:0/95791659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 msgr2=0x55ad9598f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:58.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.921+0000 7f6c500d9740 1 --2- 172.21.15.123:0/95791659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 0x55ad9598f270 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6c40005e30 tx=0x7f6c40000f40 comp rx=0 tx=0).stop 2024-09-19T04:45:58.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.921+0000 7f6c500d9740 1 -- 172.21.15.123:0/95791659 shutdown_connections 2024-09-19T04:45:58.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.921+0000 7f6c500d9740 1 --2- 172.21.15.123:0/95791659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 0x55ad9598f270 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.921+0000 7f6c500d9740 1 --2- 172.21.15.123:0/95791659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988c50 0x55ad9598ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.921+0000 7f6c500d9740 1 --2- 172.21.15.123:0/95791659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95988330 0x55ad9598fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.921+0000 7f6c500d9740 1 -- 172.21.15.123:0/95791659 >> 172.21.15.123:0/95791659 conn(0x55ad959848c0 msgr2=0x55ad95984ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:58.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.921+0000 7f6c500d9740 1 -- 172.21.15.123:0/95791659 shutdown_connections 2024-09-19T04:45:58.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.921+0000 7f6c500d9740 1 -- 172.21.15.123:0/95791659 wait complete. 2024-09-19T04:45:58.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.922+0000 7f6c500d9740 1 -- start start 2024-09-19T04:45:58.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.922+0000 7f6c500d9740 1 Processor -- start 2024-09-19T04:45:58.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.922+0000 7f6c500d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95988330 0x55ad959ade10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.922+0000 7f6c500d9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988c50 0x55ad9598d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.922+0000 7f6c500d9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 0x55ad9598d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.922+0000 7f6c500d9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ad959a5510 con 0x55ad95993510 2024-09-19T04:45:58.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.922+0000 7f6c500d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ad959a4e40 con 0x55ad95988330 2024-09-19T04:45:58.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.922+0000 7f6c500d9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ad959a5be0 con 0x55ad95988c50 2024-09-19T04:45:58.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.922+0000 7f6c500d9740 10 client.0 _ll_register_callbacks cb 0x7f6c340d2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:45:58.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.923+0000 7f6c47400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 0x55ad9598d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.923+0000 7f6c46a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95988330 0x55ad959ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.923+0000 7f6c47400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 0x55ad9598d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37218/0 (socket says 172.21.15.123:37218) 2024-09-19T04:45:58.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.923+0000 7f6c47400640 1 -- 172.21.15.123:0/1190046779 learned_addr learned my addr 172.21.15.123:0/1190046779 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:45:58.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.923+0000 7f6c46000640 1 --2- 172.21.15.123:0/1190046779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988c50 0x55ad9598d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.923+0000 7f6c47400640 1 -- 172.21.15.123:0/1190046779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988c50 msgr2=0x55ad9598d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:45:58.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.923+0000 7f6c47400640 1 --2- 172.21.15.123:0/1190046779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988c50 0x55ad9598d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.923+0000 7f6c47400640 1 -- 172.21.15.123:0/1190046779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95988330 msgr2=0x55ad959ade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:45:58.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.923+0000 7f6c47400640 1 --2- 172.21.15.123:0/1190046779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95988330 0x55ad959ade10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.923+0000 7f6c47400640 1 -- 172.21.15.123:0/1190046779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad9598db10 con 0x55ad95993510 2024-09-19T04:45:58.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.923+0000 7f6c47400640 1 --2- 172.21.15.123:0/1190046779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 0x55ad9598d5d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6c4002ade0 tx=0x7f6c40031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2485b2b4f950543 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:45:58.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.923+0000 7f6c27e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:45:58.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.923+0000 7f6c27e00640 1 -- 172.21.15.123:0/1190046779 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40003d80 con 0x55ad95993510 2024-09-19T04:45:58.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.923+0000 7f6c27e00640 1 -- 172.21.15.123:0/1190046779 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f6c40004040 con 0x55ad95993510 2024-09-19T04:45:58.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.924+0000 7f6c500d9740 1 -- 172.21.15.123:0/1190046779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c40036040 con 0x55ad95993510 2024-09-19T04:45:58.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.924+0000 7f6c27e00640 1 -- 172.21.15.123:0/1190046779 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c40004240 con 0x55ad95993510 2024-09-19T04:45:58.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.924+0000 7f6c500d9740 10 client.14892 fetch_fsmap learned FSMap version 27 2024-09-19T04:45:58.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.924+0000 7f6c500d9740 1 -- 172.21.15.123:0/1190046779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ad959ac6b0 con 0x55ad95993510 2024-09-19T04:45:58.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.924+0000 7f6c27e00640 1 -- 172.21.15.123:0/1190046779 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c40036040 con 0x55ad95993510 2024-09-19T04:45:58.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.924+0000 7f6c27e00640 1 -- 172.21.15.123:0/1190046779 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6c40043070 con 0x55ad95993510 2024-09-19T04:45:58.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.924+0000 7f6c500d9740 10 client.14892 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:45:58.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.924+0000 7f6c500d9740 10 client.14892 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T04:45:58.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.924+0000 7f6c500d9740 20 client.14892 populate_metadata read hostname 'smithi123' 2024-09-19T04:45:58.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.924+0000 7f6c500d9740 1 --2- 172.21.15.123:0/1190046779 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55ad95a12ed0 0x55ad95a11be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:45:58.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.924+0000 7f6c500d9740 4 client.14892 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:45:58.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.924+0000 7f6c500d9740 1 -- 172.21.15.123:0/1190046779 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "status"}) -- 0x55ad958f5b30 con 0x55ad95a12ed0 2024-09-19T04:45:58.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.925+0000 7f6c46a00640 1 --2- 172.21.15.123:0/1190046779 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55ad95a12ed0 0x55ad95a11be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:45:58.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.925+0000 7f6c46a00640 1 --2- 172.21.15.123:0/1190046779 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55ad95a12ed0 0x55ad95a11be0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ac49c1ed0b91136c server_cookie=87842f405065bf0c in_seq=0 out_seq=0 2024-09-19T04:45:58.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.925+0000 7f6c27e00640 10 client.14892 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:45:58.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.925+0000 7f6c27e00640 1 -- 172.21.15.123:0/1190046779 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ad958f5b30 con 0x55ad95a12ed0 2024-09-19T04:45:58.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.926+0000 7f6c27e00640 10 client.14892 handle_command_reply: tid=0 2024-09-19T04:45:58.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.926+0000 7f6c500d9740 1 client.14892 shutdown 2024-09-19T04:45:58.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.926+0000 7f6c500d9740 1 -- 172.21.15.123:0/1190046779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 msgr2=0x55ad9598d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:58.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.926+0000 7f6c500d9740 1 --2- 172.21.15.123:0/1190046779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 0x55ad9598d5d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6c4002ade0 tx=0x7f6c40031680 comp rx=0 tx=0).stop 2024-09-19T04:45:58.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.926+0000 7f6c500d9740 1 -- 172.21.15.123:0/1190046779 shutdown_connections 2024-09-19T04:45:58.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.926+0000 7f6c500d9740 1 --2- 172.21.15.123:0/1190046779 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55ad95a12ed0 0x55ad95a11be0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.926+0000 7f6c500d9740 1 --2- 172.21.15.123:0/1190046779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad95993510 0x55ad9598d5d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.926+0000 7f6c500d9740 1 --2- 172.21.15.123:0/1190046779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad95988c50 0x55ad9598d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.926+0000 7f6c27e00640 0 client.14892 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:45:58.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.926+0000 7f6c500d9740 1 --2- 172.21.15.123:0/1190046779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad95988330 0x55ad959ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.926+0000 7f6c500d9740 1 -- 172.21.15.123:0/1190046779 >> 172.21.15.123:0/1190046779 conn(0x55ad959848c0 msgr2=0x55ad9598ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:58.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.926+0000 7f6c500d9740 1 -- 172.21.15.123:0/1190046779 shutdown_connections 2024-09-19T04:45:58.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.926+0000 7f6c500d9740 1 -- 172.21.15.123:0/1190046779 wait complete. 2024-09-19T04:45:58.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.926+0000 7f6c500d9740 20 client.14892 trim_cache size 0 max 16384 2024-09-19T04:45:58.937 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:45:58.937 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "911fa052-7640-11ef-bceb-c7b262605968", 2024-09-19T04:45:58.937 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 3, 2024-09-19T04:45:58.937 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24500, 2024-09-19T04:45:58.938 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-09-19T04:45:58.938 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-09-19T04:45:58.938 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-09-19T04:45:58.938 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 76.106175132000004, 2024-09-19T04:45:58.938 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 27, 2024-09-19T04:45:58.938 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 86, 2024-09-19T04:45:58.938 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T04:45:58.938 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 112.45260555599999 2024-09-19T04:45:58.938 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:45:58.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.927+0000 7f6c4d600640 1 -- 172.21.15.123:0/3168856987 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6c2c0a32d0 msgr2=0x7f6c2c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:58.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.927+0000 7f6c4d600640 1 --2- 172.21.15.123:0/3168856987 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6c2c0a32d0 0x7f6c2c0a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55ad95985720 tx=0x7f6c3800b410 comp rx=0 tx=0).stop 2024-09-19T04:45:58.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.927+0000 7f6c4d600640 1 -- 172.21.15.123:0/3168856987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c48102790 msgr2=0x7f6c4819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:45:58.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.927+0000 7f6c4d600640 1 --2- 172.21.15.123:0/3168856987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c48102790 0x7f6c4819ab20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400d950 tx=0x7f6c3400de10 comp rx=0 tx=0).stop 2024-09-19T04:45:58.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.927+0000 7f6c4d600640 1 -- 172.21.15.123:0/3168856987 shutdown_connections 2024-09-19T04:45:58.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.927+0000 7f6c4d600640 1 --2- 172.21.15.123:0/3168856987 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f6c2c0a32d0 0x7f6c2c0a5790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.927+0000 7f6c4d600640 1 --2- 172.21.15.123:0/3168856987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c48106f10 0x7f6c4819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.927+0000 7f6c4d600640 1 --2- 172.21.15.123:0/3168856987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c48102790 0x7f6c4819ab20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.927+0000 7f6c4d600640 1 --2- 172.21.15.123:0/3168856987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c48101de0 0x7f6c4819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:45:58.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.927+0000 7f6c4d600640 1 -- 172.21.15.123:0/3168856987 >> 172.21.15.123:0/3168856987 conn(0x7f6c480fd8c0 msgr2=0x7f6c480fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:45:58.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.927+0000 7f6c4d600640 1 -- 172.21.15.123:0/3168856987 shutdown_connections 2024-09-19T04:45:58.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:45:58.927+0000 7f6c4d600640 1 -- 172.21.15.123:0/3168856987 wait complete. 2024-09-19T04:45:58.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:45:58 smithi193 ceph-mon[35592]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-19T04:45:59.708 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T04:46:00.078 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:00.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.640+0000 7f86ba000640 1 -- 172.21.15.123:0/2013462954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86b4074eb0 msgr2=0x7f86b4112490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:00.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.640+0000 7f86ba000640 1 --2- 172.21.15.123:0/2013462954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86b4074eb0 0x7f86b4112490 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f869c009900 tx=0x7f869c02a480 comp rx=0 tx=0).stop 2024-09-19T04:46:00.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.640+0000 7f86ba000640 1 -- 172.21.15.123:0/2013462954 shutdown_connections 2024-09-19T04:46:00.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.640+0000 7f86ba000640 1 --2- 172.21.15.123:0/2013462954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86b41129d0 0x7f86b4114dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.640+0000 7f86ba000640 1 --2- 172.21.15.123:0/2013462954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86b4074eb0 0x7f86b4112490 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.640+0000 7f86ba000640 1 --2- 172.21.15.123:0/2013462954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86b4074590 0x7f86b4074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.640+0000 7f86ba000640 1 -- 172.21.15.123:0/2013462954 >> 172.21.15.123:0/2013462954 conn(0x7f86b40fd6e0 msgr2=0x7f86b40ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:00.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.640+0000 7f86ba000640 1 -- 172.21.15.123:0/2013462954 shutdown_connections 2024-09-19T04:46:00.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.640+0000 7f86ba000640 1 -- 172.21.15.123:0/2013462954 wait complete. 2024-09-19T04:46:00.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.641+0000 7f86ba000640 1 Processor -- start 2024-09-19T04:46:00.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.642+0000 7f86ba000640 1 -- start start 2024-09-19T04:46:00.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.642+0000 7f86ba000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86b4074590 0x7f86b41a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.642+0000 7f86ba000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86b4074eb0 0x7f86b41a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.642+0000 7f86ba000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86b41129d0 0x7f86b419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.642+0000 7f86ba000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f86b41183f0 con 0x7f86b41129d0 2024-09-19T04:46:00.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.642+0000 7f86ba000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86b4118270 con 0x7f86b4074eb0 2024-09-19T04:46:00.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.642+0000 7f86ba000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f86b4118570 con 0x7f86b4074590 2024-09-19T04:46:00.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.642+0000 7f86b9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86b41129d0 0x7f86b419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.642+0000 7f86b9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86b41129d0 0x7f86b419e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37240/0 (socket says 172.21.15.123:37240) 2024-09-19T04:46:00.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.642+0000 7f86b9600640 1 -- 172.21.15.123:0/3527922632 learned_addr learned my addr 172.21.15.123:0/3527922632 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:00.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.642+0000 7f86b8c00640 1 --2- 172.21.15.123:0/3527922632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86b4074590 0x7f86b41a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.642+0000 7f86abe00640 1 --2- 172.21.15.123:0/3527922632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86b4074eb0 0x7f86b41a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.643+0000 7f86b9600640 1 -- 172.21.15.123:0/3527922632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86b4074590 msgr2=0x7f86b41a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:00.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.643+0000 7f86b9600640 1 --2- 172.21.15.123:0/3527922632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86b4074590 0x7f86b41a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.643+0000 7f86b9600640 1 -- 172.21.15.123:0/3527922632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86b4074eb0 msgr2=0x7f86b41a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:00.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.643+0000 7f86b9600640 1 --2- 172.21.15.123:0/3527922632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86b4074eb0 0x7f86b41a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.643+0000 7f86b9600640 1 -- 172.21.15.123:0/3527922632 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b419ef50 con 0x7f86b41129d0 2024-09-19T04:46:00.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.643+0000 7f86b9600640 1 --2- 172.21.15.123:0/3527922632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86b41129d0 0x7f86b419e6e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00e9d0 tx=0x7f86ac00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:00.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.643+0000 7f86b2000640 1 -- 172.21.15.123:0/3527922632 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac00cd80 con 0x7f86b41129d0 2024-09-19T04:46:00.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.643+0000 7f86b2000640 1 -- 172.21.15.123:0/3527922632 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ac005020 con 0x7f86b41129d0 2024-09-19T04:46:00.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.643+0000 7f86b2000640 1 -- 172.21.15.123:0/3527922632 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0116d0 con 0x7f86b41129d0 2024-09-19T04:46:00.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.643+0000 7f86ba000640 1 -- 172.21.15.123:0/3527922632 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b419f240 con 0x7f86b41129d0 2024-09-19T04:46:00.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.643+0000 7f86ba000640 1 -- 172.21.15.123:0/3527922632 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b41ab3c0 con 0x7f86b41129d0 2024-09-19T04:46:00.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.646+0000 7f86bdfde740 1 Processor -- start 2024-09-19T04:46:00.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.646+0000 7f86b2000640 1 -- 172.21.15.123:0/3527922632 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f86ac0051d0 con 0x7f86b41129d0 2024-09-19T04:46:00.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.646+0000 7f86bdfde740 1 -- start start 2024-09-19T04:46:00.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.647+0000 7f86bdfde740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24d4e0 0x5639aa24d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.647+0000 7f86bdfde740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24de00 0x5639aa258180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.647+0000 7f86bdfde740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa2586c0 0x5639aa25aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.647+0000 7f86bdfde740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5639aa1b30b0 con 0x5639aa24d4e0 2024-09-19T04:46:00.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.647+0000 7f86bdfde740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639aa15aab0 con 0x5639aa24de00 2024-09-19T04:46:00.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.647+0000 7f86bdfde740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5639aa1af070 con 0x5639aa2586c0 2024-09-19T04:46:00.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.647+0000 7f86abe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24d4e0 0x5639aa24d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.647+0000 7f86abe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24d4e0 0x5639aa24d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37248/0 (socket says 172.21.15.123:37248) 2024-09-19T04:46:00.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.647+0000 7f86abe00640 1 -- 172.21.15.123:0/1520942134 learned_addr learned my addr 172.21.15.123:0/1520942134 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:00.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.647+0000 7f86b8c00640 1 --2- 172.21.15.123:0/1520942134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24de00 0x5639aa258180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.647+0000 7f86abe00640 1 --2- 172.21.15.123:0/1520942134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa2586c0 0x5639aa25aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.648+0000 7f86b8c00640 1 -- 172.21.15.123:0/1520942134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa2586c0 msgr2=0x5639aa25aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:00.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.648+0000 7f86b8c00640 1 --2- 172.21.15.123:0/1520942134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa2586c0 0x5639aa25aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.648+0000 7f86b8c00640 1 -- 172.21.15.123:0/1520942134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24d4e0 msgr2=0x5639aa24d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:00.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.648+0000 7f86b8c00640 1 --2- 172.21.15.123:0/1520942134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24d4e0 0x5639aa24d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.648+0000 7f86b8c00640 1 -- 172.21.15.123:0/1520942134 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639aa1674a0 con 0x5639aa24de00 2024-09-19T04:46:00.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.648+0000 7f86abe00640 1 --2- 172.21.15.123:0/1520942134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa2586c0 0x5639aa25aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:46:00.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.648+0000 7f86b8c00640 1 --2- 172.21.15.123:0/1520942134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24de00 0x5639aa258180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f86a400bbf0 tx=0x7f86a40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=554245c00edac78d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:00.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.648+0000 7f86b2000640 1 --2- 172.21.15.123:0/3527922632 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f86980a32d0 0x7f86980a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.648+0000 7f86a9c00640 1 -- 172.21.15.123:0/1520942134 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4004030 con 0x5639aa24de00 2024-09-19T04:46:00.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.648+0000 7f86a9c00640 1 -- 172.21.15.123:0/1520942134 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a40041d0 con 0x5639aa24de00 2024-09-19T04:46:00.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.648+0000 7f86a9c00640 1 -- 172.21.15.123:0/1520942134 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a40116c0 con 0x5639aa24de00 2024-09-19T04:46:00.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.648+0000 7f86abe00640 1 --2- 172.21.15.123:0/3527922632 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f86980a32d0 0x7f86980a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.648+0000 7f86b2000640 1 -- 172.21.15.123:0/3527922632 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f86ac0cff40 con 0x7f86b41129d0 2024-09-19T04:46:00.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.649+0000 7f86bdfde740 1 -- 172.21.15.123:0/1520942134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24de00 msgr2=0x5639aa258180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:00.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.649+0000 7f86bdfde740 1 --2- 172.21.15.123:0/1520942134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24de00 0x5639aa258180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f86a400bbf0 tx=0x7f86a40107a0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.649+0000 7f86abe00640 1 --2- 172.21.15.123:0/3527922632 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f86980a32d0 0x7f86980a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f869c031550 tx=0x7f869c005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:00.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.649+0000 7f86bdfde740 1 -- 172.21.15.123:0/1520942134 shutdown_connections 2024-09-19T04:46:00.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.649+0000 7f86bdfde740 1 --2- 172.21.15.123:0/1520942134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa2586c0 0x5639aa25aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.649+0000 7f86bdfde740 1 --2- 172.21.15.123:0/1520942134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24de00 0x5639aa258180 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.649+0000 7f86bdfde740 1 --2- 172.21.15.123:0/1520942134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24d4e0 0x5639aa24d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.649+0000 7f86bdfde740 1 -- 172.21.15.123:0/1520942134 >> 172.21.15.123:0/1520942134 conn(0x5639aa249a70 msgr2=0x5639aa249e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:00.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.649+0000 7f86bdfde740 1 -- 172.21.15.123:0/1520942134 shutdown_connections 2024-09-19T04:46:00.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.650+0000 7f86bdfde740 1 -- 172.21.15.123:0/1520942134 wait complete. 2024-09-19T04:46:00.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.650+0000 7f86bdfde740 1 -- start start 2024-09-19T04:46:00.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.650+0000 7f86bdfde740 1 Processor -- start 2024-09-19T04:46:00.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.651+0000 7f86bdfde740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa24d4e0 0x5639aa2555b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.651+0000 7f86bdfde740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24de00 0x5639aa255af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.651+0000 7f86bdfde740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa2586c0 0x5639aa256030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.651+0000 7f86bdfde740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5639aa25b330 con 0x5639aa24de00 2024-09-19T04:46:00.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.651+0000 7f86bdfde740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639aa13b920 con 0x5639aa2586c0 2024-09-19T04:46:00.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.651+0000 7f86bdfde740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5639aa25b4b0 con 0x5639aa24d4e0 2024-09-19T04:46:00.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.651+0000 7f86bdfde740 10 client.0 _ll_register_callbacks cb 0x5639aa15f8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:00.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.651+0000 7f86b8c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa24d4e0 0x5639aa2555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.651+0000 7f86abe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24de00 0x5639aa255af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.651+0000 7f86b8c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa24d4e0 0x5639aa2555b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:33546/0 (socket says 172.21.15.123:33546) 2024-09-19T04:46:00.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.651+0000 7f86b8c00640 1 -- 172.21.15.123:0/2999277845 learned_addr learned my addr 172.21.15.123:0/2999277845 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:00.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.651+0000 7f86b9600640 1 --2- 172.21.15.123:0/2999277845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa2586c0 0x5639aa256030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.651+0000 7f86b8c00640 1 -- 172.21.15.123:0/2999277845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa2586c0 msgr2=0x5639aa256030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:00.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.651+0000 7f86b8c00640 1 --2- 172.21.15.123:0/2999277845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa2586c0 0x5639aa256030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.651+0000 7f86b8c00640 1 -- 172.21.15.123:0/2999277845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24de00 msgr2=0x5639aa255af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:00.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.651+0000 7f86b8c00640 1 --2- 172.21.15.123:0/2999277845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24de00 0x5639aa255af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.651+0000 7f86b8c00640 1 -- 172.21.15.123:0/2999277845 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a9f242b0 con 0x5639aa24d4e0 2024-09-19T04:46:00.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.652+0000 7f86b8c00640 1 --2- 172.21.15.123:0/2999277845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa24d4e0 0x5639aa2555b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f86a4000c00 tx=0x7f86a400bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8a01e8dcdee499e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:00.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.652+0000 7f86a2000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:46:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.652+0000 7f86a2000640 1 -- 172.21.15.123:0/2999277845 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4005040 con 0x5639aa24d4e0 2024-09-19T04:46:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.652+0000 7f86a2000640 1 -- 172.21.15.123:0/2999277845 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f86a40051e0 con 0x5639aa24d4e0 2024-09-19T04:46:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.652+0000 7f86bdfde740 1 -- 172.21.15.123:0/2999277845 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86a40041d0 con 0x5639aa24d4e0 2024-09-19T04:46:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.653+0000 7f86a2000640 1 -- 172.21.15.123:0/2999277845 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f86a40053e0 con 0x5639aa24d4e0 2024-09-19T04:46:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.653+0000 7f86bdfde740 10 client.24937 fetch_fsmap learned FSMap version 27 2024-09-19T04:46:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.653+0000 7f86bdfde740 1 -- 172.21.15.123:0/2999277845 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5639aa266f50 con 0x5639aa24d4e0 2024-09-19T04:46:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.653+0000 7f86a2000640 1 -- 172.21.15.123:0/2999277845 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a40041d0 con 0x5639aa24d4e0 2024-09-19T04:46:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.653+0000 7f86a2000640 1 -- 172.21.15.123:0/2999277845 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f86a4004a90 con 0x5639aa24d4e0 2024-09-19T04:46:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.653+0000 7f86bdfde740 10 client.24937 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:46:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.653+0000 7f86bdfde740 10 client.24937 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:46:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.653+0000 7f86bdfde740 20 client.24937 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.653+0000 7f86bdfde740 1 --2- 172.21.15.123:0/2999277845 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5639aa269390 0x5639aa26a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.653+0000 7f86bdfde740 4 client.24937 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:46:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.653+0000 7f86bdfde740 1 -- 172.21.15.123:0/2999277845 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639aa1c0620 con 0x5639aa269390 2024-09-19T04:46:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.654+0000 7f86abe00640 1 --2- 172.21.15.123:0/2999277845 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5639aa269390 0x5639aa26a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.654+0000 7f86abe00640 1 --2- 172.21.15.123:0/2999277845 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5639aa269390 0x5639aa26a800 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=844a3bb7ba067cc8 server_cookie=eb700cce64f7301 in_seq=0 out_seq=0 2024-09-19T04:46:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.654+0000 7f86a2000640 10 client.24937 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:46:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.655+0000 7f86a2000640 1 -- 172.21.15.123:0/2999277845 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5639aa1c0620 con 0x5639aa269390 2024-09-19T04:46:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.655+0000 7f86a2000640 10 client.24937 handle_command_reply: tid=0 2024-09-19T04:46:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.655+0000 7f86bdfde740 1 client.24937 shutdown 2024-09-19T04:46:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.655+0000 7f86bdfde740 1 -- 172.21.15.123:0/2999277845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa24d4e0 msgr2=0x5639aa2555b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.655+0000 7f86bdfde740 1 --2- 172.21.15.123:0/2999277845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa24d4e0 0x5639aa2555b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f86a4000c00 tx=0x7f86a400bfd0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.655+0000 7f86bdfde740 1 -- 172.21.15.123:0/2999277845 shutdown_connections 2024-09-19T04:46:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.655+0000 7f86bdfde740 1 --2- 172.21.15.123:0/2999277845 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5639aa269390 0x5639aa26a800 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.655+0000 7f86bdfde740 1 --2- 172.21.15.123:0/2999277845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa2586c0 0x5639aa256030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.655+0000 7f86bdfde740 1 --2- 172.21.15.123:0/2999277845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24de00 0x5639aa255af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.655+0000 7f86a2000640 0 client.24937 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:46:00.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.655+0000 7f86bdfde740 1 --2- 172.21.15.123:0/2999277845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa24d4e0 0x5639aa2555b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.655+0000 7f86bdfde740 1 -- 172.21.15.123:0/2999277845 >> 172.21.15.123:0/2999277845 conn(0x5639aa249a70 msgr2=0x5639aa24f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:00.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.656+0000 7f86bdfde740 1 -- 172.21.15.123:0/2999277845 shutdown_connections 2024-09-19T04:46:00.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.656+0000 7f86bdfde740 1 -- 172.21.15.123:0/2999277845 wait complete. 2024-09-19T04:46:00.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.656+0000 7f86bdfde740 20 client.24937 trim_cache size 0 max 16384 2024-09-19T04:46:00.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.674+0000 7f86bdfde740 1 Processor -- start 2024-09-19T04:46:00.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.674+0000 7f86bdfde740 1 -- start start 2024-09-19T04:46:00.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.674+0000 7f86bdfde740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24d4e0 0x5639aa24f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.674+0000 7f86bdfde740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24de00 0x5639aa255b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.674+0000 7f86bdfde740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa2586c0 0x5639aa256040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.674+0000 7f86bdfde740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5639aa268b40 con 0x5639aa24de00 2024-09-19T04:46:00.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.674+0000 7f86bdfde740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639aa269210 con 0x5639aa24d4e0 2024-09-19T04:46:00.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.674+0000 7f86bdfde740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5639aa2698e0 con 0x5639aa2586c0 2024-09-19T04:46:00.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.674+0000 7f86abe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24de00 0x5639aa255b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.674+0000 7f86abe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24de00 0x5639aa255b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37270/0 (socket says 172.21.15.123:37270) 2024-09-19T04:46:00.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.674+0000 7f86abe00640 1 -- 172.21.15.123:0/2436132673 learned_addr learned my addr 172.21.15.123:0/2436132673 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:00.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.674+0000 7f86b8c00640 1 --2- 172.21.15.123:0/2436132673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24d4e0 0x5639aa24f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.675+0000 7f86b9600640 1 --2- 172.21.15.123:0/2436132673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa2586c0 0x5639aa256040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.675+0000 7f86abe00640 1 -- 172.21.15.123:0/2436132673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa2586c0 msgr2=0x5639aa256040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:00.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.675+0000 7f86abe00640 1 --2- 172.21.15.123:0/2436132673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa2586c0 0x5639aa256040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.675+0000 7f86abe00640 1 -- 172.21.15.123:0/2436132673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24d4e0 msgr2=0x5639aa24f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:00.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.675+0000 7f86abe00640 1 --2- 172.21.15.123:0/2436132673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24d4e0 0x5639aa24f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.675+0000 7f86abe00640 1 -- 172.21.15.123:0/2436132673 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639aa249510 con 0x5639aa24de00 2024-09-19T04:46:00.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.675+0000 7f86b8c00640 1 --2- 172.21.15.123:0/2436132673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24d4e0 0x5639aa24f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:46:00.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.675+0000 7f86b9600640 1 --2- 172.21.15.123:0/2436132673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa2586c0 0x5639aa256040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:46:00.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.675+0000 7f86abe00640 1 --2- 172.21.15.123:0/2436132673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24de00 0x5639aa255b00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f869c02a9a0 tx=0x7f869c03f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88353abc9c73c73f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:00.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.675+0000 7f86a9200640 1 -- 172.21.15.123:0/2436132673 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c059070 con 0x5639aa24de00 2024-09-19T04:46:00.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.675+0000 7f86a9200640 1 -- 172.21.15.123:0/2436132673 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f869c0325c0 con 0x5639aa24de00 2024-09-19T04:46:00.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.675+0000 7f86a9200640 1 -- 172.21.15.123:0/2436132673 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c04f070 con 0x5639aa24de00 2024-09-19T04:46:00.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.676+0000 7f86bdfde740 1 -- 172.21.15.123:0/2436132673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24de00 msgr2=0x5639aa255b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:00.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.676+0000 7f86bdfde740 1 --2- 172.21.15.123:0/2436132673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24de00 0x5639aa255b00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f869c02a9a0 tx=0x7f869c03f070 comp rx=0 tx=0).stop 2024-09-19T04:46:00.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.676+0000 7f86bdfde740 1 -- 172.21.15.123:0/2436132673 shutdown_connections 2024-09-19T04:46:00.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.676+0000 7f86bdfde740 1 --2- 172.21.15.123:0/2436132673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa2586c0 0x5639aa256040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.676+0000 7f86bdfde740 1 --2- 172.21.15.123:0/2436132673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa24de00 0x5639aa255b00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.676+0000 7f86bdfde740 1 --2- 172.21.15.123:0/2436132673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24d4e0 0x5639aa24f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.676+0000 7f86bdfde740 1 -- 172.21.15.123:0/2436132673 >> 172.21.15.123:0/2436132673 conn(0x5639aa249a70 msgr2=0x5639aa24ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:00.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.676+0000 7f86bdfde740 1 -- 172.21.15.123:0/2436132673 shutdown_connections 2024-09-19T04:46:00.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.676+0000 7f86bdfde740 1 -- 172.21.15.123:0/2436132673 wait complete. 2024-09-19T04:46:00.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.677+0000 7f86bdfde740 1 -- start start 2024-09-19T04:46:00.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.677+0000 7f86bdfde740 1 Processor -- start 2024-09-19T04:46:00.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.677+0000 7f86bdfde740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa24d4e0 0x5639aa272fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.677+0000 7f86bdfde740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24de00 0x5639aa252200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.677+0000 7f86bdfde740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa2586c0 0x5639aa252740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.677+0000 7f86bdfde740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5639aa26a680 con 0x5639aa2586c0 2024-09-19T04:46:00.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.677+0000 7f86bdfde740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639aa269fb0 con 0x5639aa24de00 2024-09-19T04:46:00.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.677+0000 7f86bdfde740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5639aa26ad50 con 0x5639aa24d4e0 2024-09-19T04:46:00.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.677+0000 7f86bdfde740 10 client.0 _ll_register_callbacks cb 0x7f86a400b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:00.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.677+0000 7f86b9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa2586c0 0x5639aa252740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.677+0000 7f86abe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24de00 0x5639aa252200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.677+0000 7f86b9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa2586c0 0x5639aa252740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37286/0 (socket says 172.21.15.123:37286) 2024-09-19T04:46:00.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.677+0000 7f86b9600640 1 -- 172.21.15.123:0/2105294140 learned_addr learned my addr 172.21.15.123:0/2105294140 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:00.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.678+0000 7f86b8c00640 1 --2- 172.21.15.123:0/2105294140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa24d4e0 0x5639aa272fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.678+0000 7f86abe00640 1 -- 172.21.15.123:0/2105294140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa24d4e0 msgr2=0x5639aa272fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:00.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.678+0000 7f86abe00640 1 --2- 172.21.15.123:0/2105294140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa24d4e0 0x5639aa272fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.678+0000 7f86abe00640 1 -- 172.21.15.123:0/2105294140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa2586c0 msgr2=0x5639aa252740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:00.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.678+0000 7f86abe00640 1 --2- 172.21.15.123:0/2105294140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa2586c0 0x5639aa252740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.678+0000 7f86abe00640 1 -- 172.21.15.123:0/2105294140 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639aa252cb0 con 0x5639aa24de00 2024-09-19T04:46:00.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.678+0000 7f86b9600640 1 --2- 172.21.15.123:0/2105294140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa2586c0 0x5639aa252740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:00.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.678+0000 7f86abe00640 1 --2- 172.21.15.123:0/2105294140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24de00 0x5639aa252200 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f869c0096c0 tx=0x7f869c02ac00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae1887d1609a36d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:00.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.678+0000 7f86a2000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:46:00.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.678+0000 7f86a2000640 1 -- 172.21.15.123:0/2105294140 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c04f050 con 0x5639aa24de00 2024-09-19T04:46:00.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.678+0000 7f86a2000640 1 -- 172.21.15.123:0/2105294140 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f869c004ca0 con 0x5639aa24de00 2024-09-19T04:46:00.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.678+0000 7f86bdfde740 1 -- 172.21.15.123:0/2105294140 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f869c0325c0 con 0x5639aa24de00 2024-09-19T04:46:00.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.679+0000 7f86a2000640 1 -- 172.21.15.123:0/2105294140 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f869c059070 con 0x5639aa24de00 2024-09-19T04:46:00.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.679+0000 7f86bdfde740 10 client.24929 fetch_fsmap learned FSMap version 27 2024-09-19T04:46:00.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.679+0000 7f86bdfde740 1 -- 172.21.15.123:0/2105294140 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5639aa2717c0 con 0x5639aa24de00 2024-09-19T04:46:00.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.679+0000 7f86a2000640 1 -- 172.21.15.123:0/2105294140 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f869c0325c0 con 0x5639aa24de00 2024-09-19T04:46:00.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.679+0000 7f86a2000640 1 -- 172.21.15.123:0/2105294140 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f869c003c40 con 0x5639aa24de00 2024-09-19T04:46:00.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.679+0000 7f86bdfde740 10 client.24929 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:46:00.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.679+0000 7f86bdfde740 10 client.24929 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:46:00.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.679+0000 7f86bdfde740 20 client.24929 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:00.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.679+0000 7f86bdfde740 1 --2- 172.21.15.123:0/2105294140 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5639aa2d7dc0 0x5639aa2d6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:00.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.679+0000 7f86bdfde740 4 client.24929 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:46:00.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.679+0000 7f86bdfde740 1 -- 172.21.15.123:0/2105294140 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "status"}) -- 0x5639aa1bab30 con 0x5639aa2d7dc0 2024-09-19T04:46:00.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.680+0000 7f86b8c00640 1 --2- 172.21.15.123:0/2105294140 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5639aa2d7dc0 0x5639aa2d6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:00.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.680+0000 7f86b8c00640 1 --2- 172.21.15.123:0/2105294140 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5639aa2d7dc0 0x5639aa2d6ad0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1e16841c384acbb server_cookie=5bae9e4adb7b8752 in_seq=0 out_seq=0 2024-09-19T04:46:00.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.680+0000 7f86a2000640 10 client.24929 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:46:00.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.681+0000 7f86a2000640 1 -- 172.21.15.123:0/2105294140 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5639aa1bab30 con 0x5639aa2d7dc0 2024-09-19T04:46:00.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.681+0000 7f86a2000640 10 client.24929 handle_command_reply: tid=0 2024-09-19T04:46:00.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.681+0000 7f86bdfde740 1 client.24929 shutdown 2024-09-19T04:46:00.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.681+0000 7f86bdfde740 1 -- 172.21.15.123:0/2105294140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24de00 msgr2=0x5639aa252200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:00.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.681+0000 7f86bdfde740 1 --2- 172.21.15.123:0/2105294140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24de00 0x5639aa252200 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f869c0096c0 tx=0x7f869c02ac00 comp rx=0 tx=0).stop 2024-09-19T04:46:00.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.681+0000 7f86bdfde740 1 -- 172.21.15.123:0/2105294140 shutdown_connections 2024-09-19T04:46:00.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.681+0000 7f86bdfde740 1 --2- 172.21.15.123:0/2105294140 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5639aa2d7dc0 0x5639aa2d6ad0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.681+0000 7f86bdfde740 1 --2- 172.21.15.123:0/2105294140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639aa2586c0 0x5639aa252740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.681+0000 7f86bdfde740 1 --2- 172.21.15.123:0/2105294140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639aa24de00 0x5639aa252200 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.681+0000 7f86bdfde740 1 --2- 172.21.15.123:0/2105294140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639aa24d4e0 0x5639aa272fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.681+0000 7f86a2000640 0 client.24929 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:46:00.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.681+0000 7f86bdfde740 1 -- 172.21.15.123:0/2105294140 >> 172.21.15.123:0/2105294140 conn(0x5639aa249a70 msgr2=0x5639aa263700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:00.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.681+0000 7f86bdfde740 1 -- 172.21.15.123:0/2105294140 shutdown_connections 2024-09-19T04:46:00.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.681+0000 7f86bdfde740 1 -- 172.21.15.123:0/2105294140 wait complete. 2024-09-19T04:46:00.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.681+0000 7f86bdfde740 20 client.24929 trim_cache size 0 max 16384 2024-09-19T04:46:00.688 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:46:00.688 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "911fa052-7640-11ef-bceb-c7b262605968", 2024-09-19T04:46:00.688 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 2, 2024-09-19T04:46:00.688 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24505, 2024-09-19T04:46:00.688 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-09-19T04:46:00.688 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-09-19T04:46:00.688 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-09-19T04:46:00.688 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 78.875844342999997, 2024-09-19T04:46:00.689 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 27, 2024-09-19T04:46:00.689 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 86, 2024-09-19T04:46:00.689 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T04:46:00.689 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 116.86341450899999 2024-09-19T04:46:00.689 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:46:00.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.682+0000 7f86ba000640 1 -- 172.21.15.123:0/3527922632 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f86980a32d0 msgr2=0x7f86980a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:00.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.682+0000 7f86ba000640 1 --2- 172.21.15.123:0/3527922632 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f86980a32d0 0x7f86980a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f869c031550 tx=0x7f869c005bc0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.682+0000 7f86ba000640 1 -- 172.21.15.123:0/3527922632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86b41129d0 msgr2=0x7f86b419e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:00.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.682+0000 7f86ba000640 1 --2- 172.21.15.123:0/3527922632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86b41129d0 0x7f86b419e6e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00e9d0 tx=0x7f86ac00ee90 comp rx=0 tx=0).stop 2024-09-19T04:46:00.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.682+0000 7f86ba000640 1 -- 172.21.15.123:0/3527922632 shutdown_connections 2024-09-19T04:46:00.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.682+0000 7f86ba000640 1 --2- 172.21.15.123:0/3527922632 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f86980a32d0 0x7f86980a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.682+0000 7f86ba000640 1 --2- 172.21.15.123:0/3527922632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86b41129d0 0x7f86b419e6e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.682+0000 7f86ba000640 1 --2- 172.21.15.123:0/3527922632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86b4074eb0 0x7f86b41a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.682+0000 7f86ba000640 1 --2- 172.21.15.123:0/3527922632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86b4074590 0x7f86b41a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:00.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.682+0000 7f86ba000640 1 -- 172.21.15.123:0/3527922632 >> 172.21.15.123:0/3527922632 conn(0x7f86b40fd6e0 msgr2=0x7f86b40fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:00.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.682+0000 7f86ba000640 1 -- 172.21.15.123:0/3527922632 shutdown_connections 2024-09-19T04:46:00.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:00.682+0000 7f86ba000640 1 -- 172.21.15.123:0/3527922632 wait complete. 2024-09-19T04:46:00.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:00 smithi183 ceph-mon[35333]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-19T04:46:00.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:00 smithi123 ceph-mon[30511]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-19T04:46:00.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:00 smithi193 ceph-mon[35592]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-19T04:46:01.577 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T04:46:01.944 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:02.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.501+0000 7feb5d800640 1 -- 172.21.15.123:0/3407259060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb58106f10 msgr2=0x7feb5810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:02.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.501+0000 7feb5d800640 1 --2- 172.21.15.123:0/3407259060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb58106f10 0x7feb5810ea90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feb50009900 tx=0x7feb5002a410 comp rx=0 tx=0).stop 2024-09-19T04:46:02.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.501+0000 7feb5d800640 1 -- 172.21.15.123:0/3407259060 shutdown_connections 2024-09-19T04:46:02.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.501+0000 7feb5d800640 1 --2- 172.21.15.123:0/3407259060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb58106f10 0x7feb5810ea90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.501+0000 7feb5d800640 1 --2- 172.21.15.123:0/3407259060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb58102790 0x7feb581067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.501+0000 7feb5d800640 1 --2- 172.21.15.123:0/3407259060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb58101de0 0x7feb581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.501+0000 7feb5d800640 1 -- 172.21.15.123:0/3407259060 >> 172.21.15.123:0/3407259060 conn(0x7feb580fd8c0 msgr2=0x7feb580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:02.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.501+0000 7feb5d800640 1 -- 172.21.15.123:0/3407259060 shutdown_connections 2024-09-19T04:46:02.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.501+0000 7feb5d800640 1 -- 172.21.15.123:0/3407259060 wait complete. 2024-09-19T04:46:02.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.502+0000 7feb5d800640 1 Processor -- start 2024-09-19T04:46:02.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.502+0000 7feb5d800640 1 -- start start 2024-09-19T04:46:02.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.502+0000 7feb5d800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb58101de0 0x7feb581983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.502+0000 7feb5d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb58102790 0x7feb581988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.502+0000 7feb5d800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb58106f10 0x7feb5819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.502+0000 7feb5d800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb58111f70 con 0x7feb58102790 2024-09-19T04:46:02.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.502+0000 7feb5d800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feb58111df0 con 0x7feb58106f10 2024-09-19T04:46:02.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.502+0000 7feb5d800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb581120f0 con 0x7feb58101de0 2024-09-19T04:46:02.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.502+0000 7feb56000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb58102790 0x7feb581988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.502+0000 7feb56000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb58102790 0x7feb581988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53260/0 (socket says 172.21.15.123:53260) 2024-09-19T04:46:02.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.502+0000 7feb56000640 1 -- 172.21.15.123:0/1216399343 learned_addr learned my addr 172.21.15.123:0/1216399343 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:02.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.502+0000 7feb57400640 1 --2- 172.21.15.123:0/1216399343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb58106f10 0x7feb5819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.502+0000 7feb56a00640 1 --2- 172.21.15.123:0/1216399343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb58101de0 0x7feb581983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.503+0000 7feb56000640 1 -- 172.21.15.123:0/1216399343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb58101de0 msgr2=0x7feb581983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:02.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.503+0000 7feb56000640 1 --2- 172.21.15.123:0/1216399343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb58101de0 0x7feb581983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.503+0000 7feb56000640 1 -- 172.21.15.123:0/1216399343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb58106f10 msgr2=0x7feb5819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:02.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.503+0000 7feb56000640 1 --2- 172.21.15.123:0/1216399343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb58106f10 0x7feb5819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.503+0000 7feb56000640 1 -- 172.21.15.123:0/1216399343 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5819d360 con 0x7feb58102790 2024-09-19T04:46:02.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.503+0000 7feb56a00640 1 --2- 172.21.15.123:0/1216399343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb58101de0 0x7feb581983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:02.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.503+0000 7feb56000640 1 --2- 172.21.15.123:0/1216399343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb58102790 0x7feb581988f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7feb4400d950 tx=0x7feb4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:02.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.503+0000 7feb4f400640 1 -- 172.21.15.123:0/1216399343 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44017030 con 0x7feb58102790 2024-09-19T04:46:02.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.503+0000 7feb4f400640 1 -- 172.21.15.123:0/1216399343 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb44004020 con 0x7feb58102790 2024-09-19T04:46:02.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.503+0000 7feb4f400640 1 -- 172.21.15.123:0/1216399343 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb440047a0 con 0x7feb58102790 2024-09-19T04:46:02.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.503+0000 7feb5d800640 1 -- 172.21.15.123:0/1216399343 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb5819d650 con 0x7feb58102790 2024-09-19T04:46:02.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.504+0000 7feb5d800640 1 -- 172.21.15.123:0/1216399343 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb581a4f30 con 0x7feb58102790 2024-09-19T04:46:02.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.507+0000 7feb603c0740 1 Processor -- start 2024-09-19T04:46:02.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.508+0000 7feb603c0740 1 -- start start 2024-09-19T04:46:02.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.508+0000 7feb4f400640 1 -- 172.21.15.123:0/1216399343 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7feb440041d0 con 0x7feb58102790 2024-09-19T04:46:02.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.508+0000 7feb603c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414c230 0x55635414c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.508+0000 7feb603c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635414cb50 0x55635415b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.508+0000 7feb603c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635415b920 0x55635415dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.508+0000 7feb603c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5563540b20b0 con 0x55635415b920 2024-09-19T04:46:02.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.508+0000 7feb603c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556354059ab0 con 0x55635414c230 2024-09-19T04:46:02.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.508+0000 7feb603c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5563540ae070 con 0x55635414cb50 2024-09-19T04:46:02.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.509+0000 7feb56000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635415b920 0x55635415dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.509+0000 7feb57400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414c230 0x55635414c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.509+0000 7feb56000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635415b920 0x55635415dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53272/0 (socket says 172.21.15.123:53272) 2024-09-19T04:46:02.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.509+0000 7feb56000640 1 -- 172.21.15.123:0/53976758 learned_addr learned my addr 172.21.15.123:0/53976758 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:02.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.509+0000 7feb56a00640 1 --2- 172.21.15.123:0/53976758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635414cb50 0x55635415b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.509+0000 7feb56000640 1 -- 172.21.15.123:0/53976758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635414cb50 msgr2=0x55635415b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:02.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.509+0000 7feb56000640 1 --2- 172.21.15.123:0/53976758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635414cb50 0x55635415b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.509+0000 7feb56000640 1 -- 172.21.15.123:0/53976758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414c230 msgr2=0x55635414c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:02.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.509+0000 7feb56000640 1 --2- 172.21.15.123:0/53976758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414c230 0x55635414c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.509+0000 7feb56000640 1 -- 172.21.15.123:0/53976758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563540664a0 con 0x55635415b920 2024-09-19T04:46:02.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.509+0000 7feb56000640 1 --2- 172.21.15.123:0/53976758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635415b920 0x55635415dd10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7feb440ccc20 tx=0x7feb440d7860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c10ebfb65ad864 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:02.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.510+0000 7feb4f400640 1 --2- 172.21.15.123:0/1216399343 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7feb3c0a3350 0x7feb3c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.510+0000 7feb43e00640 1 -- 172.21.15.123:0/53976758 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44097a60 con 0x55635415b920 2024-09-19T04:46:02.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.510+0000 7feb4f400640 1 -- 172.21.15.123:0/1216399343 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7feb440cf2f0 con 0x7feb58102790 2024-09-19T04:46:02.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.510+0000 7feb43e00640 1 -- 172.21.15.123:0/53976758 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb4409bec0 con 0x55635415b920 2024-09-19T04:46:02.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.510+0000 7feb57400640 1 --2- 172.21.15.123:0/1216399343 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7feb3c0a3350 0x7feb3c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.510+0000 7feb43e00640 1 -- 172.21.15.123:0/53976758 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4409ac10 con 0x55635415b920 2024-09-19T04:46:02.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.511+0000 7feb603c0740 1 -- 172.21.15.123:0/53976758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635415b920 msgr2=0x55635415dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:02.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.511+0000 7feb603c0740 1 --2- 172.21.15.123:0/53976758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635415b920 0x55635415dd10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7feb440ccc20 tx=0x7feb440d7860 comp rx=0 tx=0).stop 2024-09-19T04:46:02.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.511+0000 7feb57400640 1 --2- 172.21.15.123:0/1216399343 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7feb3c0a3350 0x7feb3c0a5810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feb50005c40 tx=0x7feb50005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:02.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.511+0000 7feb603c0740 1 -- 172.21.15.123:0/53976758 shutdown_connections 2024-09-19T04:46:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.511+0000 7feb603c0740 1 --2- 172.21.15.123:0/53976758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635415b920 0x55635415dd10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.511+0000 7feb603c0740 1 --2- 172.21.15.123:0/53976758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635414cb50 0x55635415b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.511+0000 7feb603c0740 1 --2- 172.21.15.123:0/53976758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414c230 0x55635414c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.511+0000 7feb603c0740 1 -- 172.21.15.123:0/53976758 >> 172.21.15.123:0/53976758 conn(0x5563541488c0 msgr2=0x556354148ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.511+0000 7feb603c0740 1 -- 172.21.15.123:0/53976758 shutdown_connections 2024-09-19T04:46:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.511+0000 7feb603c0740 1 -- 172.21.15.123:0/53976758 wait complete. 2024-09-19T04:46:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.511+0000 7feb603c0740 1 -- start start 2024-09-19T04:46:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.511+0000 7feb603c0740 1 Processor -- start 2024-09-19T04:46:02.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.512+0000 7feb603c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635414c230 0x5563541588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.512+0000 7feb603c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414cb50 0x556354158de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.512+0000 7feb603c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635415b920 0x556354159320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.512+0000 7feb603c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55635415e5f0 con 0x55635415b920 2024-09-19T04:46:02.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.512+0000 7feb603c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55635403a920 con 0x55635414cb50 2024-09-19T04:46:02.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.512+0000 7feb603c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55635415e770 con 0x55635414c230 2024-09-19T04:46:02.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.512+0000 7feb603c0740 10 client.0 _ll_register_callbacks cb 0x55635405e8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:02.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.512+0000 7feb57400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635415b920 0x556354159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.512+0000 7feb57400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635415b920 0x556354159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53288/0 (socket says 172.21.15.123:53288) 2024-09-19T04:46:02.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.512+0000 7feb57400640 1 -- 172.21.15.123:0/4002969688 learned_addr learned my addr 172.21.15.123:0/4002969688 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:02.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.512+0000 7feb56000640 1 --2- 172.21.15.123:0/4002969688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414cb50 0x556354158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.512+0000 7feb56a00640 1 --2- 172.21.15.123:0/4002969688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635414c230 0x5563541588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.513+0000 7feb57400640 1 -- 172.21.15.123:0/4002969688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635414c230 msgr2=0x5563541588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:02.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.513+0000 7feb57400640 1 --2- 172.21.15.123:0/4002969688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635414c230 0x5563541588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.513+0000 7feb57400640 1 -- 172.21.15.123:0/4002969688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414cb50 msgr2=0x556354158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:02.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.513+0000 7feb57400640 1 --2- 172.21.15.123:0/4002969688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414cb50 0x556354158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.513+0000 7feb57400640 1 -- 172.21.15.123:0/4002969688 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556353e232b0 con 0x55635415b920 2024-09-19T04:46:02.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.513+0000 7feb57400640 1 --2- 172.21.15.123:0/4002969688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635415b920 0x556354159320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7feb50009900 tx=0x7feb50009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1469839a64d870c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:02.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.513+0000 7feb40c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:46:02.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.513+0000 7feb40c00640 1 -- 172.21.15.123:0/4002969688 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5004e070 con 0x55635415b920 2024-09-19T04:46:02.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.513+0000 7feb40c00640 1 -- 172.21.15.123:0/4002969688 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7feb500037f0 con 0x55635415b920 2024-09-19T04:46:02.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.513+0000 7feb603c0740 1 -- 172.21.15.123:0/4002969688 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb4409bec0 con 0x55635415b920 2024-09-19T04:46:02.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.514+0000 7feb40c00640 1 -- 172.21.15.123:0/4002969688 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feb5005a050 con 0x55635415b920 2024-09-19T04:46:02.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.514+0000 7feb603c0740 10 client.14922 fetch_fsmap learned FSMap version 27 2024-09-19T04:46:02.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.514+0000 7feb603c0740 1 -- 172.21.15.123:0/4002969688 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55635416a360 con 0x55635415b920 2024-09-19T04:46:02.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.514+0000 7feb40c00640 1 -- 172.21.15.123:0/4002969688 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb4409bec0 con 0x55635415b920 2024-09-19T04:46:02.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.514+0000 7feb40c00640 1 -- 172.21.15.123:0/4002969688 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feb5004ac20 con 0x55635415b920 2024-09-19T04:46:02.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.514+0000 7feb603c0740 10 client.14922 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:46:02.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.514+0000 7feb603c0740 10 client.14922 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:46:02.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.514+0000 7feb603c0740 20 client.14922 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:02.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.514+0000 7feb603c0740 1 --2- 172.21.15.123:0/4002969688 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55635416c970 0x55635416dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.514+0000 7feb603c0740 4 client.14922 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:46:02.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.514+0000 7feb603c0740 1 -- 172.21.15.123:0/4002969688 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563540bf620 con 0x55635416c970 2024-09-19T04:46:02.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.515+0000 7feb56a00640 1 --2- 172.21.15.123:0/4002969688 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55635416c970 0x55635416dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.515+0000 7feb56a00640 1 --2- 172.21.15.123:0/4002969688 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55635416c970 0x55635416dde0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10bf6093c5bdf99d server_cookie=fee3c26c3a579074 in_seq=0 out_seq=0 2024-09-19T04:46:02.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.515+0000 7feb40c00640 10 client.14922 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:46:02.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.516+0000 7feb40c00640 1 -- 172.21.15.123:0/4002969688 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563540bf620 con 0x55635416c970 2024-09-19T04:46:02.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.516+0000 7feb40c00640 10 client.14922 handle_command_reply: tid=0 2024-09-19T04:46:02.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.516+0000 7feb603c0740 1 client.14922 shutdown 2024-09-19T04:46:02.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.517+0000 7feb603c0740 1 -- 172.21.15.123:0/4002969688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635415b920 msgr2=0x556354159320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:02.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.517+0000 7feb603c0740 1 --2- 172.21.15.123:0/4002969688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635415b920 0x556354159320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7feb50009900 tx=0x7feb50009260 comp rx=0 tx=0).stop 2024-09-19T04:46:02.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.517+0000 7feb603c0740 1 -- 172.21.15.123:0/4002969688 shutdown_connections 2024-09-19T04:46:02.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.517+0000 7feb603c0740 1 --2- 172.21.15.123:0/4002969688 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55635416c970 0x55635416dde0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.517+0000 7feb603c0740 1 --2- 172.21.15.123:0/4002969688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635415b920 0x556354159320 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.517+0000 7feb603c0740 1 --2- 172.21.15.123:0/4002969688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414cb50 0x556354158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.517+0000 7feb603c0740 1 --2- 172.21.15.123:0/4002969688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635414c230 0x5563541588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.517+0000 7feb603c0740 1 -- 172.21.15.123:0/4002969688 >> 172.21.15.123:0/4002969688 conn(0x5563541488c0 msgr2=0x55635414e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:02.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.517+0000 7feb40c00640 0 client.14922 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:46:02.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.517+0000 7feb603c0740 1 -- 172.21.15.123:0/4002969688 shutdown_connections 2024-09-19T04:46:02.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.517+0000 7feb603c0740 1 -- 172.21.15.123:0/4002969688 wait complete. 2024-09-19T04:46:02.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.517+0000 7feb603c0740 20 client.14922 trim_cache size 0 max 16384 2024-09-19T04:46:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.536+0000 7feb603c0740 1 Processor -- start 2024-09-19T04:46:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.536+0000 7feb603c0740 1 -- start start 2024-09-19T04:46:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.536+0000 7feb603c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635414c230 0x55635416d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.536+0000 7feb603c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414cb50 0x556354158160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.536+0000 7feb603c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635415b920 0x556354157170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.536+0000 7feb603c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55635416c120 con 0x55635414c230 2024-09-19T04:46:02.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.536+0000 7feb603c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55635416c7f0 con 0x55635414cb50 2024-09-19T04:46:02.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.536+0000 7feb603c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55635416cec0 con 0x55635415b920 2024-09-19T04:46:02.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.536+0000 7feb56a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635414c230 0x55635416d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.536+0000 7feb57400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635415b920 0x556354157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.536+0000 7feb56a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635414c230 0x55635416d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53304/0 (socket says 172.21.15.123:53304) 2024-09-19T04:46:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.536+0000 7feb56a00640 1 -- 172.21.15.123:0/1703704135 learned_addr learned my addr 172.21.15.123:0/1703704135 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.536+0000 7feb56000640 1 --2- 172.21.15.123:0/1703704135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414cb50 0x556354158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.537+0000 7feb57400640 1 -- 172.21.15.123:0/1703704135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414cb50 msgr2=0x556354158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.537+0000 7feb57400640 1 --2- 172.21.15.123:0/1703704135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414cb50 0x556354158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.537+0000 7feb57400640 1 -- 172.21.15.123:0/1703704135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635414c230 msgr2=0x55635416d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.537+0000 7feb57400640 1 --2- 172.21.15.123:0/1703704135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635414c230 0x55635416d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.537+0000 7feb57400640 1 -- 172.21.15.123:0/1703704135 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556354156df0 con 0x55635415b920 2024-09-19T04:46:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.537+0000 7feb56000640 1 --2- 172.21.15.123:0/1703704135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414cb50 0x556354158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.537+0000 7feb56a00640 1 --2- 172.21.15.123:0/1703704135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635414c230 0x55635416d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.537+0000 7feb57400640 1 --2- 172.21.15.123:0/1703704135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635415b920 0x556354157170 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb50031550 tx=0x7feb50004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da9afbb0ff9bc4e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:02.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.538+0000 7feb43400640 1 -- 172.21.15.123:0/1703704135 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5004e040 con 0x55635415b920 2024-09-19T04:46:02.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.538+0000 7feb43400640 1 -- 172.21.15.123:0/1703704135 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb500035f0 con 0x55635415b920 2024-09-19T04:46:02.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.538+0000 7feb43400640 1 -- 172.21.15.123:0/1703704135 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50035040 con 0x55635415b920 2024-09-19T04:46:02.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.539+0000 7feb603c0740 1 -- 172.21.15.123:0/1703704135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635415b920 msgr2=0x556354157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:02.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.539+0000 7feb603c0740 1 --2- 172.21.15.123:0/1703704135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635415b920 0x556354157170 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb50031550 tx=0x7feb50004610 comp rx=0 tx=0).stop 2024-09-19T04:46:02.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.539+0000 7feb603c0740 1 -- 172.21.15.123:0/1703704135 shutdown_connections 2024-09-19T04:46:02.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.539+0000 7feb603c0740 1 --2- 172.21.15.123:0/1703704135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635415b920 0x556354157170 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.539+0000 7feb603c0740 1 --2- 172.21.15.123:0/1703704135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414cb50 0x556354158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.539+0000 7feb603c0740 1 --2- 172.21.15.123:0/1703704135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635414c230 0x55635416d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.539+0000 7feb603c0740 1 -- 172.21.15.123:0/1703704135 >> 172.21.15.123:0/1703704135 conn(0x5563541488c0 msgr2=0x556354153ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:02.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.539+0000 7feb603c0740 1 -- 172.21.15.123:0/1703704135 shutdown_connections 2024-09-19T04:46:02.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.539+0000 7feb603c0740 1 -- 172.21.15.123:0/1703704135 wait complete. 2024-09-19T04:46:02.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.540+0000 7feb603c0740 1 -- start start 2024-09-19T04:46:02.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.540+0000 7feb603c0740 1 Processor -- start 2024-09-19T04:46:02.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.541+0000 7feb603c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414c230 0x5563541589a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.541+0000 7feb603c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635414cb50 0x556354158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.541+0000 7feb603c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635415b920 0x5563541d5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.541+0000 7feb603c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55635416dc60 con 0x55635414cb50 2024-09-19T04:46:02.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.541+0000 7feb603c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55635416d590 con 0x55635414c230 2024-09-19T04:46:02.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.541+0000 7feb603c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55635416e330 con 0x55635415b920 2024-09-19T04:46:02.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.541+0000 7feb603c0740 10 client.0 _ll_register_callbacks cb 0x7feb50031830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:02.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.541+0000 7feb56a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414c230 0x5563541589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.541+0000 7feb56000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635414cb50 0x556354158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.541+0000 7feb56a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414c230 0x5563541589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:47330/0 (socket says 172.21.15.123:47330) 2024-09-19T04:46:02.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.541+0000 7feb57400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635415b920 0x5563541d5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.541+0000 7feb56a00640 1 -- 172.21.15.123:0/61882325 learned_addr learned my addr 172.21.15.123:0/61882325 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:02.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.541+0000 7feb56a00640 1 -- 172.21.15.123:0/61882325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635415b920 msgr2=0x5563541d5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:02.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.541+0000 7feb56a00640 1 --2- 172.21.15.123:0/61882325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635415b920 0x5563541d5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.541+0000 7feb56a00640 1 -- 172.21.15.123:0/61882325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635414cb50 msgr2=0x556354158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:02.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.541+0000 7feb56a00640 1 --2- 172.21.15.123:0/61882325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635414cb50 0x556354158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.541+0000 7feb56a00640 1 -- 172.21.15.123:0/61882325 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563541597c0 con 0x55635414c230 2024-09-19T04:46:02.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.542+0000 7feb56a00640 1 --2- 172.21.15.123:0/61882325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414c230 0x5563541589a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feb4800fc00 tx=0x7feb4801f770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e02770bafa19bd78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:02.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.542+0000 7feb40c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:46:02.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.542+0000 7feb40c00640 1 -- 172.21.15.123:0/61882325 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48010630 con 0x55635414c230 2024-09-19T04:46:02.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.542+0000 7feb603c0740 1 -- 172.21.15.123:0/61882325 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb500035f0 con 0x55635414c230 2024-09-19T04:46:02.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.542+0000 7feb40c00640 1 -- 172.21.15.123:0/61882325 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7feb480107d0 con 0x55635414c230 2024-09-19T04:46:02.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.542+0000 7feb40c00640 1 -- 172.21.15.123:0/61882325 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feb4800fe70 con 0x55635414c230 2024-09-19T04:46:02.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.542+0000 7feb603c0740 10 client.24947 fetch_fsmap learned FSMap version 27 2024-09-19T04:46:02.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.542+0000 7feb603c0740 1 -- 172.21.15.123:0/61882325 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5563541d66c0 con 0x55635414c230 2024-09-19T04:46:02.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.542+0000 7feb40c00640 1 -- 172.21.15.123:0/61882325 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb500035f0 con 0x55635414c230 2024-09-19T04:46:02.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.543+0000 7feb40c00640 1 -- 172.21.15.123:0/61882325 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feb48026ae0 con 0x55635414c230 2024-09-19T04:46:02.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.543+0000 7feb603c0740 10 client.24947 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:46:02.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.543+0000 7feb603c0740 10 client.24947 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:46:02.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.543+0000 7feb603c0740 20 client.24947 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:02.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.543+0000 7feb603c0740 1 --2- 172.21.15.123:0/61882325 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5563541db2c0 0x5563541d9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:02.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.543+0000 7feb603c0740 4 client.24947 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:46:02.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.543+0000 7feb603c0740 1 -- 172.21.15.123:0/61882325 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "status"}) -- 0x5563540b9b30 con 0x5563541db2c0 2024-09-19T04:46:02.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.543+0000 7feb56000640 1 --2- 172.21.15.123:0/61882325 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5563541db2c0 0x5563541d9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:02.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.544+0000 7feb56000640 1 --2- 172.21.15.123:0/61882325 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5563541db2c0 0x5563541d9fd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c98ac35cc4182d75 server_cookie=95cafe462ccdf913 in_seq=0 out_seq=0 2024-09-19T04:46:02.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.544+0000 7feb40c00640 10 client.24947 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:46:02.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.544+0000 7feb40c00640 1 -- 172.21.15.123:0/61882325 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x5563540b9b30 con 0x5563541db2c0 2024-09-19T04:46:02.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.544+0000 7feb40c00640 10 client.24947 handle_command_reply: tid=0 2024-09-19T04:46:02.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.544+0000 7feb603c0740 1 client.24947 shutdown 2024-09-19T04:46:02.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.544+0000 7feb603c0740 1 -- 172.21.15.123:0/61882325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414c230 msgr2=0x5563541589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:02.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.544+0000 7feb603c0740 1 --2- 172.21.15.123:0/61882325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414c230 0x5563541589a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feb4800fc00 tx=0x7feb4801f770 comp rx=0 tx=0).stop 2024-09-19T04:46:02.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.544+0000 7feb603c0740 1 -- 172.21.15.123:0/61882325 shutdown_connections 2024-09-19T04:46:02.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.544+0000 7feb603c0740 1 --2- 172.21.15.123:0/61882325 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5563541db2c0 0x5563541d9fd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.544+0000 7feb603c0740 1 --2- 172.21.15.123:0/61882325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55635415b920 0x5563541d5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.544+0000 7feb603c0740 1 --2- 172.21.15.123:0/61882325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635414cb50 0x556354158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.544+0000 7feb40c00640 0 client.24947 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:46:02.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.544+0000 7feb603c0740 1 --2- 172.21.15.123:0/61882325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55635414c230 0x5563541589a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.544+0000 7feb603c0740 1 -- 172.21.15.123:0/61882325 >> 172.21.15.123:0/61882325 conn(0x5563541488c0 msgr2=0x556354148ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:02.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.544+0000 7feb603c0740 1 -- 172.21.15.123:0/61882325 shutdown_connections 2024-09-19T04:46:02.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.544+0000 7feb603c0740 1 -- 172.21.15.123:0/61882325 wait complete. 2024-09-19T04:46:02.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.544+0000 7feb603c0740 20 client.24947 trim_cache size 0 max 16384 2024-09-19T04:46:02.550 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:46:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "911fa052-7640-11ef-bceb-c7b262605968", 2024-09-19T04:46:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 1, 2024-09-19T04:46:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24529, 2024-09-19T04:46:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-09-19T04:46:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-09-19T04:46:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-09-19T04:46:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 81.747393355, 2024-09-19T04:46:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 27, 2024-09-19T04:46:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 86, 2024-09-19T04:46:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T04:46:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 113.320517208 2024-09-19T04:46:02.551 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:46:02.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.545+0000 7feb5d800640 1 -- 172.21.15.123:0/1216399343 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7feb3c0a3350 msgr2=0x7feb3c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:02.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.545+0000 7feb5d800640 1 --2- 172.21.15.123:0/1216399343 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7feb3c0a3350 0x7feb3c0a5810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feb50005c40 tx=0x7feb50005b50 comp rx=0 tx=0).stop 2024-09-19T04:46:02.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.545+0000 7feb5d800640 1 -- 172.21.15.123:0/1216399343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb58102790 msgr2=0x7feb581988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:02.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.545+0000 7feb5d800640 1 --2- 172.21.15.123:0/1216399343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb58102790 0x7feb581988f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7feb4400d950 tx=0x7feb4400de10 comp rx=0 tx=0).stop 2024-09-19T04:46:02.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.545+0000 7feb5d800640 1 -- 172.21.15.123:0/1216399343 shutdown_connections 2024-09-19T04:46:02.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.545+0000 7feb5d800640 1 --2- 172.21.15.123:0/1216399343 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7feb3c0a3350 0x7feb3c0a5810 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.545+0000 7feb5d800640 1 --2- 172.21.15.123:0/1216399343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb58106f10 0x7feb5819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.545+0000 7feb5d800640 1 --2- 172.21.15.123:0/1216399343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb58102790 0x7feb581988f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.545+0000 7feb5d800640 1 --2- 172.21.15.123:0/1216399343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb58101de0 0x7feb581983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:02.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.545+0000 7feb5d800640 1 -- 172.21.15.123:0/1216399343 >> 172.21.15.123:0/1216399343 conn(0x7feb580fd8c0 msgr2=0x7feb580feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:02.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.545+0000 7feb5d800640 1 -- 172.21.15.123:0/1216399343 shutdown_connections 2024-09-19T04:46:02.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:02.545+0000 7feb5d800640 1 -- 172.21.15.123:0/1216399343 wait complete. 2024-09-19T04:46:02.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:02 smithi123 ceph-mon[30511]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:46:02.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:02 smithi183 ceph-mon[35333]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:46:02.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:02 smithi193 ceph-mon[35592]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:46:03.556 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T04:46:03.924 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:04.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.496+0000 7f8db3e00640 1 -- 172.21.15.123:0/2762866489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8db41019d0 msgr2=0x7f8db4101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:04.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.496+0000 7f8db3e00640 1 --2- 172.21.15.123:0/2762866489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8db41019d0 0x7f8db4101e30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009930 tx=0x7f8d9c02a420 comp rx=0 tx=0).stop 2024-09-19T04:46:04.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.496+0000 7f8db3e00640 1 -- 172.21.15.123:0/2762866489 shutdown_connections 2024-09-19T04:46:04.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.496+0000 7f8db3e00640 1 --2- 172.21.15.123:0/2762866489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8db4102370 0x7f8db410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.496+0000 7f8db3e00640 1 --2- 172.21.15.123:0/2762866489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8db41019d0 0x7f8db4101e30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.496+0000 7f8db3e00640 1 --2- 172.21.15.123:0/2762866489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8db41079d0 0x7f8db4107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.496+0000 7f8db3e00640 1 -- 172.21.15.123:0/2762866489 >> 172.21.15.123:0/2762866489 conn(0x7f8db40fd710 msgr2=0x7f8db40ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:04.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.496+0000 7f8db3e00640 1 -- 172.21.15.123:0/2762866489 shutdown_connections 2024-09-19T04:46:04.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.496+0000 7f8db3e00640 1 -- 172.21.15.123:0/2762866489 wait complete. 2024-09-19T04:46:04.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.497+0000 7f8db3e00640 1 Processor -- start 2024-09-19T04:46:04.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.497+0000 7f8db3e00640 1 -- start start 2024-09-19T04:46:04.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.497+0000 7f8db3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8db41019d0 0x7f8db41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.497+0000 7f8db3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8db4102370 0x7f8db4198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.498+0000 7f8db3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8db41079d0 0x7f8db419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.498+0000 7f8db3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8db4111de0 con 0x7f8db41079d0 2024-09-19T04:46:04.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.498+0000 7f8db3e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8db4111c60 con 0x7f8db41019d0 2024-09-19T04:46:04.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.498+0000 7f8db3e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8db4111f60 con 0x7f8db4102370 2024-09-19T04:46:04.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.498+0000 7f8db3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8db41079d0 0x7f8db419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.498+0000 7f8db3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8db41079d0 0x7f8db419cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53324/0 (socket says 172.21.15.123:53324) 2024-09-19T04:46:04.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.498+0000 7f8db3400640 1 -- 172.21.15.123:0/625017999 learned_addr learned my addr 172.21.15.123:0/625017999 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:04.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.498+0000 7f8db2a00640 1 --2- 172.21.15.123:0/625017999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8db41019d0 0x7f8db41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.498+0000 7f8dabe00640 1 --2- 172.21.15.123:0/625017999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8db4102370 0x7f8db4198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.498+0000 7f8db3400640 1 -- 172.21.15.123:0/625017999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8db4102370 msgr2=0x7f8db4198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:04.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.498+0000 7f8db3400640 1 --2- 172.21.15.123:0/625017999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8db4102370 0x7f8db4198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.498+0000 7f8db3400640 1 -- 172.21.15.123:0/625017999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8db41019d0 msgr2=0x7f8db41982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:04.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.498+0000 7f8db3400640 1 --2- 172.21.15.123:0/625017999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8db41019d0 0x7f8db41982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.498+0000 7f8db3400640 1 -- 172.21.15.123:0/625017999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db419d270 con 0x7f8db41079d0 2024-09-19T04:46:04.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.499+0000 7f8db3400640 1 --2- 172.21.15.123:0/625017999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8db41079d0 0x7f8db419cb90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00eaa0 tx=0x7f8dac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:04.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.499+0000 7f8db8a00640 1 -- 172.21.15.123:0/625017999 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac00cdb0 con 0x7f8db41079d0 2024-09-19T04:46:04.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.499+0000 7f8db8a00640 1 -- 172.21.15.123:0/625017999 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dac005020 con 0x7f8db41079d0 2024-09-19T04:46:04.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.499+0000 7f8db3e00640 1 -- 172.21.15.123:0/625017999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db419d560 con 0x7f8db41079d0 2024-09-19T04:46:04.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.499+0000 7f8db8a00640 1 -- 172.21.15.123:0/625017999 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac0116b0 con 0x7f8db41079d0 2024-09-19T04:46:04.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.499+0000 7f8db3e00640 1 -- 172.21.15.123:0/625017999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db41a4e40 con 0x7f8db41079d0 2024-09-19T04:46:04.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.503+0000 7f8dbca25740 1 Processor -- start 2024-09-19T04:46:04.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.503+0000 7f8db8a00640 1 -- 172.21.15.123:0/625017999 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8dac005560 con 0x7f8db41079d0 2024-09-19T04:46:04.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.503+0000 7f8dbca25740 1 -- start start 2024-09-19T04:46:04.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.504+0000 7f8dbca25740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec844e0 0x55df4ec848c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.504+0000 7f8dbca25740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec84e00 0x55df4ec8f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.504+0000 7f8dbca25740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec8f6c0 0x55df4ec91ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.504+0000 7f8dbca25740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55df4ebea0b0 con 0x55df4ec8f6c0 2024-09-19T04:46:04.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.504+0000 7f8dbca25740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55df4eb91ab0 con 0x55df4ec844e0 2024-09-19T04:46:04.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.504+0000 7f8dbca25740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55df4ebe6070 con 0x55df4ec84e00 2024-09-19T04:46:04.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.504+0000 7f8db2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec84e00 0x55df4ec8f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.504+0000 7f8db2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec84e00 0x55df4ec8f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:43116/0 (socket says 172.21.15.123:43116) 2024-09-19T04:46:04.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.504+0000 7f8db2a00640 1 -- 172.21.15.123:0/4061285335 learned_addr learned my addr 172.21.15.123:0/4061285335 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:04.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.504+0000 7f8dabe00640 1 --2- 172.21.15.123:0/4061285335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec8f6c0 0x55df4ec91ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.504+0000 7f8dabe00640 1 --2- 172.21.15.123:0/4061285335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec844e0 0x55df4ec848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.504+0000 7f8db2a00640 1 -- 172.21.15.123:0/4061285335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec844e0 msgr2=0x55df4ec848c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:04.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.504+0000 7f8db2a00640 1 --2- 172.21.15.123:0/4061285335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec844e0 0x55df4ec848c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.504+0000 7f8db2a00640 1 -- 172.21.15.123:0/4061285335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec8f6c0 msgr2=0x55df4ec91ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:04.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.504+0000 7f8db2a00640 1 --2- 172.21.15.123:0/4061285335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec8f6c0 0x55df4ec91ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.504+0000 7f8db2a00640 1 -- 172.21.15.123:0/4061285335 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df4eb9e4a0 con 0x55df4ec84e00 2024-09-19T04:46:04.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.504+0000 7f8dabe00640 1 --2- 172.21.15.123:0/4061285335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec8f6c0 0x55df4ec91ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:46:04.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.505+0000 7f8db2a00640 1 --2- 172.21.15.123:0/4061285335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec84e00 0x55df4ec8f180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009c70 tx=0x7f8da4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15a5f4002dd52181 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:04.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.505+0000 7f8da3e00640 1 -- 172.21.15.123:0/4061285335 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4004230 con 0x55df4ec84e00 2024-09-19T04:46:04.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.505+0000 7f8da3e00640 1 -- 172.21.15.123:0/4061285335 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8da40043d0 con 0x55df4ec84e00 2024-09-19T04:46:04.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.505+0000 7f8da3e00640 1 -- 172.21.15.123:0/4061285335 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da40197c0 con 0x55df4ec84e00 2024-09-19T04:46:04.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.505+0000 7f8db8a00640 1 --2- 172.21.15.123:0/625017999 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8d980a32d0 0x7f8d980a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.505+0000 7f8db8a00640 1 -- 172.21.15.123:0/625017999 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f8dac0cfc90 con 0x7f8db41079d0 2024-09-19T04:46:04.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.505+0000 7f8dabe00640 1 --2- 172.21.15.123:0/625017999 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8d980a32d0 0x7f8d980a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.506+0000 7f8dbca25740 1 -- 172.21.15.123:0/4061285335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec84e00 msgr2=0x55df4ec8f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:04.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.506+0000 7f8dbca25740 1 --2- 172.21.15.123:0/4061285335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec84e00 0x55df4ec8f180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009c70 tx=0x7f8da4011810 comp rx=0 tx=0).stop 2024-09-19T04:46:04.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.506+0000 7f8dbca25740 1 -- 172.21.15.123:0/4061285335 shutdown_connections 2024-09-19T04:46:04.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.506+0000 7f8dbca25740 1 --2- 172.21.15.123:0/4061285335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec8f6c0 0x55df4ec91ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.506+0000 7f8dbca25740 1 --2- 172.21.15.123:0/4061285335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec84e00 0x55df4ec8f180 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.506+0000 7f8dbca25740 1 --2- 172.21.15.123:0/4061285335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec844e0 0x55df4ec848c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.506+0000 7f8dbca25740 1 -- 172.21.15.123:0/4061285335 >> 172.21.15.123:0/4061285335 conn(0x55df4ec80a70 msgr2=0x55df4ec80e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:04.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.506+0000 7f8dbca25740 1 -- 172.21.15.123:0/4061285335 shutdown_connections 2024-09-19T04:46:04.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.506+0000 7f8dbca25740 1 -- 172.21.15.123:0/4061285335 wait complete. 2024-09-19T04:46:04.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.506+0000 7f8dabe00640 1 --2- 172.21.15.123:0/625017999 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8d980a32d0 0x7f8d980a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c006010 tx=0x7f8d9c02a420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:04.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.507+0000 7f8dbca25740 1 -- start start 2024-09-19T04:46:04.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.507+0000 7f8dbca25740 1 Processor -- start 2024-09-19T04:46:04.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.507+0000 7f8dbca25740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec844e0 0x55df4ec8c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.507+0000 7f8dbca25740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec84e00 0x55df4ec8caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.507+0000 7f8dbca25740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec8f6c0 0x55df4ec8cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.507+0000 7f8dbca25740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55df4ec92330 con 0x55df4ec84e00 2024-09-19T04:46:04.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.507+0000 7f8dbca25740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55df4eb72920 con 0x55df4ec8f6c0 2024-09-19T04:46:04.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.507+0000 7f8dbca25740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55df4ec924b0 con 0x55df4ec844e0 2024-09-19T04:46:04.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.507+0000 7f8dbca25740 10 client.0 _ll_register_callbacks cb 0x55df4eb968a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:04.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.508+0000 7f8db2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec844e0 0x55df4ec8c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.508+0000 7f8dabe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec84e00 0x55df4ec8caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.508+0000 7f8db3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec8f6c0 0x55df4ec8cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.508+0000 7f8db3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec8f6c0 0x55df4ec8cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:47368/0 (socket says 172.21.15.123:47368) 2024-09-19T04:46:04.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.508+0000 7f8db2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec844e0 0x55df4ec8c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:43132/0 (socket says 172.21.15.123:43132) 2024-09-19T04:46:04.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.508+0000 7f8db3400640 1 -- 172.21.15.123:0/2123761047 learned_addr learned my addr 172.21.15.123:0/2123761047 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:04.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.508+0000 7f8db3400640 1 -- 172.21.15.123:0/2123761047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec844e0 msgr2=0x55df4ec8c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:04.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.508+0000 7f8db3400640 1 --2- 172.21.15.123:0/2123761047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec844e0 0x55df4ec8c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.508+0000 7f8db3400640 1 -- 172.21.15.123:0/2123761047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec84e00 msgr2=0x55df4ec8caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:04.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.508+0000 7f8db3400640 1 --2- 172.21.15.123:0/2123761047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec84e00 0x55df4ec8caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.508+0000 7f8db3400640 1 -- 172.21.15.123:0/2123761047 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df4e95b2b0 con 0x55df4ec8f6c0 2024-09-19T04:46:04.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.508+0000 7f8db3400640 1 --2- 172.21.15.123:0/2123761047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec8f6c0 0x55df4ec8cfe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8dac09beb0 tx=0x7f8dac0a0570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b9dc2bd3642a7da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:04.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.508+0000 7f8da0c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:46:04.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.508+0000 7f8da0c00640 1 -- 172.21.15.123:0/2123761047 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac01f030 con 0x55df4ec8f6c0 2024-09-19T04:46:04.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.508+0000 7f8dbca25740 1 -- 172.21.15.123:0/2123761047 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8da40043d0 con 0x55df4ec8f6c0 2024-09-19T04:46:04.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.508+0000 7f8da0c00640 1 -- 172.21.15.123:0/2123761047 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f8dac0a0120 con 0x55df4ec8f6c0 2024-09-19T04:46:04.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.509+0000 7f8da0c00640 1 -- 172.21.15.123:0/2123761047 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8dac0cd900 con 0x55df4ec8f6c0 2024-09-19T04:46:04.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.509+0000 7f8dbca25740 10 client.24959 fetch_fsmap learned FSMap version 27 2024-09-19T04:46:04.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.509+0000 7f8dbca25740 1 -- 172.21.15.123:0/2123761047 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df4eca2450 con 0x55df4ec8f6c0 2024-09-19T04:46:04.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.509+0000 7f8da0c00640 1 -- 172.21.15.123:0/2123761047 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8da40043d0 con 0x55df4ec8f6c0 2024-09-19T04:46:04.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.509+0000 7f8da0c00640 1 -- 172.21.15.123:0/2123761047 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8dac00f040 con 0x55df4ec8f6c0 2024-09-19T04:46:04.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.509+0000 7f8dbca25740 10 client.24959 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:46:04.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.509+0000 7f8dbca25740 10 client.24959 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:46:04.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.509+0000 7f8dbca25740 20 client.24959 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:04.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.509+0000 7f8dbca25740 1 --2- 172.21.15.123:0/2123761047 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55df4eca4a80 0x55df4eca5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.509+0000 7f8dbca25740 4 client.24959 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:46:04.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.509+0000 7f8dbca25740 1 -- 172.21.15.123:0/2123761047 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df4ebf7620 con 0x55df4eca4a80 2024-09-19T04:46:04.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.510+0000 7f8db2a00640 1 --2- 172.21.15.123:0/2123761047 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55df4eca4a80 0x55df4eca5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.510+0000 7f8db2a00640 1 --2- 172.21.15.123:0/2123761047 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55df4eca4a80 0x55df4eca5ef0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9432f0eb68048a22 server_cookie=eb962dc7f2da579a in_seq=0 out_seq=0 2024-09-19T04:46:04.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.510+0000 7f8da0c00640 10 client.24959 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:46:04.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.511+0000 7f8da0c00640 1 -- 172.21.15.123:0/2123761047 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df4ebf7620 con 0x55df4eca4a80 2024-09-19T04:46:04.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.511+0000 7f8da0c00640 10 client.24959 handle_command_reply: tid=0 2024-09-19T04:46:04.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.511+0000 7f8dbca25740 1 client.24959 shutdown 2024-09-19T04:46:04.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.512+0000 7f8dbca25740 1 -- 172.21.15.123:0/2123761047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec8f6c0 msgr2=0x55df4ec8cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:04.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.512+0000 7f8dbca25740 1 --2- 172.21.15.123:0/2123761047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec8f6c0 0x55df4ec8cfe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8dac09beb0 tx=0x7f8dac0a0570 comp rx=0 tx=0).stop 2024-09-19T04:46:04.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.512+0000 7f8dbca25740 1 -- 172.21.15.123:0/2123761047 shutdown_connections 2024-09-19T04:46:04.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.512+0000 7f8dbca25740 1 --2- 172.21.15.123:0/2123761047 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55df4eca4a80 0x55df4eca5ef0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.512+0000 7f8dbca25740 1 --2- 172.21.15.123:0/2123761047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec8f6c0 0x55df4ec8cfe0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.512+0000 7f8dbca25740 1 --2- 172.21.15.123:0/2123761047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec84e00 0x55df4ec8caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.512+0000 7f8dbca25740 1 --2- 172.21.15.123:0/2123761047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec844e0 0x55df4ec8c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.512+0000 7f8dbca25740 1 -- 172.21.15.123:0/2123761047 >> 172.21.15.123:0/2123761047 conn(0x55df4ec80a70 msgr2=0x55df4ec86750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:04.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.512+0000 7f8da0c00640 0 client.24959 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:46:04.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.512+0000 7f8dbca25740 1 -- 172.21.15.123:0/2123761047 shutdown_connections 2024-09-19T04:46:04.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.512+0000 7f8dbca25740 1 -- 172.21.15.123:0/2123761047 wait complete. 2024-09-19T04:46:04.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.512+0000 7f8dbca25740 20 client.24959 trim_cache size 0 max 16384 2024-09-19T04:46:04.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.530+0000 7f8dbca25740 1 Processor -- start 2024-09-19T04:46:04.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.531+0000 7f8dbca25740 1 -- start start 2024-09-19T04:46:04.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.531+0000 7f8dbca25740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec844e0 0x55df4eca5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.531+0000 7f8dbca25740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec84e00 0x55df4ec8ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.531+0000 7f8dbca25740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec8f6c0 0x55df4ec8b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.531+0000 7f8dbca25740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55df4eca4230 con 0x55df4ec84e00 2024-09-19T04:46:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.531+0000 7f8dbca25740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55df4eca4900 con 0x55df4ec8f6c0 2024-09-19T04:46:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.531+0000 7f8dbca25740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55df4eca4fd0 con 0x55df4ec844e0 2024-09-19T04:46:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.531+0000 7f8dabe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec84e00 0x55df4ec8ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.531+0000 7f8dabe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec84e00 0x55df4ec8ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53346/0 (socket says 172.21.15.123:53346) 2024-09-19T04:46:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.531+0000 7f8dabe00640 1 -- 172.21.15.123:0/1920835758 learned_addr learned my addr 172.21.15.123:0/1920835758 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.531+0000 7f8db3400640 1 --2- 172.21.15.123:0/1920835758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec8f6c0 0x55df4ec8b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.531+0000 7f8db2a00640 1 --2- 172.21.15.123:0/1920835758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec844e0 0x55df4eca5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.532+0000 7f8dabe00640 1 -- 172.21.15.123:0/1920835758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec844e0 msgr2=0x55df4eca5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.532+0000 7f8dabe00640 1 --2- 172.21.15.123:0/1920835758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec844e0 0x55df4eca5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.532+0000 7f8dabe00640 1 -- 172.21.15.123:0/1920835758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec8f6c0 msgr2=0x55df4ec8b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.532+0000 7f8dabe00640 1 --2- 172.21.15.123:0/1920835758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec8f6c0 0x55df4ec8b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.532+0000 7f8dabe00640 1 -- 172.21.15.123:0/1920835758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df4ec8aab0 con 0x55df4ec84e00 2024-09-19T04:46:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.532+0000 7f8db3400640 1 --2- 172.21.15.123:0/1920835758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec8f6c0 0x55df4ec8b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.532+0000 7f8dabe00640 1 --2- 172.21.15.123:0/1920835758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec84e00 0x55df4ec8ae30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009930 tx=0x7f8d9c041070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82531a47327a3ebf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.532+0000 7f8da3400640 1 -- 172.21.15.123:0/1920835758 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c0317b0 con 0x55df4ec84e00 2024-09-19T04:46:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.532+0000 7f8da3400640 1 -- 172.21.15.123:0/1920835758 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d9c031e30 con 0x55df4ec84e00 2024-09-19T04:46:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.532+0000 7f8da3400640 1 -- 172.21.15.123:0/1920835758 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c03ddb0 con 0x55df4ec84e00 2024-09-19T04:46:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.533+0000 7f8dbca25740 1 -- 172.21.15.123:0/1920835758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec84e00 msgr2=0x55df4ec8ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.533+0000 7f8dbca25740 1 --2- 172.21.15.123:0/1920835758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec84e00 0x55df4ec8ae30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009930 tx=0x7f8d9c041070 comp rx=0 tx=0).stop 2024-09-19T04:46:04.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.533+0000 7f8dbca25740 1 -- 172.21.15.123:0/1920835758 shutdown_connections 2024-09-19T04:46:04.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.533+0000 7f8dbca25740 1 --2- 172.21.15.123:0/1920835758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec8f6c0 0x55df4ec8b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.533+0000 7f8dbca25740 1 --2- 172.21.15.123:0/1920835758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec84e00 0x55df4ec8ae30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.533+0000 7f8dbca25740 1 --2- 172.21.15.123:0/1920835758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec844e0 0x55df4eca5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.533+0000 7f8dbca25740 1 -- 172.21.15.123:0/1920835758 >> 172.21.15.123:0/1920835758 conn(0x55df4ec80a70 msgr2=0x55df4ec87860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:04.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.533+0000 7f8dbca25740 1 -- 172.21.15.123:0/1920835758 shutdown_connections 2024-09-19T04:46:04.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.533+0000 7f8dbca25740 1 -- 172.21.15.123:0/1920835758 wait complete. 2024-09-19T04:46:04.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.533+0000 7f8dbca25740 1 -- start start 2024-09-19T04:46:04.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.534+0000 7f8dbca25740 1 Processor -- start 2024-09-19T04:46:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.534+0000 7f8dbca25740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec844e0 0x55df4ecae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.534+0000 7f8dbca25740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec84e00 0x55df4eca7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.534+0000 7f8dbca25740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec8f6c0 0x55df4eca8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.534+0000 7f8dbca25740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55df4eca5d70 con 0x55df4ec8f6c0 2024-09-19T04:46:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.534+0000 7f8dbca25740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55df4eca56a0 con 0x55df4ec84e00 2024-09-19T04:46:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.534+0000 7f8dbca25740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55df4eca6440 con 0x55df4ec844e0 2024-09-19T04:46:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.534+0000 7f8dbca25740 10 client.0 _ll_register_callbacks cb 0x7f8dac09c220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.534+0000 7f8db3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec8f6c0 0x55df4eca8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.534+0000 7f8dabe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec84e00 0x55df4eca7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.534+0000 7f8dabe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec84e00 0x55df4eca7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:47386/0 (socket says 172.21.15.123:47386) 2024-09-19T04:46:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.534+0000 7f8dabe00640 1 -- 172.21.15.123:0/2395661038 learned_addr learned my addr 172.21.15.123:0/2395661038 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.534+0000 7f8db2a00640 1 --2- 172.21.15.123:0/2395661038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec844e0 0x55df4ecae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.535+0000 7f8db3400640 1 -- 172.21.15.123:0/2395661038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec844e0 msgr2=0x55df4ecae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.535+0000 7f8db3400640 1 --2- 172.21.15.123:0/2395661038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec844e0 0x55df4ecae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.535+0000 7f8db3400640 1 -- 172.21.15.123:0/2395661038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec84e00 msgr2=0x55df4eca7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.535+0000 7f8db3400640 1 --2- 172.21.15.123:0/2395661038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec84e00 0x55df4eca7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.535+0000 7f8db3400640 1 -- 172.21.15.123:0/2395661038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df4eca8940 con 0x55df4ec8f6c0 2024-09-19T04:46:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.535+0000 7f8dabe00640 1 --2- 172.21.15.123:0/2395661038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec84e00 0x55df4eca7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.535+0000 7f8db3400640 1 --2- 172.21.15.123:0/2395661038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec8f6c0 0x55df4eca8130 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00efd0 tx=0x7f8dac0bc2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9864e6922f2a05b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.535+0000 7f8da0c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:46:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.535+0000 7f8da0c00640 1 -- 172.21.15.123:0/2395661038 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac01f040 con 0x55df4ec8f6c0 2024-09-19T04:46:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.535+0000 7f8da0c00640 1 -- 172.21.15.123:0/2395661038 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f8dac09b600 con 0x55df4ec8f6c0 2024-09-19T04:46:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.535+0000 7f8dbca25740 1 -- 172.21.15.123:0/2395661038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d9c031e30 con 0x55df4ec8f6c0 2024-09-19T04:46:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.536+0000 7f8da0c00640 1 -- 172.21.15.123:0/2395661038 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8dac09b260 con 0x55df4ec8f6c0 2024-09-19T04:46:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.536+0000 7f8dbca25740 10 client.14946 fetch_fsmap learned FSMap version 27 2024-09-19T04:46:04.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.536+0000 7f8dbca25740 1 -- 172.21.15.123:0/2395661038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df4ed0f190 con 0x55df4ec8f6c0 2024-09-19T04:46:04.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.536+0000 7f8da0c00640 1 -- 172.21.15.123:0/2395661038 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d9c031e30 con 0x55df4ec8f6c0 2024-09-19T04:46:04.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.536+0000 7f8da0c00640 1 -- 172.21.15.123:0/2395661038 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8dac004a40 con 0x55df4ec8f6c0 2024-09-19T04:46:04.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.536+0000 7f8dbca25740 10 client.14946 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:46:04.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.536+0000 7f8dbca25740 10 client.14946 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:46:04.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.536+0000 7f8dbca25740 20 client.14946 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:04.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.536+0000 7f8dbca25740 1 --2- 172.21.15.123:0/2395661038 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55df4ed13c80 0x55df4ed12990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:04.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.536+0000 7f8dbca25740 4 client.14946 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:46:04.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.536+0000 7f8dbca25740 1 -- 172.21.15.123:0/2395661038 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "status"}) -- 0x55df4ebf1b30 con 0x55df4ed13c80 2024-09-19T04:46:04.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.537+0000 7f8db2a00640 1 --2- 172.21.15.123:0/2395661038 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55df4ed13c80 0x55df4ed12990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:04.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.537+0000 7f8db2a00640 1 --2- 172.21.15.123:0/2395661038 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55df4ed13c80 0x55df4ed12990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18c582b63afb8751 server_cookie=58fc1660b8537cea in_seq=0 out_seq=0 2024-09-19T04:46:04.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.537+0000 7f8da0c00640 10 client.14946 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:46:04.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.537+0000 7f8da0c00640 1 -- 172.21.15.123:0/2395661038 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55df4ebf1b30 con 0x55df4ed13c80 2024-09-19T04:46:04.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.537+0000 7f8da0c00640 10 client.14946 handle_command_reply: tid=0 2024-09-19T04:46:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.537+0000 7f8dbca25740 1 client.14946 shutdown 2024-09-19T04:46:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.538+0000 7f8dbca25740 1 -- 172.21.15.123:0/2395661038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec8f6c0 msgr2=0x55df4eca8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.538+0000 7f8dbca25740 1 --2- 172.21.15.123:0/2395661038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec8f6c0 0x55df4eca8130 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00efd0 tx=0x7f8dac0bc2e0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.538+0000 7f8dbca25740 1 -- 172.21.15.123:0/2395661038 shutdown_connections 2024-09-19T04:46:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.538+0000 7f8dbca25740 1 --2- 172.21.15.123:0/2395661038 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55df4ed13c80 0x55df4ed12990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.538+0000 7f8dbca25740 1 --2- 172.21.15.123:0/2395661038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4ec8f6c0 0x55df4eca8130 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.538+0000 7f8dbca25740 1 --2- 172.21.15.123:0/2395661038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df4ec84e00 0x55df4eca7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.538+0000 7f8dbca25740 1 --2- 172.21.15.123:0/2395661038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4ec844e0 0x55df4ecae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.538+0000 7f8dbca25740 1 -- 172.21.15.123:0/2395661038 >> 172.21.15.123:0/2395661038 conn(0x55df4ec80a70 msgr2=0x55df4ec8acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.538+0000 7f8da0c00640 0 client.14946 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:46:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.538+0000 7f8dbca25740 1 -- 172.21.15.123:0/2395661038 shutdown_connections 2024-09-19T04:46:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.538+0000 7f8dbca25740 1 -- 172.21.15.123:0/2395661038 wait complete. 2024-09-19T04:46:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.538+0000 7f8dbca25740 20 client.14946 trim_cache size 0 max 16384 2024-09-19T04:46:04.544 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:46:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "911fa052-7640-11ef-bceb-c7b262605968", 2024-09-19T04:46:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 0, 2024-09-19T04:46:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24539, 2024-09-19T04:46:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-09-19T04:46:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-09-19T04:46:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-09-19T04:46:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 91.791418065000002, 2024-09-19T04:46:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 27, 2024-09-19T04:46:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 86, 2024-09-19T04:46:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T04:46:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 105.28546946100001 2024-09-19T04:46:04.545 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:46:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.539+0000 7f8db3e00640 1 -- 172.21.15.123:0/625017999 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8d980a32d0 msgr2=0x7f8d980a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.539+0000 7f8db3e00640 1 --2- 172.21.15.123:0/625017999 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8d980a32d0 0x7f8d980a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c006010 tx=0x7f8d9c02a420 comp rx=0 tx=0).stop 2024-09-19T04:46:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.539+0000 7f8db3e00640 1 -- 172.21.15.123:0/625017999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8db41079d0 msgr2=0x7f8db419cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.539+0000 7f8db3e00640 1 --2- 172.21.15.123:0/625017999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8db41079d0 0x7f8db419cb90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00eaa0 tx=0x7f8dac00ef60 comp rx=0 tx=0).stop 2024-09-19T04:46:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.539+0000 7f8db3e00640 1 -- 172.21.15.123:0/625017999 shutdown_connections 2024-09-19T04:46:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.539+0000 7f8db3e00640 1 --2- 172.21.15.123:0/625017999 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8d980a32d0 0x7f8d980a5790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.539+0000 7f8db3e00640 1 --2- 172.21.15.123:0/625017999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8db41079d0 0x7f8db419cb90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.539+0000 7f8db3e00640 1 --2- 172.21.15.123:0/625017999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8db4102370 0x7f8db4198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.539+0000 7f8db3e00640 1 --2- 172.21.15.123:0/625017999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8db41019d0 0x7f8db41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.539+0000 7f8db3e00640 1 -- 172.21.15.123:0/625017999 >> 172.21.15.123:0/625017999 conn(0x7f8db40fd710 msgr2=0x7f8db40fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.539+0000 7f8db3e00640 1 -- 172.21.15.123:0/625017999 shutdown_connections 2024-09-19T04:46:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:04.539+0000 7f8db3e00640 1 -- 172.21.15.123:0/625017999 wait complete. 2024-09-19T04:46:04.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:04 smithi123 ceph-mon[30511]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:46:04.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:04 smithi183 ceph-mon[35333]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:46:04.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:04 smithi193 ceph-mon[35592]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:46:05.383 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T04:46:05.383 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T04:46:05.395 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T04:46:05.395 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T04:46:05.768 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:06.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.323+0000 7f31e0e00640 1 -- 172.21.15.123:0/3278913544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31dc102c40 msgr2=0x7f31dc1030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:06.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.323+0000 7f31e0e00640 1 --2- 172.21.15.123:0/3278913544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31dc102c40 0x7f31dc1030c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009860 tx=0x7f31c802a340 comp rx=0 tx=0).stop 2024-09-19T04:46:06.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.323+0000 7f31e0e00640 1 -- 172.21.15.123:0/3278913544 shutdown_connections 2024-09-19T04:46:06.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.323+0000 7f31e0e00640 1 --2- 172.21.15.123:0/3278913544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31dc103600 0x7f31dc109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:06.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.323+0000 7f31e0e00640 1 --2- 172.21.15.123:0/3278913544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31dc102c40 0x7f31dc1030c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:06.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.323+0000 7f31e0e00640 1 --2- 172.21.15.123:0/3278913544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31dc101a40 0x7f31dc101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:06.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.323+0000 7f31e0e00640 1 -- 172.21.15.123:0/3278913544 >> 172.21.15.123:0/3278913544 conn(0x7f31dc0fd1f0 msgr2=0x7f31dc0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:06.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.323+0000 7f31e0e00640 1 -- 172.21.15.123:0/3278913544 shutdown_connections 2024-09-19T04:46:06.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.323+0000 7f31e0e00640 1 -- 172.21.15.123:0/3278913544 wait complete. 2024-09-19T04:46:06.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.324+0000 7f31e0e00640 1 Processor -- start 2024-09-19T04:46:06.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.325+0000 7f31e0e00640 1 -- start start 2024-09-19T04:46:06.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.325+0000 7f31e0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31dc101a40 0x7f31dc199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:06.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.325+0000 7f31e0e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31dc102c40 0x7f31dc19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:06.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.325+0000 7f31e0e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31dc103600 0x7f31dc1a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:06.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.325+0000 7f31e0e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc10d6c0 con 0x7f31dc101a40 2024-09-19T04:46:06.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.325+0000 7f31e0e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc10d540 con 0x7f31dc102c40 2024-09-19T04:46:06.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.325+0000 7f31e0e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc10d840 con 0x7f31dc103600 2024-09-19T04:46:06.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.325+0000 7f31db400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31dc101a40 0x7f31dc199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:06.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.325+0000 7f31db400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31dc101a40 0x7f31dc199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53380/0 (socket says 172.21.15.123:53380) 2024-09-19T04:46:06.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.325+0000 7f31db400640 1 -- 172.21.15.123:0/3910255286 learned_addr learned my addr 172.21.15.123:0/3910255286 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:06.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.325+0000 7f31dbe00640 1 --2- 172.21.15.123:0/3910255286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31dc103600 0x7f31dc1a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:06.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.325+0000 7f31daa00640 1 --2- 172.21.15.123:0/3910255286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31dc102c40 0x7f31dc19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:06.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.326+0000 7f31db400640 1 -- 172.21.15.123:0/3910255286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31dc103600 msgr2=0x7f31dc1a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:06.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.326+0000 7f31db400640 1 --2- 172.21.15.123:0/3910255286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31dc103600 0x7f31dc1a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:06.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.326+0000 7f31db400640 1 -- 172.21.15.123:0/3910255286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31dc102c40 msgr2=0x7f31dc19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:06.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.326+0000 7f31db400640 1 --2- 172.21.15.123:0/3910255286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31dc102c40 0x7f31dc19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:06.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.326+0000 7f31db400640 1 -- 172.21.15.123:0/3910255286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31dc1a1b60 con 0x7f31dc101a40 2024-09-19T04:46:06.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.326+0000 7f31db400640 1 --2- 172.21.15.123:0/3910255286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31dc101a40 0x7f31dc199f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00e9a0 tx=0x7f31cc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:06.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.326+0000 7f31d3e00640 1 -- 172.21.15.123:0/3910255286 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc00cd80 con 0x7f31dc101a40 2024-09-19T04:46:06.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.326+0000 7f31d3e00640 1 -- 172.21.15.123:0/3910255286 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31cc010910 con 0x7f31dc101a40 2024-09-19T04:46:06.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.326+0000 7f31d3e00640 1 -- 172.21.15.123:0/3910255286 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc018a30 con 0x7f31dc101a40 2024-09-19T04:46:06.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.326+0000 7f31e0e00640 1 -- 172.21.15.123:0/3910255286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31dc1a1e50 con 0x7f31dc101a40 2024-09-19T04:46:06.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.326+0000 7f31e0e00640 1 -- 172.21.15.123:0/3910255286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31dc1a2250 con 0x7f31dc101a40 2024-09-19T04:46:06.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.328+0000 7f31e0e00640 1 -- 172.21.15.123:0/3910255286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31dc1a24e0 con 0x7f31dc101a40 2024-09-19T04:46:06.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.329+0000 7f31d3e00640 1 -- 172.21.15.123:0/3910255286 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f31cc010ac0 con 0x7f31dc101a40 2024-09-19T04:46:06.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.331+0000 7f31d3e00640 1 --2- 172.21.15.123:0/3910255286 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f31c00a3380 0x7f31c00a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:06.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.331+0000 7f31d3e00640 1 -- 172.21.15.123:0/3910255286 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f31cc0cfda0 con 0x7f31dc101a40 2024-09-19T04:46:06.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.331+0000 7f31daa00640 1 --2- 172.21.15.123:0/3910255286 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f31c00a3380 0x7f31c00a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:06.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.332+0000 7f31daa00640 1 --2- 172.21.15.123:0/3910255286 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f31c00a3380 0x7f31c00a5840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f31c802a930 tx=0x7f31c8030570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:06.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.335+0000 7f31d3e00640 1 -- 172.21.15.123:0/3910255286 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f31cc054210 con 0x7f31dc101a40 2024-09-19T04:46:06.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.525+0000 7f31e0e00640 1 -- 172.21.15.123:0/3910255286 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f31dc075b40 con 0x7f31c00a3380 2024-09-19T04:46:06.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.526+0000 7f31d3e00640 1 -- 172.21.15.123:0/3910255286 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f31dc075b40 con 0x7f31c00a3380 2024-09-19T04:46:06.528 INFO:teuthology.orchestra.run.smithi123.stdout:Backend: cephadm 2024-09-19T04:46:06.528 INFO:teuthology.orchestra.run.smithi123.stdout:Available: Yes 2024-09-19T04:46:06.528 INFO:teuthology.orchestra.run.smithi123.stdout:Paused: No 2024-09-19T04:46:06.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.529+0000 7f31e0e00640 1 -- 172.21.15.123:0/3910255286 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f31c00a3380 msgr2=0x7f31c00a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:06.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.529+0000 7f31e0e00640 1 --2- 172.21.15.123:0/3910255286 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f31c00a3380 0x7f31c00a5840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f31c802a930 tx=0x7f31c8030570 comp rx=0 tx=0).stop 2024-09-19T04:46:06.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.529+0000 7f31e0e00640 1 -- 172.21.15.123:0/3910255286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31dc101a40 msgr2=0x7f31dc199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:06.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.529+0000 7f31e0e00640 1 --2- 172.21.15.123:0/3910255286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31dc101a40 0x7f31dc199f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00e9a0 tx=0x7f31cc00ee60 comp rx=0 tx=0).stop 2024-09-19T04:46:06.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.529+0000 7f31e0e00640 1 -- 172.21.15.123:0/3910255286 shutdown_connections 2024-09-19T04:46:06.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.529+0000 7f31e0e00640 1 --2- 172.21.15.123:0/3910255286 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f31c00a3380 0x7f31c00a5840 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:06.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.529+0000 7f31e0e00640 1 --2- 172.21.15.123:0/3910255286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31dc103600 0x7f31dc1a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:06.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.529+0000 7f31e0e00640 1 --2- 172.21.15.123:0/3910255286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31dc102c40 0x7f31dc19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:06.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.529+0000 7f31e0e00640 1 --2- 172.21.15.123:0/3910255286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31dc101a40 0x7f31dc199f30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:06.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.529+0000 7f31e0e00640 1 -- 172.21.15.123:0/3910255286 >> 172.21.15.123:0/3910255286 conn(0x7f31dc0fd1f0 msgr2=0x7f31dc074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:06.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.529+0000 7f31e0e00640 1 -- 172.21.15.123:0/3910255286 shutdown_connections 2024-09-19T04:46:06.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:06.529+0000 7f31e0e00640 1 -- 172.21.15.123:0/3910255286 wait complete. 2024-09-19T04:46:06.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:06 smithi183 ceph-mon[35333]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:46:06.876 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:06 smithi123 ceph-mon[30511]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:46:06.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:06 smithi193 ceph-mon[35592]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:46:07.400 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T04:46:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:07 smithi123 ceph-mon[30511]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:07.769 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:07 smithi183 ceph-mon[35333]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:07.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:07 smithi193 ceph-mon[35592]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:08.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.374+0000 7f3078800640 1 -- 172.21.15.123:0/3855883230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30701037c0 msgr2=0x7f307010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:08.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.374+0000 7f3078800640 1 --2- 172.21.15.123:0/3855883230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30701037c0 0x7f307010a050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f306c009ca0 tx=0x7f306c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:46:08.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.374+0000 7f3078800640 1 -- 172.21.15.123:0/3855883230 shutdown_connections 2024-09-19T04:46:08.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.374+0000 7f3078800640 1 --2- 172.21.15.123:0/3855883230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30701037c0 0x7f307010a050 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:08.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.374+0000 7f3078800640 1 --2- 172.21.15.123:0/3855883230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3070102e00 0x7f3070103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:08.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.374+0000 7f3078800640 1 --2- 172.21.15.123:0/3855883230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3070101c00 0x7f3070102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:08.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.374+0000 7f3078800640 1 -- 172.21.15.123:0/3855883230 >> 172.21.15.123:0/3855883230 conn(0x7f30700fd390 msgr2=0x7f30700ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:08.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.374+0000 7f3078800640 1 -- 172.21.15.123:0/3855883230 shutdown_connections 2024-09-19T04:46:08.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.375+0000 7f3078800640 1 -- 172.21.15.123:0/3855883230 wait complete. 2024-09-19T04:46:08.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.375+0000 7f3078800640 1 Processor -- start 2024-09-19T04:46:08.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.376+0000 7f3078800640 1 -- start start 2024-09-19T04:46:08.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.376+0000 7f3078800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3070101c00 0x7f307019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:08.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.376+0000 7f3078800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3070102e00 0x7f307019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:08.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.376+0000 7f3078800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30701037c0 0x7f30701a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:08.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.376+0000 7f3078800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f307010d8d0 con 0x7f3070102e00 2024-09-19T04:46:08.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.376+0000 7f3078800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f307010d750 con 0x7f30701037c0 2024-09-19T04:46:08.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.376+0000 7f3078800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f307010da50 con 0x7f3070101c00 2024-09-19T04:46:08.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.376+0000 7f3075400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3070102e00 0x7f307019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:08.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.376+0000 7f3075e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3070101c00 0x7f307019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:08.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.376+0000 7f3075400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3070102e00 0x7f307019ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53400/0 (socket says 172.21.15.123:53400) 2024-09-19T04:46:08.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.376+0000 7f3075400640 1 -- 172.21.15.123:0/3665187865 learned_addr learned my addr 172.21.15.123:0/3665187865 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:08.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.376+0000 7f3076800640 1 --2- 172.21.15.123:0/3665187865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30701037c0 0x7f30701a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:08.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.377+0000 7f3075400640 1 -- 172.21.15.123:0/3665187865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3070101c00 msgr2=0x7f307019e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:08.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.377+0000 7f3075400640 1 --2- 172.21.15.123:0/3665187865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3070101c00 0x7f307019e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:08.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.377+0000 7f3075400640 1 -- 172.21.15.123:0/3665187865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30701037c0 msgr2=0x7f30701a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:08.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.377+0000 7f3075400640 1 --2- 172.21.15.123:0/3665187865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30701037c0 0x7f30701a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:08.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.377+0000 7f3075400640 1 -- 172.21.15.123:0/3665187865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30701a6150 con 0x7f3070102e00 2024-09-19T04:46:08.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.377+0000 7f3075e00640 1 --2- 172.21.15.123:0/3665187865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3070101c00 0x7f307019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:08.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.377+0000 7f3076800640 1 --2- 172.21.15.123:0/3665187865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30701037c0 0x7f30701a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:46:08.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.377+0000 7f3075400640 1 --2- 172.21.15.123:0/3665187865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3070102e00 0x7f307019ea60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f306000e9f0 tx=0x7f306000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:08.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.377+0000 7f306aa00640 1 -- 172.21.15.123:0/3665187865 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306000cd80 con 0x7f3070102e00 2024-09-19T04:46:08.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.377+0000 7f306aa00640 1 -- 172.21.15.123:0/3665187865 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3060005500 con 0x7f3070102e00 2024-09-19T04:46:08.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.377+0000 7f3078800640 1 -- 172.21.15.123:0/3665187865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30701a6440 con 0x7f3070102e00 2024-09-19T04:46:08.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.377+0000 7f306aa00640 1 -- 172.21.15.123:0/3665187865 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060010690 con 0x7f3070102e00 2024-09-19T04:46:08.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.377+0000 7f3078800640 1 -- 172.21.15.123:0/3665187865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30701a6930 con 0x7f3070102e00 2024-09-19T04:46:08.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.378+0000 7f3078800640 1 -- 172.21.15.123:0/3665187865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30701a6bf0 con 0x7f3070102e00 2024-09-19T04:46:08.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.381+0000 7f306aa00640 1 -- 172.21.15.123:0/3665187865 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f30600046c0 con 0x7f3070102e00 2024-09-19T04:46:08.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.382+0000 7f306aa00640 1 --2- 172.21.15.123:0/3665187865 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f30580a3300 0x7f30580a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:08.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.382+0000 7f306aa00640 1 -- 172.21.15.123:0/3665187865 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f30600cff70 con 0x7f3070102e00 2024-09-19T04:46:08.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.382+0000 7f3075e00640 1 --2- 172.21.15.123:0/3665187865 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f30580a3300 0x7f30580a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:08.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.383+0000 7f3075e00640 1 --2- 172.21.15.123:0/3665187865 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f30580a3300 0x7f30580a57c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3070102c60 tx=0x7f3064005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:08.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.386+0000 7f306aa00640 1 -- 172.21.15.123:0/3665187865 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f306009cae0 con 0x7f3070102e00 2024-09-19T04:46:08.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.570+0000 7f3078800640 1 -- 172.21.15.123:0/3665187865 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f3070075b40 con 0x7f30580a3300 2024-09-19T04:46:08.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.581+0000 7f306aa00640 1 -- 172.21.15.123:0/3665187865 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f3070075b40 con 0x7f30580a3300 2024-09-19T04:46:08.583 INFO:teuthology.orchestra.run.smithi123.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T04:46:08.583 INFO:teuthology.orchestra.run.smithi123.stdout:mds.a smithi123 running (112s) 25s ago 111s 15.5M - 19.1.1-361-g2f599a46 d2baa855c067 87f469624551 2024-09-19T04:46:08.583 INFO:teuthology.orchestra.run.smithi123.stdout:mds.b smithi183 running (109s) 39s ago 109s 25.9M - 19.1.1-361-g2f599a46 d2baa855c067 711b223d35df 2024-09-19T04:46:08.583 INFO:teuthology.orchestra.run.smithi123.stdout:mds.c smithi193 running (2m) 39s ago 2m 19.5M - 19.1.1-361-g2f599a46 d2baa855c067 dafb2eeaedcb 2024-09-19T04:46:08.583 INFO:teuthology.orchestra.run.smithi123.stdout:mds.d smithi123 running (2m) 25s ago 2m 15.6M - 19.1.1-361-g2f599a46 d2baa855c067 fd9818a3f941 2024-09-19T04:46:08.583 INFO:teuthology.orchestra.run.smithi123.stdout:mds.e smithi183 running (2m) 39s ago 2m 20.6M - 19.1.1-361-g2f599a46 d2baa855c067 c61564e5ddba 2024-09-19T04:46:08.583 INFO:teuthology.orchestra.run.smithi123.stdout:mds.f smithi193 running (2m) 39s ago 2m 18.0M - 19.1.1-361-g2f599a46 d2baa855c067 0f3bca754953 2024-09-19T04:46:08.583 INFO:teuthology.orchestra.run.smithi123.stdout:mds.g smithi123 running (115s) 25s ago 115s 12.7M - 19.1.1-361-g2f599a46 d2baa855c067 bfae55e6fe72 2024-09-19T04:46:08.583 INFO:teuthology.orchestra.run.smithi123.stdout:mds.h smithi183 running (2m) 39s ago 2m 17.8M - 19.1.1-361-g2f599a46 d2baa855c067 7bb03127b5dd 2024-09-19T04:46:08.584 INFO:teuthology.orchestra.run.smithi123.stdout:mds.i smithi193 running (119s) 39s ago 119s 17.2M - 19.1.1-361-g2f599a46 d2baa855c067 00b1c8207448 2024-09-19T04:46:08.584 INFO:teuthology.orchestra.run.smithi123.stdout:mds.j smithi123 running (2m) 25s ago 2m 12.8M - 19.1.1-361-g2f599a46 d2baa855c067 984c514baed2 2024-09-19T04:46:08.584 INFO:teuthology.orchestra.run.smithi123.stdout:mds.k smithi183 running (2m) 39s ago 2m 18.4M - 19.1.1-361-g2f599a46 d2baa855c067 a5492af672c9 2024-09-19T04:46:08.584 INFO:teuthology.orchestra.run.smithi123.stdout:mds.l smithi193 running (2m) 39s ago 2m 17.0M - 19.1.1-361-g2f599a46 d2baa855c067 c44754603a22 2024-09-19T04:46:08.584 INFO:teuthology.orchestra.run.smithi123.stdout:mgr.x smithi123 *:9283,8765 running (9m) 25s ago 9m 509M - 19.1.1-361-g2f599a46 d2baa855c067 1f7915704579 2024-09-19T04:46:08.584 INFO:teuthology.orchestra.run.smithi123.stdout:mgr.y smithi183 *:8443,8765 running (7m) 39s ago 7m 441M - 19.1.1-361-g2f599a46 d2baa855c067 3fca84f03bdc 2024-09-19T04:46:08.584 INFO:teuthology.orchestra.run.smithi123.stdout:mgr.z smithi193 *:8443,8765 running (7m) 39s ago 7m 443M - 19.1.1-361-g2f599a46 d2baa855c067 e97a36cb7ef5 2024-09-19T04:46:08.584 INFO:teuthology.orchestra.run.smithi123.stdout:mon.a smithi123 running (9m) 25s ago 9m 48.2M 2048M 19.1.1-361-g2f599a46 d2baa855c067 16e47864cb79 2024-09-19T04:46:08.584 INFO:teuthology.orchestra.run.smithi123.stdout:mon.b smithi183 running (8m) 39s ago 8m 42.1M 2048M 19.1.1-361-g2f599a46 d2baa855c067 cb785711cbc8 2024-09-19T04:46:08.584 INFO:teuthology.orchestra.run.smithi123.stdout:mon.c smithi193 running (8m) 39s ago 8m 41.0M 2048M 19.1.1-361-g2f599a46 d2baa855c067 8e497ed1e57b 2024-09-19T04:46:08.584 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0 smithi123 running (7m) 25s ago 7m 42.4M 4272M 19.1.1-361-g2f599a46 d2baa855c067 8c1269231e5d 2024-09-19T04:46:08.584 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1 smithi183 running (7m) 39s ago 7m 45.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 a35422581151 2024-09-19T04:46:08.584 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2 smithi193 running (6m) 39s ago 6m 47.3M 4272M 19.1.1-361-g2f599a46 d2baa855c067 36f82aa19108 2024-09-19T04:46:08.584 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3 smithi123 running (6m) 25s ago 6m 41.9M 4272M 19.1.1-361-g2f599a46 d2baa855c067 880479365d1c 2024-09-19T04:46:08.585 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4 smithi183 running (5m) 39s ago 5m 44.8M 4272M 19.1.1-361-g2f599a46 d2baa855c067 3f895ca06ac4 2024-09-19T04:46:08.585 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5 smithi193 running (5m) 39s ago 5m 55.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 c917ea986750 2024-09-19T04:46:08.585 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6 smithi123 running (4m) 25s ago 4m 50.4M 4272M 19.1.1-361-g2f599a46 d2baa855c067 4adbf2c8bbf8 2024-09-19T04:46:08.585 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7 smithi183 running (4m) 39s ago 4m 56.8M 4272M 19.1.1-361-g2f599a46 d2baa855c067 765461ee2d5a 2024-09-19T04:46:08.585 INFO:teuthology.orchestra.run.smithi123.stdout:osd.8 smithi193 running (4m) 39s ago 4m 43.1M 4272M 19.1.1-361-g2f599a46 d2baa855c067 4ec0e29819be 2024-09-19T04:46:08.585 INFO:teuthology.orchestra.run.smithi123.stdout:osd.9 smithi123 running (3m) 25s ago 3m 51.1M 4272M 19.1.1-361-g2f599a46 d2baa855c067 3c3100f10775 2024-09-19T04:46:08.585 INFO:teuthology.orchestra.run.smithi123.stdout:osd.10 smithi183 running (3m) 39s ago 3m 44.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 d0d5b581f1ff 2024-09-19T04:46:08.585 INFO:teuthology.orchestra.run.smithi123.stdout:osd.11 smithi193 running (2m) 39s ago 2m 53.9M 4272M 19.1.1-361-g2f599a46 d2baa855c067 ace95eee0b0b 2024-09-19T04:46:08.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.584+0000 7f3078800640 1 -- 172.21.15.123:0/3665187865 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f30580a3300 msgr2=0x7f30580a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:08.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.584+0000 7f3078800640 1 --2- 172.21.15.123:0/3665187865 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f30580a3300 0x7f30580a57c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3070102c60 tx=0x7f3064005f90 comp rx=0 tx=0).stop 2024-09-19T04:46:08.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.584+0000 7f3078800640 1 -- 172.21.15.123:0/3665187865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3070102e00 msgr2=0x7f307019ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:08.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.584+0000 7f3078800640 1 --2- 172.21.15.123:0/3665187865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3070102e00 0x7f307019ea60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f306000e9f0 tx=0x7f306000edf0 comp rx=0 tx=0).stop 2024-09-19T04:46:08.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.584+0000 7f3078800640 1 -- 172.21.15.123:0/3665187865 shutdown_connections 2024-09-19T04:46:08.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.584+0000 7f3078800640 1 --2- 172.21.15.123:0/3665187865 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f30580a3300 0x7f30580a57c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:08.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.584+0000 7f3078800640 1 --2- 172.21.15.123:0/3665187865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30701037c0 0x7f30701a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:08.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.584+0000 7f3078800640 1 --2- 172.21.15.123:0/3665187865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3070102e00 0x7f307019ea60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:08.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.584+0000 7f3078800640 1 --2- 172.21.15.123:0/3665187865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3070101c00 0x7f307019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:08.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.584+0000 7f3078800640 1 -- 172.21.15.123:0/3665187865 >> 172.21.15.123:0/3665187865 conn(0x7f30700fd390 msgr2=0x7f3070074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:08.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.584+0000 7f3078800640 1 -- 172.21.15.123:0/3665187865 shutdown_connections 2024-09-19T04:46:08.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:08.584+0000 7f3078800640 1 -- 172.21.15.123:0/3665187865 wait complete. 2024-09-19T04:46:08.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:08 smithi183 ceph-mon[35333]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-19T04:46:08.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:08 smithi123 ceph-mon[30511]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-19T04:46:08.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:08 smithi193 ceph-mon[35592]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-19T04:46:09.374 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T04:46:09.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:09 smithi123 ceph-mon[30511]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:09.743 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:09.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:09 smithi183 ceph-mon[35333]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:09.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:09 smithi193 ceph-mon[35592]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:10.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.307+0000 7fdb7ac00640 1 -- 172.21.15.123:0/2027180379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb74103c50 msgr2=0x7fdb74104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:10.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.307+0000 7fdb7ac00640 1 --2- 172.21.15.123:0/2027180379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb74103c50 0x7fdb74104050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009900 tx=0x7fdb6802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:46:10.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.307+0000 7fdb7ac00640 1 -- 172.21.15.123:0/2027180379 shutdown_connections 2024-09-19T04:46:10.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.307+0000 7fdb7ac00640 1 --2- 172.21.15.123:0/2027180379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb74105810 0x7fdb7410c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:10.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.307+0000 7fdb7ac00640 1 --2- 172.21.15.123:0/2027180379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb74104e50 0x7fdb741052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:10.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.307+0000 7fdb7ac00640 1 --2- 172.21.15.123:0/2027180379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb74103c50 0x7fdb74104050 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:10.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.307+0000 7fdb7ac00640 1 -- 172.21.15.123:0/2027180379 >> 172.21.15.123:0/2027180379 conn(0x7fdb740ff3e0 msgr2=0x7fdb74101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:10.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.307+0000 7fdb7ac00640 1 -- 172.21.15.123:0/2027180379 shutdown_connections 2024-09-19T04:46:10.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.307+0000 7fdb7ac00640 1 -- 172.21.15.123:0/2027180379 wait complete. 2024-09-19T04:46:10.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.308+0000 7fdb7ac00640 1 Processor -- start 2024-09-19T04:46:10.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.308+0000 7fdb7ac00640 1 -- start start 2024-09-19T04:46:10.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.308+0000 7fdb7ac00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb74103c50 0x7fdb74199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:10.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.308+0000 7fdb7ac00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb74104e50 0x7fdb7419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:10.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.308+0000 7fdb7ac00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb74105810 0x7fdb741a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:10.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.308+0000 7fdb7ac00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7410f900 con 0x7fdb74103c50 2024-09-19T04:46:10.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.308+0000 7fdb7ac00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7410f780 con 0x7fdb74105810 2024-09-19T04:46:10.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.308+0000 7fdb7ac00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7410fa80 con 0x7fdb74104e50 2024-09-19T04:46:10.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.308+0000 7fdb79800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb74103c50 0x7fdb74199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:10.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.309+0000 7fdb7a200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb74105810 0x7fdb741a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:10.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.309+0000 7fdb79800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb74103c50 0x7fdb74199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53408/0 (socket says 172.21.15.123:53408) 2024-09-19T04:46:10.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.309+0000 7fdb79800640 1 -- 172.21.15.123:0/1859058026 learned_addr learned my addr 172.21.15.123:0/1859058026 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:10.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.309+0000 7fdb78e00640 1 --2- 172.21.15.123:0/1859058026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb74104e50 0x7fdb7419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:10.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.309+0000 7fdb79800640 1 -- 172.21.15.123:0/1859058026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb74104e50 msgr2=0x7fdb7419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:10.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.309+0000 7fdb79800640 1 --2- 172.21.15.123:0/1859058026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb74104e50 0x7fdb7419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:10.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.309+0000 7fdb79800640 1 -- 172.21.15.123:0/1859058026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb74105810 msgr2=0x7fdb741a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:10.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.309+0000 7fdb79800640 1 --2- 172.21.15.123:0/1859058026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb74105810 0x7fdb741a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:10.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.309+0000 7fdb79800640 1 -- 172.21.15.123:0/1859058026 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb741a1b20 con 0x7fdb74103c50 2024-09-19T04:46:10.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.309+0000 7fdb79800640 1 --2- 172.21.15.123:0/1859058026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb74103c50 0x7fdb74199ef0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdb68031440 tx=0x7fdb68032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:10.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.309+0000 7fdb72000640 1 -- 172.21.15.123:0/1859058026 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6803f030 con 0x7fdb74103c50 2024-09-19T04:46:10.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.309+0000 7fdb72000640 1 -- 172.21.15.123:0/1859058026 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb68003ce0 con 0x7fdb74103c50 2024-09-19T04:46:10.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.310+0000 7fdb7ac00640 1 -- 172.21.15.123:0/1859058026 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb741a1db0 con 0x7fdb74103c50 2024-09-19T04:46:10.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.310+0000 7fdb72000640 1 -- 172.21.15.123:0/1859058026 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68035070 con 0x7fdb74103c50 2024-09-19T04:46:10.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.310+0000 7fdb7ac00640 1 -- 172.21.15.123:0/1859058026 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb741a21d0 con 0x7fdb74103c50 2024-09-19T04:46:10.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.311+0000 7fdb7ac00640 1 -- 172.21.15.123:0/1859058026 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb7410afa0 con 0x7fdb74103c50 2024-09-19T04:46:10.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.313+0000 7fdb72000640 1 -- 172.21.15.123:0/1859058026 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fdb68042030 con 0x7fdb74103c50 2024-09-19T04:46:10.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.315+0000 7fdb72000640 1 --2- 172.21.15.123:0/1859058026 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fdb5c0a3300 0x7fdb5c0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:10.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.315+0000 7fdb72000640 1 -- 172.21.15.123:0/1859058026 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fdb680eff10 con 0x7fdb74103c50 2024-09-19T04:46:10.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.315+0000 7fdb78e00640 1 --2- 172.21.15.123:0/1859058026 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fdb5c0a3300 0x7fdb5c0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:10.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.315+0000 7fdb78e00640 1 --2- 172.21.15.123:0/1859058026 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fdb5c0a3300 0x7fdb5c0a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdb7419b410 tx=0x7fdb60005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:10.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.319+0000 7fdb72000640 1 -- 172.21.15.123:0/1859058026 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdb680bc430 con 0x7fdb74103c50 2024-09-19T04:46:10.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.501+0000 7fdb7ac00640 1 -- 172.21.15.123:0/1859058026 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fdb74062630 con 0x7fdb5c0a3300 2024-09-19T04:46:10.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.507+0000 7fdb72000640 1 -- 172.21.15.123:0/1859058026 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fdb74062630 con 0x7fdb5c0a3300 2024-09-19T04:46:10.508 INFO:teuthology.orchestra.run.smithi123.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T04:46:10.509 INFO:teuthology.orchestra.run.smithi123.stdout:mds.all 12/12 41s ago 2m smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-09-19T04:46:10.509 INFO:teuthology.orchestra.run.smithi123.stdout:mgr 3/3 41s ago 8m smithi123=x;smithi183=y;smithi193=z;count:3 2024-09-19T04:46:10.509 INFO:teuthology.orchestra.run.smithi123.stdout:mon 3/3 41s ago 8m smithi123:172.21.15.123=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c;count:3 2024-09-19T04:46:10.509 INFO:teuthology.orchestra.run.smithi123.stdout:osd 12 41s ago - 2024-09-19T04:46:10.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.509+0000 7fdb7ac00640 1 -- 172.21.15.123:0/1859058026 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fdb5c0a3300 msgr2=0x7fdb5c0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:10.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.509+0000 7fdb7ac00640 1 --2- 172.21.15.123:0/1859058026 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fdb5c0a3300 0x7fdb5c0a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdb7419b410 tx=0x7fdb60005f90 comp rx=0 tx=0).stop 2024-09-19T04:46:10.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.509+0000 7fdb7ac00640 1 -- 172.21.15.123:0/1859058026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb74103c50 msgr2=0x7fdb74199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:10.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.509+0000 7fdb7ac00640 1 --2- 172.21.15.123:0/1859058026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb74103c50 0x7fdb74199ef0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdb68031440 tx=0x7fdb68032ce0 comp rx=0 tx=0).stop 2024-09-19T04:46:10.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.509+0000 7fdb7ac00640 1 -- 172.21.15.123:0/1859058026 shutdown_connections 2024-09-19T04:46:10.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.510+0000 7fdb7ac00640 1 --2- 172.21.15.123:0/1859058026 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fdb5c0a3300 0x7fdb5c0a57c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:10.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.510+0000 7fdb7ac00640 1 --2- 172.21.15.123:0/1859058026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb74105810 0x7fdb741a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:10.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.510+0000 7fdb7ac00640 1 --2- 172.21.15.123:0/1859058026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb74104e50 0x7fdb7419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:10.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.510+0000 7fdb7ac00640 1 --2- 172.21.15.123:0/1859058026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb74103c50 0x7fdb74199ef0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:10.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.510+0000 7fdb7ac00640 1 -- 172.21.15.123:0/1859058026 >> 172.21.15.123:0/1859058026 conn(0x7fdb740ff3e0 msgr2=0x7fdb74108170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:10.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.510+0000 7fdb7ac00640 1 -- 172.21.15.123:0/1859058026 shutdown_connections 2024-09-19T04:46:10.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:10.510+0000 7fdb7ac00640 1 -- 172.21.15.123:0/1859058026 wait complete. 2024-09-19T04:46:10.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:10 smithi183 ceph-mon[35333]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-19T04:46:10.859 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:10 smithi123 ceph-mon[30511]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-19T04:46:10.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:10 smithi193 ceph-mon[35592]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-19T04:46:11.250 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T04:46:11.620 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:11.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:11 smithi183 ceph-mon[35333]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:11.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:11 smithi123 ceph-mon[30511]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:11.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:11 smithi193 ceph-mon[35592]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:12.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.219+0000 7f84b9e00640 1 -- 172.21.15.123:0/392672981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b4103e00 msgr2=0x7f84b4104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:12.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.219+0000 7f84b9e00640 1 --2- 172.21.15.123:0/392672981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b4103e00 0x7f84b4104200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f849c009900 tx=0x7f849c02a410 comp rx=0 tx=0).stop 2024-09-19T04:46:12.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.219+0000 7f84b9e00640 1 -- 172.21.15.123:0/392672981 shutdown_connections 2024-09-19T04:46:12.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.219+0000 7f84b9e00640 1 --2- 172.21.15.123:0/392672981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84b4105920 0x7f84b410c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:12.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.219+0000 7f84b9e00640 1 --2- 172.21.15.123:0/392672981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84b4104f60 0x7f84b41053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:12.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.219+0000 7f84b9e00640 1 --2- 172.21.15.123:0/392672981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b4103e00 0x7f84b4104200 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:12.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.219+0000 7f84b9e00640 1 -- 172.21.15.123:0/392672981 >> 172.21.15.123:0/392672981 conn(0x7f84b40ff5b0 msgr2=0x7f84b41019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:12.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.219+0000 7f84b9e00640 1 -- 172.21.15.123:0/392672981 shutdown_connections 2024-09-19T04:46:12.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.219+0000 7f84b9e00640 1 -- 172.21.15.123:0/392672981 wait complete. 2024-09-19T04:46:12.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.220+0000 7f84b9e00640 1 Processor -- start 2024-09-19T04:46:12.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.220+0000 7f84b9e00640 1 -- start start 2024-09-19T04:46:12.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.220+0000 7f84b9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b4103e00 0x7f84b419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:12.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.220+0000 7f84b9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84b4104f60 0x7f84b419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:12.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.220+0000 7f84b9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84b4105920 0x7f84b41a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:12.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.220+0000 7f84b9e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f84b410fa90 con 0x7f84b4103e00 2024-09-19T04:46:12.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.220+0000 7f84b9e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f84b410f910 con 0x7f84b4104f60 2024-09-19T04:46:12.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.220+0000 7f84b9e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f84b410fc10 con 0x7f84b4105920 2024-09-19T04:46:12.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.220+0000 7f84b2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b4103e00 0x7f84b419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:12.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.221+0000 7f84b2000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84b4104f60 0x7f84b419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:12.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.221+0000 7f84b2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b4103e00 0x7f84b419a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45956/0 (socket says 172.21.15.123:45956) 2024-09-19T04:46:12.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.221+0000 7f84b2a00640 1 -- 172.21.15.123:0/3781727256 learned_addr learned my addr 172.21.15.123:0/3781727256 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:12.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.221+0000 7f84b3400640 1 --2- 172.21.15.123:0/3781727256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84b4105920 0x7f84b41a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:12.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.221+0000 7f84b2000640 1 -- 172.21.15.123:0/3781727256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84b4105920 msgr2=0x7f84b41a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:12.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.221+0000 7f84b2000640 1 --2- 172.21.15.123:0/3781727256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84b4105920 0x7f84b41a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:12.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.221+0000 7f84b2000640 1 -- 172.21.15.123:0/3781727256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b4103e00 msgr2=0x7f84b419a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:12.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.221+0000 7f84b2000640 1 --2- 172.21.15.123:0/3781727256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b4103e00 0x7f84b419a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:12.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.221+0000 7f84b2000640 1 -- 172.21.15.123:0/3781727256 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b41a1d10 con 0x7f84b4104f60 2024-09-19T04:46:12.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.221+0000 7f84b3400640 1 --2- 172.21.15.123:0/3781727256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84b4105920 0x7f84b41a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:46:12.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.221+0000 7f84b2a00640 1 --2- 172.21.15.123:0/3781727256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b4103e00 0x7f84b419a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:12.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.221+0000 7f84b2000640 1 --2- 172.21.15.123:0/3781727256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84b4104f60 0x7f84b419a620 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f84a400e9c0 tx=0x7f84a400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:12.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.222+0000 7f84b8a00640 1 -- 172.21.15.123:0/3781727256 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a400ccc0 con 0x7f84b4104f60 2024-09-19T04:46:12.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.222+0000 7f84b8a00640 1 -- 172.21.15.123:0/3781727256 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a40035f0 con 0x7f84b4104f60 2024-09-19T04:46:12.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.222+0000 7f84b9e00640 1 -- 172.21.15.123:0/3781727256 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b41a1fa0 con 0x7f84b4104f60 2024-09-19T04:46:12.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.222+0000 7f84b8a00640 1 -- 172.21.15.123:0/3781727256 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a400ce60 con 0x7f84b4104f60 2024-09-19T04:46:12.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.222+0000 7f84b9e00640 1 -- 172.21.15.123:0/3781727256 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b41a2500 con 0x7f84b4104f60 2024-09-19T04:46:12.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.223+0000 7f84b9e00640 1 -- 172.21.15.123:0/3781727256 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b410b190 con 0x7f84b4104f60 2024-09-19T04:46:12.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.225+0000 7f84b8a00640 1 -- 172.21.15.123:0/3781727256 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f84a40037a0 con 0x7f84b4104f60 2024-09-19T04:46:12.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.226+0000 7f84b8a00640 1 --2- 172.21.15.123:0/3781727256 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f84980a3380 0x7f84980a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:12.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.226+0000 7f84b8a00640 1 -- 172.21.15.123:0/3781727256 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f84a40d0150 con 0x7f84b4104f60 2024-09-19T04:46:12.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.226+0000 7f84b2a00640 1 --2- 172.21.15.123:0/3781727256 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f84980a3380 0x7f84980a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:12.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.227+0000 7f84b2a00640 1 --2- 172.21.15.123:0/3781727256 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f84980a3380 0x7f84980a5840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f849c031440 tx=0x7f849c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:12.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.228+0000 7f84b8a00640 1 -- 172.21.15.123:0/3781727256 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f84a409ccc0 con 0x7f84b4104f60 2024-09-19T04:46:12.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.410+0000 7f84b9e00640 1 -- 172.21.15.123:0/3781727256 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f84b4062630 con 0x7f84980a3380 2024-09-19T04:46:12.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.412+0000 7f84b8a00640 1 -- 172.21.15.123:0/3781727256 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f84b4062630 con 0x7f84980a3380 2024-09-19T04:46:12.413 INFO:teuthology.orchestra.run.smithi123.stdout:HOST ADDR LABELS STATUS 2024-09-19T04:46:12.413 INFO:teuthology.orchestra.run.smithi123.stdout:smithi123 172.21.15.123 2024-09-19T04:46:12.414 INFO:teuthology.orchestra.run.smithi123.stdout:smithi183 172.21.15.183 2024-09-19T04:46:12.414 INFO:teuthology.orchestra.run.smithi123.stdout:smithi193 172.21.15.193 2024-09-19T04:46:12.414 INFO:teuthology.orchestra.run.smithi123.stdout:3 hosts in cluster 2024-09-19T04:46:12.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.414+0000 7f84b9e00640 1 -- 172.21.15.123:0/3781727256 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f84980a3380 msgr2=0x7f84980a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:12.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.414+0000 7f84b9e00640 1 --2- 172.21.15.123:0/3781727256 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f84980a3380 0x7f84980a5840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f849c031440 tx=0x7f849c0315b0 comp rx=0 tx=0).stop 2024-09-19T04:46:12.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.414+0000 7f84b9e00640 1 -- 172.21.15.123:0/3781727256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84b4104f60 msgr2=0x7f84b419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:12.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.414+0000 7f84b9e00640 1 --2- 172.21.15.123:0/3781727256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84b4104f60 0x7f84b419a620 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f84a400e9c0 tx=0x7f84a400ee80 comp rx=0 tx=0).stop 2024-09-19T04:46:12.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.414+0000 7f84b9e00640 1 -- 172.21.15.123:0/3781727256 shutdown_connections 2024-09-19T04:46:12.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.414+0000 7f84b9e00640 1 --2- 172.21.15.123:0/3781727256 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f84980a3380 0x7f84980a5840 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:12.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.414+0000 7f84b9e00640 1 --2- 172.21.15.123:0/3781727256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84b4105920 0x7f84b41a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:12.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.415+0000 7f84b9e00640 1 --2- 172.21.15.123:0/3781727256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84b4104f60 0x7f84b419a620 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:12.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.415+0000 7f84b9e00640 1 --2- 172.21.15.123:0/3781727256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b4103e00 0x7f84b419a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:12.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.415+0000 7f84b9e00640 1 -- 172.21.15.123:0/3781727256 >> 172.21.15.123:0/3781727256 conn(0x7f84b40ff5b0 msgr2=0x7f84b4108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:12.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.415+0000 7f84b9e00640 1 -- 172.21.15.123:0/3781727256 shutdown_connections 2024-09-19T04:46:12.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:12.415+0000 7f84b9e00640 1 -- 172.21.15.123:0/3781727256 wait complete. 2024-09-19T04:46:12.835 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:12 smithi123 ceph-mon[30511]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:46:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:12 smithi183 ceph-mon[35333]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:46:12.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:12 smithi193 ceph-mon[35592]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:46:13.334 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T04:46:13.726 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:13.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:13 smithi183 ceph-mon[35333]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:13.864 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:13 smithi123 ceph-mon[30511]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:13.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:13 smithi193 ceph-mon[35592]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:14.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.348+0000 7efdbe400640 1 -- 172.21.15.123:0/691950851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdb8074c50 msgr2=0x7efdb80750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:14.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.348+0000 7efdbe400640 1 --2- 172.21.15.123:0/691950851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdb8074c50 0x7efdb80750b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efda4009900 tx=0x7efda402a410 comp rx=0 tx=0).stop 2024-09-19T04:46:14.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.348+0000 7efdbe400640 1 -- 172.21.15.123:0/691950851 shutdown_connections 2024-09-19T04:46:14.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.348+0000 7efdbe400640 1 --2- 172.21.15.123:0/691950851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdb8109a80 0x7efdb810bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:14.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.348+0000 7efdbe400640 1 --2- 172.21.15.123:0/691950851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdb8074c50 0x7efdb80750b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:14.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.348+0000 7efdbe400640 1 --2- 172.21.15.123:0/691950851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb8076220 0x7efdb8074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:14.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.348+0000 7efdbe400640 1 -- 172.21.15.123:0/691950851 >> 172.21.15.123:0/691950851 conn(0x7efdb80fd390 msgr2=0x7efdb80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:14.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.348+0000 7efdbe400640 1 -- 172.21.15.123:0/691950851 shutdown_connections 2024-09-19T04:46:14.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.348+0000 7efdbe400640 1 -- 172.21.15.123:0/691950851 wait complete. 2024-09-19T04:46:14.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.349+0000 7efdbe400640 1 Processor -- start 2024-09-19T04:46:14.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.349+0000 7efdbe400640 1 -- start start 2024-09-19T04:46:14.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.349+0000 7efdbe400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdb8074c50 0x7efdb819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:14.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.349+0000 7efdbe400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb8076220 0x7efdb819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:14.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.349+0000 7efdbe400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdb8109a80 0x7efdb81a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:14.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.349+0000 7efdbe400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efdb810fad0 con 0x7efdb8109a80 2024-09-19T04:46:14.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.349+0000 7efdbe400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efdb810f950 con 0x7efdb8074c50 2024-09-19T04:46:14.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.349+0000 7efdbe400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efdb810fc50 con 0x7efdb8076220 2024-09-19T04:46:14.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.350+0000 7efdb7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdb8074c50 0x7efdb819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:14.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.350+0000 7efdb7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdb8074c50 0x7efdb819a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:56100/0 (socket says 172.21.15.123:56100) 2024-09-19T04:46:14.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.350+0000 7efdb7400640 1 -- 172.21.15.123:0/3261840466 learned_addr learned my addr 172.21.15.123:0/3261840466 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:14.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.350+0000 7efdb7e00640 1 --2- 172.21.15.123:0/3261840466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdb8109a80 0x7efdb81a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:14.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.350+0000 7efdb6a00640 1 --2- 172.21.15.123:0/3261840466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb8076220 0x7efdb819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:14.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.350+0000 7efdb7400640 1 -- 172.21.15.123:0/3261840466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb8076220 msgr2=0x7efdb819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:14.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.350+0000 7efdb7400640 1 --2- 172.21.15.123:0/3261840466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb8076220 0x7efdb819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:14.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.350+0000 7efdb7400640 1 -- 172.21.15.123:0/3261840466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdb8109a80 msgr2=0x7efdb81a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:14.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.350+0000 7efdb7400640 1 --2- 172.21.15.123:0/3261840466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdb8109a80 0x7efdb81a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:14.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.350+0000 7efdb7400640 1 -- 172.21.15.123:0/3261840466 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb81a1d40 con 0x7efdb8074c50 2024-09-19T04:46:14.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.350+0000 7efdb7400640 1 --2- 172.21.15.123:0/3261840466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdb8074c50 0x7efdb819a110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efda800e9c0 tx=0x7efda800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:14.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.351+0000 7efdafe00640 1 -- 172.21.15.123:0/3261840466 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda800cd80 con 0x7efdb8074c50 2024-09-19T04:46:14.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.351+0000 7efdafe00640 1 -- 172.21.15.123:0/3261840466 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda8005500 con 0x7efdb8074c50 2024-09-19T04:46:14.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.351+0000 7efdbe400640 1 -- 172.21.15.123:0/3261840466 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb81a1fd0 con 0x7efdb8074c50 2024-09-19T04:46:14.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.351+0000 7efdafe00640 1 -- 172.21.15.123:0/3261840466 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8010690 con 0x7efdb8074c50 2024-09-19T04:46:14.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.351+0000 7efdbe400640 1 -- 172.21.15.123:0/3261840466 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb81a24d0 con 0x7efdb8074c50 2024-09-19T04:46:14.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.352+0000 7efdbe400640 1 -- 172.21.15.123:0/3261840466 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb8108800 con 0x7efdb8074c50 2024-09-19T04:46:14.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.354+0000 7efdafe00640 1 -- 172.21.15.123:0/3261840466 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7efda80046c0 con 0x7efdb8074c50 2024-09-19T04:46:14.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.355+0000 7efdafe00640 1 --2- 172.21.15.123:0/3261840466 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7efd9c0a32f0 0x7efd9c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:14.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.355+0000 7efdafe00640 1 -- 172.21.15.123:0/3261840466 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7efda80cfc10 con 0x7efdb8074c50 2024-09-19T04:46:14.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.356+0000 7efdb6a00640 1 --2- 172.21.15.123:0/3261840466 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7efd9c0a32f0 0x7efd9c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:14.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.356+0000 7efdb6a00640 1 --2- 172.21.15.123:0/3261840466 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7efd9c0a32f0 0x7efd9c0a57b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efda4031440 tx=0x7efda40315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:14.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.359+0000 7efdafe00640 1 -- 172.21.15.123:0/3261840466 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efda809c8e0 con 0x7efdb8074c50 2024-09-19T04:46:14.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.545+0000 7efdbe400640 1 -- 172.21.15.123:0/3261840466 --> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7efdb8062630 con 0x7efd9c0a32f0 2024-09-19T04:46:14.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.547+0000 7efdafe00640 1 -- 172.21.15.123:0/3261840466 <== mgr.14150 v2:172.21.15.123:6800/1532622003 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7efdb8062630 con 0x7efd9c0a32f0 2024-09-19T04:46:14.549 INFO:teuthology.orchestra.run.smithi123.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T04:46:14.549 INFO:teuthology.orchestra.run.smithi123.stdout:smithi123 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400YK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-19T04:46:14.549 INFO:teuthology.orchestra.run.smithi123.stdout:smithi183 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-19T04:46:14.549 INFO:teuthology.orchestra.run.smithi123.stdout:smithi193 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500HB400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T04:46:14.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.549+0000 7efdbe400640 1 -- 172.21.15.123:0/3261840466 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7efd9c0a32f0 msgr2=0x7efd9c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.549+0000 7efdbe400640 1 --2- 172.21.15.123:0/3261840466 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7efd9c0a32f0 0x7efd9c0a57b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efda4031440 tx=0x7efda40315b0 comp rx=0 tx=0).stop 2024-09-19T04:46:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.549+0000 7efdbe400640 1 -- 172.21.15.123:0/3261840466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdb8074c50 msgr2=0x7efdb819a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.549+0000 7efdbe400640 1 --2- 172.21.15.123:0/3261840466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdb8074c50 0x7efdb819a110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efda800e9c0 tx=0x7efda800ee80 comp rx=0 tx=0).stop 2024-09-19T04:46:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.549+0000 7efdbe400640 1 -- 172.21.15.123:0/3261840466 shutdown_connections 2024-09-19T04:46:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.549+0000 7efdbe400640 1 --2- 172.21.15.123:0/3261840466 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7efd9c0a32f0 0x7efd9c0a57b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.549+0000 7efdbe400640 1 --2- 172.21.15.123:0/3261840466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdb8109a80 0x7efdb81a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.549+0000 7efdbe400640 1 --2- 172.21.15.123:0/3261840466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb8076220 0x7efdb819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.549+0000 7efdbe400640 1 --2- 172.21.15.123:0/3261840466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdb8074c50 0x7efdb819a110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.549+0000 7efdbe400640 1 -- 172.21.15.123:0/3261840466 >> 172.21.15.123:0/3261840466 conn(0x7efdb80fd390 msgr2=0x7efdb81059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.549+0000 7efdbe400640 1 -- 172.21.15.123:0/3261840466 shutdown_connections 2024-09-19T04:46:14.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:14.549+0000 7efdbe400640 1 -- 172.21.15.123:0/3261840466 wait complete. 2024-09-19T04:46:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:14 smithi183 ceph-mon[35333]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:46:14.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:14 smithi183 ceph-mon[35333]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-19T04:46:14.911 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:14 smithi123 ceph-mon[30511]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:46:14.911 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:14 smithi123 ceph-mon[30511]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-19T04:46:14.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:14 smithi193 ceph-mon[35592]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:46:14.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:14 smithi193 ceph-mon[35592]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-19T04:46:15.395 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T04:46:15.406 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T04:46:15.406 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-19T04:46:15.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:15 smithi123 ceph-mon[30511]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:15.775 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:15.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:15 smithi183 ceph-mon[35333]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:15.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:15 smithi193 ceph-mon[35592]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:16.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.343+0000 7f7b64800640 1 -- 172.21.15.123:0/2300142395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b5c102e00 msgr2=0x7f7b5c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:16.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.343+0000 7f7b64800640 1 --2- 172.21.15.123:0/2300142395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b5c102e00 0x7f7b5c103280 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009860 tx=0x7f7b4c02a340 comp rx=0 tx=0).stop 2024-09-19T04:46:16.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.343+0000 7f7b64800640 1 -- 172.21.15.123:0/2300142395 shutdown_connections 2024-09-19T04:46:16.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.343+0000 7f7b64800640 1 --2- 172.21.15.123:0/2300142395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b5c1037c0 0x7f7b5c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:16.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.343+0000 7f7b64800640 1 --2- 172.21.15.123:0/2300142395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b5c102e00 0x7f7b5c103280 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:16.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.343+0000 7f7b64800640 1 --2- 172.21.15.123:0/2300142395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b5c101c00 0x7f7b5c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:16.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.343+0000 7f7b64800640 1 -- 172.21.15.123:0/2300142395 >> 172.21.15.123:0/2300142395 conn(0x7f7b5c0fd390 msgr2=0x7f7b5c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:16.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.343+0000 7f7b64800640 1 -- 172.21.15.123:0/2300142395 shutdown_connections 2024-09-19T04:46:16.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.343+0000 7f7b64800640 1 -- 172.21.15.123:0/2300142395 wait complete. 2024-09-19T04:46:16.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.344+0000 7f7b64800640 1 Processor -- start 2024-09-19T04:46:16.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.344+0000 7f7b64800640 1 -- start start 2024-09-19T04:46:16.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.344+0000 7f7b64800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b5c101c00 0x7f7b5c19e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:16.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.344+0000 7f7b64800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b5c102e00 0x7f7b5c19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:16.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.344+0000 7f7b64800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b5c1037c0 0x7f7b5c1a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:16.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.344+0000 7f7b64800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c10d880 con 0x7f7b5c1037c0 2024-09-19T04:46:16.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.344+0000 7f7b64800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c10d700 con 0x7f7b5c102e00 2024-09-19T04:46:16.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.344+0000 7f7b64800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c10da00 con 0x7f7b5c101c00 2024-09-19T04:46:16.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.345+0000 7f7b61e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b5c101c00 0x7f7b5c19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:16.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.345+0000 7f7b61e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b5c101c00 0x7f7b5c19e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53582/0 (socket says 172.21.15.123:53582) 2024-09-19T04:46:16.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.345+0000 7f7b61e00640 1 -- 172.21.15.123:0/2532297963 learned_addr learned my addr 172.21.15.123:0/2532297963 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:16.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.345+0000 7f7b61400640 1 --2- 172.21.15.123:0/2532297963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b5c102e00 0x7f7b5c19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:16.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.345+0000 7f7b62800640 1 --2- 172.21.15.123:0/2532297963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b5c1037c0 0x7f7b5c1a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:16.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.345+0000 7f7b61e00640 1 -- 172.21.15.123:0/2532297963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b5c102e00 msgr2=0x7f7b5c19eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:16.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.345+0000 7f7b61e00640 1 --2- 172.21.15.123:0/2532297963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b5c102e00 0x7f7b5c19eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:16.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.345+0000 7f7b61e00640 1 -- 172.21.15.123:0/2532297963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b5c1037c0 msgr2=0x7f7b5c1a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:16.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.345+0000 7f7b61e00640 1 --2- 172.21.15.123:0/2532297963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b5c1037c0 0x7f7b5c1a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:16.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.345+0000 7f7b61e00640 1 -- 172.21.15.123:0/2532297963 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5c1a6190 con 0x7f7b5c101c00 2024-09-19T04:46:16.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.345+0000 7f7b62800640 1 --2- 172.21.15.123:0/2532297963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b5c1037c0 0x7f7b5c1a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:46:16.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.345+0000 7f7b61e00640 1 --2- 172.21.15.123:0/2532297963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b5c101c00 0x7f7b5c19e560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000e9a0 tx=0x7f7b5000ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:16.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.346+0000 7f7b56a00640 1 -- 172.21.15.123:0/2532297963 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5000cd80 con 0x7f7b5c101c00 2024-09-19T04:46:16.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.346+0000 7f7b64800640 1 -- 172.21.15.123:0/2532297963 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5c1a6480 con 0x7f7b5c101c00 2024-09-19T04:46:16.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.346+0000 7f7b56a00640 1 -- 172.21.15.123:0/2532297963 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b50010910 con 0x7f7b5c101c00 2024-09-19T04:46:16.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.346+0000 7f7b56a00640 1 -- 172.21.15.123:0/2532297963 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50018a40 con 0x7f7b5c101c00 2024-09-19T04:46:16.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.346+0000 7f7b64800640 1 -- 172.21.15.123:0/2532297963 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b5c1a69e0 con 0x7f7b5c101c00 2024-09-19T04:46:16.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.347+0000 7f7b64800640 1 -- 172.21.15.123:0/2532297963 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b5c1a6d00 con 0x7f7b5c101c00 2024-09-19T04:46:16.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.349+0000 7f7b56a00640 1 -- 172.21.15.123:0/2532297963 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f7b50010ac0 con 0x7f7b5c101c00 2024-09-19T04:46:16.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.350+0000 7f7b56a00640 1 --2- 172.21.15.123:0/2532297963 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7b440a32f0 0x7f7b440a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:16.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.350+0000 7f7b56a00640 1 -- 172.21.15.123:0/2532297963 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f7b500d01f0 con 0x7f7b5c101c00 2024-09-19T04:46:16.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.350+0000 7f7b61400640 1 --2- 172.21.15.123:0/2532297963 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7b440a32f0 0x7f7b440a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:16.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.351+0000 7f7b61400640 1 --2- 172.21.15.123:0/2532297963 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7b440a32f0 0x7f7b440a57b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c02a930 tx=0x7f7b4c030570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:16.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.353+0000 7f7b56a00640 1 -- 172.21.15.123:0/2532297963 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7b50026020 con 0x7f7b5c101c00 2024-09-19T04:46:16.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.560+0000 7f7b64800640 1 -- 172.21.15.123:0/2532297963 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f7b5c1a6ed0 con 0x7f7b5c101c00 2024-09-19T04:46:16.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.561+0000 7f7b56a00640 1 -- 172.21.15.123:0/2532297963 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3139 (secure 0 0 0) 0x7f7b5009cec0 con 0x7f7b5c101c00 2024-09-19T04:46:16.563 INFO:teuthology.orchestra.run.smithi123.stdout:e27 2024-09-19T04:46:16.563 INFO:teuthology.orchestra.run.smithi123.stdout:btime 2024-09-19T04:44:44:834422+0000 2024-09-19T04:46:16.563 INFO:teuthology.orchestra.run.smithi123.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T04:46:16.563 INFO:teuthology.orchestra.run.smithi123.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:46:16.563 INFO:teuthology.orchestra.run.smithi123.stdout:legacy client fscid: 1 2024-09-19T04:46:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:46:16.563 INFO:teuthology.orchestra.run.smithi123.stdout:Filesystem 'cephfs' (1) 2024-09-19T04:46:16.563 INFO:teuthology.orchestra.run.smithi123.stdout:fs_name cephfs 2024-09-19T04:46:16.563 INFO:teuthology.orchestra.run.smithi123.stdout:epoch 27 2024-09-19T04:46:16.563 INFO:teuthology.orchestra.run.smithi123.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-19T04:46:16.563 INFO:teuthology.orchestra.run.smithi123.stdout:created 2024-09-19T04:44:31.766928+0000 2024-09-19T04:46:16.563 INFO:teuthology.orchestra.run.smithi123.stdout:modified 2024-09-19T04:44:44.834407+0000 2024-09-19T04:46:16.564 INFO:teuthology.orchestra.run.smithi123.stdout:tableserver 0 2024-09-19T04:46:16.564 INFO:teuthology.orchestra.run.smithi123.stdout:root 0 2024-09-19T04:46:16.564 INFO:teuthology.orchestra.run.smithi123.stdout:session_timeout 300 2024-09-19T04:46:16.564 INFO:teuthology.orchestra.run.smithi123.stdout:session_autoclose 300 2024-09-19T04:46:16.564 INFO:teuthology.orchestra.run.smithi123.stdout:max_file_size 1099511627776 2024-09-19T04:46:16.564 INFO:teuthology.orchestra.run.smithi123.stdout:max_xattr_size 65536 2024-09-19T04:46:16.564 INFO:teuthology.orchestra.run.smithi123.stdout:required_client_features {} 2024-09-19T04:46:16.564 INFO:teuthology.orchestra.run.smithi123.stdout:last_failure 0 2024-09-19T04:46:16.564 INFO:teuthology.orchestra.run.smithi123.stdout:last_failure_osd_epoch 0 2024-09-19T04:46:16.564 INFO:teuthology.orchestra.run.smithi123.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:46:16.564 INFO:teuthology.orchestra.run.smithi123.stdout:max_mds 5 2024-09-19T04:46:16.564 INFO:teuthology.orchestra.run.smithi123.stdout:in 0,1,2,3,4 2024-09-19T04:46:16.564 INFO:teuthology.orchestra.run.smithi123.stdout:up {0=24539,1=24529,2=24505,3=24500,4=24499} 2024-09-19T04:46:16.565 INFO:teuthology.orchestra.run.smithi123.stdout:failed 2024-09-19T04:46:16.565 INFO:teuthology.orchestra.run.smithi123.stdout:damaged 2024-09-19T04:46:16.565 INFO:teuthology.orchestra.run.smithi123.stdout:stopped 2024-09-19T04:46:16.565 INFO:teuthology.orchestra.run.smithi123.stdout:data_pools [3] 2024-09-19T04:46:16.565 INFO:teuthology.orchestra.run.smithi123.stdout:metadata_pool 2 2024-09-19T04:46:16.565 INFO:teuthology.orchestra.run.smithi123.stdout:inline_data disabled 2024-09-19T04:46:16.565 INFO:teuthology.orchestra.run.smithi123.stdout:balancer 2024-09-19T04:46:16.565 INFO:teuthology.orchestra.run.smithi123.stdout:bal_rank_mask -1 2024-09-19T04:46:16.565 INFO:teuthology.orchestra.run.smithi123.stdout:standby_count_wanted 0 2024-09-19T04:46:16.565 INFO:teuthology.orchestra.run.smithi123.stdout:qdb_cluster leader: 24539 members: 24529,24505,24539,24500,24499 2024-09-19T04:46:16.565 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b{0:24539} state up:active seq 5 addr [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:46:16.565 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.e{0:24479} state up:standby-replay seq 1 addr [v2:172.21.15.183:6834/309335638,v1:172.21.15.183:6835/309335638] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:46:16.568 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.i{1:24529} state up:active seq 9 addr [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:46:16.568 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d{1:24478} state up:standby-replay seq 1 addr [v2:172.21.15.123:6836/3620540498,v1:172.21.15.123:6837/3620540498] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:46:16.568 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c{2:24505} state up:active seq 11 addr [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:46:16.568 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.l{2:24460} state up:standby-replay seq 1 addr [v2:172.21.15.193:6832/964700250,v1:172.21.15.193:6833/964700250] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:46:16.568 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.h{3:24500} state up:active seq 11 addr [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:46:16.568 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.k{3:24443} state up:standby-replay seq 1 addr [v2:172.21.15.183:6832/2688234655,v1:172.21.15.183:6833/2688234655] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:46:16.569 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.f{4:24499} state up:active seq 12 addr [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:46:16.569 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a{4:14514} state up:standby-replay seq 1 addr [v2:172.21.15.123:6840/784762674,v1:172.21.15.123:6841/784762674] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:46:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:46:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:46:16.569 INFO:teuthology.orchestra.run.smithi123.stdout:Standby daemons: 2024-09-19T04:46:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:46:16.569 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.j{-1:14454} state up:standby seq 1 addr [v2:172.21.15.123:6834/1654466507,v1:172.21.15.123:6835/1654466507] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:46:16.569 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.g{-1:14508} state up:standby seq 1 addr [v2:172.21.15.123:6838/4146531731,v1:172.21.15.123:6839/4146531731] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:46:16.570 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 27 2024-09-19T04:46:16.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.563+0000 7f7b64800640 1 -- 172.21.15.123:0/2532297963 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7b440a32f0 msgr2=0x7f7b440a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:16.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.563+0000 7f7b64800640 1 --2- 172.21.15.123:0/2532297963 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7b440a32f0 0x7f7b440a57b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c02a930 tx=0x7f7b4c030570 comp rx=0 tx=0).stop 2024-09-19T04:46:16.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.563+0000 7f7b64800640 1 -- 172.21.15.123:0/2532297963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b5c101c00 msgr2=0x7f7b5c19e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:16.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.563+0000 7f7b64800640 1 --2- 172.21.15.123:0/2532297963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b5c101c00 0x7f7b5c19e560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000e9a0 tx=0x7f7b5000ee60 comp rx=0 tx=0).stop 2024-09-19T04:46:16.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.563+0000 7f7b64800640 1 -- 172.21.15.123:0/2532297963 shutdown_connections 2024-09-19T04:46:16.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.563+0000 7f7b64800640 1 --2- 172.21.15.123:0/2532297963 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7b440a32f0 0x7f7b440a57b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:16.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.563+0000 7f7b64800640 1 --2- 172.21.15.123:0/2532297963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b5c1037c0 0x7f7b5c1a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:16.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.563+0000 7f7b64800640 1 --2- 172.21.15.123:0/2532297963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b5c102e00 0x7f7b5c19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:16.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.563+0000 7f7b64800640 1 --2- 172.21.15.123:0/2532297963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b5c101c00 0x7f7b5c19e560 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:16.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.563+0000 7f7b64800640 1 -- 172.21.15.123:0/2532297963 >> 172.21.15.123:0/2532297963 conn(0x7f7b5c0fd390 msgr2=0x7f7b5c1085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:16.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.563+0000 7f7b64800640 1 -- 172.21.15.123:0/2532297963 shutdown_connections 2024-09-19T04:46:16.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:16.564+0000 7f7b64800640 1 -- 172.21.15.123:0/2532297963 wait complete. 2024-09-19T04:46:16.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:16 smithi183 ceph-mon[35333]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:46:16.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:16 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2532297963' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T04:46:16.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:16 smithi123 ceph-mon[30511]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:46:16.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:16 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2532297963' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T04:46:16.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:16 smithi193 ceph-mon[35592]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:46:16.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:16 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2532297963' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T04:46:17.438 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-19T04:46:17.806 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:18.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.346+0000 7f7c6e000640 1 -- 172.21.15.123:0/4289731655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c681037c0 msgr2=0x7f7c6810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:18.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.346+0000 7f7c6e000640 1 --2- 172.21.15.123:0/4289731655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c681037c0 0x7f7c6810a050 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009ca0 tx=0x7f7c6002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:46:18.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.346+0000 7f7c6e000640 1 -- 172.21.15.123:0/4289731655 shutdown_connections 2024-09-19T04:46:18.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.346+0000 7f7c6e000640 1 --2- 172.21.15.123:0/4289731655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c681037c0 0x7f7c6810a050 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:18.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.346+0000 7f7c6e000640 1 --2- 172.21.15.123:0/4289731655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c68102e00 0x7f7c68103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:18.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.346+0000 7f7c6e000640 1 --2- 172.21.15.123:0/4289731655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c68101c00 0x7f7c68102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:18.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.346+0000 7f7c6e000640 1 -- 172.21.15.123:0/4289731655 >> 172.21.15.123:0/4289731655 conn(0x7f7c680fd390 msgr2=0x7f7c680ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:18.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.346+0000 7f7c6e000640 1 -- 172.21.15.123:0/4289731655 shutdown_connections 2024-09-19T04:46:18.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.346+0000 7f7c6e000640 1 -- 172.21.15.123:0/4289731655 wait complete. 2024-09-19T04:46:18.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.347+0000 7f7c6e000640 1 Processor -- start 2024-09-19T04:46:18.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.347+0000 7f7c6e000640 1 -- start start 2024-09-19T04:46:18.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.348+0000 7f7c6e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c68101c00 0x7f7c6819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:18.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.348+0000 7f7c6e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c68102e00 0x7f7c6819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:18.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.348+0000 7f7c6e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c681037c0 0x7f7c681a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:18.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.348+0000 7f7c6e000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6810d8d0 con 0x7f7c68101c00 2024-09-19T04:46:18.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.348+0000 7f7c6e000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6810d750 con 0x7f7c681037c0 2024-09-19T04:46:18.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.348+0000 7f7c6e000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6810da50 con 0x7f7c68102e00 2024-09-19T04:46:18.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.348+0000 7f7c67400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c68101c00 0x7f7c6819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:18.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.348+0000 7f7c67400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c68101c00 0x7f7c6819a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46008/0 (socket says 172.21.15.123:46008) 2024-09-19T04:46:18.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.348+0000 7f7c67400640 1 -- 172.21.15.123:0/2709406120 learned_addr learned my addr 172.21.15.123:0/2709406120 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:18.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.348+0000 7f7c5ea00640 1 --2- 172.21.15.123:0/2709406120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c68102e00 0x7f7c6819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:18.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.348+0000 7f7c67e00640 1 --2- 172.21.15.123:0/2709406120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c681037c0 0x7f7c681a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:18.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.348+0000 7f7c67400640 1 -- 172.21.15.123:0/2709406120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c68102e00 msgr2=0x7f7c6819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:18.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.348+0000 7f7c67400640 1 --2- 172.21.15.123:0/2709406120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c68102e00 0x7f7c6819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:18.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.348+0000 7f7c67400640 1 -- 172.21.15.123:0/2709406120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c681037c0 msgr2=0x7f7c681a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:18.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.348+0000 7f7c67400640 1 --2- 172.21.15.123:0/2709406120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c681037c0 0x7f7c681a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:18.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.348+0000 7f7c67400640 1 -- 172.21.15.123:0/2709406120 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c681a1ca0 con 0x7f7c68101c00 2024-09-19T04:46:18.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.348+0000 7f7c5ea00640 1 --2- 172.21.15.123:0/2709406120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c68102e00 0x7f7c6819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:18.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.349+0000 7f7c67400640 1 --2- 172.21.15.123:0/2709406120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c68101c00 0x7f7c6819a070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800e9c0 tx=0x7f7c5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:18.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.349+0000 7f7c64c00640 1 -- 172.21.15.123:0/2709406120 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5800cd80 con 0x7f7c68101c00 2024-09-19T04:46:18.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.349+0000 7f7c64c00640 1 -- 172.21.15.123:0/2709406120 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c580035f0 con 0x7f7c68101c00 2024-09-19T04:46:18.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.349+0000 7f7c64c00640 1 -- 172.21.15.123:0/2709406120 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58010690 con 0x7f7c68101c00 2024-09-19T04:46:18.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.349+0000 7f7c6e000640 1 -- 172.21.15.123:0/2709406120 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c681a1f90 con 0x7f7c68101c00 2024-09-19T04:46:18.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.350+0000 7f7c6e000640 1 -- 172.21.15.123:0/2709406120 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c681a2330 con 0x7f7c68101c00 2024-09-19T04:46:18.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.350+0000 7f7c6e000640 1 -- 172.21.15.123:0/2709406120 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c681a25f0 con 0x7f7c68101c00 2024-09-19T04:46:18.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.352+0000 7f7c64c00640 1 -- 172.21.15.123:0/2709406120 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f7c580037a0 con 0x7f7c68101c00 2024-09-19T04:46:18.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.353+0000 7f7c64c00640 1 --2- 172.21.15.123:0/2709406120 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7c4c0a3270 0x7f7c4c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:18.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.354+0000 7f7c64c00640 1 -- 172.21.15.123:0/2709406120 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f7c580cfcd0 con 0x7f7c68101c00 2024-09-19T04:46:18.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.354+0000 7f7c5ea00640 1 --2- 172.21.15.123:0/2709406120 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7c4c0a3270 0x7f7c4c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:18.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.354+0000 7f7c5ea00640 1 --2- 172.21.15.123:0/2709406120 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7c4c0a3270 0x7f7c4c0a5730 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7c6819b590 tx=0x7f7c50005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:18.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.358+0000 7f7c64c00640 1 -- 172.21.15.123:0/2709406120 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7c5809c9a0 con 0x7f7c68101c00 2024-09-19T04:46:18.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.536+0000 7f7c6e000640 1 -- 172.21.15.123:0/2709406120 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f7c681a2e90 con 0x7f7c68101c00 2024-09-19T04:46:18.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.537+0000 7f7c64c00640 1 -- 172.21.15.123:0/2709406120 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v86) ==== 56+0+5313 (secure 0 0 0) 0x7f7c5809c2a0 con 0x7f7c68101c00 2024-09-19T04:46:18.539 INFO:teuthology.orchestra.run.smithi123.stdout:epoch 86 2024-09-19T04:46:18.539 INFO:teuthology.orchestra.run.smithi123.stdout:fsid 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T04:46:18.539 INFO:teuthology.orchestra.run.smithi123.stdout:created 2024-09-19T04:36:21.083333+0000 2024-09-19T04:46:18.539 INFO:teuthology.orchestra.run.smithi123.stdout:modified 2024-09-19T04:45:14.619563+0000 2024-09-19T04:46:18.539 INFO:teuthology.orchestra.run.smithi123.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T04:46:18.539 INFO:teuthology.orchestra.run.smithi123.stdout:crush_version 26 2024-09-19T04:46:18.539 INFO:teuthology.orchestra.run.smithi123.stdout:full_ratio 0.95 2024-09-19T04:46:18.539 INFO:teuthology.orchestra.run.smithi123.stdout:backfillfull_ratio 0.9 2024-09-19T04:46:18.540 INFO:teuthology.orchestra.run.smithi123.stdout:nearfull_ratio 0.85 2024-09-19T04:46:18.540 INFO:teuthology.orchestra.run.smithi123.stdout:require_min_compat_client luminous 2024-09-19T04:46:18.540 INFO:teuthology.orchestra.run.smithi123.stdout:min_compat_client luminous 2024-09-19T04:46:18.540 INFO:teuthology.orchestra.run.smithi123.stdout:require_osd_release squid 2024-09-19T04:46:18.540 INFO:teuthology.orchestra.run.smithi123.stdout:stretch_mode_enabled false 2024-09-19T04:46:18.540 INFO:teuthology.orchestra.run.smithi123.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-19T04:46:18.540 INFO:teuthology.orchestra.run.smithi123.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-19T04:46:18.540 INFO:teuthology.orchestra.run.smithi123.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-19T04:46:18.540 INFO:teuthology.orchestra.run.smithi123.stdout:max_osd 12 2024-09-19T04:46:18.540 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6802/3631596501,v1:172.21.15.123:6803/3631596501] [v2:172.21.15.123:6804/3631596501,v1:172.21.15.123:6805/3631596501] exists,up c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b 2024-09-19T04:46:18.540 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/1259721614,v1:172.21.15.183:6801/1259721614] [v2:172.21.15.183:6802/1259721614,v1:172.21.15.183:6803/1259721614] exists,up a526d4ce-586b-4d66-b720-e868df92fa24 2024-09-19T04:46:18.540 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6800/2708151726,v1:172.21.15.193:6801/2708151726] [v2:172.21.15.193:6802/2708151726,v1:172.21.15.193:6803/2708151726] exists,up 3650601a-6259-4b2a-97df-1d47b09b0127 2024-09-19T04:46:18.541 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6810/3214806282,v1:172.21.15.123:6811/3214806282] [v2:172.21.15.123:6812/3214806282,v1:172.21.15.123:6813/3214806282] exists,up 67c485ad-136e-4020-950d-9171a3c2de35 2024-09-19T04:46:18.541 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4 up in weight 1 up_from 32 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/667618665,v1:172.21.15.183:6809/667618665] [v2:172.21.15.183:6810/667618665,v1:172.21.15.183:6811/667618665] exists,up c9098540-06ab-45a8-9cf9-f90519cc852e 2024-09-19T04:46:18.541 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5 up in weight 1 up_from 38 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6808/2292251639,v1:172.21.15.193:6809/2292251639] [v2:172.21.15.193:6810/2292251639,v1:172.21.15.193:6811/2292251639] exists,up 278a8dbd-6492-42d4-b489-1f832350bc6e 2024-09-19T04:46:18.541 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6 up in weight 1 up_from 45 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6818/226007499,v1:172.21.15.123:6819/226007499] [v2:172.21.15.123:6820/226007499,v1:172.21.15.123:6821/226007499] exists,up 0fda4e39-6adb-4941-a39b-a9a15019088c 2024-09-19T04:46:18.541 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7 up in weight 1 up_from 50 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/3718518226,v1:172.21.15.183:6817/3718518226] [v2:172.21.15.183:6818/3718518226,v1:172.21.15.183:6819/3718518226] exists,up 5c0a021e-9f3b-4766-9686-0ef24b68cf25 2024-09-19T04:46:18.541 INFO:teuthology.orchestra.run.smithi123.stdout:osd.8 up in weight 1 up_from 57 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6816/3551404118,v1:172.21.15.193:6817/3551404118] [v2:172.21.15.193:6818/3551404118,v1:172.21.15.193:6819/3551404118] exists,up 9b0e67e0-6305-4f81-9211-d41960099a8b 2024-09-19T04:46:18.541 INFO:teuthology.orchestra.run.smithi123.stdout:osd.9 up in weight 1 up_from 65 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6826/1439901149,v1:172.21.15.123:6827/1439901149] [v2:172.21.15.123:6828/1439901149,v1:172.21.15.123:6829/1439901149] exists,up 978ef3fa-c6ec-421f-b194-56d44813be5b 2024-09-19T04:46:18.541 INFO:teuthology.orchestra.run.smithi123.stdout:osd.10 up in weight 1 up_from 70 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/3355349245,v1:172.21.15.183:6825/3355349245] [v2:172.21.15.183:6826/3355349245,v1:172.21.15.183:6827/3355349245] exists,up 5495e09b-7809-47b0-b7e0-b56c452f9f22 2024-09-19T04:46:18.541 INFO:teuthology.orchestra.run.smithi123.stdout:osd.11 up in weight 1 up_from 77 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6824/4291617038,v1:172.21.15.193:6825/4291617038] [v2:172.21.15.193:6826/4291617038,v1:172.21.15.193:6827/4291617038] exists,up 03046fef-b16b-42f8-9a19-6c0b78cbfe7b 2024-09-19T04:46:18.541 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 2.a [2,11] 2024-09-19T04:46:18.541 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 2.1d [8,11] 2024-09-19T04:46:18.541 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.2 [3,9] 2024-09-19T04:46:18.541 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.3 [4,9] 2024-09-19T04:46:18.542 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.6 [4,6] 2024-09-19T04:46:18.542 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.21 [5,9] 2024-09-19T04:46:18.542 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.23 [4,7] 2024-09-19T04:46:18.542 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.28 [5,8] 2024-09-19T04:46:18.542 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/4040401634 expires 2024-09-20T04:37:13.257785+0000 2024-09-19T04:46:18.542 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/3498196650 expires 2024-09-20T04:37:13.257785+0000 2024-09-19T04:46:18.542 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:6801/3323203947 expires 2024-09-20T04:37:13.257785+0000 2024-09-19T04:46:18.542 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:6800/3323203947 expires 2024-09-20T04:37:13.257785+0000 2024-09-19T04:46:18.542 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:6801/2065492981 expires 2024-09-20T04:36:45.913060+0000 2024-09-19T04:46:18.542 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:6800/2065492981 expires 2024-09-20T04:36:45.913060+0000 2024-09-19T04:46:18.542 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/471028951 expires 2024-09-20T04:37:13.257785+0000 2024-09-19T04:46:18.542 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/1009496020 expires 2024-09-20T04:36:45.913060+0000 2024-09-19T04:46:18.542 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/2000593686 expires 2024-09-20T04:36:45.913060+0000 2024-09-19T04:46:18.542 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/2580747029 expires 2024-09-20T04:36:45.913060+0000 2024-09-19T04:46:18.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.541+0000 7f7c6e000640 1 -- 172.21.15.123:0/2709406120 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7c4c0a3270 msgr2=0x7f7c4c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:18.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.541+0000 7f7c6e000640 1 --2- 172.21.15.123:0/2709406120 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7c4c0a3270 0x7f7c4c0a5730 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7c6819b590 tx=0x7f7c50005f90 comp rx=0 tx=0).stop 2024-09-19T04:46:18.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.541+0000 7f7c6e000640 1 -- 172.21.15.123:0/2709406120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c68101c00 msgr2=0x7f7c6819a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:18.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.541+0000 7f7c6e000640 1 --2- 172.21.15.123:0/2709406120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c68101c00 0x7f7c6819a070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800e9c0 tx=0x7f7c5800ee80 comp rx=0 tx=0).stop 2024-09-19T04:46:18.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.541+0000 7f7c6e000640 1 -- 172.21.15.123:0/2709406120 shutdown_connections 2024-09-19T04:46:18.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.541+0000 7f7c6e000640 1 --2- 172.21.15.123:0/2709406120 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f7c4c0a3270 0x7f7c4c0a5730 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:18.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.541+0000 7f7c6e000640 1 --2- 172.21.15.123:0/2709406120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c681037c0 0x7f7c681a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:18.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.541+0000 7f7c6e000640 1 --2- 172.21.15.123:0/2709406120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c68102e00 0x7f7c6819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:18.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.541+0000 7f7c6e000640 1 --2- 172.21.15.123:0/2709406120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c68101c00 0x7f7c6819a070 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:18.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.541+0000 7f7c6e000640 1 -- 172.21.15.123:0/2709406120 >> 172.21.15.123:0/2709406120 conn(0x7f7c680fd390 msgr2=0x7f7c68074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:18.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.541+0000 7f7c6e000640 1 -- 172.21.15.123:0/2709406120 shutdown_connections 2024-09-19T04:46:18.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:18.541+0000 7f7c6e000640 1 -- 172.21.15.123:0/2709406120 wait complete. 2024-09-19T04:46:18.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:18 smithi183 ceph-mon[35333]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T04:46:18.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:18 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2709406120' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T04:46:18.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:18 smithi123 ceph-mon[30511]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T04:46:18.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:18 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2709406120' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T04:46:18.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:18 smithi193 ceph-mon[35592]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T04:46:18.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:18 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2709406120' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T04:46:19.439 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T04:46:19.450 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:46:19.818 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:20.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.519+0000 7f8fe3e00640 1 -- 172.21.15.123:0/2394785823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe41047e0 msgr2=0x7f8fe4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:20.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.519+0000 7f8fe3e00640 1 --2- 172.21.15.123:0/2394785823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe41047e0 0x7f8fe4108830 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009900 tx=0x7f8fd002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:46:20.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.520+0000 7f8fe3e00640 1 -- 172.21.15.123:0/2394785823 shutdown_connections 2024-09-19T04:46:20.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.520+0000 7f8fe3e00640 1 --2- 172.21.15.123:0/2394785823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fe4108f60 0x7f8fe4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:20.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.520+0000 7f8fe3e00640 1 --2- 172.21.15.123:0/2394785823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe41047e0 0x7f8fe4108830 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:20.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.520+0000 7f8fe3e00640 1 --2- 172.21.15.123:0/2394785823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fe4103e30 0x7f8fe4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:20.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.520+0000 7f8fe3e00640 1 -- 172.21.15.123:0/2394785823 >> 172.21.15.123:0/2394785823 conn(0x7f8fe40ff8d0 msgr2=0x7f8fe4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:20.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.520+0000 7f8fe3e00640 1 -- 172.21.15.123:0/2394785823 shutdown_connections 2024-09-19T04:46:20.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.520+0000 7f8fe3e00640 1 -- 172.21.15.123:0/2394785823 wait complete. 2024-09-19T04:46:20.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.521+0000 7f8fe3e00640 1 Processor -- start 2024-09-19T04:46:20.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.521+0000 7f8fe3e00640 1 -- start start 2024-09-19T04:46:20.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.521+0000 7f8fe3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe4103e30 0x7f8fe419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:20.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.521+0000 7f8fe3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fe41047e0 0x7f8fe419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:20.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.521+0000 7f8fe3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fe4108f60 0x7f8fe419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:20.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.521+0000 7f8fe3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe4113f80 con 0x7f8fe4103e30 2024-09-19T04:46:20.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.521+0000 7f8fe3e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe4113e00 con 0x7f8fe4108f60 2024-09-19T04:46:20.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.521+0000 7f8fe3e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe4114100 con 0x7f8fe41047e0 2024-09-19T04:46:20.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.522+0000 7f8fe2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe4103e30 0x7f8fe419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:20.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.522+0000 7f8fe2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe4103e30 0x7f8fe419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46022/0 (socket says 172.21.15.123:46022) 2024-09-19T04:46:20.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.522+0000 7f8fe2a00640 1 -- 172.21.15.123:0/3956482823 learned_addr learned my addr 172.21.15.123:0/3956482823 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:20.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.522+0000 7f8fe3400640 1 --2- 172.21.15.123:0/3956482823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fe4108f60 0x7f8fe419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:20.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.522+0000 7f8fe2000640 1 --2- 172.21.15.123:0/3956482823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fe41047e0 0x7f8fe419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:20.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.522+0000 7f8fe2a00640 1 -- 172.21.15.123:0/3956482823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fe41047e0 msgr2=0x7f8fe419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:20.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.522+0000 7f8fe2a00640 1 --2- 172.21.15.123:0/3956482823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fe41047e0 0x7f8fe419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:20.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.522+0000 7f8fe2a00640 1 -- 172.21.15.123:0/3956482823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fe4108f60 msgr2=0x7f8fe419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:20.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.522+0000 7f8fe2a00640 1 --2- 172.21.15.123:0/3956482823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fe4108f60 0x7f8fe419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:20.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.522+0000 7f8fe2a00640 1 -- 172.21.15.123:0/3956482823 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe419f430 con 0x7f8fe4103e30 2024-09-19T04:46:20.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.522+0000 7f8fe2a00640 1 --2- 172.21.15.123:0/3956482823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe4103e30 0x7f8fe419a480 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400d950 tx=0x7f8fd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:20.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.523+0000 7f8fdb400640 1 -- 172.21.15.123:0/3956482823 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4017070 con 0x7f8fe4103e30 2024-09-19T04:46:20.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.523+0000 7f8fdb400640 1 -- 172.21.15.123:0/3956482823 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fd4009c30 con 0x7f8fe4103e30 2024-09-19T04:46:20.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.523+0000 7f8fdb400640 1 -- 172.21.15.123:0/3956482823 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd400f6b0 con 0x7f8fe4103e30 2024-09-19T04:46:20.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.523+0000 7f8fe3e00640 1 -- 172.21.15.123:0/3956482823 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe419f720 con 0x7f8fe4103e30 2024-09-19T04:46:20.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.523+0000 7f8fe3e00640 1 -- 172.21.15.123:0/3956482823 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe41a7000 con 0x7f8fe4103e30 2024-09-19T04:46:20.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.524+0000 7f8fe3e00640 1 -- 172.21.15.123:0/3956482823 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe4104210 con 0x7f8fe4103e30 2024-09-19T04:46:20.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.526+0000 7f8fdb400640 1 -- 172.21.15.123:0/3956482823 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f8fd40109d0 con 0x7f8fe4103e30 2024-09-19T04:46:20.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.527+0000 7f8fdb400640 1 --2- 172.21.15.123:0/3956482823 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8fc80a32c0 0x7f8fc80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:20.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.527+0000 7f8fdb400640 1 -- 172.21.15.123:0/3956482823 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f8fd40cf4a0 con 0x7f8fe4103e30 2024-09-19T04:46:20.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.527+0000 7f8fe2000640 1 --2- 172.21.15.123:0/3956482823 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8fc80a32c0 0x7f8fc80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:20.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.528+0000 7f8fe2000640 1 --2- 172.21.15.123:0/3956482823 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8fc80a32c0 0x7f8fc80a5780 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009230 tx=0x7f8fd00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:20.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.531+0000 7f8fdb400640 1 -- 172.21.15.123:0/3956482823 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8fd4009de0 con 0x7f8fe4103e30 2024-09-19T04:46:20.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.742+0000 7f8fe3e00640 1 -- 172.21.15.123:0/3956482823 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fe410bbb0 con 0x7f8fe4103e30 2024-09-19T04:46:20.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.742+0000 7f8fdb400640 1 -- 172.21.15.123:0/3956482823 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11990 (secure 0 0 0) 0x7f8fd409bb20 con 0x7f8fe4103e30 2024-09-19T04:46:20.744 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:46:20.744 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"btime":"2024-09-19T04:44:44:834422+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:44:44.834407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24529,24505,24539,24500,24499]},"id":1}]} 2024-09-19T04:46:20.745 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 27 2024-09-19T04:46:20.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.745+0000 7f8fe3e00640 1 -- 172.21.15.123:0/3956482823 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8fc80a32c0 msgr2=0x7f8fc80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:20.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.745+0000 7f8fe3e00640 1 --2- 172.21.15.123:0/3956482823 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8fc80a32c0 0x7f8fc80a5780 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009230 tx=0x7f8fd00058d0 comp rx=0 tx=0).stop 2024-09-19T04:46:20.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.745+0000 7f8fe3e00640 1 -- 172.21.15.123:0/3956482823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe4103e30 msgr2=0x7f8fe419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:20.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.745+0000 7f8fe3e00640 1 --2- 172.21.15.123:0/3956482823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe4103e30 0x7f8fe419a480 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400d950 tx=0x7f8fd400de10 comp rx=0 tx=0).stop 2024-09-19T04:46:20.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.745+0000 7f8fe3e00640 1 -- 172.21.15.123:0/3956482823 shutdown_connections 2024-09-19T04:46:20.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.745+0000 7f8fe3e00640 1 --2- 172.21.15.123:0/3956482823 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8fc80a32c0 0x7f8fc80a5780 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:20.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.745+0000 7f8fe3e00640 1 --2- 172.21.15.123:0/3956482823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fe4108f60 0x7f8fe419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:20.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.745+0000 7f8fe3e00640 1 --2- 172.21.15.123:0/3956482823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fe41047e0 0x7f8fe419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:20.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.745+0000 7f8fe3e00640 1 --2- 172.21.15.123:0/3956482823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe4103e30 0x7f8fe419a480 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:20.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.745+0000 7f8fe3e00640 1 -- 172.21.15.123:0/3956482823 >> 172.21.15.123:0/3956482823 conn(0x7f8fe40ff8d0 msgr2=0x7f8fe4100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:20.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.745+0000 7f8fe3e00640 1 -- 172.21.15.123:0/3956482823 shutdown_connections 2024-09-19T04:46:20.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:20.745+0000 7f8fe3e00640 1 -- 172.21.15.123:0/3956482823 wait complete. 2024-09-19T04:46:20.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:20 smithi183 ceph-mon[35333]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-19T04:46:20.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:20 smithi123 ceph-mon[30511]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-19T04:46:20.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:20 smithi193 ceph-mon[35592]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-19T04:46:21.655 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:46:21.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:21 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3956482823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:46:21.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:21 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3956482823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:46:21.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:21 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3956482823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:46:22.023 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:22.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.567+0000 7f9a8c600640 1 -- 172.21.15.123:0/4042464683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a84106f10 msgr2=0x7f9a8410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:22.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.567+0000 7f9a8c600640 1 --2- 172.21.15.123:0/4042464683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a84106f10 0x7f9a8410ea90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009ca0 tx=0x7f9a7c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:46:22.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.567+0000 7f9a8c600640 1 -- 172.21.15.123:0/4042464683 shutdown_connections 2024-09-19T04:46:22.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.567+0000 7f9a8c600640 1 --2- 172.21.15.123:0/4042464683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a84106f10 0x7f9a8410ea90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:22.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.567+0000 7f9a8c600640 1 --2- 172.21.15.123:0/4042464683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a84102790 0x7f9a841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:22.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.567+0000 7f9a8c600640 1 --2- 172.21.15.123:0/4042464683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a84101de0 0x7f9a841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:22.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.567+0000 7f9a8c600640 1 -- 172.21.15.123:0/4042464683 >> 172.21.15.123:0/4042464683 conn(0x7f9a840fd8c0 msgr2=0x7f9a840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:22.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.567+0000 7f9a8c600640 1 -- 172.21.15.123:0/4042464683 shutdown_connections 2024-09-19T04:46:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.567+0000 7f9a8c600640 1 -- 172.21.15.123:0/4042464683 wait complete. 2024-09-19T04:46:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.568+0000 7f9a8c600640 1 Processor -- start 2024-09-19T04:46:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.568+0000 7f9a8c600640 1 -- start start 2024-09-19T04:46:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.569+0000 7f9a8c600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a84101de0 0x7f9a8419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.569+0000 7f9a8c600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a84102790 0x7f9a8419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.569+0000 7f9a8c600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a84106f10 0x7f9a8419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.569+0000 7f9a8c600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9a84111f70 con 0x7f9a84102790 2024-09-19T04:46:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.569+0000 7f9a8c600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9a84111df0 con 0x7f9a84106f10 2024-09-19T04:46:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.569+0000 7f9a8c600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a841120f0 con 0x7f9a84101de0 2024-09-19T04:46:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.569+0000 7f9a89c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a84101de0 0x7f9a8419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.569+0000 7f9a89c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a84101de0 0x7f9a8419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42558/0 (socket says 172.21.15.123:42558) 2024-09-19T04:46:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.569+0000 7f9a89c00640 1 -- 172.21.15.123:0/2513785741 learned_addr learned my addr 172.21.15.123:0/2513785741 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.569+0000 7f9a89200640 1 --2- 172.21.15.123:0/2513785741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a84102790 0x7f9a8419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.569+0000 7f9a8a600640 1 --2- 172.21.15.123:0/2513785741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a84106f10 0x7f9a8419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.569+0000 7f9a89c00640 1 -- 172.21.15.123:0/2513785741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a84106f10 msgr2=0x7f9a8419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.569+0000 7f9a89c00640 1 --2- 172.21.15.123:0/2513785741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a84106f10 0x7f9a8419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.569+0000 7f9a89c00640 1 -- 172.21.15.123:0/2513785741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a84102790 msgr2=0x7f9a8419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.569+0000 7f9a89c00640 1 --2- 172.21.15.123:0/2513785741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a84102790 0x7f9a8419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.569+0000 7f9a89c00640 1 -- 172.21.15.123:0/2513785741 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8419f560 con 0x7f9a84101de0 2024-09-19T04:46:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.570+0000 7f9a89c00640 1 --2- 172.21.15.123:0/2513785741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a84101de0 0x7f9a8419a5b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400e9f0 tx=0x7f9a7400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.570+0000 7f9a82000640 1 -- 172.21.15.123:0/2513785741 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7400cd80 con 0x7f9a84101de0 2024-09-19T04:46:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.570+0000 7f9a82000640 1 -- 172.21.15.123:0/2513785741 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a74005500 con 0x7f9a84101de0 2024-09-19T04:46:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.570+0000 7f9a8c600640 1 -- 172.21.15.123:0/2513785741 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8419f850 con 0x7f9a84101de0 2024-09-19T04:46:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.570+0000 7f9a82000640 1 -- 172.21.15.123:0/2513785741 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74010690 con 0x7f9a84101de0 2024-09-19T04:46:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.570+0000 7f9a8c600640 1 -- 172.21.15.123:0/2513785741 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a841a7130 con 0x7f9a84101de0 2024-09-19T04:46:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.571+0000 7f9a8c600640 1 -- 172.21.15.123:0/2513785741 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a841021c0 con 0x7f9a84101de0 2024-09-19T04:46:22.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.573+0000 7f9a82000640 1 -- 172.21.15.123:0/2513785741 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f9a740046c0 con 0x7f9a84101de0 2024-09-19T04:46:22.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.574+0000 7f9a82000640 1 --2- 172.21.15.123:0/2513785741 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9a680a3240 0x7f9a680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:22.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.574+0000 7f9a82000640 1 -- 172.21.15.123:0/2513785741 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f9a740cffe0 con 0x7f9a84101de0 2024-09-19T04:46:22.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.575+0000 7f9a89200640 1 --2- 172.21.15.123:0/2513785741 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9a680a3240 0x7f9a680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:22.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.575+0000 7f9a89200640 1 --2- 172.21.15.123:0/2513785741 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9a680a3240 0x7f9a680a5700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9a840ff6e0 tx=0x7f9a70005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:22.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.576+0000 7f9a82000640 1 -- 172.21.15.123:0/2513785741 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9a74054210 con 0x7f9a84101de0 2024-09-19T04:46:22.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.791+0000 7f9a8c600640 1 -- 172.21.15.123:0/2513785741 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a8419b910 con 0x7f9a84101de0 2024-09-19T04:46:22.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:22 smithi123 ceph-mon[30511]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-19T04:46:22.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.792+0000 7f9a82000640 1 -- 172.21.15.123:0/2513785741 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11990 (secure 0 0 0) 0x7f9a7409ccf0 con 0x7f9a84101de0 2024-09-19T04:46:22.794 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:46:22.794 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 27 2024-09-19T04:46:22.795 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"btime":"2024-09-19T04:44:44:834422+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:44:44.834407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24529,24505,24539,24500,24499]},"id":1}]} 2024-09-19T04:46:22.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.795+0000 7f9a8c600640 1 -- 172.21.15.123:0/2513785741 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9a680a3240 msgr2=0x7f9a680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:22.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.795+0000 7f9a8c600640 1 --2- 172.21.15.123:0/2513785741 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9a680a3240 0x7f9a680a5700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9a840ff6e0 tx=0x7f9a70005f90 comp rx=0 tx=0).stop 2024-09-19T04:46:22.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.795+0000 7f9a8c600640 1 -- 172.21.15.123:0/2513785741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a84101de0 msgr2=0x7f9a8419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:22.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.795+0000 7f9a8c600640 1 --2- 172.21.15.123:0/2513785741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a84101de0 0x7f9a8419a5b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400e9f0 tx=0x7f9a7400edf0 comp rx=0 tx=0).stop 2024-09-19T04:46:22.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.795+0000 7f9a8c600640 1 -- 172.21.15.123:0/2513785741 shutdown_connections 2024-09-19T04:46:22.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.795+0000 7f9a8c600640 1 --2- 172.21.15.123:0/2513785741 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f9a680a3240 0x7f9a680a5700 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:22.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.795+0000 7f9a8c600640 1 --2- 172.21.15.123:0/2513785741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a84106f10 0x7f9a8419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:22.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.795+0000 7f9a8c600640 1 --2- 172.21.15.123:0/2513785741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a84102790 0x7f9a8419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:22.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.795+0000 7f9a8c600640 1 --2- 172.21.15.123:0/2513785741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a84101de0 0x7f9a8419a5b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:22.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.795+0000 7f9a8c600640 1 -- 172.21.15.123:0/2513785741 >> 172.21.15.123:0/2513785741 conn(0x7f9a840fd8c0 msgr2=0x7f9a840feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:22.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.795+0000 7f9a8c600640 1 -- 172.21.15.123:0/2513785741 shutdown_connections 2024-09-19T04:46:22.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:22.795+0000 7f9a8c600640 1 -- 172.21.15.123:0/2513785741 wait complete. 2024-09-19T04:46:22.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:22 smithi183 ceph-mon[35333]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-19T04:46:22.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:22 smithi193 ceph-mon[35592]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-19T04:46:23.573 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:46:23.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:23 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2513785741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:46:23.943 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:23.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:23 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2513785741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:46:24.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:23 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2513785741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:46:24.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.532+0000 7f29dd000640 1 -- 172.21.15.123:0/3536276744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29d8109140 msgr2=0x7f29d8110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:24.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.532+0000 7f29dd000640 1 --2- 172.21.15.123:0/3536276744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29d8109140 0x7f29d8110cc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009be0 tx=0x7f29d002a6c0 comp rx=0 tx=0).stop 2024-09-19T04:46:24.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.532+0000 7f29dd000640 1 -- 172.21.15.123:0/3536276744 shutdown_connections 2024-09-19T04:46:24.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.532+0000 7f29dd000640 1 --2- 172.21.15.123:0/3536276744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29d8109140 0x7f29d8110cc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:24.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.532+0000 7f29dd000640 1 --2- 172.21.15.123:0/3536276744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29d81049c0 0x7f29d8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:24.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.532+0000 7f29dd000640 1 --2- 172.21.15.123:0/3536276744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29d8104010 0x7f29d81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:24.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.532+0000 7f29dd000640 1 -- 172.21.15.123:0/3536276744 >> 172.21.15.123:0/3536276744 conn(0x7f29d80ffa50 msgr2=0x7f29d8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:24.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.532+0000 7f29dd000640 1 -- 172.21.15.123:0/3536276744 shutdown_connections 2024-09-19T04:46:24.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.532+0000 7f29dd000640 1 -- 172.21.15.123:0/3536276744 wait complete. 2024-09-19T04:46:24.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.533+0000 7f29dd000640 1 Processor -- start 2024-09-19T04:46:24.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.533+0000 7f29dd000640 1 -- start start 2024-09-19T04:46:24.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.534+0000 7f29dd000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29d8104010 0x7f29d819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:24.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.534+0000 7f29dd000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29d81049c0 0x7f29d819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:24.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.534+0000 7f29dd000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29d8109140 0x7f29d819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:24.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.534+0000 7f29dd000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29d8114120 con 0x7f29d8104010 2024-09-19T04:46:24.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.534+0000 7f29dd000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f29d8113fa0 con 0x7f29d8109140 2024-09-19T04:46:24.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.534+0000 7f29dd000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f29d81142a0 con 0x7f29d81049c0 2024-09-19T04:46:24.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.534+0000 7f29d5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29d8104010 0x7f29d819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:24.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.534+0000 7f29d5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29d8104010 0x7f29d819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49816/0 (socket says 172.21.15.123:49816) 2024-09-19T04:46:24.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.534+0000 7f29d5e00640 1 -- 172.21.15.123:0/3350603385 learned_addr learned my addr 172.21.15.123:0/3350603385 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:24.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.534+0000 7f29d5400640 1 --2- 172.21.15.123:0/3350603385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29d81049c0 0x7f29d819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:24.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.534+0000 7f29d6800640 1 --2- 172.21.15.123:0/3350603385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29d8109140 0x7f29d819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:24.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.535+0000 7f29d5e00640 1 -- 172.21.15.123:0/3350603385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29d81049c0 msgr2=0x7f29d819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:24.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.535+0000 7f29d5e00640 1 --2- 172.21.15.123:0/3350603385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29d81049c0 0x7f29d819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:24.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.535+0000 7f29d5e00640 1 -- 172.21.15.123:0/3350603385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29d8109140 msgr2=0x7f29d819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:24.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.535+0000 7f29d5e00640 1 --2- 172.21.15.123:0/3350603385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29d8109140 0x7f29d819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:24.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.535+0000 7f29d5e00640 1 -- 172.21.15.123:0/3350603385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d819f5f0 con 0x7f29d8104010 2024-09-19T04:46:24.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.535+0000 7f29d5e00640 1 --2- 172.21.15.123:0/3350603385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29d8104010 0x7f29d819a640 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f29c400e9f0 tx=0x7f29c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:24.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.535+0000 7f29cea00640 1 -- 172.21.15.123:0/3350603385 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c400ccc0 con 0x7f29d8104010 2024-09-19T04:46:24.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.535+0000 7f29dd000640 1 -- 172.21.15.123:0/3350603385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d819f8e0 con 0x7f29d8104010 2024-09-19T04:46:24.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.535+0000 7f29cea00640 1 -- 172.21.15.123:0/3350603385 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c400ce60 con 0x7f29d8104010 2024-09-19T04:46:24.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.535+0000 7f29cea00640 1 -- 172.21.15.123:0/3350603385 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4010690 con 0x7f29d8104010 2024-09-19T04:46:24.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.535+0000 7f29dd000640 1 -- 172.21.15.123:0/3350603385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d81a71c0 con 0x7f29d8104010 2024-09-19T04:46:24.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.537+0000 7f29dd000640 1 -- 172.21.15.123:0/3350603385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d81043f0 con 0x7f29d8104010 2024-09-19T04:46:24.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.539+0000 7f29cea00640 1 -- 172.21.15.123:0/3350603385 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f29c40046c0 con 0x7f29d8104010 2024-09-19T04:46:24.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.540+0000 7f29cea00640 1 --2- 172.21.15.123:0/3350603385 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f29bc0a3240 0x7f29bc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:24.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.540+0000 7f29cea00640 1 -- 172.21.15.123:0/3350603385 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f29c40cfd80 con 0x7f29d8104010 2024-09-19T04:46:24.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.540+0000 7f29d5400640 1 --2- 172.21.15.123:0/3350603385 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f29bc0a3240 0x7f29bc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:24.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.541+0000 7f29d5400640 1 --2- 172.21.15.123:0/3350603385 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f29bc0a3240 0x7f29bc0a5700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f29d810ab10 tx=0x7f29c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:24.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.544+0000 7f29cea00640 1 -- 172.21.15.123:0/3350603385 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f29c409ca50 con 0x7f29d8104010 2024-09-19T04:46:24.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:24 smithi123 ceph-mon[30511]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-19T04:46:24.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.733+0000 7f29dd000640 1 -- 172.21.15.123:0/3350603385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29d819b9a0 con 0x7f29d8104010 2024-09-19T04:46:24.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.735+0000 7f29cea00640 1 -- 172.21.15.123:0/3350603385 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22794 (secure 0 0 0) 0x7f29c409c350 con 0x7f29d8104010 2024-09-19T04:46:24.737 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:46:24.737 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":86,"fsid":"911fa052-7640-11ef-bceb-c7b262605968","created":"2024-09-19T04:36:21.083333+0000","modified":"2024-09-19T04:45:14.619563+0000","last_up_change":"2024-09-19T04:43:33.818439+0000","last_in_change":"2024-09-19T04:43:10.148272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:39:33.856981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:44:26.956483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:44:29.718625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6803","nonce":3631596501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6805","nonce":3631596501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6809","nonce":3631596501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6807","nonce":3631596501}]},"public_addr":"172.21.15.123:6803/3631596501","cluster_addr":"172.21.15.123:6805/3631596501","heartbeat_back_addr":"172.21.15.123:6809/3631596501","heartbeat_front_addr":"172.21.15.123:6807/3631596501","state":["exists","up"]},{"osd":1,"uuid":"a526d4ce-586b-4d66-b720-e868df92fa24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6801","nonce":1259721614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6803","nonce":1259721614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6807","nonce":1259721614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6805","nonce":1259721614}]},"public_addr":"172.21.15.183:6801/1259721614","cluster_addr":"172.21.15.183:6803/1259721614","heartbeat_back_addr":"172.21.15.183:6807/1259721614","heartbeat_front_addr":"172.21.15.183:6805/1259721614","state":["exists","up"]},{"osd":2,"uuid":"3650601a-6259-4b2a-97df-1d47b09b0127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6801","nonce":2708151726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6803","nonce":2708151726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6807","nonce":2708151726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6805","nonce":2708151726}]},"public_addr":"172.21.15.193:6801/2708151726","cluster_addr":"172.21.15.193:6803/2708151726","heartbeat_back_addr":"172.21.15.193:6807/2708151726","heartbeat_front_addr":"172.21.15.193:6805/2708151726","state":["exists","up"]},{"osd":3,"uuid":"67c485ad-136e-4020-950d-9171a3c2de35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6811","nonce":3214806282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6813","nonce":3214806282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6817","nonce":3214806282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6815","nonce":3214806282}]},"public_addr":"172.21.15.123:6811/3214806282","cluster_addr":"172.21.15.123:6813/3214806282","heartbeat_back_addr":"172.21.15.123:6817/3214806282","heartbeat_front_addr":"172.21.15.123:6815/3214806282","state":["exists","up"]},{"osd":4,"uuid":"c9098540-06ab-45a8-9cf9-f90519cc852e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6809","nonce":667618665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6811","nonce":667618665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6815","nonce":667618665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6813","nonce":667618665}]},"public_addr":"172.21.15.183:6809/667618665","cluster_addr":"172.21.15.183:6811/667618665","heartbeat_back_addr":"172.21.15.183:6815/667618665","heartbeat_front_addr":"172.21.15.183:6813/667618665","state":["exists","up"]},{"osd":5,"uuid":"278a8dbd-6492-42d4-b489-1f832350bc6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6809","nonce":2292251639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6811","nonce":2292251639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6815","nonce":2292251639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6813","nonce":2292251639}]},"public_addr":"172.21.15.193:6809/2292251639","cluster_addr":"172.21.15.193:6811/2292251639","heartbeat_back_addr":"172.21.15.193:6815/2292251639","heartbeat_front_addr":"172.21.15.193:6813/2292251639","state":["exists","up"]},{"osd":6,"uuid":"0fda4e39-6adb-4941-a39b-a9a15019088c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6819","nonce":226007499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6821","nonce":226007499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6825","nonce":226007499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6823","nonce":226007499}]},"public_addr":"172.21.15.123:6819/226007499","cluster_addr":"172.21.15.123:6821/226007499","heartbeat_back_addr":"172.21.15.123:6825/226007499","heartbeat_front_addr":"172.21.15.123:6823/226007499","state":["exists","up"]},{"osd":7,"uuid":"5c0a021e-9f3b-4766-9686-0ef24b68cf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6817","nonce":3718518226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6819","nonce":3718518226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6823","nonce":3718518226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6821","nonce":3718518226}]},"public_addr":"172.21.15.183:6817/3718518226","cluster_addr":"172.21.15.183:6819/3718518226","heartbeat_back_addr":"172.21.15.183:6823/3718518226","heartbeat_front_addr":"172.21.15.183:6821/3718518226","state":["exists","up"]},{"osd":8,"uuid":"9b0e67e0-6305-4f81-9211-d41960099a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6817","nonce":3551404118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6819","nonce":3551404118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6823","nonce":3551404118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6821","nonce":3551404118}]},"public_addr":"172.21.15.193:6817/3551404118","cluster_addr":"172.21.15.193:6819/3551404118","heartbeat_back_addr":"172.21.15.193:6823/3551404118","heartbeat_front_addr":"172.21.15.193:6821/3551404118","state":["exists","up"]},{"osd":9,"uuid":"978ef3fa-c6ec-421f-b194-56d44813be5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6827","nonce":1439901149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6829","nonce":1439901149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6833","nonce":1439901149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6831","nonce":1439901149}]},"public_addr":"172.21.15.123:6827/1439901149","cluster_addr":"172.21.15.123:6829/1439901149","heartbeat_back_addr":"172.21.15.123:6833/1439901149","heartbeat_front_addr":"172.21.15.123:6831/1439901149","state":["exists","up"]},{"osd":10,"uuid":"5495e09b-7809-47b0-b7e0-b56c452f9f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6825","nonce":3355349245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6827","nonce":3355349245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6831","nonce":3355349245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6829","nonce":3355349245}]},"public_addr":"172.21.15.183:6825/3355349245","cluster_addr":"172.21.15.183:6827/3355349245","heartbeat_back_addr":"172.21.15.183:6831/3355349245","heartbeat_front_addr":"172.21.15.183:6829/3355349245","state":["exists","up"]},{"osd":11,"uuid":"03046fef-b16b-42f8-9a19-6c0b78cbfe7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6825","nonce":4291617038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6827","nonce":4291617038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6831","nonce":4291617038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6829","nonce":4291617038}]},"public_addr":"172.21.15.193:6825/4291617038","cluster_addr":"172.21.15.193:6827/4291617038","heartbeat_back_addr":"172.21.15.193:6831/4291617038","heartbeat_front_addr":"172.21.15.193:6829/4291617038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:38:39.142529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:05.336911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:30.310967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:59.655973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:25.890576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:50.778689+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:19.063731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:45.424269+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:10.562149+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:40.382368+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:06.297731+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:32.352172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/4040401634":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/3498196650":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6800/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:6800/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/471028951":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/1009496020":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/2000593686":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/2580747029":"2024-09-20T04:36:45.913060+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:46:24.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.737+0000 7f29dd000640 1 -- 172.21.15.123:0/3350603385 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f29bc0a3240 msgr2=0x7f29bc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:24.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.737+0000 7f29dd000640 1 --2- 172.21.15.123:0/3350603385 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f29bc0a3240 0x7f29bc0a5700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f29d810ab10 tx=0x7f29c8009260 comp rx=0 tx=0).stop 2024-09-19T04:46:24.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.737+0000 7f29dd000640 1 -- 172.21.15.123:0/3350603385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29d8104010 msgr2=0x7f29d819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:24.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.737+0000 7f29dd000640 1 --2- 172.21.15.123:0/3350603385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29d8104010 0x7f29d819a640 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f29c400e9f0 tx=0x7f29c400edf0 comp rx=0 tx=0).stop 2024-09-19T04:46:24.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.738+0000 7f29dd000640 1 -- 172.21.15.123:0/3350603385 shutdown_connections 2024-09-19T04:46:24.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.738+0000 7f29dd000640 1 --2- 172.21.15.123:0/3350603385 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f29bc0a3240 0x7f29bc0a5700 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:24.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.738+0000 7f29dd000640 1 --2- 172.21.15.123:0/3350603385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29d8109140 0x7f29d819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:24.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.738+0000 7f29dd000640 1 --2- 172.21.15.123:0/3350603385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29d81049c0 0x7f29d819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:24.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.738+0000 7f29dd000640 1 --2- 172.21.15.123:0/3350603385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29d8104010 0x7f29d819a640 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:24.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.738+0000 7f29dd000640 1 -- 172.21.15.123:0/3350603385 >> 172.21.15.123:0/3350603385 conn(0x7f29d80ffa50 msgr2=0x7f29d8100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:24.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.738+0000 7f29dd000640 1 -- 172.21.15.123:0/3350603385 shutdown_connections 2024-09-19T04:46:24.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:24.738+0000 7f29dd000640 1 -- 172.21.15.123:0/3350603385 wait complete. 2024-09-19T04:46:24.892 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:24 smithi193 ceph-mon[35592]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-19T04:46:25.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:24 smithi183 ceph-mon[35333]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-19T04:46:25.545 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T04:44:31.766928+0000', 'modified': '2024-09-19T04:44:44.834407+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24539, 'mds_1': 24529, 'mds_2': 24505, 'mds_3': 24500, 'mds_4': 24499}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14514': {'gid': 14514, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6841/784762674', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6840', 'nonce': 784762674}, {'type': 'v1', 'addr': '172.21.15.123:6841', 'nonce': 784762674}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6833/2688234655', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6832', 'nonce': 2688234655}, {'type': 'v1', 'addr': '172.21.15.183:6833', 'nonce': 2688234655}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6833/964700250', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6832', 'nonce': 964700250}, {'type': 'v1', 'addr': '172.21.15.193:6833', 'nonce': 964700250}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'd', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6837/3620540498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 3620540498}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 3620540498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/309335638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 309335638}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 309335638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.193:6835/1351624350', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 1351624350}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 1351624350}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.183:6837/3418533099', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 3418533099}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 3418533099}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.193:6837/3444596480', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 3444596480}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 3444596480}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24529': {'gid': 24529, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.193:6839/4148250455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 4148250455}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 4148250455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.183:6839/157697996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 157697996}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 157697996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24539, 'qdb_cluster': [24529, 24505, 24539, 24500, 24499]} 2024-09-19T04:46:25.545 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T04:46:25.546 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T04:46:25.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:25 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3350603385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:46:25.934 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:25.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:25 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3350603385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:46:26.001 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:25 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3350603385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:46:26.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.509+0000 7f60e3800640 1 -- 172.21.15.123:0/597969453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60dc074620 msgr2=0x7f60dc074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:26.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.509+0000 7f60e3800640 1 --2- 172.21.15.123:0/597969453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60dc074620 0x7f60dc074a00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009900 tx=0x7f60d002a480 comp rx=0 tx=0).stop 2024-09-19T04:46:26.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.509+0000 7f60e3800640 1 -- 172.21.15.123:0/597969453 shutdown_connections 2024-09-19T04:46:26.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.509+0000 7f60e3800640 1 --2- 172.21.15.123:0/597969453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60dc10e3c0 0x7f60dc110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.509+0000 7f60e3800640 1 --2- 172.21.15.123:0/597969453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60dc074f40 0x7f60dc10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.509+0000 7f60e3800640 1 --2- 172.21.15.123:0/597969453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60dc074620 0x7f60dc074a00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.510+0000 7f60e3800640 1 -- 172.21.15.123:0/597969453 >> 172.21.15.123:0/597969453 conn(0x7f60dc0fd8c0 msgr2=0x7f60dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:26.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.510+0000 7f60e3800640 1 -- 172.21.15.123:0/597969453 shutdown_connections 2024-09-19T04:46:26.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.510+0000 7f60e3800640 1 -- 172.21.15.123:0/597969453 wait complete. 2024-09-19T04:46:26.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.510+0000 7f60e3800640 1 Processor -- start 2024-09-19T04:46:26.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.511+0000 7f60e3800640 1 -- start start 2024-09-19T04:46:26.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.511+0000 7f60e3800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60dc074620 0x7f60dc19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.511+0000 7f60e3800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60dc074f40 0x7f60dc19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.511+0000 7f60e3800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60dc10e3c0 0x7f60dc19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.511+0000 7f60e3800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc1141c0 con 0x7f60dc074f40 2024-09-19T04:46:26.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.511+0000 7f60e3800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc114040 con 0x7f60dc10e3c0 2024-09-19T04:46:26.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.511+0000 7f60e3800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc114340 con 0x7f60dc074620 2024-09-19T04:46:26.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.511+0000 7f60e1800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60dc10e3c0 0x7f60dc19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.511+0000 7f60e0e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60dc074620 0x7f60dc19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.511+0000 7f60dbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60dc074f40 0x7f60dc19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.511+0000 7f60e1800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60dc10e3c0 0x7f60dc19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:53364/0 (socket says 172.21.15.123:53364) 2024-09-19T04:46:26.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.511+0000 7f60e1800640 1 -- 172.21.15.123:0/3619888919 learned_addr learned my addr 172.21.15.123:0/3619888919 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:26.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.511+0000 7f60e0e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60dc074620 0x7f60dc19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42616/0 (socket says 172.21.15.123:42616) 2024-09-19T04:46:26.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.512+0000 7f60e1800640 1 -- 172.21.15.123:0/3619888919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60dc074620 msgr2=0x7f60dc19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:26.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.512+0000 7f60e1800640 1 --2- 172.21.15.123:0/3619888919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60dc074620 0x7f60dc19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.512+0000 7f60e1800640 1 -- 172.21.15.123:0/3619888919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60dc074f40 msgr2=0x7f60dc19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:26.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.512+0000 7f60e1800640 1 --2- 172.21.15.123:0/3619888919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60dc074f40 0x7f60dc19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.512+0000 7f60e1800640 1 -- 172.21.15.123:0/3619888919 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc19f640 con 0x7f60dc10e3c0 2024-09-19T04:46:26.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.512+0000 7f60e1800640 1 --2- 172.21.15.123:0/3619888919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60dc10e3c0 0x7f60dc19ef60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f60d400eb10 tx=0x7f60d400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:26.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.512+0000 7f60d9600640 1 -- 172.21.15.123:0/3619888919 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4004b20 con 0x7f60dc10e3c0 2024-09-19T04:46:26.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.512+0000 7f60e3800640 1 -- 172.21.15.123:0/3619888919 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60dc19f930 con 0x7f60dc10e3c0 2024-09-19T04:46:26.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.512+0000 7f60d9600640 1 -- 172.21.15.123:0/3619888919 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d4004cc0 con 0x7f60dc10e3c0 2024-09-19T04:46:26.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.512+0000 7f60d9600640 1 -- 172.21.15.123:0/3619888919 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4010760 con 0x7f60dc10e3c0 2024-09-19T04:46:26.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.512+0000 7f60e3800640 1 -- 172.21.15.123:0/3619888919 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60dc1a7210 con 0x7f60dc10e3c0 2024-09-19T04:46:26.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.516+0000 7f60e6403740 1 Processor -- start 2024-09-19T04:46:26.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.517+0000 7f60d9600640 1 -- 172.21.15.123:0/3619888919 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f60d4005020 con 0x7f60dc10e3c0 2024-09-19T04:46:26.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.517+0000 7f60e6403740 1 -- start start 2024-09-19T04:46:26.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.517+0000 7f60e6403740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897619330 0x561897619710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.517+0000 7f60e6403740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619c50 0x561897623fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.517+0000 7f60e6403740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897624510 0x561897626900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.517+0000 7f60e6403740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56189757f0b0 con 0x561897619c50 2024-09-19T04:46:26.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.517+0000 7f60e6403740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561897526ab0 con 0x561897619330 2024-09-19T04:46:26.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.517+0000 7f60e6403740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56189757b070 con 0x561897624510 2024-09-19T04:46:26.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.517+0000 7f60e0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619c50 0x561897623fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.517+0000 7f60dbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897619330 0x561897619710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.518+0000 7f60e0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619c50 0x561897623fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49834/0 (socket says 172.21.15.123:49834) 2024-09-19T04:46:26.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.518+0000 7f60e0e00640 1 -- 172.21.15.123:0/1124968784 learned_addr learned my addr 172.21.15.123:0/1124968784 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:26.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.518+0000 7f60dbe00640 1 --2- 172.21.15.123:0/1124968784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897624510 0x561897626900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.518+0000 7f60e0e00640 1 -- 172.21.15.123:0/1124968784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897624510 msgr2=0x561897626900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:26.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.518+0000 7f60e0e00640 1 --2- 172.21.15.123:0/1124968784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897624510 0x561897626900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.518+0000 7f60e0e00640 1 -- 172.21.15.123:0/1124968784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897619330 msgr2=0x561897619710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:26.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.518+0000 7f60e0e00640 1 --2- 172.21.15.123:0/1124968784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897619330 0x561897619710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.518+0000 7f60e0e00640 1 -- 172.21.15.123:0/1124968784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618975334a0 con 0x561897619c50 2024-09-19T04:46:26.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.518+0000 7f60dbe00640 1 --2- 172.21.15.123:0/1124968784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897619330 0x561897619710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:26.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.518+0000 7f60d9600640 1 --2- 172.21.15.123:0/3619888919 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f60c40a32c0 0x7f60c40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.518+0000 7f60d9600640 1 -- 172.21.15.123:0/3619888919 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f60d40cff10 con 0x7f60dc10e3c0 2024-09-19T04:46:26.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.518+0000 7f60e0e00640 1 --2- 172.21.15.123:0/1124968784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619c50 0x561897623fd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009900 tx=0x7f60d0005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ca243872f9c5ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:26.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.518+0000 7f60dbe00640 1 --2- 172.21.15.123:0/3619888919 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f60c40a32c0 0x7f60c40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.519+0000 7f60ce000640 1 -- 172.21.15.123:0/1124968784 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0035040 con 0x561897619c50 2024-09-19T04:46:26.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.519+0000 7f60ce000640 1 -- 172.21.15.123:0/1124968784 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d0004400 con 0x561897619c50 2024-09-19T04:46:26.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.519+0000 7f60ce000640 1 -- 172.21.15.123:0/1124968784 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d00035f0 con 0x561897619c50 2024-09-19T04:46:26.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.519+0000 7f60e6403740 1 -- 172.21.15.123:0/1124968784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619c50 msgr2=0x561897623fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:26.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.519+0000 7f60dbe00640 1 --2- 172.21.15.123:0/3619888919 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f60c40a32c0 0x7f60c40a5780 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x561897616c40 tx=0x7f60c80096c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:26.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.519+0000 7f60e6403740 1 --2- 172.21.15.123:0/1124968784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619c50 0x561897623fd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009900 tx=0x7f60d0005de0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.520+0000 7f60e6403740 1 -- 172.21.15.123:0/1124968784 shutdown_connections 2024-09-19T04:46:26.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.520+0000 7f60e6403740 1 --2- 172.21.15.123:0/1124968784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897624510 0x561897626900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.520+0000 7f60e6403740 1 --2- 172.21.15.123:0/1124968784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619c50 0x561897623fd0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.520+0000 7f60e6403740 1 --2- 172.21.15.123:0/1124968784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897619330 0x561897619710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.520+0000 7f60e6403740 1 -- 172.21.15.123:0/1124968784 >> 172.21.15.123:0/1124968784 conn(0x5618976158c0 msgr2=0x561897615ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:26.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.520+0000 7f60e6403740 1 -- 172.21.15.123:0/1124968784 shutdown_connections 2024-09-19T04:46:26.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.520+0000 7f60e6403740 1 -- 172.21.15.123:0/1124968784 wait complete. 2024-09-19T04:46:26.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.520+0000 7f60e6403740 1 -- start start 2024-09-19T04:46:26.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.520+0000 7f60e6403740 1 Processor -- start 2024-09-19T04:46:26.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.521+0000 7f60e6403740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619330 0x561897621670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.521+0000 7f60e6403740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897619c50 0x561897621bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.521+0000 7f60e6403740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897624510 0x561897622110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.521+0000 7f60e6403740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561897627060 con 0x561897619330 2024-09-19T04:46:26.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.521+0000 7f60e6403740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561897507920 con 0x561897624510 2024-09-19T04:46:26.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.521+0000 7f60e6403740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5618976271e0 con 0x561897619c50 2024-09-19T04:46:26.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.521+0000 7f60e6403740 10 client.0 _ll_register_callbacks cb 0x56189752b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:26.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.521+0000 7f60e0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619330 0x561897621670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.521+0000 7f60e0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619330 0x561897621670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49842/0 (socket says 172.21.15.123:49842) 2024-09-19T04:46:26.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.521+0000 7f60e0e00640 1 -- 172.21.15.123:0/487175509 learned_addr learned my addr 172.21.15.123:0/487175509 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:26.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.521+0000 7f60dbe00640 1 --2- 172.21.15.123:0/487175509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897619c50 0x561897621bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.521+0000 7f60e1800640 1 --2- 172.21.15.123:0/487175509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897624510 0x561897622110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.522+0000 7f60dbe00640 1 -- 172.21.15.123:0/487175509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897624510 msgr2=0x561897622110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:26.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.522+0000 7f60dbe00640 1 --2- 172.21.15.123:0/487175509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897624510 0x561897622110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.522+0000 7f60dbe00640 1 -- 172.21.15.123:0/487175509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619330 msgr2=0x561897621670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:26.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.522+0000 7f60dbe00640 1 --2- 172.21.15.123:0/487175509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619330 0x561897621670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.522+0000 7f60dbe00640 1 -- 172.21.15.123:0/487175509 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618972f02b0 con 0x561897619c50 2024-09-19T04:46:26.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.522+0000 7f60e0e00640 1 --2- 172.21.15.123:0/487175509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619330 0x561897621670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:26.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.522+0000 7f60dbe00640 1 --2- 172.21.15.123:0/487175509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897619c50 0x561897621bb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009730 tx=0x7f60c8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89dd50aa9a42e9be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:26.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.522+0000 7f60c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:46:26.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.522+0000 7f60c2a00640 1 -- 172.21.15.123:0/487175509 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8004480 con 0x561897619c50 2024-09-19T04:46:26.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.523+0000 7f60e6403740 1 -- 172.21.15.123:0/487175509 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60d0004400 con 0x561897619c50 2024-09-19T04:46:26.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.523+0000 7f60c2a00640 1 -- 172.21.15.123:0/487175509 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f60c801a070 con 0x561897619c50 2024-09-19T04:46:26.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.523+0000 7f60c2a00640 1 -- 172.21.15.123:0/487175509 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f60c8005020 con 0x561897619c50 2024-09-19T04:46:26.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.523+0000 7f60e6403740 10 client.25048 fetch_fsmap learned FSMap version 27 2024-09-19T04:46:26.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.523+0000 7f60e6403740 1 -- 172.21.15.123:0/487175509 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561897632f20 con 0x561897619c50 2024-09-19T04:46:26.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.523+0000 7f60c2a00640 1 -- 172.21.15.123:0/487175509 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d0004400 con 0x561897619c50 2024-09-19T04:46:26.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.523+0000 7f60c2a00640 1 -- 172.21.15.123:0/487175509 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f60c8027070 con 0x561897619c50 2024-09-19T04:46:26.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.523+0000 7f60e6403740 10 client.25048 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:46:26.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.523+0000 7f60e6403740 10 client.25048 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T04:46:26.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.523+0000 7f60e6403740 20 client.25048 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:26.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.523+0000 7f60e6403740 1 --2- 172.21.15.123:0/487175509 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x561897635510 0x561897636980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.523+0000 7f60e6403740 4 client.25048 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:46:26.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.523+0000 7f60e6403740 1 -- 172.21.15.123:0/487175509 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56189758c620 con 0x561897635510 2024-09-19T04:46:26.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.524+0000 7f60e0e00640 1 --2- 172.21.15.123:0/487175509 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x561897635510 0x561897636980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.524+0000 7f60e0e00640 1 --2- 172.21.15.123:0/487175509 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x561897635510 0x561897636980 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=72841d20351988f5 server_cookie=81fb48292893f35 in_seq=0 out_seq=0 2024-09-19T04:46:26.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.524+0000 7f60c2a00640 10 client.25048 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:46:26.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.525+0000 7f60c2a00640 1 -- 172.21.15.123:0/487175509 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56189758c620 con 0x561897635510 2024-09-19T04:46:26.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.525+0000 7f60c2a00640 10 client.25048 handle_command_reply: tid=0 2024-09-19T04:46:26.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.525+0000 7f60e6403740 1 client.25048 shutdown 2024-09-19T04:46:26.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.525+0000 7f60e6403740 1 -- 172.21.15.123:0/487175509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897619c50 msgr2=0x561897621bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:26.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.525+0000 7f60e6403740 1 --2- 172.21.15.123:0/487175509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897619c50 0x561897621bb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009730 tx=0x7f60c8004730 comp rx=0 tx=0).stop 2024-09-19T04:46:26.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.526+0000 7f60e6403740 1 -- 172.21.15.123:0/487175509 shutdown_connections 2024-09-19T04:46:26.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.526+0000 7f60e6403740 1 --2- 172.21.15.123:0/487175509 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x561897635510 0x561897636980 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.526+0000 7f60e6403740 1 --2- 172.21.15.123:0/487175509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897624510 0x561897622110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.526+0000 7f60e6403740 1 --2- 172.21.15.123:0/487175509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897619c50 0x561897621bb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.526+0000 7f60e6403740 1 --2- 172.21.15.123:0/487175509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619330 0x561897621670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.526+0000 7f60e6403740 1 -- 172.21.15.123:0/487175509 >> 172.21.15.123:0/487175509 conn(0x5618976158c0 msgr2=0x56189761b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:26.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.526+0000 7f60c2a00640 0 client.25048 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:46:26.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.526+0000 7f60e6403740 1 -- 172.21.15.123:0/487175509 shutdown_connections 2024-09-19T04:46:26.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.526+0000 7f60e6403740 1 -- 172.21.15.123:0/487175509 wait complete. 2024-09-19T04:46:26.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.526+0000 7f60e6403740 20 client.25048 trim_cache size 0 max 16384 2024-09-19T04:46:26.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.544+0000 7f60e6403740 1 Processor -- start 2024-09-19T04:46:26.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.544+0000 7f60e6403740 1 -- start start 2024-09-19T04:46:26.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.545+0000 7f60e6403740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897619330 0x561897635510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.545+0000 7f60e6403740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897619c50 0x56189761bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.545+0000 7f60e6403740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897624510 0x561897621f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.545+0000 7f60e6403740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561897634cc0 con 0x561897624510 2024-09-19T04:46:26.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.545+0000 7f60e6403740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561897635390 con 0x561897619330 2024-09-19T04:46:26.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.545+0000 7f60e6403740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561897635a60 con 0x561897619c50 2024-09-19T04:46:26.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.545+0000 7f60e1800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897624510 0x561897621f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.545+0000 7f60dbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897619c50 0x56189761bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.545+0000 7f60e1800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897624510 0x561897621f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49848/0 (socket says 172.21.15.123:49848) 2024-09-19T04:46:26.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.545+0000 7f60e1800640 1 -- 172.21.15.123:0/2002020982 learned_addr learned my addr 172.21.15.123:0/2002020982 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:26.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.545+0000 7f60e0e00640 1 --2- 172.21.15.123:0/2002020982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897619330 0x561897635510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.546+0000 7f60e1800640 1 -- 172.21.15.123:0/2002020982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897619c50 msgr2=0x56189761bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:26.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.546+0000 7f60e1800640 1 --2- 172.21.15.123:0/2002020982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897619c50 0x56189761bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.546+0000 7f60e1800640 1 -- 172.21.15.123:0/2002020982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897619330 msgr2=0x561897635510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:26.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.546+0000 7f60e1800640 1 --2- 172.21.15.123:0/2002020982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897619330 0x561897635510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.546+0000 7f60e1800640 1 -- 172.21.15.123:0/2002020982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561897623fe0 con 0x561897624510 2024-09-19T04:46:26.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.546+0000 7f60e1800640 1 --2- 172.21.15.123:0/2002020982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897624510 0x561897621f40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f60d40cd9c0 tx=0x7f60d4098840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8dee51b5fca3c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:26.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.546+0000 7f60cd600640 1 -- 172.21.15.123:0/2002020982 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d40b4020 con 0x561897624510 2024-09-19T04:46:26.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.546+0000 7f60cd600640 1 -- 172.21.15.123:0/2002020982 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d409cbe0 con 0x561897624510 2024-09-19T04:46:26.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.546+0000 7f60cd600640 1 -- 172.21.15.123:0/2002020982 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d40b5220 con 0x561897624510 2024-09-19T04:46:26.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.547+0000 7f60e6403740 1 -- 172.21.15.123:0/2002020982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897624510 msgr2=0x561897621f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:26.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.547+0000 7f60e6403740 1 --2- 172.21.15.123:0/2002020982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897624510 0x561897621f40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f60d40cd9c0 tx=0x7f60d4098840 comp rx=0 tx=0).stop 2024-09-19T04:46:26.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.547+0000 7f60e6403740 1 -- 172.21.15.123:0/2002020982 shutdown_connections 2024-09-19T04:46:26.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.547+0000 7f60e6403740 1 --2- 172.21.15.123:0/2002020982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897624510 0x561897621f40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.547+0000 7f60e6403740 1 --2- 172.21.15.123:0/2002020982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897619c50 0x56189761bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.547+0000 7f60e6403740 1 --2- 172.21.15.123:0/2002020982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897619330 0x561897635510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.547+0000 7f60e6403740 1 -- 172.21.15.123:0/2002020982 >> 172.21.15.123:0/2002020982 conn(0x5618976158c0 msgr2=0x56189761c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:26.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.547+0000 7f60e6403740 1 -- 172.21.15.123:0/2002020982 shutdown_connections 2024-09-19T04:46:26.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.547+0000 7f60e6403740 1 -- 172.21.15.123:0/2002020982 wait complete. 2024-09-19T04:46:26.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.547+0000 7f60e6403740 1 -- start start 2024-09-19T04:46:26.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.547+0000 7f60e6403740 1 Processor -- start 2024-09-19T04:46:26.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.548+0000 7f60e6403740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619330 0x56189761d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.548+0000 7f60e6403740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897619c50 0x56189761de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.548+0000 7f60e6403740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897624510 0x56189761e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.548+0000 7f60e6403740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561897636800 con 0x561897619330 2024-09-19T04:46:26.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.548+0000 7f60e6403740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561897636130 con 0x561897624510 2024-09-19T04:46:26.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.548+0000 7f60e6403740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561897636ed0 con 0x561897619c50 2024-09-19T04:46:26.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.548+0000 7f60e6403740 10 client.0 _ll_register_callbacks cb 0x7f60c8011930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:26.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.548+0000 7f60e0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619330 0x56189761d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.548+0000 7f60e0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619330 0x56189761d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49862/0 (socket says 172.21.15.123:49862) 2024-09-19T04:46:26.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.548+0000 7f60e0e00640 1 -- 172.21.15.123:0/1313604761 learned_addr learned my addr 172.21.15.123:0/1313604761 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:26.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.548+0000 7f60dbe00640 1 --2- 172.21.15.123:0/1313604761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897619c50 0x56189761de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.548+0000 7f60e1800640 1 --2- 172.21.15.123:0/1313604761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897624510 0x56189761e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.549+0000 7f60e0e00640 1 -- 172.21.15.123:0/1313604761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897619c50 msgr2=0x56189761de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:26.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.549+0000 7f60e0e00640 1 --2- 172.21.15.123:0/1313604761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897619c50 0x56189761de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.549+0000 7f60e0e00640 1 -- 172.21.15.123:0/1313604761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897624510 msgr2=0x56189761e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:26.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.549+0000 7f60e0e00640 1 --2- 172.21.15.123:0/1313604761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897624510 0x56189761e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.549+0000 7f60e0e00640 1 -- 172.21.15.123:0/1313604761 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56189763dc50 con 0x561897619330 2024-09-19T04:46:26.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.549+0000 7f60e0e00640 1 --2- 172.21.15.123:0/1313604761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619330 0x56189761d920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f60d0004ad0 tx=0x7f60d002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71f92d81747c377 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:26.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.549+0000 7f60c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:46:26.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.549+0000 7f60c2a00640 1 -- 172.21.15.123:0/1313604761 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0035040 con 0x561897619330 2024-09-19T04:46:26.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.549+0000 7f60e6403740 1 -- 172.21.15.123:0/1313604761 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60d409cbe0 con 0x561897619330 2024-09-19T04:46:26.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.549+0000 7f60c2a00640 1 -- 172.21.15.123:0/1313604761 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f60d0004400 con 0x561897619330 2024-09-19T04:46:26.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.549+0000 7f60c2a00640 1 -- 172.21.15.123:0/1313604761 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f60d00315b0 con 0x561897619330 2024-09-19T04:46:26.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.549+0000 7f60e6403740 10 client.15021 fetch_fsmap learned FSMap version 27 2024-09-19T04:46:26.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.549+0000 7f60e6403740 1 -- 172.21.15.123:0/1313604761 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56189763ef10 con 0x561897619330 2024-09-19T04:46:26.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.550+0000 7f60c2a00640 1 -- 172.21.15.123:0/1313604761 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d409cbe0 con 0x561897619330 2024-09-19T04:46:26.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.550+0000 7f60c2a00640 1 -- 172.21.15.123:0/1313604761 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f60d0003a30 con 0x561897619330 2024-09-19T04:46:26.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.550+0000 7f60e6403740 10 client.15021 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:46:26.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.550+0000 7f60e6403740 10 client.15021 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T04:46:26.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.550+0000 7f60e6403740 20 client.15021 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:26.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.550+0000 7f60e6403740 1 --2- 172.21.15.123:0/1313604761 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x5618976a3780 0x5618976a2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:26.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.550+0000 7f60e6403740 4 client.15021 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:46:26.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.550+0000 7f60e6403740 1 -- 172.21.15.123:0/1313604761 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "status"}) -- 0x561897586b30 con 0x5618976a3780 2024-09-19T04:46:26.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.550+0000 7f60dbe00640 1 --2- 172.21.15.123:0/1313604761 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x5618976a3780 0x5618976a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:26.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.551+0000 7f60dbe00640 1 --2- 172.21.15.123:0/1313604761 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x5618976a3780 0x5618976a2490 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5d8caecb881e0fe1 server_cookie=dff076b2c8435ef0 in_seq=0 out_seq=0 2024-09-19T04:46:26.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.551+0000 7f60c2a00640 10 client.15021 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:46:26.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.551+0000 7f60c2a00640 1 -- 172.21.15.123:0/1313604761 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561897586b30 con 0x5618976a3780 2024-09-19T04:46:26.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.551+0000 7f60c2a00640 10 client.15021 handle_command_reply: tid=0 2024-09-19T04:46:26.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.551+0000 7f60e6403740 1 client.15021 shutdown 2024-09-19T04:46:26.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.551+0000 7f60e6403740 1 -- 172.21.15.123:0/1313604761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619330 msgr2=0x56189761d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:26.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.551+0000 7f60e6403740 1 --2- 172.21.15.123:0/1313604761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619330 0x56189761d920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f60d0004ad0 tx=0x7f60d002a480 comp rx=0 tx=0).stop 2024-09-19T04:46:26.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.551+0000 7f60e6403740 1 -- 172.21.15.123:0/1313604761 shutdown_connections 2024-09-19T04:46:26.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.551+0000 7f60e6403740 1 --2- 172.21.15.123:0/1313604761 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x5618976a3780 0x5618976a2490 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.551+0000 7f60e6403740 1 --2- 172.21.15.123:0/1313604761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561897624510 0x56189761e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.551+0000 7f60e6403740 1 --2- 172.21.15.123:0/1313604761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561897619c50 0x56189761de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.551+0000 7f60e6403740 1 --2- 172.21.15.123:0/1313604761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561897619330 0x56189761d920 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.551+0000 7f60e6403740 1 -- 172.21.15.123:0/1313604761 >> 172.21.15.123:0/1313604761 conn(0x5618976158c0 msgr2=0x56189761ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:26.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.551+0000 7f60c2a00640 0 client.15021 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:46:26.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.552+0000 7f60e6403740 1 -- 172.21.15.123:0/1313604761 shutdown_connections 2024-09-19T04:46:26.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.552+0000 7f60e6403740 1 -- 172.21.15.123:0/1313604761 wait complete. 2024-09-19T04:46:26.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.552+0000 7f60e6403740 20 client.15021 trim_cache size 0 max 16384 2024-09-19T04:46:26.557 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:46:26.557 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "911fa052-7640-11ef-bceb-c7b262605968", 2024-09-19T04:46:26.557 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 4, 2024-09-19T04:46:26.557 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24499, 2024-09-19T04:46:26.557 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-09-19T04:46:26.557 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-09-19T04:46:26.557 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-09-19T04:46:26.557 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 102.724138098, 2024-09-19T04:46:26.557 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 27, 2024-09-19T04:46:26.557 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 86, 2024-09-19T04:46:26.557 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T04:46:26.558 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 145.38961036500001 2024-09-19T04:46:26.558 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:46:26.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.552+0000 7f60e3800640 1 -- 172.21.15.123:0/3619888919 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f60c40a32c0 msgr2=0x7f60c40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:26.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.552+0000 7f60e3800640 1 --2- 172.21.15.123:0/3619888919 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f60c40a32c0 0x7f60c40a5780 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x561897616c40 tx=0x7f60c80096c0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.552+0000 7f60e3800640 1 -- 172.21.15.123:0/3619888919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60dc10e3c0 msgr2=0x7f60dc19ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:26.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.552+0000 7f60e3800640 1 --2- 172.21.15.123:0/3619888919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60dc10e3c0 0x7f60dc19ef60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f60d400eb10 tx=0x7f60d400c2d0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.552+0000 7f60e3800640 1 -- 172.21.15.123:0/3619888919 shutdown_connections 2024-09-19T04:46:26.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.552+0000 7f60e3800640 1 --2- 172.21.15.123:0/3619888919 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f60c40a32c0 0x7f60c40a5780 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.552+0000 7f60e3800640 1 --2- 172.21.15.123:0/3619888919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60dc10e3c0 0x7f60dc19ef60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.552+0000 7f60e3800640 1 --2- 172.21.15.123:0/3619888919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60dc074f40 0x7f60dc19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.552+0000 7f60e3800640 1 --2- 172.21.15.123:0/3619888919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60dc074620 0x7f60dc19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:26.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.552+0000 7f60e3800640 1 -- 172.21.15.123:0/3619888919 >> 172.21.15.123:0/3619888919 conn(0x7f60dc0fd8c0 msgr2=0x7f60dc0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:26.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.553+0000 7f60e3800640 1 -- 172.21.15.123:0/3619888919 shutdown_connections 2024-09-19T04:46:26.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:26.553+0000 7f60e3800640 1 -- 172.21.15.123:0/3619888919 wait complete. 2024-09-19T04:46:26.910 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:26 smithi123 ceph-mon[30511]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-19T04:46:26.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:26 smithi193 ceph-mon[35592]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-19T04:46:27.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:26 smithi183 ceph-mon[35333]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-19T04:46:27.291 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T04:46:27.661 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:28.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.191+0000 7f8bf9200640 1 -- 172.21.15.123:0/3553802404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bf4101de0 msgr2=0x7f8bf41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:28.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.191+0000 7f8bf9200640 1 --2- 172.21.15.123:0/3553802404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bf4101de0 0x7f8bf41021c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009900 tx=0x7f8be402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.192+0000 7f8bf9200640 1 -- 172.21.15.123:0/3553802404 shutdown_connections 2024-09-19T04:46:28.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.192+0000 7f8bf9200640 1 --2- 172.21.15.123:0/3553802404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf4106f10 0x7f8bf410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.192+0000 7f8bf9200640 1 --2- 172.21.15.123:0/3553802404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bf4102790 0x7f8bf41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.192+0000 7f8bf9200640 1 --2- 172.21.15.123:0/3553802404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bf4101de0 0x7f8bf41021c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.192+0000 7f8bf9200640 1 -- 172.21.15.123:0/3553802404 >> 172.21.15.123:0/3553802404 conn(0x7f8bf40fd8c0 msgr2=0x7f8bf40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:28.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.192+0000 7f8bf9200640 1 -- 172.21.15.123:0/3553802404 shutdown_connections 2024-09-19T04:46:28.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.192+0000 7f8bf9200640 1 -- 172.21.15.123:0/3553802404 wait complete. 2024-09-19T04:46:28.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.193+0000 7f8bf9200640 1 Processor -- start 2024-09-19T04:46:28.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.193+0000 7f8bf9200640 1 -- start start 2024-09-19T04:46:28.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.193+0000 7f8bf9200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bf4101de0 0x7f8bf4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.193+0000 7f8bf9200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf4102790 0x7f8bf4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.193+0000 7f8bf9200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bf4106f10 0x7f8bf419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.193+0000 7f8bf9200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4111fc0 con 0x7f8bf4106f10 2024-09-19T04:46:28.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.193+0000 7f8bf9200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4111e40 con 0x7f8bf4101de0 2024-09-19T04:46:28.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.193+0000 7f8bf9200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4112140 con 0x7f8bf4102790 2024-09-19T04:46:28.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.193+0000 7f8bf1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bf4101de0 0x7f8bf4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:28.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.193+0000 7f8bf2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bf4106f10 0x7f8bf419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:28.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.193+0000 7f8bf1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bf4101de0 0x7f8bf4198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:53416/0 (socket says 172.21.15.123:53416) 2024-09-19T04:46:28.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.193+0000 7f8bf1e00640 1 -- 172.21.15.123:0/2057872826 learned_addr learned my addr 172.21.15.123:0/2057872826 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:28.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.193+0000 7f8be9400640 1 --2- 172.21.15.123:0/2057872826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf4102790 0x7f8bf4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:28.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.194+0000 7f8bf1e00640 1 -- 172.21.15.123:0/2057872826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf4102790 msgr2=0x7f8bf4198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:28.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.194+0000 7f8bf1e00640 1 --2- 172.21.15.123:0/2057872826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf4102790 0x7f8bf4198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.194+0000 7f8bf1e00640 1 -- 172.21.15.123:0/2057872826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bf4106f10 msgr2=0x7f8bf419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:28.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.194+0000 7f8bf1e00640 1 --2- 172.21.15.123:0/2057872826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bf4106f10 0x7f8bf419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.194+0000 7f8bf1e00640 1 -- 172.21.15.123:0/2057872826 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf419d3f0 con 0x7f8bf4101de0 2024-09-19T04:46:28.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.194+0000 7f8bf2800640 1 --2- 172.21.15.123:0/2057872826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bf4106f10 0x7f8bf419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:28.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.194+0000 7f8bf1e00640 1 --2- 172.21.15.123:0/2057872826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bf4101de0 0x7f8bf4198440 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8be4004770 tx=0x7f8be4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:28.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.195+0000 7f8beb400640 1 -- 172.21.15.123:0/2057872826 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be403f070 con 0x7f8bf4101de0 2024-09-19T04:46:28.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.195+0000 7f8bf9200640 1 -- 172.21.15.123:0/2057872826 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf419d680 con 0x7f8bf4101de0 2024-09-19T04:46:28.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.195+0000 7f8beb400640 1 -- 172.21.15.123:0/2057872826 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8be4031ca0 con 0x7f8bf4101de0 2024-09-19T04:46:28.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.195+0000 7f8beb400640 1 -- 172.21.15.123:0/2057872826 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be4035030 con 0x7f8bf4101de0 2024-09-19T04:46:28.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.195+0000 7f8bf9200640 1 -- 172.21.15.123:0/2057872826 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf41a4fc0 con 0x7f8bf4101de0 2024-09-19T04:46:28.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.198+0000 7f8bfbc84740 1 Processor -- start 2024-09-19T04:46:28.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.198+0000 7f8bfbc84740 1 -- start start 2024-09-19T04:46:28.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.198+0000 7f8beb400640 1 -- 172.21.15.123:0/2057872826 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f8be4042070 con 0x7f8bf4101de0 2024-09-19T04:46:28.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.198+0000 7f8bfbc84740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbc4330 0x55ec5dbc4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.198+0000 7f8bfbc84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbcefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.199+0000 7f8bfbc84740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbcf510 0x55ec5dbd1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.199+0000 7f8bfbc84740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ec5db2a0b0 con 0x55ec5dbcf510 2024-09-19T04:46:28.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.199+0000 7f8bfbc84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ec5dad1ab0 con 0x55ec5dbc4c50 2024-09-19T04:46:28.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.199+0000 7f8bfbc84740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ec5db26070 con 0x55ec5dbc4330 2024-09-19T04:46:28.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.199+0000 7f8bf2800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbc4330 0x55ec5dbc4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:28.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.199+0000 7f8bf2800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbc4330 0x55ec5dbc4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42714/0 (socket says 172.21.15.123:42714) 2024-09-19T04:46:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.199+0000 7f8bf2800640 1 -- 172.21.15.123:0/445503075 learned_addr learned my addr 172.21.15.123:0/445503075 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.199+0000 7f8bf1e00640 1 --2- 172.21.15.123:0/445503075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbcefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.199+0000 7f8be9400640 1 --2- 172.21.15.123:0/445503075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbcf510 0x55ec5dbd1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.199+0000 7f8bf2800640 1 -- 172.21.15.123:0/445503075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 msgr2=0x55ec5dbcefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.199+0000 7f8bf2800640 1 --2- 172.21.15.123:0/445503075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbcefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.199+0000 7f8bf2800640 1 -- 172.21.15.123:0/445503075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbcf510 msgr2=0x55ec5dbd1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.199+0000 7f8bf2800640 1 --2- 172.21.15.123:0/445503075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbcf510 0x55ec5dbd1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.199+0000 7f8bf2800640 1 -- 172.21.15.123:0/445503075 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec5dade4a0 con 0x55ec5dbc4330 2024-09-19T04:46:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.200+0000 7f8bf2800640 1 --2- 172.21.15.123:0/445503075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbc4330 0x55ec5dbc4710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8bec010710 tx=0x7f8bec010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=761348077f706a07 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.200+0000 7f8be3400640 1 -- 172.21.15.123:0/445503075 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec00e8d0 con 0x55ec5dbc4330 2024-09-19T04:46:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.200+0000 7f8be3400640 1 -- 172.21.15.123:0/445503075 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bec012450 con 0x55ec5dbc4330 2024-09-19T04:46:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.200+0000 7f8be3400640 1 -- 172.21.15.123:0/445503075 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec01c630 con 0x55ec5dbc4330 2024-09-19T04:46:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.200+0000 7f8beb400640 1 --2- 172.21.15.123:0/2057872826 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8bd80a3240 0x7f8bd80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.200+0000 7f8beb400640 1 -- 172.21.15.123:0/2057872826 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f8be40effe0 con 0x7f8bf4101de0 2024-09-19T04:46:28.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.200+0000 7f8bf1e00640 1 --2- 172.21.15.123:0/2057872826 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8bd80a3240 0x7f8bd80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:28.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.201+0000 7f8bfbc84740 1 -- 172.21.15.123:0/445503075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbc4330 msgr2=0x55ec5dbc4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:28.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.201+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/445503075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbc4330 0x55ec5dbc4710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8bec010710 tx=0x7f8bec010bd0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.201+0000 7f8bfbc84740 1 -- 172.21.15.123:0/445503075 shutdown_connections 2024-09-19T04:46:28.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.201+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/445503075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbcf510 0x55ec5dbd1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.201+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/445503075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbcefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.201+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/445503075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbc4330 0x55ec5dbc4710 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.201+0000 7f8bfbc84740 1 -- 172.21.15.123:0/445503075 >> 172.21.15.123:0/445503075 conn(0x55ec5dbc08c0 msgr2=0x55ec5dbc0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:28.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.201+0000 7f8bfbc84740 1 -- 172.21.15.123:0/445503075 shutdown_connections 2024-09-19T04:46:28.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.201+0000 7f8bfbc84740 1 -- 172.21.15.123:0/445503075 wait complete. 2024-09-19T04:46:28.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.201+0000 7f8bf1e00640 1 --2- 172.21.15.123:0/2057872826 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8bd80a3240 0x7f8bd80a5700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8be40b9030 tx=0x7f8be403b890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:28.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.201+0000 7f8bfbc84740 1 -- start start 2024-09-19T04:46:28.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.202+0000 7f8bfbc84740 1 Processor -- start 2024-09-19T04:46:28.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.202+0000 7f8bfbc84740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbc4330 0x55ec5dbd3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.202+0000 7f8bfbc84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbd3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.202+0000 7f8bfbc84740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbcf510 0x55ec5dbd8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.202+0000 7f8bfbc84740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ec5dbd2060 con 0x55ec5dbcf510 2024-09-19T04:46:28.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.202+0000 7f8bfbc84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ec5dab2920 con 0x55ec5dbc4c50 2024-09-19T04:46:28.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.202+0000 7f8bfbc84740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ec5dbd21e0 con 0x55ec5dbc4330 2024-09-19T04:46:28.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.202+0000 7f8bfbc84740 10 client.0 _ll_register_callbacks cb 0x55ec5dad68a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:28.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.202+0000 7f8bf2800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbc4330 0x55ec5dbd3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:28.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.202+0000 7f8be9400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbd3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:28.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.202+0000 7f8be9400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbd3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:53438/0 (socket says 172.21.15.123:53438) 2024-09-19T04:46:28.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.202+0000 7f8be9400640 1 -- 172.21.15.123:0/4178527610 learned_addr learned my addr 172.21.15.123:0/4178527610 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:28.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.202+0000 7f8bf2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbcf510 0x55ec5dbd8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:28.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.203+0000 7f8be9400640 1 -- 172.21.15.123:0/4178527610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbc4330 msgr2=0x55ec5dbd3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:28.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.203+0000 7f8be9400640 1 --2- 172.21.15.123:0/4178527610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbc4330 0x55ec5dbd3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.203+0000 7f8be9400640 1 -- 172.21.15.123:0/4178527610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbcf510 msgr2=0x55ec5dbd8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:28.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.203+0000 7f8be9400640 1 --2- 172.21.15.123:0/4178527610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbcf510 0x55ec5dbd8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.203+0000 7f8be9400640 1 -- 172.21.15.123:0/4178527610 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec5d89b2b0 con 0x55ec5dbc4c50 2024-09-19T04:46:28.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.203+0000 7f8be9400640 1 --2- 172.21.15.123:0/4178527610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbd3f90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc012700 tx=0x7f8bdc012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d11efcff4e6f475a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:28.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.203+0000 7f8bd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:46:28.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.203+0000 7f8bd7e00640 1 -- 172.21.15.123:0/4178527610 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc004050 con 0x55ec5dbc4c50 2024-09-19T04:46:28.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.203+0000 7f8bfbc84740 1 -- 172.21.15.123:0/4178527610 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bec012450 con 0x55ec5dbc4c50 2024-09-19T04:46:28.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.203+0000 7f8bd7e00640 1 -- 172.21.15.123:0/4178527610 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f8bdc0041f0 con 0x55ec5dbc4c50 2024-09-19T04:46:28.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.204+0000 7f8bd7e00640 1 -- 172.21.15.123:0/4178527610 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8bdc0043f0 con 0x55ec5dbc4c50 2024-09-19T04:46:28.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.204+0000 7f8bfbc84740 10 client.25055 fetch_fsmap learned FSMap version 27 2024-09-19T04:46:28.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.204+0000 7f8bfbc84740 1 -- 172.21.15.123:0/4178527610 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ec5dbe22b0 con 0x55ec5dbc4c50 2024-09-19T04:46:28.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.204+0000 7f8bd7e00640 1 -- 172.21.15.123:0/4178527610 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bec012450 con 0x55ec5dbc4c50 2024-09-19T04:46:28.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.204+0000 7f8bd7e00640 1 -- 172.21.15.123:0/4178527610 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8bdc003b30 con 0x55ec5dbc4c50 2024-09-19T04:46:28.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.204+0000 7f8bfbc84740 10 client.25055 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:46:28.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.204+0000 7f8bfbc84740 10 client.25055 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T04:46:28.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.204+0000 7f8bfbc84740 20 client.25055 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:28.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.204+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/4178527610 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55ec5dbe48b0 0x55ec5dbe5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.204+0000 7f8bfbc84740 4 client.25055 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:46:28.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.204+0000 7f8bfbc84740 1 -- 172.21.15.123:0/4178527610 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec5db37620 con 0x55ec5dbe48b0 2024-09-19T04:46:28.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.205+0000 7f8bf2800640 1 --2- 172.21.15.123:0/4178527610 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55ec5dbe48b0 0x55ec5dbe5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:28.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.205+0000 7f8bf2800640 1 --2- 172.21.15.123:0/4178527610 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55ec5dbe48b0 0x55ec5dbe5d20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fef8f9311b80347c server_cookie=6e7383a289c1ee03 in_seq=0 out_seq=0 2024-09-19T04:46:28.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.205+0000 7f8bd7e00640 10 client.25055 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:46:28.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.206+0000 7f8bd7e00640 1 -- 172.21.15.123:0/4178527610 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec5db37620 con 0x55ec5dbe48b0 2024-09-19T04:46:28.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.206+0000 7f8bd7e00640 10 client.25055 handle_command_reply: tid=0 2024-09-19T04:46:28.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.206+0000 7f8bfbc84740 1 client.25055 shutdown 2024-09-19T04:46:28.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.206+0000 7f8bfbc84740 1 -- 172.21.15.123:0/4178527610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 msgr2=0x55ec5dbd3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:28.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.206+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/4178527610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbd3f90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc012700 tx=0x7f8bdc012bc0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.206+0000 7f8bfbc84740 1 -- 172.21.15.123:0/4178527610 shutdown_connections 2024-09-19T04:46:28.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.206+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/4178527610 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55ec5dbe48b0 0x55ec5dbe5d20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.206+0000 7f8bd7e00640 0 client.25055 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:46:28.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.206+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/4178527610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbcf510 0x55ec5dbd8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.206+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/4178527610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbd3f90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.206+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/4178527610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbc4330 0x55ec5dbd3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.206+0000 7f8bfbc84740 1 -- 172.21.15.123:0/4178527610 >> 172.21.15.123:0/4178527610 conn(0x55ec5dbc08c0 msgr2=0x55ec5dbc65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:28.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.207+0000 7f8bfbc84740 1 -- 172.21.15.123:0/4178527610 shutdown_connections 2024-09-19T04:46:28.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.207+0000 7f8bfbc84740 1 -- 172.21.15.123:0/4178527610 wait complete. 2024-09-19T04:46:28.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.207+0000 7f8bfbc84740 20 client.25055 trim_cache size 0 max 16384 2024-09-19T04:46:28.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.225+0000 7f8bfbc84740 1 Processor -- start 2024-09-19T04:46:28.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.225+0000 7f8bfbc84740 1 -- start start 2024-09-19T04:46:28.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.226+0000 7f8bfbc84740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbc4330 0x55ec5dbd80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.226+0000 7f8bfbc84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbd23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.226+0000 7f8bfbc84740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbcf510 0x55ec5dbd2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.226+0000 7f8bfbc84740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ec5dbe4060 con 0x55ec5dbc4330 2024-09-19T04:46:28.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.226+0000 7f8bfbc84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ec5dbe4730 con 0x55ec5dbc4c50 2024-09-19T04:46:28.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.226+0000 7f8bfbc84740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ec5dbe4e00 con 0x55ec5dbcf510 2024-09-19T04:46:28.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.226+0000 7f8bf2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbc4330 0x55ec5dbd80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:28.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.226+0000 7f8bf2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbc4330 0x55ec5dbd80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49914/0 (socket says 172.21.15.123:49914) 2024-09-19T04:46:28.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.226+0000 7f8bf2800640 1 -- 172.21.15.123:0/2941482293 learned_addr learned my addr 172.21.15.123:0/2941482293 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:28.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.226+0000 7f8be9400640 1 --2- 172.21.15.123:0/2941482293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbd23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:28.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.226+0000 7f8bf2800640 1 -- 172.21.15.123:0/2941482293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbcf510 msgr2=0x55ec5dbd2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:28.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.226+0000 7f8bf2800640 1 --2- 172.21.15.123:0/2941482293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbcf510 0x55ec5dbd2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.226+0000 7f8bf2800640 1 -- 172.21.15.123:0/2941482293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 msgr2=0x55ec5dbd23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:28.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.226+0000 7f8bf2800640 1 --2- 172.21.15.123:0/2941482293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbd23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.226+0000 7f8bf2800640 1 -- 172.21.15.123:0/2941482293 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec5dbe6050 con 0x55ec5dbc4330 2024-09-19T04:46:28.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.226+0000 7f8be9400640 1 --2- 172.21.15.123:0/2941482293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbd23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:46:28.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.226+0000 7f8bf2800640 1 --2- 172.21.15.123:0/2941482293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbc4330 0x55ec5dbd80f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8bec002230 tx=0x7f8bec009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9a6f6395a70dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:28.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.227+0000 7f8be2a00640 1 -- 172.21.15.123:0/2941482293 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec031070 con 0x55ec5dbc4330 2024-09-19T04:46:28.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.227+0000 7f8be2a00640 1 -- 172.21.15.123:0/2941482293 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bec012450 con 0x55ec5dbc4330 2024-09-19T04:46:28.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.227+0000 7f8be2a00640 1 -- 172.21.15.123:0/2941482293 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec026040 con 0x55ec5dbc4330 2024-09-19T04:46:28.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.227+0000 7f8bfbc84740 1 -- 172.21.15.123:0/2941482293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbc4330 msgr2=0x55ec5dbd80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:28.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.227+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/2941482293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbc4330 0x55ec5dbd80f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8bec002230 tx=0x7f8bec009600 comp rx=0 tx=0).stop 2024-09-19T04:46:28.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.227+0000 7f8bfbc84740 1 -- 172.21.15.123:0/2941482293 shutdown_connections 2024-09-19T04:46:28.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.227+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/2941482293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbcf510 0x55ec5dbd2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.228+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/2941482293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbd23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.228+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/2941482293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbc4330 0x55ec5dbd80f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.228+0000 7f8bfbc84740 1 -- 172.21.15.123:0/2941482293 >> 172.21.15.123:0/2941482293 conn(0x55ec5dbc08c0 msgr2=0x55ec5dbd3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:28.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.228+0000 7f8bfbc84740 1 -- 172.21.15.123:0/2941482293 shutdown_connections 2024-09-19T04:46:28.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.228+0000 7f8bfbc84740 1 -- 172.21.15.123:0/2941482293 wait complete. 2024-09-19T04:46:28.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.228+0000 7f8bfbc84740 1 -- start start 2024-09-19T04:46:28.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.228+0000 7f8bfbc84740 1 Processor -- start 2024-09-19T04:46:28.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.229+0000 7f8bfbc84740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbc4330 0x55ec5dbe0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.229+0000 7f8bfbc84740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbdae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.229+0000 7f8bfbc84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbcf510 0x55ec5dbdb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.229+0000 7f8bfbc84740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ec5dbe5ba0 con 0x55ec5dbc4330 2024-09-19T04:46:28.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.229+0000 7f8bfbc84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ec5dbe54d0 con 0x55ec5dbcf510 2024-09-19T04:46:28.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.229+0000 7f8bfbc84740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ec5dbe6270 con 0x55ec5dbc4c50 2024-09-19T04:46:28.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.229+0000 7f8bfbc84740 10 client.0 _ll_register_callbacks cb 0x7f8bdc009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:28.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.229+0000 7f8bf2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbc4330 0x55ec5dbe0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:28.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.229+0000 7f8bf2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbc4330 0x55ec5dbe0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49928/0 (socket says 172.21.15.123:49928) 2024-09-19T04:46:28.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.229+0000 7f8bf2800640 1 -- 172.21.15.123:0/3052844691 learned_addr learned my addr 172.21.15.123:0/3052844691 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:28.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.229+0000 7f8be9400640 1 --2- 172.21.15.123:0/3052844691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbdae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:28.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.229+0000 7f8bf2800640 1 -- 172.21.15.123:0/3052844691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbc4c50 msgr2=0x55ec5dbdae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:28.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.229+0000 7f8bf2800640 1 --2- 172.21.15.123:0/3052844691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbdae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.229+0000 7f8bf2800640 1 -- 172.21.15.123:0/3052844691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbcf510 msgr2=0x55ec5dbdb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:28.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.229+0000 7f8bf2800640 1 --2- 172.21.15.123:0/3052844691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbcf510 0x55ec5dbdb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.229+0000 7f8bf2800640 1 -- 172.21.15.123:0/3052844691 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec5dbdb9e0 con 0x55ec5dbc4330 2024-09-19T04:46:28.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.230+0000 7f8bf2800640 1 --2- 172.21.15.123:0/3052844691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbc4330 0x55ec5dbe0030 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009770 tx=0x7f8bec009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c277d2278738410 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:28.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.230+0000 7f8bd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:46:28.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.230+0000 7f8bd7e00640 1 -- 172.21.15.123:0/3052844691 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec03a060 con 0x55ec5dbc4330 2024-09-19T04:46:28.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.230+0000 7f8bd7e00640 1 -- 172.21.15.123:0/3052844691 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f8bec00ede0 con 0x55ec5dbc4330 2024-09-19T04:46:28.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.230+0000 7f8bfbc84740 1 -- 172.21.15.123:0/3052844691 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bec012450 con 0x55ec5dbc4330 2024-09-19T04:46:28.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.230+0000 7f8bd7e00640 1 -- 172.21.15.123:0/3052844691 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8bec031070 con 0x55ec5dbc4330 2024-09-19T04:46:28.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.230+0000 7f8bfbc84740 10 client.15039 fetch_fsmap learned FSMap version 27 2024-09-19T04:46:28.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.230+0000 7f8bfbc84740 1 -- 172.21.15.123:0/3052844691 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ec5dc4fe90 con 0x55ec5dbc4330 2024-09-19T04:46:28.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.231+0000 7f8bd7e00640 1 -- 172.21.15.123:0/3052844691 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bec012450 con 0x55ec5dbc4330 2024-09-19T04:46:28.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.231+0000 7f8bd7e00640 1 -- 172.21.15.123:0/3052844691 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8bec00f040 con 0x55ec5dbc4330 2024-09-19T04:46:28.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.231+0000 7f8bfbc84740 10 client.15039 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:46:28.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.231+0000 7f8bfbc84740 10 client.15039 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T04:46:28.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.231+0000 7f8bfbc84740 20 client.15039 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:28.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.231+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/3052844691 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55ec5dbe48b0 0x55ec5dc52f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:28.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.231+0000 7f8bfbc84740 4 client.15039 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:46:28.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.231+0000 7f8bfbc84740 1 -- 172.21.15.123:0/3052844691 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "status"}) -- 0x55ec5db31b30 con 0x55ec5dbe48b0 2024-09-19T04:46:28.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.231+0000 7f8be9400640 1 --2- 172.21.15.123:0/3052844691 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55ec5dbe48b0 0x55ec5dc52f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:28.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.232+0000 7f8be9400640 1 --2- 172.21.15.123:0/3052844691 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55ec5dbe48b0 0x55ec5dc52f10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dd907ab9fa941b86 server_cookie=4ac593e3b7cb7d8e in_seq=0 out_seq=0 2024-09-19T04:46:28.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.232+0000 7f8bd7e00640 10 client.15039 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:46:28.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.232+0000 7f8bd7e00640 1 -- 172.21.15.123:0/3052844691 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55ec5db31b30 con 0x55ec5dbe48b0 2024-09-19T04:46:28.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.232+0000 7f8bd7e00640 10 client.15039 handle_command_reply: tid=0 2024-09-19T04:46:28.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.232+0000 7f8bfbc84740 1 client.15039 shutdown 2024-09-19T04:46:28.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.232+0000 7f8bfbc84740 1 -- 172.21.15.123:0/3052844691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbc4330 msgr2=0x55ec5dbe0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:28.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.232+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/3052844691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbc4330 0x55ec5dbe0030 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009770 tx=0x7f8bec009fb0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.232+0000 7f8bfbc84740 1 -- 172.21.15.123:0/3052844691 shutdown_connections 2024-09-19T04:46:28.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.232+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/3052844691 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55ec5dbe48b0 0x55ec5dc52f10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.232+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/3052844691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5dbcf510 0x55ec5dbdb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.232+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/3052844691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5dbc4c50 0x55ec5dbdae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.232+0000 7f8bfbc84740 1 --2- 172.21.15.123:0/3052844691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5dbc4330 0x55ec5dbe0030 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.232+0000 7f8bfbc84740 1 -- 172.21.15.123:0/3052844691 >> 172.21.15.123:0/3052844691 conn(0x55ec5dbc08c0 msgr2=0x55ec5dbd23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:28.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.232+0000 7f8bd7e00640 0 client.15039 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:46:28.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.233+0000 7f8bfbc84740 1 -- 172.21.15.123:0/3052844691 shutdown_connections 2024-09-19T04:46:28.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.233+0000 7f8bfbc84740 1 -- 172.21.15.123:0/3052844691 wait complete. 2024-09-19T04:46:28.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.233+0000 7f8bfbc84740 20 client.15039 trim_cache size 0 max 16384 2024-09-19T04:46:28.240 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:46:28.240 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "911fa052-7640-11ef-bceb-c7b262605968", 2024-09-19T04:46:28.240 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 3, 2024-09-19T04:46:28.240 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24500, 2024-09-19T04:46:28.241 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-09-19T04:46:28.241 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-09-19T04:46:28.241 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-09-19T04:46:28.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 105.412661329, 2024-09-19T04:46:28.241 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 27, 2024-09-19T04:46:28.241 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 86, 2024-09-19T04:46:28.241 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T04:46:28.241 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 141.759091705 2024-09-19T04:46:28.241 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:46:28.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.233+0000 7f8bf9200640 1 -- 172.21.15.123:0/2057872826 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8bd80a3240 msgr2=0x7f8bd80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:28.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.233+0000 7f8bf9200640 1 --2- 172.21.15.123:0/2057872826 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8bd80a3240 0x7f8bd80a5700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8be40b9030 tx=0x7f8be403b890 comp rx=0 tx=0).stop 2024-09-19T04:46:28.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.233+0000 7f8bf9200640 1 -- 172.21.15.123:0/2057872826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bf4101de0 msgr2=0x7f8bf4198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:28.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.233+0000 7f8bf9200640 1 --2- 172.21.15.123:0/2057872826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bf4101de0 0x7f8bf4198440 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8be4004770 tx=0x7f8be4003c10 comp rx=0 tx=0).stop 2024-09-19T04:46:28.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.234+0000 7f8bf9200640 1 -- 172.21.15.123:0/2057872826 shutdown_connections 2024-09-19T04:46:28.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.234+0000 7f8bf9200640 1 --2- 172.21.15.123:0/2057872826 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f8bd80a3240 0x7f8bd80a5700 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.234+0000 7f8bf9200640 1 --2- 172.21.15.123:0/2057872826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bf4106f10 0x7f8bf419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.234+0000 7f8bf9200640 1 --2- 172.21.15.123:0/2057872826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf4102790 0x7f8bf4198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.234+0000 7f8bf9200640 1 --2- 172.21.15.123:0/2057872826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bf4101de0 0x7f8bf4198440 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:28.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.234+0000 7f8bf9200640 1 -- 172.21.15.123:0/2057872826 >> 172.21.15.123:0/2057872826 conn(0x7f8bf40fd8c0 msgr2=0x7f8bf40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:28.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.234+0000 7f8bf9200640 1 -- 172.21.15.123:0/2057872826 shutdown_connections 2024-09-19T04:46:28.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:28.234+0000 7f8bf9200640 1 -- 172.21.15.123:0/2057872826 wait complete. 2024-09-19T04:46:28.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:28 smithi123 ceph-mon[30511]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T04:46:28.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:28 smithi193 ceph-mon[35592]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T04:46:29.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T04:46:29.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:28 smithi183 ceph-mon[35333]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T04:46:29.368 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:29.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.916+0000 7fe1c1800640 1 -- 172.21.15.123:0/736665705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1bc074590 msgr2=0x7fe1bc074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:29.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.916+0000 7fe1c1800640 1 --2- 172.21.15.123:0/736665705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1bc074590 0x7fe1bc074970 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009900 tx=0x7fe1ac02a480 comp rx=0 tx=0).stop 2024-09-19T04:46:29.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.916+0000 7fe1c1800640 1 -- 172.21.15.123:0/736665705 shutdown_connections 2024-09-19T04:46:29.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.916+0000 7fe1c1800640 1 --2- 172.21.15.123:0/736665705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1bc1129e0 0x7fe1bc114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.916+0000 7fe1c1800640 1 --2- 172.21.15.123:0/736665705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1bc074eb0 0x7fe1bc1124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.916+0000 7fe1c1800640 1 --2- 172.21.15.123:0/736665705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1bc074590 0x7fe1bc074970 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.916+0000 7fe1c1800640 1 -- 172.21.15.123:0/736665705 >> 172.21.15.123:0/736665705 conn(0x7fe1bc0fd710 msgr2=0x7fe1bc0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:29.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.916+0000 7fe1c1800640 1 -- 172.21.15.123:0/736665705 shutdown_connections 2024-09-19T04:46:29.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.917+0000 7fe1c1800640 1 -- 172.21.15.123:0/736665705 wait complete. 2024-09-19T04:46:29.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.917+0000 7fe1c1800640 1 Processor -- start 2024-09-19T04:46:29.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.918+0000 7fe1c1800640 1 -- start start 2024-09-19T04:46:29.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.918+0000 7fe1c1800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1bc074590 0x7fe1bc1a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.918+0000 7fe1c1800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1bc074eb0 0x7fe1bc1a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.918+0000 7fe1c1800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1bc1129e0 0x7fe1bc19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.918+0000 7fe1c1800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc118410 con 0x7fe1bc074eb0 2024-09-19T04:46:29.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.918+0000 7fe1c1800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc118290 con 0x7fe1bc074590 2024-09-19T04:46:29.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.918+0000 7fe1c1800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc118590 con 0x7fe1bc1129e0 2024-09-19T04:46:29.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.918+0000 7fe1bb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1bc074eb0 0x7fe1bc1a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.918+0000 7fe1bb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1bc074eb0 0x7fe1bc1a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49938/0 (socket says 172.21.15.123:49938) 2024-09-19T04:46:29.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.918+0000 7fe1bb400640 1 -- 172.21.15.123:0/4005799299 learned_addr learned my addr 172.21.15.123:0/4005799299 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:29.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.918+0000 7fe1bbe00640 1 --2- 172.21.15.123:0/4005799299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1bc074590 0x7fe1bc1a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.918+0000 7fe1c0e00640 1 --2- 172.21.15.123:0/4005799299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1bc1129e0 0x7fe1bc19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.919+0000 7fe1bb400640 1 -- 172.21.15.123:0/4005799299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1bc1129e0 msgr2=0x7fe1bc19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:29.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.919+0000 7fe1bb400640 1 --2- 172.21.15.123:0/4005799299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1bc1129e0 0x7fe1bc19e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.919+0000 7fe1bb400640 1 -- 172.21.15.123:0/4005799299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1bc074590 msgr2=0x7fe1bc1a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:29.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.919+0000 7fe1bb400640 1 --2- 172.21.15.123:0/4005799299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1bc074590 0x7fe1bc1a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.919+0000 7fe1bb400640 1 -- 172.21.15.123:0/4005799299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1bc19efa0 con 0x7fe1bc074eb0 2024-09-19T04:46:29.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.919+0000 7fe1bb400640 1 --2- 172.21.15.123:0/4005799299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1bc074eb0 0x7fe1bc1a4ba0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800d770 tx=0x7fe1a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:29.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.919+0000 7fe1b8c00640 1 -- 172.21.15.123:0/4005799299 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8017070 con 0x7fe1bc074eb0 2024-09-19T04:46:29.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.919+0000 7fe1b8c00640 1 -- 172.21.15.123:0/4005799299 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1a8005500 con 0x7fe1bc074eb0 2024-09-19T04:46:29.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.919+0000 7fe1c1800640 1 -- 172.21.15.123:0/4005799299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1bc19f290 con 0x7fe1bc074eb0 2024-09-19T04:46:29.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.919+0000 7fe1b8c00640 1 -- 172.21.15.123:0/4005799299 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8004e00 con 0x7fe1bc074eb0 2024-09-19T04:46:29.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.919+0000 7fe1c1800640 1 -- 172.21.15.123:0/4005799299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1bc1ab410 con 0x7fe1bc074eb0 2024-09-19T04:46:29.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.923+0000 7fe1c5a77740 1 Processor -- start 2024-09-19T04:46:29.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.923+0000 7fe1c5a77740 1 -- start start 2024-09-19T04:46:29.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.923+0000 7fe1c5a77740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd8244e0 0x559efd8248c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.923+0000 7fe1b8c00640 1 -- 172.21.15.123:0/4005799299 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fe1a8004fa0 con 0x7fe1bc074eb0 2024-09-19T04:46:29.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.923+0000 7fe1c5a77740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd824e00 0x559efd82f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.923+0000 7fe1c5a77740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd82f6c0 0x559efd831ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.923+0000 7fe1c5a77740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559efd78a0b0 con 0x559efd824e00 2024-09-19T04:46:29.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.923+0000 7fe1c5a77740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559efd731ab0 con 0x559efd8244e0 2024-09-19T04:46:29.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.923+0000 7fe1c5a77740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559efd786070 con 0x559efd82f6c0 2024-09-19T04:46:29.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.923+0000 7fe1bbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd824e00 0x559efd82f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.924+0000 7fe1bbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd824e00 0x559efd82f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49942/0 (socket says 172.21.15.123:49942) 2024-09-19T04:46:29.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.924+0000 7fe1bbe00640 1 -- 172.21.15.123:0/3260364444 learned_addr learned my addr 172.21.15.123:0/3260364444 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:29.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.924+0000 7fe1bb400640 1 --2- 172.21.15.123:0/3260364444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd82f6c0 0x559efd831ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.924+0000 7fe1c0e00640 1 --2- 172.21.15.123:0/3260364444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd8244e0 0x559efd8248c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.924+0000 7fe1bbe00640 1 -- 172.21.15.123:0/3260364444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd82f6c0 msgr2=0x559efd831ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:29.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.924+0000 7fe1bbe00640 1 --2- 172.21.15.123:0/3260364444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd82f6c0 0x559efd831ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.924+0000 7fe1bbe00640 1 -- 172.21.15.123:0/3260364444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd8244e0 msgr2=0x559efd8248c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:29.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.924+0000 7fe1bbe00640 1 --2- 172.21.15.123:0/3260364444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd8244e0 0x559efd8248c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.924+0000 7fe1bbe00640 1 -- 172.21.15.123:0/3260364444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559efd73e4a0 con 0x559efd824e00 2024-09-19T04:46:29.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.924+0000 7fe1bbe00640 1 --2- 172.21.15.123:0/3260364444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd824e00 0x559efd82f180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009900 tx=0x7fe1ac0040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955a96808737b4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:29.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.925+0000 7fe1b1600640 1 -- 172.21.15.123:0/3260364444 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac035040 con 0x559efd824e00 2024-09-19T04:46:29.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.925+0000 7fe1b1600640 1 -- 172.21.15.123:0/3260364444 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1ac005520 con 0x559efd824e00 2024-09-19T04:46:29.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.925+0000 7fe1b1600640 1 -- 172.21.15.123:0/3260364444 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac004e60 con 0x559efd824e00 2024-09-19T04:46:29.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.925+0000 7fe1b8c00640 1 --2- 172.21.15.123:0/4005799299 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe1a00a3240 0x7fe1a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.925+0000 7fe1b8c00640 1 -- 172.21.15.123:0/4005799299 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fe1a80cf330 con 0x7fe1bc074eb0 2024-09-19T04:46:29.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.925+0000 7fe1c0e00640 1 --2- 172.21.15.123:0/4005799299 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe1a00a3240 0x7fe1a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.926+0000 7fe1c5a77740 1 -- 172.21.15.123:0/3260364444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd824e00 msgr2=0x559efd82f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:29.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.926+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/3260364444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd824e00 0x559efd82f180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009900 tx=0x7fe1ac0040f0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.926+0000 7fe1c0e00640 1 --2- 172.21.15.123:0/4005799299 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe1a00a3240 0x7fe1a00a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc076600 tx=0x7fe1b40096c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:29.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.926+0000 7fe1c5a77740 1 -- 172.21.15.123:0/3260364444 shutdown_connections 2024-09-19T04:46:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.926+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/3260364444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd82f6c0 0x559efd831ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.926+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/3260364444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd824e00 0x559efd82f180 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.926+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/3260364444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd8244e0 0x559efd8248c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.926+0000 7fe1c5a77740 1 -- 172.21.15.123:0/3260364444 >> 172.21.15.123:0/3260364444 conn(0x559efd820a70 msgr2=0x559efd820e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.926+0000 7fe1c5a77740 1 -- 172.21.15.123:0/3260364444 shutdown_connections 2024-09-19T04:46:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.926+0000 7fe1c5a77740 1 -- 172.21.15.123:0/3260364444 wait complete. 2024-09-19T04:46:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.926+0000 7fe1c5a77740 1 -- start start 2024-09-19T04:46:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.927+0000 7fe1c5a77740 1 Processor -- start 2024-09-19T04:46:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.927+0000 7fe1c5a77740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd8244e0 0x559efd82c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.927+0000 7fe1c5a77740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd824e00 0x559efd82ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.927+0000 7fe1c5a77740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd82f6c0 0x559efd82cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.927+0000 7fe1c5a77740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559efd832330 con 0x559efd8244e0 2024-09-19T04:46:29.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.927+0000 7fe1c5a77740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559efd712920 con 0x559efd82f6c0 2024-09-19T04:46:29.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.927+0000 7fe1c5a77740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559efd8324b0 con 0x559efd824e00 2024-09-19T04:46:29.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.927+0000 7fe1c5a77740 10 client.0 _ll_register_callbacks cb 0x559efd7368a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:29.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.927+0000 7fe1bbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd8244e0 0x559efd82c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.927+0000 7fe1bbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd8244e0 0x559efd82c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49952/0 (socket says 172.21.15.123:49952) 2024-09-19T04:46:29.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.927+0000 7fe1bbe00640 1 -- 172.21.15.123:0/3443223447 learned_addr learned my addr 172.21.15.123:0/3443223447 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:29.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.928+0000 7fe1c0e00640 1 --2- 172.21.15.123:0/3443223447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd82f6c0 0x559efd82cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.928+0000 7fe1bb400640 1 --2- 172.21.15.123:0/3443223447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd824e00 0x559efd82ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.928+0000 7fe1bbe00640 1 -- 172.21.15.123:0/3443223447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd824e00 msgr2=0x559efd82ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:29.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.928+0000 7fe1bbe00640 1 --2- 172.21.15.123:0/3443223447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd824e00 0x559efd82ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.928+0000 7fe1bbe00640 1 -- 172.21.15.123:0/3443223447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd82f6c0 msgr2=0x559efd82cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:29.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.928+0000 7fe1bbe00640 1 --2- 172.21.15.123:0/3443223447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd82f6c0 0x559efd82cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.928+0000 7fe1bbe00640 1 -- 172.21.15.123:0/3443223447 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559efd4fb2b0 con 0x559efd8244e0 2024-09-19T04:46:29.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.928+0000 7fe1c0e00640 1 --2- 172.21.15.123:0/3443223447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd82f6c0 0x559efd82cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:29.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.928+0000 7fe1bbe00640 1 --2- 172.21.15.123:0/3443223447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd8244e0 0x559efd82c540 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac004600 tx=0x7fe1ac004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ad93a436bd5722 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:29.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.928+0000 7fe1a6000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:46:29.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.929+0000 7fe1a6000640 1 -- 172.21.15.123:0/3443223447 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac053060 con 0x559efd8244e0 2024-09-19T04:46:29.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.929+0000 7fe1a6000640 1 -- 172.21.15.123:0/3443223447 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fe1ac004950 con 0x559efd8244e0 2024-09-19T04:46:29.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.929+0000 7fe1c5a77740 1 -- 172.21.15.123:0/3443223447 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1ac005520 con 0x559efd8244e0 2024-09-19T04:46:29.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.929+0000 7fe1a6000640 1 -- 172.21.15.123:0/3443223447 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe1ac035050 con 0x559efd8244e0 2024-09-19T04:46:29.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.929+0000 7fe1c5a77740 10 client.15051 fetch_fsmap learned FSMap version 27 2024-09-19T04:46:29.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.929+0000 7fe1c5a77740 1 -- 172.21.15.123:0/3443223447 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559efd83dfd0 con 0x559efd8244e0 2024-09-19T04:46:29.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.929+0000 7fe1a6000640 1 -- 172.21.15.123:0/3443223447 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1ac005520 con 0x559efd8244e0 2024-09-19T04:46:29.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.930+0000 7fe1a6000640 1 -- 172.21.15.123:0/3443223447 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe1ac031c00 con 0x559efd8244e0 2024-09-19T04:46:29.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.930+0000 7fe1c5a77740 10 client.15051 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:46:29.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.930+0000 7fe1c5a77740 10 client.15051 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:46:29.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.930+0000 7fe1c5a77740 20 client.15051 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:29.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.930+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/3443223447 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x559efd83f320 0x559efd8414f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.930+0000 7fe1c5a77740 4 client.15051 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:46:29.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.930+0000 7fe1c5a77740 1 -- 172.21.15.123:0/3443223447 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559efd797620 con 0x559efd83f320 2024-09-19T04:46:29.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.930+0000 7fe1bb400640 1 --2- 172.21.15.123:0/3443223447 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x559efd83f320 0x559efd8414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.931+0000 7fe1bb400640 1 --2- 172.21.15.123:0/3443223447 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x559efd83f320 0x559efd8414f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f2c60c7d05a75e4b server_cookie=2d018c14cd21ee89 in_seq=0 out_seq=0 2024-09-19T04:46:29.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.931+0000 7fe1a6000640 10 client.15051 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:46:29.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.931+0000 7fe1a6000640 1 -- 172.21.15.123:0/3443223447 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559efd797620 con 0x559efd83f320 2024-09-19T04:46:29.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.931+0000 7fe1a6000640 10 client.15051 handle_command_reply: tid=0 2024-09-19T04:46:29.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.932+0000 7fe1c5a77740 1 client.15051 shutdown 2024-09-19T04:46:29.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.932+0000 7fe1c5a77740 1 -- 172.21.15.123:0/3443223447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd8244e0 msgr2=0x559efd82c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:29.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.932+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/3443223447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd8244e0 0x559efd82c540 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac004600 tx=0x7fe1ac004c40 comp rx=0 tx=0).stop 2024-09-19T04:46:29.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.932+0000 7fe1c5a77740 1 -- 172.21.15.123:0/3443223447 shutdown_connections 2024-09-19T04:46:29.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.932+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/3443223447 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x559efd83f320 0x559efd8414f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.932+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/3443223447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd82f6c0 0x559efd82cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.932+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/3443223447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd824e00 0x559efd82ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.932+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/3443223447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd8244e0 0x559efd82c540 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.932+0000 7fe1c5a77740 1 -- 172.21.15.123:0/3443223447 >> 172.21.15.123:0/3443223447 conn(0x559efd820a70 msgr2=0x559efd81fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:29.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.932+0000 7fe1a6000640 0 client.15051 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:46:29.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.932+0000 7fe1c5a77740 1 -- 172.21.15.123:0/3443223447 shutdown_connections 2024-09-19T04:46:29.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.932+0000 7fe1c5a77740 1 -- 172.21.15.123:0/3443223447 wait complete. 2024-09-19T04:46:29.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.932+0000 7fe1c5a77740 20 client.15051 trim_cache size 0 max 16384 2024-09-19T04:46:29.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.955+0000 7fe1c5a77740 1 Processor -- start 2024-09-19T04:46:29.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.955+0000 7fe1c5a77740 1 -- start start 2024-09-19T04:46:29.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.955+0000 7fe1c5a77740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd8244e0 0x559efd8414f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.955+0000 7fe1c5a77740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd824e00 0x559efd826eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.955+0000 7fe1c5a77740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd82f6c0 0x559efd83ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.955+0000 7fe1c5a77740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559efd83f830 con 0x559efd824e00 2024-09-19T04:46:29.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.955+0000 7fe1c5a77740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559efd83ff00 con 0x559efd82f6c0 2024-09-19T04:46:29.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.955+0000 7fe1c5a77740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559efd8405d0 con 0x559efd8244e0 2024-09-19T04:46:29.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.956+0000 7fe1bb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd824e00 0x559efd826eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.956+0000 7fe1bb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd824e00 0x559efd826eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49966/0 (socket says 172.21.15.123:49966) 2024-09-19T04:46:29.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.956+0000 7fe1bb400640 1 -- 172.21.15.123:0/1416239922 learned_addr learned my addr 172.21.15.123:0/1416239922 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:29.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.956+0000 7fe1bbe00640 1 --2- 172.21.15.123:0/1416239922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd8244e0 0x559efd8414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.956+0000 7fe1c0e00640 1 --2- 172.21.15.123:0/1416239922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd82f6c0 0x559efd83ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.956+0000 7fe1bb400640 1 -- 172.21.15.123:0/1416239922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd8244e0 msgr2=0x559efd8414f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:29.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.956+0000 7fe1bb400640 1 --2- 172.21.15.123:0/1416239922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd8244e0 0x559efd8414f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.956+0000 7fe1bb400640 1 -- 172.21.15.123:0/1416239922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd82f6c0 msgr2=0x559efd83ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:29.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.956+0000 7fe1bb400640 1 --2- 172.21.15.123:0/1416239922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd82f6c0 0x559efd83ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.956+0000 7fe1bb400640 1 -- 172.21.15.123:0/1416239922 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559efd820510 con 0x559efd824e00 2024-09-19T04:46:29.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.956+0000 7fe1bb400640 1 --2- 172.21.15.123:0/1416239922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd824e00 0x559efd826eb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8097e10 tx=0x7fe1a80cccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79839e737a5a10e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:29.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.957+0000 7fe1b0c00640 1 -- 172.21.15.123:0/1416239922 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8020040 con 0x559efd824e00 2024-09-19T04:46:29.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.957+0000 7fe1b0c00640 1 -- 172.21.15.123:0/1416239922 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1a80046c0 con 0x559efd824e00 2024-09-19T04:46:29.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.957+0000 7fe1b0c00640 1 -- 172.21.15.123:0/1416239922 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a80ccef0 con 0x559efd824e00 2024-09-19T04:46:29.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.957+0000 7fe1c5a77740 1 -- 172.21.15.123:0/1416239922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd824e00 msgr2=0x559efd826eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:29.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.957+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/1416239922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd824e00 0x559efd826eb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8097e10 tx=0x7fe1a80cccb0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.958+0000 7fe1c5a77740 1 -- 172.21.15.123:0/1416239922 shutdown_connections 2024-09-19T04:46:29.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.958+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/1416239922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd82f6c0 0x559efd83ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.958+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/1416239922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd824e00 0x559efd826eb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.958+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/1416239922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd8244e0 0x559efd8414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.958+0000 7fe1c5a77740 1 -- 172.21.15.123:0/1416239922 >> 172.21.15.123:0/1416239922 conn(0x559efd820a70 msgr2=0x559efd83a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:29.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.958+0000 7fe1c5a77740 1 -- 172.21.15.123:0/1416239922 shutdown_connections 2024-09-19T04:46:29.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.958+0000 7fe1c5a77740 1 -- 172.21.15.123:0/1416239922 wait complete. 2024-09-19T04:46:29.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.958+0000 7fe1c5a77740 1 -- start start 2024-09-19T04:46:29.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.958+0000 7fe1c5a77740 1 Processor -- start 2024-09-19T04:46:29.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.959+0000 7fe1c5a77740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd8244e0 0x559efd849ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.959+0000 7fe1c5a77740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd824e00 0x559efd829400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.959+0000 7fe1c5a77740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd82f6c0 0x559efd829a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.959+0000 7fe1c5a77740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559efd841370 con 0x559efd8244e0 2024-09-19T04:46:29.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.959+0000 7fe1c5a77740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559efd840ca0 con 0x559efd824e00 2024-09-19T04:46:29.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.959+0000 7fe1c5a77740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559efd841a40 con 0x559efd82f6c0 2024-09-19T04:46:29.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.959+0000 7fe1c5a77740 10 client.0 _ll_register_callbacks cb 0x7fe1ac004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:29.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.959+0000 7fe1bbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd8244e0 0x559efd849ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.959+0000 7fe1bbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd8244e0 0x559efd849ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49974/0 (socket says 172.21.15.123:49974) 2024-09-19T04:46:29.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.959+0000 7fe1bbe00640 1 -- 172.21.15.123:0/2746720650 learned_addr learned my addr 172.21.15.123:0/2746720650 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:29.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.959+0000 7fe1c0e00640 1 --2- 172.21.15.123:0/2746720650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd82f6c0 0x559efd829a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.959+0000 7fe1bb400640 1 --2- 172.21.15.123:0/2746720650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd824e00 0x559efd829400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.959+0000 7fe1bbe00640 1 -- 172.21.15.123:0/2746720650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd82f6c0 msgr2=0x559efd829a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:29.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.959+0000 7fe1bbe00640 1 --2- 172.21.15.123:0/2746720650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd82f6c0 0x559efd829a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.959+0000 7fe1bbe00640 1 -- 172.21.15.123:0/2746720650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd824e00 msgr2=0x559efd829400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:29.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.959+0000 7fe1bbe00640 1 --2- 172.21.15.123:0/2746720650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd824e00 0x559efd829400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.959+0000 7fe1bbe00640 1 -- 172.21.15.123:0/2746720650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559efd845a30 con 0x559efd8244e0 2024-09-19T04:46:29.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.960+0000 7fe1bbe00640 1 --2- 172.21.15.123:0/2746720650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd8244e0 0x559efd849ff0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac0045d0 tx=0x7fe1ac005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bceb5bed0e8e0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:29.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.960+0000 7fe1a6000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:46:29.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.960+0000 7fe1a6000640 1 -- 172.21.15.123:0/2746720650 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac05d070 con 0x559efd8244e0 2024-09-19T04:46:29.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.960+0000 7fe1a6000640 1 -- 172.21.15.123:0/2746720650 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fe1ac003bb0 con 0x559efd8244e0 2024-09-19T04:46:29.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.960+0000 7fe1c5a77740 1 -- 172.21.15.123:0/2746720650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559efd791b30 con 0x559efd8244e0 2024-09-19T04:46:29.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.960+0000 7fe1a6000640 1 -- 172.21.15.123:0/2746720650 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe1ac069050 con 0x559efd8244e0 2024-09-19T04:46:29.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.960+0000 7fe1c5a77740 10 client.15057 fetch_fsmap learned FSMap version 27 2024-09-19T04:46:29.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.960+0000 7fe1c5a77740 1 -- 172.21.15.123:0/2746720650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559efd8aa850 con 0x559efd8244e0 2024-09-19T04:46:29.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.960+0000 7fe1a6000640 1 -- 172.21.15.123:0/2746720650 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559efd791b30 con 0x559efd8244e0 2024-09-19T04:46:29.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.961+0000 7fe1a6000640 1 -- 172.21.15.123:0/2746720650 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe1ac031640 con 0x559efd8244e0 2024-09-19T04:46:29.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.961+0000 7fe1c5a77740 10 client.15057 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:46:29.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.961+0000 7fe1c5a77740 10 client.15057 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:46:29.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.961+0000 7fe1c5a77740 20 client.15057 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:29.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.961+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/2746720650 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x559efd8af3b0 0x559efd8ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:29.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.961+0000 7fe1c5a77740 4 client.15057 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:46:29.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.961+0000 7fe1c5a77740 1 -- 172.21.15.123:0/2746720650 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "status"}) -- 0x559efd70df80 con 0x559efd8af3b0 2024-09-19T04:46:29.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.961+0000 7fe1bb400640 1 --2- 172.21.15.123:0/2746720650 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x559efd8af3b0 0x559efd8ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:29.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.962+0000 7fe1bb400640 1 --2- 172.21.15.123:0/2746720650 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x559efd8af3b0 0x559efd8ae0c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9007e0b3aa82360 server_cookie=761d2ab211492eda in_seq=0 out_seq=0 2024-09-19T04:46:29.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.962+0000 7fe1a6000640 10 client.15057 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:46:29.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.962+0000 7fe1a6000640 1 -- 172.21.15.123:0/2746720650 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x559efd70df80 con 0x559efd8af3b0 2024-09-19T04:46:29.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.962+0000 7fe1a6000640 10 client.15057 handle_command_reply: tid=0 2024-09-19T04:46:29.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.962+0000 7fe1c5a77740 1 client.15057 shutdown 2024-09-19T04:46:29.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.962+0000 7fe1c5a77740 1 -- 172.21.15.123:0/2746720650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd8244e0 msgr2=0x559efd849ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:29.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.962+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/2746720650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd8244e0 0x559efd849ff0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac0045d0 tx=0x7fe1ac005c00 comp rx=0 tx=0).stop 2024-09-19T04:46:29.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.962+0000 7fe1c5a77740 1 -- 172.21.15.123:0/2746720650 shutdown_connections 2024-09-19T04:46:29.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.962+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/2746720650 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x559efd8af3b0 0x559efd8ae0c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.962+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/2746720650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559efd82f6c0 0x559efd829a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.962+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/2746720650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559efd824e00 0x559efd829400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.962+0000 7fe1a6000640 0 client.15057 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:46:29.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.962+0000 7fe1c5a77740 1 --2- 172.21.15.123:0/2746720650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559efd8244e0 0x559efd849ff0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.962+0000 7fe1c5a77740 1 -- 172.21.15.123:0/2746720650 >> 172.21.15.123:0/2746720650 conn(0x559efd820a70 msgr2=0x559efd83a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:29.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.962+0000 7fe1c5a77740 1 -- 172.21.15.123:0/2746720650 shutdown_connections 2024-09-19T04:46:29.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.962+0000 7fe1c5a77740 1 -- 172.21.15.123:0/2746720650 wait complete. 2024-09-19T04:46:29.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.962+0000 7fe1c5a77740 20 client.15057 trim_cache size 0 max 16384 2024-09-19T04:46:29.969 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:46:29.969 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "911fa052-7640-11ef-bceb-c7b262605968", 2024-09-19T04:46:29.969 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 2, 2024-09-19T04:46:29.969 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24505, 2024-09-19T04:46:29.969 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-09-19T04:46:29.969 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-09-19T04:46:29.969 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-09-19T04:46:29.969 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 108.156087919, 2024-09-19T04:46:29.970 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 27, 2024-09-19T04:46:29.970 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 86, 2024-09-19T04:46:29.970 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T04:46:29.970 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 146.143657214 2024-09-19T04:46:29.970 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:46:29.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.963+0000 7fe1c1800640 1 -- 172.21.15.123:0/4005799299 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe1a00a3240 msgr2=0x7fe1a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:29.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.963+0000 7fe1c1800640 1 --2- 172.21.15.123:0/4005799299 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe1a00a3240 0x7fe1a00a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc076600 tx=0x7fe1b40096c0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.963+0000 7fe1c1800640 1 -- 172.21.15.123:0/4005799299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1bc074eb0 msgr2=0x7fe1bc1a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:29.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.963+0000 7fe1c1800640 1 --2- 172.21.15.123:0/4005799299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1bc074eb0 0x7fe1bc1a4ba0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800d770 tx=0x7fe1a800dc30 comp rx=0 tx=0).stop 2024-09-19T04:46:29.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.963+0000 7fe1c1800640 1 -- 172.21.15.123:0/4005799299 shutdown_connections 2024-09-19T04:46:29.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.963+0000 7fe1c1800640 1 --2- 172.21.15.123:0/4005799299 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fe1a00a3240 0x7fe1a00a5700 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.963+0000 7fe1c1800640 1 --2- 172.21.15.123:0/4005799299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1bc1129e0 0x7fe1bc19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.963+0000 7fe1c1800640 1 --2- 172.21.15.123:0/4005799299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1bc074eb0 0x7fe1bc1a4ba0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.963+0000 7fe1c1800640 1 --2- 172.21.15.123:0/4005799299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1bc074590 0x7fe1bc1a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:29.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.963+0000 7fe1c1800640 1 -- 172.21.15.123:0/4005799299 >> 172.21.15.123:0/4005799299 conn(0x7fe1bc0fd710 msgr2=0x7fe1bc0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:29.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.963+0000 7fe1c1800640 1 -- 172.21.15.123:0/4005799299 shutdown_connections 2024-09-19T04:46:29.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:29.963+0000 7fe1c1800640 1 -- 172.21.15.123:0/4005799299 wait complete. 2024-09-19T04:46:30.708 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T04:46:30.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:30 smithi123 ceph-mon[30511]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-19T04:46:30.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:30 smithi193 ceph-mon[35592]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-19T04:46:31.081 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:31.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:30 smithi183 ceph-mon[35333]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-19T04:46:31.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.693+0000 7f47f8800640 1 -- 172.21.15.123:0/353582041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47f0102790 msgr2=0x7f47f01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:31.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.693+0000 7f47f8800640 1 --2- 172.21.15.123:0/353582041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47f0102790 0x7f47f01067e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009900 tx=0x7f47e002a410 comp rx=0 tx=0).stop 2024-09-19T04:46:31.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.693+0000 7f47f8800640 1 -- 172.21.15.123:0/353582041 shutdown_connections 2024-09-19T04:46:31.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.693+0000 7f47f8800640 1 --2- 172.21.15.123:0/353582041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47f0106f10 0x7f47f010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.693+0000 7f47f8800640 1 --2- 172.21.15.123:0/353582041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47f0102790 0x7f47f01067e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.693+0000 7f47f8800640 1 --2- 172.21.15.123:0/353582041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47f0101de0 0x7f47f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.693+0000 7f47f8800640 1 -- 172.21.15.123:0/353582041 >> 172.21.15.123:0/353582041 conn(0x7f47f00fd8c0 msgr2=0x7f47f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:31.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.693+0000 7f47f8800640 1 -- 172.21.15.123:0/353582041 shutdown_connections 2024-09-19T04:46:31.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.693+0000 7f47f8800640 1 -- 172.21.15.123:0/353582041 wait complete. 2024-09-19T04:46:31.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.694+0000 7f47f8800640 1 Processor -- start 2024-09-19T04:46:31.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.694+0000 7f47f8800640 1 -- start start 2024-09-19T04:46:31.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.694+0000 7f47f8800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47f0101de0 0x7f47f00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.694+0000 7f47f8800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47f0102790 0x7f47f0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.694+0000 7f47f8800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47f0106f10 0x7f47f0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.694+0000 7f47f8800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0111fc0 con 0x7f47f0106f10 2024-09-19T04:46:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.694+0000 7f47f8800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0111e40 con 0x7f47f0102790 2024-09-19T04:46:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.694+0000 7f47f8800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0112140 con 0x7f47f0101de0 2024-09-19T04:46:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.695+0000 7f47f6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47f0106f10 0x7f47f0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.695+0000 7f47f6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47f0106f10 0x7f47f0074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49986/0 (socket says 172.21.15.123:49986) 2024-09-19T04:46:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.695+0000 7f47f6800640 1 -- 172.21.15.123:0/3223312419 learned_addr learned my addr 172.21.15.123:0/3223312419 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.695+0000 7f47f5e00640 1 --2- 172.21.15.123:0/3223312419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47f0101de0 0x7f47f00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.695+0000 7f47f5400640 1 --2- 172.21.15.123:0/3223312419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47f0102790 0x7f47f0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.695+0000 7f47f6800640 1 -- 172.21.15.123:0/3223312419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47f0101de0 msgr2=0x7f47f00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.695+0000 7f47f6800640 1 --2- 172.21.15.123:0/3223312419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47f0101de0 0x7f47f00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.695+0000 7f47f6800640 1 -- 172.21.15.123:0/3223312419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47f0102790 msgr2=0x7f47f0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:31.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.695+0000 7f47f6800640 1 --2- 172.21.15.123:0/3223312419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47f0102790 0x7f47f0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.695+0000 7f47f6800640 1 -- 172.21.15.123:0/3223312419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f01a6950 con 0x7f47f0106f10 2024-09-19T04:46:31.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.696+0000 7f47f6800640 1 --2- 172.21.15.123:0/3223312419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47f0106f10 0x7f47f0074f70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00eb10 tx=0x7f47ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:31.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.696+0000 7f47eaa00640 1 -- 172.21.15.123:0/3223312419 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec004b20 con 0x7f47f0106f10 2024-09-19T04:46:31.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.696+0000 7f47eaa00640 1 -- 172.21.15.123:0/3223312419 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47ec004cc0 con 0x7f47f0106f10 2024-09-19T04:46:31.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.696+0000 7f47f8800640 1 -- 172.21.15.123:0/3223312419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f01a6c40 con 0x7f47f0106f10 2024-09-19T04:46:31.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.696+0000 7f47eaa00640 1 -- 172.21.15.123:0/3223312419 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec010760 con 0x7f47f0106f10 2024-09-19T04:46:31.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.696+0000 7f47f8800640 1 -- 172.21.15.123:0/3223312419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f01a7100 con 0x7f47f0106f10 2024-09-19T04:46:31.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.699+0000 7f47eaa00640 1 -- 172.21.15.123:0/3223312419 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f47ec005020 con 0x7f47f0106f10 2024-09-19T04:46:31.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.699+0000 7f47fb427740 1 Processor -- start 2024-09-19T04:46:31.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.700+0000 7f47fb427740 1 -- start start 2024-09-19T04:46:31.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.700+0000 7f47fb427740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42add330 0x55cf42add710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.700+0000 7f47fb427740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42addc50 0x55cf42ae7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.700+0000 7f47fb427740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42ae8510 0x55cf42aea900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.700+0000 7f47fb427740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cf42a430b0 con 0x55cf42addc50 2024-09-19T04:46:31.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.700+0000 7f47fb427740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cf429eaab0 con 0x55cf42add330 2024-09-19T04:46:31.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.700+0000 7f47fb427740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cf42a3f070 con 0x55cf42ae8510 2024-09-19T04:46:31.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.700+0000 7f47f5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42addc50 0x55cf42ae7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.700+0000 7f47f5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42addc50 0x55cf42ae7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49994/0 (socket says 172.21.15.123:49994) 2024-09-19T04:46:31.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.700+0000 7f47f5e00640 1 -- 172.21.15.123:0/2974009788 learned_addr learned my addr 172.21.15.123:0/2974009788 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:31.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.700+0000 7f47f5400640 1 --2- 172.21.15.123:0/2974009788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42add330 0x55cf42add710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.701+0000 7f47f5400640 1 --2- 172.21.15.123:0/2974009788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42ae8510 0x55cf42aea900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.701+0000 7f47f5e00640 1 -- 172.21.15.123:0/2974009788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42ae8510 msgr2=0x55cf42aea900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:31.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.701+0000 7f47f5e00640 1 --2- 172.21.15.123:0/2974009788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42ae8510 0x55cf42aea900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.701+0000 7f47f5e00640 1 -- 172.21.15.123:0/2974009788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42add330 msgr2=0x55cf42add710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:31.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.701+0000 7f47f5e00640 1 --2- 172.21.15.123:0/2974009788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42add330 0x55cf42add710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.701+0000 7f47f5e00640 1 -- 172.21.15.123:0/2974009788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf429f74a0 con 0x55cf42addc50 2024-09-19T04:46:31.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.701+0000 7f47f5400640 1 --2- 172.21.15.123:0/2974009788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42add330 0x55cf42add710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:46:31.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.701+0000 7f47eaa00640 1 --2- 172.21.15.123:0/3223312419 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f47d80a3240 0x7f47d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.701+0000 7f47f5e00640 1 --2- 172.21.15.123:0/2974009788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42addc50 0x55cf42ae7fd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009d20 tx=0x7f47e4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0048de435f0783d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:31.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.701+0000 7f47eaa00640 1 -- 172.21.15.123:0/3223312419 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f47ec0cfe10 con 0x7f47f0106f10 2024-09-19T04:46:31.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.701+0000 7f47df000640 1 -- 172.21.15.123:0/2974009788 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e401e070 con 0x55cf42addc50 2024-09-19T04:46:31.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.701+0000 7f47df000640 1 -- 172.21.15.123:0/2974009788 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e4019b10 con 0x55cf42addc50 2024-09-19T04:46:31.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.701+0000 7f47f5400640 1 --2- 172.21.15.123:0/3223312419 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f47d80a3240 0x7f47d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.701+0000 7f47df000640 1 -- 172.21.15.123:0/2974009788 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e400ebf0 con 0x55cf42addc50 2024-09-19T04:46:31.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.702+0000 7f47fb427740 1 -- 172.21.15.123:0/2974009788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42addc50 msgr2=0x55cf42ae7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:31.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.702+0000 7f47f5400640 1 --2- 172.21.15.123:0/3223312419 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f47d80a3240 0x7f47d80a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f47e0000c00 tx=0x7f47e002a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:31.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.702+0000 7f47fb427740 1 --2- 172.21.15.123:0/2974009788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42addc50 0x55cf42ae7fd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009d20 tx=0x7f47e4011810 comp rx=0 tx=0).stop 2024-09-19T04:46:31.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.702+0000 7f47fb427740 1 -- 172.21.15.123:0/2974009788 shutdown_connections 2024-09-19T04:46:31.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.702+0000 7f47fb427740 1 --2- 172.21.15.123:0/2974009788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42ae8510 0x55cf42aea900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.702+0000 7f47fb427740 1 --2- 172.21.15.123:0/2974009788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42addc50 0x55cf42ae7fd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.702+0000 7f47fb427740 1 --2- 172.21.15.123:0/2974009788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42add330 0x55cf42add710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.702+0000 7f47fb427740 1 -- 172.21.15.123:0/2974009788 >> 172.21.15.123:0/2974009788 conn(0x55cf42ad98c0 msgr2=0x55cf42ad9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:31.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.703+0000 7f47fb427740 1 -- 172.21.15.123:0/2974009788 shutdown_connections 2024-09-19T04:46:31.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.703+0000 7f47fb427740 1 -- 172.21.15.123:0/2974009788 wait complete. 2024-09-19T04:46:31.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.703+0000 7f47fb427740 1 -- start start 2024-09-19T04:46:31.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.703+0000 7f47fb427740 1 Processor -- start 2024-09-19T04:46:31.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47fb427740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42add330 0x55cf42ae5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47fb427740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42addc50 0x55cf42ae5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47fb427740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42ae8510 0x55cf42ae6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47fb427740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cf42aeb060 con 0x55cf42ae8510 2024-09-19T04:46:31.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47fb427740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cf429cb920 con 0x55cf42add330 2024-09-19T04:46:31.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47fb427740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cf42aeb1e0 con 0x55cf42addc50 2024-09-19T04:46:31.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47fb427740 10 client.0 _ll_register_callbacks cb 0x55cf429ef8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:31.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47f6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42ae8510 0x55cf42ae6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47f5400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42addc50 0x55cf42ae5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47f6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42ae8510 0x55cf42ae6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49996/0 (socket says 172.21.15.123:49996) 2024-09-19T04:46:31.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47f5400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42addc50 0x55cf42ae5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42840/0 (socket says 172.21.15.123:42840) 2024-09-19T04:46:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47f6800640 1 -- 172.21.15.123:0/1206266958 learned_addr learned my addr 172.21.15.123:0/1206266958 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47f5e00640 1 --2- 172.21.15.123:0/1206266958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42add330 0x55cf42ae5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47f6800640 1 -- 172.21.15.123:0/1206266958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42addc50 msgr2=0x55cf42ae5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47f6800640 1 --2- 172.21.15.123:0/1206266958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42addc50 0x55cf42ae5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47f6800640 1 -- 172.21.15.123:0/1206266958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42add330 msgr2=0x55cf42ae5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47f6800640 1 --2- 172.21.15.123:0/1206266958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42add330 0x55cf42ae5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.704+0000 7f47f6800640 1 -- 172.21.15.123:0/1206266958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf427b42b0 con 0x55cf42ae8510 2024-09-19T04:46:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.705+0000 7f47f5e00640 1 --2- 172.21.15.123:0/1206266958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42add330 0x55cf42ae5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.705+0000 7f47f6800640 1 --2- 172.21.15.123:0/1206266958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42ae8510 0x55cf42ae6010 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0a4b60 tx=0x7f47ec09c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393aa6d80e558070 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.705+0000 7f47d3400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:46:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.705+0000 7f47d3400640 1 -- 172.21.15.123:0/1206266958 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0b5060 con 0x55cf42ae8510 2024-09-19T04:46:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.705+0000 7f47d3400640 1 -- 172.21.15.123:0/1206266958 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f47ec0a4c00 con 0x55cf42ae8510 2024-09-19T04:46:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.705+0000 7f47fb427740 1 -- 172.21.15.123:0/1206266958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e4019b10 con 0x55cf42ae8510 2024-09-19T04:46:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.705+0000 7f47d3400640 1 -- 172.21.15.123:0/1206266958 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f47ec0b5400 con 0x55cf42ae8510 2024-09-19T04:46:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.705+0000 7f47fb427740 10 client.15069 fetch_fsmap learned FSMap version 27 2024-09-19T04:46:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.705+0000 7f47fb427740 1 -- 172.21.15.123:0/1206266958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf42af6eb0 con 0x55cf42ae8510 2024-09-19T04:46:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.706+0000 7f47d3400640 1 -- 172.21.15.123:0/1206266958 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e4019b10 con 0x55cf42ae8510 2024-09-19T04:46:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.706+0000 7f47d3400640 1 -- 172.21.15.123:0/1206266958 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f47ec09b870 con 0x55cf42ae8510 2024-09-19T04:46:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.706+0000 7f47fb427740 10 client.15069 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:46:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.706+0000 7f47fb427740 10 client.15069 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:46:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.706+0000 7f47fb427740 20 client.15069 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.706+0000 7f47fb427740 1 --2- 172.21.15.123:0/1206266958 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55cf42af9440 0x55cf42afa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.706+0000 7f47fb427740 4 client.15069 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:46:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.706+0000 7f47fb427740 1 -- 172.21.15.123:0/1206266958 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf42a50620 con 0x55cf42af9440 2024-09-19T04:46:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.706+0000 7f47f5e00640 1 --2- 172.21.15.123:0/1206266958 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55cf42af9440 0x55cf42afa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.707+0000 7f47f5e00640 1 --2- 172.21.15.123:0/1206266958 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55cf42af9440 0x55cf42afa8b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=766e8ee94f188db9 server_cookie=140075f0047c3d8b in_seq=0 out_seq=0 2024-09-19T04:46:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.707+0000 7f47d3400640 10 client.15069 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:46:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.708+0000 7f47d3400640 1 -- 172.21.15.123:0/1206266958 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf42a50620 con 0x55cf42af9440 2024-09-19T04:46:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.708+0000 7f47d3400640 10 client.15069 handle_command_reply: tid=0 2024-09-19T04:46:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.708+0000 7f47fb427740 1 client.15069 shutdown 2024-09-19T04:46:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.708+0000 7f47fb427740 1 -- 172.21.15.123:0/1206266958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42ae8510 msgr2=0x55cf42ae6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.708+0000 7f47fb427740 1 --2- 172.21.15.123:0/1206266958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42ae8510 0x55cf42ae6010 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0a4b60 tx=0x7f47ec09c2b0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.708+0000 7f47fb427740 1 -- 172.21.15.123:0/1206266958 shutdown_connections 2024-09-19T04:46:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.708+0000 7f47fb427740 1 --2- 172.21.15.123:0/1206266958 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55cf42af9440 0x55cf42afa8b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.708+0000 7f47fb427740 1 --2- 172.21.15.123:0/1206266958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42ae8510 0x55cf42ae6010 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.708+0000 7f47fb427740 1 --2- 172.21.15.123:0/1206266958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42addc50 0x55cf42ae5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.708+0000 7f47fb427740 1 --2- 172.21.15.123:0/1206266958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42add330 0x55cf42ae5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.708+0000 7f47d3400640 0 client.15069 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:46:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.708+0000 7f47fb427740 1 -- 172.21.15.123:0/1206266958 >> 172.21.15.123:0/1206266958 conn(0x55cf42ad98c0 msgr2=0x55cf42adf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.708+0000 7f47fb427740 1 -- 172.21.15.123:0/1206266958 shutdown_connections 2024-09-19T04:46:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.708+0000 7f47fb427740 1 -- 172.21.15.123:0/1206266958 wait complete. 2024-09-19T04:46:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.708+0000 7f47fb427740 20 client.15069 trim_cache size 0 max 16384 2024-09-19T04:46:31.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.727+0000 7f47fb427740 1 Processor -- start 2024-09-19T04:46:31.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.727+0000 7f47fb427740 1 -- start start 2024-09-19T04:46:31.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.727+0000 7f47fb427740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42add330 0x55cf42ad9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.727+0000 7f47fb427740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42addc50 0x55cf42ae3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.727+0000 7f47fb427740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42ae8510 0x55cf42ae43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.727+0000 7f47fb427740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cf42af8bf0 con 0x55cf42addc50 2024-09-19T04:46:31.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.727+0000 7f47fb427740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cf42af92c0 con 0x55cf42ae8510 2024-09-19T04:46:31.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.727+0000 7f47fb427740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cf42af9990 con 0x55cf42add330 2024-09-19T04:46:31.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.727+0000 7f47f5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42addc50 0x55cf42ae3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.727+0000 7f47f5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42addc50 0x55cf42ae3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50012/0 (socket says 172.21.15.123:50012) 2024-09-19T04:46:31.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.727+0000 7f47f5400640 1 -- 172.21.15.123:0/2870111541 learned_addr learned my addr 172.21.15.123:0/2870111541 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:31.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.727+0000 7f47f5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42add330 0x55cf42ad9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.727+0000 7f47f6800640 1 --2- 172.21.15.123:0/2870111541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42ae8510 0x55cf42ae43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.728+0000 7f47f5e00640 1 -- 172.21.15.123:0/2870111541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42ae8510 msgr2=0x55cf42ae43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:31.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.728+0000 7f47f5e00640 1 --2- 172.21.15.123:0/2870111541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42ae8510 0x55cf42ae43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.728+0000 7f47f5e00640 1 -- 172.21.15.123:0/2870111541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42addc50 msgr2=0x55cf42ae3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:31.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.728+0000 7f47f5e00640 1 --2- 172.21.15.123:0/2870111541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42addc50 0x55cf42ae3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.728+0000 7f47f5e00640 1 -- 172.21.15.123:0/2870111541 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf42ae3ae0 con 0x55cf42add330 2024-09-19T04:46:31.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.728+0000 7f47f6800640 1 --2- 172.21.15.123:0/2870111541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42ae8510 0x55cf42ae43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:31.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.728+0000 7f47f5400640 1 --2- 172.21.15.123:0/2870111541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42addc50 0x55cf42ae3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:31.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.728+0000 7f47f5e00640 1 --2- 172.21.15.123:0/2870111541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42add330 0x55cf42ad9ca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f47e401c040 tx=0x7f47e400e540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=563a4d36c25ed5f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:31.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.728+0000 7f47de600640 1 -- 172.21.15.123:0/2870111541 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e401e040 con 0x55cf42add330 2024-09-19T04:46:31.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.728+0000 7f47de600640 1 -- 172.21.15.123:0/2870111541 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e401f040 con 0x55cf42add330 2024-09-19T04:46:31.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.728+0000 7f47de600640 1 -- 172.21.15.123:0/2870111541 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e40035f0 con 0x55cf42add330 2024-09-19T04:46:31.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.729+0000 7f47fb427740 1 -- 172.21.15.123:0/2870111541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42add330 msgr2=0x55cf42ad9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:31.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.729+0000 7f47fb427740 1 --2- 172.21.15.123:0/2870111541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42add330 0x55cf42ad9ca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f47e401c040 tx=0x7f47e400e540 comp rx=0 tx=0).stop 2024-09-19T04:46:31.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.729+0000 7f47fb427740 1 -- 172.21.15.123:0/2870111541 shutdown_connections 2024-09-19T04:46:31.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.729+0000 7f47fb427740 1 --2- 172.21.15.123:0/2870111541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42ae8510 0x55cf42ae43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.729+0000 7f47fb427740 1 --2- 172.21.15.123:0/2870111541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42addc50 0x55cf42ae3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.729+0000 7f47fb427740 1 --2- 172.21.15.123:0/2870111541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42add330 0x55cf42ad9ca0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.729+0000 7f47fb427740 1 -- 172.21.15.123:0/2870111541 >> 172.21.15.123:0/2870111541 conn(0x55cf42ad98c0 msgr2=0x55cf42ae0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:31.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.729+0000 7f47fb427740 1 -- 172.21.15.123:0/2870111541 shutdown_connections 2024-09-19T04:46:31.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.729+0000 7f47fb427740 1 -- 172.21.15.123:0/2870111541 wait complete. 2024-09-19T04:46:31.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.730+0000 7f47fb427740 1 -- start start 2024-09-19T04:46:31.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.730+0000 7f47fb427740 1 Processor -- start 2024-09-19T04:46:31.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.730+0000 7f47fb427740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42add330 0x55cf42b03330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.730+0000 7f47fb427740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42addc50 0x55cf42afc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.730+0000 7f47fb427740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42ae8510 0x55cf42afcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.730+0000 7f47fb427740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cf42afa730 con 0x55cf42add330 2024-09-19T04:46:31.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.730+0000 7f47fb427740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cf42afa060 con 0x55cf42ae8510 2024-09-19T04:46:31.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.730+0000 7f47fb427740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cf42afae00 con 0x55cf42addc50 2024-09-19T04:46:31.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.730+0000 7f47fb427740 10 client.0 _ll_register_callbacks cb 0x7f47ec004e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:31.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.731+0000 7f47f5400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42addc50 0x55cf42afc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.731+0000 7f47f5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42add330 0x55cf42b03330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.731+0000 7f47f5400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42addc50 0x55cf42afc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42868/0 (socket says 172.21.15.123:42868) 2024-09-19T04:46:31.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.731+0000 7f47f5400640 1 -- 172.21.15.123:0/3019732146 learned_addr learned my addr 172.21.15.123:0/3019732146 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:31.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.731+0000 7f47f6800640 1 --2- 172.21.15.123:0/3019732146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42ae8510 0x55cf42afcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.731+0000 7f47f5400640 1 -- 172.21.15.123:0/3019732146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42ae8510 msgr2=0x55cf42afcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.731+0000 7f47f5400640 1 --2- 172.21.15.123:0/3019732146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42ae8510 0x55cf42afcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.731+0000 7f47f5400640 1 -- 172.21.15.123:0/3019732146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42add330 msgr2=0x55cf42b03330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.731+0000 7f47f5400640 1 --2- 172.21.15.123:0/3019732146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42add330 0x55cf42b03330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.731+0000 7f47f5400640 1 -- 172.21.15.123:0/3019732146 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf42afd490 con 0x55cf42addc50 2024-09-19T04:46:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.731+0000 7f47f5e00640 1 --2- 172.21.15.123:0/3019732146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42add330 0x55cf42b03330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.731+0000 7f47f5400640 1 --2- 172.21.15.123:0/3019732146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42addc50 0x55cf42afc6b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f47e002a930 tx=0x7f47e0005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4393e25e35c2f0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.731+0000 7f47d3400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:46:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.732+0000 7f47d3400640 1 -- 172.21.15.123:0/3019732146 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0035040 con 0x55cf42addc50 2024-09-19T04:46:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.732+0000 7f47d3400640 1 -- 172.21.15.123:0/3019732146 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f47e00317b0 con 0x55cf42addc50 2024-09-19T04:46:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.732+0000 7f47fb427740 1 -- 172.21.15.123:0/3019732146 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e401f040 con 0x55cf42addc50 2024-09-19T04:46:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.732+0000 7f47d3400640 1 -- 172.21.15.123:0/3019732146 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f47e0003ea0 con 0x55cf42addc50 2024-09-19T04:46:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.732+0000 7f47fb427740 10 client.25105 fetch_fsmap learned FSMap version 27 2024-09-19T04:46:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.732+0000 7f47fb427740 1 -- 172.21.15.123:0/3019732146 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf42b63be0 con 0x55cf42addc50 2024-09-19T04:46:31.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.732+0000 7f47d3400640 1 -- 172.21.15.123:0/3019732146 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e401f040 con 0x55cf42addc50 2024-09-19T04:46:31.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.732+0000 7f47d3400640 1 -- 172.21.15.123:0/3019732146 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f47e0058be0 con 0x55cf42addc50 2024-09-19T04:46:31.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.732+0000 7f47fb427740 10 client.25105 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:46:31.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.733+0000 7f47fb427740 10 client.25105 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:46:31.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.733+0000 7f47fb427740 20 client.25105 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:31.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.733+0000 7f47fb427740 1 --2- 172.21.15.123:0/3019732146 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55cf42b68800 0x55cf42b67510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:31.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.733+0000 7f47fb427740 4 client.25105 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:46:31.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.733+0000 7f47fb427740 1 -- 172.21.15.123:0/3019732146 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "status"}) -- 0x55cf42a4ab30 con 0x55cf42b68800 2024-09-19T04:46:31.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.733+0000 7f47f5e00640 1 --2- 172.21.15.123:0/3019732146 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55cf42b68800 0x55cf42b67510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:31.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.733+0000 7f47f5e00640 1 --2- 172.21.15.123:0/3019732146 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55cf42b68800 0x55cf42b67510 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61863d87624b8a4d server_cookie=51aa1af9a6fad2fc in_seq=0 out_seq=0 2024-09-19T04:46:31.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.734+0000 7f47d3400640 10 client.25105 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:46:31.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.734+0000 7f47d3400640 1 -- 172.21.15.123:0/3019732146 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cf42a4ab30 con 0x55cf42b68800 2024-09-19T04:46:31.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.734+0000 7f47d3400640 10 client.25105 handle_command_reply: tid=0 2024-09-19T04:46:31.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.734+0000 7f47fb427740 1 client.25105 shutdown 2024-09-19T04:46:31.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.734+0000 7f47fb427740 1 -- 172.21.15.123:0/3019732146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42addc50 msgr2=0x55cf42afc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:31.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.734+0000 7f47fb427740 1 --2- 172.21.15.123:0/3019732146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42addc50 0x55cf42afc6b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f47e002a930 tx=0x7f47e0005b50 comp rx=0 tx=0).stop 2024-09-19T04:46:31.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.734+0000 7f47fb427740 1 -- 172.21.15.123:0/3019732146 shutdown_connections 2024-09-19T04:46:31.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.734+0000 7f47fb427740 1 --2- 172.21.15.123:0/3019732146 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55cf42b68800 0x55cf42b67510 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.734+0000 7f47fb427740 1 --2- 172.21.15.123:0/3019732146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf42ae8510 0x55cf42afcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.734+0000 7f47fb427740 1 --2- 172.21.15.123:0/3019732146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf42addc50 0x55cf42afc6b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.734+0000 7f47fb427740 1 --2- 172.21.15.123:0/3019732146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf42add330 0x55cf42b03330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.734+0000 7f47d3400640 0 client.25105 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:46:31.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.734+0000 7f47fb427740 1 -- 172.21.15.123:0/3019732146 >> 172.21.15.123:0/3019732146 conn(0x55cf42ad98c0 msgr2=0x55cf42ae3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:31.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.734+0000 7f47fb427740 1 -- 172.21.15.123:0/3019732146 shutdown_connections 2024-09-19T04:46:31.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.734+0000 7f47fb427740 1 -- 172.21.15.123:0/3019732146 wait complete. 2024-09-19T04:46:31.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.734+0000 7f47fb427740 20 client.25105 trim_cache size 0 max 16384 2024-09-19T04:46:31.740 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:46:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "911fa052-7640-11ef-bceb-c7b262605968", 2024-09-19T04:46:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 1, 2024-09-19T04:46:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24529, 2024-09-19T04:46:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-09-19T04:46:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-09-19T04:46:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-09-19T04:46:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 110.936342478, 2024-09-19T04:46:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 27, 2024-09-19T04:46:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 86, 2024-09-19T04:46:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T04:46:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 142.50946579800001 2024-09-19T04:46:31.741 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:46:31.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.735+0000 7f47f8800640 1 -- 172.21.15.123:0/3223312419 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f47d80a3240 msgr2=0x7f47d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:31.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.735+0000 7f47f8800640 1 --2- 172.21.15.123:0/3223312419 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f47d80a3240 0x7f47d80a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f47e0000c00 tx=0x7f47e002a410 comp rx=0 tx=0).stop 2024-09-19T04:46:31.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.735+0000 7f47f8800640 1 -- 172.21.15.123:0/3223312419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47f0106f10 msgr2=0x7f47f0074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:31.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.735+0000 7f47f8800640 1 --2- 172.21.15.123:0/3223312419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47f0106f10 0x7f47f0074f70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00eb10 tx=0x7f47ec00c2d0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.735+0000 7f47f8800640 1 -- 172.21.15.123:0/3223312419 shutdown_connections 2024-09-19T04:46:31.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.735+0000 7f47f8800640 1 --2- 172.21.15.123:0/3223312419 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f47d80a3240 0x7f47d80a5700 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.735+0000 7f47f8800640 1 --2- 172.21.15.123:0/3223312419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47f0106f10 0x7f47f0074f70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.735+0000 7f47f8800640 1 --2- 172.21.15.123:0/3223312419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47f0102790 0x7f47f0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.735+0000 7f47f8800640 1 --2- 172.21.15.123:0/3223312419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47f0101de0 0x7f47f00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:31.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.735+0000 7f47f8800640 1 -- 172.21.15.123:0/3223312419 >> 172.21.15.123:0/3223312419 conn(0x7f47f00fd8c0 msgr2=0x7f47f00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:31.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.735+0000 7f47f8800640 1 -- 172.21.15.123:0/3223312419 shutdown_connections 2024-09-19T04:46:31.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:31.736+0000 7f47f8800640 1 -- 172.21.15.123:0/3223312419 wait complete. 2024-09-19T04:46:32.509 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T04:46:32.877 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:32.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:32 smithi123 ceph-mon[30511]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:46:32.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:32 smithi193 ceph-mon[35592]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:46:33.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:32 smithi183 ceph-mon[35333]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:46:33.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.470+0000 7fae78c00640 1 -- 172.21.15.123:0/2989844025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae74103e30 msgr2=0x7fae74104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:33.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.470+0000 7fae78c00640 1 --2- 172.21.15.123:0/2989844025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae74103e30 0x7fae74104210 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fae64009900 tx=0x7fae6402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.471+0000 7fae78c00640 1 -- 172.21.15.123:0/2989844025 shutdown_connections 2024-09-19T04:46:33.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.471+0000 7fae78c00640 1 --2- 172.21.15.123:0/2989844025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae74108f60 0x7fae74110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.471+0000 7fae78c00640 1 --2- 172.21.15.123:0/2989844025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae741047e0 0x7fae74108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.471+0000 7fae78c00640 1 --2- 172.21.15.123:0/2989844025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae74103e30 0x7fae74104210 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.471+0000 7fae78c00640 1 -- 172.21.15.123:0/2989844025 >> 172.21.15.123:0/2989844025 conn(0x7fae740ff8d0 msgr2=0x7fae74101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:33.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.471+0000 7fae78c00640 1 -- 172.21.15.123:0/2989844025 shutdown_connections 2024-09-19T04:46:33.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.471+0000 7fae78c00640 1 -- 172.21.15.123:0/2989844025 wait complete. 2024-09-19T04:46:33.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.472+0000 7fae78c00640 1 Processor -- start 2024-09-19T04:46:33.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.472+0000 7fae78c00640 1 -- start start 2024-09-19T04:46:33.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.472+0000 7fae78c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae74103e30 0x7fae7419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.472+0000 7fae78c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae741047e0 0x7fae7419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.472+0000 7fae78c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae74108f60 0x7fae7419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.472+0000 7fae78c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae74113fd0 con 0x7fae74108f60 2024-09-19T04:46:33.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.472+0000 7fae78c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae74113e50 con 0x7fae74103e30 2024-09-19T04:46:33.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.472+0000 7fae78c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae74114150 con 0x7fae741047e0 2024-09-19T04:46:33.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.473+0000 7fae73400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae74103e30 0x7fae7419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.473+0000 7fae73e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae74108f60 0x7fae7419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.473+0000 7fae73400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae74103e30 0x7fae7419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:34404/0 (socket says 172.21.15.123:34404) 2024-09-19T04:46:33.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.473+0000 7fae73400640 1 -- 172.21.15.123:0/2672641048 learned_addr learned my addr 172.21.15.123:0/2672641048 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:33.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.473+0000 7fae6be00640 1 --2- 172.21.15.123:0/2672641048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae741047e0 0x7fae7419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.473+0000 7fae73400640 1 -- 172.21.15.123:0/2672641048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae741047e0 msgr2=0x7fae7419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:33.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.473+0000 7fae73400640 1 --2- 172.21.15.123:0/2672641048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae741047e0 0x7fae7419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.473+0000 7fae73400640 1 -- 172.21.15.123:0/2672641048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae74108f60 msgr2=0x7fae7419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:33.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.473+0000 7fae73400640 1 --2- 172.21.15.123:0/2672641048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae74108f60 0x7fae7419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.473+0000 7fae73400640 1 -- 172.21.15.123:0/2672641048 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7419f420 con 0x7fae74103e30 2024-09-19T04:46:33.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.473+0000 7fae73e00640 1 --2- 172.21.15.123:0/2672641048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae74108f60 0x7fae7419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:33.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.473+0000 7fae6be00640 1 --2- 172.21.15.123:0/2672641048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae741047e0 0x7fae7419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:33.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.473+0000 7fae73400640 1 --2- 172.21.15.123:0/2672641048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae74103e30 0x7fae7419a470 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fae64004770 tx=0x7fae64003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:33.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.474+0000 7fae70c00640 1 -- 172.21.15.123:0/2672641048 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6403f070 con 0x7fae74103e30 2024-09-19T04:46:33.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.474+0000 7fae70c00640 1 -- 172.21.15.123:0/2672641048 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae64031ca0 con 0x7fae74103e30 2024-09-19T04:46:33.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.474+0000 7fae78c00640 1 -- 172.21.15.123:0/2672641048 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7419f6b0 con 0x7fae74103e30 2024-09-19T04:46:33.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.474+0000 7fae70c00640 1 -- 172.21.15.123:0/2672641048 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64035030 con 0x7fae74103e30 2024-09-19T04:46:33.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.474+0000 7fae78c00640 1 -- 172.21.15.123:0/2672641048 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae741a6ff0 con 0x7fae74103e30 2024-09-19T04:46:33.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.477+0000 7fae7cc34740 1 Processor -- start 2024-09-19T04:46:33.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.477+0000 7fae7cc34740 1 -- start start 2024-09-19T04:46:33.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.477+0000 7fae70c00640 1 -- 172.21.15.123:0/2672641048 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fae64042070 con 0x7fae74103e30 2024-09-19T04:46:33.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.477+0000 7fae7cc34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97c4e0 0x5578cd97c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.477+0000 7fae7cc34740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd97ce00 0x5578cd987180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.477+0000 7fae7cc34740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd9876c0 0x5578cd989ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.477+0000 7fae7cc34740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5578cd8e20b0 con 0x5578cd9876c0 2024-09-19T04:46:33.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.477+0000 7fae7cc34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5578cd889ab0 con 0x5578cd97c4e0 2024-09-19T04:46:33.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.477+0000 7fae7cc34740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5578cd8de070 con 0x5578cd97ce00 2024-09-19T04:46:33.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.478+0000 7fae6be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd9876c0 0x5578cd989ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.478+0000 7fae73e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97c4e0 0x5578cd97c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.478+0000 7fae73400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd97ce00 0x5578cd987180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.478+0000 7fae73e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97c4e0 0x5578cd97c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:34418/0 (socket says 172.21.15.123:34418) 2024-09-19T04:46:33.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.478+0000 7fae73e00640 1 -- 172.21.15.123:0/4289423247 learned_addr learned my addr 172.21.15.123:0/4289423247 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:33.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.478+0000 7fae6be00640 1 -- 172.21.15.123:0/4289423247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd97ce00 msgr2=0x5578cd987180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:33.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.478+0000 7fae6be00640 1 --2- 172.21.15.123:0/4289423247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd97ce00 0x5578cd987180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.478+0000 7fae6be00640 1 -- 172.21.15.123:0/4289423247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97c4e0 msgr2=0x5578cd97c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:33.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.478+0000 7fae6be00640 1 --2- 172.21.15.123:0/4289423247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97c4e0 0x5578cd97c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.478+0000 7fae6be00640 1 -- 172.21.15.123:0/4289423247 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578cd8964a0 con 0x5578cd9876c0 2024-09-19T04:46:33.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.478+0000 7fae6be00640 1 --2- 172.21.15.123:0/4289423247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd9876c0 0x5578cd989ab0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009fa0 tx=0x7fae5c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9befa726b81605 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:33.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.479+0000 7fae63e00640 1 -- 172.21.15.123:0/4289423247 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c004020 con 0x5578cd9876c0 2024-09-19T04:46:33.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.479+0000 7fae63e00640 1 -- 172.21.15.123:0/4289423247 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae5c0041c0 con 0x5578cd9876c0 2024-09-19T04:46:33.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.479+0000 7fae63e00640 1 -- 172.21.15.123:0/4289423247 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c00f910 con 0x5578cd9876c0 2024-09-19T04:46:33.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.479+0000 7fae70c00640 1 --2- 172.21.15.123:0/2672641048 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fae580a3240 0x7fae580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.479+0000 7fae70c00640 1 -- 172.21.15.123:0/2672641048 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fae640efbc0 con 0x7fae74103e30 2024-09-19T04:46:33.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.479+0000 7fae73e00640 1 --2- 172.21.15.123:0/2672641048 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fae580a3240 0x7fae580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.479+0000 7fae7cc34740 1 -- 172.21.15.123:0/4289423247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd9876c0 msgr2=0x5578cd989ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:33.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.479+0000 7fae7cc34740 1 --2- 172.21.15.123:0/4289423247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd9876c0 0x5578cd989ab0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009fa0 tx=0x7fae5c0118d0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.480+0000 7fae7cc34740 1 -- 172.21.15.123:0/4289423247 shutdown_connections 2024-09-19T04:46:33.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.480+0000 7fae7cc34740 1 --2- 172.21.15.123:0/4289423247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd9876c0 0x5578cd989ab0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.480+0000 7fae7cc34740 1 --2- 172.21.15.123:0/4289423247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd97ce00 0x5578cd987180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.480+0000 7fae7cc34740 1 --2- 172.21.15.123:0/4289423247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97c4e0 0x5578cd97c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.480+0000 7fae7cc34740 1 -- 172.21.15.123:0/4289423247 >> 172.21.15.123:0/4289423247 conn(0x5578cd978a70 msgr2=0x5578cd978e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:33.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.480+0000 7fae7cc34740 1 -- 172.21.15.123:0/4289423247 shutdown_connections 2024-09-19T04:46:33.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.480+0000 7fae7cc34740 1 -- 172.21.15.123:0/4289423247 wait complete. 2024-09-19T04:46:33.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.480+0000 7fae73e00640 1 --2- 172.21.15.123:0/2672641048 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fae580a3240 0x7fae580a5700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009fd0 tx=0x7fae6c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:33.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.480+0000 7fae7cc34740 1 -- start start 2024-09-19T04:46:33.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.481+0000 7fae7cc34740 1 Processor -- start 2024-09-19T04:46:33.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.481+0000 7fae7cc34740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd97c4e0 0x5578cd9845b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.481+0000 7fae7cc34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97ce00 0x5578cd984af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.481+0000 7fae7cc34740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd9876c0 0x5578cd985030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.481+0000 7fae7cc34740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5578cd98a330 con 0x5578cd97c4e0 2024-09-19T04:46:33.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.481+0000 7fae7cc34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5578cd86a920 con 0x5578cd97ce00 2024-09-19T04:46:33.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.481+0000 7fae7cc34740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5578cd98a4b0 con 0x5578cd9876c0 2024-09-19T04:46:33.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.481+0000 7fae7cc34740 10 client.0 _ll_register_callbacks cb 0x5578cd88e8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:33.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.481+0000 7fae73400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd97c4e0 0x5578cd9845b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.481+0000 7fae73400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd97c4e0 0x5578cd9845b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57434/0 (socket says 172.21.15.123:57434) 2024-09-19T04:46:33.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.481+0000 7fae73400640 1 -- 172.21.15.123:0/1198036693 learned_addr learned my addr 172.21.15.123:0/1198036693 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:33.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.481+0000 7fae73e00640 1 --2- 172.21.15.123:0/1198036693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd9876c0 0x5578cd985030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.481+0000 7fae6be00640 1 --2- 172.21.15.123:0/1198036693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97ce00 0x5578cd984af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.482+0000 7fae73400640 1 -- 172.21.15.123:0/1198036693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd9876c0 msgr2=0x5578cd985030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:33.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.482+0000 7fae73400640 1 --2- 172.21.15.123:0/1198036693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd9876c0 0x5578cd985030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.482+0000 7fae73400640 1 -- 172.21.15.123:0/1198036693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97ce00 msgr2=0x5578cd984af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:33.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.482+0000 7fae73400640 1 --2- 172.21.15.123:0/1198036693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97ce00 0x5578cd984af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.482+0000 7fae73400640 1 -- 172.21.15.123:0/1198036693 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578cd6532b0 con 0x5578cd97c4e0 2024-09-19T04:46:33.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.482+0000 7fae73400640 1 --2- 172.21.15.123:0/1198036693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd97c4e0 0x5578cd9845b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fae640c4990 tx=0x7fae64009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c4a68d9415fd44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:33.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.482+0000 7fae60c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:46:33.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.482+0000 7fae60c00640 1 -- 172.21.15.123:0/1198036693 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64049070 con 0x5578cd97c4e0 2024-09-19T04:46:33.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.482+0000 7fae60c00640 1 -- 172.21.15.123:0/1198036693 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fae640b8970 con 0x5578cd97c4e0 2024-09-19T04:46:33.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.482+0000 7fae7cc34740 1 -- 172.21.15.123:0/1198036693 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae5c0041c0 con 0x5578cd97c4e0 2024-09-19T04:46:33.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.482+0000 7fae60c00640 1 -- 172.21.15.123:0/1198036693 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae640bbc60 con 0x5578cd97c4e0 2024-09-19T04:46:33.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.482+0000 7fae7cc34740 10 client.15087 fetch_fsmap learned FSMap version 27 2024-09-19T04:46:33.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.482+0000 7fae7cc34740 1 -- 172.21.15.123:0/1198036693 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578cd995fe0 con 0x5578cd97c4e0 2024-09-19T04:46:33.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.483+0000 7fae60c00640 1 -- 172.21.15.123:0/1198036693 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae5c0041c0 con 0x5578cd97c4e0 2024-09-19T04:46:33.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.483+0000 7fae60c00640 1 -- 172.21.15.123:0/1198036693 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fae6403b4f0 con 0x5578cd97c4e0 2024-09-19T04:46:33.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.483+0000 7fae7cc34740 10 client.15087 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:46:33.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.483+0000 7fae7cc34740 10 client.15087 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:46:33.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.483+0000 7fae7cc34740 20 client.15087 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:33.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.483+0000 7fae7cc34740 1 --2- 172.21.15.123:0/1198036693 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5578cd998630 0x5578cd999aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.483+0000 7fae7cc34740 4 client.15087 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:46:33.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.483+0000 7fae7cc34740 1 -- 172.21.15.123:0/1198036693 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578cd8ef620 con 0x5578cd998630 2024-09-19T04:46:33.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.483+0000 7fae6be00640 1 --2- 172.21.15.123:0/1198036693 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5578cd998630 0x5578cd999aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.484+0000 7fae6be00640 1 --2- 172.21.15.123:0/1198036693 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5578cd998630 0x5578cd999aa0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e7b048d02e541bd server_cookie=5c5be7aeba20eaec in_seq=0 out_seq=0 2024-09-19T04:46:33.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.484+0000 7fae60c00640 10 client.15087 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:46:33.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.485+0000 7fae60c00640 1 -- 172.21.15.123:0/1198036693 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578cd8ef620 con 0x5578cd998630 2024-09-19T04:46:33.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.485+0000 7fae60c00640 10 client.15087 handle_command_reply: tid=0 2024-09-19T04:46:33.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.485+0000 7fae7cc34740 1 client.15087 shutdown 2024-09-19T04:46:33.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.485+0000 7fae7cc34740 1 -- 172.21.15.123:0/1198036693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd97c4e0 msgr2=0x5578cd9845b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:33.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.485+0000 7fae7cc34740 1 --2- 172.21.15.123:0/1198036693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd97c4e0 0x5578cd9845b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fae640c4990 tx=0x7fae64009620 comp rx=0 tx=0).stop 2024-09-19T04:46:33.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.485+0000 7fae7cc34740 1 -- 172.21.15.123:0/1198036693 shutdown_connections 2024-09-19T04:46:33.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.485+0000 7fae7cc34740 1 --2- 172.21.15.123:0/1198036693 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5578cd998630 0x5578cd999aa0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.485+0000 7fae7cc34740 1 --2- 172.21.15.123:0/1198036693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd9876c0 0x5578cd985030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.485+0000 7fae7cc34740 1 --2- 172.21.15.123:0/1198036693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97ce00 0x5578cd984af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.485+0000 7fae7cc34740 1 --2- 172.21.15.123:0/1198036693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd97c4e0 0x5578cd9845b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.485+0000 7fae7cc34740 1 -- 172.21.15.123:0/1198036693 >> 172.21.15.123:0/1198036693 conn(0x5578cd978a70 msgr2=0x5578cd97e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:33.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.485+0000 7fae60c00640 0 client.15087 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:46:33.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.485+0000 7fae7cc34740 1 -- 172.21.15.123:0/1198036693 shutdown_connections 2024-09-19T04:46:33.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.485+0000 7fae7cc34740 1 -- 172.21.15.123:0/1198036693 wait complete. 2024-09-19T04:46:33.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.485+0000 7fae7cc34740 20 client.15087 trim_cache size 0 max 16384 2024-09-19T04:46:33.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.503+0000 7fae7cc34740 1 Processor -- start 2024-09-19T04:46:33.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.504+0000 7fae7cc34740 1 -- start start 2024-09-19T04:46:33.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.504+0000 7fae7cc34740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd97c4e0 0x5578cd97ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.504+0000 7fae7cc34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97ce00 0x5578cd982e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.504+0000 7fae7cc34740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd9876c0 0x5578cd9833c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.504+0000 7fae7cc34740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5578cd997de0 con 0x5578cd97c4e0 2024-09-19T04:46:33.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.504+0000 7fae7cc34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5578cd9984b0 con 0x5578cd97ce00 2024-09-19T04:46:33.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.504+0000 7fae7cc34740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5578cd998b80 con 0x5578cd9876c0 2024-09-19T04:46:33.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.504+0000 7fae73400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd97c4e0 0x5578cd97ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.504+0000 7fae73400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd97c4e0 0x5578cd97ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57448/0 (socket says 172.21.15.123:57448) 2024-09-19T04:46:33.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.504+0000 7fae73400640 1 -- 172.21.15.123:0/4113647440 learned_addr learned my addr 172.21.15.123:0/4113647440 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:33.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.504+0000 7fae73e00640 1 --2- 172.21.15.123:0/4113647440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd9876c0 0x5578cd9833c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.504+0000 7fae6be00640 1 --2- 172.21.15.123:0/4113647440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97ce00 0x5578cd982e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.504+0000 7fae73400640 1 -- 172.21.15.123:0/4113647440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd9876c0 msgr2=0x5578cd9833c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:33.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.505+0000 7fae73400640 1 --2- 172.21.15.123:0/4113647440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd9876c0 0x5578cd9833c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.505+0000 7fae73400640 1 -- 172.21.15.123:0/4113647440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97ce00 msgr2=0x5578cd982e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:33.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.505+0000 7fae73400640 1 --2- 172.21.15.123:0/4113647440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97ce00 0x5578cd982e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.505+0000 7fae73400640 1 -- 172.21.15.123:0/4113647440 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578cd97c310 con 0x5578cd97c4e0 2024-09-19T04:46:33.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.505+0000 7fae73400640 1 --2- 172.21.15.123:0/4113647440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd97c4e0 0x5578cd97ef40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fae64031550 tx=0x7fae640ba810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f498914bc677b100 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:33.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.505+0000 7fae63400640 1 -- 172.21.15.123:0/4113647440 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64075070 con 0x5578cd97c4e0 2024-09-19T04:46:33.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.505+0000 7fae63400640 1 -- 172.21.15.123:0/4113647440 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae640bc300 con 0x5578cd97c4e0 2024-09-19T04:46:33.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.505+0000 7fae63400640 1 -- 172.21.15.123:0/4113647440 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64049070 con 0x5578cd97c4e0 2024-09-19T04:46:33.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.506+0000 7fae7cc34740 1 -- 172.21.15.123:0/4113647440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd97c4e0 msgr2=0x5578cd97ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:33.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.506+0000 7fae7cc34740 1 --2- 172.21.15.123:0/4113647440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd97c4e0 0x5578cd97ef40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fae64031550 tx=0x7fae640ba810 comp rx=0 tx=0).stop 2024-09-19T04:46:33.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.506+0000 7fae7cc34740 1 -- 172.21.15.123:0/4113647440 shutdown_connections 2024-09-19T04:46:33.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.506+0000 7fae7cc34740 1 --2- 172.21.15.123:0/4113647440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd9876c0 0x5578cd9833c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.506+0000 7fae7cc34740 1 --2- 172.21.15.123:0/4113647440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97ce00 0x5578cd982e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.506+0000 7fae7cc34740 1 --2- 172.21.15.123:0/4113647440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd97c4e0 0x5578cd97ef40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.506+0000 7fae7cc34740 1 -- 172.21.15.123:0/4113647440 >> 172.21.15.123:0/4113647440 conn(0x5578cd978a70 msgr2=0x5578cd97f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:33.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.506+0000 7fae7cc34740 1 -- 172.21.15.123:0/4113647440 shutdown_connections 2024-09-19T04:46:33.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.506+0000 7fae7cc34740 1 -- 172.21.15.123:0/4113647440 wait complete. 2024-09-19T04:46:33.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.506+0000 7fae7cc34740 1 -- start start 2024-09-19T04:46:33.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.506+0000 7fae7cc34740 1 Processor -- start 2024-09-19T04:46:33.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.507+0000 7fae7cc34740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd97c4e0 0x5578cd9a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.507+0000 7fae7cc34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97ce00 0x5578cd981220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.507+0000 7fae7cc34740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd9876c0 0x5578cd981760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.507+0000 7fae7cc34740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5578cd999920 con 0x5578cd9876c0 2024-09-19T04:46:33.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.507+0000 7fae7cc34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5578cd999250 con 0x5578cd97ce00 2024-09-19T04:46:33.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.507+0000 7fae7cc34740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5578cd999ff0 con 0x5578cd97c4e0 2024-09-19T04:46:33.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.507+0000 7fae7cc34740 10 client.0 _ll_register_callbacks cb 0x7fae64031e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:33.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.507+0000 7fae73e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd9876c0 0x5578cd981760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.507+0000 7fae73400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd97c4e0 0x5578cd9a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.507+0000 7fae73e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd9876c0 0x5578cd981760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57464/0 (socket says 172.21.15.123:57464) 2024-09-19T04:46:33.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.507+0000 7fae73e00640 1 -- 172.21.15.123:0/4227900886 learned_addr learned my addr 172.21.15.123:0/4227900886 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:33.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.507+0000 7fae6be00640 1 --2- 172.21.15.123:0/4227900886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97ce00 0x5578cd981220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.508+0000 7fae73e00640 1 -- 172.21.15.123:0/4227900886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd97c4e0 msgr2=0x5578cd9a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:33.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.508+0000 7fae73e00640 1 --2- 172.21.15.123:0/4227900886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd97c4e0 0x5578cd9a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.508+0000 7fae73e00640 1 -- 172.21.15.123:0/4227900886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97ce00 msgr2=0x5578cd981220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:33.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.508+0000 7fae73e00640 1 --2- 172.21.15.123:0/4227900886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97ce00 0x5578cd981220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.508+0000 7fae73e00640 1 -- 172.21.15.123:0/4227900886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578cd981f90 con 0x5578cd9876c0 2024-09-19T04:46:33.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.508+0000 7fae73e00640 1 --2- 172.21.15.123:0/4227900886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd9876c0 0x5578cd981760 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00f790 tx=0x7fae6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cbb0fa347316ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:33.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.508+0000 7fae60c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:46:33.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.508+0000 7fae60c00640 1 -- 172.21.15.123:0/4227900886 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c011ad0 con 0x5578cd9876c0 2024-09-19T04:46:33.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.508+0000 7fae60c00640 1 -- 172.21.15.123:0/4227900886 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7fae6c011c70 con 0x5578cd9876c0 2024-09-19T04:46:33.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.508+0000 7fae7cc34740 1 -- 172.21.15.123:0/4227900886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae640bc300 con 0x5578cd9876c0 2024-09-19T04:46:33.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.508+0000 7fae60c00640 1 -- 172.21.15.123:0/4227900886 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae6c011e70 con 0x5578cd9876c0 2024-09-19T04:46:33.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.508+0000 7fae7cc34740 10 client.15093 fetch_fsmap learned FSMap version 27 2024-09-19T04:46:33.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.509+0000 7fae7cc34740 1 -- 172.21.15.123:0/4227900886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578cda02d40 con 0x5578cd9876c0 2024-09-19T04:46:33.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.509+0000 7fae60c00640 1 -- 172.21.15.123:0/4227900886 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae640bc300 con 0x5578cd9876c0 2024-09-19T04:46:33.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.509+0000 7fae60c00640 1 -- 172.21.15.123:0/4227900886 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fae6c01f420 con 0x5578cd9876c0 2024-09-19T04:46:33.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.509+0000 7fae7cc34740 10 client.15093 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:46:33.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.509+0000 7fae7cc34740 10 client.15093 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:46:33.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.509+0000 7fae7cc34740 20 client.15093 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:33.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.509+0000 7fae7cc34740 1 --2- 172.21.15.123:0/4227900886 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5578cda07980 0x5578cda06690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:33.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.509+0000 7fae7cc34740 4 client.15093 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:46:33.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.509+0000 7fae7cc34740 1 -- 172.21.15.123:0/4227900886 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "status"}) -- 0x5578cd8e9b30 con 0x5578cda07980 2024-09-19T04:46:33.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.509+0000 7fae73400640 1 --2- 172.21.15.123:0/4227900886 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5578cda07980 0x5578cda06690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:33.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.510+0000 7fae73400640 1 --2- 172.21.15.123:0/4227900886 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5578cda07980 0x5578cda06690 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=116fba0cb68080a6 server_cookie=4ee2c6e8d95811de in_seq=0 out_seq=0 2024-09-19T04:46:33.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.510+0000 7fae60c00640 10 client.15093 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:46:33.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.510+0000 7fae60c00640 1 -- 172.21.15.123:0/4227900886 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5578cd8e9b30 con 0x5578cda07980 2024-09-19T04:46:33.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.510+0000 7fae60c00640 10 client.15093 handle_command_reply: tid=0 2024-09-19T04:46:33.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.510+0000 7fae7cc34740 1 client.15093 shutdown 2024-09-19T04:46:33.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.510+0000 7fae7cc34740 1 -- 172.21.15.123:0/4227900886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd9876c0 msgr2=0x5578cd981760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:33.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.510+0000 7fae7cc34740 1 --2- 172.21.15.123:0/4227900886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd9876c0 0x5578cd981760 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00f790 tx=0x7fae6c004a40 comp rx=0 tx=0).stop 2024-09-19T04:46:33.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.510+0000 7fae7cc34740 1 -- 172.21.15.123:0/4227900886 shutdown_connections 2024-09-19T04:46:33.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.510+0000 7fae7cc34740 1 --2- 172.21.15.123:0/4227900886 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5578cda07980 0x5578cda06690 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.510+0000 7fae7cc34740 1 --2- 172.21.15.123:0/4227900886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578cd9876c0 0x5578cd981760 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.510+0000 7fae7cc34740 1 --2- 172.21.15.123:0/4227900886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578cd97ce00 0x5578cd981220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.510+0000 7fae7cc34740 1 --2- 172.21.15.123:0/4227900886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578cd97c4e0 0x5578cd9a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.510+0000 7fae60c00640 0 client.15093 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:46:33.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.510+0000 7fae7cc34740 1 -- 172.21.15.123:0/4227900886 >> 172.21.15.123:0/4227900886 conn(0x5578cd978a70 msgr2=0x5578cd992700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:33.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.511+0000 7fae7cc34740 1 -- 172.21.15.123:0/4227900886 shutdown_connections 2024-09-19T04:46:33.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.511+0000 7fae7cc34740 1 -- 172.21.15.123:0/4227900886 wait complete. 2024-09-19T04:46:33.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.511+0000 7fae7cc34740 20 client.15093 trim_cache size 0 max 16384 2024-09-19T04:46:33.518 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:46:33.518 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "911fa052-7640-11ef-bceb-c7b262605968", 2024-09-19T04:46:33.518 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 0, 2024-09-19T04:46:33.518 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24539, 2024-09-19T04:46:33.518 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-09-19T04:46:33.518 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-09-19T04:46:33.518 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-09-19T04:46:33.518 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 120.764187056, 2024-09-19T04:46:33.518 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 27, 2024-09-19T04:46:33.519 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 86, 2024-09-19T04:46:33.519 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T04:46:33.519 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 134.25823957700001 2024-09-19T04:46:33.519 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:46:33.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.511+0000 7fae78c00640 1 -- 172.21.15.123:0/2672641048 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fae580a3240 msgr2=0x7fae580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:33.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.511+0000 7fae78c00640 1 --2- 172.21.15.123:0/2672641048 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fae580a3240 0x7fae580a5700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009fd0 tx=0x7fae6c009500 comp rx=0 tx=0).stop 2024-09-19T04:46:33.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.511+0000 7fae78c00640 1 -- 172.21.15.123:0/2672641048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae74103e30 msgr2=0x7fae7419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:33.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.511+0000 7fae78c00640 1 --2- 172.21.15.123:0/2672641048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae74103e30 0x7fae7419a470 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fae64004770 tx=0x7fae64003c10 comp rx=0 tx=0).stop 2024-09-19T04:46:33.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.511+0000 7fae78c00640 1 -- 172.21.15.123:0/2672641048 shutdown_connections 2024-09-19T04:46:33.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.511+0000 7fae78c00640 1 --2- 172.21.15.123:0/2672641048 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7fae580a3240 0x7fae580a5700 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.511+0000 7fae78c00640 1 --2- 172.21.15.123:0/2672641048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae74108f60 0x7fae7419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.511+0000 7fae78c00640 1 --2- 172.21.15.123:0/2672641048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae741047e0 0x7fae7419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.511+0000 7fae78c00640 1 --2- 172.21.15.123:0/2672641048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae74103e30 0x7fae7419a470 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:33.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.511+0000 7fae78c00640 1 -- 172.21.15.123:0/2672641048 >> 172.21.15.123:0/2672641048 conn(0x7fae740ff8d0 msgr2=0x7fae74100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:33.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.512+0000 7fae78c00640 1 -- 172.21.15.123:0/2672641048 shutdown_connections 2024-09-19T04:46:33.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:33.512+0000 7fae78c00640 1 -- 172.21.15.123:0/2672641048 wait complete. 2024-09-19T04:46:34.248 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T04:46:34.259 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T04:46:34.259 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T04:46:34.259 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T04:46:34.259 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-19T04:46:34.260 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T04:46:34.260 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T04:46:34.261 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-19T04:46:34.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.413+0000 7f5da3400640 1 -- 172.21.15.123:0/1866187619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d9c104f50 msgr2=0x7f5d9c1053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:34.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.413+0000 7f5da3400640 1 --2- 172.21.15.123:0/1866187619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d9c104f50 0x7f5d9c1053d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009900 tx=0x7f5d8802a400 comp rx=0 tx=0).stop 2024-09-19T04:46:34.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.413+0000 7f5da3400640 1 -- 172.21.15.123:0/1866187619 shutdown_connections 2024-09-19T04:46:34.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.413+0000 7f5da3400640 1 --2- 172.21.15.123:0/1866187619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d9c105910 0x7f5d9c10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:34.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.413+0000 7f5da3400640 1 --2- 172.21.15.123:0/1866187619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d9c104f50 0x7f5d9c1053d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:34.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.413+0000 7f5da3400640 1 --2- 172.21.15.123:0/1866187619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d9c103d60 0x7f5d9c104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:34.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.413+0000 7f5da3400640 1 -- 172.21.15.123:0/1866187619 >> 172.21.15.123:0/1866187619 conn(0x7f5d9c0ff4e0 msgr2=0x7f5d9c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:34.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.413+0000 7f5da3400640 1 -- 172.21.15.123:0/1866187619 shutdown_connections 2024-09-19T04:46:34.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.413+0000 7f5da3400640 1 -- 172.21.15.123:0/1866187619 wait complete. 2024-09-19T04:46:34.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.414+0000 7f5da3400640 1 Processor -- start 2024-09-19T04:46:34.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.414+0000 7f5da3400640 1 -- start start 2024-09-19T04:46:34.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.414+0000 7f5da3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d9c103d60 0x7f5d9c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:34.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.414+0000 7f5da3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d9c104f50 0x7f5d9c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:34.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.414+0000 7f5da3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d9c105910 0x7f5d9c1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:34.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.414+0000 7f5da3400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c10fa30 con 0x7f5d9c105910 2024-09-19T04:46:34.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.414+0000 7f5da3400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c10f8b0 con 0x7f5d9c104f50 2024-09-19T04:46:34.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.414+0000 7f5da3400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c10fbb0 con 0x7f5d9c103d60 2024-09-19T04:46:34.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.414+0000 7f5da0a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d9c103d60 0x7f5d9c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:34.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.414+0000 7f5da0a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d9c103d60 0x7f5d9c19a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:45172/0 (socket says 172.21.15.123:45172) 2024-09-19T04:46:34.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.414+0000 7f5da0a00640 1 -- 172.21.15.123:0/1855556764 learned_addr learned my addr 172.21.15.123:0/1855556764 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:34.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.414+0000 7f5da1400640 1 --2- 172.21.15.123:0/1855556764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d9c105910 0x7f5d9c1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:34.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.415+0000 7f5d9be00640 1 --2- 172.21.15.123:0/1855556764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d9c104f50 0x7f5d9c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:34.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.415+0000 7f5da0a00640 1 -- 172.21.15.123:0/1855556764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d9c104f50 msgr2=0x7f5d9c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:34.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.415+0000 7f5da0a00640 1 --2- 172.21.15.123:0/1855556764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d9c104f50 0x7f5d9c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:34.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.415+0000 7f5da0a00640 1 -- 172.21.15.123:0/1855556764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d9c105910 msgr2=0x7f5d9c1a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:34.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.415+0000 7f5da0a00640 1 --2- 172.21.15.123:0/1855556764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d9c105910 0x7f5d9c1a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:34.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.415+0000 7f5da0a00640 1 -- 172.21.15.123:0/1855556764 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9c1a1c20 con 0x7f5d9c103d60 2024-09-19T04:46:34.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.415+0000 7f5da0a00640 1 --2- 172.21.15.123:0/1855556764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d9c103d60 0x7f5d9c19a000 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000e9c0 tx=0x7f5d9000edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:34.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.415+0000 7f5d99600640 1 -- 172.21.15.123:0/1855556764 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9000ccc0 con 0x7f5d9c103d60 2024-09-19T04:46:34.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.415+0000 7f5d99600640 1 -- 172.21.15.123:0/1855556764 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d900035f0 con 0x7f5d9c103d60 2024-09-19T04:46:34.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.415+0000 7f5da3400640 1 -- 172.21.15.123:0/1855556764 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9c1a1f10 con 0x7f5d9c103d60 2024-09-19T04:46:34.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.415+0000 7f5d99600640 1 -- 172.21.15.123:0/1855556764 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9000ce60 con 0x7f5d9c103d60 2024-09-19T04:46:34.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.415+0000 7f5da3400640 1 -- 172.21.15.123:0/1855556764 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9c1a2320 con 0x7f5d9c103d60 2024-09-19T04:46:34.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.416+0000 7f5da3400640 1 -- 172.21.15.123:0/1855556764 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d9c10b080 con 0x7f5d9c103d60 2024-09-19T04:46:34.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.419+0000 7f5d99600640 1 -- 172.21.15.123:0/1855556764 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f5d900037a0 con 0x7f5d9c103d60 2024-09-19T04:46:34.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.420+0000 7f5d99600640 1 --2- 172.21.15.123:0/1855556764 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5d840a32f0 0x7f5d840a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:34.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.420+0000 7f5d99600640 1 -- 172.21.15.123:0/1855556764 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f5d900cfd80 con 0x7f5d9c103d60 2024-09-19T04:46:34.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.420+0000 7f5d9be00640 1 --2- 172.21.15.123:0/1855556764 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5d840a32f0 0x7f5d840a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:34.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.421+0000 7f5d9be00640 1 --2- 172.21.15.123:0/1855556764 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5d840a32f0 0x7f5d840a57b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5d88031440 tx=0x7f5d880315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:34.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.422+0000 7f5d99600640 1 -- 172.21.15.123:0/1855556764 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5d9009ca50 con 0x7f5d9c103d60 2024-09-19T04:46:34.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:34.647+0000 7f5da3400640 1 -- 172.21.15.123:0/1855556764 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f5d9c02da30 con 0x7f5d9c103d60 2024-09-19T04:46:34.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:34 smithi123 ceph-mon[30511]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:46:34.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:34 smithi193 ceph-mon[35592]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:46:35.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:34 smithi183 ceph-mon[35333]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:46:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:35.639+0000 7f5d99600640 1 -- 172.21.15.123:0/1855556764 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7f5d900989c0 con 0x7f5d9c103d60 2024-09-19T04:46:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:35.639+0000 7f5d99600640 1 -- 172.21.15.123:0/1855556764 <== mon.1 v2:172.21.15.193:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f5d900d1520 con 0x7f5d9c103d60 2024-09-19T04:46:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:35.642+0000 7f5da3400640 1 -- 172.21.15.123:0/1855556764 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5d840a32f0 msgr2=0x7f5d840a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:35.642+0000 7f5da3400640 1 --2- 172.21.15.123:0/1855556764 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5d840a32f0 0x7f5d840a57b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5d88031440 tx=0x7f5d880315b0 comp rx=0 tx=0).stop 2024-09-19T04:46:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:35.642+0000 7f5da3400640 1 -- 172.21.15.123:0/1855556764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d9c103d60 msgr2=0x7f5d9c19a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:35.642+0000 7f5da3400640 1 --2- 172.21.15.123:0/1855556764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d9c103d60 0x7f5d9c19a000 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000e9c0 tx=0x7f5d9000edc0 comp rx=0 tx=0).stop 2024-09-19T04:46:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:35.642+0000 7f5da3400640 1 -- 172.21.15.123:0/1855556764 shutdown_connections 2024-09-19T04:46:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:35.642+0000 7f5da3400640 1 --2- 172.21.15.123:0/1855556764 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f5d840a32f0 0x7f5d840a57b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:35.642+0000 7f5da3400640 1 --2- 172.21.15.123:0/1855556764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d9c105910 0x7f5d9c1a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:35.642+0000 7f5da3400640 1 --2- 172.21.15.123:0/1855556764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d9c104f50 0x7f5d9c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:35.642+0000 7f5da3400640 1 --2- 172.21.15.123:0/1855556764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d9c103d60 0x7f5d9c19a000 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:35.642+0000 7f5da3400640 1 -- 172.21.15.123:0/1855556764 >> 172.21.15.123:0/1855556764 conn(0x7f5d9c0ff4e0 msgr2=0x7f5d9c108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:35.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:35.643+0000 7f5da3400640 1 -- 172.21.15.123:0/1855556764 shutdown_connections 2024-09-19T04:46:35.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:35.643+0000 7f5da3400640 1 -- 172.21.15.123:0/1855556764 wait complete. 2024-09-19T04:46:35.664 INFO:teuthology.run_tasks:Running task kclient... 2024-09-19T04:46:35.690 INFO:tasks.kclient:Mounting kernel clients... 2024-09-19T04:46:35.690 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-19T04:46:35.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:46:35.690 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-09-19T04:46:35.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:46:35.732 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-09-19T04:46:35.834 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2024-09-19T04:46:35.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:46:35.837 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-19T04:46:35.837 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T04:46:35.837 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T04:46:35.838 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T04:46:35.886 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:35 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1855556764' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T04:46:35.886 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:35 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T04:46:35.886 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: ignoring --setuser ceph since I am not root 2024-09-19T04:46:35.886 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: ignoring --setgroup ceph since I am not root 2024-09-19T04:46:35.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:35 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1855556764' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T04:46:35.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:35 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T04:46:35.987 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: ignoring --setuser ceph since I am not root 2024-09-19T04:46:35.987 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: ignoring --setgroup ceph since I am not root 2024-09-19T04:46:35.987 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:35 smithi193 ceph-mgr[37408]: -- 172.21.15.193:0/1836623606 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ec7fb50e0 con 0x558ec7f93000 2024-09-19T04:46:35.987 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:35.779+0000 7fe5a0052040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:46:35.987 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:35.836+0000 7fe5a0052040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:46:35.987 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:35.984+0000 7fe5a0052040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:46:35.991 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:35 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1855556764' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T04:46:35.991 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:35 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T04:46:35.991 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: ignoring --setuser ceph since I am not root 2024-09-19T04:46:35.991 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: ignoring --setgroup ceph since I am not root 2024-09-19T04:46:35.991 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:35 smithi183 ceph-mgr[36698]: -- 172.21.15.183:0/4207328849 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562f99fcd0e0 con 0x562f99fab000 2024-09-19T04:46:35.991 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:35.783+0000 7f7dc6c7b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:46:35.992 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:35.841+0000 7f7dc6c7b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:46:35.992 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:35.990+0000 7f7dc6c7b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:46:36.156 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:35.888+0000 7f2a9440c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:46:36.156 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:35.961+0000 7f2a9440c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:46:36.156 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:36 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:36.154+0000 7f2a9440c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:46:36.223 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:36.240 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:36 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:36.051+0000 7fe5a0052040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:46:36.240 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:36 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:36.196+0000 7fe5a0052040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:46:36.345 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:36 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:36.057+0000 7f7dc6c7b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:46:36.345 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:36 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:36.203+0000 7f7dc6c7b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:46:36.345 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:36 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:36.344+0000 7f7dc6c7b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:46:36.424 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:36 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:36.242+0000 7f2a9440c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:46:36.632 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:36 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:36.335+0000 7fe5a0052040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:46:36.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:36 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T04:46:36.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:36 smithi123 ceph-mon[30511]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-19T04:46:36.677 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:36 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:36.422+0000 7f2a9440c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:46:36.677 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:36 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:36.591+0000 7f2a9440c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:46:36.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.796+0000 7f2391400640 1 -- 172.21.15.123:0/2319040920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f238c1049a0 msgr2=0x7f238c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:36.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.796+0000 7f2391400640 1 --2- 172.21.15.123:0/2319040920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f238c1049a0 0x7f238c1089f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2378009900 tx=0x7f237802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:46:36.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.796+0000 7f2391400640 1 -- 172.21.15.123:0/2319040920 shutdown_connections 2024-09-19T04:46:36.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.796+0000 7f2391400640 1 --2- 172.21.15.123:0/2319040920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f238c109120 0x7f238c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:36.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.796+0000 7f2391400640 1 --2- 172.21.15.123:0/2319040920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f238c1049a0 0x7f238c1089f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:36.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.796+0000 7f2391400640 1 --2- 172.21.15.123:0/2319040920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f238c103ff0 0x7f238c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:36.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.796+0000 7f2391400640 1 -- 172.21.15.123:0/2319040920 >> 172.21.15.123:0/2319040920 conn(0x7f238c0ffa90 msgr2=0x7f238c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:36.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.796+0000 7f2391400640 1 -- 172.21.15.123:0/2319040920 shutdown_connections 2024-09-19T04:46:36.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.797+0000 7f2391400640 1 -- 172.21.15.123:0/2319040920 wait complete. 2024-09-19T04:46:36.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.797+0000 7f2391400640 1 Processor -- start 2024-09-19T04:46:36.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.798+0000 7f2391400640 1 -- start start 2024-09-19T04:46:36.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.798+0000 7f2391400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f238c103ff0 0x7f238c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:36.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.798+0000 7f2391400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f238c1049a0 0x7f238c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:36.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.798+0000 7f2391400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f238c109120 0x7f238c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:36.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.798+0000 7f2391400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f238c114140 con 0x7f238c109120 2024-09-19T04:46:36.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.798+0000 7f2391400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f238c113fc0 con 0x7f238c103ff0 2024-09-19T04:46:36.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.798+0000 7f2391400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f238c1142c0 con 0x7f238c1049a0 2024-09-19T04:46:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.798+0000 7f238a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f238c103ff0 0x7f238c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.798+0000 7f2389e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f238c1049a0 0x7f238c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.798+0000 7f238a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f238c103ff0 0x7f238c19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:34496/0 (socket says 172.21.15.123:34496) 2024-09-19T04:46:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.798+0000 7f2389e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f238c1049a0 0x7f238c19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:45210/0 (socket says 172.21.15.123:45210) 2024-09-19T04:46:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.798+0000 7f238a800640 1 -- 172.21.15.123:0/2637985527 learned_addr learned my addr 172.21.15.123:0/2637985527 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.798+0000 7f238b200640 1 --2- 172.21.15.123:0/2637985527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f238c109120 0x7f238c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.799+0000 7f238a800640 1 -- 172.21.15.123:0/2637985527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f238c1049a0 msgr2=0x7f238c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.799+0000 7f238a800640 1 --2- 172.21.15.123:0/2637985527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f238c1049a0 0x7f238c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.799+0000 7f238a800640 1 -- 172.21.15.123:0/2637985527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f238c109120 msgr2=0x7f238c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.799+0000 7f238a800640 1 --2- 172.21.15.123:0/2637985527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f238c109120 0x7f238c19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.799+0000 7f238a800640 1 -- 172.21.15.123:0/2637985527 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238c19f5d0 con 0x7f238c103ff0 2024-09-19T04:46:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.799+0000 7f238a800640 1 --2- 172.21.15.123:0/2637985527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f238c103ff0 0x7f238c19a620 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f237c00e9f0 tx=0x7f237c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.799+0000 7f2383400640 1 -- 172.21.15.123:0/2637985527 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c00cd80 con 0x7f238c103ff0 2024-09-19T04:46:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.799+0000 7f2391400640 1 -- 172.21.15.123:0/2637985527 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238c19f8c0 con 0x7f238c103ff0 2024-09-19T04:46:36.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.799+0000 7f2383400640 1 -- 172.21.15.123:0/2637985527 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f237c005500 con 0x7f238c103ff0 2024-09-19T04:46:36.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.799+0000 7f2383400640 1 -- 172.21.15.123:0/2637985527 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c010690 con 0x7f238c103ff0 2024-09-19T04:46:36.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.799+0000 7f2391400640 1 -- 172.21.15.123:0/2637985527 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f238c1a71a0 con 0x7f238c103ff0 2024-09-19T04:46:36.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.800+0000 7f2391400640 1 -- 172.21.15.123:0/2637985527 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f238c1043d0 con 0x7f238c103ff0 2024-09-19T04:46:36.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.802+0000 7f2383400640 1 -- 172.21.15.123:0/2637985527 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7f237c0046c0 con 0x7f238c103ff0 2024-09-19T04:46:36.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.803+0000 7f2383400640 1 --2- 172.21.15.123:0/2637985527 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f23700a3290 0x7f23700a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:36.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.803+0000 7f2389e00640 1 -- 172.21.15.123:0/2637985527 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f23700a3290 msgr2=0x7f23700a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/1532622003 2024-09-19T04:46:36.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.804+0000 7f2383400640 1 -- 172.21.15.123:0/2637985527 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f237c0cfe50 con 0x7f238c103ff0 2024-09-19T04:46:36.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.804+0000 7f2389e00640 1 --2- 172.21.15.123:0/2637985527 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f23700a3290 0x7f23700a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:46:36.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:36.807+0000 7f2383400640 1 -- 172.21.15.123:0/2637985527 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f237c09cb20 con 0x7f238c103ff0 2024-09-19T04:46:36.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:36 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T04:46:36.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:36 smithi193 ceph-mon[35592]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-19T04:46:36.892 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:36 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:36.646+0000 7fe5a0052040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:46:36.892 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:36 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:36.706+0000 7fe5a0052040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:46:36.892 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:36 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:36.890+0000 7fe5a0052040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:46:36.905 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:36 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T04:46:36.905 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:36 smithi183 ceph-mon[35333]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-19T04:46:36.905 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:36 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:36.656+0000 7f7dc6c7b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:46:36.905 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:36 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:36.716+0000 7f7dc6c7b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:46:36.905 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:36 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:36.904+0000 7f7dc6c7b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:46:37.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:37.005+0000 7f2389e00640 1 -- 172.21.15.123:0/2637985527 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f23700a3290 msgr2=0x7f23700a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/1532622003 2024-09-19T04:46:37.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:37.005+0000 7f2389e00640 1 --2- 172.21.15.123:0/2637985527 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f23700a3290 0x7f23700a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:46:37.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:37.027+0000 7f2391400640 1 -- 172.21.15.123:0/2637985527 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f238c10c360 con 0x7f238c103ff0 2024-09-19T04:46:37.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:37.027+0000 7f2383400640 1 -- 172.21.15.123:0/2637985527 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f237c09c420 con 0x7f238c103ff0 2024-09-19T04:46:37.029 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T04:46:37.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:37.030+0000 7f2391400640 1 -- 172.21.15.123:0/2637985527 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f23700a3290 msgr2=0x7f23700a5750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:46:37.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:37.030+0000 7f2391400640 1 --2- 172.21.15.123:0/2637985527 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f23700a3290 0x7f23700a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:37.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:37.030+0000 7f2391400640 1 -- 172.21.15.123:0/2637985527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f238c103ff0 msgr2=0x7f238c19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:37.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:37.030+0000 7f2391400640 1 --2- 172.21.15.123:0/2637985527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f238c103ff0 0x7f238c19a620 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f237c00e9f0 tx=0x7f237c00edf0 comp rx=0 tx=0).stop 2024-09-19T04:46:37.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:37.030+0000 7f2391400640 1 -- 172.21.15.123:0/2637985527 shutdown_connections 2024-09-19T04:46:37.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:37.030+0000 7f2391400640 1 --2- 172.21.15.123:0/2637985527 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f23700a3290 0x7f23700a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:37.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:37.030+0000 7f2391400640 1 --2- 172.21.15.123:0/2637985527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f238c109120 0x7f238c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:37.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:37.030+0000 7f2391400640 1 --2- 172.21.15.123:0/2637985527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f238c1049a0 0x7f238c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:37.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:37.030+0000 7f2391400640 1 --2- 172.21.15.123:0/2637985527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f238c103ff0 0x7f238c19a620 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:37.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:37.030+0000 7f2391400640 1 -- 172.21.15.123:0/2637985527 >> 172.21.15.123:0/2637985527 conn(0x7f238c0ffa90 msgr2=0x7f238c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:37.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:37.030+0000 7f2391400640 1 -- 172.21.15.123:0/2637985527 shutdown_connections 2024-09-19T04:46:37.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:37.030+0000 7f2391400640 1 -- 172.21.15.123:0/2637985527 wait complete. 2024-09-19T04:46:37.265 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:36 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:36.969+0000 7f2a9440c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:46:37.265 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:37 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:37.046+0000 7f2a9440c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:46:37.632 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:37 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:37.263+0000 7f2a9440c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:46:37.798 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T04:46:37.798 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T04:46:37.798 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-09-19T04:46:37.798 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T04:46:37.798 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T04:46:37.798 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T04:46:37.798 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T04:46:37.798 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T04:46:37.798 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:46:37.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:46:37.799 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-09-19T04:46:37.835 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:46:37.836 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:46:37.836 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:46:37.836 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:46:37.836 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-09-19T04:46:37.836 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:46:37.836 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T04:46:37.836 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-09-19T04:46:37.836 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2024-09-19T04:46:37.836 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T04:46:37.836 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-09-19T04:46:37.836 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2024-09-19T04:46:37.836 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T04:46:37.836 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:46:37.837 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T04:46:37.837 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:46:37.837 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T04:46:37.837 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42100sec preferred_lft 42100sec 2024-09-19T04:46:37.837 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link noprefixroute 2024-09-19T04:46:37.837 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:46:37.838 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T04:46:37.838 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:46:37.839 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-09-19T04:46:37.839 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add name ceph-brx type bridge 2024-09-19T04:46:37.839 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr flush dev ceph-brx 2024-09-19T04:46:37.839 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set ceph-brx up 2024-09-19T04:46:37.839 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T04:46:37.839 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-09-19T04:46:37.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:37 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2637985527' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T04:46:37.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:46:37.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:37 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2637985527' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T04:46:37.990 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:37 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:37.764+0000 7fe5a0052040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:46:37.991 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:37 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:37.832+0000 7fe5a0052040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:46:38.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:37 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2637985527' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T04:46:38.100 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:37 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:37.788+0000 7f7dc6c7b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:46:38.100 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:37 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:37.856+0000 7f7dc6c7b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:46:38.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:46:38.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:46:38.256 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T04:46:38.314 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-09-19T04:46:38.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:46:38.318 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-09-19T04:46:38.368 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-09-19T04:46:38.368 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T04:46:38.368 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T04:46:38.368 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T04:46:38.368 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T04:46:38.369 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:46:38.369 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-09-19T04:46:38.369 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T04:46:38.369 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T04:46:38.370 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T04:46:38.370 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-09-19T04:46:38.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:46:38.497 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:38 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:38.411+0000 7f2a9440c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:46:38.497 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:38 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:38.495+0000 7f2a9440c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:46:38.795 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:38 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:38.533+0000 7fe5a0052040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:46:38.795 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:38 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:38.648+0000 7fe5a0052040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:46:38.826 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:38 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:38.563+0000 7f7dc6c7b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:46:38.826 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:38 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:38.677+0000 7f7dc6c7b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:46:38.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:46:38.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:46:38.967 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-09-19T04:46:39.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:46:39.003 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2024-09-19T04:46:39.073 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:38 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:38.793+0000 7fe5a0052040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:46:39.073 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:38 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:38.855+0000 7fe5a0052040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:46:39.073 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:39 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:39.004+0000 7fe5a0052040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:46:39.073 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:39 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:39.071+0000 7fe5a0052040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:46:39.080 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:46:39.080 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-09-19T04:46:39.080 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:46:39.080 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T04:46:39.080 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-09-19T04:46:39.100 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:38 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:38.825+0000 7f7dc6c7b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:46:39.100 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:38 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:38.887+0000 7f7dc6c7b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:46:39.100 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:39 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:39.036+0000 7f7dc6c7b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:46:39.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:46:39.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:46:39.261 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T04:46:39.261 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:46:39.261 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-09-19T04:46:39.262 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T04:46:39.262 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T04:46:39.262 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T04:46:39.262 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T04:46:39.262 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T04:46:39.262 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-09-19T04:46:39.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:46:39.353 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:39 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:39.130+0000 7fe5a0052040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:46:39.354 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:39 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:39.287+0000 7fe5a0052040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:46:39.354 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:39 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:39.352+0000 7fe5a0052040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:46:39.390 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:39 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:39.104+0000 7f7dc6c7b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:46:39.390 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:39 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:39.163+0000 7f7dc6c7b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:46:39.390 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:39 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:39.322+0000 7f7dc6c7b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:46:39.390 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:39 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:39.388+0000 7f7dc6c7b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:46:39.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:46:39.511 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:46:39.511 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-09-19T04:46:39.511 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2024-09-19T04:46:39.511 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T04:46:39.511 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-09-19T04:46:39.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:46:39.615 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:39 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:39.418+0000 7fe5a0052040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:46:39.615 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:39 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:39.498+0000 7fe5a0052040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:46:39.615 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:39 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:39.556+0000 7fe5a0052040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:46:39.616 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:39 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:39.614+0000 7fe5a0052040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:46:39.656 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:39 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:39.454+0000 7f7dc6c7b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:46:39.656 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:39 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:39.536+0000 7f7dc6c7b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:46:39.656 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:39 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:39.595+0000 7f7dc6c7b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:46:39.656 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:39 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:39.655+0000 7f7dc6c7b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:46:39.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:46:39.686 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:46:39.686 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T04:46:39.722 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:46:39.723 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:46:39.723 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:46:39.798 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:39 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:39.471+0000 7f2a9440c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:46:39.798 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:39 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:39.614+0000 7f2a9440c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:46:39.798 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:39 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:39.795+0000 7f2a9440c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:46:39.799 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-19T04:46:39.799 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-19T04:46:39.799 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/403e363f-2f5b-43da-b341-82f288e03744 2024-09-19T04:46:39.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:46:39.800 DEBUG:teuthology.orchestra.run.smithi123:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/403e363f-2f5b-43da-b341-82f288e03744 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-19T04:46:39.955 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-19T04:46:39.955 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-19T04:46:40.056 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:39 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:39.871+0000 7f2a9440c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:46:40.057 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:40.054+0000 7f2a9440c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:46:40.338 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:40.137+0000 7f2a9440c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:46:40.338 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:40.210+0000 7f2a9440c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:46:40.601 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-19T04:46:40.601 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-19T04:46:40.601 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.123:6789,172.21.15.183:6789,172.21.15.193:6789" 2024-09-19T04:46:40.601 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with new device syntax: 0@911fa052-7640-11ef-bceb-c7b262605968.cephfs=/volumes/qa/sv_1/403e363f-2f5b-43da-b341-82f288e03744 2024-09-19T04:46:40.601 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.123:6789/172.21.15.183:6789/172.21.15.193:6789" will pass to kernel 2024-09-19T04:46:40.604 INFO:teuthology.orchestra.run.smithi123.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-19T04:46:40.604 INFO:teuthology.orchestra.run.smithi123.stdout: You just mounted a file system that supports labels which does not 2024-09-19T04:46:40.604 INFO:teuthology.orchestra.run.smithi123.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-19T04:46:40.604 INFO:teuthology.orchestra.run.smithi123.stdout: applications will generate AVC messages and not be allowed access to 2024-09-19T04:46:40.604 INFO:teuthology.orchestra.run.smithi123.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-19T04:46:40.607 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-19T04:46:40.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:46:40.607 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:46:40.656 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:40.396+0000 7f2a9440c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:46:40.656 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:40.483+0000 7f2a9440c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:46:40.656 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:40.565+0000 7f2a9440c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:46:40.656 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-mon[30511]: Standby manager daemon z restarted 2024-09-19T04:46:40.656 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-mon[30511]: Standby manager daemon z started 2024-09-19T04:46:40.656 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-mon[30511]: from='mgr.? 172.21.15.193:0/490815833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:46:40.656 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-mon[30511]: from='mgr.? 172.21.15.193:0/490815833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:46:40.656 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-mon[30511]: from='mgr.? 172.21.15.193:0/490815833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:46:40.656 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-mon[30511]: from='mgr.? 172.21.15.193:0/490815833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:46:40.657 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-mon[30511]: Standby manager daemon y restarted 2024-09-19T04:46:40.657 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-mon[30511]: Standby manager daemon y started 2024-09-19T04:46:40.657 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-mon[30511]: from='mgr.? 172.21.15.183:0/2662592060' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:46:40.667 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T04:46:40.740 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:46:40 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z[37404]: 2024-09-19T04:46:40.397+0000 7fe5a0052040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:46:40.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:40 smithi193 ceph-mon[35592]: Standby manager daemon z restarted 2024-09-19T04:46:40.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:40 smithi193 ceph-mon[35592]: Standby manager daemon z started 2024-09-19T04:46:40.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:40 smithi193 ceph-mon[35592]: from='mgr.? 172.21.15.193:0/490815833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:46:40.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:40 smithi193 ceph-mon[35592]: from='mgr.? 172.21.15.193:0/490815833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:46:40.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:40 smithi193 ceph-mon[35592]: from='mgr.? 172.21.15.193:0/490815833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:46:40.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:40 smithi193 ceph-mon[35592]: from='mgr.? 172.21.15.193:0/490815833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:46:40.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:40 smithi193 ceph-mon[35592]: Standby manager daemon y restarted 2024-09-19T04:46:40.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:40 smithi193 ceph-mon[35592]: Standby manager daemon y started 2024-09-19T04:46:40.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:40 smithi193 ceph-mon[35592]: from='mgr.? 172.21.15.183:0/2662592060' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:46:40.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T04:46:40.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:46:40.760 INFO:teuthology.orchestra.run.smithi123.stdout:client25153 2024-09-19T04:46:40.761 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:46:40.761 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:46:40.798 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T04:46:40.850 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:46:40 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y[36694]: 2024-09-19T04:46:40.446+0000 7f7dc6c7b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:46:40.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:40 smithi183 ceph-mon[35333]: Standby manager daemon z restarted 2024-09-19T04:46:40.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:40 smithi183 ceph-mon[35333]: Standby manager daemon z started 2024-09-19T04:46:40.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:40 smithi183 ceph-mon[35333]: from='mgr.? 172.21.15.193:0/490815833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:46:40.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:40 smithi183 ceph-mon[35333]: from='mgr.? 172.21.15.193:0/490815833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:46:40.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:40 smithi183 ceph-mon[35333]: from='mgr.? 172.21.15.193:0/490815833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:46:40.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:40 smithi183 ceph-mon[35333]: from='mgr.? 172.21.15.193:0/490815833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:46:40.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:40 smithi183 ceph-mon[35333]: Standby manager daemon y restarted 2024-09-19T04:46:40.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:40 smithi183 ceph-mon[35333]: Standby manager daemon y started 2024-09-19T04:46:40.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:40 smithi183 ceph-mon[35333]: from='mgr.? 172.21.15.183:0/2662592060' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:46:40.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T04:46:40.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:46:40.927 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:40.669+0000 7f2a9440c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:46:40.927 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:40.743+0000 7f2a9440c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:46:40.927 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:40.818+0000 7f2a9440c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:46:40.929 INFO:teuthology.orchestra.run.smithi123.stdout:client25153 2024-09-19T04:46:40.929 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T04:46:41.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T04:46:41.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:46:41.021 INFO:teuthology.orchestra.run.smithi123.stdout:client25153 2024-09-19T04:46:41.022 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/911fa052-7640-11ef-bceb-c7b262605968.client25153/status') 2024-09-19T04:46:41.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:46:41.123 INFO:teuthology.orchestra.run.smithi123.stdout:instance: client.25153 (3)172.21.15.123:0/4152251600 2024-09-19T04:46:41.123 INFO:teuthology.orchestra.run.smithi123.stdout:blocklisted: false 2024-09-19T04:46:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records in 2024-09-19T04:46:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records out 2024-09-19T04:46:41.124 INFO:teuthology.orchestra.run.smithi123.stderr:72 bytes copied, 7.1463e-05 s, 1.0 MB/s 2024-09-19T04:46:41.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:46:41.134 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T04:46:41.186 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T04:46:41.187 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T04:46:41.187 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-19T04:46:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.341+0000 7f2206200640 1 -- 172.21.15.123:0/3068400195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2200103d60 msgr2=0x7f2200104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.341+0000 7f2206200640 1 --2- 172.21.15.123:0/3068400195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2200103d60 0x7f2200104160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009900 tx=0x7f21f002a400 comp rx=0 tx=0).stop 2024-09-19T04:46:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.341+0000 7f2206200640 1 -- 172.21.15.123:0/3068400195 shutdown_connections 2024-09-19T04:46:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.341+0000 7f2206200640 1 --2- 172.21.15.123:0/3068400195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2200105910 0x7f220010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.341+0000 7f2206200640 1 --2- 172.21.15.123:0/3068400195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2200104f50 0x7f22001053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.341+0000 7f2206200640 1 --2- 172.21.15.123:0/3068400195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2200103d60 0x7f2200104160 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.341+0000 7f2206200640 1 -- 172.21.15.123:0/3068400195 >> 172.21.15.123:0/3068400195 conn(0x7f22000ff4e0 msgr2=0x7f2200101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.341+0000 7f2206200640 1 -- 172.21.15.123:0/3068400195 shutdown_connections 2024-09-19T04:46:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.341+0000 7f2206200640 1 -- 172.21.15.123:0/3068400195 wait complete. 2024-09-19T04:46:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.342+0000 7f2206200640 1 Processor -- start 2024-09-19T04:46:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.342+0000 7f2206200640 1 -- start start 2024-09-19T04:46:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.343+0000 7f2206200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2200103d60 0x7f2200079fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.343+0000 7f2206200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2200104f50 0x7f220007a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.343+0000 7f2206200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2200105910 0x7f2200076b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.343+0000 7f2206200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f220010f9e0 con 0x7f2200104f50 2024-09-19T04:46:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.343+0000 7f2206200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f220010f860 con 0x7f2200103d60 2024-09-19T04:46:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.343+0000 7f2206200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f220010fb60 con 0x7f2200105910 2024-09-19T04:46:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.343+0000 7f21fea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2200104f50 0x7f220007a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.343+0000 7f21fea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2200104f50 0x7f220007a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57554/0 (socket says 172.21.15.123:57554) 2024-09-19T04:46:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.343+0000 7f21fea00640 1 -- 172.21.15.123:0/935745964 learned_addr learned my addr 172.21.15.123:0/935745964 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.343+0000 7f21ff400640 1 --2- 172.21.15.123:0/935745964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2200103d60 0x7f2200079fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.343+0000 7f21ffe00640 1 --2- 172.21.15.123:0/935745964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2200105910 0x7f2200076b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:41.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.343+0000 7f21fea00640 1 -- 172.21.15.123:0/935745964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2200105910 msgr2=0x7f2200076b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:41.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.343+0000 7f21fea00640 1 --2- 172.21.15.123:0/935745964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2200105910 0x7f2200076b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:41.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.343+0000 7f21fea00640 1 -- 172.21.15.123:0/935745964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2200103d60 msgr2=0x7f2200079fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:41.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.343+0000 7f21fea00640 1 --2- 172.21.15.123:0/935745964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2200103d60 0x7f2200079fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:41.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.343+0000 7f21fea00640 1 -- 172.21.15.123:0/935745964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22000773d0 con 0x7f2200104f50 2024-09-19T04:46:41.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.344+0000 7f21fea00640 1 --2- 172.21.15.123:0/935745964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2200104f50 0x7f220007a4e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00e9f0 tx=0x7f21ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:41.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.344+0000 7f21f7e00640 1 -- 172.21.15.123:0/935745964 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec00cd80 con 0x7f2200104f50 2024-09-19T04:46:41.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.344+0000 7f2206200640 1 -- 172.21.15.123:0/935745964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22001a6450 con 0x7f2200104f50 2024-09-19T04:46:41.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.344+0000 7f21f7e00640 1 -- 172.21.15.123:0/935745964 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21ec005500 con 0x7f2200104f50 2024-09-19T04:46:41.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.344+0000 7f21f7e00640 1 -- 172.21.15.123:0/935745964 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec010690 con 0x7f2200104f50 2024-09-19T04:46:41.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.344+0000 7f2206200640 1 -- 172.21.15.123:0/935745964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22001a6860 con 0x7f2200104f50 2024-09-19T04:46:41.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.345+0000 7f2206200640 1 -- 172.21.15.123:0/935745964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f220010b0d0 con 0x7f2200104f50 2024-09-19T04:46:41.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.351+0000 7f21f7e00640 1 -- 172.21.15.123:0/935745964 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 138373+0+0 (secure 0 0 0) 0x7f21ec0046c0 con 0x7f2200104f50 2024-09-19T04:46:41.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.352+0000 7f21f7e00640 1 --2- 172.21.15.123:0/935745964 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f21e40a3340 0x7f21e40a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:41.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.352+0000 7f21ff400640 1 -- 172.21.15.123:0/935745964 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f21e40a3340 msgr2=0x7f21e40a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/1532622003 2024-09-19T04:46:41.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.352+0000 7f21f7e00640 1 -- 172.21.15.123:0/935745964 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f21ec0cffe0 con 0x7f2200104f50 2024-09-19T04:46:41.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.352+0000 7f21ff400640 1 --2- 172.21.15.123:0/935745964 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f21e40a3340 0x7f21e40a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:46:41.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.353+0000 7f21f7e00640 1 -- 172.21.15.123:0/935745964 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f21ec0d0460 con 0x7f2200104f50 2024-09-19T04:46:41.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.553+0000 7f21ff400640 1 -- 172.21.15.123:0/935745964 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f21e40a3340 msgr2=0x7f21e40a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/1532622003 2024-09-19T04:46:41.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.553+0000 7f21ff400640 1 --2- 172.21.15.123:0/935745964 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f21e40a3340 0x7f21e40a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:46:41.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.574+0000 7f2206200640 1 -- 172.21.15.123:0/935745964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f2200062df0 con 0x7f2200104f50 2024-09-19T04:46:41.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:41 smithi193 ceph-mon[35592]: from='mgr.? 172.21.15.183:0/2662592060' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:46:41.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:41 smithi193 ceph-mon[35592]: from='mgr.? 172.21.15.183:0/2662592060' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:46:41.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:41 smithi193 ceph-mon[35592]: from='mgr.? 172.21.15.183:0/2662592060' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:46:41.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:41 smithi193 ceph-mon[35592]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-09-19T04:46:41.773 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:41 smithi123 ceph-mon[30511]: from='mgr.? 172.21.15.183:0/2662592060' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:46:41.773 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:41 smithi123 ceph-mon[30511]: from='mgr.? 172.21.15.183:0/2662592060' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:46:41.773 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:41 smithi123 ceph-mon[30511]: from='mgr.? 172.21.15.183:0/2662592060' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:46:41.774 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:41 smithi123 ceph-mon[30511]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-09-19T04:46:41.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.786+0000 7f21f7e00640 1 -- 172.21.15.123:0/935745964 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 21) ==== 138040+0+0 (secure 0 0 0) 0x7f21ec098c20 con 0x7f2200104f50 2024-09-19T04:46:41.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.786+0000 7f21f7e00640 1 -- 172.21.15.123:0/935745964 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f21e40a3340 msgr2=0x7f21e40a5800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:46:41.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:41.786+0000 7f21f7e00640 1 --2- 172.21.15.123:0/935745964 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f21e40a3340 0x7f21e40a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:41.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:41 smithi183 ceph-mon[35333]: from='mgr.? 172.21.15.183:0/2662592060' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:46:41.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:41 smithi183 ceph-mon[35333]: from='mgr.? 172.21.15.183:0/2662592060' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:46:41.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:41 smithi183 ceph-mon[35333]: from='mgr.? 172.21.15.183:0/2662592060' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:46:41.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:41 smithi183 ceph-mon[35333]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-09-19T04:46:42.043 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:41 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:41.770+0000 7f2a9440c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:46:42.427 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:42.077+0000 7f29f0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:46:42.427 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:42.077+0000 7f29f0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:46:42.427 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:42.077+0000 7f29f0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:46:42.427 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:42.077+0000 7f29f0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:46:42.428 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:42.077+0000 7f29f0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:46:42.428 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:42.078+0000 7f29eb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:46:42.428 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:42.078+0000 7f29eb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:46:42.428 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:42.078+0000 7f29eb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:46:42.428 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:42.078+0000 7f29eb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:46:42.428 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:42.078+0000 7f29eb600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:46:42.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.473+0000 7f21f7e00640 1 -- 172.21.15.123:0/935745964 <== mon.0 v2:172.21.15.123:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7f21ec09ccb0 con 0x7f2200104f50 2024-09-19T04:46:42.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.476+0000 7f2206200640 1 -- 172.21.15.123:0/935745964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2200104f50 msgr2=0x7f220007a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:42.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.476+0000 7f2206200640 1 --2- 172.21.15.123:0/935745964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2200104f50 0x7f220007a4e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00e9f0 tx=0x7f21ec00edf0 comp rx=0 tx=0).stop 2024-09-19T04:46:42.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.477+0000 7f2206200640 1 -- 172.21.15.123:0/935745964 shutdown_connections 2024-09-19T04:46:42.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.477+0000 7f2206200640 1 --2- 172.21.15.123:0/935745964 >> [v2:172.21.15.123:6800/1532622003,v1:172.21.15.123:6801/1532622003] conn(0x7f21e40a3340 0x7f21e40a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:42.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.477+0000 7f2206200640 1 --2- 172.21.15.123:0/935745964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2200105910 0x7f2200076b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:42.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.477+0000 7f2206200640 1 --2- 172.21.15.123:0/935745964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2200104f50 0x7f220007a4e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:42.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.477+0000 7f2206200640 1 --2- 172.21.15.123:0/935745964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2200103d60 0x7f2200079fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:42.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.477+0000 7f2206200640 1 -- 172.21.15.123:0/935745964 >> 172.21.15.123:0/935745964 conn(0x7f22000ff4e0 msgr2=0x7f22001082a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:42.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.477+0000 7f2206200640 1 -- 172.21.15.123:0/935745964 shutdown_connections 2024-09-19T04:46:42.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.477+0000 7f2206200640 1 -- 172.21.15.123:0/935745964 wait complete. 2024-09-19T04:46:42.498 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-19T04:46:42.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.653+0000 7fade3600640 1 -- 172.21.15.123:0/4221741310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faddc103d60 msgr2=0x7faddc104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:42.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.653+0000 7fade3600640 1 --2- 172.21.15.123:0/4221741310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faddc103d60 0x7faddc104160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009900 tx=0x7fadd002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:46:42.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.653+0000 7fade3600640 1 -- 172.21.15.123:0/4221741310 shutdown_connections 2024-09-19T04:46:42.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.653+0000 7fade3600640 1 --2- 172.21.15.123:0/4221741310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc105910 0x7faddc10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:42.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.653+0000 7fade3600640 1 --2- 172.21.15.123:0/4221741310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faddc104f50 0x7faddc1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:42.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.653+0000 7fade3600640 1 --2- 172.21.15.123:0/4221741310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faddc103d60 0x7faddc104160 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:42.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.653+0000 7fade3600640 1 -- 172.21.15.123:0/4221741310 >> 172.21.15.123:0/4221741310 conn(0x7faddc0ff4e0 msgr2=0x7faddc101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:42.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.653+0000 7fade3600640 1 -- 172.21.15.123:0/4221741310 shutdown_connections 2024-09-19T04:46:42.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.653+0000 7fade3600640 1 -- 172.21.15.123:0/4221741310 wait complete. 2024-09-19T04:46:42.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.654+0000 7fade3600640 1 Processor -- start 2024-09-19T04:46:42.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.654+0000 7fade3600640 1 -- start start 2024-09-19T04:46:42.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.655+0000 7fade3600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc103d60 0x7faddc199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:42.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.655+0000 7fade3600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faddc104f50 0x7faddc19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:42.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.655+0000 7fade3600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faddc105910 0x7faddc1a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:42.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.655+0000 7fade3600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faddc10fa10 con 0x7faddc105910 2024-09-19T04:46:42.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.655+0000 7fade3600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faddc10f890 con 0x7faddc104f50 2024-09-19T04:46:42.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.655+0000 7fade3600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faddc10fb90 con 0x7faddc103d60 2024-09-19T04:46:42.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.655+0000 7fade0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc103d60 0x7faddc199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:42.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.655+0000 7faddbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faddc104f50 0x7faddc19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:42.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.655+0000 7fade0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc103d60 0x7faddc199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:46754/0 (socket says 172.21.15.123:46754) 2024-09-19T04:46:42.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.655+0000 7fade1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faddc105910 0x7faddc1a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:42.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.655+0000 7faddbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faddc104f50 0x7faddc19a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:34418/0 (socket says 172.21.15.123:34418) 2024-09-19T04:46:42.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.655+0000 7fade0c00640 1 -- 172.21.15.123:0/1550030248 learned_addr learned my addr 172.21.15.123:0/1550030248 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:42.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.655+0000 7fade0c00640 1 -- 172.21.15.123:0/1550030248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faddc104f50 msgr2=0x7faddc19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:42.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.655+0000 7fade0c00640 1 --2- 172.21.15.123:0/1550030248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faddc104f50 0x7faddc19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:42.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.655+0000 7fade0c00640 1 -- 172.21.15.123:0/1550030248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faddc105910 msgr2=0x7faddc1a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:42.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.655+0000 7fade0c00640 1 --2- 172.21.15.123:0/1550030248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faddc105910 0x7faddc1a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:42.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.655+0000 7fade0c00640 1 -- 172.21.15.123:0/1550030248 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faddc1a1bf0 con 0x7faddc103d60 2024-09-19T04:46:42.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.656+0000 7fade0c00640 1 --2- 172.21.15.123:0/1550030248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc103d60 0x7faddc199fd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fadd0031440 tx=0x7fadd0032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:42.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.656+0000 7fadd9600640 1 -- 172.21.15.123:0/1550030248 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd003f030 con 0x7faddc103d60 2024-09-19T04:46:42.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.656+0000 7fade3600640 1 -- 172.21.15.123:0/1550030248 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faddc1a1e80 con 0x7faddc103d60 2024-09-19T04:46:42.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.656+0000 7fadd9600640 1 -- 172.21.15.123:0/1550030248 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadd0003ce0 con 0x7faddc103d60 2024-09-19T04:46:42.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.656+0000 7fadd9600640 1 -- 172.21.15.123:0/1550030248 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0035070 con 0x7faddc103d60 2024-09-19T04:46:42.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.656+0000 7fade3600640 1 -- 172.21.15.123:0/1550030248 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faddc1a22e0 con 0x7faddc103d60 2024-09-19T04:46:42.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.657+0000 7fade3600640 1 -- 172.21.15.123:0/1550030248 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faddc10b080 con 0x7faddc103d60 2024-09-19T04:46:42.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.659+0000 7fadd9600640 1 -- 172.21.15.123:0/1550030248 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 138040+0+0 (secure 0 0 0) 0x7fadd0042030 con 0x7faddc103d60 2024-09-19T04:46:42.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.661+0000 7fadd9600640 1 -- 172.21.15.123:0/1550030248 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fadd00efc50 con 0x7faddc103d60 2024-09-19T04:46:42.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.666+0000 7fadd9600640 1 -- 172.21.15.123:0/1550030248 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fadd00bcc20 con 0x7faddc103d60 2024-09-19T04:46:42.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/935745964' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T04:46:42.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: Active manager daemon x restarted 2024-09-19T04:46:42.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: Activating manager daemon x 2024-09-19T04:46:42.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T04:46:42.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: mgrmap e21: x(active, starting, since 0.014009s), standbys: y, z 2024-09-19T04:46:42.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:46:42.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:46:42.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:46:42.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:46:42.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:46:42.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:46:42.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:46:42.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:46:42.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:46:42.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:46:42.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:46:42.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:46:42.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:46:42.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:46:42.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:46:42.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:46:42.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:46:42.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:46:42.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:46:42.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:46:42.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:46:42.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:46:42.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:46:42.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:46:42.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:46:42.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:46:42.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:46:42.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:46:42.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:46:42.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:46:42.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:46:42.744 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:46:42.744 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:46:42.744 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: Manager daemon x is now available 2024-09-19T04:46:42.744 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:46:42.744 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:46:42.744 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:46:42.744 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:46:42.744 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:42 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:46:42.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.794+0000 7fadd9600640 1 -- 172.21.15.123:0/1550030248 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7fadd003b460 con 0x7faddc103d60 2024-09-19T04:46:42.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.795+0000 7fadd9600640 1 --2- 172.21.15.123:0/1550030248 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fadc40aa070 0x7fadc40ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:42.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.797+0000 7faddbe00640 1 --2- 172.21.15.123:0/1550030248 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fadc40aa070 0x7fadc40ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:42.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.803+0000 7faddbe00640 1 --2- 172.21.15.123:0/1550030248 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fadc40aa070 0x7fadc40ac460 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faddc19b4e0 tx=0x7fadc8005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/935745964' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T04:46:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: Active manager daemon x restarted 2024-09-19T04:46:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: Activating manager daemon x 2024-09-19T04:46:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T04:46:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: mgrmap e21: x(active, starting, since 0.014009s), standbys: y, z 2024-09-19T04:46:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:46:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:46:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:46:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:46:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:46:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:46:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:46:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:46:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:46:42.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:46:42.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:46:42.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:46:42.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:46:42.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:46:42.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:46:42.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:46:42.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:46:42.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:46:42.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:46:42.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:46:42.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:46:42.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:46:42.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:46:42.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:46:42.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:46:42.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:46:42.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:46:42.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:46:42.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:46:42.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:46:42.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:46:42.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:46:42.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:46:42.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: Manager daemon x is now available 2024-09-19T04:46:42.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:46:42.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:46:42.854 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:46:42.854 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:46:42.854 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:42 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:46:42.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/935745964' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T04:46:42.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: Active manager daemon x restarted 2024-09-19T04:46:42.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: Activating manager daemon x 2024-09-19T04:46:42.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T04:46:42.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: mgrmap e21: x(active, starting, since 0.014009s), standbys: y, z 2024-09-19T04:46:42.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:46:42.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:46:42.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:46:42.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:46:42.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:46:42.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:46:42.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:46:42.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:46:42.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:46:42.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:46:42.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:46:42.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:46:42.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:46:42.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:46:42.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:46:42.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:46:42.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:46:42.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:46:42.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:46:42.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:46:42.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:46:42.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:46:42.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:46:42.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:46:42.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:46:42.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:46:42.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:46:42.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:46:42.931 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:46:42.931 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:46:42.931 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:46:42.931 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:46:42.931 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:46:42.931 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: Manager daemon x is now available 2024-09-19T04:46:42.931 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:46:42.931 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:46:42.931 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:46:42.931 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:46:42.931 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:46:42.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.925+0000 7fade3600640 1 -- 172.21.15.123:0/1550030248 --> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7faddc0626d0 con 0x7fadc40aa070 2024-09-19T04:46:42.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.943+0000 7fadd9600640 1 -- 172.21.15.123:0/1550030248 <== mgr.25133 v2:172.21.15.123:6800/1578549821 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7faddc0626d0 con 0x7fadc40aa070 2024-09-19T04:46:42.946 INFO:teuthology.orchestra.run.smithi123.stdout:{} 2024-09-19T04:46:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.945+0000 7fade3600640 1 -- 172.21.15.123:0/1550030248 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fadc40aa070 msgr2=0x7fadc40ac460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.945+0000 7fade3600640 1 --2- 172.21.15.123:0/1550030248 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fadc40aa070 0x7fadc40ac460 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faddc19b4e0 tx=0x7fadc8005f90 comp rx=0 tx=0).stop 2024-09-19T04:46:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.945+0000 7fade3600640 1 -- 172.21.15.123:0/1550030248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc103d60 msgr2=0x7faddc199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.945+0000 7fade3600640 1 --2- 172.21.15.123:0/1550030248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc103d60 0x7faddc199fd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fadd0031440 tx=0x7fadd0032ce0 comp rx=0 tx=0).stop 2024-09-19T04:46:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.945+0000 7fade3600640 1 -- 172.21.15.123:0/1550030248 shutdown_connections 2024-09-19T04:46:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.945+0000 7fade3600640 1 --2- 172.21.15.123:0/1550030248 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fadc40aa070 0x7fadc40ac460 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.945+0000 7fade3600640 1 --2- 172.21.15.123:0/1550030248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faddc105910 0x7faddc1a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.945+0000 7fade3600640 1 --2- 172.21.15.123:0/1550030248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faddc104f50 0x7faddc19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.945+0000 7fade3600640 1 --2- 172.21.15.123:0/1550030248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc103d60 0x7faddc199fd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.945+0000 7fade3600640 1 -- 172.21.15.123:0/1550030248 >> 172.21.15.123:0/1550030248 conn(0x7faddc0ff4e0 msgr2=0x7faddc108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:42.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.946+0000 7fade3600640 1 -- 172.21.15.123:0/1550030248 shutdown_connections 2024-09-19T04:46:42.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:42.946+0000 7fade3600640 1 -- 172.21.15.123:0/1550030248 wait complete. 2024-09-19T04:46:42.964 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-19T04:46:42.979 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-19T04:46:43.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:46:43.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.163+0000 7fb516200640 1 -- 172.21.15.123:0/3350938533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb510077df0 msgr2=0x7fb510078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:43.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.163+0000 7fb516200640 1 --2- 172.21.15.123:0/3350938533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb510077df0 0x7fb510078250 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009900 tx=0x7fb4fc02a410 comp rx=0 tx=0).stop 2024-09-19T04:46:43.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.163+0000 7fb516200640 1 -- 172.21.15.123:0/3350938533 shutdown_connections 2024-09-19T04:46:43.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.163+0000 7fb516200640 1 --2- 172.21.15.123:0/3350938533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb510078790 0x7fb510079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.163+0000 7fb516200640 1 --2- 172.21.15.123:0/3350938533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb510077df0 0x7fb510078250 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.163+0000 7fb516200640 1 --2- 172.21.15.123:0/3350938533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb510076bb0 0x7fb510076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.163+0000 7fb516200640 1 -- 172.21.15.123:0/3350938533 >> 172.21.15.123:0/3350938533 conn(0x7fb5100ff4b0 msgr2=0x7fb5101018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:43.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.163+0000 7fb516200640 1 -- 172.21.15.123:0/3350938533 shutdown_connections 2024-09-19T04:46:43.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.163+0000 7fb516200640 1 -- 172.21.15.123:0/3350938533 wait complete. 2024-09-19T04:46:43.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.164+0000 7fb516200640 1 Processor -- start 2024-09-19T04:46:43.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.164+0000 7fb516200640 1 -- start start 2024-09-19T04:46:43.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.164+0000 7fb516200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb510076bb0 0x7fb51019e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:43.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.164+0000 7fb516200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb510077df0 0x7fb51019e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:43.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.164+0000 7fb516200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb510078790 0x7fb5101a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:43.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.164+0000 7fb516200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb510113e10 con 0x7fb510076bb0 2024-09-19T04:46:43.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.164+0000 7fb516200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb510113c90 con 0x7fb510078790 2024-09-19T04:46:43.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.164+0000 7fb516200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb510113f90 con 0x7fb510077df0 2024-09-19T04:46:43.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.164+0000 7fb50f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb510076bb0 0x7fb51019e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:43.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.164+0000 7fb50f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb510076bb0 0x7fb51019e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50538/0 (socket says 172.21.15.123:50538) 2024-09-19T04:46:43.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.164+0000 7fb50f400640 1 -- 172.21.15.123:0/3423045399 learned_addr learned my addr 172.21.15.123:0/3423045399 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:43.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.164+0000 7fb50fe00640 1 --2- 172.21.15.123:0/3423045399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb510078790 0x7fb5101a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:43.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.164+0000 7fb50ea00640 1 --2- 172.21.15.123:0/3423045399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb510077df0 0x7fb51019e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:43.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.165+0000 7fb50f400640 1 -- 172.21.15.123:0/3423045399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb510077df0 msgr2=0x7fb51019e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:43.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.165+0000 7fb50f400640 1 --2- 172.21.15.123:0/3423045399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb510077df0 0x7fb51019e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.165+0000 7fb50f400640 1 -- 172.21.15.123:0/3423045399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb510078790 msgr2=0x7fb5101a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:43.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.165+0000 7fb50f400640 1 --2- 172.21.15.123:0/3423045399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb510078790 0x7fb5101a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.165+0000 7fb50f400640 1 -- 172.21.15.123:0/3423045399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5101a60c0 con 0x7fb510076bb0 2024-09-19T04:46:43.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.165+0000 7fb50fe00640 1 --2- 172.21.15.123:0/3423045399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb510078790 0x7fb5101a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:43.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.165+0000 7fb50f400640 1 --2- 172.21.15.123:0/3423045399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb510076bb0 0x7fb51019e4a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb50000e9f0 tx=0x7fb50000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:43.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.165+0000 7fb507e00640 1 -- 172.21.15.123:0/3423045399 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50000ccc0 con 0x7fb510076bb0 2024-09-19T04:46:43.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.165+0000 7fb507e00640 1 -- 172.21.15.123:0/3423045399 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb50000ce60 con 0x7fb510076bb0 2024-09-19T04:46:43.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.165+0000 7fb516200640 1 -- 172.21.15.123:0/3423045399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5101a63b0 con 0x7fb510076bb0 2024-09-19T04:46:43.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.165+0000 7fb507e00640 1 -- 172.21.15.123:0/3423045399 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500010690 con 0x7fb510076bb0 2024-09-19T04:46:43.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.165+0000 7fb516200640 1 -- 172.21.15.123:0/3423045399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5101a6880 con 0x7fb510076bb0 2024-09-19T04:46:43.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.167+0000 7fb516200640 1 -- 172.21.15.123:0/3423045399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb510110850 con 0x7fb510076bb0 2024-09-19T04:46:43.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.168+0000 7fb507e00640 1 -- 172.21.15.123:0/3423045399 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7fb5000046c0 con 0x7fb510076bb0 2024-09-19T04:46:43.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.170+0000 7fb507e00640 1 --2- 172.21.15.123:0/3423045399 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb4f40a3050 0x7fb4f40a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:43.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.170+0000 7fb507e00640 1 -- 172.21.15.123:0/3423045399 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fb5000cfa60 con 0x7fb510076bb0 2024-09-19T04:46:43.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.170+0000 7fb50ea00640 1 --2- 172.21.15.123:0/3423045399 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb4f40a3050 0x7fb4f40a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:43.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.171+0000 7fb50ea00640 1 --2- 172.21.15.123:0/3423045399 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb4f40a3050 0x7fb4f40a5510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc031440 tx=0x7fb4fc0315b0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:43.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.174+0000 7fb507e00640 1 -- 172.21.15.123:0/3423045399 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb50009c940 con 0x7fb510076bb0 2024-09-19T04:46:43.302 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:42.932+0000 7f29e8e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:46:43.302 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:42.932+0000 7f29e8e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:46:43.302 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:42.932+0000 7f29e8e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:46:43.302 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:42.932+0000 7f29e8e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:46:43.302 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:42.932+0000 7f29e8e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:46:43.302 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:43.007+0000 7f29d4400640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:04:46:43] ENGINE Error in HTTPServer.serve 2024-09-19T04:46:43.303 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: Traceback (most recent call last): 2024-09-19T04:46:43.303 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:46:43.303 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: self._connections.run(self.expiration_interval) 2024-09-19T04:46:43.303 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:46:43.303 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: self._run(expiration_interval) 2024-09-19T04:46:43.303 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:46:43.303 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:46:43.303 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:46:43.303 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:46:43.303 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:46:43.303 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: s = self.context.wrap_socket( 2024-09-19T04:46:43.304 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:46:43.304 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: return self.sslsocket_class._create( 2024-09-19T04:46:43.304 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:46:43.304 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: self.do_handshake() 2024-09-19T04:46:43.304 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:46:43.304 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: self._sslobj.do_handshake() 2024-09-19T04:46:43.304 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:46:43.304 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:46:43.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.356+0000 7fb516200640 1 -- 172.21.15.123:0/3423045399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb510062df0 con 0x7fb510076bb0 2024-09-19T04:46:43.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.360+0000 7fb507e00640 1 -- 172.21.15.123:0/3423045399 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+23178 (secure 0 0 0) 0x7fb50009c240 con 0x7fb510076bb0 2024-09-19T04:46:43.361 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:46:43.362 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":87,"fsid":"911fa052-7640-11ef-bceb-c7b262605968","created":"2024-09-19T04:36:21.083333+0000","modified":"2024-09-19T04:46:41.775142+0000","last_up_change":"2024-09-19T04:43:33.818439+0000","last_in_change":"2024-09-19T04:43:10.148272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:39:33.856981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:44:26.956483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:44:29.718625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6803","nonce":3631596501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6805","nonce":3631596501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6809","nonce":3631596501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6807","nonce":3631596501}]},"public_addr":"172.21.15.123:6803/3631596501","cluster_addr":"172.21.15.123:6805/3631596501","heartbeat_back_addr":"172.21.15.123:6809/3631596501","heartbeat_front_addr":"172.21.15.123:6807/3631596501","state":["exists","up"]},{"osd":1,"uuid":"a526d4ce-586b-4d66-b720-e868df92fa24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6801","nonce":1259721614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6803","nonce":1259721614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6807","nonce":1259721614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6805","nonce":1259721614}]},"public_addr":"172.21.15.183:6801/1259721614","cluster_addr":"172.21.15.183:6803/1259721614","heartbeat_back_addr":"172.21.15.183:6807/1259721614","heartbeat_front_addr":"172.21.15.183:6805/1259721614","state":["exists","up"]},{"osd":2,"uuid":"3650601a-6259-4b2a-97df-1d47b09b0127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6801","nonce":2708151726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6803","nonce":2708151726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6807","nonce":2708151726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6805","nonce":2708151726}]},"public_addr":"172.21.15.193:6801/2708151726","cluster_addr":"172.21.15.193:6803/2708151726","heartbeat_back_addr":"172.21.15.193:6807/2708151726","heartbeat_front_addr":"172.21.15.193:6805/2708151726","state":["exists","up"]},{"osd":3,"uuid":"67c485ad-136e-4020-950d-9171a3c2de35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6811","nonce":3214806282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6813","nonce":3214806282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6817","nonce":3214806282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6815","nonce":3214806282}]},"public_addr":"172.21.15.123:6811/3214806282","cluster_addr":"172.21.15.123:6813/3214806282","heartbeat_back_addr":"172.21.15.123:6817/3214806282","heartbeat_front_addr":"172.21.15.123:6815/3214806282","state":["exists","up"]},{"osd":4,"uuid":"c9098540-06ab-45a8-9cf9-f90519cc852e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6809","nonce":667618665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6811","nonce":667618665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6815","nonce":667618665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6813","nonce":667618665}]},"public_addr":"172.21.15.183:6809/667618665","cluster_addr":"172.21.15.183:6811/667618665","heartbeat_back_addr":"172.21.15.183:6815/667618665","heartbeat_front_addr":"172.21.15.183:6813/667618665","state":["exists","up"]},{"osd":5,"uuid":"278a8dbd-6492-42d4-b489-1f832350bc6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6809","nonce":2292251639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6811","nonce":2292251639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6815","nonce":2292251639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6813","nonce":2292251639}]},"public_addr":"172.21.15.193:6809/2292251639","cluster_addr":"172.21.15.193:6811/2292251639","heartbeat_back_addr":"172.21.15.193:6815/2292251639","heartbeat_front_addr":"172.21.15.193:6813/2292251639","state":["exists","up"]},{"osd":6,"uuid":"0fda4e39-6adb-4941-a39b-a9a15019088c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6819","nonce":226007499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6821","nonce":226007499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6825","nonce":226007499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6823","nonce":226007499}]},"public_addr":"172.21.15.123:6819/226007499","cluster_addr":"172.21.15.123:6821/226007499","heartbeat_back_addr":"172.21.15.123:6825/226007499","heartbeat_front_addr":"172.21.15.123:6823/226007499","state":["exists","up"]},{"osd":7,"uuid":"5c0a021e-9f3b-4766-9686-0ef24b68cf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6817","nonce":3718518226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6819","nonce":3718518226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6823","nonce":3718518226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6821","nonce":3718518226}]},"public_addr":"172.21.15.183:6817/3718518226","cluster_addr":"172.21.15.183:6819/3718518226","heartbeat_back_addr":"172.21.15.183:6823/3718518226","heartbeat_front_addr":"172.21.15.183:6821/3718518226","state":["exists","up"]},{"osd":8,"uuid":"9b0e67e0-6305-4f81-9211-d41960099a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6817","nonce":3551404118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6819","nonce":3551404118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6823","nonce":3551404118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6821","nonce":3551404118}]},"public_addr":"172.21.15.193:6817/3551404118","cluster_addr":"172.21.15.193:6819/3551404118","heartbeat_back_addr":"172.21.15.193:6823/3551404118","heartbeat_front_addr":"172.21.15.193:6821/3551404118","state":["exists","up"]},{"osd":9,"uuid":"978ef3fa-c6ec-421f-b194-56d44813be5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6827","nonce":1439901149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6829","nonce":1439901149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6833","nonce":1439901149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6831","nonce":1439901149}]},"public_addr":"172.21.15.123:6827/1439901149","cluster_addr":"172.21.15.123:6829/1439901149","heartbeat_back_addr":"172.21.15.123:6833/1439901149","heartbeat_front_addr":"172.21.15.123:6831/1439901149","state":["exists","up"]},{"osd":10,"uuid":"5495e09b-7809-47b0-b7e0-b56c452f9f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6825","nonce":3355349245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6827","nonce":3355349245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6831","nonce":3355349245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6829","nonce":3355349245}]},"public_addr":"172.21.15.183:6825/3355349245","cluster_addr":"172.21.15.183:6827/3355349245","heartbeat_back_addr":"172.21.15.183:6831/3355349245","heartbeat_front_addr":"172.21.15.183:6829/3355349245","state":["exists","up"]},{"osd":11,"uuid":"03046fef-b16b-42f8-9a19-6c0b78cbfe7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6825","nonce":4291617038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6827","nonce":4291617038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6831","nonce":4291617038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6829","nonce":4291617038}]},"public_addr":"172.21.15.193:6825/4291617038","cluster_addr":"172.21.15.193:6827/4291617038","heartbeat_back_addr":"172.21.15.193:6831/4291617038","heartbeat_front_addr":"172.21.15.193:6829/4291617038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:38:39.142529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:05.336911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:30.310967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:59.655973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:25.890576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:50.778689+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:19.063731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:45.424269+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:10.562149+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:40.382368+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:06.297731+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:32.352172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/1532622003":"2024-09-20T04:46:41.775112+0000","172.21.15.123:0/2833919666":"2024-09-20T04:46:41.775112+0000","172.21.15.123:0/2580747029":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/471028951":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:6800/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6800/1532622003":"2024-09-20T04:46:41.775112+0000","172.21.15.123:6801/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/3498196650":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/4040401634":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/2000593686":"2024-09-20T04:36:45.913060+0000","172.21.15.123:6800/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/3339335756":"2024-09-20T04:46:41.775112+0000","172.21.15.123:0/1009496020":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/3244738342":"2024-09-20T04:46:41.775112+0000","172.21.15.123:0/2764863023":"2024-09-20T04:46:41.775112+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:46:43.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.361+0000 7fb516200640 1 -- 172.21.15.123:0/3423045399 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb4f40a3050 msgr2=0x7fb4f40a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:43.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.361+0000 7fb516200640 1 --2- 172.21.15.123:0/3423045399 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb4f40a3050 0x7fb4f40a5510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc031440 tx=0x7fb4fc0315b0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.361+0000 7fb516200640 1 -- 172.21.15.123:0/3423045399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb510076bb0 msgr2=0x7fb51019e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:43.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.361+0000 7fb516200640 1 --2- 172.21.15.123:0/3423045399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb510076bb0 0x7fb51019e4a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb50000e9f0 tx=0x7fb50000edf0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.361+0000 7fb516200640 1 -- 172.21.15.123:0/3423045399 shutdown_connections 2024-09-19T04:46:43.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.361+0000 7fb516200640 1 --2- 172.21.15.123:0/3423045399 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb4f40a3050 0x7fb4f40a5510 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.361+0000 7fb516200640 1 --2- 172.21.15.123:0/3423045399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb510078790 0x7fb5101a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.361+0000 7fb516200640 1 --2- 172.21.15.123:0/3423045399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb510077df0 0x7fb51019e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.361+0000 7fb516200640 1 --2- 172.21.15.123:0/3423045399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb510076bb0 0x7fb51019e4a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.361+0000 7fb516200640 1 -- 172.21.15.123:0/3423045399 >> 172.21.15.123:0/3423045399 conn(0x7fb5100ff4b0 msgr2=0x7fb510100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:43.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.361+0000 7fb516200640 1 -- 172.21.15.123:0/3423045399 shutdown_connections 2024-09-19T04:46:43.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.361+0000 7fb516200640 1 -- 172.21.15.123:0/3423045399 wait complete. 2024-09-19T04:46:43.380 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:39:33.856981+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T04:44:26.956483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T04:44:29.718625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T04:46:43.380 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T04:46:43.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.542+0000 7ff84d200640 1 -- 172.21.15.123:0/1676182397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848103d20 msgr2=0x7ff848104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:43.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.542+0000 7ff84d200640 1 --2- 172.21.15.123:0/1676182397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848103d20 0x7ff848104120 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff838009860 tx=0x7ff83802a340 comp rx=0 tx=0).stop 2024-09-19T04:46:43.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.542+0000 7ff84d200640 1 -- 172.21.15.123:0/1676182397 shutdown_connections 2024-09-19T04:46:43.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.542+0000 7ff84d200640 1 --2- 172.21.15.123:0/1676182397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8481058d0 0x7ff84810c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.542+0000 7ff84d200640 1 --2- 172.21.15.123:0/1676182397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff848104f10 0x7ff848105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.542+0000 7ff84d200640 1 --2- 172.21.15.123:0/1676182397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848103d20 0x7ff848104120 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.542+0000 7ff84d200640 1 -- 172.21.15.123:0/1676182397 >> 172.21.15.123:0/1676182397 conn(0x7ff8480ff4a0 msgr2=0x7ff8481018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:43.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.542+0000 7ff84d200640 1 -- 172.21.15.123:0/1676182397 shutdown_connections 2024-09-19T04:46:43.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.542+0000 7ff84d200640 1 -- 172.21.15.123:0/1676182397 wait complete. 2024-09-19T04:46:43.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.543+0000 7ff84d200640 1 Processor -- start 2024-09-19T04:46:43.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.543+0000 7ff84d200640 1 -- start start 2024-09-19T04:46:43.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff84d200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff848103d20 0x7ff84819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:43.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff84d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848104f10 0x7ff84819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:43.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff84d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8481058d0 0x7ff8481a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:43.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff84d200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff84810f9a0 con 0x7ff8481058d0 2024-09-19T04:46:43.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff84d200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff84810f820 con 0x7ff848104f10 2024-09-19T04:46:43.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff84d200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff84810fb20 con 0x7ff848103d20 2024-09-19T04:46:43.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff845e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff848103d20 0x7ff84819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:43.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff83d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848104f10 0x7ff84819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:43.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff83d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848104f10 0x7ff84819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:34474/0 (socket says 172.21.15.123:34474) 2024-09-19T04:46:43.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff83d400640 1 -- 172.21.15.123:0/735255883 learned_addr learned my addr 172.21.15.123:0/735255883 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:43.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff845e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff848103d20 0x7ff84819a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:46792/0 (socket says 172.21.15.123:46792) 2024-09-19T04:46:43.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff846800640 1 --2- 172.21.15.123:0/735255883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8481058d0 0x7ff8481a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:43.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff845e00640 1 -- 172.21.15.123:0/735255883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848104f10 msgr2=0x7ff84819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:43.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff845e00640 1 --2- 172.21.15.123:0/735255883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848104f10 0x7ff84819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff845e00640 1 -- 172.21.15.123:0/735255883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8481058d0 msgr2=0x7ff8481a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:43.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff845e00640 1 --2- 172.21.15.123:0/735255883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8481058d0 0x7ff8481a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff845e00640 1 -- 172.21.15.123:0/735255883 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8481a1cd0 con 0x7ff848103d20 2024-09-19T04:46:43.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff83d400640 1 --2- 172.21.15.123:0/735255883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848104f10 0x7ff84819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:43.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.544+0000 7ff846800640 1 --2- 172.21.15.123:0/735255883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8481058d0 0x7ff8481a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:43.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.545+0000 7ff845e00640 1 --2- 172.21.15.123:0/735255883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff848103d20 0x7ff84819a0b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff83802a860 tx=0x7ff838033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:43.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.545+0000 7ff83f400640 1 -- 172.21.15.123:0/735255883 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838037040 con 0x7ff848103d20 2024-09-19T04:46:43.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.545+0000 7ff84d200640 1 -- 172.21.15.123:0/735255883 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8481a1f60 con 0x7ff848103d20 2024-09-19T04:46:43.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.545+0000 7ff83f400640 1 -- 172.21.15.123:0/735255883 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff838030850 con 0x7ff848103d20 2024-09-19T04:46:43.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.545+0000 7ff83f400640 1 -- 172.21.15.123:0/735255883 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8380046c0 con 0x7ff848103d20 2024-09-19T04:46:43.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.545+0000 7ff84d200640 1 -- 172.21.15.123:0/735255883 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8481a2360 con 0x7ff848103d20 2024-09-19T04:46:43.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.546+0000 7ff84d200640 1 -- 172.21.15.123:0/735255883 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84810b150 con 0x7ff848103d20 2024-09-19T04:46:43.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.548+0000 7ff83f400640 1 -- 172.21.15.123:0/735255883 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7ff838004050 con 0x7ff848103d20 2024-09-19T04:46:43.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.549+0000 7ff83f400640 1 --2- 172.21.15.123:0/735255883 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff82c0a3050 0x7ff82c0a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:43.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.549+0000 7ff83f400640 1 -- 172.21.15.123:0/735255883 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7ff8380ef3d0 con 0x7ff848103d20 2024-09-19T04:46:43.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.549+0000 7ff83d400640 1 --2- 172.21.15.123:0/735255883 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff82c0a3050 0x7ff82c0a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:43.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.550+0000 7ff83d400640 1 --2- 172.21.15.123:0/735255883 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff82c0a3050 0x7ff82c0a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff84819b5c0 tx=0x7ff830009260 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:43.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.551+0000 7ff83f400640 1 -- 172.21.15.123:0/735255883 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff8380bc2b0 con 0x7ff848103d20 2024-09-19T04:46:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:43 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/935745964' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T04:46:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:43 smithi183 ceph-mon[35333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:46:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:43 smithi183 ceph-mon[35333]: mgrmap e22: x(active, since 1.02871s), standbys: y, z 2024-09-19T04:46:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:43 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3423045399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:46:43.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/935745964' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T04:46:43.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-mon[30511]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:46:43.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-mon[30511]: mgrmap e22: x(active, since 1.02871s), standbys: y, z 2024-09-19T04:46:43.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:43 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3423045399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:46:43.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.733+0000 7ff84d200640 1 -- 172.21.15.123:0/735255883 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff848062e90 con 0x7ff848103d20 2024-09-19T04:46:43.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.734+0000 7ff83f400640 1 -- 172.21.15.123:0/735255883 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7ff8380bbbb0 con 0x7ff848103d20 2024-09-19T04:46:43.735 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-09-19T04:46:43.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.736+0000 7ff84d200640 1 -- 172.21.15.123:0/735255883 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff82c0a3050 msgr2=0x7ff82c0a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:43.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.736+0000 7ff84d200640 1 --2- 172.21.15.123:0/735255883 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff82c0a3050 0x7ff82c0a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff84819b5c0 tx=0x7ff830009260 comp rx=0 tx=0).stop 2024-09-19T04:46:43.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.736+0000 7ff84d200640 1 -- 172.21.15.123:0/735255883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff848103d20 msgr2=0x7ff84819a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:43.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.736+0000 7ff84d200640 1 --2- 172.21.15.123:0/735255883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff848103d20 0x7ff84819a0b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff83802a860 tx=0x7ff838033d90 comp rx=0 tx=0).stop 2024-09-19T04:46:43.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.736+0000 7ff84d200640 1 -- 172.21.15.123:0/735255883 shutdown_connections 2024-09-19T04:46:43.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.736+0000 7ff84d200640 1 --2- 172.21.15.123:0/735255883 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff82c0a3050 0x7ff82c0a5510 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff84819b5c0 tx=0x7ff830009260 comp rx=0 tx=0).stop 2024-09-19T04:46:43.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.736+0000 7ff84d200640 1 --2- 172.21.15.123:0/735255883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8481058d0 0x7ff8481a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.736+0000 7ff84d200640 1 --2- 172.21.15.123:0/735255883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848104f10 0x7ff84819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.736+0000 7ff84d200640 1 --2- 172.21.15.123:0/735255883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff848103d20 0x7ff84819a0b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.736+0000 7ff84d200640 1 -- 172.21.15.123:0/735255883 >> 172.21.15.123:0/735255883 conn(0x7ff8480ff4a0 msgr2=0x7ff848108320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:43.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.736+0000 7ff84d200640 1 -- 172.21.15.123:0/735255883 shutdown_connections 2024-09-19T04:46:43.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.736+0000 7ff84d200640 1 -- 172.21.15.123:0/735255883 wait complete. 2024-09-19T04:46:43.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:43 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/935745964' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T04:46:43.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:43 smithi193 ceph-mon[35592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:46:43.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:43 smithi193 ceph-mon[35592]: mgrmap e22: x(active, since 1.02871s), standbys: y, z 2024-09-19T04:46:43.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:43 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3423045399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:46:43.754 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T04:46:43.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.944+0000 7f2fd4400640 1 -- 172.21.15.123:0/3178872465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fcc103d30 msgr2=0x7f2fcc104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:43.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.944+0000 7f2fd4400640 1 --2- 172.21.15.123:0/3178872465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fcc103d30 0x7f2fcc104130 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009900 tx=0x7f2fc002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.944+0000 7f2fd4400640 1 -- 172.21.15.123:0/3178872465 shutdown_connections 2024-09-19T04:46:43.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.944+0000 7f2fd4400640 1 --2- 172.21.15.123:0/3178872465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fcc1058e0 0x7f2fcc10c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.944+0000 7f2fd4400640 1 --2- 172.21.15.123:0/3178872465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fcc104f20 0x7f2fcc1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.944+0000 7f2fd4400640 1 --2- 172.21.15.123:0/3178872465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fcc103d30 0x7f2fcc104130 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.944+0000 7f2fd4400640 1 -- 172.21.15.123:0/3178872465 >> 172.21.15.123:0/3178872465 conn(0x7f2fcc0ff470 msgr2=0x7f2fcc1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:43.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.944+0000 7f2fd4400640 1 -- 172.21.15.123:0/3178872465 shutdown_connections 2024-09-19T04:46:43.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.944+0000 7f2fd4400640 1 -- 172.21.15.123:0/3178872465 wait complete. 2024-09-19T04:46:43.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.945+0000 7f2fd4400640 1 Processor -- start 2024-09-19T04:46:43.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.945+0000 7f2fd4400640 1 -- start start 2024-09-19T04:46:43.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.945+0000 7f2fd4400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fcc103d30 0x7f2fcc19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:43.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.945+0000 7f2fd4400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fcc104f20 0x7f2fcc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:43.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.945+0000 7f2fd4400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fcc1058e0 0x7f2fcc1a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:43.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.945+0000 7f2fd4400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc10fa00 con 0x7f2fcc104f20 2024-09-19T04:46:43.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.945+0000 7f2fd4400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc10f880 con 0x7f2fcc1058e0 2024-09-19T04:46:43.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.945+0000 7f2fd4400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc10fb80 con 0x7f2fcc103d30 2024-09-19T04:46:43.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.946+0000 7f2fd1a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fcc103d30 0x7f2fcc19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:43.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.946+0000 7f2fd1a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fcc103d30 0x7f2fcc19a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:46820/0 (socket says 172.21.15.123:46820) 2024-09-19T04:46:43.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.946+0000 7f2fd1a00640 1 -- 172.21.15.123:0/3294866839 learned_addr learned my addr 172.21.15.123:0/3294866839 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:43.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.946+0000 7f2fd2400640 1 --2- 172.21.15.123:0/3294866839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fcc1058e0 0x7f2fcc1a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:43.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.946+0000 7f2fd1000640 1 --2- 172.21.15.123:0/3294866839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fcc104f20 0x7f2fcc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:43.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.946+0000 7f2fd1a00640 1 -- 172.21.15.123:0/3294866839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fcc1058e0 msgr2=0x7f2fcc1a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:43.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.946+0000 7f2fd1a00640 1 --2- 172.21.15.123:0/3294866839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fcc1058e0 0x7f2fcc1a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.946+0000 7f2fd1a00640 1 -- 172.21.15.123:0/3294866839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fcc104f20 msgr2=0x7f2fcc19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:43.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.946+0000 7f2fd1a00640 1 --2- 172.21.15.123:0/3294866839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fcc104f20 0x7f2fcc19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:43.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.946+0000 7f2fd1a00640 1 -- 172.21.15.123:0/3294866839 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fcc1a1ca0 con 0x7f2fcc103d30 2024-09-19T04:46:43.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.946+0000 7f2fd1a00640 1 --2- 172.21.15.123:0/3294866839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fcc103d30 0x7f2fcc19a080 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0031440 tx=0x7f2fc0032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:43.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.947+0000 7f2fca000640 1 -- 172.21.15.123:0/3294866839 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc003f030 con 0x7f2fcc103d30 2024-09-19T04:46:43.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.947+0000 7f2fca000640 1 -- 172.21.15.123:0/3294866839 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fc0003ce0 con 0x7f2fcc103d30 2024-09-19T04:46:43.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.947+0000 7f2fd4400640 1 -- 172.21.15.123:0/3294866839 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fcc1a1f30 con 0x7f2fcc103d30 2024-09-19T04:46:43.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.947+0000 7f2fca000640 1 -- 172.21.15.123:0/3294866839 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0035070 con 0x7f2fcc103d30 2024-09-19T04:46:43.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.947+0000 7f2fd4400640 1 -- 172.21.15.123:0/3294866839 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fcc1a2340 con 0x7f2fcc103d30 2024-09-19T04:46:43.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.948+0000 7f2fd4400640 1 -- 172.21.15.123:0/3294866839 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fcc10b1c0 con 0x7f2fcc103d30 2024-09-19T04:46:43.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.950+0000 7f2fca000640 1 -- 172.21.15.123:0/3294866839 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f2fc0042030 con 0x7f2fcc103d30 2024-09-19T04:46:43.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.951+0000 7f2fca000640 1 --2- 172.21.15.123:0/3294866839 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2fb40a3050 0x7f2fb40a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:43.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.951+0000 7f2fca000640 1 -- 172.21.15.123:0/3294866839 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f2fc00efa90 con 0x7f2fcc103d30 2024-09-19T04:46:43.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.951+0000 7f2fd1000640 1 --2- 172.21.15.123:0/3294866839 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2fb40a3050 0x7f2fb40a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:43.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.952+0000 7f2fd1000640 1 --2- 172.21.15.123:0/3294866839 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2fb40a3050 0x7f2fb40a5510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80095f0 tx=0x7f2fb8005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:43.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:43.955+0000 7f2fca000640 1 -- 172.21.15.123:0/3294866839 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2fc00bc320 con 0x7f2fcc103d30 2024-09-19T04:46:44.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.149+0000 7f2fd4400640 1 -- 172.21.15.123:0/3294866839 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2fcc062f30 con 0x7f2fcc103d30 2024-09-19T04:46:44.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.149+0000 7f2fca000640 1 -- 172.21.15.123:0/3294866839 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f2fc00c0620 con 0x7f2fcc103d30 2024-09-19T04:46:44.151 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-09-19T04:46:44.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.152+0000 7f2fd4400640 1 -- 172.21.15.123:0/3294866839 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2fb40a3050 msgr2=0x7f2fb40a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:44.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.152+0000 7f2fd4400640 1 --2- 172.21.15.123:0/3294866839 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2fb40a3050 0x7f2fb40a5510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80095f0 tx=0x7f2fb8005f90 comp rx=0 tx=0).stop 2024-09-19T04:46:44.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.152+0000 7f2fd4400640 1 -- 172.21.15.123:0/3294866839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fcc103d30 msgr2=0x7f2fcc19a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:44.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.152+0000 7f2fd4400640 1 --2- 172.21.15.123:0/3294866839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fcc103d30 0x7f2fcc19a080 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0031440 tx=0x7f2fc0032ce0 comp rx=0 tx=0).stop 2024-09-19T04:46:44.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.152+0000 7f2fd4400640 1 -- 172.21.15.123:0/3294866839 shutdown_connections 2024-09-19T04:46:44.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.152+0000 7f2fd4400640 1 --2- 172.21.15.123:0/3294866839 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2fb40a3050 0x7f2fb40a5510 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:44.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.152+0000 7f2fd4400640 1 --2- 172.21.15.123:0/3294866839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fcc1058e0 0x7f2fcc1a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:44.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.152+0000 7f2fd4400640 1 --2- 172.21.15.123:0/3294866839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fcc104f20 0x7f2fcc19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:44.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.152+0000 7f2fd4400640 1 --2- 172.21.15.123:0/3294866839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fcc103d30 0x7f2fcc19a080 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:44.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.152+0000 7f2fd4400640 1 -- 172.21.15.123:0/3294866839 >> 172.21.15.123:0/3294866839 conn(0x7f2fcc0ff470 msgr2=0x7f2fcc108390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:44.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.152+0000 7f2fd4400640 1 -- 172.21.15.123:0/3294866839 shutdown_connections 2024-09-19T04:46:44.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.152+0000 7f2fd4400640 1 -- 172.21.15.123:0/3294866839 wait complete. 2024-09-19T04:46:44.171 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T04:46:44.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.344+0000 7f85eca00640 1 -- 172.21.15.123:0/2921850463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85e81077e0 msgr2=0x7f85e8109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:44.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.344+0000 7f85eca00640 1 --2- 172.21.15.123:0/2921850463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85e81077e0 0x7f85e8109c50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009ca0 tx=0x7f85e002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:46:44.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.344+0000 7f85eca00640 1 -- 172.21.15.123:0/2921850463 shutdown_connections 2024-09-19T04:46:44.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.344+0000 7f85eca00640 1 --2- 172.21.15.123:0/2921850463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85e81077e0 0x7f85e8109c50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:44.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.344+0000 7f85eca00640 1 --2- 172.21.15.123:0/2921850463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85e8104eb0 0x7f85e81072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:44.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.344+0000 7f85eca00640 1 --2- 172.21.15.123:0/2921850463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85e8102580 0x7f85e8104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:44.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.344+0000 7f85eca00640 1 -- 172.21.15.123:0/2921850463 >> 172.21.15.123:0/2921850463 conn(0x7f85e80fc140 msgr2=0x7f85e80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:44.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.344+0000 7f85eca00640 1 -- 172.21.15.123:0/2921850463 shutdown_connections 2024-09-19T04:46:44.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.344+0000 7f85eca00640 1 -- 172.21.15.123:0/2921850463 wait complete. 2024-09-19T04:46:44.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.345+0000 7f85eca00640 1 Processor -- start 2024-09-19T04:46:44.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.345+0000 7f85eca00640 1 -- start start 2024-09-19T04:46:44.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.345+0000 7f85eca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85e8102580 0x7f85e8101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:44.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.345+0000 7f85eca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85e8104eb0 0x7f85e8100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:44.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.345+0000 7f85eca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85e81077e0 0x7f85e8100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:44.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.345+0000 7f85eca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f85e810d7a0 con 0x7f85e8102580 2024-09-19T04:46:44.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.345+0000 7f85eca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85e810d620 con 0x7f85e81077e0 2024-09-19T04:46:44.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.345+0000 7f85eca00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f85e810d920 con 0x7f85e8104eb0 2024-09-19T04:46:44.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.346+0000 7f85e5400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85e8104eb0 0x7f85e8100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:44.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.346+0000 7f85e5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85e8102580 0x7f85e8101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:44.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.346+0000 7f85e5400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85e8104eb0 0x7f85e8100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:46842/0 (socket says 172.21.15.123:46842) 2024-09-19T04:46:44.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.346+0000 7f85e5400640 1 -- 172.21.15.123:0/4103348856 learned_addr learned my addr 172.21.15.123:0/4103348856 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:44.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.346+0000 7f85e6800640 1 --2- 172.21.15.123:0/4103348856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85e81077e0 0x7f85e8100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:44.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.346+0000 7f85e5400640 1 -- 172.21.15.123:0/4103348856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85e81077e0 msgr2=0x7f85e8100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:44.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.346+0000 7f85e5400640 1 --2- 172.21.15.123:0/4103348856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85e81077e0 0x7f85e8100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:44.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.346+0000 7f85e5400640 1 -- 172.21.15.123:0/4103348856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85e8102580 msgr2=0x7f85e8101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:44.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.346+0000 7f85e5400640 1 --2- 172.21.15.123:0/4103348856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85e8102580 0x7f85e8101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:44.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.346+0000 7f85e5400640 1 -- 172.21.15.123:0/4103348856 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e8101300 con 0x7f85e8104eb0 2024-09-19T04:46:44.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.347+0000 7f85e5400640 1 --2- 172.21.15.123:0/4103348856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85e8104eb0 0x7f85e8100530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f85d400e9c0 tx=0x7f85d400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:44.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.347+0000 7f85dea00640 1 -- 172.21.15.123:0/4103348856 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d400cd80 con 0x7f85e8104eb0 2024-09-19T04:46:44.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.347+0000 7f85dea00640 1 -- 172.21.15.123:0/4103348856 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85d4005500 con 0x7f85e8104eb0 2024-09-19T04:46:44.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.347+0000 7f85eca00640 1 -- 172.21.15.123:0/4103348856 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e81a1fa0 con 0x7f85e8104eb0 2024-09-19T04:46:44.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.347+0000 7f85dea00640 1 -- 172.21.15.123:0/4103348856 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4010690 con 0x7f85e8104eb0 2024-09-19T04:46:44.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.347+0000 7f85eca00640 1 -- 172.21.15.123:0/4103348856 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e81a2410 con 0x7f85e8104eb0 2024-09-19T04:46:44.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.348+0000 7f85eca00640 1 -- 172.21.15.123:0/4103348856 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e80fdc30 con 0x7f85e8104eb0 2024-09-19T04:46:44.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.351+0000 7f85dea00640 1 -- 172.21.15.123:0/4103348856 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f85d4005020 con 0x7f85e8104eb0 2024-09-19T04:46:44.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.352+0000 7f85dea00640 1 --2- 172.21.15.123:0/4103348856 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85cc0a30d0 0x7f85cc0a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:44.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.352+0000 7f85dea00640 1 -- 172.21.15.123:0/4103348856 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f85d40cf8d0 con 0x7f85e8104eb0 2024-09-19T04:46:44.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.352+0000 7f85e5e00640 1 --2- 172.21.15.123:0/4103348856 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85cc0a30d0 0x7f85cc0a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:44.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.353+0000 7f85e5e00640 1 --2- 172.21.15.123:0/4103348856 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85cc0a30d0 0x7f85cc0a5590 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009bd0 tx=0x7f85d8005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:44.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.356+0000 7f85dea00640 1 -- 172.21.15.123:0/4103348856 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f85d409c7f0 con 0x7f85e8104eb0 2024-09-19T04:46:44.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.487+0000 7f85dea00640 1 -- 172.21.15.123:0/4103348856 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mgrmap(e 23) ==== 138527+0+0 (secure 0 0 0) 0x7f85d4098760 con 0x7f85e8104eb0 2024-09-19T04:46:44.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.556+0000 7f85eca00640 1 -- 172.21.15.123:0/4103348856 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f85e8062df0 con 0x7f85e8104eb0 2024-09-19T04:46:44.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.556+0000 7f85dea00640 1 -- 172.21.15.123:0/4103348856 <== mon.1 v2:172.21.15.193:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f85d4046b90 con 0x7f85e8104eb0 2024-09-19T04:46:44.558 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-09-19T04:46:44.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.559+0000 7f85eca00640 1 -- 172.21.15.123:0/4103348856 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85cc0a30d0 msgr2=0x7f85cc0a5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:44.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.559+0000 7f85eca00640 1 --2- 172.21.15.123:0/4103348856 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85cc0a30d0 0x7f85cc0a5590 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009bd0 tx=0x7f85d8005f90 comp rx=0 tx=0).stop 2024-09-19T04:46:44.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.559+0000 7f85eca00640 1 -- 172.21.15.123:0/4103348856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85e8104eb0 msgr2=0x7f85e8100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:44.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.559+0000 7f85eca00640 1 --2- 172.21.15.123:0/4103348856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85e8104eb0 0x7f85e8100530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f85d400e9c0 tx=0x7f85d400ee80 comp rx=0 tx=0).stop 2024-09-19T04:46:44.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.559+0000 7f85eca00640 1 -- 172.21.15.123:0/4103348856 shutdown_connections 2024-09-19T04:46:44.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.559+0000 7f85eca00640 1 --2- 172.21.15.123:0/4103348856 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85cc0a30d0 0x7f85cc0a5590 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:44.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.559+0000 7f85eca00640 1 --2- 172.21.15.123:0/4103348856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85e81077e0 0x7f85e8100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:44.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.559+0000 7f85eca00640 1 --2- 172.21.15.123:0/4103348856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85e8104eb0 0x7f85e8100530 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:44.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.559+0000 7f85eca00640 1 --2- 172.21.15.123:0/4103348856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85e8102580 0x7f85e8101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:44.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.559+0000 7f85eca00640 1 -- 172.21.15.123:0/4103348856 >> 172.21.15.123:0/4103348856 conn(0x7f85e80fc140 msgr2=0x7f85e8102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:44.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.560+0000 7f85eca00640 1 -- 172.21.15.123:0/4103348856 shutdown_connections 2024-09-19T04:46:44.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:44.560+0000 7f85eca00640 1 -- 172.21.15.123:0/4103348856 wait complete. 2024-09-19T04:46:44.577 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-19T04:46:44.578 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:46:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: [19/Sep/2024:04:46:42] ENGINE Bus STARTING 2024-09-19T04:46:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: from='client.25192 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: [19/Sep/2024:04:46:43] ENGINE Serving on https://172.21.15.123:7150 2024-09-19T04:46:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: [19/Sep/2024:04:46:43] ENGINE Error in HTTPServer.serve 2024-09-19T04:46:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout: Traceback (most recent call last): 2024-09-19T04:46:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:46:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:46:44.851 INFO:journalctl@ceph.mon.b.smithi183.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:46:44.851 INFO:journalctl@ceph.mon.b.smithi183.stdout: self._run(expiration_interval) 2024-09-19T04:46:44.851 INFO:journalctl@ceph.mon.b.smithi183.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:46:44.851 INFO:journalctl@ceph.mon.b.smithi183.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:46:44.851 INFO:journalctl@ceph.mon.b.smithi183.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:46:44.851 INFO:journalctl@ceph.mon.b.smithi183.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:46:44.851 INFO:journalctl@ceph.mon.b.smithi183.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:46:44.851 INFO:journalctl@ceph.mon.b.smithi183.stdout: s = self.context.wrap_socket( 2024-09-19T04:46:44.851 INFO:journalctl@ceph.mon.b.smithi183.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:46:44.851 INFO:journalctl@ceph.mon.b.smithi183.stdout: return self.sslsocket_class._create( 2024-09-19T04:46:44.851 INFO:journalctl@ceph.mon.b.smithi183.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:46:44.851 INFO:journalctl@ceph.mon.b.smithi183.stdout: self.do_handshake() 2024-09-19T04:46:44.851 INFO:journalctl@ceph.mon.b.smithi183.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:46:44.851 INFO:journalctl@ceph.mon.b.smithi183.stdout: self._sslobj.do_handshake() 2024-09-19T04:46:44.852 INFO:journalctl@ceph.mon.b.smithi183.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:46:44.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: [19/Sep/2024:04:46:43] ENGINE Serving on http://172.21.15.123:8765 2024-09-19T04:46:44.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: [19/Sep/2024:04:46:43] ENGINE Bus STARTED 2024-09-19T04:46:44.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/735255883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:46:44.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3294866839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:46:44.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:44 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: [19/Sep/2024:04:46:42] ENGINE Bus STARTING 2024-09-19T04:46:44.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: from='client.25192 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:44.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: [19/Sep/2024:04:46:43] ENGINE Serving on https://172.21.15.123:7150 2024-09-19T04:46:44.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: [19/Sep/2024:04:46:43] ENGINE Error in HTTPServer.serve 2024-09-19T04:46:44.928 INFO:journalctl@ceph.mon.a.smithi123.stdout: Traceback (most recent call last): 2024-09-19T04:46:44.928 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:46:44.928 INFO:journalctl@ceph.mon.a.smithi123.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:46:44.928 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:46:44.928 INFO:journalctl@ceph.mon.a.smithi123.stdout: self._run(expiration_interval) 2024-09-19T04:46:44.928 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:46:44.928 INFO:journalctl@ceph.mon.a.smithi123.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:46:44.928 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:46:44.928 INFO:journalctl@ceph.mon.a.smithi123.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:46:44.928 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:46:44.929 INFO:journalctl@ceph.mon.a.smithi123.stdout: s = self.context.wrap_socket( 2024-09-19T04:46:44.929 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:46:44.929 INFO:journalctl@ceph.mon.a.smithi123.stdout: return self.sslsocket_class._create( 2024-09-19T04:46:44.929 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:46:44.929 INFO:journalctl@ceph.mon.a.smithi123.stdout: self.do_handshake() 2024-09-19T04:46:44.929 INFO:journalctl@ceph.mon.a.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:46:44.929 INFO:journalctl@ceph.mon.a.smithi123.stdout: self._sslobj.do_handshake() 2024-09-19T04:46:44.929 INFO:journalctl@ceph.mon.a.smithi123.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:46:44.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: [19/Sep/2024:04:46:43] ENGINE Serving on http://172.21.15.123:8765 2024-09-19T04:46:44.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: [19/Sep/2024:04:46:43] ENGINE Bus STARTED 2024-09-19T04:46:44.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/735255883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:46:44.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3294866839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:46:44.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:44 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.969 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:44.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: [19/Sep/2024:04:46:42] ENGINE Bus STARTING 2024-09-19T04:46:44.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: from='client.25192 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:44.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: [19/Sep/2024:04:46:43] ENGINE Serving on https://172.21.15.123:7150 2024-09-19T04:46:44.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: [19/Sep/2024:04:46:43] ENGINE Error in HTTPServer.serve 2024-09-19T04:46:44.992 INFO:journalctl@ceph.mon.c.smithi193.stdout: Traceback (most recent call last): 2024-09-19T04:46:44.992 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:46:44.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:46:44.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:46:44.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: self._run(expiration_interval) 2024-09-19T04:46:44.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:46:44.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:46:44.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:46:44.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:46:44.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:46:44.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: s = self.context.wrap_socket( 2024-09-19T04:46:44.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:46:44.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: return self.sslsocket_class._create( 2024-09-19T04:46:44.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:46:44.993 INFO:journalctl@ceph.mon.c.smithi193.stdout: self.do_handshake() 2024-09-19T04:46:44.994 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:46:44.994 INFO:journalctl@ceph.mon.c.smithi193.stdout: self._sslobj.do_handshake() 2024-09-19T04:46:44.994 INFO:journalctl@ceph.mon.c.smithi193.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:46:44.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: [19/Sep/2024:04:46:43] ENGINE Serving on http://172.21.15.123:8765 2024-09-19T04:46:44.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: [19/Sep/2024:04:46:43] ENGINE Bus STARTED 2024-09-19T04:46:44.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/735255883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:46:44.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3294866839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:46:44.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:44.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:44.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:44 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.584+0000 7fe7dfc00640 1 -- 172.21.15.123:0/1804534751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7d8103660 msgr2=0x7fe7d810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:45.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.584+0000 7fe7dfc00640 1 --2- 172.21.15.123:0/1804534751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7d8103660 0x7fe7d810bb50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009900 tx=0x7fe7c402a480 comp rx=0 tx=0).stop 2024-09-19T04:46:45.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.584+0000 7fe7dfc00640 1 -- 172.21.15.123:0/1804534751 shutdown_connections 2024-09-19T04:46:45.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.584+0000 7fe7dfc00640 1 --2- 172.21.15.123:0/1804534751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7d810c170 0x7fe7d810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:45.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.584+0000 7fe7dfc00640 1 --2- 172.21.15.123:0/1804534751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7d8103660 0x7fe7d810bb50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:45.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.584+0000 7fe7dfc00640 1 --2- 172.21.15.123:0/1804534751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7d8102d40 0x7fe7d8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:45.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.584+0000 7fe7dfc00640 1 -- 172.21.15.123:0/1804534751 >> 172.21.15.123:0/1804534751 conn(0x7fe7d80fc7a0 msgr2=0x7fe7d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:45.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.584+0000 7fe7dfc00640 1 -- 172.21.15.123:0/1804534751 shutdown_connections 2024-09-19T04:46:45.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.584+0000 7fe7dfc00640 1 -- 172.21.15.123:0/1804534751 wait complete. 2024-09-19T04:46:45.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.585+0000 7fe7dfc00640 1 Processor -- start 2024-09-19T04:46:45.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.586+0000 7fe7dfc00640 1 -- start start 2024-09-19T04:46:45.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.586+0000 7fe7dfc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7d8102d40 0x7fe7d819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:45.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.586+0000 7fe7dfc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7d8103660 0x7fe7d819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:45.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.586+0000 7fe7dfc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7d810c170 0x7fe7d819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:45.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.586+0000 7fe7dfc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8111fc0 con 0x7fe7d8102d40 2024-09-19T04:46:45.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.586+0000 7fe7dfc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8111e40 con 0x7fe7d810c170 2024-09-19T04:46:45.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.586+0000 7fe7dfc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8112140 con 0x7fe7d8103660 2024-09-19T04:46:45.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.586+0000 7fe7dd200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7d8102d40 0x7fe7d819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:45.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.586+0000 7fe7dd200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7d8102d40 0x7fe7d819a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50634/0 (socket says 172.21.15.123:50634) 2024-09-19T04:46:45.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.586+0000 7fe7dd200640 1 -- 172.21.15.123:0/1668281908 learned_addr learned my addr 172.21.15.123:0/1668281908 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:45.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.586+0000 7fe7d7e00640 1 --2- 172.21.15.123:0/1668281908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7d8103660 0x7fe7d819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:45.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.586+0000 7fe7ddc00640 1 --2- 172.21.15.123:0/1668281908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7d810c170 0x7fe7d819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:45.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.587+0000 7fe7d7e00640 1 -- 172.21.15.123:0/1668281908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7d810c170 msgr2=0x7fe7d819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:45.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.587+0000 7fe7d7e00640 1 --2- 172.21.15.123:0/1668281908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7d810c170 0x7fe7d819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:45.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.587+0000 7fe7d7e00640 1 -- 172.21.15.123:0/1668281908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7d8102d40 msgr2=0x7fe7d819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:45.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.587+0000 7fe7d7e00640 1 --2- 172.21.15.123:0/1668281908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7d8102d40 0x7fe7d819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:45.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.587+0000 7fe7d7e00640 1 -- 172.21.15.123:0/1668281908 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d819f630 con 0x7fe7d8103660 2024-09-19T04:46:45.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.587+0000 7fe7dd200640 1 --2- 172.21.15.123:0/1668281908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7d8102d40 0x7fe7d819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:45.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.587+0000 7fe7ddc00640 1 --2- 172.21.15.123:0/1668281908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7d810c170 0x7fe7d819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:45.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.587+0000 7fe7d7e00640 1 --2- 172.21.15.123:0/1668281908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7d8103660 0x7fe7d819abc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe7c402a9a0 tx=0x7fe7c4003a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:45.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.587+0000 7fe7d5600640 1 -- 172.21.15.123:0/1668281908 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c403f070 con 0x7fe7d8103660 2024-09-19T04:46:45.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.587+0000 7fe7dfc00640 1 -- 172.21.15.123:0/1668281908 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d819f8c0 con 0x7fe7d8103660 2024-09-19T04:46:45.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.587+0000 7fe7d5600640 1 -- 172.21.15.123:0/1668281908 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7c4031c90 con 0x7fe7d8103660 2024-09-19T04:46:45.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.588+0000 7fe7d5600640 1 -- 172.21.15.123:0/1668281908 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4035030 con 0x7fe7d8103660 2024-09-19T04:46:45.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.588+0000 7fe7dfc00640 1 -- 172.21.15.123:0/1668281908 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d81a7200 con 0x7fe7d8103660 2024-09-19T04:46:45.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.589+0000 7fe7dfc00640 1 -- 172.21.15.123:0/1668281908 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d8103120 con 0x7fe7d8103660 2024-09-19T04:46:45.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.591+0000 7fe7d5600640 1 -- 172.21.15.123:0/1668281908 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 23) ==== 138527+0+0 (secure 0 0 0) 0x7fe7c4042070 con 0x7fe7d8103660 2024-09-19T04:46:45.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.592+0000 7fe7d5600640 1 --2- 172.21.15.123:0/1668281908 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe7c00a3440 0x7fe7c00a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:45.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.592+0000 7fe7d5600640 1 -- 172.21.15.123:0/1668281908 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fe7c40f08a0 con 0x7fe7d8103660 2024-09-19T04:46:45.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.592+0000 7fe7dd200640 1 --2- 172.21.15.123:0/1668281908 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe7c00a3440 0x7fe7c00a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:45.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.593+0000 7fe7dd200640 1 --2- 172.21.15.123:0/1668281908 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe7c00a3440 0x7fe7c00a5900 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0095f0 tx=0x7fe7cc005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:45.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.594+0000 7fe7d5600640 1 -- 172.21.15.123:0/1668281908 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe7c40bca90 con 0x7fe7d8103660 2024-09-19T04:46:45.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:46:45.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-19T04:46:45.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-19T04:46:45.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: Unable to set osd_memory_target on smithi193 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T04:46:45.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:45.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:45.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:45.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:45.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.679 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.679 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.679 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.679 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-19T04:46:45.679 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: Unable to set osd_memory_target on smithi183 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T04:46:45.679 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:45 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/4103348856' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:46:45.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.822+0000 7fe7dfc00640 1 -- 172.21.15.123:0/1668281908 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7d819b9e0 con 0x7fe7d8103660 2024-09-19T04:46:45.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.823+0000 7fe7d5600640 1 -- 172.21.15.123:0/1668281908 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11990 (secure 0 0 0) 0x7fe7d819b9e0 con 0x7fe7d8103660 2024-09-19T04:46:45.825 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:46:45.825 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"btime":"2024-09-19T04:46:42:466383+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:46:41.577049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24499,24500,24539,24505,24529]},"id":1}]} 2024-09-19T04:46:45.825 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 28 2024-09-19T04:46:45.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.825+0000 7fe7dfc00640 1 -- 172.21.15.123:0/1668281908 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe7c00a3440 msgr2=0x7fe7c00a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:45.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.825+0000 7fe7dfc00640 1 --2- 172.21.15.123:0/1668281908 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe7c00a3440 0x7fe7c00a5900 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0095f0 tx=0x7fe7cc005f90 comp rx=0 tx=0).stop 2024-09-19T04:46:45.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.825+0000 7fe7dfc00640 1 -- 172.21.15.123:0/1668281908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7d8103660 msgr2=0x7fe7d819abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:45.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.825+0000 7fe7dfc00640 1 --2- 172.21.15.123:0/1668281908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7d8103660 0x7fe7d819abc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe7c402a9a0 tx=0x7fe7c4003a70 comp rx=0 tx=0).stop 2024-09-19T04:46:45.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.826+0000 7fe7dfc00640 1 -- 172.21.15.123:0/1668281908 shutdown_connections 2024-09-19T04:46:45.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.826+0000 7fe7dfc00640 1 --2- 172.21.15.123:0/1668281908 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe7c00a3440 0x7fe7c00a5900 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:45.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.826+0000 7fe7dfc00640 1 --2- 172.21.15.123:0/1668281908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7d810c170 0x7fe7d819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:45.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.826+0000 7fe7dfc00640 1 --2- 172.21.15.123:0/1668281908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7d8103660 0x7fe7d819abc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:45.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.826+0000 7fe7dfc00640 1 --2- 172.21.15.123:0/1668281908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7d8102d40 0x7fe7d819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:45.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.826+0000 7fe7dfc00640 1 -- 172.21.15.123:0/1668281908 >> 172.21.15.123:0/1668281908 conn(0x7fe7d80fc7a0 msgr2=0x7fe7d810d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:45.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.826+0000 7fe7dfc00640 1 -- 172.21.15.123:0/1668281908 shutdown_connections 2024-09-19T04:46:45.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:45.826+0000 7fe7dfc00640 1 -- 172.21.15.123:0/1668281908 wait complete. 2024-09-19T04:46:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:46:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-19T04:46:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-19T04:46:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: Unable to set osd_memory_target on smithi193 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T04:46:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-19T04:46:45.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: Unable to set osd_memory_target on smithi183 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T04:46:45.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:45 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/4103348856' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:46:45.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:46:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-19T04:46:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-19T04:46:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: Unable to set osd_memory_target on smithi193 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T04:46:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:45.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-19T04:46:45.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: Unable to set osd_memory_target on smithi183 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T04:46:45.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:45 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/4103348856' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:46:46.650 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-19T04:46:46.650 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-19T04:46:46.650 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:46:46.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.823+0000 7f979b400640 1 -- 172.21.15.123:0/4276909143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9794104f20 msgr2=0x7f97941053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:46.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.823+0000 7f979b400640 1 --2- 172.21.15.123:0/4276909143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9794104f20 0x7f97941053a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9780009900 tx=0x7f978002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:46:46.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.823+0000 7f979b400640 1 -- 172.21.15.123:0/4276909143 shutdown_connections 2024-09-19T04:46:46.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.823+0000 7f979b400640 1 --2- 172.21.15.123:0/4276909143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97941058e0 0x7f979410c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:46.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.823+0000 7f979b400640 1 --2- 172.21.15.123:0/4276909143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9794104f20 0x7f97941053a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:46.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.823+0000 7f979b400640 1 --2- 172.21.15.123:0/4276909143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9794103d30 0x7f9794104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:46.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.823+0000 7f979b400640 1 -- 172.21.15.123:0/4276909143 >> 172.21.15.123:0/4276909143 conn(0x7f97940ff4b0 msgr2=0x7f97941018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:46.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.823+0000 7f979b400640 1 -- 172.21.15.123:0/4276909143 shutdown_connections 2024-09-19T04:46:46.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.823+0000 7f979b400640 1 -- 172.21.15.123:0/4276909143 wait complete. 2024-09-19T04:46:46.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.824+0000 7f979b400640 1 Processor -- start 2024-09-19T04:46:46.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.824+0000 7f979b400640 1 -- start start 2024-09-19T04:46:46.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.825+0000 7f979b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9794103d30 0x7f979419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:46.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.825+0000 7f979b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9794104f20 0x7f979419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:46.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.825+0000 7f979b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97941058e0 0x7f97941a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:46.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.825+0000 7f979b400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f979410fa00 con 0x7f9794103d30 2024-09-19T04:46:46.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.825+0000 7f979b400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f979410f880 con 0x7f9794104f20 2024-09-19T04:46:46.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.825+0000 7f979b400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f979410fb80 con 0x7f97941058e0 2024-09-19T04:46:46.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.825+0000 7f9798a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9794103d30 0x7f979419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:46.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.825+0000 7f9799400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97941058e0 0x7f97941a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:46.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.825+0000 7f9799400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97941058e0 0x7f97941a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:46892/0 (socket says 172.21.15.123:46892) 2024-09-19T04:46:46.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.825+0000 7f9793e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9794104f20 0x7f979419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:46.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.825+0000 7f9799400640 1 -- 172.21.15.123:0/4140637471 learned_addr learned my addr 172.21.15.123:0/4140637471 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:46.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.825+0000 7f9799400640 1 -- 172.21.15.123:0/4140637471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9794104f20 msgr2=0x7f979419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:46.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.825+0000 7f9799400640 1 --2- 172.21.15.123:0/4140637471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9794104f20 0x7f979419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:46.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.825+0000 7f9799400640 1 -- 172.21.15.123:0/4140637471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9794103d30 msgr2=0x7f979419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:46.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.826+0000 7f9799400640 1 --2- 172.21.15.123:0/4140637471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9794103d30 0x7f979419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:46.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.826+0000 7f9799400640 1 -- 172.21.15.123:0/4140637471 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97941a1c50 con 0x7f97941058e0 2024-09-19T04:46:46.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.826+0000 7f9799400640 1 --2- 172.21.15.123:0/4140637471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97941058e0 0x7f97941a15e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f978c00eb10 tx=0x7f978c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:46.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.826+0000 7f9791600640 1 -- 172.21.15.123:0/4140637471 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c004b20 con 0x7f97941058e0 2024-09-19T04:46:46.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.826+0000 7f9791600640 1 -- 172.21.15.123:0/4140637471 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f978c004cc0 con 0x7f97941058e0 2024-09-19T04:46:46.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.826+0000 7f9791600640 1 -- 172.21.15.123:0/4140637471 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c010760 con 0x7f97941058e0 2024-09-19T04:46:46.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.826+0000 7f979b400640 1 -- 172.21.15.123:0/4140637471 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97941a1f40 con 0x7f97941058e0 2024-09-19T04:46:46.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.826+0000 7f979b400640 1 -- 172.21.15.123:0/4140637471 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97941a2400 con 0x7f97941058e0 2024-09-19T04:46:46.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.827+0000 7f979b400640 1 -- 172.21.15.123:0/4140637471 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f979410b050 con 0x7f97941058e0 2024-09-19T04:46:46.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.829+0000 7f9791600640 1 -- 172.21.15.123:0/4140637471 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 138583+0+0 (secure 0 0 0) 0x7f978c005020 con 0x7f97941058e0 2024-09-19T04:46:46.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.830+0000 7f9791600640 1 --2- 172.21.15.123:0/4140637471 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f977c0a34e0 0x7f977c0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:46.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.831+0000 7f9791600640 1 -- 172.21.15.123:0/4140637471 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f978c0d06a0 con 0x7f97941058e0 2024-09-19T04:46:46.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.831+0000 7f9798a00640 1 --2- 172.21.15.123:0/4140637471 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f977c0a34e0 0x7f977c0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:46.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.831+0000 7f9798a00640 1 --2- 172.21.15.123:0/4140637471 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f977c0a34e0 0x7f977c0a59a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f97880095f0 tx=0x7f9788005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:46.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:46.834+0000 7f9791600640 1 -- 172.21.15.123:0/4140637471 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f978c09ce00 con 0x7f97941058e0 2024-09-19T04:46:46.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:46.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:46.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:46.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:46.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:46.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:46.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:46.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:46.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:46.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:46.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi123 to 176.4M 2024-09-19T04:46:46.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: Unable to set osd_memory_target on smithi123 to 185036185: error parsing value: Value '185036185' is below minimum 939524096 2024-09-19T04:46:46.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:46:46.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:46:46.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-19T04:46:46.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-19T04:46:46.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-19T04:46:46.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1668281908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:46:46.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:46.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:46.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:46.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:46 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:46.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:46.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:46.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:46.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:46.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:46.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:46.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:46.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:46.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:46.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:46.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi123 to 176.4M 2024-09-19T04:46:46.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: Unable to set osd_memory_target on smithi123 to 185036185: error parsing value: Value '185036185' is below minimum 939524096 2024-09-19T04:46:46.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:46:46.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:46:46.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-19T04:46:46.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-19T04:46:46.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-19T04:46:46.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1668281908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:46:46.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:46.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:46.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:46.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:46 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:47.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.034+0000 7f979b400640 1 -- 172.21.15.123:0/4140637471 --> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9794062630 con 0x7f977c0a34e0 2024-09-19T04:46:47.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.042+0000 7f9791600640 1 -- 172.21.15.123:0/4140637471 <== mgr.25133 v2:172.21.15.123:6800/1578549821 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258051 (secure 0 0 0) 0x7f9794062630 con 0x7f977c0a34e0 2024-09-19T04:46:47.047 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:46:47.048 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-09-19T04:46:47.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.046+0000 7f979b400640 1 -- 172.21.15.123:0/4140637471 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f977c0a34e0 msgr2=0x7f977c0a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:47.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.046+0000 7f979b400640 1 --2- 172.21.15.123:0/4140637471 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f977c0a34e0 0x7f977c0a59a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f97880095f0 tx=0x7f9788005f90 comp rx=0 tx=0).stop 2024-09-19T04:46:47.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.046+0000 7f979b400640 1 -- 172.21.15.123:0/4140637471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97941058e0 msgr2=0x7f97941a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:47.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.046+0000 7f979b400640 1 --2- 172.21.15.123:0/4140637471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97941058e0 0x7f97941a15e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f978c00eb10 tx=0x7f978c00c2d0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.046+0000 7f979b400640 1 -- 172.21.15.123:0/4140637471 shutdown_connections 2024-09-19T04:46:47.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.046+0000 7f979b400640 1 --2- 172.21.15.123:0/4140637471 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f977c0a34e0 0x7f977c0a59a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.046+0000 7f979b400640 1 --2- 172.21.15.123:0/4140637471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97941058e0 0x7f97941a15e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.046+0000 7f979b400640 1 --2- 172.21.15.123:0/4140637471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9794104f20 0x7f979419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.046+0000 7f979b400640 1 --2- 172.21.15.123:0/4140637471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9794103d30 0x7f979419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.046+0000 7f979b400640 1 -- 172.21.15.123:0/4140637471 >> 172.21.15.123:0/4140637471 conn(0x7f97940ff4b0 msgr2=0x7f9794108220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:47.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.046+0000 7f979b400640 1 -- 172.21.15.123:0/4140637471 shutdown_connections 2024-09-19T04:46:47.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.046+0000 7f979b400640 1 -- 172.21.15.123:0/4140637471 wait complete. 2024-09-19T04:46:47.065 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T04:46:45.796478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190429,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1449,"num_read_kb":882,"num_write":555,"num_write_kb":4401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":938,"ondisk_log_size":938,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357032,"kb_used_data":11892,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584792,"statfs":{"total":1151940427776,"available":1151574827008,"internally_reserved":0,"allocated":12177408,"data_stored":8740503,"data_compressed":199511,"data_compressed_allocated":258048,"data_compressed_original":3022848,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.977753"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033086+0000","last_change":"2024-09-19T04:44:30.754400+0000","last_active":"2024-09-19T04:46:42.033086+0000","last_peered":"2024-09-19T04:46:42.033086+0000","last_clean":"2024-09-19T04:46:42.033086+0000","last_became_active":"2024-09-19T04:44:30.754028+0000","last_became_peered":"2024-09-19T04:44:30.754028+0000","last_unstale":"2024-09-19T04:46:42.033086+0000","last_undegraded":"2024-09-19T04:46:42.033086+0000","last_fullsized":"2024-09-19T04:46:42.033086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:25.668478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782861+0000","last_change":"2024-09-19T04:44:30.758072+0000","last_active":"2024-09-19T04:46:41.782861+0000","last_peered":"2024-09-19T04:46:41.782861+0000","last_clean":"2024-09-19T04:46:41.782861+0000","last_became_active":"2024-09-19T04:44:30.756734+0000","last_became_peered":"2024-09-19T04:44:30.756734+0000","last_unstale":"2024-09-19T04:46:41.782861+0000","last_undegraded":"2024-09-19T04:46:41.782861+0000","last_fullsized":"2024-09-19T04:46:41.782861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:56:47.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782852+0000","last_change":"2024-09-19T04:44:30.760208+0000","last_active":"2024-09-19T04:46:41.782852+0000","last_peered":"2024-09-19T04:46:41.782852+0000","last_clean":"2024-09-19T04:46:41.782852+0000","last_became_active":"2024-09-19T04:44:30.759719+0000","last_became_peered":"2024-09-19T04:44:30.759719+0000","last_unstale":"2024-09-19T04:46:41.782852+0000","last_undegraded":"2024-09-19T04:46:41.782852+0000","last_fullsized":"2024-09-19T04:46:41.782852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:14.041853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033102+0000","last_change":"2024-09-19T04:44:30.747813+0000","last_active":"2024-09-19T04:46:42.033102+0000","last_peered":"2024-09-19T04:46:42.033102+0000","last_clean":"2024-09-19T04:46:42.033102+0000","last_became_active":"2024-09-19T04:44:30.747486+0000","last_became_peered":"2024-09-19T04:44:30.747486+0000","last_unstale":"2024-09-19T04:46:42.033102+0000","last_undegraded":"2024-09-19T04:46:42.033102+0000","last_fullsized":"2024-09-19T04:46:42.033102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:11.315273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785693+0000","last_change":"2024-09-19T04:44:30.755800+0000","last_active":"2024-09-19T04:46:41.785693+0000","last_peered":"2024-09-19T04:46:41.785693+0000","last_clean":"2024-09-19T04:46:41.785693+0000","last_became_active":"2024-09-19T04:44:30.755448+0000","last_became_peered":"2024-09-19T04:44:30.755448+0000","last_unstale":"2024-09-19T04:46:41.785693+0000","last_undegraded":"2024-09-19T04:46:41.785693+0000","last_fullsized":"2024-09-19T04:46:41.785693+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:39.308614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.898818+0000","last_change":"2024-09-19T04:44:30.746230+0000","last_active":"2024-09-19T04:46:41.898818+0000","last_peered":"2024-09-19T04:46:41.898818+0000","last_clean":"2024-09-19T04:46:41.898818+0000","last_became_active":"2024-09-19T04:44:30.745884+0000","last_became_peered":"2024-09-19T04:44:30.745884+0000","last_unstale":"2024-09-19T04:46:41.898818+0000","last_undegraded":"2024-09-19T04:46:41.898818+0000","last_fullsized":"2024-09-19T04:46:41.898818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:51:58.550322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782104+0000","last_change":"2024-09-19T04:44:30.747870+0000","last_active":"2024-09-19T04:46:41.782104+0000","last_peered":"2024-09-19T04:46:41.782104+0000","last_clean":"2024-09-19T04:46:41.782104+0000","last_became_active":"2024-09-19T04:44:30.747536+0000","last_became_peered":"2024-09-19T04:44:30.747536+0000","last_unstale":"2024-09-19T04:46:41.782104+0000","last_undegraded":"2024-09-19T04:46:41.782104+0000","last_fullsized":"2024-09-19T04:46:41.782104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:20.074032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886225+0000","last_change":"2024-09-19T04:44:30.750425+0000","last_active":"2024-09-19T04:46:41.886225+0000","last_peered":"2024-09-19T04:46:41.886225+0000","last_clean":"2024-09-19T04:46:41.886225+0000","last_became_active":"2024-09-19T04:44:30.749841+0000","last_became_peered":"2024-09-19T04:44:30.749841+0000","last_unstale":"2024-09-19T04:46:41.886225+0000","last_undegraded":"2024-09-19T04:46:41.886225+0000","last_fullsized":"2024-09-19T04:46:41.886225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:12:08.420727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032832+0000","last_change":"2024-09-19T04:44:30.750624+0000","last_active":"2024-09-19T04:46:42.032832+0000","last_peered":"2024-09-19T04:46:42.032832+0000","last_clean":"2024-09-19T04:46:42.032832+0000","last_became_active":"2024-09-19T04:44:30.748370+0000","last_became_peered":"2024-09-19T04:44:30.748370+0000","last_unstale":"2024-09-19T04:46:42.032832+0000","last_undegraded":"2024-09-19T04:46:42.032832+0000","last_fullsized":"2024-09-19T04:46:42.032832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:36.131634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.899170+0000","last_change":"2024-09-19T04:44:30.750035+0000","last_active":"2024-09-19T04:46:41.899170+0000","last_peered":"2024-09-19T04:46:41.899170+0000","last_clean":"2024-09-19T04:46:41.899170+0000","last_became_active":"2024-09-19T04:44:30.749699+0000","last_became_peered":"2024-09-19T04:44:30.749699+0000","last_unstale":"2024-09-19T04:46:41.899170+0000","last_undegraded":"2024-09-19T04:46:41.899170+0000","last_fullsized":"2024-09-19T04:46:41.899170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:22.637193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782572+0000","last_change":"2024-09-19T04:44:30.760258+0000","last_active":"2024-09-19T04:46:41.782572+0000","last_peered":"2024-09-19T04:46:41.782572+0000","last_clean":"2024-09-19T04:46:41.782572+0000","last_became_active":"2024-09-19T04:44:30.759925+0000","last_became_peered":"2024-09-19T04:44:30.759925+0000","last_unstale":"2024-09-19T04:46:41.782572+0000","last_undegraded":"2024-09-19T04:46:41.782572+0000","last_fullsized":"2024-09-19T04:46:41.782572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:58.144691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033125+0000","last_change":"2024-09-19T04:44:30.747703+0000","last_active":"2024-09-19T04:46:42.033125+0000","last_peered":"2024-09-19T04:46:42.033125+0000","last_clean":"2024-09-19T04:46:42.033125+0000","last_became_active":"2024-09-19T04:44:30.747369+0000","last_became_peered":"2024-09-19T04:44:30.747369+0000","last_unstale":"2024-09-19T04:46:42.033125+0000","last_undegraded":"2024-09-19T04:46:42.033125+0000","last_fullsized":"2024-09-19T04:46:42.033125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:06.920357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.034354+0000","last_change":"2024-09-19T04:44:30.748844+0000","last_active":"2024-09-19T04:46:42.034354+0000","last_peered":"2024-09-19T04:46:42.034354+0000","last_clean":"2024-09-19T04:46:42.034354+0000","last_became_active":"2024-09-19T04:44:30.748513+0000","last_became_peered":"2024-09-19T04:44:30.748513+0000","last_unstale":"2024-09-19T04:46:42.034354+0000","last_undegraded":"2024-09-19T04:46:42.034354+0000","last_fullsized":"2024-09-19T04:46:42.034354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:45.107718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782714+0000","last_change":"2024-09-19T04:44:30.750526+0000","last_active":"2024-09-19T04:46:41.782714+0000","last_peered":"2024-09-19T04:46:41.782714+0000","last_clean":"2024-09-19T04:46:41.782714+0000","last_became_active":"2024-09-19T04:44:30.749693+0000","last_became_peered":"2024-09-19T04:44:30.749693+0000","last_unstale":"2024-09-19T04:46:41.782714+0000","last_undegraded":"2024-09-19T04:46:41.782714+0000","last_fullsized":"2024-09-19T04:46:41.782714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:43:04.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782639+0000","last_change":"2024-09-19T04:44:30.754310+0000","last_active":"2024-09-19T04:46:41.782639+0000","last_peered":"2024-09-19T04:46:41.782639+0000","last_clean":"2024-09-19T04:46:41.782639+0000","last_became_active":"2024-09-19T04:44:30.753933+0000","last_became_peered":"2024-09-19T04:44:30.753933+0000","last_unstale":"2024-09-19T04:46:41.782639+0000","last_undegraded":"2024-09-19T04:46:41.782639+0000","last_fullsized":"2024-09-19T04:46:41.782639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:40:44.985693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032386+0000","last_change":"2024-09-19T04:44:30.757458+0000","last_active":"2024-09-19T04:46:42.032386+0000","last_peered":"2024-09-19T04:46:42.032386+0000","last_clean":"2024-09-19T04:46:42.032386+0000","last_became_active":"2024-09-19T04:44:30.757127+0000","last_became_peered":"2024-09-19T04:44:30.757127+0000","last_unstale":"2024-09-19T04:46:42.032386+0000","last_undegraded":"2024-09-19T04:46:42.032386+0000","last_fullsized":"2024-09-19T04:46:42.032386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:59.325865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033118+0000","last_change":"2024-09-19T04:44:30.754685+0000","last_active":"2024-09-19T04:46:42.033118+0000","last_peered":"2024-09-19T04:46:42.033118+0000","last_clean":"2024-09-19T04:46:42.033118+0000","last_became_active":"2024-09-19T04:44:30.753625+0000","last_became_peered":"2024-09-19T04:44:30.753625+0000","last_unstale":"2024-09-19T04:46:42.033118+0000","last_undegraded":"2024-09-19T04:46:42.033118+0000","last_fullsized":"2024-09-19T04:46:42.033118+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:46:01.164693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.882473+0000","last_change":"2024-09-19T04:44:30.745823+0000","last_active":"2024-09-19T04:46:41.882473+0000","last_peered":"2024-09-19T04:46:41.882473+0000","last_clean":"2024-09-19T04:46:41.882473+0000","last_became_active":"2024-09-19T04:44:30.745224+0000","last_became_peered":"2024-09-19T04:44:30.745224+0000","last_unstale":"2024-09-19T04:46:41.882473+0000","last_undegraded":"2024-09-19T04:46:41.882473+0000","last_fullsized":"2024-09-19T04:46:41.882473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:23.682118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.012014+0000","last_change":"2024-09-19T04:44:30.752138+0000","last_active":"2024-09-19T04:46:42.012014+0000","last_peered":"2024-09-19T04:46:42.012014+0000","last_clean":"2024-09-19T04:46:42.012014+0000","last_became_active":"2024-09-19T04:44:30.751737+0000","last_became_peered":"2024-09-19T04:44:30.751737+0000","last_unstale":"2024-09-19T04:46:42.012014+0000","last_undegraded":"2024-09-19T04:46:42.012014+0000","last_fullsized":"2024-09-19T04:46:42.012014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:08.204601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032760+0000","last_change":"2024-09-19T04:44:30.748870+0000","last_active":"2024-09-19T04:46:42.032760+0000","last_peered":"2024-09-19T04:46:42.032760+0000","last_clean":"2024-09-19T04:46:42.032760+0000","last_became_active":"2024-09-19T04:44:30.748549+0000","last_became_peered":"2024-09-19T04:44:30.748549+0000","last_unstale":"2024-09-19T04:46:42.032760+0000","last_undegraded":"2024-09-19T04:46:42.032760+0000","last_fullsized":"2024-09-19T04:46:42.032760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:12.951564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.034496+0000","last_change":"2024-09-19T04:44:30.759937+0000","last_active":"2024-09-19T04:46:42.034496+0000","last_peered":"2024-09-19T04:46:42.034496+0000","last_clean":"2024-09-19T04:46:42.034496+0000","last_became_active":"2024-09-19T04:44:30.759659+0000","last_became_peered":"2024-09-19T04:44:30.759659+0000","last_unstale":"2024-09-19T04:46:42.034496+0000","last_undegraded":"2024-09-19T04:46:42.034496+0000","last_fullsized":"2024-09-19T04:46:42.034496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:07.608994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032624+0000","last_change":"2024-09-19T04:44:30.756216+0000","last_active":"2024-09-19T04:46:42.032624+0000","last_peered":"2024-09-19T04:46:42.032624+0000","last_clean":"2024-09-19T04:46:42.032624+0000","last_became_active":"2024-09-19T04:44:30.755903+0000","last_became_peered":"2024-09-19T04:44:30.755903+0000","last_unstale":"2024-09-19T04:46:42.032624+0000","last_undegraded":"2024-09-19T04:46:42.032624+0000","last_fullsized":"2024-09-19T04:46:42.032624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:16.966305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886344+0000","last_change":"2024-09-19T04:44:30.746297+0000","last_active":"2024-09-19T04:46:41.886344+0000","last_peered":"2024-09-19T04:46:41.886344+0000","last_clean":"2024-09-19T04:46:41.886344+0000","last_became_active":"2024-09-19T04:44:30.746013+0000","last_became_peered":"2024-09-19T04:44:30.746013+0000","last_unstale":"2024-09-19T04:46:41.886344+0000","last_undegraded":"2024-09-19T04:46:41.886344+0000","last_fullsized":"2024-09-19T04:46:41.886344+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:17.086180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785644+0000","last_change":"2024-09-19T04:45:14.630690+0000","last_active":"2024-09-19T04:46:41.785644+0000","last_peered":"2024-09-19T04:46:41.785644+0000","last_clean":"2024-09-19T04:46:41.785644+0000","last_became_active":"2024-09-19T04:45:14.630228+0000","last_became_peered":"2024-09-19T04:45:14.630228+0000","last_unstale":"2024-09-19T04:46:41.785644+0000","last_undegraded":"2024-09-19T04:46:41.785644+0000","last_fullsized":"2024-09-19T04:46:41.785644+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:39.901088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.887149+0000","last_change":"2024-09-19T04:44:30.753423+0000","last_active":"2024-09-19T04:46:41.887149+0000","last_peered":"2024-09-19T04:46:41.887149+0000","last_clean":"2024-09-19T04:46:41.887149+0000","last_became_active":"2024-09-19T04:44:30.752139+0000","last_became_peered":"2024-09-19T04:44:30.752139+0000","last_unstale":"2024-09-19T04:46:41.887149+0000","last_undegraded":"2024-09-19T04:46:41.887149+0000","last_fullsized":"2024-09-19T04:46:41.887149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:18.046893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.012113+0000","last_change":"2024-09-19T04:44:30.751114+0000","last_active":"2024-09-19T04:46:42.012113+0000","last_peered":"2024-09-19T04:46:42.012113+0000","last_clean":"2024-09-19T04:46:42.012113+0000","last_became_active":"2024-09-19T04:44:30.750784+0000","last_became_peered":"2024-09-19T04:44:30.750784+0000","last_unstale":"2024-09-19T04:46:42.012113+0000","last_undegraded":"2024-09-19T04:46:42.012113+0000","last_fullsized":"2024-09-19T04:46:42.012113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:22.040253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033057+0000","last_change":"2024-09-19T04:44:30.747191+0000","last_active":"2024-09-19T04:46:42.033057+0000","last_peered":"2024-09-19T04:46:42.033057+0000","last_clean":"2024-09-19T04:46:42.033057+0000","last_became_active":"2024-09-19T04:44:30.746830+0000","last_became_peered":"2024-09-19T04:44:30.746830+0000","last_unstale":"2024-09-19T04:46:42.033057+0000","last_undegraded":"2024-09-19T04:46:42.033057+0000","last_fullsized":"2024-09-19T04:46:42.033057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:24.634920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.034635+0000","last_change":"2024-09-19T04:44:30.755174+0000","last_active":"2024-09-19T04:46:42.034635+0000","last_peered":"2024-09-19T04:46:42.034635+0000","last_clean":"2024-09-19T04:46:42.034635+0000","last_became_active":"2024-09-19T04:44:30.754885+0000","last_became_peered":"2024-09-19T04:44:30.754885+0000","last_unstale":"2024-09-19T04:46:42.034635+0000","last_undegraded":"2024-09-19T04:46:42.034635+0000","last_fullsized":"2024-09-19T04:46:42.034635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:11:32.428033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782923+0000","last_change":"2024-09-19T04:45:14.891484+0000","last_active":"2024-09-19T04:46:41.782923+0000","last_peered":"2024-09-19T04:46:41.782923+0000","last_clean":"2024-09-19T04:46:41.782923+0000","last_became_active":"2024-09-19T04:45:14.891120+0000","last_became_peered":"2024-09-19T04:45:14.891120+0000","last_unstale":"2024-09-19T04:46:41.782923+0000","last_undegraded":"2024-09-19T04:46:41.782923+0000","last_fullsized":"2024-09-19T04:46:41.782923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:20.959571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.011793+0000","last_change":"2024-09-19T04:44:30.757343+0000","last_active":"2024-09-19T04:46:42.011793+0000","last_peered":"2024-09-19T04:46:42.011793+0000","last_clean":"2024-09-19T04:46:42.011793+0000","last_became_active":"2024-09-19T04:44:30.756981+0000","last_became_peered":"2024-09-19T04:44:30.756981+0000","last_unstale":"2024-09-19T04:46:42.011793+0000","last_undegraded":"2024-09-19T04:46:42.011793+0000","last_fullsized":"2024-09-19T04:46:42.011793+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:07.388384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.011916+0000","last_change":"2024-09-19T04:45:14.631104+0000","last_active":"2024-09-19T04:46:42.011916+0000","last_peered":"2024-09-19T04:46:42.011916+0000","last_clean":"2024-09-19T04:46:42.011916+0000","last_became_active":"2024-09-19T04:45:14.630654+0000","last_became_peered":"2024-09-19T04:45:14.630654+0000","last_unstale":"2024-09-19T04:46:42.011916+0000","last_undegraded":"2024-09-19T04:46:42.011916+0000","last_fullsized":"2024-09-19T04:46:42.011916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:37.659295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032830+0000","last_change":"2024-09-19T04:44:30.748408+0000","last_active":"2024-09-19T04:46:42.032830+0000","last_peered":"2024-09-19T04:46:42.032830+0000","last_clean":"2024-09-19T04:46:42.032830+0000","last_became_active":"2024-09-19T04:44:30.747960+0000","last_became_peered":"2024-09-19T04:44:30.747960+0000","last_unstale":"2024-09-19T04:46:42.032830+0000","last_undegraded":"2024-09-19T04:46:42.032830+0000","last_fullsized":"2024-09-19T04:46:42.032830+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:51.043791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785665+0000","last_change":"2024-09-19T04:44:30.747194+0000","last_active":"2024-09-19T04:46:41.785665+0000","last_peered":"2024-09-19T04:46:41.785665+0000","last_clean":"2024-09-19T04:46:41.785665+0000","last_became_active":"2024-09-19T04:44:30.746871+0000","last_became_peered":"2024-09-19T04:44:30.746871+0000","last_unstale":"2024-09-19T04:46:41.785665+0000","last_undegraded":"2024-09-19T04:46:41.785665+0000","last_fullsized":"2024-09-19T04:46:41.785665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:52.340341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":152,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.114293+0000","last_change":"2024-09-19T04:44:28.337746+0000","last_active":"2024-09-19T04:46:42.114293+0000","last_peered":"2024-09-19T04:46:42.114293+0000","last_clean":"2024-09-19T04:46:42.114293+0000","last_became_active":"2024-09-19T04:44:28.337410+0000","last_became_peered":"2024-09-19T04:44:28.337410+0000","last_unstale":"2024-09-19T04:46:42.114293+0000","last_undegraded":"2024-09-19T04:46:42.114293+0000","last_fullsized":"2024-09-19T04:46:42.114293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:41:27.436939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":2,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895593+0000","last_change":"2024-09-19T04:44:30.755512+0000","last_active":"2024-09-19T04:46:41.895593+0000","last_peered":"2024-09-19T04:46:41.895593+0000","last_clean":"2024-09-19T04:46:41.895593+0000","last_became_active":"2024-09-19T04:44:30.755086+0000","last_became_peered":"2024-09-19T04:44:30.755086+0000","last_unstale":"2024-09-19T04:46:41.895593+0000","last_undegraded":"2024-09-19T04:46:41.895593+0000","last_fullsized":"2024-09-19T04:46:41.895593+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:33.870376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.898886+0000","last_change":"2024-09-19T04:44:28.334492+0000","last_active":"2024-09-19T04:46:41.898886+0000","last_peered":"2024-09-19T04:46:41.898886+0000","last_clean":"2024-09-19T04:46:41.898886+0000","last_became_active":"2024-09-19T04:44:28.334098+0000","last_became_peered":"2024-09-19T04:44:28.334098+0000","last_unstale":"2024-09-19T04:46:41.898886+0000","last_undegraded":"2024-09-19T04:46:41.898886+0000","last_fullsized":"2024-09-19T04:46:41.898886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:48.156590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033952+0000","last_change":"2024-09-19T04:44:28.335593+0000","last_active":"2024-09-19T04:46:42.033952+0000","last_peered":"2024-09-19T04:46:42.033952+0000","last_clean":"2024-09-19T04:46:42.033952+0000","last_became_active":"2024-09-19T04:44:28.335060+0000","last_became_peered":"2024-09-19T04:44:28.335060+0000","last_unstale":"2024-09-19T04:46:42.033952+0000","last_undegraded":"2024-09-19T04:46:42.033952+0000","last_fullsized":"2024-09-19T04:46:42.033952+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:27.859960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886288+0000","last_change":"2024-09-19T04:44:30.748235+0000","last_active":"2024-09-19T04:46:41.886288+0000","last_peered":"2024-09-19T04:46:41.886288+0000","last_clean":"2024-09-19T04:46:41.886288+0000","last_became_active":"2024-09-19T04:44:30.747881+0000","last_became_peered":"2024-09-19T04:44:30.747881+0000","last_unstale":"2024-09-19T04:46:41.886288+0000","last_undegraded":"2024-09-19T04:46:41.886288+0000","last_fullsized":"2024-09-19T04:46:41.886288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:40:39.183897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895656+0000","last_change":"2024-09-19T04:44:28.337762+0000","last_active":"2024-09-19T04:46:41.895656+0000","last_peered":"2024-09-19T04:46:41.895656+0000","last_clean":"2024-09-19T04:46:41.895656+0000","last_became_active":"2024-09-19T04:44:28.337383+0000","last_became_peered":"2024-09-19T04:44:28.337383+0000","last_unstale":"2024-09-19T04:46:41.895656+0000","last_undegraded":"2024-09-19T04:46:41.895656+0000","last_fullsized":"2024-09-19T04:46:41.895656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:07:55.268958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.882010+0000","last_change":"2024-09-19T04:44:30.754318+0000","last_active":"2024-09-19T04:46:41.882010+0000","last_peered":"2024-09-19T04:46:41.882010+0000","last_clean":"2024-09-19T04:46:41.882010+0000","last_became_active":"2024-09-19T04:44:30.754025+0000","last_became_peered":"2024-09-19T04:44:30.754025+0000","last_unstale":"2024-09-19T04:46:41.882010+0000","last_undegraded":"2024-09-19T04:46:41.882010+0000","last_fullsized":"2024-09-19T04:46:41.882010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:58.190504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":266,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.010422+0000","last_change":"2024-09-19T04:44:28.335947+0000","last_active":"2024-09-19T04:46:42.010422+0000","last_peered":"2024-09-19T04:46:42.010422+0000","last_clean":"2024-09-19T04:46:42.010422+0000","last_became_active":"2024-09-19T04:44:28.335588+0000","last_became_peered":"2024-09-19T04:44:28.335588+0000","last_unstale":"2024-09-19T04:46:42.010422+0000","last_undegraded":"2024-09-19T04:46:42.010422+0000","last_fullsized":"2024-09-19T04:46:42.010422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:44.024915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":236,"num_read_kb":236,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032720+0000","last_change":"2024-09-19T04:44:30.748272+0000","last_active":"2024-09-19T04:46:42.032720+0000","last_peered":"2024-09-19T04:46:42.032720+0000","last_clean":"2024-09-19T04:46:42.032720+0000","last_became_active":"2024-09-19T04:44:30.747147+0000","last_became_peered":"2024-09-19T04:44:30.747147+0000","last_unstale":"2024-09-19T04:46:42.032720+0000","last_undegraded":"2024-09-19T04:46:42.032720+0000","last_fullsized":"2024-09-19T04:46:42.032720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:11.653693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782564+0000","last_change":"2024-09-19T04:44:28.337120+0000","last_active":"2024-09-19T04:46:41.782564+0000","last_peered":"2024-09-19T04:46:41.782564+0000","last_clean":"2024-09-19T04:46:41.782564+0000","last_became_active":"2024-09-19T04:44:28.336053+0000","last_became_peered":"2024-09-19T04:44:28.336053+0000","last_unstale":"2024-09-19T04:46:41.782564+0000","last_undegraded":"2024-09-19T04:46:41.782564+0000","last_fullsized":"2024-09-19T04:46:41.782564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:56:57.514931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782921+0000","last_change":"2024-09-19T04:44:30.755238+0000","last_active":"2024-09-19T04:46:41.782921+0000","last_peered":"2024-09-19T04:46:41.782921+0000","last_clean":"2024-09-19T04:46:41.782921+0000","last_became_active":"2024-09-19T04:44:30.754748+0000","last_became_peered":"2024-09-19T04:44:30.754748+0000","last_unstale":"2024-09-19T04:46:41.782921+0000","last_undegraded":"2024-09-19T04:46:41.782921+0000","last_fullsized":"2024-09-19T04:46:41.782921+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:38.954491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895818+0000","last_change":"2024-09-19T04:44:28.596147+0000","last_active":"2024-09-19T04:46:41.895818+0000","last_peered":"2024-09-19T04:46:41.895818+0000","last_clean":"2024-09-19T04:46:41.895818+0000","last_became_active":"2024-09-19T04:44:28.595785+0000","last_became_peered":"2024-09-19T04:44:28.595785+0000","last_unstale":"2024-09-19T04:46:41.895818+0000","last_undegraded":"2024-09-19T04:46:41.895818+0000","last_fullsized":"2024-09-19T04:46:41.895818+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:39.138724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032947+0000","last_change":"2024-09-19T04:44:30.753817+0000","last_active":"2024-09-19T04:46:42.032947+0000","last_peered":"2024-09-19T04:46:42.032947+0000","last_clean":"2024-09-19T04:46:42.032947+0000","last_became_active":"2024-09-19T04:44:30.752541+0000","last_became_peered":"2024-09-19T04:44:30.752541+0000","last_unstale":"2024-09-19T04:46:42.032947+0000","last_undegraded":"2024-09-19T04:46:42.032947+0000","last_fullsized":"2024-09-19T04:46:42.032947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:16.121822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'38","reported_seq":310,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.117197+0000","last_change":"2024-09-19T04:44:28.335449+0000","last_active":"2024-09-19T04:46:42.117197+0000","last_peered":"2024-09-19T04:46:42.117197+0000","last_clean":"2024-09-19T04:46:42.117197+0000","last_became_active":"2024-09-19T04:44:28.335080+0000","last_became_peered":"2024-09-19T04:44:28.335080+0000","last_unstale":"2024-09-19T04:46:42.117197+0000","last_undegraded":"2024-09-19T04:46:42.117197+0000","last_fullsized":"2024-09-19T04:46:42.117197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:28.436290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66483,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":247,"num_read_kb":182,"num_write":40,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.786101+0000","last_change":"2024-09-19T04:44:30.749624+0000","last_active":"2024-09-19T04:46:41.786101+0000","last_peered":"2024-09-19T04:46:41.786101+0000","last_clean":"2024-09-19T04:46:41.786101+0000","last_became_active":"2024-09-19T04:44:30.749295+0000","last_became_peered":"2024-09-19T04:44:30.749295+0000","last_unstale":"2024-09-19T04:46:41.786101+0000","last_undegraded":"2024-09-19T04:46:41.786101+0000","last_fullsized":"2024-09-19T04:46:41.786101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:48:46.942237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.011879+0000","last_change":"2024-09-19T04:44:28.337863+0000","last_active":"2024-09-19T04:46:42.011879+0000","last_peered":"2024-09-19T04:46:42.011879+0000","last_clean":"2024-09-19T04:46:42.011879+0000","last_became_active":"2024-09-19T04:44:28.337522+0000","last_became_peered":"2024-09-19T04:44:28.337522+0000","last_unstale":"2024-09-19T04:46:42.011879+0000","last_undegraded":"2024-09-19T04:46:42.011879+0000","last_fullsized":"2024-09-19T04:46:42.011879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:02.488622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033943+0000","last_change":"2024-09-19T04:44:30.759916+0000","last_active":"2024-09-19T04:46:42.033943+0000","last_peered":"2024-09-19T04:46:42.033943+0000","last_clean":"2024-09-19T04:46:42.033943+0000","last_became_active":"2024-09-19T04:44:30.758835+0000","last_became_peered":"2024-09-19T04:44:30.758835+0000","last_unstale":"2024-09-19T04:46:42.033943+0000","last_undegraded":"2024-09-19T04:46:42.033943+0000","last_fullsized":"2024-09-19T04:46:42.033943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:38.614343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895828+0000","last_change":"2024-09-19T04:44:28.337574+0000","last_active":"2024-09-19T04:46:41.895828+0000","last_peered":"2024-09-19T04:46:41.895828+0000","last_clean":"2024-09-19T04:46:41.895828+0000","last_became_active":"2024-09-19T04:44:28.337224+0000","last_became_peered":"2024-09-19T04:44:28.337224+0000","last_unstale":"2024-09-19T04:46:41.895828+0000","last_undegraded":"2024-09-19T04:46:41.895828+0000","last_fullsized":"2024-09-19T04:46:41.895828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:20.538181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782359+0000","last_change":"2024-09-19T04:44:30.758152+0000","last_active":"2024-09-19T04:46:41.782359+0000","last_peered":"2024-09-19T04:46:41.782359+0000","last_clean":"2024-09-19T04:46:41.782359+0000","last_became_active":"2024-09-19T04:44:30.757801+0000","last_became_peered":"2024-09-19T04:44:30.757801+0000","last_unstale":"2024-09-19T04:46:41.782359+0000","last_undegraded":"2024-09-19T04:46:41.782359+0000","last_fullsized":"2024-09-19T04:46:41.782359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:05.212471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'7","reported_seq":154,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032968+0000","last_change":"2024-09-19T04:44:28.339340+0000","last_active":"2024-09-19T04:46:42.032968+0000","last_peered":"2024-09-19T04:46:42.032968+0000","last_clean":"2024-09-19T04:46:42.032968+0000","last_became_active":"2024-09-19T04:44:28.339019+0000","last_became_peered":"2024-09-19T04:44:28.339019+0000","last_unstale":"2024-09-19T04:46:42.032968+0000","last_undegraded":"2024-09-19T04:46:42.032968+0000","last_fullsized":"2024-09-19T04:46:42.032968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:29:59.356748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782063+0000","last_change":"2024-09-19T04:44:30.751217+0000","last_active":"2024-09-19T04:46:41.782063+0000","last_peered":"2024-09-19T04:46:41.782063+0000","last_clean":"2024-09-19T04:46:41.782063+0000","last_became_active":"2024-09-19T04:44:30.750863+0000","last_became_peered":"2024-09-19T04:44:30.750863+0000","last_unstale":"2024-09-19T04:46:41.782063+0000","last_undegraded":"2024-09-19T04:46:41.782063+0000","last_fullsized":"2024-09-19T04:46:41.782063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:09.957572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785881+0000","last_change":"2024-09-19T04:44:28.596585+0000","last_active":"2024-09-19T04:46:41.785881+0000","last_peered":"2024-09-19T04:46:41.785881+0000","last_clean":"2024-09-19T04:46:41.785881+0000","last_became_active":"2024-09-19T04:44:28.596218+0000","last_became_peered":"2024-09-19T04:44:28.596218+0000","last_unstale":"2024-09-19T04:46:41.785881+0000","last_undegraded":"2024-09-19T04:46:41.785881+0000","last_fullsized":"2024-09-19T04:46:41.785881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:29:35.395610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895520+0000","last_change":"2024-09-19T04:44:30.746590+0000","last_active":"2024-09-19T04:46:41.895520+0000","last_peered":"2024-09-19T04:46:41.895520+0000","last_clean":"2024-09-19T04:46:41.895520+0000","last_became_active":"2024-09-19T04:44:30.746263+0000","last_became_peered":"2024-09-19T04:44:30.746263+0000","last_unstale":"2024-09-19T04:46:41.895520+0000","last_undegraded":"2024-09-19T04:46:41.895520+0000","last_fullsized":"2024-09-19T04:46:41.895520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:38.955613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.786340+0000","last_change":"2024-09-19T04:44:28.335826+0000","last_active":"2024-09-19T04:46:41.786340+0000","last_peered":"2024-09-19T04:46:41.786340+0000","last_clean":"2024-09-19T04:46:41.786340+0000","last_became_active":"2024-09-19T04:44:28.335481+0000","last_became_peered":"2024-09-19T04:44:28.335481+0000","last_unstale":"2024-09-19T04:46:41.786340+0000","last_undegraded":"2024-09-19T04:46:41.786340+0000","last_fullsized":"2024-09-19T04:46:41.786340+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:20.989500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032963+0000","last_change":"2024-09-19T04:44:30.744130+0000","last_active":"2024-09-19T04:46:42.032963+0000","last_peered":"2024-09-19T04:46:42.032963+0000","last_clean":"2024-09-19T04:46:42.032963+0000","last_became_active":"2024-09-19T04:44:30.743772+0000","last_became_peered":"2024-09-19T04:44:30.743772+0000","last_unstale":"2024-09-19T04:46:42.032963+0000","last_undegraded":"2024-09-19T04:46:42.032963+0000","last_fullsized":"2024-09-19T04:46:42.032963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:34.956670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.034296+0000","last_change":"2024-09-19T04:44:28.337543+0000","last_active":"2024-09-19T04:46:42.034296+0000","last_peered":"2024-09-19T04:46:42.034296+0000","last_clean":"2024-09-19T04:46:42.034296+0000","last_became_active":"2024-09-19T04:44:28.336128+0000","last_became_peered":"2024-09-19T04:44:28.336128+0000","last_unstale":"2024-09-19T04:46:42.034296+0000","last_undegraded":"2024-09-19T04:46:42.034296+0000","last_fullsized":"2024-09-19T04:46:42.034296+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:31.907777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782306+0000","last_change":"2024-09-19T04:44:30.757489+0000","last_active":"2024-09-19T04:46:41.782306+0000","last_peered":"2024-09-19T04:46:41.782306+0000","last_clean":"2024-09-19T04:46:41.782306+0000","last_became_active":"2024-09-19T04:44:30.757041+0000","last_became_peered":"2024-09-19T04:44:30.757041+0000","last_unstale":"2024-09-19T04:46:41.782306+0000","last_undegraded":"2024-09-19T04:46:41.782306+0000","last_fullsized":"2024-09-19T04:46:41.782306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:03.395913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'41","reported_seq":75,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.049157+0000","last_change":"2024-09-19T04:44:28.327393+0000","last_active":"2024-09-19T04:46:42.049157+0000","last_peered":"2024-09-19T04:46:42.049157+0000","last_clean":"2024-09-19T04:46:42.049157+0000","last_became_active":"2024-09-19T04:44:28.327116+0000","last_became_peered":"2024-09-19T04:44:28.327116+0000","last_unstale":"2024-09-19T04:46:42.049157+0000","last_undegraded":"2024-09-19T04:46:42.049157+0000","last_fullsized":"2024-09-19T04:46:42.049157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:42.360980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782342+0000","last_change":"2024-09-19T04:44:30.754239+0000","last_active":"2024-09-19T04:46:41.782342+0000","last_peered":"2024-09-19T04:46:41.782342+0000","last_clean":"2024-09-19T04:46:41.782342+0000","last_became_active":"2024-09-19T04:44:30.753885+0000","last_became_peered":"2024-09-19T04:44:30.753885+0000","last_unstale":"2024-09-19T04:46:41.782342+0000","last_undegraded":"2024-09-19T04:46:41.782342+0000","last_fullsized":"2024-09-19T04:46:41.782342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:27.355600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'5","reported_seq":152,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.012256+0000","last_change":"2024-09-19T04:44:28.596214+0000","last_active":"2024-09-19T04:46:42.012256+0000","last_peered":"2024-09-19T04:46:42.012256+0000","last_clean":"2024-09-19T04:46:42.012256+0000","last_became_active":"2024-09-19T04:44:28.595841+0000","last_became_peered":"2024-09-19T04:44:28.595841+0000","last_unstale":"2024-09-19T04:46:42.012256+0000","last_undegraded":"2024-09-19T04:46:42.012256+0000","last_fullsized":"2024-09-19T04:46:42.012256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:43.864702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782627+0000","last_change":"2024-09-19T04:44:30.756192+0000","last_active":"2024-09-19T04:46:41.782627+0000","last_peered":"2024-09-19T04:46:41.782627+0000","last_clean":"2024-09-19T04:46:41.782627+0000","last_became_active":"2024-09-19T04:44:30.755853+0000","last_became_peered":"2024-09-19T04:44:30.755853+0000","last_unstale":"2024-09-19T04:46:41.782627+0000","last_undegraded":"2024-09-19T04:46:41.782627+0000","last_fullsized":"2024-09-19T04:46:41.782627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:25:49.206978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":352,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782231+0000","last_change":"2024-09-19T04:43:10.156816+0000","last_active":"2024-09-19T04:46:41.782231+0000","last_peered":"2024-09-19T04:46:41.782231+0000","last_clean":"2024-09-19T04:46:41.782231+0000","last_became_active":"2024-09-19T04:43:10.156359+0000","last_became_peered":"2024-09-19T04:43:10.156359+0000","last_unstale":"2024-09-19T04:46:41.782231+0000","last_undegraded":"2024-09-19T04:46:41.782231+0000","last_fullsized":"2024-09-19T04:46:41.782231+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:39:34.200793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:39:34.200793+0000","last_clean_scrub_stamp":"2024-09-19T04:39:34.200793+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:13.453818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032920+0000","last_change":"2024-09-19T04:44:28.337637+0000","last_active":"2024-09-19T04:46:42.032920+0000","last_peered":"2024-09-19T04:46:42.032920+0000","last_clean":"2024-09-19T04:46:42.032920+0000","last_became_active":"2024-09-19T04:44:28.337235+0000","last_became_peered":"2024-09-19T04:44:28.337235+0000","last_unstale":"2024-09-19T04:46:42.032920+0000","last_undegraded":"2024-09-19T04:46:42.032920+0000","last_fullsized":"2024-09-19T04:46:42.032920+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:42.104972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.881909+0000","last_change":"2024-09-19T04:45:15.281422+0000","last_active":"2024-09-19T04:46:41.881909+0000","last_peered":"2024-09-19T04:46:41.881909+0000","last_clean":"2024-09-19T04:46:41.881909+0000","last_became_active":"2024-09-19T04:45:15.281031+0000","last_became_peered":"2024-09-19T04:45:15.281031+0000","last_unstale":"2024-09-19T04:46:41.881909+0000","last_undegraded":"2024-09-19T04:46:41.881909+0000","last_fullsized":"2024-09-19T04:46:41.881909+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:05.058469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782204+0000","last_change":"2024-09-19T04:44:28.336124+0000","last_active":"2024-09-19T04:46:41.782204+0000","last_peered":"2024-09-19T04:46:41.782204+0000","last_clean":"2024-09-19T04:46:41.782204+0000","last_became_active":"2024-09-19T04:44:28.335754+0000","last_became_peered":"2024-09-19T04:44:28.335754+0000","last_unstale":"2024-09-19T04:46:41.782204+0000","last_undegraded":"2024-09-19T04:46:41.782204+0000","last_fullsized":"2024-09-19T04:46:41.782204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:41.604367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033916+0000","last_change":"2024-09-19T04:44:30.749939+0000","last_active":"2024-09-19T04:46:42.033916+0000","last_peered":"2024-09-19T04:46:42.033916+0000","last_clean":"2024-09-19T04:46:42.033916+0000","last_became_active":"2024-09-19T04:44:30.747122+0000","last_became_peered":"2024-09-19T04:44:30.747122+0000","last_unstale":"2024-09-19T04:46:42.033916+0000","last_undegraded":"2024-09-19T04:46:42.033916+0000","last_fullsized":"2024-09-19T04:46:42.033916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:46:08.371493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":151,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.039159+0000","last_change":"2024-09-19T04:44:28.594768+0000","last_active":"2024-09-19T04:46:42.039159+0000","last_peered":"2024-09-19T04:46:42.039159+0000","last_clean":"2024-09-19T04:46:42.039159+0000","last_became_active":"2024-09-19T04:44:28.594320+0000","last_became_peered":"2024-09-19T04:44:28.594320+0000","last_unstale":"2024-09-19T04:46:42.039159+0000","last_undegraded":"2024-09-19T04:46:42.039159+0000","last_fullsized":"2024-09-19T04:46:42.039159+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:32:17.736874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895518+0000","last_change":"2024-09-19T04:44:30.746666+0000","last_active":"2024-09-19T04:46:41.895518+0000","last_peered":"2024-09-19T04:46:41.895518+0000","last_clean":"2024-09-19T04:46:41.895518+0000","last_became_active":"2024-09-19T04:44:30.746312+0000","last_became_peered":"2024-09-19T04:44:30.746312+0000","last_unstale":"2024-09-19T04:46:41.895518+0000","last_undegraded":"2024-09-19T04:46:41.895518+0000","last_fullsized":"2024-09-19T04:46:41.895518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:13.562810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"87'7","reported_seq":153,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.117972+0000","last_change":"2024-09-19T04:44:28.336492+0000","last_active":"2024-09-19T04:46:42.117972+0000","last_peered":"2024-09-19T04:46:42.117972+0000","last_clean":"2024-09-19T04:46:42.117972+0000","last_became_active":"2024-09-19T04:44:28.336044+0000","last_became_peered":"2024-09-19T04:44:28.336044+0000","last_unstale":"2024-09-19T04:46:42.117972+0000","last_undegraded":"2024-09-19T04:46:42.117972+0000","last_fullsized":"2024-09-19T04:46:42.117972+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:18.760473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.899051+0000","last_change":"2024-09-19T04:45:14.630959+0000","last_active":"2024-09-19T04:46:41.899051+0000","last_peered":"2024-09-19T04:46:41.899051+0000","last_clean":"2024-09-19T04:46:41.899051+0000","last_became_active":"2024-09-19T04:45:14.630528+0000","last_became_peered":"2024-09-19T04:45:14.630528+0000","last_unstale":"2024-09-19T04:46:41.899051+0000","last_undegraded":"2024-09-19T04:46:41.899051+0000","last_fullsized":"2024-09-19T04:46:41.899051+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:29.754355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.882436+0000","last_change":"2024-09-19T04:44:28.596322+0000","last_active":"2024-09-19T04:46:41.882436+0000","last_peered":"2024-09-19T04:46:41.882436+0000","last_clean":"2024-09-19T04:46:41.882436+0000","last_became_active":"2024-09-19T04:44:28.595566+0000","last_became_peered":"2024-09-19T04:44:28.595566+0000","last_unstale":"2024-09-19T04:46:41.882436+0000","last_undegraded":"2024-09-19T04:46:41.882436+0000","last_fullsized":"2024-09-19T04:46:41.882436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:23.878145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.899012+0000","last_change":"2024-09-19T04:44:30.752321+0000","last_active":"2024-09-19T04:46:41.899012+0000","last_peered":"2024-09-19T04:46:41.899012+0000","last_clean":"2024-09-19T04:46:41.899012+0000","last_became_active":"2024-09-19T04:44:30.751980+0000","last_became_peered":"2024-09-19T04:44:30.751980+0000","last_unstale":"2024-09-19T04:46:41.899012+0000","last_undegraded":"2024-09-19T04:46:41.899012+0000","last_fullsized":"2024-09-19T04:46:41.899012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:41.863034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.786281+0000","last_change":"2024-09-19T04:44:28.596547+0000","last_active":"2024-09-19T04:46:41.786281+0000","last_peered":"2024-09-19T04:46:41.786281+0000","last_clean":"2024-09-19T04:46:41.786281+0000","last_became_active":"2024-09-19T04:44:28.596181+0000","last_became_peered":"2024-09-19T04:44:28.596181+0000","last_unstale":"2024-09-19T04:46:41.786281+0000","last_undegraded":"2024-09-19T04:46:41.786281+0000","last_fullsized":"2024-09-19T04:46:41.786281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:53.251737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.034777+0000","last_change":"2024-09-19T04:44:30.751533+0000","last_active":"2024-09-19T04:46:42.034777+0000","last_peered":"2024-09-19T04:46:42.034777+0000","last_clean":"2024-09-19T04:46:42.034777+0000","last_became_active":"2024-09-19T04:44:30.748012+0000","last_became_peered":"2024-09-19T04:44:30.748012+0000","last_unstale":"2024-09-19T04:46:42.034777+0000","last_undegraded":"2024-09-19T04:46:42.034777+0000","last_fullsized":"2024-09-19T04:46:42.034777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:32.360147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'52","reported_seq":102,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.770637+0000","last_change":"2024-09-19T04:44:28.335884+0000","last_active":"2024-09-19T04:46:42.770637+0000","last_peered":"2024-09-19T04:46:42.770637+0000","last_clean":"2024-09-19T04:46:42.770637+0000","last_became_active":"2024-09-19T04:44:28.335521+0000","last_became_peered":"2024-09-19T04:44:28.335521+0000","last_unstale":"2024-09-19T04:46:42.770637+0000","last_undegraded":"2024-09-19T04:46:42.770637+0000","last_fullsized":"2024-09-19T04:46:42.770637+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:32:49.764893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.887262+0000","last_change":"2024-09-19T04:44:30.759578+0000","last_active":"2024-09-19T04:46:41.887262+0000","last_peered":"2024-09-19T04:46:41.887262+0000","last_clean":"2024-09-19T04:46:41.887262+0000","last_became_active":"2024-09-19T04:44:30.759245+0000","last_became_peered":"2024-09-19T04:44:30.759245+0000","last_unstale":"2024-09-19T04:46:41.887262+0000","last_undegraded":"2024-09-19T04:46:41.887262+0000","last_fullsized":"2024-09-19T04:46:41.887262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:37.121770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033302+0000","last_change":"2024-09-19T04:44:28.593404+0000","last_active":"2024-09-19T04:46:42.033302+0000","last_peered":"2024-09-19T04:46:42.033302+0000","last_clean":"2024-09-19T04:46:42.033302+0000","last_became_active":"2024-09-19T04:44:28.593037+0000","last_became_peered":"2024-09-19T04:44:28.593037+0000","last_unstale":"2024-09-19T04:46:42.033302+0000","last_undegraded":"2024-09-19T04:46:42.033302+0000","last_fullsized":"2024-09-19T04:46:42.033302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:08:29.218543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.783007+0000","last_change":"2024-09-19T04:45:14.891783+0000","last_active":"2024-09-19T04:46:41.783007+0000","last_peered":"2024-09-19T04:46:41.783007+0000","last_clean":"2024-09-19T04:46:41.783007+0000","last_became_active":"2024-09-19T04:45:14.891396+0000","last_became_peered":"2024-09-19T04:45:14.891396+0000","last_unstale":"2024-09-19T04:46:41.783007+0000","last_undegraded":"2024-09-19T04:46:41.783007+0000","last_fullsized":"2024-09-19T04:46:41.783007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:56.443270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032529+0000","last_change":"2024-09-19T04:44:28.594541+0000","last_active":"2024-09-19T04:46:42.032529+0000","last_peered":"2024-09-19T04:46:42.032529+0000","last_clean":"2024-09-19T04:46:42.032529+0000","last_became_active":"2024-09-19T04:44:28.594262+0000","last_became_peered":"2024-09-19T04:44:28.594262+0000","last_unstale":"2024-09-19T04:46:42.032529+0000","last_undegraded":"2024-09-19T04:46:42.032529+0000","last_fullsized":"2024-09-19T04:46:42.032529+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:45.685034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785728+0000","last_change":"2024-09-19T04:44:30.757861+0000","last_active":"2024-09-19T04:46:41.785728+0000","last_peered":"2024-09-19T04:46:41.785728+0000","last_clean":"2024-09-19T04:46:41.785728+0000","last_became_active":"2024-09-19T04:44:30.757509+0000","last_became_peered":"2024-09-19T04:44:30.757509+0000","last_unstale":"2024-09-19T04:46:41.785728+0000","last_undegraded":"2024-09-19T04:46:41.785728+0000","last_fullsized":"2024-09-19T04:46:41.785728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:17.737055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886469+0000","last_change":"2024-09-19T04:44:28.335225+0000","last_active":"2024-09-19T04:46:41.886469+0000","last_peered":"2024-09-19T04:46:41.886469+0000","last_clean":"2024-09-19T04:46:41.886469+0000","last_became_active":"2024-09-19T04:44:28.334804+0000","last_became_peered":"2024-09-19T04:44:28.334804+0000","last_unstale":"2024-09-19T04:46:41.886469+0000","last_undegraded":"2024-09-19T04:46:41.886469+0000","last_fullsized":"2024-09-19T04:46:41.886469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:15.639418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033015+0000","last_change":"2024-09-19T04:44:30.754018+0000","last_active":"2024-09-19T04:46:42.033015+0000","last_peered":"2024-09-19T04:46:42.033015+0000","last_clean":"2024-09-19T04:46:42.033015+0000","last_became_active":"2024-09-19T04:44:30.753639+0000","last_became_peered":"2024-09-19T04:44:30.753639+0000","last_unstale":"2024-09-19T04:46:42.033015+0000","last_undegraded":"2024-09-19T04:46:42.033015+0000","last_fullsized":"2024-09-19T04:46:42.033015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:05.012847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032760+0000","last_change":"2024-09-19T04:45:14.636102+0000","last_active":"2024-09-19T04:46:42.032760+0000","last_peered":"2024-09-19T04:46:42.032760+0000","last_clean":"2024-09-19T04:46:42.032760+0000","last_became_active":"2024-09-19T04:45:14.631201+0000","last_became_peered":"2024-09-19T04:45:14.631201+0000","last_unstale":"2024-09-19T04:46:42.032760+0000","last_undegraded":"2024-09-19T04:46:42.032760+0000","last_fullsized":"2024-09-19T04:46:42.032760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:04.331207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886472+0000","last_change":"2024-09-19T04:44:30.758543+0000","last_active":"2024-09-19T04:46:41.886472+0000","last_peered":"2024-09-19T04:46:41.886472+0000","last_clean":"2024-09-19T04:46:41.886472+0000","last_became_active":"2024-09-19T04:44:30.757228+0000","last_became_peered":"2024-09-19T04:44:30.757228+0000","last_unstale":"2024-09-19T04:46:41.886472+0000","last_undegraded":"2024-09-19T04:46:41.886472+0000","last_fullsized":"2024-09-19T04:46:41.886472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:13.877742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785932+0000","last_change":"2024-09-19T04:44:28.339152+0000","last_active":"2024-09-19T04:46:41.785932+0000","last_peered":"2024-09-19T04:46:41.785932+0000","last_clean":"2024-09-19T04:46:41.785932+0000","last_became_active":"2024-09-19T04:44:28.338778+0000","last_became_peered":"2024-09-19T04:44:28.338778+0000","last_unstale":"2024-09-19T04:46:41.785932+0000","last_undegraded":"2024-09-19T04:46:41.785932+0000","last_fullsized":"2024-09-19T04:46:41.785932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:53.906755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782285+0000","last_change":"2024-09-19T04:44:30.759330+0000","last_active":"2024-09-19T04:46:41.782285+0000","last_peered":"2024-09-19T04:46:41.782285+0000","last_clean":"2024-09-19T04:46:41.782285+0000","last_became_active":"2024-09-19T04:44:30.758987+0000","last_became_peered":"2024-09-19T04:44:30.758987+0000","last_unstale":"2024-09-19T04:46:41.782285+0000","last_undegraded":"2024-09-19T04:46:41.782285+0000","last_fullsized":"2024-09-19T04:46:41.782285+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:10.105051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895780+0000","last_change":"2024-09-19T04:44:30.748089+0000","last_active":"2024-09-19T04:46:41.895780+0000","last_peered":"2024-09-19T04:46:41.895780+0000","last_clean":"2024-09-19T04:46:41.895780+0000","last_became_active":"2024-09-19T04:44:30.747566+0000","last_became_peered":"2024-09-19T04:44:30.747566+0000","last_unstale":"2024-09-19T04:46:41.895780+0000","last_undegraded":"2024-09-19T04:46:41.895780+0000","last_fullsized":"2024-09-19T04:46:41.895780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:32.406745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.786105+0000","last_change":"2024-09-19T04:44:28.334217+0000","last_active":"2024-09-19T04:46:41.786105+0000","last_peered":"2024-09-19T04:46:41.786105+0000","last_clean":"2024-09-19T04:46:41.786105+0000","last_became_active":"2024-09-19T04:44:28.333768+0000","last_became_peered":"2024-09-19T04:44:28.333768+0000","last_unstale":"2024-09-19T04:46:41.786105+0000","last_undegraded":"2024-09-19T04:46:41.786105+0000","last_fullsized":"2024-09-19T04:46:41.786105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:52:56.626635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032821+0000","last_change":"2024-09-19T04:44:30.747498+0000","last_active":"2024-09-19T04:46:42.032821+0000","last_peered":"2024-09-19T04:46:42.032821+0000","last_clean":"2024-09-19T04:46:42.032821+0000","last_became_active":"2024-09-19T04:44:30.746822+0000","last_became_peered":"2024-09-19T04:44:30.746822+0000","last_unstale":"2024-09-19T04:46:42.032821+0000","last_undegraded":"2024-09-19T04:46:42.032821+0000","last_fullsized":"2024-09-19T04:46:42.032821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:09.460263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032840+0000","last_change":"2024-09-19T04:45:14.645843+0000","last_active":"2024-09-19T04:46:42.032840+0000","last_peered":"2024-09-19T04:46:42.032840+0000","last_clean":"2024-09-19T04:46:42.032840+0000","last_became_active":"2024-09-19T04:45:14.632156+0000","last_became_peered":"2024-09-19T04:45:14.632156+0000","last_unstale":"2024-09-19T04:46:42.032840+0000","last_undegraded":"2024-09-19T04:46:42.032840+0000","last_fullsized":"2024-09-19T04:46:42.032840+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:34.406160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782880+0000","last_change":"2024-09-19T04:44:30.762776+0000","last_active":"2024-09-19T04:46:41.782880+0000","last_peered":"2024-09-19T04:46:41.782880+0000","last_clean":"2024-09-19T04:46:41.782880+0000","last_became_active":"2024-09-19T04:44:30.761308+0000","last_became_peered":"2024-09-19T04:44:30.761308+0000","last_unstale":"2024-09-19T04:46:41.782880+0000","last_undegraded":"2024-09-19T04:46:41.782880+0000","last_fullsized":"2024-09-19T04:46:41.782880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:19.960125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.887257+0000","last_change":"2024-09-19T04:44:28.594858+0000","last_active":"2024-09-19T04:46:41.887257+0000","last_peered":"2024-09-19T04:46:41.887257+0000","last_clean":"2024-09-19T04:46:41.887257+0000","last_became_active":"2024-09-19T04:44:28.594515+0000","last_became_peered":"2024-09-19T04:44:28.594515+0000","last_unstale":"2024-09-19T04:46:41.887257+0000","last_undegraded":"2024-09-19T04:46:41.887257+0000","last_fullsized":"2024-09-19T04:46:41.887257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:53.728393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886920+0000","last_change":"2024-09-19T04:44:30.753604+0000","last_active":"2024-09-19T04:46:41.886920+0000","last_peered":"2024-09-19T04:46:41.886920+0000","last_clean":"2024-09-19T04:46:41.886920+0000","last_became_active":"2024-09-19T04:44:30.751961+0000","last_became_peered":"2024-09-19T04:44:30.751961+0000","last_unstale":"2024-09-19T04:46:41.886920+0000","last_undegraded":"2024-09-19T04:46:41.886920+0000","last_fullsized":"2024-09-19T04:46:41.886920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:46:58.759650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"87'14","reported_seq":160,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.010647+0000","last_change":"2024-09-19T04:44:28.339499+0000","last_active":"2024-09-19T04:46:42.010647+0000","last_peered":"2024-09-19T04:46:42.010647+0000","last_clean":"2024-09-19T04:46:42.010647+0000","last_became_active":"2024-09-19T04:44:28.338774+0000","last_became_peered":"2024-09-19T04:44:28.338774+0000","last_unstale":"2024-09-19T04:46:42.010647+0000","last_undegraded":"2024-09-19T04:46:42.010647+0000","last_fullsized":"2024-09-19T04:46:42.010647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:12.261237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599827,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1313,"num_read_kb":763,"num_write":372,"num_write_kb":1461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1799481,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108024,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481833,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":954368,"data_stored":464859,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710765,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":1254505,"data_compressed":45058,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874290,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":858005,"data_compressed":22040,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":465167,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364860,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1028096,"data_stored":1057828,"data_compressed":38902,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528387,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":962560,"data_stored":465292,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1189358,"data_compressed":45058,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953548,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":464901,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1024000,"data_stored":989147,"data_compressed":28195,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1052672,"data_stored":532908,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":532688,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":465845,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135206,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200353,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:46:47.074 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:46:47.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:47.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:47.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:47.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:47.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:47.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:47.102 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:47.102 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:47.102 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:47.102 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:46:47.102 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi123 to 176.4M 2024-09-19T04:46:47.102 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: Unable to set osd_memory_target on smithi123 to 185036185: error parsing value: Value '185036185' is below minimum 939524096 2024-09-19T04:46:47.102 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:46:47.102 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:46:47.102 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-19T04:46:47.102 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-19T04:46:47.102 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-19T04:46:47.102 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1668281908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:46:47.102 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:47.102 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:47.103 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:47.103 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:46 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:47.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.264+0000 7f31f3e00640 1 -- 172.21.15.123:0/4021025599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4107900 msgr2=0x7f31f4107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:47.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.264+0000 7f31f3e00640 1 --2- 172.21.15.123:0/4021025599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4107900 0x7f31f4107ce0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009930 tx=0x7f31e402a420 comp rx=0 tx=0).stop 2024-09-19T04:46:47.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.264+0000 7f31f3e00640 1 -- 172.21.15.123:0/4021025599 shutdown_connections 2024-09-19T04:46:47.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.264+0000 7f31f3e00640 1 --2- 172.21.15.123:0/4021025599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31f4102290 0x7f31f410e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.264+0000 7f31f3e00640 1 --2- 172.21.15.123:0/4021025599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f41018f0 0x7f31f4101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.264+0000 7f31f3e00640 1 --2- 172.21.15.123:0/4021025599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4107900 0x7f31f4107ce0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.264+0000 7f31f3e00640 1 -- 172.21.15.123:0/4021025599 >> 172.21.15.123:0/4021025599 conn(0x7f31f40fd610 msgr2=0x7f31f40ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.264+0000 7f31f3e00640 1 -- 172.21.15.123:0/4021025599 shutdown_connections 2024-09-19T04:46:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.264+0000 7f31f3e00640 1 -- 172.21.15.123:0/4021025599 wait complete. 2024-09-19T04:46:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.265+0000 7f31f3e00640 1 Processor -- start 2024-09-19T04:46:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.265+0000 7f31f3e00640 1 -- start start 2024-09-19T04:46:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.265+0000 7f31f3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31f41018f0 0x7f31f40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.266+0000 7f31f3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f4102290 0x7f31f4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.266+0000 7f31f3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4107900 0x7f31f4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.266+0000 7f31f3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4111d90 con 0x7f31f4107900 2024-09-19T04:46:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.266+0000 7f31f3e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4111c10 con 0x7f31f41018f0 2024-09-19T04:46:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.266+0000 7f31f3e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4111f10 con 0x7f31f4102290 2024-09-19T04:46:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.266+0000 7f31f3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4107900 0x7f31f4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.266+0000 7f31f3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4107900 0x7f31f4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50662/0 (socket says 172.21.15.123:50662) 2024-09-19T04:46:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.266+0000 7f31f3400640 1 -- 172.21.15.123:0/2667236754 learned_addr learned my addr 172.21.15.123:0/2667236754 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.266+0000 7f31f2a00640 1 --2- 172.21.15.123:0/2667236754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31f41018f0 0x7f31f40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.266+0000 7f31ebe00640 1 --2- 172.21.15.123:0/2667236754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f4102290 0x7f31f4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.266+0000 7f31f3400640 1 -- 172.21.15.123:0/2667236754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f4102290 msgr2=0x7f31f4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.266+0000 7f31f3400640 1 --2- 172.21.15.123:0/2667236754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f4102290 0x7f31f4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.266+0000 7f31f3400640 1 -- 172.21.15.123:0/2667236754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31f41018f0 msgr2=0x7f31f40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.266+0000 7f31f3400640 1 --2- 172.21.15.123:0/2667236754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31f41018f0 0x7f31f40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.266+0000 7f31f3400640 1 -- 172.21.15.123:0/2667236754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f41a67a0 con 0x7f31f4107900 2024-09-19T04:46:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.266+0000 7f31f2a00640 1 --2- 172.21.15.123:0/2667236754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31f41018f0 0x7f31f40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.267+0000 7f31f3400640 1 --2- 172.21.15.123:0/2667236754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4107900 0x7f31f4074f70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00eb10 tx=0x7f31ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.267+0000 7f31eb400640 1 -- 172.21.15.123:0/2667236754 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec004b20 con 0x7f31f4107900 2024-09-19T04:46:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.267+0000 7f31eb400640 1 -- 172.21.15.123:0/2667236754 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31ec004cc0 con 0x7f31f4107900 2024-09-19T04:46:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.267+0000 7f31f3e00640 1 -- 172.21.15.123:0/2667236754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f41a6a90 con 0x7f31f4107900 2024-09-19T04:46:47.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.267+0000 7f31eb400640 1 -- 172.21.15.123:0/2667236754 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec010760 con 0x7f31f4107900 2024-09-19T04:46:47.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.267+0000 7f31f3e00640 1 -- 172.21.15.123:0/2667236754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f41a6f90 con 0x7f31f4107900 2024-09-19T04:46:47.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.268+0000 7f31f3e00640 1 -- 172.21.15.123:0/2667236754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f4101d50 con 0x7f31f4107900 2024-09-19T04:46:47.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.270+0000 7f31eb400640 1 -- 172.21.15.123:0/2667236754 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 138583+0+0 (secure 0 0 0) 0x7f31ec005020 con 0x7f31f4107900 2024-09-19T04:46:47.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.272+0000 7f31eb400640 1 --2- 172.21.15.123:0/2667236754 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31d80a34e0 0x7f31d80a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:47.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.272+0000 7f31eb400640 1 -- 172.21.15.123:0/2667236754 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f31ec0d04f0 con 0x7f31f4107900 2024-09-19T04:46:47.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.272+0000 7f31f2a00640 1 --2- 172.21.15.123:0/2667236754 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31d80a34e0 0x7f31d80a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:47.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.273+0000 7f31f2a00640 1 --2- 172.21.15.123:0/2667236754 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31d80a34e0 0x7f31d80a59a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f31e4006010 tx=0x7f31e4031510 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:47.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.277+0000 7f31eb400640 1 -- 172.21.15.123:0/2667236754 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f31ec09cc50 con 0x7f31f4107900 2024-09-19T04:46:47.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.461+0000 7f31f3e00640 1 -- 172.21.15.123:0/2667236754 --> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f31f4062630 con 0x7f31d80a34e0 2024-09-19T04:46:47.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.468+0000 7f31eb400640 1 -- 172.21.15.123:0/2667236754 <== mgr.25133 v2:172.21.15.123:6800/1578549821 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258051 (secure 0 0 0) 0x7f31f4062630 con 0x7f31d80a34e0 2024-09-19T04:46:47.474 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:46:47.476 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-09-19T04:46:47.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.472+0000 7f31f3e00640 1 -- 172.21.15.123:0/2667236754 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31d80a34e0 msgr2=0x7f31d80a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:47.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.472+0000 7f31f3e00640 1 --2- 172.21.15.123:0/2667236754 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31d80a34e0 0x7f31d80a59a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f31e4006010 tx=0x7f31e4031510 comp rx=0 tx=0).stop 2024-09-19T04:46:47.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.472+0000 7f31f3e00640 1 -- 172.21.15.123:0/2667236754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4107900 msgr2=0x7f31f4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:47.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.472+0000 7f31f3e00640 1 --2- 172.21.15.123:0/2667236754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4107900 0x7f31f4074f70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00eb10 tx=0x7f31ec00c2d0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.473+0000 7f31f3e00640 1 -- 172.21.15.123:0/2667236754 shutdown_connections 2024-09-19T04:46:47.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.473+0000 7f31f3e00640 1 --2- 172.21.15.123:0/2667236754 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31d80a34e0 0x7f31d80a59a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.473+0000 7f31f3e00640 1 --2- 172.21.15.123:0/2667236754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4107900 0x7f31f4074f70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.473+0000 7f31f3e00640 1 --2- 172.21.15.123:0/2667236754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f4102290 0x7f31f4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.473+0000 7f31f3e00640 1 --2- 172.21.15.123:0/2667236754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31f41018f0 0x7f31f40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.473+0000 7f31f3e00640 1 -- 172.21.15.123:0/2667236754 >> 172.21.15.123:0/2667236754 conn(0x7f31f40fd610 msgr2=0x7f31f40fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:47.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.473+0000 7f31f3e00640 1 -- 172.21.15.123:0/2667236754 shutdown_connections 2024-09-19T04:46:47.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.473+0000 7f31f3e00640 1 -- 172.21.15.123:0/2667236754 wait complete. 2024-09-19T04:46:47.490 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T04:46:45.796478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190429,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1449,"num_read_kb":882,"num_write":555,"num_write_kb":4401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":938,"ondisk_log_size":938,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357032,"kb_used_data":11892,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584792,"statfs":{"total":1151940427776,"available":1151574827008,"internally_reserved":0,"allocated":12177408,"data_stored":8740503,"data_compressed":199511,"data_compressed_allocated":258048,"data_compressed_original":3022848,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.977753"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033086+0000","last_change":"2024-09-19T04:44:30.754400+0000","last_active":"2024-09-19T04:46:42.033086+0000","last_peered":"2024-09-19T04:46:42.033086+0000","last_clean":"2024-09-19T04:46:42.033086+0000","last_became_active":"2024-09-19T04:44:30.754028+0000","last_became_peered":"2024-09-19T04:44:30.754028+0000","last_unstale":"2024-09-19T04:46:42.033086+0000","last_undegraded":"2024-09-19T04:46:42.033086+0000","last_fullsized":"2024-09-19T04:46:42.033086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:25.668478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782861+0000","last_change":"2024-09-19T04:44:30.758072+0000","last_active":"2024-09-19T04:46:41.782861+0000","last_peered":"2024-09-19T04:46:41.782861+0000","last_clean":"2024-09-19T04:46:41.782861+0000","last_became_active":"2024-09-19T04:44:30.756734+0000","last_became_peered":"2024-09-19T04:44:30.756734+0000","last_unstale":"2024-09-19T04:46:41.782861+0000","last_undegraded":"2024-09-19T04:46:41.782861+0000","last_fullsized":"2024-09-19T04:46:41.782861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:56:47.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782852+0000","last_change":"2024-09-19T04:44:30.760208+0000","last_active":"2024-09-19T04:46:41.782852+0000","last_peered":"2024-09-19T04:46:41.782852+0000","last_clean":"2024-09-19T04:46:41.782852+0000","last_became_active":"2024-09-19T04:44:30.759719+0000","last_became_peered":"2024-09-19T04:44:30.759719+0000","last_unstale":"2024-09-19T04:46:41.782852+0000","last_undegraded":"2024-09-19T04:46:41.782852+0000","last_fullsized":"2024-09-19T04:46:41.782852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:14.041853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033102+0000","last_change":"2024-09-19T04:44:30.747813+0000","last_active":"2024-09-19T04:46:42.033102+0000","last_peered":"2024-09-19T04:46:42.033102+0000","last_clean":"2024-09-19T04:46:42.033102+0000","last_became_active":"2024-09-19T04:44:30.747486+0000","last_became_peered":"2024-09-19T04:44:30.747486+0000","last_unstale":"2024-09-19T04:46:42.033102+0000","last_undegraded":"2024-09-19T04:46:42.033102+0000","last_fullsized":"2024-09-19T04:46:42.033102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:11.315273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785693+0000","last_change":"2024-09-19T04:44:30.755800+0000","last_active":"2024-09-19T04:46:41.785693+0000","last_peered":"2024-09-19T04:46:41.785693+0000","last_clean":"2024-09-19T04:46:41.785693+0000","last_became_active":"2024-09-19T04:44:30.755448+0000","last_became_peered":"2024-09-19T04:44:30.755448+0000","last_unstale":"2024-09-19T04:46:41.785693+0000","last_undegraded":"2024-09-19T04:46:41.785693+0000","last_fullsized":"2024-09-19T04:46:41.785693+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:39.308614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.898818+0000","last_change":"2024-09-19T04:44:30.746230+0000","last_active":"2024-09-19T04:46:41.898818+0000","last_peered":"2024-09-19T04:46:41.898818+0000","last_clean":"2024-09-19T04:46:41.898818+0000","last_became_active":"2024-09-19T04:44:30.745884+0000","last_became_peered":"2024-09-19T04:44:30.745884+0000","last_unstale":"2024-09-19T04:46:41.898818+0000","last_undegraded":"2024-09-19T04:46:41.898818+0000","last_fullsized":"2024-09-19T04:46:41.898818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:51:58.550322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782104+0000","last_change":"2024-09-19T04:44:30.747870+0000","last_active":"2024-09-19T04:46:41.782104+0000","last_peered":"2024-09-19T04:46:41.782104+0000","last_clean":"2024-09-19T04:46:41.782104+0000","last_became_active":"2024-09-19T04:44:30.747536+0000","last_became_peered":"2024-09-19T04:44:30.747536+0000","last_unstale":"2024-09-19T04:46:41.782104+0000","last_undegraded":"2024-09-19T04:46:41.782104+0000","last_fullsized":"2024-09-19T04:46:41.782104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:20.074032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886225+0000","last_change":"2024-09-19T04:44:30.750425+0000","last_active":"2024-09-19T04:46:41.886225+0000","last_peered":"2024-09-19T04:46:41.886225+0000","last_clean":"2024-09-19T04:46:41.886225+0000","last_became_active":"2024-09-19T04:44:30.749841+0000","last_became_peered":"2024-09-19T04:44:30.749841+0000","last_unstale":"2024-09-19T04:46:41.886225+0000","last_undegraded":"2024-09-19T04:46:41.886225+0000","last_fullsized":"2024-09-19T04:46:41.886225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:12:08.420727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032832+0000","last_change":"2024-09-19T04:44:30.750624+0000","last_active":"2024-09-19T04:46:42.032832+0000","last_peered":"2024-09-19T04:46:42.032832+0000","last_clean":"2024-09-19T04:46:42.032832+0000","last_became_active":"2024-09-19T04:44:30.748370+0000","last_became_peered":"2024-09-19T04:44:30.748370+0000","last_unstale":"2024-09-19T04:46:42.032832+0000","last_undegraded":"2024-09-19T04:46:42.032832+0000","last_fullsized":"2024-09-19T04:46:42.032832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:36.131634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.899170+0000","last_change":"2024-09-19T04:44:30.750035+0000","last_active":"2024-09-19T04:46:41.899170+0000","last_peered":"2024-09-19T04:46:41.899170+0000","last_clean":"2024-09-19T04:46:41.899170+0000","last_became_active":"2024-09-19T04:44:30.749699+0000","last_became_peered":"2024-09-19T04:44:30.749699+0000","last_unstale":"2024-09-19T04:46:41.899170+0000","last_undegraded":"2024-09-19T04:46:41.899170+0000","last_fullsized":"2024-09-19T04:46:41.899170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:22.637193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782572+0000","last_change":"2024-09-19T04:44:30.760258+0000","last_active":"2024-09-19T04:46:41.782572+0000","last_peered":"2024-09-19T04:46:41.782572+0000","last_clean":"2024-09-19T04:46:41.782572+0000","last_became_active":"2024-09-19T04:44:30.759925+0000","last_became_peered":"2024-09-19T04:44:30.759925+0000","last_unstale":"2024-09-19T04:46:41.782572+0000","last_undegraded":"2024-09-19T04:46:41.782572+0000","last_fullsized":"2024-09-19T04:46:41.782572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:58.144691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033125+0000","last_change":"2024-09-19T04:44:30.747703+0000","last_active":"2024-09-19T04:46:42.033125+0000","last_peered":"2024-09-19T04:46:42.033125+0000","last_clean":"2024-09-19T04:46:42.033125+0000","last_became_active":"2024-09-19T04:44:30.747369+0000","last_became_peered":"2024-09-19T04:44:30.747369+0000","last_unstale":"2024-09-19T04:46:42.033125+0000","last_undegraded":"2024-09-19T04:46:42.033125+0000","last_fullsized":"2024-09-19T04:46:42.033125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:06.920357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.034354+0000","last_change":"2024-09-19T04:44:30.748844+0000","last_active":"2024-09-19T04:46:42.034354+0000","last_peered":"2024-09-19T04:46:42.034354+0000","last_clean":"2024-09-19T04:46:42.034354+0000","last_became_active":"2024-09-19T04:44:30.748513+0000","last_became_peered":"2024-09-19T04:44:30.748513+0000","last_unstale":"2024-09-19T04:46:42.034354+0000","last_undegraded":"2024-09-19T04:46:42.034354+0000","last_fullsized":"2024-09-19T04:46:42.034354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:45.107718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782714+0000","last_change":"2024-09-19T04:44:30.750526+0000","last_active":"2024-09-19T04:46:41.782714+0000","last_peered":"2024-09-19T04:46:41.782714+0000","last_clean":"2024-09-19T04:46:41.782714+0000","last_became_active":"2024-09-19T04:44:30.749693+0000","last_became_peered":"2024-09-19T04:44:30.749693+0000","last_unstale":"2024-09-19T04:46:41.782714+0000","last_undegraded":"2024-09-19T04:46:41.782714+0000","last_fullsized":"2024-09-19T04:46:41.782714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:43:04.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782639+0000","last_change":"2024-09-19T04:44:30.754310+0000","last_active":"2024-09-19T04:46:41.782639+0000","last_peered":"2024-09-19T04:46:41.782639+0000","last_clean":"2024-09-19T04:46:41.782639+0000","last_became_active":"2024-09-19T04:44:30.753933+0000","last_became_peered":"2024-09-19T04:44:30.753933+0000","last_unstale":"2024-09-19T04:46:41.782639+0000","last_undegraded":"2024-09-19T04:46:41.782639+0000","last_fullsized":"2024-09-19T04:46:41.782639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:40:44.985693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032386+0000","last_change":"2024-09-19T04:44:30.757458+0000","last_active":"2024-09-19T04:46:42.032386+0000","last_peered":"2024-09-19T04:46:42.032386+0000","last_clean":"2024-09-19T04:46:42.032386+0000","last_became_active":"2024-09-19T04:44:30.757127+0000","last_became_peered":"2024-09-19T04:44:30.757127+0000","last_unstale":"2024-09-19T04:46:42.032386+0000","last_undegraded":"2024-09-19T04:46:42.032386+0000","last_fullsized":"2024-09-19T04:46:42.032386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:59.325865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033118+0000","last_change":"2024-09-19T04:44:30.754685+0000","last_active":"2024-09-19T04:46:42.033118+0000","last_peered":"2024-09-19T04:46:42.033118+0000","last_clean":"2024-09-19T04:46:42.033118+0000","last_became_active":"2024-09-19T04:44:30.753625+0000","last_became_peered":"2024-09-19T04:44:30.753625+0000","last_unstale":"2024-09-19T04:46:42.033118+0000","last_undegraded":"2024-09-19T04:46:42.033118+0000","last_fullsized":"2024-09-19T04:46:42.033118+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:46:01.164693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.882473+0000","last_change":"2024-09-19T04:44:30.745823+0000","last_active":"2024-09-19T04:46:41.882473+0000","last_peered":"2024-09-19T04:46:41.882473+0000","last_clean":"2024-09-19T04:46:41.882473+0000","last_became_active":"2024-09-19T04:44:30.745224+0000","last_became_peered":"2024-09-19T04:44:30.745224+0000","last_unstale":"2024-09-19T04:46:41.882473+0000","last_undegraded":"2024-09-19T04:46:41.882473+0000","last_fullsized":"2024-09-19T04:46:41.882473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:23.682118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.012014+0000","last_change":"2024-09-19T04:44:30.752138+0000","last_active":"2024-09-19T04:46:42.012014+0000","last_peered":"2024-09-19T04:46:42.012014+0000","last_clean":"2024-09-19T04:46:42.012014+0000","last_became_active":"2024-09-19T04:44:30.751737+0000","last_became_peered":"2024-09-19T04:44:30.751737+0000","last_unstale":"2024-09-19T04:46:42.012014+0000","last_undegraded":"2024-09-19T04:46:42.012014+0000","last_fullsized":"2024-09-19T04:46:42.012014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:08.204601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032760+0000","last_change":"2024-09-19T04:44:30.748870+0000","last_active":"2024-09-19T04:46:42.032760+0000","last_peered":"2024-09-19T04:46:42.032760+0000","last_clean":"2024-09-19T04:46:42.032760+0000","last_became_active":"2024-09-19T04:44:30.748549+0000","last_became_peered":"2024-09-19T04:44:30.748549+0000","last_unstale":"2024-09-19T04:46:42.032760+0000","last_undegraded":"2024-09-19T04:46:42.032760+0000","last_fullsized":"2024-09-19T04:46:42.032760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:12.951564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.034496+0000","last_change":"2024-09-19T04:44:30.759937+0000","last_active":"2024-09-19T04:46:42.034496+0000","last_peered":"2024-09-19T04:46:42.034496+0000","last_clean":"2024-09-19T04:46:42.034496+0000","last_became_active":"2024-09-19T04:44:30.759659+0000","last_became_peered":"2024-09-19T04:44:30.759659+0000","last_unstale":"2024-09-19T04:46:42.034496+0000","last_undegraded":"2024-09-19T04:46:42.034496+0000","last_fullsized":"2024-09-19T04:46:42.034496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:07.608994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032624+0000","last_change":"2024-09-19T04:44:30.756216+0000","last_active":"2024-09-19T04:46:42.032624+0000","last_peered":"2024-09-19T04:46:42.032624+0000","last_clean":"2024-09-19T04:46:42.032624+0000","last_became_active":"2024-09-19T04:44:30.755903+0000","last_became_peered":"2024-09-19T04:44:30.755903+0000","last_unstale":"2024-09-19T04:46:42.032624+0000","last_undegraded":"2024-09-19T04:46:42.032624+0000","last_fullsized":"2024-09-19T04:46:42.032624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:16.966305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886344+0000","last_change":"2024-09-19T04:44:30.746297+0000","last_active":"2024-09-19T04:46:41.886344+0000","last_peered":"2024-09-19T04:46:41.886344+0000","last_clean":"2024-09-19T04:46:41.886344+0000","last_became_active":"2024-09-19T04:44:30.746013+0000","last_became_peered":"2024-09-19T04:44:30.746013+0000","last_unstale":"2024-09-19T04:46:41.886344+0000","last_undegraded":"2024-09-19T04:46:41.886344+0000","last_fullsized":"2024-09-19T04:46:41.886344+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:17.086180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785644+0000","last_change":"2024-09-19T04:45:14.630690+0000","last_active":"2024-09-19T04:46:41.785644+0000","last_peered":"2024-09-19T04:46:41.785644+0000","last_clean":"2024-09-19T04:46:41.785644+0000","last_became_active":"2024-09-19T04:45:14.630228+0000","last_became_peered":"2024-09-19T04:45:14.630228+0000","last_unstale":"2024-09-19T04:46:41.785644+0000","last_undegraded":"2024-09-19T04:46:41.785644+0000","last_fullsized":"2024-09-19T04:46:41.785644+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:39.901088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.887149+0000","last_change":"2024-09-19T04:44:30.753423+0000","last_active":"2024-09-19T04:46:41.887149+0000","last_peered":"2024-09-19T04:46:41.887149+0000","last_clean":"2024-09-19T04:46:41.887149+0000","last_became_active":"2024-09-19T04:44:30.752139+0000","last_became_peered":"2024-09-19T04:44:30.752139+0000","last_unstale":"2024-09-19T04:46:41.887149+0000","last_undegraded":"2024-09-19T04:46:41.887149+0000","last_fullsized":"2024-09-19T04:46:41.887149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:18.046893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.012113+0000","last_change":"2024-09-19T04:44:30.751114+0000","last_active":"2024-09-19T04:46:42.012113+0000","last_peered":"2024-09-19T04:46:42.012113+0000","last_clean":"2024-09-19T04:46:42.012113+0000","last_became_active":"2024-09-19T04:44:30.750784+0000","last_became_peered":"2024-09-19T04:44:30.750784+0000","last_unstale":"2024-09-19T04:46:42.012113+0000","last_undegraded":"2024-09-19T04:46:42.012113+0000","last_fullsized":"2024-09-19T04:46:42.012113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:22.040253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033057+0000","last_change":"2024-09-19T04:44:30.747191+0000","last_active":"2024-09-19T04:46:42.033057+0000","last_peered":"2024-09-19T04:46:42.033057+0000","last_clean":"2024-09-19T04:46:42.033057+0000","last_became_active":"2024-09-19T04:44:30.746830+0000","last_became_peered":"2024-09-19T04:44:30.746830+0000","last_unstale":"2024-09-19T04:46:42.033057+0000","last_undegraded":"2024-09-19T04:46:42.033057+0000","last_fullsized":"2024-09-19T04:46:42.033057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:24.634920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.034635+0000","last_change":"2024-09-19T04:44:30.755174+0000","last_active":"2024-09-19T04:46:42.034635+0000","last_peered":"2024-09-19T04:46:42.034635+0000","last_clean":"2024-09-19T04:46:42.034635+0000","last_became_active":"2024-09-19T04:44:30.754885+0000","last_became_peered":"2024-09-19T04:44:30.754885+0000","last_unstale":"2024-09-19T04:46:42.034635+0000","last_undegraded":"2024-09-19T04:46:42.034635+0000","last_fullsized":"2024-09-19T04:46:42.034635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:11:32.428033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782923+0000","last_change":"2024-09-19T04:45:14.891484+0000","last_active":"2024-09-19T04:46:41.782923+0000","last_peered":"2024-09-19T04:46:41.782923+0000","last_clean":"2024-09-19T04:46:41.782923+0000","last_became_active":"2024-09-19T04:45:14.891120+0000","last_became_peered":"2024-09-19T04:45:14.891120+0000","last_unstale":"2024-09-19T04:46:41.782923+0000","last_undegraded":"2024-09-19T04:46:41.782923+0000","last_fullsized":"2024-09-19T04:46:41.782923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:20.959571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.011793+0000","last_change":"2024-09-19T04:44:30.757343+0000","last_active":"2024-09-19T04:46:42.011793+0000","last_peered":"2024-09-19T04:46:42.011793+0000","last_clean":"2024-09-19T04:46:42.011793+0000","last_became_active":"2024-09-19T04:44:30.756981+0000","last_became_peered":"2024-09-19T04:44:30.756981+0000","last_unstale":"2024-09-19T04:46:42.011793+0000","last_undegraded":"2024-09-19T04:46:42.011793+0000","last_fullsized":"2024-09-19T04:46:42.011793+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:07.388384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.011916+0000","last_change":"2024-09-19T04:45:14.631104+0000","last_active":"2024-09-19T04:46:42.011916+0000","last_peered":"2024-09-19T04:46:42.011916+0000","last_clean":"2024-09-19T04:46:42.011916+0000","last_became_active":"2024-09-19T04:45:14.630654+0000","last_became_peered":"2024-09-19T04:45:14.630654+0000","last_unstale":"2024-09-19T04:46:42.011916+0000","last_undegraded":"2024-09-19T04:46:42.011916+0000","last_fullsized":"2024-09-19T04:46:42.011916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:37.659295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032830+0000","last_change":"2024-09-19T04:44:30.748408+0000","last_active":"2024-09-19T04:46:42.032830+0000","last_peered":"2024-09-19T04:46:42.032830+0000","last_clean":"2024-09-19T04:46:42.032830+0000","last_became_active":"2024-09-19T04:44:30.747960+0000","last_became_peered":"2024-09-19T04:44:30.747960+0000","last_unstale":"2024-09-19T04:46:42.032830+0000","last_undegraded":"2024-09-19T04:46:42.032830+0000","last_fullsized":"2024-09-19T04:46:42.032830+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:51.043791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785665+0000","last_change":"2024-09-19T04:44:30.747194+0000","last_active":"2024-09-19T04:46:41.785665+0000","last_peered":"2024-09-19T04:46:41.785665+0000","last_clean":"2024-09-19T04:46:41.785665+0000","last_became_active":"2024-09-19T04:44:30.746871+0000","last_became_peered":"2024-09-19T04:44:30.746871+0000","last_unstale":"2024-09-19T04:46:41.785665+0000","last_undegraded":"2024-09-19T04:46:41.785665+0000","last_fullsized":"2024-09-19T04:46:41.785665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:52.340341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":152,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.114293+0000","last_change":"2024-09-19T04:44:28.337746+0000","last_active":"2024-09-19T04:46:42.114293+0000","last_peered":"2024-09-19T04:46:42.114293+0000","last_clean":"2024-09-19T04:46:42.114293+0000","last_became_active":"2024-09-19T04:44:28.337410+0000","last_became_peered":"2024-09-19T04:44:28.337410+0000","last_unstale":"2024-09-19T04:46:42.114293+0000","last_undegraded":"2024-09-19T04:46:42.114293+0000","last_fullsized":"2024-09-19T04:46:42.114293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:41:27.436939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":2,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895593+0000","last_change":"2024-09-19T04:44:30.755512+0000","last_active":"2024-09-19T04:46:41.895593+0000","last_peered":"2024-09-19T04:46:41.895593+0000","last_clean":"2024-09-19T04:46:41.895593+0000","last_became_active":"2024-09-19T04:44:30.755086+0000","last_became_peered":"2024-09-19T04:44:30.755086+0000","last_unstale":"2024-09-19T04:46:41.895593+0000","last_undegraded":"2024-09-19T04:46:41.895593+0000","last_fullsized":"2024-09-19T04:46:41.895593+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:33.870376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.898886+0000","last_change":"2024-09-19T04:44:28.334492+0000","last_active":"2024-09-19T04:46:41.898886+0000","last_peered":"2024-09-19T04:46:41.898886+0000","last_clean":"2024-09-19T04:46:41.898886+0000","last_became_active":"2024-09-19T04:44:28.334098+0000","last_became_peered":"2024-09-19T04:44:28.334098+0000","last_unstale":"2024-09-19T04:46:41.898886+0000","last_undegraded":"2024-09-19T04:46:41.898886+0000","last_fullsized":"2024-09-19T04:46:41.898886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:48.156590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033952+0000","last_change":"2024-09-19T04:44:28.335593+0000","last_active":"2024-09-19T04:46:42.033952+0000","last_peered":"2024-09-19T04:46:42.033952+0000","last_clean":"2024-09-19T04:46:42.033952+0000","last_became_active":"2024-09-19T04:44:28.335060+0000","last_became_peered":"2024-09-19T04:44:28.335060+0000","last_unstale":"2024-09-19T04:46:42.033952+0000","last_undegraded":"2024-09-19T04:46:42.033952+0000","last_fullsized":"2024-09-19T04:46:42.033952+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:27.859960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886288+0000","last_change":"2024-09-19T04:44:30.748235+0000","last_active":"2024-09-19T04:46:41.886288+0000","last_peered":"2024-09-19T04:46:41.886288+0000","last_clean":"2024-09-19T04:46:41.886288+0000","last_became_active":"2024-09-19T04:44:30.747881+0000","last_became_peered":"2024-09-19T04:44:30.747881+0000","last_unstale":"2024-09-19T04:46:41.886288+0000","last_undegraded":"2024-09-19T04:46:41.886288+0000","last_fullsized":"2024-09-19T04:46:41.886288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:40:39.183897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895656+0000","last_change":"2024-09-19T04:44:28.337762+0000","last_active":"2024-09-19T04:46:41.895656+0000","last_peered":"2024-09-19T04:46:41.895656+0000","last_clean":"2024-09-19T04:46:41.895656+0000","last_became_active":"2024-09-19T04:44:28.337383+0000","last_became_peered":"2024-09-19T04:44:28.337383+0000","last_unstale":"2024-09-19T04:46:41.895656+0000","last_undegraded":"2024-09-19T04:46:41.895656+0000","last_fullsized":"2024-09-19T04:46:41.895656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:07:55.268958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.882010+0000","last_change":"2024-09-19T04:44:30.754318+0000","last_active":"2024-09-19T04:46:41.882010+0000","last_peered":"2024-09-19T04:46:41.882010+0000","last_clean":"2024-09-19T04:46:41.882010+0000","last_became_active":"2024-09-19T04:44:30.754025+0000","last_became_peered":"2024-09-19T04:44:30.754025+0000","last_unstale":"2024-09-19T04:46:41.882010+0000","last_undegraded":"2024-09-19T04:46:41.882010+0000","last_fullsized":"2024-09-19T04:46:41.882010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:58.190504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":266,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.010422+0000","last_change":"2024-09-19T04:44:28.335947+0000","last_active":"2024-09-19T04:46:42.010422+0000","last_peered":"2024-09-19T04:46:42.010422+0000","last_clean":"2024-09-19T04:46:42.010422+0000","last_became_active":"2024-09-19T04:44:28.335588+0000","last_became_peered":"2024-09-19T04:44:28.335588+0000","last_unstale":"2024-09-19T04:46:42.010422+0000","last_undegraded":"2024-09-19T04:46:42.010422+0000","last_fullsized":"2024-09-19T04:46:42.010422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:44.024915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":236,"num_read_kb":236,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032720+0000","last_change":"2024-09-19T04:44:30.748272+0000","last_active":"2024-09-19T04:46:42.032720+0000","last_peered":"2024-09-19T04:46:42.032720+0000","last_clean":"2024-09-19T04:46:42.032720+0000","last_became_active":"2024-09-19T04:44:30.747147+0000","last_became_peered":"2024-09-19T04:44:30.747147+0000","last_unstale":"2024-09-19T04:46:42.032720+0000","last_undegraded":"2024-09-19T04:46:42.032720+0000","last_fullsized":"2024-09-19T04:46:42.032720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:11.653693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782564+0000","last_change":"2024-09-19T04:44:28.337120+0000","last_active":"2024-09-19T04:46:41.782564+0000","last_peered":"2024-09-19T04:46:41.782564+0000","last_clean":"2024-09-19T04:46:41.782564+0000","last_became_active":"2024-09-19T04:44:28.336053+0000","last_became_peered":"2024-09-19T04:44:28.336053+0000","last_unstale":"2024-09-19T04:46:41.782564+0000","last_undegraded":"2024-09-19T04:46:41.782564+0000","last_fullsized":"2024-09-19T04:46:41.782564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:56:57.514931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782921+0000","last_change":"2024-09-19T04:44:30.755238+0000","last_active":"2024-09-19T04:46:41.782921+0000","last_peered":"2024-09-19T04:46:41.782921+0000","last_clean":"2024-09-19T04:46:41.782921+0000","last_became_active":"2024-09-19T04:44:30.754748+0000","last_became_peered":"2024-09-19T04:44:30.754748+0000","last_unstale":"2024-09-19T04:46:41.782921+0000","last_undegraded":"2024-09-19T04:46:41.782921+0000","last_fullsized":"2024-09-19T04:46:41.782921+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:38.954491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895818+0000","last_change":"2024-09-19T04:44:28.596147+0000","last_active":"2024-09-19T04:46:41.895818+0000","last_peered":"2024-09-19T04:46:41.895818+0000","last_clean":"2024-09-19T04:46:41.895818+0000","last_became_active":"2024-09-19T04:44:28.595785+0000","last_became_peered":"2024-09-19T04:44:28.595785+0000","last_unstale":"2024-09-19T04:46:41.895818+0000","last_undegraded":"2024-09-19T04:46:41.895818+0000","last_fullsized":"2024-09-19T04:46:41.895818+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:39.138724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032947+0000","last_change":"2024-09-19T04:44:30.753817+0000","last_active":"2024-09-19T04:46:42.032947+0000","last_peered":"2024-09-19T04:46:42.032947+0000","last_clean":"2024-09-19T04:46:42.032947+0000","last_became_active":"2024-09-19T04:44:30.752541+0000","last_became_peered":"2024-09-19T04:44:30.752541+0000","last_unstale":"2024-09-19T04:46:42.032947+0000","last_undegraded":"2024-09-19T04:46:42.032947+0000","last_fullsized":"2024-09-19T04:46:42.032947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:16.121822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'38","reported_seq":310,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.117197+0000","last_change":"2024-09-19T04:44:28.335449+0000","last_active":"2024-09-19T04:46:42.117197+0000","last_peered":"2024-09-19T04:46:42.117197+0000","last_clean":"2024-09-19T04:46:42.117197+0000","last_became_active":"2024-09-19T04:44:28.335080+0000","last_became_peered":"2024-09-19T04:44:28.335080+0000","last_unstale":"2024-09-19T04:46:42.117197+0000","last_undegraded":"2024-09-19T04:46:42.117197+0000","last_fullsized":"2024-09-19T04:46:42.117197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:28.436290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66483,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":247,"num_read_kb":182,"num_write":40,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.786101+0000","last_change":"2024-09-19T04:44:30.749624+0000","last_active":"2024-09-19T04:46:41.786101+0000","last_peered":"2024-09-19T04:46:41.786101+0000","last_clean":"2024-09-19T04:46:41.786101+0000","last_became_active":"2024-09-19T04:44:30.749295+0000","last_became_peered":"2024-09-19T04:44:30.749295+0000","last_unstale":"2024-09-19T04:46:41.786101+0000","last_undegraded":"2024-09-19T04:46:41.786101+0000","last_fullsized":"2024-09-19T04:46:41.786101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:48:46.942237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.011879+0000","last_change":"2024-09-19T04:44:28.337863+0000","last_active":"2024-09-19T04:46:42.011879+0000","last_peered":"2024-09-19T04:46:42.011879+0000","last_clean":"2024-09-19T04:46:42.011879+0000","last_became_active":"2024-09-19T04:44:28.337522+0000","last_became_peered":"2024-09-19T04:44:28.337522+0000","last_unstale":"2024-09-19T04:46:42.011879+0000","last_undegraded":"2024-09-19T04:46:42.011879+0000","last_fullsized":"2024-09-19T04:46:42.011879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:02.488622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033943+0000","last_change":"2024-09-19T04:44:30.759916+0000","last_active":"2024-09-19T04:46:42.033943+0000","last_peered":"2024-09-19T04:46:42.033943+0000","last_clean":"2024-09-19T04:46:42.033943+0000","last_became_active":"2024-09-19T04:44:30.758835+0000","last_became_peered":"2024-09-19T04:44:30.758835+0000","last_unstale":"2024-09-19T04:46:42.033943+0000","last_undegraded":"2024-09-19T04:46:42.033943+0000","last_fullsized":"2024-09-19T04:46:42.033943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:38.614343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895828+0000","last_change":"2024-09-19T04:44:28.337574+0000","last_active":"2024-09-19T04:46:41.895828+0000","last_peered":"2024-09-19T04:46:41.895828+0000","last_clean":"2024-09-19T04:46:41.895828+0000","last_became_active":"2024-09-19T04:44:28.337224+0000","last_became_peered":"2024-09-19T04:44:28.337224+0000","last_unstale":"2024-09-19T04:46:41.895828+0000","last_undegraded":"2024-09-19T04:46:41.895828+0000","last_fullsized":"2024-09-19T04:46:41.895828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:20.538181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782359+0000","last_change":"2024-09-19T04:44:30.758152+0000","last_active":"2024-09-19T04:46:41.782359+0000","last_peered":"2024-09-19T04:46:41.782359+0000","last_clean":"2024-09-19T04:46:41.782359+0000","last_became_active":"2024-09-19T04:44:30.757801+0000","last_became_peered":"2024-09-19T04:44:30.757801+0000","last_unstale":"2024-09-19T04:46:41.782359+0000","last_undegraded":"2024-09-19T04:46:41.782359+0000","last_fullsized":"2024-09-19T04:46:41.782359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:05.212471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'7","reported_seq":154,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032968+0000","last_change":"2024-09-19T04:44:28.339340+0000","last_active":"2024-09-19T04:46:42.032968+0000","last_peered":"2024-09-19T04:46:42.032968+0000","last_clean":"2024-09-19T04:46:42.032968+0000","last_became_active":"2024-09-19T04:44:28.339019+0000","last_became_peered":"2024-09-19T04:44:28.339019+0000","last_unstale":"2024-09-19T04:46:42.032968+0000","last_undegraded":"2024-09-19T04:46:42.032968+0000","last_fullsized":"2024-09-19T04:46:42.032968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:29:59.356748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782063+0000","last_change":"2024-09-19T04:44:30.751217+0000","last_active":"2024-09-19T04:46:41.782063+0000","last_peered":"2024-09-19T04:46:41.782063+0000","last_clean":"2024-09-19T04:46:41.782063+0000","last_became_active":"2024-09-19T04:44:30.750863+0000","last_became_peered":"2024-09-19T04:44:30.750863+0000","last_unstale":"2024-09-19T04:46:41.782063+0000","last_undegraded":"2024-09-19T04:46:41.782063+0000","last_fullsized":"2024-09-19T04:46:41.782063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:09.957572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785881+0000","last_change":"2024-09-19T04:44:28.596585+0000","last_active":"2024-09-19T04:46:41.785881+0000","last_peered":"2024-09-19T04:46:41.785881+0000","last_clean":"2024-09-19T04:46:41.785881+0000","last_became_active":"2024-09-19T04:44:28.596218+0000","last_became_peered":"2024-09-19T04:44:28.596218+0000","last_unstale":"2024-09-19T04:46:41.785881+0000","last_undegraded":"2024-09-19T04:46:41.785881+0000","last_fullsized":"2024-09-19T04:46:41.785881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:29:35.395610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895520+0000","last_change":"2024-09-19T04:44:30.746590+0000","last_active":"2024-09-19T04:46:41.895520+0000","last_peered":"2024-09-19T04:46:41.895520+0000","last_clean":"2024-09-19T04:46:41.895520+0000","last_became_active":"2024-09-19T04:44:30.746263+0000","last_became_peered":"2024-09-19T04:44:30.746263+0000","last_unstale":"2024-09-19T04:46:41.895520+0000","last_undegraded":"2024-09-19T04:46:41.895520+0000","last_fullsized":"2024-09-19T04:46:41.895520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:38.955613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.786340+0000","last_change":"2024-09-19T04:44:28.335826+0000","last_active":"2024-09-19T04:46:41.786340+0000","last_peered":"2024-09-19T04:46:41.786340+0000","last_clean":"2024-09-19T04:46:41.786340+0000","last_became_active":"2024-09-19T04:44:28.335481+0000","last_became_peered":"2024-09-19T04:44:28.335481+0000","last_unstale":"2024-09-19T04:46:41.786340+0000","last_undegraded":"2024-09-19T04:46:41.786340+0000","last_fullsized":"2024-09-19T04:46:41.786340+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:20.989500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032963+0000","last_change":"2024-09-19T04:44:30.744130+0000","last_active":"2024-09-19T04:46:42.032963+0000","last_peered":"2024-09-19T04:46:42.032963+0000","last_clean":"2024-09-19T04:46:42.032963+0000","last_became_active":"2024-09-19T04:44:30.743772+0000","last_became_peered":"2024-09-19T04:44:30.743772+0000","last_unstale":"2024-09-19T04:46:42.032963+0000","last_undegraded":"2024-09-19T04:46:42.032963+0000","last_fullsized":"2024-09-19T04:46:42.032963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:34.956670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.034296+0000","last_change":"2024-09-19T04:44:28.337543+0000","last_active":"2024-09-19T04:46:42.034296+0000","last_peered":"2024-09-19T04:46:42.034296+0000","last_clean":"2024-09-19T04:46:42.034296+0000","last_became_active":"2024-09-19T04:44:28.336128+0000","last_became_peered":"2024-09-19T04:44:28.336128+0000","last_unstale":"2024-09-19T04:46:42.034296+0000","last_undegraded":"2024-09-19T04:46:42.034296+0000","last_fullsized":"2024-09-19T04:46:42.034296+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:31.907777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782306+0000","last_change":"2024-09-19T04:44:30.757489+0000","last_active":"2024-09-19T04:46:41.782306+0000","last_peered":"2024-09-19T04:46:41.782306+0000","last_clean":"2024-09-19T04:46:41.782306+0000","last_became_active":"2024-09-19T04:44:30.757041+0000","last_became_peered":"2024-09-19T04:44:30.757041+0000","last_unstale":"2024-09-19T04:46:41.782306+0000","last_undegraded":"2024-09-19T04:46:41.782306+0000","last_fullsized":"2024-09-19T04:46:41.782306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:03.395913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'41","reported_seq":75,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.049157+0000","last_change":"2024-09-19T04:44:28.327393+0000","last_active":"2024-09-19T04:46:42.049157+0000","last_peered":"2024-09-19T04:46:42.049157+0000","last_clean":"2024-09-19T04:46:42.049157+0000","last_became_active":"2024-09-19T04:44:28.327116+0000","last_became_peered":"2024-09-19T04:44:28.327116+0000","last_unstale":"2024-09-19T04:46:42.049157+0000","last_undegraded":"2024-09-19T04:46:42.049157+0000","last_fullsized":"2024-09-19T04:46:42.049157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:42.360980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782342+0000","last_change":"2024-09-19T04:44:30.754239+0000","last_active":"2024-09-19T04:46:41.782342+0000","last_peered":"2024-09-19T04:46:41.782342+0000","last_clean":"2024-09-19T04:46:41.782342+0000","last_became_active":"2024-09-19T04:44:30.753885+0000","last_became_peered":"2024-09-19T04:44:30.753885+0000","last_unstale":"2024-09-19T04:46:41.782342+0000","last_undegraded":"2024-09-19T04:46:41.782342+0000","last_fullsized":"2024-09-19T04:46:41.782342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:27.355600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'5","reported_seq":152,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.012256+0000","last_change":"2024-09-19T04:44:28.596214+0000","last_active":"2024-09-19T04:46:42.012256+0000","last_peered":"2024-09-19T04:46:42.012256+0000","last_clean":"2024-09-19T04:46:42.012256+0000","last_became_active":"2024-09-19T04:44:28.595841+0000","last_became_peered":"2024-09-19T04:44:28.595841+0000","last_unstale":"2024-09-19T04:46:42.012256+0000","last_undegraded":"2024-09-19T04:46:42.012256+0000","last_fullsized":"2024-09-19T04:46:42.012256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:43.864702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782627+0000","last_change":"2024-09-19T04:44:30.756192+0000","last_active":"2024-09-19T04:46:41.782627+0000","last_peered":"2024-09-19T04:46:41.782627+0000","last_clean":"2024-09-19T04:46:41.782627+0000","last_became_active":"2024-09-19T04:44:30.755853+0000","last_became_peered":"2024-09-19T04:44:30.755853+0000","last_unstale":"2024-09-19T04:46:41.782627+0000","last_undegraded":"2024-09-19T04:46:41.782627+0000","last_fullsized":"2024-09-19T04:46:41.782627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:25:49.206978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":352,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782231+0000","last_change":"2024-09-19T04:43:10.156816+0000","last_active":"2024-09-19T04:46:41.782231+0000","last_peered":"2024-09-19T04:46:41.782231+0000","last_clean":"2024-09-19T04:46:41.782231+0000","last_became_active":"2024-09-19T04:43:10.156359+0000","last_became_peered":"2024-09-19T04:43:10.156359+0000","last_unstale":"2024-09-19T04:46:41.782231+0000","last_undegraded":"2024-09-19T04:46:41.782231+0000","last_fullsized":"2024-09-19T04:46:41.782231+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:39:34.200793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:39:34.200793+0000","last_clean_scrub_stamp":"2024-09-19T04:39:34.200793+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:13.453818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032920+0000","last_change":"2024-09-19T04:44:28.337637+0000","last_active":"2024-09-19T04:46:42.032920+0000","last_peered":"2024-09-19T04:46:42.032920+0000","last_clean":"2024-09-19T04:46:42.032920+0000","last_became_active":"2024-09-19T04:44:28.337235+0000","last_became_peered":"2024-09-19T04:44:28.337235+0000","last_unstale":"2024-09-19T04:46:42.032920+0000","last_undegraded":"2024-09-19T04:46:42.032920+0000","last_fullsized":"2024-09-19T04:46:42.032920+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:42.104972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.881909+0000","last_change":"2024-09-19T04:45:15.281422+0000","last_active":"2024-09-19T04:46:41.881909+0000","last_peered":"2024-09-19T04:46:41.881909+0000","last_clean":"2024-09-19T04:46:41.881909+0000","last_became_active":"2024-09-19T04:45:15.281031+0000","last_became_peered":"2024-09-19T04:45:15.281031+0000","last_unstale":"2024-09-19T04:46:41.881909+0000","last_undegraded":"2024-09-19T04:46:41.881909+0000","last_fullsized":"2024-09-19T04:46:41.881909+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:05.058469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782204+0000","last_change":"2024-09-19T04:44:28.336124+0000","last_active":"2024-09-19T04:46:41.782204+0000","last_peered":"2024-09-19T04:46:41.782204+0000","last_clean":"2024-09-19T04:46:41.782204+0000","last_became_active":"2024-09-19T04:44:28.335754+0000","last_became_peered":"2024-09-19T04:44:28.335754+0000","last_unstale":"2024-09-19T04:46:41.782204+0000","last_undegraded":"2024-09-19T04:46:41.782204+0000","last_fullsized":"2024-09-19T04:46:41.782204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:41.604367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033916+0000","last_change":"2024-09-19T04:44:30.749939+0000","last_active":"2024-09-19T04:46:42.033916+0000","last_peered":"2024-09-19T04:46:42.033916+0000","last_clean":"2024-09-19T04:46:42.033916+0000","last_became_active":"2024-09-19T04:44:30.747122+0000","last_became_peered":"2024-09-19T04:44:30.747122+0000","last_unstale":"2024-09-19T04:46:42.033916+0000","last_undegraded":"2024-09-19T04:46:42.033916+0000","last_fullsized":"2024-09-19T04:46:42.033916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:46:08.371493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":151,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.039159+0000","last_change":"2024-09-19T04:44:28.594768+0000","last_active":"2024-09-19T04:46:42.039159+0000","last_peered":"2024-09-19T04:46:42.039159+0000","last_clean":"2024-09-19T04:46:42.039159+0000","last_became_active":"2024-09-19T04:44:28.594320+0000","last_became_peered":"2024-09-19T04:44:28.594320+0000","last_unstale":"2024-09-19T04:46:42.039159+0000","last_undegraded":"2024-09-19T04:46:42.039159+0000","last_fullsized":"2024-09-19T04:46:42.039159+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:32:17.736874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895518+0000","last_change":"2024-09-19T04:44:30.746666+0000","last_active":"2024-09-19T04:46:41.895518+0000","last_peered":"2024-09-19T04:46:41.895518+0000","last_clean":"2024-09-19T04:46:41.895518+0000","last_became_active":"2024-09-19T04:44:30.746312+0000","last_became_peered":"2024-09-19T04:44:30.746312+0000","last_unstale":"2024-09-19T04:46:41.895518+0000","last_undegraded":"2024-09-19T04:46:41.895518+0000","last_fullsized":"2024-09-19T04:46:41.895518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:13.562810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"87'7","reported_seq":153,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.117972+0000","last_change":"2024-09-19T04:44:28.336492+0000","last_active":"2024-09-19T04:46:42.117972+0000","last_peered":"2024-09-19T04:46:42.117972+0000","last_clean":"2024-09-19T04:46:42.117972+0000","last_became_active":"2024-09-19T04:44:28.336044+0000","last_became_peered":"2024-09-19T04:44:28.336044+0000","last_unstale":"2024-09-19T04:46:42.117972+0000","last_undegraded":"2024-09-19T04:46:42.117972+0000","last_fullsized":"2024-09-19T04:46:42.117972+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:18.760473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.899051+0000","last_change":"2024-09-19T04:45:14.630959+0000","last_active":"2024-09-19T04:46:41.899051+0000","last_peered":"2024-09-19T04:46:41.899051+0000","last_clean":"2024-09-19T04:46:41.899051+0000","last_became_active":"2024-09-19T04:45:14.630528+0000","last_became_peered":"2024-09-19T04:45:14.630528+0000","last_unstale":"2024-09-19T04:46:41.899051+0000","last_undegraded":"2024-09-19T04:46:41.899051+0000","last_fullsized":"2024-09-19T04:46:41.899051+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:29.754355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.882436+0000","last_change":"2024-09-19T04:44:28.596322+0000","last_active":"2024-09-19T04:46:41.882436+0000","last_peered":"2024-09-19T04:46:41.882436+0000","last_clean":"2024-09-19T04:46:41.882436+0000","last_became_active":"2024-09-19T04:44:28.595566+0000","last_became_peered":"2024-09-19T04:44:28.595566+0000","last_unstale":"2024-09-19T04:46:41.882436+0000","last_undegraded":"2024-09-19T04:46:41.882436+0000","last_fullsized":"2024-09-19T04:46:41.882436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:23.878145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.899012+0000","last_change":"2024-09-19T04:44:30.752321+0000","last_active":"2024-09-19T04:46:41.899012+0000","last_peered":"2024-09-19T04:46:41.899012+0000","last_clean":"2024-09-19T04:46:41.899012+0000","last_became_active":"2024-09-19T04:44:30.751980+0000","last_became_peered":"2024-09-19T04:44:30.751980+0000","last_unstale":"2024-09-19T04:46:41.899012+0000","last_undegraded":"2024-09-19T04:46:41.899012+0000","last_fullsized":"2024-09-19T04:46:41.899012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:41.863034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.786281+0000","last_change":"2024-09-19T04:44:28.596547+0000","last_active":"2024-09-19T04:46:41.786281+0000","last_peered":"2024-09-19T04:46:41.786281+0000","last_clean":"2024-09-19T04:46:41.786281+0000","last_became_active":"2024-09-19T04:44:28.596181+0000","last_became_peered":"2024-09-19T04:44:28.596181+0000","last_unstale":"2024-09-19T04:46:41.786281+0000","last_undegraded":"2024-09-19T04:46:41.786281+0000","last_fullsized":"2024-09-19T04:46:41.786281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:53.251737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.034777+0000","last_change":"2024-09-19T04:44:30.751533+0000","last_active":"2024-09-19T04:46:42.034777+0000","last_peered":"2024-09-19T04:46:42.034777+0000","last_clean":"2024-09-19T04:46:42.034777+0000","last_became_active":"2024-09-19T04:44:30.748012+0000","last_became_peered":"2024-09-19T04:44:30.748012+0000","last_unstale":"2024-09-19T04:46:42.034777+0000","last_undegraded":"2024-09-19T04:46:42.034777+0000","last_fullsized":"2024-09-19T04:46:42.034777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:32.360147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'52","reported_seq":102,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.770637+0000","last_change":"2024-09-19T04:44:28.335884+0000","last_active":"2024-09-19T04:46:42.770637+0000","last_peered":"2024-09-19T04:46:42.770637+0000","last_clean":"2024-09-19T04:46:42.770637+0000","last_became_active":"2024-09-19T04:44:28.335521+0000","last_became_peered":"2024-09-19T04:44:28.335521+0000","last_unstale":"2024-09-19T04:46:42.770637+0000","last_undegraded":"2024-09-19T04:46:42.770637+0000","last_fullsized":"2024-09-19T04:46:42.770637+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:32:49.764893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.887262+0000","last_change":"2024-09-19T04:44:30.759578+0000","last_active":"2024-09-19T04:46:41.887262+0000","last_peered":"2024-09-19T04:46:41.887262+0000","last_clean":"2024-09-19T04:46:41.887262+0000","last_became_active":"2024-09-19T04:44:30.759245+0000","last_became_peered":"2024-09-19T04:44:30.759245+0000","last_unstale":"2024-09-19T04:46:41.887262+0000","last_undegraded":"2024-09-19T04:46:41.887262+0000","last_fullsized":"2024-09-19T04:46:41.887262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:37.121770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033302+0000","last_change":"2024-09-19T04:44:28.593404+0000","last_active":"2024-09-19T04:46:42.033302+0000","last_peered":"2024-09-19T04:46:42.033302+0000","last_clean":"2024-09-19T04:46:42.033302+0000","last_became_active":"2024-09-19T04:44:28.593037+0000","last_became_peered":"2024-09-19T04:44:28.593037+0000","last_unstale":"2024-09-19T04:46:42.033302+0000","last_undegraded":"2024-09-19T04:46:42.033302+0000","last_fullsized":"2024-09-19T04:46:42.033302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:08:29.218543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.783007+0000","last_change":"2024-09-19T04:45:14.891783+0000","last_active":"2024-09-19T04:46:41.783007+0000","last_peered":"2024-09-19T04:46:41.783007+0000","last_clean":"2024-09-19T04:46:41.783007+0000","last_became_active":"2024-09-19T04:45:14.891396+0000","last_became_peered":"2024-09-19T04:45:14.891396+0000","last_unstale":"2024-09-19T04:46:41.783007+0000","last_undegraded":"2024-09-19T04:46:41.783007+0000","last_fullsized":"2024-09-19T04:46:41.783007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:56.443270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032529+0000","last_change":"2024-09-19T04:44:28.594541+0000","last_active":"2024-09-19T04:46:42.032529+0000","last_peered":"2024-09-19T04:46:42.032529+0000","last_clean":"2024-09-19T04:46:42.032529+0000","last_became_active":"2024-09-19T04:44:28.594262+0000","last_became_peered":"2024-09-19T04:44:28.594262+0000","last_unstale":"2024-09-19T04:46:42.032529+0000","last_undegraded":"2024-09-19T04:46:42.032529+0000","last_fullsized":"2024-09-19T04:46:42.032529+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:45.685034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785728+0000","last_change":"2024-09-19T04:44:30.757861+0000","last_active":"2024-09-19T04:46:41.785728+0000","last_peered":"2024-09-19T04:46:41.785728+0000","last_clean":"2024-09-19T04:46:41.785728+0000","last_became_active":"2024-09-19T04:44:30.757509+0000","last_became_peered":"2024-09-19T04:44:30.757509+0000","last_unstale":"2024-09-19T04:46:41.785728+0000","last_undegraded":"2024-09-19T04:46:41.785728+0000","last_fullsized":"2024-09-19T04:46:41.785728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:17.737055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886469+0000","last_change":"2024-09-19T04:44:28.335225+0000","last_active":"2024-09-19T04:46:41.886469+0000","last_peered":"2024-09-19T04:46:41.886469+0000","last_clean":"2024-09-19T04:46:41.886469+0000","last_became_active":"2024-09-19T04:44:28.334804+0000","last_became_peered":"2024-09-19T04:44:28.334804+0000","last_unstale":"2024-09-19T04:46:41.886469+0000","last_undegraded":"2024-09-19T04:46:41.886469+0000","last_fullsized":"2024-09-19T04:46:41.886469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:15.639418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033015+0000","last_change":"2024-09-19T04:44:30.754018+0000","last_active":"2024-09-19T04:46:42.033015+0000","last_peered":"2024-09-19T04:46:42.033015+0000","last_clean":"2024-09-19T04:46:42.033015+0000","last_became_active":"2024-09-19T04:44:30.753639+0000","last_became_peered":"2024-09-19T04:44:30.753639+0000","last_unstale":"2024-09-19T04:46:42.033015+0000","last_undegraded":"2024-09-19T04:46:42.033015+0000","last_fullsized":"2024-09-19T04:46:42.033015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:05.012847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032760+0000","last_change":"2024-09-19T04:45:14.636102+0000","last_active":"2024-09-19T04:46:42.032760+0000","last_peered":"2024-09-19T04:46:42.032760+0000","last_clean":"2024-09-19T04:46:42.032760+0000","last_became_active":"2024-09-19T04:45:14.631201+0000","last_became_peered":"2024-09-19T04:45:14.631201+0000","last_unstale":"2024-09-19T04:46:42.032760+0000","last_undegraded":"2024-09-19T04:46:42.032760+0000","last_fullsized":"2024-09-19T04:46:42.032760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:04.331207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886472+0000","last_change":"2024-09-19T04:44:30.758543+0000","last_active":"2024-09-19T04:46:41.886472+0000","last_peered":"2024-09-19T04:46:41.886472+0000","last_clean":"2024-09-19T04:46:41.886472+0000","last_became_active":"2024-09-19T04:44:30.757228+0000","last_became_peered":"2024-09-19T04:44:30.757228+0000","last_unstale":"2024-09-19T04:46:41.886472+0000","last_undegraded":"2024-09-19T04:46:41.886472+0000","last_fullsized":"2024-09-19T04:46:41.886472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:13.877742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785932+0000","last_change":"2024-09-19T04:44:28.339152+0000","last_active":"2024-09-19T04:46:41.785932+0000","last_peered":"2024-09-19T04:46:41.785932+0000","last_clean":"2024-09-19T04:46:41.785932+0000","last_became_active":"2024-09-19T04:44:28.338778+0000","last_became_peered":"2024-09-19T04:44:28.338778+0000","last_unstale":"2024-09-19T04:46:41.785932+0000","last_undegraded":"2024-09-19T04:46:41.785932+0000","last_fullsized":"2024-09-19T04:46:41.785932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:53.906755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782285+0000","last_change":"2024-09-19T04:44:30.759330+0000","last_active":"2024-09-19T04:46:41.782285+0000","last_peered":"2024-09-19T04:46:41.782285+0000","last_clean":"2024-09-19T04:46:41.782285+0000","last_became_active":"2024-09-19T04:44:30.758987+0000","last_became_peered":"2024-09-19T04:44:30.758987+0000","last_unstale":"2024-09-19T04:46:41.782285+0000","last_undegraded":"2024-09-19T04:46:41.782285+0000","last_fullsized":"2024-09-19T04:46:41.782285+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:10.105051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895780+0000","last_change":"2024-09-19T04:44:30.748089+0000","last_active":"2024-09-19T04:46:41.895780+0000","last_peered":"2024-09-19T04:46:41.895780+0000","last_clean":"2024-09-19T04:46:41.895780+0000","last_became_active":"2024-09-19T04:44:30.747566+0000","last_became_peered":"2024-09-19T04:44:30.747566+0000","last_unstale":"2024-09-19T04:46:41.895780+0000","last_undegraded":"2024-09-19T04:46:41.895780+0000","last_fullsized":"2024-09-19T04:46:41.895780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:32.406745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.786105+0000","last_change":"2024-09-19T04:44:28.334217+0000","last_active":"2024-09-19T04:46:41.786105+0000","last_peered":"2024-09-19T04:46:41.786105+0000","last_clean":"2024-09-19T04:46:41.786105+0000","last_became_active":"2024-09-19T04:44:28.333768+0000","last_became_peered":"2024-09-19T04:44:28.333768+0000","last_unstale":"2024-09-19T04:46:41.786105+0000","last_undegraded":"2024-09-19T04:46:41.786105+0000","last_fullsized":"2024-09-19T04:46:41.786105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:52:56.626635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032821+0000","last_change":"2024-09-19T04:44:30.747498+0000","last_active":"2024-09-19T04:46:42.032821+0000","last_peered":"2024-09-19T04:46:42.032821+0000","last_clean":"2024-09-19T04:46:42.032821+0000","last_became_active":"2024-09-19T04:44:30.746822+0000","last_became_peered":"2024-09-19T04:44:30.746822+0000","last_unstale":"2024-09-19T04:46:42.032821+0000","last_undegraded":"2024-09-19T04:46:42.032821+0000","last_fullsized":"2024-09-19T04:46:42.032821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:09.460263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032840+0000","last_change":"2024-09-19T04:45:14.645843+0000","last_active":"2024-09-19T04:46:42.032840+0000","last_peered":"2024-09-19T04:46:42.032840+0000","last_clean":"2024-09-19T04:46:42.032840+0000","last_became_active":"2024-09-19T04:45:14.632156+0000","last_became_peered":"2024-09-19T04:45:14.632156+0000","last_unstale":"2024-09-19T04:46:42.032840+0000","last_undegraded":"2024-09-19T04:46:42.032840+0000","last_fullsized":"2024-09-19T04:46:42.032840+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:34.406160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782880+0000","last_change":"2024-09-19T04:44:30.762776+0000","last_active":"2024-09-19T04:46:41.782880+0000","last_peered":"2024-09-19T04:46:41.782880+0000","last_clean":"2024-09-19T04:46:41.782880+0000","last_became_active":"2024-09-19T04:44:30.761308+0000","last_became_peered":"2024-09-19T04:44:30.761308+0000","last_unstale":"2024-09-19T04:46:41.782880+0000","last_undegraded":"2024-09-19T04:46:41.782880+0000","last_fullsized":"2024-09-19T04:46:41.782880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:19.960125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.887257+0000","last_change":"2024-09-19T04:44:28.594858+0000","last_active":"2024-09-19T04:46:41.887257+0000","last_peered":"2024-09-19T04:46:41.887257+0000","last_clean":"2024-09-19T04:46:41.887257+0000","last_became_active":"2024-09-19T04:44:28.594515+0000","last_became_peered":"2024-09-19T04:44:28.594515+0000","last_unstale":"2024-09-19T04:46:41.887257+0000","last_undegraded":"2024-09-19T04:46:41.887257+0000","last_fullsized":"2024-09-19T04:46:41.887257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:53.728393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886920+0000","last_change":"2024-09-19T04:44:30.753604+0000","last_active":"2024-09-19T04:46:41.886920+0000","last_peered":"2024-09-19T04:46:41.886920+0000","last_clean":"2024-09-19T04:46:41.886920+0000","last_became_active":"2024-09-19T04:44:30.751961+0000","last_became_peered":"2024-09-19T04:44:30.751961+0000","last_unstale":"2024-09-19T04:46:41.886920+0000","last_undegraded":"2024-09-19T04:46:41.886920+0000","last_fullsized":"2024-09-19T04:46:41.886920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:46:58.759650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"87'14","reported_seq":160,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.010647+0000","last_change":"2024-09-19T04:44:28.339499+0000","last_active":"2024-09-19T04:46:42.010647+0000","last_peered":"2024-09-19T04:46:42.010647+0000","last_clean":"2024-09-19T04:46:42.010647+0000","last_became_active":"2024-09-19T04:44:28.338774+0000","last_became_peered":"2024-09-19T04:44:28.338774+0000","last_unstale":"2024-09-19T04:46:42.010647+0000","last_undegraded":"2024-09-19T04:46:42.010647+0000","last_fullsized":"2024-09-19T04:46:42.010647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:12.261237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599827,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1313,"num_read_kb":763,"num_write":372,"num_write_kb":1461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1799481,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108024,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481833,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":954368,"data_stored":464859,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710765,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":1254505,"data_compressed":45058,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874290,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":858005,"data_compressed":22040,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":465167,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364860,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1028096,"data_stored":1057828,"data_compressed":38902,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528387,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":962560,"data_stored":465292,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1189358,"data_compressed":45058,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953548,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":464901,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1024000,"data_stored":989147,"data_compressed":28195,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1052672,"data_stored":532908,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":532688,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":465845,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135206,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200353,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:46:47.496 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-19T04:46:47.497 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:46:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.691+0000 7f03b3200640 1 -- 172.21.15.123:0/798332366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ac074620 msgr2=0x7f03ac074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.691+0000 7f03b3200640 1 --2- 172.21.15.123:0/798332366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ac074620 0x7f03ac074a00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009900 tx=0x7f03a002a480 comp rx=0 tx=0).stop 2024-09-19T04:46:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.692+0000 7f03b3200640 1 -- 172.21.15.123:0/798332366 shutdown_connections 2024-09-19T04:46:47.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.692+0000 7f03b3200640 1 --2- 172.21.15.123:0/798332366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03ac10e120 0x7f03ac110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.692+0000 7f03b3200640 1 --2- 172.21.15.123:0/798332366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03ac074f40 0x7f03ac10dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.692+0000 7f03b3200640 1 --2- 172.21.15.123:0/798332366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ac074620 0x7f03ac074a00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.692+0000 7f03b3200640 1 -- 172.21.15.123:0/798332366 >> 172.21.15.123:0/798332366 conn(0x7f03ac0fd5e0 msgr2=0x7f03ac0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:47.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.692+0000 7f03b3200640 1 -- 172.21.15.123:0/798332366 shutdown_connections 2024-09-19T04:46:47.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.692+0000 7f03b3200640 1 -- 172.21.15.123:0/798332366 wait complete. 2024-09-19T04:46:47.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.692+0000 7f03b3200640 1 Processor -- start 2024-09-19T04:46:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.693+0000 7f03b3200640 1 -- start start 2024-09-19T04:46:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.693+0000 7f03b3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03ac074620 0x7f03ac19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.693+0000 7f03b3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03ac074f40 0x7f03ac19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.693+0000 7f03b3200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ac10e120 0x7f03ac19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.693+0000 7f03b3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac113ee0 con 0x7f03ac074f40 2024-09-19T04:46:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.693+0000 7f03b3200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac113d60 con 0x7f03ac074620 2024-09-19T04:46:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.693+0000 7f03b3200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac114060 con 0x7f03ac10e120 2024-09-19T04:46:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.693+0000 7f03b1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03ac074f40 0x7f03ac19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.693+0000 7f03b1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03ac074620 0x7f03ac19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:47.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.693+0000 7f03b1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03ac074f40 0x7f03ac19a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50678/0 (socket says 172.21.15.123:50678) 2024-09-19T04:46:47.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.693+0000 7f03b1400640 1 -- 172.21.15.123:0/2258462768 learned_addr learned my addr 172.21.15.123:0/2258462768 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:47.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.693+0000 7f03b2800640 1 --2- 172.21.15.123:0/2258462768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ac10e120 0x7f03ac19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:47.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.694+0000 7f03b1e00640 1 -- 172.21.15.123:0/2258462768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ac10e120 msgr2=0x7f03ac19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:47.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.694+0000 7f03b1e00640 1 --2- 172.21.15.123:0/2258462768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ac10e120 0x7f03ac19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.694+0000 7f03b1e00640 1 -- 172.21.15.123:0/2258462768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03ac074f40 msgr2=0x7f03ac19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:47.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.694+0000 7f03b1e00640 1 --2- 172.21.15.123:0/2258462768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03ac074f40 0x7f03ac19a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.694+0000 7f03b1e00640 1 -- 172.21.15.123:0/2258462768 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac19f420 con 0x7f03ac074620 2024-09-19T04:46:47.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.694+0000 7f03b2800640 1 --2- 172.21.15.123:0/2258462768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ac10e120 0x7f03ac19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:47.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.694+0000 7f03b1400640 1 --2- 172.21.15.123:0/2258462768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03ac074f40 0x7f03ac19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:47.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.694+0000 7f03b1e00640 1 --2- 172.21.15.123:0/2258462768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03ac074620 0x7f03ac19a430 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f03a002a9a0 tx=0x7f03a0003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:47.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.694+0000 7f03a6a00640 1 -- 172.21.15.123:0/2258462768 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a003f070 con 0x7f03ac074620 2024-09-19T04:46:47.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.694+0000 7f03b3200640 1 -- 172.21.15.123:0/2258462768 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ac19f6b0 con 0x7f03ac074620 2024-09-19T04:46:47.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.694+0000 7f03b3200640 1 -- 172.21.15.123:0/2258462768 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ac1a6fa0 con 0x7f03ac074620 2024-09-19T04:46:47.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.694+0000 7f03a6a00640 1 -- 172.21.15.123:0/2258462768 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03a0031c90 con 0x7f03ac074620 2024-09-19T04:46:47.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.695+0000 7f03a6a00640 1 -- 172.21.15.123:0/2258462768 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0049060 con 0x7f03ac074620 2024-09-19T04:46:47.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.695+0000 7f03b3200640 1 -- 172.21.15.123:0/2258462768 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ac074a00 con 0x7f03ac074620 2024-09-19T04:46:47.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.698+0000 7f03a6a00640 1 -- 172.21.15.123:0/2258462768 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 138583+0+0 (secure 0 0 0) 0x7f03a0042070 con 0x7f03ac074620 2024-09-19T04:46:47.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.699+0000 7f03a6a00640 1 --2- 172.21.15.123:0/2258462768 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f03940a3560 0x7f03940a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:47.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.699+0000 7f03a6a00640 1 -- 172.21.15.123:0/2258462768 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f03a00f06b0 con 0x7f03ac074620 2024-09-19T04:46:47.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.699+0000 7f03b1400640 1 --2- 172.21.15.123:0/2258462768 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f03940a3560 0x7f03940a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:47.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.700+0000 7f03b1400640 1 --2- 172.21.15.123:0/2258462768 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f03940a3560 0x7f03940a5a20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03ac10f430 tx=0x7f039c005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:47.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.701+0000 7f03a6a00640 1 -- 172.21.15.123:0/2258462768 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f03a00bc890 con 0x7f03ac074620 2024-09-19T04:46:47.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.886+0000 7f03b3200640 1 -- 172.21.15.123:0/2258462768 --> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f03ac109310 con 0x7f03940a3560 2024-09-19T04:46:47.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.895+0000 7f03a6a00640 1 -- 172.21.15.123:0/2258462768 <== mgr.25133 v2:172.21.15.123:6800/1578549821 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258051 (secure 0 0 0) 0x7f039c00f4e0 con 0x7f03940a3560 2024-09-19T04:46:47.900 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:46:47.901 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-09-19T04:46:47.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.899+0000 7f03b3200640 1 -- 172.21.15.123:0/2258462768 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f03940a3560 msgr2=0x7f03940a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:47.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.899+0000 7f03b3200640 1 --2- 172.21.15.123:0/2258462768 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f03940a3560 0x7f03940a5a20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03ac10f430 tx=0x7f039c005f90 comp rx=0 tx=0).stop 2024-09-19T04:46:47.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.899+0000 7f03b3200640 1 -- 172.21.15.123:0/2258462768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03ac074620 msgr2=0x7f03ac19a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:47.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.899+0000 7f03b3200640 1 --2- 172.21.15.123:0/2258462768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03ac074620 0x7f03ac19a430 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f03a002a9a0 tx=0x7f03a0003b20 comp rx=0 tx=0).stop 2024-09-19T04:46:47.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.899+0000 7f03b3200640 1 -- 172.21.15.123:0/2258462768 shutdown_connections 2024-09-19T04:46:47.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.899+0000 7f03b3200640 1 --2- 172.21.15.123:0/2258462768 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f03940a3560 0x7f03940a5a20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.899+0000 7f03b3200640 1 --2- 172.21.15.123:0/2258462768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ac10e120 0x7f03ac19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.899+0000 7f03b3200640 1 --2- 172.21.15.123:0/2258462768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03ac074f40 0x7f03ac19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.899+0000 7f03b3200640 1 --2- 172.21.15.123:0/2258462768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03ac074620 0x7f03ac19a430 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:47.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.899+0000 7f03b3200640 1 -- 172.21.15.123:0/2258462768 >> 172.21.15.123:0/2258462768 conn(0x7f03ac0fd5e0 msgr2=0x7f03ac0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:47.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.900+0000 7f03b3200640 1 -- 172.21.15.123:0/2258462768 shutdown_connections 2024-09-19T04:46:47.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:47.900+0000 7f03b3200640 1 -- 172.21.15.123:0/2258462768 wait complete. 2024-09-19T04:46:47.916 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:47 smithi123 ceph-mon[30511]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:46:47.916 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:47 smithi123 ceph-mon[30511]: Updating smithi183:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:46:47.916 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:47 smithi123 ceph-mon[30511]: Updating smithi193:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:46:47.916 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:47 smithi123 ceph-mon[30511]: Updating smithi123:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:46:47.916 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:47 smithi123 ceph-mon[30511]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:46:47.916 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:47 smithi123 ceph-mon[30511]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:46:47.916 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:47 smithi123 ceph-mon[30511]: Updating smithi183:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:46:47.916 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:47 smithi123 ceph-mon[30511]: Updating smithi193:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:46:47.916 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:47 smithi123 ceph-mon[30511]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-19T04:46:47.916 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:47 smithi123 ceph-mon[30511]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:46:47.916 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:47 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:47.918 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-09-19T04:46:47.796926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190429,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1449,"num_read_kb":882,"num_write":555,"num_write_kb":4401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":938,"ondisk_log_size":938,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357032,"kb_used_data":11892,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584792,"statfs":{"total":1151940427776,"available":1151574827008,"internally_reserved":0,"allocated":12177408,"data_stored":8740896,"data_compressed":199511,"data_compressed_allocated":258048,"data_compressed_original":3022848,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.978201"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033086+0000","last_change":"2024-09-19T04:44:30.754400+0000","last_active":"2024-09-19T04:46:42.033086+0000","last_peered":"2024-09-19T04:46:42.033086+0000","last_clean":"2024-09-19T04:46:42.033086+0000","last_became_active":"2024-09-19T04:44:30.754028+0000","last_became_peered":"2024-09-19T04:44:30.754028+0000","last_unstale":"2024-09-19T04:46:42.033086+0000","last_undegraded":"2024-09-19T04:46:42.033086+0000","last_fullsized":"2024-09-19T04:46:42.033086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:25.668478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782861+0000","last_change":"2024-09-19T04:44:30.758072+0000","last_active":"2024-09-19T04:46:41.782861+0000","last_peered":"2024-09-19T04:46:41.782861+0000","last_clean":"2024-09-19T04:46:41.782861+0000","last_became_active":"2024-09-19T04:44:30.756734+0000","last_became_peered":"2024-09-19T04:44:30.756734+0000","last_unstale":"2024-09-19T04:46:41.782861+0000","last_undegraded":"2024-09-19T04:46:41.782861+0000","last_fullsized":"2024-09-19T04:46:41.782861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:56:47.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782852+0000","last_change":"2024-09-19T04:44:30.760208+0000","last_active":"2024-09-19T04:46:41.782852+0000","last_peered":"2024-09-19T04:46:41.782852+0000","last_clean":"2024-09-19T04:46:41.782852+0000","last_became_active":"2024-09-19T04:44:30.759719+0000","last_became_peered":"2024-09-19T04:44:30.759719+0000","last_unstale":"2024-09-19T04:46:41.782852+0000","last_undegraded":"2024-09-19T04:46:41.782852+0000","last_fullsized":"2024-09-19T04:46:41.782852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:14.041853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033102+0000","last_change":"2024-09-19T04:44:30.747813+0000","last_active":"2024-09-19T04:46:42.033102+0000","last_peered":"2024-09-19T04:46:42.033102+0000","last_clean":"2024-09-19T04:46:42.033102+0000","last_became_active":"2024-09-19T04:44:30.747486+0000","last_became_peered":"2024-09-19T04:44:30.747486+0000","last_unstale":"2024-09-19T04:46:42.033102+0000","last_undegraded":"2024-09-19T04:46:42.033102+0000","last_fullsized":"2024-09-19T04:46:42.033102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:11.315273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785693+0000","last_change":"2024-09-19T04:44:30.755800+0000","last_active":"2024-09-19T04:46:41.785693+0000","last_peered":"2024-09-19T04:46:41.785693+0000","last_clean":"2024-09-19T04:46:41.785693+0000","last_became_active":"2024-09-19T04:44:30.755448+0000","last_became_peered":"2024-09-19T04:44:30.755448+0000","last_unstale":"2024-09-19T04:46:41.785693+0000","last_undegraded":"2024-09-19T04:46:41.785693+0000","last_fullsized":"2024-09-19T04:46:41.785693+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:39.308614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.898818+0000","last_change":"2024-09-19T04:44:30.746230+0000","last_active":"2024-09-19T04:46:41.898818+0000","last_peered":"2024-09-19T04:46:41.898818+0000","last_clean":"2024-09-19T04:46:41.898818+0000","last_became_active":"2024-09-19T04:44:30.745884+0000","last_became_peered":"2024-09-19T04:44:30.745884+0000","last_unstale":"2024-09-19T04:46:41.898818+0000","last_undegraded":"2024-09-19T04:46:41.898818+0000","last_fullsized":"2024-09-19T04:46:41.898818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:51:58.550322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782104+0000","last_change":"2024-09-19T04:44:30.747870+0000","last_active":"2024-09-19T04:46:41.782104+0000","last_peered":"2024-09-19T04:46:41.782104+0000","last_clean":"2024-09-19T04:46:41.782104+0000","last_became_active":"2024-09-19T04:44:30.747536+0000","last_became_peered":"2024-09-19T04:44:30.747536+0000","last_unstale":"2024-09-19T04:46:41.782104+0000","last_undegraded":"2024-09-19T04:46:41.782104+0000","last_fullsized":"2024-09-19T04:46:41.782104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:20.074032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886225+0000","last_change":"2024-09-19T04:44:30.750425+0000","last_active":"2024-09-19T04:46:41.886225+0000","last_peered":"2024-09-19T04:46:41.886225+0000","last_clean":"2024-09-19T04:46:41.886225+0000","last_became_active":"2024-09-19T04:44:30.749841+0000","last_became_peered":"2024-09-19T04:44:30.749841+0000","last_unstale":"2024-09-19T04:46:41.886225+0000","last_undegraded":"2024-09-19T04:46:41.886225+0000","last_fullsized":"2024-09-19T04:46:41.886225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:12:08.420727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032832+0000","last_change":"2024-09-19T04:44:30.750624+0000","last_active":"2024-09-19T04:46:42.032832+0000","last_peered":"2024-09-19T04:46:42.032832+0000","last_clean":"2024-09-19T04:46:42.032832+0000","last_became_active":"2024-09-19T04:44:30.748370+0000","last_became_peered":"2024-09-19T04:44:30.748370+0000","last_unstale":"2024-09-19T04:46:42.032832+0000","last_undegraded":"2024-09-19T04:46:42.032832+0000","last_fullsized":"2024-09-19T04:46:42.032832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:36.131634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.899170+0000","last_change":"2024-09-19T04:44:30.750035+0000","last_active":"2024-09-19T04:46:41.899170+0000","last_peered":"2024-09-19T04:46:41.899170+0000","last_clean":"2024-09-19T04:46:41.899170+0000","last_became_active":"2024-09-19T04:44:30.749699+0000","last_became_peered":"2024-09-19T04:44:30.749699+0000","last_unstale":"2024-09-19T04:46:41.899170+0000","last_undegraded":"2024-09-19T04:46:41.899170+0000","last_fullsized":"2024-09-19T04:46:41.899170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:22.637193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782572+0000","last_change":"2024-09-19T04:44:30.760258+0000","last_active":"2024-09-19T04:46:41.782572+0000","last_peered":"2024-09-19T04:46:41.782572+0000","last_clean":"2024-09-19T04:46:41.782572+0000","last_became_active":"2024-09-19T04:44:30.759925+0000","last_became_peered":"2024-09-19T04:44:30.759925+0000","last_unstale":"2024-09-19T04:46:41.782572+0000","last_undegraded":"2024-09-19T04:46:41.782572+0000","last_fullsized":"2024-09-19T04:46:41.782572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:58.144691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033125+0000","last_change":"2024-09-19T04:44:30.747703+0000","last_active":"2024-09-19T04:46:42.033125+0000","last_peered":"2024-09-19T04:46:42.033125+0000","last_clean":"2024-09-19T04:46:42.033125+0000","last_became_active":"2024-09-19T04:44:30.747369+0000","last_became_peered":"2024-09-19T04:44:30.747369+0000","last_unstale":"2024-09-19T04:46:42.033125+0000","last_undegraded":"2024-09-19T04:46:42.033125+0000","last_fullsized":"2024-09-19T04:46:42.033125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:06.920357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.034354+0000","last_change":"2024-09-19T04:44:30.748844+0000","last_active":"2024-09-19T04:46:42.034354+0000","last_peered":"2024-09-19T04:46:42.034354+0000","last_clean":"2024-09-19T04:46:42.034354+0000","last_became_active":"2024-09-19T04:44:30.748513+0000","last_became_peered":"2024-09-19T04:44:30.748513+0000","last_unstale":"2024-09-19T04:46:42.034354+0000","last_undegraded":"2024-09-19T04:46:42.034354+0000","last_fullsized":"2024-09-19T04:46:42.034354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:45.107718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782714+0000","last_change":"2024-09-19T04:44:30.750526+0000","last_active":"2024-09-19T04:46:41.782714+0000","last_peered":"2024-09-19T04:46:41.782714+0000","last_clean":"2024-09-19T04:46:41.782714+0000","last_became_active":"2024-09-19T04:44:30.749693+0000","last_became_peered":"2024-09-19T04:44:30.749693+0000","last_unstale":"2024-09-19T04:46:41.782714+0000","last_undegraded":"2024-09-19T04:46:41.782714+0000","last_fullsized":"2024-09-19T04:46:41.782714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:43:04.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782639+0000","last_change":"2024-09-19T04:44:30.754310+0000","last_active":"2024-09-19T04:46:41.782639+0000","last_peered":"2024-09-19T04:46:41.782639+0000","last_clean":"2024-09-19T04:46:41.782639+0000","last_became_active":"2024-09-19T04:44:30.753933+0000","last_became_peered":"2024-09-19T04:44:30.753933+0000","last_unstale":"2024-09-19T04:46:41.782639+0000","last_undegraded":"2024-09-19T04:46:41.782639+0000","last_fullsized":"2024-09-19T04:46:41.782639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:40:44.985693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032386+0000","last_change":"2024-09-19T04:44:30.757458+0000","last_active":"2024-09-19T04:46:42.032386+0000","last_peered":"2024-09-19T04:46:42.032386+0000","last_clean":"2024-09-19T04:46:42.032386+0000","last_became_active":"2024-09-19T04:44:30.757127+0000","last_became_peered":"2024-09-19T04:44:30.757127+0000","last_unstale":"2024-09-19T04:46:42.032386+0000","last_undegraded":"2024-09-19T04:46:42.032386+0000","last_fullsized":"2024-09-19T04:46:42.032386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:59.325865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033118+0000","last_change":"2024-09-19T04:44:30.754685+0000","last_active":"2024-09-19T04:46:42.033118+0000","last_peered":"2024-09-19T04:46:42.033118+0000","last_clean":"2024-09-19T04:46:42.033118+0000","last_became_active":"2024-09-19T04:44:30.753625+0000","last_became_peered":"2024-09-19T04:44:30.753625+0000","last_unstale":"2024-09-19T04:46:42.033118+0000","last_undegraded":"2024-09-19T04:46:42.033118+0000","last_fullsized":"2024-09-19T04:46:42.033118+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:46:01.164693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.882473+0000","last_change":"2024-09-19T04:44:30.745823+0000","last_active":"2024-09-19T04:46:41.882473+0000","last_peered":"2024-09-19T04:46:41.882473+0000","last_clean":"2024-09-19T04:46:41.882473+0000","last_became_active":"2024-09-19T04:44:30.745224+0000","last_became_peered":"2024-09-19T04:44:30.745224+0000","last_unstale":"2024-09-19T04:46:41.882473+0000","last_undegraded":"2024-09-19T04:46:41.882473+0000","last_fullsized":"2024-09-19T04:46:41.882473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:23.682118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.012014+0000","last_change":"2024-09-19T04:44:30.752138+0000","last_active":"2024-09-19T04:46:42.012014+0000","last_peered":"2024-09-19T04:46:42.012014+0000","last_clean":"2024-09-19T04:46:42.012014+0000","last_became_active":"2024-09-19T04:44:30.751737+0000","last_became_peered":"2024-09-19T04:44:30.751737+0000","last_unstale":"2024-09-19T04:46:42.012014+0000","last_undegraded":"2024-09-19T04:46:42.012014+0000","last_fullsized":"2024-09-19T04:46:42.012014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:08.204601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032760+0000","last_change":"2024-09-19T04:44:30.748870+0000","last_active":"2024-09-19T04:46:42.032760+0000","last_peered":"2024-09-19T04:46:42.032760+0000","last_clean":"2024-09-19T04:46:42.032760+0000","last_became_active":"2024-09-19T04:44:30.748549+0000","last_became_peered":"2024-09-19T04:44:30.748549+0000","last_unstale":"2024-09-19T04:46:42.032760+0000","last_undegraded":"2024-09-19T04:46:42.032760+0000","last_fullsized":"2024-09-19T04:46:42.032760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:12.951564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.034496+0000","last_change":"2024-09-19T04:44:30.759937+0000","last_active":"2024-09-19T04:46:42.034496+0000","last_peered":"2024-09-19T04:46:42.034496+0000","last_clean":"2024-09-19T04:46:42.034496+0000","last_became_active":"2024-09-19T04:44:30.759659+0000","last_became_peered":"2024-09-19T04:44:30.759659+0000","last_unstale":"2024-09-19T04:46:42.034496+0000","last_undegraded":"2024-09-19T04:46:42.034496+0000","last_fullsized":"2024-09-19T04:46:42.034496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:07.608994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032624+0000","last_change":"2024-09-19T04:44:30.756216+0000","last_active":"2024-09-19T04:46:42.032624+0000","last_peered":"2024-09-19T04:46:42.032624+0000","last_clean":"2024-09-19T04:46:42.032624+0000","last_became_active":"2024-09-19T04:44:30.755903+0000","last_became_peered":"2024-09-19T04:44:30.755903+0000","last_unstale":"2024-09-19T04:46:42.032624+0000","last_undegraded":"2024-09-19T04:46:42.032624+0000","last_fullsized":"2024-09-19T04:46:42.032624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:16.966305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886344+0000","last_change":"2024-09-19T04:44:30.746297+0000","last_active":"2024-09-19T04:46:41.886344+0000","last_peered":"2024-09-19T04:46:41.886344+0000","last_clean":"2024-09-19T04:46:41.886344+0000","last_became_active":"2024-09-19T04:44:30.746013+0000","last_became_peered":"2024-09-19T04:44:30.746013+0000","last_unstale":"2024-09-19T04:46:41.886344+0000","last_undegraded":"2024-09-19T04:46:41.886344+0000","last_fullsized":"2024-09-19T04:46:41.886344+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:17.086180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785644+0000","last_change":"2024-09-19T04:45:14.630690+0000","last_active":"2024-09-19T04:46:41.785644+0000","last_peered":"2024-09-19T04:46:41.785644+0000","last_clean":"2024-09-19T04:46:41.785644+0000","last_became_active":"2024-09-19T04:45:14.630228+0000","last_became_peered":"2024-09-19T04:45:14.630228+0000","last_unstale":"2024-09-19T04:46:41.785644+0000","last_undegraded":"2024-09-19T04:46:41.785644+0000","last_fullsized":"2024-09-19T04:46:41.785644+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:39.901088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.887149+0000","last_change":"2024-09-19T04:44:30.753423+0000","last_active":"2024-09-19T04:46:41.887149+0000","last_peered":"2024-09-19T04:46:41.887149+0000","last_clean":"2024-09-19T04:46:41.887149+0000","last_became_active":"2024-09-19T04:44:30.752139+0000","last_became_peered":"2024-09-19T04:44:30.752139+0000","last_unstale":"2024-09-19T04:46:41.887149+0000","last_undegraded":"2024-09-19T04:46:41.887149+0000","last_fullsized":"2024-09-19T04:46:41.887149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:18.046893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.012113+0000","last_change":"2024-09-19T04:44:30.751114+0000","last_active":"2024-09-19T04:46:42.012113+0000","last_peered":"2024-09-19T04:46:42.012113+0000","last_clean":"2024-09-19T04:46:42.012113+0000","last_became_active":"2024-09-19T04:44:30.750784+0000","last_became_peered":"2024-09-19T04:44:30.750784+0000","last_unstale":"2024-09-19T04:46:42.012113+0000","last_undegraded":"2024-09-19T04:46:42.012113+0000","last_fullsized":"2024-09-19T04:46:42.012113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:22.040253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033057+0000","last_change":"2024-09-19T04:44:30.747191+0000","last_active":"2024-09-19T04:46:42.033057+0000","last_peered":"2024-09-19T04:46:42.033057+0000","last_clean":"2024-09-19T04:46:42.033057+0000","last_became_active":"2024-09-19T04:44:30.746830+0000","last_became_peered":"2024-09-19T04:44:30.746830+0000","last_unstale":"2024-09-19T04:46:42.033057+0000","last_undegraded":"2024-09-19T04:46:42.033057+0000","last_fullsized":"2024-09-19T04:46:42.033057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:24.634920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.034635+0000","last_change":"2024-09-19T04:44:30.755174+0000","last_active":"2024-09-19T04:46:42.034635+0000","last_peered":"2024-09-19T04:46:42.034635+0000","last_clean":"2024-09-19T04:46:42.034635+0000","last_became_active":"2024-09-19T04:44:30.754885+0000","last_became_peered":"2024-09-19T04:44:30.754885+0000","last_unstale":"2024-09-19T04:46:42.034635+0000","last_undegraded":"2024-09-19T04:46:42.034635+0000","last_fullsized":"2024-09-19T04:46:42.034635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:11:32.428033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782923+0000","last_change":"2024-09-19T04:45:14.891484+0000","last_active":"2024-09-19T04:46:41.782923+0000","last_peered":"2024-09-19T04:46:41.782923+0000","last_clean":"2024-09-19T04:46:41.782923+0000","last_became_active":"2024-09-19T04:45:14.891120+0000","last_became_peered":"2024-09-19T04:45:14.891120+0000","last_unstale":"2024-09-19T04:46:41.782923+0000","last_undegraded":"2024-09-19T04:46:41.782923+0000","last_fullsized":"2024-09-19T04:46:41.782923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:20.959571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.011793+0000","last_change":"2024-09-19T04:44:30.757343+0000","last_active":"2024-09-19T04:46:42.011793+0000","last_peered":"2024-09-19T04:46:42.011793+0000","last_clean":"2024-09-19T04:46:42.011793+0000","last_became_active":"2024-09-19T04:44:30.756981+0000","last_became_peered":"2024-09-19T04:44:30.756981+0000","last_unstale":"2024-09-19T04:46:42.011793+0000","last_undegraded":"2024-09-19T04:46:42.011793+0000","last_fullsized":"2024-09-19T04:46:42.011793+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:07.388384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.011916+0000","last_change":"2024-09-19T04:45:14.631104+0000","last_active":"2024-09-19T04:46:42.011916+0000","last_peered":"2024-09-19T04:46:42.011916+0000","last_clean":"2024-09-19T04:46:42.011916+0000","last_became_active":"2024-09-19T04:45:14.630654+0000","last_became_peered":"2024-09-19T04:45:14.630654+0000","last_unstale":"2024-09-19T04:46:42.011916+0000","last_undegraded":"2024-09-19T04:46:42.011916+0000","last_fullsized":"2024-09-19T04:46:42.011916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:37.659295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032830+0000","last_change":"2024-09-19T04:44:30.748408+0000","last_active":"2024-09-19T04:46:42.032830+0000","last_peered":"2024-09-19T04:46:42.032830+0000","last_clean":"2024-09-19T04:46:42.032830+0000","last_became_active":"2024-09-19T04:44:30.747960+0000","last_became_peered":"2024-09-19T04:44:30.747960+0000","last_unstale":"2024-09-19T04:46:42.032830+0000","last_undegraded":"2024-09-19T04:46:42.032830+0000","last_fullsized":"2024-09-19T04:46:42.032830+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:51.043791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785665+0000","last_change":"2024-09-19T04:44:30.747194+0000","last_active":"2024-09-19T04:46:41.785665+0000","last_peered":"2024-09-19T04:46:41.785665+0000","last_clean":"2024-09-19T04:46:41.785665+0000","last_became_active":"2024-09-19T04:44:30.746871+0000","last_became_peered":"2024-09-19T04:44:30.746871+0000","last_unstale":"2024-09-19T04:46:41.785665+0000","last_undegraded":"2024-09-19T04:46:41.785665+0000","last_fullsized":"2024-09-19T04:46:41.785665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:52.340341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":152,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.114293+0000","last_change":"2024-09-19T04:44:28.337746+0000","last_active":"2024-09-19T04:46:42.114293+0000","last_peered":"2024-09-19T04:46:42.114293+0000","last_clean":"2024-09-19T04:46:42.114293+0000","last_became_active":"2024-09-19T04:44:28.337410+0000","last_became_peered":"2024-09-19T04:44:28.337410+0000","last_unstale":"2024-09-19T04:46:42.114293+0000","last_undegraded":"2024-09-19T04:46:42.114293+0000","last_fullsized":"2024-09-19T04:46:42.114293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:41:27.436939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":2,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895593+0000","last_change":"2024-09-19T04:44:30.755512+0000","last_active":"2024-09-19T04:46:41.895593+0000","last_peered":"2024-09-19T04:46:41.895593+0000","last_clean":"2024-09-19T04:46:41.895593+0000","last_became_active":"2024-09-19T04:44:30.755086+0000","last_became_peered":"2024-09-19T04:44:30.755086+0000","last_unstale":"2024-09-19T04:46:41.895593+0000","last_undegraded":"2024-09-19T04:46:41.895593+0000","last_fullsized":"2024-09-19T04:46:41.895593+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:33.870376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.898886+0000","last_change":"2024-09-19T04:44:28.334492+0000","last_active":"2024-09-19T04:46:41.898886+0000","last_peered":"2024-09-19T04:46:41.898886+0000","last_clean":"2024-09-19T04:46:41.898886+0000","last_became_active":"2024-09-19T04:44:28.334098+0000","last_became_peered":"2024-09-19T04:44:28.334098+0000","last_unstale":"2024-09-19T04:46:41.898886+0000","last_undegraded":"2024-09-19T04:46:41.898886+0000","last_fullsized":"2024-09-19T04:46:41.898886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:48.156590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033952+0000","last_change":"2024-09-19T04:44:28.335593+0000","last_active":"2024-09-19T04:46:42.033952+0000","last_peered":"2024-09-19T04:46:42.033952+0000","last_clean":"2024-09-19T04:46:42.033952+0000","last_became_active":"2024-09-19T04:44:28.335060+0000","last_became_peered":"2024-09-19T04:44:28.335060+0000","last_unstale":"2024-09-19T04:46:42.033952+0000","last_undegraded":"2024-09-19T04:46:42.033952+0000","last_fullsized":"2024-09-19T04:46:42.033952+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:27.859960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886288+0000","last_change":"2024-09-19T04:44:30.748235+0000","last_active":"2024-09-19T04:46:41.886288+0000","last_peered":"2024-09-19T04:46:41.886288+0000","last_clean":"2024-09-19T04:46:41.886288+0000","last_became_active":"2024-09-19T04:44:30.747881+0000","last_became_peered":"2024-09-19T04:44:30.747881+0000","last_unstale":"2024-09-19T04:46:41.886288+0000","last_undegraded":"2024-09-19T04:46:41.886288+0000","last_fullsized":"2024-09-19T04:46:41.886288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:40:39.183897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895656+0000","last_change":"2024-09-19T04:44:28.337762+0000","last_active":"2024-09-19T04:46:41.895656+0000","last_peered":"2024-09-19T04:46:41.895656+0000","last_clean":"2024-09-19T04:46:41.895656+0000","last_became_active":"2024-09-19T04:44:28.337383+0000","last_became_peered":"2024-09-19T04:44:28.337383+0000","last_unstale":"2024-09-19T04:46:41.895656+0000","last_undegraded":"2024-09-19T04:46:41.895656+0000","last_fullsized":"2024-09-19T04:46:41.895656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:07:55.268958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.882010+0000","last_change":"2024-09-19T04:44:30.754318+0000","last_active":"2024-09-19T04:46:41.882010+0000","last_peered":"2024-09-19T04:46:41.882010+0000","last_clean":"2024-09-19T04:46:41.882010+0000","last_became_active":"2024-09-19T04:44:30.754025+0000","last_became_peered":"2024-09-19T04:44:30.754025+0000","last_unstale":"2024-09-19T04:46:41.882010+0000","last_undegraded":"2024-09-19T04:46:41.882010+0000","last_fullsized":"2024-09-19T04:46:41.882010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:58.190504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":266,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.010422+0000","last_change":"2024-09-19T04:44:28.335947+0000","last_active":"2024-09-19T04:46:42.010422+0000","last_peered":"2024-09-19T04:46:42.010422+0000","last_clean":"2024-09-19T04:46:42.010422+0000","last_became_active":"2024-09-19T04:44:28.335588+0000","last_became_peered":"2024-09-19T04:44:28.335588+0000","last_unstale":"2024-09-19T04:46:42.010422+0000","last_undegraded":"2024-09-19T04:46:42.010422+0000","last_fullsized":"2024-09-19T04:46:42.010422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:44.024915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":236,"num_read_kb":236,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032720+0000","last_change":"2024-09-19T04:44:30.748272+0000","last_active":"2024-09-19T04:46:42.032720+0000","last_peered":"2024-09-19T04:46:42.032720+0000","last_clean":"2024-09-19T04:46:42.032720+0000","last_became_active":"2024-09-19T04:44:30.747147+0000","last_became_peered":"2024-09-19T04:44:30.747147+0000","last_unstale":"2024-09-19T04:46:42.032720+0000","last_undegraded":"2024-09-19T04:46:42.032720+0000","last_fullsized":"2024-09-19T04:46:42.032720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:11.653693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782564+0000","last_change":"2024-09-19T04:44:28.337120+0000","last_active":"2024-09-19T04:46:41.782564+0000","last_peered":"2024-09-19T04:46:41.782564+0000","last_clean":"2024-09-19T04:46:41.782564+0000","last_became_active":"2024-09-19T04:44:28.336053+0000","last_became_peered":"2024-09-19T04:44:28.336053+0000","last_unstale":"2024-09-19T04:46:41.782564+0000","last_undegraded":"2024-09-19T04:46:41.782564+0000","last_fullsized":"2024-09-19T04:46:41.782564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:56:57.514931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782921+0000","last_change":"2024-09-19T04:44:30.755238+0000","last_active":"2024-09-19T04:46:41.782921+0000","last_peered":"2024-09-19T04:46:41.782921+0000","last_clean":"2024-09-19T04:46:41.782921+0000","last_became_active":"2024-09-19T04:44:30.754748+0000","last_became_peered":"2024-09-19T04:44:30.754748+0000","last_unstale":"2024-09-19T04:46:41.782921+0000","last_undegraded":"2024-09-19T04:46:41.782921+0000","last_fullsized":"2024-09-19T04:46:41.782921+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:38.954491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895818+0000","last_change":"2024-09-19T04:44:28.596147+0000","last_active":"2024-09-19T04:46:41.895818+0000","last_peered":"2024-09-19T04:46:41.895818+0000","last_clean":"2024-09-19T04:46:41.895818+0000","last_became_active":"2024-09-19T04:44:28.595785+0000","last_became_peered":"2024-09-19T04:44:28.595785+0000","last_unstale":"2024-09-19T04:46:41.895818+0000","last_undegraded":"2024-09-19T04:46:41.895818+0000","last_fullsized":"2024-09-19T04:46:41.895818+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:39.138724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032947+0000","last_change":"2024-09-19T04:44:30.753817+0000","last_active":"2024-09-19T04:46:42.032947+0000","last_peered":"2024-09-19T04:46:42.032947+0000","last_clean":"2024-09-19T04:46:42.032947+0000","last_became_active":"2024-09-19T04:44:30.752541+0000","last_became_peered":"2024-09-19T04:44:30.752541+0000","last_unstale":"2024-09-19T04:46:42.032947+0000","last_undegraded":"2024-09-19T04:46:42.032947+0000","last_fullsized":"2024-09-19T04:46:42.032947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:16.121822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'38","reported_seq":310,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.117197+0000","last_change":"2024-09-19T04:44:28.335449+0000","last_active":"2024-09-19T04:46:42.117197+0000","last_peered":"2024-09-19T04:46:42.117197+0000","last_clean":"2024-09-19T04:46:42.117197+0000","last_became_active":"2024-09-19T04:44:28.335080+0000","last_became_peered":"2024-09-19T04:44:28.335080+0000","last_unstale":"2024-09-19T04:46:42.117197+0000","last_undegraded":"2024-09-19T04:46:42.117197+0000","last_fullsized":"2024-09-19T04:46:42.117197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:28.436290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66483,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":247,"num_read_kb":182,"num_write":40,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.786101+0000","last_change":"2024-09-19T04:44:30.749624+0000","last_active":"2024-09-19T04:46:41.786101+0000","last_peered":"2024-09-19T04:46:41.786101+0000","last_clean":"2024-09-19T04:46:41.786101+0000","last_became_active":"2024-09-19T04:44:30.749295+0000","last_became_peered":"2024-09-19T04:44:30.749295+0000","last_unstale":"2024-09-19T04:46:41.786101+0000","last_undegraded":"2024-09-19T04:46:41.786101+0000","last_fullsized":"2024-09-19T04:46:41.786101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:48:46.942237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.011879+0000","last_change":"2024-09-19T04:44:28.337863+0000","last_active":"2024-09-19T04:46:42.011879+0000","last_peered":"2024-09-19T04:46:42.011879+0000","last_clean":"2024-09-19T04:46:42.011879+0000","last_became_active":"2024-09-19T04:44:28.337522+0000","last_became_peered":"2024-09-19T04:44:28.337522+0000","last_unstale":"2024-09-19T04:46:42.011879+0000","last_undegraded":"2024-09-19T04:46:42.011879+0000","last_fullsized":"2024-09-19T04:46:42.011879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:02.488622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033943+0000","last_change":"2024-09-19T04:44:30.759916+0000","last_active":"2024-09-19T04:46:42.033943+0000","last_peered":"2024-09-19T04:46:42.033943+0000","last_clean":"2024-09-19T04:46:42.033943+0000","last_became_active":"2024-09-19T04:44:30.758835+0000","last_became_peered":"2024-09-19T04:44:30.758835+0000","last_unstale":"2024-09-19T04:46:42.033943+0000","last_undegraded":"2024-09-19T04:46:42.033943+0000","last_fullsized":"2024-09-19T04:46:42.033943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:38.614343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895828+0000","last_change":"2024-09-19T04:44:28.337574+0000","last_active":"2024-09-19T04:46:41.895828+0000","last_peered":"2024-09-19T04:46:41.895828+0000","last_clean":"2024-09-19T04:46:41.895828+0000","last_became_active":"2024-09-19T04:44:28.337224+0000","last_became_peered":"2024-09-19T04:44:28.337224+0000","last_unstale":"2024-09-19T04:46:41.895828+0000","last_undegraded":"2024-09-19T04:46:41.895828+0000","last_fullsized":"2024-09-19T04:46:41.895828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:20.538181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782359+0000","last_change":"2024-09-19T04:44:30.758152+0000","last_active":"2024-09-19T04:46:41.782359+0000","last_peered":"2024-09-19T04:46:41.782359+0000","last_clean":"2024-09-19T04:46:41.782359+0000","last_became_active":"2024-09-19T04:44:30.757801+0000","last_became_peered":"2024-09-19T04:44:30.757801+0000","last_unstale":"2024-09-19T04:46:41.782359+0000","last_undegraded":"2024-09-19T04:46:41.782359+0000","last_fullsized":"2024-09-19T04:46:41.782359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:05.212471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'7","reported_seq":154,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032968+0000","last_change":"2024-09-19T04:44:28.339340+0000","last_active":"2024-09-19T04:46:42.032968+0000","last_peered":"2024-09-19T04:46:42.032968+0000","last_clean":"2024-09-19T04:46:42.032968+0000","last_became_active":"2024-09-19T04:44:28.339019+0000","last_became_peered":"2024-09-19T04:44:28.339019+0000","last_unstale":"2024-09-19T04:46:42.032968+0000","last_undegraded":"2024-09-19T04:46:42.032968+0000","last_fullsized":"2024-09-19T04:46:42.032968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:29:59.356748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782063+0000","last_change":"2024-09-19T04:44:30.751217+0000","last_active":"2024-09-19T04:46:41.782063+0000","last_peered":"2024-09-19T04:46:41.782063+0000","last_clean":"2024-09-19T04:46:41.782063+0000","last_became_active":"2024-09-19T04:44:30.750863+0000","last_became_peered":"2024-09-19T04:44:30.750863+0000","last_unstale":"2024-09-19T04:46:41.782063+0000","last_undegraded":"2024-09-19T04:46:41.782063+0000","last_fullsized":"2024-09-19T04:46:41.782063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:09.957572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785881+0000","last_change":"2024-09-19T04:44:28.596585+0000","last_active":"2024-09-19T04:46:41.785881+0000","last_peered":"2024-09-19T04:46:41.785881+0000","last_clean":"2024-09-19T04:46:41.785881+0000","last_became_active":"2024-09-19T04:44:28.596218+0000","last_became_peered":"2024-09-19T04:44:28.596218+0000","last_unstale":"2024-09-19T04:46:41.785881+0000","last_undegraded":"2024-09-19T04:46:41.785881+0000","last_fullsized":"2024-09-19T04:46:41.785881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:29:35.395610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895520+0000","last_change":"2024-09-19T04:44:30.746590+0000","last_active":"2024-09-19T04:46:41.895520+0000","last_peered":"2024-09-19T04:46:41.895520+0000","last_clean":"2024-09-19T04:46:41.895520+0000","last_became_active":"2024-09-19T04:44:30.746263+0000","last_became_peered":"2024-09-19T04:44:30.746263+0000","last_unstale":"2024-09-19T04:46:41.895520+0000","last_undegraded":"2024-09-19T04:46:41.895520+0000","last_fullsized":"2024-09-19T04:46:41.895520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:38.955613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.786340+0000","last_change":"2024-09-19T04:44:28.335826+0000","last_active":"2024-09-19T04:46:41.786340+0000","last_peered":"2024-09-19T04:46:41.786340+0000","last_clean":"2024-09-19T04:46:41.786340+0000","last_became_active":"2024-09-19T04:44:28.335481+0000","last_became_peered":"2024-09-19T04:44:28.335481+0000","last_unstale":"2024-09-19T04:46:41.786340+0000","last_undegraded":"2024-09-19T04:46:41.786340+0000","last_fullsized":"2024-09-19T04:46:41.786340+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:20.989500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032963+0000","last_change":"2024-09-19T04:44:30.744130+0000","last_active":"2024-09-19T04:46:42.032963+0000","last_peered":"2024-09-19T04:46:42.032963+0000","last_clean":"2024-09-19T04:46:42.032963+0000","last_became_active":"2024-09-19T04:44:30.743772+0000","last_became_peered":"2024-09-19T04:44:30.743772+0000","last_unstale":"2024-09-19T04:46:42.032963+0000","last_undegraded":"2024-09-19T04:46:42.032963+0000","last_fullsized":"2024-09-19T04:46:42.032963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:34.956670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.034296+0000","last_change":"2024-09-19T04:44:28.337543+0000","last_active":"2024-09-19T04:46:42.034296+0000","last_peered":"2024-09-19T04:46:42.034296+0000","last_clean":"2024-09-19T04:46:42.034296+0000","last_became_active":"2024-09-19T04:44:28.336128+0000","last_became_peered":"2024-09-19T04:44:28.336128+0000","last_unstale":"2024-09-19T04:46:42.034296+0000","last_undegraded":"2024-09-19T04:46:42.034296+0000","last_fullsized":"2024-09-19T04:46:42.034296+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:31.907777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782306+0000","last_change":"2024-09-19T04:44:30.757489+0000","last_active":"2024-09-19T04:46:41.782306+0000","last_peered":"2024-09-19T04:46:41.782306+0000","last_clean":"2024-09-19T04:46:41.782306+0000","last_became_active":"2024-09-19T04:44:30.757041+0000","last_became_peered":"2024-09-19T04:44:30.757041+0000","last_unstale":"2024-09-19T04:46:41.782306+0000","last_undegraded":"2024-09-19T04:46:41.782306+0000","last_fullsized":"2024-09-19T04:46:41.782306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:03.395913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'41","reported_seq":75,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.049157+0000","last_change":"2024-09-19T04:44:28.327393+0000","last_active":"2024-09-19T04:46:42.049157+0000","last_peered":"2024-09-19T04:46:42.049157+0000","last_clean":"2024-09-19T04:46:42.049157+0000","last_became_active":"2024-09-19T04:44:28.327116+0000","last_became_peered":"2024-09-19T04:44:28.327116+0000","last_unstale":"2024-09-19T04:46:42.049157+0000","last_undegraded":"2024-09-19T04:46:42.049157+0000","last_fullsized":"2024-09-19T04:46:42.049157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:42.360980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782342+0000","last_change":"2024-09-19T04:44:30.754239+0000","last_active":"2024-09-19T04:46:41.782342+0000","last_peered":"2024-09-19T04:46:41.782342+0000","last_clean":"2024-09-19T04:46:41.782342+0000","last_became_active":"2024-09-19T04:44:30.753885+0000","last_became_peered":"2024-09-19T04:44:30.753885+0000","last_unstale":"2024-09-19T04:46:41.782342+0000","last_undegraded":"2024-09-19T04:46:41.782342+0000","last_fullsized":"2024-09-19T04:46:41.782342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:27.355600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'5","reported_seq":152,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.012256+0000","last_change":"2024-09-19T04:44:28.596214+0000","last_active":"2024-09-19T04:46:42.012256+0000","last_peered":"2024-09-19T04:46:42.012256+0000","last_clean":"2024-09-19T04:46:42.012256+0000","last_became_active":"2024-09-19T04:44:28.595841+0000","last_became_peered":"2024-09-19T04:44:28.595841+0000","last_unstale":"2024-09-19T04:46:42.012256+0000","last_undegraded":"2024-09-19T04:46:42.012256+0000","last_fullsized":"2024-09-19T04:46:42.012256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:43.864702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782627+0000","last_change":"2024-09-19T04:44:30.756192+0000","last_active":"2024-09-19T04:46:41.782627+0000","last_peered":"2024-09-19T04:46:41.782627+0000","last_clean":"2024-09-19T04:46:41.782627+0000","last_became_active":"2024-09-19T04:44:30.755853+0000","last_became_peered":"2024-09-19T04:44:30.755853+0000","last_unstale":"2024-09-19T04:46:41.782627+0000","last_undegraded":"2024-09-19T04:46:41.782627+0000","last_fullsized":"2024-09-19T04:46:41.782627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:25:49.206978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":352,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782231+0000","last_change":"2024-09-19T04:43:10.156816+0000","last_active":"2024-09-19T04:46:41.782231+0000","last_peered":"2024-09-19T04:46:41.782231+0000","last_clean":"2024-09-19T04:46:41.782231+0000","last_became_active":"2024-09-19T04:43:10.156359+0000","last_became_peered":"2024-09-19T04:43:10.156359+0000","last_unstale":"2024-09-19T04:46:41.782231+0000","last_undegraded":"2024-09-19T04:46:41.782231+0000","last_fullsized":"2024-09-19T04:46:41.782231+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:39:34.200793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:39:34.200793+0000","last_clean_scrub_stamp":"2024-09-19T04:39:34.200793+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:13.453818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032920+0000","last_change":"2024-09-19T04:44:28.337637+0000","last_active":"2024-09-19T04:46:42.032920+0000","last_peered":"2024-09-19T04:46:42.032920+0000","last_clean":"2024-09-19T04:46:42.032920+0000","last_became_active":"2024-09-19T04:44:28.337235+0000","last_became_peered":"2024-09-19T04:44:28.337235+0000","last_unstale":"2024-09-19T04:46:42.032920+0000","last_undegraded":"2024-09-19T04:46:42.032920+0000","last_fullsized":"2024-09-19T04:46:42.032920+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:42.104972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.881909+0000","last_change":"2024-09-19T04:45:15.281422+0000","last_active":"2024-09-19T04:46:41.881909+0000","last_peered":"2024-09-19T04:46:41.881909+0000","last_clean":"2024-09-19T04:46:41.881909+0000","last_became_active":"2024-09-19T04:45:15.281031+0000","last_became_peered":"2024-09-19T04:45:15.281031+0000","last_unstale":"2024-09-19T04:46:41.881909+0000","last_undegraded":"2024-09-19T04:46:41.881909+0000","last_fullsized":"2024-09-19T04:46:41.881909+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:05.058469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782204+0000","last_change":"2024-09-19T04:44:28.336124+0000","last_active":"2024-09-19T04:46:41.782204+0000","last_peered":"2024-09-19T04:46:41.782204+0000","last_clean":"2024-09-19T04:46:41.782204+0000","last_became_active":"2024-09-19T04:44:28.335754+0000","last_became_peered":"2024-09-19T04:44:28.335754+0000","last_unstale":"2024-09-19T04:46:41.782204+0000","last_undegraded":"2024-09-19T04:46:41.782204+0000","last_fullsized":"2024-09-19T04:46:41.782204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:41.604367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033916+0000","last_change":"2024-09-19T04:44:30.749939+0000","last_active":"2024-09-19T04:46:42.033916+0000","last_peered":"2024-09-19T04:46:42.033916+0000","last_clean":"2024-09-19T04:46:42.033916+0000","last_became_active":"2024-09-19T04:44:30.747122+0000","last_became_peered":"2024-09-19T04:44:30.747122+0000","last_unstale":"2024-09-19T04:46:42.033916+0000","last_undegraded":"2024-09-19T04:46:42.033916+0000","last_fullsized":"2024-09-19T04:46:42.033916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:46:08.371493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":151,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.039159+0000","last_change":"2024-09-19T04:44:28.594768+0000","last_active":"2024-09-19T04:46:42.039159+0000","last_peered":"2024-09-19T04:46:42.039159+0000","last_clean":"2024-09-19T04:46:42.039159+0000","last_became_active":"2024-09-19T04:44:28.594320+0000","last_became_peered":"2024-09-19T04:44:28.594320+0000","last_unstale":"2024-09-19T04:46:42.039159+0000","last_undegraded":"2024-09-19T04:46:42.039159+0000","last_fullsized":"2024-09-19T04:46:42.039159+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:32:17.736874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895518+0000","last_change":"2024-09-19T04:44:30.746666+0000","last_active":"2024-09-19T04:46:41.895518+0000","last_peered":"2024-09-19T04:46:41.895518+0000","last_clean":"2024-09-19T04:46:41.895518+0000","last_became_active":"2024-09-19T04:44:30.746312+0000","last_became_peered":"2024-09-19T04:44:30.746312+0000","last_unstale":"2024-09-19T04:46:41.895518+0000","last_undegraded":"2024-09-19T04:46:41.895518+0000","last_fullsized":"2024-09-19T04:46:41.895518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:13.562810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"87'7","reported_seq":153,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.117972+0000","last_change":"2024-09-19T04:44:28.336492+0000","last_active":"2024-09-19T04:46:42.117972+0000","last_peered":"2024-09-19T04:46:42.117972+0000","last_clean":"2024-09-19T04:46:42.117972+0000","last_became_active":"2024-09-19T04:44:28.336044+0000","last_became_peered":"2024-09-19T04:44:28.336044+0000","last_unstale":"2024-09-19T04:46:42.117972+0000","last_undegraded":"2024-09-19T04:46:42.117972+0000","last_fullsized":"2024-09-19T04:46:42.117972+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:18.760473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.899051+0000","last_change":"2024-09-19T04:45:14.630959+0000","last_active":"2024-09-19T04:46:41.899051+0000","last_peered":"2024-09-19T04:46:41.899051+0000","last_clean":"2024-09-19T04:46:41.899051+0000","last_became_active":"2024-09-19T04:45:14.630528+0000","last_became_peered":"2024-09-19T04:45:14.630528+0000","last_unstale":"2024-09-19T04:46:41.899051+0000","last_undegraded":"2024-09-19T04:46:41.899051+0000","last_fullsized":"2024-09-19T04:46:41.899051+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:29.754355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.882436+0000","last_change":"2024-09-19T04:44:28.596322+0000","last_active":"2024-09-19T04:46:41.882436+0000","last_peered":"2024-09-19T04:46:41.882436+0000","last_clean":"2024-09-19T04:46:41.882436+0000","last_became_active":"2024-09-19T04:44:28.595566+0000","last_became_peered":"2024-09-19T04:44:28.595566+0000","last_unstale":"2024-09-19T04:46:41.882436+0000","last_undegraded":"2024-09-19T04:46:41.882436+0000","last_fullsized":"2024-09-19T04:46:41.882436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:23.878145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.899012+0000","last_change":"2024-09-19T04:44:30.752321+0000","last_active":"2024-09-19T04:46:41.899012+0000","last_peered":"2024-09-19T04:46:41.899012+0000","last_clean":"2024-09-19T04:46:41.899012+0000","last_became_active":"2024-09-19T04:44:30.751980+0000","last_became_peered":"2024-09-19T04:44:30.751980+0000","last_unstale":"2024-09-19T04:46:41.899012+0000","last_undegraded":"2024-09-19T04:46:41.899012+0000","last_fullsized":"2024-09-19T04:46:41.899012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:41.863034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.786281+0000","last_change":"2024-09-19T04:44:28.596547+0000","last_active":"2024-09-19T04:46:41.786281+0000","last_peered":"2024-09-19T04:46:41.786281+0000","last_clean":"2024-09-19T04:46:41.786281+0000","last_became_active":"2024-09-19T04:44:28.596181+0000","last_became_peered":"2024-09-19T04:44:28.596181+0000","last_unstale":"2024-09-19T04:46:41.786281+0000","last_undegraded":"2024-09-19T04:46:41.786281+0000","last_fullsized":"2024-09-19T04:46:41.786281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:53.251737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.034777+0000","last_change":"2024-09-19T04:44:30.751533+0000","last_active":"2024-09-19T04:46:42.034777+0000","last_peered":"2024-09-19T04:46:42.034777+0000","last_clean":"2024-09-19T04:46:42.034777+0000","last_became_active":"2024-09-19T04:44:30.748012+0000","last_became_peered":"2024-09-19T04:44:30.748012+0000","last_unstale":"2024-09-19T04:46:42.034777+0000","last_undegraded":"2024-09-19T04:46:42.034777+0000","last_fullsized":"2024-09-19T04:46:42.034777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:32.360147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'52","reported_seq":102,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.770637+0000","last_change":"2024-09-19T04:44:28.335884+0000","last_active":"2024-09-19T04:46:42.770637+0000","last_peered":"2024-09-19T04:46:42.770637+0000","last_clean":"2024-09-19T04:46:42.770637+0000","last_became_active":"2024-09-19T04:44:28.335521+0000","last_became_peered":"2024-09-19T04:44:28.335521+0000","last_unstale":"2024-09-19T04:46:42.770637+0000","last_undegraded":"2024-09-19T04:46:42.770637+0000","last_fullsized":"2024-09-19T04:46:42.770637+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:32:49.764893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.887262+0000","last_change":"2024-09-19T04:44:30.759578+0000","last_active":"2024-09-19T04:46:41.887262+0000","last_peered":"2024-09-19T04:46:41.887262+0000","last_clean":"2024-09-19T04:46:41.887262+0000","last_became_active":"2024-09-19T04:44:30.759245+0000","last_became_peered":"2024-09-19T04:44:30.759245+0000","last_unstale":"2024-09-19T04:46:41.887262+0000","last_undegraded":"2024-09-19T04:46:41.887262+0000","last_fullsized":"2024-09-19T04:46:41.887262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:37.121770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033302+0000","last_change":"2024-09-19T04:44:28.593404+0000","last_active":"2024-09-19T04:46:42.033302+0000","last_peered":"2024-09-19T04:46:42.033302+0000","last_clean":"2024-09-19T04:46:42.033302+0000","last_became_active":"2024-09-19T04:44:28.593037+0000","last_became_peered":"2024-09-19T04:44:28.593037+0000","last_unstale":"2024-09-19T04:46:42.033302+0000","last_undegraded":"2024-09-19T04:46:42.033302+0000","last_fullsized":"2024-09-19T04:46:42.033302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:08:29.218543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.783007+0000","last_change":"2024-09-19T04:45:14.891783+0000","last_active":"2024-09-19T04:46:41.783007+0000","last_peered":"2024-09-19T04:46:41.783007+0000","last_clean":"2024-09-19T04:46:41.783007+0000","last_became_active":"2024-09-19T04:45:14.891396+0000","last_became_peered":"2024-09-19T04:45:14.891396+0000","last_unstale":"2024-09-19T04:46:41.783007+0000","last_undegraded":"2024-09-19T04:46:41.783007+0000","last_fullsized":"2024-09-19T04:46:41.783007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:56.443270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032529+0000","last_change":"2024-09-19T04:44:28.594541+0000","last_active":"2024-09-19T04:46:42.032529+0000","last_peered":"2024-09-19T04:46:42.032529+0000","last_clean":"2024-09-19T04:46:42.032529+0000","last_became_active":"2024-09-19T04:44:28.594262+0000","last_became_peered":"2024-09-19T04:44:28.594262+0000","last_unstale":"2024-09-19T04:46:42.032529+0000","last_undegraded":"2024-09-19T04:46:42.032529+0000","last_fullsized":"2024-09-19T04:46:42.032529+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:45.685034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785728+0000","last_change":"2024-09-19T04:44:30.757861+0000","last_active":"2024-09-19T04:46:41.785728+0000","last_peered":"2024-09-19T04:46:41.785728+0000","last_clean":"2024-09-19T04:46:41.785728+0000","last_became_active":"2024-09-19T04:44:30.757509+0000","last_became_peered":"2024-09-19T04:44:30.757509+0000","last_unstale":"2024-09-19T04:46:41.785728+0000","last_undegraded":"2024-09-19T04:46:41.785728+0000","last_fullsized":"2024-09-19T04:46:41.785728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:17.737055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886469+0000","last_change":"2024-09-19T04:44:28.335225+0000","last_active":"2024-09-19T04:46:41.886469+0000","last_peered":"2024-09-19T04:46:41.886469+0000","last_clean":"2024-09-19T04:46:41.886469+0000","last_became_active":"2024-09-19T04:44:28.334804+0000","last_became_peered":"2024-09-19T04:44:28.334804+0000","last_unstale":"2024-09-19T04:46:41.886469+0000","last_undegraded":"2024-09-19T04:46:41.886469+0000","last_fullsized":"2024-09-19T04:46:41.886469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:15.639418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.033015+0000","last_change":"2024-09-19T04:44:30.754018+0000","last_active":"2024-09-19T04:46:42.033015+0000","last_peered":"2024-09-19T04:46:42.033015+0000","last_clean":"2024-09-19T04:46:42.033015+0000","last_became_active":"2024-09-19T04:44:30.753639+0000","last_became_peered":"2024-09-19T04:44:30.753639+0000","last_unstale":"2024-09-19T04:46:42.033015+0000","last_undegraded":"2024-09-19T04:46:42.033015+0000","last_fullsized":"2024-09-19T04:46:42.033015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:05.012847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032760+0000","last_change":"2024-09-19T04:45:14.636102+0000","last_active":"2024-09-19T04:46:42.032760+0000","last_peered":"2024-09-19T04:46:42.032760+0000","last_clean":"2024-09-19T04:46:42.032760+0000","last_became_active":"2024-09-19T04:45:14.631201+0000","last_became_peered":"2024-09-19T04:45:14.631201+0000","last_unstale":"2024-09-19T04:46:42.032760+0000","last_undegraded":"2024-09-19T04:46:42.032760+0000","last_fullsized":"2024-09-19T04:46:42.032760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:04.331207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886472+0000","last_change":"2024-09-19T04:44:30.758543+0000","last_active":"2024-09-19T04:46:41.886472+0000","last_peered":"2024-09-19T04:46:41.886472+0000","last_clean":"2024-09-19T04:46:41.886472+0000","last_became_active":"2024-09-19T04:44:30.757228+0000","last_became_peered":"2024-09-19T04:44:30.757228+0000","last_unstale":"2024-09-19T04:46:41.886472+0000","last_undegraded":"2024-09-19T04:46:41.886472+0000","last_fullsized":"2024-09-19T04:46:41.886472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:13.877742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.785932+0000","last_change":"2024-09-19T04:44:28.339152+0000","last_active":"2024-09-19T04:46:41.785932+0000","last_peered":"2024-09-19T04:46:41.785932+0000","last_clean":"2024-09-19T04:46:41.785932+0000","last_became_active":"2024-09-19T04:44:28.338778+0000","last_became_peered":"2024-09-19T04:44:28.338778+0000","last_unstale":"2024-09-19T04:46:41.785932+0000","last_undegraded":"2024-09-19T04:46:41.785932+0000","last_fullsized":"2024-09-19T04:46:41.785932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:53.906755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782285+0000","last_change":"2024-09-19T04:44:30.759330+0000","last_active":"2024-09-19T04:46:41.782285+0000","last_peered":"2024-09-19T04:46:41.782285+0000","last_clean":"2024-09-19T04:46:41.782285+0000","last_became_active":"2024-09-19T04:44:30.758987+0000","last_became_peered":"2024-09-19T04:44:30.758987+0000","last_unstale":"2024-09-19T04:46:41.782285+0000","last_undegraded":"2024-09-19T04:46:41.782285+0000","last_fullsized":"2024-09-19T04:46:41.782285+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:10.105051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.895780+0000","last_change":"2024-09-19T04:44:30.748089+0000","last_active":"2024-09-19T04:46:41.895780+0000","last_peered":"2024-09-19T04:46:41.895780+0000","last_clean":"2024-09-19T04:46:41.895780+0000","last_became_active":"2024-09-19T04:44:30.747566+0000","last_became_peered":"2024-09-19T04:44:30.747566+0000","last_unstale":"2024-09-19T04:46:41.895780+0000","last_undegraded":"2024-09-19T04:46:41.895780+0000","last_fullsized":"2024-09-19T04:46:41.895780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:32.406745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.786105+0000","last_change":"2024-09-19T04:44:28.334217+0000","last_active":"2024-09-19T04:46:41.786105+0000","last_peered":"2024-09-19T04:46:41.786105+0000","last_clean":"2024-09-19T04:46:41.786105+0000","last_became_active":"2024-09-19T04:44:28.333768+0000","last_became_peered":"2024-09-19T04:44:28.333768+0000","last_unstale":"2024-09-19T04:46:41.786105+0000","last_undegraded":"2024-09-19T04:46:41.786105+0000","last_fullsized":"2024-09-19T04:46:41.786105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:52:56.626635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032821+0000","last_change":"2024-09-19T04:44:30.747498+0000","last_active":"2024-09-19T04:46:42.032821+0000","last_peered":"2024-09-19T04:46:42.032821+0000","last_clean":"2024-09-19T04:46:42.032821+0000","last_became_active":"2024-09-19T04:44:30.746822+0000","last_became_peered":"2024-09-19T04:44:30.746822+0000","last_unstale":"2024-09-19T04:46:42.032821+0000","last_undegraded":"2024-09-19T04:46:42.032821+0000","last_fullsized":"2024-09-19T04:46:42.032821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:09.460263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.032840+0000","last_change":"2024-09-19T04:45:14.645843+0000","last_active":"2024-09-19T04:46:42.032840+0000","last_peered":"2024-09-19T04:46:42.032840+0000","last_clean":"2024-09-19T04:46:42.032840+0000","last_became_active":"2024-09-19T04:45:14.632156+0000","last_became_peered":"2024-09-19T04:45:14.632156+0000","last_unstale":"2024-09-19T04:46:42.032840+0000","last_undegraded":"2024-09-19T04:46:42.032840+0000","last_fullsized":"2024-09-19T04:46:42.032840+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:34.406160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.782880+0000","last_change":"2024-09-19T04:44:30.762776+0000","last_active":"2024-09-19T04:46:41.782880+0000","last_peered":"2024-09-19T04:46:41.782880+0000","last_clean":"2024-09-19T04:46:41.782880+0000","last_became_active":"2024-09-19T04:44:30.761308+0000","last_became_peered":"2024-09-19T04:44:30.761308+0000","last_unstale":"2024-09-19T04:46:41.782880+0000","last_undegraded":"2024-09-19T04:46:41.782880+0000","last_fullsized":"2024-09-19T04:46:41.782880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:19.960125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.887257+0000","last_change":"2024-09-19T04:44:28.594858+0000","last_active":"2024-09-19T04:46:41.887257+0000","last_peered":"2024-09-19T04:46:41.887257+0000","last_clean":"2024-09-19T04:46:41.887257+0000","last_became_active":"2024-09-19T04:44:28.594515+0000","last_became_peered":"2024-09-19T04:44:28.594515+0000","last_unstale":"2024-09-19T04:46:41.887257+0000","last_undegraded":"2024-09-19T04:46:41.887257+0000","last_fullsized":"2024-09-19T04:46:41.887257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:53.728393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:41.886920+0000","last_change":"2024-09-19T04:44:30.753604+0000","last_active":"2024-09-19T04:46:41.886920+0000","last_peered":"2024-09-19T04:46:41.886920+0000","last_clean":"2024-09-19T04:46:41.886920+0000","last_became_active":"2024-09-19T04:44:30.751961+0000","last_became_peered":"2024-09-19T04:44:30.751961+0000","last_unstale":"2024-09-19T04:46:41.886920+0000","last_undegraded":"2024-09-19T04:46:41.886920+0000","last_fullsized":"2024-09-19T04:46:41.886920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:29.719202+0000","last_clean_scrub_stamp":"2024-09-19T04:44:29.719202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:46:58.759650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"87'14","reported_seq":160,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T04:46:42.010647+0000","last_change":"2024-09-19T04:44:28.339499+0000","last_active":"2024-09-19T04:46:42.010647+0000","last_peered":"2024-09-19T04:46:42.010647+0000","last_clean":"2024-09-19T04:46:42.010647+0000","last_became_active":"2024-09-19T04:44:28.338774+0000","last_became_peered":"2024-09-19T04:44:28.338774+0000","last_unstale":"2024-09-19T04:46:42.010647+0000","last_undegraded":"2024-09-19T04:46:42.010647+0000","last_fullsized":"2024-09-19T04:46:42.010647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:44:27.305965+0000","last_clean_scrub_stamp":"2024-09-19T04:44:27.305965+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:12.261237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599827,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1313,"num_read_kb":763,"num_write":372,"num_write_kb":1461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1799874,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108024,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481833,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":954368,"data_stored":464859,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710765,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":1254505,"data_compressed":45058,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874290,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":858005,"data_compressed":22040,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":465167,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364860,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1028096,"data_stored":1057828,"data_compressed":38902,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528387,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":962560,"data_stored":465292,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1189358,"data_compressed":45058,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953548,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":464901,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1024000,"data_stored":989147,"data_compressed":28195,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1052672,"data_stored":532908,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":532688,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":466238,"data_compressed":2894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135206,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200353,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:46:47.922 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:46:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:47 smithi193 ceph-mon[35592]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:46:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:47 smithi193 ceph-mon[35592]: Updating smithi183:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:46:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:47 smithi193 ceph-mon[35592]: Updating smithi193:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:46:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:47 smithi193 ceph-mon[35592]: Updating smithi123:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:46:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:47 smithi193 ceph-mon[35592]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:46:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:47 smithi193 ceph-mon[35592]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:46:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:47 smithi193 ceph-mon[35592]: Updating smithi183:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:46:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:47 smithi193 ceph-mon[35592]: Updating smithi193:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:46:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:47 smithi193 ceph-mon[35592]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-19T04:46:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:47 smithi193 ceph-mon[35592]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:46:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:47 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:48.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:47 smithi183 ceph-mon[35333]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:46:48.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:47 smithi183 ceph-mon[35333]: Updating smithi183:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:46:48.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:47 smithi183 ceph-mon[35333]: Updating smithi193:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:46:48.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:47 smithi183 ceph-mon[35333]: Updating smithi123:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:46:48.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:47 smithi183 ceph-mon[35333]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:46:48.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:47 smithi183 ceph-mon[35333]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:46:48.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:47 smithi183 ceph-mon[35333]: Updating smithi183:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:46:48.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:47 smithi183 ceph-mon[35333]: Updating smithi193:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:46:48.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:47 smithi183 ceph-mon[35333]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-19T04:46:48.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:47 smithi183 ceph-mon[35333]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:46:48.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:47 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:48.327 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:48.890 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:48 smithi123 ceph-mon[30511]: from='client.25240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:46:48.890 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:48 smithi123 ceph-mon[30511]: Updating smithi123:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:46:48.890 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:48 smithi123 ceph-mon[30511]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:46:48.890 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:48 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:48.890 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:48 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:48.890 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:48 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:46:48.890 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:48 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:46:48.890 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:48 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:46:48.890 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:48 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:48.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.923+0000 7fbcee800640 1 -- 172.21.15.123:0/3080957922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbce8077040 msgr2=0x7fbce8112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:48.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.923+0000 7fbcee800640 1 --2- 172.21.15.123:0/3080957922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbce8077040 0x7fbce8112330 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009900 tx=0x7fbcd402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:46:48.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.923+0000 7fbcee800640 1 -- 172.21.15.123:0/3080957922 shutdown_connections 2024-09-19T04:46:48.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.923+0000 7fbcee800640 1 --2- 172.21.15.123:0/3080957922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbce8112870 0x7fbce8114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:48.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.923+0000 7fbcee800640 1 --2- 172.21.15.123:0/3080957922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbce8077040 0x7fbce8112330 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:48.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.923+0000 7fbcee800640 1 --2- 172.21.15.123:0/3080957922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbce8076720 0x7fbce8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:48.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.923+0000 7fbcee800640 1 -- 172.21.15.123:0/3080957922 >> 172.21.15.123:0/3080957922 conn(0x7fbce80ff8d0 msgr2=0x7fbce8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:48.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.923+0000 7fbcee800640 1 -- 172.21.15.123:0/3080957922 shutdown_connections 2024-09-19T04:46:48.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.923+0000 7fbcee800640 1 -- 172.21.15.123:0/3080957922 wait complete. 2024-09-19T04:46:48.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.924+0000 7fbcee800640 1 Processor -- start 2024-09-19T04:46:48.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.924+0000 7fbcee800640 1 -- start start 2024-09-19T04:46:48.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.925+0000 7fbcee800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbce8076720 0x7fbce81a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:48.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.925+0000 7fbcee800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbce8077040 0x7fbce81a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:48.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.925+0000 7fbcee800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbce8112870 0x7fbce819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:48.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.925+0000 7fbcee800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbce8118430 con 0x7fbce8112870 2024-09-19T04:46:48.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.925+0000 7fbcee800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbce81182b0 con 0x7fbce8076720 2024-09-19T04:46:48.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.925+0000 7fbcee800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbce81185b0 con 0x7fbce8077040 2024-09-19T04:46:48.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.925+0000 7fbcede00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbce8112870 0x7fbce819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:48.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.925+0000 7fbceca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbce8077040 0x7fbce81a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:48.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.925+0000 7fbced400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbce8076720 0x7fbce81a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:48.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.925+0000 7fbcede00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbce8112870 0x7fbce819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50700/0 (socket says 172.21.15.123:50700) 2024-09-19T04:46:48.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.925+0000 7fbcede00640 1 -- 172.21.15.123:0/2121864640 learned_addr learned my addr 172.21.15.123:0/2121864640 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:48.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.926+0000 7fbcede00640 1 -- 172.21.15.123:0/2121864640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbce8077040 msgr2=0x7fbce81a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:48.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.926+0000 7fbcede00640 1 --2- 172.21.15.123:0/2121864640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbce8077040 0x7fbce81a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:48.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.926+0000 7fbcede00640 1 -- 172.21.15.123:0/2121864640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbce8076720 msgr2=0x7fbce81a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:48.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.926+0000 7fbcede00640 1 --2- 172.21.15.123:0/2121864640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbce8076720 0x7fbce81a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:48.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.926+0000 7fbcede00640 1 -- 172.21.15.123:0/2121864640 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce819efc0 con 0x7fbce8112870 2024-09-19T04:46:48.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.926+0000 7fbced400640 1 --2- 172.21.15.123:0/2121864640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbce8076720 0x7fbce81a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:48.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.926+0000 7fbcede00640 1 --2- 172.21.15.123:0/2121864640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbce8112870 0x7fbce819e750 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbce000eb10 tx=0x7fbce000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:48.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.926+0000 7fbce6000640 1 -- 172.21.15.123:0/2121864640 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0004b20 con 0x7fbce8112870 2024-09-19T04:46:48.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.926+0000 7fbce6000640 1 -- 172.21.15.123:0/2121864640 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbce0004cc0 con 0x7fbce8112870 2024-09-19T04:46:48.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.926+0000 7fbce6000640 1 -- 172.21.15.123:0/2121864640 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0010790 con 0x7fbce8112870 2024-09-19T04:46:48.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.926+0000 7fbcee800640 1 -- 172.21.15.123:0/2121864640 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce819f2b0 con 0x7fbce8112870 2024-09-19T04:46:48.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.926+0000 7fbcee800640 1 -- 172.21.15.123:0/2121864640 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce81ab600 con 0x7fbce8112870 2024-09-19T04:46:48.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.927+0000 7fbcee800640 1 -- 172.21.15.123:0/2121864640 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce8077e60 con 0x7fbce8112870 2024-09-19T04:46:48.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.929+0000 7fbce6000640 1 -- 172.21.15.123:0/2121864640 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 138583+0+0 (secure 0 0 0) 0x7fbce0005020 con 0x7fbce8112870 2024-09-19T04:46:48.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.931+0000 7fbce6000640 1 --2- 172.21.15.123:0/2121864640 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbcd00a3550 0x7fbcd00a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:48.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.931+0000 7fbce6000640 1 -- 172.21.15.123:0/2121864640 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fbce00d0680 con 0x7fbce8112870 2024-09-19T04:46:48.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.931+0000 7fbced400640 1 --2- 172.21.15.123:0/2121864640 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbcd00a3550 0x7fbcd00a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:48.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.932+0000 7fbced400640 1 --2- 172.21.15.123:0/2121864640 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbcd00a3550 0x7fbcd00a5a10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc009fc0 tx=0x7fbcdc005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:48.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:48.935+0000 7fbce6000640 1 -- 172.21.15.123:0/2121864640 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbce009cde0 con 0x7fbce8112870 2024-09-19T04:46:48.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:48 smithi193 ceph-mon[35592]: from='client.25240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:46:48.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:48 smithi193 ceph-mon[35592]: Updating smithi123:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:46:48.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:48 smithi193 ceph-mon[35592]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:46:48.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:48 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:48.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:48 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:48.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:48 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:46:48.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:48 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:46:48.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:48 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:46:48.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:48 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:49.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:48 smithi183 ceph-mon[35333]: from='client.25240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:46:49.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:48 smithi183 ceph-mon[35333]: Updating smithi123:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:46:49.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:48 smithi183 ceph-mon[35333]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:46:49.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:48 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:49.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:48 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:49.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:48 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:46:49.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:48 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:46:49.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:48 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:46:49.101 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:48 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:49.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:49.148+0000 7fbcee800640 1 -- 172.21.15.123:0/2121864640 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbce810da00 con 0x7fbce8112870 2024-09-19T04:46:49.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:49.149+0000 7fbce6000640 1 -- 172.21.15.123:0/2121864640 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11990 (secure 0 0 0) 0x7fbce009c6e0 con 0x7fbce8112870 2024-09-19T04:46:49.151 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:46:49.151 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"btime":"2024-09-19T04:46:42:466383+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:46:41.577049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24499,24500,24539,24505,24529]},"id":1}]} 2024-09-19T04:46:49.151 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 28 2024-09-19T04:46:49.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:49.151+0000 7fbcee800640 1 -- 172.21.15.123:0/2121864640 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbcd00a3550 msgr2=0x7fbcd00a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:49.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:49.151+0000 7fbcee800640 1 --2- 172.21.15.123:0/2121864640 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbcd00a3550 0x7fbcd00a5a10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc009fc0 tx=0x7fbcdc005f90 comp rx=0 tx=0).stop 2024-09-19T04:46:49.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:49.151+0000 7fbcee800640 1 -- 172.21.15.123:0/2121864640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbce8112870 msgr2=0x7fbce819e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:49.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:49.151+0000 7fbcee800640 1 --2- 172.21.15.123:0/2121864640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbce8112870 0x7fbce819e750 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbce000eb10 tx=0x7fbce000c2d0 comp rx=0 tx=0).stop 2024-09-19T04:46:49.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:49.152+0000 7fbcee800640 1 -- 172.21.15.123:0/2121864640 shutdown_connections 2024-09-19T04:46:49.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:49.152+0000 7fbcee800640 1 --2- 172.21.15.123:0/2121864640 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbcd00a3550 0x7fbcd00a5a10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:49.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:49.152+0000 7fbcee800640 1 --2- 172.21.15.123:0/2121864640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbce8112870 0x7fbce819e750 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:49.152+0000 7fbcee800640 1 --2- 172.21.15.123:0/2121864640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbce8077040 0x7fbce81a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:49.152+0000 7fbcee800640 1 --2- 172.21.15.123:0/2121864640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbce8076720 0x7fbce81a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:49.152+0000 7fbcee800640 1 -- 172.21.15.123:0/2121864640 >> 172.21.15.123:0/2121864640 conn(0x7fbce80ff8d0 msgr2=0x7fbce8101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:49.152+0000 7fbcee800640 1 -- 172.21.15.123:0/2121864640 shutdown_connections 2024-09-19T04:46:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:49.152+0000 7fbcee800640 1 -- 172.21.15.123:0/2121864640 wait complete. 2024-09-19T04:46:49.913 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:49 smithi123 ceph-mon[30511]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:46:49.913 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:49 smithi123 ceph-mon[30511]: from='client.25247 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:46:49.914 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:49 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2121864640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:46:49.950 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:46:49.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:49 smithi193 ceph-mon[35592]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:46:49.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:49 smithi193 ceph-mon[35592]: from='client.25247 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:46:49.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:49 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2121864640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:46:50.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:49 smithi183 ceph-mon[35333]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:46:50.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:49 smithi183 ceph-mon[35333]: from='client.25247 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:46:50.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:49 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2121864640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:46:50.316 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:50.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.905+0000 7f35d2600640 1 -- 172.21.15.123:0/2640006901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35cc1079c0 msgr2=0x7f35cc107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:50.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.905+0000 7f35d2600640 1 --2- 172.21.15.123:0/2640006901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35cc1079c0 0x7f35cc107da0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009900 tx=0x7f35c002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:46:50.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.905+0000 7f35d2600640 1 -- 172.21.15.123:0/2640006901 shutdown_connections 2024-09-19T04:46:50.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.905+0000 7f35d2600640 1 --2- 172.21.15.123:0/2640006901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc102360 0x7f35cc10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:50.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.905+0000 7f35d2600640 1 --2- 172.21.15.123:0/2640006901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35cc1019c0 0x7f35cc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:50.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.905+0000 7f35d2600640 1 --2- 172.21.15.123:0/2640006901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35cc1079c0 0x7f35cc107da0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:50.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.905+0000 7f35d2600640 1 -- 172.21.15.123:0/2640006901 >> 172.21.15.123:0/2640006901 conn(0x7f35cc0fd740 msgr2=0x7f35cc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:50.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.905+0000 7f35d2600640 1 -- 172.21.15.123:0/2640006901 shutdown_connections 2024-09-19T04:46:50.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.905+0000 7f35d2600640 1 -- 172.21.15.123:0/2640006901 wait complete. 2024-09-19T04:46:50.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.906+0000 7f35d2600640 1 Processor -- start 2024-09-19T04:46:50.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.906+0000 7f35d2600640 1 -- start start 2024-09-19T04:46:50.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.907+0000 7f35d2600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35cc1019c0 0x7f35cc19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:50.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.907+0000 7f35d2600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35cc102360 0x7f35cc19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:50.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.907+0000 7f35d2600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc1079c0 0x7f35cc19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:50.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.907+0000 7f35d2600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc111dc0 con 0x7f35cc1079c0 2024-09-19T04:46:50.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.907+0000 7f35d2600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc111c40 con 0x7f35cc1019c0 2024-09-19T04:46:50.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.907+0000 7f35d2600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc111f40 con 0x7f35cc102360 2024-09-19T04:46:50.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.907+0000 7f35d1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35cc1019c0 0x7f35cc19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:50.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.907+0000 7f35d1c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc1079c0 0x7f35cc19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:50.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.907+0000 7f35d1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35cc1019c0 0x7f35cc19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:34634/0 (socket says 172.21.15.123:34634) 2024-09-19T04:46:50.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.907+0000 7f35d1200640 1 -- 172.21.15.123:0/2010239699 learned_addr learned my addr 172.21.15.123:0/2010239699 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:50.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.907+0000 7f35cbe00640 1 --2- 172.21.15.123:0/2010239699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35cc102360 0x7f35cc19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:50.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.908+0000 7f35d1200640 1 -- 172.21.15.123:0/2010239699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35cc102360 msgr2=0x7f35cc19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:50.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.908+0000 7f35d1200640 1 --2- 172.21.15.123:0/2010239699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35cc102360 0x7f35cc19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:50.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.908+0000 7f35d1200640 1 -- 172.21.15.123:0/2010239699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc1079c0 msgr2=0x7f35cc19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:50.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.908+0000 7f35d1200640 1 --2- 172.21.15.123:0/2010239699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc1079c0 0x7f35cc19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:50.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.908+0000 7f35d1200640 1 -- 172.21.15.123:0/2010239699 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc19f450 con 0x7f35cc1019c0 2024-09-19T04:46:50.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.908+0000 7f35d1200640 1 --2- 172.21.15.123:0/2010239699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35cc1019c0 0x7f35cc19a4a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f35c0006010 tx=0x7f35c0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:50.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.908+0000 7f35c9600640 1 -- 172.21.15.123:0/2010239699 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c003f070 con 0x7f35cc1019c0 2024-09-19T04:46:50.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.908+0000 7f35d2600640 1 -- 172.21.15.123:0/2010239699 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35cc19f6e0 con 0x7f35cc1019c0 2024-09-19T04:46:50.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.908+0000 7f35c9600640 1 -- 172.21.15.123:0/2010239699 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c0031c30 con 0x7f35cc1019c0 2024-09-19T04:46:50.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.908+0000 7f35c9600640 1 -- 172.21.15.123:0/2010239699 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c0035030 con 0x7f35cc1019c0 2024-09-19T04:46:50.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.909+0000 7f35d2600640 1 -- 172.21.15.123:0/2010239699 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35cc1a7020 con 0x7f35cc1019c0 2024-09-19T04:46:50.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.909+0000 7f35d2600640 1 -- 172.21.15.123:0/2010239699 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35cc103100 con 0x7f35cc1019c0 2024-09-19T04:46:50.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.912+0000 7f35c9600640 1 -- 172.21.15.123:0/2010239699 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 138583+0+0 (secure 0 0 0) 0x7f35c0042070 con 0x7f35cc1019c0 2024-09-19T04:46:50.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.913+0000 7f35c9600640 1 --2- 172.21.15.123:0/2010239699 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f35b40a34d0 0x7f35b40a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:50.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.913+0000 7f35c9600640 1 -- 172.21.15.123:0/2010239699 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f35c00f0b20 con 0x7f35cc1019c0 2024-09-19T04:46:50.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.913+0000 7f35cbe00640 1 --2- 172.21.15.123:0/2010239699 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f35b40a34d0 0x7f35b40a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:50.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.914+0000 7f35cbe00640 1 --2- 172.21.15.123:0/2010239699 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f35b40a34d0 0x7f35b40a5990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f35cc10ed20 tx=0x7f35b8005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:50.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:50.915+0000 7f35c9600640 1 -- 172.21.15.123:0/2010239699 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f35c00bd350 con 0x7f35cc1019c0 2024-09-19T04:46:51.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:51.092+0000 7f35d2600640 1 -- 172.21.15.123:0/2010239699 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35cc1a71f0 con 0x7f35cc1019c0 2024-09-19T04:46:51.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:51.093+0000 7f35c9600640 1 -- 172.21.15.123:0/2010239699 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+23178 (secure 0 0 0) 0x7f35c0031de0 con 0x7f35cc1019c0 2024-09-19T04:46:51.095 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:46:51.096 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":87,"fsid":"911fa052-7640-11ef-bceb-c7b262605968","created":"2024-09-19T04:36:21.083333+0000","modified":"2024-09-19T04:46:41.775142+0000","last_up_change":"2024-09-19T04:43:33.818439+0000","last_in_change":"2024-09-19T04:43:10.148272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:39:33.856981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:44:26.956483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:44:29.718625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6803","nonce":3631596501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6805","nonce":3631596501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6809","nonce":3631596501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6807","nonce":3631596501}]},"public_addr":"172.21.15.123:6803/3631596501","cluster_addr":"172.21.15.123:6805/3631596501","heartbeat_back_addr":"172.21.15.123:6809/3631596501","heartbeat_front_addr":"172.21.15.123:6807/3631596501","state":["exists","up"]},{"osd":1,"uuid":"a526d4ce-586b-4d66-b720-e868df92fa24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6801","nonce":1259721614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6803","nonce":1259721614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6807","nonce":1259721614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6805","nonce":1259721614}]},"public_addr":"172.21.15.183:6801/1259721614","cluster_addr":"172.21.15.183:6803/1259721614","heartbeat_back_addr":"172.21.15.183:6807/1259721614","heartbeat_front_addr":"172.21.15.183:6805/1259721614","state":["exists","up"]},{"osd":2,"uuid":"3650601a-6259-4b2a-97df-1d47b09b0127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6801","nonce":2708151726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6803","nonce":2708151726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6807","nonce":2708151726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6805","nonce":2708151726}]},"public_addr":"172.21.15.193:6801/2708151726","cluster_addr":"172.21.15.193:6803/2708151726","heartbeat_back_addr":"172.21.15.193:6807/2708151726","heartbeat_front_addr":"172.21.15.193:6805/2708151726","state":["exists","up"]},{"osd":3,"uuid":"67c485ad-136e-4020-950d-9171a3c2de35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6811","nonce":3214806282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6813","nonce":3214806282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6817","nonce":3214806282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6815","nonce":3214806282}]},"public_addr":"172.21.15.123:6811/3214806282","cluster_addr":"172.21.15.123:6813/3214806282","heartbeat_back_addr":"172.21.15.123:6817/3214806282","heartbeat_front_addr":"172.21.15.123:6815/3214806282","state":["exists","up"]},{"osd":4,"uuid":"c9098540-06ab-45a8-9cf9-f90519cc852e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6809","nonce":667618665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6811","nonce":667618665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6815","nonce":667618665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6813","nonce":667618665}]},"public_addr":"172.21.15.183:6809/667618665","cluster_addr":"172.21.15.183:6811/667618665","heartbeat_back_addr":"172.21.15.183:6815/667618665","heartbeat_front_addr":"172.21.15.183:6813/667618665","state":["exists","up"]},{"osd":5,"uuid":"278a8dbd-6492-42d4-b489-1f832350bc6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6809","nonce":2292251639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6811","nonce":2292251639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6815","nonce":2292251639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6813","nonce":2292251639}]},"public_addr":"172.21.15.193:6809/2292251639","cluster_addr":"172.21.15.193:6811/2292251639","heartbeat_back_addr":"172.21.15.193:6815/2292251639","heartbeat_front_addr":"172.21.15.193:6813/2292251639","state":["exists","up"]},{"osd":6,"uuid":"0fda4e39-6adb-4941-a39b-a9a15019088c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6819","nonce":226007499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6821","nonce":226007499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6825","nonce":226007499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6823","nonce":226007499}]},"public_addr":"172.21.15.123:6819/226007499","cluster_addr":"172.21.15.123:6821/226007499","heartbeat_back_addr":"172.21.15.123:6825/226007499","heartbeat_front_addr":"172.21.15.123:6823/226007499","state":["exists","up"]},{"osd":7,"uuid":"5c0a021e-9f3b-4766-9686-0ef24b68cf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6817","nonce":3718518226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6819","nonce":3718518226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6823","nonce":3718518226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6821","nonce":3718518226}]},"public_addr":"172.21.15.183:6817/3718518226","cluster_addr":"172.21.15.183:6819/3718518226","heartbeat_back_addr":"172.21.15.183:6823/3718518226","heartbeat_front_addr":"172.21.15.183:6821/3718518226","state":["exists","up"]},{"osd":8,"uuid":"9b0e67e0-6305-4f81-9211-d41960099a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6817","nonce":3551404118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6819","nonce":3551404118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6823","nonce":3551404118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6821","nonce":3551404118}]},"public_addr":"172.21.15.193:6817/3551404118","cluster_addr":"172.21.15.193:6819/3551404118","heartbeat_back_addr":"172.21.15.193:6823/3551404118","heartbeat_front_addr":"172.21.15.193:6821/3551404118","state":["exists","up"]},{"osd":9,"uuid":"978ef3fa-c6ec-421f-b194-56d44813be5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6827","nonce":1439901149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6829","nonce":1439901149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6833","nonce":1439901149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6831","nonce":1439901149}]},"public_addr":"172.21.15.123:6827/1439901149","cluster_addr":"172.21.15.123:6829/1439901149","heartbeat_back_addr":"172.21.15.123:6833/1439901149","heartbeat_front_addr":"172.21.15.123:6831/1439901149","state":["exists","up"]},{"osd":10,"uuid":"5495e09b-7809-47b0-b7e0-b56c452f9f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6825","nonce":3355349245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6827","nonce":3355349245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6831","nonce":3355349245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6829","nonce":3355349245}]},"public_addr":"172.21.15.183:6825/3355349245","cluster_addr":"172.21.15.183:6827/3355349245","heartbeat_back_addr":"172.21.15.183:6831/3355349245","heartbeat_front_addr":"172.21.15.183:6829/3355349245","state":["exists","up"]},{"osd":11,"uuid":"03046fef-b16b-42f8-9a19-6c0b78cbfe7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6825","nonce":4291617038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6827","nonce":4291617038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6831","nonce":4291617038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6829","nonce":4291617038}]},"public_addr":"172.21.15.193:6825/4291617038","cluster_addr":"172.21.15.193:6827/4291617038","heartbeat_back_addr":"172.21.15.193:6831/4291617038","heartbeat_front_addr":"172.21.15.193:6829/4291617038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:38:39.142529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:05.336911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:30.310967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:59.655973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:25.890576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:50.778689+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:19.063731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:45.424269+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:10.562149+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:40.382368+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:06.297731+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:32.352172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/1532622003":"2024-09-20T04:46:41.775112+0000","172.21.15.123:0/2833919666":"2024-09-20T04:46:41.775112+0000","172.21.15.123:0/2580747029":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/471028951":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:6800/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6800/1532622003":"2024-09-20T04:46:41.775112+0000","172.21.15.123:6801/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/3498196650":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/4040401634":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/2000593686":"2024-09-20T04:36:45.913060+0000","172.21.15.123:6800/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/3339335756":"2024-09-20T04:46:41.775112+0000","172.21.15.123:0/1009496020":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/3244738342":"2024-09-20T04:46:41.775112+0000","172.21.15.123:0/2764863023":"2024-09-20T04:46:41.775112+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:46:51.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:51.096+0000 7f35d2600640 1 -- 172.21.15.123:0/2010239699 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f35b40a34d0 msgr2=0x7f35b40a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:51.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:51.096+0000 7f35d2600640 1 --2- 172.21.15.123:0/2010239699 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f35b40a34d0 0x7f35b40a5990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f35cc10ed20 tx=0x7f35b8005f90 comp rx=0 tx=0).stop 2024-09-19T04:46:51.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:51.096+0000 7f35d2600640 1 -- 172.21.15.123:0/2010239699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35cc1019c0 msgr2=0x7f35cc19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:51.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:51.096+0000 7f35d2600640 1 --2- 172.21.15.123:0/2010239699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35cc1019c0 0x7f35cc19a4a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f35c0006010 tx=0x7f35c0003c10 comp rx=0 tx=0).stop 2024-09-19T04:46:51.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:51.096+0000 7f35d2600640 1 -- 172.21.15.123:0/2010239699 shutdown_connections 2024-09-19T04:46:51.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:51.096+0000 7f35d2600640 1 --2- 172.21.15.123:0/2010239699 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f35b40a34d0 0x7f35b40a5990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:51.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:51.096+0000 7f35d2600640 1 --2- 172.21.15.123:0/2010239699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35cc1079c0 0x7f35cc19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:51.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:51.096+0000 7f35d2600640 1 --2- 172.21.15.123:0/2010239699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35cc102360 0x7f35cc19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:51.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:51.096+0000 7f35d2600640 1 --2- 172.21.15.123:0/2010239699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35cc1019c0 0x7f35cc19a4a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:51.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:51.096+0000 7f35d2600640 1 -- 172.21.15.123:0/2010239699 >> 172.21.15.123:0/2010239699 conn(0x7f35cc0fd740 msgr2=0x7f35cc0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:51.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:51.096+0000 7f35d2600640 1 -- 172.21.15.123:0/2010239699 shutdown_connections 2024-09-19T04:46:51.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:51.096+0000 7f35d2600640 1 -- 172.21.15.123:0/2010239699 wait complete. 2024-09-19T04:46:51.861 DEBUG:tasks.fwd_scrub:Yielding 2024-09-19T04:46:51.861 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T04:46:51.871 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T04:46:51.871 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T04:46:51.871 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-19T04:46:51.872 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-19T04:46:51.872 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:46:51.872 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:46:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:51 smithi193 ceph-mon[35592]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.3 KiB/s wr, 26 op/s 2024-09-19T04:46:51.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:51 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2010239699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:46:52.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.028+0000 7f725ea00640 1 -- 172.21.15.123:0/2896257898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72581018b0 msgr2=0x7f7258101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.029+0000 7f725ea00640 1 --2- 172.21.15.123:0/2896257898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72581018b0 0x7f7258101d10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7244009930 tx=0x7f724402a420 comp rx=0 tx=0).stop 2024-09-19T04:46:52.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.029+0000 7f725ea00640 1 -- 172.21.15.123:0/2896257898 shutdown_connections 2024-09-19T04:46:52.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.029+0000 7f725ea00640 1 --2- 172.21.15.123:0/2896257898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7258102250 0x7f725810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.029+0000 7f725ea00640 1 --2- 172.21.15.123:0/2896257898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72581018b0 0x7f7258101d10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.029+0000 7f725ea00640 1 --2- 172.21.15.123:0/2896257898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72581078c0 0x7f7258107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.029+0000 7f725ea00640 1 -- 172.21.15.123:0/2896257898 >> 172.21.15.123:0/2896257898 conn(0x7f72580fd5d0 msgr2=0x7f72580ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:52.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.029+0000 7f725ea00640 1 -- 172.21.15.123:0/2896257898 shutdown_connections 2024-09-19T04:46:52.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.029+0000 7f725ea00640 1 -- 172.21.15.123:0/2896257898 wait complete. 2024-09-19T04:46:52.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.030+0000 7f725ea00640 1 Processor -- start 2024-09-19T04:46:52.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.030+0000 7f725ea00640 1 -- start start 2024-09-19T04:46:52.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.030+0000 7f725ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72581018b0 0x7f725819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.030+0000 7f725ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7258102250 0x7f725819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.030+0000 7f725ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72581078c0 0x7f725819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.030+0000 7f725ea00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7258111d00 con 0x7f72581078c0 2024-09-19T04:46:52.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.030+0000 7f725ea00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7258111b80 con 0x7f7258102250 2024-09-19T04:46:52.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.030+0000 7f725ea00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7258111e80 con 0x7f72581018b0 2024-09-19T04:46:52.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.030+0000 7f725e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72581078c0 0x7f725819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.030+0000 7f725cc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7258102250 0x7f725819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.030+0000 7f725cc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7258102250 0x7f725819a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:35512/0 (socket says 172.21.15.123:35512) 2024-09-19T04:46:52.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.030+0000 7f725cc00640 1 -- 172.21.15.123:0/3435383657 learned_addr learned my addr 172.21.15.123:0/3435383657 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:52.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.030+0000 7f725d600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72581018b0 0x7f725819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.030+0000 7f725e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72581078c0 0x7f725819ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60360/0 (socket says 172.21.15.123:60360) 2024-09-19T04:46:52.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.031+0000 7f725e000640 1 -- 172.21.15.123:0/3435383657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72581018b0 msgr2=0x7f725819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.031+0000 7f725e000640 1 --2- 172.21.15.123:0/3435383657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72581018b0 0x7f725819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.031+0000 7f725e000640 1 -- 172.21.15.123:0/3435383657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7258102250 msgr2=0x7f725819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.031+0000 7f725e000640 1 --2- 172.21.15.123:0/3435383657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7258102250 0x7f725819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.031+0000 7f725e000640 1 -- 172.21.15.123:0/3435383657 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725819f450 con 0x7f72581078c0 2024-09-19T04:46:52.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.031+0000 7f725e000640 1 --2- 172.21.15.123:0/3435383657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72581078c0 0x7f725819ed20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f725000e970 tx=0x7f725000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:52.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.031+0000 7f7256000640 1 -- 172.21.15.123:0/3435383657 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725000ccc0 con 0x7f72581078c0 2024-09-19T04:46:52.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.031+0000 7f7256000640 1 -- 172.21.15.123:0/3435383657 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7250005020 con 0x7f72581078c0 2024-09-19T04:46:52.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.031+0000 7f725ea00640 1 -- 172.21.15.123:0/3435383657 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725819f740 con 0x7f72581078c0 2024-09-19T04:46:52.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.031+0000 7f7256000640 1 -- 172.21.15.123:0/3435383657 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250004a00 con 0x7f72581078c0 2024-09-19T04:46:52.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.032+0000 7f725ea00640 1 -- 172.21.15.123:0/3435383657 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72581a6fd0 con 0x7f72581078c0 2024-09-19T04:46:52.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.033+0000 7f725ea00640 1 -- 172.21.15.123:0/3435383657 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7258101d10 con 0x7f72581078c0 2024-09-19T04:46:52.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.034+0000 7f7256000640 1 -- 172.21.15.123:0/3435383657 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 138583+0+0 (secure 0 0 0) 0x7f72500051d0 con 0x7f72581078c0 2024-09-19T04:46:52.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.036+0000 7f7256000640 1 --2- 172.21.15.123:0/3435383657 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f72400a34d0 0x7f72400a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.036+0000 7f7256000640 1 -- 172.21.15.123:0/3435383657 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f72500cff30 con 0x7f72581078c0 2024-09-19T04:46:52.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.036+0000 7f725d600640 1 --2- 172.21.15.123:0/3435383657 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f72400a34d0 0x7f72400a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.037+0000 7f725d600640 1 --2- 172.21.15.123:0/3435383657 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f72400a34d0 0x7f72400a5990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f724c005fd0 tx=0x7f724c009760 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:52.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.040+0000 7f7256000640 1 -- 172.21.15.123:0/3435383657 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f725009c690 con 0x7f72581078c0 2024-09-19T04:46:52.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:51 smithi183 ceph-mon[35333]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.3 KiB/s wr, 26 op/s 2024-09-19T04:46:52.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:51 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2010239699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:46:52.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:51 smithi123 ceph-mon[30511]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.3 KiB/s wr, 26 op/s 2024-09-19T04:46:52.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:51 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2010239699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:46:52.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.229+0000 7f725ea00640 1 -- 172.21.15.123:0/3435383657 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f72581a7310 con 0x7f72581078c0 2024-09-19T04:46:52.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.234+0000 7f7256000640 1 -- 172.21.15.123:0/3435383657 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v28)=0 v28) ==== 129+0+0 (secure 0 0 0) 0x7f725009bf90 con 0x7f72581078c0 2024-09-19T04:46:52.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.236+0000 7f725ea00640 1 -- 172.21.15.123:0/3435383657 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f72400a34d0 msgr2=0x7f72400a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.236+0000 7f725ea00640 1 --2- 172.21.15.123:0/3435383657 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f72400a34d0 0x7f72400a5990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f724c005fd0 tx=0x7f724c009760 comp rx=0 tx=0).stop 2024-09-19T04:46:52.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.236+0000 7f725ea00640 1 -- 172.21.15.123:0/3435383657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72581078c0 msgr2=0x7f725819ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.236+0000 7f725ea00640 1 --2- 172.21.15.123:0/3435383657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72581078c0 0x7f725819ed20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f725000e970 tx=0x7f725000ee30 comp rx=0 tx=0).stop 2024-09-19T04:46:52.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.236+0000 7f725ea00640 1 -- 172.21.15.123:0/3435383657 shutdown_connections 2024-09-19T04:46:52.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.236+0000 7f725ea00640 1 --2- 172.21.15.123:0/3435383657 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f72400a34d0 0x7f72400a5990 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f724c005fd0 tx=0x7f724c009760 comp rx=0 tx=0).stop 2024-09-19T04:46:52.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.236+0000 7f725ea00640 1 --2- 172.21.15.123:0/3435383657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72581078c0 0x7f725819ed20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.236+0000 7f725ea00640 1 --2- 172.21.15.123:0/3435383657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7258102250 0x7f725819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.236+0000 7f725ea00640 1 --2- 172.21.15.123:0/3435383657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72581018b0 0x7f725819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.236+0000 7f725ea00640 1 -- 172.21.15.123:0/3435383657 >> 172.21.15.123:0/3435383657 conn(0x7f72580fd5d0 msgr2=0x7f725810cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:52.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.237+0000 7f725ea00640 1 -- 172.21.15.123:0/3435383657 shutdown_connections 2024-09-19T04:46:52.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.237+0000 7f725ea00640 1 -- 172.21.15.123:0/3435383657 wait complete. 2024-09-19T04:46:52.258 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-19T04:46:52.262 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.419+0000 7f7def200640 1 -- 172.21.15.123:0/2321810619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7df01064c0 msgr2=0x7f7df0100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.419+0000 7f7def200640 1 --2- 172.21.15.123:0/2321810619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7df01064c0 0x7f7df0100040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009900 tx=0x7f7ddc02a410 comp rx=0 tx=0).stop 2024-09-19T04:46:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.419+0000 7f7def200640 1 -- 172.21.15.123:0/2321810619 shutdown_connections 2024-09-19T04:46:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.419+0000 7f7def200640 1 --2- 172.21.15.123:0/2321810619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7df010a9c0 0x7f7df010cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.419+0000 7f7def200640 1 --2- 172.21.15.123:0/2321810619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7df01064c0 0x7f7df0100040 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.419+0000 7f7def200640 1 --2- 172.21.15.123:0/2321810619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df0068bc0 0x7f7df00ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.419+0000 7f7def200640 1 -- 172.21.15.123:0/2321810619 >> 172.21.15.123:0/2321810619 conn(0x7f7df00fb820 msgr2=0x7f7df00fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.419+0000 7f7def200640 1 -- 172.21.15.123:0/2321810619 shutdown_connections 2024-09-19T04:46:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.419+0000 7f7def200640 1 -- 172.21.15.123:0/2321810619 wait complete. 2024-09-19T04:46:52.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.420+0000 7f7def200640 1 Processor -- start 2024-09-19T04:46:52.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.420+0000 7f7def200640 1 -- start start 2024-09-19T04:46:52.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.420+0000 7f7def200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7df0068bc0 0x7f7df019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.420+0000 7f7def200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df01064c0 0x7f7df0109600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.420+0000 7f7def200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7df010a9c0 0x7f7df0109b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.420+0000 7f7def200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7df0111d50 con 0x7f7df01064c0 2024-09-19T04:46:52.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.420+0000 7f7def200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7df0111bd0 con 0x7f7df0068bc0 2024-09-19T04:46:52.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.420+0000 7f7def200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7df0111ed0 con 0x7f7df010a9c0 2024-09-19T04:46:52.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.421+0000 7f7ded400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df01064c0 0x7f7df0109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.421+0000 7f7dee800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7df010a9c0 0x7f7df0109b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.421+0000 7f7ded400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df01064c0 0x7f7df0109600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60372/0 (socket says 172.21.15.123:60372) 2024-09-19T04:46:52.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.421+0000 7f7ded400640 1 -- 172.21.15.123:0/3389864293 learned_addr learned my addr 172.21.15.123:0/3389864293 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:52.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.421+0000 7f7dee800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7df010a9c0 0x7f7df0109b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:36282/0 (socket says 172.21.15.123:36282) 2024-09-19T04:46:52.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.421+0000 7f7dede00640 1 --2- 172.21.15.123:0/3389864293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7df0068bc0 0x7f7df019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.421+0000 7f7ded400640 1 -- 172.21.15.123:0/3389864293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7df010a9c0 msgr2=0x7f7df0109b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.421+0000 7f7ded400640 1 --2- 172.21.15.123:0/3389864293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7df010a9c0 0x7f7df0109b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.421+0000 7f7ded400640 1 -- 172.21.15.123:0/3389864293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7df0068bc0 msgr2=0x7f7df019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.421+0000 7f7ded400640 1 --2- 172.21.15.123:0/3389864293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7df0068bc0 0x7f7df019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.421+0000 7f7ded400640 1 -- 172.21.15.123:0/3389864293 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df010a2c0 con 0x7f7df01064c0 2024-09-19T04:46:52.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.421+0000 7f7dee800640 1 --2- 172.21.15.123:0/3389864293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7df010a9c0 0x7f7df0109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:52.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.422+0000 7f7ded400640 1 --2- 172.21.15.123:0/3389864293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df01064c0 0x7f7df0109600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc02a930 tx=0x7f7ddc003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:52.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.422+0000 7f7de6a00640 1 -- 172.21.15.123:0/3389864293 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc03f070 con 0x7f7df01064c0 2024-09-19T04:46:52.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.422+0000 7f7de6a00640 1 -- 172.21.15.123:0/3389864293 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ddc031c90 con 0x7f7df01064c0 2024-09-19T04:46:52.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.422+0000 7f7def200640 1 -- 172.21.15.123:0/3389864293 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df01aae60 con 0x7f7df01064c0 2024-09-19T04:46:52.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.422+0000 7f7de6a00640 1 -- 172.21.15.123:0/3389864293 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc035030 con 0x7f7df01064c0 2024-09-19T04:46:52.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.422+0000 7f7def200640 1 -- 172.21.15.123:0/3389864293 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df01ab210 con 0x7f7df01064c0 2024-09-19T04:46:52.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.423+0000 7f7def200640 1 -- 172.21.15.123:0/3389864293 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df0101200 con 0x7f7df01064c0 2024-09-19T04:46:52.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.425+0000 7f7de6a00640 1 -- 172.21.15.123:0/3389864293 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 138583+0+0 (secure 0 0 0) 0x7f7ddc042070 con 0x7f7df01064c0 2024-09-19T04:46:52.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.426+0000 7f7de6a00640 1 --2- 172.21.15.123:0/3389864293 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7dd40a34d0 0x7f7dd40a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.426+0000 7f7de6a00640 1 -- 172.21.15.123:0/3389864293 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f7ddc0f05b0 con 0x7f7df01064c0 2024-09-19T04:46:52.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.427+0000 7f7dede00640 1 --2- 172.21.15.123:0/3389864293 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7dd40a34d0 0x7f7dd40a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.427+0000 7f7dede00640 1 --2- 172.21.15.123:0/3389864293 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7dd40a34d0 0x7f7dd40a5990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009bd0 tx=0x7f7de0005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:52.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.432+0000 7f7de6a00640 1 -- 172.21.15.123:0/3389864293 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7ddc0bc790 con 0x7f7df01064c0 2024-09-19T04:46:52.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.614+0000 7f7def200640 1 -- 172.21.15.123:0/3389864293 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f7df01ab550 con 0x7f7df01064c0 2024-09-19T04:46:52.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.619+0000 7f7de6a00640 1 -- 172.21.15.123:0/3389864293 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f7ddc0c0a90 con 0x7f7df01064c0 2024-09-19T04:46:52.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.621+0000 7f7def200640 1 -- 172.21.15.123:0/3389864293 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7dd40a34d0 msgr2=0x7f7dd40a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.621+0000 7f7def200640 1 --2- 172.21.15.123:0/3389864293 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7dd40a34d0 0x7f7dd40a5990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009bd0 tx=0x7f7de0005f90 comp rx=0 tx=0).stop 2024-09-19T04:46:52.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.621+0000 7f7def200640 1 -- 172.21.15.123:0/3389864293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df01064c0 msgr2=0x7f7df0109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.621+0000 7f7def200640 1 --2- 172.21.15.123:0/3389864293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df01064c0 0x7f7df0109600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc02a930 tx=0x7f7ddc003b90 comp rx=0 tx=0).stop 2024-09-19T04:46:52.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.621+0000 7f7def200640 1 -- 172.21.15.123:0/3389864293 shutdown_connections 2024-09-19T04:46:52.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.621+0000 7f7def200640 1 --2- 172.21.15.123:0/3389864293 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7dd40a34d0 0x7f7dd40a5990 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.621+0000 7f7def200640 1 --2- 172.21.15.123:0/3389864293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7df010a9c0 0x7f7df0109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.621+0000 7f7def200640 1 --2- 172.21.15.123:0/3389864293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df01064c0 0x7f7df0109600 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.621+0000 7f7def200640 1 --2- 172.21.15.123:0/3389864293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7df0068bc0 0x7f7df019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.621+0000 7f7def200640 1 -- 172.21.15.123:0/3389864293 >> 172.21.15.123:0/3389864293 conn(0x7f7df00fb820 msgr2=0x7f7df010e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:52.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.621+0000 7f7def200640 1 -- 172.21.15.123:0/3389864293 shutdown_connections 2024-09-19T04:46:52.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.621+0000 7f7def200640 1 -- 172.21.15.123:0/3389864293 wait complete. 2024-09-19T04:46:52.640 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-19T04:46:52.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.802+0000 7fc65f600640 1 -- 172.21.15.123:0/287298498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc658074ee0 msgr2=0x7fc65810de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.802+0000 7fc65f600640 1 --2- 172.21.15.123:0/287298498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc658074ee0 0x7fc65810de40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc644009900 tx=0x7fc64402a480 comp rx=0 tx=0).stop 2024-09-19T04:46:52.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.802+0000 7fc65f600640 1 -- 172.21.15.123:0/287298498 shutdown_connections 2024-09-19T04:46:52.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.802+0000 7fc65f600640 1 --2- 172.21.15.123:0/287298498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc65810e380 0x7fc658110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.802+0000 7fc65f600640 1 --2- 172.21.15.123:0/287298498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc658074ee0 0x7fc65810de40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.802+0000 7fc65f600640 1 --2- 172.21.15.123:0/287298498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6580745c0 0x7fc6580749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.802+0000 7fc65f600640 1 -- 172.21.15.123:0/287298498 >> 172.21.15.123:0/287298498 conn(0x7fc6580fd7e0 msgr2=0x7fc6580ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:52.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.803+0000 7fc65f600640 1 -- 172.21.15.123:0/287298498 shutdown_connections 2024-09-19T04:46:52.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.803+0000 7fc65f600640 1 -- 172.21.15.123:0/287298498 wait complete. 2024-09-19T04:46:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.803+0000 7fc65f600640 1 Processor -- start 2024-09-19T04:46:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.804+0000 7fc65f600640 1 -- start start 2024-09-19T04:46:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.804+0000 7fc65f600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6580745c0 0x7fc65819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.804+0000 7fc65f600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc658074ee0 0x7fc65819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.804+0000 7fc65f600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc65810e380 0x7fc65819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.804+0000 7fc65f600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc658114140 con 0x7fc658074ee0 2024-09-19T04:46:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.804+0000 7fc65f600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc658113fc0 con 0x7fc6580745c0 2024-09-19T04:46:52.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.804+0000 7fc65f600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc6581142c0 con 0x7fc65810e380 2024-09-19T04:46:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.804+0000 7fc657e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc658074ee0 0x7fc65819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.804+0000 7fc65d600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc65810e380 0x7fc65819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.804+0000 7fc65cc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6580745c0 0x7fc65819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.804+0000 7fc65d600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc65810e380 0x7fc65819ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:36296/0 (socket says 172.21.15.123:36296) 2024-09-19T04:46:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.804+0000 7fc65d600640 1 -- 172.21.15.123:0/3369325828 learned_addr learned my addr 172.21.15.123:0/3369325828 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.805+0000 7fc65d600640 1 -- 172.21.15.123:0/3369325828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6580745c0 msgr2=0x7fc65819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.805+0000 7fc65d600640 1 --2- 172.21.15.123:0/3369325828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6580745c0 0x7fc65819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.805+0000 7fc65d600640 1 -- 172.21.15.123:0/3369325828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc658074ee0 msgr2=0x7fc65819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.805+0000 7fc65d600640 1 --2- 172.21.15.123:0/3369325828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc658074ee0 0x7fc65819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.805+0000 7fc65d600640 1 -- 172.21.15.123:0/3369325828 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65819f5c0 con 0x7fc65810e380 2024-09-19T04:46:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.805+0000 7fc65cc00640 1 --2- 172.21.15.123:0/3369325828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6580745c0 0x7fc65819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.805+0000 7fc65d600640 1 --2- 172.21.15.123:0/3369325828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc65810e380 0x7fc65819ee90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc65000ec50 tx=0x7fc65000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:52.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.805+0000 7fc655600640 1 -- 172.21.15.123:0/3369325828 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc650003b70 con 0x7fc65810e380 2024-09-19T04:46:52.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.805+0000 7fc655600640 1 -- 172.21.15.123:0/3369325828 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc650003d10 con 0x7fc65810e380 2024-09-19T04:46:52.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.805+0000 7fc65f600640 1 -- 172.21.15.123:0/3369325828 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65819f8b0 con 0x7fc65810e380 2024-09-19T04:46:52.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.805+0000 7fc655600640 1 -- 172.21.15.123:0/3369325828 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6500109e0 con 0x7fc65810e380 2024-09-19T04:46:52.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.806+0000 7fc65f600640 1 -- 172.21.15.123:0/3369325828 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6581a7140 con 0x7fc65810e380 2024-09-19T04:46:52.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.807+0000 7fc65f600640 1 -- 172.21.15.123:0/3369325828 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6580749a0 con 0x7fc65810e380 2024-09-19T04:46:52.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.809+0000 7fc655600640 1 -- 172.21.15.123:0/3369325828 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 138583+0+0 (secure 0 0 0) 0x7fc650010b80 con 0x7fc65810e380 2024-09-19T04:46:52.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.810+0000 7fc655600640 1 --2- 172.21.15.123:0/3369325828 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc6400a3600 0x7fc6400a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.810+0000 7fc655600640 1 -- 172.21.15.123:0/3369325828 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fc6500d0720 con 0x7fc65810e380 2024-09-19T04:46:52.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.811+0000 7fc65cc00640 1 --2- 172.21.15.123:0/3369325828 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc6400a3600 0x7fc6400a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.811+0000 7fc65cc00640 1 --2- 172.21.15.123:0/3369325828 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc6400a3600 0x7fc6400a5ac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc64c0095f0 tx=0x7fc64c005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:52.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.812+0000 7fc655600640 1 -- 172.21.15.123:0/3369325828 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc65009ce80 con 0x7fc65810e380 2024-09-19T04:46:52.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.882+0000 7f6283200640 1 -- 172.21.15.123:0/3861848387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62841079c0 msgr2=0x7f6284107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.882+0000 7f6283200640 1 --2- 172.21.15.123:0/3861848387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62841079c0 0x7f6284107da0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6274009930 tx=0x7f627402a420 comp rx=0 tx=0).stop 2024-09-19T04:46:52.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.883+0000 7f6283200640 1 -- 172.21.15.123:0/3861848387 shutdown_connections 2024-09-19T04:46:52.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.883+0000 7f6283200640 1 --2- 172.21.15.123:0/3861848387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6284102360 0x7f628410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.883+0000 7f6283200640 1 --2- 172.21.15.123:0/3861848387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62841019c0 0x7f6284101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.883+0000 7f6283200640 1 --2- 172.21.15.123:0/3861848387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62841079c0 0x7f6284107da0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.883+0000 7f6283200640 1 -- 172.21.15.123:0/3861848387 >> 172.21.15.123:0/3861848387 conn(0x7f62840fd740 msgr2=0x7f62840ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:52.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.883+0000 7f6283200640 1 -- 172.21.15.123:0/3861848387 shutdown_connections 2024-09-19T04:46:52.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.883+0000 7f6283200640 1 -- 172.21.15.123:0/3861848387 wait complete. 2024-09-19T04:46:52.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.884+0000 7f6283200640 1 Processor -- start 2024-09-19T04:46:52.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.884+0000 7f6283200640 1 -- start start 2024-09-19T04:46:52.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.884+0000 7f6283200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62841019c0 0x7f62840744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.884+0000 7f6283200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6284102360 0x7f6284074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.884+0000 7f6283200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62841079c0 0x7f6284074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.884+0000 7f6283200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6284111e10 con 0x7f6284102360 2024-09-19T04:46:52.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.884+0000 7f6283200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6284111c90 con 0x7f62841019c0 2024-09-19T04:46:52.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.884+0000 7f6283200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6284111f90 con 0x7f62841079c0 2024-09-19T04:46:52.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.884+0000 7f6281400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6284102360 0x7f6284074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.884+0000 7f6281400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6284102360 0x7f6284074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60408/0 (socket says 172.21.15.123:60408) 2024-09-19T04:46:52.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.884+0000 7f6281400640 1 -- 172.21.15.123:0/1980744355 learned_addr learned my addr 172.21.15.123:0/1980744355 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:52.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.884+0000 7f6281e00640 1 --2- 172.21.15.123:0/1980744355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62841019c0 0x7f62840744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.885+0000 7f6282800640 1 --2- 172.21.15.123:0/1980744355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62841079c0 0x7f6284074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.885+0000 7f6281400640 1 -- 172.21.15.123:0/1980744355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62841079c0 msgr2=0x7f6284074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.885+0000 7f6281400640 1 --2- 172.21.15.123:0/1980744355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62841079c0 0x7f6284074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.885+0000 7f6281400640 1 -- 172.21.15.123:0/1980744355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62841019c0 msgr2=0x7f62840744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.885+0000 7f6281400640 1 --2- 172.21.15.123:0/1980744355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62841019c0 0x7f62840744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.885+0000 7f6281400640 1 -- 172.21.15.123:0/1980744355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62841a6750 con 0x7f6284102360 2024-09-19T04:46:52.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.885+0000 7f6281400640 1 --2- 172.21.15.123:0/1980744355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6284102360 0x7f6284074a30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f627000e9f0 tx=0x7f627000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:52.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.885+0000 7f627aa00640 1 -- 172.21.15.123:0/1980744355 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627000cd80 con 0x7f6284102360 2024-09-19T04:46:52.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.886+0000 7f6283200640 1 -- 172.21.15.123:0/1980744355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62841a6a40 con 0x7f6284102360 2024-09-19T04:46:52.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.886+0000 7f627aa00640 1 -- 172.21.15.123:0/1980744355 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6270010910 con 0x7f6284102360 2024-09-19T04:46:52.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.886+0000 7f6283200640 1 -- 172.21.15.123:0/1980744355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62841a6fd0 con 0x7f6284102360 2024-09-19T04:46:52.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.886+0000 7f627aa00640 1 -- 172.21.15.123:0/1980744355 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6270018910 con 0x7f6284102360 2024-09-19T04:46:52.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.889+0000 7f628b8d5740 1 Processor -- start 2024-09-19T04:46:52.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.889+0000 7f628b8d5740 1 -- start start 2024-09-19T04:46:52.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.890+0000 7f628b8d5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e20675dcb0 0x55e20675e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.890+0000 7f628b8d5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 0x55e206768950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.890+0000 7f628b8d5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e206768e90 0x55e20676b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.890+0000 7f628b8d5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e2066c4290 con 0x55e206768e90 2024-09-19T04:46:52.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.890+0000 7f628b8d5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e20666bb40 con 0x55e20675dcb0 2024-09-19T04:46:52.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.890+0000 7f628b8d5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e2066c0250 con 0x55e20675e5d0 2024-09-19T04:46:52.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.890+0000 7f6281e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 0x55e206768950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.890+0000 7f6282800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e20675dcb0 0x55e20675e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.890+0000 7f6281e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 0x55e206768950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:36324/0 (socket says 172.21.15.123:36324) 2024-09-19T04:46:52.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.890+0000 7f6281e00640 1 -- 172.21.15.123:0/580569496 learned_addr learned my addr 172.21.15.123:0/580569496 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:52.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.890+0000 7f6281e00640 1 -- 172.21.15.123:0/580569496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e20675dcb0 msgr2=0x55e20675e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:52.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.890+0000 7f6281e00640 1 --2- 172.21.15.123:0/580569496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e20675dcb0 0x55e20675e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.890+0000 7f6281e00640 1 -- 172.21.15.123:0/580569496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e206768e90 msgr2=0x55e20676b280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:46:52.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.890+0000 7f6281e00640 1 --2- 172.21.15.123:0/580569496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e206768e90 0x55e20676b280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.890+0000 7f6281e00640 1 -- 172.21.15.123:0/580569496 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2064356a0 con 0x55e20675e5d0 2024-09-19T04:46:52.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.891+0000 7f627aa00640 1 -- 172.21.15.123:0/1980744355 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 138583+0+0 (secure 0 0 0) 0x7f6270010ac0 con 0x7f6284102360 2024-09-19T04:46:52.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.891+0000 7f6281e00640 1 --2- 172.21.15.123:0/580569496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 0x55e206768950 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6274009930 tx=0x7f6274005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df35df1e16c28fc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:52.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.891+0000 7f626f400640 1 -- 172.21.15.123:0/580569496 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6274035040 con 0x55e20675e5d0 2024-09-19T04:46:52.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.891+0000 7f626f400640 1 -- 172.21.15.123:0/580569496 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6274005520 con 0x55e20675e5d0 2024-09-19T04:46:52.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.891+0000 7f626f400640 1 -- 172.21.15.123:0/580569496 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62740035f0 con 0x55e20675e5d0 2024-09-19T04:46:52.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.892+0000 7f628b8d5740 1 -- 172.21.15.123:0/580569496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 msgr2=0x55e206768950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.892+0000 7f628b8d5740 1 --2- 172.21.15.123:0/580569496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 0x55e206768950 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6274009930 tx=0x7f6274005de0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.892+0000 7f628b8d5740 1 -- 172.21.15.123:0/580569496 shutdown_connections 2024-09-19T04:46:52.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.892+0000 7f628b8d5740 1 --2- 172.21.15.123:0/580569496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e206768e90 0x55e20676b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.892+0000 7f628b8d5740 1 --2- 172.21.15.123:0/580569496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 0x55e206768950 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.892+0000 7f628b8d5740 1 --2- 172.21.15.123:0/580569496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e20675dcb0 0x55e20675e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.892+0000 7f628b8d5740 1 -- 172.21.15.123:0/580569496 >> 172.21.15.123:0/580569496 conn(0x55e20675a240 msgr2=0x55e20675a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:52.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.892+0000 7f628b8d5740 1 -- 172.21.15.123:0/580569496 shutdown_connections 2024-09-19T04:46:52.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.892+0000 7f628b8d5740 1 -- 172.21.15.123:0/580569496 wait complete. 2024-09-19T04:46:52.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.892+0000 7f627aa00640 1 --2- 172.21.15.123:0/1980744355 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f62680a33e0 0x7f62680a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.892+0000 7f627aa00640 1 -- 172.21.15.123:0/1980744355 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f62700d02f0 con 0x7f6284102360 2024-09-19T04:46:52.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.893+0000 7f628b8d5740 1 -- start start 2024-09-19T04:46:52.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.893+0000 7f6281e00640 1 --2- 172.21.15.123:0/1980744355 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f62680a33e0 0x7f62680a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.893+0000 7f628b8d5740 1 Processor -- start 2024-09-19T04:46:52.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.893+0000 7f6281e00640 1 --2- 172.21.15.123:0/1980744355 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f62680a33e0 0x7f62680a58a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f627402aa10 tx=0x7f6274005fd0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:52.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f628b8d5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e20675dcb0 0x55e2067687d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f628b8d5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 0x55e206765a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f628b8d5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e206768e90 0x55e206765fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f628b8d5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e20676bb00 con 0x55e20675dcb0 2024-09-19T04:46:52.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f628b8d5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e20667efa0 con 0x55e206768e90 2024-09-19T04:46:52.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f628b8d5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e20676bc80 con 0x55e20675e5d0 2024-09-19T04:46:52.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f628b8d5740 10 client.0 _ll_register_callbacks cb 0x55e2063d2ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:52.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f6281e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e20675dcb0 0x55e2067687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f6281400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 0x55e206765a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f6281400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 0x55e206765a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:36328/0 (socket says 172.21.15.123:36328) 2024-09-19T04:46:52.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f6281e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e20675dcb0 0x55e2067687d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60414/0 (socket says 172.21.15.123:60414) 2024-09-19T04:46:52.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f6281400640 1 -- 172.21.15.123:0/3324425209 learned_addr learned my addr 172.21.15.123:0/3324425209 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f6282800640 1 --2- 172.21.15.123:0/3324425209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e206768e90 0x55e206765fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f6282800640 1 -- 172.21.15.123:0/3324425209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 msgr2=0x55e206765a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f6282800640 1 --2- 172.21.15.123:0/3324425209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 0x55e206765a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f6282800640 1 -- 172.21.15.123:0/3324425209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e20675dcb0 msgr2=0x55e2067687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f6282800640 1 --2- 172.21.15.123:0/3324425209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e20675dcb0 0x55e2067687d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f6282800640 1 -- 172.21.15.123:0/3324425209 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2067667f0 con 0x55e206768e90 2024-09-19T04:46:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.894+0000 7f6281400640 1 --2- 172.21.15.123:0/3324425209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 0x55e206765a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.895+0000 7f6282800640 1 --2- 172.21.15.123:0/3324425209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e206768e90 0x55e206765fc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f627c011ce0 tx=0x7f627c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=269173ca144c8140 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.895+0000 7f6263e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:46:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.895+0000 7f6263e00640 1 -- 172.21.15.123:0/3324425209 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c004b20 con 0x55e206768e90 2024-09-19T04:46:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.895+0000 7f628b8d5740 1 -- 172.21.15.123:0/3324425209 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6274005520 con 0x55e206768e90 2024-09-19T04:46:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.895+0000 7f6263e00640 1 -- 172.21.15.123:0/3324425209 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f627c004cc0 con 0x55e206768e90 2024-09-19T04:46:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.895+0000 7f6263e00640 1 -- 172.21.15.123:0/3324425209 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f627c003970 con 0x55e206768e90 2024-09-19T04:46:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.895+0000 7f628b8d5740 10 client.25289 fetch_fsmap learned FSMap version 28 2024-09-19T04:46:52.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.895+0000 7f628b8d5740 1 -- 172.21.15.123:0/3324425209 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e206777a60 con 0x55e206768e90 2024-09-19T04:46:52.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.896+0000 7f6263e00640 1 -- 172.21.15.123:0/3324425209 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6274005520 con 0x55e206768e90 2024-09-19T04:46:52.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.896+0000 7f6263e00640 1 -- 172.21.15.123:0/3324425209 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f627c005020 con 0x55e206768e90 2024-09-19T04:46:52.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.896+0000 7f628b8d5740 10 client.25289 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:46:52.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.896+0000 7f628b8d5740 10 client.25289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:46:52.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.896+0000 7f628b8d5740 20 client.25289 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.896+0000 7f628b8d5740 1 --2- 172.21.15.123:0/3324425209 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55e206778e70 0x55e20677b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.896+0000 7f628b8d5740 4 client.25289 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:46:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.896+0000 7f628b8d5740 1 -- 172.21.15.123:0/3324425209 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2066d1800 con 0x55e206778e70 2024-09-19T04:46:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.896+0000 7f6281e00640 1 --2- 172.21.15.123:0/3324425209 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55e206778e70 0x55e20677b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.897+0000 7f6281e00640 1 --2- 172.21.15.123:0/3324425209 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55e206778e70 0x55e20677b260 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb94be6c5674650f server_cookie=ceeb6b5a47ffbc66 in_seq=0 out_seq=0 2024-09-19T04:46:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.897+0000 7f6263e00640 10 client.25289 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:46:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.898+0000 7f6263e00640 1 -- 172.21.15.123:0/3324425209 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e2066d1800 con 0x55e206778e70 2024-09-19T04:46:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.898+0000 7f6263e00640 10 client.25289 handle_command_reply: tid=0 2024-09-19T04:46:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.898+0000 7f628b8d5740 1 client.25289 shutdown 2024-09-19T04:46:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.898+0000 7f628b8d5740 1 -- 172.21.15.123:0/3324425209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e206768e90 msgr2=0x55e206765fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.898+0000 7f628b8d5740 1 --2- 172.21.15.123:0/3324425209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e206768e90 0x55e206765fc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f627c011ce0 tx=0x7f627c00f790 comp rx=0 tx=0).stop 2024-09-19T04:46:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.898+0000 7f628b8d5740 1 -- 172.21.15.123:0/3324425209 shutdown_connections 2024-09-19T04:46:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.898+0000 7f628b8d5740 1 --2- 172.21.15.123:0/3324425209 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55e206778e70 0x55e20677b260 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.898+0000 7f628b8d5740 1 --2- 172.21.15.123:0/3324425209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e206768e90 0x55e206765fc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.898+0000 7f628b8d5740 1 --2- 172.21.15.123:0/3324425209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 0x55e206765a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.898+0000 7f628b8d5740 1 --2- 172.21.15.123:0/3324425209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e20675dcb0 0x55e2067687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.898+0000 7f628b8d5740 1 -- 172.21.15.123:0/3324425209 >> 172.21.15.123:0/3324425209 conn(0x55e20675a240 msgr2=0x55e20675ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:52.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.898+0000 7f6263e00640 0 client.25289 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:46:52.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.898+0000 7f628b8d5740 1 -- 172.21.15.123:0/3324425209 shutdown_connections 2024-09-19T04:46:52.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.898+0000 7f628b8d5740 1 -- 172.21.15.123:0/3324425209 wait complete. 2024-09-19T04:46:52.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.899+0000 7f628b8d5740 20 client.25289 trim_cache size 0 max 16384 2024-09-19T04:46:52.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.916+0000 7f628b8d5740 1 Processor -- start 2024-09-19T04:46:52.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.917+0000 7f628b8d5740 1 -- start start 2024-09-19T04:46:52.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.917+0000 7f628b8d5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e20675dcb0 0x55e20675fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.917+0000 7f628b8d5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e20675e5d0 0x55e206780b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.917+0000 7f628b8d5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e206768e90 0x55e206781060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.917+0000 7f628b8d5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62841af8d0 con 0x55e20675dcb0 2024-09-19T04:46:52.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.917+0000 7f628b8d5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62841b1410 con 0x55e20675e5d0 2024-09-19T04:46:52.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.917+0000 7f628b8d5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f62841affa0 con 0x55e206768e90 2024-09-19T04:46:52.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.917+0000 7f6281400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e20675e5d0 0x55e206780b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.917+0000 7f6281e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e20675dcb0 0x55e20675fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.917+0000 7f6281400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e20675e5d0 0x55e206780b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:35602/0 (socket says 172.21.15.123:35602) 2024-09-19T04:46:52.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.917+0000 7f6282800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e206768e90 0x55e206781060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.917+0000 7f6281400640 1 -- 172.21.15.123:0/2442364053 learned_addr learned my addr 172.21.15.123:0/2442364053 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:52.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.918+0000 7f6281400640 1 -- 172.21.15.123:0/2442364053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e206768e90 msgr2=0x55e206781060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:52.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.918+0000 7f6281400640 1 --2- 172.21.15.123:0/2442364053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e206768e90 0x55e206781060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.918+0000 7f6281400640 1 -- 172.21.15.123:0/2442364053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e20675dcb0 msgr2=0x55e20675fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:52.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.918+0000 7f6281400640 1 --2- 172.21.15.123:0/2442364053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e20675dcb0 0x55e20675fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.918+0000 7f6281400640 1 -- 172.21.15.123:0/2442364053 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e206764a50 con 0x55e20675e5d0 2024-09-19T04:46:52.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.918+0000 7f6281e00640 1 --2- 172.21.15.123:0/2442364053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e20675dcb0 0x55e20675fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:46:52.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.918+0000 7f6281400640 1 --2- 172.21.15.123:0/2442364053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e20675e5d0 0x55e206780b20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6270009380 tx=0x7f6270097fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afeb3a72b8ea3e3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:52.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.918+0000 7f626ea00640 1 -- 172.21.15.123:0/2442364053 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62700b5210 con 0x55e20675e5d0 2024-09-19T04:46:52.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.918+0000 7f626ea00640 1 -- 172.21.15.123:0/2442364053 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62700cdbd0 con 0x55e20675e5d0 2024-09-19T04:46:52.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.918+0000 7f626ea00640 1 -- 172.21.15.123:0/2442364053 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627009b8b0 con 0x55e20675e5d0 2024-09-19T04:46:52.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.919+0000 7f628b8d5740 1 -- 172.21.15.123:0/2442364053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e20675e5d0 msgr2=0x55e206780b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.919+0000 7f628b8d5740 1 --2- 172.21.15.123:0/2442364053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e20675e5d0 0x55e206780b20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6270009380 tx=0x7f6270097fe0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.919+0000 7f628b8d5740 1 -- 172.21.15.123:0/2442364053 shutdown_connections 2024-09-19T04:46:52.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.919+0000 7f628b8d5740 1 --2- 172.21.15.123:0/2442364053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e206768e90 0x55e206781060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.919+0000 7f628b8d5740 1 --2- 172.21.15.123:0/2442364053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e20675e5d0 0x55e206780b20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.919+0000 7f628b8d5740 1 --2- 172.21.15.123:0/2442364053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e20675dcb0 0x55e20675fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.919+0000 7f628b8d5740 1 -- 172.21.15.123:0/2442364053 >> 172.21.15.123:0/2442364053 conn(0x55e20675a240 msgr2=0x55e20675f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:52.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.919+0000 7f628b8d5740 1 -- 172.21.15.123:0/2442364053 shutdown_connections 2024-09-19T04:46:52.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.919+0000 7f628b8d5740 1 -- 172.21.15.123:0/2442364053 wait complete. 2024-09-19T04:46:52.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.920+0000 7f628b8d5740 1 -- start start 2024-09-19T04:46:52.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.920+0000 7f628b8d5740 1 Processor -- start 2024-09-19T04:46:52.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.920+0000 7f628b8d5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e20675dcb0 0x55e20676d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.920+0000 7f628b8d5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 0x55e20676d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.920+0000 7f628b8d5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e206768e90 0x55e2067e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.920+0000 7f628b8d5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62841b0d40 con 0x55e206768e90 2024-09-19T04:46:52.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.920+0000 7f628b8d5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62841b0670 con 0x55e20675dcb0 2024-09-19T04:46:52.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.920+0000 7f628b8d5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f62841ad8b0 con 0x55e20675e5d0 2024-09-19T04:46:52.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.921+0000 7f628b8d5740 10 client.0 _ll_register_callbacks cb 0x7f627c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:52.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.921+0000 7f6282800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e206768e90 0x55e2067e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.921+0000 7f6282800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e206768e90 0x55e2067e2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60436/0 (socket says 172.21.15.123:60436) 2024-09-19T04:46:52.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.921+0000 7f6282800640 1 -- 172.21.15.123:0/1922846026 learned_addr learned my addr 172.21.15.123:0/1922846026 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:52.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.921+0000 7f6281400640 1 --2- 172.21.15.123:0/1922846026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 0x55e20676d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.921+0000 7f6281e00640 1 --2- 172.21.15.123:0/1922846026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e20675dcb0 0x55e20676d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.921+0000 7f6282800640 1 -- 172.21.15.123:0/1922846026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 msgr2=0x55e20676d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:52.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.921+0000 7f6282800640 1 --2- 172.21.15.123:0/1922846026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 0x55e20676d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.921+0000 7f6282800640 1 -- 172.21.15.123:0/1922846026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e20675dcb0 msgr2=0x55e20676d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:52.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.921+0000 7f6282800640 1 --2- 172.21.15.123:0/1922846026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e20675dcb0 0x55e20676d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.921+0000 7f6282800640 1 -- 172.21.15.123:0/1922846026 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2067e3130 con 0x55e206768e90 2024-09-19T04:46:52.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.921+0000 7f6281400640 1 --2- 172.21.15.123:0/1922846026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 0x55e20676d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:52.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.922+0000 7f6282800640 1 --2- 172.21.15.123:0/1922846026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e206768e90 0x55e2067e2af0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f627c005440 tx=0x7f627c005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907b182840f414f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:52.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.922+0000 7f6263e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:46:52.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.922+0000 7f6263e00640 1 -- 172.21.15.123:0/1922846026 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c003d20 con 0x55e206768e90 2024-09-19T04:46:52.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.922+0000 7f628b8d5740 1 -- 172.21.15.123:0/1922846026 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62700cdbd0 con 0x55e206768e90 2024-09-19T04:46:52.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.922+0000 7f6263e00640 1 -- 172.21.15.123:0/1922846026 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f627c0056b0 con 0x55e206768e90 2024-09-19T04:46:52.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.922+0000 7f6263e00640 1 -- 172.21.15.123:0/1922846026 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f627c03a050 con 0x55e206768e90 2024-09-19T04:46:52.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.922+0000 7f628b8d5740 10 client.15270 fetch_fsmap learned FSMap version 28 2024-09-19T04:46:52.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.922+0000 7f628b8d5740 1 -- 172.21.15.123:0/1922846026 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e2067e4430 con 0x55e206768e90 2024-09-19T04:46:52.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.923+0000 7f6263e00640 1 -- 172.21.15.123:0/1922846026 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62700cdbd0 con 0x55e206768e90 2024-09-19T04:46:52.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.923+0000 7f6263e00640 1 -- 172.21.15.123:0/1922846026 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f627c02cc10 con 0x55e206768e90 2024-09-19T04:46:52.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.923+0000 7f628b8d5740 10 client.15270 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:46:52.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.923+0000 7f628b8d5740 10 client.15270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:46:52.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.923+0000 7f628b8d5740 20 client.15270 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:52.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.923+0000 7f628b8d5740 1 --2- 172.21.15.123:0/1922846026 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55e2067e4b70 0x55e2067e6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:52.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.923+0000 7f628b8d5740 4 client.15270 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:46:52.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.923+0000 7f628b8d5740 1 -- 172.21.15.123:0/1922846026 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e2066cbd10 con 0x55e2067e4b70 2024-09-19T04:46:52.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.923+0000 7f6281e00640 1 --2- 172.21.15.123:0/1922846026 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55e2067e4b70 0x55e2067e6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:52.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.924+0000 7f6281e00640 1 --2- 172.21.15.123:0/1922846026 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55e2067e4b70 0x55e2067e6f90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d786f0a9da2bfcb5 server_cookie=f6d2bac8ae82a86b in_seq=0 out_seq=0 2024-09-19T04:46:52.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.924+0000 7f6263e00640 10 client.15270 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:46:52.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.925+0000 7f6263e00640 1 -- 172.21.15.123:0/1922846026 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e2066cbd10 con 0x55e2067e4b70 2024-09-19T04:46:52.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.925+0000 7f6263e00640 10 client.15270 handle_command_reply: tid=0 2024-09-19T04:46:52.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.925+0000 7f628b8d5740 1 client.15270 shutdown 2024-09-19T04:46:52.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.925+0000 7f628b8d5740 1 -- 172.21.15.123:0/1922846026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e206768e90 msgr2=0x55e2067e2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.925+0000 7f628b8d5740 1 --2- 172.21.15.123:0/1922846026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e206768e90 0x55e2067e2af0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f627c005440 tx=0x7f627c005280 comp rx=0 tx=0).stop 2024-09-19T04:46:52.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.925+0000 7f628b8d5740 1 -- 172.21.15.123:0/1922846026 shutdown_connections 2024-09-19T04:46:52.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.925+0000 7f628b8d5740 1 --2- 172.21.15.123:0/1922846026 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55e2067e4b70 0x55e2067e6f90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.925+0000 7f628b8d5740 1 --2- 172.21.15.123:0/1922846026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e206768e90 0x55e2067e2af0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.925+0000 7f6263e00640 0 client.15270 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:46:52.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.925+0000 7f628b8d5740 1 --2- 172.21.15.123:0/1922846026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e20675e5d0 0x55e20676d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.925+0000 7f628b8d5740 1 --2- 172.21.15.123:0/1922846026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e20675dcb0 0x55e20676d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.925+0000 7f628b8d5740 1 -- 172.21.15.123:0/1922846026 >> 172.21.15.123:0/1922846026 conn(0x55e20675a240 msgr2=0x55e206759270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:52.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.925+0000 7f628b8d5740 1 -- 172.21.15.123:0/1922846026 shutdown_connections 2024-09-19T04:46:52.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.925+0000 7f628b8d5740 1 -- 172.21.15.123:0/1922846026 wait complete. 2024-09-19T04:46:52.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.926+0000 7f628b8d5740 20 client.15270 trim_cache size 0 max 16384 2024-09-19T04:46:52.930 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:46:52.930 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-09-19T04:46:52.930 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "5acf54a5-fef7-4ca9-9a57-2c90575994be", 2024-09-19T04:46:52.930 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-09-19T04:46:52.930 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:46:52.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.926+0000 7f6283200640 1 -- 172.21.15.123:0/1980744355 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f62680a33e0 msgr2=0x7f62680a58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.926+0000 7f6283200640 1 --2- 172.21.15.123:0/1980744355 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f62680a33e0 0x7f62680a58a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f627402aa10 tx=0x7f6274005fd0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.926+0000 7f6283200640 1 -- 172.21.15.123:0/1980744355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6284102360 msgr2=0x7f6284074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:52.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.926+0000 7f6283200640 1 --2- 172.21.15.123:0/1980744355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6284102360 0x7f6284074a30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f627000e9f0 tx=0x7f627000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.926+0000 7f6283200640 1 -- 172.21.15.123:0/1980744355 shutdown_connections 2024-09-19T04:46:52.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.926+0000 7f6283200640 1 --2- 172.21.15.123:0/1980744355 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f62680a33e0 0x7f62680a58a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.926+0000 7f6283200640 1 --2- 172.21.15.123:0/1980744355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62841079c0 0x7f6284074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.926+0000 7f6283200640 1 --2- 172.21.15.123:0/1980744355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6284102360 0x7f6284074a30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.926+0000 7f6283200640 1 --2- 172.21.15.123:0/1980744355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62841019c0 0x7f62840744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:52.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.926+0000 7f6283200640 1 -- 172.21.15.123:0/1980744355 >> 172.21.15.123:0/1980744355 conn(0x7f62840fd740 msgr2=0x7f62840fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:52.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.927+0000 7f6283200640 1 -- 172.21.15.123:0/1980744355 shutdown_connections 2024-09-19T04:46:52.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:52.927+0000 7f6283200640 1 -- 172.21.15.123:0/1980744355 wait complete. 2024-09-19T04:46:53.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.006+0000 7fc65f600640 1 -- 172.21.15.123:0/3369325828 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fc65819b920 con 0x7fc65810e380 2024-09-19T04:46:53.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.013+0000 7fc655600640 1 -- 172.21.15.123:0/3369325828 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7fc65819b920 con 0x7fc65810e380 2024-09-19T04:46:53.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.015+0000 7fc65f600640 1 -- 172.21.15.123:0/3369325828 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc6400a3600 msgr2=0x7fc6400a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:53.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.015+0000 7fc65f600640 1 --2- 172.21.15.123:0/3369325828 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc6400a3600 0x7fc6400a5ac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc64c0095f0 tx=0x7fc64c005f90 comp rx=0 tx=0).stop 2024-09-19T04:46:53.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.015+0000 7fc65f600640 1 -- 172.21.15.123:0/3369325828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc65810e380 msgr2=0x7fc65819ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:53.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.015+0000 7fc65f600640 1 --2- 172.21.15.123:0/3369325828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc65810e380 0x7fc65819ee90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc65000ec50 tx=0x7fc65000c6a0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.015+0000 7fc65f600640 1 -- 172.21.15.123:0/3369325828 shutdown_connections 2024-09-19T04:46:53.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.015+0000 7fc65f600640 1 --2- 172.21.15.123:0/3369325828 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc6400a3600 0x7fc6400a5ac0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.015+0000 7fc65f600640 1 --2- 172.21.15.123:0/3369325828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc65810e380 0x7fc65819ee90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.015+0000 7fc65f600640 1 --2- 172.21.15.123:0/3369325828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc658074ee0 0x7fc65819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.015+0000 7fc65f600640 1 --2- 172.21.15.123:0/3369325828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6580745c0 0x7fc65819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.015+0000 7fc65f600640 1 -- 172.21.15.123:0/3369325828 >> 172.21.15.123:0/3369325828 conn(0x7fc6580fd7e0 msgr2=0x7fc6580fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:53.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.016+0000 7fc65f600640 1 -- 172.21.15.123:0/3369325828 shutdown_connections 2024-09-19T04:46:53.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.016+0000 7fc65f600640 1 -- 172.21.15.123:0/3369325828 wait complete. 2024-09-19T04:46:53.035 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-19T04:46:53.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.197+0000 7fadc2c00640 1 -- 172.21.15.123:0/1430806033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadbc103f50 msgr2=0x7fadbc104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:53.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.197+0000 7fadc2c00640 1 --2- 172.21.15.123:0/1430806033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadbc103f50 0x7fadbc104330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fadac009860 tx=0x7fadac02a340 comp rx=0 tx=0).stop 2024-09-19T04:46:53.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.198+0000 7fadc2c00640 1 -- 172.21.15.123:0/1430806033 shutdown_connections 2024-09-19T04:46:53.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.198+0000 7fadc2c00640 1 --2- 172.21.15.123:0/1430806033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadbc1090a0 0x7fadbc110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.198+0000 7fadc2c00640 1 --2- 172.21.15.123:0/1430806033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadbc104900 0x7fadbc108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.198+0000 7fadc2c00640 1 --2- 172.21.15.123:0/1430806033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadbc103f50 0x7fadbc104330 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.198+0000 7fadc2c00640 1 -- 172.21.15.123:0/1430806033 >> 172.21.15.123:0/1430806033 conn(0x7fadbc0ff9e0 msgr2=0x7fadbc101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:53.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.198+0000 7fadc2c00640 1 -- 172.21.15.123:0/1430806033 shutdown_connections 2024-09-19T04:46:53.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.198+0000 7fadc2c00640 1 -- 172.21.15.123:0/1430806033 wait complete. 2024-09-19T04:46:53.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.199+0000 7fadc2c00640 1 Processor -- start 2024-09-19T04:46:53.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.199+0000 7fadc2c00640 1 -- start start 2024-09-19T04:46:53.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.199+0000 7fadc2c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadbc103f50 0x7fadbc19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:53.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.199+0000 7fadc2c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadbc1090a0 0x7fadbc19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:53.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.199+0000 7fadc2c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadbc19eea0 0x7fadbc19f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:53.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.199+0000 7fadc2c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1140f0 con 0x7fadbc103f50 2024-09-19T04:46:53.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.199+0000 7fadc2c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc113f70 con 0x7fadbc1090a0 2024-09-19T04:46:53.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.199+0000 7fadc2c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc114270 con 0x7fadbc19eea0 2024-09-19T04:46:53.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.200+0000 7fadbbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadbc103f50 0x7fadbc19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:53.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.200+0000 7fadb3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadbc1090a0 0x7fadbc19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:53.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.200+0000 7fadbbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadbc103f50 0x7fadbc19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60450/0 (socket says 172.21.15.123:60450) 2024-09-19T04:46:53.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.200+0000 7fadbbe00640 1 -- 172.21.15.123:0/16125719 learned_addr learned my addr 172.21.15.123:0/16125719 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:53.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.200+0000 7fadb3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadbc1090a0 0x7fadbc19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:35638/0 (socket says 172.21.15.123:35638) 2024-09-19T04:46:53.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.200+0000 7fadc0c00640 1 --2- 172.21.15.123:0/16125719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadbc19eea0 0x7fadbc19f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:53.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.200+0000 7fadb3e00640 1 -- 172.21.15.123:0/16125719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadbc19eea0 msgr2=0x7fadbc19f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:53.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.200+0000 7fadb3e00640 1 --2- 172.21.15.123:0/16125719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadbc19eea0 0x7fadbc19f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.200+0000 7fadb3e00640 1 -- 172.21.15.123:0/16125719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadbc103f50 msgr2=0x7fadbc19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:53.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.200+0000 7fadb3e00640 1 --2- 172.21.15.123:0/16125719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadbc103f50 0x7fadbc19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.200+0000 7fadb3e00640 1 -- 172.21.15.123:0/16125719 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadbc19f990 con 0x7fadbc1090a0 2024-09-19T04:46:53.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.200+0000 7fadbbe00640 1 --2- 172.21.15.123:0/16125719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadbc103f50 0x7fadbc19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:53.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.200+0000 7fadb3e00640 1 --2- 172.21.15.123:0/16125719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadbc1090a0 0x7fadbc19ab20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fada4005fd0 tx=0x7fada400ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:53.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.201+0000 7fadb9600640 1 -- 172.21.15.123:0/16125719 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada400cc90 con 0x7fadbc1090a0 2024-09-19T04:46:53.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.201+0000 7fadc2c00640 1 -- 172.21.15.123:0/16125719 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadbc19fc80 con 0x7fadbc1090a0 2024-09-19T04:46:53.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.201+0000 7fadb9600640 1 -- 172.21.15.123:0/16125719 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada400ee70 con 0x7fadbc1090a0 2024-09-19T04:46:53.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.201+0000 7fadb9600640 1 -- 172.21.15.123:0/16125719 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4004a00 con 0x7fadbc1090a0 2024-09-19T04:46:53.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.201+0000 7fadc2c00640 1 -- 172.21.15.123:0/16125719 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadbc1a7420 con 0x7fadbc1090a0 2024-09-19T04:46:53.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.202+0000 7fadc2c00640 1 -- 172.21.15.123:0/16125719 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadbc104330 con 0x7fadbc1090a0 2024-09-19T04:46:53.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.204+0000 7fadb9600640 1 -- 172.21.15.123:0/16125719 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 138583+0+0 (secure 0 0 0) 0x7fada4004ba0 con 0x7fadbc1090a0 2024-09-19T04:46:53.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.205+0000 7fadb9600640 1 --2- 172.21.15.123:0/16125719 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fada00a3580 0x7fada00a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:53.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.205+0000 7fadb9600640 1 -- 172.21.15.123:0/16125719 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fada40cff50 con 0x7fadbc1090a0 2024-09-19T04:46:53.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.205+0000 7fadbbe00640 1 --2- 172.21.15.123:0/16125719 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fada00a3580 0x7fada00a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:53.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.206+0000 7fadbbe00640 1 --2- 172.21.15.123:0/16125719 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fada00a3580 0x7fada00a5a40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fadac0057c0 tx=0x7fadac0056d0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:53.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.207+0000 7fadb9600640 1 -- 172.21.15.123:0/16125719 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fada409c740 con 0x7fadbc1090a0 2024-09-19T04:46:53.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.413+0000 7fadc2c00640 1 -- 172.21.15.123:0/16125719 --> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fadbc104900 con 0x7fada00a3580 2024-09-19T04:46:53.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.486+0000 7fadb9600640 1 -- 172.21.15.123:0/16125719 <== mgr.25133 v2:172.21.15.123:6800/1578549821 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fadbc104900 con 0x7fada00a3580 2024-09-19T04:46:53.487 INFO:teuthology.orchestra.run.smithi123.stdout:Schedule set for path / 2024-09-19T04:46:53.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.488+0000 7fadc2c00640 1 -- 172.21.15.123:0/16125719 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fada00a3580 msgr2=0x7fada00a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:53.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.488+0000 7fadc2c00640 1 --2- 172.21.15.123:0/16125719 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fada00a3580 0x7fada00a5a40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fadac0057c0 tx=0x7fadac0056d0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.488+0000 7fadc2c00640 1 -- 172.21.15.123:0/16125719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadbc1090a0 msgr2=0x7fadbc19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:53.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.488+0000 7fadc2c00640 1 --2- 172.21.15.123:0/16125719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadbc1090a0 0x7fadbc19ab20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fada4005fd0 tx=0x7fada400ec20 comp rx=0 tx=0).stop 2024-09-19T04:46:53.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.488+0000 7fadc2c00640 1 -- 172.21.15.123:0/16125719 shutdown_connections 2024-09-19T04:46:53.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.488+0000 7fadc2c00640 1 --2- 172.21.15.123:0/16125719 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fada00a3580 0x7fada00a5a40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.488+0000 7fadc2c00640 1 --2- 172.21.15.123:0/16125719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadbc19eea0 0x7fadbc19f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.488+0000 7fadc2c00640 1 --2- 172.21.15.123:0/16125719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadbc1090a0 0x7fadbc19ab20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.488+0000 7fadc2c00640 1 --2- 172.21.15.123:0/16125719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadbc103f50 0x7fadbc19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.488+0000 7fadc2c00640 1 -- 172.21.15.123:0/16125719 >> 172.21.15.123:0/16125719 conn(0x7fadbc0ff9e0 msgr2=0x7fadbc100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:53.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.488+0000 7fadc2c00640 1 -- 172.21.15.123:0/16125719 shutdown_connections 2024-09-19T04:46:53.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.488+0000 7fadc2c00640 1 -- 172.21.15.123:0/16125719 wait complete. 2024-09-19T04:46:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:53 smithi193 ceph-mon[35592]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.0 KiB/s wr, 20 op/s 2024-09-19T04:46:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:53 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3435383657' entity='client.admin' 2024-09-19T04:46:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:53 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:46:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:53 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:46:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:53 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:46:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:53 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:53 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3389864293' entity='client.admin' 2024-09-19T04:46:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:53 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:46:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:53 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:46:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:53 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:46:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:53 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:53 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' 2024-09-19T04:46:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:53 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:46:53.492 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:53 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:46:53.492 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:53 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:46:53.492 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:53 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:53.504 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:53 smithi123 ceph-mon[30511]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.0 KiB/s wr, 20 op/s 2024-09-19T04:46:53.504 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:53 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3435383657' entity='client.admin' 2024-09-19T04:46:53.504 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:53 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:46:53.504 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:53 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:46:53.504 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:53 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:46:53.504 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:53 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:53.504 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:53 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3389864293' entity='client.admin' 2024-09-19T04:46:53.504 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:53 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:46:53.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:53 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:46:53.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:53 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:46:53.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:53 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:53.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:53 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' 2024-09-19T04:46:53.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:53 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:46:53.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:53 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:46:53.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:53 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:46:53.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:53 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:53.506 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-19T04:46:53.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:53 smithi183 ceph-mon[35333]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.0 KiB/s wr, 20 op/s 2024-09-19T04:46:53.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:53 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3435383657' entity='client.admin' 2024-09-19T04:46:53.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:53 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:46:53.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:53 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:46:53.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:53 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:46:53.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:53 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:53.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:53 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3389864293' entity='client.admin' 2024-09-19T04:46:53.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:53 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:46:53.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:53 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:46:53.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:53 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:46:53.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:53 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:53.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:53 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' 2024-09-19T04:46:53.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:53 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:46:53.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:53 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:46:53.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:53 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:46:53.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:53 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:46:53.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.659+0000 7f1fde200640 1 -- 172.21.15.123:0/3605937065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fd8103b00 msgr2=0x7f1fd8103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:53.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.659+0000 7f1fde200640 1 --2- 172.21.15.123:0/3605937065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fd8103b00 0x7f1fd8103f60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009930 tx=0x7f1fc402a420 comp rx=0 tx=0).stop 2024-09-19T04:46:53.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.659+0000 7f1fde200640 1 -- 172.21.15.123:0/3605937065 shutdown_connections 2024-09-19T04:46:53.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.659+0000 7f1fde200640 1 --2- 172.21.15.123:0/3605937065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fd81044a0 0x7f1fd81109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.659+0000 7f1fde200640 1 --2- 172.21.15.123:0/3605937065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fd8103b00 0x7f1fd8103f60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.659+0000 7f1fde200640 1 --2- 172.21.15.123:0/3605937065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd8109ba0 0x7f1fd8109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.659+0000 7f1fde200640 1 -- 172.21.15.123:0/3605937065 >> 172.21.15.123:0/3605937065 conn(0x7f1fd80ff7c0 msgr2=0x7f1fd8101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:53.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.659+0000 7f1fde200640 1 -- 172.21.15.123:0/3605937065 shutdown_connections 2024-09-19T04:46:53.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.659+0000 7f1fde200640 1 -- 172.21.15.123:0/3605937065 wait complete. 2024-09-19T04:46:53.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.660+0000 7f1fde200640 1 Processor -- start 2024-09-19T04:46:53.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.660+0000 7f1fde200640 1 -- start start 2024-09-19T04:46:53.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.661+0000 7f1fde200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fd8103b00 0x7f1fd819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:53.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.661+0000 7f1fde200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fd81044a0 0x7f1fd819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:53.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.661+0000 7f1fde200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd8109ba0 0x7f1fd819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:53.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.661+0000 7f1fde200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd8113fa0 con 0x7f1fd8103b00 2024-09-19T04:46:53.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.661+0000 7f1fde200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd8113e20 con 0x7f1fd8109ba0 2024-09-19T04:46:53.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.661+0000 7f1fde200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd8114120 con 0x7f1fd81044a0 2024-09-19T04:46:53.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.661+0000 7f1fdce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fd8103b00 0x7f1fd819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:53.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.661+0000 7f1fdce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fd8103b00 0x7f1fd819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60464/0 (socket says 172.21.15.123:60464) 2024-09-19T04:46:53.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.661+0000 7f1fdce00640 1 -- 172.21.15.123:0/2750294205 learned_addr learned my addr 172.21.15.123:0/2750294205 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:53.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.661+0000 7f1fdd800640 1 --2- 172.21.15.123:0/2750294205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd8109ba0 0x7f1fd819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:53.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.662+0000 7f1fd7e00640 1 --2- 172.21.15.123:0/2750294205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fd81044a0 0x7f1fd819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:53.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.662+0000 7f1fdce00640 1 -- 172.21.15.123:0/2750294205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fd81044a0 msgr2=0x7f1fd819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:53.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.662+0000 7f1fdce00640 1 --2- 172.21.15.123:0/2750294205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fd81044a0 0x7f1fd819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.662+0000 7f1fdce00640 1 -- 172.21.15.123:0/2750294205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd8109ba0 msgr2=0x7f1fd819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:53.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.662+0000 7f1fdce00640 1 --2- 172.21.15.123:0/2750294205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd8109ba0 0x7f1fd819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.662+0000 7f1fdce00640 1 -- 172.21.15.123:0/2750294205 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd819f460 con 0x7f1fd8103b00 2024-09-19T04:46:53.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.662+0000 7f1fd7e00640 1 --2- 172.21.15.123:0/2750294205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fd81044a0 0x7f1fd819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:46:53.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.662+0000 7f1fdce00640 1 --2- 172.21.15.123:0/2750294205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fd8103b00 0x7f1fd819a470 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00e9f0 tx=0x7f1fcc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:53.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.662+0000 7f1fd5600640 1 -- 172.21.15.123:0/2750294205 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc00cd80 con 0x7f1fd8103b00 2024-09-19T04:46:53.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.662+0000 7f1fde200640 1 -- 172.21.15.123:0/2750294205 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd819f750 con 0x7f1fd8103b00 2024-09-19T04:46:53.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.662+0000 7f1fd5600640 1 -- 172.21.15.123:0/2750294205 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fcc010910 con 0x7f1fd8103b00 2024-09-19T04:46:53.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.662+0000 7f1fd5600640 1 -- 172.21.15.123:0/2750294205 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc0188f0 con 0x7f1fd8103b00 2024-09-19T04:46:53.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.662+0000 7f1fde200640 1 -- 172.21.15.123:0/2750294205 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd81a6fe0 con 0x7f1fd8103b00 2024-09-19T04:46:53.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.664+0000 7f1fde200640 1 -- 172.21.15.123:0/2750294205 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd8103f60 con 0x7f1fd8103b00 2024-09-19T04:46:53.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.666+0000 7f1fd5600640 1 -- 172.21.15.123:0/2750294205 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 138583+0+0 (secure 0 0 0) 0x7f1fcc010ac0 con 0x7f1fd8103b00 2024-09-19T04:46:53.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.667+0000 7f1fd5600640 1 --2- 172.21.15.123:0/2750294205 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1fc00a34d0 0x7f1fc00a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:53.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.667+0000 7f1fd5600640 1 -- 172.21.15.123:0/2750294205 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f1fcc0d03f0 con 0x7f1fd8103b00 2024-09-19T04:46:53.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.667+0000 7f1fd7e00640 1 --2- 172.21.15.123:0/2750294205 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1fc00a34d0 0x7f1fc00a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:53.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.668+0000 7f1fd7e00640 1 --2- 172.21.15.123:0/2750294205 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1fc00a34d0 0x7f1fc00a5990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1fc402a940 tx=0x7f1fc4005fd0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:53.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.671+0000 7f1fd5600640 1 -- 172.21.15.123:0/2750294205 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1fcc09cb50 con 0x7f1fd8103b00 2024-09-19T04:46:53.705 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:46:53.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.880+0000 7f1fde200640 1 -- 172.21.15.123:0/2750294205 --> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f1fd8062630 con 0x7f1fc00a34d0 2024-09-19T04:46:53.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.948+0000 7f1fd5600640 1 -- 172.21.15.123:0/2750294205 <== mgr.25133 v2:172.21.15.123:6800/1578549821 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f1fd8062630 con 0x7f1fc00a34d0 2024-09-19T04:46:53.950 INFO:teuthology.orchestra.run.smithi123.stdout:Retention added to path / 2024-09-19T04:46:53.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.951+0000 7f1fde200640 1 -- 172.21.15.123:0/2750294205 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1fc00a34d0 msgr2=0x7f1fc00a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:53.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.951+0000 7f1fde200640 1 --2- 172.21.15.123:0/2750294205 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1fc00a34d0 0x7f1fc00a5990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1fc402a940 tx=0x7f1fc4005fd0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.951+0000 7f1fde200640 1 -- 172.21.15.123:0/2750294205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fd8103b00 msgr2=0x7f1fd819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:53.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.951+0000 7f1fde200640 1 --2- 172.21.15.123:0/2750294205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fd8103b00 0x7f1fd819a470 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00e9f0 tx=0x7f1fcc00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.951+0000 7f1fde200640 1 -- 172.21.15.123:0/2750294205 shutdown_connections 2024-09-19T04:46:53.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.951+0000 7f1fde200640 1 --2- 172.21.15.123:0/2750294205 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1fc00a34d0 0x7f1fc00a5990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.951+0000 7f1fde200640 1 --2- 172.21.15.123:0/2750294205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd8109ba0 0x7f1fd819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.951+0000 7f1fde200640 1 --2- 172.21.15.123:0/2750294205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fd81044a0 0x7f1fd819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.951+0000 7f1fde200640 1 --2- 172.21.15.123:0/2750294205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fd8103b00 0x7f1fd819a470 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:53.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.951+0000 7f1fde200640 1 -- 172.21.15.123:0/2750294205 >> 172.21.15.123:0/2750294205 conn(0x7f1fd80ff7c0 msgr2=0x7f1fd80fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:53.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.951+0000 7f1fde200640 1 -- 172.21.15.123:0/2750294205 shutdown_connections 2024-09-19T04:46:53.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:53.951+0000 7f1fde200640 1 -- 172.21.15.123:0/2750294205 wait complete. 2024-09-19T04:46:53.970 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T04:46:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:46:54.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.147+0000 7f2b85000640 1 -- 172.21.15.123:0/1286837054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b80074eb0 msgr2=0x7f2b801125a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.147+0000 7f2b85000640 1 --2- 172.21.15.123:0/1286837054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b80074eb0 0x7f2b801125a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009900 tx=0x7f2b6c02a480 comp rx=0 tx=0).stop 2024-09-19T04:46:54.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.148+0000 7f2b85000640 1 -- 172.21.15.123:0/1286837054 shutdown_connections 2024-09-19T04:46:54.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.148+0000 7f2b85000640 1 --2- 172.21.15.123:0/1286837054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b80112ae0 0x7f2b80114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.148+0000 7f2b85000640 1 --2- 172.21.15.123:0/1286837054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b80074eb0 0x7f2b801125a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.148+0000 7f2b85000640 1 --2- 172.21.15.123:0/1286837054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b80074590 0x7f2b80074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.148+0000 7f2b85000640 1 -- 172.21.15.123:0/1286837054 >> 172.21.15.123:0/1286837054 conn(0x7f2b800fd7d0 msgr2=0x7f2b800ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:54.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.148+0000 7f2b85000640 1 -- 172.21.15.123:0/1286837054 shutdown_connections 2024-09-19T04:46:54.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.148+0000 7f2b85000640 1 -- 172.21.15.123:0/1286837054 wait complete. 2024-09-19T04:46:54.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.149+0000 7f2b85000640 1 Processor -- start 2024-09-19T04:46:54.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.150+0000 7f2b85000640 1 -- start start 2024-09-19T04:46:54.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.150+0000 7f2b85000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b80074590 0x7f2b801a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.150+0000 7f2b85000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b80074eb0 0x7f2b801a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.150+0000 7f2b85000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b80112ae0 0x7f2b8019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.150+0000 7f2b85000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b801184e0 con 0x7f2b80112ae0 2024-09-19T04:46:54.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.150+0000 7f2b85000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b80118360 con 0x7f2b80074eb0 2024-09-19T04:46:54.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.150+0000 7f2b85000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2b80118660 con 0x7f2b80074590 2024-09-19T04:46:54.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.150+0000 7f2b7e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b80112ae0 0x7f2b8019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.150+0000 7f2b7de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b80074590 0x7f2b801a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.150+0000 7f2b7d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b80074eb0 0x7f2b801a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.150+0000 7f2b7de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b80074590 0x7f2b801a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:36422/0 (socket says 172.21.15.123:36422) 2024-09-19T04:46:54.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.150+0000 7f2b7d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b80074eb0 0x7f2b801a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:35680/0 (socket says 172.21.15.123:35680) 2024-09-19T04:46:54.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.150+0000 7f2b7de00640 1 -- 172.21.15.123:0/2719885627 learned_addr learned my addr 172.21.15.123:0/2719885627 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:54.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.151+0000 7f2b7de00640 1 -- 172.21.15.123:0/2719885627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b80074eb0 msgr2=0x7f2b801a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.151+0000 7f2b7de00640 1 --2- 172.21.15.123:0/2719885627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b80074eb0 0x7f2b801a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.151+0000 7f2b7de00640 1 -- 172.21.15.123:0/2719885627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b80112ae0 msgr2=0x7f2b8019e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.151+0000 7f2b7de00640 1 --2- 172.21.15.123:0/2719885627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b80112ae0 0x7f2b8019e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.151+0000 7f2b7de00640 1 -- 172.21.15.123:0/2719885627 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8019f030 con 0x7f2b80074590 2024-09-19T04:46:54.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.151+0000 7f2b7d400640 1 --2- 172.21.15.123:0/2719885627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b80074eb0 0x7f2b801a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:54.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.151+0000 7f2b7de00640 1 --2- 172.21.15.123:0/2719885627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b80074590 0x7f2b801a46b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000e9f0 tx=0x7f2b7000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:54.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.152+0000 7f2b76a00640 1 -- 172.21.15.123:0/2719885627 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7000cd80 con 0x7f2b80074590 2024-09-19T04:46:54.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.152+0000 7f2b76a00640 1 -- 172.21.15.123:0/2719885627 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b70010910 con 0x7f2b80074590 2024-09-19T04:46:54.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.152+0000 7f2b85000640 1 -- 172.21.15.123:0/2719885627 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8019f320 con 0x7f2b80074590 2024-09-19T04:46:54.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.152+0000 7f2b76a00640 1 -- 172.21.15.123:0/2719885627 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70018990 con 0x7f2b80074590 2024-09-19T04:46:54.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.152+0000 7f2b85000640 1 -- 172.21.15.123:0/2719885627 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b801ab630 con 0x7f2b80074590 2024-09-19T04:46:54.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.153+0000 7f2b85000640 1 -- 172.21.15.123:0/2719885627 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b80075cd0 con 0x7f2b80074590 2024-09-19T04:46:54.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.156+0000 7f2b76a00640 1 -- 172.21.15.123:0/2719885627 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 138583+0+0 (secure 0 0 0) 0x7f2b70010ac0 con 0x7f2b80074590 2024-09-19T04:46:54.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.158+0000 7f2b76a00640 1 --2- 172.21.15.123:0/2719885627 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2b640a3600 0x7f2b640a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.158+0000 7f2b76a00640 1 -- 172.21.15.123:0/2719885627 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f2b700d0660 con 0x7f2b80074590 2024-09-19T04:46:54.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.158+0000 7f2b7d400640 1 --2- 172.21.15.123:0/2719885627 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2b640a3600 0x7f2b640a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.158+0000 7f2b76a00640 1 -- 172.21.15.123:0/2719885627 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2b7009c770 con 0x7f2b80074590 2024-09-19T04:46:54.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.159+0000 7f2b7d400640 1 --2- 172.21.15.123:0/2719885627 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2b640a3600 0x7f2b640a5ac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c000c00 tx=0x7f2b6c031510 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:54.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:54 smithi123 ceph-mon[30511]: scrub queued for path: / 2024-09-19T04:46:54.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:54 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:46:54.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:54 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:46:54.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:54 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:46:54.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:54 smithi123 ceph-mon[30511]: from='client.25301 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:54.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.368+0000 7f2b85000640 1 -- 172.21.15.123:0/2719885627 --> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f2b800ff4c0 con 0x7f2b640a3600 2024-09-19T04:46:54.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.376+0000 7f2b76a00640 1 -- 172.21.15.123:0/2719885627 <== mgr.25133 v2:172.21.15.123:6800/1578549821 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f2b800ff4c0 con 0x7f2b640a3600 2024-09-19T04:46:54.377 INFO:teuthology.orchestra.run.smithi123.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T04:46:53", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-19T04:46:54.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.378+0000 7f2b85000640 1 -- 172.21.15.123:0/2719885627 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2b640a3600 msgr2=0x7f2b640a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.378+0000 7f2b85000640 1 --2- 172.21.15.123:0/2719885627 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2b640a3600 0x7f2b640a5ac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c000c00 tx=0x7f2b6c031510 comp rx=0 tx=0).stop 2024-09-19T04:46:54.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.378+0000 7f2b85000640 1 -- 172.21.15.123:0/2719885627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b80074590 msgr2=0x7f2b801a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.378+0000 7f2b85000640 1 --2- 172.21.15.123:0/2719885627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b80074590 0x7f2b801a46b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000e9f0 tx=0x7f2b7000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.378+0000 7f2b85000640 1 -- 172.21.15.123:0/2719885627 shutdown_connections 2024-09-19T04:46:54.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.378+0000 7f2b85000640 1 --2- 172.21.15.123:0/2719885627 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2b640a3600 0x7f2b640a5ac0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.378+0000 7f2b85000640 1 --2- 172.21.15.123:0/2719885627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b80112ae0 0x7f2b8019e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.378+0000 7f2b85000640 1 --2- 172.21.15.123:0/2719885627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b80074eb0 0x7f2b801a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.378+0000 7f2b85000640 1 --2- 172.21.15.123:0/2719885627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b80074590 0x7f2b801a46b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.378+0000 7f2b85000640 1 -- 172.21.15.123:0/2719885627 >> 172.21.15.123:0/2719885627 conn(0x7f2b800fd7d0 msgr2=0x7f2b800fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:54.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.378+0000 7f2b85000640 1 -- 172.21.15.123:0/2719885627 shutdown_connections 2024-09-19T04:46:54.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.379+0000 7f2b85000640 1 -- 172.21.15.123:0/2719885627 wait complete. 2024-09-19T04:46:54.397 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T04:46:54.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.558+0000 7fd3dfe00640 1 -- 172.21.15.123:0/2718780386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3e01018b0 msgr2=0x7fd3e0101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.558+0000 7fd3dfe00640 1 --2- 172.21.15.123:0/2718780386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3e01018b0 0x7fd3e0101d10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009930 tx=0x7fd3cc02a420 comp rx=0 tx=0).stop 2024-09-19T04:46:54.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.558+0000 7fd3dfe00640 1 -- 172.21.15.123:0/2718780386 shutdown_connections 2024-09-19T04:46:54.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.558+0000 7fd3dfe00640 1 --2- 172.21.15.123:0/2718780386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3e0102250 0x7fd3e010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.558+0000 7fd3dfe00640 1 --2- 172.21.15.123:0/2718780386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3e01018b0 0x7fd3e0101d10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.558+0000 7fd3dfe00640 1 --2- 172.21.15.123:0/2718780386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3e01078c0 0x7fd3e0107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.558+0000 7fd3dfe00640 1 -- 172.21.15.123:0/2718780386 >> 172.21.15.123:0/2718780386 conn(0x7fd3e00fd5d0 msgr2=0x7fd3e00ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:54.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.559+0000 7fd3dfe00640 1 -- 172.21.15.123:0/2718780386 shutdown_connections 2024-09-19T04:46:54.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.559+0000 7fd3dfe00640 1 -- 172.21.15.123:0/2718780386 wait complete. 2024-09-19T04:46:54.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.560+0000 7fd3dfe00640 1 Processor -- start 2024-09-19T04:46:54.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.560+0000 7fd3dfe00640 1 -- start start 2024-09-19T04:46:54.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.560+0000 7fd3dfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3e01018b0 0x7fd3e019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.560+0000 7fd3dfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3e0102250 0x7fd3e019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.560+0000 7fd3dfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3e01078c0 0x7fd3e019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.560+0000 7fd3dfe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e0111d00 con 0x7fd3e0102250 2024-09-19T04:46:54.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.560+0000 7fd3dfe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e0111b80 con 0x7fd3e01018b0 2024-09-19T04:46:54.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.560+0000 7fd3dfe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e0111e80 con 0x7fd3e01078c0 2024-09-19T04:46:54.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.560+0000 7fd3de000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3e0102250 0x7fd3e019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.560+0000 7fd3dea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3e01018b0 0x7fd3e019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.560+0000 7fd3dea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3e01018b0 0x7fd3e019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:35700/0 (socket says 172.21.15.123:35700) 2024-09-19T04:46:54.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.560+0000 7fd3dea00640 1 -- 172.21.15.123:0/1113782777 learned_addr learned my addr 172.21.15.123:0/1113782777 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:54.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.560+0000 7fd3df400640 1 --2- 172.21.15.123:0/1113782777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3e01078c0 0x7fd3e019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.561+0000 7fd3dea00640 1 -- 172.21.15.123:0/1113782777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3e01078c0 msgr2=0x7fd3e019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.561+0000 7fd3dea00640 1 --2- 172.21.15.123:0/1113782777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3e01078c0 0x7fd3e019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.561+0000 7fd3dea00640 1 -- 172.21.15.123:0/1113782777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3e0102250 msgr2=0x7fd3e019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.561+0000 7fd3dea00640 1 --2- 172.21.15.123:0/1113782777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3e0102250 0x7fd3e019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.561+0000 7fd3dea00640 1 -- 172.21.15.123:0/1113782777 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e019f4c0 con 0x7fd3e01018b0 2024-09-19T04:46:54.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.561+0000 7fd3dea00640 1 --2- 172.21.15.123:0/1113782777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3e01018b0 0x7fd3e019a4d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000e9f0 tx=0x7fd3d000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:54.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.562+0000 7fd3d7400640 1 -- 172.21.15.123:0/1113782777 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d000cd80 con 0x7fd3e01018b0 2024-09-19T04:46:54.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.562+0000 7fd3dfe00640 1 -- 172.21.15.123:0/1113782777 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e019f7b0 con 0x7fd3e01018b0 2024-09-19T04:46:54.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.562+0000 7fd3d7400640 1 -- 172.21.15.123:0/1113782777 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3d0005500 con 0x7fd3e01018b0 2024-09-19T04:46:54.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.562+0000 7fd3d7400640 1 -- 172.21.15.123:0/1113782777 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0010690 con 0x7fd3e01018b0 2024-09-19T04:46:54.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.562+0000 7fd3dfe00640 1 -- 172.21.15.123:0/1113782777 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e01a7040 con 0x7fd3e01018b0 2024-09-19T04:46:54.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.563+0000 7fd3dfe00640 1 -- 172.21.15.123:0/1113782777 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e0101d10 con 0x7fd3e01018b0 2024-09-19T04:46:54.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.565+0000 7fd3d7400640 1 -- 172.21.15.123:0/1113782777 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 138583+0+0 (secure 0 0 0) 0x7fd3d00046c0 con 0x7fd3e01018b0 2024-09-19T04:46:54.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.566+0000 7fd3d7400640 1 --2- 172.21.15.123:0/1113782777 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd3c40a3550 0x7fd3c40a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.566+0000 7fd3d7400640 1 -- 172.21.15.123:0/1113782777 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd3d00d0290 con 0x7fd3e01018b0 2024-09-19T04:46:54.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.566+0000 7fd3de000640 1 --2- 172.21.15.123:0/1113782777 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd3c40a3550 0x7fd3c40a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.567+0000 7fd3de000640 1 --2- 172.21.15.123:0/1113782777 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd3c40a3550 0x7fd3c40a5a10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009900 tx=0x7fd3cc0058d0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:54.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.570+0000 7fd3d7400640 1 -- 172.21.15.123:0/1113782777 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd3d009c9f0 con 0x7fd3e01018b0 2024-09-19T04:46:54.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:54 smithi183 ceph-mon[35333]: scrub queued for path: / 2024-09-19T04:46:54.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:54 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:46:54.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:54 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:46:54.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:54 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:46:54.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:54 smithi183 ceph-mon[35333]: from='client.25301 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:54.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.711+0000 7fb03f600640 1 -- 172.21.15.123:0/501294748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0381049c0 msgr2=0x7fb038108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.711+0000 7fb03f600640 1 --2- 172.21.15.123:0/501294748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0381049c0 0x7fb038108a10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb024009900 tx=0x7fb02402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.711+0000 7fb03f600640 1 -- 172.21.15.123:0/501294748 shutdown_connections 2024-09-19T04:46:54.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.711+0000 7fb03f600640 1 --2- 172.21.15.123:0/501294748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb038109140 0x7fb038110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.711+0000 7fb03f600640 1 --2- 172.21.15.123:0/501294748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0381049c0 0x7fb038108a10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.711+0000 7fb03f600640 1 --2- 172.21.15.123:0/501294748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb038104010 0x7fb0381043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.711+0000 7fb03f600640 1 -- 172.21.15.123:0/501294748 >> 172.21.15.123:0/501294748 conn(0x7fb0380ffa50 msgr2=0x7fb038101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:54.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.711+0000 7fb03f600640 1 -- 172.21.15.123:0/501294748 shutdown_connections 2024-09-19T04:46:54.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.711+0000 7fb03f600640 1 -- 172.21.15.123:0/501294748 wait complete. 2024-09-19T04:46:54.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.712+0000 7fb03f600640 1 Processor -- start 2024-09-19T04:46:54.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.712+0000 7fb03f600640 1 -- start start 2024-09-19T04:46:54.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.713+0000 7fb03f600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb038104010 0x7fb03819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.713+0000 7fb03f600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0381049c0 0x7fb03819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.713+0000 7fb03f600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb038109140 0x7fb03819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.713+0000 7fb03f600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb038114190 con 0x7fb038109140 2024-09-19T04:46:54.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.713+0000 7fb03f600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb038114010 con 0x7fb038104010 2024-09-19T04:46:54.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.713+0000 7fb03f600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb038114310 con 0x7fb0381049c0 2024-09-19T04:46:54.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.713+0000 7fb03d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb038109140 0x7fb03819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.713+0000 7fb037e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0381049c0 0x7fb03819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.713+0000 7fb03cc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb038104010 0x7fb03819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.713+0000 7fb037e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0381049c0 0x7fb03819ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:36468/0 (socket says 172.21.15.123:36468) 2024-09-19T04:46:54.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.713+0000 7fb037e00640 1 -- 172.21.15.123:0/1619951718 learned_addr learned my addr 172.21.15.123:0/1619951718 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:54.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.714+0000 7fb037e00640 1 -- 172.21.15.123:0/1619951718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb038104010 msgr2=0x7fb03819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.714+0000 7fb037e00640 1 --2- 172.21.15.123:0/1619951718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb038104010 0x7fb03819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.714+0000 7fb037e00640 1 -- 172.21.15.123:0/1619951718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb038109140 msgr2=0x7fb03819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.714+0000 7fb037e00640 1 --2- 172.21.15.123:0/1619951718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb038109140 0x7fb03819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.714+0000 7fb037e00640 1 -- 172.21.15.123:0/1619951718 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03819f6a0 con 0x7fb0381049c0 2024-09-19T04:46:54.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.714+0000 7fb037e00640 1 --2- 172.21.15.123:0/1619951718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0381049c0 0x7fb03819ac30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb024004770 tx=0x7fb024003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:54.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.714+0000 7fb035600640 1 -- 172.21.15.123:0/1619951718 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02403f070 con 0x7fb0381049c0 2024-09-19T04:46:54.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.714+0000 7fb035600640 1 -- 172.21.15.123:0/1619951718 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb024031ca0 con 0x7fb0381049c0 2024-09-19T04:46:54.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.714+0000 7fb03f600640 1 -- 172.21.15.123:0/1619951718 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03819f930 con 0x7fb0381049c0 2024-09-19T04:46:54.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.715+0000 7fb035600640 1 -- 172.21.15.123:0/1619951718 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024035030 con 0x7fb0381049c0 2024-09-19T04:46:54.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.715+0000 7fb03f600640 1 -- 172.21.15.123:0/1619951718 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0381a7270 con 0x7fb0381049c0 2024-09-19T04:46:54.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.718+0000 7fb0421d0740 1 Processor -- start 2024-09-19T04:46:54.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.719+0000 7fb0421d0740 1 -- start start 2024-09-19T04:46:54.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.719+0000 7fb0421d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51c0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.719+0000 7fb0421d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51c14b0 0x5641c51cb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.719+0000 7fb0421d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51cbd70 0x5641c51ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.719+0000 7fb0421d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5641c51273e0 con 0x5641c51c0b90 2024-09-19T04:46:54.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.719+0000 7fb0421d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5641c50ced80 con 0x5641c51cbd70 2024-09-19T04:46:54.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.719+0000 7fb0421d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5641c51233a0 con 0x5641c51c14b0 2024-09-19T04:46:54.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.719+0000 7fb035600640 1 -- 172.21.15.123:0/1619951718 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 138583+0+0 (secure 0 0 0) 0x7fb024042070 con 0x7fb0381049c0 2024-09-19T04:46:54.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.720+0000 7fb03d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51c0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.720+0000 7fb03d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51c0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60536/0 (socket says 172.21.15.123:60536) 2024-09-19T04:46:54.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.720+0000 7fb03d600640 1 -- 172.21.15.123:0/3884019524 learned_addr learned my addr 172.21.15.123:0/3884019524 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:54.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.720+0000 7fb037e00640 1 --2- 172.21.15.123:0/3884019524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51cbd70 0x5641c51ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.720+0000 7fb03cc00640 1 --2- 172.21.15.123:0/3884019524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51c14b0 0x5641c51cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.720+0000 7fb037e00640 1 -- 172.21.15.123:0/3884019524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51c14b0 msgr2=0x5641c51cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:54.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.721+0000 7fb037e00640 1 --2- 172.21.15.123:0/3884019524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51c14b0 0x5641c51cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.721+0000 7fb037e00640 1 -- 172.21.15.123:0/3884019524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 msgr2=0x5641c51c0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:54.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.721+0000 7fb037e00640 1 --2- 172.21.15.123:0/3884019524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51c0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.721+0000 7fb037e00640 1 -- 172.21.15.123:0/3884019524 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641c50edae0 con 0x5641c51cbd70 2024-09-19T04:46:54.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.721+0000 7fb03d600640 1 --2- 172.21.15.123:0/3884019524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51c0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:54.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.721+0000 7fb03cc00640 1 --2- 172.21.15.123:0/3884019524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51c14b0 0x5641c51cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:46:54.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.721+0000 7fb037e00640 1 --2- 172.21.15.123:0/3884019524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51cbd70 0x5641c51ce160 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb0240edab0 tx=0x7fb0240f66f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30e2aa489ea907e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:54.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.722+0000 7fb035600640 1 --2- 172.21.15.123:0/1619951718 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb0200a34d0 0x7fb0200a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.722+0000 7fb029c00640 1 -- 172.21.15.123:0/3884019524 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0240b8a50 con 0x5641c51cbd70 2024-09-19T04:46:54.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.722+0000 7fb029c00640 1 -- 172.21.15.123:0/3884019524 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0240fd030 con 0x5641c51cbd70 2024-09-19T04:46:54.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.722+0000 7fb03d600640 1 --2- 172.21.15.123:0/1619951718 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb0200a34d0 0x7fb0200a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.722+0000 7fb035600640 1 -- 172.21.15.123:0/1619951718 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fb0240f0510 con 0x7fb0381049c0 2024-09-19T04:46:54.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.723+0000 7fb029c00640 1 -- 172.21.15.123:0/3884019524 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0240bbc00 con 0x5641c51cbd70 2024-09-19T04:46:54.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.723+0000 7fb0421d0740 1 -- 172.21.15.123:0/3884019524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51cbd70 msgr2=0x5641c51ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.723+0000 7fb0421d0740 1 --2- 172.21.15.123:0/3884019524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51cbd70 0x5641c51ce160 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb0240edab0 tx=0x7fb0240f66f0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.723+0000 7fb03d600640 1 --2- 172.21.15.123:0/1619951718 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb0200a34d0 0x7fb0200a5990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb030009fd0 tx=0x7fb030009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:54.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.723+0000 7fb0421d0740 1 -- 172.21.15.123:0/3884019524 shutdown_connections 2024-09-19T04:46:54.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.723+0000 7fb0421d0740 1 --2- 172.21.15.123:0/3884019524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51cbd70 0x5641c51ce160 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.723+0000 7fb0421d0740 1 --2- 172.21.15.123:0/3884019524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51c14b0 0x5641c51cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.723+0000 7fb0421d0740 1 --2- 172.21.15.123:0/3884019524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51c0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.723+0000 7fb0421d0740 1 -- 172.21.15.123:0/3884019524 >> 172.21.15.123:0/3884019524 conn(0x5641c51bd120 msgr2=0x5641c51bd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:54.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.723+0000 7fb0421d0740 1 -- 172.21.15.123:0/3884019524 shutdown_connections 2024-09-19T04:46:54.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.723+0000 7fb0421d0740 1 -- 172.21.15.123:0/3884019524 wait complete. 2024-09-19T04:46:54.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.724+0000 7fb0421d0740 1 -- start start 2024-09-19T04:46:54.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.724+0000 7fb0421d0740 1 Processor -- start 2024-09-19T04:46:54.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.725+0000 7fb0421d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.725+0000 7fb0421d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51c14b0 0x5641c51c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.725+0000 7fb0421d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51cbd70 0x5641c51c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.725+0000 7fb0421d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5641c51ce700 con 0x5641c51c0b90 2024-09-19T04:46:54.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.725+0000 7fb0421d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5641c50e20f0 con 0x5641c51c14b0 2024-09-19T04:46:54.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.725+0000 7fb0421d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5641c51ce880 con 0x5641c51cbd70 2024-09-19T04:46:54.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.725+0000 7fb0421d0740 10 client.0 _ll_register_callbacks cb 0x5641c50d39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:54.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.725+0000 7fb03cc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.725+0000 7fb037e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51c14b0 0x5641c51c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.725+0000 7fb03cc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60544/0 (socket says 172.21.15.123:60544) 2024-09-19T04:46:54.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.725+0000 7fb03cc00640 1 -- 172.21.15.123:0/3242141448 learned_addr learned my addr 172.21.15.123:0/3242141448 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:54.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.725+0000 7fb03d600640 1 --2- 172.21.15.123:0/3242141448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51cbd70 0x5641c51c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.726+0000 7fb037e00640 1 -- 172.21.15.123:0/3242141448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51cbd70 msgr2=0x5641c51c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:54.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.726+0000 7fb037e00640 1 --2- 172.21.15.123:0/3242141448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51cbd70 0x5641c51c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.726+0000 7fb037e00640 1 -- 172.21.15.123:0/3242141448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 msgr2=0x5641c51c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.726+0000 7fb037e00640 1 --2- 172.21.15.123:0/3242141448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.726+0000 7fb037e00640 1 -- 172.21.15.123:0/3242141448 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641c4e983c0 con 0x5641c51c14b0 2024-09-19T04:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.726+0000 7fb037e00640 1 --2- 172.21.15.123:0/3242141448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51c14b0 0x5641c51c9050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb0240b9180 tx=0x7fb024009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4884d3d7eece156c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.726+0000 7fb01e000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.726+0000 7fb01e000640 1 -- 172.21.15.123:0/3242141448 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024049070 con 0x5641c51c14b0 2024-09-19T04:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.726+0000 7fb0421d0740 1 -- 172.21.15.123:0/3242141448 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0240fd030 con 0x5641c51c14b0 2024-09-19T04:46:54.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.726+0000 7fb01e000640 1 -- 172.21.15.123:0/3242141448 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fb0240baab0 con 0x5641c51c14b0 2024-09-19T04:46:54.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.727+0000 7fb01e000640 1 -- 172.21.15.123:0/3242141448 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb0240bbf30 con 0x5641c51c14b0 2024-09-19T04:46:54.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.727+0000 7fb0421d0740 10 client.25331 fetch_fsmap learned FSMap version 28 2024-09-19T04:46:54.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.727+0000 7fb0421d0740 1 -- 172.21.15.123:0/3242141448 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641c51ce700 con 0x5641c51c14b0 2024-09-19T04:46:54.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.727+0000 7fb01e000640 1 -- 172.21.15.123:0/3242141448 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0240fd030 con 0x5641c51c14b0 2024-09-19T04:46:54.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.727+0000 7fb01e000640 1 -- 172.21.15.123:0/3242141448 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb0240bbc00 con 0x5641c51c14b0 2024-09-19T04:46:54.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.727+0000 7fb0421d0740 10 client.25331 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:46:54.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.727+0000 7fb0421d0740 10 client.25331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:46:54.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.727+0000 7fb0421d0740 20 client.25331 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:54.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.727+0000 7fb0421d0740 1 --2- 172.21.15.123:0/3242141448 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5641c51e3cb0 0x5641c51e6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.727+0000 7fb0421d0740 4 client.25331 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:46:54.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.727+0000 7fb0421d0740 1 -- 172.21.15.123:0/3242141448 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641c5134950 con 0x5641c51e3cb0 2024-09-19T04:46:54.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.728+0000 7fb03cc00640 1 --2- 172.21.15.123:0/3242141448 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5641c51e3cb0 0x5641c51e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.729+0000 7fb03cc00640 1 --2- 172.21.15.123:0/3242141448 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5641c51e3cb0 0x5641c51e6100 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6a95ef3f88f1716 server_cookie=240d1bb520f9504 in_seq=0 out_seq=0 2024-09-19T04:46:54.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.729+0000 7fb01e000640 10 client.25331 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:46:54.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.729+0000 7fb01e000640 1 -- 172.21.15.123:0/3242141448 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641c5134950 con 0x5641c51e3cb0 2024-09-19T04:46:54.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.729+0000 7fb01e000640 10 client.25331 handle_command_reply: tid=0 2024-09-19T04:46:54.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.730+0000 7fb0421d0740 1 client.25331 shutdown 2024-09-19T04:46:54.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.730+0000 7fb0421d0740 1 -- 172.21.15.123:0/3242141448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51c14b0 msgr2=0x5641c51c9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.730+0000 7fb0421d0740 1 --2- 172.21.15.123:0/3242141448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51c14b0 0x5641c51c9050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb0240b9180 tx=0x7fb024009260 comp rx=0 tx=0).stop 2024-09-19T04:46:54.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.730+0000 7fb0421d0740 1 -- 172.21.15.123:0/3242141448 shutdown_connections 2024-09-19T04:46:54.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.730+0000 7fb0421d0740 1 --2- 172.21.15.123:0/3242141448 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5641c51e3cb0 0x5641c51e6100 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.730+0000 7fb0421d0740 1 --2- 172.21.15.123:0/3242141448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51cbd70 0x5641c51c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.730+0000 7fb0421d0740 1 --2- 172.21.15.123:0/3242141448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51c14b0 0x5641c51c9050 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.730+0000 7fb0421d0740 1 --2- 172.21.15.123:0/3242141448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.730+0000 7fb0421d0740 1 -- 172.21.15.123:0/3242141448 >> 172.21.15.123:0/3242141448 conn(0x5641c51bd120 msgr2=0x5641c51c2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:54.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.730+0000 7fb01e000640 0 client.25331 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:46:54.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.730+0000 7fb0421d0740 1 -- 172.21.15.123:0/3242141448 shutdown_connections 2024-09-19T04:46:54.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.730+0000 7fb0421d0740 1 -- 172.21.15.123:0/3242141448 wait complete. 2024-09-19T04:46:54.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.730+0000 7fb0421d0740 20 client.25331 trim_cache size 0 max 16384 2024-09-19T04:46:54.746 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:54 smithi193 ceph-mon[35592]: scrub queued for path: / 2024-09-19T04:46:54.746 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:54 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:46:54.746 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:54 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:46:54.747 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:54 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:46:54.747 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:54 smithi193 ceph-mon[35592]: from='client.25301 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:54.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.771+0000 7fb0421d0740 1 Processor -- start 2024-09-19T04:46:54.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.771+0000 7fb0421d0740 1 -- start start 2024-09-19T04:46:54.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.772+0000 7fb0421d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51eb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.772+0000 7fb0421d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51c14b0 0x5641c51ebf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.772+0000 7fb0421d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51cbd70 0x5641c51c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.772+0000 7fb0421d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381ab6f0 con 0x5641c51c0b90 2024-09-19T04:46:54.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.772+0000 7fb0421d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381ad230 con 0x5641c51cbd70 2024-09-19T04:46:54.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.772+0000 7fb0421d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381abdc0 con 0x5641c51c14b0 2024-09-19T04:46:54.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.772+0000 7fb037e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51c14b0 0x5641c51ebf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.772+0000 7fb03cc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51eb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.772+0000 7fb03d600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51cbd70 0x5641c51c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.772+0000 7fb037e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51c14b0 0x5641c51ebf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:36498/0 (socket says 172.21.15.123:36498) 2024-09-19T04:46:54.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.772+0000 7fb037e00640 1 -- 172.21.15.123:0/2948643213 learned_addr learned my addr 172.21.15.123:0/2948643213 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:54.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.773+0000 7fb037e00640 1 -- 172.21.15.123:0/2948643213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51cbd70 msgr2=0x5641c51c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:54.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.773+0000 7fb037e00640 1 --2- 172.21.15.123:0/2948643213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51cbd70 0x5641c51c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.773+0000 7fb037e00640 1 -- 172.21.15.123:0/2948643213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 msgr2=0x5641c51eb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:54.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.773+0000 7fb037e00640 1 --2- 172.21.15.123:0/2948643213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51eb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.773+0000 7fb037e00640 1 -- 172.21.15.123:0/2948643213 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641c51bcf50 con 0x5641c51c14b0 2024-09-19T04:46:54.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.773+0000 7fb037e00640 1 --2- 172.21.15.123:0/2948643213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51c14b0 0x5641c51ebf10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb024000c00 tx=0x7fb0240f0870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a32a16d033d9bcae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:54.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.773+0000 7fb029200640 1 -- 172.21.15.123:0/2948643213 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024049030 con 0x5641c51c14b0 2024-09-19T04:46:54.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.773+0000 7fb029200640 1 -- 172.21.15.123:0/2948643213 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0240fd070 con 0x5641c51c14b0 2024-09-19T04:46:54.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.773+0000 7fb029200640 1 -- 172.21.15.123:0/2948643213 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0240bc110 con 0x5641c51c14b0 2024-09-19T04:46:54.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.774+0000 7fb0421d0740 1 -- 172.21.15.123:0/2948643213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51c14b0 msgr2=0x5641c51ebf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.774+0000 7fb0421d0740 1 --2- 172.21.15.123:0/2948643213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51c14b0 0x5641c51ebf10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb024000c00 tx=0x7fb0240f0870 comp rx=0 tx=0).stop 2024-09-19T04:46:54.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.774+0000 7fb0421d0740 1 -- 172.21.15.123:0/2948643213 shutdown_connections 2024-09-19T04:46:54.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.774+0000 7fb0421d0740 1 --2- 172.21.15.123:0/2948643213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51cbd70 0x5641c51c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.775+0000 7fb0421d0740 1 --2- 172.21.15.123:0/2948643213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51c14b0 0x5641c51ebf10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.775+0000 7fb0421d0740 1 --2- 172.21.15.123:0/2948643213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51eb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.775+0000 7fb0421d0740 1 -- 172.21.15.123:0/2948643213 >> 172.21.15.123:0/2948643213 conn(0x5641c51bd120 msgr2=0x5641c51c2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:54.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.775+0000 7fb0421d0740 1 -- 172.21.15.123:0/2948643213 shutdown_connections 2024-09-19T04:46:54.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.775+0000 7fb0421d0740 1 -- 172.21.15.123:0/2948643213 wait complete. 2024-09-19T04:46:54.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.775+0000 7fb0421d0740 1 -- start start 2024-09-19T04:46:54.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.775+0000 7fb0421d0740 1 Processor -- start 2024-09-19T04:46:54.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.776+0000 7fb0421d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.776+0000 7fb0421d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51c14b0 0x5641c51c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.776+0000 7fb0421d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51cbd70 0x5641c51c6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.776+0000 7fb0421d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381acb60 con 0x5641c51c0b90 2024-09-19T04:46:54.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.776+0000 7fb0421d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381ac490 con 0x5641c51c14b0 2024-09-19T04:46:54.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.776+0000 7fb0421d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381a96f0 con 0x5641c51cbd70 2024-09-19T04:46:54.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.776+0000 7fb0421d0740 10 client.0 _ll_register_callbacks cb 0x7fb0240f64b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:46:54.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.776+0000 7fb03cc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.776+0000 7fb03cc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51c5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60552/0 (socket says 172.21.15.123:60552) 2024-09-19T04:46:54.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.776+0000 7fb03cc00640 1 -- 172.21.15.123:0/3571229141 learned_addr learned my addr 172.21.15.123:0/3571229141 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:46:54.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.776+0000 7fb037e00640 1 --2- 172.21.15.123:0/3571229141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51c14b0 0x5641c51c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.776+0000 7fb03d600640 1 --2- 172.21.15.123:0/3571229141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51cbd70 0x5641c51c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.777+0000 7fb037e00640 1 -- 172.21.15.123:0/3571229141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51cbd70 msgr2=0x5641c51c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:54.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.777+0000 7fb037e00640 1 --2- 172.21.15.123:0/3571229141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51cbd70 0x5641c51c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.777+0000 7fb037e00640 1 -- 172.21.15.123:0/3571229141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 msgr2=0x5641c51c5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:46:54.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.777+0000 7fb037e00640 1 --2- 172.21.15.123:0/3571229141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51c5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.777+0000 7fb037e00640 1 -- 172.21.15.123:0/3571229141 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641c51be200 con 0x5641c51c14b0 2024-09-19T04:46:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.777+0000 7fb03cc00640 1 --2- 172.21.15.123:0/3571229141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:46:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.777+0000 7fb037e00640 1 --2- 172.21.15.123:0/3571229141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51c14b0 0x5641c51c6010 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x5641c51c9d80 tx=0x7fb024005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=566266cd532720 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:46:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.777+0000 7fb01e000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:46:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.777+0000 7fb01e000640 1 -- 172.21.15.123:0/3571229141 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024049070 con 0x5641c51c14b0 2024-09-19T04:46:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.777+0000 7fb01e000640 1 -- 172.21.15.123:0/3571229141 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fb0240bbc00 con 0x5641c51c14b0 2024-09-19T04:46:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.777+0000 7fb0421d0740 1 -- 172.21.15.123:0/3571229141 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0240fd070 con 0x5641c51c14b0 2024-09-19T04:46:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.778+0000 7fb01e000640 1 -- 172.21.15.123:0/3571229141 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb0240bc2f0 con 0x5641c51c14b0 2024-09-19T04:46:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.778+0000 7fb0421d0740 10 client.25337 fetch_fsmap learned FSMap version 28 2024-09-19T04:46:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.778+0000 7fb0421d0740 1 -- 172.21.15.123:0/3571229141 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641c51be200 con 0x5641c51c14b0 2024-09-19T04:46:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.778+0000 7fb01e000640 1 -- 172.21.15.123:0/3571229141 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0240fd070 con 0x5641c51c14b0 2024-09-19T04:46:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.778+0000 7fb01e000640 1 -- 172.21.15.123:0/3571229141 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb0240b8e20 con 0x5641c51c14b0 2024-09-19T04:46:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.778+0000 7fb0421d0740 10 client.25337 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:46:54.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.778+0000 7fb0421d0740 10 client.25337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:46:54.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.778+0000 7fb0421d0740 20 client.25337 populate_metadata read hostname 'smithi123' 2024-09-19T04:46:54.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.778+0000 7fb0421d0740 1 --2- 172.21.15.123:0/3571229141 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5641c524edd0 0x5641c52511c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:46:54.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.778+0000 7fb0421d0740 4 client.25337 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:46:54.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.778+0000 7fb0421d0740 1 -- 172.21.15.123:0/3571229141 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5641c512ee60 con 0x5641c524edd0 2024-09-19T04:46:54.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.779+0000 7fb03cc00640 1 --2- 172.21.15.123:0/3571229141 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5641c524edd0 0x5641c52511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:46:54.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.779+0000 7fb03cc00640 1 --2- 172.21.15.123:0/3571229141 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5641c524edd0 0x5641c52511c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d84037acdf110d51 server_cookie=74981aef89c21792 in_seq=0 out_seq=0 2024-09-19T04:46:54.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.779+0000 7fb01e000640 10 client.25337 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:46:54.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.780+0000 7fb01e000640 1 -- 172.21.15.123:0/3571229141 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5641c512ee60 con 0x5641c524edd0 2024-09-19T04:46:54.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.780+0000 7fb01e000640 10 client.25337 handle_command_reply: tid=0 2024-09-19T04:46:54.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.780+0000 7fb0421d0740 1 client.25337 shutdown 2024-09-19T04:46:54.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.780+0000 7fb0421d0740 1 -- 172.21.15.123:0/3571229141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51c14b0 msgr2=0x5641c51c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.780+0000 7fb0421d0740 1 --2- 172.21.15.123:0/3571229141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51c14b0 0x5641c51c6010 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x5641c51c9d80 tx=0x7fb024005b50 comp rx=0 tx=0).stop 2024-09-19T04:46:54.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.780+0000 7fb0421d0740 1 -- 172.21.15.123:0/3571229141 shutdown_connections 2024-09-19T04:46:54.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.780+0000 7fb0421d0740 1 --2- 172.21.15.123:0/3571229141 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5641c524edd0 0x5641c52511c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.780+0000 7fb0421d0740 1 --2- 172.21.15.123:0/3571229141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641c51cbd70 0x5641c51c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.780+0000 7fb0421d0740 1 --2- 172.21.15.123:0/3571229141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641c51c14b0 0x5641c51c6010 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.780+0000 7fb0421d0740 1 --2- 172.21.15.123:0/3571229141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641c51c0b90 0x5641c51c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.780+0000 7fb0421d0740 1 -- 172.21.15.123:0/3571229141 >> 172.21.15.123:0/3571229141 conn(0x5641c51bd120 msgr2=0x5641c51c8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:54.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.780+0000 7fb01e000640 0 client.25337 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:46:54.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.780+0000 7fb0421d0740 1 -- 172.21.15.123:0/3571229141 shutdown_connections 2024-09-19T04:46:54.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.780+0000 7fb0421d0740 1 -- 172.21.15.123:0/3571229141 wait complete. 2024-09-19T04:46:54.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.780+0000 7fb0421d0740 20 client.25337 trim_cache size 0 max 16384 2024-09-19T04:46:54.787 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:46:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:46:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": { 2024-09-19T04:46:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "5acf54a5-fef7-4ca9-9a57-2c90575994be": { 2024-09-19T04:46:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "path": "/", 2024-09-19T04:46:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "tag": "5acf54a5-fef7-4ca9-9a57-2c90575994be", 2024-09-19T04:46:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: "options": "recursive,force" 2024-09-19T04:46:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:46:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:46:54.788 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:46:54.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.781+0000 7fb03f600640 1 -- 172.21.15.123:0/1619951718 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb0200a34d0 msgr2=0x7fb0200a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.781+0000 7fb03f600640 1 --2- 172.21.15.123:0/1619951718 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb0200a34d0 0x7fb0200a5990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb030009fd0 tx=0x7fb030009500 comp rx=0 tx=0).stop 2024-09-19T04:46:54.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.781+0000 7fb03f600640 1 -- 172.21.15.123:0/1619951718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0381049c0 msgr2=0x7fb03819ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.781+0000 7fb03f600640 1 --2- 172.21.15.123:0/1619951718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0381049c0 0x7fb03819ac30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb024004770 tx=0x7fb024003c10 comp rx=0 tx=0).stop 2024-09-19T04:46:54.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.781+0000 7fb03f600640 1 -- 172.21.15.123:0/1619951718 shutdown_connections 2024-09-19T04:46:54.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.781+0000 7fb03f600640 1 --2- 172.21.15.123:0/1619951718 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb0200a34d0 0x7fb0200a5990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.781+0000 7fb03f600640 1 --2- 172.21.15.123:0/1619951718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb038109140 0x7fb03819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.781+0000 7fb03f600640 1 --2- 172.21.15.123:0/1619951718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0381049c0 0x7fb03819ac30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.781+0000 7fb03f600640 1 --2- 172.21.15.123:0/1619951718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb038104010 0x7fb03819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.781+0000 7fb03f600640 1 -- 172.21.15.123:0/1619951718 >> 172.21.15.123:0/1619951718 conn(0x7fb0380ffa50 msgr2=0x7fb038100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:54.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.782+0000 7fb03f600640 1 -- 172.21.15.123:0/1619951718 shutdown_connections 2024-09-19T04:46:54.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.782+0000 7fb03f600640 1 -- 172.21.15.123:0/1619951718 wait complete. 2024-09-19T04:46:54.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.815+0000 7fd3dfe00640 1 -- 172.21.15.123:0/1113782777 --> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fd3e00febb0 con 0x7fd3c40a3550 2024-09-19T04:46:54.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.824+0000 7fd3d7400640 1 -- 172.21.15.123:0/1113782777 <== mgr.25133 v2:172.21.15.123:6800/1578549821 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fd3e00febb0 con 0x7fd3c40a3550 2024-09-19T04:46:54.826 INFO:teuthology.orchestra.run.smithi123.stdout:/ 1m 6m3h 2024-09-19T04:46:54.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.827+0000 7fd3dfe00640 1 -- 172.21.15.123:0/1113782777 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd3c40a3550 msgr2=0x7fd3c40a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.827+0000 7fd3dfe00640 1 --2- 172.21.15.123:0/1113782777 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd3c40a3550 0x7fd3c40a5a10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009900 tx=0x7fd3cc0058d0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.827+0000 7fd3dfe00640 1 -- 172.21.15.123:0/1113782777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3e01018b0 msgr2=0x7fd3e019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:46:54.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.827+0000 7fd3dfe00640 1 --2- 172.21.15.123:0/1113782777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3e01018b0 0x7fd3e019a4d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000e9f0 tx=0x7fd3d000edf0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.827+0000 7fd3dfe00640 1 -- 172.21.15.123:0/1113782777 shutdown_connections 2024-09-19T04:46:54.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.827+0000 7fd3dfe00640 1 --2- 172.21.15.123:0/1113782777 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd3c40a3550 0x7fd3c40a5a10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.827+0000 7fd3dfe00640 1 --2- 172.21.15.123:0/1113782777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3e01078c0 0x7fd3e019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.827+0000 7fd3dfe00640 1 --2- 172.21.15.123:0/1113782777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3e0102250 0x7fd3e019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.827+0000 7fd3dfe00640 1 --2- 172.21.15.123:0/1113782777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3e01018b0 0x7fd3e019a4d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:46:54.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.827+0000 7fd3dfe00640 1 -- 172.21.15.123:0/1113782777 >> 172.21.15.123:0/1113782777 conn(0x7fd3e00fd5d0 msgr2=0x7fd3e010cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:46:54.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.828+0000 7fd3dfe00640 1 -- 172.21.15.123:0/1113782777 shutdown_connections 2024-09-19T04:46:54.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:46:54.828+0000 7fd3dfe00640 1 -- 172.21.15.123:0/1113782777 wait complete. 2024-09-19T04:46:54.847 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-19T04:46:54.908 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-19T04:46:54.919 INFO:teuthology.run_tasks:Running task workunit... 2024-09-19T04:46:54.967 INFO:tasks.workunit:Pulling workunits from ref 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T04:46:54.968 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-19T04:46:54.968 INFO:tasks.workunit:timeout=3h 2024-09-19T04:46:54.968 INFO:tasks.workunit:cleanup=True 2024-09-19T04:46:54.968 DEBUG:teuthology.orchestra.run.smithi123:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-19T04:46:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-19T04:46:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-19T04:46:55.006 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-19T04:46:55.006 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:46:55.006 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-19T04:46:55.006 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-19 04:44:52.757190275 +0000 2024-09-19T04:46:55.006 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-19 04:44:52.757190275 +0000 2024-09-19T04:46:55.006 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-19 04:46:40.659966037 +0000 2024-09-19T04:46:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-19 04:44:52.757190275 +0000 2024-09-19T04:46:55.007 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-19T04:46:55.007 DEBUG:teuthology.orchestra.run.smithi123:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-19T04:46:55.118 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T04:46:55.157 INFO:tasks.workunit.client.0.smithi123.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-19T04:46:55.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:55 smithi123 ceph-mon[30511]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 839 B/s wr, 21 op/s 2024-09-19T04:46:55.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:55 smithi123 ceph-mon[30511]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:55.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:55 smithi123 ceph-mon[30511]: from='client.25315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:55.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:55 smithi183 ceph-mon[35333]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 839 B/s wr, 21 op/s 2024-09-19T04:46:55.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:55 smithi183 ceph-mon[35333]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:55.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:55 smithi183 ceph-mon[35333]: from='client.25315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:55.644 INFO:tasks.cephfs.filesystem:scrub status for tag:5acf54a5-fef7-4ca9-9a57-2c90575994be - {'path': '/', 'tag': '5acf54a5-fef7-4ca9-9a57-2c90575994be', 'options': 'recursive,force'} 2024-09-19T04:46:55.660 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:55 smithi193 ceph-mon[35592]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 839 B/s wr, 21 op/s 2024-09-19T04:46:55.660 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:55 smithi193 ceph-mon[35592]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:55.660 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:55 smithi193 ceph-mon[35592]: from='client.25315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:56.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:56 smithi183 ceph-mon[35333]: from='client.25319 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:56.605 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:56 smithi193 ceph-mon[35592]: from='client.25319 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:56.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:56 smithi123 ceph-mon[30511]: from='client.25319 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:46:57.517 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:57 smithi183 ceph-mon[35333]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 767 B/s wr, 19 op/s 2024-09-19T04:46:57.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:57 smithi123 ceph-mon[30511]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 767 B/s wr, 19 op/s 2024-09-19T04:46:57.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:57 smithi193 ceph-mon[35592]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 767 B/s wr, 19 op/s 2024-09-19T04:46:59.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:46:59 smithi183 ceph-mon[35333]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 767 B/s wr, 19 op/s 2024-09-19T04:46:59.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:46:59 smithi123 ceph-mon[30511]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 767 B/s wr, 19 op/s 2024-09-19T04:46:59.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:46:59 smithi193 ceph-mon[35592]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 767 B/s wr, 19 op/s 2024-09-19T04:47:01.259 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:47:00 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:47:00.961+0000 7f29bb400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:47:01.259 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:47:00 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:47:00.961+0000 7f29bb400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:47:01.259 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:47:00 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:47:00.961+0000 7f29bb400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:47:01.259 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:47:00 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:47:00.961+0000 7f29bb400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:47:01.259 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:47:00 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:47:00.961+0000 7f29bb400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:47:01.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:01 smithi183 ceph-mon[35333]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 78 KiB/s wr, 56 op/s 2024-09-19T04:47:01.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:01 smithi123 ceph-mon[30511]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 78 KiB/s wr, 56 op/s 2024-09-19T04:47:01.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:01 smithi193 ceph-mon[35592]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 78 KiB/s wr, 56 op/s 2024-09-19T04:47:03.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:03 smithi183 ceph-mon[35333]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T04:47:03.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:03 smithi183 ceph-mon[35333]: mgrmap e25: x(active, since 20s), standbys: y, z 2024-09-19T04:47:03.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:03 smithi123 ceph-mon[30511]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T04:47:03.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:03 smithi123 ceph-mon[30511]: mgrmap e25: x(active, since 20s), standbys: y, z 2024-09-19T04:47:03.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:03 smithi193 ceph-mon[35592]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T04:47:03.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:03 smithi193 ceph-mon[35592]: mgrmap e25: x(active, since 20s), standbys: y, z 2024-09-19T04:47:05.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:05 smithi123 ceph-mon[30511]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 99 KiB/s wr, 60 op/s 2024-09-19T04:47:05.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:05 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:47:05.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:05 smithi183 ceph-mon[35333]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 99 KiB/s wr, 60 op/s 2024-09-19T04:47:05.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:05 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:47:05.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:05 smithi193 ceph-mon[35592]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 99 KiB/s wr, 60 op/s 2024-09-19T04:47:05.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:05 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:47:07.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:07 smithi183 ceph-mon[35333]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 99 KiB/s wr, 56 op/s 2024-09-19T04:47:07.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:07 smithi123 ceph-mon[30511]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 99 KiB/s wr, 56 op/s 2024-09-19T04:47:07.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:07 smithi193 ceph-mon[35592]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 99 KiB/s wr, 56 op/s 2024-09-19T04:47:09.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:09 smithi183 ceph-mon[35333]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 99 KiB/s wr, 56 op/s 2024-09-19T04:47:09.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:09 smithi123 ceph-mon[30511]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 99 KiB/s wr, 56 op/s 2024-09-19T04:47:09.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:09 smithi193 ceph-mon[35592]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 99 KiB/s wr, 56 op/s 2024-09-19T04:47:11.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:11 smithi183 ceph-mon[35333]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 100 KiB/s wr, 60 op/s 2024-09-19T04:47:11.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:11 smithi123 ceph-mon[30511]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 100 KiB/s wr, 60 op/s 2024-09-19T04:47:11.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:11 smithi193 ceph-mon[35592]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 100 KiB/s wr, 60 op/s 2024-09-19T04:47:13.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:13 smithi183 ceph-mon[35333]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T04:47:13.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:13 smithi123 ceph-mon[30511]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T04:47:13.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:13 smithi193 ceph-mon[35592]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T04:47:15.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:15 smithi183 ceph-mon[35333]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-19T04:47:15.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:15 smithi123 ceph-mon[30511]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-19T04:47:15.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:15 smithi193 ceph-mon[35592]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-19T04:47:17.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:17 smithi183 ceph-mon[35333]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T04:47:17.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:17 smithi123 ceph-mon[30511]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T04:47:17.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:17 smithi193 ceph-mon[35592]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T04:47:19.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:19 smithi183 ceph-mon[35333]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T04:47:19.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:19 smithi123 ceph-mon[30511]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T04:47:19.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:19 smithi193 ceph-mon[35592]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T04:47:21.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:21 smithi183 ceph-mon[35333]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T04:47:21.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:21 smithi123 ceph-mon[30511]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T04:47:21.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:21 smithi193 ceph-mon[35592]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T04:47:23.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:23 smithi183 ceph-mon[35333]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T04:47:23.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:23 smithi123 ceph-mon[30511]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T04:47:23.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:23 smithi193 ceph-mon[35592]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T04:47:25.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:25 smithi183 ceph-mon[35333]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T04:47:25.646 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:47:25.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:25 smithi123 ceph-mon[30511]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T04:47:25.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:25 smithi193 ceph-mon[35592]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T04:47:26.083 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:47:26.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.722+0000 7f46dfe00640 1 -- 172.21.15.123:0/1561963670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46e01019c0 msgr2=0x7f46e0101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:26.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.722+0000 7f46dfe00640 1 --2- 172.21.15.123:0/1561963670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46e01019c0 0x7f46e0101e20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009860 tx=0x7f46cc02a340 comp rx=0 tx=0).stop 2024-09-19T04:47:26.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.722+0000 7f46dfe00640 1 -- 172.21.15.123:0/1561963670 shutdown_connections 2024-09-19T04:47:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.722+0000 7f46dfe00640 1 --2- 172.21.15.123:0/1561963670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e0102360 0x7f46e010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.722+0000 7f46dfe00640 1 --2- 172.21.15.123:0/1561963670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46e01019c0 0x7f46e0101e20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.722+0000 7f46dfe00640 1 --2- 172.21.15.123:0/1561963670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46e01079c0 0x7f46e0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.722+0000 7f46dfe00640 1 -- 172.21.15.123:0/1561963670 >> 172.21.15.123:0/1561963670 conn(0x7f46e00fd740 msgr2=0x7f46e00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.722+0000 7f46dfe00640 1 -- 172.21.15.123:0/1561963670 shutdown_connections 2024-09-19T04:47:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.722+0000 7f46dfe00640 1 -- 172.21.15.123:0/1561963670 wait complete. 2024-09-19T04:47:26.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.723+0000 7f46dfe00640 1 Processor -- start 2024-09-19T04:47:26.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.723+0000 7f46dfe00640 1 -- start start 2024-09-19T04:47:26.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.724+0000 7f46dfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e01019c0 0x7f46e019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.724+0000 7f46dfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46e0102360 0x7f46e019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.724+0000 7f46dfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46e01079c0 0x7f46e019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.724+0000 7f46dfe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46e0111dc0 con 0x7f46e01019c0 2024-09-19T04:47:26.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.724+0000 7f46dfe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46e0111c40 con 0x7f46e0102360 2024-09-19T04:47:26.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.724+0000 7f46dfe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f46e0111f40 con 0x7f46e01079c0 2024-09-19T04:47:26.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.724+0000 7f46dea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e01019c0 0x7f46e019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:26.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.724+0000 7f46dea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e01019c0 0x7f46e019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59916/0 (socket says 172.21.15.123:59916) 2024-09-19T04:47:26.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.724+0000 7f46dea00640 1 -- 172.21.15.123:0/2030084627 learned_addr learned my addr 172.21.15.123:0/2030084627 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:26.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.725+0000 7f46df400640 1 --2- 172.21.15.123:0/2030084627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46e01079c0 0x7f46e019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:26.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.725+0000 7f46de000640 1 --2- 172.21.15.123:0/2030084627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46e0102360 0x7f46e019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:26.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.725+0000 7f46de000640 1 -- 172.21.15.123:0/2030084627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46e01079c0 msgr2=0x7f46e019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:26.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.725+0000 7f46de000640 1 --2- 172.21.15.123:0/2030084627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46e01079c0 0x7f46e019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.725+0000 7f46de000640 1 -- 172.21.15.123:0/2030084627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e01019c0 msgr2=0x7f46e019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:26.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.725+0000 7f46de000640 1 --2- 172.21.15.123:0/2030084627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e01019c0 0x7f46e019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.725+0000 7f46de000640 1 -- 172.21.15.123:0/2030084627 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e019f500 con 0x7f46e0102360 2024-09-19T04:47:26.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.725+0000 7f46dea00640 1 --2- 172.21.15.123:0/2030084627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e01019c0 0x7f46e019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:47:26.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.725+0000 7f46df400640 1 --2- 172.21.15.123:0/2030084627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46e01079c0 0x7f46e019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:47:26.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.726+0000 7f46de000640 1 --2- 172.21.15.123:0/2030084627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46e0102360 0x7f46e019aa90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f46cc02a860 tx=0x7f46cc033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:26.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.726+0000 7f46d7400640 1 -- 172.21.15.123:0/2030084627 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc037040 con 0x7f46e0102360 2024-09-19T04:47:26.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.726+0000 7f46d7400640 1 -- 172.21.15.123:0/2030084627 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46cc033d50 con 0x7f46e0102360 2024-09-19T04:47:26.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.726+0000 7f46dfe00640 1 -- 172.21.15.123:0/2030084627 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e019f790 con 0x7f46e0102360 2024-09-19T04:47:26.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.726+0000 7f46d7400640 1 -- 172.21.15.123:0/2030084627 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc0039e0 con 0x7f46e0102360 2024-09-19T04:47:26.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.726+0000 7f46dfe00640 1 -- 172.21.15.123:0/2030084627 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e01a70d0 con 0x7f46e0102360 2024-09-19T04:47:26.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.731+0000 7f46e8678740 1 Processor -- start 2024-09-19T04:47:26.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.731+0000 7f46e8678740 1 -- start start 2024-09-19T04:47:26.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.731+0000 7f46e8678740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 0x5607ee102130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.731+0000 7f46e8678740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee102670 0x5607ee10c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.732+0000 7f46e8678740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee10cf30 0x5607ee10f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.732+0000 7f46e8678740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5607ee0683e0 con 0x5607ee102670 2024-09-19T04:47:26.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.732+0000 7f46e8678740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5607ee00fd80 con 0x5607ee10cf30 2024-09-19T04:47:26.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.732+0000 7f46e8678740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5607ee0643a0 con 0x5607ee101d50 2024-09-19T04:47:26.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.732+0000 7f46d7400640 1 -- 172.21.15.123:0/2030084627 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7f46cc042020 con 0x7f46e0102360 2024-09-19T04:47:26.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.732+0000 7f46df400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 0x5607ee102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:26.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.732+0000 7f46df400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 0x5607ee102130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:48178/0 (socket says 172.21.15.123:48178) 2024-09-19T04:47:26.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.732+0000 7f46df400640 1 -- 172.21.15.123:0/2564039611 learned_addr learned my addr 172.21.15.123:0/2564039611 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:26.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.732+0000 7f46dea00640 1 --2- 172.21.15.123:0/2564039611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee102670 0x5607ee10c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:26.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.732+0000 7f46df400640 1 -- 172.21.15.123:0/2564039611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee10cf30 msgr2=0x5607ee10f320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:47:26.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.732+0000 7f46df400640 1 --2- 172.21.15.123:0/2564039611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee10cf30 0x5607ee10f320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.732+0000 7f46df400640 1 -- 172.21.15.123:0/2564039611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee102670 msgr2=0x5607ee10c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:26.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.732+0000 7f46df400640 1 --2- 172.21.15.123:0/2564039611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee102670 0x5607ee10c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.732+0000 7f46df400640 1 -- 172.21.15.123:0/2564039611 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607ee02eae0 con 0x5607ee101d50 2024-09-19T04:47:26.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.733+0000 7f46dea00640 1 --2- 172.21.15.123:0/2564039611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee102670 0x5607ee10c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:26.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.733+0000 7f46df400640 1 --2- 172.21.15.123:0/2564039611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 0x5607ee102130 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009fd0 tx=0x7f46d800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f131d294372c6452 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:26.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.733+0000 7f46cbe00640 1 -- 172.21.15.123:0/2564039611 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8004d20 con 0x5607ee101d50 2024-09-19T04:47:26.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.733+0000 7f46cbe00640 1 -- 172.21.15.123:0/2564039611 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46d8003970 con 0x5607ee101d50 2024-09-19T04:47:26.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.734+0000 7f46cbe00640 1 -- 172.21.15.123:0/2564039611 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8004490 con 0x5607ee101d50 2024-09-19T04:47:26.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.734+0000 7f46d7400640 1 --2- 172.21.15.123:0/2030084627 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46c40a3660 0x7f46c40a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.734+0000 7f46d7400640 1 -- 172.21.15.123:0/2030084627 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f46cc0ef190 con 0x7f46e0102360 2024-09-19T04:47:26.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.734+0000 7f46dea00640 1 --2- 172.21.15.123:0/2030084627 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46c40a3660 0x7f46c40a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:26.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.735+0000 7f46e8678740 1 -- 172.21.15.123:0/2564039611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 msgr2=0x5607ee102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:26.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.735+0000 7f46e8678740 1 --2- 172.21.15.123:0/2564039611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 0x5607ee102130 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009fd0 tx=0x7f46d800ec80 comp rx=0 tx=0).stop 2024-09-19T04:47:26.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.735+0000 7f46e8678740 1 -- 172.21.15.123:0/2564039611 shutdown_connections 2024-09-19T04:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.735+0000 7f46e8678740 1 --2- 172.21.15.123:0/2564039611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee10cf30 0x5607ee10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.735+0000 7f46e8678740 1 --2- 172.21.15.123:0/2564039611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee102670 0x5607ee10c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.735+0000 7f46e8678740 1 --2- 172.21.15.123:0/2564039611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 0x5607ee102130 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.735+0000 7f46e8678740 1 -- 172.21.15.123:0/2564039611 >> 172.21.15.123:0/2564039611 conn(0x5607ee0fe2e0 msgr2=0x5607ee0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.735+0000 7f46e8678740 1 -- 172.21.15.123:0/2564039611 shutdown_connections 2024-09-19T04:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.735+0000 7f46dea00640 1 --2- 172.21.15.123:0/2030084627 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46c40a3660 0x7f46c40a5b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5607ee0ff140 tx=0x7f46d0005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.735+0000 7f46e8678740 1 -- 172.21.15.123:0/2564039611 wait complete. 2024-09-19T04:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.736+0000 7f46e8678740 1 -- start start 2024-09-19T04:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.736+0000 7f46e8678740 1 Processor -- start 2024-09-19T04:47:26.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.736+0000 7f46e8678740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee101d50 0x5607ee109e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.736+0000 7f46e8678740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee102670 0x5607ee10a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.736+0000 7f46e8678740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee10cf30 0x5607ee10a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.737+0000 7f46e8678740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5607ee120bb0 con 0x5607ee10cf30 2024-09-19T04:47:26.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.737+0000 7f46e8678740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5607ee0230f0 con 0x5607ee101d50 2024-09-19T04:47:26.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.737+0000 7f46e8678740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5607ee120d30 con 0x5607ee102670 2024-09-19T04:47:26.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.737+0000 7f46e8678740 10 client.0 _ll_register_callbacks cb 0x5607ee0149e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:26.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.737+0000 7f46dea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee101d50 0x5607ee109e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:26.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.737+0000 7f46dea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee101d50 0x5607ee109e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:52806/0 (socket says 172.21.15.123:52806) 2024-09-19T04:47:26.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.737+0000 7f46dea00640 1 -- 172.21.15.123:0/1888222281 learned_addr learned my addr 172.21.15.123:0/1888222281 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:26.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.737+0000 7f46de000640 1 --2- 172.21.15.123:0/1888222281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee102670 0x5607ee10a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:26.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.737+0000 7f46df400640 1 --2- 172.21.15.123:0/1888222281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee10cf30 0x5607ee10a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:26.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.738+0000 7f46de000640 1 -- 172.21.15.123:0/1888222281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee101d50 msgr2=0x5607ee109e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:26.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.738+0000 7f46de000640 1 --2- 172.21.15.123:0/1888222281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee101d50 0x5607ee109e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.738+0000 7f46de000640 1 -- 172.21.15.123:0/1888222281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee10cf30 msgr2=0x5607ee10a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:26.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.738+0000 7f46de000640 1 --2- 172.21.15.123:0/1888222281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee10cf30 0x5607ee10a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.738+0000 7f46de000640 1 -- 172.21.15.123:0/1888222281 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607eddd93c0 con 0x5607ee102670 2024-09-19T04:47:26.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.738+0000 7f46de000640 1 --2- 172.21.15.123:0/1888222281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee102670 0x5607ee10a3d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0ec7b0 tx=0x7f46cc0baf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae53a7e3331a709f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:26.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.738+0000 7f46c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:47:26.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.739+0000 7f46c8c00640 1 -- 172.21.15.123:0/1888222281 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc0c3d50 con 0x5607ee102670 2024-09-19T04:47:26.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.739+0000 7f46e8678740 1 -- 172.21.15.123:0/1888222281 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46d8003970 con 0x5607ee102670 2024-09-19T04:47:26.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.739+0000 7f46c8c00640 1 -- 172.21.15.123:0/1888222281 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f46cc0ba7c0 con 0x5607ee102670 2024-09-19T04:47:26.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.739+0000 7f46c8c00640 1 -- 172.21.15.123:0/1888222281 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f46cc0bf720 con 0x5607ee102670 2024-09-19T04:47:26.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.739+0000 7f46e8678740 10 client.25354 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:26.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.739+0000 7f46e8678740 1 -- 172.21.15.123:0/1888222281 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5607ee120bb0 con 0x5607ee102670 2024-09-19T04:47:26.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.739+0000 7f46c8c00640 1 -- 172.21.15.123:0/1888222281 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46d8003970 con 0x5607ee102670 2024-09-19T04:47:26.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.739+0000 7f46c8c00640 1 -- 172.21.15.123:0/1888222281 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f46cc03a070 con 0x5607ee102670 2024-09-19T04:47:26.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.740+0000 7f46e8678740 10 client.25354 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:26.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.740+0000 7f46e8678740 10 client.25354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:47:26.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.740+0000 7f46e8678740 20 client.25354 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:26.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.740+0000 7f46e8678740 1 --2- 172.21.15.123:0/1888222281 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5607ee136470 0x5607ee138860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.740+0000 7f46e8678740 4 client.25354 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:47:26.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.740+0000 7f46e8678740 1 -- 172.21.15.123:0/1888222281 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607ee075950 con 0x5607ee136470 2024-09-19T04:47:26.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.740+0000 7f46dea00640 1 --2- 172.21.15.123:0/1888222281 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5607ee136470 0x5607ee138860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:26.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.741+0000 7f46dea00640 1 --2- 172.21.15.123:0/1888222281 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5607ee136470 0x5607ee138860 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c963b76a3dc7af8 server_cookie=c978c91c0bd6d53e in_seq=0 out_seq=0 2024-09-19T04:47:26.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.741+0000 7f46c8c00640 10 client.25354 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:26.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.741+0000 7f46c8c00640 1 -- 172.21.15.123:0/1888222281 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607ee075950 con 0x5607ee136470 2024-09-19T04:47:26.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.741+0000 7f46c8c00640 10 client.25354 handle_command_reply: tid=0 2024-09-19T04:47:26.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.742+0000 7f46e8678740 1 client.25354 shutdown 2024-09-19T04:47:26.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.742+0000 7f46e8678740 1 -- 172.21.15.123:0/1888222281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee102670 msgr2=0x5607ee10a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:26.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.742+0000 7f46e8678740 1 --2- 172.21.15.123:0/1888222281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee102670 0x5607ee10a3d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0ec7b0 tx=0x7f46cc0baf70 comp rx=0 tx=0).stop 2024-09-19T04:47:26.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.742+0000 7f46e8678740 1 -- 172.21.15.123:0/1888222281 shutdown_connections 2024-09-19T04:47:26.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.742+0000 7f46e8678740 1 --2- 172.21.15.123:0/1888222281 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5607ee136470 0x5607ee138860 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.742+0000 7f46e8678740 1 --2- 172.21.15.123:0/1888222281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee10cf30 0x5607ee10a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.742+0000 7f46e8678740 1 --2- 172.21.15.123:0/1888222281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee102670 0x5607ee10a3d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.742+0000 7f46e8678740 1 --2- 172.21.15.123:0/1888222281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee101d50 0x5607ee109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.742+0000 7f46e8678740 1 -- 172.21.15.123:0/1888222281 >> 172.21.15.123:0/1888222281 conn(0x5607ee0fe2e0 msgr2=0x5607ee104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:26.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.742+0000 7f46c8c00640 0 client.25354 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:26.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.742+0000 7f46e8678740 1 -- 172.21.15.123:0/1888222281 shutdown_connections 2024-09-19T04:47:26.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.742+0000 7f46e8678740 1 -- 172.21.15.123:0/1888222281 wait complete. 2024-09-19T04:47:26.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.742+0000 7f46e8678740 20 client.25354 trim_cache size 0 max 16384 2024-09-19T04:47:26.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.765+0000 7f46e8678740 1 Processor -- start 2024-09-19T04:47:26.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.765+0000 7f46e8678740 1 -- start start 2024-09-19T04:47:26.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.766+0000 7f46e8678740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 0x5607ee10a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.766+0000 7f46e8678740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee102670 0x5607ee13e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.766+0000 7f46e8678740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee10cf30 0x5607ee13e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.766+0000 7f46e8678740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46e006fd90 con 0x5607ee10cf30 2024-09-19T04:47:26.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.766+0000 7f46e8678740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46e00718d0 con 0x5607ee102670 2024-09-19T04:47:26.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.766+0000 7f46e8678740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f46e0070460 con 0x5607ee101d50 2024-09-19T04:47:26.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.766+0000 7f46de000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee102670 0x5607ee13e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:26.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.766+0000 7f46df400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee10cf30 0x5607ee13e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:26.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.766+0000 7f46de000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee102670 0x5607ee13e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:52822/0 (socket says 172.21.15.123:52822) 2024-09-19T04:47:26.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.766+0000 7f46de000640 1 -- 172.21.15.123:0/1072957840 learned_addr learned my addr 172.21.15.123:0/1072957840 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:26.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.766+0000 7f46dea00640 1 --2- 172.21.15.123:0/1072957840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 0x5607ee10a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:26.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.767+0000 7f46de000640 1 -- 172.21.15.123:0/1072957840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 msgr2=0x5607ee10a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:26.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.767+0000 7f46de000640 1 --2- 172.21.15.123:0/1072957840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 0x5607ee10a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.767+0000 7f46de000640 1 -- 172.21.15.123:0/1072957840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee10cf30 msgr2=0x5607ee13e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:26.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.767+0000 7f46de000640 1 --2- 172.21.15.123:0/1072957840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee10cf30 0x5607ee13e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.767+0000 7f46de000640 1 -- 172.21.15.123:0/1072957840 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607ee10c800 con 0x5607ee102670 2024-09-19T04:47:26.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.767+0000 7f46de000640 1 --2- 172.21.15.123:0/1072957840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee102670 0x5607ee13e130 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f46cc02a930 tx=0x7f46cc0bafe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7eb850f4f2e80e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:26.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.767+0000 7f46cb400640 1 -- 172.21.15.123:0/1072957840 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc0c3fe0 con 0x5607ee102670 2024-09-19T04:47:26.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.767+0000 7f46cb400640 1 -- 172.21.15.123:0/1072957840 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46cc0ba3d0 con 0x5607ee102670 2024-09-19T04:47:26.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.767+0000 7f46cb400640 1 -- 172.21.15.123:0/1072957840 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc0c3db0 con 0x5607ee102670 2024-09-19T04:47:26.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.768+0000 7f46e8678740 1 -- 172.21.15.123:0/1072957840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee102670 msgr2=0x5607ee13e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:26.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.768+0000 7f46e8678740 1 --2- 172.21.15.123:0/1072957840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee102670 0x5607ee13e130 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f46cc02a930 tx=0x7f46cc0bafe0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.768+0000 7f46e8678740 1 -- 172.21.15.123:0/1072957840 shutdown_connections 2024-09-19T04:47:26.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.768+0000 7f46e8678740 1 --2- 172.21.15.123:0/1072957840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee10cf30 0x5607ee13e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.768+0000 7f46e8678740 1 --2- 172.21.15.123:0/1072957840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee102670 0x5607ee13e130 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.768+0000 7f46e8678740 1 --2- 172.21.15.123:0/1072957840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 0x5607ee10a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.768+0000 7f46e8678740 1 -- 172.21.15.123:0/1072957840 >> 172.21.15.123:0/1072957840 conn(0x5607ee0fe2e0 msgr2=0x5607ee10bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:26.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.769+0000 7f46e8678740 1 -- 172.21.15.123:0/1072957840 shutdown_connections 2024-09-19T04:47:26.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.769+0000 7f46e8678740 1 -- 172.21.15.123:0/1072957840 wait complete. 2024-09-19T04:47:26.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.769+0000 7f46e8678740 1 -- start start 2024-09-19T04:47:26.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.770+0000 7f46e8678740 1 Processor -- start 2024-09-19T04:47:26.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.770+0000 7f46e8678740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 0x5607ee121c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.770+0000 7f46e8678740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee102670 0x5607ee122180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.770+0000 7f46e8678740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee10cf30 0x5607ee1226c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.770+0000 7f46e8678740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46e0071200 con 0x5607ee10cf30 2024-09-19T04:47:26.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.770+0000 7f46e8678740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46e0070b30 con 0x5607ee102670 2024-09-19T04:47:26.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.770+0000 7f46e8678740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f46e006dd90 con 0x5607ee101d50 2024-09-19T04:47:26.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.770+0000 7f46e8678740 10 client.0 _ll_register_callbacks cb 0x7f46cc0b7610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:26.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.771+0000 7f46dea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 0x5607ee121c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:26.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.771+0000 7f46dea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 0x5607ee121c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:48186/0 (socket says 172.21.15.123:48186) 2024-09-19T04:47:26.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.771+0000 7f46dea00640 1 -- 172.21.15.123:0/1756184123 learned_addr learned my addr 172.21.15.123:0/1756184123 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:26.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.771+0000 7f46df400640 1 --2- 172.21.15.123:0/1756184123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee10cf30 0x5607ee1226c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:26.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.771+0000 7f46de000640 1 --2- 172.21.15.123:0/1756184123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee102670 0x5607ee122180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:26.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.771+0000 7f46dea00640 1 -- 172.21.15.123:0/1756184123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee102670 msgr2=0x5607ee122180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:26.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.771+0000 7f46dea00640 1 --2- 172.21.15.123:0/1756184123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee102670 0x5607ee122180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.771+0000 7f46dea00640 1 -- 172.21.15.123:0/1756184123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee10cf30 msgr2=0x5607ee1226c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:26.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.771+0000 7f46dea00640 1 --2- 172.21.15.123:0/1756184123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee10cf30 0x5607ee1226c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.771+0000 7f46dea00640 1 -- 172.21.15.123:0/1756184123 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607ee1a0100 con 0x5607ee101d50 2024-09-19T04:47:26.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.772+0000 7f46de000640 1 --2- 172.21.15.123:0/1756184123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee102670 0x5607ee122180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:26.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.772+0000 7f46df400640 1 --2- 172.21.15.123:0/1756184123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee10cf30 0x5607ee1226c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:47:26.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.772+0000 7f46dea00640 1 --2- 172.21.15.123:0/1756184123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 0x5607ee121c40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5607ee1390a0 tx=0x7f46d00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=956b02ca735ed2d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:26.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.772+0000 7f46c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:47:26.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.772+0000 7f46c8c00640 1 -- 172.21.15.123:0/1756184123 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0005020 con 0x5607ee101d50 2024-09-19T04:47:26.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.772+0000 7f46e8678740 1 -- 172.21.15.123:0/1756184123 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46cc0ba3d0 con 0x5607ee101d50 2024-09-19T04:47:26.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.772+0000 7f46c8c00640 1 -- 172.21.15.123:0/1756184123 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f46d00051c0 con 0x5607ee101d50 2024-09-19T04:47:26.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.773+0000 7f46c8c00640 1 -- 172.21.15.123:0/1756184123 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f46d00053c0 con 0x5607ee101d50 2024-09-19T04:47:26.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.773+0000 7f46e8678740 10 client.25360 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:26.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.773+0000 7f46e8678740 1 -- 172.21.15.123:0/1756184123 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5607ee1a13b0 con 0x5607ee101d50 2024-09-19T04:47:26.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.773+0000 7f46c8c00640 1 -- 172.21.15.123:0/1756184123 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46cc0ba3d0 con 0x5607ee101d50 2024-09-19T04:47:26.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.773+0000 7f46c8c00640 1 -- 172.21.15.123:0/1756184123 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f46d002a840 con 0x5607ee101d50 2024-09-19T04:47:26.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.773+0000 7f46e8678740 10 client.25360 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:26.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.773+0000 7f46e8678740 10 client.25360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:47:26.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.773+0000 7f46e8678740 20 client.25360 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:26.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.773+0000 7f46e8678740 1 --2- 172.21.15.123:0/1756184123 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5607ee1a1a30 0x5607ee1a3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:26.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.773+0000 7f46e8678740 4 client.25360 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:47:26.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.773+0000 7f46e8678740 1 -- 172.21.15.123:0/1756184123 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5607ee06fe60 con 0x5607ee1a1a30 2024-09-19T04:47:26.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.774+0000 7f46de000640 1 --2- 172.21.15.123:0/1756184123 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5607ee1a1a30 0x5607ee1a3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:26.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.775+0000 7f46de000640 1 --2- 172.21.15.123:0/1756184123 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5607ee1a1a30 0x5607ee1a3e20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec68c26c71c97d7f server_cookie=6d575a8ece9912bd in_seq=0 out_seq=0 2024-09-19T04:47:26.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.775+0000 7f46c8c00640 10 client.25360 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:26.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.775+0000 7f46c8c00640 1 -- 172.21.15.123:0/1756184123 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5607ee06fe60 con 0x5607ee1a1a30 2024-09-19T04:47:26.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.775+0000 7f46c8c00640 10 client.25360 handle_command_reply: tid=0 2024-09-19T04:47:26.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.775+0000 7f46e8678740 1 client.25360 shutdown 2024-09-19T04:47:26.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.775+0000 7f46e8678740 1 -- 172.21.15.123:0/1756184123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 msgr2=0x5607ee121c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:26.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.775+0000 7f46e8678740 1 --2- 172.21.15.123:0/1756184123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 0x5607ee121c40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5607ee1390a0 tx=0x7f46d00046c0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.775+0000 7f46e8678740 1 -- 172.21.15.123:0/1756184123 shutdown_connections 2024-09-19T04:47:26.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.775+0000 7f46e8678740 1 --2- 172.21.15.123:0/1756184123 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5607ee1a1a30 0x5607ee1a3e20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.775+0000 7f46e8678740 1 --2- 172.21.15.123:0/1756184123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607ee10cf30 0x5607ee1226c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.775+0000 7f46e8678740 1 --2- 172.21.15.123:0/1756184123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607ee102670 0x5607ee122180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.775+0000 7f46e8678740 1 --2- 172.21.15.123:0/1756184123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607ee101d50 0x5607ee121c40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.775+0000 7f46c8c00640 0 client.25360 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:26.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.775+0000 7f46e8678740 1 -- 172.21.15.123:0/1756184123 >> 172.21.15.123:0/1756184123 conn(0x5607ee0fe2e0 msgr2=0x5607ee1037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:26.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.776+0000 7f46e8678740 1 -- 172.21.15.123:0/1756184123 shutdown_connections 2024-09-19T04:47:26.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.776+0000 7f46e8678740 1 -- 172.21.15.123:0/1756184123 wait complete. 2024-09-19T04:47:26.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.776+0000 7f46e8678740 20 client.25360 trim_cache size 0 max 16384 2024-09-19T04:47:26.780 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:47:26.780 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-09-19T04:47:26.780 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-09-19T04:47:26.780 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:47:26.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.777+0000 7f46dfe00640 1 -- 172.21.15.123:0/2030084627 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46c40a3660 msgr2=0x7f46c40a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:26.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.777+0000 7f46dfe00640 1 --2- 172.21.15.123:0/2030084627 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46c40a3660 0x7f46c40a5b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5607ee0ff140 tx=0x7f46d0005f90 comp rx=0 tx=0).stop 2024-09-19T04:47:26.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.777+0000 7f46dfe00640 1 -- 172.21.15.123:0/2030084627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46e0102360 msgr2=0x7f46e019aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:26.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.777+0000 7f46dfe00640 1 --2- 172.21.15.123:0/2030084627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46e0102360 0x7f46e019aa90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f46cc02a860 tx=0x7f46cc033b60 comp rx=0 tx=0).stop 2024-09-19T04:47:26.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.777+0000 7f46dfe00640 1 -- 172.21.15.123:0/2030084627 shutdown_connections 2024-09-19T04:47:26.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.777+0000 7f46dfe00640 1 --2- 172.21.15.123:0/2030084627 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46c40a3660 0x7f46c40a5b20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.777+0000 7f46dfe00640 1 --2- 172.21.15.123:0/2030084627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46e01079c0 0x7f46e019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.777+0000 7f46dfe00640 1 --2- 172.21.15.123:0/2030084627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46e0102360 0x7f46e019aa90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.777+0000 7f46dfe00640 1 --2- 172.21.15.123:0/2030084627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e01019c0 0x7f46e019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:26.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.777+0000 7f46dfe00640 1 -- 172.21.15.123:0/2030084627 >> 172.21.15.123:0/2030084627 conn(0x7f46e00fd740 msgr2=0x7f46e010cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:26.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.777+0000 7f46dfe00640 1 -- 172.21.15.123:0/2030084627 shutdown_connections 2024-09-19T04:47:26.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:26.777+0000 7f46dfe00640 1 -- 172.21.15.123:0/2030084627 wait complete. 2024-09-19T04:47:27.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:27 smithi123 ceph-mon[30511]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:47:27.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:27 smithi183 ceph-mon[35333]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:47:27.734 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:47:27.734 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:47:27.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:27 smithi193 ceph-mon[35592]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:47:28.221 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:47:28.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.905+0000 7f0e53200640 1 -- 172.21.15.123:0/3166106138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e54103e30 msgr2=0x7f0e54104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:28.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.905+0000 7f0e53200640 1 --2- 172.21.15.123:0/3166106138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e54103e30 0x7f0e54104210 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009900 tx=0x7f0e4402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:47:28.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.905+0000 7f0e53200640 1 -- 172.21.15.123:0/3166106138 shutdown_connections 2024-09-19T04:47:28.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.905+0000 7f0e53200640 1 --2- 172.21.15.123:0/3166106138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e54108f60 0x7f0e54110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:28.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.905+0000 7f0e53200640 1 --2- 172.21.15.123:0/3166106138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e541047e0 0x7f0e54108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:28.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.905+0000 7f0e53200640 1 --2- 172.21.15.123:0/3166106138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e54103e30 0x7f0e54104210 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:28.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.905+0000 7f0e53200640 1 -- 172.21.15.123:0/3166106138 >> 172.21.15.123:0/3166106138 conn(0x7f0e540ff910 msgr2=0x7f0e54101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:28.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.905+0000 7f0e53200640 1 -- 172.21.15.123:0/3166106138 shutdown_connections 2024-09-19T04:47:28.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.905+0000 7f0e53200640 1 -- 172.21.15.123:0/3166106138 wait complete. 2024-09-19T04:47:28.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.906+0000 7f0e53200640 1 Processor -- start 2024-09-19T04:47:28.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.907+0000 7f0e53200640 1 -- start start 2024-09-19T04:47:28.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.907+0000 7f0e53200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e54103e30 0x7f0e54110910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:28.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.907+0000 7f0e53200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e541047e0 0x7f0e5410d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:28.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.907+0000 7f0e53200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e54108f60 0x7f0e5410df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:28.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.907+0000 7f0e53200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e54114010 con 0x7f0e54103e30 2024-09-19T04:47:28.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.907+0000 7f0e53200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0e54113e90 con 0x7f0e54108f60 2024-09-19T04:47:28.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.907+0000 7f0e53200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e54114190 con 0x7f0e541047e0 2024-09-19T04:47:28.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.907+0000 7f0e51400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e541047e0 0x7f0e5410d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:28.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.908+0000 7f0e51400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e541047e0 0x7f0e5410d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:48202/0 (socket says 172.21.15.123:48202) 2024-09-19T04:47:28.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.908+0000 7f0e51400640 1 -- 172.21.15.123:0/371691854 learned_addr learned my addr 172.21.15.123:0/371691854 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:28.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.908+0000 7f0e51e00640 1 --2- 172.21.15.123:0/371691854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e54103e30 0x7f0e54110910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:28.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.908+0000 7f0e52800640 1 --2- 172.21.15.123:0/371691854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e54108f60 0x7f0e5410df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:28.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.908+0000 7f0e51400640 1 -- 172.21.15.123:0/371691854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e54108f60 msgr2=0x7f0e5410df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:28.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.908+0000 7f0e51400640 1 --2- 172.21.15.123:0/371691854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e54108f60 0x7f0e5410df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:28.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.908+0000 7f0e51400640 1 -- 172.21.15.123:0/371691854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e54103e30 msgr2=0x7f0e54110910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:28.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.908+0000 7f0e51400640 1 --2- 172.21.15.123:0/371691854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e54103e30 0x7f0e54110910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:28.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.908+0000 7f0e51400640 1 -- 172.21.15.123:0/371691854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5410e7c0 con 0x7f0e541047e0 2024-09-19T04:47:28.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.908+0000 7f0e52800640 1 --2- 172.21.15.123:0/371691854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e54108f60 0x7f0e5410df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:28.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.909+0000 7f0e51400640 1 --2- 172.21.15.123:0/371691854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e541047e0 0x7f0e5410d9c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00e9f0 tx=0x7f0e3c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:28.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.909+0000 7f0e4aa00640 1 -- 172.21.15.123:0/371691854 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c00cd80 con 0x7f0e541047e0 2024-09-19T04:47:28.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.909+0000 7f0e4aa00640 1 -- 172.21.15.123:0/371691854 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e3c010910 con 0x7f0e541047e0 2024-09-19T04:47:28.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.909+0000 7f0e4aa00640 1 -- 172.21.15.123:0/371691854 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c018980 con 0x7f0e541047e0 2024-09-19T04:47:28.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.909+0000 7f0e53200640 1 -- 172.21.15.123:0/371691854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e541aadd0 con 0x7f0e541047e0 2024-09-19T04:47:28.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.909+0000 7f0e53200640 1 -- 172.21.15.123:0/371691854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e541ab230 con 0x7f0e541047e0 2024-09-19T04:47:28.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.910+0000 7f0e53200640 1 -- 172.21.15.123:0/371691854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e54105310 con 0x7f0e541047e0 2024-09-19T04:47:28.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.912+0000 7f0e4aa00640 1 -- 172.21.15.123:0/371691854 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7f0e3c010ac0 con 0x7f0e541047e0 2024-09-19T04:47:28.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.914+0000 7f0e4aa00640 1 --2- 172.21.15.123:0/371691854 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0e380a35e0 0x7f0e380a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:28.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.914+0000 7f0e4aa00640 1 -- 172.21.15.123:0/371691854 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f0e3c0d02a0 con 0x7f0e541047e0 2024-09-19T04:47:28.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.914+0000 7f0e51e00640 1 --2- 172.21.15.123:0/371691854 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0e380a35e0 0x7f0e380a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:28.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.915+0000 7f0e51e00640 1 --2- 172.21.15.123:0/371691854 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0e380a35e0 0x7f0e380a5aa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0e44000c00 tx=0x7f0e44005b50 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:28.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:28.918+0000 7f0e4aa00640 1 -- 172.21.15.123:0/371691854 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0e3c09ca70 con 0x7f0e541047e0 2024-09-19T04:47:29.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:29.171+0000 7f0e53200640 1 -- 172.21.15.123:0/371691854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e5410eef0 con 0x7f0e541047e0 2024-09-19T04:47:29.174 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:47:29.174 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"btime":"2024-09-19T04:46:42:466383+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:46:41.577049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24499,24500,24539,24505,24529]},"id":1}]} 2024-09-19T04:47:29.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:29.172+0000 7f0e4aa00640 1 -- 172.21.15.123:0/371691854 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11990 (secure 0 0 0) 0x7f0e3c09c370 con 0x7f0e541047e0 2024-09-19T04:47:29.174 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 28 2024-09-19T04:47:29.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:29.175+0000 7f0e53200640 1 -- 172.21.15.123:0/371691854 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0e380a35e0 msgr2=0x7f0e380a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:29.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:29.175+0000 7f0e53200640 1 --2- 172.21.15.123:0/371691854 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0e380a35e0 0x7f0e380a5aa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0e44000c00 tx=0x7f0e44005b50 comp rx=0 tx=0).stop 2024-09-19T04:47:29.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:29.175+0000 7f0e53200640 1 -- 172.21.15.123:0/371691854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e541047e0 msgr2=0x7f0e5410d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:29.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:29.175+0000 7f0e53200640 1 --2- 172.21.15.123:0/371691854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e541047e0 0x7f0e5410d9c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00e9f0 tx=0x7f0e3c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:47:29.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:29.175+0000 7f0e53200640 1 -- 172.21.15.123:0/371691854 shutdown_connections 2024-09-19T04:47:29.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:29.175+0000 7f0e53200640 1 --2- 172.21.15.123:0/371691854 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0e380a35e0 0x7f0e380a5aa0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:29.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:29.175+0000 7f0e53200640 1 --2- 172.21.15.123:0/371691854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e54108f60 0x7f0e5410df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:29.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:29.175+0000 7f0e53200640 1 --2- 172.21.15.123:0/371691854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e541047e0 0x7f0e5410d9c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:29.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:29.176+0000 7f0e53200640 1 --2- 172.21.15.123:0/371691854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e54103e30 0x7f0e54110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:29.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:29.176+0000 7f0e53200640 1 -- 172.21.15.123:0/371691854 >> 172.21.15.123:0/371691854 conn(0x7f0e540ff910 msgr2=0x7f0e54100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:29.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:29.176+0000 7f0e53200640 1 -- 172.21.15.123:0/371691854 shutdown_connections 2024-09-19T04:47:29.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:29.176+0000 7f0e53200640 1 -- 172.21.15.123:0/371691854 wait complete. 2024-09-19T04:47:29.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:29 smithi123 ceph-mon[30511]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:47:29.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:29 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/371691854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:47:29.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:29 smithi183 ceph-mon[35333]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:47:29.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:29 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/371691854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:47:29.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:29 smithi193 ceph-mon[35592]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:47:29.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:29 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/371691854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:47:30.085 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:47:30.593 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:47:31.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.229+0000 7f6474a00640 1 -- 172.21.15.123:0/3624569409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6470108f50 msgr2=0x7f6470110ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:31.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.229+0000 7f6474a00640 1 --2- 172.21.15.123:0/3624569409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6470108f50 0x7f6470110ad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6468009ca0 tx=0x7f646802a8b0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.229+0000 7f6474a00640 1 -- 172.21.15.123:0/3624569409 shutdown_connections 2024-09-19T04:47:31.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.229+0000 7f6474a00640 1 --2- 172.21.15.123:0/3624569409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6470108f50 0x7f6470110ad0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.229+0000 7f6474a00640 1 --2- 172.21.15.123:0/3624569409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64701047d0 0x7f6470108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.229+0000 7f6474a00640 1 --2- 172.21.15.123:0/3624569409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6470103e20 0x7f6470104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.229+0000 7f6474a00640 1 -- 172.21.15.123:0/3624569409 >> 172.21.15.123:0/3624569409 conn(0x7f64700ff880 msgr2=0x7f6470101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:31.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.229+0000 7f6474a00640 1 -- 172.21.15.123:0/3624569409 shutdown_connections 2024-09-19T04:47:31.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.229+0000 7f6474a00640 1 -- 172.21.15.123:0/3624569409 wait complete. 2024-09-19T04:47:31.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.230+0000 7f6474a00640 1 Processor -- start 2024-09-19T04:47:31.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.230+0000 7f6474a00640 1 -- start start 2024-09-19T04:47:31.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.230+0000 7f6474a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6470103e20 0x7f6470110980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.230+0000 7f6474a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64701047d0 0x7f647010da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.230+0000 7f6474a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6470108f50 0x7f647010df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.230+0000 7f6474a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6470113fb0 con 0x7f6470103e20 2024-09-19T04:47:31.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.230+0000 7f6474a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6470113e30 con 0x7f64701047d0 2024-09-19T04:47:31.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.230+0000 7f6474a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6470114130 con 0x7f6470108f50 2024-09-19T04:47:31.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.231+0000 7f646f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6470108f50 0x7f647010df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.231+0000 7f646f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6470108f50 0x7f647010df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:48216/0 (socket says 172.21.15.123:48216) 2024-09-19T04:47:31.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.231+0000 7f646f200640 1 -- 172.21.15.123:0/3327854372 learned_addr learned my addr 172.21.15.123:0/3327854372 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:31.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.231+0000 7f646e800640 1 --2- 172.21.15.123:0/3327854372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6470103e20 0x7f6470110980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.231+0000 7f646de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64701047d0 0x7f647010da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.232+0000 7f646f200640 1 -- 172.21.15.123:0/3327854372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64701047d0 msgr2=0x7f647010da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:31.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.232+0000 7f646f200640 1 --2- 172.21.15.123:0/3327854372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64701047d0 0x7f647010da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.232+0000 7f646f200640 1 -- 172.21.15.123:0/3327854372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6470103e20 msgr2=0x7f6470110980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:31.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.232+0000 7f646f200640 1 --2- 172.21.15.123:0/3327854372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6470103e20 0x7f6470110980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.232+0000 7f646f200640 1 -- 172.21.15.123:0/3327854372 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647010e850 con 0x7f6470108f50 2024-09-19T04:47:31.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.232+0000 7f646f200640 1 --2- 172.21.15.123:0/3327854372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6470108f50 0x7f647010df90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6468006010 tx=0x7f6468004a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:31.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.232+0000 7f6467400640 1 -- 172.21.15.123:0/3327854372 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468003d80 con 0x7f6470108f50 2024-09-19T04:47:31.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.232+0000 7f6467400640 1 -- 172.21.15.123:0/3327854372 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6468040070 con 0x7f6470108f50 2024-09-19T04:47:31.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.232+0000 7f6467400640 1 -- 172.21.15.123:0/3327854372 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468045ad0 con 0x7f6470108f50 2024-09-19T04:47:31.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.232+0000 7f6474a00640 1 -- 172.21.15.123:0/3327854372 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64701aad50 con 0x7f6470108f50 2024-09-19T04:47:31.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.233+0000 7f6474a00640 1 -- 172.21.15.123:0/3327854372 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64701ab240 con 0x7f6470108f50 2024-09-19T04:47:31.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.237+0000 7f6477e8f740 1 Processor -- start 2024-09-19T04:47:31.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.237+0000 7f6467400640 1 -- 172.21.15.123:0/3327854372 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7f6468045c70 con 0x7f6470108f50 2024-09-19T04:47:31.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.237+0000 7f6477e8f740 1 -- start start 2024-09-19T04:47:31.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.238+0000 7f6477e8f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f23078ae40 0x55f23078b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.238+0000 7f6477e8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078b760 0x55f230795ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.238+0000 7f6477e8f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f230796020 0x55f230798410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.238+0000 7f6477e8f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f2306f13e0 con 0x55f23078ae40 2024-09-19T04:47:31.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.238+0000 7f6477e8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f230698d80 con 0x55f23078b760 2024-09-19T04:47:31.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.238+0000 7f6477e8f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f2306ed3a0 con 0x55f230796020 2024-09-19T04:47:31.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.238+0000 7f646de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f23078ae40 0x55f23078b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.238+0000 7f646e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078b760 0x55f230795ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.238+0000 7f646e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078b760 0x55f230795ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:52898/0 (socket says 172.21.15.123:52898) 2024-09-19T04:47:31.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.239+0000 7f646e800640 1 -- 172.21.15.123:0/3306211475 learned_addr learned my addr 172.21.15.123:0/3306211475 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:31.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.239+0000 7f646de00640 1 --2- 172.21.15.123:0/3306211475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f230796020 0x55f230798410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.239+0000 7f646e800640 1 -- 172.21.15.123:0/3306211475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f230796020 msgr2=0x55f230798410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:31.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.239+0000 7f646e800640 1 --2- 172.21.15.123:0/3306211475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f230796020 0x55f230798410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.239+0000 7f646e800640 1 -- 172.21.15.123:0/3306211475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f23078ae40 msgr2=0x55f23078b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:31.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.239+0000 7f646e800640 1 --2- 172.21.15.123:0/3306211475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f23078ae40 0x55f23078b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.239+0000 7f646e800640 1 -- 172.21.15.123:0/3306211475 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2306b7ae0 con 0x55f23078b760 2024-09-19T04:47:31.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.239+0000 7f6467400640 1 --2- 172.21.15.123:0/3327854372 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f64540a35e0 0x7f64540a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.239+0000 7f6467400640 1 -- 172.21.15.123:0/3327854372 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f64680f0ee0 con 0x7f6470108f50 2024-09-19T04:47:31.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.240+0000 7f646de00640 1 --2- 172.21.15.123:0/3327854372 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f64540a35e0 0x7f64540a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.240+0000 7f646e800640 1 --2- 172.21.15.123:0/3306211475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078b760 0x55f230795ae0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6460009cf0 tx=0x7f6460011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72a382294ee09a55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:31.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.241+0000 7f645b400640 1 -- 172.21.15.123:0/3306211475 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646001e070 con 0x55f23078b760 2024-09-19T04:47:31.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.241+0000 7f645b400640 1 -- 172.21.15.123:0/3306211475 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6460019b10 con 0x55f23078b760 2024-09-19T04:47:31.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.241+0000 7f645b400640 1 -- 172.21.15.123:0/3306211475 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646000e9f0 con 0x55f23078b760 2024-09-19T04:47:31.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.241+0000 7f646de00640 1 --2- 172.21.15.123:0/3327854372 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f64540a35e0 0x7f64540a5aa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55f230788750 tx=0x7f645c0096c0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:31.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.241+0000 7f6477e8f740 1 -- 172.21.15.123:0/3306211475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078b760 msgr2=0x55f230795ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:31.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.241+0000 7f6477e8f740 1 --2- 172.21.15.123:0/3306211475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078b760 0x55f230795ae0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6460009cf0 tx=0x7f6460011810 comp rx=0 tx=0).stop 2024-09-19T04:47:31.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.242+0000 7f6477e8f740 1 -- 172.21.15.123:0/3306211475 shutdown_connections 2024-09-19T04:47:31.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.242+0000 7f6477e8f740 1 --2- 172.21.15.123:0/3306211475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f230796020 0x55f230798410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.242+0000 7f6477e8f740 1 --2- 172.21.15.123:0/3306211475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078b760 0x55f230795ae0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.242+0000 7f6477e8f740 1 --2- 172.21.15.123:0/3306211475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f23078ae40 0x55f23078b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.242+0000 7f6477e8f740 1 -- 172.21.15.123:0/3306211475 >> 172.21.15.123:0/3306211475 conn(0x55f2307873d0 msgr2=0x55f2307877b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:31.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.242+0000 7f6477e8f740 1 -- 172.21.15.123:0/3306211475 shutdown_connections 2024-09-19T04:47:31.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.242+0000 7f6477e8f740 1 -- 172.21.15.123:0/3306211475 wait complete. 2024-09-19T04:47:31.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.243+0000 7f6477e8f740 1 -- start start 2024-09-19T04:47:31.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.243+0000 7f6477e8f740 1 Processor -- start 2024-09-19T04:47:31.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.244+0000 7f6477e8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078ae40 0x55f230792e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.244+0000 7f6477e8f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 0x55f2307933c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.244+0000 7f6477e8f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f230796020 0x55f230793900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.244+0000 7f6477e8f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f230798c40 con 0x55f230796020 2024-09-19T04:47:31.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.244+0000 7f6477e8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f2306ac0f0 con 0x55f23078ae40 2024-09-19T04:47:31.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.244+0000 7f6477e8f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f230798dc0 con 0x55f23078b760 2024-09-19T04:47:31.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.244+0000 7f6477e8f740 10 client.0 _ll_register_callbacks cb 0x55f230693bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:31.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.244+0000 7f646e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078ae40 0x55f230792e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.244+0000 7f646de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 0x55f2307933c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.244+0000 7f646e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078ae40 0x55f230792e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:52900/0 (socket says 172.21.15.123:52900) 2024-09-19T04:47:31.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.244+0000 7f646e800640 1 -- 172.21.15.123:0/1551784339 learned_addr learned my addr 172.21.15.123:0/1551784339 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:31.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.244+0000 7f646f200640 1 --2- 172.21.15.123:0/1551784339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f230796020 0x55f230793900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.245+0000 7f646de00640 1 -- 172.21.15.123:0/1551784339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078ae40 msgr2=0x55f230792e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:31.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.245+0000 7f646de00640 1 --2- 172.21.15.123:0/1551784339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078ae40 0x55f230792e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.245+0000 7f646de00640 1 -- 172.21.15.123:0/1551784339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f230796020 msgr2=0x55f230793900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:31.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.245+0000 7f646de00640 1 --2- 172.21.15.123:0/1551784339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f230796020 0x55f230793900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.245+0000 7f646de00640 1 -- 172.21.15.123:0/1551784339 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2304623c0 con 0x55f23078b760 2024-09-19T04:47:31.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.245+0000 7f646de00640 1 --2- 172.21.15.123:0/1551784339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 0x55f2307933c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f647010f1f0 tx=0x7f645c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=885e347da019bc33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:31.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.245+0000 7f644fe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:47:31.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.245+0000 7f644fe00640 1 -- 172.21.15.123:0/1551784339 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c02e060 con 0x55f23078b760 2024-09-19T04:47:31.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.245+0000 7f644fe00640 1 -- 172.21.15.123:0/1551784339 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f645c009040 con 0x55f23078b760 2024-09-19T04:47:31.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.245+0000 7f6477e8f740 1 -- 172.21.15.123:0/1551784339 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6460019b10 con 0x55f23078b760 2024-09-19T04:47:31.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.246+0000 7f644fe00640 1 -- 172.21.15.123:0/1551784339 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f645c021980 con 0x55f23078b760 2024-09-19T04:47:31.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.246+0000 7f6477e8f740 10 client.25378 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:31.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.246+0000 7f6477e8f740 1 -- 172.21.15.123:0/1551784339 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2307a4c60 con 0x55f23078b760 2024-09-19T04:47:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.246+0000 7f644fe00640 1 -- 172.21.15.123:0/1551784339 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6460019b10 con 0x55f23078b760 2024-09-19T04:47:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.246+0000 7f644fe00640 1 -- 172.21.15.123:0/1551784339 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f645c01d070 con 0x55f23078b760 2024-09-19T04:47:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.246+0000 7f6477e8f740 10 client.25378 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.246+0000 7f6477e8f740 10 client.25378 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:47:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.247+0000 7f6477e8f740 20 client.25378 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.247+0000 7f6477e8f740 1 --2- 172.21.15.123:0/1551784339 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55f2307a61c0 0x55f2307a8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.247+0000 7f6477e8f740 4 client.25378 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:47:31.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.247+0000 7f6477e8f740 1 -- 172.21.15.123:0/1551784339 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2306fe950 con 0x55f2307a61c0 2024-09-19T04:47:31.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.247+0000 7f646e800640 1 --2- 172.21.15.123:0/1551784339 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55f2307a61c0 0x55f2307a8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.248+0000 7f644fe00640 10 client.25378 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:47:31.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.248+0000 7f646e800640 1 --2- 172.21.15.123:0/1551784339 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55f2307a61c0 0x55f2307a8600 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c2b9126efdf17950 server_cookie=3ef552fd3c6279a8 in_seq=0 out_seq=0 2024-09-19T04:47:31.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.248+0000 7f644fe00640 1 -- 172.21.15.123:0/1551784339 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f2306fe950 con 0x55f2307a61c0 2024-09-19T04:47:31.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.248+0000 7f644fe00640 10 client.25378 handle_command_reply: tid=0 2024-09-19T04:47:31.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.249+0000 7f6477e8f740 1 client.25378 shutdown 2024-09-19T04:47:31.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.249+0000 7f6477e8f740 1 -- 172.21.15.123:0/1551784339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 msgr2=0x55f2307933c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:31.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.249+0000 7f6477e8f740 1 --2- 172.21.15.123:0/1551784339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 0x55f2307933c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f647010f1f0 tx=0x7f645c004730 comp rx=0 tx=0).stop 2024-09-19T04:47:31.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.249+0000 7f6477e8f740 1 -- 172.21.15.123:0/1551784339 shutdown_connections 2024-09-19T04:47:31.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.249+0000 7f6477e8f740 1 --2- 172.21.15.123:0/1551784339 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55f2307a61c0 0x55f2307a8600 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.249+0000 7f6477e8f740 1 --2- 172.21.15.123:0/1551784339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f230796020 0x55f230793900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.249+0000 7f644fe00640 0 client.25378 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:47:31.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.249+0000 7f6477e8f740 1 --2- 172.21.15.123:0/1551784339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 0x55f2307933c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.249+0000 7f6477e8f740 1 --2- 172.21.15.123:0/1551784339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078ae40 0x55f230792e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.249+0000 7f6477e8f740 1 -- 172.21.15.123:0/1551784339 >> 172.21.15.123:0/1551784339 conn(0x55f2307873d0 msgr2=0x55f23078d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:31.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.249+0000 7f6477e8f740 1 -- 172.21.15.123:0/1551784339 shutdown_connections 2024-09-19T04:47:31.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.250+0000 7f6477e8f740 1 -- 172.21.15.123:0/1551784339 wait complete. 2024-09-19T04:47:31.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.250+0000 7f6477e8f740 20 client.25378 trim_cache size 0 max 16384 2024-09-19T04:47:31.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.269+0000 7f6477e8f740 1 Processor -- start 2024-09-19T04:47:31.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.269+0000 7f6477e8f740 1 -- start start 2024-09-19T04:47:31.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.270+0000 7f6477e8f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f23078ae40 0x55f23078d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.270+0000 7f6477e8f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 0x55f230791770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.270+0000 7f6477e8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f230796020 0x55f2307aded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.270+0000 7f6477e8f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64701b3c50 con 0x55f23078ae40 2024-09-19T04:47:31.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.270+0000 7f6477e8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f64701b5790 con 0x55f230796020 2024-09-19T04:47:31.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.270+0000 7f6477e8f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f64701b4320 con 0x55f23078b760 2024-09-19T04:47:31.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.270+0000 7f646e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f23078ae40 0x55f23078d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.270+0000 7f646de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 0x55f230791770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.270+0000 7f646e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f23078ae40 0x55f23078d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59998/0 (socket says 172.21.15.123:59998) 2024-09-19T04:47:31.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.270+0000 7f646e800640 1 -- 172.21.15.123:0/1913941465 learned_addr learned my addr 172.21.15.123:0/1913941465 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:31.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.270+0000 7f646f200640 1 --2- 172.21.15.123:0/1913941465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f230796020 0x55f2307aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.271+0000 7f646e800640 1 -- 172.21.15.123:0/1913941465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 msgr2=0x55f230791770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:31.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.271+0000 7f646e800640 1 --2- 172.21.15.123:0/1913941465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 0x55f230791770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.271+0000 7f646e800640 1 -- 172.21.15.123:0/1913941465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f230796020 msgr2=0x55f2307aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:31.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.271+0000 7f646e800640 1 --2- 172.21.15.123:0/1913941465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f230796020 0x55f2307aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.271+0000 7f646e800640 1 -- 172.21.15.123:0/1913941465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f230787200 con 0x55f23078ae40 2024-09-19T04:47:31.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.271+0000 7f646e800640 1 --2- 172.21.15.123:0/1913941465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f23078ae40 0x55f23078d8a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6460011d10 tx=0x7f646001af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a81c239501a5f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:31.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.272+0000 7f645aa00640 1 -- 172.21.15.123:0/1913941465 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6460033070 con 0x55f23078ae40 2024-09-19T04:47:31.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.272+0000 7f645aa00640 1 -- 172.21.15.123:0/1913941465 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f646001f040 con 0x55f23078ae40 2024-09-19T04:47:31.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.272+0000 7f645aa00640 1 -- 172.21.15.123:0/1913941465 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646001e040 con 0x55f23078ae40 2024-09-19T04:47:31.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.273+0000 7f6477e8f740 1 -- 172.21.15.123:0/1913941465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f23078ae40 msgr2=0x55f23078d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:31.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.273+0000 7f6477e8f740 1 --2- 172.21.15.123:0/1913941465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f23078ae40 0x55f23078d8a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6460011d10 tx=0x7f646001af00 comp rx=0 tx=0).stop 2024-09-19T04:47:31.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.273+0000 7f6477e8f740 1 -- 172.21.15.123:0/1913941465 shutdown_connections 2024-09-19T04:47:31.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.273+0000 7f6477e8f740 1 --2- 172.21.15.123:0/1913941465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f230796020 0x55f2307aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.273+0000 7f6477e8f740 1 --2- 172.21.15.123:0/1913941465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 0x55f230791770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.273+0000 7f6477e8f740 1 --2- 172.21.15.123:0/1913941465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f23078ae40 0x55f23078d8a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.273+0000 7f6477e8f740 1 -- 172.21.15.123:0/1913941465 >> 172.21.15.123:0/1913941465 conn(0x55f2307873d0 msgr2=0x55f230794f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:31.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.273+0000 7f6477e8f740 1 -- 172.21.15.123:0/1913941465 shutdown_connections 2024-09-19T04:47:31.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.273+0000 7f6477e8f740 1 -- 172.21.15.123:0/1913941465 wait complete. 2024-09-19T04:47:31.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.274+0000 7f6477e8f740 1 -- start start 2024-09-19T04:47:31.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.274+0000 7f6477e8f740 1 Processor -- start 2024-09-19T04:47:31.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.274+0000 7f6477e8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078ae40 0x55f23079e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.275+0000 7f6477e8f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 0x55f23078fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.275+0000 7f6477e8f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f230796020 0x55f2307901a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.275+0000 7f6477e8f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64701b50c0 con 0x55f230796020 2024-09-19T04:47:31.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.275+0000 7f6477e8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f64701b49f0 con 0x55f23078ae40 2024-09-19T04:47:31.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.275+0000 7f6477e8f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f64701b1c30 con 0x55f23078b760 2024-09-19T04:47:31.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.275+0000 7f6477e8f740 10 client.0 _ll_register_callbacks cb 0x7f645c003f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:31.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.275+0000 7f646e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078ae40 0x55f23079e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.275+0000 7f646f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f230796020 0x55f2307901a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.275+0000 7f646de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 0x55f23078fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.275+0000 7f646e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078ae40 0x55f23079e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:52914/0 (socket says 172.21.15.123:52914) 2024-09-19T04:47:31.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.275+0000 7f646de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 0x55f23078fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:48250/0 (socket says 172.21.15.123:48250) 2024-09-19T04:47:31.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.275+0000 7f646de00640 1 -- 172.21.15.123:0/2221618339 learned_addr learned my addr 172.21.15.123:0/2221618339 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:31.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.276+0000 7f646de00640 1 -- 172.21.15.123:0/2221618339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078ae40 msgr2=0x55f23079e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:31.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.276+0000 7f646de00640 1 --2- 172.21.15.123:0/2221618339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078ae40 0x55f23079e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.276+0000 7f646de00640 1 -- 172.21.15.123:0/2221618339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f230796020 msgr2=0x55f2307901a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:31.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.276+0000 7f646de00640 1 --2- 172.21.15.123:0/2221618339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f230796020 0x55f2307901a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.276+0000 7f646de00640 1 -- 172.21.15.123:0/2221618339 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2307909d0 con 0x55f23078b760 2024-09-19T04:47:31.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.276+0000 7f646f200640 1 --2- 172.21.15.123:0/2221618339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f230796020 0x55f2307901a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:31.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.276+0000 7f646de00640 1 --2- 172.21.15.123:0/2221618339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 0x55f23078fc60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f645c003f40 tx=0x7f645c01e6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7de771e7d59517 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:31.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.276+0000 7f644fe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:47:31.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.277+0000 7f644fe00640 1 -- 172.21.15.123:0/2221618339 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c02e070 con 0x55f23078b760 2024-09-19T04:47:31.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.277+0000 7f644fe00640 1 -- 172.21.15.123:0/2221618339 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f645c01a070 con 0x55f23078b760 2024-09-19T04:47:31.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.277+0000 7f6477e8f740 1 -- 172.21.15.123:0/2221618339 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f646001f040 con 0x55f23078b760 2024-09-19T04:47:31.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.277+0000 7f644fe00640 1 -- 172.21.15.123:0/2221618339 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f645c040050 con 0x55f23078b760 2024-09-19T04:47:31.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.277+0000 7f6477e8f740 10 client.25384 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:31.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.277+0000 7f6477e8f740 1 -- 172.21.15.123:0/2221618339 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f230810f00 con 0x55f23078b760 2024-09-19T04:47:31.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.278+0000 7f644fe00640 1 -- 172.21.15.123:0/2221618339 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f646001f040 con 0x55f23078b760 2024-09-19T04:47:31.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.278+0000 7f644fe00640 1 -- 172.21.15.123:0/2221618339 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f645c01d040 con 0x55f23078b760 2024-09-19T04:47:31.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.279+0000 7f6477e8f740 10 client.25384 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:31.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.279+0000 7f6477e8f740 10 client.25384 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:47:31.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.279+0000 7f6477e8f740 20 client.25384 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:31.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.279+0000 7f6477e8f740 1 --2- 172.21.15.123:0/2221618339 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55f2308115c0 0x55f2308139b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:31.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.279+0000 7f6477e8f740 4 client.25384 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:47:31.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.279+0000 7f6477e8f740 1 -- 172.21.15.123:0/2221618339 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2306f8e60 con 0x55f2308115c0 2024-09-19T04:47:31.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.279+0000 7f646e800640 1 --2- 172.21.15.123:0/2221618339 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55f2308115c0 0x55f2308139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:31.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.280+0000 7f646e800640 1 --2- 172.21.15.123:0/2221618339 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55f2308115c0 0x55f2308139b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=99d6e99ef1ae4e4e server_cookie=e5362db37b0fcda3 in_seq=0 out_seq=0 2024-09-19T04:47:31.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.280+0000 7f644fe00640 10 client.25384 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:47:31.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.280+0000 7f644fe00640 1 -- 172.21.15.123:0/2221618339 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2306f8e60 con 0x55f2308115c0 2024-09-19T04:47:31.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.280+0000 7f644fe00640 10 client.25384 handle_command_reply: tid=0 2024-09-19T04:47:31.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.280+0000 7f6477e8f740 1 client.25384 shutdown 2024-09-19T04:47:31.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.280+0000 7f6477e8f740 1 -- 172.21.15.123:0/2221618339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 msgr2=0x55f23078fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:31.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.280+0000 7f6477e8f740 1 --2- 172.21.15.123:0/2221618339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 0x55f23078fc60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f645c003f40 tx=0x7f645c01e6a0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.280+0000 7f6477e8f740 1 -- 172.21.15.123:0/2221618339 shutdown_connections 2024-09-19T04:47:31.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.280+0000 7f6477e8f740 1 --2- 172.21.15.123:0/2221618339 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55f2308115c0 0x55f2308139b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.280+0000 7f6477e8f740 1 --2- 172.21.15.123:0/2221618339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f230796020 0x55f2307901a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.280+0000 7f6477e8f740 1 --2- 172.21.15.123:0/2221618339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f23078b760 0x55f23078fc60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.280+0000 7f6477e8f740 1 --2- 172.21.15.123:0/2221618339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f23078ae40 0x55f23079e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.280+0000 7f6477e8f740 1 -- 172.21.15.123:0/2221618339 >> 172.21.15.123:0/2221618339 conn(0x55f2307873d0 msgr2=0x55f230785c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:31.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.280+0000 7f644fe00640 0 client.25384 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:47:31.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.281+0000 7f6477e8f740 1 -- 172.21.15.123:0/2221618339 shutdown_connections 2024-09-19T04:47:31.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.281+0000 7f6477e8f740 1 -- 172.21.15.123:0/2221618339 wait complete. 2024-09-19T04:47:31.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.281+0000 7f6477e8f740 20 client.25384 trim_cache size 0 max 16384 2024-09-19T04:47:31.285 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:47:31.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.282+0000 7f6474a00640 1 -- 172.21.15.123:0/3327854372 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f64540a35e0 msgr2=0x7f64540a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:31.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.282+0000 7f6474a00640 1 --2- 172.21.15.123:0/3327854372 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f64540a35e0 0x7f64540a5aa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55f230788750 tx=0x7f645c0096c0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.282+0000 7f6474a00640 1 -- 172.21.15.123:0/3327854372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6470108f50 msgr2=0x7f647010df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:31.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.282+0000 7f6474a00640 1 --2- 172.21.15.123:0/3327854372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6470108f50 0x7f647010df90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6468006010 tx=0x7f6468004a60 comp rx=0 tx=0).stop 2024-09-19T04:47:31.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.282+0000 7f6474a00640 1 -- 172.21.15.123:0/3327854372 shutdown_connections 2024-09-19T04:47:31.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.282+0000 7f6474a00640 1 --2- 172.21.15.123:0/3327854372 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f64540a35e0 0x7f64540a5aa0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.282+0000 7f6474a00640 1 --2- 172.21.15.123:0/3327854372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6470108f50 0x7f647010df90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.282+0000 7f6474a00640 1 --2- 172.21.15.123:0/3327854372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64701047d0 0x7f647010da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.282+0000 7f6474a00640 1 --2- 172.21.15.123:0/3327854372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6470103e20 0x7f6470110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:31.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.282+0000 7f6474a00640 1 -- 172.21.15.123:0/3327854372 >> 172.21.15.123:0/3327854372 conn(0x7f64700ff880 msgr2=0x7f6470100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:31.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.282+0000 7f6474a00640 1 -- 172.21.15.123:0/3327854372 shutdown_connections 2024-09-19T04:47:31.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:31.282+0000 7f6474a00640 1 -- 172.21.15.123:0/3327854372 wait complete. 2024-09-19T04:47:31.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:31 smithi123 ceph-mon[30511]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:47:31.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:31 smithi183 ceph-mon[35333]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:47:31.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:31 smithi193 ceph-mon[35592]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:47:32.361 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:47:32.789 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:47:33.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.415+0000 7f83fd400640 1 -- 172.21.15.123:0/2315931864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f81019a0 msgr2=0x7f83f8101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:33.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.415+0000 7f83fd400640 1 --2- 172.21.15.123:0/2315931864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f81019a0 0x7f83f8101e00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009930 tx=0x7f83e402a420 comp rx=0 tx=0).stop 2024-09-19T04:47:33.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.416+0000 7f83fd400640 1 -- 172.21.15.123:0/2315931864 shutdown_connections 2024-09-19T04:47:33.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.416+0000 7f83fd400640 1 --2- 172.21.15.123:0/2315931864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f8102340 0x7f83f810e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.416+0000 7f83fd400640 1 --2- 172.21.15.123:0/2315931864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f81019a0 0x7f83f8101e00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.416+0000 7f83fd400640 1 --2- 172.21.15.123:0/2315931864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83f81079a0 0x7f83f8107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.416+0000 7f83fd400640 1 -- 172.21.15.123:0/2315931864 >> 172.21.15.123:0/2315931864 conn(0x7f83f80fd700 msgr2=0x7f83f80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:33.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.417+0000 7f83fd400640 1 -- 172.21.15.123:0/2315931864 shutdown_connections 2024-09-19T04:47:33.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.417+0000 7f83fd400640 1 -- 172.21.15.123:0/2315931864 wait complete. 2024-09-19T04:47:33.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.418+0000 7f83fd400640 1 Processor -- start 2024-09-19T04:47:33.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.418+0000 7f83fd400640 1 -- start start 2024-09-19T04:47:33.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.419+0000 7f83fd400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f81019a0 0x7f83f819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.419+0000 7f83fd400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f8102340 0x7f83f819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.419+0000 7f83fd400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83f81079a0 0x7f83f819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.419+0000 7f83fd400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83f8111df0 con 0x7f83f8102340 2024-09-19T04:47:33.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.419+0000 7f83fd400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f83f8111c70 con 0x7f83f81079a0 2024-09-19T04:47:33.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.419+0000 7f83fd400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f83f8111f70 con 0x7f83f81019a0 2024-09-19T04:47:33.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.419+0000 7f83f7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f81019a0 0x7f83f819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:33.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.419+0000 7f83f7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f81019a0 0x7f83f819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58656/0 (socket says 172.21.15.123:58656) 2024-09-19T04:47:33.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.419+0000 7f83f7e00640 1 -- 172.21.15.123:0/2126495476 learned_addr learned my addr 172.21.15.123:0/2126495476 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:33.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.420+0000 7f83fca00640 1 --2- 172.21.15.123:0/2126495476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83f81079a0 0x7f83f819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:33.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.420+0000 7f83f7400640 1 --2- 172.21.15.123:0/2126495476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f8102340 0x7f83f819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:33.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.420+0000 7f83f7e00640 1 -- 172.21.15.123:0/2126495476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83f81079a0 msgr2=0x7f83f819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:33.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.420+0000 7f83f7e00640 1 --2- 172.21.15.123:0/2126495476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83f81079a0 0x7f83f819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.420+0000 7f83f7e00640 1 -- 172.21.15.123:0/2126495476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f8102340 msgr2=0x7f83f819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:33.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.420+0000 7f83f7e00640 1 --2- 172.21.15.123:0/2126495476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f8102340 0x7f83f819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.420+0000 7f83f7e00640 1 -- 172.21.15.123:0/2126495476 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f819f440 con 0x7f83f81019a0 2024-09-19T04:47:33.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.420+0000 7f83f7400640 1 --2- 172.21.15.123:0/2126495476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f8102340 0x7f83f819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:47:33.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.421+0000 7f83f7e00640 1 --2- 172.21.15.123:0/2126495476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f81019a0 0x7f83f819a490 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f83e800e9f0 tx=0x7f83e800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:33.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.421+0000 7f83f4c00640 1 -- 172.21.15.123:0/2126495476 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e800cd80 con 0x7f83f81019a0 2024-09-19T04:47:33.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.421+0000 7f83f4c00640 1 -- 172.21.15.123:0/2126495476 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e8010910 con 0x7f83f81019a0 2024-09-19T04:47:33.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.421+0000 7f83fd400640 1 -- 172.21.15.123:0/2126495476 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f819f730 con 0x7f83f81019a0 2024-09-19T04:47:33.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.421+0000 7f83f4c00640 1 -- 172.21.15.123:0/2126495476 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8018990 con 0x7f83f81019a0 2024-09-19T04:47:33.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.422+0000 7f83fd400640 1 -- 172.21.15.123:0/2126495476 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f81a7010 con 0x7f83f81019a0 2024-09-19T04:47:33.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.426+0000 7f840144f740 1 Processor -- start 2024-09-19T04:47:33.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.426+0000 7f840144f740 1 -- start start 2024-09-19T04:47:33.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.426+0000 7f840144f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a6e40 0x55d8471a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.426+0000 7f840144f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471a7760 0x55d8471b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.426+0000 7f840144f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471b2020 0x55d8471b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.426+0000 7f840144f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d84710d3e0 con 0x55d8471a7760 2024-09-19T04:47:33.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.426+0000 7f840144f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d8470b4d80 con 0x55d8471a6e40 2024-09-19T04:47:33.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.426+0000 7f840144f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d8471093a0 con 0x55d8471b2020 2024-09-19T04:47:33.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.427+0000 7f83fca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a6e40 0x55d8471a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:33.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.427+0000 7f83fca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a6e40 0x55d8471a7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:43294/0 (socket says 172.21.15.123:43294) 2024-09-19T04:47:33.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.427+0000 7f83fca00640 1 -- 172.21.15.123:0/302583742 learned_addr learned my addr 172.21.15.123:0/302583742 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:33.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.427+0000 7f83f7400640 1 --2- 172.21.15.123:0/302583742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471b2020 0x55d8471b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:33.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.427+0000 7f83fca00640 1 -- 172.21.15.123:0/302583742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471b2020 msgr2=0x55d8471b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:33.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.427+0000 7f83fca00640 1 --2- 172.21.15.123:0/302583742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471b2020 0x55d8471b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.427+0000 7f83f4c00640 1 -- 172.21.15.123:0/2126495476 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7f83e8010ac0 con 0x7f83f81019a0 2024-09-19T04:47:33.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.427+0000 7f83fca00640 1 -- 172.21.15.123:0/302583742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471a7760 msgr2=0x55d8471b1ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:47:33.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.427+0000 7f83fca00640 1 --2- 172.21.15.123:0/302583742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471a7760 0x55d8471b1ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.427+0000 7f83fca00640 1 -- 172.21.15.123:0/302583742 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8470d3ae0 con 0x55d8471a6e40 2024-09-19T04:47:33.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.428+0000 7f83fca00640 1 --2- 172.21.15.123:0/302583742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a6e40 0x55d8471a7220 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009fd0 tx=0x7f83f00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbfb68269cfb305a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:33.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.428+0000 7f83ed200640 1 -- 172.21.15.123:0/302583742 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f0004d20 con 0x55d8471a6e40 2024-09-19T04:47:33.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.428+0000 7f83ed200640 1 -- 172.21.15.123:0/302583742 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83f0004050 con 0x55d8471a6e40 2024-09-19T04:47:33.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.428+0000 7f83ed200640 1 -- 172.21.15.123:0/302583742 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f0005280 con 0x55d8471a6e40 2024-09-19T04:47:33.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.429+0000 7f840144f740 1 -- 172.21.15.123:0/302583742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a6e40 msgr2=0x55d8471a7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:33.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.429+0000 7f840144f740 1 --2- 172.21.15.123:0/302583742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a6e40 0x55d8471a7220 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009fd0 tx=0x7f83f00118d0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.429+0000 7f83f4c00640 1 --2- 172.21.15.123:0/2126495476 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f83dc0a35e0 0x7f83dc0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.430+0000 7f83f4c00640 1 -- 172.21.15.123:0/2126495476 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f83e80d0260 con 0x7f83f81019a0 2024-09-19T04:47:33.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.430+0000 7f840144f740 1 -- 172.21.15.123:0/302583742 shutdown_connections 2024-09-19T04:47:33.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.430+0000 7f840144f740 1 --2- 172.21.15.123:0/302583742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471b2020 0x55d8471b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.430+0000 7f840144f740 1 --2- 172.21.15.123:0/302583742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471a7760 0x55d8471b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.430+0000 7f840144f740 1 --2- 172.21.15.123:0/302583742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a6e40 0x55d8471a7220 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.430+0000 7f840144f740 1 -- 172.21.15.123:0/302583742 >> 172.21.15.123:0/302583742 conn(0x55d8471a33d0 msgr2=0x55d8471a37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:33.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.430+0000 7f83f7e00640 1 --2- 172.21.15.123:0/2126495476 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f83dc0a35e0 0x7f83dc0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:33.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.431+0000 7f840144f740 1 -- 172.21.15.123:0/302583742 shutdown_connections 2024-09-19T04:47:33.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.431+0000 7f840144f740 1 -- 172.21.15.123:0/302583742 wait complete. 2024-09-19T04:47:33.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.431+0000 7f83f7e00640 1 --2- 172.21.15.123:0/2126495476 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f83dc0a35e0 0x7f83dc0a5aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f83e80cd800 tx=0x7f83e8009310 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:33.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.431+0000 7f840144f740 1 -- start start 2024-09-19T04:47:33.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.432+0000 7f840144f740 1 Processor -- start 2024-09-19T04:47:33.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.432+0000 7f840144f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471a6e40 0x55d8471aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.432+0000 7f840144f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471a7760 0x55d8471af500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.432+0000 7f840144f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471b2020 0x55d8471afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.432+0000 7f840144f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d8471c5c00 con 0x55d8471a6e40 2024-09-19T04:47:33.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.432+0000 7f840144f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d8470c80f0 con 0x55d8471b2020 2024-09-19T04:47:33.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.432+0000 7f840144f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d8471c5d80 con 0x55d8471a7760 2024-09-19T04:47:33.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.432+0000 7f840144f740 10 client.0 _ll_register_callbacks cb 0x55d8470afbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:33.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.433+0000 7f83f7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471a7760 0x55d8471af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:33.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.433+0000 7f83f7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471a7760 0x55d8471af500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58676/0 (socket says 172.21.15.123:58676) 2024-09-19T04:47:33.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.433+0000 7f83f7400640 1 -- 172.21.15.123:0/3383685950 learned_addr learned my addr 172.21.15.123:0/3383685950 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:33.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.433+0000 7f83fca00640 1 --2- 172.21.15.123:0/3383685950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471a6e40 0x55d8471aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:33.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.433+0000 7f83f7400640 1 -- 172.21.15.123:0/3383685950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471b2020 msgr2=0x55d8471afa40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:47:33.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.433+0000 7f83f7400640 1 --2- 172.21.15.123:0/3383685950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471b2020 0x55d8471afa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.433+0000 7f83f7400640 1 -- 172.21.15.123:0/3383685950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471a6e40 msgr2=0x55d8471aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:33.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.433+0000 7f83f7400640 1 --2- 172.21.15.123:0/3383685950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471a6e40 0x55d8471aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.433+0000 7f83f7400640 1 -- 172.21.15.123:0/3383685950 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d846e7e3c0 con 0x55d8471a7760 2024-09-19T04:47:33.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.433+0000 7f83fca00640 1 --2- 172.21.15.123:0/3383685950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471a6e40 0x55d8471aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:47:33.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.434+0000 7f83f7400640 1 --2- 172.21.15.123:0/3383685950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471a7760 0x55d8471af500 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009230 tx=0x7f83e4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9daa8202d3d38b23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:33.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.434+0000 7f83e1600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:47:33.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.434+0000 7f83e1600640 1 -- 172.21.15.123:0/3383685950 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4035040 con 0x55d8471a7760 2024-09-19T04:47:33.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.434+0000 7f840144f740 1 -- 172.21.15.123:0/3383685950 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83f0004050 con 0x55d8471a7760 2024-09-19T04:47:33.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.434+0000 7f83e1600640 1 -- 172.21.15.123:0/3383685950 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f83e4005230 con 0x55d8471a7760 2024-09-19T04:47:33.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.434+0000 7f83e1600640 1 -- 172.21.15.123:0/3383685950 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f83e4005430 con 0x55d8471a7760 2024-09-19T04:47:33.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.434+0000 7f840144f740 10 client.25396 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:33.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.434+0000 7f840144f740 1 -- 172.21.15.123:0/3383685950 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8471da540 con 0x55d8471a7760 2024-09-19T04:47:33.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.435+0000 7f83e1600640 1 -- 172.21.15.123:0/3383685950 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83f0004050 con 0x55d8471a7760 2024-09-19T04:47:33.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.435+0000 7f83e1600640 1 -- 172.21.15.123:0/3383685950 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f83e4003950 con 0x55d8471a7760 2024-09-19T04:47:33.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.435+0000 7f840144f740 10 client.25396 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:33.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.435+0000 7f840144f740 10 client.25396 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:47:33.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.435+0000 7f840144f740 20 client.25396 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:33.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.435+0000 7f840144f740 1 --2- 172.21.15.123:0/3383685950 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55d8471db950 0x55d8471dddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.435+0000 7f840144f740 4 client.25396 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:47:33.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.435+0000 7f840144f740 1 -- 172.21.15.123:0/3383685950 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d84711a950 con 0x55d8471db950 2024-09-19T04:47:33.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.435+0000 7f83fca00640 1 --2- 172.21.15.123:0/3383685950 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55d8471db950 0x55d8471dddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:33.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.436+0000 7f83fca00640 1 --2- 172.21.15.123:0/3383685950 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55d8471db950 0x55d8471dddf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b35249de7198faf2 server_cookie=8c0d33b23d4762a7 in_seq=0 out_seq=0 2024-09-19T04:47:33.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.436+0000 7f83e1600640 10 client.25396 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:47:33.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.437+0000 7f83e1600640 1 -- 172.21.15.123:0/3383685950 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d84711a950 con 0x55d8471db950 2024-09-19T04:47:33.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.437+0000 7f83e1600640 10 client.25396 handle_command_reply: tid=0 2024-09-19T04:47:33.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.437+0000 7f840144f740 1 client.25396 shutdown 2024-09-19T04:47:33.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.437+0000 7f840144f740 1 -- 172.21.15.123:0/3383685950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471a7760 msgr2=0x55d8471af500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:33.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.437+0000 7f840144f740 1 --2- 172.21.15.123:0/3383685950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471a7760 0x55d8471af500 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009230 tx=0x7f83e4004270 comp rx=0 tx=0).stop 2024-09-19T04:47:33.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.437+0000 7f840144f740 1 -- 172.21.15.123:0/3383685950 shutdown_connections 2024-09-19T04:47:33.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.437+0000 7f840144f740 1 --2- 172.21.15.123:0/3383685950 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55d8471db950 0x55d8471dddf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.437+0000 7f840144f740 1 --2- 172.21.15.123:0/3383685950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471b2020 0x55d8471afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.437+0000 7f840144f740 1 --2- 172.21.15.123:0/3383685950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471a7760 0x55d8471af500 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.437+0000 7f840144f740 1 --2- 172.21.15.123:0/3383685950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471a6e40 0x55d8471aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.438+0000 7f840144f740 1 -- 172.21.15.123:0/3383685950 >> 172.21.15.123:0/3383685950 conn(0x55d8471a33d0 msgr2=0x55d8471a9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:33.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.438+0000 7f83e1600640 0 client.25396 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:47:33.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.438+0000 7f840144f740 1 -- 172.21.15.123:0/3383685950 shutdown_connections 2024-09-19T04:47:33.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.438+0000 7f840144f740 1 -- 172.21.15.123:0/3383685950 wait complete. 2024-09-19T04:47:33.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.438+0000 7f840144f740 20 client.25396 trim_cache size 0 max 16384 2024-09-19T04:47:33.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.464+0000 7f840144f740 1 Processor -- start 2024-09-19T04:47:33.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.464+0000 7f840144f740 1 -- start start 2024-09-19T04:47:33.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.464+0000 7f840144f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471a6e40 0x55d8471af890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.464+0000 7f840144f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a7760 0x55d8471a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.465+0000 7f840144f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471b2020 0x55d8471e36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.465+0000 7f840144f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83f806ff20 con 0x55d8471b2020 2024-09-19T04:47:33.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.465+0000 7f840144f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f83f8071a60 con 0x55d8471a7760 2024-09-19T04:47:33.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.465+0000 7f840144f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f83f80705f0 con 0x55d8471a6e40 2024-09-19T04:47:33.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.465+0000 7f83fca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471a6e40 0x55d8471af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:33.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.465+0000 7f83f7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a7760 0x55d8471a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:33.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.465+0000 7f83fca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471a6e40 0x55d8471af890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58680/0 (socket says 172.21.15.123:58680) 2024-09-19T04:47:33.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.465+0000 7f83fca00640 1 -- 172.21.15.123:0/884150186 learned_addr learned my addr 172.21.15.123:0/884150186 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:33.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.465+0000 7f83fca00640 1 --2- 172.21.15.123:0/884150186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471b2020 0x55d8471e36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:33.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.465+0000 7f83f7400640 1 -- 172.21.15.123:0/884150186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471a6e40 msgr2=0x55d8471af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:33.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.465+0000 7f83f7400640 1 --2- 172.21.15.123:0/884150186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471a6e40 0x55d8471af890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.465+0000 7f83f7400640 1 -- 172.21.15.123:0/884150186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471b2020 msgr2=0x55d8471e36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:33.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.465+0000 7f83f7400640 1 --2- 172.21.15.123:0/884150186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471b2020 0x55d8471e36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.465+0000 7f83f7400640 1 -- 172.21.15.123:0/884150186 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8471b4410 con 0x55d8471a7760 2024-09-19T04:47:33.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.466+0000 7f83f7400640 1 --2- 172.21.15.123:0/884150186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a7760 0x55d8471a8fa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f83e4003ec0 tx=0x7f83e402a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57986ba8d127fa4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:33.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.467+0000 7f83e3e00640 1 -- 172.21.15.123:0/884150186 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4000e60 con 0x55d8471a7760 2024-09-19T04:47:33.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.467+0000 7f83e3e00640 1 -- 172.21.15.123:0/884150186 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e4003ad0 con 0x55d8471a7760 2024-09-19T04:47:33.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.467+0000 7f83e3e00640 1 -- 172.21.15.123:0/884150186 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4035070 con 0x55d8471a7760 2024-09-19T04:47:33.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.468+0000 7f840144f740 1 -- 172.21.15.123:0/884150186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a7760 msgr2=0x55d8471a8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:33.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.468+0000 7f840144f740 1 --2- 172.21.15.123:0/884150186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a7760 0x55d8471a8fa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f83e4003ec0 tx=0x7f83e402a420 comp rx=0 tx=0).stop 2024-09-19T04:47:33.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.468+0000 7f840144f740 1 -- 172.21.15.123:0/884150186 shutdown_connections 2024-09-19T04:47:33.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.468+0000 7f840144f740 1 --2- 172.21.15.123:0/884150186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471b2020 0x55d8471e36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.468+0000 7f840144f740 1 --2- 172.21.15.123:0/884150186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a7760 0x55d8471a8fa0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.468+0000 7f840144f740 1 --2- 172.21.15.123:0/884150186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471a6e40 0x55d8471af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.468+0000 7f840144f740 1 -- 172.21.15.123:0/884150186 >> 172.21.15.123:0/884150186 conn(0x55d8471a33d0 msgr2=0x55d8471a8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:33.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.468+0000 7f840144f740 1 -- 172.21.15.123:0/884150186 shutdown_connections 2024-09-19T04:47:33.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.468+0000 7f840144f740 1 -- 172.21.15.123:0/884150186 wait complete. 2024-09-19T04:47:33.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.469+0000 7f840144f740 1 -- start start 2024-09-19T04:47:33.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.469+0000 7f840144f740 1 Processor -- start 2024-09-19T04:47:33.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.470+0000 7f840144f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471a6e40 0x55d8471d4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.470+0000 7f840144f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a7760 0x55d8471abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.470+0000 7f840144f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471b2020 0x55d8471ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.470+0000 7f840144f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83f8071390 con 0x55d8471a6e40 2024-09-19T04:47:33.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.470+0000 7f840144f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f83f8070cc0 con 0x55d8471a7760 2024-09-19T04:47:33.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.470+0000 7f840144f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f83f806df40 con 0x55d8471b2020 2024-09-19T04:47:33.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.470+0000 7f840144f740 10 client.0 _ll_register_callbacks cb 0x7f83f00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:33.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.470+0000 7f83fca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471a6e40 0x55d8471d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:33.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.470+0000 7f83f7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a7760 0x55d8471abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:33.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.470+0000 7f83fca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471a6e40 0x55d8471d4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53524/0 (socket says 172.21.15.123:53524) 2024-09-19T04:47:33.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.470+0000 7f83fca00640 1 -- 172.21.15.123:0/3543601243 learned_addr learned my addr 172.21.15.123:0/3543601243 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:33.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.471+0000 7f83fca00640 1 --2- 172.21.15.123:0/3543601243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471b2020 0x55d8471ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:33.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.471+0000 7f83f7400640 1 -- 172.21.15.123:0/3543601243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471b2020 msgr2=0x55d8471ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:33.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.471+0000 7f83f7400640 1 --2- 172.21.15.123:0/3543601243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471b2020 0x55d8471ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.471+0000 7f83f7400640 1 -- 172.21.15.123:0/3543601243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471a6e40 msgr2=0x55d8471d4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:33.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.471+0000 7f83f7400640 1 --2- 172.21.15.123:0/3543601243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471a6e40 0x55d8471d4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.471+0000 7f83f7400640 1 -- 172.21.15.123:0/3543601243 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8471aca60 con 0x55d8471a7760 2024-09-19T04:47:33.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.471+0000 7f83fca00640 1 --2- 172.21.15.123:0/3543601243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471a6e40 0x55d8471d4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:47:33.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.471+0000 7f83f7400640 1 --2- 172.21.15.123:0/3543601243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a7760 0x55d8471abcf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55d8471d89e0 tx=0x7f83e4000e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4271fc86092cdb7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:33.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.471+0000 7f83e1600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:47:33.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.472+0000 7f83e1600640 1 -- 172.21.15.123:0/3543601243 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4035070 con 0x55d8471a7760 2024-09-19T04:47:33.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.472+0000 7f83e1600640 1 -- 172.21.15.123:0/3543601243 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f83e40037f0 con 0x55d8471a7760 2024-09-19T04:47:33.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.472+0000 7f840144f740 1 -- 172.21.15.123:0/3543601243 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83e4003ad0 con 0x55d8471a7760 2024-09-19T04:47:33.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.472+0000 7f83e1600640 1 -- 172.21.15.123:0/3543601243 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f83e4005650 con 0x55d8471a7760 2024-09-19T04:47:33.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.472+0000 7f840144f740 10 client.25397 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:33.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.472+0000 7f840144f740 1 -- 172.21.15.123:0/3543601243 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d847246660 con 0x55d8471a7760 2024-09-19T04:47:33.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.472+0000 7f83e1600640 1 -- 172.21.15.123:0/3543601243 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e4003ad0 con 0x55d8471a7760 2024-09-19T04:47:33.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.473+0000 7f83e1600640 1 -- 172.21.15.123:0/3543601243 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f83e40318f0 con 0x55d8471a7760 2024-09-19T04:47:33.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.473+0000 7f840144f740 10 client.25397 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:33.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.473+0000 7f840144f740 10 client.25397 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:47:33.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.473+0000 7f840144f740 20 client.25397 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:33.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.473+0000 7f840144f740 1 --2- 172.21.15.123:0/3543601243 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55d847246d10 0x55d847249130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:33.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.473+0000 7f840144f740 4 client.25397 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:47:33.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.473+0000 7f840144f740 1 -- 172.21.15.123:0/3543601243 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d847114e60 con 0x55d847246d10 2024-09-19T04:47:33.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.473+0000 7f83fca00640 1 --2- 172.21.15.123:0/3543601243 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55d847246d10 0x55d847249130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:33.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.474+0000 7f83fca00640 1 --2- 172.21.15.123:0/3543601243 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55d847246d10 0x55d847249130 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4990cfb0de24ee39 server_cookie=3505e9f95a71084f in_seq=0 out_seq=0 2024-09-19T04:47:33.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.474+0000 7f83e1600640 10 client.25397 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:47:33.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.474+0000 7f83e1600640 1 -- 172.21.15.123:0/3543601243 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d847114e60 con 0x55d847246d10 2024-09-19T04:47:33.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.474+0000 7f83e1600640 10 client.25397 handle_command_reply: tid=0 2024-09-19T04:47:33.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.474+0000 7f840144f740 1 client.25397 shutdown 2024-09-19T04:47:33.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.474+0000 7f840144f740 1 -- 172.21.15.123:0/3543601243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a7760 msgr2=0x55d8471abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:33.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.474+0000 7f840144f740 1 --2- 172.21.15.123:0/3543601243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a7760 0x55d8471abcf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55d8471d89e0 tx=0x7f83e4000e60 comp rx=0 tx=0).stop 2024-09-19T04:47:33.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.475+0000 7f840144f740 1 -- 172.21.15.123:0/3543601243 shutdown_connections 2024-09-19T04:47:33.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.475+0000 7f840144f740 1 --2- 172.21.15.123:0/3543601243 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55d847246d10 0x55d847249130 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.475+0000 7f840144f740 1 --2- 172.21.15.123:0/3543601243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8471b2020 0x55d8471ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.475+0000 7f840144f740 1 --2- 172.21.15.123:0/3543601243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8471a7760 0x55d8471abcf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.475+0000 7f840144f740 1 --2- 172.21.15.123:0/3543601243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8471a6e40 0x55d8471d4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.475+0000 7f840144f740 1 -- 172.21.15.123:0/3543601243 >> 172.21.15.123:0/3543601243 conn(0x55d8471a33d0 msgr2=0x55d8471b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:33.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.475+0000 7f83e1600640 0 client.25397 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:47:33.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.475+0000 7f840144f740 1 -- 172.21.15.123:0/3543601243 shutdown_connections 2024-09-19T04:47:33.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.475+0000 7f840144f740 1 -- 172.21.15.123:0/3543601243 wait complete. 2024-09-19T04:47:33.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.476+0000 7f840144f740 20 client.25397 trim_cache size 0 max 16384 2024-09-19T04:47:33.480 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:47:33.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.477+0000 7f83fd400640 1 -- 172.21.15.123:0/2126495476 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f83dc0a35e0 msgr2=0x7f83dc0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:33.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.477+0000 7f83fd400640 1 --2- 172.21.15.123:0/2126495476 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f83dc0a35e0 0x7f83dc0a5aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f83e80cd800 tx=0x7f83e8009310 comp rx=0 tx=0).stop 2024-09-19T04:47:33.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.477+0000 7f83fd400640 1 -- 172.21.15.123:0/2126495476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f81019a0 msgr2=0x7f83f819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:33.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.477+0000 7f83fd400640 1 --2- 172.21.15.123:0/2126495476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f81019a0 0x7f83f819a490 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f83e800e9f0 tx=0x7f83e800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.477+0000 7f83fd400640 1 -- 172.21.15.123:0/2126495476 shutdown_connections 2024-09-19T04:47:33.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.477+0000 7f83fd400640 1 --2- 172.21.15.123:0/2126495476 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f83dc0a35e0 0x7f83dc0a5aa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.477+0000 7f83fd400640 1 --2- 172.21.15.123:0/2126495476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83f81079a0 0x7f83f819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.477+0000 7f83fd400640 1 --2- 172.21.15.123:0/2126495476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f8102340 0x7f83f819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.477+0000 7f83fd400640 1 --2- 172.21.15.123:0/2126495476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f81019a0 0x7f83f819a490 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:33.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.477+0000 7f83fd400640 1 -- 172.21.15.123:0/2126495476 >> 172.21.15.123:0/2126495476 conn(0x7f83f80fd700 msgr2=0x7f83f810cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:33.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.478+0000 7f83fd400640 1 -- 172.21.15.123:0/2126495476 shutdown_connections 2024-09-19T04:47:33.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:33.478+0000 7f83fd400640 1 -- 172.21.15.123:0/2126495476 wait complete. 2024-09-19T04:47:33.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:33 smithi183 ceph-mon[35333]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:47:33.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:33 smithi123 ceph-mon[30511]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:47:33.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:33 smithi193 ceph-mon[35592]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:47:34.215 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:47:34.642 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:47:35.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.354+0000 7fae1d200640 1 -- 172.21.15.123:0/1233609553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae181049c0 msgr2=0x7fae18108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:35.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.354+0000 7fae1d200640 1 --2- 172.21.15.123:0/1233609553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae181049c0 0x7fae18108a10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fae04009900 tx=0x7fae0402a400 comp rx=0 tx=0).stop 2024-09-19T04:47:35.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.354+0000 7fae1d200640 1 -- 172.21.15.123:0/1233609553 shutdown_connections 2024-09-19T04:47:35.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.354+0000 7fae1d200640 1 --2- 172.21.15.123:0/1233609553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae18109140 0x7fae18110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.354+0000 7fae1d200640 1 --2- 172.21.15.123:0/1233609553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae181049c0 0x7fae18108a10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.354+0000 7fae1d200640 1 --2- 172.21.15.123:0/1233609553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae18104010 0x7fae181043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.354+0000 7fae1d200640 1 -- 172.21.15.123:0/1233609553 >> 172.21.15.123:0/1233609553 conn(0x7fae180ffa50 msgr2=0x7fae18101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:35.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.354+0000 7fae1d200640 1 -- 172.21.15.123:0/1233609553 shutdown_connections 2024-09-19T04:47:35.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.355+0000 7fae1d200640 1 -- 172.21.15.123:0/1233609553 wait complete. 2024-09-19T04:47:35.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.356+0000 7fae1d200640 1 Processor -- start 2024-09-19T04:47:35.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.357+0000 7fae1d200640 1 -- start start 2024-09-19T04:47:35.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.357+0000 7fae1d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae18104010 0x7fae1819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.357+0000 7fae1d200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae181049c0 0x7fae1819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.357+0000 7fae1d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae18109140 0x7fae1819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.357+0000 7fae1d200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae18114190 con 0x7fae18104010 2024-09-19T04:47:35.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.357+0000 7fae1d200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae18114010 con 0x7fae18109140 2024-09-19T04:47:35.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.357+0000 7fae1d200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae18114310 con 0x7fae181049c0 2024-09-19T04:47:35.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.358+0000 7fae15e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae18104010 0x7fae1819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:35.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.358+0000 7fae15e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae18104010 0x7fae1819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53536/0 (socket says 172.21.15.123:53536) 2024-09-19T04:47:35.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.358+0000 7fae15e00640 1 -- 172.21.15.123:0/1005448983 learned_addr learned my addr 172.21.15.123:0/1005448983 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:35.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.358+0000 7fae15400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae181049c0 0x7fae1819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:35.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.358+0000 7fae16800640 1 --2- 172.21.15.123:0/1005448983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae18109140 0x7fae1819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:35.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.359+0000 7fae15e00640 1 -- 172.21.15.123:0/1005448983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae181049c0 msgr2=0x7fae1819ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:35.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.359+0000 7fae15e00640 1 --2- 172.21.15.123:0/1005448983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae181049c0 0x7fae1819ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.359+0000 7fae15e00640 1 -- 172.21.15.123:0/1005448983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae18109140 msgr2=0x7fae1819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:35.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.359+0000 7fae15e00640 1 --2- 172.21.15.123:0/1005448983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae18109140 0x7fae1819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.359+0000 7fae15e00640 1 -- 172.21.15.123:0/1005448983 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1819f6c0 con 0x7fae18104010 2024-09-19T04:47:35.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.359+0000 7fae15e00640 1 --2- 172.21.15.123:0/1005448983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae18104010 0x7fae1819a710 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fae0800ea60 tx=0x7fae0800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:35.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.360+0000 7fae0ea00640 1 -- 172.21.15.123:0/1005448983 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0800cd20 con 0x7fae18104010 2024-09-19T04:47:35.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.360+0000 7fae0ea00640 1 -- 172.21.15.123:0/1005448983 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae080035f0 con 0x7fae18104010 2024-09-19T04:47:35.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.360+0000 7fae1d200640 1 -- 172.21.15.123:0/1005448983 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae1819f9b0 con 0x7fae18104010 2024-09-19T04:47:35.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.360+0000 7fae0ea00640 1 -- 172.21.15.123:0/1005448983 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08010690 con 0x7fae18104010 2024-09-19T04:47:35.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.360+0000 7fae1d200640 1 -- 172.21.15.123:0/1005448983 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae181a7290 con 0x7fae18104010 2024-09-19T04:47:35.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.365+0000 7fae1fc90740 1 Processor -- start 2024-09-19T04:47:35.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.365+0000 7fae1fc90740 1 -- start start 2024-09-19T04:47:35.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.366+0000 7fae0ea00640 1 -- 172.21.15.123:0/1005448983 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7fae080037a0 con 0x7fae18104010 2024-09-19T04:47:35.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.366+0000 7fae1fc90740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.366+0000 7fae1fc90740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8d1590 0x55c9aa8db910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.366+0000 7fae1fc90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8dbe50 0x55c9aa8de240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.366+0000 7fae1fc90740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa8373e0 con 0x55c9aa8d1590 2024-09-19T04:47:35.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.366+0000 7fae1fc90740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa7ded80 con 0x55c9aa8dbe50 2024-09-19T04:47:35.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.366+0000 7fae1fc90740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa8333a0 con 0x55c9aa8d0c70 2024-09-19T04:47:35.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.367+0000 7fae16800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:35.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.367+0000 7fae16800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8d1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58720/0 (socket says 172.21.15.123:58720) 2024-09-19T04:47:35.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.367+0000 7fae16800640 1 -- 172.21.15.123:0/3891042217 learned_addr learned my addr 172.21.15.123:0/3891042217 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:35.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.367+0000 7fae15400640 1 --2- 172.21.15.123:0/3891042217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8dbe50 0x55c9aa8de240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:35.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.367+0000 7fae16800640 1 -- 172.21.15.123:0/3891042217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8dbe50 msgr2=0x55c9aa8de240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:35.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.367+0000 7fae16800640 1 --2- 172.21.15.123:0/3891042217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8dbe50 0x55c9aa8de240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.367+0000 7fae16800640 1 -- 172.21.15.123:0/3891042217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8d1590 msgr2=0x55c9aa8db910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:47:35.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.367+0000 7fae16800640 1 --2- 172.21.15.123:0/3891042217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8d1590 0x55c9aa8db910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.367+0000 7fae16800640 1 -- 172.21.15.123:0/3891042217 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9aa7fdae0 con 0x55c9aa8d0c70 2024-09-19T04:47:35.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.368+0000 7fae0ea00640 1 --2- 172.21.15.123:0/1005448983 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fadfc0a35e0 0x7fadfc0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.368+0000 7fae16800640 1 --2- 172.21.15.123:0/3891042217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8d1050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae10009f60 tx=0x7fae10009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef8b8639a40800f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:35.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.369+0000 7fae0ea00640 1 -- 172.21.15.123:0/1005448983 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fae080d0190 con 0x7fae18104010 2024-09-19T04:47:35.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.369+0000 7fae03400640 1 -- 172.21.15.123:0/3891042217 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1001e070 con 0x55c9aa8d0c70 2024-09-19T04:47:35.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.369+0000 7fae15e00640 1 --2- 172.21.15.123:0/1005448983 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fadfc0a35e0 0x7fadfc0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:35.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.370+0000 7fae03400640 1 -- 172.21.15.123:0/3891042217 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae10004030 con 0x55c9aa8d0c70 2024-09-19T04:47:35.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.370+0000 7fae03400640 1 -- 172.21.15.123:0/3891042217 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1000f6e0 con 0x55c9aa8d0c70 2024-09-19T04:47:35.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.371+0000 7fae1fc90740 1 -- 172.21.15.123:0/3891042217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d0c70 msgr2=0x55c9aa8d1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:35.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.371+0000 7fae15e00640 1 --2- 172.21.15.123:0/1005448983 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fadfc0a35e0 0x7fadfc0a5aa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fae080cd770 tx=0x7fae0800ef90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:35.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.371+0000 7fae1fc90740 1 --2- 172.21.15.123:0/3891042217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8d1050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae10009f60 tx=0x7fae10009f90 comp rx=0 tx=0).stop 2024-09-19T04:47:35.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.371+0000 7fae1fc90740 1 -- 172.21.15.123:0/3891042217 shutdown_connections 2024-09-19T04:47:35.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.371+0000 7fae1fc90740 1 --2- 172.21.15.123:0/3891042217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8dbe50 0x55c9aa8de240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.371+0000 7fae1fc90740 1 --2- 172.21.15.123:0/3891042217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8d1590 0x55c9aa8db910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.371+0000 7fae1fc90740 1 --2- 172.21.15.123:0/3891042217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8d1050 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.371+0000 7fae1fc90740 1 -- 172.21.15.123:0/3891042217 >> 172.21.15.123:0/3891042217 conn(0x55c9aa8cd200 msgr2=0x55c9aa8cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:35.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.372+0000 7fae1fc90740 1 -- 172.21.15.123:0/3891042217 shutdown_connections 2024-09-19T04:47:35.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.372+0000 7fae1fc90740 1 -- 172.21.15.123:0/3891042217 wait complete. 2024-09-19T04:47:35.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.372+0000 7fae1fc90740 1 -- start start 2024-09-19T04:47:35.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.373+0000 7fae1fc90740 1 Processor -- start 2024-09-19T04:47:35.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.373+0000 7fae1fc90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8d8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.373+0000 7fae1fc90740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d1590 0x55c9aa8d92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.373+0000 7fae1fc90740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8dbe50 0x55c9aa8d9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.373+0000 7fae1fc90740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa8dea70 con 0x55c9aa8dbe50 2024-09-19T04:47:35.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.373+0000 7fae1fc90740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa7f20f0 con 0x55c9aa8d0c70 2024-09-19T04:47:35.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.373+0000 7fae1fc90740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa8debf0 con 0x55c9aa8d1590 2024-09-19T04:47:35.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.373+0000 7fae1fc90740 10 client.0 _ll_register_callbacks cb 0x55c9aa7d9bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:35.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.374+0000 7fae15400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d1590 0x55c9aa8d92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:35.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.374+0000 7fae15400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d1590 0x55c9aa8d92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58722/0 (socket says 172.21.15.123:58722) 2024-09-19T04:47:35.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.374+0000 7fae15400640 1 -- 172.21.15.123:0/3734694935 learned_addr learned my addr 172.21.15.123:0/3734694935 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:35.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.374+0000 7fae16800640 1 --2- 172.21.15.123:0/3734694935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8d8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:35.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.374+0000 7fae16800640 1 --2- 172.21.15.123:0/3734694935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8dbe50 0x55c9aa8d9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:35.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.374+0000 7fae15400640 1 -- 172.21.15.123:0/3734694935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8d0c70 msgr2=0x55c9aa8d8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:35.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.374+0000 7fae15400640 1 --2- 172.21.15.123:0/3734694935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8d8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.374+0000 7fae15400640 1 -- 172.21.15.123:0/3734694935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8dbe50 msgr2=0x55c9aa8d9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:35.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.374+0000 7fae15400640 1 --2- 172.21.15.123:0/3734694935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8dbe50 0x55c9aa8d9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.374+0000 7fae15400640 1 -- 172.21.15.123:0/3734694935 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9aa5a83c0 con 0x55c9aa8d1590 2024-09-19T04:47:35.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.374+0000 7fae16800640 1 --2- 172.21.15.123:0/3734694935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8dbe50 0x55c9aa8d9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:35.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.375+0000 7fae15400640 1 --2- 172.21.15.123:0/3734694935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d1590 0x55c9aa8d92d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fae04009230 tx=0x7fae040042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57651e6101c7cb48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:35.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.375+0000 7fadf7e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:47:35.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.375+0000 7fadf7e00640 1 -- 172.21.15.123:0/3734694935 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae04035040 con 0x55c9aa8d1590 2024-09-19T04:47:35.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.375+0000 7fae1fc90740 1 -- 172.21.15.123:0/3734694935 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae10004030 con 0x55c9aa8d1590 2024-09-19T04:47:35.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.375+0000 7fadf7e00640 1 -- 172.21.15.123:0/3734694935 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fae04005230 con 0x55c9aa8d1590 2024-09-19T04:47:35.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.376+0000 7fadf7e00640 1 -- 172.21.15.123:0/3734694935 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fae04005430 con 0x55c9aa8d1590 2024-09-19T04:47:35.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.376+0000 7fae1fc90740 10 client.25414 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:35.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.376+0000 7fae1fc90740 1 -- 172.21.15.123:0/3734694935 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c9aa8dee80 con 0x55c9aa8d1590 2024-09-19T04:47:35.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.376+0000 7fadf7e00640 1 -- 172.21.15.123:0/3734694935 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae10004030 con 0x55c9aa8d1590 2024-09-19T04:47:35.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.376+0000 7fadf7e00640 1 -- 172.21.15.123:0/3734694935 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fae04031980 con 0x55c9aa8d1590 2024-09-19T04:47:35.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.376+0000 7fae1fc90740 10 client.25414 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:35.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.376+0000 7fae1fc90740 10 client.25414 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:47:35.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.376+0000 7fae1fc90740 20 client.25414 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:35.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.376+0000 7fae1fc90740 1 --2- 172.21.15.123:0/3734694935 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55c9aa8e0340 0x55c9aa8e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.377+0000 7fae1fc90740 4 client.25414 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:47:35.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.377+0000 7fae1fc90740 1 -- 172.21.15.123:0/3734694935 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9aa844950 con 0x55c9aa8e0340 2024-09-19T04:47:35.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.377+0000 7fae16800640 1 --2- 172.21.15.123:0/3734694935 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55c9aa8e0340 0x55c9aa8e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:35.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.378+0000 7fae16800640 1 --2- 172.21.15.123:0/3734694935 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55c9aa8e0340 0x55c9aa8e2760 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=733fe08d0060ce38 server_cookie=57b17170a658fe37 in_seq=0 out_seq=0 2024-09-19T04:47:35.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.378+0000 7fadf7e00640 10 client.25414 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:47:35.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.378+0000 7fadf7e00640 1 -- 172.21.15.123:0/3734694935 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9aa844950 con 0x55c9aa8e0340 2024-09-19T04:47:35.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.378+0000 7fadf7e00640 10 client.25414 handle_command_reply: tid=0 2024-09-19T04:47:35.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.379+0000 7fae1fc90740 1 client.25414 shutdown 2024-09-19T04:47:35.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.379+0000 7fae1fc90740 1 -- 172.21.15.123:0/3734694935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d1590 msgr2=0x55c9aa8d92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:35.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.379+0000 7fae1fc90740 1 --2- 172.21.15.123:0/3734694935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d1590 0x55c9aa8d92d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fae04009230 tx=0x7fae040042e0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.379+0000 7fae1fc90740 1 -- 172.21.15.123:0/3734694935 shutdown_connections 2024-09-19T04:47:35.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.379+0000 7fae1fc90740 1 --2- 172.21.15.123:0/3734694935 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55c9aa8e0340 0x55c9aa8e2760 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.379+0000 7fae1fc90740 1 --2- 172.21.15.123:0/3734694935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8dbe50 0x55c9aa8d9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.379+0000 7fae1fc90740 1 --2- 172.21.15.123:0/3734694935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d1590 0x55c9aa8d92d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.379+0000 7fae1fc90740 1 --2- 172.21.15.123:0/3734694935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8d8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.379+0000 7fadf7e00640 0 client.25414 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:47:35.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.379+0000 7fae1fc90740 1 -- 172.21.15.123:0/3734694935 >> 172.21.15.123:0/3734694935 conn(0x55c9aa8cd200 msgr2=0x55c9aa8d2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:35.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.379+0000 7fae1fc90740 1 -- 172.21.15.123:0/3734694935 shutdown_connections 2024-09-19T04:47:35.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.379+0000 7fae1fc90740 1 -- 172.21.15.123:0/3734694935 wait complete. 2024-09-19T04:47:35.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.380+0000 7fae1fc90740 20 client.25414 trim_cache size 0 max 16384 2024-09-19T04:47:35.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.402+0000 7fae1fc90740 1 Processor -- start 2024-09-19T04:47:35.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.402+0000 7fae1fc90740 1 -- start start 2024-09-19T04:47:35.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.402+0000 7fae1fc90740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8efef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.402+0000 7fae1fc90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8d1590 0x55c9aa909350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.403+0000 7fae1fc90740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8dbe50 0x55c9aa909890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.403+0000 7fae1fc90740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae1806fe40 con 0x55c9aa8d0c70 2024-09-19T04:47:35.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.403+0000 7fae1fc90740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae18071980 con 0x55c9aa8d1590 2024-09-19T04:47:35.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.403+0000 7fae1fc90740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae18070510 con 0x55c9aa8dbe50 2024-09-19T04:47:35.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.403+0000 7fae16800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8efef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:35.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.403+0000 7fae16800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8efef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53552/0 (socket says 172.21.15.123:53552) 2024-09-19T04:47:35.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.403+0000 7fae16800640 1 -- 172.21.15.123:0/846146588 learned_addr learned my addr 172.21.15.123:0/846146588 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:35.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.403+0000 7fae15400640 1 --2- 172.21.15.123:0/846146588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8d1590 0x55c9aa909350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:35.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.403+0000 7fae16800640 1 --2- 172.21.15.123:0/846146588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8dbe50 0x55c9aa909890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:35.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.404+0000 7fae15400640 1 -- 172.21.15.123:0/846146588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8dbe50 msgr2=0x55c9aa909890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:35.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.404+0000 7fae15400640 1 --2- 172.21.15.123:0/846146588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8dbe50 0x55c9aa909890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.404+0000 7fae15400640 1 -- 172.21.15.123:0/846146588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8d0c70 msgr2=0x55c9aa8efef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:35.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.404+0000 7fae15400640 1 --2- 172.21.15.123:0/846146588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8efef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.404+0000 7fae15400640 1 -- 172.21.15.123:0/846146588 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9aa8c8770 con 0x55c9aa8d1590 2024-09-19T04:47:35.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.404+0000 7fae15400640 1 --2- 172.21.15.123:0/846146588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8d1590 0x55c9aa909350 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fae04031d50 tx=0x7fae0402a400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3048f35e49a069a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.404+0000 7fae02a00640 1 -- 172.21.15.123:0/846146588 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae04035030 con 0x55c9aa8d1590 2024-09-19T04:47:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.404+0000 7fae02a00640 1 -- 172.21.15.123:0/846146588 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae04040070 con 0x55c9aa8d1590 2024-09-19T04:47:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.404+0000 7fae02a00640 1 -- 172.21.15.123:0/846146588 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae04051070 con 0x55c9aa8d1590 2024-09-19T04:47:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.405+0000 7fae1fc90740 1 -- 172.21.15.123:0/846146588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8d1590 msgr2=0x55c9aa909350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.405+0000 7fae1fc90740 1 --2- 172.21.15.123:0/846146588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8d1590 0x55c9aa909350 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fae04031d50 tx=0x7fae0402a400 comp rx=0 tx=0).stop 2024-09-19T04:47:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.405+0000 7fae1fc90740 1 -- 172.21.15.123:0/846146588 shutdown_connections 2024-09-19T04:47:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.405+0000 7fae1fc90740 1 --2- 172.21.15.123:0/846146588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8dbe50 0x55c9aa909890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.405+0000 7fae1fc90740 1 --2- 172.21.15.123:0/846146588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8d1590 0x55c9aa909350 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.405+0000 7fae1fc90740 1 --2- 172.21.15.123:0/846146588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8efef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.405+0000 7fae1fc90740 1 -- 172.21.15.123:0/846146588 >> 172.21.15.123:0/846146588 conn(0x55c9aa8cd200 msgr2=0x55c9aa8d7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:35.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.406+0000 7fae1fc90740 1 -- 172.21.15.123:0/846146588 shutdown_connections 2024-09-19T04:47:35.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.406+0000 7fae1fc90740 1 -- 172.21.15.123:0/846146588 wait complete. 2024-09-19T04:47:35.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.406+0000 7fae1fc90740 1 -- start start 2024-09-19T04:47:35.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.406+0000 7fae1fc90740 1 Processor -- start 2024-09-19T04:47:35.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.407+0000 7fae1fc90740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8e3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.407+0000 7fae1fc90740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8d1590 0x55c9aa8e3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.407+0000 7fae1fc90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8dbe50 0x55c9aa9662f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.407+0000 7fae1fc90740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae180712b0 con 0x55c9aa8d1590 2024-09-19T04:47:35.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.407+0000 7fae1fc90740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae18070be0 con 0x55c9aa8dbe50 2024-09-19T04:47:35.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.407+0000 7fae1fc90740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae1806de20 con 0x55c9aa8d0c70 2024-09-19T04:47:35.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.407+0000 7fae1fc90740 10 client.0 _ll_register_callbacks cb 0x7fae100093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:35.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.407+0000 7fae16800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:35.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.407+0000 7fae16800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8e3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58736/0 (socket says 172.21.15.123:58736) 2024-09-19T04:47:35.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.407+0000 7fae16800640 1 -- 172.21.15.123:0/1061635846 learned_addr learned my addr 172.21.15.123:0/1061635846 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:35.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.407+0000 7fae16800640 1 --2- 172.21.15.123:0/1061635846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8dbe50 0x55c9aa9662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:35.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.407+0000 7fae15400640 1 --2- 172.21.15.123:0/1061635846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8d1590 0x55c9aa8e3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:35.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.408+0000 7fae16800640 1 -- 172.21.15.123:0/1061635846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8dbe50 msgr2=0x55c9aa9662f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:35.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.408+0000 7fae16800640 1 --2- 172.21.15.123:0/1061635846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8dbe50 0x55c9aa9662f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.408+0000 7fae16800640 1 -- 172.21.15.123:0/1061635846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8d1590 msgr2=0x55c9aa8e3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:35.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.408+0000 7fae16800640 1 --2- 172.21.15.123:0/1061635846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8d1590 0x55c9aa8e3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.408+0000 7fae16800640 1 -- 172.21.15.123:0/1061635846 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9aa9668e0 con 0x55c9aa8d0c70 2024-09-19T04:47:35.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.408+0000 7fae16800640 1 --2- 172.21.15.123:0/1061635846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8e3570 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fae10009f60 tx=0x7fae10005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48a2a2b0384467fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:35.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.408+0000 7fadf7e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:47:35.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.408+0000 7fadf7e00640 1 -- 172.21.15.123:0/1061635846 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10049060 con 0x55c9aa8d0c70 2024-09-19T04:47:35.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.408+0000 7fadf7e00640 1 -- 172.21.15.123:0/1061635846 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fae10043440 con 0x55c9aa8d0c70 2024-09-19T04:47:35.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.408+0000 7fae1fc90740 1 -- 172.21.15.123:0/1061635846 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae04040070 con 0x55c9aa8d0c70 2024-09-19T04:47:35.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.409+0000 7fadf7e00640 1 -- 172.21.15.123:0/1061635846 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fae10043640 con 0x55c9aa8d0c70 2024-09-19T04:47:35.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.409+0000 7fae1fc90740 10 client.25420 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:35.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.409+0000 7fae1fc90740 1 -- 172.21.15.123:0/1061635846 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c9aa967c20 con 0x55c9aa8d0c70 2024-09-19T04:47:35.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.409+0000 7fadf7e00640 1 -- 172.21.15.123:0/1061635846 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae04040070 con 0x55c9aa8d0c70 2024-09-19T04:47:35.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.409+0000 7fadf7e00640 1 -- 172.21.15.123:0/1061635846 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fae1000f780 con 0x55c9aa8d0c70 2024-09-19T04:47:35.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.409+0000 7fae1fc90740 10 client.25420 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:35.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.409+0000 7fae1fc90740 10 client.25420 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:47:35.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.409+0000 7fae1fc90740 20 client.25420 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:35.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.409+0000 7fae1fc90740 1 --2- 172.21.15.123:0/1061635846 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55c9aa9681b0 0x55c9aa96a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:35.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.409+0000 7fae1fc90740 4 client.25420 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:47:35.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.409+0000 7fae1fc90740 1 -- 172.21.15.123:0/1061635846 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9aa83ee60 con 0x55c9aa9681b0 2024-09-19T04:47:35.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.410+0000 7fae15400640 1 --2- 172.21.15.123:0/1061635846 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55c9aa9681b0 0x55c9aa96a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:35.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.410+0000 7fae15400640 1 --2- 172.21.15.123:0/1061635846 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55c9aa9681b0 0x55c9aa96a5a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df1af5f449aaab84 server_cookie=491a59aa9d817c5e in_seq=0 out_seq=0 2024-09-19T04:47:35.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.410+0000 7fadf7e00640 10 client.25420 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:47:35.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.410+0000 7fadf7e00640 1 -- 172.21.15.123:0/1061635846 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9aa83ee60 con 0x55c9aa9681b0 2024-09-19T04:47:35.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.410+0000 7fadf7e00640 10 client.25420 handle_command_reply: tid=0 2024-09-19T04:47:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.410+0000 7fae1fc90740 1 client.25420 shutdown 2024-09-19T04:47:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.411+0000 7fae1fc90740 1 -- 172.21.15.123:0/1061635846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d0c70 msgr2=0x55c9aa8e3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.411+0000 7fae1fc90740 1 --2- 172.21.15.123:0/1061635846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8e3570 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fae10009f60 tx=0x7fae10005fd0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.411+0000 7fae1fc90740 1 -- 172.21.15.123:0/1061635846 shutdown_connections 2024-09-19T04:47:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.411+0000 7fae1fc90740 1 --2- 172.21.15.123:0/1061635846 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55c9aa9681b0 0x55c9aa96a5a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.411+0000 7fae1fc90740 1 --2- 172.21.15.123:0/1061635846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9aa8dbe50 0x55c9aa9662f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.411+0000 7fae1fc90740 1 --2- 172.21.15.123:0/1061635846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9aa8d1590 0x55c9aa8e3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.411+0000 7fae1fc90740 1 --2- 172.21.15.123:0/1061635846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9aa8d0c70 0x55c9aa8e3570 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.411+0000 7fae1fc90740 1 -- 172.21.15.123:0/1061635846 >> 172.21.15.123:0/1061635846 conn(0x55c9aa8cd200 msgr2=0x55c9aa8cbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.411+0000 7fadf7e00640 0 client.25420 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:47:35.415 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:47:35.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.411+0000 7fae1fc90740 1 -- 172.21.15.123:0/1061635846 shutdown_connections 2024-09-19T04:47:35.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.411+0000 7fae1fc90740 1 -- 172.21.15.123:0/1061635846 wait complete. 2024-09-19T04:47:35.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.411+0000 7fae1fc90740 20 client.25420 trim_cache size 0 max 16384 2024-09-19T04:47:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.412+0000 7fae1d200640 1 -- 172.21.15.123:0/1005448983 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fadfc0a35e0 msgr2=0x7fadfc0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.412+0000 7fae1d200640 1 --2- 172.21.15.123:0/1005448983 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fadfc0a35e0 0x7fadfc0a5aa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fae080cd770 tx=0x7fae0800ef90 comp rx=0 tx=0).stop 2024-09-19T04:47:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.412+0000 7fae1d200640 1 -- 172.21.15.123:0/1005448983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae18104010 msgr2=0x7fae1819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.412+0000 7fae1d200640 1 --2- 172.21.15.123:0/1005448983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae18104010 0x7fae1819a710 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fae0800ea60 tx=0x7fae0800ee60 comp rx=0 tx=0).stop 2024-09-19T04:47:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.412+0000 7fae1d200640 1 -- 172.21.15.123:0/1005448983 shutdown_connections 2024-09-19T04:47:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.412+0000 7fae1d200640 1 --2- 172.21.15.123:0/1005448983 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fadfc0a35e0 0x7fadfc0a5aa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.412+0000 7fae1d200640 1 --2- 172.21.15.123:0/1005448983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae18109140 0x7fae1819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.412+0000 7fae1d200640 1 --2- 172.21.15.123:0/1005448983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae181049c0 0x7fae1819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.412+0000 7fae1d200640 1 --2- 172.21.15.123:0/1005448983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae18104010 0x7fae1819a710 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.412+0000 7fae1d200640 1 -- 172.21.15.123:0/1005448983 >> 172.21.15.123:0/1005448983 conn(0x7fae180ffa50 msgr2=0x7fae18100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.413+0000 7fae1d200640 1 -- 172.21.15.123:0/1005448983 shutdown_connections 2024-09-19T04:47:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:35.413+0000 7fae1d200640 1 -- 172.21.15.123:0/1005448983 wait complete. 2024-09-19T04:47:35.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:35 smithi123 ceph-mon[30511]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T04:47:35.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:35 smithi183 ceph-mon[35333]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T04:47:35.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:35 smithi193 ceph-mon[35592]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T04:47:36.473 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:47:36.884 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:47:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.516+0000 7f9493200640 1 -- 172.21.15.123:0/76108167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f948c074e30 msgr2=0x7f948c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.516+0000 7f9493200640 1 --2- 172.21.15.123:0/76108167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f948c074e30 0x7f948c10dd40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f947c009860 tx=0x7f947c02a3d0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.516+0000 7f9493200640 1 -- 172.21.15.123:0/76108167 shutdown_connections 2024-09-19T04:47:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.516+0000 7f9493200640 1 --2- 172.21.15.123:0/76108167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f948c10e280 0x7f948c110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.516+0000 7f9493200640 1 --2- 172.21.15.123:0/76108167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f948c074e30 0x7f948c10dd40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.516+0000 7f9493200640 1 --2- 172.21.15.123:0/76108167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f948c074510 0x7f948c0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.516+0000 7f9493200640 1 -- 172.21.15.123:0/76108167 >> 172.21.15.123:0/76108167 conn(0x7f948c0fd740 msgr2=0x7f948c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:37.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.516+0000 7f9493200640 1 -- 172.21.15.123:0/76108167 shutdown_connections 2024-09-19T04:47:37.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.516+0000 7f9493200640 1 -- 172.21.15.123:0/76108167 wait complete. 2024-09-19T04:47:37.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.518+0000 7f9493200640 1 Processor -- start 2024-09-19T04:47:37.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.518+0000 7f9493200640 1 -- start start 2024-09-19T04:47:37.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.518+0000 7f9493200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f948c074510 0x7f948c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.518+0000 7f9493200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f948c074e30 0x7f948c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.518+0000 7f9493200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f948c10e280 0x7f948c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.518+0000 7f9493200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f948c113fc0 con 0x7f948c074510 2024-09-19T04:47:37.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.518+0000 7f9493200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f948c113e40 con 0x7f948c10e280 2024-09-19T04:47:37.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.518+0000 7f9493200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f948c114140 con 0x7f948c074e30 2024-09-19T04:47:37.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.519+0000 7f9491400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f948c074e30 0x7f948c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:37.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.519+0000 7f9491400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f948c074e30 0x7f948c19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58748/0 (socket says 172.21.15.123:58748) 2024-09-19T04:47:37.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.519+0000 7f9491400640 1 -- 172.21.15.123:0/3299603798 learned_addr learned my addr 172.21.15.123:0/3299603798 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:37.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.519+0000 7f9492800640 1 --2- 172.21.15.123:0/3299603798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f948c10e280 0x7f948c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:37.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.519+0000 7f9491e00640 1 --2- 172.21.15.123:0/3299603798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f948c074510 0x7f948c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:37.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.519+0000 7f9491400640 1 -- 172.21.15.123:0/3299603798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f948c10e280 msgr2=0x7f948c19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.520+0000 7f9491400640 1 --2- 172.21.15.123:0/3299603798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f948c10e280 0x7f948c19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.520+0000 7f9491400640 1 -- 172.21.15.123:0/3299603798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f948c074510 msgr2=0x7f948c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.520+0000 7f9491400640 1 --2- 172.21.15.123:0/3299603798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f948c074510 0x7f948c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.520+0000 7f9491400640 1 -- 172.21.15.123:0/3299603798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948c19f4b0 con 0x7f948c074e30 2024-09-19T04:47:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.520+0000 7f9492800640 1 --2- 172.21.15.123:0/3299603798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f948c10e280 0x7f948c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.520+0000 7f9491400640 1 --2- 172.21.15.123:0/3299603798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f948c074e30 0x7f948c19aa40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f947c009980 tx=0x7f947c0309a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.520+0000 7f9491e00640 1 --2- 172.21.15.123:0/3299603798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f948c074510 0x7f948c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:47:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.520+0000 7f9486a00640 1 -- 172.21.15.123:0/3299603798 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c040070 con 0x7f948c074e30 2024-09-19T04:47:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.521+0000 7f9486a00640 1 -- 172.21.15.123:0/3299603798 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f947c005620 con 0x7f948c074e30 2024-09-19T04:47:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.521+0000 7f9486a00640 1 -- 172.21.15.123:0/3299603798 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c004920 con 0x7f948c074e30 2024-09-19T04:47:37.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.521+0000 7f9493200640 1 -- 172.21.15.123:0/3299603798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f948c19f740 con 0x7f948c074e30 2024-09-19T04:47:37.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.521+0000 7f9493200640 1 -- 172.21.15.123:0/3299603798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f948c19fbd0 con 0x7f948c074e30 2024-09-19T04:47:37.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.524+0000 7f94973e4740 1 Processor -- start 2024-09-19T04:47:37.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.524+0000 7f94973e4740 1 -- start start 2024-09-19T04:47:37.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.525+0000 7f94973e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562284694e40 0x562284695220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.525+0000 7f94973e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284695760 0x56228469fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.525+0000 7f94973e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622846a0020 0x5622846a2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.525+0000 7f94973e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5622845fb3e0 con 0x562284694e40 2024-09-19T04:47:37.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.525+0000 7f94973e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5622845a2d80 con 0x5622846a0020 2024-09-19T04:47:37.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.525+0000 7f94973e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5622845f73a0 con 0x562284695760 2024-09-19T04:47:37.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.525+0000 7f9491e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284695760 0x56228469fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:37.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.525+0000 7f9491e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284695760 0x56228469fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58758/0 (socket says 172.21.15.123:58758) 2024-09-19T04:47:37.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.525+0000 7f9491e00640 1 -- 172.21.15.123:0/3531960495 learned_addr learned my addr 172.21.15.123:0/3531960495 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:37.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.525+0000 7f9492800640 1 --2- 172.21.15.123:0/3531960495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562284694e40 0x562284695220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:37.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.526+0000 7f9491e00640 1 -- 172.21.15.123:0/3531960495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622846a0020 msgr2=0x5622846a2410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:47:37.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.526+0000 7f9491e00640 1 --2- 172.21.15.123:0/3531960495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622846a0020 0x5622846a2410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.526+0000 7f9491e00640 1 -- 172.21.15.123:0/3531960495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562284694e40 msgr2=0x562284695220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:37.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.526+0000 7f9491e00640 1 --2- 172.21.15.123:0/3531960495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562284694e40 0x562284695220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.526+0000 7f9491e00640 1 -- 172.21.15.123:0/3531960495 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622845c1ae0 con 0x562284695760 2024-09-19T04:47:37.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.526+0000 7f9492800640 1 --2- 172.21.15.123:0/3531960495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562284694e40 0x562284695220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:47:37.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.526+0000 7f9486a00640 1 -- 172.21.15.123:0/3299603798 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7f947c043070 con 0x7f948c074e30 2024-09-19T04:47:37.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.526+0000 7f9491e00640 1 --2- 172.21.15.123:0/3531960495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284695760 0x56228469fae0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9480005ab0 tx=0x7f9480010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=771b73c9fa3198cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:37.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.527+0000 7f947b000640 1 -- 172.21.15.123:0/3531960495 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480005020 con 0x562284695760 2024-09-19T04:47:37.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.527+0000 7f947b000640 1 -- 172.21.15.123:0/3531960495 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94800047a0 con 0x562284695760 2024-09-19T04:47:37.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.527+0000 7f947b000640 1 -- 172.21.15.123:0/3531960495 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94800126d0 con 0x562284695760 2024-09-19T04:47:37.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.528+0000 7f94973e4740 1 -- 172.21.15.123:0/3531960495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284695760 msgr2=0x56228469fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:37.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.528+0000 7f94973e4740 1 --2- 172.21.15.123:0/3531960495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284695760 0x56228469fae0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9480005ab0 tx=0x7f9480010740 comp rx=0 tx=0).stop 2024-09-19T04:47:37.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.528+0000 7f94973e4740 1 -- 172.21.15.123:0/3531960495 shutdown_connections 2024-09-19T04:47:37.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.528+0000 7f94973e4740 1 --2- 172.21.15.123:0/3531960495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622846a0020 0x5622846a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.528+0000 7f94973e4740 1 --2- 172.21.15.123:0/3531960495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284695760 0x56228469fae0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.528+0000 7f94973e4740 1 --2- 172.21.15.123:0/3531960495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562284694e40 0x562284695220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.528+0000 7f94973e4740 1 -- 172.21.15.123:0/3531960495 >> 172.21.15.123:0/3531960495 conn(0x5622846913d0 msgr2=0x5622846917b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:37.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.529+0000 7f94973e4740 1 -- 172.21.15.123:0/3531960495 shutdown_connections 2024-09-19T04:47:37.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.529+0000 7f94973e4740 1 -- 172.21.15.123:0/3531960495 wait complete. 2024-09-19T04:47:37.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.529+0000 7f9486a00640 1 --2- 172.21.15.123:0/3299603798 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94740a34f0 0x7f94740a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.530+0000 7f94973e4740 1 -- start start 2024-09-19T04:47:37.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.530+0000 7f94973e4740 1 Processor -- start 2024-09-19T04:47:37.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.530+0000 7f9491e00640 1 --2- 172.21.15.123:0/3299603798 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94740a34f0 0x7f94740a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:37.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.530+0000 7f9486a00640 1 -- 172.21.15.123:0/3299603798 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f947c0f0aa0 con 0x7f948c074e30 2024-09-19T04:47:37.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.531+0000 7f94973e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562284694e40 0x56228469d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.531+0000 7f94973e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284695760 0x56228469d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.531+0000 7f94973e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622846a0020 0x56228469dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.531+0000 7f94973e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5622846a2bf0 con 0x5622846a0020 2024-09-19T04:47:37.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.531+0000 7f94973e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5622845b60f0 con 0x562284694e40 2024-09-19T04:47:37.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.531+0000 7f94973e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5622846a2d70 con 0x562284695760 2024-09-19T04:47:37.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.531+0000 7f94973e4740 10 client.0 _ll_register_callbacks cb 0x56228459dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:37.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.531+0000 7f9491400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284695760 0x56228469d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:37.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.531+0000 7f9491400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284695760 0x56228469d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58772/0 (socket says 172.21.15.123:58772) 2024-09-19T04:47:37.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.531+0000 7f9491400640 1 -- 172.21.15.123:0/3302374762 learned_addr learned my addr 172.21.15.123:0/3302374762 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:37.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.531+0000 7f9492800640 1 --2- 172.21.15.123:0/3302374762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622846a0020 0x56228469dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:37.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.532+0000 7f9491e00640 1 --2- 172.21.15.123:0/3299603798 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94740a34f0 0x7f94740a59b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9480011010 tx=0x7f9480005fd0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:37.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.532+0000 7f9491400640 1 -- 172.21.15.123:0/3302374762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562284694e40 msgr2=0x56228469d060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:47:37.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.532+0000 7f9491400640 1 --2- 172.21.15.123:0/3302374762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562284694e40 0x56228469d060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.532+0000 7f9491400640 1 -- 172.21.15.123:0/3302374762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622846a0020 msgr2=0x56228469dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:37.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.532+0000 7f9491400640 1 --2- 172.21.15.123:0/3302374762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622846a0020 0x56228469dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.532+0000 7f9491400640 1 -- 172.21.15.123:0/3302374762 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56228436c3c0 con 0x562284695760 2024-09-19T04:47:37.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.532+0000 7f9492800640 1 --2- 172.21.15.123:0/3302374762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622846a0020 0x56228469dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:37.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.533+0000 7f9491400640 1 --2- 172.21.15.123:0/3302374762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284695760 0x56228469d5a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f947c009830 tx=0x7f947c0bc4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d66db54d1f803bb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:37.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.533+0000 7f946f400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:47:37.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.533+0000 7f946f400640 1 -- 172.21.15.123:0/3302374762 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c0bcfd0 con 0x562284695760 2024-09-19T04:47:37.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.533+0000 7f946f400640 1 -- 172.21.15.123:0/3302374762 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f947c0c5350 con 0x562284695760 2024-09-19T04:47:37.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.533+0000 7f94973e4740 1 -- 172.21.15.123:0/3302374762 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94800047a0 con 0x562284695760 2024-09-19T04:47:37.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.533+0000 7f946f400640 1 -- 172.21.15.123:0/3302374762 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f947c0bcb70 con 0x562284695760 2024-09-19T04:47:37.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.534+0000 7f94973e4740 10 client.25432 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:37.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.534+0000 7f94973e4740 1 -- 172.21.15.123:0/3302374762 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622846a2bf0 con 0x562284695760 2024-09-19T04:47:37.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.534+0000 7f946f400640 1 -- 172.21.15.123:0/3302374762 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94800047a0 con 0x562284695760 2024-09-19T04:47:37.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.534+0000 7f946f400640 1 -- 172.21.15.123:0/3302374762 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f947c0a7100 con 0x562284695760 2024-09-19T04:47:37.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.534+0000 7f94973e4740 10 client.25432 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:37.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.534+0000 7f94973e4740 10 client.25432 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:47:37.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.534+0000 7f94973e4740 20 client.25432 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:37.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.534+0000 7f94973e4740 1 --2- 172.21.15.123:0/3302374762 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5622846b8850 0x5622846bacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.534+0000 7f94973e4740 4 client.25432 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:47:37.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.534+0000 7f94973e4740 1 -- 172.21.15.123:0/3302374762 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562284608950 con 0x5622846b8850 2024-09-19T04:47:37.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.535+0000 7f9491e00640 1 --2- 172.21.15.123:0/3302374762 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5622846b8850 0x5622846bacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:37.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.535+0000 7f9491e00640 1 --2- 172.21.15.123:0/3302374762 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5622846b8850 0x5622846bacb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8cfe18f34839eb68 server_cookie=10ec488bcd6cef7b in_seq=0 out_seq=0 2024-09-19T04:47:37.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.535+0000 7f946f400640 10 client.25432 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:47:37.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.536+0000 7f946f400640 1 -- 172.21.15.123:0/3302374762 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562284608950 con 0x5622846b8850 2024-09-19T04:47:37.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.536+0000 7f946f400640 10 client.25432 handle_command_reply: tid=0 2024-09-19T04:47:37.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.537+0000 7f94973e4740 1 client.25432 shutdown 2024-09-19T04:47:37.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.537+0000 7f94973e4740 1 -- 172.21.15.123:0/3302374762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284695760 msgr2=0x56228469d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:37.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.537+0000 7f94973e4740 1 --2- 172.21.15.123:0/3302374762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284695760 0x56228469d5a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f947c009830 tx=0x7f947c0bc4d0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.537+0000 7f94973e4740 1 -- 172.21.15.123:0/3302374762 shutdown_connections 2024-09-19T04:47:37.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.537+0000 7f94973e4740 1 --2- 172.21.15.123:0/3302374762 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5622846b8850 0x5622846bacb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.537+0000 7f94973e4740 1 --2- 172.21.15.123:0/3302374762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622846a0020 0x56228469dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.537+0000 7f94973e4740 1 --2- 172.21.15.123:0/3302374762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284695760 0x56228469d5a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.537+0000 7f94973e4740 1 --2- 172.21.15.123:0/3302374762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562284694e40 0x56228469d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.537+0000 7f94973e4740 1 -- 172.21.15.123:0/3302374762 >> 172.21.15.123:0/3302374762 conn(0x5622846913d0 msgr2=0x562284697100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:37.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.537+0000 7f946f400640 0 client.25432 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:47:37.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.537+0000 7f94973e4740 1 -- 172.21.15.123:0/3302374762 shutdown_connections 2024-09-19T04:47:37.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.538+0000 7f94973e4740 1 -- 172.21.15.123:0/3302374762 wait complete. 2024-09-19T04:47:37.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.538+0000 7f94973e4740 20 client.25432 trim_cache size 0 max 16384 2024-09-19T04:47:37.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.564+0000 7f94973e4740 1 Processor -- start 2024-09-19T04:47:37.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.564+0000 7f94973e4740 1 -- start start 2024-09-19T04:47:37.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.565+0000 7f94973e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284694e40 0x5622846c05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.565+0000 7f94973e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562284695760 0x5622846c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.565+0000 7f94973e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622846a0020 0x56228469a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.565+0000 7f94973e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1ab4b0 con 0x562284695760 2024-09-19T04:47:37.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.565+0000 7f94973e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1acff0 con 0x5622846a0020 2024-09-19T04:47:37.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.565+0000 7f94973e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1abb80 con 0x562284694e40 2024-09-19T04:47:37.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.565+0000 7f9491e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284694e40 0x5622846c05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:37.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.565+0000 7f9492800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622846a0020 0x56228469a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:37.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.565+0000 7f9491e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284694e40 0x5622846c05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58788/0 (socket says 172.21.15.123:58788) 2024-09-19T04:47:37.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.565+0000 7f9491e00640 1 -- 172.21.15.123:0/3159553318 learned_addr learned my addr 172.21.15.123:0/3159553318 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:37.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.565+0000 7f9491400640 1 --2- 172.21.15.123:0/3159553318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562284695760 0x5622846c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:37.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.566+0000 7f9492800640 1 -- 172.21.15.123:0/3159553318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284694e40 msgr2=0x5622846c05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:37.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.566+0000 7f9492800640 1 --2- 172.21.15.123:0/3159553318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284694e40 0x5622846c05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.566+0000 7f9492800640 1 -- 172.21.15.123:0/3159553318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562284695760 msgr2=0x5622846c0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:37.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.566+0000 7f9492800640 1 --2- 172.21.15.123:0/3159553318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562284695760 0x5622846c0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.566+0000 7f9492800640 1 -- 172.21.15.123:0/3159553318 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562284692540 con 0x5622846a0020 2024-09-19T04:47:37.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.566+0000 7f9491400640 1 --2- 172.21.15.123:0/3159553318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562284695760 0x5622846c0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:37.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.567+0000 7f9492800640 1 --2- 172.21.15.123:0/3159553318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622846a0020 0x56228469a300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9488005fd0 tx=0x7f94880138f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb6170e006f00e25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:37.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.567+0000 7f947a600640 1 -- 172.21.15.123:0/3159553318 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488004d20 con 0x5622846a0020 2024-09-19T04:47:37.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.567+0000 7f947a600640 1 -- 172.21.15.123:0/3159553318 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9488005500 con 0x5622846a0020 2024-09-19T04:47:37.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.567+0000 7f947a600640 1 -- 172.21.15.123:0/3159553318 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488011b50 con 0x5622846a0020 2024-09-19T04:47:37.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.568+0000 7f94973e4740 1 -- 172.21.15.123:0/3159553318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622846a0020 msgr2=0x56228469a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:37.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.568+0000 7f94973e4740 1 --2- 172.21.15.123:0/3159553318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622846a0020 0x56228469a300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9488005fd0 tx=0x7f94880138f0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.568+0000 7f94973e4740 1 -- 172.21.15.123:0/3159553318 shutdown_connections 2024-09-19T04:47:37.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.568+0000 7f94973e4740 1 --2- 172.21.15.123:0/3159553318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622846a0020 0x56228469a300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.568+0000 7f94973e4740 1 --2- 172.21.15.123:0/3159553318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562284695760 0x5622846c0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.568+0000 7f94973e4740 1 --2- 172.21.15.123:0/3159553318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562284694e40 0x5622846c05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.568+0000 7f94973e4740 1 -- 172.21.15.123:0/3159553318 >> 172.21.15.123:0/3159553318 conn(0x5622846913d0 msgr2=0x56228469f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:37.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.568+0000 7f94973e4740 1 -- 172.21.15.123:0/3159553318 shutdown_connections 2024-09-19T04:47:37.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.568+0000 7f94973e4740 1 -- 172.21.15.123:0/3159553318 wait complete. 2024-09-19T04:47:37.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.569+0000 7f94973e4740 1 -- start start 2024-09-19T04:47:37.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.569+0000 7f94973e4740 1 Processor -- start 2024-09-19T04:47:37.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.569+0000 7f94973e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562284694e40 0x5622846a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.569+0000 7f94973e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562284695760 0x5622846a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.569+0000 7f94973e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622846a0020 0x5622846a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.569+0000 7f94973e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1ac920 con 0x562284694e40 2024-09-19T04:47:37.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.569+0000 7f94973e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1ac250 con 0x562284695760 2024-09-19T04:47:37.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.569+0000 7f94973e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1a94b0 con 0x5622846a0020 2024-09-19T04:47:37.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.569+0000 7f94973e4740 10 client.0 _ll_register_callbacks cb 0x7f947c004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:37.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.570+0000 7f9491400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562284695760 0x5622846a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:37.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.570+0000 7f9491400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562284695760 0x5622846a4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:43398/0 (socket says 172.21.15.123:43398) 2024-09-19T04:47:37.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.570+0000 7f9491400640 1 -- 172.21.15.123:0/3806441430 learned_addr learned my addr 172.21.15.123:0/3806441430 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:37.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.570+0000 7f9491e00640 1 --2- 172.21.15.123:0/3806441430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562284694e40 0x5622846a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:37.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.570+0000 7f9492800640 1 --2- 172.21.15.123:0/3806441430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622846a0020 0x5622846a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:37.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.570+0000 7f9491400640 1 -- 172.21.15.123:0/3806441430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622846a0020 msgr2=0x5622846a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:37.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.570+0000 7f9491400640 1 --2- 172.21.15.123:0/3806441430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622846a0020 0x5622846a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.570+0000 7f9491400640 1 -- 172.21.15.123:0/3806441430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562284694e40 msgr2=0x5622846a83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:37.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.570+0000 7f9491400640 1 --2- 172.21.15.123:0/3806441430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562284694e40 0x5622846a83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.570+0000 7f9491400640 1 -- 172.21.15.123:0/3806441430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562284692060 con 0x562284695760 2024-09-19T04:47:37.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.571+0000 7f9491400640 1 --2- 172.21.15.123:0/3806441430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562284695760 0x5622846a4f90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f947c0b8fd0 tx=0x7f947c005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c253032ca6a76ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:37.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.571+0000 7f946f400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:47:37.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.571+0000 7f946f400640 1 -- 172.21.15.123:0/3806441430 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c0a0080 con 0x562284695760 2024-09-19T04:47:37.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.571+0000 7f946f400640 1 -- 172.21.15.123:0/3806441430 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f947c0c5550 con 0x562284695760 2024-09-19T04:47:37.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.571+0000 7f94973e4740 1 -- 172.21.15.123:0/3806441430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9488005500 con 0x562284695760 2024-09-19T04:47:37.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.572+0000 7f946f400640 1 -- 172.21.15.123:0/3806441430 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f947c031400 con 0x562284695760 2024-09-19T04:47:37.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.572+0000 7f94973e4740 10 client.25427 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:37.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.572+0000 7f94973e4740 1 -- 172.21.15.123:0/3806441430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562284692060 con 0x562284695760 2024-09-19T04:47:37.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.572+0000 7f946f400640 1 -- 172.21.15.123:0/3806441430 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9488005500 con 0x562284695760 2024-09-19T04:47:37.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.573+0000 7f946f400640 1 -- 172.21.15.123:0/3806441430 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f947c0bcdd0 con 0x562284695760 2024-09-19T04:47:37.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.573+0000 7f94973e4740 10 client.25427 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:37.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.573+0000 7f94973e4740 10 client.25427 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:47:37.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.573+0000 7f94973e4740 20 client.25427 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:37.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.573+0000 7f94973e4740 1 --2- 172.21.15.123:0/3806441430 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5622847244d0 0x5622847268f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:37.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.573+0000 7f94973e4740 4 client.25427 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:47:37.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.573+0000 7f94973e4740 1 -- 172.21.15.123:0/3806441430 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562284602e60 con 0x5622847244d0 2024-09-19T04:47:37.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.573+0000 7f9491e00640 1 --2- 172.21.15.123:0/3806441430 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5622847244d0 0x5622847268f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:37.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.574+0000 7f9491e00640 1 --2- 172.21.15.123:0/3806441430 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5622847244d0 0x5622847268f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=326b8bef1bb2771a server_cookie=f240e4243b846aae in_seq=0 out_seq=0 2024-09-19T04:47:37.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.574+0000 7f946f400640 10 client.25427 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:47:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.574+0000 7f946f400640 1 -- 172.21.15.123:0/3806441430 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562284602e60 con 0x5622847244d0 2024-09-19T04:47:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.574+0000 7f946f400640 10 client.25427 handle_command_reply: tid=0 2024-09-19T04:47:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.574+0000 7f94973e4740 1 client.25427 shutdown 2024-09-19T04:47:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.574+0000 7f94973e4740 1 -- 172.21.15.123:0/3806441430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562284695760 msgr2=0x5622846a4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.574+0000 7f94973e4740 1 --2- 172.21.15.123:0/3806441430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562284695760 0x5622846a4f90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f947c0b8fd0 tx=0x7f947c005bd0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.574+0000 7f94973e4740 1 -- 172.21.15.123:0/3806441430 shutdown_connections 2024-09-19T04:47:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.574+0000 7f94973e4740 1 --2- 172.21.15.123:0/3806441430 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5622847244d0 0x5622847268f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.574+0000 7f94973e4740 1 --2- 172.21.15.123:0/3806441430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622846a0020 0x5622846a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.574+0000 7f94973e4740 1 --2- 172.21.15.123:0/3806441430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562284695760 0x5622846a4f90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.574+0000 7f94973e4740 1 --2- 172.21.15.123:0/3806441430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562284694e40 0x5622846a83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.574+0000 7f94973e4740 1 -- 172.21.15.123:0/3806441430 >> 172.21.15.123:0/3806441430 conn(0x5622846913d0 msgr2=0x5622846968e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.575+0000 7f946f400640 0 client.25427 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:47:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.575+0000 7f94973e4740 1 -- 172.21.15.123:0/3806441430 shutdown_connections 2024-09-19T04:47:37.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.575+0000 7f94973e4740 1 -- 172.21.15.123:0/3806441430 wait complete. 2024-09-19T04:47:37.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.575+0000 7f94973e4740 20 client.25427 trim_cache size 0 max 16384 2024-09-19T04:47:37.578 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:47:37.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.576+0000 7f9493200640 1 -- 172.21.15.123:0/3299603798 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94740a34f0 msgr2=0x7f94740a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.576+0000 7f9493200640 1 --2- 172.21.15.123:0/3299603798 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94740a34f0 0x7f94740a59b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9480011010 tx=0x7f9480005fd0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.576+0000 7f9493200640 1 -- 172.21.15.123:0/3299603798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f948c074e30 msgr2=0x7f948c19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.576+0000 7f9493200640 1 --2- 172.21.15.123:0/3299603798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f948c074e30 0x7f948c19aa40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f947c009980 tx=0x7f947c0309a0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.577+0000 7f9493200640 1 -- 172.21.15.123:0/3299603798 shutdown_connections 2024-09-19T04:47:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.577+0000 7f9493200640 1 --2- 172.21.15.123:0/3299603798 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94740a34f0 0x7f94740a59b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.577+0000 7f9493200640 1 --2- 172.21.15.123:0/3299603798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f948c10e280 0x7f948c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.577+0000 7f9493200640 1 --2- 172.21.15.123:0/3299603798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f948c074e30 0x7f948c19aa40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.577+0000 7f9493200640 1 --2- 172.21.15.123:0/3299603798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f948c074510 0x7f948c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.577+0000 7f9493200640 1 -- 172.21.15.123:0/3299603798 >> 172.21.15.123:0/3299603798 conn(0x7f948c0fd740 msgr2=0x7f948c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:37.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.577+0000 7f9493200640 1 -- 172.21.15.123:0/3299603798 shutdown_connections 2024-09-19T04:47:37.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:37.577+0000 7f9493200640 1 -- 172.21.15.123:0/3299603798 wait complete. 2024-09-19T04:47:37.594 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:37 smithi123 ceph-mon[30511]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T04:47:37.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:37 smithi183 ceph-mon[35333]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T04:47:37.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:37 smithi193 ceph-mon[35592]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T04:47:38.543 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:47:39.042 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:47:39.521 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:39 smithi123 ceph-mon[30511]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T04:47:39.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:39 smithi183 ceph-mon[35333]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T04:47:39.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.639+0000 7fd0a1400640 1 -- 172.21.15.123:0/407781231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c102360 msgr2=0x7fd09c10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:39.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.639+0000 7fd0a1400640 1 --2- 172.21.15.123:0/407781231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c102360 0x7fd09c10e890 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd094009cd0 tx=0x7fd09402a8f0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.640+0000 7fd0a1400640 1 -- 172.21.15.123:0/407781231 shutdown_connections 2024-09-19T04:47:39.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.640+0000 7fd0a1400640 1 --2- 172.21.15.123:0/407781231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c102360 0x7fd09c10e890 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.640+0000 7fd0a1400640 1 --2- 172.21.15.123:0/407781231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd09c1019c0 0x7fd09c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.640+0000 7fd0a1400640 1 --2- 172.21.15.123:0/407781231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd09c1079c0 0x7fd09c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.640+0000 7fd0a1400640 1 -- 172.21.15.123:0/407781231 >> 172.21.15.123:0/407781231 conn(0x7fd09c0fd740 msgr2=0x7fd09c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:39.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.641+0000 7fd0a1400640 1 -- 172.21.15.123:0/407781231 shutdown_connections 2024-09-19T04:47:39.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.641+0000 7fd0a1400640 1 -- 172.21.15.123:0/407781231 wait complete. 2024-09-19T04:47:39.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.642+0000 7fd0a1400640 1 Processor -- start 2024-09-19T04:47:39.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.643+0000 7fd0a1400640 1 -- start start 2024-09-19T04:47:39.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.643+0000 7fd0a1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd09c1019c0 0x7fd09c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.643+0000 7fd0a1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd09c102360 0x7fd09c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.643+0000 7fd0a1400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c1079c0 0x7fd09c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.643+0000 7fd0a1400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c111e10 con 0x7fd09c102360 2024-09-19T04:47:39.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.643+0000 7fd0a1400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c111c90 con 0x7fd09c1019c0 2024-09-19T04:47:39.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.643+0000 7fd0a1400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c111f90 con 0x7fd09c1079c0 2024-09-19T04:47:39.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.644+0000 7fd09be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd09c1019c0 0x7fd09c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.644+0000 7fd09be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd09c1019c0 0x7fd09c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:43420/0 (socket says 172.21.15.123:43420) 2024-09-19T04:47:39.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.644+0000 7fd09be00640 1 -- 172.21.15.123:0/1935334471 learned_addr learned my addr 172.21.15.123:0/1935334471 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:39.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.644+0000 7fd09b400640 1 --2- 172.21.15.123:0/1935334471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd09c102360 0x7fd09c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.644+0000 7fd0a0a00640 1 --2- 172.21.15.123:0/1935334471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c1079c0 0x7fd09c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.644+0000 7fd0a0a00640 1 -- 172.21.15.123:0/1935334471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd09c1019c0 msgr2=0x7fd09c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:39.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.644+0000 7fd0a0a00640 1 --2- 172.21.15.123:0/1935334471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd09c1019c0 0x7fd09c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.644+0000 7fd0a0a00640 1 -- 172.21.15.123:0/1935334471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd09c102360 msgr2=0x7fd09c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:39.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.644+0000 7fd0a0a00640 1 --2- 172.21.15.123:0/1935334471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd09c102360 0x7fd09c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.644+0000 7fd0a0a00640 1 -- 172.21.15.123:0/1935334471 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09c1a6730 con 0x7fd09c1079c0 2024-09-19T04:47:39.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.645+0000 7fd0a0a00640 1 --2- 172.21.15.123:0/1935334471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c1079c0 0x7fd09c074f70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd09402ae10 tx=0x7fd094003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:39.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.645+0000 7fd098c00640 1 -- 172.21.15.123:0/1935334471 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094030dc0 con 0x7fd09c1079c0 2024-09-19T04:47:39.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.645+0000 7fd098c00640 1 -- 172.21.15.123:0/1935334471 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd094040070 con 0x7fd09c1079c0 2024-09-19T04:47:39.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.645+0000 7fd098c00640 1 -- 172.21.15.123:0/1935334471 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094045a30 con 0x7fd09c1079c0 2024-09-19T04:47:39.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.645+0000 7fd0a1400640 1 -- 172.21.15.123:0/1935334471 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09c1a69c0 con 0x7fd09c1079c0 2024-09-19T04:47:39.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.645+0000 7fd0a1400640 1 -- 172.21.15.123:0/1935334471 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd09c1a6e80 con 0x7fd09c1079c0 2024-09-19T04:47:39.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.649+0000 7fd098c00640 1 -- 172.21.15.123:0/1935334471 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7fd094004690 con 0x7fd09c1079c0 2024-09-19T04:47:39.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.650+0000 7fd0a5401740 1 Processor -- start 2024-09-19T04:47:39.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.650+0000 7fd0a5401740 1 -- start start 2024-09-19T04:47:39.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.651+0000 7fd0a5401740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c801e40 0x55af3c802220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.651+0000 7fd0a5401740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c802760 0x55af3c80cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.651+0000 7fd0a5401740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c80d020 0x55af3c80f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.651+0000 7fd0a5401740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55af3c7683e0 con 0x55af3c80d020 2024-09-19T04:47:39.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.651+0000 7fd0a5401740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55af3c70fd80 con 0x55af3c801e40 2024-09-19T04:47:39.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.651+0000 7fd0a5401740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55af3c7643a0 con 0x55af3c802760 2024-09-19T04:47:39.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.651+0000 7fd09be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c802760 0x55af3c80cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.651+0000 7fd09be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c802760 0x55af3c80cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58816/0 (socket says 172.21.15.123:58816) 2024-09-19T04:47:39.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.651+0000 7fd09be00640 1 -- 172.21.15.123:0/708780718 learned_addr learned my addr 172.21.15.123:0/708780718 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:39.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.651+0000 7fd098c00640 1 --2- 172.21.15.123:0/1935334471 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd0800a35e0 0x7fd0800a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.651+0000 7fd09b400640 1 --2- 172.21.15.123:0/708780718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c801e40 0x55af3c802220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.651+0000 7fd09b400640 1 --2- 172.21.15.123:0/708780718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c80d020 0x55af3c80f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.651+0000 7fd098c00640 1 -- 172.21.15.123:0/1935334471 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd0940f13c0 con 0x7fd09c1079c0 2024-09-19T04:47:39.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.652+0000 7fd09be00640 1 -- 172.21.15.123:0/708780718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c801e40 msgr2=0x55af3c802220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:39.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.652+0000 7fd09be00640 1 --2- 172.21.15.123:0/708780718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c801e40 0x55af3c802220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.652+0000 7fd09be00640 1 -- 172.21.15.123:0/708780718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c80d020 msgr2=0x55af3c80f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:39.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.652+0000 7fd09be00640 1 --2- 172.21.15.123:0/708780718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c80d020 0x55af3c80f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.652+0000 7fd09be00640 1 -- 172.21.15.123:0/708780718 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af3c72eae0 con 0x55af3c802760 2024-09-19T04:47:39.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.652+0000 7fd09b400640 1 --2- 172.21.15.123:0/708780718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c80d020 0x55af3c80f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:39.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.652+0000 7fd09be00640 1 --2- 172.21.15.123:0/708780718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c802760 0x55af3c80cae0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009d40 tx=0x7fd08c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5aaa4047545120c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:39.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.653+0000 7fd091200640 1 -- 172.21.15.123:0/708780718 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c004220 con 0x55af3c802760 2024-09-19T04:47:39.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.653+0000 7fd091200640 1 -- 172.21.15.123:0/708780718 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd08c0043c0 con 0x55af3c802760 2024-09-19T04:47:39.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.653+0000 7fd091200640 1 -- 172.21.15.123:0/708780718 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c0035f0 con 0x55af3c802760 2024-09-19T04:47:39.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.653+0000 7fd0a5401740 1 -- 172.21.15.123:0/708780718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c802760 msgr2=0x55af3c80cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:39.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.653+0000 7fd0a5401740 1 --2- 172.21.15.123:0/708780718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c802760 0x55af3c80cae0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009d40 tx=0x7fd08c011810 comp rx=0 tx=0).stop 2024-09-19T04:47:39.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.655+0000 7fd0a0a00640 1 --2- 172.21.15.123:0/1935334471 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd0800a35e0 0x7fd0800a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.655+0000 7fd0a5401740 1 -- 172.21.15.123:0/708780718 shutdown_connections 2024-09-19T04:47:39.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.655+0000 7fd0a5401740 1 --2- 172.21.15.123:0/708780718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c80d020 0x55af3c80f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.655+0000 7fd0a5401740 1 --2- 172.21.15.123:0/708780718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c802760 0x55af3c80cae0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.655+0000 7fd0a5401740 1 --2- 172.21.15.123:0/708780718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c801e40 0x55af3c802220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.655+0000 7fd0a5401740 1 -- 172.21.15.123:0/708780718 >> 172.21.15.123:0/708780718 conn(0x55af3c7fe3d0 msgr2=0x55af3c7fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:39.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.655+0000 7fd0a5401740 1 -- 172.21.15.123:0/708780718 shutdown_connections 2024-09-19T04:47:39.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.655+0000 7fd0a5401740 1 -- 172.21.15.123:0/708780718 wait complete. 2024-09-19T04:47:39.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.656+0000 7fd0a0a00640 1 --2- 172.21.15.123:0/1935334471 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd0800a35e0 0x7fd0800a5aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd0940eea70 tx=0x7fd094005df0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.656+0000 7fd0a5401740 1 -- start start 2024-09-19T04:47:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.656+0000 7fd0a5401740 1 Processor -- start 2024-09-19T04:47:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.657+0000 7fd0a5401740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c801e40 0x55af3c809ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.657+0000 7fd0a5401740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c802760 0x55af3c80a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.657+0000 7fd0a5401740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c80d020 0x55af3c80a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.657+0000 7fd0a5401740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55af3c80fbf0 con 0x55af3c802760 2024-09-19T04:47:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.657+0000 7fd0a5401740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55af3c7230f0 con 0x55af3c801e40 2024-09-19T04:47:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.657+0000 7fd0a5401740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55af3c80fd70 con 0x55af3c80d020 2024-09-19T04:47:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.657+0000 7fd0a5401740 10 client.0 _ll_register_callbacks cb 0x55af3c70abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.657+0000 7fd09b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c80d020 0x55af3c80a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.657+0000 7fd09be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c802760 0x55af3c80a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.657+0000 7fd09b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c80d020 0x55af3c80a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58828/0 (socket says 172.21.15.123:58828) 2024-09-19T04:47:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.657+0000 7fd09b400640 1 -- 172.21.15.123:0/550582724 learned_addr learned my addr 172.21.15.123:0/550582724 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.657+0000 7fd09b400640 1 --2- 172.21.15.123:0/550582724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c801e40 0x55af3c809ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.658+0000 7fd09be00640 1 -- 172.21.15.123:0/550582724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c80d020 msgr2=0x55af3c80a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.658+0000 7fd09be00640 1 --2- 172.21.15.123:0/550582724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c80d020 0x55af3c80a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.658+0000 7fd09be00640 1 -- 172.21.15.123:0/550582724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c801e40 msgr2=0x55af3c809ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.658+0000 7fd09be00640 1 --2- 172.21.15.123:0/550582724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c801e40 0x55af3c809ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.658+0000 7fd09be00640 1 -- 172.21.15.123:0/550582724 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af3c4d93c0 con 0x55af3c802760 2024-09-19T04:47:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.658+0000 7fd09b400640 1 --2- 172.21.15.123:0/550582724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c801e40 0x55af3c809ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.658+0000 7fd09be00640 1 --2- 172.21.15.123:0/550582724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c802760 0x55af3c80a400 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd08c000c00 tx=0x7fd08c004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca81282320e2e353 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.658+0000 7fd085600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:47:39.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.659+0000 7fd085600640 1 -- 172.21.15.123:0/550582724 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c024b90 con 0x55af3c802760 2024-09-19T04:47:39.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.659+0000 7fd085600640 1 -- 172.21.15.123:0/550582724 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd08c024d30 con 0x55af3c802760 2024-09-19T04:47:39.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.659+0000 7fd0a5401740 1 -- 172.21.15.123:0/550582724 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd08c0043c0 con 0x55af3c802760 2024-09-19T04:47:39.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.659+0000 7fd085600640 1 -- 172.21.15.123:0/550582724 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd08c004f20 con 0x55af3c802760 2024-09-19T04:47:39.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.659+0000 7fd0a5401740 10 client.15405 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.659+0000 7fd0a5401740 1 -- 172.21.15.123:0/550582724 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55af3c81bbe0 con 0x55af3c802760 2024-09-19T04:47:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.660+0000 7fd085600640 1 -- 172.21.15.123:0/550582724 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd08c0043c0 con 0x55af3c802760 2024-09-19T04:47:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.660+0000 7fd085600640 1 -- 172.21.15.123:0/550582724 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd08c026070 con 0x55af3c802760 2024-09-19T04:47:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.660+0000 7fd0a5401740 10 client.15405 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.660+0000 7fd0a5401740 10 client.15405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:47:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.660+0000 7fd0a5401740 20 client.15405 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.660+0000 7fd0a5401740 1 --2- 172.21.15.123:0/550582724 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55af3c81cfe0 0x55af3c81f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.660+0000 7fd0a5401740 4 client.15405 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:47:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.660+0000 7fd0a5401740 1 -- 172.21.15.123:0/550582724 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af3c775950 con 0x55af3c81cfe0 2024-09-19T04:47:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.661+0000 7fd09b400640 1 --2- 172.21.15.123:0/550582724 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55af3c81cfe0 0x55af3c81f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.661+0000 7fd085600640 10 client.15405 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.661+0000 7fd09b400640 1 --2- 172.21.15.123:0/550582724 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55af3c81cfe0 0x55af3c81f3d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acc1fa85334737d7 server_cookie=408671cb73190e76 in_seq=0 out_seq=0 2024-09-19T04:47:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.662+0000 7fd085600640 1 -- 172.21.15.123:0/550582724 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af3c775950 con 0x55af3c81cfe0 2024-09-19T04:47:39.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.662+0000 7fd085600640 10 client.15405 handle_command_reply: tid=0 2024-09-19T04:47:39.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.662+0000 7fd0a5401740 1 client.15405 shutdown 2024-09-19T04:47:39.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.663+0000 7fd0a5401740 1 -- 172.21.15.123:0/550582724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c802760 msgr2=0x55af3c80a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:39.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.663+0000 7fd0a5401740 1 --2- 172.21.15.123:0/550582724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c802760 0x55af3c80a400 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd08c000c00 tx=0x7fd08c004e30 comp rx=0 tx=0).stop 2024-09-19T04:47:39.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.663+0000 7fd0a5401740 1 -- 172.21.15.123:0/550582724 shutdown_connections 2024-09-19T04:47:39.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.663+0000 7fd0a5401740 1 --2- 172.21.15.123:0/550582724 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55af3c81cfe0 0x55af3c81f3d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.663+0000 7fd0a5401740 1 --2- 172.21.15.123:0/550582724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c80d020 0x55af3c80a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.663+0000 7fd0a5401740 1 --2- 172.21.15.123:0/550582724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c802760 0x55af3c80a400 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.663+0000 7fd0a5401740 1 --2- 172.21.15.123:0/550582724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c801e40 0x55af3c809ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.663+0000 7fd0a5401740 1 -- 172.21.15.123:0/550582724 >> 172.21.15.123:0/550582724 conn(0x55af3c7fe3d0 msgr2=0x55af3c8040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:39.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.663+0000 7fd085600640 0 client.15405 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:39.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.663+0000 7fd0a5401740 1 -- 172.21.15.123:0/550582724 shutdown_connections 2024-09-19T04:47:39.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.663+0000 7fd0a5401740 1 -- 172.21.15.123:0/550582724 wait complete. 2024-09-19T04:47:39.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.663+0000 7fd0a5401740 20 client.15405 trim_cache size 0 max 16384 2024-09-19T04:47:39.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.690+0000 7fd0a5401740 1 Processor -- start 2024-09-19T04:47:39.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.690+0000 7fd0a5401740 1 -- start start 2024-09-19T04:47:39.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.690+0000 7fd0a5401740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c801e40 0x55af3c824b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.690+0000 7fd0a5401740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c802760 0x55af3c825050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.690+0000 7fd0a5401740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c80d020 0x55af3c809e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.690+0000 7fd0a5401740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1ab480 con 0x55af3c802760 2024-09-19T04:47:39.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.690+0000 7fd0a5401740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1acfc0 con 0x55af3c80d020 2024-09-19T04:47:39.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.690+0000 7fd0a5401740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1abb50 con 0x55af3c801e40 2024-09-19T04:47:39.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.691+0000 7fd09b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c801e40 0x55af3c824b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.691+0000 7fd09b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c801e40 0x55af3c824b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58832/0 (socket says 172.21.15.123:58832) 2024-09-19T04:47:39.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.691+0000 7fd09b400640 1 -- 172.21.15.123:0/2114336261 learned_addr learned my addr 172.21.15.123:0/2114336261 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:39.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.691+0000 7fd09b400640 1 --2- 172.21.15.123:0/2114336261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c80d020 0x55af3c809e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.691+0000 7fd09be00640 1 --2- 172.21.15.123:0/2114336261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c802760 0x55af3c825050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.691+0000 7fd09b400640 1 -- 172.21.15.123:0/2114336261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c801e40 msgr2=0x55af3c824b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:39.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.691+0000 7fd09b400640 1 --2- 172.21.15.123:0/2114336261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c801e40 0x55af3c824b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.691+0000 7fd09b400640 1 -- 172.21.15.123:0/2114336261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c802760 msgr2=0x55af3c825050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:39.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.691+0000 7fd09b400640 1 --2- 172.21.15.123:0/2114336261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c802760 0x55af3c825050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.691+0000 7fd09b400640 1 -- 172.21.15.123:0/2114336261 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af3c804cb0 con 0x55af3c80d020 2024-09-19T04:47:39.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.692+0000 7fd09b400640 1 --2- 172.21.15.123:0/2114336261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c80d020 0x55af3c809e80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd088012630 tx=0x7fd088009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f294538205b540b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:39.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.692+0000 7fd087e00640 1 -- 172.21.15.123:0/2114336261 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088011ca0 con 0x55af3c80d020 2024-09-19T04:47:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.692+0000 7fd087e00640 1 -- 172.21.15.123:0/2114336261 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd088011e40 con 0x55af3c80d020 2024-09-19T04:47:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.692+0000 7fd087e00640 1 -- 172.21.15.123:0/2114336261 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088021380 con 0x55af3c80d020 2024-09-19T04:47:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.694+0000 7fd0a5401740 1 -- 172.21.15.123:0/2114336261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c80d020 msgr2=0x55af3c809e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.694+0000 7fd0a5401740 1 --2- 172.21.15.123:0/2114336261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c80d020 0x55af3c809e80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd088012630 tx=0x7fd088009670 comp rx=0 tx=0).stop 2024-09-19T04:47:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.694+0000 7fd0a5401740 1 -- 172.21.15.123:0/2114336261 shutdown_connections 2024-09-19T04:47:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.694+0000 7fd0a5401740 1 --2- 172.21.15.123:0/2114336261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c80d020 0x55af3c809e80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.694+0000 7fd0a5401740 1 --2- 172.21.15.123:0/2114336261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c802760 0x55af3c825050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.694+0000 7fd0a5401740 1 --2- 172.21.15.123:0/2114336261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c801e40 0x55af3c824b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.694+0000 7fd0a5401740 1 -- 172.21.15.123:0/2114336261 >> 172.21.15.123:0/2114336261 conn(0x55af3c7fe3d0 msgr2=0x55af3c808fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.694+0000 7fd0a5401740 1 -- 172.21.15.123:0/2114336261 shutdown_connections 2024-09-19T04:47:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.694+0000 7fd0a5401740 1 -- 172.21.15.123:0/2114336261 wait complete. 2024-09-19T04:47:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.695+0000 7fd0a5401740 1 -- start start 2024-09-19T04:47:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.695+0000 7fd0a5401740 1 Processor -- start 2024-09-19T04:47:39.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.696+0000 7fd0a5401740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c801e40 0x55af3c8157b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.696+0000 7fd0a5401740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c802760 0x55af3c810a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.697+0000 7fd0a5401740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c810f80 0x55af3c8113c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.697+0000 7fd0a5401740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1ac8f0 con 0x55af3c801e40 2024-09-19T04:47:39.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.697+0000 7fd0a5401740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1ac220 con 0x55af3c810f80 2024-09-19T04:47:39.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.697+0000 7fd0a5401740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1a94a0 con 0x55af3c802760 2024-09-19T04:47:39.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.697+0000 7fd0a5401740 10 client.0 _ll_register_callbacks cb 0x7fd0880098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:39.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.697+0000 7fd09b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c801e40 0x55af3c8157b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.697+0000 7fd09be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c802760 0x55af3c810a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.697+0000 7fd09b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c801e40 0x55af3c8157b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53662/0 (socket says 172.21.15.123:53662) 2024-09-19T04:47:39.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.697+0000 7fd09b400640 1 -- 172.21.15.123:0/403544253 learned_addr learned my addr 172.21.15.123:0/403544253 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:39.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.698+0000 7fd09b400640 1 --2- 172.21.15.123:0/403544253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c810f80 0x55af3c8113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.698+0000 7fd09b400640 1 -- 172.21.15.123:0/403544253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c802760 msgr2=0x55af3c810a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:39.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.698+0000 7fd09b400640 1 --2- 172.21.15.123:0/403544253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c802760 0x55af3c810a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.698+0000 7fd09b400640 1 -- 172.21.15.123:0/403544253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c810f80 msgr2=0x55af3c8113c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:39.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.698+0000 7fd09b400640 1 --2- 172.21.15.123:0/403544253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c810f80 0x55af3c8113c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.698+0000 7fd09b400640 1 -- 172.21.15.123:0/403544253 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af3c7fdc80 con 0x55af3c801e40 2024-09-19T04:47:39.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.698+0000 7fd09b400640 1 --2- 172.21.15.123:0/403544253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c801e40 0x55af3c8157b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd088006010 tx=0x7fd088009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5767066c7c12c11d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:39.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.698+0000 7fd085600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:47:39.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.699+0000 7fd085600640 1 -- 172.21.15.123:0/403544253 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088036020 con 0x55af3c801e40 2024-09-19T04:47:39.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.699+0000 7fd0a5401740 1 -- 172.21.15.123:0/403544253 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd088011e40 con 0x55af3c801e40 2024-09-19T04:47:39.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.699+0000 7fd085600640 1 -- 172.21.15.123:0/403544253 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd088003f10 con 0x55af3c801e40 2024-09-19T04:47:39.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.699+0000 7fd085600640 1 -- 172.21.15.123:0/403544253 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd088011ca0 con 0x55af3c801e40 2024-09-19T04:47:39.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.699+0000 7fd0a5401740 10 client.15411 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:39.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.700+0000 7fd0a5401740 1 -- 172.21.15.123:0/403544253 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55af3c887d90 con 0x55af3c801e40 2024-09-19T04:47:39.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.700+0000 7fd085600640 1 -- 172.21.15.123:0/403544253 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd088011e40 con 0x55af3c801e40 2024-09-19T04:47:39.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.700+0000 7fd085600640 1 -- 172.21.15.123:0/403544253 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd08801f070 con 0x55af3c801e40 2024-09-19T04:47:39.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.700+0000 7fd0a5401740 10 client.15411 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:39.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.700+0000 7fd0a5401740 10 client.15411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:47:39.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.700+0000 7fd0a5401740 20 client.15411 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:39.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.700+0000 7fd0a5401740 1 --2- 172.21.15.123:0/403544253 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55af3c888250 0x55af3c88a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:39.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.700+0000 7fd0a5401740 4 client.15411 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:47:39.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.700+0000 7fd0a5401740 1 -- 172.21.15.123:0/403544253 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af3c76fe60 con 0x55af3c888250 2024-09-19T04:47:39.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.701+0000 7fd09be00640 1 --2- 172.21.15.123:0/403544253 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55af3c888250 0x55af3c88a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:39.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.702+0000 7fd09be00640 1 --2- 172.21.15.123:0/403544253 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55af3c888250 0x55af3c88a640 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c115e1fc0315cc2 server_cookie=f0b89f708ef87113 in_seq=0 out_seq=0 2024-09-19T04:47:39.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.702+0000 7fd085600640 10 client.15411 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:39.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.702+0000 7fd085600640 1 -- 172.21.15.123:0/403544253 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af3c76fe60 con 0x55af3c888250 2024-09-19T04:47:39.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.702+0000 7fd085600640 10 client.15411 handle_command_reply: tid=0 2024-09-19T04:47:39.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.702+0000 7fd0a5401740 1 client.15411 shutdown 2024-09-19T04:47:39.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.702+0000 7fd0a5401740 1 -- 172.21.15.123:0/403544253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c801e40 msgr2=0x55af3c8157b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:39.706 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:47:39.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.702+0000 7fd0a5401740 1 --2- 172.21.15.123:0/403544253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c801e40 0x55af3c8157b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd088006010 tx=0x7fd088009260 comp rx=0 tx=0).stop 2024-09-19T04:47:39.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.702+0000 7fd0a5401740 1 -- 172.21.15.123:0/403544253 shutdown_connections 2024-09-19T04:47:39.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.702+0000 7fd0a5401740 1 --2- 172.21.15.123:0/403544253 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55af3c888250 0x55af3c88a640 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.702+0000 7fd0a5401740 1 --2- 172.21.15.123:0/403544253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3c810f80 0x55af3c8113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.702+0000 7fd0a5401740 1 --2- 172.21.15.123:0/403544253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3c802760 0x55af3c810a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.702+0000 7fd0a5401740 1 --2- 172.21.15.123:0/403544253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af3c801e40 0x55af3c8157b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.702+0000 7fd0a5401740 1 -- 172.21.15.123:0/403544253 >> 172.21.15.123:0/403544253 conn(0x55af3c7fe3d0 msgr2=0x55af3c7fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:39.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.702+0000 7fd085600640 0 client.15411 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:39.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.702+0000 7fd0a5401740 1 -- 172.21.15.123:0/403544253 shutdown_connections 2024-09-19T04:47:39.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.703+0000 7fd0a5401740 1 -- 172.21.15.123:0/403544253 wait complete. 2024-09-19T04:47:39.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.703+0000 7fd0a5401740 20 client.15411 trim_cache size 0 max 16384 2024-09-19T04:47:39.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.703+0000 7fd0a1400640 1 -- 172.21.15.123:0/1935334471 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd0800a35e0 msgr2=0x7fd0800a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:39.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.703+0000 7fd0a1400640 1 --2- 172.21.15.123:0/1935334471 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd0800a35e0 0x7fd0800a5aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd0940eea70 tx=0x7fd094005df0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.703+0000 7fd0a1400640 1 -- 172.21.15.123:0/1935334471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c1079c0 msgr2=0x7fd09c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:39.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.703+0000 7fd0a1400640 1 --2- 172.21.15.123:0/1935334471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c1079c0 0x7fd09c074f70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd09402ae10 tx=0x7fd094003f10 comp rx=0 tx=0).stop 2024-09-19T04:47:39.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.704+0000 7fd0a1400640 1 -- 172.21.15.123:0/1935334471 shutdown_connections 2024-09-19T04:47:39.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.704+0000 7fd0a1400640 1 --2- 172.21.15.123:0/1935334471 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd0800a35e0 0x7fd0800a5aa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.704+0000 7fd0a1400640 1 --2- 172.21.15.123:0/1935334471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c1079c0 0x7fd09c074f70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.704+0000 7fd0a1400640 1 --2- 172.21.15.123:0/1935334471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd09c102360 0x7fd09c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.704+0000 7fd0a1400640 1 --2- 172.21.15.123:0/1935334471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd09c1019c0 0x7fd09c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:39.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.704+0000 7fd0a1400640 1 -- 172.21.15.123:0/1935334471 >> 172.21.15.123:0/1935334471 conn(0x7fd09c0fd740 msgr2=0x7fd09c0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:39.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.704+0000 7fd0a1400640 1 -- 172.21.15.123:0/1935334471 shutdown_connections 2024-09-19T04:47:39.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:39.704+0000 7fd0a1400640 1 -- 172.21.15.123:0/1935334471 wait complete. 2024-09-19T04:47:39.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:39 smithi193 ceph-mon[35592]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T04:47:41.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:41 smithi183 ceph-mon[35333]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T04:47:41.641 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:47:41.641 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:47:41.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:41 smithi123 ceph-mon[30511]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T04:47:41.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:41 smithi193 ceph-mon[35592]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T04:47:42.202 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:47:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.882+0000 7f2161400640 1 -- 172.21.15.123:0/308098213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f215c10e260 msgr2=0x7f215c110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.882+0000 7f2161400640 1 --2- 172.21.15.123:0/308098213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f215c10e260 0x7f215c110650 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2154009ca0 tx=0x7f215402a940 comp rx=0 tx=0).stop 2024-09-19T04:47:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.882+0000 7f2161400640 1 -- 172.21.15.123:0/308098213 shutdown_connections 2024-09-19T04:47:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.882+0000 7f2161400640 1 --2- 172.21.15.123:0/308098213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f215c10e260 0x7f215c110650 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.882+0000 7f2161400640 1 --2- 172.21.15.123:0/308098213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f215c074ee0 0x7f215c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.882+0000 7f2161400640 1 --2- 172.21.15.123:0/308098213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f215c0745c0 0x7f215c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.882+0000 7f2161400640 1 -- 172.21.15.123:0/308098213 >> 172.21.15.123:0/308098213 conn(0x7f215c0fd740 msgr2=0x7f215c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.883+0000 7f2161400640 1 -- 172.21.15.123:0/308098213 shutdown_connections 2024-09-19T04:47:42.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.883+0000 7f2161400640 1 -- 172.21.15.123:0/308098213 wait complete. 2024-09-19T04:47:42.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.884+0000 7f2161400640 1 Processor -- start 2024-09-19T04:47:42.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.885+0000 7f2161400640 1 -- start start 2024-09-19T04:47:42.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.885+0000 7f2161400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f215c0745c0 0x7f215c10db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.885+0000 7f2161400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f215c074ee0 0x7f215c10ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.885+0000 7f2161400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f215c10e260 0x7f215c10b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.885+0000 7f2161400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f215c114000 con 0x7f215c0745c0 2024-09-19T04:47:42.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.885+0000 7f2161400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f215c113e80 con 0x7f215c10e260 2024-09-19T04:47:42.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.885+0000 7f2161400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f215c114180 con 0x7f215c074ee0 2024-09-19T04:47:42.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.886+0000 7f2160a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f215c10e260 0x7f215c10b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:42.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.886+0000 7f215b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f215c074ee0 0x7f215c10ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:42.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.886+0000 7f2160a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f215c10e260 0x7f215c10b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:56294/0 (socket says 172.21.15.123:56294) 2024-09-19T04:47:42.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.886+0000 7f2160a00640 1 -- 172.21.15.123:0/3749449883 learned_addr learned my addr 172.21.15.123:0/3749449883 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:42.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.886+0000 7f215be00640 1 --2- 172.21.15.123:0/3749449883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f215c0745c0 0x7f215c10db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:42.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.886+0000 7f2160a00640 1 -- 172.21.15.123:0/3749449883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f215c074ee0 msgr2=0x7f215c10ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:42.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.886+0000 7f2160a00640 1 --2- 172.21.15.123:0/3749449883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f215c074ee0 0x7f215c10ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.886+0000 7f2160a00640 1 -- 172.21.15.123:0/3749449883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f215c0745c0 msgr2=0x7f215c10db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:42.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.886+0000 7f2160a00640 1 --2- 172.21.15.123:0/3749449883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f215c0745c0 0x7f215c10db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.887+0000 7f2160a00640 1 -- 172.21.15.123:0/3749449883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c10bab0 con 0x7f215c10e260 2024-09-19T04:47:42.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.887+0000 7f2160a00640 1 --2- 172.21.15.123:0/3749449883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f215c10e260 0x7f215c10b1f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2154005e30 tx=0x7f215402f630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:42.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.888+0000 7f2158c00640 1 -- 172.21.15.123:0/3749449883 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154034040 con 0x7f215c10e260 2024-09-19T04:47:42.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.888+0000 7f2158c00640 1 -- 172.21.15.123:0/3749449883 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f215402fe10 con 0x7f215c10e260 2024-09-19T04:47:42.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.888+0000 7f2161400640 1 -- 172.21.15.123:0/3749449883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f215c1aadc0 con 0x7f215c10e260 2024-09-19T04:47:42.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.888+0000 7f2158c00640 1 -- 172.21.15.123:0/3749449883 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21540045f0 con 0x7f215c10e260 2024-09-19T04:47:42.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.888+0000 7f2161400640 1 -- 172.21.15.123:0/3749449883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f215c1ab2a0 con 0x7f215c10e260 2024-09-19T04:47:42.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.892+0000 7f21654b7740 1 Processor -- start 2024-09-19T04:47:42.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.893+0000 7f2158c00640 1 -- 172.21.15.123:0/3749449883 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7f2154037020 con 0x7f215c10e260 2024-09-19T04:47:42.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.893+0000 7f21654b7740 1 -- start start 2024-09-19T04:47:42.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.893+0000 7f21654b7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd08701cb0 0x55bd08702090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.893+0000 7f21654b7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd087025d0 0x55bd0870c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.894+0000 7f21654b7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd0870ce90 0x55bd0870f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.894+0000 7f21654b7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bd08668290 con 0x55bd0870ce90 2024-09-19T04:47:42.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.894+0000 7f21654b7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bd0860fb40 con 0x55bd087025d0 2024-09-19T04:47:42.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.894+0000 7f21654b7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bd08664250 con 0x55bd08701cb0 2024-09-19T04:47:42.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.894+0000 7f215be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd087025d0 0x55bd0870c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:42.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.894+0000 7f215be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd087025d0 0x55bd0870c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:56296/0 (socket says 172.21.15.123:56296) 2024-09-19T04:47:42.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.894+0000 7f215be00640 1 -- 172.21.15.123:0/914534835 learned_addr learned my addr 172.21.15.123:0/914534835 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:42.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.895+0000 7f215b400640 1 --2- 172.21.15.123:0/914534835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd0870ce90 0x55bd0870f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:42.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.895+0000 7f215b400640 1 --2- 172.21.15.123:0/914534835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd08701cb0 0x55bd08702090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:42.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.895+0000 7f215be00640 1 -- 172.21.15.123:0/914534835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd08701cb0 msgr2=0x55bd08702090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:42.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.895+0000 7f215be00640 1 --2- 172.21.15.123:0/914534835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd08701cb0 0x55bd08702090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.895+0000 7f215be00640 1 -- 172.21.15.123:0/914534835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd0870ce90 msgr2=0x55bd0870f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:42.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.895+0000 7f215be00640 1 --2- 172.21.15.123:0/914534835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd0870ce90 0x55bd0870f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.895+0000 7f215be00640 1 -- 172.21.15.123:0/914534835 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd083d96a0 con 0x55bd087025d0 2024-09-19T04:47:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.895+0000 7f215b400640 1 --2- 172.21.15.123:0/914534835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd0870ce90 0x55bd0870f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:47:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.895+0000 7f2158c00640 1 --2- 172.21.15.123:0/3749449883 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f21400a3560 0x7f21400a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.895+0000 7f2158c00640 1 -- 172.21.15.123:0/3749449883 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f21540f0e40 con 0x7f215c10e260 2024-09-19T04:47:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.896+0000 7f215be00640 1 --2- 172.21.15.123:0/914534835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd087025d0 0x55bd0870c950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f214c009c80 tx=0x7f214c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0729b1b86831d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.896+0000 7f215b400640 1 --2- 172.21.15.123:0/3749449883 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f21400a3560 0x7f21400a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.896+0000 7f2151200640 1 -- 172.21.15.123:0/914534835 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c012450 con 0x55bd087025d0 2024-09-19T04:47:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.896+0000 7f2151200640 1 -- 172.21.15.123:0/914534835 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f214c012ad0 con 0x55bd087025d0 2024-09-19T04:47:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.897+0000 7f21654b7740 1 -- 172.21.15.123:0/914534835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd087025d0 msgr2=0x55bd0870c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:42.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.897+0000 7f2151200640 1 -- 172.21.15.123:0/914534835 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c012450 con 0x55bd087025d0 2024-09-19T04:47:42.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.897+0000 7f215b400640 1 --2- 172.21.15.123:0/3749449883 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f21400a3560 0x7f21400a5a20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55bd086ff5c0 tx=0x7f2148005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:42.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.897+0000 7f21654b7740 1 --2- 172.21.15.123:0/914534835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd087025d0 0x55bd0870c950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f214c009c80 tx=0x7f214c0117b0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.897+0000 7f21654b7740 1 -- 172.21.15.123:0/914534835 shutdown_connections 2024-09-19T04:47:42.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.897+0000 7f21654b7740 1 --2- 172.21.15.123:0/914534835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd0870ce90 0x55bd0870f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.897+0000 7f21654b7740 1 --2- 172.21.15.123:0/914534835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd087025d0 0x55bd0870c950 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.897+0000 7f21654b7740 1 --2- 172.21.15.123:0/914534835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd08701cb0 0x55bd08702090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.897+0000 7f21654b7740 1 -- 172.21.15.123:0/914534835 >> 172.21.15.123:0/914534835 conn(0x55bd086fe240 msgr2=0x55bd086fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:42.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.898+0000 7f21654b7740 1 -- 172.21.15.123:0/914534835 shutdown_connections 2024-09-19T04:47:42.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.898+0000 7f21654b7740 1 -- 172.21.15.123:0/914534835 wait complete. 2024-09-19T04:47:42.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.898+0000 7f21654b7740 1 -- start start 2024-09-19T04:47:42.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.899+0000 7f21654b7740 1 Processor -- start 2024-09-19T04:47:42.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.899+0000 7f21654b7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd08701cb0 0x55bd0870c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.900+0000 7f21654b7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd087025d0 0x55bd087098f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.900+0000 7f21654b7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0870ce90 0x55bd08709e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.900+0000 7f21654b7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bd0870fb50 con 0x55bd08701cb0 2024-09-19T04:47:42.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.900+0000 7f21654b7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bd08622fa0 con 0x55bd087025d0 2024-09-19T04:47:42.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.900+0000 7f21654b7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bd0870fcd0 con 0x55bd0870ce90 2024-09-19T04:47:42.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.900+0000 7f215be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd08701cb0 0x55bd0870c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:42.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.900+0000 7f21654b7740 10 client.0 _ll_register_callbacks cb 0x55bd08376ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:42.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.900+0000 7f215be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd08701cb0 0x55bd0870c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41238/0 (socket says 172.21.15.123:41238) 2024-09-19T04:47:42.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.900+0000 7f215be00640 1 -- 172.21.15.123:0/2416186722 learned_addr learned my addr 172.21.15.123:0/2416186722 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:42.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.900+0000 7f2160a00640 1 --2- 172.21.15.123:0/2416186722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0870ce90 0x55bd08709e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:42.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.900+0000 7f215be00640 1 -- 172.21.15.123:0/2416186722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0870ce90 msgr2=0x55bd08709e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:42.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.900+0000 7f215be00640 1 --2- 172.21.15.123:0/2416186722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0870ce90 0x55bd08709e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.900+0000 7f215be00640 1 -- 172.21.15.123:0/2416186722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd087025d0 msgr2=0x55bd087098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:42.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.900+0000 7f215be00640 1 --2- 172.21.15.123:0/2416186722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd087025d0 0x55bd087098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.900+0000 7f215be00640 1 -- 172.21.15.123:0/2416186722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd0870a520 con 0x55bd08701cb0 2024-09-19T04:47:42.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.900+0000 7f2160a00640 1 --2- 172.21.15.123:0/2416186722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0870ce90 0x55bd08709e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:47:42.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.901+0000 7f215be00640 1 --2- 172.21.15.123:0/2416186722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd08701cb0 0x55bd0870c640 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f214c000c00 tx=0x7f214c0098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458787e873bd62d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:42.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.901+0000 7f2145600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:47:42.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.902+0000 7f2145600640 1 -- 172.21.15.123:0/2416186722 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c028020 con 0x55bd08701cb0 2024-09-19T04:47:42.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.902+0000 7f2145600640 1 -- 172.21.15.123:0/2416186722 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f214c012820 con 0x55bd08701cb0 2024-09-19T04:47:42.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.902+0000 7f21654b7740 1 -- 172.21.15.123:0/2416186722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f214c012ad0 con 0x55bd08701cb0 2024-09-19T04:47:42.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.903+0000 7f2145600640 1 -- 172.21.15.123:0/2416186722 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f214c005050 con 0x55bd08701cb0 2024-09-19T04:47:42.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.903+0000 7f21654b7740 10 client.15420 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:42.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.903+0000 7f21654b7740 1 -- 172.21.15.123:0/2416186722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bd0871b8c0 con 0x55bd08701cb0 2024-09-19T04:47:42.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.903+0000 7f2145600640 1 -- 172.21.15.123:0/2416186722 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f214c012ad0 con 0x55bd08701cb0 2024-09-19T04:47:42.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.903+0000 7f2145600640 1 -- 172.21.15.123:0/2416186722 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f214c004790 con 0x55bd08701cb0 2024-09-19T04:47:42.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.904+0000 7f21654b7740 10 client.15420 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:42.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.904+0000 7f21654b7740 10 client.15420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:47:42.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.904+0000 7f21654b7740 20 client.15420 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:42.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.904+0000 7f21654b7740 1 --2- 172.21.15.123:0/2416186722 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55bd0871ccd0 0x55bd0871f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.904+0000 7f21654b7740 4 client.15420 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:47:42.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.904+0000 7f21654b7740 1 -- 172.21.15.123:0/2416186722 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd08675800 con 0x55bd0871ccd0 2024-09-19T04:47:42.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.904+0000 7f215b400640 1 --2- 172.21.15.123:0/2416186722 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55bd0871ccd0 0x55bd0871f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:42.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.905+0000 7f2145600640 10 client.15420 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:42.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.905+0000 7f215b400640 1 --2- 172.21.15.123:0/2416186722 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55bd0871ccd0 0x55bd0871f0e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c3684180d5bdc1f server_cookie=5593e86a6cb10caa in_seq=0 out_seq=0 2024-09-19T04:47:42.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.906+0000 7f2145600640 1 -- 172.21.15.123:0/2416186722 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd08675800 con 0x55bd0871ccd0 2024-09-19T04:47:42.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.906+0000 7f2145600640 10 client.15420 handle_command_reply: tid=0 2024-09-19T04:47:42.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.906+0000 7f21654b7740 1 client.15420 shutdown 2024-09-19T04:47:42.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.906+0000 7f21654b7740 1 -- 172.21.15.123:0/2416186722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd08701cb0 msgr2=0x55bd0870c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:42.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.906+0000 7f21654b7740 1 --2- 172.21.15.123:0/2416186722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd08701cb0 0x55bd0870c640 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f214c000c00 tx=0x7f214c0098b0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.906+0000 7f21654b7740 1 -- 172.21.15.123:0/2416186722 shutdown_connections 2024-09-19T04:47:42.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.906+0000 7f21654b7740 1 --2- 172.21.15.123:0/2416186722 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55bd0871ccd0 0x55bd0871f0e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.907+0000 7f21654b7740 1 --2- 172.21.15.123:0/2416186722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0870ce90 0x55bd08709e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.907+0000 7f21654b7740 1 --2- 172.21.15.123:0/2416186722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd087025d0 0x55bd087098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.907+0000 7f21654b7740 1 --2- 172.21.15.123:0/2416186722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd08701cb0 0x55bd0870c640 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.907+0000 7f21654b7740 1 -- 172.21.15.123:0/2416186722 >> 172.21.15.123:0/2416186722 conn(0x55bd086fe240 msgr2=0x55bd08703f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:42.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.907+0000 7f2145600640 0 client.15420 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:42.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.907+0000 7f21654b7740 1 -- 172.21.15.123:0/2416186722 shutdown_connections 2024-09-19T04:47:42.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.907+0000 7f21654b7740 1 -- 172.21.15.123:0/2416186722 wait complete. 2024-09-19T04:47:42.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.907+0000 7f21654b7740 20 client.15420 trim_cache size 0 max 16384 2024-09-19T04:47:42.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.942+0000 7f21654b7740 1 Processor -- start 2024-09-19T04:47:42.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.942+0000 7f21654b7740 1 -- start start 2024-09-19T04:47:42.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.943+0000 7f21654b7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd087025d0 0x55bd0870b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.943+0000 7f21654b7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0870ce90 0x55bd0870e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.943+0000 7f21654b7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0870e7c0 0x55bd08706b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.943+0000 7f21654b7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1b3bf0 con 0x55bd087025d0 2024-09-19T04:47:42.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.943+0000 7f21654b7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1b5730 con 0x55bd0870ce90 2024-09-19T04:47:42.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.943+0000 7f21654b7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1b42c0 con 0x55bd0870e7c0 2024-09-19T04:47:42.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.943+0000 7f2160a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0870e7c0 0x55bd08706b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:42.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.943+0000 7f215be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd087025d0 0x55bd0870b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:42.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.943+0000 7f215b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0870ce90 0x55bd0870e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:42.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.943+0000 7f215b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0870ce90 0x55bd0870e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:56324/0 (socket says 172.21.15.123:56324) 2024-09-19T04:47:42.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.943+0000 7f215b400640 1 -- 172.21.15.123:0/56992603 learned_addr learned my addr 172.21.15.123:0/56992603 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:42.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.943+0000 7f2160a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0870e7c0 0x55bd08706b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:48994/0 (socket says 172.21.15.123:48994) 2024-09-19T04:47:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.944+0000 7f215b400640 1 -- 172.21.15.123:0/56992603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0870e7c0 msgr2=0x55bd08706b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.944+0000 7f215b400640 1 --2- 172.21.15.123:0/56992603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0870e7c0 0x55bd08706b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.944+0000 7f215b400640 1 -- 172.21.15.123:0/56992603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd087025d0 msgr2=0x55bd0870b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.944+0000 7f215b400640 1 --2- 172.21.15.123:0/56992603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd087025d0 0x55bd0870b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.944+0000 7f215b400640 1 -- 172.21.15.123:0/56992603 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd086ff3d0 con 0x55bd0870ce90 2024-09-19T04:47:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.944+0000 7f215be00640 1 --2- 172.21.15.123:0/56992603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd087025d0 0x55bd0870b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:47:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.945+0000 7f215b400640 1 --2- 172.21.15.123:0/56992603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0870ce90 0x55bd0870e280 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55bd08709180 tx=0x7f21480046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbe2355910b40ce1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:42.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.945+0000 7f2147e00640 1 -- 172.21.15.123:0/56992603 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148010a40 con 0x55bd0870ce90 2024-09-19T04:47:42.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.945+0000 7f2147e00640 1 -- 172.21.15.123:0/56992603 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2148010be0 con 0x55bd0870ce90 2024-09-19T04:47:42.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.945+0000 7f2147e00640 1 -- 172.21.15.123:0/56992603 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214801c8b0 con 0x55bd0870ce90 2024-09-19T04:47:42.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.946+0000 7f21654b7740 1 -- 172.21.15.123:0/56992603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0870ce90 msgr2=0x55bd0870e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:42.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.946+0000 7f21654b7740 1 --2- 172.21.15.123:0/56992603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0870ce90 0x55bd0870e280 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55bd08709180 tx=0x7f21480046c0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.946+0000 7f21654b7740 1 -- 172.21.15.123:0/56992603 shutdown_connections 2024-09-19T04:47:42.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.946+0000 7f21654b7740 1 --2- 172.21.15.123:0/56992603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0870e7c0 0x55bd08706b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.946+0000 7f21654b7740 1 --2- 172.21.15.123:0/56992603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0870ce90 0x55bd0870e280 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.947+0000 7f21654b7740 1 --2- 172.21.15.123:0/56992603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd087025d0 0x55bd0870b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.947+0000 7f21654b7740 1 -- 172.21.15.123:0/56992603 >> 172.21.15.123:0/56992603 conn(0x55bd086fe240 msgr2=0x55bd086fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:42.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.947+0000 7f21654b7740 1 -- 172.21.15.123:0/56992603 shutdown_connections 2024-09-19T04:47:42.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.948+0000 7f21654b7740 1 -- 172.21.15.123:0/56992603 wait complete. 2024-09-19T04:47:42.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.948+0000 7f21654b7740 1 -- start start 2024-09-19T04:47:42.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.948+0000 7f21654b7740 1 Processor -- start 2024-09-19T04:47:42.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.949+0000 7f21654b7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd087025d0 0x55bd08724c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.949+0000 7f21654b7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0870ce90 0x55bd08725180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.949+0000 7f21654b7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd0870e7c0 0x55bd087211b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.949+0000 7f21654b7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1b5060 con 0x55bd0870e7c0 2024-09-19T04:47:42.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.949+0000 7f21654b7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1b4990 con 0x55bd0870ce90 2024-09-19T04:47:42.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.949+0000 7f21654b7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1b1bd0 con 0x55bd087025d0 2024-09-19T04:47:42.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.949+0000 7f21654b7740 10 client.0 _ll_register_callbacks cb 0x7f214c009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:42.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.950+0000 7f215be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd087025d0 0x55bd08724c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:42.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.950+0000 7f215b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0870ce90 0x55bd08725180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:42.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.950+0000 7f215b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0870ce90 0x55bd08725180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:56330/0 (socket says 172.21.15.123:56330) 2024-09-19T04:47:42.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.950+0000 7f215b400640 1 -- 172.21.15.123:0/3990740528 learned_addr learned my addr 172.21.15.123:0/3990740528 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:42.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.950+0000 7f2160a00640 1 --2- 172.21.15.123:0/3990740528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd0870e7c0 0x55bd087211b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:42.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.951+0000 7f215b400640 1 -- 172.21.15.123:0/3990740528 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd087025d0 msgr2=0x55bd08724c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:42.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.951+0000 7f215b400640 1 --2- 172.21.15.123:0/3990740528 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd087025d0 0x55bd08724c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.951+0000 7f215b400640 1 -- 172.21.15.123:0/3990740528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd0870e7c0 msgr2=0x55bd087211b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:42.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.951+0000 7f215b400640 1 --2- 172.21.15.123:0/3990740528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd0870e7c0 0x55bd087211b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.951+0000 7f215b400640 1 -- 172.21.15.123:0/3990740528 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd087219e0 con 0x55bd0870ce90 2024-09-19T04:47:42.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.951+0000 7f215be00640 1 --2- 172.21.15.123:0/3990740528 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd087025d0 0x55bd08724c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:47:42.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.951+0000 7f2160a00640 1 --2- 172.21.15.123:0/3990740528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd0870e7c0 0x55bd087211b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:42.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.951+0000 7f215b400640 1 --2- 172.21.15.123:0/3990740528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0870ce90 0x55bd08725180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2148018040 tx=0x7f2148028cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=458d91040c224519 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:42.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.951+0000 7f2145600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:47:42.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.952+0000 7f2145600640 1 -- 172.21.15.123:0/3990740528 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148010d90 con 0x55bd0870ce90 2024-09-19T04:47:42.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.952+0000 7f2145600640 1 -- 172.21.15.123:0/3990740528 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f2148028430 con 0x55bd0870ce90 2024-09-19T04:47:42.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.952+0000 7f21654b7740 1 -- 172.21.15.123:0/3990740528 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2148010be0 con 0x55bd0870ce90 2024-09-19T04:47:42.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.952+0000 7f2145600640 1 -- 172.21.15.123:0/3990740528 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2148028630 con 0x55bd0870ce90 2024-09-19T04:47:42.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.952+0000 7f21654b7740 10 client.25460 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:42.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.952+0000 7f21654b7740 1 -- 172.21.15.123:0/3990740528 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bd08788570 con 0x55bd0870ce90 2024-09-19T04:47:42.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.953+0000 7f2145600640 1 -- 172.21.15.123:0/3990740528 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2148010be0 con 0x55bd0870ce90 2024-09-19T04:47:42.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.953+0000 7f2145600640 1 -- 172.21.15.123:0/3990740528 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2148004ab0 con 0x55bd0870ce90 2024-09-19T04:47:42.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.954+0000 7f21654b7740 10 client.25460 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:42.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.954+0000 7f21654b7740 10 client.25460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:47:42.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.954+0000 7f21654b7740 20 client.25460 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:42.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.954+0000 7f21654b7740 1 --2- 172.21.15.123:0/3990740528 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55bd08788c50 0x55bd0878b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:42.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.954+0000 7f21654b7740 4 client.25460 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:47:42.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.954+0000 7f21654b7740 1 -- 172.21.15.123:0/3990740528 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bd0866fd10 con 0x55bd08788c50 2024-09-19T04:47:42.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.954+0000 7f215be00640 1 --2- 172.21.15.123:0/3990740528 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55bd08788c50 0x55bd0878b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:42.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.955+0000 7f215be00640 1 --2- 172.21.15.123:0/3990740528 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55bd08788c50 0x55bd0878b070 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec78cbca2b5e475 server_cookie=c0c2254a1136fb9a in_seq=0 out_seq=0 2024-09-19T04:47:42.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.955+0000 7f2145600640 10 client.25460 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:42.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.956+0000 7f2145600640 1 -- 172.21.15.123:0/3990740528 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bd0866fd10 con 0x55bd08788c50 2024-09-19T04:47:42.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.956+0000 7f2145600640 10 client.25460 handle_command_reply: tid=0 2024-09-19T04:47:42.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.956+0000 7f21654b7740 1 client.25460 shutdown 2024-09-19T04:47:42.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.956+0000 7f21654b7740 1 -- 172.21.15.123:0/3990740528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0870ce90 msgr2=0x55bd08725180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:42.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.956+0000 7f21654b7740 1 --2- 172.21.15.123:0/3990740528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0870ce90 0x55bd08725180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2148018040 tx=0x7f2148028cc0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.956+0000 7f21654b7740 1 -- 172.21.15.123:0/3990740528 shutdown_connections 2024-09-19T04:47:42.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.956+0000 7f21654b7740 1 --2- 172.21.15.123:0/3990740528 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55bd08788c50 0x55bd0878b070 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.956+0000 7f21654b7740 1 --2- 172.21.15.123:0/3990740528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd0870e7c0 0x55bd087211b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.956+0000 7f21654b7740 1 --2- 172.21.15.123:0/3990740528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0870ce90 0x55bd08725180 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.956+0000 7f21654b7740 1 --2- 172.21.15.123:0/3990740528 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd087025d0 0x55bd08724c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.957+0000 7f21654b7740 1 -- 172.21.15.123:0/3990740528 >> 172.21.15.123:0/3990740528 conn(0x55bd086fe240 msgr2=0x55bd08717fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:42.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.957+0000 7f2145600640 0 client.25460 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:42.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.958+0000 7f21654b7740 1 -- 172.21.15.123:0/3990740528 shutdown_connections 2024-09-19T04:47:42.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.958+0000 7f21654b7740 1 -- 172.21.15.123:0/3990740528 wait complete. 2024-09-19T04:47:42.960 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:47:42.960 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-09-19T04:47:42.960 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "98cee490-ef25-48bf-aad7-8a22c41f8b01", 2024-09-19T04:47:42.960 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-09-19T04:47:42.961 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:47:42.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.958+0000 7f21654b7740 20 client.25460 trim_cache size 0 max 16384 2024-09-19T04:47:42.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.959+0000 7f2161400640 1 -- 172.21.15.123:0/3749449883 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f21400a3560 msgr2=0x7f21400a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:42.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.959+0000 7f2161400640 1 --2- 172.21.15.123:0/3749449883 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f21400a3560 0x7f21400a5a20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55bd086ff5c0 tx=0x7f2148005f90 comp rx=0 tx=0).stop 2024-09-19T04:47:42.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.959+0000 7f2161400640 1 -- 172.21.15.123:0/3749449883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f215c10e260 msgr2=0x7f215c10b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:42.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.959+0000 7f2161400640 1 --2- 172.21.15.123:0/3749449883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f215c10e260 0x7f215c10b1f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2154005e30 tx=0x7f215402f630 comp rx=0 tx=0).stop 2024-09-19T04:47:42.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.960+0000 7f2161400640 1 -- 172.21.15.123:0/3749449883 shutdown_connections 2024-09-19T04:47:42.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.960+0000 7f2161400640 1 --2- 172.21.15.123:0/3749449883 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f21400a3560 0x7f21400a5a20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.960+0000 7f2161400640 1 --2- 172.21.15.123:0/3749449883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f215c10e260 0x7f215c10b1f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.960+0000 7f2161400640 1 --2- 172.21.15.123:0/3749449883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f215c074ee0 0x7f215c10ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.960+0000 7f2161400640 1 --2- 172.21.15.123:0/3749449883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f215c0745c0 0x7f215c10db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:42.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.961+0000 7f2161400640 1 -- 172.21.15.123:0/3749449883 >> 172.21.15.123:0/3749449883 conn(0x7f215c0fd740 msgr2=0x7f215c0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:42.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.961+0000 7f2161400640 1 -- 172.21.15.123:0/3749449883 shutdown_connections 2024-09-19T04:47:42.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:42.961+0000 7f2161400640 1 -- 172.21.15.123:0/3749449883 wait complete. 2024-09-19T04:47:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:43 smithi183 ceph-mon[35333]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T04:47:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:43 smithi183 ceph-mon[35333]: scrub queued for path: / 2024-09-19T04:47:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:43 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:47:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:43 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:47:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:43 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:47:43.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:43 smithi123 ceph-mon[30511]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T04:47:43.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:43 smithi123 ceph-mon[30511]: scrub queued for path: / 2024-09-19T04:47:43.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:43 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:47:43.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:43 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:47:43.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:43 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:47:43.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:43 smithi193 ceph-mon[35592]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T04:47:43.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:43 smithi193 ceph-mon[35592]: scrub queued for path: / 2024-09-19T04:47:43.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:43 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:47:43.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:43 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:47:43.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:43 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:47:43.868 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:47:44.430 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:47:44.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:44 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:47:44.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:44 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:47:44.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:44 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:47:45.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.050+0000 7fd771400640 1 -- 172.21.15.123:0/669037630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd76c1019c0 msgr2=0x7fd76c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:45.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.050+0000 7fd771400640 1 --2- 172.21.15.123:0/669037630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd76c1019c0 0x7fd76c101e20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd758009930 tx=0x7fd75802a420 comp rx=0 tx=0).stop 2024-09-19T04:47:45.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.050+0000 7fd771400640 1 -- 172.21.15.123:0/669037630 shutdown_connections 2024-09-19T04:47:45.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.050+0000 7fd771400640 1 --2- 172.21.15.123:0/669037630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd76c102360 0x7fd76c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.050+0000 7fd771400640 1 --2- 172.21.15.123:0/669037630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd76c1019c0 0x7fd76c101e20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.050+0000 7fd771400640 1 --2- 172.21.15.123:0/669037630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd76c1079c0 0x7fd76c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.050+0000 7fd771400640 1 -- 172.21.15.123:0/669037630 >> 172.21.15.123:0/669037630 conn(0x7fd76c0fd740 msgr2=0x7fd76c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:45.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.051+0000 7fd771400640 1 -- 172.21.15.123:0/669037630 shutdown_connections 2024-09-19T04:47:45.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.051+0000 7fd771400640 1 -- 172.21.15.123:0/669037630 wait complete. 2024-09-19T04:47:45.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.052+0000 7fd771400640 1 Processor -- start 2024-09-19T04:47:45.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.052+0000 7fd771400640 1 -- start start 2024-09-19T04:47:45.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.053+0000 7fd771400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd76c1019c0 0x7fd76c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.053+0000 7fd771400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd76c102360 0x7fd76c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.053+0000 7fd771400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd76c1079c0 0x7fd76c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.053+0000 7fd771400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c111dd0 con 0x7fd76c1079c0 2024-09-19T04:47:45.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.053+0000 7fd771400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c111c50 con 0x7fd76c102360 2024-09-19T04:47:45.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.053+0000 7fd771400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c111f50 con 0x7fd76c1019c0 2024-09-19T04:47:45.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.053+0000 7fd76b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd76c102360 0x7fd76c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:45.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.053+0000 7fd76b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd76c102360 0x7fd76c1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:56358/0 (socket says 172.21.15.123:56358) 2024-09-19T04:47:45.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.053+0000 7fd76b400640 1 -- 172.21.15.123:0/855293176 learned_addr learned my addr 172.21.15.123:0/855293176 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:45.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.053+0000 7fd76be00640 1 --2- 172.21.15.123:0/855293176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd76c1019c0 0x7fd76c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:45.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.054+0000 7fd770a00640 1 --2- 172.21.15.123:0/855293176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd76c1079c0 0x7fd76c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:45.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.054+0000 7fd76b400640 1 -- 172.21.15.123:0/855293176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd76c1019c0 msgr2=0x7fd76c1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:45.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.054+0000 7fd76b400640 1 --2- 172.21.15.123:0/855293176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd76c1019c0 0x7fd76c1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.054+0000 7fd76b400640 1 -- 172.21.15.123:0/855293176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd76c1079c0 msgr2=0x7fd76c19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:45.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.054+0000 7fd76b400640 1 --2- 172.21.15.123:0/855293176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd76c1079c0 0x7fd76c19cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.054+0000 7fd76b400640 1 -- 172.21.15.123:0/855293176 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c19d260 con 0x7fd76c102360 2024-09-19T04:47:45.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.054+0000 7fd76be00640 1 --2- 172.21.15.123:0/855293176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd76c1019c0 0x7fd76c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:47:45.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.054+0000 7fd770a00640 1 --2- 172.21.15.123:0/855293176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd76c1079c0 0x7fd76c19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:47:45.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.055+0000 7fd76b400640 1 --2- 172.21.15.123:0/855293176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd76c102360 0x7fd76c1987f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd75802a940 tx=0x7fd758005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:45.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.055+0000 7fd768c00640 1 -- 172.21.15.123:0/855293176 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd758035030 con 0x7fd76c102360 2024-09-19T04:47:45.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.055+0000 7fd771400640 1 -- 172.21.15.123:0/855293176 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76c19d4f0 con 0x7fd76c102360 2024-09-19T04:47:45.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.055+0000 7fd768c00640 1 -- 172.21.15.123:0/855293176 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd758041070 con 0x7fd76c102360 2024-09-19T04:47:45.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.055+0000 7fd768c00640 1 -- 172.21.15.123:0/855293176 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd758031da0 con 0x7fd76c102360 2024-09-19T04:47:45.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.055+0000 7fd771400640 1 -- 172.21.15.123:0/855293176 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd76c074420 con 0x7fd76c102360 2024-09-19T04:47:45.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.060+0000 7fd77564f740 1 Processor -- start 2024-09-19T04:47:45.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.060+0000 7fd77564f740 1 -- start start 2024-09-19T04:47:45.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.061+0000 7fd768c00640 1 -- 172.21.15.123:0/855293176 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7fd75803d760 con 0x7fd76c102360 2024-09-19T04:47:45.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.061+0000 7fd77564f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece1e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.061+0000 7fd77564f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1e670 0x55a5ece289f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.061+0000 7fd77564f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5ece28f30 0x55a5ece2b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.061+0000 7fd77564f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a5ecd843e0 con 0x55a5ece1e670 2024-09-19T04:47:45.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.061+0000 7fd77564f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a5ecd2bd80 con 0x55a5ece28f30 2024-09-19T04:47:45.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.061+0000 7fd77564f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a5ecd803a0 con 0x55a5ece1dd50 2024-09-19T04:47:45.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.061+0000 7fd770a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece1e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:45.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.061+0000 7fd770a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece1e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49042/0 (socket says 172.21.15.123:49042) 2024-09-19T04:47:45.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.061+0000 7fd770a00640 1 -- 172.21.15.123:0/2369007238 learned_addr learned my addr 172.21.15.123:0/2369007238 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:45.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.062+0000 7fd76be00640 1 --2- 172.21.15.123:0/2369007238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1e670 0x55a5ece289f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:45.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.062+0000 7fd770a00640 1 -- 172.21.15.123:0/2369007238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5ece28f30 msgr2=0x55a5ece2b320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:47:45.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.062+0000 7fd770a00640 1 --2- 172.21.15.123:0/2369007238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5ece28f30 0x55a5ece2b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.062+0000 7fd770a00640 1 -- 172.21.15.123:0/2369007238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1e670 msgr2=0x55a5ece289f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:45.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.062+0000 7fd770a00640 1 --2- 172.21.15.123:0/2369007238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1e670 0x55a5ece289f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.062+0000 7fd770a00640 1 -- 172.21.15.123:0/2369007238 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ecd4aae0 con 0x55a5ece1dd50 2024-09-19T04:47:45.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.062+0000 7fd76be00640 1 --2- 172.21.15.123:0/2369007238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1e670 0x55a5ece289f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:45.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.062+0000 7fd770a00640 1 --2- 172.21.15.123:0/2369007238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece1e130 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd7640117e0 tx=0x7fd764011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c6cd94f8aab3059 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:45.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.063+0000 7fd768c00640 1 --2- 172.21.15.123:0/855293176 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd7500a35e0 0x7fd7500a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.063+0000 7fd768c00640 1 -- 172.21.15.123:0/855293176 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd7580be970 con 0x7fd76c102360 2024-09-19T04:47:45.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.063+0000 7fd76be00640 1 --2- 172.21.15.123:0/855293176 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd7500a35e0 0x7fd7500a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:45.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.063+0000 7fd761200640 1 -- 172.21.15.123:0/2369007238 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764011310 con 0x55a5ece1dd50 2024-09-19T04:47:45.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.064+0000 7fd761200640 1 -- 172.21.15.123:0/2369007238 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd764003e50 con 0x55a5ece1dd50 2024-09-19T04:47:45.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.064+0000 7fd761200640 1 -- 172.21.15.123:0/2369007238 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764004530 con 0x55a5ece1dd50 2024-09-19T04:47:45.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.064+0000 7fd77564f740 1 -- 172.21.15.123:0/2369007238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 msgr2=0x55a5ece1e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:45.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.064+0000 7fd77564f740 1 --2- 172.21.15.123:0/2369007238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece1e130 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd7640117e0 tx=0x7fd764011ca0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.064+0000 7fd76be00640 1 --2- 172.21.15.123:0/855293176 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd7500a35e0 0x7fd7500a5aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55a5ece1b140 tx=0x7fd75c005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:45.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.065+0000 7fd77564f740 1 -- 172.21.15.123:0/2369007238 shutdown_connections 2024-09-19T04:47:45.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.065+0000 7fd77564f740 1 --2- 172.21.15.123:0/2369007238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5ece28f30 0x55a5ece2b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.065+0000 7fd77564f740 1 --2- 172.21.15.123:0/2369007238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1e670 0x55a5ece289f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.065+0000 7fd77564f740 1 --2- 172.21.15.123:0/2369007238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece1e130 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.065+0000 7fd77564f740 1 -- 172.21.15.123:0/2369007238 >> 172.21.15.123:0/2369007238 conn(0x55a5ece1a2e0 msgr2=0x55a5ece1a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:45.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.065+0000 7fd77564f740 1 -- 172.21.15.123:0/2369007238 shutdown_connections 2024-09-19T04:47:45.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.065+0000 7fd77564f740 1 -- 172.21.15.123:0/2369007238 wait complete. 2024-09-19T04:47:45.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.066+0000 7fd77564f740 1 -- start start 2024-09-19T04:47:45.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.066+0000 7fd77564f740 1 Processor -- start 2024-09-19T04:47:45.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.067+0000 7fd77564f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece25f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.067+0000 7fd77564f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1e670 0x55a5ece26490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.067+0000 7fd76be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece25f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:45.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.067+0000 7fd76b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1e670 0x55a5ece26490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:45.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.067+0000 7fd76be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece25f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49048/0 (socket says 172.21.15.123:49048) 2024-09-19T04:47:45.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.067+0000 7fd76be00640 1 -- 172.21.15.123:0/2148324528 learned_addr learned my addr 172.21.15.123:0/2148324528 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:45.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.068+0000 7fd77564f740 1 --2- 172.21.15.123:0/2148324528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5ece28f30 0x55a5ece269d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.068+0000 7fd77564f740 1 -- 172.21.15.123:0/2148324528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a5ece2bba0 con 0x55a5ece1e670 2024-09-19T04:47:45.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.068+0000 7fd77564f740 1 -- 172.21.15.123:0/2148324528 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a5ecd3f0f0 con 0x55a5ece28f30 2024-09-19T04:47:45.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.068+0000 7fd77564f740 1 -- 172.21.15.123:0/2148324528 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a5ece2bd20 con 0x55a5ece1dd50 2024-09-19T04:47:45.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.068+0000 7fd77564f740 10 client.0 _ll_register_callbacks cb 0x55a5ecd309e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:45.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.068+0000 7fd76be00640 1 -- 172.21.15.123:0/2148324528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5ece28f30 msgr2=0x55a5ece269d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:47:45.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.068+0000 7fd76be00640 1 --2- 172.21.15.123:0/2148324528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5ece28f30 0x55a5ece269d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.068+0000 7fd76be00640 1 -- 172.21.15.123:0/2148324528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1e670 msgr2=0x55a5ece26490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:45.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.069+0000 7fd76be00640 1 --2- 172.21.15.123:0/2148324528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1e670 0x55a5ece26490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.069+0000 7fd76be00640 1 -- 172.21.15.123:0/2148324528 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5ecaf53c0 con 0x55a5ece1dd50 2024-09-19T04:47:45.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.069+0000 7fd76b400640 1 --2- 172.21.15.123:0/2148324528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1e670 0x55a5ece26490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:47:45.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.069+0000 7fd76be00640 1 --2- 172.21.15.123:0/2148324528 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece25f50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009fd0 tx=0x7fd75c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff5f7226a904cc5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:45.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.069+0000 7fd755600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:47:45.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.069+0000 7fd755600640 1 -- 172.21.15.123:0/2148324528 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c01e920 con 0x55a5ece1dd50 2024-09-19T04:47:45.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.070+0000 7fd755600640 1 -- 172.21.15.123:0/2148324528 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd75c01eac0 con 0x55a5ece1dd50 2024-09-19T04:47:45.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.070+0000 7fd77564f740 1 -- 172.21.15.123:0/2148324528 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd764003e50 con 0x55a5ece1dd50 2024-09-19T04:47:45.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.070+0000 7fd755600640 1 -- 172.21.15.123:0/2148324528 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd75c01ecc0 con 0x55a5ece1dd50 2024-09-19T04:47:45.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.070+0000 7fd77564f740 10 client.25483 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:45.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.070+0000 7fd77564f740 1 -- 172.21.15.123:0/2148324528 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5ece379d0 con 0x55a5ece1dd50 2024-09-19T04:47:45.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.070+0000 7fd755600640 1 -- 172.21.15.123:0/2148324528 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd764003e50 con 0x55a5ece1dd50 2024-09-19T04:47:45.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.071+0000 7fd755600640 1 -- 172.21.15.123:0/2148324528 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd75c034a30 con 0x55a5ece1dd50 2024-09-19T04:47:45.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.071+0000 7fd77564f740 10 client.25483 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:45.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.071+0000 7fd77564f740 10 client.25483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:47:45.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.071+0000 7fd77564f740 20 client.25483 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:45.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.071+0000 7fd77564f740 1 --2- 172.21.15.123:0/2148324528 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a5ece38de0 0x55a5ece3b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.071+0000 7fd77564f740 4 client.25483 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:47:45.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.071+0000 7fd77564f740 1 -- 172.21.15.123:0/2148324528 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5ecd91950 con 0x55a5ece38de0 2024-09-19T04:47:45.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.071+0000 7fd76b400640 1 --2- 172.21.15.123:0/2148324528 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a5ece38de0 0x55a5ece3b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:45.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.072+0000 7fd76b400640 1 --2- 172.21.15.123:0/2148324528 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a5ece38de0 0x55a5ece3b250 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fd98ecf28f7e937 server_cookie=f2c1e9c8e6b33a3f in_seq=0 out_seq=0 2024-09-19T04:47:45.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.072+0000 7fd755600640 10 client.25483 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:45.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.073+0000 7fd755600640 1 -- 172.21.15.123:0/2148324528 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5ecd91950 con 0x55a5ece38de0 2024-09-19T04:47:45.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.073+0000 7fd755600640 10 client.25483 handle_command_reply: tid=0 2024-09-19T04:47:45.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.073+0000 7fd77564f740 1 client.25483 shutdown 2024-09-19T04:47:45.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.073+0000 7fd77564f740 1 -- 172.21.15.123:0/2148324528 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 msgr2=0x55a5ece25f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:45.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.073+0000 7fd77564f740 1 --2- 172.21.15.123:0/2148324528 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece25f50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009fd0 tx=0x7fd75c004730 comp rx=0 tx=0).stop 2024-09-19T04:47:45.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.073+0000 7fd77564f740 1 -- 172.21.15.123:0/2148324528 shutdown_connections 2024-09-19T04:47:45.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.073+0000 7fd77564f740 1 --2- 172.21.15.123:0/2148324528 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a5ece38de0 0x55a5ece3b250 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.073+0000 7fd77564f740 1 --2- 172.21.15.123:0/2148324528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5ece28f30 0x55a5ece269d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.073+0000 7fd77564f740 1 --2- 172.21.15.123:0/2148324528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1e670 0x55a5ece26490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.073+0000 7fd77564f740 1 --2- 172.21.15.123:0/2148324528 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece25f50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.073+0000 7fd755600640 0 client.25483 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:45.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.073+0000 7fd77564f740 1 -- 172.21.15.123:0/2148324528 >> 172.21.15.123:0/2148324528 conn(0x55a5ece1a2e0 msgr2=0x55a5ece20010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:45.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.074+0000 7fd77564f740 1 -- 172.21.15.123:0/2148324528 shutdown_connections 2024-09-19T04:47:45.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.074+0000 7fd77564f740 1 -- 172.21.15.123:0/2148324528 wait complete. 2024-09-19T04:47:45.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.074+0000 7fd77564f740 20 client.25483 trim_cache size 0 max 16384 2024-09-19T04:47:45.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.108+0000 7fd77564f740 1 Processor -- start 2024-09-19T04:47:45.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.108+0000 7fd77564f740 1 -- start start 2024-09-19T04:47:45.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.109+0000 7fd77564f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1dd50 0x55a5ece27ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.109+0000 7fd77564f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1e670 0x55a5ece40b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.109+0000 7fd77564f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5ece28f30 0x55a5ece41060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.109+0000 7fd77564f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1ab450 con 0x55a5ece1dd50 2024-09-19T04:47:45.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.109+0000 7fd77564f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1acf90 con 0x55a5ece28f30 2024-09-19T04:47:45.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.109+0000 7fd77564f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1abb20 con 0x55a5ece1e670 2024-09-19T04:47:45.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.109+0000 7fd76be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1dd50 0x55a5ece27ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:45.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.109+0000 7fd76b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1e670 0x55a5ece40b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:45.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.109+0000 7fd76be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1dd50 0x55a5ece27ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41294/0 (socket says 172.21.15.123:41294) 2024-09-19T04:47:45.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.109+0000 7fd76b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1e670 0x55a5ece40b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49050/0 (socket says 172.21.15.123:49050) 2024-09-19T04:47:45.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.109+0000 7fd76be00640 1 -- 172.21.15.123:0/1819507370 learned_addr learned my addr 172.21.15.123:0/1819507370 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:45.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.110+0000 7fd770a00640 1 --2- 172.21.15.123:0/1819507370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5ece28f30 0x55a5ece41060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:45.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.110+0000 7fd76b400640 1 -- 172.21.15.123:0/1819507370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5ece28f30 msgr2=0x55a5ece41060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:45.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.110+0000 7fd76b400640 1 --2- 172.21.15.123:0/1819507370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5ece28f30 0x55a5ece41060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.110+0000 7fd76b400640 1 -- 172.21.15.123:0/1819507370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1dd50 msgr2=0x55a5ece27ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:45.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.110+0000 7fd76b400640 1 --2- 172.21.15.123:0/1819507370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1dd50 0x55a5ece27ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.110+0000 7fd76b400640 1 -- 172.21.15.123:0/1819507370 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ece288c0 con 0x55a5ece1e670 2024-09-19T04:47:45.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.110+0000 7fd76b400640 1 --2- 172.21.15.123:0/1819507370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1e670 0x55a5ece40b20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd7580f1d40 tx=0x7fd7580c5da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24a4c7af69d0ee78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:45.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.110+0000 7fd757e00640 1 -- 172.21.15.123:0/1819507370 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7580bd300 con 0x55a5ece1e670 2024-09-19T04:47:45.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.111+0000 7fd757e00640 1 -- 172.21.15.123:0/1819507370 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7580c0fd0 con 0x55a5ece1e670 2024-09-19T04:47:45.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.111+0000 7fd757e00640 1 -- 172.21.15.123:0/1819507370 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7580bd4a0 con 0x55a5ece1e670 2024-09-19T04:47:45.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.111+0000 7fd77564f740 1 -- 172.21.15.123:0/1819507370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1e670 msgr2=0x55a5ece40b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:45.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.111+0000 7fd77564f740 1 --2- 172.21.15.123:0/1819507370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1e670 0x55a5ece40b20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd7580f1d40 tx=0x7fd7580c5da0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.111+0000 7fd77564f740 1 -- 172.21.15.123:0/1819507370 shutdown_connections 2024-09-19T04:47:45.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.111+0000 7fd77564f740 1 --2- 172.21.15.123:0/1819507370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5ece28f30 0x55a5ece41060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.111+0000 7fd77564f740 1 --2- 172.21.15.123:0/1819507370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1e670 0x55a5ece40b20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.111+0000 7fd77564f740 1 --2- 172.21.15.123:0/1819507370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1dd50 0x55a5ece27ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.111+0000 7fd77564f740 1 -- 172.21.15.123:0/1819507370 >> 172.21.15.123:0/1819507370 conn(0x55a5ece1a2e0 msgr2=0x55a5ece277b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:45.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.112+0000 7fd77564f740 1 -- 172.21.15.123:0/1819507370 shutdown_connections 2024-09-19T04:47:45.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.112+0000 7fd77564f740 1 -- 172.21.15.123:0/1819507370 wait complete. 2024-09-19T04:47:45.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.112+0000 7fd77564f740 1 -- start start 2024-09-19T04:47:45.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.113+0000 7fd77564f740 1 Processor -- start 2024-09-19T04:47:45.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.113+0000 7fd77564f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece2cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.113+0000 7fd77564f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1e670 0x55a5ece2d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.113+0000 7fd77564f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5ece28f30 0x55a5ece2d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.113+0000 7fd77564f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1ac8c0 con 0x55a5ece1e670 2024-09-19T04:47:45.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.113+0000 7fd77564f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1ac1f0 con 0x55a5ece28f30 2024-09-19T04:47:45.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.113+0000 7fd77564f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1a9430 con 0x55a5ece1dd50 2024-09-19T04:47:45.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.113+0000 7fd77564f740 10 client.0 _ll_register_callbacks cb 0x7fd75c00fa60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:45.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.114+0000 7fd76be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece2cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:45.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.114+0000 7fd76be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece2cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49064/0 (socket says 172.21.15.123:49064) 2024-09-19T04:47:45.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.114+0000 7fd76be00640 1 -- 172.21.15.123:0/2600619250 learned_addr learned my addr 172.21.15.123:0/2600619250 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:45.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.114+0000 7fd76b400640 1 --2- 172.21.15.123:0/2600619250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1e670 0x55a5ece2d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:45.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.114+0000 7fd76be00640 1 -- 172.21.15.123:0/2600619250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5ece28f30 msgr2=0x55a5ece2d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:45.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.114+0000 7fd76be00640 1 --2- 172.21.15.123:0/2600619250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5ece28f30 0x55a5ece2d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.114+0000 7fd76be00640 1 -- 172.21.15.123:0/2600619250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1e670 msgr2=0x55a5ece2d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:45.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.115+0000 7fd76be00640 1 --2- 172.21.15.123:0/2600619250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1e670 0x55a5ece2d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.115+0000 7fd76be00640 1 -- 172.21.15.123:0/2600619250 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5ecea2af0 con 0x55a5ece1dd50 2024-09-19T04:47:45.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.115+0000 7fd76be00640 1 --2- 172.21.15.123:0/2600619250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece2cc10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd75c034e80 tx=0x7fd75c01ff60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42b8d4f7de698114 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:45.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.115+0000 7fd755600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:47:45.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.115+0000 7fd755600640 1 -- 172.21.15.123:0/2600619250 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c012030 con 0x55a5ece1dd50 2024-09-19T04:47:45.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.115+0000 7fd755600640 1 -- 172.21.15.123:0/2600619250 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd75c0121d0 con 0x55a5ece1dd50 2024-09-19T04:47:45.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.116+0000 7fd77564f740 1 -- 172.21.15.123:0/2600619250 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7580c0fd0 con 0x55a5ece1dd50 2024-09-19T04:47:45.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.116+0000 7fd755600640 1 -- 172.21.15.123:0/2600619250 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd75c03d8f0 con 0x55a5ece1dd50 2024-09-19T04:47:45.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.116+0000 7fd77564f740 10 client.25489 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:45.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.116+0000 7fd77564f740 1 -- 172.21.15.123:0/2600619250 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5ecea3d40 con 0x55a5ece1dd50 2024-09-19T04:47:45.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.116+0000 7fd755600640 1 -- 172.21.15.123:0/2600619250 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7580c0fd0 con 0x55a5ece1dd50 2024-09-19T04:47:45.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.117+0000 7fd755600640 1 -- 172.21.15.123:0/2600619250 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd75c029cb0 con 0x55a5ece1dd50 2024-09-19T04:47:45.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.117+0000 7fd77564f740 10 client.25489 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:45.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.117+0000 7fd77564f740 10 client.25489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:47:45.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.117+0000 7fd77564f740 20 client.25489 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:45.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.117+0000 7fd77564f740 1 --2- 172.21.15.123:0/2600619250 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a5ecea4430 0x55a5ecea6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:45.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.117+0000 7fd77564f740 4 client.25489 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:47:45.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.117+0000 7fd77564f740 1 -- 172.21.15.123:0/2600619250 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a5ecd8be60 con 0x55a5ecea4430 2024-09-19T04:47:45.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.118+0000 7fd76b400640 1 --2- 172.21.15.123:0/2600619250 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a5ecea4430 0x55a5ecea6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:45.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.118+0000 7fd76b400640 1 --2- 172.21.15.123:0/2600619250 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a5ecea4430 0x55a5ecea6890 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2057af0437e847a1 server_cookie=d981366c7e36b075 in_seq=0 out_seq=0 2024-09-19T04:47:45.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.118+0000 7fd755600640 10 client.25489 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:45.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.119+0000 7fd755600640 1 -- 172.21.15.123:0/2600619250 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a5ecd8be60 con 0x55a5ecea4430 2024-09-19T04:47:45.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.119+0000 7fd755600640 10 client.25489 handle_command_reply: tid=0 2024-09-19T04:47:45.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.119+0000 7fd77564f740 1 client.25489 shutdown 2024-09-19T04:47:45.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.119+0000 7fd77564f740 1 -- 172.21.15.123:0/2600619250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 msgr2=0x55a5ece2cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:45.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.119+0000 7fd77564f740 1 --2- 172.21.15.123:0/2600619250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece2cc10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd75c034e80 tx=0x7fd75c01ff60 comp rx=0 tx=0).stop 2024-09-19T04:47:45.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.119+0000 7fd77564f740 1 -- 172.21.15.123:0/2600619250 shutdown_connections 2024-09-19T04:47:45.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.119+0000 7fd77564f740 1 --2- 172.21.15.123:0/2600619250 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a5ecea4430 0x55a5ecea6890 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.119+0000 7fd77564f740 1 --2- 172.21.15.123:0/2600619250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5ece28f30 0x55a5ece2d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.119+0000 7fd77564f740 1 --2- 172.21.15.123:0/2600619250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5ece1e670 0x55a5ece2d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.119+0000 7fd77564f740 1 --2- 172.21.15.123:0/2600619250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5ece1dd50 0x55a5ece2cc10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.119+0000 7fd77564f740 1 -- 172.21.15.123:0/2600619250 >> 172.21.15.123:0/2600619250 conn(0x55a5ece1a2e0 msgr2=0x55a5ece20010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:45.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.119+0000 7fd755600640 0 client.25489 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:45.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.119+0000 7fd77564f740 1 -- 172.21.15.123:0/2600619250 shutdown_connections 2024-09-19T04:47:45.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.119+0000 7fd77564f740 1 -- 172.21.15.123:0/2600619250 wait complete. 2024-09-19T04:47:45.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.120+0000 7fd77564f740 20 client.25489 trim_cache size 0 max 16384 2024-09-19T04:47:45.122 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:47:45.123 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-09-19T04:47:45.123 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-09-19T04:47:45.123 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:47:45.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.120+0000 7fd771400640 1 -- 172.21.15.123:0/855293176 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd7500a35e0 msgr2=0x7fd7500a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:45.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.120+0000 7fd771400640 1 --2- 172.21.15.123:0/855293176 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd7500a35e0 0x7fd7500a5aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55a5ece1b140 tx=0x7fd75c005f90 comp rx=0 tx=0).stop 2024-09-19T04:47:45.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.120+0000 7fd771400640 1 -- 172.21.15.123:0/855293176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd76c102360 msgr2=0x7fd76c1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:45.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.120+0000 7fd771400640 1 --2- 172.21.15.123:0/855293176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd76c102360 0x7fd76c1987f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd75802a940 tx=0x7fd758005d50 comp rx=0 tx=0).stop 2024-09-19T04:47:45.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.121+0000 7fd771400640 1 -- 172.21.15.123:0/855293176 shutdown_connections 2024-09-19T04:47:45.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.121+0000 7fd771400640 1 --2- 172.21.15.123:0/855293176 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd7500a35e0 0x7fd7500a5aa0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.121+0000 7fd771400640 1 --2- 172.21.15.123:0/855293176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd76c1079c0 0x7fd76c19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.121+0000 7fd771400640 1 --2- 172.21.15.123:0/855293176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd76c102360 0x7fd76c1987f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.121+0000 7fd771400640 1 --2- 172.21.15.123:0/855293176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd76c1019c0 0x7fd76c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:45.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.121+0000 7fd771400640 1 -- 172.21.15.123:0/855293176 >> 172.21.15.123:0/855293176 conn(0x7fd76c0fd740 msgr2=0x7fd76c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:45.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.122+0000 7fd771400640 1 -- 172.21.15.123:0/855293176 shutdown_connections 2024-09-19T04:47:45.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:45.122+0000 7fd771400640 1 -- 172.21.15.123:0/855293176 wait complete. 2024-09-19T04:47:45.609 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:45 smithi123 ceph-mon[30511]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T04:47:45.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:45 smithi193 ceph-mon[35592]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T04:47:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:45 smithi183 ceph-mon[35333]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T04:47:45.929 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:47:45.930 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:47:46.356 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:47:47.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.013+0000 7ff483e00640 1 -- 172.21.15.123:0/2638552482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4841047e0 msgr2=0x7ff484108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:47.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.013+0000 7ff483e00640 1 --2- 172.21.15.123:0/2638552482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4841047e0 0x7ff484108830 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff470009900 tx=0x7ff47002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:47:47.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.013+0000 7ff483e00640 1 -- 172.21.15.123:0/2638552482 shutdown_connections 2024-09-19T04:47:47.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.013+0000 7ff483e00640 1 --2- 172.21.15.123:0/2638552482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff484108f60 0x7ff484110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:47.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.013+0000 7ff483e00640 1 --2- 172.21.15.123:0/2638552482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4841047e0 0x7ff484108830 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:47.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.013+0000 7ff483e00640 1 --2- 172.21.15.123:0/2638552482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff484103e30 0x7ff484104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:47.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.013+0000 7ff483e00640 1 -- 172.21.15.123:0/2638552482 >> 172.21.15.123:0/2638552482 conn(0x7ff4840ff910 msgr2=0x7ff484101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:47.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.013+0000 7ff483e00640 1 -- 172.21.15.123:0/2638552482 shutdown_connections 2024-09-19T04:47:47.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.014+0000 7ff483e00640 1 -- 172.21.15.123:0/2638552482 wait complete. 2024-09-19T04:47:47.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.015+0000 7ff483e00640 1 Processor -- start 2024-09-19T04:47:47.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.015+0000 7ff483e00640 1 -- start start 2024-09-19T04:47:47.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.016+0000 7ff483e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff484103e30 0x7ff48419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:47.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.016+0000 7ff483e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4841047e0 0x7ff48419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:47.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.016+0000 7ff483e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff484108f60 0x7ff48419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:47.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.016+0000 7ff483e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff484114010 con 0x7ff484103e30 2024-09-19T04:47:47.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.016+0000 7ff483e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff484113e90 con 0x7ff4841047e0 2024-09-19T04:47:47.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.016+0000 7ff483e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff484114190 con 0x7ff484108f60 2024-09-19T04:47:47.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.016+0000 7ff482000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4841047e0 0x7ff48419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:47.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.016+0000 7ff482000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4841047e0 0x7ff48419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:56418/0 (socket says 172.21.15.123:56418) 2024-09-19T04:47:47.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.016+0000 7ff482000640 1 -- 172.21.15.123:0/1624534191 learned_addr learned my addr 172.21.15.123:0/1624534191 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:47.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.016+0000 7ff482a00640 1 --2- 172.21.15.123:0/1624534191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff484103e30 0x7ff48419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:47.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.016+0000 7ff483400640 1 --2- 172.21.15.123:0/1624534191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff484108f60 0x7ff48419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:47.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.017+0000 7ff482000640 1 -- 172.21.15.123:0/1624534191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff484108f60 msgr2=0x7ff48419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.017+0000 7ff482000640 1 --2- 172.21.15.123:0/1624534191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff484108f60 0x7ff48419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.017+0000 7ff482000640 1 -- 172.21.15.123:0/1624534191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff484103e30 msgr2=0x7ff48419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.017+0000 7ff482000640 1 --2- 172.21.15.123:0/1624534191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff484103e30 0x7ff48419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.017+0000 7ff482000640 1 -- 172.21.15.123:0/1624534191 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48419f500 con 0x7ff4841047e0 2024-09-19T04:47:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.017+0000 7ff482000640 1 --2- 172.21.15.123:0/1624534191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4841047e0 0x7ff48419aa90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff470004770 tx=0x7ff470003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.018+0000 7ff47b400640 1 -- 172.21.15.123:0/1624534191 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47003f070 con 0x7ff4841047e0 2024-09-19T04:47:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.018+0000 7ff483e00640 1 -- 172.21.15.123:0/1624534191 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48419f790 con 0x7ff4841047e0 2024-09-19T04:47:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.018+0000 7ff47b400640 1 -- 172.21.15.123:0/1624534191 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff470031ca0 con 0x7ff4841047e0 2024-09-19T04:47:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.018+0000 7ff47b400640 1 -- 172.21.15.123:0/1624534191 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff470035030 con 0x7ff4841047e0 2024-09-19T04:47:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.018+0000 7ff483e00640 1 -- 172.21.15.123:0/1624534191 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4841a70d0 con 0x7ff4841047e0 2024-09-19T04:47:47.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.019+0000 7ff483e00640 1 -- 172.21.15.123:0/1624534191 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff484105310 con 0x7ff4841047e0 2024-09-19T04:47:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.022+0000 7ff47b400640 1 -- 172.21.15.123:0/1624534191 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7ff470042070 con 0x7ff4841047e0 2024-09-19T04:47:47.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.024+0000 7ff47b400640 1 --2- 172.21.15.123:0/1624534191 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff4680a35e0 0x7ff4680a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:47.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.024+0000 7ff47b400640 1 -- 172.21.15.123:0/1624534191 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7ff4700f0320 con 0x7ff4841047e0 2024-09-19T04:47:47.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.024+0000 7ff482a00640 1 --2- 172.21.15.123:0/1624534191 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff4680a35e0 0x7ff4680a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:47.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.025+0000 7ff47b400640 1 -- 172.21.15.123:0/1624534191 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff4700bc560 con 0x7ff4841047e0 2024-09-19T04:47:47.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.025+0000 7ff482a00640 1 --2- 172.21.15.123:0/1624534191 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff4680a35e0 0x7ff4680a5aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff474009bd0 tx=0x7ff474005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:47.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.265+0000 7ff483e00640 1 -- 172.21.15.123:0/1624534191 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff484104210 con 0x7ff4841047e0 2024-09-19T04:47:47.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.266+0000 7ff47b400640 1 -- 172.21.15.123:0/1624534191 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11990 (secure 0 0 0) 0x7ff484104210 con 0x7ff4841047e0 2024-09-19T04:47:47.268 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:47:47.268 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"btime":"2024-09-19T04:46:42:466383+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:46:41.577049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24499,24500,24539,24505,24529]},"id":1}]} 2024-09-19T04:47:47.269 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 28 2024-09-19T04:47:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.270+0000 7ff483e00640 1 -- 172.21.15.123:0/1624534191 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff4680a35e0 msgr2=0x7ff4680a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.270+0000 7ff483e00640 1 --2- 172.21.15.123:0/1624534191 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff4680a35e0 0x7ff4680a5aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff474009bd0 tx=0x7ff474005f90 comp rx=0 tx=0).stop 2024-09-19T04:47:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.270+0000 7ff483e00640 1 -- 172.21.15.123:0/1624534191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4841047e0 msgr2=0x7ff48419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.270+0000 7ff483e00640 1 --2- 172.21.15.123:0/1624534191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4841047e0 0x7ff48419aa90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff470004770 tx=0x7ff470003c10 comp rx=0 tx=0).stop 2024-09-19T04:47:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.270+0000 7ff483e00640 1 -- 172.21.15.123:0/1624534191 shutdown_connections 2024-09-19T04:47:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.270+0000 7ff483e00640 1 --2- 172.21.15.123:0/1624534191 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff4680a35e0 0x7ff4680a5aa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.270+0000 7ff483e00640 1 --2- 172.21.15.123:0/1624534191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff484108f60 0x7ff48419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.270+0000 7ff483e00640 1 --2- 172.21.15.123:0/1624534191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4841047e0 0x7ff48419aa90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.270+0000 7ff483e00640 1 --2- 172.21.15.123:0/1624534191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff484103e30 0x7ff48419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.270+0000 7ff483e00640 1 -- 172.21.15.123:0/1624534191 >> 172.21.15.123:0/1624534191 conn(0x7ff4840ff910 msgr2=0x7ff484100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.270+0000 7ff483e00640 1 -- 172.21.15.123:0/1624534191 shutdown_connections 2024-09-19T04:47:47.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:47.270+0000 7ff483e00640 1 -- 172.21.15.123:0/1624534191 wait complete. 2024-09-19T04:47:47.675 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:47 smithi123 ceph-mon[30511]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:47:47.675 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:47 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1624534191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:47:47.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:47 smithi193 ceph-mon[35592]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:47:47.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:47 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1624534191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:47:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:47 smithi183 ceph-mon[35333]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:47:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:47 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1624534191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:47:48.063 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:47:48.486 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:47:49.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.124+0000 7f14f7e00640 1 -- 172.21.15.123:0/1243724224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14f81047e0 msgr2=0x7f14f8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:49.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.124+0000 7f14f7e00640 1 --2- 172.21.15.123:0/1243724224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14f81047e0 0x7f14f8108830 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009900 tx=0x7f14e402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.124+0000 7f14f7e00640 1 -- 172.21.15.123:0/1243724224 shutdown_connections 2024-09-19T04:47:49.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.124+0000 7f14f7e00640 1 --2- 172.21.15.123:0/1243724224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14f8108f60 0x7f14f8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.124+0000 7f14f7e00640 1 --2- 172.21.15.123:0/1243724224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14f81047e0 0x7f14f8108830 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.124+0000 7f14f7e00640 1 --2- 172.21.15.123:0/1243724224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14f8103e30 0x7f14f8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.124+0000 7f14f7e00640 1 -- 172.21.15.123:0/1243724224 >> 172.21.15.123:0/1243724224 conn(0x7f14f80ff910 msgr2=0x7f14f8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:49.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.125+0000 7f14f7e00640 1 -- 172.21.15.123:0/1243724224 shutdown_connections 2024-09-19T04:47:49.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.125+0000 7f14f7e00640 1 -- 172.21.15.123:0/1243724224 wait complete. 2024-09-19T04:47:49.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.126+0000 7f14f7e00640 1 Processor -- start 2024-09-19T04:47:49.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.127+0000 7f14f7e00640 1 -- start start 2024-09-19T04:47:49.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.127+0000 7f14f7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14f8103e30 0x7f14f819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.127+0000 7f14f7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14f81047e0 0x7f14f819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.127+0000 7f14f7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14f8108f60 0x7f14f819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.127+0000 7f14f7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14f8114010 con 0x7f14f8103e30 2024-09-19T04:47:49.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.127+0000 7f14f7e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f14f8113e90 con 0x7f14f81047e0 2024-09-19T04:47:49.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.127+0000 7f14f7e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f14f8114190 con 0x7f14f8108f60 2024-09-19T04:47:49.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.128+0000 7f14f7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14f8108f60 0x7f14f819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.128+0000 7f14f7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14f8108f60 0x7f14f819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49118/0 (socket says 172.21.15.123:49118) 2024-09-19T04:47:49.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.128+0000 7f14f7400640 1 -- 172.21.15.123:0/3046878076 learned_addr learned my addr 172.21.15.123:0/3046878076 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:49.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.128+0000 7f14f6a00640 1 --2- 172.21.15.123:0/3046878076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14f8103e30 0x7f14f819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.128+0000 7f14f6000640 1 --2- 172.21.15.123:0/3046878076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14f81047e0 0x7f14f819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.128+0000 7f14f6a00640 1 -- 172.21.15.123:0/3046878076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14f8108f60 msgr2=0x7f14f819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:49.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.128+0000 7f14f6a00640 1 --2- 172.21.15.123:0/3046878076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14f8108f60 0x7f14f819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.128+0000 7f14f6a00640 1 -- 172.21.15.123:0/3046878076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14f81047e0 msgr2=0x7f14f819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:49.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.128+0000 7f14f6a00640 1 --2- 172.21.15.123:0/3046878076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14f81047e0 0x7f14f819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.128+0000 7f14f6a00640 1 -- 172.21.15.123:0/3046878076 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f819f460 con 0x7f14f8103e30 2024-09-19T04:47:49.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.129+0000 7f14f6a00640 1 --2- 172.21.15.123:0/3046878076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14f8103e30 0x7f14f819a4b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f14e800e9f0 tx=0x7f14e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:49.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.129+0000 7f14ef400640 1 -- 172.21.15.123:0/3046878076 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e800ccc0 con 0x7f14f8103e30 2024-09-19T04:47:49.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.130+0000 7f14f7e00640 1 -- 172.21.15.123:0/3046878076 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f819f750 con 0x7f14f8103e30 2024-09-19T04:47:49.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.130+0000 7f14ef400640 1 -- 172.21.15.123:0/3046878076 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e8005020 con 0x7f14f8103e30 2024-09-19T04:47:49.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.130+0000 7f14ef400640 1 -- 172.21.15.123:0/3046878076 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8004b20 con 0x7f14f8103e30 2024-09-19T04:47:49.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.130+0000 7f14f7e00640 1 -- 172.21.15.123:0/3046878076 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f81a7030 con 0x7f14f8103e30 2024-09-19T04:47:49.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.135+0000 7f15007d1740 1 Processor -- start 2024-09-19T04:47:49.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.135+0000 7f15007d1740 1 -- start start 2024-09-19T04:47:49.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.136+0000 7f15007d1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd1178d60 0x556fd1179140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.136+0000 7f15007d1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd1179680 0x556fd118c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.136+0000 7f15007d1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd118c830 0x556fd118ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.136+0000 7f15007d1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556fd10df3e0 con 0x556fd1179680 2024-09-19T04:47:49.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.136+0000 7f15007d1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556fd1086d80 con 0x556fd118c830 2024-09-19T04:47:49.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.136+0000 7f15007d1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556fd10db3a0 con 0x556fd1178d60 2024-09-19T04:47:49.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.136+0000 7f14ef400640 1 -- 172.21.15.123:0/3046878076 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7f14e8004cc0 con 0x7f14f8103e30 2024-09-19T04:47:49.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.136+0000 7f14f6000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd118c830 0x556fd118ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.136+0000 7f14f6000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd118c830 0x556fd118ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:56460/0 (socket says 172.21.15.123:56460) 2024-09-19T04:47:49.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.136+0000 7f14f6000640 1 -- 172.21.15.123:0/226524991 learned_addr learned my addr 172.21.15.123:0/226524991 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:49.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.137+0000 7f14f7400640 1 --2- 172.21.15.123:0/226524991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd1178d60 0x556fd1179140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.137+0000 7f14f6000640 1 -- 172.21.15.123:0/226524991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd1178d60 msgr2=0x556fd1179140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:49.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.137+0000 7f14f6a00640 1 --2- 172.21.15.123:0/226524991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd1179680 0x556fd118c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.137+0000 7f14f6000640 1 --2- 172.21.15.123:0/226524991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd1178d60 0x556fd1179140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.137+0000 7f14f6000640 1 -- 172.21.15.123:0/226524991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd1179680 msgr2=0x556fd118c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:49.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.137+0000 7f14f6000640 1 --2- 172.21.15.123:0/226524991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd1179680 0x556fd118c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.137+0000 7f14f6000640 1 -- 172.21.15.123:0/226524991 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fd10a5ae0 con 0x556fd118c830 2024-09-19T04:47:49.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.137+0000 7f14f6a00640 1 --2- 172.21.15.123:0/226524991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd1179680 0x556fd118c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:47:49.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.138+0000 7f14f6000640 1 --2- 172.21.15.123:0/226524991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd118c830 0x556fd118ec20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009900 tx=0x7f14e4005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=650e39a97da01766 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:49.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.138+0000 7f14e3e00640 1 -- 172.21.15.123:0/226524991 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4035030 con 0x556fd118c830 2024-09-19T04:47:49.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.138+0000 7f14e3e00640 1 -- 172.21.15.123:0/226524991 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e40043c0 con 0x556fd118c830 2024-09-19T04:47:49.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.138+0000 7f14e3e00640 1 -- 172.21.15.123:0/226524991 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e40035f0 con 0x556fd118c830 2024-09-19T04:47:49.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.139+0000 7f14ef400640 1 --2- 172.21.15.123:0/3046878076 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f14dc0a35e0 0x7f14dc0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.139+0000 7f14ef400640 1 -- 172.21.15.123:0/3046878076 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f14e80cff60 con 0x7f14f8103e30 2024-09-19T04:47:49.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.139+0000 7f15007d1740 1 -- 172.21.15.123:0/226524991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd118c830 msgr2=0x556fd118ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:49.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.139+0000 7f15007d1740 1 --2- 172.21.15.123:0/226524991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd118c830 0x556fd118ec20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009900 tx=0x7f14e4005e60 comp rx=0 tx=0).stop 2024-09-19T04:47:49.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.140+0000 7f14f7400640 1 --2- 172.21.15.123:0/3046878076 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f14dc0a35e0 0x7f14dc0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.141+0000 7f15007d1740 1 -- 172.21.15.123:0/226524991 shutdown_connections 2024-09-19T04:47:49.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.141+0000 7f15007d1740 1 --2- 172.21.15.123:0/226524991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd118c830 0x556fd118ec20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.141+0000 7f15007d1740 1 --2- 172.21.15.123:0/226524991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd1179680 0x556fd118c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.141+0000 7f15007d1740 1 --2- 172.21.15.123:0/226524991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd1178d60 0x556fd1179140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.141+0000 7f15007d1740 1 -- 172.21.15.123:0/226524991 >> 172.21.15.123:0/226524991 conn(0x556fd11753d0 msgr2=0x556fd11757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:49.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.141+0000 7f14f7400640 1 --2- 172.21.15.123:0/3046878076 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f14dc0a35e0 0x7f14dc0a5aa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f14f00049d0 tx=0x7f14f0009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:49.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.141+0000 7f15007d1740 1 -- 172.21.15.123:0/226524991 shutdown_connections 2024-09-19T04:47:49.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.141+0000 7f15007d1740 1 -- 172.21.15.123:0/226524991 wait complete. 2024-09-19T04:47:49.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.142+0000 7f15007d1740 1 -- start start 2024-09-19T04:47:49.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.142+0000 7f15007d1740 1 Processor -- start 2024-09-19T04:47:49.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.143+0000 7f15007d1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd1178d60 0x556fd11897f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.143+0000 7f15007d1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd1179680 0x556fd1189d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.143+0000 7f15007d1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd118c830 0x556fd118a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.143+0000 7f15007d1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556fd118f340 con 0x556fd118c830 2024-09-19T04:47:49.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.143+0000 7f15007d1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556fd109a0f0 con 0x556fd1178d60 2024-09-19T04:47:49.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.143+0000 7f15007d1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556fd118f4c0 con 0x556fd1179680 2024-09-19T04:47:49.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.143+0000 7f15007d1740 10 client.0 _ll_register_callbacks cb 0x556fd1081bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:49.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.143+0000 7f14f6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd1178d60 0x556fd11897f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.143+0000 7f14f6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd1178d60 0x556fd11897f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:56466/0 (socket says 172.21.15.123:56466) 2024-09-19T04:47:49.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.143+0000 7f14f6a00640 1 -- 172.21.15.123:0/496729068 learned_addr learned my addr 172.21.15.123:0/496729068 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:49.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.144+0000 7f14f6000640 1 --2- 172.21.15.123:0/496729068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd1179680 0x556fd1189d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.144+0000 7f14f7400640 1 --2- 172.21.15.123:0/496729068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd118c830 0x556fd118a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.144+0000 7f14f6a00640 1 -- 172.21.15.123:0/496729068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd1179680 msgr2=0x556fd1189d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:49.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.144+0000 7f14f6a00640 1 --2- 172.21.15.123:0/496729068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd1179680 0x556fd1189d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.144+0000 7f14f6a00640 1 -- 172.21.15.123:0/496729068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd118c830 msgr2=0x556fd118a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:49.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.144+0000 7f14f6a00640 1 --2- 172.21.15.123:0/496729068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd118c830 0x556fd118a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.144+0000 7f14f6a00640 1 -- 172.21.15.123:0/496729068 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fd0e503c0 con 0x556fd1178d60 2024-09-19T04:47:49.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.144+0000 7f14f6a00640 1 --2- 172.21.15.123:0/496729068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd1178d60 0x556fd11897f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f14e800e9c0 tx=0x7f14e809c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1f953cf87f932d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:49.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.144+0000 7f14e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:47:49.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.145+0000 7f14e0c00640 1 -- 172.21.15.123:0/496729068 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e801c040 con 0x556fd1178d60 2024-09-19T04:47:49.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.145+0000 7f14e0c00640 1 -- 172.21.15.123:0/496729068 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f14e80a00d0 con 0x556fd1178d60 2024-09-19T04:47:49.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.145+0000 7f15007d1740 1 -- 172.21.15.123:0/496729068 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e40043c0 con 0x556fd1178d60 2024-09-19T04:47:49.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.145+0000 7f14e0c00640 1 -- 172.21.15.123:0/496729068 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14e80a4ca0 con 0x556fd1178d60 2024-09-19T04:47:49.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.145+0000 7f15007d1740 10 client.25487 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:49.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.145+0000 7f15007d1740 1 -- 172.21.15.123:0/496729068 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556fd119b440 con 0x556fd1178d60 2024-09-19T04:47:49.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.145+0000 7f14e0c00640 1 -- 172.21.15.123:0/496729068 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e40043c0 con 0x556fd1178d60 2024-09-19T04:47:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.147+0000 7f14e0c00640 1 -- 172.21.15.123:0/496729068 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f14e809b480 con 0x556fd1178d60 2024-09-19T04:47:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.147+0000 7f15007d1740 10 client.25487 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.147+0000 7f15007d1740 10 client.25487 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:47:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.147+0000 7f15007d1740 20 client.25487 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.147+0000 7f15007d1740 1 --2- 172.21.15.123:0/496729068 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556fd119c850 0x556fd119ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.147+0000 7f15007d1740 4 client.25487 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:47:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.147+0000 7f15007d1740 1 -- 172.21.15.123:0/496729068 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fd10ec950 con 0x556fd119c850 2024-09-19T04:47:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.147+0000 7f14f6000640 1 --2- 172.21.15.123:0/496729068 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556fd119c850 0x556fd119ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.148+0000 7f14f6000640 1 --2- 172.21.15.123:0/496729068 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556fd119c850 0x556fd119ec40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d0620c5be8ae0cd5 server_cookie=3eb9469570adab5a in_seq=0 out_seq=0 2024-09-19T04:47:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.148+0000 7f14e0c00640 10 client.25487 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:47:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.149+0000 7f14e0c00640 1 -- 172.21.15.123:0/496729068 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556fd10ec950 con 0x556fd119c850 2024-09-19T04:47:49.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.149+0000 7f14e0c00640 10 client.25487 handle_command_reply: tid=0 2024-09-19T04:47:49.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.149+0000 7f15007d1740 1 client.25487 shutdown 2024-09-19T04:47:49.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.149+0000 7f15007d1740 1 -- 172.21.15.123:0/496729068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd1178d60 msgr2=0x556fd11897f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:49.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.149+0000 7f15007d1740 1 --2- 172.21.15.123:0/496729068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd1178d60 0x556fd11897f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f14e800e9c0 tx=0x7f14e809c730 comp rx=0 tx=0).stop 2024-09-19T04:47:49.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.150+0000 7f15007d1740 1 -- 172.21.15.123:0/496729068 shutdown_connections 2024-09-19T04:47:49.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.150+0000 7f15007d1740 1 --2- 172.21.15.123:0/496729068 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556fd119c850 0x556fd119ec40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.150+0000 7f15007d1740 1 --2- 172.21.15.123:0/496729068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd118c830 0x556fd118a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.150+0000 7f15007d1740 1 --2- 172.21.15.123:0/496729068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd1179680 0x556fd1189d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.150+0000 7f15007d1740 1 --2- 172.21.15.123:0/496729068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd1178d60 0x556fd11897f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.150+0000 7f14e0c00640 0 client.25487 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:47:49.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.150+0000 7f15007d1740 1 -- 172.21.15.123:0/496729068 >> 172.21.15.123:0/496729068 conn(0x556fd11753d0 msgr2=0x556fd117b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:49.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.150+0000 7f15007d1740 1 -- 172.21.15.123:0/496729068 shutdown_connections 2024-09-19T04:47:49.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.150+0000 7f15007d1740 1 -- 172.21.15.123:0/496729068 wait complete. 2024-09-19T04:47:49.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.151+0000 7f15007d1740 20 client.25487 trim_cache size 0 max 16384 2024-09-19T04:47:49.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.177+0000 7f15007d1740 1 Processor -- start 2024-09-19T04:47:49.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.177+0000 7f15007d1740 1 -- start start 2024-09-19T04:47:49.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.178+0000 7f15007d1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd1178d60 0x556fd1170300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.178+0000 7f15007d1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd1179680 0x556fd11a44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.178+0000 7f15007d1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd118c830 0x556fd11a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.178+0000 7f15007d1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81ab3a0 con 0x556fd1178d60 2024-09-19T04:47:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.178+0000 7f15007d1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81acee0 con 0x556fd1179680 2024-09-19T04:47:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.178+0000 7f15007d1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81aba70 con 0x556fd118c830 2024-09-19T04:47:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.178+0000 7f14f6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd1178d60 0x556fd1170300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.178+0000 7f14f6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd1178d60 0x556fd1170300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41354/0 (socket says 172.21.15.123:41354) 2024-09-19T04:47:49.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.178+0000 7f14f6a00640 1 -- 172.21.15.123:0/1389227318 learned_addr learned my addr 172.21.15.123:0/1389227318 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:49.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.178+0000 7f14f6000640 1 --2- 172.21.15.123:0/1389227318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd1179680 0x556fd11a44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.178+0000 7f14f7400640 1 --2- 172.21.15.123:0/1389227318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd118c830 0x556fd11a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.179+0000 7f14f7400640 1 -- 172.21.15.123:0/1389227318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd1179680 msgr2=0x556fd11a44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:49.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.179+0000 7f14f7400640 1 --2- 172.21.15.123:0/1389227318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd1179680 0x556fd11a44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.179+0000 7f14f7400640 1 -- 172.21.15.123:0/1389227318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd1178d60 msgr2=0x556fd1170300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:49.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.179+0000 7f14f7400640 1 --2- 172.21.15.123:0/1389227318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd1178d60 0x556fd1170300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.179+0000 7f14f7400640 1 -- 172.21.15.123:0/1389227318 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fd11887a0 con 0x556fd118c830 2024-09-19T04:47:49.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.180+0000 7f14f7400640 1 --2- 172.21.15.123:0/1389227318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd118c830 0x556fd11a4a30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f14f0011e90 tx=0x7f14f0016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=907d0a184051d456 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:49.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.180+0000 7f14e3400640 1 -- 172.21.15.123:0/1389227318 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f000fdc0 con 0x556fd118c830 2024-09-19T04:47:49.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.180+0000 7f14e3400640 1 -- 172.21.15.123:0/1389227318 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14f002b070 con 0x556fd118c830 2024-09-19T04:47:49.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.180+0000 7f14e3400640 1 -- 172.21.15.123:0/1389227318 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f0026770 con 0x556fd118c830 2024-09-19T04:47:49.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.181+0000 7f15007d1740 1 -- 172.21.15.123:0/1389227318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd118c830 msgr2=0x556fd11a4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:49.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.181+0000 7f15007d1740 1 --2- 172.21.15.123:0/1389227318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd118c830 0x556fd11a4a30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f14f0011e90 tx=0x7f14f0016000 comp rx=0 tx=0).stop 2024-09-19T04:47:49.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.181+0000 7f15007d1740 1 -- 172.21.15.123:0/1389227318 shutdown_connections 2024-09-19T04:47:49.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.181+0000 7f15007d1740 1 --2- 172.21.15.123:0/1389227318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd118c830 0x556fd11a4a30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.181+0000 7f15007d1740 1 --2- 172.21.15.123:0/1389227318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd1179680 0x556fd11a44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.181+0000 7f15007d1740 1 --2- 172.21.15.123:0/1389227318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd1178d60 0x556fd1170300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.181+0000 7f15007d1740 1 -- 172.21.15.123:0/1389227318 >> 172.21.15.123:0/1389227318 conn(0x556fd11753d0 msgr2=0x556fd117afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:49.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.182+0000 7f15007d1740 1 -- 172.21.15.123:0/1389227318 shutdown_connections 2024-09-19T04:47:49.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.182+0000 7f15007d1740 1 -- 172.21.15.123:0/1389227318 wait complete. 2024-09-19T04:47:49.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.182+0000 7f15007d1740 1 -- start start 2024-09-19T04:47:49.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.182+0000 7f15007d1740 1 Processor -- start 2024-09-19T04:47:49.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.183+0000 7f15007d1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd1178d60 0x556fd1194750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.183+0000 7f15007d1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd1179680 0x556fd1194c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.183+0000 7f15007d1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd118c830 0x556fd1191720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.183+0000 7f15007d1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81ac810 con 0x556fd118c830 2024-09-19T04:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.183+0000 7f15007d1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81ac140 con 0x556fd1179680 2024-09-19T04:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.183+0000 7f15007d1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81a9380 con 0x556fd1178d60 2024-09-19T04:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.183+0000 7f15007d1740 10 client.0 _ll_register_callbacks cb 0x7f14e80046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.183+0000 7f14f6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd1178d60 0x556fd1194750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.183+0000 7f14f6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd1178d60 0x556fd1194750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49152/0 (socket says 172.21.15.123:49152) 2024-09-19T04:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.183+0000 7f14f6a00640 1 -- 172.21.15.123:0/1521108292 learned_addr learned my addr 172.21.15.123:0/1521108292 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.184+0000 7f14f7400640 1 --2- 172.21.15.123:0/1521108292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd118c830 0x556fd1191720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.184+0000 7f14f6000640 1 --2- 172.21.15.123:0/1521108292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd1179680 0x556fd1194c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.184+0000 7f14f6a00640 1 -- 172.21.15.123:0/1521108292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd1179680 msgr2=0x556fd1194c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.184+0000 7f14f6a00640 1 --2- 172.21.15.123:0/1521108292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd1179680 0x556fd1194c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.184+0000 7f14f6a00640 1 -- 172.21.15.123:0/1521108292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd118c830 msgr2=0x556fd1191720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.184+0000 7f14f6a00640 1 --2- 172.21.15.123:0/1521108292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd118c830 0x556fd1191720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.184+0000 7f14f6a00640 1 -- 172.21.15.123:0/1521108292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fd1191c60 con 0x556fd1178d60 2024-09-19T04:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.184+0000 7f14f6a00640 1 --2- 172.21.15.123:0/1521108292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd1178d60 0x556fd1194750 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f14e8098490 tx=0x7f14e80a04a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bea67e164eab98f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.184+0000 7f14e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.184+0000 7f14e0c00640 1 -- 172.21.15.123:0/1521108292 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e801c040 con 0x556fd1178d60 2024-09-19T04:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.184+0000 7f14e0c00640 1 -- 172.21.15.123:0/1521108292 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f14e809b0e0 con 0x556fd1178d60 2024-09-19T04:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.185+0000 7f15007d1740 1 -- 172.21.15.123:0/1521108292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14f002b070 con 0x556fd1178d60 2024-09-19T04:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.185+0000 7f14e0c00640 1 -- 172.21.15.123:0/1521108292 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14e80d30f0 con 0x556fd1178d60 2024-09-19T04:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.185+0000 7f15007d1740 10 client.25513 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.185+0000 7f15007d1740 1 -- 172.21.15.123:0/1521108292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556fd1207b50 con 0x556fd1178d60 2024-09-19T04:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.185+0000 7f14e0c00640 1 -- 172.21.15.123:0/1521108292 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14f002b070 con 0x556fd1178d60 2024-09-19T04:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.186+0000 7f14e0c00640 1 -- 172.21.15.123:0/1521108292 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f14e80cd7b0 con 0x556fd1178d60 2024-09-19T04:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.186+0000 7f15007d1740 10 client.25513 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.186+0000 7f15007d1740 10 client.25513 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.186+0000 7f15007d1740 20 client.25513 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.186+0000 7f15007d1740 1 --2- 172.21.15.123:0/1521108292 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556fd1208080 0x556fd120a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.186+0000 7f15007d1740 4 client.25513 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.186+0000 7f15007d1740 1 -- 172.21.15.123:0/1521108292 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556fd10e6e60 con 0x556fd1208080 2024-09-19T04:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.186+0000 7f14f6000640 1 --2- 172.21.15.123:0/1521108292 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556fd1208080 0x556fd120a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.187+0000 7f14e0c00640 10 client.25513 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.187+0000 7f14f6000640 1 --2- 172.21.15.123:0/1521108292 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556fd1208080 0x556fd120a470 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=70e34eb8f2ff2884 server_cookie=69c4b843bd2982bf in_seq=0 out_seq=0 2024-09-19T04:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.187+0000 7f14e0c00640 1 -- 172.21.15.123:0/1521108292 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556fd10e6e60 con 0x556fd1208080 2024-09-19T04:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.187+0000 7f14e0c00640 10 client.25513 handle_command_reply: tid=0 2024-09-19T04:47:49.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.187+0000 7f15007d1740 1 client.25513 shutdown 2024-09-19T04:47:49.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.188+0000 7f15007d1740 1 -- 172.21.15.123:0/1521108292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd1178d60 msgr2=0x556fd1194750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:49.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.188+0000 7f15007d1740 1 --2- 172.21.15.123:0/1521108292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd1178d60 0x556fd1194750 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f14e8098490 tx=0x7f14e80a04a0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.188+0000 7f15007d1740 1 -- 172.21.15.123:0/1521108292 shutdown_connections 2024-09-19T04:47:49.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.188+0000 7f15007d1740 1 --2- 172.21.15.123:0/1521108292 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556fd1208080 0x556fd120a470 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.188+0000 7f14e0c00640 0 client.25513 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:47:49.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.188+0000 7f15007d1740 1 --2- 172.21.15.123:0/1521108292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fd118c830 0x556fd1191720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.188+0000 7f15007d1740 1 --2- 172.21.15.123:0/1521108292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fd1179680 0x556fd1194c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.188+0000 7f15007d1740 1 --2- 172.21.15.123:0/1521108292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fd1178d60 0x556fd1194750 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.188+0000 7f15007d1740 1 -- 172.21.15.123:0/1521108292 >> 172.21.15.123:0/1521108292 conn(0x556fd11753d0 msgr2=0x556fd1173c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:49.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.188+0000 7f15007d1740 1 -- 172.21.15.123:0/1521108292 shutdown_connections 2024-09-19T04:47:49.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.189+0000 7f15007d1740 1 -- 172.21.15.123:0/1521108292 wait complete. 2024-09-19T04:47:49.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.189+0000 7f15007d1740 20 client.25513 trim_cache size 0 max 16384 2024-09-19T04:47:49.193 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:47:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.190+0000 7f14f7e00640 1 -- 172.21.15.123:0/3046878076 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f14dc0a35e0 msgr2=0x7f14dc0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.190+0000 7f14f7e00640 1 --2- 172.21.15.123:0/3046878076 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f14dc0a35e0 0x7f14dc0a5aa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f14f00049d0 tx=0x7f14f0009500 comp rx=0 tx=0).stop 2024-09-19T04:47:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.190+0000 7f14f7e00640 1 -- 172.21.15.123:0/3046878076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14f8103e30 msgr2=0x7f14f819a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.190+0000 7f14f7e00640 1 --2- 172.21.15.123:0/3046878076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14f8103e30 0x7f14f819a4b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f14e800e9f0 tx=0x7f14e800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.190+0000 7f14f7e00640 1 -- 172.21.15.123:0/3046878076 shutdown_connections 2024-09-19T04:47:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.190+0000 7f14f7e00640 1 --2- 172.21.15.123:0/3046878076 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f14dc0a35e0 0x7f14dc0a5aa0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.190+0000 7f14f7e00640 1 --2- 172.21.15.123:0/3046878076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14f8108f60 0x7f14f819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.190+0000 7f14f7e00640 1 --2- 172.21.15.123:0/3046878076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14f81047e0 0x7f14f819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.190+0000 7f14f7e00640 1 --2- 172.21.15.123:0/3046878076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14f8103e30 0x7f14f819a4b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:49.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.190+0000 7f14f7e00640 1 -- 172.21.15.123:0/3046878076 >> 172.21.15.123:0/3046878076 conn(0x7f14f80ff910 msgr2=0x7f14f8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:49.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.191+0000 7f14f7e00640 1 -- 172.21.15.123:0/3046878076 shutdown_connections 2024-09-19T04:47:49.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:49.191+0000 7f14f7e00640 1 -- 172.21.15.123:0/3046878076 wait complete. 2024-09-19T04:47:49.586 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:49 smithi123 ceph-mon[30511]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:47:49.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:49 smithi193 ceph-mon[35592]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:47:49.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:49 smithi183 ceph-mon[35333]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:47:49.972 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:47:50.397 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:47:51.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.067+0000 7f88cee00640 1 -- 172.21.15.123:0/1086231440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88c8077040 msgr2=0x7f88c8112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:51.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.067+0000 7f88cee00640 1 --2- 172.21.15.123:0/1086231440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88c8077040 0x7f88c8112510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009900 tx=0x7f88b402a410 comp rx=0 tx=0).stop 2024-09-19T04:47:51.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.067+0000 7f88cee00640 1 -- 172.21.15.123:0/1086231440 shutdown_connections 2024-09-19T04:47:51.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.067+0000 7f88cee00640 1 --2- 172.21.15.123:0/1086231440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88c8112a50 0x7f88c8114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.067+0000 7f88cee00640 1 --2- 172.21.15.123:0/1086231440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88c8077040 0x7f88c8112510 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.067+0000 7f88cee00640 1 --2- 172.21.15.123:0/1086231440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88c8076720 0x7f88c8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.067+0000 7f88cee00640 1 -- 172.21.15.123:0/1086231440 >> 172.21.15.123:0/1086231440 conn(0x7f88c80ffa50 msgr2=0x7f88c8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:51.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.067+0000 7f88cee00640 1 -- 172.21.15.123:0/1086231440 shutdown_connections 2024-09-19T04:47:51.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.068+0000 7f88cee00640 1 -- 172.21.15.123:0/1086231440 wait complete. 2024-09-19T04:47:51.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.068+0000 7f88cee00640 1 Processor -- start 2024-09-19T04:47:51.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.069+0000 7f88cee00640 1 -- start start 2024-09-19T04:47:51.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.069+0000 7f88cee00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88c8076720 0x7f88c81a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.069+0000 7f88cee00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88c8077040 0x7f88c81a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.069+0000 7f88cee00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88c8112a50 0x7f88c819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.069+0000 7f88cee00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f88c8118620 con 0x7f88c8077040 2024-09-19T04:47:51.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.069+0000 7f88cee00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88c81184a0 con 0x7f88c8112a50 2024-09-19T04:47:51.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.069+0000 7f88cee00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f88c81187a0 con 0x7f88c8076720 2024-09-19T04:47:51.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.069+0000 7f88c7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88c8076720 0x7f88c81a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:51.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.069+0000 7f88c7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88c8076720 0x7f88c81a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49172/0 (socket says 172.21.15.123:49172) 2024-09-19T04:47:51.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.069+0000 7f88c7e00640 1 -- 172.21.15.123:0/3616657347 learned_addr learned my addr 172.21.15.123:0/3616657347 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:51.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.070+0000 7f88c7400640 1 --2- 172.21.15.123:0/3616657347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88c8077040 0x7f88c81a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:51.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.070+0000 7f88c7e00640 1 -- 172.21.15.123:0/3616657347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88c8112a50 msgr2=0x7f88c819e8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:47:51.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.070+0000 7f88c7e00640 1 --2- 172.21.15.123:0/3616657347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88c8112a50 0x7f88c819e8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.070+0000 7f88c7e00640 1 -- 172.21.15.123:0/3616657347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88c8077040 msgr2=0x7f88c81a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:51.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.070+0000 7f88c7e00640 1 --2- 172.21.15.123:0/3616657347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88c8077040 0x7f88c81a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.070+0000 7f88c7e00640 1 -- 172.21.15.123:0/3616657347 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c819f020 con 0x7f88c8076720 2024-09-19T04:47:51.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.070+0000 7f88c7400640 1 --2- 172.21.15.123:0/3616657347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88c8077040 0x7f88c81a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:47:51.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.071+0000 7f88c7e00640 1 --2- 172.21.15.123:0/3616657347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88c8076720 0x7f88c81a4730 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f88b800e9f0 tx=0x7f88b800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:51.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.071+0000 7f88c4c00640 1 -- 172.21.15.123:0/3616657347 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b800cd80 con 0x7f88c8076720 2024-09-19T04:47:51.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.071+0000 7f88c4c00640 1 -- 172.21.15.123:0/3616657347 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b8010910 con 0x7f88c8076720 2024-09-19T04:47:51.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.071+0000 7f88c4c00640 1 -- 172.21.15.123:0/3616657347 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8018990 con 0x7f88c8076720 2024-09-19T04:47:51.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.071+0000 7f88cee00640 1 -- 172.21.15.123:0/3616657347 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c819f310 con 0x7f88c8076720 2024-09-19T04:47:51.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.071+0000 7f88cee00640 1 -- 172.21.15.123:0/3616657347 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c806bc10 con 0x7f88c8076720 2024-09-19T04:47:51.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.076+0000 7f88d1a16740 1 Processor -- start 2024-09-19T04:47:51.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.076+0000 7f88d1a16740 1 -- start start 2024-09-19T04:47:51.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.076+0000 7f88c4c00640 1 -- 172.21.15.123:0/3616657347 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7f88b8010ac0 con 0x7f88c8076720 2024-09-19T04:47:51.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.077+0000 7f88d1a16740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 0x562f3566d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.077+0000 7f88d1a16740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f3566d590 0x562f35677910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.077+0000 7f88d1a16740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f35677e50 0x562f3567a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.077+0000 7f88d1a16740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562f355d33e0 con 0x562f35677e50 2024-09-19T04:47:51.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.077+0000 7f88d1a16740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562f3557ad80 con 0x562f3566d590 2024-09-19T04:47:51.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.077+0000 7f88d1a16740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562f355cf3a0 con 0x562f3566cc70 2024-09-19T04:47:51.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.077+0000 7f88c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f3566d590 0x562f35677910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:51.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.077+0000 7f88c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f3566d590 0x562f35677910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:56504/0 (socket says 172.21.15.123:56504) 2024-09-19T04:47:51.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.077+0000 7f88c7e00640 1 -- 172.21.15.123:0/2018211693 learned_addr learned my addr 172.21.15.123:0/2018211693 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:51.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.077+0000 7f88cce00640 1 --2- 172.21.15.123:0/2018211693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 0x562f3566d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:51.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.077+0000 7f88c7400640 1 --2- 172.21.15.123:0/2018211693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f35677e50 0x562f3567a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:51.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.078+0000 7f88cce00640 1 -- 172.21.15.123:0/2018211693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f3566d590 msgr2=0x562f35677910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:51.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.078+0000 7f88cce00640 1 --2- 172.21.15.123:0/2018211693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f3566d590 0x562f35677910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.078+0000 7f88cce00640 1 -- 172.21.15.123:0/2018211693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f35677e50 msgr2=0x562f3567a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:51.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.078+0000 7f88cce00640 1 --2- 172.21.15.123:0/2018211693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f35677e50 0x562f3567a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.078+0000 7f88cce00640 1 -- 172.21.15.123:0/2018211693 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f35599ae0 con 0x562f3566cc70 2024-09-19T04:47:51.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.078+0000 7f88cce00640 1 --2- 172.21.15.123:0/2018211693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 0x562f3566d050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f88c807aa10 tx=0x7f88c000e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7210a45cfcecd721 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:51.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.078+0000 7f88bd600640 1 -- 172.21.15.123:0/2018211693 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0016070 con 0x562f3566cc70 2024-09-19T04:47:51.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.079+0000 7f88bd600640 1 -- 172.21.15.123:0/2018211693 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c0003970 con 0x562f3566cc70 2024-09-19T04:47:51.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.079+0000 7f88bd600640 1 -- 172.21.15.123:0/2018211693 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0004520 con 0x562f3566cc70 2024-09-19T04:47:51.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.079+0000 7f88c4c00640 1 --2- 172.21.15.123:0/3616657347 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f88ac0a35e0 0x7f88ac0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.079+0000 7f88c4c00640 1 -- 172.21.15.123:0/3616657347 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f88b80d03a0 con 0x7f88c8076720 2024-09-19T04:47:51.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.080+0000 7f88d1a16740 1 -- 172.21.15.123:0/2018211693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 msgr2=0x562f3566d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:51.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.080+0000 7f88d1a16740 1 --2- 172.21.15.123:0/2018211693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 0x562f3566d050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f88c807aa10 tx=0x7f88c000e6c0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.080+0000 7f88d1a16740 1 -- 172.21.15.123:0/2018211693 shutdown_connections 2024-09-19T04:47:51.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.080+0000 7f88d1a16740 1 --2- 172.21.15.123:0/2018211693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f35677e50 0x562f3567a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.080+0000 7f88d1a16740 1 --2- 172.21.15.123:0/2018211693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f3566d590 0x562f35677910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.080+0000 7f88d1a16740 1 --2- 172.21.15.123:0/2018211693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 0x562f3566d050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.080+0000 7f88d1a16740 1 -- 172.21.15.123:0/2018211693 >> 172.21.15.123:0/2018211693 conn(0x562f35669200 msgr2=0x562f356695e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:51.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.080+0000 7f88c7e00640 1 --2- 172.21.15.123:0/3616657347 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f88ac0a35e0 0x7f88ac0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:51.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.080+0000 7f88d1a16740 1 -- 172.21.15.123:0/2018211693 shutdown_connections 2024-09-19T04:47:51.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.081+0000 7f88d1a16740 1 -- 172.21.15.123:0/2018211693 wait complete. 2024-09-19T04:47:51.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.081+0000 7f88c7e00640 1 --2- 172.21.15.123:0/3616657347 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f88ac0a35e0 0x7f88ac0a5aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f88b80988d0 tx=0x7f88b80d3e50 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:51.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.081+0000 7f88d1a16740 1 -- start start 2024-09-19T04:47:51.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.082+0000 7f88d1a16740 1 Processor -- start 2024-09-19T04:47:51.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.082+0000 7f88d1a16740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 0x562f35674d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.082+0000 7f88d1a16740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f3566d590 0x562f35675270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.082+0000 7f88d1a16740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f35677e50 0x562f356757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.082+0000 7f88d1a16740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562f3567aa20 con 0x562f3566d590 2024-09-19T04:47:51.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.082+0000 7f88d1a16740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562f3558e0f0 con 0x562f35677e50 2024-09-19T04:47:51.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.082+0000 7f88d1a16740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562f3567aba0 con 0x562f3566cc70 2024-09-19T04:47:51.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.082+0000 7f88d1a16740 10 client.0 _ll_register_callbacks cb 0x562f35575bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:51.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.083+0000 7f88cce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 0x562f35674d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:51.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.083+0000 7f88cce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 0x562f35674d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49190/0 (socket says 172.21.15.123:49190) 2024-09-19T04:47:51.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.083+0000 7f88cce00640 1 -- 172.21.15.123:0/2176295489 learned_addr learned my addr 172.21.15.123:0/2176295489 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:51.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.083+0000 7f88cce00640 1 --2- 172.21.15.123:0/2176295489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f35677e50 0x562f356757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:51.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.083+0000 7f88c7400640 1 --2- 172.21.15.123:0/2176295489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f3566d590 0x562f35675270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:51.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.083+0000 7f88cce00640 1 -- 172.21.15.123:0/2176295489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f35677e50 msgr2=0x562f356757b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:51.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.083+0000 7f88cce00640 1 --2- 172.21.15.123:0/2176295489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f35677e50 0x562f356757b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.083+0000 7f88cce00640 1 -- 172.21.15.123:0/2176295489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f3566d590 msgr2=0x562f35675270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:51.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.083+0000 7f88cce00640 1 --2- 172.21.15.123:0/2176295489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f3566d590 0x562f35675270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.083+0000 7f88cce00640 1 -- 172.21.15.123:0/2176295489 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f353443c0 con 0x562f3566cc70 2024-09-19T04:47:51.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.084+0000 7f88cce00640 1 --2- 172.21.15.123:0/2176295489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 0x562f35674d30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f88c0004370 tx=0x7f88c00040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1aaf19e95115adf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:51.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.084+0000 7f88b2000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:47:51.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.084+0000 7f88b2000640 1 -- 172.21.15.123:0/2176295489 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0016070 con 0x562f3566cc70 2024-09-19T04:47:51.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.084+0000 7f88b2000640 1 -- 172.21.15.123:0/2176295489 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f88c00045e0 con 0x562f3566cc70 2024-09-19T04:47:51.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.084+0000 7f88d1a16740 1 -- 172.21.15.123:0/2176295489 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88c0003970 con 0x562f3566cc70 2024-09-19T04:47:51.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.085+0000 7f88b2000640 1 -- 172.21.15.123:0/2176295489 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f88c002e050 con 0x562f3566cc70 2024-09-19T04:47:51.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.085+0000 7f88d1a16740 10 client.25525 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:51.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.085+0000 7f88d1a16740 1 -- 172.21.15.123:0/2176295489 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562f35686a60 con 0x562f3566cc70 2024-09-19T04:47:51.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.085+0000 7f88b2000640 1 -- 172.21.15.123:0/2176295489 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c0003970 con 0x562f3566cc70 2024-09-19T04:47:51.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.086+0000 7f88b2000640 1 -- 172.21.15.123:0/2176295489 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f88c000f040 con 0x562f3566cc70 2024-09-19T04:47:51.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.086+0000 7f88d1a16740 10 client.25525 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:51.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.086+0000 7f88d1a16740 10 client.25525 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:47:51.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.086+0000 7f88d1a16740 20 client.25525 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:51.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.086+0000 7f88d1a16740 1 --2- 172.21.15.123:0/2176295489 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x562f35687ec0 0x562f3568a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.086+0000 7f88d1a16740 4 client.25525 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:47:51.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.086+0000 7f88d1a16740 1 -- 172.21.15.123:0/2176295489 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f355e0950 con 0x562f35687ec0 2024-09-19T04:47:51.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.086+0000 7f88c7400640 1 --2- 172.21.15.123:0/2176295489 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x562f35687ec0 0x562f3568a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:51.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.087+0000 7f88c7400640 1 --2- 172.21.15.123:0/2176295489 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x562f35687ec0 0x562f3568a2b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b65498adfeb1996 server_cookie=559a8e06d55f989 in_seq=0 out_seq=0 2024-09-19T04:47:51.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.087+0000 7f88b2000640 10 client.25525 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:47:51.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.088+0000 7f88b2000640 1 -- 172.21.15.123:0/2176295489 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f355e0950 con 0x562f35687ec0 2024-09-19T04:47:51.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.088+0000 7f88b2000640 10 client.25525 handle_command_reply: tid=0 2024-09-19T04:47:51.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.088+0000 7f88d1a16740 1 client.25525 shutdown 2024-09-19T04:47:51.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.088+0000 7f88d1a16740 1 -- 172.21.15.123:0/2176295489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 msgr2=0x562f35674d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:51.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.088+0000 7f88d1a16740 1 --2- 172.21.15.123:0/2176295489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 0x562f35674d30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f88c0004370 tx=0x7f88c00040c0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.088+0000 7f88d1a16740 1 -- 172.21.15.123:0/2176295489 shutdown_connections 2024-09-19T04:47:51.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.088+0000 7f88d1a16740 1 --2- 172.21.15.123:0/2176295489 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x562f35687ec0 0x562f3568a2b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.088+0000 7f88d1a16740 1 --2- 172.21.15.123:0/2176295489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f35677e50 0x562f356757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.088+0000 7f88d1a16740 1 --2- 172.21.15.123:0/2176295489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f3566d590 0x562f35675270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.088+0000 7f88d1a16740 1 --2- 172.21.15.123:0/2176295489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 0x562f35674d30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.088+0000 7f88b2000640 0 client.25525 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:47:51.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.088+0000 7f88d1a16740 1 -- 172.21.15.123:0/2176295489 >> 172.21.15.123:0/2176295489 conn(0x562f35669200 msgr2=0x562f35668380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:51.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.089+0000 7f88d1a16740 1 -- 172.21.15.123:0/2176295489 shutdown_connections 2024-09-19T04:47:51.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.089+0000 7f88d1a16740 1 -- 172.21.15.123:0/2176295489 wait complete. 2024-09-19T04:47:51.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.089+0000 7f88d1a16740 20 client.25525 trim_cache size 0 max 16384 2024-09-19T04:47:51.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.114+0000 7f88d1a16740 1 Processor -- start 2024-09-19T04:47:51.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.114+0000 7f88d1a16740 1 -- start start 2024-09-19T04:47:51.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.115+0000 7f88d1a16740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 0x562f3566f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.115+0000 7f88d1a16740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f3566d590 0x562f35674290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.115+0000 7f88d1a16740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f35677e50 0x562f3568fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.115+0000 7f88d1a16740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f88c81bc7b0 con 0x562f3566d590 2024-09-19T04:47:51.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.115+0000 7f88d1a16740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88c81be2f0 con 0x562f35677e50 2024-09-19T04:47:51.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.115+0000 7f88d1a16740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f88c81bce80 con 0x562f3566cc70 2024-09-19T04:47:51.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.115+0000 7f88c7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f3566d590 0x562f35674290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:51.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.115+0000 7f88c7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f3566d590 0x562f35674290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41422/0 (socket says 172.21.15.123:41422) 2024-09-19T04:47:51.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.115+0000 7f88c7400640 1 -- 172.21.15.123:0/3915641282 learned_addr learned my addr 172.21.15.123:0/3915641282 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:51.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.115+0000 7f88cce00640 1 --2- 172.21.15.123:0/3915641282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 0x562f3566f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:51.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.115+0000 7f88cce00640 1 --2- 172.21.15.123:0/3915641282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f35677e50 0x562f3568fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:51.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.116+0000 7f88c7400640 1 -- 172.21.15.123:0/3915641282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 msgr2=0x562f3566f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:51.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.116+0000 7f88c7400640 1 --2- 172.21.15.123:0/3915641282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 0x562f3566f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.116+0000 7f88c7400640 1 -- 172.21.15.123:0/3915641282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f35677e50 msgr2=0x562f3568fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:51.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.116+0000 7f88c7400640 1 --2- 172.21.15.123:0/3915641282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f35677e50 0x562f3568fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.116+0000 7f88c7400640 1 -- 172.21.15.123:0/3915641282 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f35664770 con 0x562f3566d590 2024-09-19T04:47:51.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.116+0000 7f88cce00640 1 --2- 172.21.15.123:0/3915641282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f35677e50 0x562f3568fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:51.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.116+0000 7f88c7400640 1 --2- 172.21.15.123:0/3915641282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f3566d590 0x562f35674290 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f88b402a930 tx=0x7f88b4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7298ddef59c37386 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:51.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.116+0000 7f88bcc00640 1 -- 172.21.15.123:0/3915641282 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4035040 con 0x562f3566d590 2024-09-19T04:47:51.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.116+0000 7f88bcc00640 1 -- 172.21.15.123:0/3915641282 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b4049070 con 0x562f3566d590 2024-09-19T04:47:51.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.116+0000 7f88bcc00640 1 -- 172.21.15.123:0/3915641282 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4004bf0 con 0x562f3566d590 2024-09-19T04:47:51.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.117+0000 7f88d1a16740 1 -- 172.21.15.123:0/3915641282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f3566d590 msgr2=0x562f35674290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:51.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.117+0000 7f88d1a16740 1 --2- 172.21.15.123:0/3915641282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f3566d590 0x562f35674290 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f88b402a930 tx=0x7f88b4000bc0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.118+0000 7f88d1a16740 1 -- 172.21.15.123:0/3915641282 shutdown_connections 2024-09-19T04:47:51.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.118+0000 7f88d1a16740 1 --2- 172.21.15.123:0/3915641282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f35677e50 0x562f3568fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.118+0000 7f88d1a16740 1 --2- 172.21.15.123:0/3915641282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f3566d590 0x562f35674290 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.118+0000 7f88d1a16740 1 --2- 172.21.15.123:0/3915641282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566cc70 0x562f3566f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.118+0000 7f88d1a16740 1 -- 172.21.15.123:0/3915641282 >> 172.21.15.123:0/3915641282 conn(0x562f35669200 msgr2=0x562f35668380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:51.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.118+0000 7f88d1a16740 1 -- 172.21.15.123:0/3915641282 shutdown_connections 2024-09-19T04:47:51.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.118+0000 7f88d1a16740 1 -- 172.21.15.123:0/3915641282 wait complete. 2024-09-19T04:47:51.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.118+0000 7f88d1a16740 1 -- start start 2024-09-19T04:47:51.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.119+0000 7f88d1a16740 1 Processor -- start 2024-09-19T04:47:51.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.119+0000 7f88d1a16740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f3566cc70 0x562f356802e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.119+0000 7f88d1a16740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566d590 0x562f3567b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.119+0000 7f88d1a16740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f35677e50 0x562f3567bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.119+0000 7f88d1a16740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f88c81bdc20 con 0x562f35677e50 2024-09-19T04:47:51.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.119+0000 7f88d1a16740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88c81bd550 con 0x562f3566cc70 2024-09-19T04:47:51.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.119+0000 7f88d1a16740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f88c8076b00 con 0x562f3566d590 2024-09-19T04:47:51.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.119+0000 7f88d1a16740 10 client.0 _ll_register_callbacks cb 0x7f88b80d3d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:51.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.120+0000 7f88cce00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f3566cc70 0x562f356802e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:51.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.120+0000 7f88cce00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f3566cc70 0x562f356802e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:56536/0 (socket says 172.21.15.123:56536) 2024-09-19T04:47:51.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.120+0000 7f88cce00640 1 -- 172.21.15.123:0/2569562336 learned_addr learned my addr 172.21.15.123:0/2569562336 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:51.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.120+0000 7f88c7400640 1 --2- 172.21.15.123:0/2569562336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566d590 0x562f3567b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:51.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.120+0000 7f88c7400640 1 -- 172.21.15.123:0/2569562336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f3566cc70 msgr2=0x562f356802e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:51.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.120+0000 7f88c7400640 1 --2- 172.21.15.123:0/2569562336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f3566cc70 0x562f356802e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.120+0000 7f88c7400640 1 -- 172.21.15.123:0/2569562336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f35677e50 msgr2=0x562f3567bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:51.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.120+0000 7f88cce00640 1 --2- 172.21.15.123:0/2569562336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f35677e50 0x562f3567bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:51.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.120+0000 7f88c7400640 1 --2- 172.21.15.123:0/2569562336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f35677e50 0x562f3567bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.120+0000 7f88c7400640 1 -- 172.21.15.123:0/2569562336 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f3567c110 con 0x562f3566d590 2024-09-19T04:47:51.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.120+0000 7f88cce00640 1 --2- 172.21.15.123:0/2569562336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f35677e50 0x562f3567bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:47:51.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.121+0000 7f88c7400640 1 --2- 172.21.15.123:0/2569562336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566d590 0x562f3567b610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009900 tx=0x7f88b40041e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36bf7ba8cfa1f54f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:51.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.121+0000 7f88b2000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:47:51.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.121+0000 7f88b2000640 1 -- 172.21.15.123:0/2569562336 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4035070 con 0x562f3566d590 2024-09-19T04:47:51.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.121+0000 7f88b2000640 1 -- 172.21.15.123:0/2569562336 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f88b4005250 con 0x562f3566d590 2024-09-19T04:47:51.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.121+0000 7f88d1a16740 1 -- 172.21.15.123:0/2569562336 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88b4049070 con 0x562f3566d590 2024-09-19T04:47:51.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.121+0000 7f88b2000640 1 -- 172.21.15.123:0/2569562336 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f88b4004460 con 0x562f3566d590 2024-09-19T04:47:51.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.121+0000 7f88d1a16740 10 client.25531 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:51.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.121+0000 7f88d1a16740 1 -- 172.21.15.123:0/2569562336 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562f356f2780 con 0x562f3566d590 2024-09-19T04:47:51.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.122+0000 7f88b2000640 1 -- 172.21.15.123:0/2569562336 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b4049070 con 0x562f3566d590 2024-09-19T04:47:51.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.122+0000 7f88b2000640 1 -- 172.21.15.123:0/2569562336 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f88b4031920 con 0x562f3566d590 2024-09-19T04:47:51.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.123+0000 7f88d1a16740 10 client.25531 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:51.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.123+0000 7f88d1a16740 10 client.25531 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:47:51.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.123+0000 7f88d1a16740 20 client.25531 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:51.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.123+0000 7f88d1a16740 1 --2- 172.21.15.123:0/2569562336 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x562f356f2e70 0x562f356f5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:51.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.123+0000 7f88d1a16740 4 client.25531 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:47:51.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.123+0000 7f88d1a16740 1 -- 172.21.15.123:0/2569562336 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f355dae60 con 0x562f356f2e70 2024-09-19T04:47:51.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.123+0000 7f88cce00640 1 --2- 172.21.15.123:0/2569562336 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x562f356f2e70 0x562f356f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:51.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.124+0000 7f88cce00640 1 --2- 172.21.15.123:0/2569562336 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x562f356f2e70 0x562f356f5290 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dbb9f48fabe9d380 server_cookie=63593ec7e2edea51 in_seq=0 out_seq=0 2024-09-19T04:47:51.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.124+0000 7f88b2000640 10 client.25531 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:47:51.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.124+0000 7f88b2000640 1 -- 172.21.15.123:0/2569562336 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f355dae60 con 0x562f356f2e70 2024-09-19T04:47:51.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.124+0000 7f88b2000640 10 client.25531 handle_command_reply: tid=0 2024-09-19T04:47:51.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.124+0000 7f88d1a16740 1 client.25531 shutdown 2024-09-19T04:47:51.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.124+0000 7f88d1a16740 1 -- 172.21.15.123:0/2569562336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566d590 msgr2=0x562f3567b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:51.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.124+0000 7f88d1a16740 1 --2- 172.21.15.123:0/2569562336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566d590 0x562f3567b610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009900 tx=0x7f88b40041e0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.124+0000 7f88d1a16740 1 -- 172.21.15.123:0/2569562336 shutdown_connections 2024-09-19T04:47:51.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.124+0000 7f88d1a16740 1 --2- 172.21.15.123:0/2569562336 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x562f356f2e70 0x562f356f5290 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.124+0000 7f88d1a16740 1 --2- 172.21.15.123:0/2569562336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f35677e50 0x562f3567bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.124+0000 7f88d1a16740 1 --2- 172.21.15.123:0/2569562336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f3566d590 0x562f3567b610 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.124+0000 7f88d1a16740 1 --2- 172.21.15.123:0/2569562336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f3566cc70 0x562f356802e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.124+0000 7f88d1a16740 1 -- 172.21.15.123:0/2569562336 >> 172.21.15.123:0/2569562336 conn(0x562f35669200 msgr2=0x562f35682c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:51.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.124+0000 7f88b2000640 0 client.25531 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:47:51.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.125+0000 7f88d1a16740 1 -- 172.21.15.123:0/2569562336 shutdown_connections 2024-09-19T04:47:51.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.125+0000 7f88d1a16740 1 -- 172.21.15.123:0/2569562336 wait complete. 2024-09-19T04:47:51.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.125+0000 7f88d1a16740 20 client.25531 trim_cache size 0 max 16384 2024-09-19T04:47:51.128 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:47:51.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.126+0000 7f88cee00640 1 -- 172.21.15.123:0/3616657347 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f88ac0a35e0 msgr2=0x7f88ac0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:51.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.126+0000 7f88cee00640 1 --2- 172.21.15.123:0/3616657347 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f88ac0a35e0 0x7f88ac0a5aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f88b80988d0 tx=0x7f88b80d3e50 comp rx=0 tx=0).stop 2024-09-19T04:47:51.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.126+0000 7f88cee00640 1 -- 172.21.15.123:0/3616657347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88c8076720 msgr2=0x7f88c81a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:51.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.126+0000 7f88cee00640 1 --2- 172.21.15.123:0/3616657347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88c8076720 0x7f88c81a4730 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f88b800e9f0 tx=0x7f88b800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.126+0000 7f88cee00640 1 -- 172.21.15.123:0/3616657347 shutdown_connections 2024-09-19T04:47:51.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.126+0000 7f88cee00640 1 --2- 172.21.15.123:0/3616657347 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f88ac0a35e0 0x7f88ac0a5aa0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.126+0000 7f88cee00640 1 --2- 172.21.15.123:0/3616657347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88c8112a50 0x7f88c819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.126+0000 7f88cee00640 1 --2- 172.21.15.123:0/3616657347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88c8077040 0x7f88c81a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.126+0000 7f88cee00640 1 --2- 172.21.15.123:0/3616657347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88c8076720 0x7f88c81a4730 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:51.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.126+0000 7f88cee00640 1 -- 172.21.15.123:0/3616657347 >> 172.21.15.123:0/3616657347 conn(0x7f88c80ffa50 msgr2=0x7f88c8101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:51.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.126+0000 7f88cee00640 1 -- 172.21.15.123:0/3616657347 shutdown_connections 2024-09-19T04:47:51.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:51.126+0000 7f88cee00640 1 -- 172.21.15.123:0/3616657347 wait complete. 2024-09-19T04:47:51.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:51 smithi123 ceph-mon[30511]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:47:51.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:51 smithi193 ceph-mon[35592]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:47:51.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:51 smithi183 ceph-mon[35333]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:47:52.010 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:47:52.403 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:47:53.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.019+0000 7eff61c00640 1 -- 172.21.15.123:0/3002295508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff5c1027a0 msgr2=0x7eff5c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:53.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.019+0000 7eff61c00640 1 --2- 172.21.15.123:0/3002295508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff5c1027a0 0x7eff5c1067f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7eff48009860 tx=0x7eff4802a340 comp rx=0 tx=0).stop 2024-09-19T04:47:53.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.019+0000 7eff61c00640 1 -- 172.21.15.123:0/3002295508 shutdown_connections 2024-09-19T04:47:53.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.019+0000 7eff61c00640 1 --2- 172.21.15.123:0/3002295508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff5c106f20 0x7eff5c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.019+0000 7eff61c00640 1 --2- 172.21.15.123:0/3002295508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff5c1027a0 0x7eff5c1067f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.019+0000 7eff61c00640 1 --2- 172.21.15.123:0/3002295508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff5c101df0 0x7eff5c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.019+0000 7eff61c00640 1 -- 172.21.15.123:0/3002295508 >> 172.21.15.123:0/3002295508 conn(0x7eff5c0fd8b0 msgr2=0x7eff5c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:53.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.019+0000 7eff61c00640 1 -- 172.21.15.123:0/3002295508 shutdown_connections 2024-09-19T04:47:53.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.019+0000 7eff61c00640 1 -- 172.21.15.123:0/3002295508 wait complete. 2024-09-19T04:47:53.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.021+0000 7eff61c00640 1 Processor -- start 2024-09-19T04:47:53.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.021+0000 7eff61c00640 1 -- start start 2024-09-19T04:47:53.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.021+0000 7eff61c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff5c101df0 0x7eff5c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.021+0000 7eff61c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff5c1027a0 0x7eff5c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.021+0000 7eff61c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff5c106f20 0x7eff5c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.021+0000 7eff61c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c111f80 con 0x7eff5c101df0 2024-09-19T04:47:53.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.021+0000 7eff61c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c111e00 con 0x7eff5c106f20 2024-09-19T04:47:53.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.021+0000 7eff61c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c112100 con 0x7eff5c1027a0 2024-09-19T04:47:53.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.022+0000 7eff5a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff5c1027a0 0x7eff5c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:53.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.022+0000 7eff5a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff5c1027a0 0x7eff5c19ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:55102/0 (socket says 172.21.15.123:55102) 2024-09-19T04:47:53.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.022+0000 7eff5a000640 1 -- 172.21.15.123:0/1196912474 learned_addr learned my addr 172.21.15.123:0/1196912474 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:53.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.022+0000 7eff5b400640 1 --2- 172.21.15.123:0/1196912474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff5c106f20 0x7eff5c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:53.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.022+0000 7eff5a000640 1 -- 172.21.15.123:0/1196912474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff5c106f20 msgr2=0x7eff5c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:53.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.022+0000 7eff5a000640 1 --2- 172.21.15.123:0/1196912474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff5c106f20 0x7eff5c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.022+0000 7eff5a000640 1 -- 172.21.15.123:0/1196912474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff5c101df0 msgr2=0x7eff5c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:53.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.022+0000 7eff5a000640 1 --2- 172.21.15.123:0/1196912474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff5c101df0 0x7eff5c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.022+0000 7eff5a000640 1 -- 172.21.15.123:0/1196912474 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c19f600 con 0x7eff5c1027a0 2024-09-19T04:47:53.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.022+0000 7eff5b400640 1 --2- 172.21.15.123:0/1196912474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff5c106f20 0x7eff5c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:53.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.022+0000 7eff5a000640 1 --2- 172.21.15.123:0/1196912474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff5c1027a0 0x7eff5c19ab90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7eff48035040 tx=0x7eff48033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:53.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.023+0000 7eff53400640 1 -- 172.21.15.123:0/1196912474 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48037040 con 0x7eff5c1027a0 2024-09-19T04:47:53.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.023+0000 7eff61c00640 1 -- 172.21.15.123:0/1196912474 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5c19f890 con 0x7eff5c1027a0 2024-09-19T04:47:53.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.023+0000 7eff61c00640 1 -- 172.21.15.123:0/1196912474 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff5c108a50 con 0x7eff5c1027a0 2024-09-19T04:47:53.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.023+0000 7eff53400640 1 -- 172.21.15.123:0/1196912474 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff48043070 con 0x7eff5c1027a0 2024-09-19T04:47:53.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.023+0000 7eff53400640 1 -- 172.21.15.123:0/1196912474 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff480035f0 con 0x7eff5c1027a0 2024-09-19T04:47:53.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.026+0000 7eff645ee740 1 Processor -- start 2024-09-19T04:47:53.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.026+0000 7eff645ee740 1 -- start start 2024-09-19T04:47:53.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.027+0000 7eff645ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 0x558f95c60050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.027+0000 7eff645ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c60590 0x558f95c6a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.027+0000 7eff645ee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c6ae50 0x558f95c6d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.027+0000 7eff645ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558f95bc63e0 con 0x558f95c5fc70 2024-09-19T04:47:53.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.027+0000 7eff645ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558f95b6dd80 con 0x558f95c60590 2024-09-19T04:47:53.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.027+0000 7eff645ee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558f95bc23a0 con 0x558f95c6ae50 2024-09-19T04:47:53.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.027+0000 7eff5b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 0x558f95c60050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:53.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.027+0000 7eff5b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 0x558f95c60050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46376/0 (socket says 172.21.15.123:46376) 2024-09-19T04:47:53.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.027+0000 7eff5b400640 1 -- 172.21.15.123:0/3435868190 learned_addr learned my addr 172.21.15.123:0/3435868190 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:53.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.027+0000 7eff5aa00640 1 --2- 172.21.15.123:0/3435868190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c60590 0x558f95c6a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:53.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.028+0000 7eff5b400640 1 -- 172.21.15.123:0/3435868190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c6ae50 msgr2=0x558f95c6d240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:47:53.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.028+0000 7eff5b400640 1 --2- 172.21.15.123:0/3435868190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c6ae50 0x558f95c6d240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.028+0000 7eff5b400640 1 -- 172.21.15.123:0/3435868190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c60590 msgr2=0x558f95c6a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:53.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.028+0000 7eff5b400640 1 --2- 172.21.15.123:0/3435868190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c60590 0x558f95c6a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.028+0000 7eff5b400640 1 -- 172.21.15.123:0/3435868190 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f95b8cae0 con 0x558f95c5fc70 2024-09-19T04:47:53.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.028+0000 7eff5b400640 1 --2- 172.21.15.123:0/3435868190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 0x558f95c60050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7eff54009fd0 tx=0x7eff540107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0d19397bb275f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:53.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.029+0000 7eff47e00640 1 -- 172.21.15.123:0/3435868190 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54011640 con 0x558f95c5fc70 2024-09-19T04:47:53.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.029+0000 7eff47e00640 1 -- 172.21.15.123:0/3435868190 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff54011cc0 con 0x558f95c5fc70 2024-09-19T04:47:53.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.029+0000 7eff47e00640 1 -- 172.21.15.123:0/3435868190 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5400e3c0 con 0x558f95c5fc70 2024-09-19T04:47:53.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.029+0000 7eff53400640 1 -- 172.21.15.123:0/1196912474 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7eff48003790 con 0x7eff5c1027a0 2024-09-19T04:47:53.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.029+0000 7eff645ee740 1 -- 172.21.15.123:0/3435868190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 msgr2=0x558f95c60050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:53.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.029+0000 7eff645ee740 1 --2- 172.21.15.123:0/3435868190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 0x558f95c60050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7eff54009fd0 tx=0x7eff540107a0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.030+0000 7eff645ee740 1 -- 172.21.15.123:0/3435868190 shutdown_connections 2024-09-19T04:47:53.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.030+0000 7eff645ee740 1 --2- 172.21.15.123:0/3435868190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c6ae50 0x558f95c6d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.030+0000 7eff645ee740 1 --2- 172.21.15.123:0/3435868190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c60590 0x558f95c6a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.030+0000 7eff645ee740 1 --2- 172.21.15.123:0/3435868190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 0x558f95c60050 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.030+0000 7eff645ee740 1 -- 172.21.15.123:0/3435868190 >> 172.21.15.123:0/3435868190 conn(0x558f95c5c200 msgr2=0x558f95c5c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:53.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.030+0000 7eff645ee740 1 -- 172.21.15.123:0/3435868190 shutdown_connections 2024-09-19T04:47:53.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.031+0000 7eff645ee740 1 -- 172.21.15.123:0/3435868190 wait complete. 2024-09-19T04:47:53.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.031+0000 7eff645ee740 1 -- start start 2024-09-19T04:47:53.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.031+0000 7eff645ee740 1 Processor -- start 2024-09-19T04:47:53.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.032+0000 7eff645ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 0x558f95c67d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.032+0000 7eff645ee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c60590 0x558f95c68290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.032+0000 7eff645ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c6ae50 0x558f95c687d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.032+0000 7eff645ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558f95c6da20 con 0x558f95c5fc70 2024-09-19T04:47:53.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.032+0000 7eff645ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558f95b810f0 con 0x558f95c6ae50 2024-09-19T04:47:53.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.032+0000 7eff645ee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558f95c6dba0 con 0x558f95c60590 2024-09-19T04:47:53.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.032+0000 7eff645ee740 10 client.0 _ll_register_callbacks cb 0x558f95b68bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:53.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.032+0000 7eff5aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c60590 0x558f95c68290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:53.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.032+0000 7eff5aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c60590 0x558f95c68290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:55106/0 (socket says 172.21.15.123:55106) 2024-09-19T04:47:53.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.032+0000 7eff5aa00640 1 -- 172.21.15.123:0/1459801552 learned_addr learned my addr 172.21.15.123:0/1459801552 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:53.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.032+0000 7eff5b400640 1 --2- 172.21.15.123:0/1459801552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 0x558f95c67d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:53.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.033+0000 7eff5a000640 1 --2- 172.21.15.123:0/1459801552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c6ae50 0x558f95c687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:53.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.033+0000 7eff5b400640 1 -- 172.21.15.123:0/1459801552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c60590 msgr2=0x558f95c68290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:53.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.033+0000 7eff5b400640 1 --2- 172.21.15.123:0/1459801552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c60590 0x558f95c68290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.033+0000 7eff5b400640 1 -- 172.21.15.123:0/1459801552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c6ae50 msgr2=0x558f95c687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:53.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.033+0000 7eff5b400640 1 --2- 172.21.15.123:0/1459801552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c6ae50 0x558f95c687d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.033+0000 7eff5b400640 1 -- 172.21.15.123:0/1459801552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f959373c0 con 0x558f95c5fc70 2024-09-19T04:47:53.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.033+0000 7eff5b400640 1 --2- 172.21.15.123:0/1459801552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 0x558f95c67d50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7eff54010ca0 tx=0x7eff54005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcc800c8ea4bd76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:53.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.034+0000 7eff44c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:47:53.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.034+0000 7eff44c00640 1 -- 172.21.15.123:0/1459801552 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5402b060 con 0x558f95c5fc70 2024-09-19T04:47:53.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.034+0000 7eff44c00640 1 -- 172.21.15.123:0/1459801552 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7eff54011440 con 0x558f95c5fc70 2024-09-19T04:47:53.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.034+0000 7eff53400640 1 --2- 172.21.15.123:0/1196912474 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7eff400a34f0 0x7eff400a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.034+0000 7eff645ee740 1 -- 172.21.15.123:0/1459801552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff54011cc0 con 0x558f95c5fc70 2024-09-19T04:47:53.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.035+0000 7eff44c00640 1 -- 172.21.15.123:0/1459801552 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff54009bf0 con 0x558f95c5fc70 2024-09-19T04:47:53.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.035+0000 7eff645ee740 10 client.15483 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:53.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.035+0000 7eff645ee740 1 -- 172.21.15.123:0/1459801552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558f95c79b00 con 0x558f95c5fc70 2024-09-19T04:47:53.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.035+0000 7eff53400640 1 -- 172.21.15.123:0/1196912474 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7eff480f8280 con 0x7eff5c1027a0 2024-09-19T04:47:53.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.035+0000 7eff44c00640 1 -- 172.21.15.123:0/1459801552 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff54011cc0 con 0x558f95c5fc70 2024-09-19T04:47:53.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.036+0000 7eff5aa00640 1 --2- 172.21.15.123:0/1196912474 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7eff400a34f0 0x7eff400a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:53.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.036+0000 7eff44c00640 1 -- 172.21.15.123:0/1459801552 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7eff54005740 con 0x558f95c5fc70 2024-09-19T04:47:53.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.036+0000 7eff645ee740 10 client.15483 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:53.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.036+0000 7eff645ee740 10 client.15483 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:47:53.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.036+0000 7eff645ee740 20 client.15483 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:53.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.036+0000 7eff645ee740 1 --2- 172.21.15.123:0/1459801552 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x558f95c7af10 0x558f95c7d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.036+0000 7eff645ee740 4 client.15483 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:47:53.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.036+0000 7eff645ee740 1 -- 172.21.15.123:0/1459801552 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f95bd3950 con 0x558f95c7af10 2024-09-19T04:47:53.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.036+0000 7eff5a000640 1 --2- 172.21.15.123:0/1459801552 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x558f95c7af10 0x558f95c7d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:53.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.037+0000 7eff44c00640 10 client.15483 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:47:53.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.037+0000 7eff5a000640 1 --2- 172.21.15.123:0/1459801552 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x558f95c7af10 0x558f95c7d300 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f83443ac7dc4c08 server_cookie=6d2212da7b80fad9 in_seq=0 out_seq=0 2024-09-19T04:47:53.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.038+0000 7eff5aa00640 1 --2- 172.21.15.123:0/1196912474 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7eff400a34f0 0x7eff400a59b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x558f95c69300 tx=0x7eff4c009920 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:53.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.038+0000 7eff44c00640 1 -- 172.21.15.123:0/1459801552 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f95bd3950 con 0x558f95c7af10 2024-09-19T04:47:53.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.038+0000 7eff44c00640 10 client.15483 handle_command_reply: tid=0 2024-09-19T04:47:53.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.039+0000 7eff645ee740 1 client.15483 shutdown 2024-09-19T04:47:53.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.039+0000 7eff645ee740 1 -- 172.21.15.123:0/1459801552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 msgr2=0x558f95c67d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:53.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.039+0000 7eff645ee740 1 --2- 172.21.15.123:0/1459801552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 0x558f95c67d50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7eff54010ca0 tx=0x7eff54005c80 comp rx=0 tx=0).stop 2024-09-19T04:47:53.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.039+0000 7eff645ee740 1 -- 172.21.15.123:0/1459801552 shutdown_connections 2024-09-19T04:47:53.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.039+0000 7eff645ee740 1 --2- 172.21.15.123:0/1459801552 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x558f95c7af10 0x558f95c7d300 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.039+0000 7eff645ee740 1 --2- 172.21.15.123:0/1459801552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c6ae50 0x558f95c687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.039+0000 7eff645ee740 1 --2- 172.21.15.123:0/1459801552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c60590 0x558f95c68290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.039+0000 7eff645ee740 1 --2- 172.21.15.123:0/1459801552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 0x558f95c67d50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.039+0000 7eff645ee740 1 -- 172.21.15.123:0/1459801552 >> 172.21.15.123:0/1459801552 conn(0x558f95c5c200 msgr2=0x558f95c61f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:53.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.039+0000 7eff44c00640 0 client.15483 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:47:53.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.040+0000 7eff645ee740 1 -- 172.21.15.123:0/1459801552 shutdown_connections 2024-09-19T04:47:53.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.040+0000 7eff645ee740 1 -- 172.21.15.123:0/1459801552 wait complete. 2024-09-19T04:47:53.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.040+0000 7eff645ee740 20 client.15483 trim_cache size 0 max 16384 2024-09-19T04:47:53.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.066+0000 7eff645ee740 1 Processor -- start 2024-09-19T04:47:53.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.066+0000 7eff645ee740 1 -- start start 2024-09-19T04:47:53.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.066+0000 7eff645ee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c5fc70 0x558f95c62800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.066+0000 7eff645ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c60590 0x558f95c828a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.066+0000 7eff645ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c6ae50 0x558f95c82de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.066+0000 7eff645ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c00e790 con 0x558f95c6ae50 2024-09-19T04:47:53.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.066+0000 7eff645ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c0102d0 con 0x558f95c60590 2024-09-19T04:47:53.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.066+0000 7eff645ee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c00ee60 con 0x558f95c5fc70 2024-09-19T04:47:53.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.067+0000 7eff5a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c60590 0x558f95c828a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:53.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.067+0000 7eff5a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c60590 0x558f95c828a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:43742/0 (socket says 172.21.15.123:43742) 2024-09-19T04:47:53.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.067+0000 7eff5b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c6ae50 0x558f95c82de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:53.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.067+0000 7eff5a000640 1 -- 172.21.15.123:0/990623564 learned_addr learned my addr 172.21.15.123:0/990623564 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:53.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.067+0000 7eff5aa00640 1 --2- 172.21.15.123:0/990623564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c5fc70 0x558f95c62800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:53.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.067+0000 7eff5a000640 1 -- 172.21.15.123:0/990623564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c5fc70 msgr2=0x558f95c62800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:53.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.067+0000 7eff5a000640 1 --2- 172.21.15.123:0/990623564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c5fc70 0x558f95c62800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.067+0000 7eff5a000640 1 -- 172.21.15.123:0/990623564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c6ae50 msgr2=0x558f95c82de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:53.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.067+0000 7eff5a000640 1 --2- 172.21.15.123:0/990623564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c6ae50 0x558f95c82de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.067+0000 7eff5a000640 1 -- 172.21.15.123:0/990623564 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f95c66d20 con 0x558f95c60590 2024-09-19T04:47:53.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.067+0000 7eff5b400640 1 --2- 172.21.15.123:0/990623564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c6ae50 0x558f95c82de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:53.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.067+0000 7eff5aa00640 1 --2- 172.21.15.123:0/990623564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c5fc70 0x558f95c62800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:47:53.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.068+0000 7eff5a000640 1 --2- 172.21.15.123:0/990623564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c60590 0x558f95c828a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7eff480befd0 tx=0x7eff480fa6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f3606ca0b1afb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:53.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.068+0000 7eff47400640 1 -- 172.21.15.123:0/990623564 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff480c4460 con 0x558f95c60590 2024-09-19T04:47:53.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.069+0000 7eff47400640 1 -- 172.21.15.123:0/990623564 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff480c8810 con 0x558f95c60590 2024-09-19T04:47:53.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.069+0000 7eff47400640 1 -- 172.21.15.123:0/990623564 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff480c46c0 con 0x558f95c60590 2024-09-19T04:47:53.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.069+0000 7eff645ee740 1 -- 172.21.15.123:0/990623564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c60590 msgr2=0x558f95c828a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:53.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.069+0000 7eff645ee740 1 --2- 172.21.15.123:0/990623564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c60590 0x558f95c828a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7eff480befd0 tx=0x7eff480fa6d0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.070+0000 7eff645ee740 1 -- 172.21.15.123:0/990623564 shutdown_connections 2024-09-19T04:47:53.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.070+0000 7eff645ee740 1 --2- 172.21.15.123:0/990623564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c6ae50 0x558f95c82de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.070+0000 7eff645ee740 1 --2- 172.21.15.123:0/990623564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c60590 0x558f95c828a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.070+0000 7eff645ee740 1 --2- 172.21.15.123:0/990623564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c5fc70 0x558f95c62800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.070+0000 7eff645ee740 1 -- 172.21.15.123:0/990623564 >> 172.21.15.123:0/990623564 conn(0x558f95c5c200 msgr2=0x558f95c61f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:53.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.070+0000 7eff645ee740 1 -- 172.21.15.123:0/990623564 shutdown_connections 2024-09-19T04:47:53.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.070+0000 7eff645ee740 1 -- 172.21.15.123:0/990623564 wait complete. 2024-09-19T04:47:53.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.070+0000 7eff645ee740 1 -- start start 2024-09-19T04:47:53.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.070+0000 7eff645ee740 1 Processor -- start 2024-09-19T04:47:53.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.071+0000 7eff645ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 0x558f95c72e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.071+0000 7eff645ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c6ae50 0x558f95c733c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.071+0000 7eff645ee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c73900 0x558f95c6fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.071+0000 7eff645ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c00fc00 con 0x558f95c5fc70 2024-09-19T04:47:53.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.071+0000 7eff645ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c00f530 con 0x558f95c6ae50 2024-09-19T04:47:53.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.071+0000 7eff645ee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c00c770 con 0x558f95c73900 2024-09-19T04:47:53.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.071+0000 7eff645ee740 10 client.0 _ll_register_callbacks cb 0x7eff54009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:53.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.071+0000 7eff5aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 0x558f95c72e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:53.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.071+0000 7eff5aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 0x558f95c72e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46400/0 (socket says 172.21.15.123:46400) 2024-09-19T04:47:53.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.071+0000 7eff5aa00640 1 -- 172.21.15.123:0/4211161458 learned_addr learned my addr 172.21.15.123:0/4211161458 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:53.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.072+0000 7eff5aa00640 1 -- 172.21.15.123:0/4211161458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c73900 msgr2=0x558f95c6fe50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:47:53.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.072+0000 7eff5aa00640 1 --2- 172.21.15.123:0/4211161458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c73900 0x558f95c6fe50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.072+0000 7eff5aa00640 1 -- 172.21.15.123:0/4211161458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c6ae50 msgr2=0x558f95c733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:53.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.072+0000 7eff5aa00640 1 --2- 172.21.15.123:0/4211161458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c6ae50 0x558f95c733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.072+0000 7eff5aa00640 1 -- 172.21.15.123:0/4211161458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f95c70680 con 0x558f95c5fc70 2024-09-19T04:47:53.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.072+0000 7eff5aa00640 1 --2- 172.21.15.123:0/4211161458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 0x558f95c72e80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7eff4c01c9a0 tx=0x7eff4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3043759ef8b111c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:53.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.073+0000 7eff44c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:47:53.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.073+0000 7eff44c00640 1 -- 172.21.15.123:0/4211161458 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c011e60 con 0x558f95c5fc70 2024-09-19T04:47:53.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.073+0000 7eff44c00640 1 -- 172.21.15.123:0/4211161458 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7eff4c00f5d0 con 0x558f95c5fc70 2024-09-19T04:47:53.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.073+0000 7eff645ee740 1 -- 172.21.15.123:0/4211161458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff480c8810 con 0x558f95c5fc70 2024-09-19T04:47:53.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.073+0000 7eff44c00640 1 -- 172.21.15.123:0/4211161458 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff4c034050 con 0x558f95c5fc70 2024-09-19T04:47:53.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.073+0000 7eff645ee740 10 client.15489 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:53.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.073+0000 7eff645ee740 1 -- 172.21.15.123:0/4211161458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558f95ce62f0 con 0x558f95c5fc70 2024-09-19T04:47:53.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.074+0000 7eff44c00640 1 -- 172.21.15.123:0/4211161458 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff480c8810 con 0x558f95c5fc70 2024-09-19T04:47:53.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.075+0000 7eff44c00640 1 -- 172.21.15.123:0/4211161458 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7eff4c026860 con 0x558f95c5fc70 2024-09-19T04:47:53.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.075+0000 7eff645ee740 10 client.15489 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:53.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.075+0000 7eff645ee740 10 client.15489 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:47:53.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.075+0000 7eff645ee740 20 client.15489 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:53.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.075+0000 7eff645ee740 1 --2- 172.21.15.123:0/4211161458 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x558f95ce6a60 0x558f95ce8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:53.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.075+0000 7eff645ee740 4 client.15489 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:47:53.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.075+0000 7eff645ee740 1 -- 172.21.15.123:0/4211161458 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f95bcde60 con 0x558f95ce6a60 2024-09-19T04:47:53.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.076+0000 7eff5a000640 1 --2- 172.21.15.123:0/4211161458 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x558f95ce6a60 0x558f95ce8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:53.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.076+0000 7eff44c00640 10 client.15489 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:47:53.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.076+0000 7eff5a000640 1 --2- 172.21.15.123:0/4211161458 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x558f95ce6a60 0x558f95ce8ee0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3326a79290ed1f9d server_cookie=54ffa18e6191fc7a in_seq=0 out_seq=0 2024-09-19T04:47:53.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.077+0000 7eff44c00640 1 -- 172.21.15.123:0/4211161458 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f95bcde60 con 0x558f95ce6a60 2024-09-19T04:47:53.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.077+0000 7eff44c00640 10 client.15489 handle_command_reply: tid=0 2024-09-19T04:47:53.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.077+0000 7eff645ee740 1 client.15489 shutdown 2024-09-19T04:47:53.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.077+0000 7eff645ee740 1 -- 172.21.15.123:0/4211161458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 msgr2=0x558f95c72e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:53.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.077+0000 7eff645ee740 1 --2- 172.21.15.123:0/4211161458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 0x558f95c72e80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7eff4c01c9a0 tx=0x7eff4c0046c0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.077+0000 7eff645ee740 1 -- 172.21.15.123:0/4211161458 shutdown_connections 2024-09-19T04:47:53.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.077+0000 7eff645ee740 1 --2- 172.21.15.123:0/4211161458 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x558f95ce6a60 0x558f95ce8ee0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.077+0000 7eff645ee740 1 --2- 172.21.15.123:0/4211161458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f95c73900 0x558f95c6fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.077+0000 7eff645ee740 1 --2- 172.21.15.123:0/4211161458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f95c6ae50 0x558f95c733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.077+0000 7eff645ee740 1 --2- 172.21.15.123:0/4211161458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f95c5fc70 0x558f95c72e80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.077+0000 7eff645ee740 1 -- 172.21.15.123:0/4211161458 >> 172.21.15.123:0/4211161458 conn(0x558f95c5c200 msgr2=0x558f95c5ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:53.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.077+0000 7eff44c00640 0 client.15489 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:47:53.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.078+0000 7eff645ee740 1 -- 172.21.15.123:0/4211161458 shutdown_connections 2024-09-19T04:47:53.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.078+0000 7eff645ee740 1 -- 172.21.15.123:0/4211161458 wait complete. 2024-09-19T04:47:53.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.079+0000 7eff645ee740 20 client.15489 trim_cache size 0 max 16384 2024-09-19T04:47:53.081 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:47:53.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.080+0000 7eff61c00640 1 -- 172.21.15.123:0/1196912474 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7eff400a34f0 msgr2=0x7eff400a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:53.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.080+0000 7eff61c00640 1 --2- 172.21.15.123:0/1196912474 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7eff400a34f0 0x7eff400a59b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x558f95c69300 tx=0x7eff4c009920 comp rx=0 tx=0).stop 2024-09-19T04:47:53.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.080+0000 7eff61c00640 1 -- 172.21.15.123:0/1196912474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff5c1027a0 msgr2=0x7eff5c19ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:53.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.080+0000 7eff61c00640 1 --2- 172.21.15.123:0/1196912474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff5c1027a0 0x7eff5c19ab90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7eff48035040 tx=0x7eff48033b60 comp rx=0 tx=0).stop 2024-09-19T04:47:53.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.080+0000 7eff61c00640 1 -- 172.21.15.123:0/1196912474 shutdown_connections 2024-09-19T04:47:53.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.080+0000 7eff61c00640 1 --2- 172.21.15.123:0/1196912474 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7eff400a34f0 0x7eff400a59b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.080+0000 7eff61c00640 1 --2- 172.21.15.123:0/1196912474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff5c106f20 0x7eff5c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.080+0000 7eff61c00640 1 --2- 172.21.15.123:0/1196912474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff5c1027a0 0x7eff5c19ab90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.080+0000 7eff61c00640 1 --2- 172.21.15.123:0/1196912474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff5c101df0 0x7eff5c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:53.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.080+0000 7eff61c00640 1 -- 172.21.15.123:0/1196912474 >> 172.21.15.123:0/1196912474 conn(0x7eff5c0fd8b0 msgr2=0x7eff5c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:53.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.081+0000 7eff61c00640 1 -- 172.21.15.123:0/1196912474 shutdown_connections 2024-09-19T04:47:53.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:53.081+0000 7eff61c00640 1 -- 172.21.15.123:0/1196912474 wait complete. 2024-09-19T04:47:53.650 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:53 smithi123 ceph-mon[30511]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:47:53.650 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:53 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:47:53.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:53 smithi193 ceph-mon[35592]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:47:53.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:53 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:47:53.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:53 smithi183 ceph-mon[35333]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:47:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:53 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:47:54.089 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:47:54.515 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:47:54.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:54 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:54.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:54 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:54.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:54 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:54.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:54 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:54.841 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:54 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:54.842 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:54 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:54 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:54 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:54 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:54 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:54 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:54.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:54 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:54.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:54 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:54.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:54 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:54.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:54 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:54.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:54 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:54.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:54 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:54.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:54 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:55.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.169+0000 7f85d2600640 1 -- 172.21.15.123:0/2066063719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc077040 msgr2=0x7f85cc1124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:55.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.169+0000 7f85d2600640 1 --2- 172.21.15.123:0/2066063719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc077040 0x7f85cc1124f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009900 tx=0x7f85b802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.169+0000 7f85d2600640 1 -- 172.21.15.123:0/2066063719 shutdown_connections 2024-09-19T04:47:55.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.169+0000 7f85d2600640 1 --2- 172.21.15.123:0/2066063719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85cc112a30 0x7f85cc114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.169+0000 7f85d2600640 1 --2- 172.21.15.123:0/2066063719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc077040 0x7f85cc1124f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.169+0000 7f85d2600640 1 --2- 172.21.15.123:0/2066063719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85cc076720 0x7f85cc076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.169+0000 7f85d2600640 1 -- 172.21.15.123:0/2066063719 >> 172.21.15.123:0/2066063719 conn(0x7f85cc0ffa90 msgr2=0x7f85cc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:55.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.169+0000 7f85d2600640 1 -- 172.21.15.123:0/2066063719 shutdown_connections 2024-09-19T04:47:55.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.169+0000 7f85d2600640 1 -- 172.21.15.123:0/2066063719 wait complete. 2024-09-19T04:47:55.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.170+0000 7f85d2600640 1 Processor -- start 2024-09-19T04:47:55.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.170+0000 7f85d2600640 1 -- start start 2024-09-19T04:47:55.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.171+0000 7f85d2600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85cc076720 0x7f85cc1a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.171+0000 7f85d2600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85cc077040 0x7f85cc1a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.171+0000 7f85d2600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc112a30 0x7f85cc19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.171+0000 7f85d2600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc1185f0 con 0x7f85cc076720 2024-09-19T04:47:55.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.171+0000 7f85d2600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc118470 con 0x7f85cc112a30 2024-09-19T04:47:55.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.171+0000 7f85d2600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc118770 con 0x7f85cc077040 2024-09-19T04:47:55.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.171+0000 7f85caa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85cc077040 0x7f85cc1a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:55.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.171+0000 7f85caa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85cc077040 0x7f85cc1a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:55146/0 (socket says 172.21.15.123:55146) 2024-09-19T04:47:55.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.171+0000 7f85caa00640 1 -- 172.21.15.123:0/1200986576 learned_addr learned my addr 172.21.15.123:0/1200986576 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:55.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.171+0000 7f85cb400640 1 --2- 172.21.15.123:0/1200986576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85cc076720 0x7f85cc1a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:55.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.172+0000 7f85cbe00640 1 --2- 172.21.15.123:0/1200986576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc112a30 0x7f85cc19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.172+0000 7f85cb400640 1 -- 172.21.15.123:0/1200986576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85cc077040 msgr2=0x7f85cc1a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.172+0000 7f85cb400640 1 --2- 172.21.15.123:0/1200986576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85cc077040 0x7f85cc1a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.172+0000 7f85cb400640 1 -- 172.21.15.123:0/1200986576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc112a30 msgr2=0x7f85cc19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.172+0000 7f85cb400640 1 --2- 172.21.15.123:0/1200986576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc112a30 0x7f85cc19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.172+0000 7f85cb400640 1 -- 172.21.15.123:0/1200986576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85cc19efd0 con 0x7f85cc076720 2024-09-19T04:47:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.172+0000 7f85cbe00640 1 --2- 172.21.15.123:0/1200986576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc112a30 0x7f85cc19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:47:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.172+0000 7f85cb400640 1 --2- 172.21.15.123:0/1200986576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85cc076720 0x7f85cc1a46e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00e9f0 tx=0x7f85bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.173+0000 7f85c3e00640 1 -- 172.21.15.123:0/1200986576 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc00ccc0 con 0x7f85cc076720 2024-09-19T04:47:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.173+0000 7f85d2600640 1 -- 172.21.15.123:0/1200986576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85cc19f2c0 con 0x7f85cc076720 2024-09-19T04:47:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.173+0000 7f85d2600640 1 -- 172.21.15.123:0/1200986576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85cc1ab5b0 con 0x7f85cc076720 2024-09-19T04:47:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.173+0000 7f85c3e00640 1 -- 172.21.15.123:0/1200986576 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85bc010910 con 0x7f85cc076720 2024-09-19T04:47:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.173+0000 7f85c3e00640 1 -- 172.21.15.123:0/1200986576 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc026060 con 0x7f85cc076720 2024-09-19T04:47:55.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.178+0000 7f85c3e00640 1 -- 172.21.15.123:0/1200986576 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7f85bc010ac0 con 0x7f85cc076720 2024-09-19T04:47:55.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.178+0000 7f85d5055740 1 Processor -- start 2024-09-19T04:47:55.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.178+0000 7f85d5055740 1 -- start start 2024-09-19T04:47:55.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.178+0000 7f85d5055740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7ec70 0x55d09dc7f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.178+0000 7f85d5055740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc7f590 0x55d09dc89910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.179+0000 7f85d5055740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc89e50 0x55d09dc8c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.179+0000 7f85d5055740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d09dbe53e0 con 0x55d09dc7f590 2024-09-19T04:47:55.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.179+0000 7f85d5055740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d09db8cd80 con 0x55d09dc7ec70 2024-09-19T04:47:55.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.179+0000 7f85d5055740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d09dbe13a0 con 0x55d09dc89e50 2024-09-19T04:47:55.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.179+0000 7f85cbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7ec70 0x55d09dc7f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:55.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.179+0000 7f85caa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc89e50 0x55d09dc8c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:55.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.179+0000 7f85cbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7ec70 0x55d09dc7f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:43778/0 (socket says 172.21.15.123:43778) 2024-09-19T04:47:55.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.179+0000 7f85cbe00640 1 -- 172.21.15.123:0/1158743462 learned_addr learned my addr 172.21.15.123:0/1158743462 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.179+0000 7f85caa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc89e50 0x55d09dc8c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:55156/0 (socket says 172.21.15.123:55156) 2024-09-19T04:47:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.179+0000 7f85cb400640 1 --2- 172.21.15.123:0/1158743462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc7f590 0x55d09dc89910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.179+0000 7f85caa00640 1 -- 172.21.15.123:0/1158743462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7ec70 msgr2=0x55d09dc7f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.179+0000 7f85caa00640 1 --2- 172.21.15.123:0/1158743462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7ec70 0x55d09dc7f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.179+0000 7f85caa00640 1 -- 172.21.15.123:0/1158743462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc7f590 msgr2=0x55d09dc89910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.179+0000 7f85caa00640 1 --2- 172.21.15.123:0/1158743462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc7f590 0x55d09dc89910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.179+0000 7f85caa00640 1 -- 172.21.15.123:0/1158743462 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d09dbabae0 con 0x55d09dc89e50 2024-09-19T04:47:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.179+0000 7f85cb400640 1 --2- 172.21.15.123:0/1158743462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc7f590 0x55d09dc89910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:47:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.180+0000 7f85c3e00640 1 --2- 172.21.15.123:0/1200986576 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85b00a3660 0x7f85b00a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.180+0000 7f85c3e00640 1 -- 172.21.15.123:0/1200986576 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f85bc0d0280 con 0x7f85cc076720 2024-09-19T04:47:55.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.180+0000 7f85cbe00640 1 --2- 172.21.15.123:0/1200986576 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85b00a3660 0x7f85b00a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:55.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.180+0000 7f85caa00640 1 --2- 172.21.15.123:0/1158743462 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc89e50 0x55d09dc8c240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f85b8000c00 tx=0x7f85b8005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee8ad29d7deb40d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:55.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.181+0000 7f85c0c00640 1 -- 172.21.15.123:0/1158743462 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8040070 con 0x55d09dc89e50 2024-09-19T04:47:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.181+0000 7f85c0c00640 1 -- 172.21.15.123:0/1158743462 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85b8005500 con 0x55d09dc89e50 2024-09-19T04:47:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.181+0000 7f85c0c00640 1 -- 172.21.15.123:0/1158743462 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8035040 con 0x55d09dc89e50 2024-09-19T04:47:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.182+0000 7f85d5055740 1 -- 172.21.15.123:0/1158743462 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc89e50 msgr2=0x55d09dc8c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.182+0000 7f85cbe00640 1 --2- 172.21.15.123:0/1200986576 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85b00a3660 0x7f85b00a5b20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009fd0 tx=0x7f85c4009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.182+0000 7f85d5055740 1 --2- 172.21.15.123:0/1158743462 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc89e50 0x55d09dc8c240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f85b8000c00 tx=0x7f85b8005c00 comp rx=0 tx=0).stop 2024-09-19T04:47:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.182+0000 7f85d5055740 1 -- 172.21.15.123:0/1158743462 shutdown_connections 2024-09-19T04:47:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.182+0000 7f85d5055740 1 --2- 172.21.15.123:0/1158743462 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc89e50 0x55d09dc8c240 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.182+0000 7f85d5055740 1 --2- 172.21.15.123:0/1158743462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc7f590 0x55d09dc89910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.182+0000 7f85d5055740 1 --2- 172.21.15.123:0/1158743462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7ec70 0x55d09dc7f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.182+0000 7f85d5055740 1 -- 172.21.15.123:0/1158743462 >> 172.21.15.123:0/1158743462 conn(0x55d09dc7b200 msgr2=0x55d09dc7b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.182+0000 7f85d5055740 1 -- 172.21.15.123:0/1158743462 shutdown_connections 2024-09-19T04:47:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.182+0000 7f85d5055740 1 -- 172.21.15.123:0/1158743462 wait complete. 2024-09-19T04:47:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.183+0000 7f85d5055740 1 -- start start 2024-09-19T04:47:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.183+0000 7f85d5055740 1 Processor -- start 2024-09-19T04:47:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.184+0000 7f85d5055740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc7ec70 0x55d09dc86d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.184+0000 7f85d5055740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7f590 0x55d09dc87270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.184+0000 7f85d5055740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc89e50 0x55d09dc877b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.184+0000 7f85d5055740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d09dc8ca20 con 0x55d09dc89e50 2024-09-19T04:47:55.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.184+0000 7f85d5055740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d09dba00f0 con 0x55d09dc7f590 2024-09-19T04:47:55.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.184+0000 7f85d5055740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d09dc8cba0 con 0x55d09dc7ec70 2024-09-19T04:47:55.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.184+0000 7f85d5055740 10 client.0 _ll_register_callbacks cb 0x55d09db87bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:55.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.184+0000 7f85cb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc7ec70 0x55d09dc86d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:55.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.184+0000 7f85cb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc7ec70 0x55d09dc86d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:55170/0 (socket says 172.21.15.123:55170) 2024-09-19T04:47:55.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.184+0000 7f85cb400640 1 -- 172.21.15.123:0/3032818811 learned_addr learned my addr 172.21.15.123:0/3032818811 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:55.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.184+0000 7f85caa00640 1 --2- 172.21.15.123:0/3032818811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7f590 0x55d09dc87270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:55.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.184+0000 7f85cbe00640 1 --2- 172.21.15.123:0/3032818811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc89e50 0x55d09dc877b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:55.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.185+0000 7f85caa00640 1 -- 172.21.15.123:0/3032818811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc7ec70 msgr2=0x55d09dc86d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:55.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.185+0000 7f85caa00640 1 --2- 172.21.15.123:0/3032818811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc7ec70 0x55d09dc86d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.185+0000 7f85caa00640 1 -- 172.21.15.123:0/3032818811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc89e50 msgr2=0x55d09dc877b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:55.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.185+0000 7f85caa00640 1 --2- 172.21.15.123:0/3032818811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc89e50 0x55d09dc877b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.185+0000 7f85caa00640 1 -- 172.21.15.123:0/3032818811 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d09d9563c0 con 0x55d09dc7f590 2024-09-19T04:47:55.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.185+0000 7f85cb400640 1 --2- 172.21.15.123:0/3032818811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc7ec70 0x55d09dc86d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:55.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.185+0000 7f85cbe00640 1 --2- 172.21.15.123:0/3032818811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc89e50 0x55d09dc877b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:55.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.185+0000 7f85caa00640 1 --2- 172.21.15.123:0/3032818811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7f590 0x55d09dc87270 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f85b8031550 tx=0x7f85b8004f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ee3d53c03f9b8ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:55.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.185+0000 7f85b5600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:47:55.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.186+0000 7f85b5600640 1 -- 172.21.15.123:0/3032818811 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8040070 con 0x55d09dc7f590 2024-09-19T04:47:55.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.186+0000 7f85b5600640 1 -- 172.21.15.123:0/3032818811 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f85b8004380 con 0x55d09dc7f590 2024-09-19T04:47:55.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.186+0000 7f85d5055740 1 -- 172.21.15.123:0/3032818811 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85b8005500 con 0x55d09dc7f590 2024-09-19T04:47:55.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.186+0000 7f85b5600640 1 -- 172.21.15.123:0/3032818811 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f85b8056050 con 0x55d09dc7f590 2024-09-19T04:47:55.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.186+0000 7f85d5055740 10 client.25532 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:55.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.186+0000 7f85d5055740 1 -- 172.21.15.123:0/3032818811 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d09dc98a00 con 0x55d09dc7f590 2024-09-19T04:47:55.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.186+0000 7f85b5600640 1 -- 172.21.15.123:0/3032818811 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85b8005500 con 0x55d09dc7f590 2024-09-19T04:47:55.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.187+0000 7f85b5600640 1 -- 172.21.15.123:0/3032818811 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f85b8031b60 con 0x55d09dc7f590 2024-09-19T04:47:55.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.187+0000 7f85d5055740 10 client.25532 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:55.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.187+0000 7f85d5055740 10 client.25532 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:47:55.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.187+0000 7f85d5055740 20 client.25532 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:55.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.187+0000 7f85d5055740 1 --2- 172.21.15.123:0/3032818811 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d09dc99d50 0x55d09dc9c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.187+0000 7f85d5055740 4 client.25532 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:47:55.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.187+0000 7f85d5055740 1 -- 172.21.15.123:0/3032818811 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d09dbf2950 con 0x55d09dc99d50 2024-09-19T04:47:55.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.187+0000 7f85cb400640 1 --2- 172.21.15.123:0/3032818811 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d09dc99d50 0x55d09dc9c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:55.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.188+0000 7f85cb400640 1 --2- 172.21.15.123:0/3032818811 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d09dc99d50 0x55d09dc9c140 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7322a87b043d62e6 server_cookie=7d2874ade1688cec in_seq=0 out_seq=0 2024-09-19T04:47:55.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.188+0000 7f85b5600640 10 client.25532 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:47:55.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.188+0000 7f85b5600640 1 -- 172.21.15.123:0/3032818811 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d09dbf2950 con 0x55d09dc99d50 2024-09-19T04:47:55.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.188+0000 7f85b5600640 10 client.25532 handle_command_reply: tid=0 2024-09-19T04:47:55.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.188+0000 7f85d5055740 1 client.25532 shutdown 2024-09-19T04:47:55.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.189+0000 7f85d5055740 1 -- 172.21.15.123:0/3032818811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7f590 msgr2=0x55d09dc87270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:55.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.189+0000 7f85d5055740 1 --2- 172.21.15.123:0/3032818811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7f590 0x55d09dc87270 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f85b8031550 tx=0x7f85b8004f70 comp rx=0 tx=0).stop 2024-09-19T04:47:55.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.189+0000 7f85d5055740 1 -- 172.21.15.123:0/3032818811 shutdown_connections 2024-09-19T04:47:55.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.189+0000 7f85d5055740 1 --2- 172.21.15.123:0/3032818811 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d09dc99d50 0x55d09dc9c140 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.189+0000 7f85d5055740 1 --2- 172.21.15.123:0/3032818811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc89e50 0x55d09dc877b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.189+0000 7f85d5055740 1 --2- 172.21.15.123:0/3032818811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7f590 0x55d09dc87270 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.189+0000 7f85d5055740 1 --2- 172.21.15.123:0/3032818811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc7ec70 0x55d09dc86d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.189+0000 7f85d5055740 1 -- 172.21.15.123:0/3032818811 >> 172.21.15.123:0/3032818811 conn(0x55d09dc7b200 msgr2=0x55d09dc80f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:55.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.189+0000 7f85b5600640 0 client.25532 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:47:55.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.189+0000 7f85d5055740 1 -- 172.21.15.123:0/3032818811 shutdown_connections 2024-09-19T04:47:55.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.189+0000 7f85d5055740 1 -- 172.21.15.123:0/3032818811 wait complete. 2024-09-19T04:47:55.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.189+0000 7f85d5055740 20 client.25532 trim_cache size 0 max 16384 2024-09-19T04:47:55.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.203+0000 7f85d5055740 1 Processor -- start 2024-09-19T04:47:55.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.203+0000 7f85d5055740 1 -- start start 2024-09-19T04:47:55.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.204+0000 7f85d5055740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc7ec70 0x55d09dc88ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.204+0000 7f85d5055740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7f590 0x55d09dc9f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.204+0000 7f85d5055740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc89e50 0x55d09dc9f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.204+0000 7f85d5055740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac00e700 con 0x55d09dc89e50 2024-09-19T04:47:55.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.204+0000 7f85d5055740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac010240 con 0x55d09dc7f590 2024-09-19T04:47:55.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.204+0000 7f85d5055740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac00edd0 con 0x55d09dc7ec70 2024-09-19T04:47:55.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.204+0000 7f85cb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc7ec70 0x55d09dc88ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:55.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.204+0000 7f85cb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc7ec70 0x55d09dc88ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:55174/0 (socket says 172.21.15.123:55174) 2024-09-19T04:47:55.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.204+0000 7f85cb400640 1 -- 172.21.15.123:0/387529326 learned_addr learned my addr 172.21.15.123:0/387529326 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:55.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.204+0000 7f85caa00640 1 --2- 172.21.15.123:0/387529326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7f590 0x55d09dc9f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:55.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.204+0000 7f85cbe00640 1 --2- 172.21.15.123:0/387529326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc89e50 0x55d09dc9f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:55.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.205+0000 7f85caa00640 1 -- 172.21.15.123:0/387529326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc7ec70 msgr2=0x55d09dc88ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:55.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.205+0000 7f85caa00640 1 --2- 172.21.15.123:0/387529326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc7ec70 0x55d09dc88ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.205+0000 7f85caa00640 1 -- 172.21.15.123:0/387529326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc89e50 msgr2=0x55d09dc9f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:55.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.205+0000 7f85caa00640 1 --2- 172.21.15.123:0/387529326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc89e50 0x55d09dc9f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.205+0000 7f85caa00640 1 -- 172.21.15.123:0/387529326 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d09dc8caa0 con 0x55d09dc7f590 2024-09-19T04:47:55.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.205+0000 7f85caa00640 1 --2- 172.21.15.123:0/387529326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7f590 0x55d09dc9f160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f85b8031f30 tx=0x7f85b8005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa38bd035d22bd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:55.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.205+0000 7f85b7e00640 1 -- 172.21.15.123:0/387529326 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b805f070 con 0x55d09dc7f590 2024-09-19T04:47:55.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.206+0000 7f85b7e00640 1 -- 172.21.15.123:0/387529326 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85b804d070 con 0x55d09dc7f590 2024-09-19T04:47:55.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.206+0000 7f85b7e00640 1 -- 172.21.15.123:0/387529326 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8040040 con 0x55d09dc7f590 2024-09-19T04:47:55.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.206+0000 7f85d5055740 1 -- 172.21.15.123:0/387529326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7f590 msgr2=0x55d09dc9f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:55.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.206+0000 7f85d5055740 1 --2- 172.21.15.123:0/387529326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7f590 0x55d09dc9f160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f85b8031f30 tx=0x7f85b8005c00 comp rx=0 tx=0).stop 2024-09-19T04:47:55.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.206+0000 7f85d5055740 1 -- 172.21.15.123:0/387529326 shutdown_connections 2024-09-19T04:47:55.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.206+0000 7f85d5055740 1 --2- 172.21.15.123:0/387529326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc89e50 0x55d09dc9f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.206+0000 7f85d5055740 1 --2- 172.21.15.123:0/387529326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7f590 0x55d09dc9f160 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.206+0000 7f85d5055740 1 --2- 172.21.15.123:0/387529326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc7ec70 0x55d09dc88ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.206+0000 7f85d5055740 1 -- 172.21.15.123:0/387529326 >> 172.21.15.123:0/387529326 conn(0x55d09dc7b200 msgr2=0x55d09dc88590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:55.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.207+0000 7f85d5055740 1 -- 172.21.15.123:0/387529326 shutdown_connections 2024-09-19T04:47:55.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.207+0000 7f85d5055740 1 -- 172.21.15.123:0/387529326 wait complete. 2024-09-19T04:47:55.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.207+0000 7f85d5055740 1 -- start start 2024-09-19T04:47:55.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.207+0000 7f85d5055740 1 Processor -- start 2024-09-19T04:47:55.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.208+0000 7f85d5055740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc7ec70 0x55d09dc9d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.208+0000 7f85d5055740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7f590 0x55d09dc9d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.208+0000 7f85d5055740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc89e50 0x55d09dd03690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.208+0000 7f85d5055740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac00fb70 con 0x55d09dc7ec70 2024-09-19T04:47:55.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.208+0000 7f85d5055740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac00f4a0 con 0x55d09dc7f590 2024-09-19T04:47:55.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.208+0000 7f85d5055740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac00c6e0 con 0x55d09dc89e50 2024-09-19T04:47:55.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.208+0000 7f85d5055740 10 client.0 _ll_register_callbacks cb 0x7f85c4009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:55.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.208+0000 7f85cb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc7ec70 0x55d09dc9d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:55.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.208+0000 7f85cb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc7ec70 0x55d09dc9d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46454/0 (socket says 172.21.15.123:46454) 2024-09-19T04:47:55.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.208+0000 7f85cb400640 1 -- 172.21.15.123:0/3551464560 learned_addr learned my addr 172.21.15.123:0/3551464560 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:55.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.209+0000 7f85caa00640 1 --2- 172.21.15.123:0/3551464560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7f590 0x55d09dc9d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:55.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.209+0000 7f85cb400640 1 -- 172.21.15.123:0/3551464560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc89e50 msgr2=0x55d09dd03690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:47:55.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.209+0000 7f85cb400640 1 --2- 172.21.15.123:0/3551464560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc89e50 0x55d09dd03690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.209+0000 7f85cb400640 1 -- 172.21.15.123:0/3551464560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7f590 msgr2=0x55d09dc9d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:55.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.209+0000 7f85cb400640 1 --2- 172.21.15.123:0/3551464560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7f590 0x55d09dc9d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.209+0000 7f85cb400640 1 -- 172.21.15.123:0/3551464560 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d09dc7b030 con 0x55d09dc7ec70 2024-09-19T04:47:55.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.209+0000 7f85cb400640 1 --2- 172.21.15.123:0/3551464560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc7ec70 0x55d09dc9d250 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f85bc098010 tx=0x7f85bc0baf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0f263626f15be7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:55.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.209+0000 7f85b5600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:47:55.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.210+0000 7f85b5600640 1 -- 172.21.15.123:0/3551464560 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc0a0700 con 0x55d09dc7ec70 2024-09-19T04:47:55.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.210+0000 7f85b5600640 1 -- 172.21.15.123:0/3551464560 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f85bc0b5290 con 0x55d09dc7ec70 2024-09-19T04:47:55.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.210+0000 7f85d5055740 1 -- 172.21.15.123:0/3551464560 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85b804d070 con 0x55d09dc7ec70 2024-09-19T04:47:55.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.210+0000 7f85b5600640 1 -- 172.21.15.123:0/3551464560 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f85bc0a4fc0 con 0x55d09dc7ec70 2024-09-19T04:47:55.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.210+0000 7f85d5055740 10 client.15504 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:55.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.210+0000 7f85d5055740 1 -- 172.21.15.123:0/3551464560 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d09dd04af0 con 0x55d09dc7ec70 2024-09-19T04:47:55.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.210+0000 7f85b5600640 1 -- 172.21.15.123:0/3551464560 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85b804d070 con 0x55d09dc7ec70 2024-09-19T04:47:55.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.211+0000 7f85b5600640 1 -- 172.21.15.123:0/3551464560 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f85bc0a01b0 con 0x55d09dc7ec70 2024-09-19T04:47:55.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.211+0000 7f85d5055740 10 client.15504 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:55.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.211+0000 7f85d5055740 10 client.15504 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:47:55.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.211+0000 7f85d5055740 20 client.15504 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:55.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.211+0000 7f85d5055740 1 --2- 172.21.15.123:0/3551464560 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d09dd051a0 0x55d09dd07600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:55.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.211+0000 7f85d5055740 4 client.15504 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:47:55.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.211+0000 7f85d5055740 1 -- 172.21.15.123:0/3551464560 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d09dbece60 con 0x55d09dd051a0 2024-09-19T04:47:55.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.211+0000 7f85caa00640 1 --2- 172.21.15.123:0/3551464560 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d09dd051a0 0x55d09dd07600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:55.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.212+0000 7f85caa00640 1 --2- 172.21.15.123:0/3551464560 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d09dd051a0 0x55d09dd07600 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d16e33826e3a1289 server_cookie=be6eeff550decdfe in_seq=0 out_seq=0 2024-09-19T04:47:55.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.212+0000 7f85b5600640 10 client.15504 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:47:55.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.212+0000 7f85b5600640 1 -- 172.21.15.123:0/3551464560 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d09dbece60 con 0x55d09dd051a0 2024-09-19T04:47:55.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.212+0000 7f85b5600640 10 client.15504 handle_command_reply: tid=0 2024-09-19T04:47:55.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.212+0000 7f85d5055740 1 client.15504 shutdown 2024-09-19T04:47:55.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.212+0000 7f85d5055740 1 -- 172.21.15.123:0/3551464560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc7ec70 msgr2=0x55d09dc9d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:55.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.212+0000 7f85d5055740 1 --2- 172.21.15.123:0/3551464560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc7ec70 0x55d09dc9d250 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f85bc098010 tx=0x7f85bc0baf90 comp rx=0 tx=0).stop 2024-09-19T04:47:55.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.213+0000 7f85d5055740 1 -- 172.21.15.123:0/3551464560 shutdown_connections 2024-09-19T04:47:55.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.213+0000 7f85d5055740 1 --2- 172.21.15.123:0/3551464560 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d09dd051a0 0x55d09dd07600 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.213+0000 7f85d5055740 1 --2- 172.21.15.123:0/3551464560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d09dc89e50 0x55d09dd03690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.213+0000 7f85d5055740 1 --2- 172.21.15.123:0/3551464560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d09dc7f590 0x55d09dc9d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.213+0000 7f85d5055740 1 --2- 172.21.15.123:0/3551464560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d09dc7ec70 0x55d09dc9d250 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.213+0000 7f85d5055740 1 -- 172.21.15.123:0/3551464560 >> 172.21.15.123:0/3551464560 conn(0x55d09dc7b200 msgr2=0x55d09dc87510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:55.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.213+0000 7f85b5600640 0 client.15504 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:47:55.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.213+0000 7f85d5055740 1 -- 172.21.15.123:0/3551464560 shutdown_connections 2024-09-19T04:47:55.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.213+0000 7f85d5055740 1 -- 172.21.15.123:0/3551464560 wait complete. 2024-09-19T04:47:55.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.213+0000 7f85d5055740 20 client.15504 trim_cache size 0 max 16384 2024-09-19T04:47:55.216 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:47:55.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.214+0000 7f85d2600640 1 -- 172.21.15.123:0/1200986576 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85b00a3660 msgr2=0x7f85b00a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:55.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.214+0000 7f85d2600640 1 --2- 172.21.15.123:0/1200986576 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85b00a3660 0x7f85b00a5b20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009fd0 tx=0x7f85c4009500 comp rx=0 tx=0).stop 2024-09-19T04:47:55.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.214+0000 7f85d2600640 1 -- 172.21.15.123:0/1200986576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85cc076720 msgr2=0x7f85cc1a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:55.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.214+0000 7f85d2600640 1 --2- 172.21.15.123:0/1200986576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85cc076720 0x7f85cc1a46e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00e9f0 tx=0x7f85bc00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.214+0000 7f85d2600640 1 -- 172.21.15.123:0/1200986576 shutdown_connections 2024-09-19T04:47:55.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.214+0000 7f85d2600640 1 --2- 172.21.15.123:0/1200986576 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85b00a3660 0x7f85b00a5b20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.214+0000 7f85d2600640 1 --2- 172.21.15.123:0/1200986576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc112a30 0x7f85cc19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.214+0000 7f85d2600640 1 --2- 172.21.15.123:0/1200986576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85cc077040 0x7f85cc1a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.214+0000 7f85d2600640 1 --2- 172.21.15.123:0/1200986576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85cc076720 0x7f85cc1a46e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:55.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.214+0000 7f85d2600640 1 -- 172.21.15.123:0/1200986576 >> 172.21.15.123:0/1200986576 conn(0x7f85cc0ffa90 msgr2=0x7f85cc101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:55.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.215+0000 7f85d2600640 1 -- 172.21.15.123:0/1200986576 shutdown_connections 2024-09-19T04:47:55.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:55.215+0000 7f85d2600640 1 -- 172.21.15.123:0/1200986576 wait complete. 2024-09-19T04:47:55.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:55 smithi183 ceph-mon[35333]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:47:55.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:55 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:47:55.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:55 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:47:55.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:55 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:55.892 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:55 smithi193 ceph-mon[35592]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:47:55.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:55 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:47:55.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:55 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:47:55.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:55 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:55.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:55 smithi123 ceph-mon[30511]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:47:55.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:55 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:47:55.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:55 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:47:55.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:55 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:47:56.052 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:47:56.645 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:47:57.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.275+0000 7f26cbe00640 1 -- 172.21.15.123:0/4176300708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26cc102360 msgr2=0x7f26cc10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:57.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.275+0000 7f26cbe00640 1 --2- 172.21.15.123:0/4176300708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26cc102360 0x7f26cc10e890 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009ca0 tx=0x7f26c402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.275+0000 7f26cbe00640 1 -- 172.21.15.123:0/4176300708 shutdown_connections 2024-09-19T04:47:57.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.275+0000 7f26cbe00640 1 --2- 172.21.15.123:0/4176300708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26cc102360 0x7f26cc10e890 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.275+0000 7f26cbe00640 1 --2- 172.21.15.123:0/4176300708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26cc1019c0 0x7f26cc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.275+0000 7f26cbe00640 1 --2- 172.21.15.123:0/4176300708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26cc1079c0 0x7f26cc107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.275+0000 7f26cbe00640 1 -- 172.21.15.123:0/4176300708 >> 172.21.15.123:0/4176300708 conn(0x7f26cc0fd740 msgr2=0x7f26cc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:57.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.275+0000 7f26cbe00640 1 -- 172.21.15.123:0/4176300708 shutdown_connections 2024-09-19T04:47:57.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.275+0000 7f26cbe00640 1 -- 172.21.15.123:0/4176300708 wait complete. 2024-09-19T04:47:57.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.276+0000 7f26cbe00640 1 Processor -- start 2024-09-19T04:47:57.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.276+0000 7f26cbe00640 1 -- start start 2024-09-19T04:47:57.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.277+0000 7f26cbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26cc1019c0 0x7f26cc1a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.277+0000 7f26cbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26cc102360 0x7f26cc1a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.277+0000 7f26cbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26cc1079c0 0x7f26cc10d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.277+0000 7f26cbe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc111e10 con 0x7f26cc1079c0 2024-09-19T04:47:57.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.277+0000 7f26cbe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc111c90 con 0x7f26cc102360 2024-09-19T04:47:57.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.277+0000 7f26cbe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc111f90 con 0x7f26cc1019c0 2024-09-19T04:47:57.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.277+0000 7f26ca000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26cc102360 0x7f26cc1a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.277+0000 7f26ca000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26cc102360 0x7f26cc1a4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:43836/0 (socket says 172.21.15.123:43836) 2024-09-19T04:47:57.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.277+0000 7f26ca000640 1 -- 172.21.15.123:0/2162178469 learned_addr learned my addr 172.21.15.123:0/2162178469 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:57.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.277+0000 7f26cb400640 1 --2- 172.21.15.123:0/2162178469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26cc1079c0 0x7f26cc10d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.278+0000 7f26caa00640 1 --2- 172.21.15.123:0/2162178469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26cc1019c0 0x7f26cc1a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.278+0000 7f26ca000640 1 -- 172.21.15.123:0/2162178469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26cc1019c0 msgr2=0x7f26cc1a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:57.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.278+0000 7f26ca000640 1 --2- 172.21.15.123:0/2162178469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26cc1019c0 0x7f26cc1a4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.278+0000 7f26ca000640 1 -- 172.21.15.123:0/2162178469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26cc1079c0 msgr2=0x7f26cc10d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.278+0000 7f26ca000640 1 --2- 172.21.15.123:0/2162178469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26cc1079c0 0x7f26cc10d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.278+0000 7f26ca000640 1 -- 172.21.15.123:0/2162178469 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26cc10e1d0 con 0x7f26cc102360 2024-09-19T04:47:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.278+0000 7f26ca000640 1 --2- 172.21.15.123:0/2162178469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26cc102360 0x7f26cc1a4ab0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f26b800e9f0 tx=0x7f26b800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.278+0000 7f26c3400640 1 -- 172.21.15.123:0/2162178469 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b800cd80 con 0x7f26cc102360 2024-09-19T04:47:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.278+0000 7f26cbe00640 1 -- 172.21.15.123:0/2162178469 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26cc10e4c0 con 0x7f26cc102360 2024-09-19T04:47:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.278+0000 7f26c3400640 1 -- 172.21.15.123:0/2162178469 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b8005500 con 0x7f26cc102360 2024-09-19T04:47:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.278+0000 7f26c3400640 1 -- 172.21.15.123:0/2162178469 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8010690 con 0x7f26cc102360 2024-09-19T04:47:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.279+0000 7f26cbe00640 1 -- 172.21.15.123:0/2162178469 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26cc1ab4f0 con 0x7f26cc102360 2024-09-19T04:47:57.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.283+0000 7f26d45f6740 1 Processor -- start 2024-09-19T04:47:57.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.283+0000 7f26d45f6740 1 -- start start 2024-09-19T04:47:57.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.284+0000 7f26d45f6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed5638220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.284+0000 7f26d45f6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 0x557ed5642ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.284+0000 7f26d45f6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5643020 0x557ed5645410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.284+0000 7f26c3400640 1 -- 172.21.15.123:0/2162178469 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7f26b8004f50 con 0x7f26cc102360 2024-09-19T04:47:57.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.284+0000 7f26d45f6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557ed559e3e0 con 0x557ed5637e40 2024-09-19T04:47:57.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.284+0000 7f26d45f6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557ed5545d80 con 0x557ed5638760 2024-09-19T04:47:57.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.284+0000 7f26d45f6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557ed559a3a0 con 0x557ed5643020 2024-09-19T04:47:57.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.285+0000 7f26ca000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5643020 0x557ed5645410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.285+0000 7f26caa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 0x557ed5642ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.285+0000 7f26caa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 0x557ed5642ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:43852/0 (socket says 172.21.15.123:43852) 2024-09-19T04:47:57.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.285+0000 7f26caa00640 1 -- 172.21.15.123:0/3975178931 learned_addr learned my addr 172.21.15.123:0/3975178931 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:57.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.285+0000 7f26caa00640 1 -- 172.21.15.123:0/3975178931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5643020 msgr2=0x557ed5645410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:57.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.285+0000 7f26cb400640 1 --2- 172.21.15.123:0/3975178931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed5638220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.285+0000 7f26caa00640 1 --2- 172.21.15.123:0/3975178931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5643020 0x557ed5645410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.285+0000 7f26caa00640 1 -- 172.21.15.123:0/3975178931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 msgr2=0x557ed5638220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:57.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.285+0000 7f26caa00640 1 --2- 172.21.15.123:0/3975178931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed5638220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.285+0000 7f26caa00640 1 -- 172.21.15.123:0/3975178931 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed5564ae0 con 0x557ed5638760 2024-09-19T04:47:57.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.286+0000 7f26caa00640 1 --2- 172.21.15.123:0/3975178931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 0x557ed5642ae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009ad0 tx=0x7f26bc009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=428d2b1179dc015 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:57.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.286+0000 7f26ca000640 1 --2- 172.21.15.123:0/3975178931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5643020 0x557ed5645410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:47:57.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.286+0000 7f26cb400640 1 --2- 172.21.15.123:0/3975178931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed5638220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:47:57.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.286+0000 7f26c3400640 1 --2- 172.21.15.123:0/2162178469 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f26b00a3560 0x7f26b00a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.286+0000 7f26b7e00640 1 -- 172.21.15.123:0/3975178931 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc010d30 con 0x557ed5638760 2024-09-19T04:47:57.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.286+0000 7f26b7e00640 1 -- 172.21.15.123:0/3975178931 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26bc012930 con 0x557ed5638760 2024-09-19T04:47:57.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.286+0000 7f26b7e00640 1 -- 172.21.15.123:0/3975178931 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc011940 con 0x557ed5638760 2024-09-19T04:47:57.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.287+0000 7f26c3400640 1 -- 172.21.15.123:0/2162178469 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f26b8044060 con 0x7f26cc102360 2024-09-19T04:47:57.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.287+0000 7f26cb400640 1 --2- 172.21.15.123:0/2162178469 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f26b00a3560 0x7f26b00a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.288+0000 7f26d45f6740 1 -- 172.21.15.123:0/3975178931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 msgr2=0x557ed5642ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:57.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.288+0000 7f26d45f6740 1 --2- 172.21.15.123:0/3975178931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 0x557ed5642ae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009ad0 tx=0x7f26bc009f90 comp rx=0 tx=0).stop 2024-09-19T04:47:57.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.288+0000 7f26cb400640 1 --2- 172.21.15.123:0/2162178469 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f26b00a3560 0x7f26b00a5a20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f26c402f540 tx=0x7f26c4009980 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:57.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.288+0000 7f26d45f6740 1 -- 172.21.15.123:0/3975178931 shutdown_connections 2024-09-19T04:47:57.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.288+0000 7f26d45f6740 1 --2- 172.21.15.123:0/3975178931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5643020 0x557ed5645410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.288+0000 7f26d45f6740 1 --2- 172.21.15.123:0/3975178931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 0x557ed5642ae0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.288+0000 7f26d45f6740 1 --2- 172.21.15.123:0/3975178931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed5638220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.288+0000 7f26d45f6740 1 -- 172.21.15.123:0/3975178931 >> 172.21.15.123:0/3975178931 conn(0x557ed56343d0 msgr2=0x557ed56347b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:57.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.288+0000 7f26d45f6740 1 -- 172.21.15.123:0/3975178931 shutdown_connections 2024-09-19T04:47:57.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.288+0000 7f26d45f6740 1 -- 172.21.15.123:0/3975178931 wait complete. 2024-09-19T04:47:57.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.289+0000 7f26d45f6740 1 -- start start 2024-09-19T04:47:57.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.289+0000 7f26d45f6740 1 Processor -- start 2024-09-19T04:47:57.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.290+0000 7f26d45f6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed563fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.290+0000 7f26d45f6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 0x557ed5640410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.290+0000 7f26d45f6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5643020 0x557ed5640950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.290+0000 7f26d45f6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557ed5656c00 con 0x557ed5637e40 2024-09-19T04:47:57.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.290+0000 7f26d45f6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557ed55590f0 con 0x557ed5638760 2024-09-19T04:47:57.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.290+0000 7f26d45f6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557ed5656d80 con 0x557ed5643020 2024-09-19T04:47:57.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.290+0000 7f26d45f6740 10 client.0 _ll_register_callbacks cb 0x557ed5540bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:57.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.290+0000 7f26cb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5643020 0x557ed5640950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.290+0000 7f26caa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed563fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.291+0000 7f26cb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5643020 0x557ed5640950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:55220/0 (socket says 172.21.15.123:55220) 2024-09-19T04:47:57.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.291+0000 7f26cb400640 1 -- 172.21.15.123:0/3595632369 learned_addr learned my addr 172.21.15.123:0/3595632369 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:57.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.291+0000 7f26ca000640 1 --2- 172.21.15.123:0/3595632369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 0x557ed5640410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.291+0000 7f26ca000640 1 -- 172.21.15.123:0/3595632369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5643020 msgr2=0x557ed5640950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:57.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.291+0000 7f26ca000640 1 --2- 172.21.15.123:0/3595632369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5643020 0x557ed5640950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.291+0000 7f26ca000640 1 -- 172.21.15.123:0/3595632369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 msgr2=0x557ed563fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:57.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.292+0000 7f26ca000640 1 --2- 172.21.15.123:0/3595632369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed563fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.292+0000 7f26ca000640 1 -- 172.21.15.123:0/3595632369 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ed530f3c0 con 0x557ed5638760 2024-09-19T04:47:57.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.292+0000 7f26caa00640 1 --2- 172.21.15.123:0/3595632369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed563fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:47:57.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.292+0000 7f26ca000640 1 --2- 172.21.15.123:0/3595632369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 0x557ed5640410 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f26b80cd5b0 tx=0x7f26b809bb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45df36216b3a1874 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:57.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.292+0000 7f26b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:47:57.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.292+0000 7f26b4c00640 1 -- 172.21.15.123:0/3595632369 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b80cde60 con 0x557ed5638760 2024-09-19T04:47:57.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.292+0000 7f26d45f6740 1 -- 172.21.15.123:0/3595632369 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26bc012930 con 0x557ed5638760 2024-09-19T04:47:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.292+0000 7f26b4c00640 1 -- 172.21.15.123:0/3595632369 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f26b80a0120 con 0x557ed5638760 2024-09-19T04:47:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.293+0000 7f26b4c00640 1 -- 172.21.15.123:0/3595632369 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f26b80a0320 con 0x557ed5638760 2024-09-19T04:47:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.293+0000 7f26d45f6740 10 client.25547 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.293+0000 7f26d45f6740 1 -- 172.21.15.123:0/3595632369 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ed5662c60 con 0x557ed5638760 2024-09-19T04:47:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.293+0000 7f26b4c00640 1 -- 172.21.15.123:0/3595632369 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26bc012930 con 0x557ed5638760 2024-09-19T04:47:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.294+0000 7f26b4c00640 1 -- 172.21.15.123:0/3595632369 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f26b8096ce0 con 0x557ed5638760 2024-09-19T04:47:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.294+0000 7f26d45f6740 10 client.25547 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.294+0000 7f26d45f6740 10 client.25547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:47:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.294+0000 7f26d45f6740 20 client.25547 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.294+0000 7f26d45f6740 1 --2- 172.21.15.123:0/3595632369 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x557ed5663fb0 0x557ed5666420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.294+0000 7f26d45f6740 4 client.25547 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:47:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.294+0000 7f26d45f6740 1 -- 172.21.15.123:0/3595632369 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ed55ab950 con 0x557ed5663fb0 2024-09-19T04:47:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.294+0000 7f26caa00640 1 --2- 172.21.15.123:0/3595632369 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x557ed5663fb0 0x557ed5666420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.295+0000 7f26caa00640 1 --2- 172.21.15.123:0/3595632369 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x557ed5663fb0 0x557ed5666420 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e61e8c7e26c02a1 server_cookie=e96ed7d73282f65 in_seq=0 out_seq=0 2024-09-19T04:47:57.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.295+0000 7f26b4c00640 10 client.25547 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:57.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.296+0000 7f26b4c00640 1 -- 172.21.15.123:0/3595632369 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ed55ab950 con 0x557ed5663fb0 2024-09-19T04:47:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.296+0000 7f26b4c00640 10 client.25547 handle_command_reply: tid=0 2024-09-19T04:47:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.296+0000 7f26d45f6740 1 client.25547 shutdown 2024-09-19T04:47:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.296+0000 7f26d45f6740 1 -- 172.21.15.123:0/3595632369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 msgr2=0x557ed5640410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.296+0000 7f26d45f6740 1 --2- 172.21.15.123:0/3595632369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 0x557ed5640410 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f26b80cd5b0 tx=0x7f26b809bb90 comp rx=0 tx=0).stop 2024-09-19T04:47:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.297+0000 7f26d45f6740 1 -- 172.21.15.123:0/3595632369 shutdown_connections 2024-09-19T04:47:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.297+0000 7f26d45f6740 1 --2- 172.21.15.123:0/3595632369 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x557ed5663fb0 0x557ed5666420 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.297+0000 7f26d45f6740 1 --2- 172.21.15.123:0/3595632369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5643020 0x557ed5640950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.297+0000 7f26d45f6740 1 --2- 172.21.15.123:0/3595632369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 0x557ed5640410 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.297+0000 7f26b4c00640 0 client.25547 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.297+0000 7f26d45f6740 1 --2- 172.21.15.123:0/3595632369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed563fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.297+0000 7f26d45f6740 1 -- 172.21.15.123:0/3595632369 >> 172.21.15.123:0/3595632369 conn(0x557ed56343d0 msgr2=0x557ed563a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.297+0000 7f26d45f6740 1 -- 172.21.15.123:0/3595632369 shutdown_connections 2024-09-19T04:47:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.297+0000 7f26d45f6740 1 -- 172.21.15.123:0/3595632369 wait complete. 2024-09-19T04:47:57.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.297+0000 7f26d45f6740 20 client.25547 trim_cache size 0 max 16384 2024-09-19T04:47:57.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.311+0000 7f26d45f6740 1 Processor -- start 2024-09-19T04:47:57.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.311+0000 7f26d45f6740 1 -- start start 2024-09-19T04:47:57.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.311+0000 7f26d45f6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed562f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.311+0000 7f26d45f6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5638760 0x557ed56407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.311+0000 7f26d45f6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5643020 0x557ed566bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.311+0000 7f26d45f6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac00e770 con 0x557ed5637e40 2024-09-19T04:47:57.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.311+0000 7f26d45f6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac0102b0 con 0x557ed5643020 2024-09-19T04:47:57.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.311+0000 7f26d45f6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac00ee40 con 0x557ed5638760 2024-09-19T04:47:57.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.312+0000 7f26ca000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5638760 0x557ed56407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.312+0000 7f26ca000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5638760 0x557ed56407a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:55232/0 (socket says 172.21.15.123:55232) 2024-09-19T04:47:57.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.312+0000 7f26ca000640 1 -- 172.21.15.123:0/1252950298 learned_addr learned my addr 172.21.15.123:0/1252950298 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:57.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.312+0000 7f26caa00640 1 --2- 172.21.15.123:0/1252950298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed562f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.312+0000 7f26cb400640 1 --2- 172.21.15.123:0/1252950298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5643020 0x557ed566bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.312+0000 7f26ca000640 1 -- 172.21.15.123:0/1252950298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5643020 msgr2=0x557ed566bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:57.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.312+0000 7f26ca000640 1 --2- 172.21.15.123:0/1252950298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5643020 0x557ed566bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.312+0000 7f26ca000640 1 -- 172.21.15.123:0/1252950298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 msgr2=0x557ed562f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:57.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.312+0000 7f26ca000640 1 --2- 172.21.15.123:0/1252950298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed562f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.312+0000 7f26ca000640 1 -- 172.21.15.123:0/1252950298 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed563ee80 con 0x557ed5638760 2024-09-19T04:47:57.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.312+0000 7f26caa00640 1 --2- 172.21.15.123:0/1252950298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed562f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:47:57.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.313+0000 7f26ca000640 1 --2- 172.21.15.123:0/1252950298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5638760 0x557ed56407a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f26b8004d40 tx=0x7f26b80b5240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f68f75dcaecbf7ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:57.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.313+0000 7f26b7400640 1 -- 172.21.15.123:0/1252950298 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b80cde60 con 0x557ed5638760 2024-09-19T04:47:57.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.313+0000 7f26b7400640 1 -- 172.21.15.123:0/1252950298 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b8043020 con 0x557ed5638760 2024-09-19T04:47:57.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.313+0000 7f26b7400640 1 -- 172.21.15.123:0/1252950298 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8096ce0 con 0x557ed5638760 2024-09-19T04:47:57.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.314+0000 7f26d45f6740 1 -- 172.21.15.123:0/1252950298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5638760 msgr2=0x557ed56407a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:57.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.314+0000 7f26d45f6740 1 --2- 172.21.15.123:0/1252950298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5638760 0x557ed56407a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f26b8004d40 tx=0x7f26b80b5240 comp rx=0 tx=0).stop 2024-09-19T04:47:57.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.314+0000 7f26d45f6740 1 -- 172.21.15.123:0/1252950298 shutdown_connections 2024-09-19T04:47:57.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.314+0000 7f26d45f6740 1 --2- 172.21.15.123:0/1252950298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5643020 0x557ed566bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.314+0000 7f26d45f6740 1 --2- 172.21.15.123:0/1252950298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5638760 0x557ed56407a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.314+0000 7f26d45f6740 1 --2- 172.21.15.123:0/1252950298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed562f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.314+0000 7f26d45f6740 1 -- 172.21.15.123:0/1252950298 >> 172.21.15.123:0/1252950298 conn(0x557ed56343d0 msgr2=0x557ed563a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:57.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.314+0000 7f26d45f6740 1 -- 172.21.15.123:0/1252950298 shutdown_connections 2024-09-19T04:47:57.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.314+0000 7f26d45f6740 1 -- 172.21.15.123:0/1252950298 wait complete. 2024-09-19T04:47:57.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.315+0000 7f26d45f6740 1 -- start start 2024-09-19T04:47:57.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.315+0000 7f26d45f6740 1 Processor -- start 2024-09-19T04:47:57.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.316+0000 7f26d45f6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed565c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.316+0000 7f26d45f6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 0x557ed565c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.316+0000 7f26d45f6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5643020 0x557ed5659000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.316+0000 7f26d45f6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac00fbe0 con 0x557ed5637e40 2024-09-19T04:47:57.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.316+0000 7f26d45f6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac00f510 con 0x557ed5638760 2024-09-19T04:47:57.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.316+0000 7f26d45f6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac00c6f0 con 0x557ed5643020 2024-09-19T04:47:57.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.316+0000 7f26d45f6740 10 client.0 _ll_register_callbacks cb 0x7f26b80056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:47:57.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.317+0000 7f26ca000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 0x557ed565c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.317+0000 7f26ca000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 0x557ed565c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:43866/0 (socket says 172.21.15.123:43866) 2024-09-19T04:47:57.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.317+0000 7f26ca000640 1 -- 172.21.15.123:0/2894457771 learned_addr learned my addr 172.21.15.123:0/2894457771 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:47:57.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.317+0000 7f26caa00640 1 --2- 172.21.15.123:0/2894457771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed565c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.317+0000 7f26cb400640 1 --2- 172.21.15.123:0/2894457771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5643020 0x557ed5659000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.317+0000 7f26ca000640 1 -- 172.21.15.123:0/2894457771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5643020 msgr2=0x557ed5659000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:57.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.317+0000 7f26ca000640 1 --2- 172.21.15.123:0/2894457771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5643020 0x557ed5659000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.317+0000 7f26ca000640 1 -- 172.21.15.123:0/2894457771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 msgr2=0x557ed565c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:47:57.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.318+0000 7f26ca000640 1 --2- 172.21.15.123:0/2894457771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed565c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.318+0000 7f26ca000640 1 -- 172.21.15.123:0/2894457771 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ed5659710 con 0x557ed5638760 2024-09-19T04:47:57.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.318+0000 7f26caa00640 1 --2- 172.21.15.123:0/2894457771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed565c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:47:57.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.318+0000 7f26ca000640 1 --2- 172.21.15.123:0/2894457771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 0x557ed565c750 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f26b80a48b0 tx=0x7f26b8098490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f7a005787718361 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:47:57.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.318+0000 7f26b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:47:57.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.318+0000 7f26b4c00640 1 -- 172.21.15.123:0/2894457771 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8067050 con 0x557ed5638760 2024-09-19T04:47:57.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.318+0000 7f26b4c00640 1 -- 172.21.15.123:0/2894457771 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f26b8097ec0 con 0x557ed5638760 2024-09-19T04:47:57.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.319+0000 7f26d45f6740 1 -- 172.21.15.123:0/2894457771 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26b8043020 con 0x557ed5638760 2024-09-19T04:47:57.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.319+0000 7f26b4c00640 1 -- 172.21.15.123:0/2894457771 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f26b80cde60 con 0x557ed5638760 2024-09-19T04:47:57.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.319+0000 7f26d45f6740 10 client.25553 fetch_fsmap learned FSMap version 28 2024-09-19T04:47:57.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.319+0000 7f26d45f6740 1 -- 172.21.15.123:0/2894457771 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ed56cf6d0 con 0x557ed5638760 2024-09-19T04:47:57.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.320+0000 7f26b4c00640 1 -- 172.21.15.123:0/2894457771 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b8043020 con 0x557ed5638760 2024-09-19T04:47:57.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.320+0000 7f26b4c00640 1 -- 172.21.15.123:0/2894457771 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f26b809b030 con 0x557ed5638760 2024-09-19T04:47:57.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.320+0000 7f26d45f6740 10 client.25553 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:47:57.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.321+0000 7f26d45f6740 10 client.25553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:47:57.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.321+0000 7f26d45f6740 20 client.25553 populate_metadata read hostname 'smithi123' 2024-09-19T04:47:57.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.321+0000 7f26d45f6740 1 --2- 172.21.15.123:0/2894457771 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x557ed56cfba0 0x557ed56d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:47:57.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.321+0000 7f26d45f6740 4 client.25553 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:47:57.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.321+0000 7f26d45f6740 1 -- 172.21.15.123:0/2894457771 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ed55a5e60 con 0x557ed56cfba0 2024-09-19T04:47:57.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.321+0000 7f26caa00640 1 --2- 172.21.15.123:0/2894457771 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x557ed56cfba0 0x557ed56d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:47:57.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.321+0000 7f26caa00640 1 --2- 172.21.15.123:0/2894457771 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x557ed56cfba0 0x557ed56d1f90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=612a514cbdd7d4e server_cookie=5598196512f76497 in_seq=0 out_seq=0 2024-09-19T04:47:57.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.321+0000 7f26b4c00640 10 client.25553 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:57.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.322+0000 7f26b4c00640 1 -- 172.21.15.123:0/2894457771 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ed55a5e60 con 0x557ed56cfba0 2024-09-19T04:47:57.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.322+0000 7f26b4c00640 10 client.25553 handle_command_reply: tid=0 2024-09-19T04:47:57.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.322+0000 7f26d45f6740 1 client.25553 shutdown 2024-09-19T04:47:57.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.322+0000 7f26d45f6740 1 -- 172.21.15.123:0/2894457771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 msgr2=0x557ed565c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:57.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.322+0000 7f26d45f6740 1 --2- 172.21.15.123:0/2894457771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 0x557ed565c750 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f26b80a48b0 tx=0x7f26b8098490 comp rx=0 tx=0).stop 2024-09-19T04:47:57.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.322+0000 7f26d45f6740 1 -- 172.21.15.123:0/2894457771 shutdown_connections 2024-09-19T04:47:57.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.322+0000 7f26d45f6740 1 --2- 172.21.15.123:0/2894457771 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x557ed56cfba0 0x557ed56d1f90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.322+0000 7f26d45f6740 1 --2- 172.21.15.123:0/2894457771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ed5643020 0x557ed5659000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.322+0000 7f26d45f6740 1 --2- 172.21.15.123:0/2894457771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5638760 0x557ed565c750 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.322+0000 7f26b4c00640 0 client.25553 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:47:57.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.322+0000 7f26d45f6740 1 --2- 172.21.15.123:0/2894457771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ed5637e40 0x557ed565c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.322+0000 7f26d45f6740 1 -- 172.21.15.123:0/2894457771 >> 172.21.15.123:0/2894457771 conn(0x557ed56343d0 msgr2=0x557ed5632be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:57.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.323+0000 7f26d45f6740 1 -- 172.21.15.123:0/2894457771 shutdown_connections 2024-09-19T04:47:57.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.323+0000 7f26d45f6740 1 -- 172.21.15.123:0/2894457771 wait complete. 2024-09-19T04:47:57.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.323+0000 7f26d45f6740 20 client.25553 trim_cache size 0 max 16384 2024-09-19T04:47:57.330 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:47:57.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.324+0000 7f26cbe00640 1 -- 172.21.15.123:0/2162178469 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f26b00a3560 msgr2=0x7f26b00a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:57.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.324+0000 7f26cbe00640 1 --2- 172.21.15.123:0/2162178469 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f26b00a3560 0x7f26b00a5a20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f26c402f540 tx=0x7f26c4009980 comp rx=0 tx=0).stop 2024-09-19T04:47:57.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.324+0000 7f26cbe00640 1 -- 172.21.15.123:0/2162178469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26cc102360 msgr2=0x7f26cc1a4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:47:57.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.324+0000 7f26cbe00640 1 --2- 172.21.15.123:0/2162178469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26cc102360 0x7f26cc1a4ab0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f26b800e9f0 tx=0x7f26b800edf0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.324+0000 7f26cbe00640 1 -- 172.21.15.123:0/2162178469 shutdown_connections 2024-09-19T04:47:57.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.324+0000 7f26cbe00640 1 --2- 172.21.15.123:0/2162178469 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f26b00a3560 0x7f26b00a5a20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.324+0000 7f26cbe00640 1 --2- 172.21.15.123:0/2162178469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26cc1079c0 0x7f26cc10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.324+0000 7f26cbe00640 1 --2- 172.21.15.123:0/2162178469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26cc102360 0x7f26cc1a4ab0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.324+0000 7f26cbe00640 1 --2- 172.21.15.123:0/2162178469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26cc1019c0 0x7f26cc1a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:47:57.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.324+0000 7f26cbe00640 1 -- 172.21.15.123:0/2162178469 >> 172.21.15.123:0/2162178469 conn(0x7f26cc0fd740 msgr2=0x7f26cc0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:47:57.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.324+0000 7f26cbe00640 1 -- 172.21.15.123:0/2162178469 shutdown_connections 2024-09-19T04:47:57.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:47:57.324+0000 7f26cbe00640 1 -- 172.21.15.123:0/2162178469 wait complete. 2024-09-19T04:47:57.842 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:57 smithi123 ceph-mon[30511]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:47:57.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:57 smithi183 ceph-mon[35333]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:47:57.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:57 smithi193 ceph-mon[35592]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:47:59.181 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:47:59.182 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:47:59.570 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:47:59.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:47:59 smithi183 ceph-mon[35333]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T04:47:59.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:47:59 smithi123 ceph-mon[30511]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T04:47:59.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:47:59 smithi193 ceph-mon[35592]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T04:48:01.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:01 smithi183 ceph-mon[35333]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:48:01.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:01 smithi123 ceph-mon[30511]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:48:01.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:01 smithi193 ceph-mon[35592]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:48:03.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.599+0000 7f31fac00640 1 -- 172.21.15.123:0/2336260332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4102b90 msgr2=0x7f31f4102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:03.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.599+0000 7f31fac00640 1 --2- 172.21.15.123:0/2336260332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4102b90 0x7f31f4102f70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009900 tx=0x7f31e802a480 comp rx=0 tx=0).stop 2024-09-19T04:48:03.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.599+0000 7f31fac00640 1 -- 172.21.15.123:0/2336260332 shutdown_connections 2024-09-19T04:48:03.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.599+0000 7f31fac00640 1 --2- 172.21.15.123:0/2336260332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f410bfc0 0x7f31f410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.599+0000 7f31fac00640 1 --2- 172.21.15.123:0/2336260332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31f41034b0 0x7f31f410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.599+0000 7f31fac00640 1 --2- 172.21.15.123:0/2336260332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4102b90 0x7f31f4102f70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.599+0000 7f31fac00640 1 -- 172.21.15.123:0/2336260332 >> 172.21.15.123:0/2336260332 conn(0x7f31f40fc610 msgr2=0x7f31f40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:03.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.599+0000 7f31fac00640 1 -- 172.21.15.123:0/2336260332 shutdown_connections 2024-09-19T04:48:03.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.599+0000 7f31fac00640 1 -- 172.21.15.123:0/2336260332 wait complete. 2024-09-19T04:48:03.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.600+0000 7f31fac00640 1 Processor -- start 2024-09-19T04:48:03.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.601+0000 7f31fac00640 1 -- start start 2024-09-19T04:48:03.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.601+0000 7f31fac00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31f4102b90 0x7f31f419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.601+0000 7f31fac00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f41034b0 0x7f31f419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.601+0000 7f31fac00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f410bfc0 0x7f31f419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.601+0000 7f31fac00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4111e10 con 0x7f31f410bfc0 2024-09-19T04:48:03.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.601+0000 7f31fac00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4111c90 con 0x7f31f4102b90 2024-09-19T04:48:03.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.601+0000 7f31fac00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4111f90 con 0x7f31f41034b0 2024-09-19T04:48:03.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.601+0000 7f31fa200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f410bfc0 0x7f31f419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.601+0000 7f31f9800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31f4102b90 0x7f31f419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.601+0000 7f31f9800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31f4102b90 0x7f31f419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:46724/0 (socket says 172.21.15.123:46724) 2024-09-19T04:48:03.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.601+0000 7f31f8e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f41034b0 0x7f31f419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.601+0000 7f31f9800640 1 -- 172.21.15.123:0/3394477484 learned_addr learned my addr 172.21.15.123:0/3394477484 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:03.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.602+0000 7f31f8e00640 1 -- 172.21.15.123:0/3394477484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31f4102b90 msgr2=0x7f31f419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:03.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.602+0000 7f31f8e00640 1 --2- 172.21.15.123:0/3394477484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31f4102b90 0x7f31f419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.602+0000 7f31f8e00640 1 -- 172.21.15.123:0/3394477484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f410bfc0 msgr2=0x7f31f419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:03.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.602+0000 7f31f8e00640 1 --2- 172.21.15.123:0/3394477484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f410bfc0 0x7f31f419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.602+0000 7f31f8e00640 1 -- 172.21.15.123:0/3394477484 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f419f460 con 0x7f31f41034b0 2024-09-19T04:48:03.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.602+0000 7f31f9800640 1 --2- 172.21.15.123:0/3394477484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31f4102b90 0x7f31f419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:03.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.602+0000 7f31f8e00640 1 --2- 172.21.15.123:0/3394477484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f41034b0 0x7f31f419a9f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f31e000e7a0 tx=0x7f31e000ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:03.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.602+0000 7f31f2000640 1 -- 172.21.15.123:0/3394477484 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e000ccc0 con 0x7f31f41034b0 2024-09-19T04:48:03.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.602+0000 7f31f2000640 1 -- 172.21.15.123:0/3394477484 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e0009530 con 0x7f31f41034b0 2024-09-19T04:48:03.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.602+0000 7f31f2000640 1 -- 172.21.15.123:0/3394477484 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0018880 con 0x7f31f41034b0 2024-09-19T04:48:03.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.602+0000 7f31fac00640 1 -- 172.21.15.123:0/3394477484 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f419f750 con 0x7f31f41034b0 2024-09-19T04:48:03.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.603+0000 7f31fac00640 1 -- 172.21.15.123:0/3394477484 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f41a7030 con 0x7f31f41034b0 2024-09-19T04:48:03.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.606+0000 7f31fee34740 1 Processor -- start 2024-09-19T04:48:03.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.606+0000 7f31fee34740 1 -- start start 2024-09-19T04:48:03.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.606+0000 7f31f2000640 1 -- 172.21.15.123:0/3394477484 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7f31e0018a20 con 0x7f31f41034b0 2024-09-19T04:48:03.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.606+0000 7f31fee34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0a0cb0 0x558fbe0a1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.606+0000 7f31fee34740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a15d0 0x558fbe0ab950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.606+0000 7f31fee34740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0abe90 0x558fbe0ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.606+0000 7f31fee34740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558fbe007290 con 0x558fbe0a15d0 2024-09-19T04:48:03.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.606+0000 7f31fee34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558fbdfaeb40 con 0x558fbe0a0cb0 2024-09-19T04:48:03.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.606+0000 7f31fee34740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558fbe003250 con 0x558fbe0abe90 2024-09-19T04:48:03.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.607+0000 7f31f9800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a15d0 0x558fbe0ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.607+0000 7f31f9800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a15d0 0x558fbe0ab950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39248/0 (socket says 172.21.15.123:39248) 2024-09-19T04:48:03.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.607+0000 7f31f9800640 1 -- 172.21.15.123:0/1667001283 learned_addr learned my addr 172.21.15.123:0/1667001283 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:03.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.607+0000 7f31fa200640 1 --2- 172.21.15.123:0/1667001283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0a0cb0 0x558fbe0a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.607+0000 7f31f8e00640 1 --2- 172.21.15.123:0/1667001283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0abe90 0x558fbe0ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.607+0000 7f31f9800640 1 -- 172.21.15.123:0/1667001283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0abe90 msgr2=0x558fbe0ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:03.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.607+0000 7f31f9800640 1 --2- 172.21.15.123:0/1667001283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0abe90 0x558fbe0ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.607+0000 7f31f9800640 1 -- 172.21.15.123:0/1667001283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0a0cb0 msgr2=0x558fbe0a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:03.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.607+0000 7f31f9800640 1 --2- 172.21.15.123:0/1667001283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0a0cb0 0x558fbe0a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.607+0000 7f31f9800640 1 -- 172.21.15.123:0/1667001283 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fbdd786a0 con 0x558fbe0a15d0 2024-09-19T04:48:03.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.608+0000 7f31f9800640 1 --2- 172.21.15.123:0/1667001283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a15d0 0x558fbe0ab950 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f31e8005c40 tx=0x7f31e8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26710916f4854cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:03.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.608+0000 7f31f2000640 1 --2- 172.21.15.123:0/3394477484 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31dc0a35e0 0x7f31dc0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.608+0000 7f31e6a00640 1 -- 172.21.15.123:0/1667001283 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8040070 con 0x558fbe0a15d0 2024-09-19T04:48:03.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.608+0000 7f31e6a00640 1 -- 172.21.15.123:0/1667001283 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e80047a0 con 0x558fbe0a15d0 2024-09-19T04:48:03.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.608+0000 7f31e6a00640 1 -- 172.21.15.123:0/1667001283 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8035040 con 0x558fbe0a15d0 2024-09-19T04:48:03.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.608+0000 7f31f2000640 1 -- 172.21.15.123:0/3394477484 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f31e00d00c0 con 0x7f31f41034b0 2024-09-19T04:48:03.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.608+0000 7f31fa200640 1 --2- 172.21.15.123:0/3394477484 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31dc0a35e0 0x7f31dc0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.609+0000 7f31fee34740 1 -- 172.21.15.123:0/1667001283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a15d0 msgr2=0x558fbe0ab950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:03.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.609+0000 7f31fee34740 1 --2- 172.21.15.123:0/1667001283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a15d0 0x558fbe0ab950 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f31e8005c40 tx=0x7f31e8005dd0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.609+0000 7f31fee34740 1 -- 172.21.15.123:0/1667001283 shutdown_connections 2024-09-19T04:48:03.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.609+0000 7f31fee34740 1 --2- 172.21.15.123:0/1667001283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0abe90 0x558fbe0ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.609+0000 7f31fee34740 1 --2- 172.21.15.123:0/1667001283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a15d0 0x558fbe0ab950 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.609+0000 7f31fee34740 1 --2- 172.21.15.123:0/1667001283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0a0cb0 0x558fbe0a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.609+0000 7f31fee34740 1 -- 172.21.15.123:0/1667001283 >> 172.21.15.123:0/1667001283 conn(0x558fbe09d240 msgr2=0x558fbe09d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:03.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.609+0000 7f31fa200640 1 --2- 172.21.15.123:0/3394477484 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31dc0a35e0 0x7f31dc0a5aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f31ec005fd0 tx=0x7f31ec009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:03.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.609+0000 7f31fee34740 1 -- 172.21.15.123:0/1667001283 shutdown_connections 2024-09-19T04:48:03.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.609+0000 7f31fee34740 1 -- 172.21.15.123:0/1667001283 wait complete. 2024-09-19T04:48:03.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.609+0000 7f31fee34740 1 -- start start 2024-09-19T04:48:03.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.610+0000 7f31fee34740 1 Processor -- start 2024-09-19T04:48:03.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.610+0000 7f31fee34740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a0cb0 0x558fbe0ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.610+0000 7f31fee34740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a15d0 0x558fbe0a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.610+0000 7f31fee34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0abe90 0x558fbe0a8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.610+0000 7f31fee34740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558fbe0aeb00 con 0x558fbe0a15d0 2024-09-19T04:48:03.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.610+0000 7f31fee34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558fbdfc1fa0 con 0x558fbe0abe90 2024-09-19T04:48:03.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.610+0000 7f31fee34740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558fbe0aec80 con 0x558fbe0a0cb0 2024-09-19T04:48:03.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.610+0000 7f31fee34740 10 client.0 _ll_register_callbacks cb 0x558fbdd15ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:03.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.610+0000 7f31f8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a15d0 0x558fbe0a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.610+0000 7f31f8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a15d0 0x558fbe0a8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39262/0 (socket says 172.21.15.123:39262) 2024-09-19T04:48:03.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.610+0000 7f31f8e00640 1 -- 172.21.15.123:0/15283292 learned_addr learned my addr 172.21.15.123:0/15283292 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:03.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.611+0000 7f31f9800640 1 --2- 172.21.15.123:0/15283292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a0cb0 0x558fbe0ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.611+0000 7f31fa200640 1 --2- 172.21.15.123:0/15283292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0abe90 0x558fbe0a8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.611+0000 7f31f8e00640 1 -- 172.21.15.123:0/15283292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a0cb0 msgr2=0x558fbe0ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:03.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.611+0000 7f31f8e00640 1 --2- 172.21.15.123:0/15283292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a0cb0 0x558fbe0ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.611+0000 7f31f8e00640 1 -- 172.21.15.123:0/15283292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0abe90 msgr2=0x558fbe0a8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:03.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.611+0000 7f31f8e00640 1 --2- 172.21.15.123:0/15283292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0abe90 0x558fbe0a8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.611+0000 7f31f8e00640 1 -- 172.21.15.123:0/15283292 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fbe0a9780 con 0x558fbe0a15d0 2024-09-19T04:48:03.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.611+0000 7f31f9800640 1 --2- 172.21.15.123:0/15283292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a0cb0 0x558fbe0ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:03.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.611+0000 7f31f8e00640 1 --2- 172.21.15.123:0/15283292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a15d0 0x558fbe0a8a10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f31e000e770 tx=0x7f31e009be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefd3aaa2a3bba6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:03.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.611+0000 7f31db400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:48:03.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.611+0000 7f31db400640 1 -- 172.21.15.123:0/15283292 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e00a0540 con 0x558fbe0a15d0 2024-09-19T04:48:03.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.611+0000 7f31fee34740 1 -- 172.21.15.123:0/15283292 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31e80047a0 con 0x558fbe0a15d0 2024-09-19T04:48:03.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.611+0000 7f31db400640 1 -- 172.21.15.123:0/15283292 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f31e00d2070 con 0x558fbe0a15d0 2024-09-19T04:48:03.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.612+0000 7f31db400640 1 -- 172.21.15.123:0/15283292 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f31e00a4e00 con 0x558fbe0a15d0 2024-09-19T04:48:03.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.612+0000 7f31fee34740 10 client.15531 fetch_fsmap learned FSMap version 28 2024-09-19T04:48:03.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.612+0000 7f31fee34740 1 -- 172.21.15.123:0/15283292 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558fbe0ba9a0 con 0x558fbe0a15d0 2024-09-19T04:48:03.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.612+0000 7f31db400640 1 -- 172.21.15.123:0/15283292 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e80047a0 con 0x558fbe0a15d0 2024-09-19T04:48:03.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.612+0000 7f31db400640 1 -- 172.21.15.123:0/15283292 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f31e009b720 con 0x558fbe0a15d0 2024-09-19T04:48:03.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.612+0000 7f31fee34740 10 client.15531 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:48:03.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.612+0000 7f31fee34740 10 client.15531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:48:03.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.612+0000 7f31fee34740 20 client.15531 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:03.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.612+0000 7f31fee34740 1 --2- 172.21.15.123:0/15283292 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x558fbe0bbdb0 0x558fbe0be1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.612+0000 7f31fee34740 4 client.15531 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:48:03.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.612+0000 7f31fee34740 1 -- 172.21.15.123:0/15283292 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fbe014800 con 0x558fbe0bbdb0 2024-09-19T04:48:03.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.613+0000 7f31f9800640 1 --2- 172.21.15.123:0/15283292 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x558fbe0bbdb0 0x558fbe0be1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.613+0000 7f31db400640 10 client.15531 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:03.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.613+0000 7f31f9800640 1 --2- 172.21.15.123:0/15283292 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x558fbe0bbdb0 0x558fbe0be1c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ffed586e6ef998f server_cookie=84ca2661d416da6 in_seq=0 out_seq=0 2024-09-19T04:48:03.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.614+0000 7f31db400640 1 -- 172.21.15.123:0/15283292 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558fbe014800 con 0x558fbe0bbdb0 2024-09-19T04:48:03.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.614+0000 7f31db400640 10 client.15531 handle_command_reply: tid=0 2024-09-19T04:48:03.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.614+0000 7f31fee34740 1 client.15531 shutdown 2024-09-19T04:48:03.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.615+0000 7f31fee34740 1 -- 172.21.15.123:0/15283292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a15d0 msgr2=0x558fbe0a8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:03.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.615+0000 7f31fee34740 1 --2- 172.21.15.123:0/15283292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a15d0 0x558fbe0a8a10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f31e000e770 tx=0x7f31e009be70 comp rx=0 tx=0).stop 2024-09-19T04:48:03.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.615+0000 7f31fee34740 1 -- 172.21.15.123:0/15283292 shutdown_connections 2024-09-19T04:48:03.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.615+0000 7f31fee34740 1 --2- 172.21.15.123:0/15283292 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x558fbe0bbdb0 0x558fbe0be1c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.615+0000 7f31fee34740 1 --2- 172.21.15.123:0/15283292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0abe90 0x558fbe0a8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.615+0000 7f31fee34740 1 --2- 172.21.15.123:0/15283292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a15d0 0x558fbe0a8a10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.615+0000 7f31fee34740 1 --2- 172.21.15.123:0/15283292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a0cb0 0x558fbe0ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.615+0000 7f31fee34740 1 -- 172.21.15.123:0/15283292 >> 172.21.15.123:0/15283292 conn(0x558fbe09d240 msgr2=0x558fbe0a2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:03.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.615+0000 7f31db400640 0 client.15531 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:03.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.615+0000 7f31fee34740 1 -- 172.21.15.123:0/15283292 shutdown_connections 2024-09-19T04:48:03.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.615+0000 7f31fee34740 1 -- 172.21.15.123:0/15283292 wait complete. 2024-09-19T04:48:03.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.615+0000 7f31fee34740 20 client.15531 trim_cache size 0 max 16384 2024-09-19T04:48:03.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.633+0000 7f31fee34740 1 Processor -- start 2024-09-19T04:48:03.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.633+0000 7f31fee34740 1 -- start start 2024-09-19T04:48:03.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.634+0000 7f31fee34740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a0cb0 0x558fbe0c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.634+0000 7f31fee34740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a15d0 0x558fbe0c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.634+0000 7f31fee34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0abe90 0x558fbe0b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.634+0000 7f31fee34740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31f41af8d0 con 0x558fbe0a0cb0 2024-09-19T04:48:03.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.634+0000 7f31fee34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f31f41b1410 con 0x558fbe0abe90 2024-09-19T04:48:03.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.634+0000 7f31fee34740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f31f41affa0 con 0x558fbe0a15d0 2024-09-19T04:48:03.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.634+0000 7f31f9800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a0cb0 0x558fbe0c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.634+0000 7f31f9800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a0cb0 0x558fbe0c3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39270/0 (socket says 172.21.15.123:39270) 2024-09-19T04:48:03.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.634+0000 7f31f9800640 1 -- 172.21.15.123:0/2619516780 learned_addr learned my addr 172.21.15.123:0/2619516780 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:03.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.634+0000 7f31fa200640 1 --2- 172.21.15.123:0/2619516780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0abe90 0x558fbe0b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.634+0000 7f31f8e00640 1 --2- 172.21.15.123:0/2619516780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a15d0 0x558fbe0c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.635+0000 7f31f9800640 1 -- 172.21.15.123:0/2619516780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a15d0 msgr2=0x558fbe0c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:03.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.635+0000 7f31f9800640 1 --2- 172.21.15.123:0/2619516780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a15d0 0x558fbe0c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.635+0000 7f31f9800640 1 -- 172.21.15.123:0/2619516780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0abe90 msgr2=0x558fbe0b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:03.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.635+0000 7f31f9800640 1 --2- 172.21.15.123:0/2619516780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0abe90 0x558fbe0b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.635+0000 7f31f9800640 1 -- 172.21.15.123:0/2619516780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fbe0a6f80 con 0x558fbe0a0cb0 2024-09-19T04:48:03.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.635+0000 7f31fa200640 1 --2- 172.21.15.123:0/2619516780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0abe90 0x558fbe0b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:03.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.635+0000 7f31f9800640 1 --2- 172.21.15.123:0/2619516780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a0cb0 0x558fbe0c3a90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f31e8031520 tx=0x7f31e802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682d83d6e7a0d441 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:03.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.635+0000 7f31e6000640 1 -- 172.21.15.123:0/2619516780 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8058070 con 0x558fbe0a0cb0 2024-09-19T04:48:03.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.635+0000 7f31e6000640 1 -- 172.21.15.123:0/2619516780 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e8003b90 con 0x558fbe0a0cb0 2024-09-19T04:48:03.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.635+0000 7f31e6000640 1 -- 172.21.15.123:0/2619516780 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8040040 con 0x558fbe0a0cb0 2024-09-19T04:48:03.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.636+0000 7f31fee34740 1 -- 172.21.15.123:0/2619516780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a0cb0 msgr2=0x558fbe0c3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:03.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.636+0000 7f31fee34740 1 --2- 172.21.15.123:0/2619516780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a0cb0 0x558fbe0c3a90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f31e8031520 tx=0x7f31e802a480 comp rx=0 tx=0).stop 2024-09-19T04:48:03.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.636+0000 7f31fee34740 1 -- 172.21.15.123:0/2619516780 shutdown_connections 2024-09-19T04:48:03.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.636+0000 7f31fee34740 1 --2- 172.21.15.123:0/2619516780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0abe90 0x558fbe0b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.636+0000 7f31fee34740 1 --2- 172.21.15.123:0/2619516780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a15d0 0x558fbe0c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.636+0000 7f31fee34740 1 --2- 172.21.15.123:0/2619516780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0a0cb0 0x558fbe0c3a90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.636+0000 7f31fee34740 1 -- 172.21.15.123:0/2619516780 >> 172.21.15.123:0/2619516780 conn(0x558fbe09d240 msgr2=0x558fbe0a7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:03.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.637+0000 7f31fee34740 1 -- 172.21.15.123:0/2619516780 shutdown_connections 2024-09-19T04:48:03.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.637+0000 7f31fee34740 1 -- 172.21.15.123:0/2619516780 wait complete. 2024-09-19T04:48:03.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.637+0000 7f31fee34740 1 -- start start 2024-09-19T04:48:03.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.637+0000 7f31fee34740 1 Processor -- start 2024-09-19T04:48:03.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.638+0000 7f31fee34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0a0cb0 0x558fbe0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.638+0000 7f31fee34740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a15d0 0x558fbe0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.638+0000 7f31fee34740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0abe90 0x558fbe0b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.638+0000 7f31fee34740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31f41b0d40 con 0x558fbe0abe90 2024-09-19T04:48:03.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.638+0000 7f31fee34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f31f41b0670 con 0x558fbe0a0cb0 2024-09-19T04:48:03.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.638+0000 7f31fee34740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f31f41ad8b0 con 0x558fbe0a15d0 2024-09-19T04:48:03.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.638+0000 7f31fee34740 10 client.0 _ll_register_callbacks cb 0x7f31e000ef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:03.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.638+0000 7f31f8e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a15d0 0x558fbe0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.638+0000 7f31fa200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0abe90 0x558fbe0b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.638+0000 7f31f8e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a15d0 0x558fbe0b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:54260/0 (socket says 172.21.15.123:54260) 2024-09-19T04:48:03.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.638+0000 7f31f8e00640 1 -- 172.21.15.123:0/4275946508 learned_addr learned my addr 172.21.15.123:0/4275946508 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:03.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.638+0000 7f31f9800640 1 --2- 172.21.15.123:0/4275946508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0a0cb0 0x558fbe0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.638+0000 7f31f8e00640 1 -- 172.21.15.123:0/4275946508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0a0cb0 msgr2=0x558fbe0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:03.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.638+0000 7f31f8e00640 1 --2- 172.21.15.123:0/4275946508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0a0cb0 0x558fbe0b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.638+0000 7f31f8e00640 1 -- 172.21.15.123:0/4275946508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0abe90 msgr2=0x558fbe0b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:03.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.638+0000 7f31f8e00640 1 --2- 172.21.15.123:0/4275946508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0abe90 0x558fbe0b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.638+0000 7f31f8e00640 1 -- 172.21.15.123:0/4275946508 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fbe125a70 con 0x558fbe0a15d0 2024-09-19T04:48:03.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.639+0000 7f31f8e00640 1 --2- 172.21.15.123:0/4275946508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a15d0 0x558fbe0b2160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f31e0098bc0 tx=0x7f31e009c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf6f2d14ba3798a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:03.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.639+0000 7f31db400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:48:03.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.639+0000 7f31db400640 1 -- 172.21.15.123:0/4275946508 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e00a0540 con 0x558fbe0a15d0 2024-09-19T04:48:03.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.639+0000 7f31db400640 1 -- 172.21.15.123:0/4275946508 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f31e009b890 con 0x558fbe0a15d0 2024-09-19T04:48:03.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.639+0000 7f31fee34740 1 -- 172.21.15.123:0/4275946508 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31e8003b90 con 0x558fbe0a15d0 2024-09-19T04:48:03.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.639+0000 7f31db400640 1 -- 172.21.15.123:0/4275946508 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f31e009c010 con 0x558fbe0a15d0 2024-09-19T04:48:03.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.639+0000 7f31fee34740 10 client.25585 fetch_fsmap learned FSMap version 28 2024-09-19T04:48:03.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.639+0000 7f31fee34740 1 -- 172.21.15.123:0/4275946508 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558fbe126d10 con 0x558fbe0a15d0 2024-09-19T04:48:03.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.639+0000 7f31db400640 1 -- 172.21.15.123:0/4275946508 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e8003b90 con 0x558fbe0a15d0 2024-09-19T04:48:03.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.640+0000 7f31db400640 1 -- 172.21.15.123:0/4275946508 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f31e00a16d0 con 0x558fbe0a15d0 2024-09-19T04:48:03.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.640+0000 7f31fee34740 10 client.25585 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:48:03.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.640+0000 7f31fee34740 10 client.25585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:48:03.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.640+0000 7f31fee34740 20 client.25585 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:03.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.640+0000 7f31fee34740 1 --2- 172.21.15.123:0/4275946508 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x558fbe127430 0x558fbe129850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:03.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.640+0000 7f31fee34740 4 client.25585 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:48:03.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.640+0000 7f31fee34740 1 -- 172.21.15.123:0/4275946508 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558fbe00ed10 con 0x558fbe127430 2024-09-19T04:48:03.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.640+0000 7f31f9800640 1 --2- 172.21.15.123:0/4275946508 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x558fbe127430 0x558fbe129850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:03.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.641+0000 7f31f9800640 1 --2- 172.21.15.123:0/4275946508 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x558fbe127430 0x558fbe129850 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddbec1ef366511ad server_cookie=539d6971b8772d52 in_seq=0 out_seq=0 2024-09-19T04:48:03.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.641+0000 7f31db400640 10 client.25585 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:03.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.642+0000 7f31db400640 1 -- 172.21.15.123:0/4275946508 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558fbe00ed10 con 0x558fbe127430 2024-09-19T04:48:03.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.642+0000 7f31db400640 10 client.25585 handle_command_reply: tid=0 2024-09-19T04:48:03.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.642+0000 7f31fee34740 1 client.25585 shutdown 2024-09-19T04:48:03.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.642+0000 7f31fee34740 1 -- 172.21.15.123:0/4275946508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a15d0 msgr2=0x558fbe0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:03.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.642+0000 7f31fee34740 1 --2- 172.21.15.123:0/4275946508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a15d0 0x558fbe0b2160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f31e0098bc0 tx=0x7f31e009c3c0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.642+0000 7f31fee34740 1 -- 172.21.15.123:0/4275946508 shutdown_connections 2024-09-19T04:48:03.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.642+0000 7f31fee34740 1 --2- 172.21.15.123:0/4275946508 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x558fbe127430 0x558fbe129850 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.642+0000 7f31fee34740 1 --2- 172.21.15.123:0/4275946508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbe0abe90 0x558fbe0b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.642+0000 7f31fee34740 1 --2- 172.21.15.123:0/4275946508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbe0a15d0 0x558fbe0b2160 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.642+0000 7f31fee34740 1 --2- 172.21.15.123:0/4275946508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbe0a0cb0 0x558fbe0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.642+0000 7f31fee34740 1 -- 172.21.15.123:0/4275946508 >> 172.21.15.123:0/4275946508 conn(0x558fbe09d240 msgr2=0x558fbe09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:03.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.642+0000 7f31db400640 0 client.25585 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:03.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.642+0000 7f31fee34740 1 -- 172.21.15.123:0/4275946508 shutdown_connections 2024-09-19T04:48:03.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.642+0000 7f31fee34740 1 -- 172.21.15.123:0/4275946508 wait complete. 2024-09-19T04:48:03.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.642+0000 7f31fee34740 20 client.25585 trim_cache size 0 max 16384 2024-09-19T04:48:03.656 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:48:03.656 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-09-19T04:48:03.656 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "aa63c3c0-6153-40a6-aca4-76049ae9e9ea", 2024-09-19T04:48:03.656 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-09-19T04:48:03.656 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:48:03.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.643+0000 7f31fac00640 1 -- 172.21.15.123:0/3394477484 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31dc0a35e0 msgr2=0x7f31dc0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:03.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.643+0000 7f31fac00640 1 --2- 172.21.15.123:0/3394477484 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31dc0a35e0 0x7f31dc0a5aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f31ec005fd0 tx=0x7f31ec009500 comp rx=0 tx=0).stop 2024-09-19T04:48:03.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.643+0000 7f31fac00640 1 -- 172.21.15.123:0/3394477484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f41034b0 msgr2=0x7f31f419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:03.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.643+0000 7f31fac00640 1 --2- 172.21.15.123:0/3394477484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f41034b0 0x7f31f419a9f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f31e000e7a0 tx=0x7f31e000ec60 comp rx=0 tx=0).stop 2024-09-19T04:48:03.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.643+0000 7f31fac00640 1 -- 172.21.15.123:0/3394477484 shutdown_connections 2024-09-19T04:48:03.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.643+0000 7f31fac00640 1 --2- 172.21.15.123:0/3394477484 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31dc0a35e0 0x7f31dc0a5aa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.643+0000 7f31fac00640 1 --2- 172.21.15.123:0/3394477484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f410bfc0 0x7f31f419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.643+0000 7f31fac00640 1 --2- 172.21.15.123:0/3394477484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f41034b0 0x7f31f419a9f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.643+0000 7f31fac00640 1 --2- 172.21.15.123:0/3394477484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31f4102b90 0x7f31f419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:03.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.643+0000 7f31fac00640 1 -- 172.21.15.123:0/3394477484 >> 172.21.15.123:0/3394477484 conn(0x7f31f40fc610 msgr2=0x7f31f40fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:03.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.643+0000 7f31fac00640 1 -- 172.21.15.123:0/3394477484 shutdown_connections 2024-09-19T04:48:03.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:03.643+0000 7f31fac00640 1 -- 172.21.15.123:0/3394477484 wait complete. 2024-09-19T04:48:03.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:03 smithi183 ceph-mon[35333]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:48:03.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:03 smithi123 ceph-mon[30511]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:48:03.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:03 smithi193 ceph-mon[35592]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:48:04.464 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:48:04.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:04 smithi123 ceph-mon[30511]: scrub queued for path: / 2024-09-19T04:48:04.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:04 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:04.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:04 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:48:04.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:04 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:04.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:04 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:48:04.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:04 smithi183 ceph-mon[35333]: scrub queued for path: / 2024-09-19T04:48:04.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:04 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:04.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:04 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:48:04.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:04 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:04.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:04 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:48:04.851 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:48:04.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:04 smithi193 ceph-mon[35592]: scrub queued for path: / 2024-09-19T04:48:04.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:04 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:04.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:04 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:48:04.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:04 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:04.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:04 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:48:05.231 INFO:tasks.workunit.client.0.smithi123.stderr:Updating files: 62% (8087/12896) Updating files: 63% (8125/12896) Updating files: 64% (8254/12896) Updating files: 65% (8383/12896) Updating files: 66% (8512/12896) Updating files: 67% (8641/12896) Updating files: 68% (8770/12896) Updating files: 69% (8899/12896) Updating files: 70% (9028/12896) Updating files: 71% (9157/12896) Updating files: 72% (9286/12896) Updating files: 73% (9415/12896) Updating files: 74% (9544/12896) Updating files: 75% (9672/12896) Updating files: 76% (9801/12896) Updating files: 77% (9930/12896) Updating files: 78% (10059/12896) Updating files: 79% (10188/12896) Updating files: 80% (10317/12896) Updating files: 81% (10446/12896) Updating files: 82% (10575/12896) Updating files: 83% (10704/12896) Updating files: 84% (10833/12896) Updating files: 85% (10962/12896) Updating files: 86% (11091/12896) Updating files: 87% (11220/12896) Updating files: 88% (11349/12896) Updating files: 89% (11478/12896) Updating files: 90% (11607/12896) Updating files: 91% (11736/12896) Updating files: 92% (11865/12896) Updating files: 93% (11994/12896) Updating files: 94% (12123/12896) Updating files: 95% (12252/12896) Updating files: 96% (12381/12896) Updating files: 97% (12510/12896) Updating files: 97% (12612/12896) Updating files: 98% (12639/12896) Updating files: 99% (12768/12896) Updating files: 100% (12896/12896) Updating files: 100% (12896/12896), done. 2024-09-19T04:48:05.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.451+0000 7f2bfaa00640 1 -- 172.21.15.123:0/116741346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bf4106f20 msgr2=0x7f2bf410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:05.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.451+0000 7f2bfaa00640 1 --2- 172.21.15.123:0/116741346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bf4106f20 0x7f2bf410eaa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009ca0 tx=0x7f2bec02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.451+0000 7f2bfaa00640 1 -- 172.21.15.123:0/116741346 shutdown_connections 2024-09-19T04:48:05.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.451+0000 7f2bfaa00640 1 --2- 172.21.15.123:0/116741346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bf4106f20 0x7f2bf410eaa0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.451+0000 7f2bfaa00640 1 --2- 172.21.15.123:0/116741346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bf41027a0 0x7f2bf41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.451+0000 7f2bfaa00640 1 --2- 172.21.15.123:0/116741346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bf4101df0 0x7f2bf41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.451+0000 7f2bfaa00640 1 -- 172.21.15.123:0/116741346 >> 172.21.15.123:0/116741346 conn(0x7f2bf40fd8b0 msgr2=0x7f2bf40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:05.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.451+0000 7f2bfaa00640 1 -- 172.21.15.123:0/116741346 shutdown_connections 2024-09-19T04:48:05.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.451+0000 7f2bfaa00640 1 -- 172.21.15.123:0/116741346 wait complete. 2024-09-19T04:48:05.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.452+0000 7f2bfaa00640 1 Processor -- start 2024-09-19T04:48:05.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.452+0000 7f2bfaa00640 1 -- start start 2024-09-19T04:48:05.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.454+0000 7f2bfaa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bf4101df0 0x7f2bf4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.454+0000 7f2bfaa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bf41027a0 0x7f2bf4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.454+0000 7f2bfaa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bf4106f20 0x7f2bf419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.454+0000 7f2bfaa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf4111f80 con 0x7f2bf41027a0 2024-09-19T04:48:05.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.454+0000 7f2bfaa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf4111e00 con 0x7f2bf4106f20 2024-09-19T04:48:05.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.454+0000 7f2bfaa00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf4112100 con 0x7f2bf4101df0 2024-09-19T04:48:05.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.454+0000 7f2bf3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bf4101df0 0x7f2bf4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.454+0000 7f2bebe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bf41027a0 0x7f2bf4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.454+0000 7f2bf3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bf4101df0 0x7f2bf4198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:54276/0 (socket says 172.21.15.123:54276) 2024-09-19T04:48:05.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.454+0000 7f2bf3e00640 1 -- 172.21.15.123:0/3869247207 learned_addr learned my addr 172.21.15.123:0/3869247207 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:05.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.454+0000 7f2bf8a00640 1 --2- 172.21.15.123:0/3869247207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bf4106f20 0x7f2bf419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.455+0000 7f2bebe00640 1 -- 172.21.15.123:0/3869247207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bf4101df0 msgr2=0x7f2bf4198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:05.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.455+0000 7f2bebe00640 1 --2- 172.21.15.123:0/3869247207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bf4101df0 0x7f2bf4198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.455+0000 7f2bebe00640 1 -- 172.21.15.123:0/3869247207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bf4106f20 msgr2=0x7f2bf419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:05.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.455+0000 7f2bebe00640 1 --2- 172.21.15.123:0/3869247207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bf4106f20 0x7f2bf419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.455+0000 7f2bebe00640 1 -- 172.21.15.123:0/3869247207 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf419d3b0 con 0x7f2bf41027a0 2024-09-19T04:48:05.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.455+0000 7f2bf8a00640 1 --2- 172.21.15.123:0/3869247207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bf4106f20 0x7f2bf419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:48:05.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.455+0000 7f2bf3e00640 1 --2- 172.21.15.123:0/3869247207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bf4101df0 0x7f2bf4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:05.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.455+0000 7f2bebe00640 1 --2- 172.21.15.123:0/3869247207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bf41027a0 0x7f2bf4198940 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00d6d0 tx=0x7f2bdc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:05.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.455+0000 7f2bf1600640 1 -- 172.21.15.123:0/3869247207 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc017070 con 0x7f2bf41027a0 2024-09-19T04:48:05.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.455+0000 7f2bf1600640 1 -- 172.21.15.123:0/3869247207 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bdc009820 con 0x7f2bf41027a0 2024-09-19T04:48:05.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.455+0000 7f2bf1600640 1 -- 172.21.15.123:0/3869247207 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc004d00 con 0x7f2bf41027a0 2024-09-19T04:48:05.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.455+0000 7f2bfaa00640 1 -- 172.21.15.123:0/3869247207 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf419d6a0 con 0x7f2bf41027a0 2024-09-19T04:48:05.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.456+0000 7f2bfaa00640 1 -- 172.21.15.123:0/3869247207 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf41a4f80 con 0x7f2bf41027a0 2024-09-19T04:48:05.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.459+0000 7f2bf1600640 1 -- 172.21.15.123:0/3869247207 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7f2bdc0099d0 con 0x7f2bf41027a0 2024-09-19T04:48:05.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.459+0000 7f2bfd5e7740 1 Processor -- start 2024-09-19T04:48:05.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.459+0000 7f2bfd5e7740 1 -- start start 2024-09-19T04:48:05.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.459+0000 7f2bfd5e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089cfdb90 0x560089cfdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.459+0000 7f2bfd5e7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfe4b0 0x560089d08830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.459+0000 7f2bfd5e7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089d08d70 0x560089d0b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.459+0000 7f2bfd5e7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560089c643e0 con 0x560089d08d70 2024-09-19T04:48:05.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.459+0000 7f2bfd5e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560089c0bd80 con 0x560089cfdb90 2024-09-19T04:48:05.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.459+0000 7f2bfd5e7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560089c603a0 con 0x560089cfe4b0 2024-09-19T04:48:05.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.460+0000 7f2bf3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfe4b0 0x560089d08830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.460+0000 7f2bf8a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089cfdb90 0x560089cfdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.460+0000 7f2bf3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfe4b0 0x560089d08830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:54278/0 (socket says 172.21.15.123:54278) 2024-09-19T04:48:05.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.460+0000 7f2bf8a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089cfdb90 0x560089cfdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:46782/0 (socket says 172.21.15.123:46782) 2024-09-19T04:48:05.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.460+0000 7f2bf3e00640 1 -- 172.21.15.123:0/3759669195 learned_addr learned my addr 172.21.15.123:0/3759669195 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:05.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.460+0000 7f2bebe00640 1 --2- 172.21.15.123:0/3759669195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089d08d70 0x560089d0b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.460+0000 7f2bf3e00640 1 -- 172.21.15.123:0/3759669195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089cfdb90 msgr2=0x560089cfdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:05.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.460+0000 7f2bf3e00640 1 --2- 172.21.15.123:0/3759669195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089cfdb90 0x560089cfdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.460+0000 7f2bf3e00640 1 -- 172.21.15.123:0/3759669195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089d08d70 msgr2=0x560089d0b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.460+0000 7f2bf3e00640 1 --2- 172.21.15.123:0/3759669195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089d08d70 0x560089d0b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.460+0000 7f2bf3e00640 1 -- 172.21.15.123:0/3759669195 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560089c2aae0 con 0x560089cfe4b0 2024-09-19T04:48:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.460+0000 7f2bf1600640 1 --2- 172.21.15.123:0/3869247207 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2bd80a35e0 0x7f2bd80a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.461+0000 7f2bf1600640 1 -- 172.21.15.123:0/3869247207 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f2bdc0cfa80 con 0x7f2bf41027a0 2024-09-19T04:48:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.461+0000 7f2bf3e00640 1 --2- 172.21.15.123:0/3759669195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfe4b0 0x560089d08830 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2be4010580 tx=0x7f2be4010a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c208534859094254 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.461+0000 7f2bf8a00640 1 --2- 172.21.15.123:0/3869247207 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2bd80a35e0 0x7f2bd80a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.461+0000 7f2be9200640 1 -- 172.21.15.123:0/3759669195 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be400e890 con 0x560089cfe4b0 2024-09-19T04:48:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.461+0000 7f2be9200640 1 -- 172.21.15.123:0/3759669195 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be401c480 con 0x560089cfe4b0 2024-09-19T04:48:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.461+0000 7f2be9200640 1 -- 172.21.15.123:0/3759669195 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be40116a0 con 0x560089cfe4b0 2024-09-19T04:48:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.462+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/3759669195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfe4b0 msgr2=0x560089d08830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.462+0000 7f2bf8a00640 1 --2- 172.21.15.123:0/3869247207 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2bd80a35e0 0x7f2bd80a5aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2bec02f540 tx=0x7f2bec02f310 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:05.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.462+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/3759669195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfe4b0 0x560089d08830 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2be4010580 tx=0x7f2be4010a40 comp rx=0 tx=0).stop 2024-09-19T04:48:05.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.462+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/3759669195 shutdown_connections 2024-09-19T04:48:05.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.462+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/3759669195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089d08d70 0x560089d0b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.462+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/3759669195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfe4b0 0x560089d08830 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.462+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/3759669195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089cfdb90 0x560089cfdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.462+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/3759669195 >> 172.21.15.123:0/3759669195 conn(0x560089cfa120 msgr2=0x560089cfa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:05.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.462+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/3759669195 shutdown_connections 2024-09-19T04:48:05.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.462+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/3759669195 wait complete. 2024-09-19T04:48:05.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.463+0000 7f2bfd5e7740 1 -- start start 2024-09-19T04:48:05.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.463+0000 7f2bfd5e7740 1 Processor -- start 2024-09-19T04:48:05.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.464+0000 7f2bfd5e7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfdb90 0x560089d05c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.464+0000 7f2bfd5e7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfe4b0 0x560089d06190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.464+0000 7f2bfd5e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089d08d70 0x560089d066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.464+0000 7f2bfd5e7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560089d0b9e0 con 0x560089cfdb90 2024-09-19T04:48:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.464+0000 7f2bfd5e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560089c1f0f0 con 0x560089d08d70 2024-09-19T04:48:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.464+0000 7f2bfd5e7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560089d0bb60 con 0x560089cfe4b0 2024-09-19T04:48:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.464+0000 7f2bfd5e7740 10 client.0 _ll_register_callbacks cb 0x560089c109e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.464+0000 7f2bf3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfdb90 0x560089d05c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.464+0000 7f2bebe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfe4b0 0x560089d06190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.464+0000 7f2bf8a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089d08d70 0x560089d066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.464+0000 7f2bf3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfdb90 0x560089d05c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39326/0 (socket says 172.21.15.123:39326) 2024-09-19T04:48:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.464+0000 7f2bf3e00640 1 -- 172.21.15.123:0/740948551 learned_addr learned my addr 172.21.15.123:0/740948551 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:05.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.464+0000 7f2bf3e00640 1 -- 172.21.15.123:0/740948551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfe4b0 msgr2=0x560089d06190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:05.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.464+0000 7f2bf3e00640 1 --2- 172.21.15.123:0/740948551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfe4b0 0x560089d06190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.465+0000 7f2bf3e00640 1 -- 172.21.15.123:0/740948551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089d08d70 msgr2=0x560089d066d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:05.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.465+0000 7f2bf3e00640 1 --2- 172.21.15.123:0/740948551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089d08d70 0x560089d066d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.465+0000 7f2bf3e00640 1 -- 172.21.15.123:0/740948551 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600899d53c0 con 0x560089cfdb90 2024-09-19T04:48:05.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.465+0000 7f2bebe00640 1 --2- 172.21.15.123:0/740948551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfe4b0 0x560089d06190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:05.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.465+0000 7f2bf8a00640 1 --2- 172.21.15.123:0/740948551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089d08d70 0x560089d066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:05.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.465+0000 7f2bf3e00640 1 --2- 172.21.15.123:0/740948551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfdb90 0x560089d05c50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2be4010f40 tx=0x7f2be401c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37badc68bace2dd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:05.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.465+0000 7f2be1600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:48:05.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.465+0000 7f2be1600640 1 -- 172.21.15.123:0/740948551 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be401cc90 con 0x560089cfdb90 2024-09-19T04:48:05.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.465+0000 7f2be1600640 1 -- 172.21.15.123:0/740948551 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f2be4004030 con 0x560089cfdb90 2024-09-19T04:48:05.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.465+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/740948551 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be401c480 con 0x560089cfdb90 2024-09-19T04:48:05.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.465+0000 7f2be1600640 1 -- 172.21.15.123:0/740948551 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2be4004230 con 0x560089cfdb90 2024-09-19T04:48:05.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.466+0000 7f2bfd5e7740 10 client.15549 fetch_fsmap learned FSMap version 28 2024-09-19T04:48:05.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.466+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/740948551 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560089d17690 con 0x560089cfdb90 2024-09-19T04:48:05.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.466+0000 7f2be1600640 1 -- 172.21.15.123:0/740948551 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be401c480 con 0x560089cfdb90 2024-09-19T04:48:05.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.466+0000 7f2be1600640 1 -- 172.21.15.123:0/740948551 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2be4003710 con 0x560089cfdb90 2024-09-19T04:48:05.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.466+0000 7f2bfd5e7740 10 client.15549 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:48:05.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.466+0000 7f2bfd5e7740 10 client.15549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:48:05.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.466+0000 7f2bfd5e7740 20 client.15549 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:05.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.466+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/740948551 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x560089d18aa0 0x560089d1ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.466+0000 7f2bfd5e7740 4 client.15549 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:48:05.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.466+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/740948551 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560089c71950 con 0x560089d18aa0 2024-09-19T04:48:05.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.467+0000 7f2bebe00640 1 --2- 172.21.15.123:0/740948551 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x560089d18aa0 0x560089d1ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.467+0000 7f2be1600640 10 client.15549 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.467+0000 7f2bebe00640 1 --2- 172.21.15.123:0/740948551 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x560089d18aa0 0x560089d1ae90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ba216f52e9bc8f server_cookie=44625e5f1abb3134 in_seq=0 out_seq=0 2024-09-19T04:48:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.468+0000 7f2be1600640 1 -- 172.21.15.123:0/740948551 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560089c71950 con 0x560089d18aa0 2024-09-19T04:48:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.468+0000 7f2be1600640 10 client.15549 handle_command_reply: tid=0 2024-09-19T04:48:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.468+0000 7f2bfd5e7740 1 client.15549 shutdown 2024-09-19T04:48:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.468+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/740948551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfdb90 msgr2=0x560089d05c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.468+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/740948551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfdb90 0x560089d05c50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2be4010f40 tx=0x7f2be401c860 comp rx=0 tx=0).stop 2024-09-19T04:48:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.468+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/740948551 shutdown_connections 2024-09-19T04:48:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.469+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/740948551 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x560089d18aa0 0x560089d1ae90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.469+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/740948551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089d08d70 0x560089d066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.469+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/740948551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfe4b0 0x560089d06190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.469+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/740948551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfdb90 0x560089d05c50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.469+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/740948551 >> 172.21.15.123:0/740948551 conn(0x560089cfa120 msgr2=0x560089cffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.469+0000 7f2be1600640 0 client.15549 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:05.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.469+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/740948551 shutdown_connections 2024-09-19T04:48:05.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.469+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/740948551 wait complete. 2024-09-19T04:48:05.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.469+0000 7f2bfd5e7740 20 client.15549 trim_cache size 0 max 16384 2024-09-19T04:48:05.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.486+0000 7f2bfd5e7740 1 Processor -- start 2024-09-19T04:48:05.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.487+0000 7f2bfd5e7740 1 -- start start 2024-09-19T04:48:05.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.487+0000 7f2bfd5e7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfdb90 0x560089d204c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.487+0000 7f2bfd5e7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfe4b0 0x560089d20a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.487+0000 7f2bfd5e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089d08d70 0x560089cf89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.487+0000 7f2bfd5e7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41ab610 con 0x560089cfe4b0 2024-09-19T04:48:05.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.487+0000 7f2bfd5e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41ad150 con 0x560089d08d70 2024-09-19T04:48:05.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.487+0000 7f2bfd5e7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41abce0 con 0x560089cfdb90 2024-09-19T04:48:05.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.487+0000 7f2bf3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfdb90 0x560089d204c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.487+0000 7f2bf3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfdb90 0x560089d204c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:54296/0 (socket says 172.21.15.123:54296) 2024-09-19T04:48:05.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.487+0000 7f2bf3e00640 1 -- 172.21.15.123:0/637028406 learned_addr learned my addr 172.21.15.123:0/637028406 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:05.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.487+0000 7f2bebe00640 1 --2- 172.21.15.123:0/637028406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfe4b0 0x560089d20a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.487+0000 7f2bf8a00640 1 --2- 172.21.15.123:0/637028406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089d08d70 0x560089cf89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.488+0000 7f2bf3e00640 1 -- 172.21.15.123:0/637028406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089d08d70 msgr2=0x560089cf89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:05.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.488+0000 7f2bf3e00640 1 --2- 172.21.15.123:0/637028406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089d08d70 0x560089cf89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.488+0000 7f2bf3e00640 1 -- 172.21.15.123:0/637028406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfe4b0 msgr2=0x560089d20a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:05.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.488+0000 7f2bf3e00640 1 --2- 172.21.15.123:0/637028406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfe4b0 0x560089d20a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.488+0000 7f2bf3e00640 1 -- 172.21.15.123:0/637028406 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560089cfae10 con 0x560089cfdb90 2024-09-19T04:48:05.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.488+0000 7f2bf3e00640 1 --2- 172.21.15.123:0/637028406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfdb90 0x560089d204c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2be400f010 tx=0x7f2be4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ca51611c206036d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:05.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.488+0000 7f2be3e00640 1 -- 172.21.15.123:0/637028406 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be401c8d0 con 0x560089cfdb90 2024-09-19T04:48:05.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.489+0000 7f2be3e00640 1 -- 172.21.15.123:0/637028406 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be4035030 con 0x560089cfdb90 2024-09-19T04:48:05.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.489+0000 7f2be3e00640 1 -- 172.21.15.123:0/637028406 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4028340 con 0x560089cfdb90 2024-09-19T04:48:05.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.489+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/637028406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfdb90 msgr2=0x560089d204c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:05.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.489+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/637028406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfdb90 0x560089d204c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2be400f010 tx=0x7f2be4000bc0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.489+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/637028406 shutdown_connections 2024-09-19T04:48:05.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.489+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/637028406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089d08d70 0x560089cf89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.489+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/637028406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfe4b0 0x560089d20a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.489+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/637028406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089cfdb90 0x560089d204c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.489+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/637028406 >> 172.21.15.123:0/637028406 conn(0x560089cfa120 msgr2=0x560089d04d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:05.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.490+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/637028406 shutdown_connections 2024-09-19T04:48:05.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.490+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/637028406 wait complete. 2024-09-19T04:48:05.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.490+0000 7f2bfd5e7740 1 -- start start 2024-09-19T04:48:05.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.490+0000 7f2bfd5e7740 1 Processor -- start 2024-09-19T04:48:05.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bfd5e7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfdb90 0x560089d10f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bfd5e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089cfe4b0 0x560089d114c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bfd5e7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089d08d70 0x560089d0c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bfd5e7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41aca80 con 0x560089cfdb90 2024-09-19T04:48:05.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bfd5e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41ac3b0 con 0x560089cfe4b0 2024-09-19T04:48:05.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bfd5e7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41a95f0 con 0x560089d08d70 2024-09-19T04:48:05.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bfd5e7740 10 client.0 _ll_register_callbacks cb 0x7f2be400bd20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:05.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bf3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfdb90 0x560089d10f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bf8a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089d08d70 0x560089d0c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bf8a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089d08d70 0x560089d0c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:54306/0 (socket says 172.21.15.123:54306) 2024-09-19T04:48:05.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bf8a00640 1 -- 172.21.15.123:0/3069584158 learned_addr learned my addr 172.21.15.123:0/3069584158 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:05.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bebe00640 1 --2- 172.21.15.123:0/3069584158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089cfe4b0 0x560089d114c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bf8a00640 1 -- 172.21.15.123:0/3069584158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089cfe4b0 msgr2=0x560089d114c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:05.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bf8a00640 1 --2- 172.21.15.123:0/3069584158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089cfe4b0 0x560089d114c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bf8a00640 1 -- 172.21.15.123:0/3069584158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfdb90 msgr2=0x560089d10f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:05.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bf8a00640 1 --2- 172.21.15.123:0/3069584158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfdb90 0x560089d10f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bf8a00640 1 -- 172.21.15.123:0/3069584158 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560089d0cb70 con 0x560089d08d70 2024-09-19T04:48:05.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.491+0000 7f2bf3e00640 1 --2- 172.21.15.123:0/3069584158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfdb90 0x560089d10f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:05.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.492+0000 7f2bf8a00640 1 --2- 172.21.15.123:0/3069584158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089d08d70 0x560089d0c340 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009ca0 tx=0x7f2bec02a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7891ba096c7d6ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:05.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.492+0000 7f2be1600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:48:05.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.492+0000 7f2be1600640 1 -- 172.21.15.123:0/3069584158 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec058060 con 0x560089d08d70 2024-09-19T04:48:05.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.492+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/3069584158 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be4035030 con 0x560089d08d70 2024-09-19T04:48:05.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.492+0000 7f2be1600640 1 -- 172.21.15.123:0/3069584158 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f2bec0046e0 con 0x560089d08d70 2024-09-19T04:48:05.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.492+0000 7f2be1600640 1 -- 172.21.15.123:0/3069584158 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2bec02f610 con 0x560089d08d70 2024-09-19T04:48:05.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.492+0000 7f2bfd5e7740 10 client.25603 fetch_fsmap learned FSMap version 28 2024-09-19T04:48:05.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.492+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/3069584158 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560089d83020 con 0x560089d08d70 2024-09-19T04:48:05.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.493+0000 7f2be1600640 1 -- 172.21.15.123:0/3069584158 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be4035030 con 0x560089d08d70 2024-09-19T04:48:05.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.493+0000 7f2be1600640 1 -- 172.21.15.123:0/3069584158 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2bec052070 con 0x560089d08d70 2024-09-19T04:48:05.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.493+0000 7f2bfd5e7740 10 client.25603 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:48:05.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.493+0000 7f2bfd5e7740 10 client.25603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:48:05.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.493+0000 7f2bfd5e7740 20 client.25603 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:05.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.493+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/3069584158 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x560089d83750 0x560089d85b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:05.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.493+0000 7f2bfd5e7740 4 client.25603 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:48:05.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.493+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/3069584158 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560089c6be60 con 0x560089d83750 2024-09-19T04:48:05.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.493+0000 7f2bf3e00640 1 --2- 172.21.15.123:0/3069584158 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x560089d83750 0x560089d85b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:05.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.494+0000 7f2be1600640 10 client.25603 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:05.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.494+0000 7f2bf3e00640 1 --2- 172.21.15.123:0/3069584158 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x560089d83750 0x560089d85b70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=769510188e54f276 server_cookie=d9ce13f669cc58a8 in_seq=0 out_seq=0 2024-09-19T04:48:05.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.494+0000 7f2be1600640 1 -- 172.21.15.123:0/3069584158 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560089c6be60 con 0x560089d83750 2024-09-19T04:48:05.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.494+0000 7f2be1600640 10 client.25603 handle_command_reply: tid=0 2024-09-19T04:48:05.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.494+0000 7f2bfd5e7740 1 client.25603 shutdown 2024-09-19T04:48:05.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.495+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/3069584158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089d08d70 msgr2=0x560089d0c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:05.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.495+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/3069584158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089d08d70 0x560089d0c340 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009ca0 tx=0x7f2bec02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.495+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/3069584158 shutdown_connections 2024-09-19T04:48:05.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.495+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/3069584158 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x560089d83750 0x560089d85b70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.495+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/3069584158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560089d08d70 0x560089d0c340 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.495+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/3069584158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560089cfe4b0 0x560089d114c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.495+0000 7f2bfd5e7740 1 --2- 172.21.15.123:0/3069584158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560089cfdb90 0x560089d10f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.495+0000 7f2be1600640 0 client.25603 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:05.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.495+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/3069584158 >> 172.21.15.123:0/3069584158 conn(0x560089cfa120 msgr2=0x560089cff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:05.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.495+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/3069584158 shutdown_connections 2024-09-19T04:48:05.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.495+0000 7f2bfd5e7740 1 -- 172.21.15.123:0/3069584158 wait complete. 2024-09-19T04:48:05.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.495+0000 7f2bfd5e7740 20 client.25603 trim_cache size 0 max 16384 2024-09-19T04:48:05.502 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:48:05.502 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-09-19T04:48:05.503 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-09-19T04:48:05.503 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:48:05.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.496+0000 7f2bfaa00640 1 -- 172.21.15.123:0/3869247207 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2bd80a35e0 msgr2=0x7f2bd80a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:05.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.496+0000 7f2bfaa00640 1 --2- 172.21.15.123:0/3869247207 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2bd80a35e0 0x7f2bd80a5aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2bec02f540 tx=0x7f2bec02f310 comp rx=0 tx=0).stop 2024-09-19T04:48:05.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.496+0000 7f2bfaa00640 1 -- 172.21.15.123:0/3869247207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bf41027a0 msgr2=0x7f2bf4198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:05.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.496+0000 7f2bfaa00640 1 --2- 172.21.15.123:0/3869247207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bf41027a0 0x7f2bf4198940 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00d6d0 tx=0x7f2bdc00db90 comp rx=0 tx=0).stop 2024-09-19T04:48:05.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.496+0000 7f2bfaa00640 1 -- 172.21.15.123:0/3869247207 shutdown_connections 2024-09-19T04:48:05.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.496+0000 7f2bfaa00640 1 --2- 172.21.15.123:0/3869247207 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2bd80a35e0 0x7f2bd80a5aa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.496+0000 7f2bfaa00640 1 --2- 172.21.15.123:0/3869247207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bf4106f20 0x7f2bf419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.496+0000 7f2bfaa00640 1 --2- 172.21.15.123:0/3869247207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bf41027a0 0x7f2bf4198940 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.496+0000 7f2bfaa00640 1 --2- 172.21.15.123:0/3869247207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bf4101df0 0x7f2bf4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:05.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.496+0000 7f2bfaa00640 1 -- 172.21.15.123:0/3869247207 >> 172.21.15.123:0/3869247207 conn(0x7f2bf40fd8b0 msgr2=0x7f2bf40feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:05.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.496+0000 7f2bfaa00640 1 -- 172.21.15.123:0/3869247207 shutdown_connections 2024-09-19T04:48:05.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:05.496+0000 7f2bfaa00640 1 -- 172.21.15.123:0/3869247207 wait complete. 2024-09-19T04:48:05.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:05 smithi123 ceph-mon[30511]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T04:48:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:05 smithi183 ceph-mon[35333]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T04:48:05.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:05 smithi193 ceph-mon[35592]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T04:48:05.995 INFO:tasks.workunit.client.0.smithi123.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T04:48:05.995 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-19T04:48:05.996 INFO:tasks.workunit.client.0.smithi123.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T04:48:05.996 INFO:tasks.workunit.client.0.smithi123.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T04:48:05.996 INFO:tasks.workunit.client.0.smithi123.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T04:48:05.996 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-19T04:48:05.996 INFO:tasks.workunit.client.0.smithi123.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T04:48:05.996 INFO:tasks.workunit.client.0.smithi123.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T04:48:05.996 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-19T04:48:05.996 INFO:tasks.workunit.client.0.smithi123.stderr: git switch -c 2024-09-19T04:48:05.996 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-19T04:48:05.996 INFO:tasks.workunit.client.0.smithi123.stderr:Or undo this operation with: 2024-09-19T04:48:05.996 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-19T04:48:05.996 INFO:tasks.workunit.client.0.smithi123.stderr: git switch - 2024-09-19T04:48:05.997 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-19T04:48:05.997 INFO:tasks.workunit.client.0.smithi123.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T04:48:05.997 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-19T04:48:05.997 INFO:tasks.workunit.client.0.smithi123.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T04:48:06.010 DEBUG:teuthology.orchestra.run.smithi123:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-19T04:48:06.105 INFO:tasks.workunit.client.0.smithi123.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T04:48:06.110 INFO:tasks.workunit.client.0.smithi123.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T04:48:06.110 INFO:tasks.workunit.client.0.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T04:48:06.512 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:48:06.513 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:48:06.887 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:48:07.468 INFO:tasks.workunit.client.0.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T04:48:07.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.522+0000 7f5737e00640 1 -- 172.21.15.123:0/3750610977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57381019d0 msgr2=0x7f5738101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:07.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.522+0000 7f5737e00640 1 --2- 172.21.15.123:0/3750610977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57381019d0 0x7f5738101e30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5724009930 tx=0x7f572402a420 comp rx=0 tx=0).stop 2024-09-19T04:48:07.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.522+0000 7f5737e00640 1 -- 172.21.15.123:0/3750610977 shutdown_connections 2024-09-19T04:48:07.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.522+0000 7f5737e00640 1 --2- 172.21.15.123:0/3750610977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5738102370 0x7f573810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:07.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.522+0000 7f5737e00640 1 --2- 172.21.15.123:0/3750610977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57381019d0 0x7f5738101e30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:07.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.522+0000 7f5737e00640 1 --2- 172.21.15.123:0/3750610977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57381079d0 0x7f5738107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:07.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.522+0000 7f5737e00640 1 -- 172.21.15.123:0/3750610977 >> 172.21.15.123:0/3750610977 conn(0x7f57380fd710 msgr2=0x7f57380ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:07.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.522+0000 7f5737e00640 1 -- 172.21.15.123:0/3750610977 shutdown_connections 2024-09-19T04:48:07.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.522+0000 7f5737e00640 1 -- 172.21.15.123:0/3750610977 wait complete. 2024-09-19T04:48:07.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.523+0000 7f5737e00640 1 Processor -- start 2024-09-19T04:48:07.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.523+0000 7f5737e00640 1 -- start start 2024-09-19T04:48:07.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.523+0000 7f5737e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57381019d0 0x7f57381982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:07.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.523+0000 7f5737e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5738102370 0x7f5738198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:07.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.523+0000 7f5737e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57381079d0 0x7f573819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:07.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.523+0000 7f5737e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5738111de0 con 0x7f57381019d0 2024-09-19T04:48:07.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.523+0000 7f5737e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5738111c60 con 0x7f57381079d0 2024-09-19T04:48:07.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.523+0000 7f5737e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5738111f60 con 0x7f5738102370 2024-09-19T04:48:07.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.524+0000 7f5736a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57381019d0 0x7f57381982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:07.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.524+0000 7f5736000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5738102370 0x7f5738198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:07.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.524+0000 7f5736000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5738102370 0x7f5738198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:54336/0 (socket says 172.21.15.123:54336) 2024-09-19T04:48:07.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.524+0000 7f5736000640 1 -- 172.21.15.123:0/2942554087 learned_addr learned my addr 172.21.15.123:0/2942554087 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:07.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.524+0000 7f5737400640 1 --2- 172.21.15.123:0/2942554087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57381079d0 0x7f573819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:07.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.524+0000 7f5736a00640 1 -- 172.21.15.123:0/2942554087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5738102370 msgr2=0x7f5738198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:07.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.524+0000 7f5736a00640 1 --2- 172.21.15.123:0/2942554087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5738102370 0x7f5738198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:07.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.524+0000 7f5736a00640 1 -- 172.21.15.123:0/2942554087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57381079d0 msgr2=0x7f573819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:07.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.524+0000 7f5736a00640 1 --2- 172.21.15.123:0/2942554087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57381079d0 0x7f573819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:07.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.524+0000 7f5736a00640 1 -- 172.21.15.123:0/2942554087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573819d290 con 0x7f57381019d0 2024-09-19T04:48:07.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.524+0000 7f5736000640 1 --2- 172.21.15.123:0/2942554087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5738102370 0x7f5738198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:07.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.525+0000 7f5736a00640 1 --2- 172.21.15.123:0/2942554087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57381019d0 0x7f57381982e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f572800e9f0 tx=0x7f572800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:07.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.525+0000 7f572f400640 1 -- 172.21.15.123:0/2942554087 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572800ccc0 con 0x7f57381019d0 2024-09-19T04:48:07.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.525+0000 7f572f400640 1 -- 172.21.15.123:0/2942554087 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5728010910 con 0x7f57381019d0 2024-09-19T04:48:07.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.525+0000 7f572f400640 1 -- 172.21.15.123:0/2942554087 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728018960 con 0x7f57381019d0 2024-09-19T04:48:07.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.525+0000 7f5737e00640 1 -- 172.21.15.123:0/2942554087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573819d580 con 0x7f57381019d0 2024-09-19T04:48:07.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.525+0000 7f5737e00640 1 -- 172.21.15.123:0/2942554087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57381a4e60 con 0x7f57381019d0 2024-09-19T04:48:07.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.526+0000 7f5737e00640 1 -- 172.21.15.123:0/2942554087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5738101e30 con 0x7f57381019d0 2024-09-19T04:48:07.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.528+0000 7f572f400640 1 -- 172.21.15.123:0/2942554087 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7f5728010ac0 con 0x7f57381019d0 2024-09-19T04:48:07.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.529+0000 7f572f400640 1 --2- 172.21.15.123:0/2942554087 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f571c0a35e0 0x7f571c0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:07.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.529+0000 7f572f400640 1 -- 172.21.15.123:0/2942554087 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f57280d0260 con 0x7f57381019d0 2024-09-19T04:48:07.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.529+0000 7f5736000640 1 --2- 172.21.15.123:0/2942554087 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f571c0a35e0 0x7f571c0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:07.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.530+0000 7f5736000640 1 --2- 172.21.15.123:0/2942554087 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f571c0a35e0 0x7f571c0a5aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f57380ffce0 tx=0x7f5724031510 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:07.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.533+0000 7f572f400640 1 -- 172.21.15.123:0/2942554087 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f572809c3e0 con 0x7f57381019d0 2024-09-19T04:48:07.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:07 smithi123 ceph-mon[30511]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T04:48:07.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.750+0000 7f5737e00640 1 -- 172.21.15.123:0/2942554087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5738109f60 con 0x7f57381019d0 2024-09-19T04:48:07.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.751+0000 7f572f400640 1 -- 172.21.15.123:0/2942554087 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11990 (secure 0 0 0) 0x7f57280a06e0 con 0x7f57381019d0 2024-09-19T04:48:07.753 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:48:07.753 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"btime":"2024-09-19T04:46:42:466383+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:46:41.577049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24499,24500,24539,24505,24529]},"id":1}]} 2024-09-19T04:48:07.754 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 28 2024-09-19T04:48:07.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.754+0000 7f5737e00640 1 -- 172.21.15.123:0/2942554087 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f571c0a35e0 msgr2=0x7f571c0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:07.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.754+0000 7f5737e00640 1 --2- 172.21.15.123:0/2942554087 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f571c0a35e0 0x7f571c0a5aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f57380ffce0 tx=0x7f5724031510 comp rx=0 tx=0).stop 2024-09-19T04:48:07.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.754+0000 7f5737e00640 1 -- 172.21.15.123:0/2942554087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57381019d0 msgr2=0x7f57381982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:07.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.754+0000 7f5737e00640 1 --2- 172.21.15.123:0/2942554087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57381019d0 0x7f57381982e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f572800e9f0 tx=0x7f572800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:48:07.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.754+0000 7f5737e00640 1 -- 172.21.15.123:0/2942554087 shutdown_connections 2024-09-19T04:48:07.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.754+0000 7f5737e00640 1 --2- 172.21.15.123:0/2942554087 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f571c0a35e0 0x7f571c0a5aa0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:07.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.754+0000 7f5737e00640 1 --2- 172.21.15.123:0/2942554087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57381079d0 0x7f573819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:07.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.754+0000 7f5737e00640 1 --2- 172.21.15.123:0/2942554087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5738102370 0x7f5738198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:07.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.754+0000 7f5737e00640 1 --2- 172.21.15.123:0/2942554087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57381019d0 0x7f57381982e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:07.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.754+0000 7f5737e00640 1 -- 172.21.15.123:0/2942554087 >> 172.21.15.123:0/2942554087 conn(0x7f57380fd710 msgr2=0x7f57380fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:07.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.754+0000 7f5737e00640 1 -- 172.21.15.123:0/2942554087 shutdown_connections 2024-09-19T04:48:07.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:07.754+0000 7f5737e00640 1 -- 172.21.15.123:0/2942554087 wait complete. 2024-09-19T04:48:07.764 INFO:tasks.workunit.client.0.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T04:48:07.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:07 smithi183 ceph-mon[35333]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T04:48:07.905 INFO:tasks.workunit.client.0.smithi123.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T04:48:07.909 INFO:tasks.workunit.client.0.smithi123.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T04:48:07.909 INFO:tasks.workunit.client.0.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T04:48:07.975 INFO:tasks.workunit.client.0.smithi123.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T04:48:07.985 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:48:07.985 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-19T04:48:07.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:07 smithi193 ceph-mon[35592]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T04:48:08.022 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-09-19T04:48:08.024 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-09-19T04:48:08.024 DEBUG:teuthology.orchestra.run.smithi123:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-09-19T04:48:08.111 INFO:tasks.workunit.client.0.smithi123.stderr:+ set -e 2024-09-19T04:48:08.112 INFO:tasks.workunit.client.0.smithi123.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-09-19T04:48:08.116 INFO:tasks.workunit.client.0.smithi123.stderr:Cloning into 'xfstests-dev'... 2024-09-19T04:48:08.675 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:48:08.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:08 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2942554087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:48:08.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:08 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2942554087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:48:08.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:08 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2942554087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:48:09.044 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:48:09.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.622+0000 7fdb53e00640 1 -- 172.21.15.123:0/2461037629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb54102360 msgr2=0x7fdb5410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:09.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.622+0000 7fdb53e00640 1 --2- 172.21.15.123:0/2461037629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb54102360 0x7fdb5410e890 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009cd0 tx=0x7fdb4c02a8f0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.622+0000 7fdb53e00640 1 -- 172.21.15.123:0/2461037629 shutdown_connections 2024-09-19T04:48:09.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.622+0000 7fdb53e00640 1 --2- 172.21.15.123:0/2461037629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb54102360 0x7fdb5410e890 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.622+0000 7fdb53e00640 1 --2- 172.21.15.123:0/2461037629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb541019c0 0x7fdb54101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.622+0000 7fdb53e00640 1 --2- 172.21.15.123:0/2461037629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb541079c0 0x7fdb54107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.622+0000 7fdb53e00640 1 -- 172.21.15.123:0/2461037629 >> 172.21.15.123:0/2461037629 conn(0x7fdb540fd740 msgr2=0x7fdb540ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:09.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.622+0000 7fdb53e00640 1 -- 172.21.15.123:0/2461037629 shutdown_connections 2024-09-19T04:48:09.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.622+0000 7fdb53e00640 1 -- 172.21.15.123:0/2461037629 wait complete. 2024-09-19T04:48:09.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.623+0000 7fdb53e00640 1 Processor -- start 2024-09-19T04:48:09.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.624+0000 7fdb53e00640 1 -- start start 2024-09-19T04:48:09.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.624+0000 7fdb53e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb541019c0 0x7fdb541982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.624+0000 7fdb53e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb54102360 0x7fdb541987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.624+0000 7fdb53e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb541079c0 0x7fdb5419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.624+0000 7fdb53e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb54111dc0 con 0x7fdb54102360 2024-09-19T04:48:09.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.624+0000 7fdb53e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb54111c40 con 0x7fdb541019c0 2024-09-19T04:48:09.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.624+0000 7fdb53e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb54111f40 con 0x7fdb541079c0 2024-09-19T04:48:09.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.624+0000 7fdb4be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb54102360 0x7fdb541987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.624+0000 7fdb52a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb541019c0 0x7fdb541982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.624+0000 7fdb4be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb54102360 0x7fdb541987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39390/0 (socket says 172.21.15.123:39390) 2024-09-19T04:48:09.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.624+0000 7fdb4be00640 1 -- 172.21.15.123:0/131025959 learned_addr learned my addr 172.21.15.123:0/131025959 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:09.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.624+0000 7fdb53400640 1 --2- 172.21.15.123:0/131025959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb541079c0 0x7fdb5419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.625+0000 7fdb4be00640 1 -- 172.21.15.123:0/131025959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb541079c0 msgr2=0x7fdb5419cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:09.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.625+0000 7fdb4be00640 1 --2- 172.21.15.123:0/131025959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb541079c0 0x7fdb5419cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.625+0000 7fdb4be00640 1 -- 172.21.15.123:0/131025959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb541019c0 msgr2=0x7fdb541982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:09.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.625+0000 7fdb4be00640 1 --2- 172.21.15.123:0/131025959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb541019c0 0x7fdb541982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.625+0000 7fdb4be00640 1 -- 172.21.15.123:0/131025959 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5419d250 con 0x7fdb54102360 2024-09-19T04:48:09.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.625+0000 7fdb52a00640 1 --2- 172.21.15.123:0/131025959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb541019c0 0x7fdb541982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:09.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.625+0000 7fdb4be00640 1 --2- 172.21.15.123:0/131025959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb54102360 0x7fdb541987e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00e9f0 tx=0x7fdb3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:09.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.625+0000 7fdb4b400640 1 -- 172.21.15.123:0/131025959 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c00cd80 con 0x7fdb54102360 2024-09-19T04:48:09.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.626+0000 7fdb4b400640 1 -- 172.21.15.123:0/131025959 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb3c010910 con 0x7fdb54102360 2024-09-19T04:48:09.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.626+0000 7fdb4b400640 1 -- 172.21.15.123:0/131025959 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c018990 con 0x7fdb54102360 2024-09-19T04:48:09.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.626+0000 7fdb53e00640 1 -- 172.21.15.123:0/131025959 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5419d540 con 0x7fdb54102360 2024-09-19T04:48:09.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.626+0000 7fdb53e00640 1 -- 172.21.15.123:0/131025959 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb541a4e20 con 0x7fdb54102360 2024-09-19T04:48:09.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.629+0000 7fdb5c20b740 1 Processor -- start 2024-09-19T04:48:09.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.629+0000 7fdb4b400640 1 -- 172.21.15.123:0/131025959 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7fdb3c010ac0 con 0x7fdb54102360 2024-09-19T04:48:09.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.629+0000 7fdb5c20b740 1 -- start start 2024-09-19T04:48:09.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.629+0000 7fdb5c20b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 0x55ec5ddf0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.629+0000 7fdb5c20b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddf0760 0x55ec5ddfaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.630+0000 7fdb5c20b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddfb020 0x55ec5ddfd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.630+0000 7fdb5c20b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ec5dd563e0 con 0x55ec5ddefe40 2024-09-19T04:48:09.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.630+0000 7fdb5c20b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ec5dcfdd80 con 0x55ec5ddf0760 2024-09-19T04:48:09.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.630+0000 7fdb5c20b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ec5dd523a0 con 0x55ec5ddfb020 2024-09-19T04:48:09.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.630+0000 7fdb52a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddf0760 0x55ec5ddfaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.630+0000 7fdb52a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddf0760 0x55ec5ddfaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:46840/0 (socket says 172.21.15.123:46840) 2024-09-19T04:48:09.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.630+0000 7fdb52a00640 1 -- 172.21.15.123:0/1249490665 learned_addr learned my addr 172.21.15.123:0/1249490665 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:09.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.630+0000 7fdb4be00640 1 --2- 172.21.15.123:0/1249490665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddfb020 0x55ec5ddfd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.630+0000 7fdb53400640 1 --2- 172.21.15.123:0/1249490665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 0x55ec5ddf0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.630+0000 7fdb52a00640 1 -- 172.21.15.123:0/1249490665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddfb020 msgr2=0x55ec5ddfd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:09.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.630+0000 7fdb52a00640 1 --2- 172.21.15.123:0/1249490665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddfb020 0x55ec5ddfd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.630+0000 7fdb52a00640 1 -- 172.21.15.123:0/1249490665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 msgr2=0x55ec5ddf0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:09.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.630+0000 7fdb52a00640 1 --2- 172.21.15.123:0/1249490665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 0x55ec5ddf0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.630+0000 7fdb52a00640 1 -- 172.21.15.123:0/1249490665 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec5dd1cae0 con 0x55ec5ddf0760 2024-09-19T04:48:09.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.630+0000 7fdb53400640 1 --2- 172.21.15.123:0/1249490665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 0x55ec5ddf0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:48:09.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.631+0000 7fdb52a00640 1 --2- 172.21.15.123:0/1249490665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddf0760 0x55ec5ddfaae0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdb440114c0 tx=0x7fdb44011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=394c8831b9d1b287 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:09.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.631+0000 7fdb43400640 1 -- 172.21.15.123:0/1249490665 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4400f5c0 con 0x55ec5ddf0760 2024-09-19T04:48:09.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.631+0000 7fdb43400640 1 -- 172.21.15.123:0/1249490665 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb4400fc40 con 0x55ec5ddf0760 2024-09-19T04:48:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.631+0000 7fdb43400640 1 -- 172.21.15.123:0/1249490665 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44019dc0 con 0x55ec5ddf0760 2024-09-19T04:48:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.631+0000 7fdb4b400640 1 --2- 172.21.15.123:0/131025959 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb380a35e0 0x7fdb380a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.631+0000 7fdb4b400640 1 -- 172.21.15.123:0/131025959 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fdb3c0d0210 con 0x7fdb54102360 2024-09-19T04:48:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.632+0000 7fdb53400640 1 --2- 172.21.15.123:0/131025959 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb380a35e0 0x7fdb380a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.632+0000 7fdb5c20b740 1 -- 172.21.15.123:0/1249490665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddf0760 msgr2=0x55ec5ddfaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.632+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/1249490665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddf0760 0x55ec5ddfaae0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdb440114c0 tx=0x7fdb44011980 comp rx=0 tx=0).stop 2024-09-19T04:48:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.632+0000 7fdb5c20b740 1 -- 172.21.15.123:0/1249490665 shutdown_connections 2024-09-19T04:48:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.632+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/1249490665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddfb020 0x55ec5ddfd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.632+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/1249490665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddf0760 0x55ec5ddfaae0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.632+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/1249490665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 0x55ec5ddf0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.632+0000 7fdb5c20b740 1 -- 172.21.15.123:0/1249490665 >> 172.21.15.123:0/1249490665 conn(0x55ec5ddec3d0 msgr2=0x55ec5ddec7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.632+0000 7fdb5c20b740 1 -- 172.21.15.123:0/1249490665 shutdown_connections 2024-09-19T04:48:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.632+0000 7fdb5c20b740 1 -- 172.21.15.123:0/1249490665 wait complete. 2024-09-19T04:48:09.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.633+0000 7fdb53400640 1 --2- 172.21.15.123:0/131025959 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb380a35e0 0x7fdb380a5aa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009430 tx=0x7fdb4c0093c0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:09.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.633+0000 7fdb5c20b740 1 -- start start 2024-09-19T04:48:09.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.633+0000 7fdb5c20b740 1 Processor -- start 2024-09-19T04:48:09.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.633+0000 7fdb5c20b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddefe40 0x55ec5ddf7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.633+0000 7fdb5c20b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddf0760 0x55ec5ddf8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.633+0000 7fdb5c20b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddfb020 0x55ec5ddf8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.633+0000 7fdb5c20b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ec5ddfdbf0 con 0x55ec5ddf0760 2024-09-19T04:48:09.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.633+0000 7fdb5c20b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ec5dd110f0 con 0x55ec5ddfb020 2024-09-19T04:48:09.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.633+0000 7fdb5c20b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ec5ddfdd70 con 0x55ec5ddefe40 2024-09-19T04:48:09.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.634+0000 7fdb5c20b740 10 client.0 _ll_register_callbacks cb 0x55ec5dcf8bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:09.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.634+0000 7fdb4be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddf0760 0x55ec5ddf8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.634+0000 7fdb4be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddf0760 0x55ec5ddf8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39400/0 (socket says 172.21.15.123:39400) 2024-09-19T04:48:09.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.634+0000 7fdb4be00640 1 -- 172.21.15.123:0/4189650634 learned_addr learned my addr 172.21.15.123:0/4189650634 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:09.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.634+0000 7fdb52a00640 1 --2- 172.21.15.123:0/4189650634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddefe40 0x55ec5ddf7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.634+0000 7fdb53400640 1 --2- 172.21.15.123:0/4189650634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddfb020 0x55ec5ddf8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.634+0000 7fdb4be00640 1 -- 172.21.15.123:0/4189650634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddefe40 msgr2=0x55ec5ddf7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:09.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.634+0000 7fdb4be00640 1 --2- 172.21.15.123:0/4189650634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddefe40 0x55ec5ddf7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.634+0000 7fdb4be00640 1 -- 172.21.15.123:0/4189650634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddfb020 msgr2=0x55ec5ddf8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:09.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.634+0000 7fdb4be00640 1 --2- 172.21.15.123:0/4189650634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddfb020 0x55ec5ddf8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.634+0000 7fdb4be00640 1 -- 172.21.15.123:0/4189650634 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec5dac73c0 con 0x55ec5ddf0760 2024-09-19T04:48:09.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.634+0000 7fdb52a00640 1 --2- 172.21.15.123:0/4189650634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddefe40 0x55ec5ddf7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:09.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.635+0000 7fdb4be00640 1 --2- 172.21.15.123:0/4189650634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddf0760 0x55ec5ddf8400 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00c6a0 tx=0x7fdb3c09bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d59bb9e46f36b9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:09.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.635+0000 7fdb37e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:48:09.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.635+0000 7fdb37e00640 1 -- 172.21.15.123:0/4189650634 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c0a4f50 con 0x55ec5ddf0760 2024-09-19T04:48:09.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.635+0000 7fdb37e00640 1 -- 172.21.15.123:0/4189650634 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fdb3c018b30 con 0x55ec5ddf0760 2024-09-19T04:48:09.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.635+0000 7fdb5c20b740 1 -- 172.21.15.123:0/4189650634 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb4400fc40 con 0x55ec5ddf0760 2024-09-19T04:48:09.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.635+0000 7fdb37e00640 1 -- 172.21.15.123:0/4189650634 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb3c018d30 con 0x55ec5ddf0760 2024-09-19T04:48:09.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.635+0000 7fdb5c20b740 10 client.15573 fetch_fsmap learned FSMap version 29 2024-09-19T04:48:09.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.635+0000 7fdb5c20b740 1 -- 172.21.15.123:0/4189650634 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec5de09c40 con 0x55ec5ddf0760 2024-09-19T04:48:09.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.636+0000 7fdb37e00640 1 -- 172.21.15.123:0/4189650634 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb4400fc40 con 0x55ec5ddf0760 2024-09-19T04:48:09.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.636+0000 7fdb37e00640 1 -- 172.21.15.123:0/4189650634 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fdb3c0980e0 con 0x55ec5ddf0760 2024-09-19T04:48:09.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.636+0000 7fdb5c20b740 10 client.15573 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:48:09.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.636+0000 7fdb5c20b740 10 client.15573 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:48:09.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.636+0000 7fdb5c20b740 20 client.15573 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:09.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.636+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/4189650634 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55ec5de0b050 0x55ec5de0d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.636+0000 7fdb5c20b740 4 client.15573 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:48:09.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.636+0000 7fdb5c20b740 1 -- 172.21.15.123:0/4189650634 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec5dd63950 con 0x55ec5de0b050 2024-09-19T04:48:09.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.636+0000 7fdb52a00640 1 --2- 172.21.15.123:0/4189650634 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55ec5de0b050 0x55ec5de0d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.637+0000 7fdb52a00640 1 --2- 172.21.15.123:0/4189650634 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55ec5de0b050 0x55ec5de0d4a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=183a5294d91fc0e server_cookie=40b33064e590b383 in_seq=0 out_seq=0 2024-09-19T04:48:09.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.637+0000 7fdb37e00640 10 client.15573 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:48:09.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.638+0000 7fdb37e00640 1 -- 172.21.15.123:0/4189650634 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec5dd63950 con 0x55ec5de0b050 2024-09-19T04:48:09.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.638+0000 7fdb37e00640 10 client.15573 handle_command_reply: tid=0 2024-09-19T04:48:09.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.638+0000 7fdb5c20b740 1 client.15573 shutdown 2024-09-19T04:48:09.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.638+0000 7fdb5c20b740 1 -- 172.21.15.123:0/4189650634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddf0760 msgr2=0x55ec5ddf8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:09.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.638+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/4189650634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddf0760 0x55ec5ddf8400 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00c6a0 tx=0x7fdb3c09bf90 comp rx=0 tx=0).stop 2024-09-19T04:48:09.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.638+0000 7fdb5c20b740 1 -- 172.21.15.123:0/4189650634 shutdown_connections 2024-09-19T04:48:09.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.638+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/4189650634 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55ec5de0b050 0x55ec5de0d4a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.638+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/4189650634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddfb020 0x55ec5ddf8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.638+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/4189650634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddf0760 0x55ec5ddf8400 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.638+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/4189650634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddefe40 0x55ec5ddf7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.638+0000 7fdb37e00640 0 client.15573 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:48:09.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.638+0000 7fdb5c20b740 1 -- 172.21.15.123:0/4189650634 >> 172.21.15.123:0/4189650634 conn(0x55ec5ddec3d0 msgr2=0x55ec5ddf2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:09.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.638+0000 7fdb5c20b740 1 -- 172.21.15.123:0/4189650634 shutdown_connections 2024-09-19T04:48:09.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.638+0000 7fdb5c20b740 1 -- 172.21.15.123:0/4189650634 wait complete. 2024-09-19T04:48:09.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.639+0000 7fdb5c20b740 20 client.15573 trim_cache size 0 max 16384 2024-09-19T04:48:09.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.652+0000 7fdb5c20b740 1 Processor -- start 2024-09-19T04:48:09.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.652+0000 7fdb5c20b740 1 -- start start 2024-09-19T04:48:09.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.653+0000 7fdb5c20b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 0x55ec5ddf28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.653+0000 7fdb5c20b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddf0760 0x55ec5de12d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.653+0000 7fdb5c20b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddfb020 0x55ec5de132b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.653+0000 7fdb5c20b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541ab4b0 con 0x55ec5ddefe40 2024-09-19T04:48:09.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.653+0000 7fdb5c20b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541ad020 con 0x55ec5ddf0760 2024-09-19T04:48:09.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.653+0000 7fdb5c20b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541abb80 con 0x55ec5ddfb020 2024-09-19T04:48:09.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.653+0000 7fdb52a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 0x55ec5ddf28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.653+0000 7fdb53400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddfb020 0x55ec5de132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.653+0000 7fdb4be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddf0760 0x55ec5de12d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.653+0000 7fdb53400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddfb020 0x55ec5de132b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:54398/0 (socket says 172.21.15.123:54398) 2024-09-19T04:48:09.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.653+0000 7fdb53400640 1 -- 172.21.15.123:0/307847496 learned_addr learned my addr 172.21.15.123:0/307847496 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:09.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.653+0000 7fdb4be00640 1 -- 172.21.15.123:0/307847496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddfb020 msgr2=0x55ec5de132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:09.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.653+0000 7fdb4be00640 1 --2- 172.21.15.123:0/307847496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddfb020 0x55ec5de132b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.653+0000 7fdb4be00640 1 -- 172.21.15.123:0/307847496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 msgr2=0x55ec5ddf28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:09.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.653+0000 7fdb4be00640 1 --2- 172.21.15.123:0/307847496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 0x55ec5ddf28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.653+0000 7fdb4be00640 1 -- 172.21.15.123:0/307847496 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec5ddebc80 con 0x55ec5ddf0760 2024-09-19T04:48:09.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.654+0000 7fdb4be00640 1 --2- 172.21.15.123:0/307847496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddf0760 0x55ec5de12d70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55ec5ddf9470 tx=0x7fdb3c098540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b879208d4eec6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:09.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.654+0000 7fdb42a00640 1 -- 172.21.15.123:0/307847496 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c0bc2f0 con 0x55ec5ddf0760 2024-09-19T04:48:09.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.654+0000 7fdb42a00640 1 -- 172.21.15.123:0/307847496 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb3c0980e0 con 0x55ec5ddf0760 2024-09-19T04:48:09.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.654+0000 7fdb42a00640 1 -- 172.21.15.123:0/307847496 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c0a00f0 con 0x55ec5ddf0760 2024-09-19T04:48:09.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.655+0000 7fdb5c20b740 1 -- 172.21.15.123:0/307847496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddf0760 msgr2=0x55ec5de12d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:09.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.655+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/307847496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddf0760 0x55ec5de12d70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55ec5ddf9470 tx=0x7fdb3c098540 comp rx=0 tx=0).stop 2024-09-19T04:48:09.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.655+0000 7fdb5c20b740 1 -- 172.21.15.123:0/307847496 shutdown_connections 2024-09-19T04:48:09.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.655+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/307847496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddfb020 0x55ec5de132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.655+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/307847496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddf0760 0x55ec5de12d70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.655+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/307847496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 0x55ec5ddf28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.655+0000 7fdb5c20b740 1 -- 172.21.15.123:0/307847496 >> 172.21.15.123:0/307847496 conn(0x55ec5ddec3d0 msgr2=0x55ec5ddf6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:09.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.655+0000 7fdb5c20b740 1 -- 172.21.15.123:0/307847496 shutdown_connections 2024-09-19T04:48:09.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.655+0000 7fdb5c20b740 1 -- 172.21.15.123:0/307847496 wait complete. 2024-09-19T04:48:09.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.656+0000 7fdb5c20b740 1 -- start start 2024-09-19T04:48:09.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.656+0000 7fdb5c20b740 1 Processor -- start 2024-09-19T04:48:09.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.656+0000 7fdb5c20b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 0x55ec5de031a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.656+0000 7fdb5c20b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddf0760 0x55ec5de036e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.656+0000 7fdb5c20b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddfb020 0x55ec5ddf46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.656+0000 7fdb5c20b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541ac920 con 0x55ec5ddefe40 2024-09-19T04:48:09.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.656+0000 7fdb5c20b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541ac250 con 0x55ec5ddfb020 2024-09-19T04:48:09.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.656+0000 7fdb5c20b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541a94b0 con 0x55ec5ddf0760 2024-09-19T04:48:09.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.656+0000 7fdb5c20b740 10 client.0 _ll_register_callbacks cb 0x7fdb3c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:09.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.657+0000 7fdb52a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 0x55ec5de031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.657+0000 7fdb52a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 0x55ec5de031a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39410/0 (socket says 172.21.15.123:39410) 2024-09-19T04:48:09.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.657+0000 7fdb52a00640 1 -- 172.21.15.123:0/145887958 learned_addr learned my addr 172.21.15.123:0/145887958 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:09.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.657+0000 7fdb4be00640 1 --2- 172.21.15.123:0/145887958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddf0760 0x55ec5de036e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.657+0000 7fdb53400640 1 --2- 172.21.15.123:0/145887958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddfb020 0x55ec5ddf46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.657+0000 7fdb52a00640 1 -- 172.21.15.123:0/145887958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddf0760 msgr2=0x55ec5de036e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:09.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.657+0000 7fdb52a00640 1 --2- 172.21.15.123:0/145887958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddf0760 0x55ec5de036e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.657+0000 7fdb52a00640 1 -- 172.21.15.123:0/145887958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddfb020 msgr2=0x55ec5ddf46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:09.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.657+0000 7fdb52a00640 1 --2- 172.21.15.123:0/145887958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddfb020 0x55ec5ddf46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.657+0000 7fdb52a00640 1 -- 172.21.15.123:0/145887958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec5ddf4d70 con 0x55ec5ddefe40 2024-09-19T04:48:09.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.657+0000 7fdb53400640 1 --2- 172.21.15.123:0/145887958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddfb020 0x55ec5ddf46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:48:09.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.657+0000 7fdb52a00640 1 --2- 172.21.15.123:0/145887958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 0x55ec5de031a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdb44019a90 tx=0x7fdb440196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4932abcc9690a04b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:09.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.657+0000 7fdb37e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:48:09.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.658+0000 7fdb37e00640 1 -- 172.21.15.123:0/145887958 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44031060 con 0x55ec5ddefe40 2024-09-19T04:48:09.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.658+0000 7fdb5c20b740 1 -- 172.21.15.123:0/145887958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb3c0980e0 con 0x55ec5ddefe40 2024-09-19T04:48:09.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.658+0000 7fdb37e00640 1 -- 172.21.15.123:0/145887958 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fdb44005040 con 0x55ec5ddefe40 2024-09-19T04:48:09.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.658+0000 7fdb37e00640 1 -- 172.21.15.123:0/145887958 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb44005260 con 0x55ec5ddefe40 2024-09-19T04:48:09.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.658+0000 7fdb5c20b740 10 client.15579 fetch_fsmap learned FSMap version 29 2024-09-19T04:48:09.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.658+0000 7fdb5c20b740 1 -- 172.21.15.123:0/145887958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec5de756c0 con 0x55ec5ddefe40 2024-09-19T04:48:09.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.658+0000 7fdb37e00640 1 -- 172.21.15.123:0/145887958 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb3c0980e0 con 0x55ec5ddefe40 2024-09-19T04:48:09.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.659+0000 7fdb37e00640 1 -- 172.21.15.123:0/145887958 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fdb4400f5c0 con 0x55ec5ddefe40 2024-09-19T04:48:09.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.659+0000 7fdb5c20b740 10 client.15579 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:48:09.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.659+0000 7fdb5c20b740 10 client.15579 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:48:09.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.659+0000 7fdb5c20b740 20 client.15579 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:09.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.659+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/145887958 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55ec5de75c80 0x55ec5de780a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:09.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.659+0000 7fdb5c20b740 4 client.15579 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:48:09.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.659+0000 7fdb5c20b740 1 -- 172.21.15.123:0/145887958 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec5dd5de60 con 0x55ec5de75c80 2024-09-19T04:48:09.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.659+0000 7fdb4be00640 1 --2- 172.21.15.123:0/145887958 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55ec5de75c80 0x55ec5de780a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:09.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.660+0000 7fdb4be00640 1 --2- 172.21.15.123:0/145887958 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55ec5de75c80 0x55ec5de780a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5aa694857228e46 server_cookie=f691a0b5cb5965aa in_seq=0 out_seq=0 2024-09-19T04:48:09.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.660+0000 7fdb37e00640 10 client.15579 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:48:09.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.660+0000 7fdb37e00640 1 -- 172.21.15.123:0/145887958 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec5dd5de60 con 0x55ec5de75c80 2024-09-19T04:48:09.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.660+0000 7fdb37e00640 10 client.15579 handle_command_reply: tid=0 2024-09-19T04:48:09.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.660+0000 7fdb5c20b740 1 client.15579 shutdown 2024-09-19T04:48:09.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.660+0000 7fdb5c20b740 1 -- 172.21.15.123:0/145887958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 msgr2=0x55ec5de031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:09.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.660+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/145887958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 0x55ec5de031a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdb44019a90 tx=0x7fdb440196c0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.660+0000 7fdb5c20b740 1 -- 172.21.15.123:0/145887958 shutdown_connections 2024-09-19T04:48:09.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.660+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/145887958 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55ec5de75c80 0x55ec5de780a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.660+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/145887958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec5ddfb020 0x55ec5ddf46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.660+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/145887958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5ddf0760 0x55ec5de036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.660+0000 7fdb5c20b740 1 --2- 172.21.15.123:0/145887958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec5ddefe40 0x55ec5de031a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.660+0000 7fdb5c20b740 1 -- 172.21.15.123:0/145887958 >> 172.21.15.123:0/145887958 conn(0x55ec5ddec3d0 msgr2=0x55ec5ddf7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:09.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.660+0000 7fdb37e00640 0 client.15579 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:48:09.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.661+0000 7fdb5c20b740 1 -- 172.21.15.123:0/145887958 shutdown_connections 2024-09-19T04:48:09.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.661+0000 7fdb5c20b740 1 -- 172.21.15.123:0/145887958 wait complete. 2024-09-19T04:48:09.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.661+0000 7fdb5c20b740 20 client.15579 trim_cache size 0 max 16384 2024-09-19T04:48:09.667 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:48:09.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.662+0000 7fdb53e00640 1 -- 172.21.15.123:0/131025959 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb380a35e0 msgr2=0x7fdb380a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:09.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.662+0000 7fdb53e00640 1 --2- 172.21.15.123:0/131025959 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb380a35e0 0x7fdb380a5aa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009430 tx=0x7fdb4c0093c0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.662+0000 7fdb53e00640 1 -- 172.21.15.123:0/131025959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb54102360 msgr2=0x7fdb541987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:09.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.662+0000 7fdb53e00640 1 --2- 172.21.15.123:0/131025959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb54102360 0x7fdb541987e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00e9f0 tx=0x7fdb3c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.662+0000 7fdb53e00640 1 -- 172.21.15.123:0/131025959 shutdown_connections 2024-09-19T04:48:09.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.662+0000 7fdb53e00640 1 --2- 172.21.15.123:0/131025959 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb380a35e0 0x7fdb380a5aa0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.662+0000 7fdb53e00640 1 --2- 172.21.15.123:0/131025959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb541079c0 0x7fdb5419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.662+0000 7fdb53e00640 1 --2- 172.21.15.123:0/131025959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb54102360 0x7fdb541987e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.662+0000 7fdb53e00640 1 --2- 172.21.15.123:0/131025959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb541019c0 0x7fdb541982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:09.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.662+0000 7fdb53e00640 1 -- 172.21.15.123:0/131025959 >> 172.21.15.123:0/131025959 conn(0x7fdb540fd740 msgr2=0x7fdb540fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:09.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.662+0000 7fdb53e00640 1 -- 172.21.15.123:0/131025959 shutdown_connections 2024-09-19T04:48:09.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:09.662+0000 7fdb53e00640 1 -- 172.21.15.123:0/131025959 wait complete. 2024-09-19T04:48:09.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:09 smithi183 ceph-mon[35333]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-19T04:48:09.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:09 smithi123 ceph-mon[30511]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-19T04:48:09.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:09 smithi193 ceph-mon[35592]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-19T04:48:10.436 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:48:10.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:10 smithi123 ceph-mon[30511]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:48:10.824 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:48:10.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:10 smithi183 ceph-mon[35333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:48:10.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:10 smithi193 ceph-mon[35592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:48:11.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.385+0000 7ff995400640 1 -- 172.21.15.123:0/2616616044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990106f10 msgr2=0x7ff99010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:11.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.385+0000 7ff995400640 1 --2- 172.21.15.123:0/2616616044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990106f10 0x7ff99010ea90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff988009ca0 tx=0x7ff98802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.385+0000 7ff995400640 1 -- 172.21.15.123:0/2616616044 shutdown_connections 2024-09-19T04:48:11.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.385+0000 7ff995400640 1 --2- 172.21.15.123:0/2616616044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990106f10 0x7ff99010ea90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.385+0000 7ff995400640 1 --2- 172.21.15.123:0/2616616044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff990102790 0x7ff9901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.385+0000 7ff995400640 1 --2- 172.21.15.123:0/2616616044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff990101de0 0x7ff9901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.385+0000 7ff995400640 1 -- 172.21.15.123:0/2616616044 >> 172.21.15.123:0/2616616044 conn(0x7ff9900fd8c0 msgr2=0x7ff9900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:11.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.385+0000 7ff995400640 1 -- 172.21.15.123:0/2616616044 shutdown_connections 2024-09-19T04:48:11.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.385+0000 7ff995400640 1 -- 172.21.15.123:0/2616616044 wait complete. 2024-09-19T04:48:11.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.386+0000 7ff995400640 1 Processor -- start 2024-09-19T04:48:11.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.386+0000 7ff995400640 1 -- start start 2024-09-19T04:48:11.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.387+0000 7ff995400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff990101de0 0x7ff99019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.387+0000 7ff995400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff990102790 0x7ff99019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.387+0000 7ff995400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990106f10 0x7ff99019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.387+0000 7ff995400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff990111fc0 con 0x7ff990101de0 2024-09-19T04:48:11.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.387+0000 7ff995400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff990111e40 con 0x7ff990106f10 2024-09-19T04:48:11.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.387+0000 7ff995400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff990112140 con 0x7ff990102790 2024-09-19T04:48:11.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.387+0000 7ff98e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff990101de0 0x7ff99019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.387+0000 7ff98de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff990102790 0x7ff99019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.387+0000 7ff98de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff990102790 0x7ff99019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:54428/0 (socket says 172.21.15.123:54428) 2024-09-19T04:48:11.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.387+0000 7ff98de00640 1 -- 172.21.15.123:0/974273188 learned_addr learned my addr 172.21.15.123:0/974273188 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:11.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.387+0000 7ff98f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990106f10 0x7ff99019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.387+0000 7ff98de00640 1 -- 172.21.15.123:0/974273188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990106f10 msgr2=0x7ff99019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:11.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.387+0000 7ff98de00640 1 --2- 172.21.15.123:0/974273188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990106f10 0x7ff99019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.387+0000 7ff98de00640 1 -- 172.21.15.123:0/974273188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff990101de0 msgr2=0x7ff99019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:11.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.387+0000 7ff98de00640 1 --2- 172.21.15.123:0/974273188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff990101de0 0x7ff99019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.387+0000 7ff98de00640 1 -- 172.21.15.123:0/974273188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99019f600 con 0x7ff990102790 2024-09-19T04:48:11.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.388+0000 7ff98de00640 1 --2- 172.21.15.123:0/974273188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff990102790 0x7ff99019ab90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00e9f0 tx=0x7ff97c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:11.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.388+0000 7ff987400640 1 -- 172.21.15.123:0/974273188 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c00cd80 con 0x7ff990102790 2024-09-19T04:48:11.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.388+0000 7ff987400640 1 -- 172.21.15.123:0/974273188 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff97c005500 con 0x7ff990102790 2024-09-19T04:48:11.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.388+0000 7ff995400640 1 -- 172.21.15.123:0/974273188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99019f8f0 con 0x7ff990102790 2024-09-19T04:48:11.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.388+0000 7ff987400640 1 -- 172.21.15.123:0/974273188 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c010690 con 0x7ff990102790 2024-09-19T04:48:11.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.388+0000 7ff995400640 1 -- 172.21.15.123:0/974273188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9901a71d0 con 0x7ff990102790 2024-09-19T04:48:11.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.391+0000 7ff99800a740 1 Processor -- start 2024-09-19T04:48:11.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.391+0000 7ff99800a740 1 -- start start 2024-09-19T04:48:11.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.392+0000 7ff99800a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf5c70 0x55d27bbf6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.392+0000 7ff987400640 1 -- 172.21.15.123:0/974273188 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7ff97c0046c0 con 0x7ff990102790 2024-09-19T04:48:11.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.392+0000 7ff99800a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf6590 0x55d27bc00910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.392+0000 7ff99800a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bc00e50 0x55d27bc03240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.392+0000 7ff99800a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d27bb5c3e0 con 0x55d27bbf6590 2024-09-19T04:48:11.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.392+0000 7ff99800a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d27bb03d80 con 0x55d27bbf5c70 2024-09-19T04:48:11.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.392+0000 7ff99800a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d27bb583a0 con 0x55d27bc00e50 2024-09-19T04:48:11.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.392+0000 7ff98f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf5c70 0x55d27bbf6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.392+0000 7ff98e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf6590 0x55d27bc00910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.392+0000 7ff98f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf5c70 0x55d27bbf6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:46898/0 (socket says 172.21.15.123:46898) 2024-09-19T04:48:11.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.392+0000 7ff98f200640 1 -- 172.21.15.123:0/4143351064 learned_addr learned my addr 172.21.15.123:0/4143351064 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:11.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.392+0000 7ff98de00640 1 --2- 172.21.15.123:0/4143351064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bc00e50 0x55d27bc03240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.392+0000 7ff98f200640 1 -- 172.21.15.123:0/4143351064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bc00e50 msgr2=0x55d27bc03240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:11.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.392+0000 7ff98f200640 1 --2- 172.21.15.123:0/4143351064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bc00e50 0x55d27bc03240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.392+0000 7ff98f200640 1 -- 172.21.15.123:0/4143351064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf6590 msgr2=0x55d27bc00910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:11.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.392+0000 7ff98f200640 1 --2- 172.21.15.123:0/4143351064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf6590 0x55d27bc00910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.392+0000 7ff98f200640 1 -- 172.21.15.123:0/4143351064 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d27bb22ae0 con 0x55d27bbf5c70 2024-09-19T04:48:11.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.393+0000 7ff98f200640 1 --2- 172.21.15.123:0/4143351064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf5c70 0x55d27bbf6050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff988009ca0 tx=0x7ff9880093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5d47ecf2c5452b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:11.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.393+0000 7ff97b400640 1 -- 172.21.15.123:0/4143351064 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988031db0 con 0x55d27bbf5c70 2024-09-19T04:48:11.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.393+0000 7ff97b400640 1 -- 172.21.15.123:0/4143351064 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff988036030 con 0x55d27bbf5c70 2024-09-19T04:48:11.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.393+0000 7ff97b400640 1 -- 172.21.15.123:0/4143351064 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988004330 con 0x55d27bbf5c70 2024-09-19T04:48:11.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.393+0000 7ff987400640 1 --2- 172.21.15.123:0/974273188 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff9740a3560 0x7ff9740a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.394+0000 7ff987400640 1 -- 172.21.15.123:0/974273188 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7ff97c0d0320 con 0x7ff990102790 2024-09-19T04:48:11.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.394+0000 7ff98e800640 1 --2- 172.21.15.123:0/974273188 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff9740a3560 0x7ff9740a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.394+0000 7ff99800a740 1 -- 172.21.15.123:0/4143351064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf5c70 msgr2=0x55d27bbf6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:11.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.394+0000 7ff99800a740 1 --2- 172.21.15.123:0/4143351064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf5c70 0x55d27bbf6050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff988009ca0 tx=0x7ff9880093e0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.394+0000 7ff99800a740 1 -- 172.21.15.123:0/4143351064 shutdown_connections 2024-09-19T04:48:11.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.394+0000 7ff99800a740 1 --2- 172.21.15.123:0/4143351064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bc00e50 0x55d27bc03240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.394+0000 7ff99800a740 1 --2- 172.21.15.123:0/4143351064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf6590 0x55d27bc00910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.394+0000 7ff99800a740 1 --2- 172.21.15.123:0/4143351064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf5c70 0x55d27bbf6050 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.394+0000 7ff99800a740 1 -- 172.21.15.123:0/4143351064 >> 172.21.15.123:0/4143351064 conn(0x55d27bbf2200 msgr2=0x55d27bbf25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:11.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.394+0000 7ff99800a740 1 -- 172.21.15.123:0/4143351064 shutdown_connections 2024-09-19T04:48:11.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.394+0000 7ff99800a740 1 -- 172.21.15.123:0/4143351064 wait complete. 2024-09-19T04:48:11.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.395+0000 7ff98e800640 1 --2- 172.21.15.123:0/974273188 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff9740a3560 0x7ff9740a5a20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55d27bbf3060 tx=0x7ff980005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:11.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.395+0000 7ff99800a740 1 -- start start 2024-09-19T04:48:11.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.395+0000 7ff99800a740 1 Processor -- start 2024-09-19T04:48:11.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.395+0000 7ff99800a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bbf5c70 0x55d27bbfdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.395+0000 7ff99800a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf6590 0x55d27bbfe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.395+0000 7ff99800a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bc00e50 0x55d27bbfe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.395+0000 7ff99800a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d27bc038a0 con 0x55d27bbf6590 2024-09-19T04:48:11.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.395+0000 7ff99800a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d27bb170f0 con 0x55d27bc00e50 2024-09-19T04:48:11.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.395+0000 7ff99800a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d27bc03a20 con 0x55d27bbf5c70 2024-09-19T04:48:11.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.395+0000 7ff99800a740 10 client.0 _ll_register_callbacks cb 0x55d27bafebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:11.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.396+0000 7ff98e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bbf5c70 0x55d27bbfdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.396+0000 7ff98de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf6590 0x55d27bbfe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.396+0000 7ff98e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bbf5c70 0x55d27bbfdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:54456/0 (socket says 172.21.15.123:54456) 2024-09-19T04:48:11.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.396+0000 7ff98de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf6590 0x55d27bbfe260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39434/0 (socket says 172.21.15.123:39434) 2024-09-19T04:48:11.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.396+0000 7ff98e800640 1 -- 172.21.15.123:0/291225273 learned_addr learned my addr 172.21.15.123:0/291225273 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:11.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.396+0000 7ff98f200640 1 --2- 172.21.15.123:0/291225273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bc00e50 0x55d27bbfe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.396+0000 7ff98e800640 1 -- 172.21.15.123:0/291225273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bc00e50 msgr2=0x55d27bbfe7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:11.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.396+0000 7ff98e800640 1 --2- 172.21.15.123:0/291225273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bc00e50 0x55d27bbfe7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.396+0000 7ff98e800640 1 -- 172.21.15.123:0/291225273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf6590 msgr2=0x55d27bbfe260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:11.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.396+0000 7ff98e800640 1 --2- 172.21.15.123:0/291225273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf6590 0x55d27bbfe260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.396+0000 7ff98e800640 1 -- 172.21.15.123:0/291225273 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d27b8cd3c0 con 0x55d27bbf5c70 2024-09-19T04:48:11.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.397+0000 7ff98e800640 1 --2- 172.21.15.123:0/291225273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bbf5c70 0x55d27bbfdd20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff98001e9f0 tx=0x7ff980004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e013222110027885 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:11.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.397+0000 7ff96fe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:48:11.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.397+0000 7ff96fe00640 1 -- 172.21.15.123:0/291225273 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98002e060 con 0x55d27bbf5c70 2024-09-19T04:48:11.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.397+0000 7ff96fe00640 1 -- 172.21.15.123:0/291225273 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7ff980018030 con 0x55d27bbf5c70 2024-09-19T04:48:11.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.397+0000 7ff99800a740 1 -- 172.21.15.123:0/291225273 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff988036030 con 0x55d27bbf5c70 2024-09-19T04:48:11.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.397+0000 7ff96fe00640 1 -- 172.21.15.123:0/291225273 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff980009e40 con 0x55d27bbf5c70 2024-09-19T04:48:11.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.397+0000 7ff99800a740 10 client.25639 fetch_fsmap learned FSMap version 29 2024-09-19T04:48:11.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.397+0000 7ff99800a740 1 -- 172.21.15.123:0/291225273 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d27bc0f970 con 0x55d27bbf5c70 2024-09-19T04:48:11.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.397+0000 7ff96fe00640 1 -- 172.21.15.123:0/291225273 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff988036030 con 0x55d27bbf5c70 2024-09-19T04:48:11.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.398+0000 7ff96fe00640 1 -- 172.21.15.123:0/291225273 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7ff98001c600 con 0x55d27bbf5c70 2024-09-19T04:48:11.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.398+0000 7ff99800a740 10 client.25639 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:48:11.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.398+0000 7ff99800a740 10 client.25639 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:48:11.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.398+0000 7ff99800a740 20 client.25639 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:11.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.398+0000 7ff99800a740 1 --2- 172.21.15.123:0/291225273 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55d27bc10e30 0x55d27bc132a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.398+0000 7ff99800a740 4 client.25639 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:48:11.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.398+0000 7ff99800a740 1 -- 172.21.15.123:0/291225273 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d27bb69950 con 0x55d27bc10e30 2024-09-19T04:48:11.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.398+0000 7ff98de00640 1 --2- 172.21.15.123:0/291225273 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55d27bc10e30 0x55d27bc132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.399+0000 7ff98de00640 1 --2- 172.21.15.123:0/291225273 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55d27bc10e30 0x55d27bc132a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3455424ac19243a9 server_cookie=2594c8157f658090 in_seq=0 out_seq=0 2024-09-19T04:48:11.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.399+0000 7ff96fe00640 10 client.25639 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:48:11.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.399+0000 7ff96fe00640 1 -- 172.21.15.123:0/291225273 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d27bb69950 con 0x55d27bc10e30 2024-09-19T04:48:11.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.399+0000 7ff96fe00640 10 client.25639 handle_command_reply: tid=0 2024-09-19T04:48:11.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.399+0000 7ff99800a740 1 client.25639 shutdown 2024-09-19T04:48:11.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.400+0000 7ff99800a740 1 -- 172.21.15.123:0/291225273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bbf5c70 msgr2=0x55d27bbfdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:11.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.400+0000 7ff99800a740 1 --2- 172.21.15.123:0/291225273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bbf5c70 0x55d27bbfdd20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff98001e9f0 tx=0x7ff980004730 comp rx=0 tx=0).stop 2024-09-19T04:48:11.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.400+0000 7ff99800a740 1 -- 172.21.15.123:0/291225273 shutdown_connections 2024-09-19T04:48:11.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.400+0000 7ff99800a740 1 --2- 172.21.15.123:0/291225273 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55d27bc10e30 0x55d27bc132a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.400+0000 7ff99800a740 1 --2- 172.21.15.123:0/291225273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bc00e50 0x55d27bbfe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.400+0000 7ff99800a740 1 --2- 172.21.15.123:0/291225273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf6590 0x55d27bbfe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.400+0000 7ff96fe00640 0 client.25639 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:48:11.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.400+0000 7ff99800a740 1 --2- 172.21.15.123:0/291225273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bbf5c70 0x55d27bbfdd20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.400+0000 7ff99800a740 1 -- 172.21.15.123:0/291225273 >> 172.21.15.123:0/291225273 conn(0x55d27bbf2200 msgr2=0x55d27bbf1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:11.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.400+0000 7ff99800a740 1 -- 172.21.15.123:0/291225273 shutdown_connections 2024-09-19T04:48:11.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.400+0000 7ff99800a740 1 -- 172.21.15.123:0/291225273 wait complete. 2024-09-19T04:48:11.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.400+0000 7ff99800a740 20 client.25639 trim_cache size 0 max 16384 2024-09-19T04:48:11.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.413+0000 7ff99800a740 1 Processor -- start 2024-09-19T04:48:11.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.413+0000 7ff99800a740 1 -- start start 2024-09-19T04:48:11.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.414+0000 7ff99800a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf5c70 0x55d27bbf8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.414+0000 7ff99800a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bbf6590 0x55d27bbfc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.414+0000 7ff99800a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bc00e50 0x55d27bc18b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.414+0000 7ff99800a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901af8e0 con 0x55d27bc00e50 2024-09-19T04:48:11.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.414+0000 7ff99800a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901b1450 con 0x55d27bbf5c70 2024-09-19T04:48:11.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.414+0000 7ff99800a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901affb0 con 0x55d27bbf6590 2024-09-19T04:48:11.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.414+0000 7ff98f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bc00e50 0x55d27bc18b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.414+0000 7ff98f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bc00e50 0x55d27bc18b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39446/0 (socket says 172.21.15.123:39446) 2024-09-19T04:48:11.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.414+0000 7ff98de00640 1 --2- 172.21.15.123:0/3180200350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bbf6590 0x55d27bbfc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.414+0000 7ff98f200640 1 -- 172.21.15.123:0/3180200350 learned_addr learned my addr 172.21.15.123:0/3180200350 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:11.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.414+0000 7ff98e800640 1 --2- 172.21.15.123:0/3180200350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf5c70 0x55d27bbf8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.414+0000 7ff98de00640 1 -- 172.21.15.123:0/3180200350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf5c70 msgr2=0x55d27bbf8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:11.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.414+0000 7ff98de00640 1 --2- 172.21.15.123:0/3180200350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf5c70 0x55d27bbf8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.414+0000 7ff98de00640 1 -- 172.21.15.123:0/3180200350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bc00e50 msgr2=0x55d27bc18b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:11.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.414+0000 7ff98de00640 1 --2- 172.21.15.123:0/3180200350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bc00e50 0x55d27bc18b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.415+0000 7ff98de00640 1 -- 172.21.15.123:0/3180200350 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d27bbf32b0 con 0x55d27bbf6590 2024-09-19T04:48:11.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.415+0000 7ff98de00640 1 --2- 172.21.15.123:0/3180200350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bbf6590 0x55d27bbfc5f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0cd860 tx=0x7ff97c098040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=264df7517818dca6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:11.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.415+0000 7ff97aa00640 1 -- 172.21.15.123:0/3180200350 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c09bd50 con 0x55d27bbf6590 2024-09-19T04:48:11.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.415+0000 7ff97aa00640 1 -- 172.21.15.123:0/3180200350 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff97c09caf0 con 0x55d27bbf6590 2024-09-19T04:48:11.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.415+0000 7ff97aa00640 1 -- 172.21.15.123:0/3180200350 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c0a4bd0 con 0x55d27bbf6590 2024-09-19T04:48:11.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.416+0000 7ff99800a740 1 -- 172.21.15.123:0/3180200350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bbf6590 msgr2=0x55d27bbfc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:11.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.416+0000 7ff99800a740 1 --2- 172.21.15.123:0/3180200350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bbf6590 0x55d27bbfc5f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0cd860 tx=0x7ff97c098040 comp rx=0 tx=0).stop 2024-09-19T04:48:11.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.416+0000 7ff99800a740 1 -- 172.21.15.123:0/3180200350 shutdown_connections 2024-09-19T04:48:11.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.416+0000 7ff99800a740 1 --2- 172.21.15.123:0/3180200350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bc00e50 0x55d27bc18b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.416+0000 7ff99800a740 1 --2- 172.21.15.123:0/3180200350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bbf6590 0x55d27bbfc5f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.416+0000 7ff99800a740 1 --2- 172.21.15.123:0/3180200350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf5c70 0x55d27bbf8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.416+0000 7ff99800a740 1 -- 172.21.15.123:0/3180200350 >> 172.21.15.123:0/3180200350 conn(0x55d27bbf2200 msgr2=0x55d27bbffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:11.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.416+0000 7ff99800a740 1 -- 172.21.15.123:0/3180200350 shutdown_connections 2024-09-19T04:48:11.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.416+0000 7ff99800a740 1 -- 172.21.15.123:0/3180200350 wait complete. 2024-09-19T04:48:11.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.416+0000 7ff99800a740 1 -- start start 2024-09-19T04:48:11.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.416+0000 7ff99800a740 1 Processor -- start 2024-09-19T04:48:11.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.417+0000 7ff99800a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf5c70 0x55d27bc09550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.417+0000 7ff99800a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf6590 0x55d27bbfabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.417+0000 7ff99800a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bc00e50 0x55d27bbfb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.417+0000 7ff99800a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901b0d50 con 0x55d27bbf5c70 2024-09-19T04:48:11.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.417+0000 7ff99800a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901b0680 con 0x55d27bbf6590 2024-09-19T04:48:11.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.417+0000 7ff99800a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901ad8c0 con 0x55d27bc00e50 2024-09-19T04:48:11.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.417+0000 7ff99800a740 10 client.0 _ll_register_callbacks cb 0x7ff980009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:11.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.417+0000 7ff98e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf5c70 0x55d27bc09550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.417+0000 7ff98e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf5c70 0x55d27bc09550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39448/0 (socket says 172.21.15.123:39448) 2024-09-19T04:48:11.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.417+0000 7ff98e800640 1 -- 172.21.15.123:0/1278390134 learned_addr learned my addr 172.21.15.123:0/1278390134 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:11.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.417+0000 7ff98f200640 1 --2- 172.21.15.123:0/1278390134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bc00e50 0x55d27bbfb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.417+0000 7ff98de00640 1 --2- 172.21.15.123:0/1278390134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf6590 0x55d27bbfabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.418+0000 7ff98e800640 1 -- 172.21.15.123:0/1278390134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bc00e50 msgr2=0x55d27bbfb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:11.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.418+0000 7ff98e800640 1 --2- 172.21.15.123:0/1278390134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bc00e50 0x55d27bbfb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.418+0000 7ff98e800640 1 -- 172.21.15.123:0/1278390134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf6590 msgr2=0x55d27bbfabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:11.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.418+0000 7ff98e800640 1 --2- 172.21.15.123:0/1278390134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf6590 0x55d27bbfabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.418+0000 7ff98e800640 1 -- 172.21.15.123:0/1278390134 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d27bbf2e90 con 0x55d27bbf5c70 2024-09-19T04:48:11.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.418+0000 7ff98de00640 1 --2- 172.21.15.123:0/1278390134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf6590 0x55d27bbfabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:11.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.418+0000 7ff98e800640 1 --2- 172.21.15.123:0/1278390134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf5c70 0x55d27bc09550 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff9800092e0 tx=0x7ff980012d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a091392aba54472 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:11.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.418+0000 7ff96fe00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:48:11.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.418+0000 7ff96fe00640 1 -- 172.21.15.123:0/1278390134 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98001c7d0 con 0x55d27bbf5c70 2024-09-19T04:48:11.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.418+0000 7ff96fe00640 1 -- 172.21.15.123:0/1278390134 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7ff980033690 con 0x55d27bbf5c70 2024-09-19T04:48:11.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.418+0000 7ff99800a740 1 -- 172.21.15.123:0/1278390134 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff97c09caf0 con 0x55d27bbf5c70 2024-09-19T04:48:11.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.418+0000 7ff96fe00640 1 -- 172.21.15.123:0/1278390134 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff980033890 con 0x55d27bbf5c70 2024-09-19T04:48:11.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.418+0000 7ff99800a740 10 client.15597 fetch_fsmap learned FSMap version 29 2024-09-19T04:48:11.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.418+0000 7ff99800a740 1 -- 172.21.15.123:0/1278390134 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d27bbed770 con 0x55d27bbf5c70 2024-09-19T04:48:11.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.419+0000 7ff96fe00640 1 -- 172.21.15.123:0/1278390134 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff97c09caf0 con 0x55d27bbf5c70 2024-09-19T04:48:11.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.419+0000 7ff96fe00640 1 -- 172.21.15.123:0/1278390134 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7ff9800459a0 con 0x55d27bbf5c70 2024-09-19T04:48:11.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.419+0000 7ff99800a740 10 client.15597 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:48:11.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.419+0000 7ff99800a740 10 client.15597 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:48:11.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.419+0000 7ff99800a740 20 client.15597 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:11.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.419+0000 7ff99800a740 1 --2- 172.21.15.123:0/1278390134 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55d27bc7c0e0 0x55d27bc7e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:11.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.419+0000 7ff99800a740 4 client.15597 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:48:11.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.419+0000 7ff99800a740 1 -- 172.21.15.123:0/1278390134 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d27bb63e60 con 0x55d27bc7c0e0 2024-09-19T04:48:11.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.419+0000 7ff98de00640 1 --2- 172.21.15.123:0/1278390134 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55d27bc7c0e0 0x55d27bc7e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:11.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.420+0000 7ff98de00640 1 --2- 172.21.15.123:0/1278390134 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55d27bc7c0e0 0x55d27bc7e500 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2bd794acfd663147 server_cookie=2607bc97f3bef63f in_seq=0 out_seq=0 2024-09-19T04:48:11.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.420+0000 7ff96fe00640 10 client.15597 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:48:11.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.420+0000 7ff96fe00640 1 -- 172.21.15.123:0/1278390134 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d27bb63e60 con 0x55d27bc7c0e0 2024-09-19T04:48:11.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.420+0000 7ff96fe00640 10 client.15597 handle_command_reply: tid=0 2024-09-19T04:48:11.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.420+0000 7ff99800a740 1 client.15597 shutdown 2024-09-19T04:48:11.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.420+0000 7ff99800a740 1 -- 172.21.15.123:0/1278390134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf5c70 msgr2=0x55d27bc09550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:11.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.420+0000 7ff99800a740 1 --2- 172.21.15.123:0/1278390134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf5c70 0x55d27bc09550 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff9800092e0 tx=0x7ff980012d90 comp rx=0 tx=0).stop 2024-09-19T04:48:11.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.420+0000 7ff99800a740 1 -- 172.21.15.123:0/1278390134 shutdown_connections 2024-09-19T04:48:11.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.420+0000 7ff99800a740 1 --2- 172.21.15.123:0/1278390134 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55d27bc7c0e0 0x55d27bc7e500 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.420+0000 7ff99800a740 1 --2- 172.21.15.123:0/1278390134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d27bc00e50 0x55d27bbfb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.420+0000 7ff99800a740 1 --2- 172.21.15.123:0/1278390134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27bbf6590 0x55d27bbfabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.420+0000 7ff99800a740 1 --2- 172.21.15.123:0/1278390134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d27bbf5c70 0x55d27bc09550 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.420+0000 7ff96fe00640 0 client.15597 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:48:11.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.420+0000 7ff99800a740 1 -- 172.21.15.123:0/1278390134 >> 172.21.15.123:0/1278390134 conn(0x55d27bbf2200 msgr2=0x55d27bbf0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:11.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.420+0000 7ff99800a740 1 -- 172.21.15.123:0/1278390134 shutdown_connections 2024-09-19T04:48:11.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.420+0000 7ff99800a740 1 -- 172.21.15.123:0/1278390134 wait complete. 2024-09-19T04:48:11.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.420+0000 7ff99800a740 20 client.15597 trim_cache size 0 max 16384 2024-09-19T04:48:11.427 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:48:11.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.421+0000 7ff995400640 1 -- 172.21.15.123:0/974273188 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff9740a3560 msgr2=0x7ff9740a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:11.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.421+0000 7ff995400640 1 --2- 172.21.15.123:0/974273188 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff9740a3560 0x7ff9740a5a20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55d27bbf3060 tx=0x7ff980005f90 comp rx=0 tx=0).stop 2024-09-19T04:48:11.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.421+0000 7ff995400640 1 -- 172.21.15.123:0/974273188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff990102790 msgr2=0x7ff99019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:11.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.421+0000 7ff995400640 1 --2- 172.21.15.123:0/974273188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff990102790 0x7ff99019ab90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00e9f0 tx=0x7ff97c00edf0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.421+0000 7ff995400640 1 -- 172.21.15.123:0/974273188 shutdown_connections 2024-09-19T04:48:11.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.421+0000 7ff995400640 1 --2- 172.21.15.123:0/974273188 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff9740a3560 0x7ff9740a5a20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.421+0000 7ff995400640 1 --2- 172.21.15.123:0/974273188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990106f10 0x7ff99019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.421+0000 7ff995400640 1 --2- 172.21.15.123:0/974273188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff990102790 0x7ff99019ab90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.421+0000 7ff995400640 1 --2- 172.21.15.123:0/974273188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff990101de0 0x7ff99019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:11.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.421+0000 7ff995400640 1 -- 172.21.15.123:0/974273188 >> 172.21.15.123:0/974273188 conn(0x7ff9900fd8c0 msgr2=0x7ff9900ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:11.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.421+0000 7ff995400640 1 -- 172.21.15.123:0/974273188 shutdown_connections 2024-09-19T04:48:11.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:11.421+0000 7ff995400640 1 -- 172.21.15.123:0/974273188 wait complete. 2024-09-19T04:48:11.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:11 smithi183 ceph-mon[35333]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-19T04:48:11.922 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:11 smithi123 ceph-mon[30511]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-19T04:48:11.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:11 smithi193 ceph-mon[35592]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-19T04:48:12.300 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:48:12.669 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:48:13.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.212+0000 7fdde8a00640 1 -- 172.21.15.123:0/2756100756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde410e280 msgr2=0x7fdde4110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:13.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.212+0000 7fdde8a00640 1 --2- 172.21.15.123:0/2756100756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde410e280 0x7fdde4110670 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdddc009ca0 tx=0x7fdddc02a950 comp rx=0 tx=0).stop 2024-09-19T04:48:13.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.212+0000 7fdde8a00640 1 -- 172.21.15.123:0/2756100756 shutdown_connections 2024-09-19T04:48:13.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.212+0000 7fdde8a00640 1 --2- 172.21.15.123:0/2756100756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde410e280 0x7fdde4110670 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.212+0000 7fdde8a00640 1 --2- 172.21.15.123:0/2756100756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdde4074e30 0x7fdde410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.212+0000 7fdde8a00640 1 --2- 172.21.15.123:0/2756100756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdde4074510 0x7fdde40748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.212+0000 7fdde8a00640 1 -- 172.21.15.123:0/2756100756 >> 172.21.15.123:0/2756100756 conn(0x7fdde40fd740 msgr2=0x7fdde40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:13.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.212+0000 7fdde8a00640 1 -- 172.21.15.123:0/2756100756 shutdown_connections 2024-09-19T04:48:13.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.212+0000 7fdde8a00640 1 -- 172.21.15.123:0/2756100756 wait complete. 2024-09-19T04:48:13.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.213+0000 7fdde8a00640 1 Processor -- start 2024-09-19T04:48:13.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.213+0000 7fdde8a00640 1 -- start start 2024-09-19T04:48:13.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.214+0000 7fdde8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde4074510 0x7fdde419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.214+0000 7fdde8a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdde4074e30 0x7fdde419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.214+0000 7fdde8a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdde410e280 0x7fdde419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.214+0000 7fdde8a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdde4114000 con 0x7fdde4074510 2024-09-19T04:48:13.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.214+0000 7fdde8a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdde4113e80 con 0x7fdde410e280 2024-09-19T04:48:13.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.214+0000 7fdde8a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdde4114180 con 0x7fdde4074e30 2024-09-19T04:48:13.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.214+0000 7fdde3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde4074510 0x7fdde419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.214+0000 7fdde3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde4074510 0x7fdde419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38234/0 (socket says 172.21.15.123:38234) 2024-09-19T04:48:13.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.214+0000 7fdde3400640 1 -- 172.21.15.123:0/1539602698 learned_addr learned my addr 172.21.15.123:0/1539602698 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:13.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.214+0000 7fdde2a00640 1 --2- 172.21.15.123:0/1539602698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdde4074e30 0x7fdde419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.214+0000 7fdde3e00640 1 --2- 172.21.15.123:0/1539602698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdde410e280 0x7fdde419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.215+0000 7fdde3400640 1 -- 172.21.15.123:0/1539602698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdde4074e30 msgr2=0x7fdde419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:13.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.215+0000 7fdde3400640 1 --2- 172.21.15.123:0/1539602698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdde4074e30 0x7fdde419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.215+0000 7fdde3400640 1 -- 172.21.15.123:0/1539602698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdde410e280 msgr2=0x7fdde419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:13.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.215+0000 7fdde3400640 1 --2- 172.21.15.123:0/1539602698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdde410e280 0x7fdde419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.215+0000 7fdde3400640 1 -- 172.21.15.123:0/1539602698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde419f3f0 con 0x7fdde4074510 2024-09-19T04:48:13.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.215+0000 7fdde3400640 1 --2- 172.21.15.123:0/1539602698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde4074510 0x7fdde419a440 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fddd400e9f0 tx=0x7fddd400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:13.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.215+0000 7fdddbe00640 1 -- 172.21.15.123:0/1539602698 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd400cd80 con 0x7fdde4074510 2024-09-19T04:48:13.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.215+0000 7fdddbe00640 1 -- 172.21.15.123:0/1539602698 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddd4010910 con 0x7fdde4074510 2024-09-19T04:48:13.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.215+0000 7fdddbe00640 1 -- 172.21.15.123:0/1539602698 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4018990 con 0x7fdde4074510 2024-09-19T04:48:13.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.215+0000 7fdde8a00640 1 -- 172.21.15.123:0/1539602698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde419f6e0 con 0x7fdde4074510 2024-09-19T04:48:13.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.215+0000 7fdde8a00640 1 -- 172.21.15.123:0/1539602698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde41a6fc0 con 0x7fdde4074510 2024-09-19T04:48:13.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.219+0000 7fddeca2c740 1 Processor -- start 2024-09-19T04:48:13.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.220+0000 7fddeca2c740 1 -- start start 2024-09-19T04:48:13.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.220+0000 7fdddbe00640 1 -- 172.21.15.123:0/1539602698 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138640+0+0 (secure 0 0 0) 0x7fddd4010ac0 con 0x7fdde4074510 2024-09-19T04:48:13.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.220+0000 7fddeca2c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 0x562ef7c5d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.220+0000 7fddeca2c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c5d760 0x562ef7c67ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.220+0000 7fddeca2c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c68020 0x562ef7c6a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.220+0000 7fddeca2c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562ef7bc33e0 con 0x562ef7c68020 2024-09-19T04:48:13.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.220+0000 7fddeca2c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562ef7b6ad80 con 0x562ef7c5ce40 2024-09-19T04:48:13.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.220+0000 7fddeca2c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562ef7bbf3a0 con 0x562ef7c5d760 2024-09-19T04:48:13.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.220+0000 7fdde3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 0x562ef7c5d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.220+0000 7fdde2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c68020 0x562ef7c6a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.220+0000 7fdde3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 0x562ef7c5d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:39856/0 (socket says 172.21.15.123:39856) 2024-09-19T04:48:13.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.220+0000 7fdde3e00640 1 -- 172.21.15.123:0/569763808 learned_addr learned my addr 172.21.15.123:0/569763808 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:13.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.220+0000 7fdde3400640 1 --2- 172.21.15.123:0/569763808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c5d760 0x562ef7c67ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.221+0000 7fdde3e00640 1 -- 172.21.15.123:0/569763808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c5d760 msgr2=0x562ef7c67ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:13.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.221+0000 7fdde3e00640 1 --2- 172.21.15.123:0/569763808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c5d760 0x562ef7c67ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.221+0000 7fdde3e00640 1 -- 172.21.15.123:0/569763808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c68020 msgr2=0x562ef7c6a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:13.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.221+0000 7fdde3e00640 1 --2- 172.21.15.123:0/569763808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c68020 0x562ef7c6a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.221+0000 7fdde3e00640 1 -- 172.21.15.123:0/569763808 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ef7b89ae0 con 0x562ef7c5ce40 2024-09-19T04:48:13.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.221+0000 7fdde3e00640 1 --2- 172.21.15.123:0/569763808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 0x562ef7c5d220 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdde40754f0 tx=0x7fdddc005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bc135ec2473630d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:13.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.222+0000 7fddcfe00640 1 -- 172.21.15.123:0/569763808 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc048070 con 0x562ef7c5ce40 2024-09-19T04:48:13.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.222+0000 7fddcfe00640 1 -- 172.21.15.123:0/569763808 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdddc003e50 con 0x562ef7c5ce40 2024-09-19T04:48:13.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.222+0000 7fddcfe00640 1 -- 172.21.15.123:0/569763808 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc034040 con 0x562ef7c5ce40 2024-09-19T04:48:13.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.222+0000 7fdddbe00640 1 --2- 172.21.15.123:0/1539602698 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fddc80a35e0 0x7fddc80a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.222+0000 7fdddbe00640 1 -- 172.21.15.123:0/1539602698 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fddd40d01d0 con 0x7fdde4074510 2024-09-19T04:48:13.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.222+0000 7fdde3400640 1 --2- 172.21.15.123:0/1539602698 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fddc80a35e0 0x7fddc80a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.222+0000 7fddeca2c740 1 -- 172.21.15.123:0/569763808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 msgr2=0x562ef7c5d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:13.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.222+0000 7fddeca2c740 1 --2- 172.21.15.123:0/569763808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 0x562ef7c5d220 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdde40754f0 tx=0x7fdddc005f80 comp rx=0 tx=0).stop 2024-09-19T04:48:13.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.223+0000 7fddeca2c740 1 -- 172.21.15.123:0/569763808 shutdown_connections 2024-09-19T04:48:13.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.223+0000 7fddeca2c740 1 --2- 172.21.15.123:0/569763808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c68020 0x562ef7c6a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.223+0000 7fddeca2c740 1 --2- 172.21.15.123:0/569763808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c5d760 0x562ef7c67ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.223+0000 7fddeca2c740 1 --2- 172.21.15.123:0/569763808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 0x562ef7c5d220 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.223+0000 7fddeca2c740 1 -- 172.21.15.123:0/569763808 >> 172.21.15.123:0/569763808 conn(0x562ef7c593d0 msgr2=0x562ef7c597b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:13.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.223+0000 7fddeca2c740 1 -- 172.21.15.123:0/569763808 shutdown_connections 2024-09-19T04:48:13.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.223+0000 7fddeca2c740 1 -- 172.21.15.123:0/569763808 wait complete. 2024-09-19T04:48:13.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.223+0000 7fdde3400640 1 --2- 172.21.15.123:0/1539602698 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fddc80a35e0 0x7fddc80a5aa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fddd40cd7b0 tx=0x7fddd40d3ce0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:13.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.223+0000 7fddeca2c740 1 -- start start 2024-09-19T04:48:13.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.223+0000 7fddeca2c740 1 Processor -- start 2024-09-19T04:48:13.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.224+0000 7fddeca2c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 0x562ef7c64f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.224+0000 7fddeca2c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c65480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.224+0000 7fddeca2c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c68020 0x562ef7c659c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.224+0000 7fddeca2c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562ef7c6abf0 con 0x562ef7c5d760 2024-09-19T04:48:13.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.224+0000 7fddeca2c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562ef7b7e0f0 con 0x562ef7c5ce40 2024-09-19T04:48:13.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.224+0000 7fddeca2c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562ef7c6ad70 con 0x562ef7c68020 2024-09-19T04:48:13.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.224+0000 7fddeca2c740 10 client.0 _ll_register_callbacks cb 0x562ef7b65bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:13.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.224+0000 7fdde2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c65480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.224+0000 7fdde2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c65480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38250/0 (socket says 172.21.15.123:38250) 2024-09-19T04:48:13.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.224+0000 7fdde2a00640 1 -- 172.21.15.123:0/2545363241 learned_addr learned my addr 172.21.15.123:0/2545363241 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:13.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.224+0000 7fdde3e00640 1 --2- 172.21.15.123:0/2545363241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c68020 0x562ef7c659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.224+0000 7fdde3e00640 1 --2- 172.21.15.123:0/2545363241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 0x562ef7c64f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.225+0000 7fdde2a00640 1 -- 172.21.15.123:0/2545363241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c68020 msgr2=0x562ef7c659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:13.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.225+0000 7fdde2a00640 1 --2- 172.21.15.123:0/2545363241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c68020 0x562ef7c659c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.225+0000 7fdde2a00640 1 -- 172.21.15.123:0/2545363241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 msgr2=0x562ef7c64f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:13.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.225+0000 7fdde2a00640 1 --2- 172.21.15.123:0/2545363241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 0x562ef7c64f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.225+0000 7fdde2a00640 1 -- 172.21.15.123:0/2545363241 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ef79343c0 con 0x562ef7c5d760 2024-09-19T04:48:13.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.225+0000 7fdde2a00640 1 --2- 172.21.15.123:0/2545363241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c65480 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fddd0012700 tx=0x7fddd0012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8b517860f8f37c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:13.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.225+0000 7fddccc00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:48:13.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.225+0000 7fddccc00640 1 -- 172.21.15.123:0/2545363241 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd000f8c0 con 0x562ef7c5d760 2024-09-19T04:48:13.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.225+0000 7fddccc00640 1 -- 172.21.15.123:0/2545363241 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fddd000fa60 con 0x562ef7c5d760 2024-09-19T04:48:13.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.225+0000 7fddeca2c740 1 -- 172.21.15.123:0/2545363241 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdddc003e50 con 0x562ef7c5d760 2024-09-19T04:48:13.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.225+0000 7fddccc00640 1 -- 172.21.15.123:0/2545363241 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fddd000fc60 con 0x562ef7c5d760 2024-09-19T04:48:13.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.225+0000 7fddeca2c740 10 client.15609 fetch_fsmap learned FSMap version 29 2024-09-19T04:48:13.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.226+0000 7fddeca2c740 1 -- 172.21.15.123:0/2545363241 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ef7c76d10 con 0x562ef7c5d760 2024-09-19T04:48:13.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.226+0000 7fddccc00640 1 -- 172.21.15.123:0/2545363241 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdddc003e50 con 0x562ef7c5d760 2024-09-19T04:48:13.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.226+0000 7fddccc00640 1 -- 172.21.15.123:0/2545363241 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fddd0025550 con 0x562ef7c5d760 2024-09-19T04:48:13.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.226+0000 7fddeca2c740 10 client.15609 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:48:13.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.226+0000 7fddeca2c740 10 client.15609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:48:13.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.226+0000 7fddeca2c740 20 client.15609 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:13.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.226+0000 7fddeca2c740 1 --2- 172.21.15.123:0/2545363241 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x562ef7c78120 0x562ef7c7a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.226+0000 7fddeca2c740 4 client.15609 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:48:13.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.226+0000 7fddeca2c740 1 -- 172.21.15.123:0/2545363241 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ef7bd0950 con 0x562ef7c78120 2024-09-19T04:48:13.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.227+0000 7fdde3e00640 1 --2- 172.21.15.123:0/2545363241 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x562ef7c78120 0x562ef7c7a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.227+0000 7fdde3e00640 1 --2- 172.21.15.123:0/2545363241 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x562ef7c78120 0x562ef7c7a530 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec65ed40fde29c69 server_cookie=814abe7b1f1650b8 in_seq=0 out_seq=0 2024-09-19T04:48:13.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.227+0000 7fddccc00640 10 client.15609 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:48:13.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.228+0000 7fddccc00640 1 -- 172.21.15.123:0/2545363241 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ef7bd0950 con 0x562ef7c78120 2024-09-19T04:48:13.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.228+0000 7fddccc00640 10 client.15609 handle_command_reply: tid=0 2024-09-19T04:48:13.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.228+0000 7fddeca2c740 1 client.15609 shutdown 2024-09-19T04:48:13.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.228+0000 7fddeca2c740 1 -- 172.21.15.123:0/2545363241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 msgr2=0x562ef7c65480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:13.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.228+0000 7fddeca2c740 1 --2- 172.21.15.123:0/2545363241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c65480 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fddd0012700 tx=0x7fddd0012bc0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.228+0000 7fddeca2c740 1 -- 172.21.15.123:0/2545363241 shutdown_connections 2024-09-19T04:48:13.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.228+0000 7fddeca2c740 1 --2- 172.21.15.123:0/2545363241 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x562ef7c78120 0x562ef7c7a530 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.228+0000 7fddeca2c740 1 --2- 172.21.15.123:0/2545363241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c68020 0x562ef7c659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.228+0000 7fddeca2c740 1 --2- 172.21.15.123:0/2545363241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c65480 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.228+0000 7fddeca2c740 1 --2- 172.21.15.123:0/2545363241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 0x562ef7c64f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.228+0000 7fddccc00640 0 client.15609 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:48:13.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.228+0000 7fddeca2c740 1 -- 172.21.15.123:0/2545363241 >> 172.21.15.123:0/2545363241 conn(0x562ef7c593d0 msgr2=0x562ef7c5f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:13.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.229+0000 7fddeca2c740 1 -- 172.21.15.123:0/2545363241 shutdown_connections 2024-09-19T04:48:13.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.229+0000 7fddeca2c740 1 -- 172.21.15.123:0/2545363241 wait complete. 2024-09-19T04:48:13.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.229+0000 7fddeca2c740 20 client.15609 trim_cache size 0 max 16384 2024-09-19T04:48:13.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.242+0000 7fddeca2c740 1 Processor -- start 2024-09-19T04:48:13.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.242+0000 7fddeca2c740 1 -- start start 2024-09-19T04:48:13.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.243+0000 7fddeca2c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 0x562ef7c5efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.243+0000 7fddeca2c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c7fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.243+0000 7fddeca2c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c68020 0x562ef7c80340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.243+0000 7fddeca2c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41af7b0 con 0x562ef7c5d760 2024-09-19T04:48:13.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.243+0000 7fddeca2c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41b1320 con 0x562ef7c5ce40 2024-09-19T04:48:13.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.243+0000 7fddeca2c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41afe80 con 0x562ef7c68020 2024-09-19T04:48:13.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.243+0000 7fdde2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c7fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.243+0000 7fdde2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c7fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38260/0 (socket says 172.21.15.123:38260) 2024-09-19T04:48:13.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.243+0000 7fdde2a00640 1 -- 172.21.15.123:0/3189269149 learned_addr learned my addr 172.21.15.123:0/3189269149 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:13.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.243+0000 7fdde3e00640 1 --2- 172.21.15.123:0/3189269149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c68020 0x562ef7c80340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.243+0000 7fdde3e00640 1 --2- 172.21.15.123:0/3189269149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 0x562ef7c5efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.243+0000 7fdde2a00640 1 -- 172.21.15.123:0/3189269149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c68020 msgr2=0x562ef7c80340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:13.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.243+0000 7fdde2a00640 1 --2- 172.21.15.123:0/3189269149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c68020 0x562ef7c80340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.243+0000 7fdde2a00640 1 -- 172.21.15.123:0/3189269149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 msgr2=0x562ef7c5efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:13.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.243+0000 7fdde2a00640 1 --2- 172.21.15.123:0/3189269149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 0x562ef7c5efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.243+0000 7fdde2a00640 1 -- 172.21.15.123:0/3189269149 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ef7c5a480 con 0x562ef7c5d760 2024-09-19T04:48:13.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.243+0000 7fdde3e00640 1 --2- 172.21.15.123:0/3189269149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 0x562ef7c5efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:48:13.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.244+0000 7fdde2a00640 1 --2- 172.21.15.123:0/3189269149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c7fe00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fddd0000c00 tx=0x7fddd0005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45eb49691c9541ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:13.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.244+0000 7fddcf400640 1 -- 172.21.15.123:0/3189269149 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd002a030 con 0x562ef7c5d760 2024-09-19T04:48:13.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.244+0000 7fddcf400640 1 -- 172.21.15.123:0/3189269149 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddd0025b90 con 0x562ef7c5d760 2024-09-19T04:48:13.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.244+0000 7fddcf400640 1 -- 172.21.15.123:0/3189269149 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd000f790 con 0x562ef7c5d760 2024-09-19T04:48:13.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.244+0000 7fddeca2c740 1 -- 172.21.15.123:0/3189269149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 msgr2=0x562ef7c7fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:13.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.244+0000 7fddeca2c740 1 --2- 172.21.15.123:0/3189269149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c7fe00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fddd0000c00 tx=0x7fddd0005e70 comp rx=0 tx=0).stop 2024-09-19T04:48:13.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.244+0000 7fddeca2c740 1 -- 172.21.15.123:0/3189269149 shutdown_connections 2024-09-19T04:48:13.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.244+0000 7fddeca2c740 1 --2- 172.21.15.123:0/3189269149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c68020 0x562ef7c80340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.245+0000 7fddeca2c740 1 --2- 172.21.15.123:0/3189269149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c7fe00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.245+0000 7fddeca2c740 1 --2- 172.21.15.123:0/3189269149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c5ce40 0x562ef7c5efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.245+0000 7fddeca2c740 1 -- 172.21.15.123:0/3189269149 >> 172.21.15.123:0/3189269149 conn(0x562ef7c593d0 msgr2=0x562ef7c5e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:13.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.245+0000 7fddeca2c740 1 -- 172.21.15.123:0/3189269149 shutdown_connections 2024-09-19T04:48:13.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.245+0000 7fddeca2c740 1 -- 172.21.15.123:0/3189269149 wait complete. 2024-09-19T04:48:13.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.245+0000 7fddeca2c740 1 -- start start 2024-09-19T04:48:13.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.245+0000 7fddeca2c740 1 Processor -- start 2024-09-19T04:48:13.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.246+0000 7fddeca2c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c5ce40 0x562ef7c6c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.246+0000 7fddeca2c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c6c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.246+0000 7fddeca2c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c68020 0x562ef7cea5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.246+0000 7fddeca2c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41b0c20 con 0x562ef7c5d760 2024-09-19T04:48:13.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.246+0000 7fddeca2c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41b0550 con 0x562ef7c68020 2024-09-19T04:48:13.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.246+0000 7fddeca2c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41ad790 con 0x562ef7c5ce40 2024-09-19T04:48:13.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.246+0000 7fddeca2c740 10 client.0 _ll_register_callbacks cb 0x7fddd0009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:13.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.246+0000 7fdde2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c6c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.246+0000 7fdde3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c5ce40 0x562ef7c6c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.246+0000 7fdde2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c6c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38276/0 (socket says 172.21.15.123:38276) 2024-09-19T04:48:13.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.246+0000 7fdde2a00640 1 -- 172.21.15.123:0/3748477688 learned_addr learned my addr 172.21.15.123:0/3748477688 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:13.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.246+0000 7fdde3e00640 1 --2- 172.21.15.123:0/3748477688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c68020 0x562ef7cea5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.246+0000 7fdde2a00640 1 -- 172.21.15.123:0/3748477688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c5ce40 msgr2=0x562ef7c6c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:13.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.246+0000 7fdde2a00640 1 --2- 172.21.15.123:0/3748477688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c5ce40 0x562ef7c6c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.246+0000 7fdde2a00640 1 -- 172.21.15.123:0/3748477688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c68020 msgr2=0x562ef7cea5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:13.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.246+0000 7fdde2a00640 1 --2- 172.21.15.123:0/3748477688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c68020 0x562ef7cea5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.246+0000 7fdde2a00640 1 -- 172.21.15.123:0/3748477688 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ef7c58c80 con 0x562ef7c5d760 2024-09-19T04:48:13.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.247+0000 7fdde2a00640 1 --2- 172.21.15.123:0/3748477688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c6c890 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x562ef7c5dd80 tx=0x7fddd0009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd1173cc956b5e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:13.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.247+0000 7fddccc00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:48:13.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.247+0000 7fddccc00640 1 -- 172.21.15.123:0/3748477688 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd002a070 con 0x562ef7c5d760 2024-09-19T04:48:13.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.247+0000 7fddccc00640 1 -- 172.21.15.123:0/3748477688 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fddd000f790 con 0x562ef7c5d760 2024-09-19T04:48:13.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.247+0000 7fddeca2c740 1 -- 172.21.15.123:0/3748477688 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddd0025b90 con 0x562ef7c5d760 2024-09-19T04:48:13.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.247+0000 7fddccc00640 1 -- 172.21.15.123:0/3748477688 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fddd000f990 con 0x562ef7c5d760 2024-09-19T04:48:13.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.247+0000 7fddeca2c740 10 client.15615 fetch_fsmap learned FSMap version 29 2024-09-19T04:48:13.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.247+0000 7fddeca2c740 1 -- 172.21.15.123:0/3748477688 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ef7cebac0 con 0x562ef7c5d760 2024-09-19T04:48:13.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.248+0000 7fddccc00640 1 -- 172.21.15.123:0/3748477688 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddd0025b90 con 0x562ef7c5d760 2024-09-19T04:48:13.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.248+0000 7fddccc00640 1 -- 172.21.15.123:0/3748477688 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fddd00037b0 con 0x562ef7c5d760 2024-09-19T04:48:13.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.248+0000 7fddeca2c740 10 client.15615 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:48:13.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.248+0000 7fddeca2c740 10 client.15615 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:48:13.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.248+0000 7fddeca2c740 20 client.15615 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:13.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.248+0000 7fddeca2c740 1 --2- 172.21.15.123:0/3748477688 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x562ef7cec130 0x562ef7cee550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:13.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.248+0000 7fddeca2c740 4 client.15615 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:48:13.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.248+0000 7fddeca2c740 1 -- 172.21.15.123:0/3748477688 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ef7bcae60 con 0x562ef7cec130 2024-09-19T04:48:13.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.248+0000 7fdde3e00640 1 --2- 172.21.15.123:0/3748477688 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x562ef7cec130 0x562ef7cee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:13.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.249+0000 7fdde3e00640 1 --2- 172.21.15.123:0/3748477688 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x562ef7cec130 0x562ef7cee550 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=debe2722b101ae16 server_cookie=8e778d895bfea3ed in_seq=0 out_seq=0 2024-09-19T04:48:13.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.249+0000 7fddccc00640 10 client.15615 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:48:13.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.249+0000 7fddccc00640 1 -- 172.21.15.123:0/3748477688 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ef7bcae60 con 0x562ef7cec130 2024-09-19T04:48:13.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.249+0000 7fddccc00640 10 client.15615 handle_command_reply: tid=0 2024-09-19T04:48:13.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.249+0000 7fddeca2c740 1 client.15615 shutdown 2024-09-19T04:48:13.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.249+0000 7fddeca2c740 1 -- 172.21.15.123:0/3748477688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 msgr2=0x562ef7c6c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:13.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.249+0000 7fddeca2c740 1 --2- 172.21.15.123:0/3748477688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c6c890 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x562ef7c5dd80 tx=0x7fddd0009280 comp rx=0 tx=0).stop 2024-09-19T04:48:13.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.249+0000 7fddeca2c740 1 -- 172.21.15.123:0/3748477688 shutdown_connections 2024-09-19T04:48:13.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.249+0000 7fddeca2c740 1 --2- 172.21.15.123:0/3748477688 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x562ef7cec130 0x562ef7cee550 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.249+0000 7fddeca2c740 1 --2- 172.21.15.123:0/3748477688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ef7c68020 0x562ef7cea5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.249+0000 7fddccc00640 0 client.15615 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:48:13.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.249+0000 7fddeca2c740 1 --2- 172.21.15.123:0/3748477688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef7c5d760 0x562ef7c6c890 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.249+0000 7fddeca2c740 1 --2- 172.21.15.123:0/3748477688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ef7c5ce40 0x562ef7c6c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.249+0000 7fddeca2c740 1 -- 172.21.15.123:0/3748477688 >> 172.21.15.123:0/3748477688 conn(0x562ef7c593d0 msgr2=0x562ef7c5f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:13.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.249+0000 7fddeca2c740 1 -- 172.21.15.123:0/3748477688 shutdown_connections 2024-09-19T04:48:13.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.249+0000 7fddeca2c740 1 -- 172.21.15.123:0/3748477688 wait complete. 2024-09-19T04:48:13.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.250+0000 7fddeca2c740 20 client.15615 trim_cache size 0 max 16384 2024-09-19T04:48:13.255 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:48:13.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.250+0000 7fdde8a00640 1 -- 172.21.15.123:0/1539602698 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fddc80a35e0 msgr2=0x7fddc80a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:13.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.250+0000 7fdde8a00640 1 --2- 172.21.15.123:0/1539602698 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fddc80a35e0 0x7fddc80a5aa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fddd40cd7b0 tx=0x7fddd40d3ce0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.250+0000 7fdde8a00640 1 -- 172.21.15.123:0/1539602698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde4074510 msgr2=0x7fdde419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:13.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.250+0000 7fdde8a00640 1 --2- 172.21.15.123:0/1539602698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde4074510 0x7fdde419a440 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fddd400e9f0 tx=0x7fddd400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.250+0000 7fdde8a00640 1 -- 172.21.15.123:0/1539602698 shutdown_connections 2024-09-19T04:48:13.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.250+0000 7fdde8a00640 1 --2- 172.21.15.123:0/1539602698 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fddc80a35e0 0x7fddc80a5aa0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.250+0000 7fdde8a00640 1 --2- 172.21.15.123:0/1539602698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdde410e280 0x7fdde419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.250+0000 7fdde8a00640 1 --2- 172.21.15.123:0/1539602698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdde4074e30 0x7fdde419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.250+0000 7fdde8a00640 1 --2- 172.21.15.123:0/1539602698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde4074510 0x7fdde419a440 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:13.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.250+0000 7fdde8a00640 1 -- 172.21.15.123:0/1539602698 >> 172.21.15.123:0/1539602698 conn(0x7fdde40fd740 msgr2=0x7fdde40fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:13.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.250+0000 7fdde8a00640 1 -- 172.21.15.123:0/1539602698 shutdown_connections 2024-09-19T04:48:13.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:13.250+0000 7fdde8a00640 1 -- 172.21.15.123:0/1539602698 wait complete. 2024-09-19T04:48:13.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:13 smithi123 ceph-mon[30511]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T04:48:13.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:13 smithi183 ceph-mon[35333]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T04:48:13.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:13 smithi193 ceph-mon[35592]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T04:48:14.071 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:48:14.439 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:48:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:14.999+0000 7f1520c00640 1 -- 172.21.15.123:0/2864365709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f151c1049a0 msgr2=0x7f151c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:14.999+0000 7f1520c00640 1 --2- 172.21.15.123:0/2864365709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f151c1049a0 0x7f151c1089f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1508009900 tx=0x7f150802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:14.999+0000 7f1520c00640 1 -- 172.21.15.123:0/2864365709 shutdown_connections 2024-09-19T04:48:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:14.999+0000 7f1520c00640 1 --2- 172.21.15.123:0/2864365709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f151c109120 0x7f151c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:14.999+0000 7f1520c00640 1 --2- 172.21.15.123:0/2864365709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f151c1049a0 0x7f151c1089f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:14.999+0000 7f1520c00640 1 --2- 172.21.15.123:0/2864365709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f151c103ff0 0x7f151c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:14.999+0000 7f1520c00640 1 -- 172.21.15.123:0/2864365709 >> 172.21.15.123:0/2864365709 conn(0x7f151c0ffa90 msgr2=0x7f151c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:15.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:14.999+0000 7f1520c00640 1 -- 172.21.15.123:0/2864365709 shutdown_connections 2024-09-19T04:48:15.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:14.999+0000 7f1520c00640 1 -- 172.21.15.123:0/2864365709 wait complete. 2024-09-19T04:48:15.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.000+0000 7f1520c00640 1 Processor -- start 2024-09-19T04:48:15.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.000+0000 7f1520c00640 1 -- start start 2024-09-19T04:48:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.001+0000 7f1520c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f151c103ff0 0x7f151c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.001+0000 7f1520c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f151c1049a0 0x7f151c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.001+0000 7f1520c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f151c109120 0x7f151c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.001+0000 7f1520c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f151c114190 con 0x7f151c109120 2024-09-19T04:48:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.001+0000 7f1520c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f151c114010 con 0x7f151c1049a0 2024-09-19T04:48:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.001+0000 7f1520c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f151c114310 con 0x7f151c103ff0 2024-09-19T04:48:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.001+0000 7f1519e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f151c103ff0 0x7f151c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.001+0000 7f1519e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f151c103ff0 0x7f151c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:37886/0 (socket says 172.21.15.123:37886) 2024-09-19T04:48:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.001+0000 7f1519e00640 1 -- 172.21.15.123:0/2360664254 learned_addr learned my addr 172.21.15.123:0/2360664254 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.001+0000 7f151a800640 1 --2- 172.21.15.123:0/2360664254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f151c109120 0x7f151c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.001+0000 7f1519400640 1 --2- 172.21.15.123:0/2360664254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f151c1049a0 0x7f151c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.001+0000 7f1519e00640 1 -- 172.21.15.123:0/2360664254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f151c1049a0 msgr2=0x7f151c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:15.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.001+0000 7f1519e00640 1 --2- 172.21.15.123:0/2360664254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f151c1049a0 0x7f151c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.001+0000 7f1519e00640 1 -- 172.21.15.123:0/2360664254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f151c109120 msgr2=0x7f151c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:15.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.001+0000 7f1519e00640 1 --2- 172.21.15.123:0/2360664254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f151c109120 0x7f151c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.001+0000 7f1519e00640 1 -- 172.21.15.123:0/2360664254 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151c19f5c0 con 0x7f151c103ff0 2024-09-19T04:48:15.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.001+0000 7f151a800640 1 --2- 172.21.15.123:0/2360664254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f151c109120 0x7f151c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:48:15.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.002+0000 7f1519e00640 1 --2- 172.21.15.123:0/2360664254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f151c103ff0 0x7f151c19a610 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f150c00d770 tx=0x7f150c00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:15.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.002+0000 7f1512a00640 1 -- 172.21.15.123:0/2360664254 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c017070 con 0x7f151c103ff0 2024-09-19T04:48:15.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.002+0000 7f1512a00640 1 -- 172.21.15.123:0/2360664254 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f150c005500 con 0x7f151c103ff0 2024-09-19T04:48:15.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.002+0000 7f1512a00640 1 -- 172.21.15.123:0/2360664254 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c004e00 con 0x7f151c103ff0 2024-09-19T04:48:15.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.002+0000 7f1520c00640 1 -- 172.21.15.123:0/2360664254 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151c19f8b0 con 0x7f151c103ff0 2024-09-19T04:48:15.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.002+0000 7f1520c00640 1 -- 172.21.15.123:0/2360664254 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151c1a7190 con 0x7f151c103ff0 2024-09-19T04:48:15.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.005+0000 7f1523879740 1 Processor -- start 2024-09-19T04:48:15.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.005+0000 7f1512a00640 1 -- 172.21.15.123:0/2360664254 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f150c004fa0 con 0x7f151c103ff0 2024-09-19T04:48:15.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.005+0000 7f1523879740 1 -- start start 2024-09-19T04:48:15.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.006+0000 7f1523879740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b27c70 0x563622b28050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.006+0000 7f1523879740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b28590 0x563622b32910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.006+0000 7f1523879740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b32e50 0x563622b35240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.006+0000 7f1523879740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563622a8e3e0 con 0x563622b27c70 2024-09-19T04:48:15.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.006+0000 7f1523879740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563622a35d80 con 0x563622b28590 2024-09-19T04:48:15.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.006+0000 7f1523879740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563622a8a3a0 con 0x563622b32e50 2024-09-19T04:48:15.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.006+0000 7f151a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b27c70 0x563622b28050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.006+0000 7f151a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b27c70 0x563622b28050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38306/0 (socket says 172.21.15.123:38306) 2024-09-19T04:48:15.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.006+0000 7f151a800640 1 -- 172.21.15.123:0/3338784438 learned_addr learned my addr 172.21.15.123:0/3338784438 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:15.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.006+0000 7f1519400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b32e50 0x563622b35240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.006+0000 7f1519e00640 1 --2- 172.21.15.123:0/3338784438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b28590 0x563622b32910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.007+0000 7f1519400640 1 -- 172.21.15.123:0/3338784438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b28590 msgr2=0x563622b32910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:15.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.007+0000 7f1519400640 1 --2- 172.21.15.123:0/3338784438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b28590 0x563622b32910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.007+0000 7f1519400640 1 -- 172.21.15.123:0/3338784438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b27c70 msgr2=0x563622b28050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:15.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.007+0000 7f1519400640 1 --2- 172.21.15.123:0/3338784438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b27c70 0x563622b28050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.007+0000 7f1519400640 1 -- 172.21.15.123:0/3338784438 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563622a54ae0 con 0x563622b32e50 2024-09-19T04:48:15.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.007+0000 7f1519400640 1 --2- 172.21.15.123:0/3338784438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b32e50 0x563622b35240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f150802a9e0 tx=0x7f1508005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb37cdec2c855fb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:15.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.007+0000 7f1507000640 1 -- 172.21.15.123:0/3338784438 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508035030 con 0x563622b32e50 2024-09-19T04:48:15.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.007+0000 7f1512a00640 1 --2- 172.21.15.123:0/2360664254 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f15000a3430 0x7f15000a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.007+0000 7f1507000640 1 -- 172.21.15.123:0/3338784438 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15080043c0 con 0x563622b32e50 2024-09-19T04:48:15.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.007+0000 7f1507000640 1 -- 172.21.15.123:0/3338784438 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15080035f0 con 0x563622b32e50 2024-09-19T04:48:15.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.008+0000 7f1512a00640 1 -- 172.21.15.123:0/2360664254 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f150c0cf7d0 con 0x7f151c103ff0 2024-09-19T04:48:15.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.008+0000 7f151a800640 1 --2- 172.21.15.123:0/2360664254 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f15000a3430 0x7f15000a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.008+0000 7f1523879740 1 -- 172.21.15.123:0/3338784438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b32e50 msgr2=0x563622b35240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:15.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.008+0000 7f1523879740 1 --2- 172.21.15.123:0/3338784438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b32e50 0x563622b35240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f150802a9e0 tx=0x7f1508005e60 comp rx=0 tx=0).stop 2024-09-19T04:48:15.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.008+0000 7f1523879740 1 -- 172.21.15.123:0/3338784438 shutdown_connections 2024-09-19T04:48:15.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.008+0000 7f1523879740 1 --2- 172.21.15.123:0/3338784438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b32e50 0x563622b35240 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.008+0000 7f1523879740 1 --2- 172.21.15.123:0/3338784438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b28590 0x563622b32910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.008+0000 7f1523879740 1 --2- 172.21.15.123:0/3338784438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b27c70 0x563622b28050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.008+0000 7f151a800640 1 --2- 172.21.15.123:0/2360664254 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f15000a3430 0x7f15000a58f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1514009fd0 tx=0x7f1514009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:15.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.008+0000 7f1523879740 1 -- 172.21.15.123:0/3338784438 >> 172.21.15.123:0/3338784438 conn(0x563622b24200 msgr2=0x563622b245e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:15.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.008+0000 7f1523879740 1 -- 172.21.15.123:0/3338784438 shutdown_connections 2024-09-19T04:48:15.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.009+0000 7f1523879740 1 -- 172.21.15.123:0/3338784438 wait complete. 2024-09-19T04:48:15.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.010+0000 7f1523879740 1 -- start start 2024-09-19T04:48:15.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.010+0000 7f1523879740 1 Processor -- start 2024-09-19T04:48:15.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.010+0000 7f1523879740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b27c70 0x563622b2fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.010+0000 7f1523879740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b28590 0x563622b302c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.010+0000 7f1523879740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b32e50 0x563622b30800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.010+0000 7f1523879740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563622b35a20 con 0x563622b28590 2024-09-19T04:48:15.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.010+0000 7f1523879740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563622a490f0 con 0x563622b32e50 2024-09-19T04:48:15.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.010+0000 7f1523879740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563622b35ba0 con 0x563622b27c70 2024-09-19T04:48:15.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.010+0000 7f1523879740 10 client.0 _ll_register_callbacks cb 0x563622a30bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:15.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.011+0000 7f1519400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b28590 0x563622b302c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.011+0000 7f1519e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b27c70 0x563622b2fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.011+0000 7f1519400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b28590 0x563622b302c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38318/0 (socket says 172.21.15.123:38318) 2024-09-19T04:48:15.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.011+0000 7f1519400640 1 -- 172.21.15.123:0/3180683148 learned_addr learned my addr 172.21.15.123:0/3180683148 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:15.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.011+0000 7f151a800640 1 --2- 172.21.15.123:0/3180683148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b32e50 0x563622b30800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.011+0000 7f1519e00640 1 -- 172.21.15.123:0/3180683148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b32e50 msgr2=0x563622b30800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:15.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.011+0000 7f1519e00640 1 --2- 172.21.15.123:0/3180683148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b32e50 0x563622b30800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.011+0000 7f1519e00640 1 -- 172.21.15.123:0/3180683148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b28590 msgr2=0x563622b302c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:15.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.011+0000 7f1519e00640 1 --2- 172.21.15.123:0/3180683148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b28590 0x563622b302c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.011+0000 7f1519e00640 1 -- 172.21.15.123:0/3180683148 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636227ff3c0 con 0x563622b27c70 2024-09-19T04:48:15.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.011+0000 7f1519400640 1 --2- 172.21.15.123:0/3180683148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b28590 0x563622b302c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:15.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.011+0000 7f1519e00640 1 --2- 172.21.15.123:0/3180683148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b27c70 0x563622b2fd80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f150c0ccef0 tx=0x7f150c09b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7afe6fc440bfa2b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:15.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.011+0000 7f14fb400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:48:15.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.011+0000 7f14fb400640 1 -- 172.21.15.123:0/3180683148 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c020070 con 0x563622b27c70 2024-09-19T04:48:15.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.012+0000 7f14fb400640 1 -- 172.21.15.123:0/3180683148 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f150c0860b0 con 0x563622b27c70 2024-09-19T04:48:15.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.012+0000 7f1523879740 1 -- 172.21.15.123:0/3180683148 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15080043c0 con 0x563622b27c70 2024-09-19T04:48:15.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.012+0000 7f14fb400640 1 -- 172.21.15.123:0/3180683148 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f150c0046c0 con 0x563622b27c70 2024-09-19T04:48:15.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.012+0000 7f1523879740 10 client.25675 fetch_fsmap learned FSMap version 30 2024-09-19T04:48:15.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.012+0000 7f1523879740 1 -- 172.21.15.123:0/3180683148 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563622b41ab0 con 0x563622b27c70 2024-09-19T04:48:15.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.012+0000 7f14fb400640 1 -- 172.21.15.123:0/3180683148 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15080043c0 con 0x563622b27c70 2024-09-19T04:48:15.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.012+0000 7f14fb400640 1 -- 172.21.15.123:0/3180683148 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8382+0+0 (secure 0 0 0) 0x7f150c0980f0 con 0x563622b27c70 2024-09-19T04:48:15.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.012+0000 7f1523879740 10 client.25675 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:48:15.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.012+0000 7f1523879740 10 client.25675 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:48:15.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.012+0000 7f1523879740 20 client.25675 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:15.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.012+0000 7f1523879740 1 --2- 172.21.15.123:0/3180683148 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x563622b42f70 0x563622b45380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.012+0000 7f1523879740 4 client.25675 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:48:15.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.012+0000 7f1523879740 1 -- 172.21.15.123:0/3180683148 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563622a9b950 con 0x563622b42f70 2024-09-19T04:48:15.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.013+0000 7f1519400640 1 --2- 172.21.15.123:0/3180683148 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x563622b42f70 0x563622b45380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.013+0000 7f1519400640 1 --2- 172.21.15.123:0/3180683148 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x563622b42f70 0x563622b45380 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cba6a21823208452 server_cookie=d92387d7f2cc3ba8 in_seq=0 out_seq=0 2024-09-19T04:48:15.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.013+0000 7f14fb400640 10 client.25675 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:48:15.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.014+0000 7f14fb400640 1 -- 172.21.15.123:0/3180683148 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563622a9b950 con 0x563622b42f70 2024-09-19T04:48:15.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.014+0000 7f14fb400640 10 client.25675 handle_command_reply: tid=0 2024-09-19T04:48:15.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.014+0000 7f1523879740 1 client.25675 shutdown 2024-09-19T04:48:15.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.014+0000 7f1523879740 1 -- 172.21.15.123:0/3180683148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b27c70 msgr2=0x563622b2fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:15.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.014+0000 7f1523879740 1 --2- 172.21.15.123:0/3180683148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b27c70 0x563622b2fd80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f150c0ccef0 tx=0x7f150c09b850 comp rx=0 tx=0).stop 2024-09-19T04:48:15.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.014+0000 7f1523879740 1 -- 172.21.15.123:0/3180683148 shutdown_connections 2024-09-19T04:48:15.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.014+0000 7f1523879740 1 --2- 172.21.15.123:0/3180683148 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x563622b42f70 0x563622b45380 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.014+0000 7f1523879740 1 --2- 172.21.15.123:0/3180683148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b32e50 0x563622b30800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.014+0000 7f1523879740 1 --2- 172.21.15.123:0/3180683148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b28590 0x563622b302c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.014+0000 7f14fb400640 0 client.25675 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:48:15.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.014+0000 7f1523879740 1 --2- 172.21.15.123:0/3180683148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b27c70 0x563622b2fd80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.014+0000 7f1523879740 1 -- 172.21.15.123:0/3180683148 >> 172.21.15.123:0/3180683148 conn(0x563622b24200 msgr2=0x563622b29f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:15.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.014+0000 7f1523879740 1 -- 172.21.15.123:0/3180683148 shutdown_connections 2024-09-19T04:48:15.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.015+0000 7f1523879740 1 -- 172.21.15.123:0/3180683148 wait complete. 2024-09-19T04:48:15.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.015+0000 7f1523879740 20 client.25675 trim_cache size 0 max 16384 2024-09-19T04:48:15.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.029+0000 7f1523879740 1 Processor -- start 2024-09-19T04:48:15.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.029+0000 7f1523879740 1 -- start start 2024-09-19T04:48:15.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.029+0000 7f1523879740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b27c70 0x563622b320b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.029+0000 7f1523879740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b28590 0x563622b2f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.029+0000 7f1523879740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b32e50 0x563622b4ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.029+0000 7f1523879740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1af980 con 0x563622b32e50 2024-09-19T04:48:15.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.029+0000 7f1523879740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1b15b0 con 0x563622b27c70 2024-09-19T04:48:15.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.029+0000 7f1523879740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1b0050 con 0x563622b28590 2024-09-19T04:48:15.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.029+0000 7f151a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b32e50 0x563622b4ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.029+0000 7f1519400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b28590 0x563622b2f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.029+0000 7f1519400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b28590 0x563622b2f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:37924/0 (socket says 172.21.15.123:37924) 2024-09-19T04:48:15.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.029+0000 7f1519400640 1 -- 172.21.15.123:0/1193174772 learned_addr learned my addr 172.21.15.123:0/1193174772 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:15.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.029+0000 7f151a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b32e50 0x563622b4ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38320/0 (socket says 172.21.15.123:38320) 2024-09-19T04:48:15.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.030+0000 7f1519e00640 1 --2- 172.21.15.123:0/1193174772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b27c70 0x563622b320b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.030+0000 7f1519400640 1 -- 172.21.15.123:0/1193174772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b27c70 msgr2=0x563622b320b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:15.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.030+0000 7f1519400640 1 --2- 172.21.15.123:0/1193174772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b27c70 0x563622b320b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.030+0000 7f1519400640 1 -- 172.21.15.123:0/1193174772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b32e50 msgr2=0x563622b4ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:15.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.030+0000 7f1519400640 1 --2- 172.21.15.123:0/1193174772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b32e50 0x563622b4ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.030+0000 7f1519400640 1 -- 172.21.15.123:0/1193174772 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563622b27aa0 con 0x563622b28590 2024-09-19T04:48:15.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.030+0000 7f1519e00640 1 --2- 172.21.15.123:0/1193174772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b27c70 0x563622b320b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:48:15.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.030+0000 7f1519400640 1 --2- 172.21.15.123:0/1193174772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b28590 0x563622b2f6c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1508009850 tx=0x7f1508004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e580b3fd33523bae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:15.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.030+0000 7f1506600640 1 -- 172.21.15.123:0/1193174772 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508035040 con 0x563622b28590 2024-09-19T04:48:15.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.030+0000 7f1506600640 1 -- 172.21.15.123:0/1193174772 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1508049040 con 0x563622b28590 2024-09-19T04:48:15.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.030+0000 7f1506600640 1 -- 172.21.15.123:0/1193174772 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508031960 con 0x563622b28590 2024-09-19T04:48:15.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.031+0000 7f1523879740 1 -- 172.21.15.123:0/1193174772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b28590 msgr2=0x563622b2f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:15.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.031+0000 7f1523879740 1 --2- 172.21.15.123:0/1193174772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b28590 0x563622b2f6c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1508009850 tx=0x7f1508004430 comp rx=0 tx=0).stop 2024-09-19T04:48:15.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.031+0000 7f1523879740 1 -- 172.21.15.123:0/1193174772 shutdown_connections 2024-09-19T04:48:15.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.031+0000 7f1523879740 1 --2- 172.21.15.123:0/1193174772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b32e50 0x563622b4ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.031+0000 7f1523879740 1 --2- 172.21.15.123:0/1193174772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b28590 0x563622b2f6c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.031+0000 7f1523879740 1 --2- 172.21.15.123:0/1193174772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b27c70 0x563622b320b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.031+0000 7f1523879740 1 -- 172.21.15.123:0/1193174772 >> 172.21.15.123:0/1193174772 conn(0x563622b24200 msgr2=0x563622b23380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:15.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.031+0000 7f1523879740 1 -- 172.21.15.123:0/1193174772 shutdown_connections 2024-09-19T04:48:15.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.031+0000 7f1523879740 1 -- 172.21.15.123:0/1193174772 wait complete. 2024-09-19T04:48:15.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.032+0000 7f1523879740 1 -- start start 2024-09-19T04:48:15.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.032+0000 7f1523879740 1 Processor -- start 2024-09-19T04:48:15.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.032+0000 7f1523879740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b27c70 0x563622b2cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.032+0000 7f1523879740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b28590 0x563622b2d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.032+0000 7f1523879740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b32e50 0x563622b2d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.032+0000 7f1523879740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1b0e20 con 0x563622b28590 2024-09-19T04:48:15.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.032+0000 7f1523879740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1b0750 con 0x563622b27c70 2024-09-19T04:48:15.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.032+0000 7f1523879740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1ad960 con 0x563622b32e50 2024-09-19T04:48:15.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.032+0000 7f1523879740 10 client.0 _ll_register_callbacks cb 0x7f150c0d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:15.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.033+0000 7f151a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b32e50 0x563622b2d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.033+0000 7f151a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b32e50 0x563622b2d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:37938/0 (socket says 172.21.15.123:37938) 2024-09-19T04:48:15.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.033+0000 7f151a800640 1 -- 172.21.15.123:0/3265967583 learned_addr learned my addr 172.21.15.123:0/3265967583 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:15.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.033+0000 7f1519e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b27c70 0x563622b2cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.033+0000 7f1519400640 1 --2- 172.21.15.123:0/3265967583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b28590 0x563622b2d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.033+0000 7f151a800640 1 -- 172.21.15.123:0/3265967583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b27c70 msgr2=0x563622b2cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:15.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.033+0000 7f151a800640 1 --2- 172.21.15.123:0/3265967583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b27c70 0x563622b2cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.033+0000 7f151a800640 1 -- 172.21.15.123:0/3265967583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b28590 msgr2=0x563622b2d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:15.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.033+0000 7f151a800640 1 --2- 172.21.15.123:0/3265967583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b28590 0x563622b2d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.033+0000 7f151a800640 1 -- 172.21.15.123:0/3265967583 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563622b1f770 con 0x563622b32e50 2024-09-19T04:48:15.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.033+0000 7f151a800640 1 --2- 172.21.15.123:0/3265967583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b32e50 0x563622b2d790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1514010b00 tx=0x7f1514004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24ac7ba15d1fc874 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:15.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.033+0000 7f14fb400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:48:15.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.034+0000 7f14fb400640 1 -- 172.21.15.123:0/3265967583 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151401ebb0 con 0x563622b32e50 2024-09-19T04:48:15.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.034+0000 7f1523879740 1 -- 172.21.15.123:0/3265967583 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1508049040 con 0x563622b32e50 2024-09-19T04:48:15.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.034+0000 7f14fb400640 1 -- 172.21.15.123:0/3265967583 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f151401ed50 con 0x563622b32e50 2024-09-19T04:48:15.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.034+0000 7f14fb400640 1 -- 172.21.15.123:0/3265967583 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1514004b20 con 0x563622b32e50 2024-09-19T04:48:15.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.034+0000 7f1523879740 10 client.25681 fetch_fsmap learned FSMap version 30 2024-09-19T04:48:15.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.034+0000 7f1523879740 1 -- 172.21.15.123:0/3265967583 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563622badcf0 con 0x563622b32e50 2024-09-19T04:48:15.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.034+0000 7f14fb400640 1 -- 172.21.15.123:0/3265967583 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1508049040 con 0x563622b32e50 2024-09-19T04:48:15.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.034+0000 7f14fb400640 1 -- 172.21.15.123:0/3265967583 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8382+0+0 (secure 0 0 0) 0x7f1514028070 con 0x563622b32e50 2024-09-19T04:48:15.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.034+0000 7f1523879740 10 client.25681 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:48:15.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.035+0000 7f1523879740 10 client.25681 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:48:15.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.035+0000 7f1523879740 20 client.25681 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:15.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.035+0000 7f1523879740 1 --2- 172.21.15.123:0/3265967583 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x563622bae440 0x563622bb0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:15.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.035+0000 7f1523879740 4 client.25681 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:48:15.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.035+0000 7f1523879740 1 -- 172.21.15.123:0/3265967583 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563622a95e60 con 0x563622bae440 2024-09-19T04:48:15.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.035+0000 7f1519e00640 1 --2- 172.21.15.123:0/3265967583 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x563622bae440 0x563622bb0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:15.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.035+0000 7f1519e00640 1 --2- 172.21.15.123:0/3265967583 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x563622bae440 0x563622bb0860 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf8f080c9079cc71 server_cookie=5326304d13c2b925 in_seq=0 out_seq=0 2024-09-19T04:48:15.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.035+0000 7f14fb400640 10 client.25681 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:48:15.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.035+0000 7f14fb400640 1 -- 172.21.15.123:0/3265967583 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563622a95e60 con 0x563622bae440 2024-09-19T04:48:15.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.035+0000 7f14fb400640 10 client.25681 handle_command_reply: tid=0 2024-09-19T04:48:15.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.035+0000 7f1523879740 1 client.25681 shutdown 2024-09-19T04:48:15.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.036+0000 7f1523879740 1 -- 172.21.15.123:0/3265967583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b32e50 msgr2=0x563622b2d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:15.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.036+0000 7f1523879740 1 --2- 172.21.15.123:0/3265967583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b32e50 0x563622b2d790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1514010b00 tx=0x7f1514004ab0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.036+0000 7f1523879740 1 -- 172.21.15.123:0/3265967583 shutdown_connections 2024-09-19T04:48:15.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.036+0000 7f1523879740 1 --2- 172.21.15.123:0/3265967583 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x563622bae440 0x563622bb0860 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.036+0000 7f1523879740 1 --2- 172.21.15.123:0/3265967583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563622b32e50 0x563622b2d790 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.036+0000 7f14fb400640 0 client.25681 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:48:15.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.036+0000 7f1523879740 1 --2- 172.21.15.123:0/3265967583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563622b28590 0x563622b2d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.036+0000 7f1523879740 1 --2- 172.21.15.123:0/3265967583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563622b27c70 0x563622b2cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.036+0000 7f1523879740 1 -- 172.21.15.123:0/3265967583 >> 172.21.15.123:0/3265967583 conn(0x563622b24200 msgr2=0x563622b29710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:15.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.036+0000 7f1523879740 1 -- 172.21.15.123:0/3265967583 shutdown_connections 2024-09-19T04:48:15.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.036+0000 7f1523879740 1 -- 172.21.15.123:0/3265967583 wait complete. 2024-09-19T04:48:15.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.036+0000 7f1523879740 20 client.25681 trim_cache size 0 max 16384 2024-09-19T04:48:15.042 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:48:15.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.037+0000 7f1520c00640 1 -- 172.21.15.123:0/2360664254 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f15000a3430 msgr2=0x7f15000a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:15.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.037+0000 7f1520c00640 1 --2- 172.21.15.123:0/2360664254 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f15000a3430 0x7f15000a58f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1514009fd0 tx=0x7f1514009500 comp rx=0 tx=0).stop 2024-09-19T04:48:15.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.037+0000 7f1520c00640 1 -- 172.21.15.123:0/2360664254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f151c103ff0 msgr2=0x7f151c19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:15.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.037+0000 7f1520c00640 1 --2- 172.21.15.123:0/2360664254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f151c103ff0 0x7f151c19a610 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f150c00d770 tx=0x7f150c00dc30 comp rx=0 tx=0).stop 2024-09-19T04:48:15.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.037+0000 7f1520c00640 1 -- 172.21.15.123:0/2360664254 shutdown_connections 2024-09-19T04:48:15.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.037+0000 7f1520c00640 1 --2- 172.21.15.123:0/2360664254 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f15000a3430 0x7f15000a58f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.037+0000 7f1520c00640 1 --2- 172.21.15.123:0/2360664254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f151c109120 0x7f151c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.037+0000 7f1520c00640 1 --2- 172.21.15.123:0/2360664254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f151c1049a0 0x7f151c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.037+0000 7f1520c00640 1 --2- 172.21.15.123:0/2360664254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f151c103ff0 0x7f151c19a610 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:15.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.037+0000 7f1520c00640 1 -- 172.21.15.123:0/2360664254 >> 172.21.15.123:0/2360664254 conn(0x7f151c0ffa90 msgr2=0x7f151c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:15.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.037+0000 7f1520c00640 1 -- 172.21.15.123:0/2360664254 shutdown_connections 2024-09-19T04:48:15.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:15.037+0000 7f1520c00640 1 -- 172.21.15.123:0/2360664254 wait complete. 2024-09-19T04:48:15.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:15 smithi123 ceph-mon[30511]: pgmap v49: 97 pgs: 97 active+clean; 15 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 1.2 MiB/s wr, 33 op/s 2024-09-19T04:48:15.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:15 smithi123 ceph-mon[30511]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:48:15.808 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:15 smithi123 ceph-mon[30511]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-19T04:48:15.847 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:48:15.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:15 smithi183 ceph-mon[35333]: pgmap v49: 97 pgs: 97 active+clean; 15 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 1.2 MiB/s wr, 33 op/s 2024-09-19T04:48:15.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:15 smithi183 ceph-mon[35333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:48:15.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:15 smithi183 ceph-mon[35333]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-19T04:48:15.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:15 smithi193 ceph-mon[35592]: pgmap v49: 97 pgs: 97 active+clean; 15 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 1.2 MiB/s wr, 33 op/s 2024-09-19T04:48:15.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:15 smithi193 ceph-mon[35592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:48:15.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:15 smithi193 ceph-mon[35592]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-19T04:48:16.256 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:48:16.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.850+0000 7ffac7c00640 1 -- 172.21.15.123:0/3746397905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffac0102790 msgr2=0x7ffac01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:16.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.850+0000 7ffac7c00640 1 --2- 172.21.15.123:0/3746397905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffac0102790 0x7ffac01067e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009900 tx=0x7ffaac02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.850+0000 7ffac7c00640 1 -- 172.21.15.123:0/3746397905 shutdown_connections 2024-09-19T04:48:16.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.850+0000 7ffac7c00640 1 --2- 172.21.15.123:0/3746397905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffac0106f10 0x7ffac010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.850+0000 7ffac7c00640 1 --2- 172.21.15.123:0/3746397905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffac0102790 0x7ffac01067e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.850+0000 7ffac7c00640 1 --2- 172.21.15.123:0/3746397905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffac0101de0 0x7ffac01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.850+0000 7ffac7c00640 1 -- 172.21.15.123:0/3746397905 >> 172.21.15.123:0/3746397905 conn(0x7ffac00fd8c0 msgr2=0x7ffac00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:16.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.850+0000 7ffac7c00640 1 -- 172.21.15.123:0/3746397905 shutdown_connections 2024-09-19T04:48:16.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.850+0000 7ffac7c00640 1 -- 172.21.15.123:0/3746397905 wait complete. 2024-09-19T04:48:16.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.851+0000 7ffac7c00640 1 Processor -- start 2024-09-19T04:48:16.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.851+0000 7ffac7c00640 1 -- start start 2024-09-19T04:48:16.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.851+0000 7ffac7c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffac0101de0 0x7ffac019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.851+0000 7ffac7c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffac0102790 0x7ffac019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.851+0000 7ffac7c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffac0106f10 0x7ffac019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.851+0000 7ffac7c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffac0111fc0 con 0x7ffac0101de0 2024-09-19T04:48:16.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.851+0000 7ffac7c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffac0111e40 con 0x7ffac0106f10 2024-09-19T04:48:16.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.851+0000 7ffac7c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffac0112140 con 0x7ffac0102790 2024-09-19T04:48:16.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.852+0000 7ffac5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffac0101de0 0x7ffac019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.852+0000 7ffac5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffac0101de0 0x7ffac019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38334/0 (socket says 172.21.15.123:38334) 2024-09-19T04:48:16.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.852+0000 7ffac5200640 1 -- 172.21.15.123:0/1222589488 learned_addr learned my addr 172.21.15.123:0/1222589488 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:16.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.852+0000 7ffac5c00640 1 --2- 172.21.15.123:0/1222589488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffac0106f10 0x7ffac019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.852+0000 7ffabfe00640 1 --2- 172.21.15.123:0/1222589488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffac0102790 0x7ffac019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.852+0000 7ffac5200640 1 -- 172.21.15.123:0/1222589488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffac0102790 msgr2=0x7ffac019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:16.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.852+0000 7ffac5200640 1 --2- 172.21.15.123:0/1222589488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffac0102790 0x7ffac019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.852+0000 7ffac5200640 1 -- 172.21.15.123:0/1222589488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffac0106f10 msgr2=0x7ffac019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:16.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.852+0000 7ffac5200640 1 --2- 172.21.15.123:0/1222589488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffac0106f10 0x7ffac019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.852+0000 7ffac5200640 1 -- 172.21.15.123:0/1222589488 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac019f5e0 con 0x7ffac0101de0 2024-09-19T04:48:16.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.852+0000 7ffac5200640 1 --2- 172.21.15.123:0/1222589488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffac0101de0 0x7ffac019a630 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffab000ea60 tx=0x7ffab000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:16.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.853+0000 7ffabd600640 1 -- 172.21.15.123:0/1222589488 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab000cde0 con 0x7ffac0101de0 2024-09-19T04:48:16.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.853+0000 7ffabd600640 1 -- 172.21.15.123:0/1222589488 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffab00035f0 con 0x7ffac0101de0 2024-09-19T04:48:16.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.853+0000 7ffac7c00640 1 -- 172.21.15.123:0/1222589488 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac019f8d0 con 0x7ffac0101de0 2024-09-19T04:48:16.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.853+0000 7ffabd600640 1 -- 172.21.15.123:0/1222589488 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0010670 con 0x7ffac0101de0 2024-09-19T04:48:16.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.853+0000 7ffac7c00640 1 -- 172.21.15.123:0/1222589488 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac01a71b0 con 0x7ffac0101de0 2024-09-19T04:48:16.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.856+0000 7ffabd600640 1 -- 172.21.15.123:0/1222589488 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ffab00037a0 con 0x7ffac0101de0 2024-09-19T04:48:16.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.856+0000 7ffaca662740 1 Processor -- start 2024-09-19T04:48:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.856+0000 7ffaca662740 1 -- start start 2024-09-19T04:48:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.857+0000 7ffaca662740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f1c70 0x55ffce3f2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.857+0000 7ffaca662740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3f2590 0x55ffce3fc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.857+0000 7ffaca662740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3fce50 0x55ffce3ff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.857+0000 7ffaca662740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ffce3583e0 con 0x55ffce3f1c70 2024-09-19T04:48:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.857+0000 7ffaca662740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ffce2ffd80 con 0x55ffce3fce50 2024-09-19T04:48:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.857+0000 7ffaca662740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ffce3543a0 con 0x55ffce3f2590 2024-09-19T04:48:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.857+0000 7ffac5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3f2590 0x55ffce3fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.857+0000 7ffac5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f1c70 0x55ffce3f2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.857+0000 7ffac5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3f2590 0x55ffce3fc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:37952/0 (socket says 172.21.15.123:37952) 2024-09-19T04:48:16.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.857+0000 7ffac5200640 1 -- 172.21.15.123:0/3876716075 learned_addr learned my addr 172.21.15.123:0/3876716075 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:16.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.857+0000 7ffabfe00640 1 --2- 172.21.15.123:0/3876716075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3fce50 0x55ffce3ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.857+0000 7ffabd600640 1 --2- 172.21.15.123:0/1222589488 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffaa40a33b0 0x7ffaa40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.857+0000 7ffac5c00640 1 -- 172.21.15.123:0/3876716075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3f2590 msgr2=0x55ffce3fc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:16.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.857+0000 7ffac5c00640 1 --2- 172.21.15.123:0/3876716075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3f2590 0x55ffce3fc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.858+0000 7ffac5c00640 1 -- 172.21.15.123:0/3876716075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3fce50 msgr2=0x55ffce3ff240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:16.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.858+0000 7ffac5c00640 1 --2- 172.21.15.123:0/3876716075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3fce50 0x55ffce3ff240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.858+0000 7ffac5c00640 1 -- 172.21.15.123:0/3876716075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffce31eae0 con 0x55ffce3f1c70 2024-09-19T04:48:16.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.858+0000 7ffac5200640 1 --2- 172.21.15.123:0/3876716075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3f2590 0x55ffce3fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:16.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.858+0000 7ffabd600640 1 -- 172.21.15.123:0/1222589488 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7ffab00cffe0 con 0x7ffac0101de0 2024-09-19T04:48:16.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.858+0000 7ffac5c00640 1 --2- 172.21.15.123:0/3876716075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f1c70 0x55ffce3f2050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009f00 tx=0x7ffab8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0f20f70e41b2b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:16.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.858+0000 7ffac5c00640 1 --2- 172.21.15.123:0/1222589488 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffaa40a33b0 0x7ffaa40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.858+0000 7ffab6000640 1 -- 172.21.15.123:0/3876716075 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab801f070 con 0x55ffce3f1c70 2024-09-19T04:48:16.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.859+0000 7ffab6000640 1 -- 172.21.15.123:0/3876716075 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffab8003cb0 con 0x55ffce3f1c70 2024-09-19T04:48:16.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.859+0000 7ffab6000640 1 -- 172.21.15.123:0/3876716075 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab800e630 con 0x55ffce3f1c70 2024-09-19T04:48:16.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.859+0000 7ffaca662740 1 -- 172.21.15.123:0/3876716075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f1c70 msgr2=0x55ffce3f2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:16.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.859+0000 7ffac5c00640 1 --2- 172.21.15.123:0/1222589488 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffaa40a33b0 0x7ffaa40a5870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffab801c000 tx=0x7ffab8019950 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:16.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.859+0000 7ffaca662740 1 --2- 172.21.15.123:0/3876716075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f1c70 0x55ffce3f2050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009f00 tx=0x7ffab8011810 comp rx=0 tx=0).stop 2024-09-19T04:48:16.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.859+0000 7ffaca662740 1 -- 172.21.15.123:0/3876716075 shutdown_connections 2024-09-19T04:48:16.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.859+0000 7ffaca662740 1 --2- 172.21.15.123:0/3876716075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3fce50 0x55ffce3ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.859+0000 7ffaca662740 1 --2- 172.21.15.123:0/3876716075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3f2590 0x55ffce3fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.859+0000 7ffaca662740 1 --2- 172.21.15.123:0/3876716075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f1c70 0x55ffce3f2050 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.859+0000 7ffaca662740 1 -- 172.21.15.123:0/3876716075 >> 172.21.15.123:0/3876716075 conn(0x55ffce3ee200 msgr2=0x55ffce3ee5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:16.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.860+0000 7ffaca662740 1 -- 172.21.15.123:0/3876716075 shutdown_connections 2024-09-19T04:48:16.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.860+0000 7ffaca662740 1 -- 172.21.15.123:0/3876716075 wait complete. 2024-09-19T04:48:16.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.860+0000 7ffaca662740 1 -- start start 2024-09-19T04:48:16.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.860+0000 7ffaca662740 1 Processor -- start 2024-09-19T04:48:16.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.861+0000 7ffaca662740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3f1c70 0x55ffce3f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.861+0000 7ffaca662740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3f2590 0x55ffce3fa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.861+0000 7ffaca662740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3fce50 0x55ffce3fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.861+0000 7ffaca662740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ffce3ffa70 con 0x55ffce3fce50 2024-09-19T04:48:16.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.861+0000 7ffaca662740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ffce3130f0 con 0x55ffce3f2590 2024-09-19T04:48:16.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.861+0000 7ffaca662740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ffce3ffbf0 con 0x55ffce3f1c70 2024-09-19T04:48:16.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.861+0000 7ffaca662740 10 client.0 _ll_register_callbacks cb 0x55ffce2fabd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:16.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.861+0000 7ffac5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3f1c70 0x55ffce3f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.861+0000 7ffac5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3f1c70 0x55ffce3f9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:37964/0 (socket says 172.21.15.123:37964) 2024-09-19T04:48:16.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.861+0000 7ffac5200640 1 -- 172.21.15.123:0/758032478 learned_addr learned my addr 172.21.15.123:0/758032478 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:16.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.861+0000 7ffac5c00640 1 --2- 172.21.15.123:0/758032478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3fce50 0x55ffce3fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.861+0000 7ffabfe00640 1 --2- 172.21.15.123:0/758032478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3f2590 0x55ffce3fa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.861+0000 7ffac5200640 1 -- 172.21.15.123:0/758032478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3f2590 msgr2=0x55ffce3fa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:16.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.861+0000 7ffac5200640 1 --2- 172.21.15.123:0/758032478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3f2590 0x55ffce3fa270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.861+0000 7ffac5200640 1 -- 172.21.15.123:0/758032478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3fce50 msgr2=0x55ffce3fa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:16.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.861+0000 7ffac5200640 1 --2- 172.21.15.123:0/758032478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3fce50 0x55ffce3fa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.861+0000 7ffac5200640 1 -- 172.21.15.123:0/758032478 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffce0c93c0 con 0x55ffce3f1c70 2024-09-19T04:48:16.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.862+0000 7ffac5200640 1 --2- 172.21.15.123:0/758032478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3f1c70 0x55ffce3f9d30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffab009c240 tx=0x7ffab0009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b10a55d3770be12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:16.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.862+0000 7ffaaaa00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:48:16.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.862+0000 7ffaaaa00640 1 -- 172.21.15.123:0/758032478 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab009bdf0 con 0x55ffce3f1c70 2024-09-19T04:48:16.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.862+0000 7ffaca662740 1 -- 172.21.15.123:0/758032478 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffab8003cb0 con 0x55ffce3f1c70 2024-09-19T04:48:16.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.862+0000 7ffaaaa00640 1 -- 172.21.15.123:0/758032478 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7ffab00d20c0 con 0x55ffce3f1c70 2024-09-19T04:48:16.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.862+0000 7ffaaaa00640 1 -- 172.21.15.123:0/758032478 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffab009bc10 con 0x55ffce3f1c70 2024-09-19T04:48:16.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.862+0000 7ffaca662740 10 client.25693 fetch_fsmap learned FSMap version 30 2024-09-19T04:48:16.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.862+0000 7ffaca662740 1 -- 172.21.15.123:0/758032478 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ffce40bb60 con 0x55ffce3f1c70 2024-09-19T04:48:16.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.863+0000 7ffaaaa00640 1 -- 172.21.15.123:0/758032478 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffab8003cb0 con 0x55ffce3f1c70 2024-09-19T04:48:16.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.863+0000 7ffaaaa00640 1 -- 172.21.15.123:0/758032478 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8382+0+0 (secure 0 0 0) 0x7ffab00cd9a0 con 0x55ffce3f1c70 2024-09-19T04:48:16.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.863+0000 7ffaca662740 10 client.25693 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:48:16.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.863+0000 7ffaca662740 10 client.25693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:48:16.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.863+0000 7ffaca662740 20 client.25693 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:16.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.863+0000 7ffaca662740 1 --2- 172.21.15.123:0/758032478 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ffce40cf70 0x55ffce40f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.863+0000 7ffaca662740 4 client.25693 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:48:16.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.863+0000 7ffaca662740 1 -- 172.21.15.123:0/758032478 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffce365950 con 0x55ffce40cf70 2024-09-19T04:48:16.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.863+0000 7ffabfe00640 1 --2- 172.21.15.123:0/758032478 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ffce40cf70 0x55ffce40f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.864+0000 7ffabfe00640 1 --2- 172.21.15.123:0/758032478 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ffce40cf70 0x55ffce40f3c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11f34e52b1692f69 server_cookie=90faa55b2b13bd8f in_seq=0 out_seq=0 2024-09-19T04:48:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.864+0000 7ffaaaa00640 10 client.25693 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.864+0000 7ffaaaa00640 1 -- 172.21.15.123:0/758032478 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffce365950 con 0x55ffce40cf70 2024-09-19T04:48:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.864+0000 7ffaaaa00640 10 client.25693 handle_command_reply: tid=0 2024-09-19T04:48:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.865+0000 7ffaca662740 1 client.25693 shutdown 2024-09-19T04:48:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.865+0000 7ffaca662740 1 -- 172.21.15.123:0/758032478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3f1c70 msgr2=0x55ffce3f9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.865+0000 7ffaca662740 1 --2- 172.21.15.123:0/758032478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3f1c70 0x55ffce3f9d30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffab009c240 tx=0x7ffab0009510 comp rx=0 tx=0).stop 2024-09-19T04:48:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.865+0000 7ffaca662740 1 -- 172.21.15.123:0/758032478 shutdown_connections 2024-09-19T04:48:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.865+0000 7ffaca662740 1 --2- 172.21.15.123:0/758032478 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ffce40cf70 0x55ffce40f3c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.865+0000 7ffaca662740 1 --2- 172.21.15.123:0/758032478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3fce50 0x55ffce3fa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.865+0000 7ffaaaa00640 0 client.25693 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.865+0000 7ffaca662740 1 --2- 172.21.15.123:0/758032478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3f2590 0x55ffce3fa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.865+0000 7ffaca662740 1 --2- 172.21.15.123:0/758032478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3f1c70 0x55ffce3f9d30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.865+0000 7ffaca662740 1 -- 172.21.15.123:0/758032478 >> 172.21.15.123:0/758032478 conn(0x55ffce3ee200 msgr2=0x55ffce3ed380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.865+0000 7ffaca662740 1 -- 172.21.15.123:0/758032478 shutdown_connections 2024-09-19T04:48:16.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.865+0000 7ffaca662740 1 -- 172.21.15.123:0/758032478 wait complete. 2024-09-19T04:48:16.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.865+0000 7ffaca662740 20 client.25693 trim_cache size 0 max 16384 2024-09-19T04:48:16.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.879+0000 7ffaca662740 1 Processor -- start 2024-09-19T04:48:16.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.879+0000 7ffaca662740 1 -- start start 2024-09-19T04:48:16.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.879+0000 7ffaca662740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3f1c70 0x55ffce3f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.879+0000 7ffaca662740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f2590 0x55ffce3ed270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.879+0000 7ffaca662740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3fce50 0x55ffce414c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.879+0000 7ffaca662740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffac01afaf0 con 0x55ffce3f2590 2024-09-19T04:48:16.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.879+0000 7ffaca662740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffac01b1720 con 0x55ffce3f1c70 2024-09-19T04:48:16.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.879+0000 7ffaca662740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffac01b01c0 con 0x55ffce3fce50 2024-09-19T04:48:16.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.880+0000 7ffabfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f2590 0x55ffce3ed270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.880+0000 7ffabfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f2590 0x55ffce3ed270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38346/0 (socket says 172.21.15.123:38346) 2024-09-19T04:48:16.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.880+0000 7ffac5200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3f1c70 0x55ffce3f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.880+0000 7ffabfe00640 1 -- 172.21.15.123:0/466940721 learned_addr learned my addr 172.21.15.123:0/466940721 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:16.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.880+0000 7ffac5c00640 1 --2- 172.21.15.123:0/466940721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3fce50 0x55ffce414c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.880+0000 7ffabfe00640 1 -- 172.21.15.123:0/466940721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3fce50 msgr2=0x55ffce414c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:16.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.880+0000 7ffabfe00640 1 --2- 172.21.15.123:0/466940721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3fce50 0x55ffce414c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.880+0000 7ffabfe00640 1 -- 172.21.15.123:0/466940721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3f1c70 msgr2=0x55ffce3f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:16.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.880+0000 7ffabfe00640 1 --2- 172.21.15.123:0/466940721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3f1c70 0x55ffce3f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.880+0000 7ffabfe00640 1 -- 172.21.15.123:0/466940721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffce3e9770 con 0x55ffce3f2590 2024-09-19T04:48:16.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.880+0000 7ffac5200640 1 --2- 172.21.15.123:0/466940721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3f1c70 0x55ffce3f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:16.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.880+0000 7ffabfe00640 1 --2- 172.21.15.123:0/466940721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f2590 0x55ffce3ed270 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009850 tx=0x7ffaac02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a80e0b17c66644 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:16.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.881+0000 7ffab5600640 1 -- 172.21.15.123:0/466940721 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac035040 con 0x55ffce3f2590 2024-09-19T04:48:16.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.881+0000 7ffab5600640 1 -- 172.21.15.123:0/466940721 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaac04b070 con 0x55ffce3f2590 2024-09-19T04:48:16.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.881+0000 7ffab5600640 1 -- 172.21.15.123:0/466940721 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac031810 con 0x55ffce3f2590 2024-09-19T04:48:16.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.881+0000 7ffaca662740 1 -- 172.21.15.123:0/466940721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f2590 msgr2=0x55ffce3ed270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:16.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.881+0000 7ffaca662740 1 --2- 172.21.15.123:0/466940721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f2590 0x55ffce3ed270 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009850 tx=0x7ffaac02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.881+0000 7ffaca662740 1 -- 172.21.15.123:0/466940721 shutdown_connections 2024-09-19T04:48:16.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.881+0000 7ffaca662740 1 --2- 172.21.15.123:0/466940721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3fce50 0x55ffce414c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.881+0000 7ffaca662740 1 --2- 172.21.15.123:0/466940721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f2590 0x55ffce3ed270 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.881+0000 7ffaca662740 1 --2- 172.21.15.123:0/466940721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3f1c70 0x55ffce3f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.881+0000 7ffaca662740 1 -- 172.21.15.123:0/466940721 >> 172.21.15.123:0/466940721 conn(0x55ffce3ee200 msgr2=0x55ffce3ecb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:16.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.881+0000 7ffaca662740 1 -- 172.21.15.123:0/466940721 shutdown_connections 2024-09-19T04:48:16.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.882+0000 7ffaca662740 1 -- 172.21.15.123:0/466940721 wait complete. 2024-09-19T04:48:16.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.882+0000 7ffaca662740 1 -- start start 2024-09-19T04:48:16.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.882+0000 7ffaca662740 1 Processor -- start 2024-09-19T04:48:16.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.883+0000 7ffaca662740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f1c70 0x55ffce400ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.883+0000 7ffaca662740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3f2590 0x55ffce401010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.883+0000 7ffaca662740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3fce50 0x55ffce401550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.883+0000 7ffaca662740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffac01b0f90 con 0x55ffce3f1c70 2024-09-19T04:48:16.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.883+0000 7ffaca662740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffac01b08c0 con 0x55ffce3f2590 2024-09-19T04:48:16.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.883+0000 7ffaca662740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffac01adad0 con 0x55ffce3fce50 2024-09-19T04:48:16.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.883+0000 7ffaca662740 10 client.0 _ll_register_callbacks cb 0x7ffab00d37a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:16.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.883+0000 7ffac5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f1c70 0x55ffce400ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.883+0000 7ffac5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f1c70 0x55ffce400ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38356/0 (socket says 172.21.15.123:38356) 2024-09-19T04:48:16.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.883+0000 7ffac5200640 1 -- 172.21.15.123:0/866626025 learned_addr learned my addr 172.21.15.123:0/866626025 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:16.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.883+0000 7ffac5c00640 1 --2- 172.21.15.123:0/866626025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3fce50 0x55ffce401550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.883+0000 7ffabfe00640 1 --2- 172.21.15.123:0/866626025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3f2590 0x55ffce401010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.883+0000 7ffac5200640 1 -- 172.21.15.123:0/866626025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3fce50 msgr2=0x55ffce401550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:16.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.883+0000 7ffac5200640 1 --2- 172.21.15.123:0/866626025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3fce50 0x55ffce401550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.883+0000 7ffac5200640 1 -- 172.21.15.123:0/866626025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3f2590 msgr2=0x55ffce401010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.883+0000 7ffac5200640 1 --2- 172.21.15.123:0/866626025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3f2590 0x55ffce401010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.883+0000 7ffac5200640 1 -- 172.21.15.123:0/866626025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffce476c60 con 0x55ffce3f1c70 2024-09-19T04:48:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.884+0000 7ffac5200640 1 --2- 172.21.15.123:0/866626025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f1c70 0x55ffce400ad0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ffab009bfb0 tx=0x7ffab0003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c8e3d06ca6484a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.884+0000 7ffaaaa00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:48:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.884+0000 7ffaaaa00640 1 -- 172.21.15.123:0/866626025 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab009bdf0 con 0x55ffce3f1c70 2024-09-19T04:48:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.884+0000 7ffaaaa00640 1 -- 172.21.15.123:0/866626025 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7ffab009b700 con 0x55ffce3f1c70 2024-09-19T04:48:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.884+0000 7ffaca662740 1 -- 172.21.15.123:0/866626025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaac04b070 con 0x55ffce3f1c70 2024-09-19T04:48:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.884+0000 7ffaaaa00640 1 -- 172.21.15.123:0/866626025 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffab009c9b0 con 0x55ffce3f1c70 2024-09-19T04:48:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.884+0000 7ffaca662740 10 client.15651 fetch_fsmap learned FSMap version 30 2024-09-19T04:48:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.884+0000 7ffaca662740 1 -- 172.21.15.123:0/866626025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ffce477e80 con 0x55ffce3f1c70 2024-09-19T04:48:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.884+0000 7ffaaaa00640 1 -- 172.21.15.123:0/866626025 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaac04b070 con 0x55ffce3f1c70 2024-09-19T04:48:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.885+0000 7ffaaaa00640 1 -- 172.21.15.123:0/866626025 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8382+0+0 (secure 0 0 0) 0x7ffab0004000 con 0x55ffce3f1c70 2024-09-19T04:48:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.885+0000 7ffaca662740 10 client.15651 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:48:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.885+0000 7ffaca662740 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:48:16.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.885+0000 7ffaca662740 20 client.15651 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:16.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.885+0000 7ffaca662740 1 --2- 172.21.15.123:0/866626025 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ffce4784a0 0x55ffce47a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:16.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.885+0000 7ffaca662740 4 client.15651 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:48:16.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.885+0000 7ffaca662740 1 -- 172.21.15.123:0/866626025 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffce35fe60 con 0x55ffce4784a0 2024-09-19T04:48:16.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.885+0000 7ffabfe00640 1 --2- 172.21.15.123:0/866626025 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ffce4784a0 0x55ffce47a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:16.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.886+0000 7ffabfe00640 1 --2- 172.21.15.123:0/866626025 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ffce4784a0 0x55ffce47a8c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bab561aac9712d7 server_cookie=99d27f1fade88894 in_seq=0 out_seq=0 2024-09-19T04:48:16.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.886+0000 7ffaaaa00640 10 client.15651 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.886+0000 7ffaaaa00640 1 -- 172.21.15.123:0/866626025 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffce35fe60 con 0x55ffce4784a0 2024-09-19T04:48:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.886+0000 7ffaaaa00640 10 client.15651 handle_command_reply: tid=0 2024-09-19T04:48:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.886+0000 7ffaca662740 1 client.15651 shutdown 2024-09-19T04:48:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.886+0000 7ffaca662740 1 -- 172.21.15.123:0/866626025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f1c70 msgr2=0x55ffce400ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.886+0000 7ffaca662740 1 --2- 172.21.15.123:0/866626025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f1c70 0x55ffce400ad0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ffab009bfb0 tx=0x7ffab0003f30 comp rx=0 tx=0).stop 2024-09-19T04:48:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.886+0000 7ffaca662740 1 -- 172.21.15.123:0/866626025 shutdown_connections 2024-09-19T04:48:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.886+0000 7ffaca662740 1 --2- 172.21.15.123:0/866626025 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ffce4784a0 0x55ffce47a8c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.886+0000 7ffaca662740 1 --2- 172.21.15.123:0/866626025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ffce3fce50 0x55ffce401550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.886+0000 7ffaca662740 1 --2- 172.21.15.123:0/866626025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffce3f2590 0x55ffce401010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.886+0000 7ffaca662740 1 --2- 172.21.15.123:0/866626025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffce3f1c70 0x55ffce400ad0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.886+0000 7ffaaaa00640 0 client.15651 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:16.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.886+0000 7ffaca662740 1 -- 172.21.15.123:0/866626025 >> 172.21.15.123:0/866626025 conn(0x55ffce3ee200 msgr2=0x55ffce3ed380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:16.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.886+0000 7ffaca662740 1 -- 172.21.15.123:0/866626025 shutdown_connections 2024-09-19T04:48:16.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.886+0000 7ffaca662740 1 -- 172.21.15.123:0/866626025 wait complete. 2024-09-19T04:48:16.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.887+0000 7ffaca662740 20 client.15651 trim_cache size 0 max 16384 2024-09-19T04:48:16.893 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:48:16.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.887+0000 7ffac7c00640 1 -- 172.21.15.123:0/1222589488 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffaa40a33b0 msgr2=0x7ffaa40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:16.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.887+0000 7ffac7c00640 1 --2- 172.21.15.123:0/1222589488 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffaa40a33b0 0x7ffaa40a5870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffab801c000 tx=0x7ffab8019950 comp rx=0 tx=0).stop 2024-09-19T04:48:16.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.887+0000 7ffac7c00640 1 -- 172.21.15.123:0/1222589488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffac0101de0 msgr2=0x7ffac019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:16.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.887+0000 7ffac7c00640 1 --2- 172.21.15.123:0/1222589488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffac0101de0 0x7ffac019a630 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffab000ea60 tx=0x7ffab000ef20 comp rx=0 tx=0).stop 2024-09-19T04:48:16.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.887+0000 7ffac7c00640 1 -- 172.21.15.123:0/1222589488 shutdown_connections 2024-09-19T04:48:16.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.887+0000 7ffac7c00640 1 --2- 172.21.15.123:0/1222589488 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffaa40a33b0 0x7ffaa40a5870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.887+0000 7ffac7c00640 1 --2- 172.21.15.123:0/1222589488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffac0106f10 0x7ffac019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.887+0000 7ffac7c00640 1 --2- 172.21.15.123:0/1222589488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffac0102790 0x7ffac019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.887+0000 7ffac7c00640 1 --2- 172.21.15.123:0/1222589488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffac0101de0 0x7ffac019a630 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:16.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.887+0000 7ffac7c00640 1 -- 172.21.15.123:0/1222589488 >> 172.21.15.123:0/1222589488 conn(0x7ffac00fd8c0 msgr2=0x7ffac00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:16.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.888+0000 7ffac7c00640 1 -- 172.21.15.123:0/1222589488 shutdown_connections 2024-09-19T04:48:16.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:16.888+0000 7ffac7c00640 1 -- 172.21.15.123:0/1222589488 wait complete. 2024-09-19T04:48:17.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:17 smithi183 ceph-mon[35333]: pgmap v50: 97 pgs: 97 active+clean; 15 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-09-19T04:48:17.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:17 smithi123 ceph-mon[30511]: pgmap v50: 97 pgs: 97 active+clean; 15 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-09-19T04:48:17.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:17 smithi193 ceph-mon[35592]: pgmap v50: 97 pgs: 97 active+clean; 15 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-09-19T04:48:18.690 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:48:18.690 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:48:19.087 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:48:19.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.613+0000 7f16c7e00640 1 -- 172.21.15.123:0/17262185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16c01049a0 msgr2=0x7f16c01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:19.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.613+0000 7f16c7e00640 1 --2- 172.21.15.123:0/17262185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16c01049a0 0x7f16c01089f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009860 tx=0x7f16ac02a340 comp rx=0 tx=0).stop 2024-09-19T04:48:19.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.613+0000 7f16c7e00640 1 -- 172.21.15.123:0/17262185 shutdown_connections 2024-09-19T04:48:19.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.613+0000 7f16c7e00640 1 --2- 172.21.15.123:0/17262185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16c0109120 0x7f16c0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.613+0000 7f16c7e00640 1 --2- 172.21.15.123:0/17262185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16c01049a0 0x7f16c01089f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.613+0000 7f16c7e00640 1 --2- 172.21.15.123:0/17262185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c0103ff0 0x7f16c01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.614+0000 7f16c7e00640 1 -- 172.21.15.123:0/17262185 >> 172.21.15.123:0/17262185 conn(0x7f16c00ffa90 msgr2=0x7f16c0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:19.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.614+0000 7f16c7e00640 1 -- 172.21.15.123:0/17262185 shutdown_connections 2024-09-19T04:48:19.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.614+0000 7f16c7e00640 1 -- 172.21.15.123:0/17262185 wait complete. 2024-09-19T04:48:19.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.615+0000 7f16c7e00640 1 Processor -- start 2024-09-19T04:48:19.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.615+0000 7f16c7e00640 1 -- start start 2024-09-19T04:48:19.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.615+0000 7f16c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16c0103ff0 0x7f16c019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.615+0000 7f16c7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16c01049a0 0x7f16c019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.615+0000 7f16c7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c0109120 0x7f16c019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.616+0000 7f16c7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f16c0114140 con 0x7f16c0109120 2024-09-19T04:48:19.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.616+0000 7f16c7e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f16c0113fc0 con 0x7f16c0103ff0 2024-09-19T04:48:19.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.616+0000 7f16c7e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16c01142c0 con 0x7f16c01049a0 2024-09-19T04:48:19.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.616+0000 7f16c5400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16c0103ff0 0x7f16c019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.616+0000 7f16c4a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16c01049a0 0x7f16c019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.616+0000 7f16c5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c0109120 0x7f16c019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.616+0000 7f16c5400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16c0103ff0 0x7f16c019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:40008/0 (socket says 172.21.15.123:40008) 2024-09-19T04:48:19.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.616+0000 7f16c4a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16c01049a0 0x7f16c019abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:38002/0 (socket says 172.21.15.123:38002) 2024-09-19T04:48:19.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.616+0000 7f16c5400640 1 -- 172.21.15.123:0/2224462526 learned_addr learned my addr 172.21.15.123:0/2224462526 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:19.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.616+0000 7f16c4a00640 1 -- 172.21.15.123:0/2224462526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16c0103ff0 msgr2=0x7f16c019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:19.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.616+0000 7f16c4a00640 1 --2- 172.21.15.123:0/2224462526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16c0103ff0 0x7f16c019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.616+0000 7f16c4a00640 1 -- 172.21.15.123:0/2224462526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c0109120 msgr2=0x7f16c019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:19.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.616+0000 7f16c4a00640 1 --2- 172.21.15.123:0/2224462526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c0109120 0x7f16c019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.616+0000 7f16c4a00640 1 -- 172.21.15.123:0/2224462526 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c019f650 con 0x7f16c01049a0 2024-09-19T04:48:19.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.617+0000 7f16c4a00640 1 --2- 172.21.15.123:0/2224462526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16c01049a0 0x7f16c019abe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f16ac02a860 tx=0x7f16ac033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:19.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.617+0000 7f16be000640 1 -- 172.21.15.123:0/2224462526 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac037040 con 0x7f16c01049a0 2024-09-19T04:48:19.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.617+0000 7f16c7e00640 1 -- 172.21.15.123:0/2224462526 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c019f8e0 con 0x7f16c01049a0 2024-09-19T04:48:19.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.617+0000 7f16be000640 1 -- 172.21.15.123:0/2224462526 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16ac030550 con 0x7f16c01049a0 2024-09-19T04:48:19.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.617+0000 7f16be000640 1 -- 172.21.15.123:0/2224462526 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac003a30 con 0x7f16c01049a0 2024-09-19T04:48:19.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.617+0000 7f16c7e00640 1 -- 172.21.15.123:0/2224462526 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c01a7220 con 0x7f16c01049a0 2024-09-19T04:48:19.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.620+0000 7f16be000640 1 -- 172.21.15.123:0/2224462526 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f16ac042020 con 0x7f16c01049a0 2024-09-19T04:48:19.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.620+0000 7f16ca7c7740 1 Processor -- start 2024-09-19T04:48:19.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.620+0000 7f16ca7c7740 1 -- start start 2024-09-19T04:48:19.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.621+0000 7f16ca7c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561333207ac0 0x561333207ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.621+0000 7f16ca7c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613332083e0 0x561333212760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.621+0000 7f16ca7c7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 0x561333215090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.621+0000 7f16ca7c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56133316e290 con 0x561333207ac0 2024-09-19T04:48:19.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.621+0000 7f16ca7c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561333115b40 con 0x5613332083e0 2024-09-19T04:48:19.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.621+0000 7f16ca7c7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56133316a250 con 0x561333212ca0 2024-09-19T04:48:19.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.621+0000 7f16c5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561333207ac0 0x561333207ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.621+0000 7f16c5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561333207ac0 0x561333207ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38372/0 (socket says 172.21.15.123:38372) 2024-09-19T04:48:19.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.621+0000 7f16c5e00640 1 -- 172.21.15.123:0/1796494845 learned_addr learned my addr 172.21.15.123:0/1796494845 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:19.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.621+0000 7f16c4a00640 1 --2- 172.21.15.123:0/1796494845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 0x561333215090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.621+0000 7f16c5400640 1 --2- 172.21.15.123:0/1796494845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613332083e0 0x561333212760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.622+0000 7f16be000640 1 --2- 172.21.15.123:0/2224462526 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f16a80a33b0 0x7f16a80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.622+0000 7f16c5400640 1 -- 172.21.15.123:0/1796494845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 msgr2=0x561333215090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:19.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.622+0000 7f16c5400640 1 --2- 172.21.15.123:0/1796494845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 0x561333215090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.622+0000 7f16c5400640 1 -- 172.21.15.123:0/1796494845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561333207ac0 msgr2=0x561333207ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:19.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.622+0000 7f16c5400640 1 --2- 172.21.15.123:0/1796494845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561333207ac0 0x561333207ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.622+0000 7f16c5400640 1 -- 172.21.15.123:0/1796494845 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561332edf6a0 con 0x5613332083e0 2024-09-19T04:48:19.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.622+0000 7f16c4a00640 1 --2- 172.21.15.123:0/1796494845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 0x561333215090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:19.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.622+0000 7f16be000640 1 -- 172.21.15.123:0/2224462526 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f16ac0ef290 con 0x7f16c01049a0 2024-09-19T04:48:19.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.622+0000 7f16c5e00640 1 --2- 172.21.15.123:0/2224462526 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f16a80a33b0 0x7f16a80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.622+0000 7f16c5400640 1 --2- 172.21.15.123:0/1796494845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613332083e0 0x561333212760 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009ca0 tx=0x7f16b4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a914c1ca7f344b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:19.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.623+0000 7f16b2600640 1 -- 172.21.15.123:0/1796494845 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4004230 con 0x5613332083e0 2024-09-19T04:48:19.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.623+0000 7f16b2600640 1 -- 172.21.15.123:0/1796494845 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16b40043d0 con 0x5613332083e0 2024-09-19T04:48:19.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.623+0000 7f16c5e00640 1 --2- 172.21.15.123:0/2224462526 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f16a80a33b0 0x7f16a80a5870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16b8005f40 tx=0x7f16b8004a40 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:19.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.623+0000 7f16b2600640 1 -- 172.21.15.123:0/1796494845 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4019840 con 0x5613332083e0 2024-09-19T04:48:19.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.623+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1796494845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613332083e0 msgr2=0x561333212760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:19.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.623+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/1796494845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613332083e0 0x561333212760 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009ca0 tx=0x7f16b4011810 comp rx=0 tx=0).stop 2024-09-19T04:48:19.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.624+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1796494845 shutdown_connections 2024-09-19T04:48:19.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.624+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/1796494845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 0x561333215090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.624+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/1796494845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613332083e0 0x561333212760 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.624+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/1796494845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561333207ac0 0x561333207ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.624+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1796494845 >> 172.21.15.123:0/1796494845 conn(0x561333204050 msgr2=0x561333204430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:19.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.624+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1796494845 shutdown_connections 2024-09-19T04:48:19.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.624+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1796494845 wait complete. 2024-09-19T04:48:19.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.624+0000 7f16ca7c7740 1 -- start start 2024-09-19T04:48:19.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.625+0000 7f16ca7c7740 1 Processor -- start 2024-09-19T04:48:19.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.625+0000 7f16ca7c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561333207ac0 0x5613332124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.625+0000 7f16ca7c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613332083e0 0x56133320f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.625+0000 7f16ca7c7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 0x56133320fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.625+0000 7f16ca7c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561333215910 con 0x5613332083e0 2024-09-19T04:48:19.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.625+0000 7f16ca7c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561333128fa0 con 0x561333207ac0 2024-09-19T04:48:19.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.625+0000 7f16ca7c7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561333215a90 con 0x561333212ca0 2024-09-19T04:48:19.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.625+0000 7f16ca7c7740 10 client.0 _ll_register_callbacks cb 0x561332e7cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:19.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.625+0000 7f16c5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 0x56133320fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.625+0000 7f16c4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613332083e0 0x56133320f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.625+0000 7f16c5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 0x56133320fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:38026/0 (socket says 172.21.15.123:38026) 2024-09-19T04:48:19.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.625+0000 7f16c5e00640 1 -- 172.21.15.123:0/1860328010 learned_addr learned my addr 172.21.15.123:0/1860328010 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:19.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.625+0000 7f16c5400640 1 --2- 172.21.15.123:0/1860328010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561333207ac0 0x5613332124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.626+0000 7f16c5e00640 1 -- 172.21.15.123:0/1860328010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561333207ac0 msgr2=0x5613332124c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:19.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.626+0000 7f16c5e00640 1 --2- 172.21.15.123:0/1860328010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561333207ac0 0x5613332124c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.626+0000 7f16c5e00640 1 -- 172.21.15.123:0/1860328010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613332083e0 msgr2=0x56133320f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:19.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.626+0000 7f16c5e00640 1 --2- 172.21.15.123:0/1860328010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613332083e0 0x56133320f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.626+0000 7f16c5e00640 1 -- 172.21.15.123:0/1860328010 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561333210490 con 0x561333212ca0 2024-09-19T04:48:19.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.626+0000 7f16c4a00640 1 --2- 172.21.15.123:0/1860328010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613332083e0 0x56133320f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:19.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.626+0000 7f16c5e00640 1 --2- 172.21.15.123:0/1860328010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 0x56133320fcb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f16b80109c0 tx=0x7f16b8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c81320159f09e6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:19.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.626+0000 7f16a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:48:19.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.626+0000 7f16a6a00640 1 -- 172.21.15.123:0/1860328010 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b802b060 con 0x561333212ca0 2024-09-19T04:48:19.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.626+0000 7f16a6a00640 1 -- 172.21.15.123:0/1860328010 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f16b8018070 con 0x561333212ca0 2024-09-19T04:48:19.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.626+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1860328010 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16b40043d0 con 0x561333212ca0 2024-09-19T04:48:19.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.627+0000 7f16a6a00640 1 -- 172.21.15.123:0/1860328010 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f16b801e770 con 0x561333212ca0 2024-09-19T04:48:19.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.627+0000 7f16ca7c7740 10 client.25711 fetch_fsmap learned FSMap version 30 2024-09-19T04:48:19.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.627+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1860328010 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561333221770 con 0x561333212ca0 2024-09-19T04:48:19.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.627+0000 7f16a6a00640 1 -- 172.21.15.123:0/1860328010 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16b40043d0 con 0x561333212ca0 2024-09-19T04:48:19.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.627+0000 7f16a6a00640 1 -- 172.21.15.123:0/1860328010 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8382+0+0 (secure 0 0 0) 0x7f16b800f040 con 0x561333212ca0 2024-09-19T04:48:19.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.627+0000 7f16ca7c7740 10 client.25711 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:48:19.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.627+0000 7f16ca7c7740 10 client.25711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:48:19.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.627+0000 7f16ca7c7740 20 client.25711 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:19.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.627+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/1860328010 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x561333222c80 0x561333225100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.627+0000 7f16ca7c7740 4 client.25711 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:48:19.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.627+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1860328010 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56133317b800 con 0x561333222c80 2024-09-19T04:48:19.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.628+0000 7f16c5400640 1 --2- 172.21.15.123:0/1860328010 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x561333222c80 0x561333225100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.628+0000 7f16c5400640 1 --2- 172.21.15.123:0/1860328010 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x561333222c80 0x561333225100 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f41fd9265d57452d server_cookie=7774b918c2770f01 in_seq=0 out_seq=0 2024-09-19T04:48:19.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.628+0000 7f16a6a00640 10 client.25711 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:19.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.629+0000 7f16a6a00640 1 -- 172.21.15.123:0/1860328010 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56133317b800 con 0x561333222c80 2024-09-19T04:48:19.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.629+0000 7f16a6a00640 10 client.25711 handle_command_reply: tid=0 2024-09-19T04:48:19.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.629+0000 7f16ca7c7740 1 client.25711 shutdown 2024-09-19T04:48:19.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.629+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1860328010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 msgr2=0x56133320fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:19.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.629+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/1860328010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 0x56133320fcb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f16b80109c0 tx=0x7f16b8004ab0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.629+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1860328010 shutdown_connections 2024-09-19T04:48:19.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.629+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/1860328010 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x561333222c80 0x561333225100 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.629+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/1860328010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 0x56133320fcb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.629+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/1860328010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613332083e0 0x56133320f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.629+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/1860328010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561333207ac0 0x5613332124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.629+0000 7f16a6a00640 0 client.25711 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:19.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.629+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1860328010 >> 172.21.15.123:0/1860328010 conn(0x561333204050 msgr2=0x561333209d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:19.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.629+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1860328010 shutdown_connections 2024-09-19T04:48:19.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.629+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1860328010 wait complete. 2024-09-19T04:48:19.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.630+0000 7f16ca7c7740 20 client.25711 trim_cache size 0 max 16384 2024-09-19T04:48:19.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.649+0000 7f16ca7c7740 1 Processor -- start 2024-09-19T04:48:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.649+0000 7f16ca7c7740 1 -- start start 2024-09-19T04:48:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.650+0000 7f16ca7c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561333207ac0 0x5613332116e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.650+0000 7f16ca7c7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613332083e0 0x56133322a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.650+0000 7f16ca7c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561333212ca0 0x56133322af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.650+0000 7f16ca7c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f16c01afa70 con 0x561333212ca0 2024-09-19T04:48:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.650+0000 7f16ca7c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f16c01b16a0 con 0x561333207ac0 2024-09-19T04:48:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.650+0000 7f16ca7c7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16c01b0140 con 0x5613332083e0 2024-09-19T04:48:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.650+0000 7f16c5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561333212ca0 0x56133322af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.650+0000 7f16c5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561333212ca0 0x56133322af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38388/0 (socket says 172.21.15.123:38388) 2024-09-19T04:48:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.650+0000 7f16c5e00640 1 -- 172.21.15.123:0/1801325729 learned_addr learned my addr 172.21.15.123:0/1801325729 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.650+0000 7f16c5400640 1 --2- 172.21.15.123:0/1801325729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561333207ac0 0x5613332116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.650+0000 7f16c4a00640 1 --2- 172.21.15.123:0/1801325729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613332083e0 0x56133322a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.650+0000 7f16c5e00640 1 -- 172.21.15.123:0/1801325729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613332083e0 msgr2=0x56133322a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:19.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.650+0000 7f16c5e00640 1 --2- 172.21.15.123:0/1801325729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613332083e0 0x56133322a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.650+0000 7f16c5e00640 1 -- 172.21.15.123:0/1801325729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561333207ac0 msgr2=0x5613332116e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:19.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.650+0000 7f16c5e00640 1 --2- 172.21.15.123:0/1801325729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561333207ac0 0x5613332116e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.650+0000 7f16c5e00640 1 -- 172.21.15.123:0/1801325729 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561333204c00 con 0x561333212ca0 2024-09-19T04:48:19.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.650+0000 7f16c5400640 1 --2- 172.21.15.123:0/1801325729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561333207ac0 0x5613332116e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:48:19.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.651+0000 7f16c5e00640 1 --2- 172.21.15.123:0/1801325729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561333212ca0 0x56133322af10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009f70 tx=0x7f16b8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdc44d664e659cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:19.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.651+0000 7f16b1c00640 1 -- 172.21.15.123:0/1801325729 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b801e240 con 0x561333212ca0 2024-09-19T04:48:19.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.651+0000 7f16b1c00640 1 -- 172.21.15.123:0/1801325729 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16b801edf0 con 0x561333212ca0 2024-09-19T04:48:19.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.651+0000 7f16b1c00640 1 -- 172.21.15.123:0/1801325729 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b801c800 con 0x561333212ca0 2024-09-19T04:48:19.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.652+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1801325729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561333212ca0 msgr2=0x56133322af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:19.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.652+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/1801325729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561333212ca0 0x56133322af10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009f70 tx=0x7f16b8005c90 comp rx=0 tx=0).stop 2024-09-19T04:48:19.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.652+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1801325729 shutdown_connections 2024-09-19T04:48:19.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.652+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/1801325729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561333212ca0 0x56133322af10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.652+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/1801325729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613332083e0 0x56133322a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.652+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/1801325729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561333207ac0 0x5613332116e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.652+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1801325729 >> 172.21.15.123:0/1801325729 conn(0x561333204050 msgr2=0x561333210fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:19.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.652+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1801325729 shutdown_connections 2024-09-19T04:48:19.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.652+0000 7f16ca7c7740 1 -- 172.21.15.123:0/1801325729 wait complete. 2024-09-19T04:48:19.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.652+0000 7f16ca7c7740 1 -- start start 2024-09-19T04:48:19.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.652+0000 7f16ca7c7740 1 Processor -- start 2024-09-19T04:48:19.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.653+0000 7f16ca7c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561333207ac0 0x561333218ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.653+0000 7f16ca7c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613332083e0 0x561333219020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.653+0000 7f16ca7c7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 0x561333219560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.653+0000 7f16ca7c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f16c01b0f10 con 0x5613332083e0 2024-09-19T04:48:19.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.653+0000 7f16ca7c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f16c01b0840 con 0x561333207ac0 2024-09-19T04:48:19.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.653+0000 7f16ca7c7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16c01ada50 con 0x561333212ca0 2024-09-19T04:48:19.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.653+0000 7f16ca7c7740 10 client.0 _ll_register_callbacks cb 0x7f16b8009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:19.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.653+0000 7f16c4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613332083e0 0x561333219020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.653+0000 7f16c4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613332083e0 0x561333219020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38392/0 (socket says 172.21.15.123:38392) 2024-09-19T04:48:19.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.653+0000 7f16c4a00640 1 -- 172.21.15.123:0/2230806474 learned_addr learned my addr 172.21.15.123:0/2230806474 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:19.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.653+0000 7f16c5e00640 1 --2- 172.21.15.123:0/2230806474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 0x561333219560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.653+0000 7f16c5400640 1 --2- 172.21.15.123:0/2230806474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561333207ac0 0x561333218ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.654+0000 7f16c4a00640 1 -- 172.21.15.123:0/2230806474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 msgr2=0x561333219560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:19.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.654+0000 7f16c4a00640 1 --2- 172.21.15.123:0/2230806474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 0x561333219560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.654+0000 7f16c4a00640 1 -- 172.21.15.123:0/2230806474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561333207ac0 msgr2=0x561333218ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:19.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.654+0000 7f16c4a00640 1 --2- 172.21.15.123:0/2230806474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561333207ac0 0x561333218ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.654+0000 7f16c4a00640 1 -- 172.21.15.123:0/2230806474 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56133328ca30 con 0x5613332083e0 2024-09-19T04:48:19.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.654+0000 7f16c4a00640 1 --2- 172.21.15.123:0/2230806474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613332083e0 0x561333219020 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0b7fd0 tx=0x7f16ac032840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f7c1e069d6a7ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.654+0000 7f16a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:48:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.654+0000 7f16a6a00640 1 -- 172.21.15.123:0/2230806474 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac0b78f0 con 0x5613332083e0 2024-09-19T04:48:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.654+0000 7f16a6a00640 1 -- 172.21.15.123:0/2230806474 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f16ac003dd0 con 0x5613332083e0 2024-09-19T04:48:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.654+0000 7f16ca7c7740 1 -- 172.21.15.123:0/2230806474 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16b801edf0 con 0x5613332083e0 2024-09-19T04:48:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.655+0000 7f16a6a00640 1 -- 172.21.15.123:0/2230806474 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f16ac0ece40 con 0x5613332083e0 2024-09-19T04:48:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.655+0000 7f16ca7c7740 10 client.15669 fetch_fsmap learned FSMap version 30 2024-09-19T04:48:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.655+0000 7f16ca7c7740 1 -- 172.21.15.123:0/2230806474 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56133328ddb0 con 0x5613332083e0 2024-09-19T04:48:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.655+0000 7f16a6a00640 1 -- 172.21.15.123:0/2230806474 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16b801edf0 con 0x5613332083e0 2024-09-19T04:48:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.655+0000 7f16a6a00640 1 -- 172.21.15.123:0/2230806474 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8382+0+0 (secure 0 0 0) 0x7f16ac03a070 con 0x5613332083e0 2024-09-19T04:48:19.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.655+0000 7f16ca7c7740 10 client.15669 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:48:19.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.655+0000 7f16ca7c7740 10 client.15669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:48:19.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.655+0000 7f16ca7c7740 20 client.15669 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:19.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.655+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/2230806474 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56133328e5b0 0x5613332909d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:19.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.655+0000 7f16ca7c7740 4 client.15669 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:48:19.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.655+0000 7f16ca7c7740 1 -- 172.21.15.123:0/2230806474 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561333175d10 con 0x56133328e5b0 2024-09-19T04:48:19.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.655+0000 7f16c5400640 1 --2- 172.21.15.123:0/2230806474 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56133328e5b0 0x5613332909d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:19.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.656+0000 7f16c5400640 1 --2- 172.21.15.123:0/2230806474 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56133328e5b0 0x5613332909d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d96c98096b456d4 server_cookie=eb3ad74d9b32a853 in_seq=0 out_seq=0 2024-09-19T04:48:19.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.656+0000 7f16a6a00640 10 client.15669 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:19.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.657+0000 7f16a6a00640 1 -- 172.21.15.123:0/2230806474 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561333175d10 con 0x56133328e5b0 2024-09-19T04:48:19.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.657+0000 7f16a6a00640 10 client.15669 handle_command_reply: tid=0 2024-09-19T04:48:19.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.657+0000 7f16ca7c7740 1 client.15669 shutdown 2024-09-19T04:48:19.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.657+0000 7f16ca7c7740 1 -- 172.21.15.123:0/2230806474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613332083e0 msgr2=0x561333219020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:19.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.657+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/2230806474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613332083e0 0x561333219020 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0b7fd0 tx=0x7f16ac032840 comp rx=0 tx=0).stop 2024-09-19T04:48:19.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.657+0000 7f16ca7c7740 1 -- 172.21.15.123:0/2230806474 shutdown_connections 2024-09-19T04:48:19.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.657+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/2230806474 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56133328e5b0 0x5613332909d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.657+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/2230806474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561333212ca0 0x561333219560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.657+0000 7f16a6a00640 0 client.15669 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:19.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.657+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/2230806474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613332083e0 0x561333219020 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.657+0000 7f16ca7c7740 1 --2- 172.21.15.123:0/2230806474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561333207ac0 0x561333218ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.657+0000 7f16ca7c7740 1 -- 172.21.15.123:0/2230806474 >> 172.21.15.123:0/2230806474 conn(0x561333204050 msgr2=0x561333210fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:19.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.658+0000 7f16ca7c7740 1 -- 172.21.15.123:0/2230806474 shutdown_connections 2024-09-19T04:48:19.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.658+0000 7f16ca7c7740 1 -- 172.21.15.123:0/2230806474 wait complete. 2024-09-19T04:48:19.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.658+0000 7f16ca7c7740 20 client.15669 trim_cache size 0 max 16384 2024-09-19T04:48:19.664 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:48:19.664 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-09-19T04:48:19.664 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "804d4dda-f6c6-4a99-88d8-fee4a2dec0b8", 2024-09-19T04:48:19.664 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-09-19T04:48:19.664 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:48:19.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.658+0000 7f16c7e00640 1 -- 172.21.15.123:0/2224462526 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f16a80a33b0 msgr2=0x7f16a80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:19.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.658+0000 7f16c7e00640 1 --2- 172.21.15.123:0/2224462526 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f16a80a33b0 0x7f16a80a5870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16b8005f40 tx=0x7f16b8004a40 comp rx=0 tx=0).stop 2024-09-19T04:48:19.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.658+0000 7f16c7e00640 1 -- 172.21.15.123:0/2224462526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16c01049a0 msgr2=0x7f16c019abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:19.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.658+0000 7f16c7e00640 1 --2- 172.21.15.123:0/2224462526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16c01049a0 0x7f16c019abe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f16ac02a860 tx=0x7f16ac033d90 comp rx=0 tx=0).stop 2024-09-19T04:48:19.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.659+0000 7f16c7e00640 1 -- 172.21.15.123:0/2224462526 shutdown_connections 2024-09-19T04:48:19.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.659+0000 7f16c7e00640 1 --2- 172.21.15.123:0/2224462526 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f16a80a33b0 0x7f16a80a5870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.659+0000 7f16c7e00640 1 --2- 172.21.15.123:0/2224462526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c0109120 0x7f16c019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.659+0000 7f16c7e00640 1 --2- 172.21.15.123:0/2224462526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16c01049a0 0x7f16c019abe0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.659+0000 7f16c7e00640 1 --2- 172.21.15.123:0/2224462526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16c0103ff0 0x7f16c019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:19.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.659+0000 7f16c7e00640 1 -- 172.21.15.123:0/2224462526 >> 172.21.15.123:0/2224462526 conn(0x7f16c00ffa90 msgr2=0x7f16c0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:19.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.659+0000 7f16c7e00640 1 -- 172.21.15.123:0/2224462526 shutdown_connections 2024-09-19T04:48:19.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:19.659+0000 7f16c7e00640 1 -- 172.21.15.123:0/2224462526 wait complete. 2024-09-19T04:48:19.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:19 smithi183 ceph-mon[35333]: pgmap v51: 97 pgs: 97 active+clean; 16 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-09-19T04:48:19.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:19 smithi123 ceph-mon[30511]: pgmap v51: 97 pgs: 97 active+clean; 16 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-09-19T04:48:19.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:19 smithi193 ceph-mon[35592]: pgmap v51: 97 pgs: 97 active+clean; 16 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-09-19T04:48:20.459 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:48:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:20 smithi123 ceph-mon[30511]: scrub queued for path: / 2024-09-19T04:48:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:20 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:20 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:48:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:20 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:48:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:20 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:20.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:20 smithi183 ceph-mon[35333]: scrub queued for path: / 2024-09-19T04:48:20.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:20 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:20.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:20 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:48:20.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:20 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:48:20.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:20 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:20.872 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:48:20.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:20 smithi193 ceph-mon[35592]: scrub queued for path: / 2024-09-19T04:48:20.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:20 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:20.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:20 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:48:20.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:20 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:48:20.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:20 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:21.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.410+0000 7fb4a9600640 1 -- 172.21.15.123:0/631363134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a4106f10 msgr2=0x7fb4a410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:21.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.410+0000 7fb4a9600640 1 --2- 172.21.15.123:0/631363134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a4106f10 0x7fb4a410ea90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009ca0 tx=0x7fb49c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.411+0000 7fb4a9600640 1 -- 172.21.15.123:0/631363134 shutdown_connections 2024-09-19T04:48:21.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.411+0000 7fb4a9600640 1 --2- 172.21.15.123:0/631363134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a4106f10 0x7fb4a410ea90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.411+0000 7fb4a9600640 1 --2- 172.21.15.123:0/631363134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4a4102790 0x7fb4a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.411+0000 7fb4a9600640 1 --2- 172.21.15.123:0/631363134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4a4101de0 0x7fb4a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.411+0000 7fb4a9600640 1 -- 172.21.15.123:0/631363134 >> 172.21.15.123:0/631363134 conn(0x7fb4a40fd8c0 msgr2=0x7fb4a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:21.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.411+0000 7fb4a9600640 1 -- 172.21.15.123:0/631363134 shutdown_connections 2024-09-19T04:48:21.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.411+0000 7fb4a9600640 1 -- 172.21.15.123:0/631363134 wait complete. 2024-09-19T04:48:21.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.412+0000 7fb4a9600640 1 Processor -- start 2024-09-19T04:48:21.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.412+0000 7fb4a9600640 1 -- start start 2024-09-19T04:48:21.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.412+0000 7fb4a9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4a4101de0 0x7fb4a419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.412+0000 7fb4a9600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a4102790 0x7fb4a419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.412+0000 7fb4a9600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4a4106f10 0x7fb4a419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.412+0000 7fb4a9600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a4111fc0 con 0x7fb4a4101de0 2024-09-19T04:48:21.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.412+0000 7fb4a9600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a4111e40 con 0x7fb4a4106f10 2024-09-19T04:48:21.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.412+0000 7fb4a9600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a4112140 con 0x7fb4a4102790 2024-09-19T04:48:21.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.412+0000 7fb4a2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4a4101de0 0x7fb4a419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:21.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.412+0000 7fb49be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a4102790 0x7fb4a419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:21.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.412+0000 7fb4a2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4a4101de0 0x7fb4a419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38412/0 (socket says 172.21.15.123:38412) 2024-09-19T04:48:21.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.412+0000 7fb4a2a00640 1 -- 172.21.15.123:0/3194785735 learned_addr learned my addr 172.21.15.123:0/3194785735 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:21.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.412+0000 7fb49be00640 1 --2- 172.21.15.123:0/3194785735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a4102790 0x7fb4a419ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:38044/0 (socket says 172.21.15.123:38044) 2024-09-19T04:48:21.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.412+0000 7fb4a3400640 1 --2- 172.21.15.123:0/3194785735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4a4106f10 0x7fb4a419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:21.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.413+0000 7fb49be00640 1 -- 172.21.15.123:0/3194785735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4a4106f10 msgr2=0x7fb4a419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:21.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.413+0000 7fb49be00640 1 --2- 172.21.15.123:0/3194785735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4a4106f10 0x7fb4a419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.413+0000 7fb49be00640 1 -- 172.21.15.123:0/3194785735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4a4101de0 msgr2=0x7fb4a419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:21.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.413+0000 7fb49be00640 1 --2- 172.21.15.123:0/3194785735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4a4101de0 0x7fb4a419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.413+0000 7fb49be00640 1 -- 172.21.15.123:0/3194785735 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a419f590 con 0x7fb4a4102790 2024-09-19T04:48:21.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.413+0000 7fb4a2a00640 1 --2- 172.21.15.123:0/3194785735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4a4101de0 0x7fb4a419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:21.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.413+0000 7fb49be00640 1 --2- 172.21.15.123:0/3194785735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a4102790 0x7fb4a419ab20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00e9f0 tx=0x7fb48c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:21.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.413+0000 7fb49b400640 1 -- 172.21.15.123:0/3194785735 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c00cd80 con 0x7fb4a4102790 2024-09-19T04:48:21.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.413+0000 7fb49b400640 1 -- 172.21.15.123:0/3194785735 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb48c0035f0 con 0x7fb4a4102790 2024-09-19T04:48:21.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.413+0000 7fb4a9600640 1 -- 172.21.15.123:0/3194785735 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a419f880 con 0x7fb4a4102790 2024-09-19T04:48:21.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.413+0000 7fb49b400640 1 -- 172.21.15.123:0/3194785735 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c010690 con 0x7fb4a4102790 2024-09-19T04:48:21.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.414+0000 7fb4a9600640 1 -- 172.21.15.123:0/3194785735 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a41a7160 con 0x7fb4a4102790 2024-09-19T04:48:21.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.417+0000 7fb4ac200740 1 Processor -- start 2024-09-19T04:48:21.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.417+0000 7fb4ac200740 1 -- start start 2024-09-19T04:48:21.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.418+0000 7fb4ac200740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41cb90 0x556a4d41cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.418+0000 7fb4ac200740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d41d4b0 0x556a4d427830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.418+0000 7fb4ac200740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d427d70 0x556a4d42a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.418+0000 7fb4ac200740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556a4d3833e0 con 0x556a4d427d70 2024-09-19T04:48:21.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.418+0000 7fb4ac200740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556a4d32ad80 con 0x556a4d41d4b0 2024-09-19T04:48:21.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.418+0000 7fb4ac200740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556a4d37f3a0 con 0x556a4d41cb90 2024-09-19T04:48:21.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.418+0000 7fb4a3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41cb90 0x556a4d41cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:21.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.418+0000 7fb4a3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41cb90 0x556a4d41cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:38046/0 (socket says 172.21.15.123:38046) 2024-09-19T04:48:21.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.418+0000 7fb4a3400640 1 -- 172.21.15.123:0/1767013020 learned_addr learned my addr 172.21.15.123:0/1767013020 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:21.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.418+0000 7fb4a2a00640 1 --2- 172.21.15.123:0/1767013020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d41d4b0 0x556a4d427830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:21.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.418+0000 7fb49b400640 1 -- 172.21.15.123:0/3194785735 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb48c003b30 con 0x7fb4a4102790 2024-09-19T04:48:21.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.418+0000 7fb4a3400640 1 -- 172.21.15.123:0/1767013020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d41d4b0 msgr2=0x556a4d427830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:21.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.418+0000 7fb4a3400640 1 --2- 172.21.15.123:0/1767013020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d41d4b0 0x556a4d427830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.418+0000 7fb4a3400640 1 -- 172.21.15.123:0/1767013020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d427d70 msgr2=0x556a4d42a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:21.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.418+0000 7fb4a3400640 1 --2- 172.21.15.123:0/1767013020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d427d70 0x556a4d42a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.418+0000 7fb4a3400640 1 -- 172.21.15.123:0/1767013020 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a4d349ae0 con 0x556a4d41cb90 2024-09-19T04:48:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.419+0000 7fb4a3400640 1 --2- 172.21.15.123:0/1767013020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41cb90 0x556a4d41cf70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009ca0 tx=0x7fb49c0093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d9713d4bcd3d5b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.419+0000 7fb493400640 1 -- 172.21.15.123:0/1767013020 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c031db0 con 0x556a4d41cb90 2024-09-19T04:48:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.419+0000 7fb493400640 1 -- 172.21.15.123:0/1767013020 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb49c036030 con 0x556a4d41cb90 2024-09-19T04:48:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.419+0000 7fb493400640 1 -- 172.21.15.123:0/1767013020 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c0042d0 con 0x556a4d41cb90 2024-09-19T04:48:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.420+0000 7fb49b400640 1 --2- 172.21.15.123:0/3194785735 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb4880a33b0 0x7fb4880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.420+0000 7fb4ac200740 1 -- 172.21.15.123:0/1767013020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41cb90 msgr2=0x556a4d41cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.420+0000 7fb4ac200740 1 --2- 172.21.15.123:0/1767013020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41cb90 0x556a4d41cf70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009ca0 tx=0x7fb49c0093e0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.420+0000 7fb49b400640 1 -- 172.21.15.123:0/3194785735 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fb48c0cffd0 con 0x7fb4a4102790 2024-09-19T04:48:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.420+0000 7fb4a2a00640 1 --2- 172.21.15.123:0/3194785735 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb4880a33b0 0x7fb4880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.420+0000 7fb4ac200740 1 -- 172.21.15.123:0/1767013020 shutdown_connections 2024-09-19T04:48:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.420+0000 7fb4ac200740 1 --2- 172.21.15.123:0/1767013020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d427d70 0x556a4d42a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.420+0000 7fb4ac200740 1 --2- 172.21.15.123:0/1767013020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d41d4b0 0x556a4d427830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.420+0000 7fb4ac200740 1 --2- 172.21.15.123:0/1767013020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41cb90 0x556a4d41cf70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.420+0000 7fb4ac200740 1 -- 172.21.15.123:0/1767013020 >> 172.21.15.123:0/1767013020 conn(0x556a4d419120 msgr2=0x556a4d419500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.420+0000 7fb4ac200740 1 -- 172.21.15.123:0/1767013020 shutdown_connections 2024-09-19T04:48:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.420+0000 7fb4ac200740 1 -- 172.21.15.123:0/1767013020 wait complete. 2024-09-19T04:48:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.421+0000 7fb4ac200740 1 -- start start 2024-09-19T04:48:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.421+0000 7fb4a2a00640 1 --2- 172.21.15.123:0/3194785735 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb4880a33b0 0x7fb4880a5870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x556a4d419f80 tx=0x7fb494005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.421+0000 7fb4ac200740 1 Processor -- start 2024-09-19T04:48:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.421+0000 7fb4ac200740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d41cb90 0x556a4d424d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.421+0000 7fb4ac200740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d41d4b0 0x556a4d425240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.421+0000 7fb4ac200740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d427d70 0x556a4d425780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.421+0000 7fb4ac200740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556a4d42a860 con 0x556a4d41d4b0 2024-09-19T04:48:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.421+0000 7fb4ac200740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556a4d33e0f0 con 0x556a4d41cb90 2024-09-19T04:48:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.421+0000 7fb4ac200740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556a4d42a9e0 con 0x556a4d427d70 2024-09-19T04:48:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.421+0000 7fb4ac200740 10 client.0 _ll_register_callbacks cb 0x556a4d32f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.422+0000 7fb49be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d41d4b0 0x556a4d425240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.422+0000 7fb49be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d41d4b0 0x556a4d425240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38442/0 (socket says 172.21.15.123:38442) 2024-09-19T04:48:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.422+0000 7fb4a3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d427d70 0x556a4d425780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.422+0000 7fb49be00640 1 -- 172.21.15.123:0/3022290013 learned_addr learned my addr 172.21.15.123:0/3022290013 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.422+0000 7fb49be00640 1 -- 172.21.15.123:0/3022290013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d427d70 msgr2=0x556a4d425780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.422+0000 7fb49be00640 1 --2- 172.21.15.123:0/3022290013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d427d70 0x556a4d425780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.422+0000 7fb49be00640 1 -- 172.21.15.123:0/3022290013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d41cb90 msgr2=0x556a4d424d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.422+0000 7fb49be00640 1 --2- 172.21.15.123:0/3022290013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d41cb90 0x556a4d424d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.422+0000 7fb49be00640 1 -- 172.21.15.123:0/3022290013 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a4d0f43c0 con 0x556a4d41d4b0 2024-09-19T04:48:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.422+0000 7fb4a3400640 1 --2- 172.21.15.123:0/3022290013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d427d70 0x556a4d425780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.422+0000 7fb49be00640 1 --2- 172.21.15.123:0/3022290013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d41d4b0 0x556a4d425240 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00e9c0 tx=0x7fb48c09bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27d6d79474a9c7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.422+0000 7fb487e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:48:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.422+0000 7fb487e00640 1 -- 172.21.15.123:0/3022290013 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c0a4ef0 con 0x556a4d41d4b0 2024-09-19T04:48:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.422+0000 7fb487e00640 1 -- 172.21.15.123:0/3022290013 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fb48c0988f0 con 0x556a4d41d4b0 2024-09-19T04:48:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.422+0000 7fb4ac200740 1 -- 172.21.15.123:0/3022290013 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb49c036030 con 0x556a4d41d4b0 2024-09-19T04:48:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.423+0000 7fb487e00640 1 -- 172.21.15.123:0/3022290013 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb48c0b5280 con 0x556a4d41d4b0 2024-09-19T04:48:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.423+0000 7fb4ac200740 10 client.15678 fetch_fsmap learned FSMap version 30 2024-09-19T04:48:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.423+0000 7fb4ac200740 1 -- 172.21.15.123:0/3022290013 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556a4d4365b0 con 0x556a4d41d4b0 2024-09-19T04:48:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.423+0000 7fb487e00640 1 -- 172.21.15.123:0/3022290013 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb49c036030 con 0x556a4d41d4b0 2024-09-19T04:48:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.423+0000 7fb487e00640 1 -- 172.21.15.123:0/3022290013 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8382+0+0 (secure 0 0 0) 0x7fb48c0bc170 con 0x556a4d41d4b0 2024-09-19T04:48:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.423+0000 7fb4ac200740 10 client.15678 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:48:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.423+0000 7fb4ac200740 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:48:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.423+0000 7fb4ac200740 20 client.15678 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.423+0000 7fb4ac200740 1 --2- 172.21.15.123:0/3022290013 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556a4d437a70 0x556a4d439eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.423+0000 7fb4ac200740 4 client.15678 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:48:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.423+0000 7fb4ac200740 1 -- 172.21.15.123:0/3022290013 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a4d390950 con 0x556a4d437a70 2024-09-19T04:48:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.424+0000 7fb4a2a00640 1 --2- 172.21.15.123:0/3022290013 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556a4d437a70 0x556a4d439eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.424+0000 7fb4a2a00640 1 --2- 172.21.15.123:0/3022290013 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556a4d437a70 0x556a4d439eb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=624d842a150d2ad server_cookie=6b9ef69cf4275705 in_seq=0 out_seq=0 2024-09-19T04:48:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.424+0000 7fb487e00640 10 client.15678 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.425+0000 7fb487e00640 1 -- 172.21.15.123:0/3022290013 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a4d390950 con 0x556a4d437a70 2024-09-19T04:48:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.425+0000 7fb487e00640 10 client.15678 handle_command_reply: tid=0 2024-09-19T04:48:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.425+0000 7fb4ac200740 1 client.15678 shutdown 2024-09-19T04:48:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.425+0000 7fb4ac200740 1 -- 172.21.15.123:0/3022290013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d41d4b0 msgr2=0x556a4d425240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.425+0000 7fb4ac200740 1 --2- 172.21.15.123:0/3022290013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d41d4b0 0x556a4d425240 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00e9c0 tx=0x7fb48c09bcd0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.425+0000 7fb4ac200740 1 -- 172.21.15.123:0/3022290013 shutdown_connections 2024-09-19T04:48:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.425+0000 7fb4ac200740 1 --2- 172.21.15.123:0/3022290013 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556a4d437a70 0x556a4d439eb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.425+0000 7fb4ac200740 1 --2- 172.21.15.123:0/3022290013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d427d70 0x556a4d425780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.425+0000 7fb4ac200740 1 --2- 172.21.15.123:0/3022290013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d41d4b0 0x556a4d425240 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.425+0000 7fb4ac200740 1 --2- 172.21.15.123:0/3022290013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d41cb90 0x556a4d424d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.425+0000 7fb487e00640 0 client.15678 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:21.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.426+0000 7fb4ac200740 1 -- 172.21.15.123:0/3022290013 >> 172.21.15.123:0/3022290013 conn(0x556a4d419120 msgr2=0x556a4d41ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:21.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.426+0000 7fb4ac200740 1 -- 172.21.15.123:0/3022290013 shutdown_connections 2024-09-19T04:48:21.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.426+0000 7fb4ac200740 1 -- 172.21.15.123:0/3022290013 wait complete. 2024-09-19T04:48:21.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.426+0000 7fb4ac200740 20 client.15678 trim_cache size 0 max 16384 2024-09-19T04:48:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.443+0000 7fb4ac200740 1 Processor -- start 2024-09-19T04:48:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.444+0000 7fb4ac200740 1 -- start start 2024-09-19T04:48:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.444+0000 7fb4ac200740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d41cb90 0x556a4d43f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.444+0000 7fb4ac200740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41d4b0 0x556a4d43fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.444+0000 7fb4ac200740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d427d70 0x556a4d421f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.444+0000 7fb4ac200740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a41af950 con 0x556a4d41cb90 2024-09-19T04:48:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.444+0000 7fb4ac200740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a41b1580 con 0x556a4d427d70 2024-09-19T04:48:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.444+0000 7fb4ac200740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a41b0020 con 0x556a4d41d4b0 2024-09-19T04:48:21.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.444+0000 7fb4a2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d41cb90 0x556a4d43f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:21.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.444+0000 7fb4a2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d41cb90 0x556a4d43f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38444/0 (socket says 172.21.15.123:38444) 2024-09-19T04:48:21.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.444+0000 7fb49be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41d4b0 0x556a4d43fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:21.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.444+0000 7fb4a2a00640 1 -- 172.21.15.123:0/1659747592 learned_addr learned my addr 172.21.15.123:0/1659747592 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:21.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.444+0000 7fb4a3400640 1 --2- 172.21.15.123:0/1659747592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d427d70 0x556a4d421f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:21.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.444+0000 7fb49be00640 1 -- 172.21.15.123:0/1659747592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d427d70 msgr2=0x556a4d421f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:21.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.444+0000 7fb49be00640 1 --2- 172.21.15.123:0/1659747592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d427d70 0x556a4d421f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.445+0000 7fb49be00640 1 -- 172.21.15.123:0/1659747592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d41cb90 msgr2=0x556a4d43f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:21.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.445+0000 7fb49be00640 1 --2- 172.21.15.123:0/1659747592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d41cb90 0x556a4d43f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.445+0000 7fb49be00640 1 -- 172.21.15.123:0/1659747592 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a4d41d2e0 con 0x556a4d41d4b0 2024-09-19T04:48:21.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.445+0000 7fb4a2a00640 1 --2- 172.21.15.123:0/1659747592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d41cb90 0x556a4d43f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:21.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.445+0000 7fb49be00640 1 --2- 172.21.15.123:0/1659747592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41d4b0 0x556a4d43fcc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00c6a0 tx=0x7fb48c0a07b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=428547e66e52c550 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.445+0000 7fb492a00640 1 -- 172.21.15.123:0/1659747592 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c0a4ef0 con 0x556a4d41d4b0 2024-09-19T04:48:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.445+0000 7fb492a00640 1 -- 172.21.15.123:0/1659747592 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb48c009430 con 0x556a4d41d4b0 2024-09-19T04:48:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.445+0000 7fb492a00640 1 -- 172.21.15.123:0/1659747592 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c09b7f0 con 0x556a4d41d4b0 2024-09-19T04:48:21.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.446+0000 7fb4ac200740 1 -- 172.21.15.123:0/1659747592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41d4b0 msgr2=0x556a4d43fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:21.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.446+0000 7fb4ac200740 1 --2- 172.21.15.123:0/1659747592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41d4b0 0x556a4d43fcc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00c6a0 tx=0x7fb48c0a07b0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.446+0000 7fb4ac200740 1 -- 172.21.15.123:0/1659747592 shutdown_connections 2024-09-19T04:48:21.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.446+0000 7fb4ac200740 1 --2- 172.21.15.123:0/1659747592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d427d70 0x556a4d421f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.446+0000 7fb4ac200740 1 --2- 172.21.15.123:0/1659747592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41d4b0 0x556a4d43fcc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.446+0000 7fb4ac200740 1 --2- 172.21.15.123:0/1659747592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d41cb90 0x556a4d43f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.446+0000 7fb4ac200740 1 -- 172.21.15.123:0/1659747592 >> 172.21.15.123:0/1659747592 conn(0x556a4d419120 msgr2=0x556a4d426d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.446+0000 7fb4ac200740 1 -- 172.21.15.123:0/1659747592 shutdown_connections 2024-09-19T04:48:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.446+0000 7fb4ac200740 1 -- 172.21.15.123:0/1659747592 wait complete. 2024-09-19T04:48:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.446+0000 7fb4ac200740 1 -- start start 2024-09-19T04:48:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.447+0000 7fb4ac200740 1 Processor -- start 2024-09-19T04:48:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.447+0000 7fb4ac200740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d41cb90 0x556a4d42bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.447+0000 7fb4ac200740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41d4b0 0x556a4d42c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.447+0000 7fb4ac200740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d427d70 0x556a4d4a9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.447+0000 7fb4ac200740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a41b0df0 con 0x556a4d427d70 2024-09-19T04:48:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.447+0000 7fb4ac200740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a41b0720 con 0x556a4d41cb90 2024-09-19T04:48:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.447+0000 7fb4ac200740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a41ad930 con 0x556a4d41d4b0 2024-09-19T04:48:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.447+0000 7fb4ac200740 10 client.0 _ll_register_callbacks cb 0x7fb48c09be50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.447+0000 7fb4a3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d427d70 0x556a4d4a9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.447+0000 7fb49be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41d4b0 0x556a4d42c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.447+0000 7fb4a3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d427d70 0x556a4d4a9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38446/0 (socket says 172.21.15.123:38446) 2024-09-19T04:48:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.447+0000 7fb4a3400640 1 -- 172.21.15.123:0/2002740789 learned_addr learned my addr 172.21.15.123:0/2002740789 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.447+0000 7fb4a2a00640 1 --2- 172.21.15.123:0/2002740789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d41cb90 0x556a4d42bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.448+0000 7fb49be00640 1 -- 172.21.15.123:0/2002740789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d41cb90 msgr2=0x556a4d42bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.448+0000 7fb49be00640 1 --2- 172.21.15.123:0/2002740789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d41cb90 0x556a4d42bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.448+0000 7fb49be00640 1 -- 172.21.15.123:0/2002740789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d427d70 msgr2=0x556a4d4a9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.448+0000 7fb49be00640 1 --2- 172.21.15.123:0/2002740789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d427d70 0x556a4d4a9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.448+0000 7fb49be00640 1 -- 172.21.15.123:0/2002740789 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a4d42cb60 con 0x556a4d41d4b0 2024-09-19T04:48:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.448+0000 7fb49be00640 1 --2- 172.21.15.123:0/2002740789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41d4b0 0x556a4d42c260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x556a4d434990 tx=0x7fb48c0d02c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df75141d81265eba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.448+0000 7fb487e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:48:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.448+0000 7fb487e00640 1 -- 172.21.15.123:0/2002740789 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c09c280 con 0x556a4d41d4b0 2024-09-19T04:48:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.448+0000 7fb487e00640 1 -- 172.21.15.123:0/2002740789 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fb48c0bc110 con 0x556a4d41d4b0 2024-09-19T04:48:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.448+0000 7fb4ac200740 1 -- 172.21.15.123:0/2002740789 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb48c009430 con 0x556a4d41d4b0 2024-09-19T04:48:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.448+0000 7fb487e00640 1 -- 172.21.15.123:0/2002740789 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb48c09b7d0 con 0x556a4d41d4b0 2024-09-19T04:48:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.448+0000 7fb4ac200740 10 client.25735 fetch_fsmap learned FSMap version 30 2024-09-19T04:48:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.448+0000 7fb4ac200740 1 -- 172.21.15.123:0/2002740789 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556a4d4ab5e0 con 0x556a4d41d4b0 2024-09-19T04:48:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.449+0000 7fb487e00640 1 -- 172.21.15.123:0/2002740789 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb48c009430 con 0x556a4d41d4b0 2024-09-19T04:48:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.449+0000 7fb487e00640 1 -- 172.21.15.123:0/2002740789 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8382+0+0 (secure 0 0 0) 0x7fb48c0a17c0 con 0x556a4d41d4b0 2024-09-19T04:48:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.449+0000 7fb4ac200740 10 client.25735 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T04:48:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.449+0000 7fb4ac200740 10 client.25735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:48:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.449+0000 7fb4ac200740 20 client.25735 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.449+0000 7fb4ac200740 1 --2- 172.21.15.123:0/2002740789 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556a4d4abaa0 0x556a4d4adec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.449+0000 7fb4ac200740 4 client.25735 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:48:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.449+0000 7fb4ac200740 1 -- 172.21.15.123:0/2002740789 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a4d38ae60 con 0x556a4d4abaa0 2024-09-19T04:48:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.449+0000 7fb4a2a00640 1 --2- 172.21.15.123:0/2002740789 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556a4d4abaa0 0x556a4d4adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.450+0000 7fb4a2a00640 1 --2- 172.21.15.123:0/2002740789 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556a4d4abaa0 0x556a4d4adec0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad1e54080fc72114 server_cookie=34c85ec2ca69e6cf in_seq=0 out_seq=0 2024-09-19T04:48:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.450+0000 7fb487e00640 10 client.25735 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.450+0000 7fb487e00640 1 -- 172.21.15.123:0/2002740789 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556a4d38ae60 con 0x556a4d4abaa0 2024-09-19T04:48:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.450+0000 7fb487e00640 10 client.25735 handle_command_reply: tid=0 2024-09-19T04:48:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.450+0000 7fb4ac200740 1 client.25735 shutdown 2024-09-19T04:48:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.450+0000 7fb4ac200740 1 -- 172.21.15.123:0/2002740789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41d4b0 msgr2=0x556a4d42c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.450+0000 7fb4ac200740 1 --2- 172.21.15.123:0/2002740789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41d4b0 0x556a4d42c260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x556a4d434990 tx=0x7fb48c0d02c0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.450+0000 7fb4ac200740 1 -- 172.21.15.123:0/2002740789 shutdown_connections 2024-09-19T04:48:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.450+0000 7fb4ac200740 1 --2- 172.21.15.123:0/2002740789 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556a4d4abaa0 0x556a4d4adec0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.450+0000 7fb4ac200740 1 --2- 172.21.15.123:0/2002740789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a4d427d70 0x556a4d4a9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.450+0000 7fb4ac200740 1 --2- 172.21.15.123:0/2002740789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a4d41d4b0 0x556a4d42c260 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.450+0000 7fb4ac200740 1 --2- 172.21.15.123:0/2002740789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a4d41cb90 0x556a4d42bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.450+0000 7fb487e00640 0 client.25735 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.450+0000 7fb4ac200740 1 -- 172.21.15.123:0/2002740789 >> 172.21.15.123:0/2002740789 conn(0x556a4d419120 msgr2=0x556a4d4179a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.450+0000 7fb4ac200740 1 -- 172.21.15.123:0/2002740789 shutdown_connections 2024-09-19T04:48:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.450+0000 7fb4ac200740 1 -- 172.21.15.123:0/2002740789 wait complete. 2024-09-19T04:48:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.451+0000 7fb4ac200740 20 client.25735 trim_cache size 0 max 16384 2024-09-19T04:48:21.456 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:48:21.456 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:48:21.456 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": { 2024-09-19T04:48:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: "804d4dda-f6c6-4a99-88d8-fee4a2dec0b8": { 2024-09-19T04:48:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: "path": "/", 2024-09-19T04:48:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: "tag": "804d4dda-f6c6-4a99-88d8-fee4a2dec0b8", 2024-09-19T04:48:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: "options": "recursive,force" 2024-09-19T04:48:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:48:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:48:21.457 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:48:21.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.451+0000 7fb4a9600640 1 -- 172.21.15.123:0/3194785735 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb4880a33b0 msgr2=0x7fb4880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:21.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.451+0000 7fb4a9600640 1 --2- 172.21.15.123:0/3194785735 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb4880a33b0 0x7fb4880a5870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x556a4d419f80 tx=0x7fb494005f90 comp rx=0 tx=0).stop 2024-09-19T04:48:21.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.451+0000 7fb4a9600640 1 -- 172.21.15.123:0/3194785735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a4102790 msgr2=0x7fb4a419ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:21.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.451+0000 7fb4a9600640 1 --2- 172.21.15.123:0/3194785735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a4102790 0x7fb4a419ab20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00e9f0 tx=0x7fb48c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.451+0000 7fb4a9600640 1 -- 172.21.15.123:0/3194785735 shutdown_connections 2024-09-19T04:48:21.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.451+0000 7fb4a9600640 1 --2- 172.21.15.123:0/3194785735 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb4880a33b0 0x7fb4880a5870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.451+0000 7fb4a9600640 1 --2- 172.21.15.123:0/3194785735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4a4106f10 0x7fb4a419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.451+0000 7fb4a9600640 1 --2- 172.21.15.123:0/3194785735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a4102790 0x7fb4a419ab20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.451+0000 7fb4a9600640 1 --2- 172.21.15.123:0/3194785735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4a4101de0 0x7fb4a419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:21.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.451+0000 7fb4a9600640 1 -- 172.21.15.123:0/3194785735 >> 172.21.15.123:0/3194785735 conn(0x7fb4a40fd8c0 msgr2=0x7fb4a40fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:21.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.451+0000 7fb4a9600640 1 -- 172.21.15.123:0/3194785735 shutdown_connections 2024-09-19T04:48:21.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:21.452+0000 7fb4a9600640 1 -- 172.21.15.123:0/3194785735 wait complete. 2024-09-19T04:48:21.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:21 smithi183 ceph-mon[35333]: pgmap v52: 97 pgs: 97 active+clean; 16 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.3 MiB/s wr, 23 op/s 2024-09-19T04:48:21.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:21 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:48:21.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:21 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:21.859 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:21 smithi123 ceph-mon[30511]: pgmap v52: 97 pgs: 97 active+clean; 16 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.3 MiB/s wr, 23 op/s 2024-09-19T04:48:21.859 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:21 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:48:21.859 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:21 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:21.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:21 smithi193 ceph-mon[35592]: pgmap v52: 97 pgs: 97 active+clean; 16 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.3 MiB/s wr, 23 op/s 2024-09-19T04:48:21.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:21 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:48:21.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:21 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:22.232 INFO:tasks.cephfs.filesystem:scrub status for tag:804d4dda-f6c6-4a99-88d8-fee4a2dec0b8 - {'path': '/', 'tag': '804d4dda-f6c6-4a99-88d8-fee4a2dec0b8', 'options': 'recursive,force'} 2024-09-19T04:48:22.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:22 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:22.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:22 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:23.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:22 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:48:23.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:23 smithi123 ceph-mon[30511]: pgmap v53: 97 pgs: 97 active+clean; 16 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-09-19T04:48:23.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:23 smithi193 ceph-mon[35592]: pgmap v53: 97 pgs: 97 active+clean; 16 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-09-19T04:48:24.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:23 smithi183 ceph-mon[35333]: pgmap v53: 97 pgs: 97 active+clean; 16 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-09-19T04:48:25.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:25 smithi123 ceph-mon[30511]: pgmap v54: 97 pgs: 97 active+clean; 21 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.7 MiB/s wr, 43 op/s 2024-09-19T04:48:25.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:25 smithi193 ceph-mon[35592]: pgmap v54: 97 pgs: 97 active+clean; 21 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.7 MiB/s wr, 43 op/s 2024-09-19T04:48:26.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:25 smithi183 ceph-mon[35333]: pgmap v54: 97 pgs: 97 active+clean; 21 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.7 MiB/s wr, 43 op/s 2024-09-19T04:48:27.902 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:27 smithi183 ceph-mon[35333]: pgmap v55: 97 pgs: 97 active+clean; 21 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 576 KiB/s wr, 33 op/s 2024-09-19T04:48:27.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:27 smithi123 ceph-mon[30511]: pgmap v55: 97 pgs: 97 active+clean; 21 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 576 KiB/s wr, 33 op/s 2024-09-19T04:48:27.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:27 smithi193 ceph-mon[35592]: pgmap v55: 97 pgs: 97 active+clean; 21 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 576 KiB/s wr, 33 op/s 2024-09-19T04:48:29.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:29 smithi123 ceph-mon[30511]: pgmap v56: 97 pgs: 97 active+clean; 21 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 581 KiB/s wr, 36 op/s 2024-09-19T04:48:29.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:29 smithi193 ceph-mon[35592]: pgmap v56: 97 pgs: 97 active+clean; 21 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 581 KiB/s wr, 36 op/s 2024-09-19T04:48:30.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:29 smithi183 ceph-mon[35333]: pgmap v56: 97 pgs: 97 active+clean; 21 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 581 KiB/s wr, 36 op/s 2024-09-19T04:48:31.050 INFO:tasks.workunit.client.0.smithi123.stderr:Updating files: 7% (322/4351) Updating files: 8% (349/4351) Updating files: 9% (392/4351) Updating files: 10% (436/4351) Updating files: 11% (479/4351) Updating files: 12% (523/4351) Updating files: 13% (566/4351) Updating files: 14% (610/4351) Updating files: 14% (638/4351) Updating files: 15% (653/4351) Updating files: 16% (697/4351) Updating files: 17% (740/4351) Updating files: 18% (784/4351) Updating files: 19% (827/4351) Updating files: 20% (871/4351) Updating files: 21% (914/4351) Updating files: 22% (958/4351) Updating files: 23% (1001/4351) Updating files: 24% (1045/4351) Updating files: 25% (1088/4351) Updating files: 26% (1132/4351) Updating files: 27% (1175/4351) Updating files: 28% (1219/4351) Updating files: 29% (1262/4351) Updating files: 29% (1303/4351) Updating files: 30% (1306/4351) Updating files: 31% (1349/4351) Updating files: 32% (1393/4351) Updating files: 33% (1436/4351) Updating files: 34% (1480/4351) Updating files: 35% (1523/4351) Updating files: 36% (1567/4351) Updating files: 37% (1610/4351) Updating files: 38% (1654/4351) Updating files: 39% (1697/4351) Updating files: 40% (1741/4351) Updating files: 41% (1784/4351) Updating files: 42% (1828/4351) Updating files: 43% (1871/4351) Updating files: 44% (1915/4351) Updating files: 44% (1928/4351) Updating files: 45% (1958/4351) Updating files: 46% (2002/4351) Updating files: 47% (2045/4351) Updating files: 48% (2089/4351) Updating files: 49% (2132/4351) Updating files: 50% (2176/4351) Updating files: 51% (2220/4351) Updating files: 52% (2263/4351) Updating files: 53% (2307/4351) Updating files: 54% (2350/4351) Updating files: 55% (2394/4351) Updating files: 56% (2437/4351) Updating files: 56% (2454/4351) Updating files: 57% (2481/4351) Updating files: 58% (2524/4351) Updating files: 59% (2568/4351) Updating files: 60% (2611/4351) Updating files: 61% (2655/4351) Updating files: 62% (2698/4351) Updating files: 63% (2742/4351) Updating files: 64% (2785/4351) Updating files: 64% (2809/4351) Updating files: 65% (2829/4351) Updating files: 66% (2872/4351) Updating files: 67% (2916/4351) Updating files: 68% (2959/4351) Updating files: 69% (3003/4351) Updating files: 69% (3040/4351) Updating files: 70% (3046/4351) Updating files: 71% (3090/4351) Updating files: 72% (3133/4351) Updating files: 73% (3177/4351) Updating files: 74% (3220/4351) Updating files: 75% (3264/4351) Updating files: 76% (3307/4351) Updating files: 77% (3351/4351) Updating files: 78% (3394/4351) Updating files: 79% (3438/4351) Updating files: 80% (3481/4351) Updating files: 81% (3525/4351) Updating files: 82% (3568/4351) Updating files: 83% (3612/4351) Updating files: 84% (3655/4351) Updating files: 85% (3699/4351) Updating files: 85% (3734/4351) Updating files: 86% (3742/4351) Updating files: 87% (3786/4351) Updating files: 88% (3829/4351) Updating files: 89% (3873/4351) Updating files: 90% (3916/4351) Updating files: 91% (3960/4351) Updating files: 92% (4003/4351) Updating files: 93% (4047/4351) Updating files: 94% (4090/4351) Updating files: 95% (4134/4351) Updating files: 96% (4177/4351) Updating files: 97% (4221/4351) Updating files: 97% (4223/4351) Updating files: 98% (4264/4351) Updating files: 99% (4308/4351) Updating files: 100% (4351/4351) Updating files: 100% (4351/4351), done. 2024-09-19T04:48:31.059 INFO:tasks.workunit.client.0.smithi123.stderr:+ cd xfstests-dev 2024-09-19T04:48:31.091 INFO:tasks.workunit.client.0.smithi123.stderr:+ git checkout 0e5c12dfd008efc2848c98108c9237487e91ef35 2024-09-19T04:48:31.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:31 smithi123 ceph-mon[30511]: pgmap v57: 97 pgs: 97 active+clean; 21 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 538 KiB/s wr, 37 op/s 2024-09-19T04:48:31.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:31 smithi193 ceph-mon[35592]: pgmap v57: 97 pgs: 97 active+clean; 21 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 538 KiB/s wr, 37 op/s 2024-09-19T04:48:32.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:31 smithi183 ceph-mon[35333]: pgmap v57: 97 pgs: 97 active+clean; 21 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 538 KiB/s wr, 37 op/s 2024-09-19T04:48:33.892 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:33 smithi123 ceph-mon[30511]: pgmap v58: 97 pgs: 97 active+clean; 21 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 516 KiB/s wr, 30 op/s 2024-09-19T04:48:33.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:33 smithi193 ceph-mon[35592]: pgmap v58: 97 pgs: 97 active+clean; 21 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 516 KiB/s wr, 30 op/s 2024-09-19T04:48:34.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:33 smithi183 ceph-mon[35333]: pgmap v58: 97 pgs: 97 active+clean; 21 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 516 KiB/s wr, 30 op/s 2024-09-19T04:48:34.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:34 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:48:34.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:34 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:48:35.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:34 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:48:35.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:35 smithi123 ceph-mon[30511]: pgmap v59: 97 pgs: 97 active+clean; 27 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.1 MiB/s wr, 51 op/s 2024-09-19T04:48:35.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:35 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:48:35.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:35 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:48:35.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:35 smithi123 ceph-mon[30511]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:48:35.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:35 smithi193 ceph-mon[35592]: pgmap v59: 97 pgs: 97 active+clean; 27 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.1 MiB/s wr, 51 op/s 2024-09-19T04:48:35.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:35 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:48:35.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:35 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:48:35.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:35 smithi193 ceph-mon[35592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:48:36.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:35 smithi183 ceph-mon[35333]: pgmap v59: 97 pgs: 97 active+clean; 27 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.1 MiB/s wr, 51 op/s 2024-09-19T04:48:36.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:35 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:48:36.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:35 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:48:36.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:35 smithi183 ceph-mon[35333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:48:37.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:37 smithi123 ceph-mon[30511]: pgmap v60: 97 pgs: 97 active+clean; 27 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 654 KiB/s wr, 27 op/s 2024-09-19T04:48:37.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:37 smithi193 ceph-mon[35592]: pgmap v60: 97 pgs: 97 active+clean; 27 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 654 KiB/s wr, 27 op/s 2024-09-19T04:48:38.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:37 smithi183 ceph-mon[35333]: pgmap v60: 97 pgs: 97 active+clean; 27 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 654 KiB/s wr, 27 op/s 2024-09-19T04:48:39.717 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:39 smithi123 ceph-mon[30511]: pgmap v61: 97 pgs: 97 active+clean; 42 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 238 op/s 2024-09-19T04:48:39.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:39 smithi193 ceph-mon[35592]: pgmap v61: 97 pgs: 97 active+clean; 42 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 238 op/s 2024-09-19T04:48:40.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:39 smithi183 ceph-mon[35333]: pgmap v61: 97 pgs: 97 active+clean; 42 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 238 op/s 2024-09-19T04:48:41.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:41 smithi123 ceph-mon[30511]: pgmap v62: 97 pgs: 97 active+clean; 42 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 234 op/s 2024-09-19T04:48:41.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:41 smithi193 ceph-mon[35592]: pgmap v62: 97 pgs: 97 active+clean; 42 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 234 op/s 2024-09-19T04:48:42.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:41 smithi183 ceph-mon[35333]: pgmap v62: 97 pgs: 97 active+clean; 42 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 234 op/s 2024-09-19T04:48:42.392 INFO:tasks.workunit.client.0.smithi123.stderr:Note: switching to '0e5c12dfd008efc2848c98108c9237487e91ef35'. 2024-09-19T04:48:42.393 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-19T04:48:42.393 INFO:tasks.workunit.client.0.smithi123.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T04:48:42.393 INFO:tasks.workunit.client.0.smithi123.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T04:48:42.393 INFO:tasks.workunit.client.0.smithi123.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T04:48:42.393 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-19T04:48:42.393 INFO:tasks.workunit.client.0.smithi123.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T04:48:42.393 INFO:tasks.workunit.client.0.smithi123.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T04:48:42.393 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-19T04:48:42.393 INFO:tasks.workunit.client.0.smithi123.stderr: git switch -c 2024-09-19T04:48:42.393 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-19T04:48:42.393 INFO:tasks.workunit.client.0.smithi123.stderr:Or undo this operation with: 2024-09-19T04:48:42.393 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-19T04:48:42.393 INFO:tasks.workunit.client.0.smithi123.stderr: git switch - 2024-09-19T04:48:42.393 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-19T04:48:42.394 INFO:tasks.workunit.client.0.smithi123.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T04:48:42.394 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-19T04:48:42.394 INFO:tasks.workunit.client.0.smithi123.stderr:HEAD is now at 0e5c12df xfs/522: use reflink instead of crc as test feature 2024-09-19T04:48:42.395 INFO:tasks.workunit.client.0.smithi123.stderr:+ make -j4 2024-09-19T04:48:42.412 INFO:tasks.workunit.client.0.smithi123.stdout:libtoolize -cfi 2024-09-19T04:48:43.302 INFO:tasks.workunit.client.0.smithi123.stdout:libtoolize: putting auxiliary files in '.'. 2024-09-19T04:48:43.302 INFO:tasks.workunit.client.0.smithi123.stdout:libtoolize: copying file './config.guess' 2024-09-19T04:48:43.333 INFO:tasks.workunit.client.0.smithi123.stdout:libtoolize: copying file './config.sub' 2024-09-19T04:48:43.348 INFO:tasks.workunit.client.0.smithi123.stdout:libtoolize: copying file './install-sh' 2024-09-19T04:48:43.364 INFO:tasks.workunit.client.0.smithi123.stdout:libtoolize: copying file './ltmain.sh' 2024-09-19T04:48:43.428 INFO:tasks.workunit.client.0.smithi123.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-09-19T04:48:43.428 INFO:tasks.workunit.client.0.smithi123.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-09-19T04:48:43.441 INFO:tasks.workunit.client.0.smithi123.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-09-19T04:48:43.455 INFO:tasks.workunit.client.0.smithi123.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-09-19T04:48:43.468 INFO:tasks.workunit.client.0.smithi123.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-09-19T04:48:43.481 INFO:tasks.workunit.client.0.smithi123.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-09-19T04:48:43.498 INFO:tasks.workunit.client.0.smithi123.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-09-19T04:48:43.501 INFO:tasks.workunit.client.0.smithi123.stdout:cp include/install-sh . 2024-09-19T04:48:43.512 INFO:tasks.workunit.client.0.smithi123.stdout:aclocal -I m4 2024-09-19T04:48:43.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:43 smithi123 ceph-mon[30511]: pgmap v63: 97 pgs: 97 active+clean; 42 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 231 op/s 2024-09-19T04:48:43.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:43 smithi193 ceph-mon[35592]: pgmap v63: 97 pgs: 97 active+clean; 42 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 231 op/s 2024-09-19T04:48:44.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:43 smithi183 ceph-mon[35333]: pgmap v63: 97 pgs: 97 active+clean; 42 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 231 op/s 2024-09-19T04:48:44.655 INFO:tasks.workunit.client.0.smithi123.stdout:autoheader 2024-09-19T04:48:45.033 INFO:tasks.workunit.client.0.smithi123.stdout:autoconf 2024-09-19T04:48:45.260 INFO:tasks.workunit.client.0.smithi123.stdout:./configure \ 2024-09-19T04:48:45.260 INFO:tasks.workunit.client.0.smithi123.stdout: --libexecdir=/usr/lib \ 2024-09-19T04:48:45.260 INFO:tasks.workunit.client.0.smithi123.stdout: --exec_prefix=/var/lib 2024-09-19T04:48:45.573 INFO:tasks.workunit.client.0.smithi123.stdout:checking build system type... x86_64-pc-linux-gnu 2024-09-19T04:48:45.573 INFO:tasks.workunit.client.0.smithi123.stdout:checking host system type... x86_64-pc-linux-gnu 2024-09-19T04:48:45.576 INFO:tasks.workunit.client.0.smithi123.stdout:checking how to print strings... printf 2024-09-19T04:48:45.576 INFO:tasks.workunit.client.0.smithi123.stdout:checking for gcc... gcc 2024-09-19T04:48:45.725 INFO:tasks.workunit.client.0.smithi123.stdout:checking whether the C compiler works... yes 2024-09-19T04:48:45.725 INFO:tasks.workunit.client.0.smithi123.stdout:checking for C compiler default output file name... a.out 2024-09-19T04:48:45.801 INFO:tasks.workunit.client.0.smithi123.stdout:checking for suffix of executables... 2024-09-19T04:48:45.871 INFO:tasks.workunit.client.0.smithi123.stdout:checking whether we are cross compiling... no 2024-09-19T04:48:45.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:45 smithi123 ceph-mon[30511]: pgmap v64: 97 pgs: 97 active+clean; 59 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 626 op/s 2024-09-19T04:48:45.931 INFO:tasks.workunit.client.0.smithi123.stdout:checking for suffix of object files... o 2024-09-19T04:48:45.981 INFO:tasks.workunit.client.0.smithi123.stdout:checking whether we are using the GNU C compiler... yes 2024-09-19T04:48:45.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:45 smithi193 ceph-mon[35592]: pgmap v64: 97 pgs: 97 active+clean; 59 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 626 op/s 2024-09-19T04:48:46.030 INFO:tasks.workunit.client.0.smithi123.stdout:checking whether gcc accepts -g... yes 2024-09-19T04:48:46.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:45 smithi183 ceph-mon[35333]: pgmap v64: 97 pgs: 97 active+clean; 59 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 626 op/s 2024-09-19T04:48:46.138 INFO:tasks.workunit.client.0.smithi123.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-19T04:48:46.146 INFO:tasks.workunit.client.0.smithi123.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-09-19T04:48:46.153 INFO:tasks.workunit.client.0.smithi123.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-19T04:48:46.156 INFO:tasks.workunit.client.0.smithi123.stdout:checking for egrep... /usr/bin/grep -E 2024-09-19T04:48:46.159 INFO:tasks.workunit.client.0.smithi123.stdout:checking for fgrep... /usr/bin/grep -F 2024-09-19T04:48:46.174 INFO:tasks.workunit.client.0.smithi123.stdout:checking for ld used by gcc... /usr/bin/ld 2024-09-19T04:48:46.177 INFO:tasks.workunit.client.0.smithi123.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-09-19T04:48:46.187 INFO:tasks.workunit.client.0.smithi123.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-09-19T04:48:47.154 INFO:tasks.workunit.client.0.smithi123.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-09-19T04:48:47.154 INFO:tasks.workunit.client.0.smithi123.stdout:checking whether ln -s works... yes 2024-09-19T04:48:47.174 INFO:tasks.workunit.client.0.smithi123.stdout:checking the maximum length of command line arguments... 1572864 2024-09-19T04:48:47.180 INFO:tasks.workunit.client.0.smithi123.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-09-19T04:48:47.180 INFO:tasks.workunit.client.0.smithi123.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-09-19T04:48:47.180 INFO:tasks.workunit.client.0.smithi123.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-09-19T04:48:47.180 INFO:tasks.workunit.client.0.smithi123.stdout:checking for objdump... objdump 2024-09-19T04:48:47.180 INFO:tasks.workunit.client.0.smithi123.stdout:checking how to recognize dependent libraries... pass_all 2024-09-19T04:48:47.181 INFO:tasks.workunit.client.0.smithi123.stdout:checking for dlltool... no 2024-09-19T04:48:47.181 INFO:tasks.workunit.client.0.smithi123.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-09-19T04:48:47.182 INFO:tasks.workunit.client.0.smithi123.stdout:checking for ar... ar 2024-09-19T04:48:47.332 INFO:tasks.workunit.client.0.smithi123.stdout:checking for archiver @FILE support... @ 2024-09-19T04:48:47.332 INFO:tasks.workunit.client.0.smithi123.stdout:checking for strip... strip 2024-09-19T04:48:47.332 INFO:tasks.workunit.client.0.smithi123.stdout:checking for ranlib... ranlib 2024-09-19T04:48:47.332 INFO:tasks.workunit.client.0.smithi123.stdout:checking for gawk... gawk 2024-09-19T04:48:47.517 INFO:tasks.workunit.client.0.smithi123.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-09-19T04:48:47.526 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sysroot... no 2024-09-19T04:48:47.545 INFO:tasks.workunit.client.0.smithi123.stdout:checking for a working dd... /usr/bin/dd 2024-09-19T04:48:47.559 INFO:tasks.workunit.client.0.smithi123.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-09-19T04:48:47.615 INFO:tasks.workunit.client.0.smithi123.stdout:checking for mt... no 2024-09-19T04:48:47.632 INFO:tasks.workunit.client.0.smithi123.stdout:checking if : is a manifest tool... no 2024-09-19T04:48:47.745 INFO:tasks.workunit.client.0.smithi123.stdout:checking how to run the C preprocessor... gcc -E 2024-09-19T04:48:47.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:47 smithi123 ceph-mon[30511]: pgmap v65: 97 pgs: 97 active+clean; 59 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 605 op/s 2024-09-19T04:48:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:47 smithi193 ceph-mon[35592]: pgmap v65: 97 pgs: 97 active+clean; 59 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 605 op/s 2024-09-19T04:48:48.068 INFO:tasks.workunit.client.0.smithi123.stdout:checking for ANSI C header files... yes 2024-09-19T04:48:48.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:47 smithi183 ceph-mon[35333]: pgmap v65: 97 pgs: 97 active+clean; 59 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 605 op/s 2024-09-19T04:48:48.131 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/types.h... yes 2024-09-19T04:48:48.207 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/stat.h... yes 2024-09-19T04:48:48.280 INFO:tasks.workunit.client.0.smithi123.stdout:checking for stdlib.h... yes 2024-09-19T04:48:48.353 INFO:tasks.workunit.client.0.smithi123.stdout:checking for string.h... yes 2024-09-19T04:48:48.428 INFO:tasks.workunit.client.0.smithi123.stdout:checking for memory.h... yes 2024-09-19T04:48:48.503 INFO:tasks.workunit.client.0.smithi123.stdout:checking for strings.h... yes 2024-09-19T04:48:48.577 INFO:tasks.workunit.client.0.smithi123.stdout:checking for inttypes.h... yes 2024-09-19T04:48:48.652 INFO:tasks.workunit.client.0.smithi123.stdout:checking for stdint.h... yes 2024-09-19T04:48:48.727 INFO:tasks.workunit.client.0.smithi123.stdout:checking for unistd.h... yes 2024-09-19T04:48:48.807 INFO:tasks.workunit.client.0.smithi123.stdout:checking for dlfcn.h... yes 2024-09-19T04:48:48.818 INFO:tasks.workunit.client.0.smithi123.stdout:checking for objdir... .libs 2024-09-19T04:48:49.026 INFO:tasks.workunit.client.0.smithi123.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-09-19T04:48:49.026 INFO:tasks.workunit.client.0.smithi123.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-09-19T04:48:49.094 INFO:tasks.workunit.client.0.smithi123.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-09-19T04:48:49.166 INFO:tasks.workunit.client.0.smithi123.stdout:checking if gcc static flag -static works... no 2024-09-19T04:48:49.265 INFO:tasks.workunit.client.0.smithi123.stdout:checking if gcc supports -c -o file.o... yes 2024-09-19T04:48:49.265 INFO:tasks.workunit.client.0.smithi123.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-09-19T04:48:49.280 INFO:tasks.workunit.client.0.smithi123.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-09-19T04:48:49.370 INFO:tasks.workunit.client.0.smithi123.stdout:checking whether -lc should be explicitly linked in... no 2024-09-19T04:48:49.500 INFO:tasks.workunit.client.0.smithi123.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-09-19T04:48:49.500 INFO:tasks.workunit.client.0.smithi123.stdout:checking how to hardcode library paths into programs... immediate 2024-09-19T04:48:49.503 INFO:tasks.workunit.client.0.smithi123.stdout:checking whether stripping libraries is possible... yes 2024-09-19T04:48:49.503 INFO:tasks.workunit.client.0.smithi123.stdout:checking if libtool supports shared libraries... yes 2024-09-19T04:48:49.503 INFO:tasks.workunit.client.0.smithi123.stdout:checking whether to build shared libraries... yes 2024-09-19T04:48:49.503 INFO:tasks.workunit.client.0.smithi123.stdout:checking whether to build static libraries... yes 2024-09-19T04:48:49.517 INFO:tasks.workunit.client.0.smithi123.stdout:checking for gcc... (cached) gcc 2024-09-19T04:48:49.596 INFO:tasks.workunit.client.0.smithi123.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-19T04:48:49.597 INFO:tasks.workunit.client.0.smithi123.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-19T04:48:49.597 INFO:tasks.workunit.client.0.smithi123.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-19T04:48:49.597 INFO:tasks.workunit.client.0.smithi123.stdout:checking for gmake... /usr/bin/gmake 2024-09-19T04:48:49.597 INFO:tasks.workunit.client.0.smithi123.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-09-19T04:48:49.598 INFO:tasks.workunit.client.0.smithi123.stdout:checking for tar... /bin/tar 2024-09-19T04:48:49.598 INFO:tasks.workunit.client.0.smithi123.stdout:checking for gzip... /bin/gzip 2024-09-19T04:48:49.637 INFO:tasks.workunit.client.0.smithi123.stdout:checking whether gcc -MM is supported... yes 2024-09-19T04:48:49.638 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sort... /bin/sort 2024-09-19T04:48:49.638 INFO:tasks.workunit.client.0.smithi123.stdout:checking whether ln -s works... yes 2024-09-19T04:48:49.638 INFO:tasks.workunit.client.0.smithi123.stdout:checking for rpm... /bin/rpm 2024-09-19T04:48:49.646 INFO:tasks.workunit.client.0.smithi123.stdout:checking for rpmbuild... no 2024-09-19T04:48:49.722 INFO:tasks.workunit.client.0.smithi123.stdout:checking assert.h usability... yes 2024-09-19T04:48:49.748 INFO:tasks.workunit.client.0.smithi123.stdout:checking assert.h presence... yes 2024-09-19T04:48:49.748 INFO:tasks.workunit.client.0.smithi123.stdout:checking for assert.h... yes 2024-09-19T04:48:49.835 INFO:tasks.workunit.client.0.smithi123.stdout:checking bstring.h usability... no 2024-09-19T04:48:49.881 INFO:tasks.workunit.client.0.smithi123.stdout:checking bstring.h presence... no 2024-09-19T04:48:49.881 INFO:tasks.workunit.client.0.smithi123.stdout:checking for bstring.h... no 2024-09-19T04:48:49.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:49 smithi123 ceph-mon[30511]: pgmap v66: 97 pgs: 97 active+clean; 59 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 648 op/s 2024-09-19T04:48:49.951 INFO:tasks.workunit.client.0.smithi123.stdout:checking libgen.h usability... yes 2024-09-19T04:48:49.978 INFO:tasks.workunit.client.0.smithi123.stdout:checking libgen.h presence... yes 2024-09-19T04:48:49.978 INFO:tasks.workunit.client.0.smithi123.stdout:checking for libgen.h... yes 2024-09-19T04:48:49.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:49 smithi193 ceph-mon[35592]: pgmap v66: 97 pgs: 97 active+clean; 59 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 648 op/s 2024-09-19T04:48:50.072 INFO:tasks.workunit.client.0.smithi123.stdout:checking dirent.h usability... yes 2024-09-19T04:48:50.100 INFO:tasks.workunit.client.0.smithi123.stdout:checking dirent.h presence... yes 2024-09-19T04:48:50.100 INFO:tasks.workunit.client.0.smithi123.stdout:checking for dirent.h... yes 2024-09-19T04:48:50.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:49 smithi183 ceph-mon[35333]: pgmap v66: 97 pgs: 97 active+clean; 59 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 648 op/s 2024-09-19T04:48:50.181 INFO:tasks.workunit.client.0.smithi123.stdout:checking errno.h usability... yes 2024-09-19T04:48:50.208 INFO:tasks.workunit.client.0.smithi123.stdout:checking errno.h presence... yes 2024-09-19T04:48:50.208 INFO:tasks.workunit.client.0.smithi123.stdout:checking for errno.h... yes 2024-09-19T04:48:50.291 INFO:tasks.workunit.client.0.smithi123.stdout:checking malloc.h usability... yes 2024-09-19T04:48:50.320 INFO:tasks.workunit.client.0.smithi123.stdout:checking malloc.h presence... yes 2024-09-19T04:48:50.320 INFO:tasks.workunit.client.0.smithi123.stdout:checking for malloc.h... yes 2024-09-19T04:48:50.407 INFO:tasks.workunit.client.0.smithi123.stdout:checking uuid.h usability... no 2024-09-19T04:48:50.452 INFO:tasks.workunit.client.0.smithi123.stdout:checking uuid.h presence... no 2024-09-19T04:48:50.452 INFO:tasks.workunit.client.0.smithi123.stdout:checking for uuid.h... no 2024-09-19T04:48:50.537 INFO:tasks.workunit.client.0.smithi123.stdout:checking uuid/uuid.h usability... yes 2024-09-19T04:48:50.571 INFO:tasks.workunit.client.0.smithi123.stdout:checking uuid/uuid.h presence... yes 2024-09-19T04:48:50.571 INFO:tasks.workunit.client.0.smithi123.stdout:checking for uuid/uuid.h... yes 2024-09-19T04:48:50.661 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/uuid.h usability... no 2024-09-19T04:48:50.705 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/uuid.h presence... no 2024-09-19T04:48:50.705 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/uuid.h... no 2024-09-19T04:48:50.792 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/file.h usability... yes 2024-09-19T04:48:50.823 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/file.h presence... yes 2024-09-19T04:48:50.823 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/file.h... yes 2024-09-19T04:48:50.914 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/fcntl.h usability... yes 2024-09-19T04:48:50.943 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/fcntl.h presence... yes 2024-09-19T04:48:50.943 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/fcntl.h... yes 2024-09-19T04:48:51.031 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/syssgi.h usability... no 2024-09-19T04:48:51.075 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/syssgi.h presence... no 2024-09-19T04:48:51.075 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/syssgi.h... no 2024-09-19T04:48:51.215 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/param.h usability... yes 2024-09-19T04:48:51.251 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/param.h presence... yes 2024-09-19T04:48:51.251 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/param.h... yes 2024-09-19T04:48:51.273 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/stat.h... (cached) yes 2024-09-19T04:48:51.353 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/statvfs.h usability... yes 2024-09-19T04:48:51.380 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/statvfs.h presence... yes 2024-09-19T04:48:51.380 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/statvfs.h... yes 2024-09-19T04:48:51.465 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/time.h usability... yes 2024-09-19T04:48:51.497 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/time.h presence... yes 2024-09-19T04:48:51.497 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/time.h... yes 2024-09-19T04:48:51.583 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/ioctl.h usability... yes 2024-09-19T04:48:51.611 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/ioctl.h presence... yes 2024-09-19T04:48:51.611 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/ioctl.h... yes 2024-09-19T04:48:51.704 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/wait.h usability... yes 2024-09-19T04:48:51.735 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/wait.h presence... yes 2024-09-19T04:48:51.735 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/wait.h... yes 2024-09-19T04:48:51.757 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/types.h... (cached) yes 2024-09-19T04:48:51.769 INFO:tasks.workunit.client.0.smithi123.stdout:checking for strings.h... (cached) yes 2024-09-19T04:48:51.846 INFO:tasks.workunit.client.0.smithi123.stdout:checking err.h usability... yes 2024-09-19T04:48:51.874 INFO:tasks.workunit.client.0.smithi123.stdout:checking err.h presence... yes 2024-09-19T04:48:51.874 INFO:tasks.workunit.client.0.smithi123.stdout:checking for err.h... yes 2024-09-19T04:48:51.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:51 smithi123 ceph-mon[30511]: pgmap v67: 97 pgs: 97 active+clean; 59 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 438 op/s 2024-09-19T04:48:51.956 INFO:tasks.workunit.client.0.smithi123.stdout:checking linux/falloc.h usability... yes 2024-09-19T04:48:51.982 INFO:tasks.workunit.client.0.smithi123.stdout:checking linux/falloc.h presence... yes 2024-09-19T04:48:51.982 INFO:tasks.workunit.client.0.smithi123.stdout:checking for linux/falloc.h... yes 2024-09-19T04:48:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:51 smithi193 ceph-mon[35592]: pgmap v67: 97 pgs: 97 active+clean; 59 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 438 op/s 2024-09-19T04:48:52.075 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-09-19T04:48:52.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:51 smithi183 ceph-mon[35333]: pgmap v67: 97 pgs: 97 active+clean; 59 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 438 op/s 2024-09-19T04:48:52.122 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-09-19T04:48:52.122 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/fs/xfs_fsops.h... no 2024-09-19T04:48:52.196 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/fs/xfs_itable.h usability... no 2024-09-19T04:48:52.233 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:48:52.239 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/fs/xfs_itable.h presence... no 2024-09-19T04:48:52.239 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/fs/xfs_itable.h... no 2024-09-19T04:48:52.316 INFO:tasks.workunit.client.0.smithi123.stdout:checking xfs/platform_defs.h usability... no 2024-09-19T04:48:52.361 INFO:tasks.workunit.client.0.smithi123.stdout:checking xfs/platform_defs.h presence... no 2024-09-19T04:48:52.361 INFO:tasks.workunit.client.0.smithi123.stdout:checking for xfs/platform_defs.h... no 2024-09-19T04:48:52.436 INFO:tasks.workunit.client.0.smithi123.stdout:checking btrfs/ioctl.h usability... no 2024-09-19T04:48:52.482 INFO:tasks.workunit.client.0.smithi123.stdout:checking btrfs/ioctl.h presence... no 2024-09-19T04:48:52.482 INFO:tasks.workunit.client.0.smithi123.stdout:checking for btrfs/ioctl.h... no 2024-09-19T04:48:52.567 INFO:tasks.workunit.client.0.smithi123.stdout:checking cifs/ioctl.h usability... no 2024-09-19T04:48:52.618 INFO:tasks.workunit.client.0.smithi123.stdout:checking cifs/ioctl.h presence... no 2024-09-19T04:48:52.618 INFO:tasks.workunit.client.0.smithi123.stdout:checking for cifs/ioctl.h... no 2024-09-19T04:48:52.625 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:48:52.734 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/mman.h usability... yes 2024-09-19T04:48:52.762 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/mman.h presence... yes 2024-09-19T04:48:52.763 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/mman.h... yes 2024-09-19T04:48:52.855 INFO:tasks.workunit.client.0.smithi123.stdout:checking linux/ext4.h usability... no 2024-09-19T04:48:52.899 INFO:tasks.workunit.client.0.smithi123.stdout:checking linux/ext4.h presence... no 2024-09-19T04:48:52.899 INFO:tasks.workunit.client.0.smithi123.stdout:checking for linux/ext4.h... no 2024-09-19T04:48:52.951 INFO:tasks.workunit.client.0.smithi123.stdout:checking for xfs/xfs_log_format.h... no 2024-09-19T04:48:53.057 INFO:tasks.workunit.client.0.smithi123.stdout:checking for uuid_compare... no 2024-09-19T04:48:53.132 INFO:tasks.workunit.client.0.smithi123.stdout:checking for uuid_compare in -luuid... yes 2024-09-19T04:48:53.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.239+0000 7fea03000640 1 -- 172.21.15.123:0/2511483844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc1049a0 msgr2=0x7fe9fc1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:53.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.239+0000 7fea03000640 1 --2- 172.21.15.123:0/2511483844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc1049a0 0x7fe9fc1089f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009860 tx=0x7fe9e402a340 comp rx=0 tx=0).stop 2024-09-19T04:48:53.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.239+0000 7fea03000640 1 -- 172.21.15.123:0/2511483844 shutdown_connections 2024-09-19T04:48:53.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.239+0000 7fea03000640 1 --2- 172.21.15.123:0/2511483844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9fc109120 0x7fe9fc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.239+0000 7fea03000640 1 --2- 172.21.15.123:0/2511483844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc1049a0 0x7fe9fc1089f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.239+0000 7fea03000640 1 --2- 172.21.15.123:0/2511483844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9fc103ff0 0x7fe9fc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.239+0000 7fea03000640 1 -- 172.21.15.123:0/2511483844 >> 172.21.15.123:0/2511483844 conn(0x7fe9fc0ffa90 msgr2=0x7fe9fc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:53.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.239+0000 7fea03000640 1 -- 172.21.15.123:0/2511483844 shutdown_connections 2024-09-19T04:48:53.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.239+0000 7fea03000640 1 -- 172.21.15.123:0/2511483844 wait complete. 2024-09-19T04:48:53.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.240+0000 7fea03000640 1 Processor -- start 2024-09-19T04:48:53.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.241+0000 7fea03000640 1 -- start start 2024-09-19T04:48:53.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.241+0000 7fea03000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc103ff0 0x7fe9fc19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.241+0000 7fea03000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9fc1049a0 0x7fe9fc19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.241+0000 7fea03000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9fc109120 0x7fe9fc19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.241+0000 7fea03000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc114140 con 0x7fe9fc109120 2024-09-19T04:48:53.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.241+0000 7fea03000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc113fc0 con 0x7fe9fc103ff0 2024-09-19T04:48:53.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.241+0000 7fea03000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc1142c0 con 0x7fe9fc1049a0 2024-09-19T04:48:53.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.241+0000 7fea01000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9fc109120 0x7fe9fc19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.241+0000 7fe9f3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9fc1049a0 0x7fe9fc19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.241+0000 7fe9fbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc103ff0 0x7fe9fc19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.241+0000 7fe9f3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9fc1049a0 0x7fe9fc19abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:41438/0 (socket says 172.21.15.123:41438) 2024-09-19T04:48:53.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.241+0000 7fe9f3400640 1 -- 172.21.15.123:0/3911817622 learned_addr learned my addr 172.21.15.123:0/3911817622 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:53.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.241+0000 7fea01000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9fc109120 0x7fe9fc19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59724/0 (socket says 172.21.15.123:59724) 2024-09-19T04:48:53.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.242+0000 7fe9f3400640 1 -- 172.21.15.123:0/3911817622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc103ff0 msgr2=0x7fe9fc19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:53.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.242+0000 7fe9f3400640 1 --2- 172.21.15.123:0/3911817622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc103ff0 0x7fe9fc19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.242+0000 7fe9f3400640 1 -- 172.21.15.123:0/3911817622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9fc109120 msgr2=0x7fe9fc19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:53.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.242+0000 7fe9f3400640 1 --2- 172.21.15.123:0/3911817622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9fc109120 0x7fe9fc19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.242+0000 7fe9f3400640 1 -- 172.21.15.123:0/3911817622 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc19f650 con 0x7fe9fc1049a0 2024-09-19T04:48:53.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.242+0000 7fe9f3400640 1 --2- 172.21.15.123:0/3911817622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9fc1049a0 0x7fe9fc19abe0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4035040 tx=0x7fe9e4033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:53.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.242+0000 7fe9f9600640 1 -- 172.21.15.123:0/3911817622 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4037040 con 0x7fe9fc1049a0 2024-09-19T04:48:53.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.242+0000 7fe9f9600640 1 -- 172.21.15.123:0/3911817622 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9e4040070 con 0x7fe9fc1049a0 2024-09-19T04:48:53.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.242+0000 7fe9f9600640 1 -- 172.21.15.123:0/3911817622 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40039b0 con 0x7fe9fc1049a0 2024-09-19T04:48:53.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.243+0000 7fea03000640 1 -- 172.21.15.123:0/3911817622 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9fc19f8e0 con 0x7fe9fc1049a0 2024-09-19T04:48:53.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.243+0000 7fea03000640 1 -- 172.21.15.123:0/3911817622 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9fc19fd70 con 0x7fe9fc1049a0 2024-09-19T04:48:53.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.246+0000 7fe9f9600640 1 -- 172.21.15.123:0/3911817622 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe9e40305e0 con 0x7fe9fc1049a0 2024-09-19T04:48:53.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.246+0000 7fea05ab3740 1 Processor -- start 2024-09-19T04:48:53.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.246+0000 7fea05ab3740 1 -- start start 2024-09-19T04:48:53.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.246+0000 7fea05ab3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560748350b90 0x560748350f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.246+0000 7fea05ab3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607483514b0 0x56074835b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.246+0000 7fea05ab3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56074835bd70 0x56074835e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.246+0000 7fea05ab3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5607482b73e0 con 0x5607483514b0 2024-09-19T04:48:53.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.246+0000 7fea05ab3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56074825ed80 con 0x56074835bd70 2024-09-19T04:48:53.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.246+0000 7fea05ab3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5607482b33a0 con 0x560748350b90 2024-09-19T04:48:53.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.246+0000 7fea01000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560748350b90 0x560748350f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.247+0000 7fe9fbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607483514b0 0x56074835b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.247+0000 7fea01000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560748350b90 0x560748350f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:41450/0 (socket says 172.21.15.123:41450) 2024-09-19T04:48:53.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.247+0000 7fea01000640 1 -- 172.21.15.123:0/3074023851 learned_addr learned my addr 172.21.15.123:0/3074023851 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:53.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.247+0000 7fe9f3400640 1 --2- 172.21.15.123:0/3074023851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56074835bd70 0x56074835e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.247+0000 7fea01000640 1 -- 172.21.15.123:0/3074023851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56074835bd70 msgr2=0x56074835e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:53.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.247+0000 7fea01000640 1 --2- 172.21.15.123:0/3074023851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56074835bd70 0x56074835e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.247+0000 7fea01000640 1 -- 172.21.15.123:0/3074023851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607483514b0 msgr2=0x56074835b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:53.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.247+0000 7fea01000640 1 --2- 172.21.15.123:0/3074023851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607483514b0 0x56074835b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.247+0000 7fea01000640 1 -- 172.21.15.123:0/3074023851 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56074827dae0 con 0x560748350b90 2024-09-19T04:48:53.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.247+0000 7fea01000640 1 --2- 172.21.15.123:0/3074023851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560748350b90 0x560748350f70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009c20 tx=0x7fe9f4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a11da6823ec95a08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:53.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.248+0000 7fe9f9600640 1 --2- 172.21.15.123:0/3911817622 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9e00a3430 0x7fe9e00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.248+0000 7fe9f1600640 1 -- 172.21.15.123:0/3074023851 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f401e070 con 0x560748350b90 2024-09-19T04:48:53.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.248+0000 7fe9f1600640 1 -- 172.21.15.123:0/3074023851 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9f4019b10 con 0x560748350b90 2024-09-19T04:48:53.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.248+0000 7fe9f1600640 1 -- 172.21.15.123:0/3074023851 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f400eae0 con 0x560748350b90 2024-09-19T04:48:53.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.248+0000 7fe9f9600640 1 -- 172.21.15.123:0/3911817622 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fe9e40efb30 con 0x7fe9fc1049a0 2024-09-19T04:48:53.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.248+0000 7fe9fbe00640 1 --2- 172.21.15.123:0/3911817622 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9e00a3430 0x7fe9e00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.249+0000 7fe9fbe00640 1 --2- 172.21.15.123:0/3911817622 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9e00a3430 0x7fe9e00a58f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x56074834df80 tx=0x7fe9ec010040 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:53.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.249+0000 7fea05ab3740 1 -- 172.21.15.123:0/3074023851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560748350b90 msgr2=0x560748350f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:53.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.249+0000 7fea05ab3740 1 --2- 172.21.15.123:0/3074023851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560748350b90 0x560748350f70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009c20 tx=0x7fe9f4011810 comp rx=0 tx=0).stop 2024-09-19T04:48:53.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.249+0000 7fea05ab3740 1 -- 172.21.15.123:0/3074023851 shutdown_connections 2024-09-19T04:48:53.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.249+0000 7fea05ab3740 1 --2- 172.21.15.123:0/3074023851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56074835bd70 0x56074835e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.249+0000 7fea05ab3740 1 --2- 172.21.15.123:0/3074023851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607483514b0 0x56074835b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.249+0000 7fea05ab3740 1 --2- 172.21.15.123:0/3074023851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560748350b90 0x560748350f70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.249+0000 7fea05ab3740 1 -- 172.21.15.123:0/3074023851 >> 172.21.15.123:0/3074023851 conn(0x56074834d120 msgr2=0x56074834d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:53.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.249+0000 7fea05ab3740 1 -- 172.21.15.123:0/3074023851 shutdown_connections 2024-09-19T04:48:53.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.249+0000 7fea05ab3740 1 -- 172.21.15.123:0/3074023851 wait complete. 2024-09-19T04:48:53.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.249+0000 7fea05ab3740 1 -- start start 2024-09-19T04:48:53.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.250+0000 7fea05ab3740 1 Processor -- start 2024-09-19T04:48:53.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.250+0000 7fea05ab3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560748350b90 0x560748358c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.250+0000 7fea05ab3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607483514b0 0x5607483591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.250+0000 7fea05ab3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56074835bd70 0x560748359710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.250+0000 7fea05ab3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56074835e9e0 con 0x5607483514b0 2024-09-19T04:48:53.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.250+0000 7fea05ab3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5607482720f0 con 0x560748350b90 2024-09-19T04:48:53.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.250+0000 7fea05ab3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56074835eb60 con 0x56074835bd70 2024-09-19T04:48:53.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.250+0000 7fea05ab3740 10 client.0 _ll_register_callbacks cb 0x5607482639e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:53.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.250+0000 7fe9fbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560748350b90 0x560748358c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.250+0000 7fea01000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56074835bd70 0x560748359710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.250+0000 7fe9f3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607483514b0 0x5607483591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.250+0000 7fe9fbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560748350b90 0x560748358c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:53318/0 (socket says 172.21.15.123:53318) 2024-09-19T04:48:53.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.250+0000 7fea01000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56074835bd70 0x560748359710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:41460/0 (socket says 172.21.15.123:41460) 2024-09-19T04:48:53.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.250+0000 7fe9fbe00640 1 -- 172.21.15.123:0/1699942274 learned_addr learned my addr 172.21.15.123:0/1699942274 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:53.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.251+0000 7fe9f3400640 1 -- 172.21.15.123:0/1699942274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56074835bd70 msgr2=0x560748359710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:53.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.251+0000 7fe9f3400640 1 --2- 172.21.15.123:0/1699942274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56074835bd70 0x560748359710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.251+0000 7fe9f3400640 1 -- 172.21.15.123:0/1699942274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560748350b90 msgr2=0x560748358c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:53.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.251+0000 7fe9f3400640 1 --2- 172.21.15.123:0/1699942274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560748350b90 0x560748358c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.251+0000 7fe9f3400640 1 -- 172.21.15.123:0/1699942274 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607480283c0 con 0x5607483514b0 2024-09-19T04:48:53.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.251+0000 7fea01000640 1 --2- 172.21.15.123:0/1699942274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56074835bd70 0x560748359710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:53.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.251+0000 7fe9fbe00640 1 --2- 172.21.15.123:0/1699942274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560748350b90 0x560748358c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:53.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.251+0000 7fe9f3400640 1 --2- 172.21.15.123:0/1699942274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607483514b0 0x5607483591d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40ed360 tx=0x7fe9e4030980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4926526fc31521 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:53.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.251+0000 7fe9ea000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:48:53.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.251+0000 7fe9ea000640 1 -- 172.21.15.123:0/1699942274 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40bba30 con 0x5607483514b0 2024-09-19T04:48:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.251+0000 7fe9ea000640 1 -- 172.21.15.123:0/1699942274 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fe9e40c48d0 con 0x5607483514b0 2024-09-19T04:48:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.251+0000 7fea05ab3740 1 -- 172.21.15.123:0/1699942274 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9f4019b10 con 0x5607483514b0 2024-09-19T04:48:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.252+0000 7fe9ea000640 1 -- 172.21.15.123:0/1699942274 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe9e40ed6e0 con 0x5607483514b0 2024-09-19T04:48:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.252+0000 7fea05ab3740 10 client.15696 fetch_fsmap learned FSMap version 31 2024-09-19T04:48:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.252+0000 7fea05ab3740 1 -- 172.21.15.123:0/1699942274 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56074836a6c0 con 0x5607483514b0 2024-09-19T04:48:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.252+0000 7fe9ea000640 1 -- 172.21.15.123:0/1699942274 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9f4019b10 con 0x5607483514b0 2024-09-19T04:48:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.252+0000 7fe9ea000640 1 -- 172.21.15.123:0/1699942274 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fe9e403a070 con 0x5607483514b0 2024-09-19T04:48:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.252+0000 7fea05ab3740 10 client.15696 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:48:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.252+0000 7fea05ab3740 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:48:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.252+0000 7fea05ab3740 20 client.15696 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:53.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.252+0000 7fea05ab3740 1 --2- 172.21.15.123:0/1699942274 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56074836bab0 0x56074836df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.252+0000 7fea05ab3740 4 client.15696 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:48:53.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.252+0000 7fea05ab3740 1 -- 172.21.15.123:0/1699942274 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607482c4950 con 0x56074836bab0 2024-09-19T04:48:53.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.253+0000 7fe9fbe00640 1 --2- 172.21.15.123:0/1699942274 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56074836bab0 0x56074836df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.253+0000 7fe9fbe00640 1 --2- 172.21.15.123:0/1699942274 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56074836bab0 0x56074836df30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3da1415b5542819 server_cookie=438208f30e2c3b16 in_seq=0 out_seq=0 2024-09-19T04:48:53.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.253+0000 7fe9ea000640 10 client.15696 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:53.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.254+0000 7fe9ea000640 1 -- 172.21.15.123:0/1699942274 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607482c4950 con 0x56074836bab0 2024-09-19T04:48:53.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.254+0000 7fe9ea000640 10 client.15696 handle_command_reply: tid=0 2024-09-19T04:48:53.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.254+0000 7fea05ab3740 1 client.15696 shutdown 2024-09-19T04:48:53.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.254+0000 7fea05ab3740 1 -- 172.21.15.123:0/1699942274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607483514b0 msgr2=0x5607483591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:53.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.254+0000 7fea05ab3740 1 --2- 172.21.15.123:0/1699942274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607483514b0 0x5607483591d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40ed360 tx=0x7fe9e4030980 comp rx=0 tx=0).stop 2024-09-19T04:48:53.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.254+0000 7fea05ab3740 1 -- 172.21.15.123:0/1699942274 shutdown_connections 2024-09-19T04:48:53.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.254+0000 7fea05ab3740 1 --2- 172.21.15.123:0/1699942274 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56074836bab0 0x56074836df30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.254+0000 7fea05ab3740 1 --2- 172.21.15.123:0/1699942274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56074835bd70 0x560748359710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.254+0000 7fea05ab3740 1 --2- 172.21.15.123:0/1699942274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607483514b0 0x5607483591d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.254+0000 7fea05ab3740 1 --2- 172.21.15.123:0/1699942274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560748350b90 0x560748358c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.254+0000 7fea05ab3740 1 -- 172.21.15.123:0/1699942274 >> 172.21.15.123:0/1699942274 conn(0x56074834d120 msgr2=0x560748352e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:53.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.254+0000 7fe9ea000640 0 client.15696 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:53.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.255+0000 7fea05ab3740 1 -- 172.21.15.123:0/1699942274 shutdown_connections 2024-09-19T04:48:53.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.255+0000 7fea05ab3740 1 -- 172.21.15.123:0/1699942274 wait complete. 2024-09-19T04:48:53.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.255+0000 7fea05ab3740 20 client.15696 trim_cache size 0 max 16384 2024-09-19T04:48:53.265 INFO:tasks.workunit.client.0.smithi123.stdout:checking for xfs/xfs.h... yes 2024-09-19T04:48:53.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.274+0000 7fea05ab3740 1 Processor -- start 2024-09-19T04:48:53.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.274+0000 7fea05ab3740 1 -- start start 2024-09-19T04:48:53.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.274+0000 7fea05ab3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560748350b90 0x560748353640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.274+0000 7fea05ab3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607483514b0 0x560748373840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.274+0000 7fea05ab3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56074835bd70 0x560748373d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.274+0000 7fea05ab3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc1af9e0 con 0x560748350b90 2024-09-19T04:48:53.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.274+0000 7fea05ab3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc1b1670 con 0x56074835bd70 2024-09-19T04:48:53.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.274+0000 7fea05ab3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc1b00b0 con 0x5607483514b0 2024-09-19T04:48:53.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.275+0000 7fe9fbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560748350b90 0x560748353640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.275+0000 7fe9f3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607483514b0 0x560748373840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.275+0000 7fe9f3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607483514b0 0x560748373840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:41462/0 (socket says 172.21.15.123:41462) 2024-09-19T04:48:53.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.275+0000 7fe9fbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560748350b90 0x560748353640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59752/0 (socket says 172.21.15.123:59752) 2024-09-19T04:48:53.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.275+0000 7fe9f3400640 1 -- 172.21.15.123:0/220491248 learned_addr learned my addr 172.21.15.123:0/220491248 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:53.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.275+0000 7fea01000640 1 --2- 172.21.15.123:0/220491248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56074835bd70 0x560748373d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.275+0000 7fe9f3400640 1 -- 172.21.15.123:0/220491248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56074835bd70 msgr2=0x560748373d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:53.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.275+0000 7fe9f3400640 1 --2- 172.21.15.123:0/220491248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56074835bd70 0x560748373d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.275+0000 7fe9f3400640 1 -- 172.21.15.123:0/220491248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560748350b90 msgr2=0x560748353640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:53.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.275+0000 7fe9f3400640 1 --2- 172.21.15.123:0/220491248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560748350b90 0x560748353640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.275+0000 7fe9f3400640 1 -- 172.21.15.123:0/220491248 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56074836e5b0 con 0x5607483514b0 2024-09-19T04:48:53.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.275+0000 7fe9fbe00640 1 --2- 172.21.15.123:0/220491248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560748350b90 0x560748353640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:53.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.275+0000 7fe9f3400640 1 --2- 172.21.15.123:0/220491248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607483514b0 0x560748373840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4005b60 tx=0x7fe9e4033f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=416f1810436e6e57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:53.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.275+0000 7fe9f0c00640 1 -- 172.21.15.123:0/220491248 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40f3070 con 0x5607483514b0 2024-09-19T04:48:53.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.276+0000 7fe9f0c00640 1 -- 172.21.15.123:0/220491248 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9e40f5060 con 0x5607483514b0 2024-09-19T04:48:53.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.276+0000 7fe9f0c00640 1 -- 172.21.15.123:0/220491248 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40bb850 con 0x5607483514b0 2024-09-19T04:48:53.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.276+0000 7fea05ab3740 1 -- 172.21.15.123:0/220491248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607483514b0 msgr2=0x560748373840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:53.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.276+0000 7fea05ab3740 1 --2- 172.21.15.123:0/220491248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607483514b0 0x560748373840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4005b60 tx=0x7fe9e4033f50 comp rx=0 tx=0).stop 2024-09-19T04:48:53.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.276+0000 7fea05ab3740 1 -- 172.21.15.123:0/220491248 shutdown_connections 2024-09-19T04:48:53.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.276+0000 7fea05ab3740 1 --2- 172.21.15.123:0/220491248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56074835bd70 0x560748373d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.276+0000 7fea05ab3740 1 --2- 172.21.15.123:0/220491248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607483514b0 0x560748373840 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.276+0000 7fea05ab3740 1 --2- 172.21.15.123:0/220491248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560748350b90 0x560748353640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.276+0000 7fea05ab3740 1 -- 172.21.15.123:0/220491248 >> 172.21.15.123:0/220491248 conn(0x56074834d120 msgr2=0x560748357d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:53.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.277+0000 7fea05ab3740 1 -- 172.21.15.123:0/220491248 shutdown_connections 2024-09-19T04:48:53.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.277+0000 7fea05ab3740 1 -- 172.21.15.123:0/220491248 wait complete. 2024-09-19T04:48:53.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.277+0000 7fea05ab3740 1 -- start start 2024-09-19T04:48:53.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.277+0000 7fea05ab3740 1 Processor -- start 2024-09-19T04:48:53.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.278+0000 7fea05ab3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560748350b90 0x5607483ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.278+0000 7fea05ab3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607483514b0 0x5607483cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.278+0000 7fea05ab3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074835bd70 0x56074835f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.278+0000 7fea05ab3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc1b0ee0 con 0x56074835bd70 2024-09-19T04:48:53.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.278+0000 7fea05ab3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc1b0810 con 0x5607483514b0 2024-09-19T04:48:53.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.278+0000 7fea05ab3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc1ad9c0 con 0x560748350b90 2024-09-19T04:48:53.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.278+0000 7fea05ab3740 10 client.0 _ll_register_callbacks cb 0x7fe9e4005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:53.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.278+0000 7fea01000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074835bd70 0x56074835f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.278+0000 7fe9fbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560748350b90 0x5607483ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.278+0000 7fea01000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074835bd70 0x56074835f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59758/0 (socket says 172.21.15.123:59758) 2024-09-19T04:48:53.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.278+0000 7fea01000640 1 -- 172.21.15.123:0/4145012279 learned_addr learned my addr 172.21.15.123:0/4145012279 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:53.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.278+0000 7fe9fbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560748350b90 0x5607483ceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:41466/0 (socket says 172.21.15.123:41466) 2024-09-19T04:48:53.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.278+0000 7fe9f3400640 1 --2- 172.21.15.123:0/4145012279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607483514b0 0x5607483cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.279+0000 7fe9fbe00640 1 -- 172.21.15.123:0/4145012279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607483514b0 msgr2=0x5607483cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:53.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.279+0000 7fe9fbe00640 1 --2- 172.21.15.123:0/4145012279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607483514b0 0x5607483cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.279+0000 7fe9fbe00640 1 -- 172.21.15.123:0/4145012279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074835bd70 msgr2=0x56074835f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:53.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.279+0000 7fe9fbe00640 1 --2- 172.21.15.123:0/4145012279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074835bd70 0x56074835f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.279+0000 7fe9fbe00640 1 -- 172.21.15.123:0/4145012279 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56074834c9d0 con 0x560748350b90 2024-09-19T04:48:53.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.279+0000 7fe9f3400640 1 --2- 172.21.15.123:0/4145012279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607483514b0 0x5607483cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:53.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.279+0000 7fea01000640 1 --2- 172.21.15.123:0/4145012279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074835bd70 0x56074835f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:53.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.279+0000 7fe9fbe00640 1 --2- 172.21.15.123:0/4145012279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560748350b90 0x5607483ceb70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40ba1a0 tx=0x7fe9ec004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cf63e40c10b5c47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:53.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.279+0000 7fe9ea000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:48:53.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.279+0000 7fe9ea000640 1 -- 172.21.15.123:0/4145012279 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec035070 con 0x560748350b90 2024-09-19T04:48:53.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.279+0000 7fe9ea000640 1 -- 172.21.15.123:0/4145012279 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fe9ec031660 con 0x560748350b90 2024-09-19T04:48:53.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.279+0000 7fea05ab3740 1 -- 172.21.15.123:0/4145012279 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9e40f5060 con 0x560748350b90 2024-09-19T04:48:53.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.280+0000 7fe9ea000640 1 -- 172.21.15.123:0/4145012279 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe9ec00de60 con 0x560748350b90 2024-09-19T04:48:53.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.280+0000 7fea05ab3740 10 client.25753 fetch_fsmap learned FSMap version 31 2024-09-19T04:48:53.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.280+0000 7fea05ab3740 1 -- 172.21.15.123:0/4145012279 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56074834e1d0 con 0x560748350b90 2024-09-19T04:48:53.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.280+0000 7fe9ea000640 1 -- 172.21.15.123:0/4145012279 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9e40f5060 con 0x560748350b90 2024-09-19T04:48:53.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.280+0000 7fe9ea000640 1 -- 172.21.15.123:0/4145012279 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fe9ec00b040 con 0x560748350b90 2024-09-19T04:48:53.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.280+0000 7fea05ab3740 10 client.25753 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:48:53.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.280+0000 7fea05ab3740 10 client.25753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:48:53.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.280+0000 7fea05ab3740 20 client.25753 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:53.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.280+0000 7fea05ab3740 1 --2- 172.21.15.123:0/4145012279 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5607483d6730 0x5607483d8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:53.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.280+0000 7fea05ab3740 4 client.25753 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:48:53.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.280+0000 7fea05ab3740 1 -- 172.21.15.123:0/4145012279 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5607482bee60 con 0x5607483d6730 2024-09-19T04:48:53.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.280+0000 7fe9f3400640 1 --2- 172.21.15.123:0/4145012279 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5607483d6730 0x5607483d8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:53.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.281+0000 7fe9ea000640 10 client.25753 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:53.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.281+0000 7fe9f3400640 1 --2- 172.21.15.123:0/4145012279 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5607483d6730 0x5607483d8b50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d009f05a38e0ede6 server_cookie=516399747e525342 in_seq=0 out_seq=0 2024-09-19T04:48:53.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.281+0000 7fe9ea000640 1 -- 172.21.15.123:0/4145012279 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5607482bee60 con 0x5607483d6730 2024-09-19T04:48:53.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.281+0000 7fe9ea000640 10 client.25753 handle_command_reply: tid=0 2024-09-19T04:48:53.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.281+0000 7fea05ab3740 1 client.25753 shutdown 2024-09-19T04:48:53.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.282+0000 7fea05ab3740 1 -- 172.21.15.123:0/4145012279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560748350b90 msgr2=0x5607483ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:53.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.282+0000 7fea05ab3740 1 --2- 172.21.15.123:0/4145012279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560748350b90 0x5607483ceb70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40ba1a0 tx=0x7fe9ec004a40 comp rx=0 tx=0).stop 2024-09-19T04:48:53.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.282+0000 7fea05ab3740 1 -- 172.21.15.123:0/4145012279 shutdown_connections 2024-09-19T04:48:53.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.282+0000 7fea05ab3740 1 --2- 172.21.15.123:0/4145012279 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5607483d6730 0x5607483d8b50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.282+0000 7fea05ab3740 1 --2- 172.21.15.123:0/4145012279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074835bd70 0x56074835f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.282+0000 7fea05ab3740 1 --2- 172.21.15.123:0/4145012279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607483514b0 0x5607483cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.282+0000 7fea05ab3740 1 --2- 172.21.15.123:0/4145012279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560748350b90 0x5607483ceb70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.282+0000 7fe9ea000640 0 client.25753 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:48:53.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.282+0000 7fea05ab3740 1 -- 172.21.15.123:0/4145012279 >> 172.21.15.123:0/4145012279 conn(0x56074834d120 msgr2=0x56074834b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:53.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.282+0000 7fea05ab3740 1 -- 172.21.15.123:0/4145012279 shutdown_connections 2024-09-19T04:48:53.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.282+0000 7fea05ab3740 1 -- 172.21.15.123:0/4145012279 wait complete. 2024-09-19T04:48:53.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.282+0000 7fea05ab3740 20 client.25753 trim_cache size 0 max 16384 2024-09-19T04:48:53.288 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:48:53.288 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-09-19T04:48:53.288 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-09-19T04:48:53.288 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:48:53.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.283+0000 7fea03000640 1 -- 172.21.15.123:0/3911817622 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9e00a3430 msgr2=0x7fe9e00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:53.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.283+0000 7fea03000640 1 --2- 172.21.15.123:0/3911817622 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9e00a3430 0x7fe9e00a58f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x56074834df80 tx=0x7fe9ec010040 comp rx=0 tx=0).stop 2024-09-19T04:48:53.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.283+0000 7fea03000640 1 -- 172.21.15.123:0/3911817622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9fc1049a0 msgr2=0x7fe9fc19abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:53.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.283+0000 7fea03000640 1 --2- 172.21.15.123:0/3911817622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9fc1049a0 0x7fe9fc19abe0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4035040 tx=0x7fe9e4033b60 comp rx=0 tx=0).stop 2024-09-19T04:48:53.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.283+0000 7fea03000640 1 -- 172.21.15.123:0/3911817622 shutdown_connections 2024-09-19T04:48:53.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.283+0000 7fea03000640 1 --2- 172.21.15.123:0/3911817622 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9e00a3430 0x7fe9e00a58f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.283+0000 7fea03000640 1 --2- 172.21.15.123:0/3911817622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9fc109120 0x7fe9fc19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.283+0000 7fea03000640 1 --2- 172.21.15.123:0/3911817622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9fc1049a0 0x7fe9fc19abe0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.283+0000 7fea03000640 1 --2- 172.21.15.123:0/3911817622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc103ff0 0x7fe9fc19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:53.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.283+0000 7fea03000640 1 -- 172.21.15.123:0/3911817622 >> 172.21.15.123:0/3911817622 conn(0x7fe9fc0ffa90 msgr2=0x7fe9fc100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:53.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.283+0000 7fea03000640 1 -- 172.21.15.123:0/3911817622 shutdown_connections 2024-09-19T04:48:53.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:53.283+0000 7fea03000640 1 -- 172.21.15.123:0/3911817622 wait complete. 2024-09-19T04:48:53.359 INFO:tasks.workunit.client.0.smithi123.stdout:checking for xfs/libxfs.h... no 2024-09-19T04:48:53.416 INFO:tasks.workunit.client.0.smithi123.stdout:checking whether xlog_assign_lsn is declared... no 2024-09-19T04:48:53.487 INFO:tasks.workunit.client.0.smithi123.stdout:checking for xfs/xqm.h... yes 2024-09-19T04:48:53.585 INFO:tasks.workunit.client.0.smithi123.stdout:checking xfsctl from xfs/xfs.h... ok 2024-09-19T04:48:53.651 INFO:tasks.workunit.client.0.smithi123.stdout:checking xfs/handle.h usability... yes 2024-09-19T04:48:53.681 INFO:tasks.workunit.client.0.smithi123.stdout:checking xfs/handle.h presence... yes 2024-09-19T04:48:53.682 INFO:tasks.workunit.client.0.smithi123.stdout:checking for xfs/handle.h... yes 2024-09-19T04:48:53.776 INFO:tasks.workunit.client.0.smithi123.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-09-19T04:48:53.853 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/acl.h usability... yes 2024-09-19T04:48:53.881 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/acl.h presence... yes 2024-09-19T04:48:53.881 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/acl.h... yes 2024-09-19T04:48:53.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:53 smithi123 ceph-mon[30511]: pgmap v68: 97 pgs: 97 active+clean; 59 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 438 op/s 2024-09-19T04:48:53.977 INFO:tasks.workunit.client.0.smithi123.stdout:checking acl/libacl.h usability... yes 2024-09-19T04:48:53.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:53 smithi193 ceph-mon[35592]: pgmap v68: 97 pgs: 97 active+clean; 59 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 438 op/s 2024-09-19T04:48:54.006 INFO:tasks.workunit.client.0.smithi123.stdout:checking acl/libacl.h presence... yes 2024-09-19T04:48:54.006 INFO:tasks.workunit.client.0.smithi123.stdout:checking for acl/libacl.h... yes 2024-09-19T04:48:54.088 INFO:tasks.workunit.client.0.smithi123.stdout:checking for acl_init in -lacl... yes 2024-09-19T04:48:54.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:53 smithi183 ceph-mon[35333]: pgmap v68: 97 pgs: 97 active+clean; 59 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 438 op/s 2024-09-19T04:48:54.156 INFO:tasks.workunit.client.0.smithi123.stdout:checking gdbm-ndbm.h usability... no 2024-09-19T04:48:54.200 INFO:tasks.workunit.client.0.smithi123.stdout:checking gdbm-ndbm.h presence... no 2024-09-19T04:48:54.200 INFO:tasks.workunit.client.0.smithi123.stdout:checking for gdbm-ndbm.h... no 2024-09-19T04:48:54.267 INFO:tasks.workunit.client.0.smithi123.stdout:checking gdbm/ndbm.h usability... no 2024-09-19T04:48:54.311 INFO:tasks.workunit.client.0.smithi123.stdout:checking gdbm/ndbm.h presence... no 2024-09-19T04:48:54.311 INFO:tasks.workunit.client.0.smithi123.stdout:checking for gdbm/ndbm.h... no 2024-09-19T04:48:54.352 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:48:54.352 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:48:54.381 INFO:tasks.workunit.client.0.smithi123.stdout:checking ndbm.h usability... no 2024-09-19T04:48:54.425 INFO:tasks.workunit.client.0.smithi123.stdout:checking ndbm.h presence... no 2024-09-19T04:48:54.425 INFO:tasks.workunit.client.0.smithi123.stdout:checking for ndbm.h... no 2024-09-19T04:48:54.487 INFO:tasks.workunit.client.0.smithi123.stdout:checking libaio.h usability... yes 2024-09-19T04:48:54.517 INFO:tasks.workunit.client.0.smithi123.stdout:checking libaio.h presence... yes 2024-09-19T04:48:54.517 INFO:tasks.workunit.client.0.smithi123.stdout:checking for libaio.h... yes 2024-09-19T04:48:54.587 INFO:tasks.workunit.client.0.smithi123.stdout:checking liburing.h usability... no 2024-09-19T04:48:54.632 INFO:tasks.workunit.client.0.smithi123.stdout:checking liburing.h presence... no 2024-09-19T04:48:54.633 INFO:tasks.workunit.client.0.smithi123.stdout:checking for liburing.h... no 2024-09-19T04:48:54.701 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/capability.h usability... no 2024-09-19T04:48:54.747 INFO:tasks.workunit.client.0.smithi123.stdout:checking sys/capability.h presence... no 2024-09-19T04:48:54.747 INFO:tasks.workunit.client.0.smithi123.stdout:checking for sys/capability.h... no 2024-09-19T04:48:54.773 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:48:54.815 INFO:tasks.workunit.client.0.smithi123.stdout:checking linux/fiemap.h usability... yes 2024-09-19T04:48:54.843 INFO:tasks.workunit.client.0.smithi123.stdout:checking linux/fiemap.h presence... yes 2024-09-19T04:48:54.843 INFO:tasks.workunit.client.0.smithi123.stdout:checking for linux/fiemap.h... yes 2024-09-19T04:48:54.921 INFO:tasks.workunit.client.0.smithi123.stdout:checking for fallocate... yes 2024-09-19T04:48:55.021 INFO:tasks.workunit.client.0.smithi123.stdout:checking for open_by_handle_at... yes 2024-09-19T04:48:55.096 INFO:tasks.workunit.client.0.smithi123.stdout:checking linux/fs.h usability... yes 2024-09-19T04:48:55.126 INFO:tasks.workunit.client.0.smithi123.stdout:checking linux/fs.h presence... yes 2024-09-19T04:48:55.127 INFO:tasks.workunit.client.0.smithi123.stdout:checking for linux/fs.h... yes 2024-09-19T04:48:55.202 INFO:tasks.workunit.client.0.smithi123.stdout:checking btrfsutil.h usability... no 2024-09-19T04:48:55.244 INFO:tasks.workunit.client.0.smithi123.stdout:checking btrfsutil.h presence... no 2024-09-19T04:48:55.244 INFO:tasks.workunit.client.0.smithi123.stdout:checking for btrfsutil.h... no 2024-09-19T04:48:55.366 INFO:tasks.workunit.client.0.smithi123.stdout:checking for copy_file_range... yes 2024-09-19T04:48:55.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.433+0000 7fa2f3200640 1 -- 172.21.15.123:0/2499867633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2ec1079d0 msgr2=0x7fa2ec107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:55.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.433+0000 7fa2f3200640 1 --2- 172.21.15.123:0/2499867633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2ec1079d0 0x7fa2ec107db0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009930 tx=0x7fa2e002a440 comp rx=0 tx=0).stop 2024-09-19T04:48:55.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.433+0000 7fa2f3200640 1 -- 172.21.15.123:0/2499867633 shutdown_connections 2024-09-19T04:48:55.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.433+0000 7fa2f3200640 1 --2- 172.21.15.123:0/2499867633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2ec102370 0x7fa2ec10e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:55.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.433+0000 7fa2f3200640 1 --2- 172.21.15.123:0/2499867633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ec1019d0 0x7fa2ec101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:55.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.433+0000 7fa2f3200640 1 --2- 172.21.15.123:0/2499867633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2ec1079d0 0x7fa2ec107db0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:55.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.433+0000 7fa2f3200640 1 -- 172.21.15.123:0/2499867633 >> 172.21.15.123:0/2499867633 conn(0x7fa2ec0fd710 msgr2=0x7fa2ec0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:55.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.434+0000 7fa2f3200640 1 -- 172.21.15.123:0/2499867633 shutdown_connections 2024-09-19T04:48:55.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.434+0000 7fa2f3200640 1 -- 172.21.15.123:0/2499867633 wait complete. 2024-09-19T04:48:55.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.435+0000 7fa2f3200640 1 Processor -- start 2024-09-19T04:48:55.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.435+0000 7fa2f3200640 1 -- start start 2024-09-19T04:48:55.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.436+0000 7fa2f3200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ec1019d0 0x7fa2ec19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:55.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.436+0000 7fa2f3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2ec102370 0x7fa2ec19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:55.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.436+0000 7fa2f3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2ec1079d0 0x7fa2ec19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:55.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.436+0000 7fa2f3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec111e20 con 0x7fa2ec102370 2024-09-19T04:48:55.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.436+0000 7fa2f3200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec111ca0 con 0x7fa2ec1079d0 2024-09-19T04:48:55.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.436+0000 7fa2f3200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec111fa0 con 0x7fa2ec1019d0 2024-09-19T04:48:55.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.436+0000 7fa2f1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ec1019d0 0x7fa2ec19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:55.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.436+0000 7fa2f1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2ec102370 0x7fa2ec19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:55.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.436+0000 7fa2f1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ec1019d0 0x7fa2ec19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:41496/0 (socket says 172.21.15.123:41496) 2024-09-19T04:48:55.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.436+0000 7fa2f1e00640 1 -- 172.21.15.123:0/1502004977 learned_addr learned my addr 172.21.15.123:0/1502004977 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:55.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.436+0000 7fa2f2800640 1 --2- 172.21.15.123:0/1502004977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2ec1079d0 0x7fa2ec19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:55.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.436+0000 7fa2f1e00640 1 -- 172.21.15.123:0/1502004977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2ec1079d0 msgr2=0x7fa2ec19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:55.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.436+0000 7fa2f1e00640 1 --2- 172.21.15.123:0/1502004977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2ec1079d0 0x7fa2ec19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:55.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.436+0000 7fa2f1e00640 1 -- 172.21.15.123:0/1502004977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2ec102370 msgr2=0x7fa2ec19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:55.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.436+0000 7fa2f1e00640 1 --2- 172.21.15.123:0/1502004977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2ec102370 0x7fa2ec19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:55.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.436+0000 7fa2f1e00640 1 -- 172.21.15.123:0/1502004977 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ec19f4a0 con 0x7fa2ec1019d0 2024-09-19T04:48:55.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.437+0000 7fa2f1e00640 1 --2- 172.21.15.123:0/1502004977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ec1019d0 0x7fa2ec19a4f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa2e002a9b0 tx=0x7fa2e0003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:55.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.437+0000 7fa2e6a00640 1 -- 172.21.15.123:0/1502004977 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e003f070 con 0x7fa2ec1019d0 2024-09-19T04:48:55.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.437+0000 7fa2f3200640 1 -- 172.21.15.123:0/1502004977 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ec19f730 con 0x7fa2ec1019d0 2024-09-19T04:48:55.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.437+0000 7fa2e6a00640 1 -- 172.21.15.123:0/1502004977 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2e0031c90 con 0x7fa2ec1019d0 2024-09-19T04:48:55.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.437+0000 7fa2e6a00640 1 -- 172.21.15.123:0/1502004977 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0035030 con 0x7fa2ec1019d0 2024-09-19T04:48:55.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.437+0000 7fa2f3200640 1 -- 172.21.15.123:0/1502004977 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ec1a7070 con 0x7fa2ec1019d0 2024-09-19T04:48:55.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.439+0000 7fa2f3200640 1 -- 172.21.15.123:0/1502004977 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2ec101e30 con 0x7fa2ec1019d0 2024-09-19T04:48:55.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.441+0000 7fa2e6a00640 1 -- 172.21.15.123:0/1502004977 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa2e0042070 con 0x7fa2ec1019d0 2024-09-19T04:48:55.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.442+0000 7fa2e6a00640 1 --2- 172.21.15.123:0/1502004977 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa2d40a3400 0x7fa2d40a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:55.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.443+0000 7fa2e6a00640 1 -- 172.21.15.123:0/1502004977 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fa2e00f0830 con 0x7fa2ec1019d0 2024-09-19T04:48:55.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.443+0000 7fa2f1400640 1 --2- 172.21.15.123:0/1502004977 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa2d40a3400 0x7fa2d40a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:55.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.444+0000 7fa2f1400640 1 --2- 172.21.15.123:0/1502004977 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa2d40a3400 0x7fa2d40a58c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec10ed30 tx=0x7fa2dc00b410 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:55.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.446+0000 7fa2e6a00640 1 -- 172.21.15.123:0/1502004977 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa2e00bd1e0 con 0x7fa2ec1019d0 2024-09-19T04:48:55.469 INFO:tasks.workunit.client.0.smithi123.stdout:checking for SEEK_DATA... yes 2024-09-19T04:48:55.582 INFO:tasks.workunit.client.0.smithi123.stdout:checking for BMV_OF_SHARED... yes 2024-09-19T04:48:55.664 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:55 smithi123 ceph-mon[30511]: pgmap v69: 97 pgs: 97 active+clean; 61 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 455 op/s 2024-09-19T04:48:55.664 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:55 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:48:55.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.678+0000 7fa2f3200640 1 -- 172.21.15.123:0/1502004977 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2ec1a7240 con 0x7fa2ec1019d0 2024-09-19T04:48:55.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.679+0000 7fa2e6a00640 1 -- 172.21.15.123:0/1502004977 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+12002 (secure 0 0 0) 0x7fa2e00bcae0 con 0x7fa2ec1019d0 2024-09-19T04:48:55.681 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:48:55.684 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":31,"btime":"2024-09-19T04:48:34:620594+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:48:33.828912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24529,24505,24539,24500,24499]},"id":1}]} 2024-09-19T04:48:55.684 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 31 2024-09-19T04:48:55.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.683+0000 7fa2f3200640 1 -- 172.21.15.123:0/1502004977 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa2d40a3400 msgr2=0x7fa2d40a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:55.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.683+0000 7fa2f3200640 1 --2- 172.21.15.123:0/1502004977 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa2d40a3400 0x7fa2d40a58c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec10ed30 tx=0x7fa2dc00b410 comp rx=0 tx=0).stop 2024-09-19T04:48:55.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.683+0000 7fa2f3200640 1 -- 172.21.15.123:0/1502004977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ec1019d0 msgr2=0x7fa2ec19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:55.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.683+0000 7fa2f3200640 1 --2- 172.21.15.123:0/1502004977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ec1019d0 0x7fa2ec19a4f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa2e002a9b0 tx=0x7fa2e0003b90 comp rx=0 tx=0).stop 2024-09-19T04:48:55.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.683+0000 7fa2f3200640 1 -- 172.21.15.123:0/1502004977 shutdown_connections 2024-09-19T04:48:55.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.683+0000 7fa2f3200640 1 --2- 172.21.15.123:0/1502004977 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa2d40a3400 0x7fa2d40a58c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:55.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.683+0000 7fa2f3200640 1 --2- 172.21.15.123:0/1502004977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2ec1079d0 0x7fa2ec19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:55.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.683+0000 7fa2f3200640 1 --2- 172.21.15.123:0/1502004977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2ec102370 0x7fa2ec19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:55.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.683+0000 7fa2f3200640 1 --2- 172.21.15.123:0/1502004977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ec1019d0 0x7fa2ec19a4f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:55.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.683+0000 7fa2f3200640 1 -- 172.21.15.123:0/1502004977 >> 172.21.15.123:0/1502004977 conn(0x7fa2ec0fd710 msgr2=0x7fa2ec0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:55.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.683+0000 7fa2f3200640 1 -- 172.21.15.123:0/1502004977 shutdown_connections 2024-09-19T04:48:55.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:55.684+0000 7fa2f3200640 1 -- 172.21.15.123:0/1502004977 wait complete. 2024-09-19T04:48:55.686 INFO:tasks.workunit.client.0.smithi123.stdout:checking for nftw... yes 2024-09-19T04:48:55.796 INFO:tasks.workunit.client.0.smithi123.stdout:checking for RLIMIT_NOFILE... yes 2024-09-19T04:48:55.912 INFO:tasks.workunit.client.0.smithi123.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-09-19T04:48:55.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:55 smithi193 ceph-mon[35592]: pgmap v69: 97 pgs: 97 active+clean; 61 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 455 op/s 2024-09-19T04:48:55.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:55 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:48:55.995 INFO:tasks.workunit.client.0.smithi123.stdout:checking for FICLONE... yes 2024-09-19T04:48:56.081 INFO:tasks.workunit.client.0.smithi123.stdout:checking for renameat2... yes 2024-09-19T04:48:56.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:55 smithi183 ceph-mon[35333]: pgmap v69: 97 pgs: 97 active+clean; 61 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 455 op/s 2024-09-19T04:48:56.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:55 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:48:56.160 INFO:tasks.workunit.client.0.smithi123.stdout:checking for reallocarray... yes 2024-09-19T04:48:56.273 INFO:tasks.workunit.client.0.smithi123.stdout:checking for struct mount_attr... yes 2024-09-19T04:48:56.391 INFO:tasks.workunit.client.0.smithi123.stdout:checking for struct btrfs_qgroup_limit... yes 2024-09-19T04:48:56.504 INFO:tasks.workunit.client.0.smithi123.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-09-19T04:48:56.570 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:48:56.615 INFO:tasks.workunit.client.0.smithi123.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-09-19T04:48:56.726 INFO:tasks.workunit.client.0.smithi123.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-09-19T04:48:56.836 INFO:tasks.workunit.client.0.smithi123.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-09-19T04:48:56.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:56 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1502004977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:48:56.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:56 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:48:56.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:56 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:48:56.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:56 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:48:56.946 INFO:tasks.workunit.client.0.smithi123.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-09-19T04:48:56.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:56 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1502004977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:48:56.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:56 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:48:56.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:56 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:48:56.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:56 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:48:57.001 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:56 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1502004977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:48:57.001 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:56 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:48:57.001 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:56 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:48:57.001 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:56 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:48:57.004 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:48:57.061 INFO:tasks.workunit.client.0.smithi123.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-09-19T04:48:57.136 INFO:tasks.workunit.client.0.smithi123.stdout:checking linux/btrfs.h usability... yes 2024-09-19T04:48:57.163 INFO:tasks.workunit.client.0.smithi123.stdout:checking linux/btrfs.h presence... yes 2024-09-19T04:48:57.164 INFO:tasks.workunit.client.0.smithi123.stdout:checking for linux/btrfs.h... yes 2024-09-19T04:48:57.269 INFO:tasks.workunit.client.0.smithi123.stdout:checking linux/btrfs_tree.h usability... yes 2024-09-19T04:48:57.296 INFO:tasks.workunit.client.0.smithi123.stdout:checking linux/btrfs_tree.h presence... yes 2024-09-19T04:48:57.296 INFO:tasks.workunit.client.0.smithi123.stdout:checking for linux/btrfs_tree.h... yes 2024-09-19T04:48:57.372 INFO:tasks.workunit.client.0.smithi123.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-09-19T04:48:57.438 INFO:tasks.workunit.client.0.smithi123.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-09-19T04:48:57.470 INFO:tasks.workunit.client.0.smithi123.stdout:configure: creating ./config.status 2024-09-19T04:48:57.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.638+0000 7fe940c00640 1 -- 172.21.15.123:0/2542253443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe93c076720 msgr2=0x7fe93c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:57.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.638+0000 7fe940c00640 1 --2- 172.21.15.123:0/2542253443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe93c076720 0x7fe93c076b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009860 tx=0x7fe92c02a340 comp rx=0 tx=0).stop 2024-09-19T04:48:57.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.638+0000 7fe940c00640 1 -- 172.21.15.123:0/2542253443 shutdown_connections 2024-09-19T04:48:57.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.638+0000 7fe940c00640 1 --2- 172.21.15.123:0/2542253443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe93c112a50 0x7fe93c114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.638+0000 7fe940c00640 1 --2- 172.21.15.123:0/2542253443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe93c077040 0x7fe93c112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.638+0000 7fe940c00640 1 --2- 172.21.15.123:0/2542253443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe93c076720 0x7fe93c076b00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.638+0000 7fe940c00640 1 -- 172.21.15.123:0/2542253443 >> 172.21.15.123:0/2542253443 conn(0x7fe93c0ffa50 msgr2=0x7fe93c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:57.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.638+0000 7fe940c00640 1 -- 172.21.15.123:0/2542253443 shutdown_connections 2024-09-19T04:48:57.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.638+0000 7fe940c00640 1 -- 172.21.15.123:0/2542253443 wait complete. 2024-09-19T04:48:57.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.639+0000 7fe940c00640 1 Processor -- start 2024-09-19T04:48:57.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.639+0000 7fe940c00640 1 -- start start 2024-09-19T04:48:57.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.640+0000 7fe940c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe93c076720 0x7fe93c1a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.640+0000 7fe940c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe93c077040 0x7fe93c1a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.640+0000 7fe940c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe93c112a50 0x7fe93c19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.640+0000 7fe940c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1185c0 con 0x7fe93c076720 2024-09-19T04:48:57.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.640+0000 7fe940c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c118440 con 0x7fe93c077040 2024-09-19T04:48:57.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.640+0000 7fe940c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c118740 con 0x7fe93c112a50 2024-09-19T04:48:57.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.640+0000 7fe939e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe93c076720 0x7fe93c1a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.640+0000 7fe939e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe93c076720 0x7fe93c1a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59800/0 (socket says 172.21.15.123:59800) 2024-09-19T04:48:57.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.640+0000 7fe939e00640 1 -- 172.21.15.123:0/3356841511 learned_addr learned my addr 172.21.15.123:0/3356841511 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:57.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.640+0000 7fe939400640 1 --2- 172.21.15.123:0/3356841511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe93c077040 0x7fe93c1a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.640+0000 7fe93a800640 1 --2- 172.21.15.123:0/3356841511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe93c112a50 0x7fe93c19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.641+0000 7fe939400640 1 -- 172.21.15.123:0/3356841511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe93c112a50 msgr2=0x7fe93c19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:57.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.641+0000 7fe939400640 1 --2- 172.21.15.123:0/3356841511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe93c112a50 0x7fe93c19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.641+0000 7fe939400640 1 -- 172.21.15.123:0/3356841511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe93c076720 msgr2=0x7fe93c1a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:57.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.641+0000 7fe939400640 1 --2- 172.21.15.123:0/3356841511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe93c076720 0x7fe93c1a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.641+0000 7fe939400640 1 -- 172.21.15.123:0/3356841511 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c19f020 con 0x7fe93c077040 2024-09-19T04:48:57.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.641+0000 7fe939e00640 1 --2- 172.21.15.123:0/3356841511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe93c076720 0x7fe93c1a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:57.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.641+0000 7fe93a800640 1 --2- 172.21.15.123:0/3356841511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe93c112a50 0x7fe93c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:48:57.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.641+0000 7fe939400640 1 --2- 172.21.15.123:0/3356841511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe93c077040 0x7fe93c1a4c70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe928005fd0 tx=0x7fe92800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:57.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.641+0000 7fe932a00640 1 -- 172.21.15.123:0/3356841511 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928009c30 con 0x7fe93c077040 2024-09-19T04:48:57.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.641+0000 7fe932a00640 1 -- 172.21.15.123:0/3356841511 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe928009dd0 con 0x7fe93c077040 2024-09-19T04:48:57.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.641+0000 7fe932a00640 1 -- 172.21.15.123:0/3356841511 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928010460 con 0x7fe93c077040 2024-09-19T04:48:57.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.641+0000 7fe940c00640 1 -- 172.21.15.123:0/3356841511 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93c19f310 con 0x7fe93c077040 2024-09-19T04:48:57.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.642+0000 7fe940c00640 1 -- 172.21.15.123:0/3356841511 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93c1ab600 con 0x7fe93c077040 2024-09-19T04:48:57.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.645+0000 7fe943816740 1 Processor -- start 2024-09-19T04:48:57.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.645+0000 7fe932a00640 1 -- 172.21.15.123:0/3356841511 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe928003ef0 con 0x7fe93c077040 2024-09-19T04:48:57.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.645+0000 7fe943816740 1 -- start start 2024-09-19T04:48:57.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.645+0000 7fe943816740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d6c70 0x55b0216d7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.645+0000 7fe943816740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 0x55b0216e1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.645+0000 7fe943816740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216e1e50 0x55b0216e4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.645+0000 7fe943816740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b02163d3e0 con 0x55b0216e1e50 2024-09-19T04:48:57.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.645+0000 7fe943816740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b0215e4d80 con 0x55b0216d7590 2024-09-19T04:48:57.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.645+0000 7fe943816740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b0216393a0 con 0x55b0216d6c70 2024-09-19T04:48:57.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.646+0000 7fe939400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216e1e50 0x55b0216e4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.646+0000 7fe93a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d6c70 0x55b0216d7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.646+0000 7fe939e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 0x55b0216e1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.646+0000 7fe93a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d6c70 0x55b0216d7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:41522/0 (socket says 172.21.15.123:41522) 2024-09-19T04:48:57.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.646+0000 7fe93a800640 1 -- 172.21.15.123:0/2537129507 learned_addr learned my addr 172.21.15.123:0/2537129507 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.646+0000 7fe93a800640 1 -- 172.21.15.123:0/2537129507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 msgr2=0x55b0216e1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.646+0000 7fe93a800640 1 --2- 172.21.15.123:0/2537129507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 0x55b0216e1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.646+0000 7fe93a800640 1 -- 172.21.15.123:0/2537129507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216e1e50 msgr2=0x55b0216e4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.646+0000 7fe93a800640 1 --2- 172.21.15.123:0/2537129507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216e1e50 0x55b0216e4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.646+0000 7fe93a800640 1 -- 172.21.15.123:0/2537129507 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b021603ae0 con 0x55b0216d6c70 2024-09-19T04:48:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.646+0000 7fe939400640 1 --2- 172.21.15.123:0/2537129507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216e1e50 0x55b0216e4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.647+0000 7fe93a800640 1 --2- 172.21.15.123:0/2537129507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d6c70 0x55b0216d7050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe934009f80 tx=0x7fe93400f890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a11e57031974d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.647+0000 7fe932a00640 1 --2- 172.21.15.123:0/3356841511 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9200a33b0 0x7fe9200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.647+0000 7fe927000640 1 -- 172.21.15.123:0/2537129507 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934004b20 con 0x55b0216d6c70 2024-09-19T04:48:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.647+0000 7fe927000640 1 -- 172.21.15.123:0/2537129507 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe934004cc0 con 0x55b0216d6c70 2024-09-19T04:48:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.647+0000 7fe927000640 1 -- 172.21.15.123:0/2537129507 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934005550 con 0x55b0216d6c70 2024-09-19T04:48:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.647+0000 7fe932a00640 1 -- 172.21.15.123:0/3356841511 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fe9280cf910 con 0x7fe93c077040 2024-09-19T04:48:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.647+0000 7fe939e00640 1 --2- 172.21.15.123:0/3356841511 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9200a33b0 0x7fe9200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.648+0000 7fe943816740 1 -- 172.21.15.123:0/2537129507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d6c70 msgr2=0x55b0216d7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.648+0000 7fe943816740 1 --2- 172.21.15.123:0/2537129507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d6c70 0x55b0216d7050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe934009f80 tx=0x7fe93400f890 comp rx=0 tx=0).stop 2024-09-19T04:48:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.648+0000 7fe939e00640 1 --2- 172.21.15.123:0/3356841511 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9200a33b0 0x7fe9200a5870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe92c031000 tx=0x7fe92c0056d0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.648+0000 7fe943816740 1 -- 172.21.15.123:0/2537129507 shutdown_connections 2024-09-19T04:48:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.648+0000 7fe943816740 1 --2- 172.21.15.123:0/2537129507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216e1e50 0x55b0216e4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.648+0000 7fe943816740 1 --2- 172.21.15.123:0/2537129507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 0x55b0216e1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.648+0000 7fe943816740 1 --2- 172.21.15.123:0/2537129507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d6c70 0x55b0216d7050 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.648+0000 7fe943816740 1 -- 172.21.15.123:0/2537129507 >> 172.21.15.123:0/2537129507 conn(0x55b0216d3200 msgr2=0x55b0216d35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.648+0000 7fe943816740 1 -- 172.21.15.123:0/2537129507 shutdown_connections 2024-09-19T04:48:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.648+0000 7fe943816740 1 -- 172.21.15.123:0/2537129507 wait complete. 2024-09-19T04:48:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.649+0000 7fe943816740 1 -- start start 2024-09-19T04:48:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.649+0000 7fe943816740 1 Processor -- start 2024-09-19T04:48:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.650+0000 7fe943816740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d6c70 0x55b0216decb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.650+0000 7fe943816740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d7590 0x55b0216df1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.650+0000 7fe943816740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216e1e50 0x55b0216df730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.650+0000 7fe943816740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b0216e4a20 con 0x55b0216e1e50 2024-09-19T04:48:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.650+0000 7fe943816740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b0215f80f0 con 0x55b0216d6c70 2024-09-19T04:48:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.650+0000 7fe943816740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b0216e4ba0 con 0x55b0216d7590 2024-09-19T04:48:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.650+0000 7fe943816740 10 client.0 _ll_register_callbacks cb 0x55b0215dfbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.650+0000 7fe939400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d7590 0x55b0216df1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.650+0000 7fe939400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d7590 0x55b0216df1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:41536/0 (socket says 172.21.15.123:41536) 2024-09-19T04:48:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.650+0000 7fe939400640 1 -- 172.21.15.123:0/2214501441 learned_addr learned my addr 172.21.15.123:0/2214501441 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.650+0000 7fe93a800640 1 --2- 172.21.15.123:0/2214501441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216e1e50 0x55b0216df730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.650+0000 7fe939e00640 1 --2- 172.21.15.123:0/2214501441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d6c70 0x55b0216decb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.650+0000 7fe939400640 1 -- 172.21.15.123:0/2214501441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d6c70 msgr2=0x55b0216decb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.650+0000 7fe939400640 1 --2- 172.21.15.123:0/2214501441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d6c70 0x55b0216decb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.650+0000 7fe939400640 1 -- 172.21.15.123:0/2214501441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216e1e50 msgr2=0x55b0216df730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.650+0000 7fe939400640 1 --2- 172.21.15.123:0/2214501441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216e1e50 0x55b0216df730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.650+0000 7fe939400640 1 -- 172.21.15.123:0/2214501441 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0213ae3c0 con 0x55b0216d7590 2024-09-19T04:48:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.651+0000 7fe939400640 1 --2- 172.21.15.123:0/2214501441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d7590 0x55b0216df1f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe9280cd090 tx=0x7fe928005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10fd37067c5a8e75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.651+0000 7fe91b400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:48:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.651+0000 7fe91b400640 1 -- 172.21.15.123:0/2214501441 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928020030 con 0x55b0216d7590 2024-09-19T04:48:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.651+0000 7fe91b400640 1 -- 172.21.15.123:0/2214501441 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fe9280cd2b0 con 0x55b0216d7590 2024-09-19T04:48:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.651+0000 7fe943816740 1 -- 172.21.15.123:0/2214501441 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe934004cc0 con 0x55b0216d7590 2024-09-19T04:48:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.651+0000 7fe91b400640 1 -- 172.21.15.123:0/2214501441 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe9280d1e60 con 0x55b0216d7590 2024-09-19T04:48:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.651+0000 7fe943816740 10 client.25771 fetch_fsmap learned FSMap version 31 2024-09-19T04:48:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.651+0000 7fe943816740 1 -- 172.21.15.123:0/2214501441 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b0216f0a40 con 0x55b0216d7590 2024-09-19T04:48:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.651+0000 7fe91b400640 1 -- 172.21.15.123:0/2214501441 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe934004cc0 con 0x55b0216d7590 2024-09-19T04:48:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.652+0000 7fe91b400640 1 -- 172.21.15.123:0/2214501441 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fe9280a43a0 con 0x55b0216d7590 2024-09-19T04:48:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.652+0000 7fe943816740 10 client.25771 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:48:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.652+0000 7fe943816740 10 client.25771 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:48:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.652+0000 7fe943816740 20 client.25771 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.652+0000 7fe943816740 1 --2- 172.21.15.123:0/2214501441 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b0216f1ea0 0x55b0216f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.652+0000 7fe943816740 4 client.25771 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:48:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.652+0000 7fe943816740 1 -- 172.21.15.123:0/2214501441 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b02164a950 con 0x55b0216f1ea0 2024-09-19T04:48:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.652+0000 7fe939e00640 1 --2- 172.21.15.123:0/2214501441 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b0216f1ea0 0x55b0216f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.653+0000 7fe939e00640 1 --2- 172.21.15.123:0/2214501441 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b0216f1ea0 0x55b0216f42b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4d9bb6699d5c0d33 server_cookie=49c6c2a4890fc353 in_seq=0 out_seq=0 2024-09-19T04:48:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.653+0000 7fe91b400640 10 client.25771 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:48:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.653+0000 7fe91b400640 1 -- 172.21.15.123:0/2214501441 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b02164a950 con 0x55b0216f1ea0 2024-09-19T04:48:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.653+0000 7fe91b400640 10 client.25771 handle_command_reply: tid=0 2024-09-19T04:48:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.653+0000 7fe943816740 1 client.25771 shutdown 2024-09-19T04:48:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.654+0000 7fe943816740 1 -- 172.21.15.123:0/2214501441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d7590 msgr2=0x55b0216df1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.654+0000 7fe943816740 1 --2- 172.21.15.123:0/2214501441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d7590 0x55b0216df1f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe9280cd090 tx=0x7fe928005dc0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.654+0000 7fe943816740 1 -- 172.21.15.123:0/2214501441 shutdown_connections 2024-09-19T04:48:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.654+0000 7fe943816740 1 --2- 172.21.15.123:0/2214501441 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b0216f1ea0 0x55b0216f42b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.654+0000 7fe943816740 1 --2- 172.21.15.123:0/2214501441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216e1e50 0x55b0216df730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.654+0000 7fe943816740 1 --2- 172.21.15.123:0/2214501441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d7590 0x55b0216df1f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.654+0000 7fe91b400640 0 client.25771 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:48:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.654+0000 7fe943816740 1 --2- 172.21.15.123:0/2214501441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d6c70 0x55b0216decb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.654+0000 7fe943816740 1 -- 172.21.15.123:0/2214501441 >> 172.21.15.123:0/2214501441 conn(0x55b0216d3200 msgr2=0x55b0216d8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.654+0000 7fe943816740 1 -- 172.21.15.123:0/2214501441 shutdown_connections 2024-09-19T04:48:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.654+0000 7fe943816740 1 -- 172.21.15.123:0/2214501441 wait complete. 2024-09-19T04:48:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.654+0000 7fe943816740 20 client.25771 trim_cache size 0 max 16384 2024-09-19T04:48:57.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.668+0000 7fe943816740 1 Processor -- start 2024-09-19T04:48:57.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.668+0000 7fe943816740 1 -- start start 2024-09-19T04:48:57.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.668+0000 7fe943816740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d6c70 0x55b0216dd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.668+0000 7fe943816740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 0x55b0216f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.668+0000 7fe943816740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216e1e50 0x55b0216fa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.668+0000 7fe943816740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1b3f00 con 0x55b0216e1e50 2024-09-19T04:48:57.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.668+0000 7fe943816740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1b5b90 con 0x55b0216d7590 2024-09-19T04:48:57.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.668+0000 7fe943816740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1b45d0 con 0x55b0216d6c70 2024-09-19T04:48:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.668+0000 7fe939e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d6c70 0x55b0216dd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.668+0000 7fe939400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 0x55b0216f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.668+0000 7fe939e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d6c70 0x55b0216dd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:41546/0 (socket says 172.21.15.123:41546) 2024-09-19T04:48:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.668+0000 7fe939e00640 1 -- 172.21.15.123:0/1372597413 learned_addr learned my addr 172.21.15.123:0/1372597413 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.669+0000 7fe93a800640 1 --2- 172.21.15.123:0/1372597413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216e1e50 0x55b0216fa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.669+0000 7fe939400640 1 -- 172.21.15.123:0/1372597413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d6c70 msgr2=0x55b0216dd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.669+0000 7fe939400640 1 --2- 172.21.15.123:0/1372597413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d6c70 0x55b0216dd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.669+0000 7fe939400640 1 -- 172.21.15.123:0/1372597413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216e1e50 msgr2=0x55b0216fa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.669+0000 7fe939400640 1 --2- 172.21.15.123:0/1372597413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216e1e50 0x55b0216fa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.669+0000 7fe939400640 1 -- 172.21.15.123:0/1372597413 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0216d3030 con 0x55b0216d7590 2024-09-19T04:48:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.669+0000 7fe939400640 1 --2- 172.21.15.123:0/1372597413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 0x55b0216f9b80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe92809bb50 tx=0x7fe928003950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=789140662df2d50c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.669+0000 7fe926600640 1 -- 172.21.15.123:0/1372597413 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928020070 con 0x55b0216d7590 2024-09-19T04:48:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.670+0000 7fe926600640 1 -- 172.21.15.123:0/1372597413 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe92809c2c0 con 0x55b0216d7590 2024-09-19T04:48:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.670+0000 7fe926600640 1 -- 172.21.15.123:0/1372597413 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9280cd2b0 con 0x55b0216d7590 2024-09-19T04:48:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.670+0000 7fe943816740 1 -- 172.21.15.123:0/1372597413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 msgr2=0x55b0216f9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.670+0000 7fe943816740 1 --2- 172.21.15.123:0/1372597413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 0x55b0216f9b80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe92809bb50 tx=0x7fe928003950 comp rx=0 tx=0).stop 2024-09-19T04:48:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.671+0000 7fe943816740 1 -- 172.21.15.123:0/1372597413 shutdown_connections 2024-09-19T04:48:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.671+0000 7fe943816740 1 --2- 172.21.15.123:0/1372597413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216e1e50 0x55b0216fa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.671+0000 7fe943816740 1 --2- 172.21.15.123:0/1372597413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 0x55b0216f9b80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.671+0000 7fe943816740 1 --2- 172.21.15.123:0/1372597413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216d6c70 0x55b0216dd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.671+0000 7fe943816740 1 -- 172.21.15.123:0/1372597413 >> 172.21.15.123:0/1372597413 conn(0x55b0216d3200 msgr2=0x55b0216e0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.671+0000 7fe943816740 1 -- 172.21.15.123:0/1372597413 shutdown_connections 2024-09-19T04:48:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.671+0000 7fe943816740 1 -- 172.21.15.123:0/1372597413 wait complete. 2024-09-19T04:48:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.671+0000 7fe943816740 1 -- start start 2024-09-19T04:48:57.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.672+0000 7fe943816740 1 Processor -- start 2024-09-19T04:48:57.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.672+0000 7fe943816740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216d6c70 0x55b0216ea030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.672+0000 7fe943816740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 0x55b0216ea570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.672+0000 7fe943816740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216e1e50 0x55b0216e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.672+0000 7fe943816740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1b5400 con 0x55b0216d6c70 2024-09-19T04:48:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.672+0000 7fe943816740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1b4d30 con 0x55b0216d7590 2024-09-19T04:48:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.672+0000 7fe943816740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1b1ee0 con 0x55b0216e1e50 2024-09-19T04:48:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.672+0000 7fe943816740 10 client.0 _ll_register_callbacks cb 0x7fe9280d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.672+0000 7fe939400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 0x55b0216ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.672+0000 7fe939400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 0x55b0216ea570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:53410/0 (socket says 172.21.15.123:53410) 2024-09-19T04:48:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.672+0000 7fe939400640 1 -- 172.21.15.123:0/4209716835 learned_addr learned my addr 172.21.15.123:0/4209716835 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.672+0000 7fe93a800640 1 --2- 172.21.15.123:0/4209716835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216e1e50 0x55b0216e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.672+0000 7fe939e00640 1 --2- 172.21.15.123:0/4209716835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216d6c70 0x55b0216ea030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.673+0000 7fe939400640 1 -- 172.21.15.123:0/4209716835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216e1e50 msgr2=0x55b0216e6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.673+0000 7fe939400640 1 --2- 172.21.15.123:0/4209716835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216e1e50 0x55b0216e6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.673+0000 7fe939400640 1 -- 172.21.15.123:0/4209716835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216d6c70 msgr2=0x55b0216ea030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:57.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.673+0000 7fe939400640 1 --2- 172.21.15.123:0/4209716835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216d6c70 0x55b0216ea030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.673+0000 7fe939400640 1 -- 172.21.15.123:0/4209716835 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0216d3e90 con 0x55b0216d7590 2024-09-19T04:48:57.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.673+0000 7fe93a800640 1 --2- 172.21.15.123:0/4209716835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216e1e50 0x55b0216e6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:57.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.673+0000 7fe939400640 1 --2- 172.21.15.123:0/4209716835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 0x55b0216ea570 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe928098020 tx=0x7fe92809a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b2ff700274367a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:57.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.673+0000 7fe91b400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:48:57.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.676+0000 7fe91b400640 1 -- 172.21.15.123:0/4209716835 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928020070 con 0x55b0216d7590 2024-09-19T04:48:57.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.676+0000 7fe91b400640 1 -- 172.21.15.123:0/4209716835 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fe92809b010 con 0x55b0216d7590 2024-09-19T04:48:57.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.676+0000 7fe943816740 1 -- 172.21.15.123:0/4209716835 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe92809c2c0 con 0x55b0216d7590 2024-09-19T04:48:57.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.676+0000 7fe91b400640 1 -- 172.21.15.123:0/4209716835 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe9280a45b0 con 0x55b0216d7590 2024-09-19T04:48:57.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.676+0000 7fe943816740 10 client.25757 fetch_fsmap learned FSMap version 31 2024-09-19T04:48:57.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.676+0000 7fe943816740 1 -- 172.21.15.123:0/4209716835 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b0216ce770 con 0x55b0216d7590 2024-09-19T04:48:57.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.676+0000 7fe91b400640 1 -- 172.21.15.123:0/4209716835 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe92809c2c0 con 0x55b0216d7590 2024-09-19T04:48:57.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.677+0000 7fe91b400640 1 -- 172.21.15.123:0/4209716835 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fe92809ac40 con 0x55b0216d7590 2024-09-19T04:48:57.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.677+0000 7fe943816740 10 client.25757 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:48:57.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.677+0000 7fe943816740 10 client.25757 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:48:57.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.677+0000 7fe943816740 20 client.25757 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:57.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.677+0000 7fe943816740 1 --2- 172.21.15.123:0/4209716835 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b02175d790 0x55b02175fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:57.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.677+0000 7fe943816740 4 client.25757 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:48:57.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.677+0000 7fe943816740 1 -- 172.21.15.123:0/4209716835 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b021644e60 con 0x55b02175d790 2024-09-19T04:48:57.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.677+0000 7fe939e00640 1 --2- 172.21.15.123:0/4209716835 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b02175d790 0x55b02175fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:57.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.678+0000 7fe91b400640 10 client.25757 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:48:57.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.678+0000 7fe939e00640 1 --2- 172.21.15.123:0/4209716835 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b02175d790 0x55b02175fb80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a74472ece09af6a0 server_cookie=a1074ce548781b8f in_seq=0 out_seq=0 2024-09-19T04:48:57.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.678+0000 7fe91b400640 1 -- 172.21.15.123:0/4209716835 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b021644e60 con 0x55b02175d790 2024-09-19T04:48:57.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.678+0000 7fe91b400640 10 client.25757 handle_command_reply: tid=0 2024-09-19T04:48:57.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.678+0000 7fe943816740 1 client.25757 shutdown 2024-09-19T04:48:57.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.678+0000 7fe943816740 1 -- 172.21.15.123:0/4209716835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 msgr2=0x55b0216ea570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:57.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.678+0000 7fe943816740 1 --2- 172.21.15.123:0/4209716835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 0x55b0216ea570 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe928098020 tx=0x7fe92809a0d0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.678+0000 7fe943816740 1 -- 172.21.15.123:0/4209716835 shutdown_connections 2024-09-19T04:48:57.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.678+0000 7fe943816740 1 --2- 172.21.15.123:0/4209716835 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b02175d790 0x55b02175fb80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.678+0000 7fe943816740 1 --2- 172.21.15.123:0/4209716835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0216e1e50 0x55b0216e6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.678+0000 7fe943816740 1 --2- 172.21.15.123:0/4209716835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0216d7590 0x55b0216ea570 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.678+0000 7fe943816740 1 --2- 172.21.15.123:0/4209716835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0216d6c70 0x55b0216ea030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.678+0000 7fe943816740 1 -- 172.21.15.123:0/4209716835 >> 172.21.15.123:0/4209716835 conn(0x55b0216d3200 msgr2=0x55b0216d1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:57.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.678+0000 7fe91b400640 0 client.25757 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:48:57.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.679+0000 7fe943816740 1 -- 172.21.15.123:0/4209716835 shutdown_connections 2024-09-19T04:48:57.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.679+0000 7fe943816740 1 -- 172.21.15.123:0/4209716835 wait complete. 2024-09-19T04:48:57.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.679+0000 7fe943816740 20 client.25757 trim_cache size 0 max 16384 2024-09-19T04:48:57.681 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:48:57.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.679+0000 7fe940c00640 1 -- 172.21.15.123:0/3356841511 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9200a33b0 msgr2=0x7fe9200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:57.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.679+0000 7fe940c00640 1 --2- 172.21.15.123:0/3356841511 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9200a33b0 0x7fe9200a5870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe92c031000 tx=0x7fe92c0056d0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.679+0000 7fe940c00640 1 -- 172.21.15.123:0/3356841511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe93c077040 msgr2=0x7fe93c1a4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:57.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.679+0000 7fe940c00640 1 --2- 172.21.15.123:0/3356841511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe93c077040 0x7fe93c1a4c70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe928005fd0 tx=0x7fe92800dde0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.679+0000 7fe940c00640 1 -- 172.21.15.123:0/3356841511 shutdown_connections 2024-09-19T04:48:57.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.679+0000 7fe940c00640 1 --2- 172.21.15.123:0/3356841511 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9200a33b0 0x7fe9200a5870 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.679+0000 7fe940c00640 1 --2- 172.21.15.123:0/3356841511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe93c112a50 0x7fe93c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.679+0000 7fe940c00640 1 --2- 172.21.15.123:0/3356841511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe93c077040 0x7fe93c1a4c70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.679+0000 7fe940c00640 1 --2- 172.21.15.123:0/3356841511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe93c076720 0x7fe93c1a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:57.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.679+0000 7fe940c00640 1 -- 172.21.15.123:0/3356841511 >> 172.21.15.123:0/3356841511 conn(0x7fe93c0ffa50 msgr2=0x7fe93c101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:57.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.680+0000 7fe940c00640 1 -- 172.21.15.123:0/3356841511 shutdown_connections 2024-09-19T04:48:57.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:57.680+0000 7fe940c00640 1 -- 172.21.15.123:0/3356841511 wait complete. 2024-09-19T04:48:57.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:57 smithi123 ceph-mon[30511]: pgmap v70: 97 pgs: 97 active+clean; 61 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 250 KiB/s wr, 59 op/s 2024-09-19T04:48:57.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:57 smithi193 ceph-mon[35592]: pgmap v70: 97 pgs: 97 active+clean; 61 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 250 KiB/s wr, 59 op/s 2024-09-19T04:48:58.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:57 smithi183 ceph-mon[35333]: pgmap v70: 97 pgs: 97 active+clean; 61 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 250 KiB/s wr, 59 op/s 2024-09-19T04:48:58.356 INFO:tasks.workunit.client.0.smithi123.stdout:config.status: creating include/builddefs 2024-09-19T04:48:58.388 INFO:tasks.workunit.client.0.smithi123.stdout:config.status: creating include/config.h 2024-09-19T04:48:58.412 INFO:tasks.workunit.client.0.smithi123.stdout:config.status: executing libtool commands 2024-09-19T04:48:58.534 INFO:tasks.workunit.client.0.smithi123.stdout:Building include 2024-09-19T04:48:58.535 INFO:tasks.workunit.client.0.smithi123.stdout:Building lib 2024-09-19T04:48:58.535 INFO:tasks.workunit.client.0.smithi123.stdout:Building tests 2024-09-19T04:48:58.544 INFO:tasks.workunit.client.0.smithi123.stdout: [LTDEP] 2024-09-19T04:48:58.551 INFO:tasks.workunit.client.0.smithi123.stdout:Building btrfs 2024-09-19T04:48:58.551 INFO:tasks.workunit.client.0.smithi123.stdout:Building ceph 2024-09-19T04:48:58.552 INFO:tasks.workunit.client.0.smithi123.stdout:Building cifs 2024-09-19T04:48:58.576 INFO:tasks.workunit.client.0.smithi123.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-09-19T04:48:58.576 INFO:tasks.workunit.client.0.smithi123.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-09-19T04:48:58.595 INFO:tasks.workunit.client.0.smithi123.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-09-19T04:48:58.610 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:48:58.630 INFO:tasks.workunit.client.0.smithi123.stdout:Building ext4 2024-09-19T04:48:58.635 INFO:tasks.workunit.client.0.smithi123.stdout:Building f2fs 2024-09-19T04:48:58.649 INFO:tasks.workunit.client.0.smithi123.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-09-19T04:48:58.660 INFO:tasks.workunit.client.0.smithi123.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-09-19T04:48:58.715 INFO:tasks.workunit.client.0.smithi123.stdout:Building generic 2024-09-19T04:48:58.789 INFO:tasks.workunit.client.0.smithi123.stdout:Building nfs 2024-09-19T04:48:58.805 INFO:tasks.workunit.client.0.smithi123.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-09-19T04:48:58.818 INFO:tasks.workunit.client.0.smithi123.stdout:Building ocfs2 2024-09-19T04:48:58.830 INFO:tasks.workunit.client.0.smithi123.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-09-19T04:48:58.856 INFO:tasks.workunit.client.0.smithi123.stdout:Building overlay 2024-09-19T04:48:58.876 INFO:tasks.workunit.client.0.smithi123.stdout:Building perf 2024-09-19T04:48:58.885 INFO:tasks.workunit.client.0.smithi123.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-09-19T04:48:58.888 INFO:tasks.workunit.client.0.smithi123.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-09-19T04:48:58.929 INFO:tasks.workunit.client.0.smithi123.stdout:Building selftest 2024-09-19T04:48:58.936 INFO:tasks.workunit.client.0.smithi123.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-09-19T04:48:58.942 INFO:tasks.workunit.client.0.smithi123.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-09-19T04:48:58.992 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] dataascii.lo 2024-09-19T04:48:58.994 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] databin.lo 2024-09-19T04:48:59.047 INFO:tasks.workunit.client.0.smithi123.stdout:Building shared 2024-09-19T04:48:59.062 INFO:tasks.workunit.client.0.smithi123.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-09-19T04:48:59.123 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:48:59.124 INFO:tasks.workunit.client.0.smithi123.stdout:Building tmpfs 2024-09-19T04:48:59.141 INFO:tasks.workunit.client.0.smithi123.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-09-19T04:48:59.190 INFO:tasks.workunit.client.0.smithi123.stdout:Building udf 2024-09-19T04:48:59.203 INFO:tasks.workunit.client.0.smithi123.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-09-19T04:48:59.250 INFO:tasks.workunit.client.0.smithi123.stdout:Building xfs 2024-09-19T04:48:59.266 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] datapid.lo 2024-09-19T04:48:59.288 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] file_lock.lo 2024-09-19T04:48:59.348 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] forker.lo 2024-09-19T04:48:59.424 INFO:tasks.workunit.client.0.smithi123.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-09-19T04:48:59.485 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] pattern.lo 2024-09-19T04:48:59.575 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] open_flags.lo 2024-09-19T04:48:59.599 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] random_range.lo 2024-09-19T04:48:59.725 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] string_to_tokens.lo 2024-09-19T04:48:59.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.817+0000 7faeadc00640 1 -- 172.21.15.123:0/2047720639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea00a4820 msgr2=0x7faea00a4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:59.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.817+0000 7faeadc00640 1 --2- 172.21.15.123:0/2047720639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea00a4820 0x7faea00a4c40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fae94007510 tx=0x7fae9402b0a0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.819+0000 7faeadc00640 1 -- 172.21.15.123:0/2047720639 shutdown_connections 2024-09-19T04:48:59.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.819+0000 7faeadc00640 1 --2- 172.21.15.123:0/2047720639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea00a5180 0x7faea00b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.819+0000 7faeadc00640 1 --2- 172.21.15.123:0/2047720639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea00a4820 0x7faea00a4c40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.819+0000 7faeadc00640 1 --2- 172.21.15.123:0/2047720639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea00aa7d0 0x7faea00aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.819+0000 7faeadc00640 1 -- 172.21.15.123:0/2047720639 >> 172.21.15.123:0/2047720639 conn(0x7faea001a0e0 msgr2=0x7faea001a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:59.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.819+0000 7faeadc00640 1 -- 172.21.15.123:0/2047720639 shutdown_connections 2024-09-19T04:48:59.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.819+0000 7faeadc00640 1 -- 172.21.15.123:0/2047720639 wait complete. 2024-09-19T04:48:59.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.823+0000 7faeadc00640 1 Processor -- start 2024-09-19T04:48:59.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.824+0000 7faeadc00640 1 -- start start 2024-09-19T04:48:59.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.825+0000 7faeadc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea00a4820 0x7faea013adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.825+0000 7faeadc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea00a5180 0x7faea013b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.825+0000 7faea7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea00a4820 0x7faea013adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:59.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.825+0000 7faea7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea00a4820 0x7faea013adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:53420/0 (socket says 172.21.15.123:53420) 2024-09-19T04:48:59.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.826+0000 7faea7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea00a5180 0x7faea013b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:59.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.826+0000 7faea7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea00a5180 0x7faea013b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59846/0 (socket says 172.21.15.123:59846) 2024-09-19T04:48:59.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.826+0000 7faea7e00640 1 -- 172.21.15.123:0/3475787782 learned_addr learned my addr 172.21.15.123:0/3475787782 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:59.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.826+0000 7faeadc00640 1 --2- 172.21.15.123:0/3475787782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea00aa7d0 0x7faea013f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.826+0000 7faeadc00640 1 -- 172.21.15.123:0/3475787782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faea00b47f0 con 0x7faea00a5180 2024-09-19T04:48:59.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.826+0000 7faeadc00640 1 -- 172.21.15.123:0/3475787782 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faea00b4670 con 0x7faea00a4820 2024-09-19T04:48:59.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.826+0000 7faeadc00640 1 -- 172.21.15.123:0/3475787782 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faea00b4970 con 0x7faea00aa7d0 2024-09-19T04:48:59.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.827+0000 7faea7e00640 1 -- 172.21.15.123:0/3475787782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea00aa7d0 msgr2=0x7faea013f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:59.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.827+0000 7faea7e00640 1 --2- 172.21.15.123:0/3475787782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea00aa7d0 0x7faea013f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.827+0000 7faea7e00640 1 -- 172.21.15.123:0/3475787782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea00a5180 msgr2=0x7faea013b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:59.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.827+0000 7faea7e00640 1 --2- 172.21.15.123:0/3475787782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea00a5180 0x7faea013b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.827+0000 7faea7e00640 1 -- 172.21.15.123:0/3475787782 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea013fda0 con 0x7faea00a4820 2024-09-19T04:48:59.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.827+0000 7faea7400640 1 --2- 172.21.15.123:0/3475787782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea00a5180 0x7faea013b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:48:59.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.829+0000 7faea7e00640 1 --2- 172.21.15.123:0/3475787782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea00a4820 0x7faea013adf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faea8065a40 tx=0x7faea80691d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:59.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.830+0000 7faea4c00640 1 -- 172.21.15.123:0/3475787782 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea80715f0 con 0x7faea00a4820 2024-09-19T04:48:59.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.830+0000 7faea4c00640 1 -- 172.21.15.123:0/3475787782 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faea8071c70 con 0x7faea00a4820 2024-09-19T04:48:59.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.830+0000 7faea4c00640 1 -- 172.21.15.123:0/3475787782 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea8077e30 con 0x7faea00a4820 2024-09-19T04:48:59.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.830+0000 7faeadc00640 1 -- 172.21.15.123:0/3475787782 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea0140090 con 0x7faea00a4820 2024-09-19T04:48:59.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.831+0000 7faeadc00640 1 -- 172.21.15.123:0/3475787782 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea0147970 con 0x7faea00a4820 2024-09-19T04:48:59.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.839+0000 7faeb1e6a740 1 Processor -- start 2024-09-19T04:48:59.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.840+0000 7faea4c00640 1 -- 172.21.15.123:0/3475787782 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7faea807e030 con 0x7faea00a4820 2024-09-19T04:48:59.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.840+0000 7faeb1e6a740 1 -- start start 2024-09-19T04:48:59.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.841+0000 7faeb1e6a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0de70 0x5612f4e0e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.843+0000 7faea4c00640 1 --2- 172.21.15.123:0/3475787782 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fae900a5710 0x7fae900a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.843+0000 7faead200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0de70 0x5612f4e0e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:59.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.843+0000 7faead200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0de70 0x5612f4e0e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:41576/0 (socket says 172.21.15.123:41576) 2024-09-19T04:48:59.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.843+0000 7faea4c00640 1 -- 172.21.15.123:0/3475787782 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7faea8132c90 con 0x7faea00a4820 2024-09-19T04:48:59.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.844+0000 7faea7e00640 1 --2- 172.21.15.123:0/3475787782 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fae900a5710 0x7fae900a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:59.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.844+0000 7faead200640 1 -- 172.21.15.123:0/1581460756 learned_addr learned my addr 172.21.15.123:0/1581460756 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:59.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.844+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/1581460756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e0e790 0x5612f4e18b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.845+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/1581460756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e19050 0x5612f4e1b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.846+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1581460756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5612f4d743e0 con 0x5612f4e0e790 2024-09-19T04:48:59.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.846+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1581460756 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5612f4d1bd80 con 0x5612f4e19050 2024-09-19T04:48:59.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.846+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1581460756 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5612f4d703a0 con 0x5612f4e0de70 2024-09-19T04:48:59.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.847+0000 7faea7e00640 1 --2- 172.21.15.123:0/3475787782 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fae900a5710 0x7fae900a7bd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faea8068040 tx=0x7faea8067f40 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:59.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.848+0000 7faead200640 1 -- 172.21.15.123:0/1581460756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e19050 msgr2=0x5612f4e1b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:59.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.848+0000 7faead200640 1 --2- 172.21.15.123:0/1581460756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e19050 0x5612f4e1b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.848+0000 7faead200640 1 -- 172.21.15.123:0/1581460756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e0e790 msgr2=0x5612f4e18b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:59.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.848+0000 7faead200640 1 --2- 172.21.15.123:0/1581460756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e0e790 0x5612f4e18b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.848+0000 7faead200640 1 -- 172.21.15.123:0/1581460756 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f4d3aae0 con 0x5612f4e0de70 2024-09-19T04:48:59.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.849+0000 7faead200640 1 --2- 172.21.15.123:0/1581460756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0de70 0x5612f4e0e250 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0098e0 tx=0x7fae9c009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf98ab0fa1adc669 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:59.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.850+0000 7fae99600640 1 -- 172.21.15.123:0/1581460756 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c015020 con 0x5612f4e0de70 2024-09-19T04:48:59.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.850+0000 7fae99600640 1 -- 172.21.15.123:0/1581460756 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae9c011420 con 0x5612f4e0de70 2024-09-19T04:48:59.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.850+0000 7fae99600640 1 -- 172.21.15.123:0/1581460756 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0105e0 con 0x5612f4e0de70 2024-09-19T04:48:59.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.851+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1581460756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0de70 msgr2=0x5612f4e0e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:59.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.851+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/1581460756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0de70 0x5612f4e0e250 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0098e0 tx=0x7fae9c009da0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.855+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1581460756 shutdown_connections 2024-09-19T04:48:59.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.855+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/1581460756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e19050 0x5612f4e1b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.855+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/1581460756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e0e790 0x5612f4e18b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.855+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/1581460756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0de70 0x5612f4e0e250 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.855+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1581460756 >> 172.21.15.123:0/1581460756 conn(0x5612f4e0a400 msgr2=0x5612f4e0a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:59.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.855+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1581460756 shutdown_connections 2024-09-19T04:48:59.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.856+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1581460756 wait complete. 2024-09-19T04:48:59.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.856+0000 7faeb1e6a740 1 -- start start 2024-09-19T04:48:59.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.857+0000 7faeb1e6a740 1 Processor -- start 2024-09-19T04:48:59.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.858+0000 7faeb1e6a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 0x5612f4e15f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.858+0000 7faeb1e6a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e0e790 0x5612f4e164a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.860+0000 7faead200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 0x5612f4e15f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:59.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.860+0000 7faead200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 0x5612f4e15f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:53428/0 (socket says 172.21.15.123:53428) 2024-09-19T04:48:59.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.861+0000 7faea7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e0e790 0x5612f4e164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:59.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.861+0000 7faead200640 1 -- 172.21.15.123:0/1165114891 learned_addr learned my addr 172.21.15.123:0/1165114891 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:59.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.861+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/1165114891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e19050 0x5612f4e169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.862+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1165114891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5612f4e1bc20 con 0x5612f4e0e790 2024-09-19T04:48:59.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.862+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1165114891 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5612f4d2f0f0 con 0x5612f4e0de70 2024-09-19T04:48:59.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.862+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1165114891 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5612f4e1bda0 con 0x5612f4e19050 2024-09-19T04:48:59.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.865+0000 7faea7400640 1 -- 172.21.15.123:0/1165114891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e19050 msgr2=0x5612f4e169e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:59.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.865+0000 7faea7400640 1 --2- 172.21.15.123:0/1165114891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e19050 0x5612f4e169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.865+0000 7faea7400640 1 -- 172.21.15.123:0/1165114891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 msgr2=0x5612f4e15f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:59.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.865+0000 7faea7400640 1 --2- 172.21.15.123:0/1165114891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 0x5612f4e15f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.865+0000 7faea7400640 1 -- 172.21.15.123:0/1165114891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612f4ae53c0 con 0x5612f4e0e790 2024-09-19T04:48:59.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.866+0000 7faeb1e6a740 10 client.0 _ll_register_callbacks cb 0x5612f4d16bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:59.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.866+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1165114891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae9c011420 con 0x5612f4e0e790 2024-09-19T04:48:59.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.866+0000 7faea7400640 1 --2- 172.21.15.123:0/1165114891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e0e790 0x5612f4e164a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x5612f4e0b260 tx=0x7fae9402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9664ece31e67086d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:59.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.867+0000 7fae8e000640 10 client.15732 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:48:59.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.869+0000 7fae8e000640 1 -- 172.21.15.123:0/1165114891 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae94041030 con 0x5612f4e0e790 2024-09-19T04:48:59.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.870+0000 7fae8e000640 1 -- 172.21.15.123:0/1165114891 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fae94038070 con 0x5612f4e0e790 2024-09-19T04:48:59.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.870+0000 7fae8e000640 1 -- 172.21.15.123:0/1165114891 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fae94007da0 con 0x5612f4e0e790 2024-09-19T04:48:59.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.874+0000 7faeb1e6a740 10 client.15732 fetch_fsmap learned FSMap version 31 2024-09-19T04:48:59.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.874+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1165114891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=31}) -- 0x5612f4e303f0 con 0x5612f4e0e790 2024-09-19T04:48:59.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.876+0000 7fae8e000640 1 -- 172.21.15.123:0/1165114891 <== mon.0 v2:172.21.15.123:3300/0 4 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fae94008040 con 0x5612f4e0e790 2024-09-19T04:48:59.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.876+0000 7faeb1e6a740 10 client.15732 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:48:59.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.876+0000 7faeb1e6a740 10 client.15732 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:48:59.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.876+0000 7faeb1e6a740 20 client.15732 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:59.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.876+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/1165114891 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5612f4e31c10 0x5612f4e34090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.876+0000 7faeb1e6a740 4 client.15732 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:48:59.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.876+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1165114891 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612f4d81950 con 0x5612f4e31c10 2024-09-19T04:48:59.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.878+0000 7faead200640 1 --2- 172.21.15.123:0/1165114891 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5612f4e31c10 0x5612f4e34090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:59.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.881+0000 7faead200640 1 --2- 172.21.15.123:0/1165114891 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5612f4e31c10 0x5612f4e34090 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dee4a0666c2e4640 server_cookie=583ef0f4b78066f0 in_seq=0 out_seq=0 2024-09-19T04:48:59.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.881+0000 7fae8e000640 10 client.15732 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:48:59.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.882+0000 7fae8e000640 1 -- 172.21.15.123:0/1165114891 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612f4d81950 con 0x5612f4e31c10 2024-09-19T04:48:59.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.882+0000 7fae8e000640 10 client.15732 handle_command_reply: tid=0 2024-09-19T04:48:59.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.882+0000 7faeb1e6a740 1 client.15732 shutdown 2024-09-19T04:48:59.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.884+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1165114891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e0e790 msgr2=0x5612f4e164a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:59.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.884+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/1165114891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e0e790 0x5612f4e164a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x5612f4e0b260 tx=0x7fae9402beb0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.884+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1165114891 shutdown_connections 2024-09-19T04:48:59.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.884+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/1165114891 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5612f4e31c10 0x5612f4e34090 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.885+0000 7fae8e000640 0 client.15732 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:48:59.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.885+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/1165114891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e19050 0x5612f4e169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.885+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/1165114891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e0e790 0x5612f4e164a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.885+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/1165114891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 0x5612f4e15f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.885+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1165114891 >> 172.21.15.123:0/1165114891 conn(0x5612f4e0a400 msgr2=0x5612f4e10130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:59.887 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:48:59 smithi123 ceph-mon[30511]: pgmap v71: 97 pgs: 97 active+clean; 63 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 400 KiB/s wr, 75 op/s 2024-09-19T04:48:59.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.886+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1165114891 shutdown_connections 2024-09-19T04:48:59.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.887+0000 7faeb1e6a740 1 -- 172.21.15.123:0/1165114891 wait complete. 2024-09-19T04:48:59.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.887+0000 7faeb1e6a740 20 client.15732 trim_cache size 0 max 16384 2024-09-19T04:48:59.922 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] str_to_bytes.lo 2024-09-19T04:48:59.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.922+0000 7faeb1e6a740 1 Processor -- start 2024-09-19T04:48:59.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.924+0000 7faeb1e6a740 1 -- start start 2024-09-19T04:48:59.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.924+0000 7faeb1e6a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 0x5612f4e108d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.924+0000 7faeb1e6a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0e790 0x5612f4e16830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.925+0000 7faeb1e6a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e19050 0x5612f4e17ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.925+0000 7faead200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 0x5612f4e108d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:59.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.925+0000 7faead200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 0x5612f4e108d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:53430/0 (socket says 172.21.15.123:53430) 2024-09-19T04:48:59.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.925+0000 7faeb1e6a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faea0150180 con 0x5612f4e19050 2024-09-19T04:48:59.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.925+0000 7faeb1e6a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faea0151e10 con 0x5612f4e0de70 2024-09-19T04:48:59.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.925+0000 7faeb1e6a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faea0150850 con 0x5612f4e0e790 2024-09-19T04:48:59.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.926+0000 7faea7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0e790 0x5612f4e16830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:59.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.926+0000 7faead200640 1 -- 172.21.15.123:0/2211503388 learned_addr learned my addr 172.21.15.123:0/2211503388 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:59.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.927+0000 7faead200640 1 -- 172.21.15.123:0/2211503388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0e790 msgr2=0x5612f4e16830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:59.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.927+0000 7faead200640 1 --2- 172.21.15.123:0/2211503388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0e790 0x5612f4e16830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.927+0000 7faead200640 1 -- 172.21.15.123:0/2211503388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e19050 msgr2=0x5612f4e17ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:59.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.927+0000 7faead200640 1 --2- 172.21.15.123:0/2211503388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e19050 0x5612f4e17ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.927+0000 7faead200640 1 -- 172.21.15.123:0/2211503388 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f4e0dca0 con 0x5612f4e0de70 2024-09-19T04:48:59.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.929+0000 7faead200640 1 --2- 172.21.15.123:0/2211503388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 0x5612f4e108d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fae9c011e20 tx=0x7fae9c006bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=488bd1c284c27abc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:59.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.930+0000 7fae98c00640 1 -- 172.21.15.123:0/2211503388 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c029070 con 0x5612f4e0de70 2024-09-19T04:48:59.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.930+0000 7fae98c00640 1 -- 172.21.15.123:0/2211503388 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae9c021070 con 0x5612f4e0de70 2024-09-19T04:48:59.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.930+0000 7fae98c00640 1 -- 172.21.15.123:0/2211503388 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c015040 con 0x5612f4e0de70 2024-09-19T04:48:59.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.931+0000 7faeb1e6a740 1 -- 172.21.15.123:0/2211503388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 msgr2=0x5612f4e108d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:59.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.931+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/2211503388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 0x5612f4e108d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fae9c011e20 tx=0x7fae9c006bd0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.932+0000 7faeb1e6a740 1 -- 172.21.15.123:0/2211503388 shutdown_connections 2024-09-19T04:48:59.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.932+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/2211503388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e19050 0x5612f4e17ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.932+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/2211503388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0e790 0x5612f4e16830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.933+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/2211503388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 0x5612f4e108d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.933+0000 7faeb1e6a740 1 -- 172.21.15.123:0/2211503388 >> 172.21.15.123:0/2211503388 conn(0x5612f4e0a400 msgr2=0x5612f4e177c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:59.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.933+0000 7faeb1e6a740 1 -- 172.21.15.123:0/2211503388 shutdown_connections 2024-09-19T04:48:59.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.933+0000 7faeb1e6a740 1 -- 172.21.15.123:0/2211503388 wait complete. 2024-09-19T04:48:59.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.935+0000 7faeb1e6a740 1 -- start start 2024-09-19T04:48:59.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.935+0000 7faeb1e6a740 1 Processor -- start 2024-09-19T04:48:59.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.937+0000 7faeb1e6a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 0x5612f4e29cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.937+0000 7faeb1e6a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0e790 0x5612f4e24fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.937+0000 7faead200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 0x5612f4e29cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:59.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.938+0000 7faead200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 0x5612f4e29cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:53432/0 (socket says 172.21.15.123:53432) 2024-09-19T04:48:59.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.939+0000 7faead200640 1 -- 172.21.15.123:0/3106706764 learned_addr learned my addr 172.21.15.123:0/3106706764 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:48:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.941+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/3106706764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e19050 0x5612f4e25520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.941+0000 7faeb1e6a740 1 -- 172.21.15.123:0/3106706764 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faea0151680 con 0x5612f4e19050 2024-09-19T04:48:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.941+0000 7faeb1e6a740 1 -- 172.21.15.123:0/3106706764 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faea0150fb0 con 0x5612f4e0de70 2024-09-19T04:48:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.941+0000 7faeb1e6a740 1 -- 172.21.15.123:0/3106706764 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faea014e160 con 0x5612f4e0e790 2024-09-19T04:48:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.941+0000 7faeb1e6a740 10 client.0 _ll_register_callbacks cb 0x7fae9402bd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:48:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.938+0000 7faea7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0e790 0x5612f4e24fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.941+0000 7faead200640 1 --2- 172.21.15.123:0/3106706764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e19050 0x5612f4e25520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.942+0000 7faea7400640 1 -- 172.21.15.123:0/3106706764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 msgr2=0x5612f4e29cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.942+0000 7faea7400640 1 --2- 172.21.15.123:0/3106706764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 0x5612f4e29cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.942+0000 7faea7400640 1 -- 172.21.15.123:0/3106706764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e19050 msgr2=0x5612f4e25520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:48:59.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.942+0000 7faea7400640 1 --2- 172.21.15.123:0/3106706764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e19050 0x5612f4e25520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.942+0000 7faea7400640 1 -- 172.21.15.123:0/3106706764 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612f4e25a60 con 0x5612f4e0e790 2024-09-19T04:48:59.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.943+0000 7faead200640 1 --2- 172.21.15.123:0/3106706764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e19050 0x5612f4e25520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:48:59.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.943+0000 7faea7400640 1 --2- 172.21.15.123:0/3106706764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0e790 0x5612f4e24fe0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fae9400d090 tx=0x7fae9403b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3c092a5a0b86d9f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:48:59.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.943+0000 7fae8e000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:48:59.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.944+0000 7fae8e000640 1 -- 172.21.15.123:0/3106706764 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae94055060 con 0x5612f4e0e790 2024-09-19T04:48:59.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.944+0000 7faeb1e6a740 1 -- 172.21.15.123:0/3106706764 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae9c021070 con 0x5612f4e0e790 2024-09-19T04:48:59.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.944+0000 7fae8e000640 1 -- 172.21.15.123:0/3106706764 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fae9400a7f0 con 0x5612f4e0e790 2024-09-19T04:48:59.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.945+0000 7fae8e000640 1 -- 172.21.15.123:0/3106706764 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fae94000e60 con 0x5612f4e0e790 2024-09-19T04:48:59.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.945+0000 7faeb1e6a740 10 client.25789 fetch_fsmap learned FSMap version 31 2024-09-19T04:48:59.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.945+0000 7faeb1e6a740 1 -- 172.21.15.123:0/3106706764 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5612f4e9c660 con 0x5612f4e0e790 2024-09-19T04:48:59.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.947+0000 7fae8e000640 1 -- 172.21.15.123:0/3106706764 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae9c021070 con 0x5612f4e0e790 2024-09-19T04:48:59.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.947+0000 7fae8e000640 1 -- 172.21.15.123:0/3106706764 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fae94058050 con 0x5612f4e0e790 2024-09-19T04:48:59.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.950+0000 7faeb1e6a740 10 client.25789 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:48:59.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.950+0000 7faeb1e6a740 10 client.25789 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:48:59.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.950+0000 7faeb1e6a740 20 client.25789 populate_metadata read hostname 'smithi123' 2024-09-19T04:48:59.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.950+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/3106706764 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5612f4e31c10 0x5612f4e9ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:48:59.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.950+0000 7faeb1e6a740 4 client.25789 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:48:59.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.950+0000 7faeb1e6a740 1 -- 172.21.15.123:0/3106706764 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612f4d7be60 con 0x5612f4e31c10 2024-09-19T04:48:59.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.953+0000 7faead200640 1 --2- 172.21.15.123:0/3106706764 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5612f4e31c10 0x5612f4e9ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:48:59.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.960+0000 7faead200640 1 --2- 172.21.15.123:0/3106706764 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5612f4e31c10 0x5612f4e9ed90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=365aaef31dbda64b server_cookie=f881a505acef96c7 in_seq=0 out_seq=0 2024-09-19T04:48:59.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.960+0000 7fae8e000640 10 client.25789 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:48:59.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.960+0000 7fae8e000640 1 -- 172.21.15.123:0/3106706764 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612f4d7be60 con 0x5612f4e31c10 2024-09-19T04:48:59.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.960+0000 7fae8e000640 10 client.25789 handle_command_reply: tid=0 2024-09-19T04:48:59.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.961+0000 7faeb1e6a740 1 client.25789 shutdown 2024-09-19T04:48:59.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.961+0000 7faeb1e6a740 1 -- 172.21.15.123:0/3106706764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0e790 msgr2=0x5612f4e24fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:59.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.961+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/3106706764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0e790 0x5612f4e24fe0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fae9400d090 tx=0x7fae9403b000 comp rx=0 tx=0).stop 2024-09-19T04:48:59.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.961+0000 7faeb1e6a740 1 -- 172.21.15.123:0/3106706764 shutdown_connections 2024-09-19T04:48:59.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.961+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/3106706764 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5612f4e31c10 0x5612f4e9ed90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.961+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/3106706764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612f4e19050 0x5612f4e25520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.961+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/3106706764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612f4e0e790 0x5612f4e24fe0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.961+0000 7faeb1e6a740 1 --2- 172.21.15.123:0/3106706764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f4e0de70 0x5612f4e29cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.961+0000 7faeb1e6a740 1 -- 172.21.15.123:0/3106706764 >> 172.21.15.123:0/3106706764 conn(0x5612f4e0a400 msgr2=0x5612f4e17fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:59.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.961+0000 7fae8e000640 0 client.25789 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:48:59.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.962+0000 7faeb1e6a740 1 -- 172.21.15.123:0/3106706764 shutdown_connections 2024-09-19T04:48:59.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.963+0000 7faeb1e6a740 1 -- 172.21.15.123:0/3106706764 wait complete. 2024-09-19T04:48:59.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.963+0000 7faeb1e6a740 20 client.25789 trim_cache size 0 max 16384 2024-09-19T04:48:59.966 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:48:59.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.966+0000 7faeadc00640 1 -- 172.21.15.123:0/3475787782 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fae900a5710 msgr2=0x7fae900a7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:59.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.966+0000 7faeadc00640 1 --2- 172.21.15.123:0/3475787782 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fae900a5710 0x7fae900a7bd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faea8068040 tx=0x7faea8067f40 comp rx=0 tx=0).stop 2024-09-19T04:48:59.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.966+0000 7faeadc00640 1 -- 172.21.15.123:0/3475787782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea00a4820 msgr2=0x7faea013adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:48:59.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.966+0000 7faeadc00640 1 --2- 172.21.15.123:0/3475787782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea00a4820 0x7faea013adf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faea8065a40 tx=0x7faea80691d0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.966+0000 7faeadc00640 1 -- 172.21.15.123:0/3475787782 shutdown_connections 2024-09-19T04:48:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.966+0000 7faeadc00640 1 --2- 172.21.15.123:0/3475787782 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fae900a5710 0x7fae900a7bd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.966+0000 7faeadc00640 1 --2- 172.21.15.123:0/3475787782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea00aa7d0 0x7faea013f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.966+0000 7faeadc00640 1 --2- 172.21.15.123:0/3475787782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea00a5180 0x7faea013b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.966+0000 7faeadc00640 1 --2- 172.21.15.123:0/3475787782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea00a4820 0x7faea013adf0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:48:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.966+0000 7faeadc00640 1 -- 172.21.15.123:0/3475787782 >> 172.21.15.123:0/3475787782 conn(0x7faea001a0e0 msgr2=0x7faea00a26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:48:59.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.967+0000 7faeadc00640 1 -- 172.21.15.123:0/3475787782 shutdown_connections 2024-09-19T04:48:59.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:48:59.967+0000 7faeadc00640 1 -- 172.21.15.123:0/3475787782 wait complete. 2024-09-19T04:48:59.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:48:59 smithi193 ceph-mon[35592]: pgmap v71: 97 pgs: 97 active+clean; 63 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 400 KiB/s wr, 75 op/s 2024-09-19T04:49:00.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:48:59 smithi183 ceph-mon[35333]: pgmap v71: 97 pgs: 97 active+clean; 63 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 400 KiB/s wr, 75 op/s 2024-09-19T04:49:00.838 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:49:01.041 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] tlibio.lo 2024-09-19T04:49:01.042 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] write_log.lo 2024-09-19T04:49:01.053 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] random.lo 2024-09-19T04:49:01.330 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:49:01.429 INFO:tasks.workunit.client.0.smithi123.stdout: [LD] 2024-09-19T04:49:01.758 INFO:tasks.workunit.client.0.smithi123.stdout:Building ltp 2024-09-19T04:49:01.759 INFO:tasks.workunit.client.0.smithi123.stdout:Building src 2024-09-19T04:49:01.759 INFO:tasks.workunit.client.0.smithi123.stdout:Building m4 2024-09-19T04:49:01.759 INFO:tasks.workunit.client.0.smithi123.stdout:Building common 2024-09-19T04:49:01.769 INFO:tasks.workunit.client.0.smithi123.stdout: [DEP] 2024-09-19T04:49:01.784 INFO:tasks.workunit.client.0.smithi123.stdout: [DEP] 2024-09-19T04:49:01.795 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:01 smithi123 ceph-mon[30511]: pgmap v72: 97 pgs: 97 active+clean; 63 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 317 KiB/s wr, 33 op/s 2024-09-19T04:49:01.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:01 smithi193 ceph-mon[35592]: pgmap v72: 97 pgs: 97 active+clean; 63 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 317 KiB/s wr, 33 op/s 2024-09-19T04:49:02.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:01 smithi183 ceph-mon[35333]: pgmap v72: 97 pgs: 97 active+clean; 63 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 317 KiB/s wr, 33 op/s 2024-09-19T04:49:02.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.913+0000 7f971be00640 1 -- 172.21.15.123:0/1121479259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97141049a0 msgr2=0x7f97141089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:02.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.913+0000 7f971be00640 1 --2- 172.21.15.123:0/1121479259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97141049a0 0x7f97141089f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9700009900 tx=0x7f970002a410 comp rx=0 tx=0).stop 2024-09-19T04:49:02.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.913+0000 7f971be00640 1 -- 172.21.15.123:0/1121479259 shutdown_connections 2024-09-19T04:49:02.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.913+0000 7f971be00640 1 --2- 172.21.15.123:0/1121479259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9714109120 0x7f9714110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.913+0000 7f971be00640 1 --2- 172.21.15.123:0/1121479259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97141049a0 0x7f97141089f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.913+0000 7f971be00640 1 --2- 172.21.15.123:0/1121479259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9714103ff0 0x7f97141043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.913+0000 7f971be00640 1 -- 172.21.15.123:0/1121479259 >> 172.21.15.123:0/1121479259 conn(0x7f97140ffa90 msgr2=0x7f9714101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:02.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.913+0000 7f971be00640 1 -- 172.21.15.123:0/1121479259 shutdown_connections 2024-09-19T04:49:02.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.913+0000 7f971be00640 1 -- 172.21.15.123:0/1121479259 wait complete. 2024-09-19T04:49:02.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.914+0000 7f971be00640 1 Processor -- start 2024-09-19T04:49:02.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.914+0000 7f971be00640 1 -- start start 2024-09-19T04:49:02.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.914+0000 7f971be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9714103ff0 0x7f971419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.914+0000 7f971be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97141049a0 0x7f971419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.914+0000 7f971be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9714109120 0x7f971419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.914+0000 7f971be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9714114190 con 0x7f9714109120 2024-09-19T04:49:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.914+0000 7f971be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9714114010 con 0x7f97141049a0 2024-09-19T04:49:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.914+0000 7f971be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9714114310 con 0x7f9714103ff0 2024-09-19T04:49:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.915+0000 7f9718a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97141049a0 0x7f971419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.915+0000 7f9718a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97141049a0 0x7f971419ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:53458/0 (socket says 172.21.15.123:53458) 2024-09-19T04:49:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.915+0000 7f9718a00640 1 -- 172.21.15.123:0/163502536 learned_addr learned my addr 172.21.15.123:0/163502536 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.915+0000 7f9719400640 1 --2- 172.21.15.123:0/163502536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9714103ff0 0x7f971419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.915+0000 7f9719e00640 1 --2- 172.21.15.123:0/163502536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9714109120 0x7f971419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.915+0000 7f9718a00640 1 -- 172.21.15.123:0/163502536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9714103ff0 msgr2=0x7f971419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.915+0000 7f9718a00640 1 --2- 172.21.15.123:0/163502536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9714103ff0 0x7f971419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.915+0000 7f9718a00640 1 -- 172.21.15.123:0/163502536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9714109120 msgr2=0x7f971419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.915+0000 7f9718a00640 1 --2- 172.21.15.123:0/163502536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9714109120 0x7f971419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.915+0000 7f9718a00640 1 -- 172.21.15.123:0/163502536 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971419f5a0 con 0x7f97141049a0 2024-09-19T04:49:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.915+0000 7f9718a00640 1 --2- 172.21.15.123:0/163502536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97141049a0 0x7f971419ab30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9700004770 tx=0x7f9700003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.916+0000 7f9712000640 1 -- 172.21.15.123:0/163502536 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970003f070 con 0x7f97141049a0 2024-09-19T04:49:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.916+0000 7f971be00640 1 -- 172.21.15.123:0/163502536 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971419f830 con 0x7f97141049a0 2024-09-19T04:49:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.916+0000 7f971be00640 1 -- 172.21.15.123:0/163502536 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97141a7170 con 0x7f97141049a0 2024-09-19T04:49:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.916+0000 7f9712000640 1 -- 172.21.15.123:0/163502536 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970003b940 con 0x7f97141049a0 2024-09-19T04:49:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.916+0000 7f9712000640 1 -- 172.21.15.123:0/163502536 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700049060 con 0x7f97141049a0 2024-09-19T04:49:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.919+0000 7f971e80f740 1 Processor -- start 2024-09-19T04:49:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.919+0000 7f9712000640 1 -- 172.21.15.123:0/163502536 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9700042070 con 0x7f97141049a0 2024-09-19T04:49:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.919+0000 7f971e80f740 1 -- start start 2024-09-19T04:49:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.920+0000 7f971e80f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d71c70 0x5654d5d72050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.920+0000 7f971e80f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d72590 0x5654d5d7c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.920+0000 7f971e80f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d7ce50 0x5654d5d7f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.920+0000 7f971e80f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5654d5cd83e0 con 0x5654d5d71c70 2024-09-19T04:49:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.920+0000 7f971e80f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5654d5c7fd80 con 0x5654d5d72590 2024-09-19T04:49:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.920+0000 7f971e80f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5654d5cd43a0 con 0x5654d5d7ce50 2024-09-19T04:49:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.920+0000 7f9719e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d71c70 0x5654d5d72050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.920+0000 7f9718a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d7ce50 0x5654d5d7f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.920+0000 7f9719e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d71c70 0x5654d5d72050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59900/0 (socket says 172.21.15.123:59900) 2024-09-19T04:49:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.920+0000 7f9719e00640 1 -- 172.21.15.123:0/2859919836 learned_addr learned my addr 172.21.15.123:0/2859919836 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.920+0000 7f9719400640 1 --2- 172.21.15.123:0/2859919836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d72590 0x5654d5d7c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.920+0000 7f9719e00640 1 -- 172.21.15.123:0/2859919836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d7ce50 msgr2=0x5654d5d7f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.920+0000 7f9719e00640 1 --2- 172.21.15.123:0/2859919836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d7ce50 0x5654d5d7f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.920+0000 7f9719e00640 1 -- 172.21.15.123:0/2859919836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d72590 msgr2=0x5654d5d7c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:02.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.920+0000 7f9719e00640 1 --2- 172.21.15.123:0/2859919836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d72590 0x5654d5d7c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.920+0000 7f9719e00640 1 -- 172.21.15.123:0/2859919836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d5c9eae0 con 0x5654d5d71c70 2024-09-19T04:49:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.921+0000 7f9719e00640 1 --2- 172.21.15.123:0/2859919836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d71c70 0x5654d5d72050 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f970c009f50 tx=0x7f970c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88ae3984a9f4c02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.921+0000 7f9706600640 1 -- 172.21.15.123:0/2859919836 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c004d20 con 0x5654d5d71c70 2024-09-19T04:49:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.921+0000 7f9712000640 1 --2- 172.21.15.123:0/163502536 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f96fc0a33b0 0x7f96fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.921+0000 7f9706600640 1 -- 172.21.15.123:0/2859919836 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970c004050 con 0x5654d5d71c70 2024-09-19T04:49:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.921+0000 7f9706600640 1 -- 172.21.15.123:0/2859919836 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c005280 con 0x5654d5d71c70 2024-09-19T04:49:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.921+0000 7f9712000640 1 -- 172.21.15.123:0/163502536 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f97000efea0 con 0x7f97141049a0 2024-09-19T04:49:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.921+0000 7f9719400640 1 --2- 172.21.15.123:0/163502536 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f96fc0a33b0 0x7f96fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.922+0000 7f971e80f740 1 -- 172.21.15.123:0/2859919836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d71c70 msgr2=0x5654d5d72050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.922+0000 7f971e80f740 1 --2- 172.21.15.123:0/2859919836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d71c70 0x5654d5d72050 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f970c009f50 tx=0x7f970c0118d0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.922+0000 7f971e80f740 1 -- 172.21.15.123:0/2859919836 shutdown_connections 2024-09-19T04:49:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.922+0000 7f971e80f740 1 --2- 172.21.15.123:0/2859919836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d7ce50 0x5654d5d7f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.922+0000 7f971e80f740 1 --2- 172.21.15.123:0/2859919836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d72590 0x5654d5d7c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.922+0000 7f971e80f740 1 --2- 172.21.15.123:0/2859919836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d71c70 0x5654d5d72050 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.922+0000 7f971e80f740 1 -- 172.21.15.123:0/2859919836 >> 172.21.15.123:0/2859919836 conn(0x5654d5d6e200 msgr2=0x5654d5d6e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.922+0000 7f9719400640 1 --2- 172.21.15.123:0/163502536 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f96fc0a33b0 0x7f96fc0a5870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5654d5d6f060 tx=0x7f9708005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.922+0000 7f971e80f740 1 -- 172.21.15.123:0/2859919836 shutdown_connections 2024-09-19T04:49:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.923+0000 7f971e80f740 1 -- 172.21.15.123:0/2859919836 wait complete. 2024-09-19T04:49:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.923+0000 7f971e80f740 1 -- start start 2024-09-19T04:49:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.923+0000 7f971e80f740 1 Processor -- start 2024-09-19T04:49:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.924+0000 7f971e80f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d71c70 0x5654d5d79c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.924+0000 7f971e80f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d72590 0x5654d5d7a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.924+0000 7f971e80f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d7ce50 0x5654d5d7a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.924+0000 7f971e80f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5654d5d7fa20 con 0x5654d5d7ce50 2024-09-19T04:49:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.924+0000 7f971e80f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5654d5c930f0 con 0x5654d5d71c70 2024-09-19T04:49:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.924+0000 7f971e80f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5654d5d7fba0 con 0x5654d5d72590 2024-09-19T04:49:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.924+0000 7f971e80f740 10 client.0 _ll_register_callbacks cb 0x5654d5c7abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.924+0000 7f9719e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d7ce50 0x5654d5d7a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.924+0000 7f9719e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d7ce50 0x5654d5d7a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59910/0 (socket says 172.21.15.123:59910) 2024-09-19T04:49:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.924+0000 7f9719e00640 1 -- 172.21.15.123:0/3958906159 learned_addr learned my addr 172.21.15.123:0/3958906159 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.924+0000 7f9718a00640 1 --2- 172.21.15.123:0/3958906159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d72590 0x5654d5d7a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.924+0000 7f9719400640 1 --2- 172.21.15.123:0/3958906159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d71c70 0x5654d5d79c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.924+0000 7f9719e00640 1 -- 172.21.15.123:0/3958906159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d72590 msgr2=0x5654d5d7a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.925+0000 7f9719e00640 1 --2- 172.21.15.123:0/3958906159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d72590 0x5654d5d7a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.925+0000 7f9719e00640 1 -- 172.21.15.123:0/3958906159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d71c70 msgr2=0x5654d5d79c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:02.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.925+0000 7f9719e00640 1 --2- 172.21.15.123:0/3958906159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d71c70 0x5654d5d79c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.925+0000 7f9719e00640 1 -- 172.21.15.123:0/3958906159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d5a493c0 con 0x5654d5d7ce50 2024-09-19T04:49:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.925+0000 7f9719e00640 1 --2- 172.21.15.123:0/3958906159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d7ce50 0x5654d5d7a6e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f970c009250 tx=0x7f970c0044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f7118b9165ce08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.925+0000 7f96faa00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:49:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.925+0000 7f96faa00640 1 -- 172.21.15.123:0/3958906159 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c029070 con 0x5654d5d7ce50 2024-09-19T04:49:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.925+0000 7f96faa00640 1 -- 172.21.15.123:0/3958906159 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f970c00f480 con 0x5654d5d7ce50 2024-09-19T04:49:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.925+0000 7f971e80f740 1 -- 172.21.15.123:0/3958906159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f970c004050 con 0x5654d5d7ce50 2024-09-19T04:49:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.925+0000 7f96faa00640 1 -- 172.21.15.123:0/3958906159 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f970c035050 con 0x5654d5d7ce50 2024-09-19T04:49:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.925+0000 7f971e80f740 10 client.15744 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.925+0000 7f971e80f740 1 -- 172.21.15.123:0/3958906159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5654d5d8b9e0 con 0x5654d5d7ce50 2024-09-19T04:49:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.926+0000 7f96faa00640 1 -- 172.21.15.123:0/3958906159 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970c004050 con 0x5654d5d7ce50 2024-09-19T04:49:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.926+0000 7f96faa00640 1 -- 172.21.15.123:0/3958906159 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f970c019610 con 0x5654d5d7ce50 2024-09-19T04:49:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.926+0000 7f971e80f740 10 client.15744 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.926+0000 7f971e80f740 10 client.15744 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:49:02.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.926+0000 7f971e80f740 20 client.15744 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:02.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.926+0000 7f971e80f740 1 --2- 172.21.15.123:0/3958906159 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5654d5d8cd30 0x5654d5d8f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.926+0000 7f971e80f740 4 client.15744 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:49:02.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.926+0000 7f971e80f740 1 -- 172.21.15.123:0/3958906159 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654d5ce5950 con 0x5654d5d8cd30 2024-09-19T04:49:02.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.926+0000 7f9719400640 1 --2- 172.21.15.123:0/3958906159 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5654d5d8cd30 0x5654d5d8f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.927+0000 7f96faa00640 10 client.15744 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:49:02.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.927+0000 7f9719400640 1 --2- 172.21.15.123:0/3958906159 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5654d5d8cd30 0x5654d5d8f120 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c641199b9eef554c server_cookie=125765680627dc87 in_seq=0 out_seq=0 2024-09-19T04:49:02.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.928+0000 7f96faa00640 1 -- 172.21.15.123:0/3958906159 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654d5ce5950 con 0x5654d5d8cd30 2024-09-19T04:49:02.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.928+0000 7f96faa00640 10 client.15744 handle_command_reply: tid=0 2024-09-19T04:49:02.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.928+0000 7f971e80f740 1 client.15744 shutdown 2024-09-19T04:49:02.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.928+0000 7f971e80f740 1 -- 172.21.15.123:0/3958906159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d7ce50 msgr2=0x5654d5d7a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:02.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.928+0000 7f971e80f740 1 --2- 172.21.15.123:0/3958906159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d7ce50 0x5654d5d7a6e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f970c009250 tx=0x7f970c0044b0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.928+0000 7f971e80f740 1 -- 172.21.15.123:0/3958906159 shutdown_connections 2024-09-19T04:49:02.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.928+0000 7f971e80f740 1 --2- 172.21.15.123:0/3958906159 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5654d5d8cd30 0x5654d5d8f120 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.928+0000 7f971e80f740 1 --2- 172.21.15.123:0/3958906159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d7ce50 0x5654d5d7a6e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.928+0000 7f971e80f740 1 --2- 172.21.15.123:0/3958906159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d72590 0x5654d5d7a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.928+0000 7f971e80f740 1 --2- 172.21.15.123:0/3958906159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d71c70 0x5654d5d79c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.928+0000 7f971e80f740 1 -- 172.21.15.123:0/3958906159 >> 172.21.15.123:0/3958906159 conn(0x5654d5d6e200 msgr2=0x5654d5d6d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:02.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.928+0000 7f96faa00640 0 client.15744 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:49:02.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.928+0000 7f971e80f740 1 -- 172.21.15.123:0/3958906159 shutdown_connections 2024-09-19T04:49:02.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.929+0000 7f971e80f740 1 -- 172.21.15.123:0/3958906159 wait complete. 2024-09-19T04:49:02.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.929+0000 7f971e80f740 20 client.15744 trim_cache size 0 max 16384 2024-09-19T04:49:02.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.942+0000 7f971e80f740 1 Processor -- start 2024-09-19T04:49:02.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.942+0000 7f971e80f740 1 -- start start 2024-09-19T04:49:02.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.943+0000 7f971e80f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d71c70 0x5654d5d746d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.943+0000 7f971e80f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d72590 0x5654d5d949e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.943+0000 7f971e80f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d7ce50 0x5654d5d94f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.943+0000 7f971e80f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96f400e6d0 con 0x5654d5d71c70 2024-09-19T04:49:02.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.943+0000 7f971e80f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f96f4010360 con 0x5654d5d7ce50 2024-09-19T04:49:02.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.943+0000 7f971e80f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f96f400eda0 con 0x5654d5d72590 2024-09-19T04:49:02.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.943+0000 7f9718a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d72590 0x5654d5d949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.943+0000 7f9719400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d71c70 0x5654d5d746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.943+0000 7f9718a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d72590 0x5654d5d949e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:41624/0 (socket says 172.21.15.123:41624) 2024-09-19T04:49:02.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.943+0000 7f9718a00640 1 -- 172.21.15.123:0/3696702593 learned_addr learned my addr 172.21.15.123:0/3696702593 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:02.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.943+0000 7f9719e00640 1 --2- 172.21.15.123:0/3696702593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d7ce50 0x5654d5d94f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.943+0000 7f9718a00640 1 -- 172.21.15.123:0/3696702593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d7ce50 msgr2=0x5654d5d94f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:02.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.943+0000 7f9718a00640 1 --2- 172.21.15.123:0/3696702593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d7ce50 0x5654d5d94f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.943+0000 7f9718a00640 1 -- 172.21.15.123:0/3696702593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d71c70 msgr2=0x5654d5d746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:02.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.943+0000 7f9718a00640 1 --2- 172.21.15.123:0/3696702593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d71c70 0x5654d5d746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.944+0000 7f9718a00640 1 -- 172.21.15.123:0/3696702593 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d5d6e030 con 0x5654d5d72590 2024-09-19T04:49:02.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.944+0000 7f9719e00640 1 --2- 172.21.15.123:0/3696702593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d7ce50 0x5654d5d94f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:49:02.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.944+0000 7f9718a00640 1 --2- 172.21.15.123:0/3696702593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d72590 0x5654d5d949e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f97000bc2b0 tx=0x7f97000f39f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7a83e7cee9f74d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:02.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.944+0000 7f9705c00640 1 -- 172.21.15.123:0/3696702593 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97000bc0f0 con 0x5654d5d72590 2024-09-19T04:49:02.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.944+0000 7f9705c00640 1 -- 172.21.15.123:0/3696702593 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97000bc320 con 0x5654d5d72590 2024-09-19T04:49:02.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.944+0000 7f9705c00640 1 -- 172.21.15.123:0/3696702593 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97000bbc80 con 0x5654d5d72590 2024-09-19T04:49:02.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.945+0000 7f971e80f740 1 -- 172.21.15.123:0/3696702593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d72590 msgr2=0x5654d5d949e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:02.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.945+0000 7f971e80f740 1 --2- 172.21.15.123:0/3696702593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d72590 0x5654d5d949e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f97000bc2b0 tx=0x7f97000f39f0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.945+0000 7f971e80f740 1 -- 172.21.15.123:0/3696702593 shutdown_connections 2024-09-19T04:49:02.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.945+0000 7f971e80f740 1 --2- 172.21.15.123:0/3696702593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d7ce50 0x5654d5d94f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.945+0000 7f971e80f740 1 --2- 172.21.15.123:0/3696702593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d72590 0x5654d5d949e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.945+0000 7f971e80f740 1 --2- 172.21.15.123:0/3696702593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d71c70 0x5654d5d746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.945+0000 7f971e80f740 1 -- 172.21.15.123:0/3696702593 >> 172.21.15.123:0/3696702593 conn(0x5654d5d6e200 msgr2=0x5654d5d78d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:02.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.945+0000 7f971e80f740 1 -- 172.21.15.123:0/3696702593 shutdown_connections 2024-09-19T04:49:02.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.945+0000 7f971e80f740 1 -- 172.21.15.123:0/3696702593 wait complete. 2024-09-19T04:49:02.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.946+0000 7f971e80f740 1 -- start start 2024-09-19T04:49:02.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.946+0000 7f971e80f740 1 Processor -- start 2024-09-19T04:49:02.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.946+0000 7f971e80f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d71c70 0x5654d5d85470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.946+0000 7f971e80f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d72590 0x5654d5d807a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.946+0000 7f971e80f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d7ce50 0x5654d5d80ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.946+0000 7f971e80f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96f400fbd0 con 0x5654d5d72590 2024-09-19T04:49:02.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.946+0000 7f971e80f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f96f400f500 con 0x5654d5d7ce50 2024-09-19T04:49:02.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.946+0000 7f971e80f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f96f400c6b0 con 0x5654d5d71c70 2024-09-19T04:49:02.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.946+0000 7f971e80f740 10 client.0 _ll_register_callbacks cb 0x7f97080097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:02.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.947+0000 7f9718a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d72590 0x5654d5d807a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.947+0000 7f9719400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d71c70 0x5654d5d85470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.947+0000 7f9718a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d72590 0x5654d5d807a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59918/0 (socket says 172.21.15.123:59918) 2024-09-19T04:49:02.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.947+0000 7f9718a00640 1 -- 172.21.15.123:0/3909619597 learned_addr learned my addr 172.21.15.123:0/3909619597 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:02.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.947+0000 7f9719e00640 1 --2- 172.21.15.123:0/3909619597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d7ce50 0x5654d5d80ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.947+0000 7f9718a00640 1 -- 172.21.15.123:0/3909619597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d71c70 msgr2=0x5654d5d85470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:02.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.947+0000 7f9718a00640 1 --2- 172.21.15.123:0/3909619597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d71c70 0x5654d5d85470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.947+0000 7f9718a00640 1 -- 172.21.15.123:0/3909619597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d7ce50 msgr2=0x5654d5d80ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:02.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.947+0000 7f9718a00640 1 --2- 172.21.15.123:0/3909619597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d7ce50 0x5654d5d80ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.947+0000 7f9718a00640 1 -- 172.21.15.123:0/3909619597 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d5d6f2b0 con 0x5654d5d72590 2024-09-19T04:49:02.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.947+0000 7f9719400640 1 --2- 172.21.15.123:0/3909619597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d71c70 0x5654d5d85470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:49:02.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.947+0000 7f9718a00640 1 --2- 172.21.15.123:0/3909619597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d72590 0x5654d5d807a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f97000edc20 tx=0x7f9700009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd184cfa187e6d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:02.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.947+0000 7f96faa00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:49:02.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.947+0000 7f96faa00640 1 -- 172.21.15.123:0/3909619597 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97000bbe60 con 0x5654d5d72590 2024-09-19T04:49:02.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.947+0000 7f96faa00640 1 -- 172.21.15.123:0/3909619597 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f97000b8990 con 0x5654d5d72590 2024-09-19T04:49:02.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.947+0000 7f971e80f740 1 -- 172.21.15.123:0/3909619597 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97000bc320 con 0x5654d5d72590 2024-09-19T04:49:02.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.948+0000 7f96faa00640 1 -- 172.21.15.123:0/3909619597 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f97000bc0f0 con 0x5654d5d72590 2024-09-19T04:49:02.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.948+0000 7f971e80f740 10 client.15750 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:02.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.948+0000 7f971e80f740 1 -- 172.21.15.123:0/3909619597 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5654d5d69770 con 0x5654d5d72590 2024-09-19T04:49:02.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.948+0000 7f96faa00640 1 -- 172.21.15.123:0/3909619597 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97000bc320 con 0x5654d5d72590 2024-09-19T04:49:02.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.948+0000 7f96faa00640 1 -- 172.21.15.123:0/3909619597 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f970004c060 con 0x5654d5d72590 2024-09-19T04:49:02.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.948+0000 7f971e80f740 10 client.15750 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:02.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.948+0000 7f971e80f740 10 client.15750 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:49:02.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.948+0000 7f971e80f740 20 client.15750 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:02.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.948+0000 7f971e80f740 1 --2- 172.21.15.123:0/3909619597 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5654d5df7be0 0x5654d5dfa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:02.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.948+0000 7f971e80f740 4 client.15750 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:49:02.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.948+0000 7f971e80f740 1 -- 172.21.15.123:0/3909619597 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654d5cdfe60 con 0x5654d5df7be0 2024-09-19T04:49:02.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.949+0000 7f9719400640 1 --2- 172.21.15.123:0/3909619597 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5654d5df7be0 0x5654d5dfa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:02.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.949+0000 7f9719400640 1 --2- 172.21.15.123:0/3909619597 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5654d5df7be0 0x5654d5dfa000 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bc4127fdc274452 server_cookie=bfc6895487ad037a in_seq=0 out_seq=0 2024-09-19T04:49:02.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.949+0000 7f96faa00640 10 client.15750 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:49:02.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.949+0000 7f96faa00640 1 -- 172.21.15.123:0/3909619597 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654d5cdfe60 con 0x5654d5df7be0 2024-09-19T04:49:02.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.949+0000 7f96faa00640 10 client.15750 handle_command_reply: tid=0 2024-09-19T04:49:02.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.949+0000 7f971e80f740 1 client.15750 shutdown 2024-09-19T04:49:02.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.949+0000 7f971e80f740 1 -- 172.21.15.123:0/3909619597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d72590 msgr2=0x5654d5d807a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:02.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.949+0000 7f971e80f740 1 --2- 172.21.15.123:0/3909619597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d72590 0x5654d5d807a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f97000edc20 tx=0x7f9700009650 comp rx=0 tx=0).stop 2024-09-19T04:49:02.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.949+0000 7f971e80f740 1 -- 172.21.15.123:0/3909619597 shutdown_connections 2024-09-19T04:49:02.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.949+0000 7f971e80f740 1 --2- 172.21.15.123:0/3909619597 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5654d5df7be0 0x5654d5dfa000 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.949+0000 7f971e80f740 1 --2- 172.21.15.123:0/3909619597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654d5d7ce50 0x5654d5d80ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.949+0000 7f971e80f740 1 --2- 172.21.15.123:0/3909619597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654d5d72590 0x5654d5d807a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.949+0000 7f971e80f740 1 --2- 172.21.15.123:0/3909619597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654d5d71c70 0x5654d5d85470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.949+0000 7f96faa00640 0 client.15750 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:49:02.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.949+0000 7f971e80f740 1 -- 172.21.15.123:0/3909619597 >> 172.21.15.123:0/3909619597 conn(0x5654d5d6e200 msgr2=0x5654d5d7b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:02.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.950+0000 7f971e80f740 1 -- 172.21.15.123:0/3909619597 shutdown_connections 2024-09-19T04:49:02.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.950+0000 7f971e80f740 1 -- 172.21.15.123:0/3909619597 wait complete. 2024-09-19T04:49:02.268 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:49:02.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.950+0000 7f971e80f740 20 client.15750 trim_cache size 0 max 16384 2024-09-19T04:49:02.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.950+0000 7f971be00640 1 -- 172.21.15.123:0/163502536 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f96fc0a33b0 msgr2=0x7f96fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:02.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.950+0000 7f971be00640 1 --2- 172.21.15.123:0/163502536 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f96fc0a33b0 0x7f96fc0a5870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5654d5d6f060 tx=0x7f9708005f90 comp rx=0 tx=0).stop 2024-09-19T04:49:02.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.950+0000 7f971be00640 1 -- 172.21.15.123:0/163502536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97141049a0 msgr2=0x7f971419ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:02.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.950+0000 7f971be00640 1 --2- 172.21.15.123:0/163502536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97141049a0 0x7f971419ab30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9700004770 tx=0x7f9700003c10 comp rx=0 tx=0).stop 2024-09-19T04:49:02.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.951+0000 7f971be00640 1 -- 172.21.15.123:0/163502536 shutdown_connections 2024-09-19T04:49:02.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.951+0000 7f971be00640 1 --2- 172.21.15.123:0/163502536 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f96fc0a33b0 0x7f96fc0a5870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.951+0000 7f971be00640 1 --2- 172.21.15.123:0/163502536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9714109120 0x7f971419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.951+0000 7f971be00640 1 --2- 172.21.15.123:0/163502536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97141049a0 0x7f971419ab30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.951+0000 7f971be00640 1 --2- 172.21.15.123:0/163502536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9714103ff0 0x7f971419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:02.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.951+0000 7f971be00640 1 -- 172.21.15.123:0/163502536 >> 172.21.15.123:0/163502536 conn(0x7f97140ffa90 msgr2=0x7f9714101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:02.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.951+0000 7f971be00640 1 -- 172.21.15.123:0/163502536 shutdown_connections 2024-09-19T04:49:02.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:01.951+0000 7f971be00640 1 -- 172.21.15.123:0/163502536 wait complete. 2024-09-19T04:49:02.640 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] doio 2024-09-19T04:49:02.640 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] fsstress 2024-09-19T04:49:02.641 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] fsx 2024-09-19T04:49:03.474 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:49:03.600 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] growfiles 2024-09-19T04:49:03.615 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] iogen 2024-09-19T04:49:03.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:03 smithi123 ceph-mon[30511]: pgmap v73: 97 pgs: 97 active+clean; 63 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 317 KiB/s wr, 33 op/s 2024-09-19T04:49:03.967 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:49:03.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:03 smithi193 ceph-mon[35592]: pgmap v73: 97 pgs: 97 active+clean; 63 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 317 KiB/s wr, 33 op/s 2024-09-19T04:49:04.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:03 smithi183 ceph-mon[35333]: pgmap v73: 97 pgs: 97 active+clean; 63 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 317 KiB/s wr, 33 op/s 2024-09-19T04:49:04.261 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aio-stress 2024-09-19T04:49:04.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.701+0000 7fd605400640 1 -- 172.21.15.123:0/211787623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd600076720 msgr2=0x7fd600076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:04.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.701+0000 7fd605400640 1 --2- 172.21.15.123:0/211787623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd600076720 0x7fd600076b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009900 tx=0x7fd5f002a410 comp rx=0 tx=0).stop 2024-09-19T04:49:04.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.701+0000 7fd605400640 1 -- 172.21.15.123:0/211787623 shutdown_connections 2024-09-19T04:49:04.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.701+0000 7fd605400640 1 --2- 172.21.15.123:0/211787623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd600112a30 0x7fd600114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.701+0000 7fd605400640 1 --2- 172.21.15.123:0/211787623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd600077040 0x7fd6001124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.701+0000 7fd605400640 1 --2- 172.21.15.123:0/211787623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd600076720 0x7fd600076b00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.701+0000 7fd605400640 1 -- 172.21.15.123:0/211787623 >> 172.21.15.123:0/211787623 conn(0x7fd6000ffa90 msgr2=0x7fd600101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:04.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.701+0000 7fd605400640 1 -- 172.21.15.123:0/211787623 shutdown_connections 2024-09-19T04:49:04.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.701+0000 7fd605400640 1 -- 172.21.15.123:0/211787623 wait complete. 2024-09-19T04:49:04.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.703+0000 7fd605400640 1 Processor -- start 2024-09-19T04:49:04.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.703+0000 7fd605400640 1 -- start start 2024-09-19T04:49:04.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.703+0000 7fd605400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd600076720 0x7fd6001a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.703+0000 7fd605400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd600077040 0x7fd6001a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.703+0000 7fd605400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd600112a30 0x7fd60019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.703+0000 7fd605400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001185c0 con 0x7fd600077040 2024-09-19T04:49:04.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.703+0000 7fd605400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd600118440 con 0x7fd600112a30 2024-09-19T04:49:04.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.703+0000 7fd605400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd600118740 con 0x7fd600076720 2024-09-19T04:49:04.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.704+0000 7fd5fde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd600077040 0x7fd6001a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.704+0000 7fd5fde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd600077040 0x7fd6001a4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45980/0 (socket says 172.21.15.123:45980) 2024-09-19T04:49:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.704+0000 7fd5fde00640 1 -- 172.21.15.123:0/3494378023 learned_addr learned my addr 172.21.15.123:0/3494378023 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.704+0000 7fd5fe800640 1 --2- 172.21.15.123:0/3494378023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd600076720 0x7fd6001a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.704+0000 7fd5ff200640 1 --2- 172.21.15.123:0/3494378023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd600112a30 0x7fd60019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:04.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.704+0000 7fd5fde00640 1 -- 172.21.15.123:0/3494378023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd600076720 msgr2=0x7fd6001a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.704+0000 7fd5fde00640 1 --2- 172.21.15.123:0/3494378023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd600076720 0x7fd6001a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.704+0000 7fd5fde00640 1 -- 172.21.15.123:0/3494378023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd600112a30 msgr2=0x7fd60019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.704+0000 7fd5fde00640 1 --2- 172.21.15.123:0/3494378023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd600112a30 0x7fd60019e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.704+0000 7fd5fde00640 1 -- 172.21.15.123:0/3494378023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60019f010 con 0x7fd600077040 2024-09-19T04:49:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.705+0000 7fd5fe800640 1 --2- 172.21.15.123:0/3494378023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd600076720 0x7fd6001a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:49:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.705+0000 7fd5ff200640 1 --2- 172.21.15.123:0/3494378023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd600112a30 0x7fd60019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:49:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.705+0000 7fd5fde00640 1 --2- 172.21.15.123:0/3494378023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd600077040 0x7fd6001a4c60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00d950 tx=0x7fd5ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.705+0000 7fd5f7400640 1 -- 172.21.15.123:0/3494378023 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec017030 con 0x7fd600077040 2024-09-19T04:49:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.705+0000 7fd5f7400640 1 -- 172.21.15.123:0/3494378023 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ec004020 con 0x7fd600077040 2024-09-19T04:49:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.705+0000 7fd5f7400640 1 -- 172.21.15.123:0/3494378023 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec0047a0 con 0x7fd600077040 2024-09-19T04:49:04.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.705+0000 7fd605400640 1 -- 172.21.15.123:0/3494378023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60019f300 con 0x7fd600077040 2024-09-19T04:49:04.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.706+0000 7fd605400640 1 -- 172.21.15.123:0/3494378023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6001ab5f0 con 0x7fd600077040 2024-09-19T04:49:04.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.709+0000 7fd5f7400640 1 -- 172.21.15.123:0/3494378023 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd5ec0041d0 con 0x7fd600077040 2024-09-19T04:49:04.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.710+0000 7fd5f7400640 1 --2- 172.21.15.123:0/3494378023 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd5e40a3430 0x7fd5e40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.710+0000 7fd5f7400640 1 -- 172.21.15.123:0/3494378023 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd5ec0cf610 con 0x7fd600077040 2024-09-19T04:49:04.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.711+0000 7fd607e09740 1 Processor -- start 2024-09-19T04:49:04.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.711+0000 7fd5fe800640 1 --2- 172.21.15.123:0/3494378023 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd5e40a3430 0x7fd5e40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:04.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.711+0000 7fd607e09740 1 -- start start 2024-09-19T04:49:04.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.711+0000 7fd607e09740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb0dc70 0x56323cb0e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.711+0000 7fd607e09740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0e590 0x56323cb18910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.711+0000 7fd607e09740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb18e50 0x56323cb1b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.711+0000 7fd607e09740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56323ca743e0 con 0x56323cb0dc70 2024-09-19T04:49:04.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.711+0000 7fd607e09740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56323ca1bd80 con 0x56323cb0e590 2024-09-19T04:49:04.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.711+0000 7fd607e09740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56323ca703a0 con 0x56323cb18e50 2024-09-19T04:49:04.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.712+0000 7fd5fde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0e590 0x56323cb18910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:04.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.712+0000 7fd5fde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0e590 0x56323cb18910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:58528/0 (socket says 172.21.15.123:58528) 2024-09-19T04:49:04.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.712+0000 7fd5fde00640 1 -- 172.21.15.123:0/2201295246 learned_addr learned my addr 172.21.15.123:0/2201295246 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:04.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.712+0000 7fd5fe800640 1 --2- 172.21.15.123:0/2201295246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb0dc70 0x56323cb0e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:04.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.712+0000 7fd5fde00640 1 -- 172.21.15.123:0/2201295246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb18e50 msgr2=0x56323cb1b240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:49:04.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.712+0000 7fd5fde00640 1 --2- 172.21.15.123:0/2201295246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb18e50 0x56323cb1b240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.712+0000 7fd5fde00640 1 -- 172.21.15.123:0/2201295246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb0dc70 msgr2=0x56323cb0e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:04.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.712+0000 7fd5fde00640 1 --2- 172.21.15.123:0/2201295246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb0dc70 0x56323cb0e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.712+0000 7fd5fde00640 1 -- 172.21.15.123:0/2201295246 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56323ca3aae0 con 0x56323cb0e590 2024-09-19T04:49:04.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.712+0000 7fd5fe800640 1 --2- 172.21.15.123:0/2201295246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb0dc70 0x56323cb0e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:49:04.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.712+0000 7fd5fe800640 1 --2- 172.21.15.123:0/3494378023 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd5e40a3430 0x7fd5e40a58f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd60019fa20 tx=0x7fd5f00058d0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:04.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.713+0000 7fd5fde00640 1 --2- 172.21.15.123:0/2201295246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0e590 0x56323cb18910 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec098290 tx=0x7fd5ec09b2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7956fa9095991f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:04.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.713+0000 7fd5eb400640 1 -- 172.21.15.123:0/2201295246 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec09fc70 con 0x56323cb0e590 2024-09-19T04:49:04.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.713+0000 7fd5eb400640 1 -- 172.21.15.123:0/2201295246 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ec0a4530 con 0x56323cb0e590 2024-09-19T04:49:04.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.713+0000 7fd5eb400640 1 -- 172.21.15.123:0/2201295246 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec084e20 con 0x56323cb0e590 2024-09-19T04:49:04.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.714+0000 7fd607e09740 1 -- 172.21.15.123:0/2201295246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0e590 msgr2=0x56323cb18910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:04.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.714+0000 7fd607e09740 1 --2- 172.21.15.123:0/2201295246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0e590 0x56323cb18910 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec098290 tx=0x7fd5ec09b2e0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.714+0000 7fd607e09740 1 -- 172.21.15.123:0/2201295246 shutdown_connections 2024-09-19T04:49:04.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.714+0000 7fd607e09740 1 --2- 172.21.15.123:0/2201295246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb18e50 0x56323cb1b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.714+0000 7fd607e09740 1 --2- 172.21.15.123:0/2201295246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0e590 0x56323cb18910 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.714+0000 7fd607e09740 1 --2- 172.21.15.123:0/2201295246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb0dc70 0x56323cb0e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.714+0000 7fd607e09740 1 -- 172.21.15.123:0/2201295246 >> 172.21.15.123:0/2201295246 conn(0x56323cb0a200 msgr2=0x56323cb0a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:04.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.714+0000 7fd607e09740 1 -- 172.21.15.123:0/2201295246 shutdown_connections 2024-09-19T04:49:04.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.714+0000 7fd607e09740 1 -- 172.21.15.123:0/2201295246 wait complete. 2024-09-19T04:49:04.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.715+0000 7fd607e09740 1 -- start start 2024-09-19T04:49:04.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.715+0000 7fd607e09740 1 Processor -- start 2024-09-19T04:49:04.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.716+0000 7fd607e09740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0dc70 0x56323cb15e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.716+0000 7fd607e09740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 0x56323cb16370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.716+0000 7fd607e09740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb18e50 0x56323cb168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.716+0000 7fd607e09740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56323cb1ba20 con 0x56323cb18e50 2024-09-19T04:49:04.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.716+0000 7fd607e09740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56323ca2f0f0 con 0x56323cb0dc70 2024-09-19T04:49:04.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.716+0000 7fd607e09740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56323cb1bba0 con 0x56323cb0e590 2024-09-19T04:49:04.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.716+0000 7fd607e09740 10 client.0 _ll_register_callbacks cb 0x56323ca16bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:04.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.716+0000 7fd5fde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 0x56323cb16370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:04.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.717+0000 7fd5fde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 0x56323cb16370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53122/0 (socket says 172.21.15.123:53122) 2024-09-19T04:49:04.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.717+0000 7fd5fde00640 1 -- 172.21.15.123:0/2767175247 learned_addr learned my addr 172.21.15.123:0/2767175247 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:04.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.717+0000 7fd5fe800640 1 --2- 172.21.15.123:0/2767175247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0dc70 0x56323cb15e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:04.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.717+0000 7fd5ff200640 1 --2- 172.21.15.123:0/2767175247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb18e50 0x56323cb168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:04.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.717+0000 7fd5fde00640 1 -- 172.21.15.123:0/2767175247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0dc70 msgr2=0x56323cb15e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:04.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.717+0000 7fd5fde00640 1 --2- 172.21.15.123:0/2767175247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0dc70 0x56323cb15e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.717+0000 7fd5fde00640 1 -- 172.21.15.123:0/2767175247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb18e50 msgr2=0x56323cb168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:04.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.717+0000 7fd5fde00640 1 --2- 172.21.15.123:0/2767175247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb18e50 0x56323cb168b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.717+0000 7fd5fde00640 1 -- 172.21.15.123:0/2767175247 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56323c7e53c0 con 0x56323cb0e590 2024-09-19T04:49:04.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.717+0000 7fd5fe800640 1 --2- 172.21.15.123:0/2767175247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0dc70 0x56323cb15e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:49:04.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.717+0000 7fd5fde00640 1 --2- 172.21.15.123:0/2767175247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 0x56323cb16370 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0a4000 tx=0x7fd5ec09b350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1826cbf055935f72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:04.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.718+0000 7fd5dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:49:04.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.718+0000 7fd5dfe00640 1 -- 172.21.15.123:0/2767175247 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec097f30 con 0x56323cb0e590 2024-09-19T04:49:04.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.718+0000 7fd5dfe00640 1 -- 172.21.15.123:0/2767175247 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd5ec0ba270 con 0x56323cb0e590 2024-09-19T04:49:04.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.718+0000 7fd607e09740 1 -- 172.21.15.123:0/2767175247 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5ec0a4530 con 0x56323cb0e590 2024-09-19T04:49:04.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.718+0000 7fd5dfe00640 1 -- 172.21.15.123:0/2767175247 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd5ec09fc70 con 0x56323cb0e590 2024-09-19T04:49:04.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.718+0000 7fd607e09740 10 client.25816 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:04.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.718+0000 7fd607e09740 1 -- 172.21.15.123:0/2767175247 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56323cb1ba20 con 0x56323cb0e590 2024-09-19T04:49:04.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.718+0000 7fd5dfe00640 1 -- 172.21.15.123:0/2767175247 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ec0a4530 con 0x56323cb0e590 2024-09-19T04:49:04.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.719+0000 7fd5dfe00640 1 -- 172.21.15.123:0/2767175247 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fd5ec09ad10 con 0x56323cb0e590 2024-09-19T04:49:04.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.719+0000 7fd607e09740 10 client.25816 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:04.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.719+0000 7fd607e09740 10 client.25816 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:49:04.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.719+0000 7fd607e09740 20 client.25816 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:04.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.719+0000 7fd607e09740 1 --2- 172.21.15.123:0/2767175247 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x56323cb28cc0 0x56323cb2b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.719+0000 7fd607e09740 4 client.25816 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:49:04.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.719+0000 7fd607e09740 1 -- 172.21.15.123:0/2767175247 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56323ca81950 con 0x56323cb28cc0 2024-09-19T04:49:04.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.719+0000 7fd5fe800640 1 --2- 172.21.15.123:0/2767175247 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x56323cb28cc0 0x56323cb2b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:04.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.720+0000 7fd5fe800640 1 --2- 172.21.15.123:0/2767175247 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x56323cb28cc0 0x56323cb2b0b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3fa9201bf25260b1 server_cookie=d307d22c00d2adac in_seq=0 out_seq=0 2024-09-19T04:49:04.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.720+0000 7fd5dfe00640 10 client.25816 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:49:04.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.721+0000 7fd5dfe00640 1 -- 172.21.15.123:0/2767175247 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56323ca81950 con 0x56323cb28cc0 2024-09-19T04:49:04.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.721+0000 7fd5dfe00640 10 client.25816 handle_command_reply: tid=0 2024-09-19T04:49:04.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.721+0000 7fd607e09740 1 client.25816 shutdown 2024-09-19T04:49:04.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.721+0000 7fd607e09740 1 -- 172.21.15.123:0/2767175247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 msgr2=0x56323cb16370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:04.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.721+0000 7fd607e09740 1 --2- 172.21.15.123:0/2767175247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 0x56323cb16370 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0a4000 tx=0x7fd5ec09b350 comp rx=0 tx=0).stop 2024-09-19T04:49:04.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.721+0000 7fd607e09740 1 -- 172.21.15.123:0/2767175247 shutdown_connections 2024-09-19T04:49:04.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.721+0000 7fd607e09740 1 --2- 172.21.15.123:0/2767175247 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x56323cb28cc0 0x56323cb2b0b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.721+0000 7fd5dfe00640 0 client.25816 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:49:04.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.721+0000 7fd607e09740 1 --2- 172.21.15.123:0/2767175247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb18e50 0x56323cb168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.721+0000 7fd607e09740 1 --2- 172.21.15.123:0/2767175247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 0x56323cb16370 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.721+0000 7fd607e09740 1 --2- 172.21.15.123:0/2767175247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0dc70 0x56323cb15e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.722+0000 7fd607e09740 1 -- 172.21.15.123:0/2767175247 >> 172.21.15.123:0/2767175247 conn(0x56323cb0a200 msgr2=0x56323cb09380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:04.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.722+0000 7fd607e09740 1 -- 172.21.15.123:0/2767175247 shutdown_connections 2024-09-19T04:49:04.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.722+0000 7fd607e09740 1 -- 172.21.15.123:0/2767175247 wait complete. 2024-09-19T04:49:04.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.722+0000 7fd607e09740 20 client.25816 trim_cache size 0 max 16384 2024-09-19T04:49:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.748+0000 7fd607e09740 1 Processor -- start 2024-09-19T04:49:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.748+0000 7fd607e09740 1 -- start start 2024-09-19T04:49:04.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.749+0000 7fd607e09740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb0dc70 0x56323cb14700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.749+0000 7fd607e09740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 0x56323cb30840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.749+0000 7fd607e09740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb18e50 0x56323cb30d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.749+0000 7fd607e09740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001b3f40 con 0x56323cb0dc70 2024-09-19T04:49:04.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.749+0000 7fd607e09740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001b5bd0 con 0x56323cb18e50 2024-09-19T04:49:04.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.749+0000 7fd607e09740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001b4610 con 0x56323cb0e590 2024-09-19T04:49:04.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.749+0000 7fd5fe800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb0dc70 0x56323cb14700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:04.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.749+0000 7fd5fde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 0x56323cb30840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:04.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.749+0000 7fd5fde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 0x56323cb30840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53132/0 (socket says 172.21.15.123:53132) 2024-09-19T04:49:04.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.749+0000 7fd5fde00640 1 -- 172.21.15.123:0/970350438 learned_addr learned my addr 172.21.15.123:0/970350438 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:04.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.749+0000 7fd5ff200640 1 --2- 172.21.15.123:0/970350438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb18e50 0x56323cb30d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:04.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.750+0000 7fd5fde00640 1 -- 172.21.15.123:0/970350438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb18e50 msgr2=0x56323cb30d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:04.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.750+0000 7fd5fde00640 1 --2- 172.21.15.123:0/970350438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb18e50 0x56323cb30d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.750+0000 7fd5fde00640 1 -- 172.21.15.123:0/970350438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb0dc70 msgr2=0x56323cb14700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:04.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.750+0000 7fd5fde00640 1 --2- 172.21.15.123:0/970350438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb0dc70 0x56323cb14700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.750+0000 7fd5fde00640 1 -- 172.21.15.123:0/970350438 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56323cb05770 con 0x56323cb0e590 2024-09-19T04:49:04.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.750+0000 7fd5fde00640 1 --2- 172.21.15.123:0/970350438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 0x56323cb30840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec09b5c0 tx=0x7fd5ec0cce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50ea55fb4dbde6cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:04.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.750+0000 7fd5eaa00640 1 -- 172.21.15.123:0/970350438 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec0b4e60 con 0x56323cb0e590 2024-09-19T04:49:04.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.751+0000 7fd5eaa00640 1 -- 172.21.15.123:0/970350438 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ec0b9070 con 0x56323cb0e590 2024-09-19T04:49:04.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.751+0000 7fd5eaa00640 1 -- 172.21.15.123:0/970350438 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec097f30 con 0x56323cb0e590 2024-09-19T04:49:04.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.751+0000 7fd607e09740 1 -- 172.21.15.123:0/970350438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 msgr2=0x56323cb30840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:04.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.751+0000 7fd607e09740 1 --2- 172.21.15.123:0/970350438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 0x56323cb30840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec09b5c0 tx=0x7fd5ec0cce30 comp rx=0 tx=0).stop 2024-09-19T04:49:04.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.751+0000 7fd607e09740 1 -- 172.21.15.123:0/970350438 shutdown_connections 2024-09-19T04:49:04.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.751+0000 7fd607e09740 1 --2- 172.21.15.123:0/970350438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb18e50 0x56323cb30d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.751+0000 7fd607e09740 1 --2- 172.21.15.123:0/970350438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 0x56323cb30840 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.751+0000 7fd607e09740 1 --2- 172.21.15.123:0/970350438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb0dc70 0x56323cb14700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.751+0000 7fd607e09740 1 -- 172.21.15.123:0/970350438 >> 172.21.15.123:0/970350438 conn(0x56323cb0a200 msgr2=0x56323cb17eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:04.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.751+0000 7fd607e09740 1 -- 172.21.15.123:0/970350438 shutdown_connections 2024-09-19T04:49:04.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.752+0000 7fd607e09740 1 -- 172.21.15.123:0/970350438 wait complete. 2024-09-19T04:49:04.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.752+0000 7fd607e09740 1 -- start start 2024-09-19T04:49:04.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.752+0000 7fd607e09740 1 Processor -- start 2024-09-19T04:49:04.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.753+0000 7fd607e09740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0dc70 0x56323cb1ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.753+0000 7fd607e09740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 0x56323cb1d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.753+0000 7fd607e09740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb18e50 0x56323cb1d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.753+0000 7fd607e09740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001b5440 con 0x56323cb18e50 2024-09-19T04:49:04.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.753+0000 7fd607e09740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001b4d70 con 0x56323cb0dc70 2024-09-19T04:49:04.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.753+0000 7fd607e09740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001b1f20 con 0x56323cb0e590 2024-09-19T04:49:04.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.753+0000 7fd607e09740 10 client.0 _ll_register_callbacks cb 0x7fd5ec09b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:04.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.753+0000 7fd5fe800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0dc70 0x56323cb1ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:04.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.753+0000 7fd5fe800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0dc70 0x56323cb1ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:58540/0 (socket says 172.21.15.123:58540) 2024-09-19T04:49:04.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.753+0000 7fd5fe800640 1 -- 172.21.15.123:0/1698720545 learned_addr learned my addr 172.21.15.123:0/1698720545 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:04.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.753+0000 7fd5fde00640 1 --2- 172.21.15.123:0/1698720545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 0x56323cb1d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:04.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.753+0000 7fd5ff200640 1 --2- 172.21.15.123:0/1698720545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb18e50 0x56323cb1d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:04.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.753+0000 7fd5fe800640 1 -- 172.21.15.123:0/1698720545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 msgr2=0x56323cb1d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:04.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.754+0000 7fd5fe800640 1 --2- 172.21.15.123:0/1698720545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 0x56323cb1d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.754+0000 7fd5fe800640 1 -- 172.21.15.123:0/1698720545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb18e50 msgr2=0x56323cb1d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:04.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.754+0000 7fd5fe800640 1 --2- 172.21.15.123:0/1698720545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb18e50 0x56323cb1d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.754+0000 7fd5fe800640 1 -- 172.21.15.123:0/1698720545 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56323cb92980 con 0x56323cb0dc70 2024-09-19T04:49:04.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.754+0000 7fd5ff200640 1 --2- 172.21.15.123:0/1698720545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb18e50 0x56323cb1d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:49:04.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.754+0000 7fd5fe800640 1 --2- 172.21.15.123:0/1698720545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0dc70 0x56323cb1ccb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00096c0 tx=0x7fd5f002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f6444766205d633 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:04.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.754+0000 7fd5dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:49:04.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.754+0000 7fd5dfe00640 1 -- 172.21.15.123:0/1698720545 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0035040 con 0x56323cb0dc70 2024-09-19T04:49:04.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.754+0000 7fd5dfe00640 1 -- 172.21.15.123:0/1698720545 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd5f00035f0 con 0x56323cb0dc70 2024-09-19T04:49:04.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.754+0000 7fd607e09740 1 -- 172.21.15.123:0/1698720545 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5ec0b9070 con 0x56323cb0dc70 2024-09-19T04:49:04.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.755+0000 7fd5dfe00640 1 -- 172.21.15.123:0/1698720545 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd5f00037f0 con 0x56323cb0dc70 2024-09-19T04:49:04.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.755+0000 7fd607e09740 10 client.25805 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:04.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.755+0000 7fd607e09740 1 -- 172.21.15.123:0/1698720545 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56323cb93ca0 con 0x56323cb0dc70 2024-09-19T04:49:04.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.755+0000 7fd5dfe00640 1 -- 172.21.15.123:0/1698720545 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ec0b9070 con 0x56323cb0dc70 2024-09-19T04:49:04.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.755+0000 7fd5dfe00640 1 -- 172.21.15.123:0/1698720545 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fd5f003cab0 con 0x56323cb0dc70 2024-09-19T04:49:04.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.755+0000 7fd607e09740 10 client.25805 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:04.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.755+0000 7fd607e09740 10 client.25805 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:49:04.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.755+0000 7fd607e09740 20 client.25805 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:04.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.755+0000 7fd607e09740 1 --2- 172.21.15.123:0/1698720545 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x56323cb28cc0 0x56323cb96430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:04.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.755+0000 7fd607e09740 4 client.25805 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:49:04.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.755+0000 7fd607e09740 1 -- 172.21.15.123:0/1698720545 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56323ca7be60 con 0x56323cb28cc0 2024-09-19T04:49:04.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.755+0000 7fd5fde00640 1 --2- 172.21.15.123:0/1698720545 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x56323cb28cc0 0x56323cb96430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:04.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.756+0000 7fd5dfe00640 10 client.25805 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:49:04.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.756+0000 7fd5fde00640 1 --2- 172.21.15.123:0/1698720545 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x56323cb28cc0 0x56323cb96430 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53f5a5227883c031 server_cookie=6355c7529bcc837c in_seq=0 out_seq=0 2024-09-19T04:49:04.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.756+0000 7fd5dfe00640 1 -- 172.21.15.123:0/1698720545 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56323ca7be60 con 0x56323cb28cc0 2024-09-19T04:49:04.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.756+0000 7fd5dfe00640 10 client.25805 handle_command_reply: tid=0 2024-09-19T04:49:04.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.756+0000 7fd607e09740 1 client.25805 shutdown 2024-09-19T04:49:04.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.757+0000 7fd607e09740 1 -- 172.21.15.123:0/1698720545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0dc70 msgr2=0x56323cb1ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:04.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.757+0000 7fd607e09740 1 --2- 172.21.15.123:0/1698720545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0dc70 0x56323cb1ccb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00096c0 tx=0x7fd5f002a410 comp rx=0 tx=0).stop 2024-09-19T04:49:04.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.757+0000 7fd607e09740 1 -- 172.21.15.123:0/1698720545 shutdown_connections 2024-09-19T04:49:04.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.757+0000 7fd607e09740 1 --2- 172.21.15.123:0/1698720545 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x56323cb28cc0 0x56323cb96430 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.757+0000 7fd607e09740 1 --2- 172.21.15.123:0/1698720545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56323cb18e50 0x56323cb1d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.757+0000 7fd607e09740 1 --2- 172.21.15.123:0/1698720545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56323cb0e590 0x56323cb1d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.757+0000 7fd607e09740 1 --2- 172.21.15.123:0/1698720545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56323cb0dc70 0x56323cb1ccb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.757+0000 7fd607e09740 1 -- 172.21.15.123:0/1698720545 >> 172.21.15.123:0/1698720545 conn(0x56323cb0a200 msgr2=0x56323cb23c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:04.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.757+0000 7fd5dfe00640 0 client.25805 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:49:04.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.757+0000 7fd607e09740 1 -- 172.21.15.123:0/1698720545 shutdown_connections 2024-09-19T04:49:04.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.757+0000 7fd607e09740 1 -- 172.21.15.123:0/1698720545 wait complete. 2024-09-19T04:49:04.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.757+0000 7fd607e09740 20 client.25805 trim_cache size 0 max 16384 2024-09-19T04:49:04.765 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:49:04.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.758+0000 7fd605400640 1 -- 172.21.15.123:0/3494378023 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd5e40a3430 msgr2=0x7fd5e40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:04.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.758+0000 7fd605400640 1 --2- 172.21.15.123:0/3494378023 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd5e40a3430 0x7fd5e40a58f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd60019fa20 tx=0x7fd5f00058d0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.758+0000 7fd605400640 1 -- 172.21.15.123:0/3494378023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd600077040 msgr2=0x7fd6001a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:04.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.758+0000 7fd605400640 1 --2- 172.21.15.123:0/3494378023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd600077040 0x7fd6001a4c60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00d950 tx=0x7fd5ec00de10 comp rx=0 tx=0).stop 2024-09-19T04:49:04.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.758+0000 7fd605400640 1 -- 172.21.15.123:0/3494378023 shutdown_connections 2024-09-19T04:49:04.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.758+0000 7fd605400640 1 --2- 172.21.15.123:0/3494378023 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd5e40a3430 0x7fd5e40a58f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.758+0000 7fd605400640 1 --2- 172.21.15.123:0/3494378023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd600112a30 0x7fd60019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.758+0000 7fd605400640 1 --2- 172.21.15.123:0/3494378023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd600077040 0x7fd6001a4c60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.758+0000 7fd605400640 1 --2- 172.21.15.123:0/3494378023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd600076720 0x7fd6001a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:04.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.758+0000 7fd605400640 1 -- 172.21.15.123:0/3494378023 >> 172.21.15.123:0/3494378023 conn(0x7fd6000ffa90 msgr2=0x7fd600101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:04.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.759+0000 7fd605400640 1 -- 172.21.15.123:0/3494378023 shutdown_connections 2024-09-19T04:49:04.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:04.759+0000 7fd605400640 1 -- 172.21.15.123:0/3494378023 wait complete. 2024-09-19T04:49:05.763 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:05 smithi123 ceph-mon[30511]: pgmap v74: 97 pgs: 97 active+clean; 70 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 1.0 MiB/s wr, 159 op/s 2024-09-19T04:49:05.941 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:49:05.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:05 smithi193 ceph-mon[35592]: pgmap v74: 97 pgs: 97 active+clean; 70 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 1.0 MiB/s wr, 159 op/s 2024-09-19T04:49:06.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:05 smithi183 ceph-mon[35333]: pgmap v74: 97 pgs: 97 active+clean; 70 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 1.0 MiB/s wr, 159 op/s 2024-09-19T04:49:06.329 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:49:07.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.083+0000 7f95ef400640 1 -- 172.21.15.123:0/3687661614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95e8106a80 msgr2=0x7f95e8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:07.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.083+0000 7f95ef400640 1 --2- 172.21.15.123:0/3687661614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95e8106a80 0x7f95e8106e60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009900 tx=0x7f95dc02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.083+0000 7f95ef400640 1 -- 172.21.15.123:0/3687661614 shutdown_connections 2024-09-19T04:49:07.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.083+0000 7f95ef400640 1 --2- 172.21.15.123:0/3687661614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e8101420 0x7f95e810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.083+0000 7f95ef400640 1 --2- 172.21.15.123:0/3687661614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95e8100a80 0x7f95e8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.083+0000 7f95ef400640 1 --2- 172.21.15.123:0/3687661614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95e8106a80 0x7f95e8106e60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.083+0000 7f95ef400640 1 -- 172.21.15.123:0/3687661614 >> 172.21.15.123:0/3687661614 conn(0x7f95e80fc7a0 msgr2=0x7f95e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:07.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.083+0000 7f95ef400640 1 -- 172.21.15.123:0/3687661614 shutdown_connections 2024-09-19T04:49:07.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.083+0000 7f95ef400640 1 -- 172.21.15.123:0/3687661614 wait complete. 2024-09-19T04:49:07.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.084+0000 7f95ef400640 1 Processor -- start 2024-09-19T04:49:07.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.084+0000 7f95ef400640 1 -- start start 2024-09-19T04:49:07.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.084+0000 7f95ef400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95e8100a80 0x7f95e819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.084+0000 7f95ef400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95e8101420 0x7f95e819ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.084+0000 7f95ef400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e8106a80 0x7f95e819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.084+0000 7f95ef400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95e8110ed0 con 0x7f95e8106a80 2024-09-19T04:49:07.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.084+0000 7f95ef400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f95e8110d50 con 0x7f95e8101420 2024-09-19T04:49:07.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.084+0000 7f95ef400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f95e8111050 con 0x7f95e8100a80 2024-09-19T04:49:07.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.085+0000 7f95e7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95e8101420 0x7f95e819ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:07.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.085+0000 7f95e7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95e8101420 0x7f95e819ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:58556/0 (socket says 172.21.15.123:58556) 2024-09-19T04:49:07.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.085+0000 7f95e7e00640 1 -- 172.21.15.123:0/3658214781 learned_addr learned my addr 172.21.15.123:0/3658214781 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:07.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.085+0000 7f95ed400640 1 --2- 172.21.15.123:0/3658214781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e8106a80 0x7f95e819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:07.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.085+0000 7f95e7e00640 1 -- 172.21.15.123:0/3658214781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95e8100a80 msgr2=0x7f95e819a940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:49:07.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.085+0000 7f95e7e00640 1 --2- 172.21.15.123:0/3658214781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95e8100a80 0x7f95e819a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.085+0000 7f95e7e00640 1 -- 172.21.15.123:0/3658214781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e8106a80 msgr2=0x7f95e819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:07.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.085+0000 7f95e7e00640 1 --2- 172.21.15.123:0/3658214781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e8106a80 0x7f95e819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.085+0000 7f95e7e00640 1 -- 172.21.15.123:0/3658214781 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e819f690 con 0x7f95e8101420 2024-09-19T04:49:07.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.085+0000 7f95e7e00640 1 --2- 172.21.15.123:0/3658214781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95e8101420 0x7f95e819ae80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f95d400e9f0 tx=0x7f95d400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:07.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.085+0000 7f95e5600640 1 -- 172.21.15.123:0/3658214781 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d400cd80 con 0x7f95e8101420 2024-09-19T04:49:07.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.086+0000 7f95ef400640 1 -- 172.21.15.123:0/3658214781 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e819f980 con 0x7f95e8101420 2024-09-19T04:49:07.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.086+0000 7f95e5600640 1 -- 172.21.15.123:0/3658214781 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95d4005500 con 0x7f95e8101420 2024-09-19T04:49:07.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.086+0000 7f95e5600640 1 -- 172.21.15.123:0/3658214781 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d4010690 con 0x7f95e8101420 2024-09-19T04:49:07.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.086+0000 7f95ef400640 1 -- 172.21.15.123:0/3658214781 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e81a71d0 con 0x7f95e8101420 2024-09-19T04:49:07.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.088+0000 7f95f1e36740 1 Processor -- start 2024-09-19T04:49:07.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.089+0000 7f95f1e36740 1 -- start start 2024-09-19T04:49:07.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.089+0000 7f95f1e36740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c91c70 0x5581d5c92050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.089+0000 7f95f1e36740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c92590 0x5581d5c9c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.089+0000 7f95e5600640 1 -- 172.21.15.123:0/3658214781 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f95d40046c0 con 0x7f95e8101420 2024-09-19T04:49:07.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.089+0000 7f95f1e36740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c9ce50 0x5581d5c9f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.089+0000 7f95f1e36740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5581d5bf83e0 con 0x5581d5c9ce50 2024-09-19T04:49:07.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.089+0000 7f95f1e36740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5581d5b9fd80 con 0x5581d5c91c70 2024-09-19T04:49:07.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.089+0000 7f95f1e36740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5581d5bf43a0 con 0x5581d5c92590 2024-09-19T04:49:07.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.089+0000 7f95ed400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c91c70 0x5581d5c92050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:07.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.089+0000 7f95e7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c9ce50 0x5581d5c9f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:07.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.089+0000 7f95ed400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c91c70 0x5581d5c92050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:58566/0 (socket says 172.21.15.123:58566) 2024-09-19T04:49:07.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.089+0000 7f95ed400640 1 -- 172.21.15.123:0/1223011633 learned_addr learned my addr 172.21.15.123:0/1223011633 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:07.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.090+0000 7f95eca00640 1 --2- 172.21.15.123:0/1223011633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c92590 0x5581d5c9c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:07.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.090+0000 7f95ed400640 1 -- 172.21.15.123:0/1223011633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c92590 msgr2=0x5581d5c9c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:07.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.090+0000 7f95ed400640 1 --2- 172.21.15.123:0/1223011633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c92590 0x5581d5c9c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.090+0000 7f95ed400640 1 -- 172.21.15.123:0/1223011633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c9ce50 msgr2=0x5581d5c9f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:07.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.090+0000 7f95ed400640 1 --2- 172.21.15.123:0/1223011633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c9ce50 0x5581d5c9f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.090+0000 7f95ed400640 1 -- 172.21.15.123:0/1223011633 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d5bbeae0 con 0x5581d5c91c70 2024-09-19T04:49:07.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.090+0000 7f95e7e00640 1 --2- 172.21.15.123:0/1223011633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c9ce50 0x5581d5c9f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:49:07.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.090+0000 7f95ed400640 1 --2- 172.21.15.123:0/1223011633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c91c70 0x5581d5c92050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009f50 tx=0x7f95e00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f9f6cc4b8bfa8a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.090+0000 7f95d9c00640 1 -- 172.21.15.123:0/1223011633 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0004d20 con 0x5581d5c91c70 2024-09-19T04:49:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.090+0000 7f95d9c00640 1 -- 172.21.15.123:0/1223011633 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e0004050 con 0x5581d5c91c70 2024-09-19T04:49:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.090+0000 7f95d9c00640 1 -- 172.21.15.123:0/1223011633 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0005280 con 0x5581d5c91c70 2024-09-19T04:49:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.091+0000 7f95e5600640 1 --2- 172.21.15.123:0/3658214781 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f95d00a33b0 0x7f95d00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.091+0000 7f95f1e36740 1 -- 172.21.15.123:0/1223011633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c91c70 msgr2=0x5581d5c92050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.091+0000 7f95f1e36740 1 --2- 172.21.15.123:0/1223011633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c91c70 0x5581d5c92050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009f50 tx=0x7f95e00118d0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.091+0000 7f95e5600640 1 -- 172.21.15.123:0/3658214781 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f95d40d0180 con 0x7f95e8101420 2024-09-19T04:49:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.091+0000 7f95f1e36740 1 -- 172.21.15.123:0/1223011633 shutdown_connections 2024-09-19T04:49:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.091+0000 7f95f1e36740 1 --2- 172.21.15.123:0/1223011633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c9ce50 0x5581d5c9f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.091+0000 7f95f1e36740 1 --2- 172.21.15.123:0/1223011633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c92590 0x5581d5c9c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.091+0000 7f95f1e36740 1 --2- 172.21.15.123:0/1223011633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c91c70 0x5581d5c92050 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.091+0000 7f95f1e36740 1 -- 172.21.15.123:0/1223011633 >> 172.21.15.123:0/1223011633 conn(0x5581d5c8e200 msgr2=0x5581d5c8e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:07.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.091+0000 7f95eca00640 1 --2- 172.21.15.123:0/3658214781 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f95d00a33b0 0x7f95d00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:07.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.091+0000 7f95f1e36740 1 -- 172.21.15.123:0/1223011633 shutdown_connections 2024-09-19T04:49:07.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.092+0000 7f95f1e36740 1 -- 172.21.15.123:0/1223011633 wait complete. 2024-09-19T04:49:07.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.092+0000 7f95eca00640 1 --2- 172.21.15.123:0/3658214781 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f95d00a33b0 0x7f95d00a5870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f95dc006010 tx=0x7f95dc000bc0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:07.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.092+0000 7f95f1e36740 1 -- start start 2024-09-19T04:49:07.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.093+0000 7f95f1e36740 1 Processor -- start 2024-09-19T04:49:07.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.093+0000 7f95f1e36740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c91c70 0x5581d5c99d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.093+0000 7f95f1e36740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c92590 0x5581d5c9a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.093+0000 7f95f1e36740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c9ce50 0x5581d5c9a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.093+0000 7f95f1e36740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5581d5c9fa20 con 0x5581d5c92590 2024-09-19T04:49:07.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.093+0000 7f95f1e36740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5581d5bb30f0 con 0x5581d5c9ce50 2024-09-19T04:49:07.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.093+0000 7f95f1e36740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5581d5c9fba0 con 0x5581d5c91c70 2024-09-19T04:49:07.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.093+0000 7f95f1e36740 10 client.0 _ll_register_callbacks cb 0x5581d5b9abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:07.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.093+0000 7f95eca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c91c70 0x5581d5c99d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:07.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.094+0000 7f95eca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c91c70 0x5581d5c99d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53172/0 (socket says 172.21.15.123:53172) 2024-09-19T04:49:07.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.094+0000 7f95eca00640 1 -- 172.21.15.123:0/431195041 learned_addr learned my addr 172.21.15.123:0/431195041 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:07.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.094+0000 7f95e7e00640 1 --2- 172.21.15.123:0/431195041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c92590 0x5581d5c9a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:07.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.094+0000 7f95ed400640 1 --2- 172.21.15.123:0/431195041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c9ce50 0x5581d5c9a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:07.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.094+0000 7f95eca00640 1 -- 172.21.15.123:0/431195041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c9ce50 msgr2=0x5581d5c9a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:07.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.094+0000 7f95eca00640 1 --2- 172.21.15.123:0/431195041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c9ce50 0x5581d5c9a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.094+0000 7f95eca00640 1 -- 172.21.15.123:0/431195041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c92590 msgr2=0x5581d5c9a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:07.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.094+0000 7f95eca00640 1 --2- 172.21.15.123:0/431195041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c92590 0x5581d5c9a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.094+0000 7f95eca00640 1 -- 172.21.15.123:0/431195041 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d59693c0 con 0x5581d5c91c70 2024-09-19T04:49:07.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.094+0000 7f95eca00640 1 --2- 172.21.15.123:0/431195041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c91c70 0x5581d5c99d00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f95dc033000 tx=0x7f95dc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f129f7f43011d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:07.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.095+0000 7f95ce000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:49:07.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.095+0000 7f95ce000640 1 -- 172.21.15.123:0/431195041 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc035040 con 0x5581d5c91c70 2024-09-19T04:49:07.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.095+0000 7f95ce000640 1 -- 172.21.15.123:0/431195041 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f95dc004b90 con 0x5581d5c91c70 2024-09-19T04:49:07.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.095+0000 7f95f1e36740 1 -- 172.21.15.123:0/431195041 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e0004050 con 0x5581d5c91c70 2024-09-19T04:49:07.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.095+0000 7f95ce000640 1 -- 172.21.15.123:0/431195041 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f95dc056050 con 0x5581d5c91c70 2024-09-19T04:49:07.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.095+0000 7f95f1e36740 10 client.25831 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:07.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.095+0000 7f95f1e36740 1 -- 172.21.15.123:0/431195041 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5581d5cb4240 con 0x5581d5c91c70 2024-09-19T04:49:07.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.096+0000 7f95ce000640 1 -- 172.21.15.123:0/431195041 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95dc04d070 con 0x5581d5c91c70 2024-09-19T04:49:07.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.096+0000 7f95ce000640 1 -- 172.21.15.123:0/431195041 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f95dc03a040 con 0x5581d5c91c70 2024-09-19T04:49:07.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.096+0000 7f95f1e36740 10 client.25831 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:07.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.096+0000 7f95f1e36740 10 client.25831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:49:07.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.096+0000 7f95f1e36740 20 client.25831 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:07.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.096+0000 7f95f1e36740 1 --2- 172.21.15.123:0/431195041 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5581d5cb57d0 0x5581d5cb7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.096+0000 7f95f1e36740 4 client.25831 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:49:07.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.096+0000 7f95f1e36740 1 -- 172.21.15.123:0/431195041 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f95e0004050 con 0x5581d5cb57d0 2024-09-19T04:49:07.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.096+0000 7f95e7e00640 1 --2- 172.21.15.123:0/431195041 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5581d5cb57d0 0x5581d5cb7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:07.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.097+0000 7f95ce000640 10 client.25831 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.097+0000 7f95e7e00640 1 --2- 172.21.15.123:0/431195041 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5581d5cb57d0 0x5581d5cb7c70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b35d9f30c1908dd server_cookie=c9bdb0d2625433c6 in_seq=0 out_seq=0 2024-09-19T04:49:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.098+0000 7f95ce000640 1 -- 172.21.15.123:0/431195041 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f95e0004050 con 0x5581d5cb57d0 2024-09-19T04:49:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.098+0000 7f95ce000640 10 client.25831 handle_command_reply: tid=0 2024-09-19T04:49:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.098+0000 7f95f1e36740 1 client.25831 shutdown 2024-09-19T04:49:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.098+0000 7f95f1e36740 1 -- 172.21.15.123:0/431195041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c91c70 msgr2=0x5581d5c99d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.098+0000 7f95f1e36740 1 --2- 172.21.15.123:0/431195041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c91c70 0x5581d5c99d00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f95dc033000 tx=0x7f95dc009260 comp rx=0 tx=0).stop 2024-09-19T04:49:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.099+0000 7f95f1e36740 1 -- 172.21.15.123:0/431195041 shutdown_connections 2024-09-19T04:49:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.099+0000 7f95f1e36740 1 --2- 172.21.15.123:0/431195041 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5581d5cb57d0 0x5581d5cb7c70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.099+0000 7f95f1e36740 1 --2- 172.21.15.123:0/431195041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c9ce50 0x5581d5c9a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.099+0000 7f95f1e36740 1 --2- 172.21.15.123:0/431195041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c92590 0x5581d5c9a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.099+0000 7f95f1e36740 1 --2- 172.21.15.123:0/431195041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c91c70 0x5581d5c99d00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.099+0000 7f95f1e36740 1 -- 172.21.15.123:0/431195041 >> 172.21.15.123:0/431195041 conn(0x5581d5c8e200 msgr2=0x5581d5c93f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.099+0000 7f95ce000640 0 client.25831 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.099+0000 7f95f1e36740 1 -- 172.21.15.123:0/431195041 shutdown_connections 2024-09-19T04:49:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.099+0000 7f95f1e36740 1 -- 172.21.15.123:0/431195041 wait complete. 2024-09-19T04:49:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.099+0000 7f95f1e36740 20 client.25831 trim_cache size 0 max 16384 2024-09-19T04:49:07.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.113+0000 7f95f1e36740 1 Processor -- start 2024-09-19T04:49:07.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.113+0000 7f95f1e36740 1 -- start start 2024-09-19T04:49:07.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.114+0000 7f95f1e36740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c91c70 0x5581d5c94770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.114+0000 7f95f1e36740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c92590 0x5581d5cbd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.114+0000 7f95f1e36740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c9ce50 0x5581d5cbda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.114+0000 7f95f1e36740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95e81ab5a0 con 0x5581d5c9ce50 2024-09-19T04:49:07.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.114+0000 7f95f1e36740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f95e81ad230 con 0x5581d5c92590 2024-09-19T04:49:07.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.114+0000 7f95f1e36740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f95e81abc70 con 0x5581d5c91c70 2024-09-19T04:49:07.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.114+0000 7f95eca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c91c70 0x5581d5c94770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:07.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.114+0000 7f95ed400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c9ce50 0x5581d5cbda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:07.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.114+0000 7f95eca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c91c70 0x5581d5c94770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53182/0 (socket says 172.21.15.123:53182) 2024-09-19T04:49:07.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.114+0000 7f95e7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c92590 0x5581d5cbd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:07.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.114+0000 7f95eca00640 1 -- 172.21.15.123:0/2278128780 learned_addr learned my addr 172.21.15.123:0/2278128780 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:07.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.114+0000 7f95eca00640 1 -- 172.21.15.123:0/2278128780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c92590 msgr2=0x5581d5cbd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:07.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.114+0000 7f95eca00640 1 --2- 172.21.15.123:0/2278128780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c92590 0x5581d5cbd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.114+0000 7f95eca00640 1 -- 172.21.15.123:0/2278128780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c9ce50 msgr2=0x5581d5cbda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:07.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.114+0000 7f95eca00640 1 --2- 172.21.15.123:0/2278128780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c9ce50 0x5581d5cbda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.114+0000 7f95eca00640 1 -- 172.21.15.123:0/2278128780 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d5c89770 con 0x5581d5c91c70 2024-09-19T04:49:07.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.115+0000 7f95eca00640 1 --2- 172.21.15.123:0/2278128780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c91c70 0x5581d5c94770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f95dc005d00 tx=0x7f95dc0325f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb47f5f555ad7cf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:07.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.115+0000 7f95d9200640 1 -- 172.21.15.123:0/2278128780 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc05f070 con 0x5581d5c91c70 2024-09-19T04:49:07.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.115+0000 7f95d9200640 1 -- 172.21.15.123:0/2278128780 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95dc04d040 con 0x5581d5c91c70 2024-09-19T04:49:07.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.115+0000 7f95d9200640 1 -- 172.21.15.123:0/2278128780 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc035030 con 0x5581d5c91c70 2024-09-19T04:49:07.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.116+0000 7f95f1e36740 1 -- 172.21.15.123:0/2278128780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c91c70 msgr2=0x5581d5c94770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:07.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.116+0000 7f95f1e36740 1 --2- 172.21.15.123:0/2278128780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c91c70 0x5581d5c94770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f95dc005d00 tx=0x7f95dc0325f0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.116+0000 7f95f1e36740 1 -- 172.21.15.123:0/2278128780 shutdown_connections 2024-09-19T04:49:07.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.116+0000 7f95f1e36740 1 --2- 172.21.15.123:0/2278128780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c9ce50 0x5581d5cbda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.116+0000 7f95f1e36740 1 --2- 172.21.15.123:0/2278128780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c92590 0x5581d5cbd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.116+0000 7f95f1e36740 1 --2- 172.21.15.123:0/2278128780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c91c70 0x5581d5c94770 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.116+0000 7f95f1e36740 1 -- 172.21.15.123:0/2278128780 >> 172.21.15.123:0/2278128780 conn(0x5581d5c8e200 msgr2=0x5581d5c98df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:07.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.116+0000 7f95f1e36740 1 -- 172.21.15.123:0/2278128780 shutdown_connections 2024-09-19T04:49:07.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.116+0000 7f95f1e36740 1 -- 172.21.15.123:0/2278128780 wait complete. 2024-09-19T04:49:07.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.116+0000 7f95f1e36740 1 -- start start 2024-09-19T04:49:07.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.117+0000 7f95f1e36740 1 Processor -- start 2024-09-19T04:49:07.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.117+0000 7f95f1e36740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c91c70 0x5581d5ca5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.117+0000 7f95f1e36740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c92590 0x5581d5ca0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.117+0000 7f95f1e36740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c9ce50 0x5581d5ca0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.117+0000 7f95f1e36740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95e81acaa0 con 0x5581d5c92590 2024-09-19T04:49:07.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.117+0000 7f95f1e36740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f95e81ac3d0 con 0x5581d5c91c70 2024-09-19T04:49:07.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.117+0000 7f95f1e36740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f95e81a95c0 con 0x5581d5c9ce50 2024-09-19T04:49:07.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.117+0000 7f95f1e36740 10 client.0 _ll_register_callbacks cb 0x7f95e00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:07.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.118+0000 7f95e7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c92590 0x5581d5ca0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:07.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.118+0000 7f95e7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c92590 0x5581d5ca0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46064/0 (socket says 172.21.15.123:46064) 2024-09-19T04:49:07.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.118+0000 7f95e7e00640 1 -- 172.21.15.123:0/1284725369 learned_addr learned my addr 172.21.15.123:0/1284725369 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:07.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.118+0000 7f95ed400640 1 --2- 172.21.15.123:0/1284725369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c9ce50 0x5581d5ca0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:07.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.118+0000 7f95eca00640 1 --2- 172.21.15.123:0/1284725369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c91c70 0x5581d5ca5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:07.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.119+0000 7f95e7e00640 1 -- 172.21.15.123:0/1284725369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c9ce50 msgr2=0x5581d5ca0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:07.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.119+0000 7f95e7e00640 1 --2- 172.21.15.123:0/1284725369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c9ce50 0x5581d5ca0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.119+0000 7f95e7e00640 1 -- 172.21.15.123:0/1284725369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c91c70 msgr2=0x5581d5ca5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:07.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.119+0000 7f95e7e00640 1 --2- 172.21.15.123:0/1284725369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c91c70 0x5581d5ca5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.119+0000 7f95e7e00640 1 -- 172.21.15.123:0/1284725369 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d5ca1000 con 0x5581d5c92590 2024-09-19T04:49:07.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.119+0000 7f95e7e00640 1 --2- 172.21.15.123:0/1284725369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c92590 0x5581d5ca0580 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f95d409c190 tx=0x7f95d4005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72da26ab54daae21 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:07.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.119+0000 7f95ce000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:49:07.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.119+0000 7f95ce000640 1 -- 172.21.15.123:0/1284725369 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d409c260 con 0x5581d5c92590 2024-09-19T04:49:07.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.120+0000 7f95ce000640 1 -- 172.21.15.123:0/1284725369 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f95d4098b00 con 0x5581d5c92590 2024-09-19T04:49:07.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.120+0000 7f95f1e36740 1 -- 172.21.15.123:0/1284725369 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95dc04d040 con 0x5581d5c92590 2024-09-19T04:49:07.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.121+0000 7f95ce000640 1 -- 172.21.15.123:0/1284725369 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f95d409bdf0 con 0x5581d5c92590 2024-09-19T04:49:07.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.121+0000 7f95f1e36740 10 client.15783 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:07.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.121+0000 7f95f1e36740 1 -- 172.21.15.123:0/1284725369 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5581d5d20280 con 0x5581d5c92590 2024-09-19T04:49:07.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.122+0000 7f95ce000640 1 -- 172.21.15.123:0/1284725369 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95dc04d040 con 0x5581d5c92590 2024-09-19T04:49:07.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.122+0000 7f95ce000640 1 -- 172.21.15.123:0/1284725369 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f95d40af060 con 0x5581d5c92590 2024-09-19T04:49:07.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.122+0000 7f95f1e36740 10 client.15783 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:07.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.122+0000 7f95f1e36740 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:49:07.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.122+0000 7f95f1e36740 20 client.15783 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:07.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.122+0000 7f95f1e36740 1 --2- 172.21.15.123:0/1284725369 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5581d5d20a70 0x5581d5d22e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:07.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.122+0000 7f95f1e36740 4 client.15783 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:49:07.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.122+0000 7f95f1e36740 1 -- 172.21.15.123:0/1284725369 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581d5c05950 con 0x5581d5d20a70 2024-09-19T04:49:07.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.122+0000 7f95eca00640 1 --2- 172.21.15.123:0/1284725369 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5581d5d20a70 0x5581d5d22e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:07.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.123+0000 7f95ce000640 10 client.15783 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:07.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.123+0000 7f95eca00640 1 --2- 172.21.15.123:0/1284725369 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5581d5d20a70 0x5581d5d22e90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9c5f3f8df14b2e8 server_cookie=717f2edce3c0a14 in_seq=0 out_seq=0 2024-09-19T04:49:07.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.123+0000 7f95ce000640 1 -- 172.21.15.123:0/1284725369 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581d5c05950 con 0x5581d5d20a70 2024-09-19T04:49:07.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.123+0000 7f95ce000640 10 client.15783 handle_command_reply: tid=0 2024-09-19T04:49:07.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.123+0000 7f95f1e36740 1 client.15783 shutdown 2024-09-19T04:49:07.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.124+0000 7f95f1e36740 1 -- 172.21.15.123:0/1284725369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c92590 msgr2=0x5581d5ca0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:07.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.124+0000 7f95f1e36740 1 --2- 172.21.15.123:0/1284725369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c92590 0x5581d5ca0580 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f95d409c190 tx=0x7f95d4005f10 comp rx=0 tx=0).stop 2024-09-19T04:49:07.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.124+0000 7f95f1e36740 1 -- 172.21.15.123:0/1284725369 shutdown_connections 2024-09-19T04:49:07.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.124+0000 7f95f1e36740 1 --2- 172.21.15.123:0/1284725369 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5581d5d20a70 0x5581d5d22e90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.124+0000 7f95f1e36740 1 --2- 172.21.15.123:0/1284725369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581d5c9ce50 0x5581d5ca0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.124+0000 7f95f1e36740 1 --2- 172.21.15.123:0/1284725369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581d5c92590 0x5581d5ca0580 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.124+0000 7f95f1e36740 1 --2- 172.21.15.123:0/1284725369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d5c91c70 0x5581d5ca5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.124+0000 7f95f1e36740 1 -- 172.21.15.123:0/1284725369 >> 172.21.15.123:0/1284725369 conn(0x5581d5c8e200 msgr2=0x5581d5c9b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:07.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.124+0000 7f95ce000640 0 client.15783 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:07.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.124+0000 7f95f1e36740 1 -- 172.21.15.123:0/1284725369 shutdown_connections 2024-09-19T04:49:07.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.124+0000 7f95f1e36740 1 -- 172.21.15.123:0/1284725369 wait complete. 2024-09-19T04:49:07.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.124+0000 7f95f1e36740 20 client.15783 trim_cache size 0 max 16384 2024-09-19T04:49:07.135 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:49:07.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.125+0000 7f95ef400640 1 -- 172.21.15.123:0/3658214781 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f95d00a33b0 msgr2=0x7f95d00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:07.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.125+0000 7f95ef400640 1 --2- 172.21.15.123:0/3658214781 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f95d00a33b0 0x7f95d00a5870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f95dc006010 tx=0x7f95dc000bc0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.125+0000 7f95ef400640 1 -- 172.21.15.123:0/3658214781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95e8101420 msgr2=0x7f95e819ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:07.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.125+0000 7f95ef400640 1 --2- 172.21.15.123:0/3658214781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95e8101420 0x7f95e819ae80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f95d400e9f0 tx=0x7f95d400edf0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.125+0000 7f95ef400640 1 -- 172.21.15.123:0/3658214781 shutdown_connections 2024-09-19T04:49:07.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.125+0000 7f95ef400640 1 --2- 172.21.15.123:0/3658214781 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f95d00a33b0 0x7f95d00a5870 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.125+0000 7f95ef400640 1 --2- 172.21.15.123:0/3658214781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e8106a80 0x7f95e819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.125+0000 7f95ef400640 1 --2- 172.21.15.123:0/3658214781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95e8101420 0x7f95e819ae80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.125+0000 7f95ef400640 1 --2- 172.21.15.123:0/3658214781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95e8100a80 0x7f95e819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:07.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.125+0000 7f95ef400640 1 -- 172.21.15.123:0/3658214781 >> 172.21.15.123:0/3658214781 conn(0x7f95e80fc7a0 msgr2=0x7f95e8102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:07.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.125+0000 7f95ef400640 1 -- 172.21.15.123:0/3658214781 shutdown_connections 2024-09-19T04:49:07.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:07.125+0000 7f95ef400640 1 -- 172.21.15.123:0/3658214781 wait complete. 2024-09-19T04:49:07.731 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:07 smithi123 ceph-mon[30511]: pgmap v75: 97 pgs: 97 active+clean; 70 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 863 KiB/s wr, 142 op/s 2024-09-19T04:49:07.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:07 smithi193 ceph-mon[35592]: pgmap v75: 97 pgs: 97 active+clean; 70 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 863 KiB/s wr, 142 op/s 2024-09-19T04:49:08.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:07 smithi183 ceph-mon[35333]: pgmap v75: 97 pgs: 97 active+clean; 70 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 863 KiB/s wr, 142 op/s 2024-09-19T04:49:08.991 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:49:08.992 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:49:09.397 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:49:09.754 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:09 smithi123 ceph-mon[30511]: pgmap v76: 97 pgs: 97 active+clean; 75 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.5 MiB/s wr, 228 op/s 2024-09-19T04:49:09.910 INFO:tasks.workunit.client.0.smithi123.stdout: [DEP] 2024-09-19T04:49:09.911 INFO:tasks.workunit.client.0.smithi123.stdout: [DEP] 2024-09-19T04:49:09.911 INFO:tasks.workunit.client.0.smithi123.stdout: [DEP] 2024-09-19T04:49:09.912 INFO:tasks.workunit.client.0.smithi123.stdout: [DEP] 2024-09-19T04:49:09.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:09 smithi193 ceph-mon[35592]: pgmap v76: 97 pgs: 97 active+clean; 75 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.5 MiB/s wr, 228 op/s 2024-09-19T04:49:10.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:09 smithi183 ceph-mon[35333]: pgmap v76: 97 pgs: 97 active+clean; 75 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.5 MiB/s wr, 228 op/s 2024-09-19T04:49:10.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.124+0000 7f944fa00640 1 -- 172.21.15.123:0/3619593921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9448109140 msgr2=0x7f9448110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:10.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.124+0000 7f944fa00640 1 --2- 172.21.15.123:0/3619593921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9448109140 0x7f9448110cc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9440009ca0 tx=0x7f944002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.124+0000 7f944fa00640 1 -- 172.21.15.123:0/3619593921 shutdown_connections 2024-09-19T04:49:10.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.124+0000 7f944fa00640 1 --2- 172.21.15.123:0/3619593921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9448109140 0x7f9448110cc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.124+0000 7f944fa00640 1 --2- 172.21.15.123:0/3619593921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94481049c0 0x7f9448108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.124+0000 7f944fa00640 1 --2- 172.21.15.123:0/3619593921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9448104010 0x7f94481043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.124+0000 7f944fa00640 1 -- 172.21.15.123:0/3619593921 >> 172.21.15.123:0/3619593921 conn(0x7f94480ffa50 msgr2=0x7f9448101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:10.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.124+0000 7f944fa00640 1 -- 172.21.15.123:0/3619593921 shutdown_connections 2024-09-19T04:49:10.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.124+0000 7f944fa00640 1 -- 172.21.15.123:0/3619593921 wait complete. 2024-09-19T04:49:10.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.125+0000 7f944fa00640 1 Processor -- start 2024-09-19T04:49:10.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.125+0000 7f944fa00640 1 -- start start 2024-09-19T04:49:10.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.126+0000 7f944fa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9448104010 0x7f944819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.126+0000 7f944fa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94481049c0 0x7f944819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.126+0000 7f944fa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9448109140 0x7f944819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.126+0000 7f944fa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9448114150 con 0x7f9448109140 2024-09-19T04:49:10.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.126+0000 7f944fa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9448113fd0 con 0x7f9448104010 2024-09-19T04:49:10.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.126+0000 7f944fa00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f94481142d0 con 0x7f94481049c0 2024-09-19T04:49:10.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.126+0000 7f9447e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94481049c0 0x7f944819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.126+0000 7f944da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9448109140 0x7f944819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.126+0000 7f9447e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94481049c0 0x7f944819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53210/0 (socket says 172.21.15.123:53210) 2024-09-19T04:49:10.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.126+0000 7f9447e00640 1 -- 172.21.15.123:0/3980453116 learned_addr learned my addr 172.21.15.123:0/3980453116 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:10.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.126+0000 7f944d000640 1 --2- 172.21.15.123:0/3980453116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9448104010 0x7f944819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.126+0000 7f9447e00640 1 -- 172.21.15.123:0/3980453116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9448104010 msgr2=0x7f944819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:10.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.126+0000 7f9447e00640 1 --2- 172.21.15.123:0/3980453116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9448104010 0x7f944819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.126+0000 7f9447e00640 1 -- 172.21.15.123:0/3980453116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9448109140 msgr2=0x7f944819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:10.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.126+0000 7f9447e00640 1 --2- 172.21.15.123:0/3980453116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9448109140 0x7f944819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.126+0000 7f9447e00640 1 -- 172.21.15.123:0/3980453116 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944819f640 con 0x7f94481049c0 2024-09-19T04:49:10.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.127+0000 7f9447e00640 1 --2- 172.21.15.123:0/3980453116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94481049c0 0x7f944819abd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f943400e9f0 tx=0x7f943400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:10.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.127+0000 7f9445600640 1 -- 172.21.15.123:0/3980453116 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943400cd80 con 0x7f94481049c0 2024-09-19T04:49:10.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.127+0000 7f944fa00640 1 -- 172.21.15.123:0/3980453116 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944819f930 con 0x7f94481049c0 2024-09-19T04:49:10.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.127+0000 7f9445600640 1 -- 172.21.15.123:0/3980453116 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9434010910 con 0x7f94481049c0 2024-09-19T04:49:10.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.127+0000 7f9445600640 1 -- 172.21.15.123:0/3980453116 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94340188f0 con 0x7f94481049c0 2024-09-19T04:49:10.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.127+0000 7f944fa00640 1 -- 172.21.15.123:0/3980453116 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94481a7210 con 0x7f94481049c0 2024-09-19T04:49:10.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.131+0000 7f9445600640 1 -- 172.21.15.123:0/3980453116 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9434010ac0 con 0x7f94481049c0 2024-09-19T04:49:10.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.131+0000 7f94523bb740 1 Processor -- start 2024-09-19T04:49:10.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.131+0000 7f94523bb740 1 -- start start 2024-09-19T04:49:10.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.132+0000 7f94523bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a371ac0 0x55832a371ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.132+0000 7f94523bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a3723e0 0x55832a37c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.132+0000 7f94523bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a37cca0 0x55832a37f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.132+0000 7f94523bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55832a2d8290 con 0x55832a3723e0 2024-09-19T04:49:10.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.132+0000 7f94523bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55832a27fb40 con 0x55832a371ac0 2024-09-19T04:49:10.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.132+0000 7f94523bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55832a2d4250 con 0x55832a37cca0 2024-09-19T04:49:10.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.132+0000 7f944d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a3723e0 0x55832a37c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.132+0000 7f9445600640 1 --2- 172.21.15.123:0/3980453116 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94300a33b0 0x7f94300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.132+0000 7f9447e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a37cca0 0x55832a37f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.132+0000 7f944d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a3723e0 0x55832a37c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46108/0 (socket says 172.21.15.123:46108) 2024-09-19T04:49:10.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.132+0000 7f944d000640 1 -- 172.21.15.123:0/1492897295 learned_addr learned my addr 172.21.15.123:0/1492897295 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:10.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.132+0000 7f944da00640 1 --2- 172.21.15.123:0/1492897295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a371ac0 0x55832a371ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.133+0000 7f9445600640 1 -- 172.21.15.123:0/3980453116 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f94340cffd0 con 0x7f94481049c0 2024-09-19T04:49:10.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.133+0000 7f9447e00640 1 -- 172.21.15.123:0/1492897295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a371ac0 msgr2=0x55832a371ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:10.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.133+0000 7f9447e00640 1 --2- 172.21.15.123:0/1492897295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a371ac0 0x55832a371ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.133+0000 7f9447e00640 1 -- 172.21.15.123:0/1492897295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a3723e0 msgr2=0x55832a37c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:10.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.133+0000 7f9447e00640 1 --2- 172.21.15.123:0/1492897295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a3723e0 0x55832a37c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.133+0000 7f9447e00640 1 -- 172.21.15.123:0/1492897295 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55832a0496a0 con 0x55832a37cca0 2024-09-19T04:49:10.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.133+0000 7f944d000640 1 --2- 172.21.15.123:0/1492897295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a3723e0 0x55832a37c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:49:10.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.133+0000 7f944da00640 1 --2- 172.21.15.123:0/3980453116 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94300a33b0 0x7f94300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.133+0000 7f9447e00640 1 --2- 172.21.15.123:0/1492897295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a37cca0 0x55832a37f090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f94340cd750 tx=0x7f943401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d49a4f1beb199ca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:10.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.133+0000 7f943a000640 1 -- 172.21.15.123:0/1492897295 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94340a0690 con 0x55832a37cca0 2024-09-19T04:49:10.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.134+0000 7f943a000640 1 -- 172.21.15.123:0/1492897295 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94340a4f50 con 0x55832a37cca0 2024-09-19T04:49:10.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.134+0000 7f943a000640 1 -- 172.21.15.123:0/1492897295 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94340d2080 con 0x55832a37cca0 2024-09-19T04:49:10.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.134+0000 7f944da00640 1 --2- 172.21.15.123:0/3980453116 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94300a33b0 0x7f94300a5870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9440009430 tx=0x7f944002a8c0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:10.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.134+0000 7f94523bb740 1 -- 172.21.15.123:0/1492897295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a37cca0 msgr2=0x55832a37f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:10.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.134+0000 7f94523bb740 1 --2- 172.21.15.123:0/1492897295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a37cca0 0x55832a37f090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f94340cd750 tx=0x7f943401f000 comp rx=0 tx=0).stop 2024-09-19T04:49:10.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.134+0000 7f94523bb740 1 -- 172.21.15.123:0/1492897295 shutdown_connections 2024-09-19T04:49:10.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.134+0000 7f94523bb740 1 --2- 172.21.15.123:0/1492897295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a37cca0 0x55832a37f090 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.134+0000 7f94523bb740 1 --2- 172.21.15.123:0/1492897295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a3723e0 0x55832a37c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.134+0000 7f94523bb740 1 --2- 172.21.15.123:0/1492897295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a371ac0 0x55832a371ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.134+0000 7f94523bb740 1 -- 172.21.15.123:0/1492897295 >> 172.21.15.123:0/1492897295 conn(0x55832a36e050 msgr2=0x55832a36e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:10.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.135+0000 7f94523bb740 1 -- 172.21.15.123:0/1492897295 shutdown_connections 2024-09-19T04:49:10.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.135+0000 7f94523bb740 1 -- 172.21.15.123:0/1492897295 wait complete. 2024-09-19T04:49:10.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.135+0000 7f94523bb740 1 -- start start 2024-09-19T04:49:10.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.135+0000 7f94523bb740 1 Processor -- start 2024-09-19T04:49:10.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.136+0000 7f94523bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a371ac0 0x55832a37c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.136+0000 7f94523bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a3723e0 0x55832a3799b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.136+0000 7f94523bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a37cca0 0x55832a37a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.136+0000 7f94523bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55832a37f910 con 0x55832a3723e0 2024-09-19T04:49:10.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.136+0000 7f94523bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55832a292fa0 con 0x55832a37cca0 2024-09-19T04:49:10.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.136+0000 7f94523bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55832a37fa90 con 0x55832a371ac0 2024-09-19T04:49:10.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.136+0000 7f94523bb740 10 client.0 _ll_register_callbacks cb 0x558329fe6ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:10.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.136+0000 7f944d000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a371ac0 0x55832a37c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.136+0000 7f944d000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a371ac0 0x55832a37c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53242/0 (socket says 172.21.15.123:53242) 2024-09-19T04:49:10.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.136+0000 7f944d000640 1 -- 172.21.15.123:0/1458870397 learned_addr learned my addr 172.21.15.123:0/1458870397 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:10.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.136+0000 7f9447e00640 1 --2- 172.21.15.123:0/1458870397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a3723e0 0x55832a3799b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.136+0000 7f944da00640 1 --2- 172.21.15.123:0/1458870397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a37cca0 0x55832a37a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.136+0000 7f944d000640 1 -- 172.21.15.123:0/1458870397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a37cca0 msgr2=0x55832a37a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:10.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.136+0000 7f944d000640 1 --2- 172.21.15.123:0/1458870397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a37cca0 0x55832a37a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.137+0000 7f944d000640 1 -- 172.21.15.123:0/1458870397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a3723e0 msgr2=0x55832a3799b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:10.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.137+0000 7f944d000640 1 --2- 172.21.15.123:0/1458870397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a3723e0 0x55832a3799b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.137+0000 7f944d000640 1 -- 172.21.15.123:0/1458870397 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55832a388d80 con 0x55832a371ac0 2024-09-19T04:49:10.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.137+0000 7f944d000640 1 --2- 172.21.15.123:0/1458870397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a371ac0 0x55832a37c700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f943c0126e0 tx=0x7f943c012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53c2e4f8f182f77e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:10.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.137+0000 7f942ea00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:49:10.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.137+0000 7f942ea00640 1 -- 172.21.15.123:0/1458870397 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c0121d0 con 0x55832a371ac0 2024-09-19T04:49:10.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.137+0000 7f942ea00640 1 -- 172.21.15.123:0/1458870397 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f943c004050 con 0x55832a371ac0 2024-09-19T04:49:10.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.137+0000 7f94523bb740 1 -- 172.21.15.123:0/1458870397 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94340a4f50 con 0x55832a371ac0 2024-09-19T04:49:10.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.138+0000 7f942ea00640 1 -- 172.21.15.123:0/1458870397 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f943c004250 con 0x55832a371ac0 2024-09-19T04:49:10.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.138+0000 7f94523bb740 10 client.25849 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:10.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.138+0000 7f94523bb740 1 -- 172.21.15.123:0/1458870397 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55832a38ba00 con 0x55832a371ac0 2024-09-19T04:49:10.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.138+0000 7f942ea00640 1 -- 172.21.15.123:0/1458870397 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94340a4f50 con 0x55832a371ac0 2024-09-19T04:49:10.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.138+0000 7f942ea00640 1 -- 172.21.15.123:0/1458870397 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f943c0039c0 con 0x55832a371ac0 2024-09-19T04:49:10.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.138+0000 7f94523bb740 10 client.25849 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:10.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.138+0000 7f94523bb740 10 client.25849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:49:10.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.138+0000 7f94523bb740 20 client.25849 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:10.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.138+0000 7f94523bb740 1 --2- 172.21.15.123:0/1458870397 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55832a38ce10 0x55832a38f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.139+0000 7f94523bb740 4 client.25849 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:49:10.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.139+0000 7f94523bb740 1 -- 172.21.15.123:0/1458870397 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55832a2e5800 con 0x55832a38ce10 2024-09-19T04:49:10.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.139+0000 7f9447e00640 1 --2- 172.21.15.123:0/1458870397 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55832a38ce10 0x55832a38f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.139+0000 7f9447e00640 1 --2- 172.21.15.123:0/1458870397 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55832a38ce10 0x55832a38f290 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5430fb1166cd2df server_cookie=4802f7bfd5a3ec51 in_seq=0 out_seq=0 2024-09-19T04:49:10.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.139+0000 7f942ea00640 10 client.25849 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:10.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.140+0000 7f942ea00640 1 -- 172.21.15.123:0/1458870397 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55832a2e5800 con 0x55832a38ce10 2024-09-19T04:49:10.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.140+0000 7f942ea00640 10 client.25849 handle_command_reply: tid=0 2024-09-19T04:49:10.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.140+0000 7f94523bb740 1 client.25849 shutdown 2024-09-19T04:49:10.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.141+0000 7f94523bb740 1 -- 172.21.15.123:0/1458870397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a371ac0 msgr2=0x55832a37c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:10.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.141+0000 7f94523bb740 1 --2- 172.21.15.123:0/1458870397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a371ac0 0x55832a37c700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f943c0126e0 tx=0x7f943c012ba0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.141+0000 7f94523bb740 1 -- 172.21.15.123:0/1458870397 shutdown_connections 2024-09-19T04:49:10.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.141+0000 7f94523bb740 1 --2- 172.21.15.123:0/1458870397 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55832a38ce10 0x55832a38f290 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.141+0000 7f94523bb740 1 --2- 172.21.15.123:0/1458870397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a37cca0 0x55832a37a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.141+0000 7f94523bb740 1 --2- 172.21.15.123:0/1458870397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a3723e0 0x55832a3799b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.141+0000 7f94523bb740 1 --2- 172.21.15.123:0/1458870397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a371ac0 0x55832a37c700 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.141+0000 7f94523bb740 1 -- 172.21.15.123:0/1458870397 >> 172.21.15.123:0/1458870397 conn(0x55832a36e050 msgr2=0x55832a373d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:10.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.141+0000 7f942ea00640 0 client.25849 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:10.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.141+0000 7f94523bb740 1 -- 172.21.15.123:0/1458870397 shutdown_connections 2024-09-19T04:49:10.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.141+0000 7f94523bb740 1 -- 172.21.15.123:0/1458870397 wait complete. 2024-09-19T04:49:10.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.141+0000 7f94523bb740 20 client.25849 trim_cache size 0 max 16384 2024-09-19T04:49:10.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.159+0000 7f94523bb740 1 Processor -- start 2024-09-19T04:49:10.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.160+0000 7f94523bb740 1 -- start start 2024-09-19T04:49:10.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.160+0000 7f94523bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a371ac0 0x55832a36d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.160+0000 7f94523bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a3723e0 0x55832a37c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.160+0000 7f94523bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a37cca0 0x55832a37c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.160+0000 7f94523bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94481af9b0 con 0x55832a37cca0 2024-09-19T04:49:10.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.160+0000 7f94523bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94481b1640 con 0x55832a3723e0 2024-09-19T04:49:10.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.160+0000 7f94523bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f94481b0080 con 0x55832a371ac0 2024-09-19T04:49:10.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.160+0000 7f9447e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a3723e0 0x55832a37c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.160+0000 7f9447e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a3723e0 0x55832a37c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:58666/0 (socket says 172.21.15.123:58666) 2024-09-19T04:49:10.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.160+0000 7f9447e00640 1 -- 172.21.15.123:0/1835605439 learned_addr learned my addr 172.21.15.123:0/1835605439 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:10.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.160+0000 7f944da00640 1 --2- 172.21.15.123:0/1835605439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a37cca0 0x55832a37c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.160+0000 7f944d000640 1 --2- 172.21.15.123:0/1835605439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a371ac0 0x55832a36d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.161+0000 7f9447e00640 1 -- 172.21.15.123:0/1835605439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a371ac0 msgr2=0x55832a36d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:10.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.161+0000 7f9447e00640 1 --2- 172.21.15.123:0/1835605439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a371ac0 0x55832a36d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.161+0000 7f9447e00640 1 -- 172.21.15.123:0/1835605439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a37cca0 msgr2=0x55832a37c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:10.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.161+0000 7f9447e00640 1 --2- 172.21.15.123:0/1835605439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a37cca0 0x55832a37c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.161+0000 7f9447e00640 1 -- 172.21.15.123:0/1835605439 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55832a37aae0 con 0x55832a3723e0 2024-09-19T04:49:10.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.161+0000 7f944d000640 1 --2- 172.21.15.123:0/1835605439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a371ac0 0x55832a36d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:49:10.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.161+0000 7f944da00640 1 --2- 172.21.15.123:0/1835605439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a37cca0 0x55832a37c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:49:10.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.161+0000 7f9447e00640 1 --2- 172.21.15.123:0/1835605439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a3723e0 0x55832a37c220 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f94340cd800 tx=0x7f94340b5050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=631ec74d66e3f2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:10.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.161+0000 7f9439600640 1 -- 172.21.15.123:0/1835605439 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434098a00 con 0x55832a3723e0 2024-09-19T04:49:10.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.161+0000 7f9439600640 1 -- 172.21.15.123:0/1835605439 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94340b50c0 con 0x55832a3723e0 2024-09-19T04:49:10.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.161+0000 7f9439600640 1 -- 172.21.15.123:0/1835605439 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94340bc0d0 con 0x55832a3723e0 2024-09-19T04:49:10.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.162+0000 7f94523bb740 1 -- 172.21.15.123:0/1835605439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a3723e0 msgr2=0x55832a37c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:10.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.162+0000 7f94523bb740 1 --2- 172.21.15.123:0/1835605439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a3723e0 0x55832a37c220 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f94340cd800 tx=0x7f94340b5050 comp rx=0 tx=0).stop 2024-09-19T04:49:10.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.162+0000 7f94523bb740 1 -- 172.21.15.123:0/1835605439 shutdown_connections 2024-09-19T04:49:10.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.162+0000 7f94523bb740 1 --2- 172.21.15.123:0/1835605439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a37cca0 0x55832a37c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.162+0000 7f94523bb740 1 --2- 172.21.15.123:0/1835605439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a3723e0 0x55832a37c220 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.162+0000 7f94523bb740 1 --2- 172.21.15.123:0/1835605439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a371ac0 0x55832a36d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.162+0000 7f94523bb740 1 -- 172.21.15.123:0/1835605439 >> 172.21.15.123:0/1835605439 conn(0x55832a36e050 msgr2=0x55832a36c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:10.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.162+0000 7f94523bb740 1 -- 172.21.15.123:0/1835605439 shutdown_connections 2024-09-19T04:49:10.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.162+0000 7f94523bb740 1 -- 172.21.15.123:0/1835605439 wait complete. 2024-09-19T04:49:10.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.163+0000 7f94523bb740 1 -- start start 2024-09-19T04:49:10.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.163+0000 7f94523bb740 1 Processor -- start 2024-09-19T04:49:10.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.163+0000 7f94523bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a371ac0 0x55832a381180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.163+0000 7f94523bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a3723e0 0x55832a3816c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.164+0000 7f94523bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a37cca0 0x55832a3f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.164+0000 7f94523bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94481b0eb0 con 0x55832a37cca0 2024-09-19T04:49:10.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.164+0000 7f94523bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94481b07e0 con 0x55832a371ac0 2024-09-19T04:49:10.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.164+0000 7f94523bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f94481ad990 con 0x55832a3723e0 2024-09-19T04:49:10.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.164+0000 7f94523bb740 10 client.0 _ll_register_callbacks cb 0x7f943c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:10.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.164+0000 7f9447e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a3723e0 0x55832a3816c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.164+0000 7f944d000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a371ac0 0x55832a381180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.164+0000 7f9447e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a3723e0 0x55832a3816c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53260/0 (socket says 172.21.15.123:53260) 2024-09-19T04:49:10.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.164+0000 7f944da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a37cca0 0x55832a3f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.164+0000 7f9447e00640 1 -- 172.21.15.123:0/1390151571 learned_addr learned my addr 172.21.15.123:0/1390151571 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:10.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.164+0000 7f944d000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a371ac0 0x55832a381180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:58674/0 (socket says 172.21.15.123:58674) 2024-09-19T04:49:10.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.164+0000 7f9447e00640 1 -- 172.21.15.123:0/1390151571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a371ac0 msgr2=0x55832a381180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:10.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.164+0000 7f9447e00640 1 --2- 172.21.15.123:0/1390151571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a371ac0 0x55832a381180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.164+0000 7f9447e00640 1 -- 172.21.15.123:0/1390151571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a37cca0 msgr2=0x55832a3f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:10.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.164+0000 7f9447e00640 1 --2- 172.21.15.123:0/1390151571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a37cca0 0x55832a3f6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.164+0000 7f9447e00640 1 -- 172.21.15.123:0/1390151571 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55832a3f7070 con 0x55832a3723e0 2024-09-19T04:49:10.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.164+0000 7f944d000640 1 --2- 172.21.15.123:0/1390151571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a371ac0 0x55832a381180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:49:10.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.165+0000 7f9447e00640 1 --2- 172.21.15.123:0/1390151571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a3723e0 0x55832a3816c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f94340987f0 tx=0x7f943409c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2759e07c5e19127 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:10.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.165+0000 7f942ea00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:49:10.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.165+0000 7f942ea00640 1 -- 172.21.15.123:0/1390151571 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434098a00 con 0x55832a3723e0 2024-09-19T04:49:10.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.165+0000 7f942ea00640 1 -- 172.21.15.123:0/1390151571 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f94340cd9d0 con 0x55832a3723e0 2024-09-19T04:49:10.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.165+0000 7f94523bb740 1 -- 172.21.15.123:0/1390151571 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94340b50c0 con 0x55832a3723e0 2024-09-19T04:49:10.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.165+0000 7f942ea00640 1 -- 172.21.15.123:0/1390151571 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f94340ba050 con 0x55832a3723e0 2024-09-19T04:49:10.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.165+0000 7f94523bb740 10 client.25855 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:10.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.165+0000 7f94523bb740 1 -- 172.21.15.123:0/1390151571 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55832a3f83b0 con 0x55832a3723e0 2024-09-19T04:49:10.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.165+0000 7f942ea00640 1 -- 172.21.15.123:0/1390151571 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94340b50c0 con 0x55832a3723e0 2024-09-19T04:49:10.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.166+0000 7f942ea00640 1 -- 172.21.15.123:0/1390151571 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f943409b900 con 0x55832a3723e0 2024-09-19T04:49:10.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.166+0000 7f94523bb740 10 client.25855 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:10.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.166+0000 7f94523bb740 10 client.25855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:49:10.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.166+0000 7f94523bb740 20 client.25855 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:10.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.166+0000 7f94523bb740 1 --2- 172.21.15.123:0/1390151571 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55832a3f85e0 0x55832a3fa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:10.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.166+0000 7f94523bb740 4 client.25855 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:49:10.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.166+0000 7f94523bb740 1 -- 172.21.15.123:0/1390151571 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55832a2dfd10 con 0x55832a3f85e0 2024-09-19T04:49:10.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.166+0000 7f944d000640 1 --2- 172.21.15.123:0/1390151571 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55832a3f85e0 0x55832a3fa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:10.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.167+0000 7f942ea00640 10 client.25855 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:10.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.167+0000 7f944d000640 1 --2- 172.21.15.123:0/1390151571 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55832a3f85e0 0x55832a3fa9d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b79c07c1eb16544 server_cookie=69042ebeff22f42d in_seq=0 out_seq=0 2024-09-19T04:49:10.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.168+0000 7f942ea00640 1 -- 172.21.15.123:0/1390151571 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55832a2dfd10 con 0x55832a3f85e0 2024-09-19T04:49:10.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.168+0000 7f942ea00640 10 client.25855 handle_command_reply: tid=0 2024-09-19T04:49:10.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.168+0000 7f94523bb740 1 client.25855 shutdown 2024-09-19T04:49:10.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.168+0000 7f94523bb740 1 -- 172.21.15.123:0/1390151571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a3723e0 msgr2=0x55832a3816c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:10.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.168+0000 7f94523bb740 1 --2- 172.21.15.123:0/1390151571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a3723e0 0x55832a3816c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f94340987f0 tx=0x7f943409c510 comp rx=0 tx=0).stop 2024-09-19T04:49:10.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.168+0000 7f94523bb740 1 -- 172.21.15.123:0/1390151571 shutdown_connections 2024-09-19T04:49:10.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.168+0000 7f94523bb740 1 --2- 172.21.15.123:0/1390151571 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55832a3f85e0 0x55832a3fa9d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.168+0000 7f94523bb740 1 --2- 172.21.15.123:0/1390151571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55832a37cca0 0x55832a3f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.168+0000 7f94523bb740 1 --2- 172.21.15.123:0/1390151571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55832a3723e0 0x55832a3816c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.168+0000 7f94523bb740 1 --2- 172.21.15.123:0/1390151571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55832a371ac0 0x55832a381180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.168+0000 7f94523bb740 1 -- 172.21.15.123:0/1390151571 >> 172.21.15.123:0/1390151571 conn(0x55832a36e050 msgr2=0x55832a373d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:10.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.168+0000 7f942ea00640 0 client.25855 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:10.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.169+0000 7f94523bb740 1 -- 172.21.15.123:0/1390151571 shutdown_connections 2024-09-19T04:49:10.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.169+0000 7f94523bb740 1 -- 172.21.15.123:0/1390151571 wait complete. 2024-09-19T04:49:10.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.169+0000 7f94523bb740 20 client.25855 trim_cache size 0 max 16384 2024-09-19T04:49:10.176 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:49:10.176 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-09-19T04:49:10.176 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "f8a5315d-9b00-46db-aa2a-c9156aefa539", 2024-09-19T04:49:10.176 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-09-19T04:49:10.177 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:49:10.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.169+0000 7f944fa00640 1 -- 172.21.15.123:0/3980453116 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94300a33b0 msgr2=0x7f94300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:10.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.169+0000 7f944fa00640 1 --2- 172.21.15.123:0/3980453116 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94300a33b0 0x7f94300a5870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9440009430 tx=0x7f944002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.169+0000 7f944fa00640 1 -- 172.21.15.123:0/3980453116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94481049c0 msgr2=0x7f944819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:10.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.169+0000 7f944fa00640 1 --2- 172.21.15.123:0/3980453116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94481049c0 0x7f944819abd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f943400e9f0 tx=0x7f943400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.170+0000 7f944fa00640 1 -- 172.21.15.123:0/3980453116 shutdown_connections 2024-09-19T04:49:10.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.170+0000 7f944fa00640 1 --2- 172.21.15.123:0/3980453116 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94300a33b0 0x7f94300a5870 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.170+0000 7f944fa00640 1 --2- 172.21.15.123:0/3980453116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9448109140 0x7f944819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.170+0000 7f944fa00640 1 --2- 172.21.15.123:0/3980453116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94481049c0 0x7f944819abd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.170+0000 7f944fa00640 1 --2- 172.21.15.123:0/3980453116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9448104010 0x7f944819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:10.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.170+0000 7f944fa00640 1 -- 172.21.15.123:0/3980453116 >> 172.21.15.123:0/3980453116 conn(0x7f94480ffa50 msgr2=0x7f9448100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:10.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.170+0000 7f944fa00640 1 -- 172.21.15.123:0/3980453116 shutdown_connections 2024-09-19T04:49:10.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:10.170+0000 7f944fa00640 1 -- 172.21.15.123:0/3980453116 wait complete. 2024-09-19T04:49:10.359 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] dirstress 2024-09-19T04:49:10.360 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] fill 2024-09-19T04:49:10.361 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] fill2 2024-09-19T04:49:10.362 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] getpagesize 2024-09-19T04:49:10.743 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] holes 2024-09-19T04:49:10.760 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] lstat64 2024-09-19T04:49:10.843 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] nametest 2024-09-19T04:49:10.874 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] permname 2024-09-19T04:49:10.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:10 smithi123 ceph-mon[30511]: scrub queued for path: / 2024-09-19T04:49:10.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:10 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:10.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:10 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:49:10.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:10 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:10.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:10 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:49:10.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:10 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:10.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:10 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:49:10.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:10 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:10.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:10 smithi193 ceph-mon[35592]: scrub queued for path: / 2024-09-19T04:49:10.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:10 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:10.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:10 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:49:10.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:10 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:10.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:10 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:49:10.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:10 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:10.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:10 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:49:10.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:10 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:11.087 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] randholes 2024-09-19T04:49:11.098 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] runas 2024-09-19T04:49:11.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:10 smithi183 ceph-mon[35333]: scrub queued for path: / 2024-09-19T04:49:11.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:10 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:11.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:10 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:49:11.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:10 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:11.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:10 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:49:11.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:10 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:11.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:10 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:49:11.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:10 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:11.214 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] truncfile 2024-09-19T04:49:11.249 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] usemem 2024-09-19T04:49:11.268 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:49:11.417 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] mmapcat 2024-09-19T04:49:11.533 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] append_reader 2024-09-19T04:49:11.557 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] append_writer 2024-09-19T04:49:11.561 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] dirperf 2024-09-19T04:49:11.753 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] metaperf 2024-09-19T04:49:11.763 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:49:11.842 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] devzero 2024-09-19T04:49:11.878 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] feature 2024-09-19T04:49:11.960 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:11 smithi123 ceph-mon[30511]: pgmap v77: 97 pgs: 97 active+clean; 75 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-09-19T04:49:11.960 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:11 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:49:11.960 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:11 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:49:11.960 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:11 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:11.960 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:11 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:11.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:11 smithi193 ceph-mon[35592]: pgmap v77: 97 pgs: 97 active+clean; 75 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-09-19T04:49:11.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:11 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:49:11.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:11 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:49:11.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:11 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:11.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:11 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:12.013 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] alloc 2024-09-19T04:49:12.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:11 smithi183 ceph-mon[35333]: pgmap v77: 97 pgs: 97 active+clean; 75 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-09-19T04:49:12.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:11 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:49:12.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:11 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:49:12.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:11 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:12.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:11 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:49:12.167 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] fault 2024-09-19T04:49:12.209 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] fstest 2024-09-19T04:49:12.254 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_access_root 2024-09-19T04:49:12.387 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] godown 2024-09-19T04:49:12.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.438+0000 7f308be00640 1 -- 172.21.15.123:0/2801025048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f308410e440 msgr2=0x7f3084110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:12.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.438+0000 7f308be00640 1 --2- 172.21.15.123:0/2801025048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f308410e440 0x7f3084110830 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f307c009ca0 tx=0x7f307c02a950 comp rx=0 tx=0).stop 2024-09-19T04:49:12.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.439+0000 7f308be00640 1 -- 172.21.15.123:0/2801025048 shutdown_connections 2024-09-19T04:49:12.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.439+0000 7f308be00640 1 --2- 172.21.15.123:0/2801025048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f308410e440 0x7f3084110830 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.439+0000 7f308be00640 1 --2- 172.21.15.123:0/2801025048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3084074e30 0x7f308410df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.439+0000 7f308be00640 1 --2- 172.21.15.123:0/2801025048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3084074510 0x7f30840748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.439+0000 7f308be00640 1 -- 172.21.15.123:0/2801025048 >> 172.21.15.123:0/2801025048 conn(0x7f30840fd8e0 msgr2=0x7f30840ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:12.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.439+0000 7f308be00640 1 -- 172.21.15.123:0/2801025048 shutdown_connections 2024-09-19T04:49:12.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.439+0000 7f308be00640 1 -- 172.21.15.123:0/2801025048 wait complete. 2024-09-19T04:49:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.440+0000 7f308be00640 1 Processor -- start 2024-09-19T04:49:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.441+0000 7f308be00640 1 -- start start 2024-09-19T04:49:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.441+0000 7f308be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3084074510 0x7f308419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.441+0000 7f308be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3084074e30 0x7f308419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.441+0000 7f308be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f308410e440 0x7f308419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.441+0000 7f308be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f30841141a0 con 0x7f3084074510 2024-09-19T04:49:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.441+0000 7f308be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3084114020 con 0x7f3084074e30 2024-09-19T04:49:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.441+0000 7f308be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3084114320 con 0x7f308410e440 2024-09-19T04:49:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.442+0000 7f3088a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3084074e30 0x7f308419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.442+0000 7f3088a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3084074e30 0x7f308419ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:45260/0 (socket says 172.21.15.123:45260) 2024-09-19T04:49:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.442+0000 7f3088a00640 1 -- 172.21.15.123:0/2870684164 learned_addr learned my addr 172.21.15.123:0/2870684164 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.442+0000 7f3089e00640 1 --2- 172.21.15.123:0/2870684164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f308410e440 0x7f308419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.442+0000 7f3088a00640 1 -- 172.21.15.123:0/2870684164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f308410e440 msgr2=0x7f308419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.442+0000 7f3088a00640 1 --2- 172.21.15.123:0/2870684164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f308410e440 0x7f308419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.442+0000 7f3089400640 1 --2- 172.21.15.123:0/2870684164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3084074510 0x7f308419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.442+0000 7f3088a00640 1 -- 172.21.15.123:0/2870684164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3084074510 msgr2=0x7f308419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.443+0000 7f3088a00640 1 --2- 172.21.15.123:0/2870684164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3084074510 0x7f308419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.443+0000 7f3088a00640 1 -- 172.21.15.123:0/2870684164 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308419f5d0 con 0x7f3084074e30 2024-09-19T04:49:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.443+0000 7f3089400640 1 --2- 172.21.15.123:0/2870684164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3084074510 0x7f308419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:49:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.444+0000 7f3088a00640 1 --2- 172.21.15.123:0/2870684164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3084074e30 0x7f308419ab60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f307000e9f0 tx=0x7f307000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.444+0000 7f3082000640 1 -- 172.21.15.123:0/2870684164 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307000cd80 con 0x7f3084074e30 2024-09-19T04:49:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.444+0000 7f3082000640 1 -- 172.21.15.123:0/2870684164 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3070010910 con 0x7f3084074e30 2024-09-19T04:49:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.444+0000 7f3082000640 1 -- 172.21.15.123:0/2870684164 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070018980 con 0x7f3084074e30 2024-09-19T04:49:12.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.444+0000 7f308be00640 1 -- 172.21.15.123:0/2870684164 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308419f8c0 con 0x7f3084074e30 2024-09-19T04:49:12.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.444+0000 7f308be00640 1 -- 172.21.15.123:0/2870684164 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30841a71a0 con 0x7f3084074e30 2024-09-19T04:49:12.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.449+0000 7f308e86a740 1 Processor -- start 2024-09-19T04:49:12.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.449+0000 7f308e86a740 1 -- start start 2024-09-19T04:49:12.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.450+0000 7f308e86a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20e9b90 0x5572e20e9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.450+0000 7f308e86a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20ea4b0 0x5572e20f4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.450+0000 7f308e86a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20f4d70 0x5572e20f7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.450+0000 7f308e86a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5572e20503e0 con 0x5572e20ea4b0 2024-09-19T04:49:12.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.450+0000 7f308e86a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5572e1ff7d80 con 0x5572e20e9b90 2024-09-19T04:49:12.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.450+0000 7f308e86a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5572e204c3a0 con 0x5572e20f4d70 2024-09-19T04:49:12.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.450+0000 7f3082000640 1 -- 172.21.15.123:0/2870684164 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3070010ac0 con 0x7f3084074e30 2024-09-19T04:49:12.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.450+0000 7f3089e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20e9b90 0x5572e20e9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.450+0000 7f3089e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20e9b90 0x5572e20e9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:45272/0 (socket says 172.21.15.123:45272) 2024-09-19T04:49:12.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.450+0000 7f3089e00640 1 -- 172.21.15.123:0/3088983234 learned_addr learned my addr 172.21.15.123:0/3088983234 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:12.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.451+0000 7f3088a00640 1 --2- 172.21.15.123:0/3088983234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20f4d70 0x5572e20f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.451+0000 7f3089400640 1 --2- 172.21.15.123:0/3088983234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20ea4b0 0x5572e20f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.451+0000 7f3089e00640 1 -- 172.21.15.123:0/3088983234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20f4d70 msgr2=0x5572e20f7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:12.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.451+0000 7f3089e00640 1 --2- 172.21.15.123:0/3088983234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20f4d70 0x5572e20f7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.451+0000 7f3089e00640 1 -- 172.21.15.123:0/3088983234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20ea4b0 msgr2=0x5572e20f4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:12.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.451+0000 7f3089e00640 1 --2- 172.21.15.123:0/3088983234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20ea4b0 0x5572e20f4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.451+0000 7f3089e00640 1 -- 172.21.15.123:0/3088983234 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572e2016ae0 con 0x5572e20e9b90 2024-09-19T04:49:12.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.452+0000 7f3089e00640 1 --2- 172.21.15.123:0/3088983234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20e9b90 0x5572e20e9f70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f30840754f0 tx=0x7f307c009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c60d9768b46b9959 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:12.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.452+0000 7f3076600640 1 -- 172.21.15.123:0/3088983234 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c034040 con 0x5572e20e9b90 2024-09-19T04:49:12.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.452+0000 7f3082000640 1 --2- 172.21.15.123:0/2870684164 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f306c0a3500 0x7f306c0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.452+0000 7f3076600640 1 -- 172.21.15.123:0/3088983234 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f307c003e50 con 0x5572e20e9b90 2024-09-19T04:49:12.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.452+0000 7f3076600640 1 -- 172.21.15.123:0/3088983234 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c0042a0 con 0x5572e20e9b90 2024-09-19T04:49:12.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.452+0000 7f3082000640 1 -- 172.21.15.123:0/2870684164 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f30700d01e0 con 0x7f3084074e30 2024-09-19T04:49:12.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.453+0000 7f3089400640 1 --2- 172.21.15.123:0/2870684164 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f306c0a3500 0x7f306c0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.453+0000 7f308e86a740 1 -- 172.21.15.123:0/3088983234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20e9b90 msgr2=0x5572e20e9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:12.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.453+0000 7f308e86a740 1 --2- 172.21.15.123:0/3088983234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20e9b90 0x5572e20e9f70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f30840754f0 tx=0x7f307c009a20 comp rx=0 tx=0).stop 2024-09-19T04:49:12.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.454+0000 7f3089400640 1 --2- 172.21.15.123:0/2870684164 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f306c0a3500 0x7f306c0a59c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5572e20e6f80 tx=0x7f3078005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:12.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.454+0000 7f308e86a740 1 -- 172.21.15.123:0/3088983234 shutdown_connections 2024-09-19T04:49:12.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.454+0000 7f308e86a740 1 --2- 172.21.15.123:0/3088983234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20f4d70 0x5572e20f7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.454+0000 7f308e86a740 1 --2- 172.21.15.123:0/3088983234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20ea4b0 0x5572e20f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.454+0000 7f308e86a740 1 --2- 172.21.15.123:0/3088983234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20e9b90 0x5572e20e9f70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.454+0000 7f308e86a740 1 -- 172.21.15.123:0/3088983234 >> 172.21.15.123:0/3088983234 conn(0x5572e20e6120 msgr2=0x5572e20e6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:12.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.456+0000 7f308e86a740 1 -- 172.21.15.123:0/3088983234 shutdown_connections 2024-09-19T04:49:12.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.456+0000 7f308e86a740 1 -- 172.21.15.123:0/3088983234 wait complete. 2024-09-19T04:49:12.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.456+0000 7f308e86a740 1 -- start start 2024-09-19T04:49:12.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.456+0000 7f308e86a740 1 Processor -- start 2024-09-19T04:49:12.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.457+0000 7f308e86a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20e9b90 0x5572e20f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.457+0000 7f308e86a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20ea4b0 0x5572e20f2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.457+0000 7f308e86a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20f4d70 0x5572e20f2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.457+0000 7f308e86a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5572e21001f0 con 0x5572e20e9b90 2024-09-19T04:49:12.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.457+0000 7f308e86a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5572e200b0f0 con 0x5572e20ea4b0 2024-09-19T04:49:12.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.457+0000 7f308e86a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5572e2100370 con 0x5572e20f4d70 2024-09-19T04:49:12.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.457+0000 7f308e86a740 10 client.0 _ll_register_callbacks cb 0x5572e1ffc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:12.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.458+0000 7f3089e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20f4d70 0x5572e20f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.458+0000 7f3089e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20f4d70 0x5572e20f2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:40094/0 (socket says 172.21.15.123:40094) 2024-09-19T04:49:12.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.458+0000 7f3089e00640 1 -- 172.21.15.123:0/1630379100 learned_addr learned my addr 172.21.15.123:0/1630379100 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:12.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.458+0000 7f3088a00640 1 --2- 172.21.15.123:0/1630379100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20ea4b0 0x5572e20f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.458+0000 7f3089e00640 1 -- 172.21.15.123:0/1630379100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20ea4b0 msgr2=0x5572e20f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:12.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.458+0000 7f3089400640 1 --2- 172.21.15.123:0/1630379100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20e9b90 0x5572e20f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.458+0000 7f3089e00640 1 --2- 172.21.15.123:0/1630379100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20ea4b0 0x5572e20f2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.458+0000 7f3089e00640 1 -- 172.21.15.123:0/1630379100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20e9b90 msgr2=0x5572e20f1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:12.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.458+0000 7f3089e00640 1 --2- 172.21.15.123:0/1630379100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20e9b90 0x5572e20f1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.458+0000 7f3089e00640 1 -- 172.21.15.123:0/1630379100 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572e1dc13c0 con 0x5572e20f4d70 2024-09-19T04:49:12.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.458+0000 7f3089400640 1 --2- 172.21.15.123:0/1630379100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20e9b90 0x5572e20f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:49:12.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.459+0000 7f3089e00640 1 --2- 172.21.15.123:0/1630379100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20f4d70 0x5572e20f2690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f307c009bf0 tx=0x7f307c02f750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f8020d26bd5f8c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:12.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.459+0000 7f306aa00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:49:12.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.459+0000 7f306aa00640 1 -- 172.21.15.123:0/1630379100 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c053060 con 0x5572e20f4d70 2024-09-19T04:49:12.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.459+0000 7f306aa00640 1 -- 172.21.15.123:0/1630379100 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f307c004000 con 0x5572e20f4d70 2024-09-19T04:49:12.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.459+0000 7f308e86a740 1 -- 172.21.15.123:0/1630379100 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f307c003e50 con 0x5572e20f4d70 2024-09-19T04:49:12.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.460+0000 7f306aa00640 1 -- 172.21.15.123:0/1630379100 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f307c034050 con 0x5572e20f4d70 2024-09-19T04:49:12.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.460+0000 7f308e86a740 10 client.25867 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:12.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.460+0000 7f308e86a740 1 -- 172.21.15.123:0/1630379100 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5572e21001f0 con 0x5572e20f4d70 2024-09-19T04:49:12.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.460+0000 7f306aa00640 1 -- 172.21.15.123:0/1630379100 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f307c003e50 con 0x5572e20f4d70 2024-09-19T04:49:12.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.460+0000 7f306aa00640 1 -- 172.21.15.123:0/1630379100 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f307c042070 con 0x5572e20f4d70 2024-09-19T04:49:12.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.461+0000 7f308e86a740 10 client.25867 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:12.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.461+0000 7f308e86a740 10 client.25867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:49:12.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.461+0000 7f308e86a740 20 client.25867 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:12.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.461+0000 7f308e86a740 1 --2- 172.21.15.123:0/1630379100 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5572e20fb000 0x5572e20fd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.461+0000 7f308e86a740 4 client.25867 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:49:12.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.461+0000 7f308e86a740 1 -- 172.21.15.123:0/1630379100 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572e205d950 con 0x5572e20fb000 2024-09-19T04:49:12.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.462+0000 7f3089400640 1 --2- 172.21.15.123:0/1630379100 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5572e20fb000 0x5572e20fd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.462+0000 7f3089400640 1 --2- 172.21.15.123:0/1630379100 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5572e20fb000 0x5572e20fd440 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3689d35fcb01f4fe server_cookie=1fa1f8e0872d7624 in_seq=0 out_seq=0 2024-09-19T04:49:12.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.462+0000 7f306aa00640 10 client.25867 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:12.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.463+0000 7f306aa00640 1 -- 172.21.15.123:0/1630379100 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572e205d950 con 0x5572e20fb000 2024-09-19T04:49:12.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.463+0000 7f306aa00640 10 client.25867 handle_command_reply: tid=0 2024-09-19T04:49:12.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.463+0000 7f308e86a740 1 client.25867 shutdown 2024-09-19T04:49:12.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.463+0000 7f308e86a740 1 -- 172.21.15.123:0/1630379100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20f4d70 msgr2=0x5572e20f2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:12.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.464+0000 7f308e86a740 1 --2- 172.21.15.123:0/1630379100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20f4d70 0x5572e20f2690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f307c009bf0 tx=0x7f307c02f750 comp rx=0 tx=0).stop 2024-09-19T04:49:12.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.464+0000 7f308e86a740 1 -- 172.21.15.123:0/1630379100 shutdown_connections 2024-09-19T04:49:12.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.464+0000 7f308e86a740 1 --2- 172.21.15.123:0/1630379100 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5572e20fb000 0x5572e20fd440 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.464+0000 7f308e86a740 1 --2- 172.21.15.123:0/1630379100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20f4d70 0x5572e20f2690 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.464+0000 7f308e86a740 1 --2- 172.21.15.123:0/1630379100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20ea4b0 0x5572e20f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.464+0000 7f308e86a740 1 --2- 172.21.15.123:0/1630379100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20e9b90 0x5572e20f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.464+0000 7f308e86a740 1 -- 172.21.15.123:0/1630379100 >> 172.21.15.123:0/1630379100 conn(0x5572e20e6120 msgr2=0x5572e20ebe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:12.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.464+0000 7f306aa00640 0 client.25867 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:12.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.464+0000 7f308e86a740 1 -- 172.21.15.123:0/1630379100 shutdown_connections 2024-09-19T04:49:12.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.464+0000 7f308e86a740 1 -- 172.21.15.123:0/1630379100 wait complete. 2024-09-19T04:49:12.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.465+0000 7f308e86a740 20 client.25867 trim_cache size 0 max 16384 2024-09-19T04:49:12.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.500+0000 7f308e86a740 1 Processor -- start 2024-09-19T04:49:12.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.500+0000 7f308e86a740 1 -- start start 2024-09-19T04:49:12.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.500+0000 7f308e86a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20e9b90 0x5572e20ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.500+0000 7f308e86a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20ea4b0 0x5572e211f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.500+0000 7f308e86a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20f4d70 0x5572e211fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.500+0000 7f308e86a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f30841ab6f0 con 0x5572e20f4d70 2024-09-19T04:49:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.500+0000 7f308e86a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f30841ad380 con 0x5572e20ea4b0 2024-09-19T04:49:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.500+0000 7f308e86a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30841abdc0 con 0x5572e20e9b90 2024-09-19T04:49:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.501+0000 7f3088a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20ea4b0 0x5572e211f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.501+0000 7f3088a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20ea4b0 0x5572e211f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:45284/0 (socket says 172.21.15.123:45284) 2024-09-19T04:49:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.501+0000 7f3088a00640 1 -- 172.21.15.123:0/1335555225 learned_addr learned my addr 172.21.15.123:0/1335555225 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.501+0000 7f3089e00640 1 --2- 172.21.15.123:0/1335555225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20f4d70 0x5572e211fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.501+0000 7f3089400640 1 --2- 172.21.15.123:0/1335555225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20e9b90 0x5572e20ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.501+0000 7f3088a00640 1 -- 172.21.15.123:0/1335555225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20e9b90 msgr2=0x5572e20ec5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:12.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.501+0000 7f3088a00640 1 --2- 172.21.15.123:0/1335555225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20e9b90 0x5572e20ec5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.502+0000 7f3088a00640 1 -- 172.21.15.123:0/1335555225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20f4d70 msgr2=0x5572e211fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:12.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.502+0000 7f3088a00640 1 --2- 172.21.15.123:0/1335555225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20f4d70 0x5572e211fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.502+0000 7f3088a00640 1 -- 172.21.15.123:0/1335555225 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572e20e5f50 con 0x5572e20ea4b0 2024-09-19T04:49:12.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.502+0000 7f3089e00640 1 --2- 172.21.15.123:0/1335555225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20f4d70 0x5572e211fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:49:12.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.502+0000 7f3088a00640 1 --2- 172.21.15.123:0/1335555225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20ea4b0 0x5572e211f6a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3070098ec0 tx=0x7f3070018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76ea74b4597fdbe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:12.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.503+0000 7f3075c00640 1 -- 172.21.15.123:0/1335555225 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307009be50 con 0x5572e20ea4b0 2024-09-19T04:49:12.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.503+0000 7f3075c00640 1 -- 172.21.15.123:0/1335555225 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f307009cbf0 con 0x5572e20ea4b0 2024-09-19T04:49:12.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.503+0000 7f3075c00640 1 -- 172.21.15.123:0/1335555225 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307009c9b0 con 0x5572e20ea4b0 2024-09-19T04:49:12.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.504+0000 7f308e86a740 1 -- 172.21.15.123:0/1335555225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20ea4b0 msgr2=0x5572e211f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.504+0000 7f308e86a740 1 --2- 172.21.15.123:0/1335555225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20ea4b0 0x5572e211f6a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3070098ec0 tx=0x7f3070018420 comp rx=0 tx=0).stop 2024-09-19T04:49:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.504+0000 7f308e86a740 1 -- 172.21.15.123:0/1335555225 shutdown_connections 2024-09-19T04:49:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.504+0000 7f308e86a740 1 --2- 172.21.15.123:0/1335555225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20f4d70 0x5572e211fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.504+0000 7f308e86a740 1 --2- 172.21.15.123:0/1335555225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20ea4b0 0x5572e211f6a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.504+0000 7f308e86a740 1 --2- 172.21.15.123:0/1335555225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20e9b90 0x5572e20ec5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.504+0000 7f308e86a740 1 -- 172.21.15.123:0/1335555225 >> 172.21.15.123:0/1335555225 conn(0x5572e20e6120 msgr2=0x5572e20f0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.504+0000 7f308e86a740 1 -- 172.21.15.123:0/1335555225 shutdown_connections 2024-09-19T04:49:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.505+0000 7f308e86a740 1 -- 172.21.15.123:0/1335555225 wait complete. 2024-09-19T04:49:12.507 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] resvtest 2024-09-19T04:49:12.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.505+0000 7f308e86a740 1 -- start start 2024-09-19T04:49:12.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.505+0000 7f308e86a740 1 Processor -- start 2024-09-19T04:49:12.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.506+0000 7f308e86a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20e9b90 0x5572e2105ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.506+0000 7f308e86a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20ea4b0 0x5572e2100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.506+0000 7f308e86a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20f4d70 0x5572e2101410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.506+0000 7f308e86a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f30841acbf0 con 0x5572e20e9b90 2024-09-19T04:49:12.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.506+0000 7f308e86a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f30841ac520 con 0x5572e20f4d70 2024-09-19T04:49:12.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.506+0000 7f308e86a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30841a96d0 con 0x5572e20ea4b0 2024-09-19T04:49:12.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.506+0000 7f308e86a740 10 client.0 _ll_register_callbacks cb 0x7f307c009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:12.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.507+0000 7f3089400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20e9b90 0x5572e2105ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.507+0000 7f3089400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20e9b90 0x5572e2105ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53298/0 (socket says 172.21.15.123:53298) 2024-09-19T04:49:12.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.507+0000 7f3089400640 1 -- 172.21.15.123:0/3926822317 learned_addr learned my addr 172.21.15.123:0/3926822317 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:12.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.507+0000 7f3088a00640 1 --2- 172.21.15.123:0/3926822317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20ea4b0 0x5572e2100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.507+0000 7f3089e00640 1 --2- 172.21.15.123:0/3926822317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20f4d70 0x5572e2101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.507+0000 7f3089400640 1 -- 172.21.15.123:0/3926822317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20ea4b0 msgr2=0x5572e2100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:12.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.507+0000 7f3089400640 1 --2- 172.21.15.123:0/3926822317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20ea4b0 0x5572e2100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.507+0000 7f3089400640 1 -- 172.21.15.123:0/3926822317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20f4d70 msgr2=0x5572e2101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:12.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.507+0000 7f3089400640 1 --2- 172.21.15.123:0/3926822317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20f4d70 0x5572e2101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.507+0000 7f3089400640 1 -- 172.21.15.123:0/3926822317 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572e2101ab0 con 0x5572e20e9b90 2024-09-19T04:49:12.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.508+0000 7f3089400640 1 --2- 172.21.15.123:0/3926822317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20e9b90 0x5572e2105ba0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x5572e20fdad0 tx=0x7f30780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3400593135bed573 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:12.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.508+0000 7f306aa00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:49:12.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.508+0000 7f306aa00640 1 -- 172.21.15.123:0/3926822317 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078012450 con 0x5572e20e9b90 2024-09-19T04:49:12.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.508+0000 7f306aa00640 1 -- 172.21.15.123:0/3926822317 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f30780125f0 con 0x5572e20e9b90 2024-09-19T04:49:12.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.508+0000 7f308e86a740 1 -- 172.21.15.123:0/3926822317 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f307009cbf0 con 0x5572e20e9b90 2024-09-19T04:49:12.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.509+0000 7f306aa00640 1 -- 172.21.15.123:0/3926822317 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f30780127f0 con 0x5572e20e9b90 2024-09-19T04:49:12.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.509+0000 7f308e86a740 10 client.15810 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:12.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.509+0000 7f308e86a740 1 -- 172.21.15.123:0/3926822317 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5572e2180560 con 0x5572e20e9b90 2024-09-19T04:49:12.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.509+0000 7f306aa00640 1 -- 172.21.15.123:0/3926822317 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f307009cbf0 con 0x5572e20e9b90 2024-09-19T04:49:12.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.509+0000 7f306aa00640 1 -- 172.21.15.123:0/3926822317 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f307802f070 con 0x5572e20e9b90 2024-09-19T04:49:12.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.510+0000 7f308e86a740 10 client.15810 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:12.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.510+0000 7f308e86a740 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:49:12.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.510+0000 7f308e86a740 20 client.15810 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:12.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.510+0000 7f308e86a740 1 --2- 172.21.15.123:0/3926822317 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5572e2180d20 0x5572e2183140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:12.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.510+0000 7f308e86a740 4 client.15810 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:49:12.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.510+0000 7f308e86a740 1 -- 172.21.15.123:0/3926822317 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5572e2057e60 con 0x5572e2180d20 2024-09-19T04:49:12.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.510+0000 7f3088a00640 1 --2- 172.21.15.123:0/3926822317 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5572e2180d20 0x5572e2183140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:12.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.511+0000 7f3088a00640 1 --2- 172.21.15.123:0/3926822317 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5572e2180d20 0x5572e2183140 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e2acb7cfeb948f server_cookie=b49cd81277eab34d in_seq=0 out_seq=0 2024-09-19T04:49:12.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.511+0000 7f306aa00640 10 client.15810 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:12.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.511+0000 7f306aa00640 1 -- 172.21.15.123:0/3926822317 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5572e2057e60 con 0x5572e2180d20 2024-09-19T04:49:12.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.511+0000 7f306aa00640 10 client.15810 handle_command_reply: tid=0 2024-09-19T04:49:12.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.511+0000 7f308e86a740 1 client.15810 shutdown 2024-09-19T04:49:12.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.512+0000 7f308e86a740 1 -- 172.21.15.123:0/3926822317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20e9b90 msgr2=0x5572e2105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:12.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.512+0000 7f308e86a740 1 --2- 172.21.15.123:0/3926822317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20e9b90 0x5572e2105ba0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x5572e20fdad0 tx=0x7f30780046c0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.512+0000 7f308e86a740 1 -- 172.21.15.123:0/3926822317 shutdown_connections 2024-09-19T04:49:12.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.512+0000 7f308e86a740 1 --2- 172.21.15.123:0/3926822317 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5572e2180d20 0x5572e2183140 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.512+0000 7f308e86a740 1 --2- 172.21.15.123:0/3926822317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e20f4d70 0x5572e2101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.512+0000 7f308e86a740 1 --2- 172.21.15.123:0/3926822317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572e20ea4b0 0x5572e2100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.512+0000 7f308e86a740 1 --2- 172.21.15.123:0/3926822317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572e20e9b90 0x5572e2105ba0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.512+0000 7f308e86a740 1 -- 172.21.15.123:0/3926822317 >> 172.21.15.123:0/3926822317 conn(0x5572e20e6120 msgr2=0x5572e20f3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:12.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.512+0000 7f306aa00640 0 client.15810 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:12.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.514+0000 7f308e86a740 1 -- 172.21.15.123:0/3926822317 shutdown_connections 2024-09-19T04:49:12.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.514+0000 7f308e86a740 1 -- 172.21.15.123:0/3926822317 wait complete. 2024-09-19T04:49:12.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.514+0000 7f308e86a740 20 client.15810 trim_cache size 0 max 16384 2024-09-19T04:49:12.516 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:49:12.516 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:49:12.516 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": { 2024-09-19T04:49:12.516 INFO:teuthology.orchestra.run.smithi123.stdout: "f8a5315d-9b00-46db-aa2a-c9156aefa539": { 2024-09-19T04:49:12.516 INFO:teuthology.orchestra.run.smithi123.stdout: "path": "/", 2024-09-19T04:49:12.516 INFO:teuthology.orchestra.run.smithi123.stdout: "tag": "f8a5315d-9b00-46db-aa2a-c9156aefa539", 2024-09-19T04:49:12.516 INFO:teuthology.orchestra.run.smithi123.stdout: "options": "recursive,force" 2024-09-19T04:49:12.517 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:49:12.517 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:49:12.517 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:49:12.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.515+0000 7f308be00640 1 -- 172.21.15.123:0/2870684164 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f306c0a3500 msgr2=0x7f306c0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:12.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.515+0000 7f308be00640 1 --2- 172.21.15.123:0/2870684164 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f306c0a3500 0x7f306c0a59c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5572e20e6f80 tx=0x7f3078005f90 comp rx=0 tx=0).stop 2024-09-19T04:49:12.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.515+0000 7f308be00640 1 -- 172.21.15.123:0/2870684164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3084074e30 msgr2=0x7f308419ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:12.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.515+0000 7f308be00640 1 --2- 172.21.15.123:0/2870684164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3084074e30 0x7f308419ab60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f307000e9f0 tx=0x7f307000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.515+0000 7f308be00640 1 -- 172.21.15.123:0/2870684164 shutdown_connections 2024-09-19T04:49:12.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.515+0000 7f308be00640 1 --2- 172.21.15.123:0/2870684164 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f306c0a3500 0x7f306c0a59c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.515+0000 7f308be00640 1 --2- 172.21.15.123:0/2870684164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f308410e440 0x7f308419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.515+0000 7f308be00640 1 --2- 172.21.15.123:0/2870684164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3084074e30 0x7f308419ab60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.515+0000 7f308be00640 1 --2- 172.21.15.123:0/2870684164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3084074510 0x7f308419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:12.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.515+0000 7f308be00640 1 -- 172.21.15.123:0/2870684164 >> 172.21.15.123:0/2870684164 conn(0x7f30840fd8e0 msgr2=0x7f30840fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:12.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.515+0000 7f308be00640 1 -- 172.21.15.123:0/2870684164 shutdown_connections 2024-09-19T04:49:12.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:12.516+0000 7f308be00640 1 -- 172.21.15.123:0/2870684164 wait complete. 2024-09-19T04:49:12.574 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] writemod 2024-09-19T04:49:12.656 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] writev_on_pagefault 2024-09-19T04:49:12.732 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] makeextents 2024-09-19T04:49:12.844 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] itrash 2024-09-19T04:49:12.863 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] rename 2024-09-19T04:49:12.953 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] multi_open_unlink 2024-09-19T04:49:13.062 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] unwritten_sync 2024-09-19T04:49:13.168 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] genhashnames 2024-09-19T04:49:13.168 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_holes 2024-09-19T04:49:13.268 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_mmap_writev 2024-09-19T04:49:13.383 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_truncate_cmtime 2024-09-19T04:49:13.450 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] dirhash_collide 2024-09-19T04:49:13.496 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_rename_overwrite 2024-09-19T04:49:13.556 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] holetest 2024-09-19T04:49:13.673 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_truncate_self 2024-09-19T04:49:13.698 INFO:tasks.cephfs.filesystem:scrub status for tag:f8a5315d-9b00-46db-aa2a-c9156aefa539 - {'path': '/', 'tag': 'f8a5315d-9b00-46db-aa2a-c9156aefa539', 'options': 'recursive,force'} 2024-09-19T04:49:13.765 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] af_unix 2024-09-19T04:49:13.769 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_mmap_stale_pmd 2024-09-19T04:49:13.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:13 smithi123 ceph-mon[30511]: pgmap v78: 97 pgs: 97 active+clean; 75 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-09-19T04:49:13.939 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_mmap_cow_race 2024-09-19T04:49:13.955 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_mmap_fallocate 2024-09-19T04:49:13.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:13 smithi193 ceph-mon[35592]: pgmap v78: 97 pgs: 97 active+clean; 75 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-09-19T04:49:14.052 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] fsync-err 2024-09-19T04:49:14.068 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_mmap_write_ro 2024-09-19T04:49:14.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:13 smithi183 ceph-mon[35333]: pgmap v78: 97 pgs: 97 active+clean; 75 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-09-19T04:49:14.227 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_ext4_dax_journal_corruption 2024-09-19T04:49:14.236 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_ext4_dax_inline_corruption 2024-09-19T04:49:14.344 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_ofd_locks 2024-09-19T04:49:14.380 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_mmap_collision 2024-09-19T04:49:14.504 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] mmap-write-concurrent 2024-09-19T04:49:14.536 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_get_file_time 2024-09-19T04:49:14.677 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_create_short_dirs 2024-09-19T04:49:14.731 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_create_long_dirs 2024-09-19T04:49:14.805 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_enospc 2024-09-19T04:49:14.820 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_mmap_writev_overlap 2024-09-19T04:49:15.003 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] checkpoint_journal 2024-09-19T04:49:15.030 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] mmap-rw-fault 2024-09-19T04:49:15.109 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] allocstale 2024-09-19T04:49:15.149 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_mmap_cow_memory_failure 2024-09-19T04:49:15.288 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] fake-dump-rootino 2024-09-19T04:49:15.357 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] dio-buf-fault 2024-09-19T04:49:15.414 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] rewinddir-test 2024-09-19T04:49:15.477 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] readdir-while-renames 2024-09-19T04:49:15.585 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] xfsctl 2024-09-19T04:49:15.658 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] bstat 2024-09-19T04:49:15.723 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_mtab 2024-09-19T04:49:15.801 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] getdevicesize 2024-09-19T04:49:15.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:15 smithi123 ceph-mon[30511]: pgmap v79: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.3 MiB/s wr, 220 op/s 2024-09-19T04:49:15.933 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] preallo_rw_pattern_reader 2024-09-19T04:49:15.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:15 smithi193 ceph-mon[35592]: pgmap v79: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.3 MiB/s wr, 220 op/s 2024-09-19T04:49:15.994 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] preallo_rw_pattern_writer 2024-09-19T04:49:16.054 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] ftrunc 2024-09-19T04:49:16.059 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] trunc 2024-09-19T04:49:16.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:15 smithi183 ceph-mon[35333]: pgmap v79: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.3 MiB/s wr, 220 op/s 2024-09-19T04:49:16.230 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] fs_perms 2024-09-19T04:49:16.288 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] testx 2024-09-19T04:49:16.326 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] looptest 2024-09-19T04:49:16.370 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] locktest 2024-09-19T04:49:16.533 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] unwritten_mmap 2024-09-19T04:49:16.534 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] bulkstat_unlink_test 2024-09-19T04:49:16.681 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] deduperace 2024-09-19T04:49:16.834 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] bulkstat_unlink_test_modified 2024-09-19T04:49:16.834 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_dir_offset 2024-09-19T04:49:16.894 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_futimens 2024-09-19T04:49:17.049 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_immutable 2024-09-19T04:49:17.130 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] stale_handle 2024-09-19T04:49:17.152 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] pwrite_mmap_blocked 2024-09-19T04:49:17.163 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_dir_offset2 2024-09-19T04:49:17.435 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] seek_sanity_test 2024-09-19T04:49:17.435 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] seek_copy_test 2024-09-19T04:49:17.488 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_readdir_1 2024-09-19T04:49:17.736 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_readdir_2 2024-09-19T04:49:17.771 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] fsync-tester 2024-09-19T04:49:17.926 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] nsexec 2024-09-19T04:49:17.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:17 smithi193 ceph-mon[35592]: pgmap v80: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 639 KiB/s wr, 93 op/s 2024-09-19T04:49:17.994 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] cloner 2024-09-19T04:49:18.070 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] renameat2 2024-09-19T04:49:18.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:17 smithi183 ceph-mon[35333]: pgmap v80: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 639 KiB/s wr, 93 op/s 2024-09-19T04:49:18.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:17 smithi123 ceph-mon[30511]: pgmap v80: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 639 KiB/s wr, 93 op/s 2024-09-19T04:49:18.211 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_getcwd 2024-09-19T04:49:18.237 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] e4compact 2024-09-19T04:49:18.331 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] test-nextquota 2024-09-19T04:49:18.384 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] punch-alternating 2024-09-19T04:49:18.502 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] attr-list-by-handle-cursor-test 2024-09-19T04:49:18.594 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] listxattr 2024-09-19T04:49:18.651 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] dio-interleaved 2024-09-19T04:49:18.674 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_dir_type 2024-09-19T04:49:18.819 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] dio-invalidate-cache 2024-09-19T04:49:18.888 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] stat_test 2024-09-19T04:49:18.956 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_encrypted_d_revalidate 2024-09-19T04:49:18.966 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] attr_replace_test 2024-09-19T04:49:19.177 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] swapon 2024-09-19T04:49:19.237 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] mkswap 2024-09-19T04:49:19.264 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_attr_corruption 2024-09-19T04:49:19.335 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_open_tmpfiles 2024-09-19T04:49:19.473 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] fscrypt-crypt-util 2024-09-19T04:49:19.533 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] bulkstat_null_ocount 2024-09-19T04:49:19.536 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] splice-test 2024-09-19T04:49:19.680 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] chprojid_fail 2024-09-19T04:49:19.827 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] detached_mounts_propagation 2024-09-19T04:49:19.861 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] ext4_resize 2024-09-19T04:49:19.984 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_readdir_3 2024-09-19T04:49:19.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:19 smithi193 ceph-mon[35592]: pgmap v81: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 645 KiB/s wr, 100 op/s 2024-09-19T04:49:20.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:19 smithi183 ceph-mon[35333]: pgmap v81: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 645 KiB/s wr, 100 op/s 2024-09-19T04:49:20.132 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] splice2pipe 2024-09-19T04:49:20.151 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] uuid_ioctl 2024-09-19T04:49:20.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:19 smithi123 ceph-mon[30511]: pgmap v81: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 645 KiB/s wr, 100 op/s 2024-09-19T04:49:20.346 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_snapshot_deleted_subvolume 2024-09-19T04:49:20.368 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] fiemap-fault 2024-09-19T04:49:20.422 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] fiemap-tester 2024-09-19T04:49:20.440 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_stripealign 2024-09-19T04:49:20.647 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] open_by_handle 2024-09-19T04:49:20.668 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_mmap_dio 2024-09-19T04:49:20.739 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] xfsfind 2024-09-19T04:49:20.839 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] t_reflink_read_race 2024-09-19T04:49:20.952 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] fssum 2024-09-19T04:49:21.061 INFO:tasks.workunit.client.0.smithi123.stdout:Building log-writes 2024-09-19T04:49:21.071 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] replay-log 2024-09-19T04:49:21.084 INFO:tasks.workunit.client.0.smithi123.stdout:Building perf 2024-09-19T04:49:21.092 INFO:tasks.workunit.client.0.smithi123.stdout:Building aio-dio-regress 2024-09-19T04:49:21.105 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aiocp 2024-09-19T04:49:21.184 INFO:tasks.workunit.client.0.smithi123.stdout:Building vfs 2024-09-19T04:49:21.195 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] vfstest 2024-09-19T04:49:21.479 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aio-dio-append-write-fallocate-race 2024-09-19T04:49:21.540 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] mount-idmapped 2024-09-19T04:49:21.567 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aio-dio-append-write-read-race 2024-09-19T04:49:21.801 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aio-dio-cow-race 2024-09-19T04:49:21.867 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aio-dio-cycle-write 2024-09-19T04:49:21.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:21 smithi193 ceph-mon[35592]: pgmap v82: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-19T04:49:22.070 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aio-dio-eof-race 2024-09-19T04:49:22.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:21 smithi183 ceph-mon[35333]: pgmap v82: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-19T04:49:22.119 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aio-dio-extend-stat 2024-09-19T04:49:22.170 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aio-dio-fcntl-race 2024-09-19T04:49:22.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:21 smithi123 ceph-mon[30511]: pgmap v82: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-19T04:49:22.379 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aio-dio-hole-filling-race 2024-09-19T04:49:22.416 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aio-dio-invalidate-failure 2024-09-19T04:49:22.463 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aio-dio-invalidate-readahead 2024-09-19T04:49:22.652 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aiodio_sparse2 2024-09-19T04:49:22.706 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aio-dio-subblock-eof-read 2024-09-19T04:49:22.783 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aio-dio-write-verify 2024-09-19T04:49:22.979 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-09-19T04:49:23.009 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-09-19T04:49:23.123 INFO:tasks.workunit.client.0.smithi123.stdout: [CC] aio-last-ref-held-by-io 2024-09-19T04:49:23.694 INFO:tasks.workunit.client.0.smithi123.stderr:+ cd .. 2024-09-19T04:49:23.694 INFO:tasks.workunit.client.0.smithi123.stderr:+ cp xfstests-dev/ltp/fsx . 2024-09-19T04:49:23.698 INFO:tasks.workunit.client.0.smithi123.stderr:+ OPTIONS=-z 2024-09-19T04:49:23.698 INFO:tasks.workunit.client.0.smithi123.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-09-19T04:49:23.699 INFO:tasks.workunit.client.0.smithi123.stdout:Seed set to 1 2024-09-19T04:49:23.745 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-19T04:49:23.745 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-19T04:49:23.746 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-19T04:49:23.747 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-19T04:49:23.747 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support clone range, disabling! 2024-09-19T04:49:23.749 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-19T04:49:23.751 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support exchange range, disabling! 2024-09-19T04:49:23.751 INFO:tasks.workunit.client.0.smithi123.stdout:skipping zero size read 2024-09-19T04:49:23.751 INFO:tasks.workunit.client.0.smithi123.stdout:skipping zero size read 2024-09-19T04:49:23.751 INFO:tasks.workunit.client.0.smithi123.stdout:truncating to largest ever: 0x81d54 2024-09-19T04:49:23.755 INFO:tasks.workunit.client.0.smithi123.stdout:copying to largest ever: 0xb86d7 2024-09-19T04:49:23.763 INFO:tasks.workunit.client.0.smithi123.stdout:copying to largest ever: 0xe627e 2024-09-19T04:49:23.804 INFO:tasks.workunit.client.0.smithi123.stdout:truncating to largest ever: 0xf3194 2024-09-19T04:49:23.826 INFO:tasks.workunit.client.0.smithi123.stdout:copying to largest ever: 0xf3ceb 2024-09-19T04:49:23.833 INFO:tasks.workunit.client.0.smithi123.stdout:truncating to largest ever: 0xf47e4 2024-09-19T04:49:23.927 INFO:tasks.workunit.client.0.smithi123.stdout:copying to largest ever: 0xfaccc 2024-09-19T04:49:23.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:23 smithi193 ceph-mon[35592]: pgmap v83: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-19T04:49:24.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:23 smithi183 ceph-mon[35333]: pgmap v83: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-19T04:49:24.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:23 smithi123 ceph-mon[30511]: pgmap v83: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-19T04:49:24.329 INFO:tasks.workunit.client.0.smithi123.stdout:copying to largest ever: 0xfe423 2024-09-19T04:49:24.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:24 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:49:24.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:24 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:49:24.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:24 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:49:25.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:24 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:49:25.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:24 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:49:25.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:24 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:49:25.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:24 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:49:25.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:24 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:49:25.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:24 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:49:25.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:25 smithi193 ceph-mon[35592]: pgmap v84: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 19 op/s 2024-09-19T04:49:25.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:25 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:49:25.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:25 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:49:26.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:25 smithi183 ceph-mon[35333]: pgmap v84: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 19 op/s 2024-09-19T04:49:26.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:25 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:49:26.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:25 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:49:26.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:25 smithi123 ceph-mon[30511]: pgmap v84: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 19 op/s 2024-09-19T04:49:26.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:25 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:49:26.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:25 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:49:26.732 INFO:tasks.workunit.client.0.smithi123.stdout:copying to largest ever: 0xfe492 2024-09-19T04:49:27.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:27 smithi193 ceph-mon[35592]: pgmap v85: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-19T04:49:28.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:27 smithi183 ceph-mon[35333]: pgmap v85: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-19T04:49:28.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:27 smithi123 ceph-mon[30511]: pgmap v85: 97 pgs: 97 active+clean; 76 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-19T04:49:28.756 INFO:tasks.workunit.client.0.smithi123.stdout:copying to largest ever: 0xff038 2024-09-19T04:49:29.598 INFO:tasks.workunit.client.0.smithi123.stdout:truncating to largest ever: 0xffcac 2024-09-19T04:49:29.722 INFO:tasks.workunit.client.0.smithi123.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-09-19T04:49:29.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:29 smithi193 ceph-mon[35592]: pgmap v86: 97 pgs: 97 active+clean; 79 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-09-19T04:49:30.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:29 smithi183 ceph-mon[35333]: pgmap v86: 97 pgs: 97 active+clean; 79 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-09-19T04:49:30.176 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:29 smithi123 ceph-mon[30511]: pgmap v86: 97 pgs: 97 active+clean; 79 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-09-19T04:49:31.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:31 smithi193 ceph-mon[35592]: pgmap v87: 97 pgs: 97 active+clean; 79 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 276 op/s 2024-09-19T04:49:32.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:31 smithi183 ceph-mon[35333]: pgmap v87: 97 pgs: 97 active+clean; 79 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 276 op/s 2024-09-19T04:49:32.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:31 smithi123 ceph-mon[30511]: pgmap v87: 97 pgs: 97 active+clean; 79 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 276 op/s 2024-09-19T04:49:33.995 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:33 smithi123 ceph-mon[30511]: pgmap v88: 97 pgs: 97 active+clean; 79 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 276 op/s 2024-09-19T04:49:34.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:33 smithi183 ceph-mon[35333]: pgmap v88: 97 pgs: 97 active+clean; 79 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 276 op/s 2024-09-19T04:49:34.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:33 smithi193 ceph-mon[35592]: pgmap v88: 97 pgs: 97 active+clean; 79 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 276 op/s 2024-09-19T04:49:35.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:35 smithi123 ceph-mon[30511]: pgmap v89: 97 pgs: 97 active+clean; 83 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 588 op/s 2024-09-19T04:49:36.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:35 smithi183 ceph-mon[35333]: pgmap v89: 97 pgs: 97 active+clean; 83 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 588 op/s 2024-09-19T04:49:36.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:35 smithi193 ceph-mon[35592]: pgmap v89: 97 pgs: 97 active+clean; 83 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 588 op/s 2024-09-19T04:49:38.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:37 smithi183 ceph-mon[35333]: pgmap v90: 97 pgs: 97 active+clean; 83 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 582 op/s 2024-09-19T04:49:38.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:37 smithi123 ceph-mon[30511]: pgmap v90: 97 pgs: 97 active+clean; 83 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 582 op/s 2024-09-19T04:49:38.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:37 smithi193 ceph-mon[35592]: pgmap v90: 97 pgs: 97 active+clean; 83 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 582 op/s 2024-09-19T04:49:40.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:39 smithi183 ceph-mon[35333]: pgmap v91: 97 pgs: 97 active+clean; 87 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 846 op/s 2024-09-19T04:49:40.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:39 smithi123 ceph-mon[30511]: pgmap v91: 97 pgs: 97 active+clean; 87 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 846 op/s 2024-09-19T04:49:40.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:39 smithi193 ceph-mon[35592]: pgmap v91: 97 pgs: 97 active+clean; 87 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 846 op/s 2024-09-19T04:49:42.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:41 smithi183 ceph-mon[35333]: pgmap v92: 97 pgs: 97 active+clean; 87 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 575 op/s 2024-09-19T04:49:42.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:41 smithi123 ceph-mon[30511]: pgmap v92: 97 pgs: 97 active+clean; 87 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 575 op/s 2024-09-19T04:49:42.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:41 smithi193 ceph-mon[35592]: pgmap v92: 97 pgs: 97 active+clean; 87 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 575 op/s 2024-09-19T04:49:43.699 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:49:44.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:43 smithi183 ceph-mon[35333]: pgmap v93: 97 pgs: 97 active+clean; 87 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 575 op/s 2024-09-19T04:49:44.167 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:49:44.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:43 smithi123 ceph-mon[30511]: pgmap v93: 97 pgs: 97 active+clean; 87 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 575 op/s 2024-09-19T04:49:44.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:43 smithi193 ceph-mon[35592]: pgmap v93: 97 pgs: 97 active+clean; 87 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 575 op/s 2024-09-19T04:49:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.964+0000 7f7d87200640 1 -- 172.21.15.123:0/2930664501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d88103e30 msgr2=0x7f7d88104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.964+0000 7f7d87200640 1 --2- 172.21.15.123:0/2930664501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d88103e30 0x7f7d88104210 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009860 tx=0x7f7d7802a340 comp rx=0 tx=0).stop 2024-09-19T04:49:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.964+0000 7f7d87200640 1 -- 172.21.15.123:0/2930664501 shutdown_connections 2024-09-19T04:49:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.964+0000 7f7d87200640 1 --2- 172.21.15.123:0/2930664501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d88108f60 0x7f7d88110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.964+0000 7f7d87200640 1 --2- 172.21.15.123:0/2930664501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d881047e0 0x7f7d88108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.964+0000 7f7d87200640 1 --2- 172.21.15.123:0/2930664501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d88103e30 0x7f7d88104210 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:44.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.964+0000 7f7d87200640 1 -- 172.21.15.123:0/2930664501 >> 172.21.15.123:0/2930664501 conn(0x7f7d880ff910 msgr2=0x7f7d88101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:44.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.964+0000 7f7d87200640 1 -- 172.21.15.123:0/2930664501 shutdown_connections 2024-09-19T04:49:44.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.964+0000 7f7d87200640 1 -- 172.21.15.123:0/2930664501 wait complete. 2024-09-19T04:49:44.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.965+0000 7f7d87200640 1 Processor -- start 2024-09-19T04:49:44.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.965+0000 7f7d87200640 1 -- start start 2024-09-19T04:49:44.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.966+0000 7f7d87200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d88103e30 0x7f7d8819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:44.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.966+0000 7f7d87200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d881047e0 0x7f7d8819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:44.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.966+0000 7f7d87200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d88108f60 0x7f7d8819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:44.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.966+0000 7f7d87200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d88113fc0 con 0x7f7d88108f60 2024-09-19T04:49:44.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.966+0000 7f7d87200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d88113e40 con 0x7f7d88103e30 2024-09-19T04:49:44.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.966+0000 7f7d87200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d88114140 con 0x7f7d881047e0 2024-09-19T04:49:44.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.966+0000 7f7d85e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d88103e30 0x7f7d8819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:44.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.966+0000 7f7d85e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d88103e30 0x7f7d8819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:50208/0 (socket says 172.21.15.123:50208) 2024-09-19T04:49:44.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.966+0000 7f7d85e00640 1 -- 172.21.15.123:0/4273534006 learned_addr learned my addr 172.21.15.123:0/4273534006 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:44.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.966+0000 7f7d85400640 1 --2- 172.21.15.123:0/4273534006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d881047e0 0x7f7d8819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:44.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.966+0000 7f7d86800640 1 --2- 172.21.15.123:0/4273534006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d88108f60 0x7f7d8819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:44.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.966+0000 7f7d85e00640 1 -- 172.21.15.123:0/4273534006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d881047e0 msgr2=0x7f7d8819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:44.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.966+0000 7f7d85e00640 1 --2- 172.21.15.123:0/4273534006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d881047e0 0x7f7d8819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:44.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.966+0000 7f7d85e00640 1 -- 172.21.15.123:0/4273534006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d88108f60 msgr2=0x7f7d8819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:44.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.966+0000 7f7d85e00640 1 --2- 172.21.15.123:0/4273534006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d88108f60 0x7f7d8819ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:44.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.966+0000 7f7d85e00640 1 -- 172.21.15.123:0/4273534006 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8819f470 con 0x7f7d88103e30 2024-09-19T04:49:44.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.967+0000 7f7d85e00640 1 --2- 172.21.15.123:0/4273534006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d88103e30 0x7f7d8819a4c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7d78035040 tx=0x7f7d78033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:44.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.967+0000 7f7d7ea00640 1 -- 172.21.15.123:0/4273534006 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78037040 con 0x7f7d88103e30 2024-09-19T04:49:44.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.967+0000 7f7d87200640 1 -- 172.21.15.123:0/4273534006 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8819f700 con 0x7f7d88103e30 2024-09-19T04:49:44.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.967+0000 7f7d7ea00640 1 -- 172.21.15.123:0/4273534006 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d78040070 con 0x7f7d88103e30 2024-09-19T04:49:44.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.967+0000 7f7d7ea00640 1 -- 172.21.15.123:0/4273534006 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d780039b0 con 0x7f7d88103e30 2024-09-19T04:49:44.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.967+0000 7f7d87200640 1 -- 172.21.15.123:0/4273534006 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d881a7040 con 0x7f7d88103e30 2024-09-19T04:49:44.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.971+0000 7f7d8fe05740 1 Processor -- start 2024-09-19T04:49:44.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.972+0000 7f7d8fe05740 1 -- start start 2024-09-19T04:49:44.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.972+0000 7f7d7ea00640 1 -- 172.21.15.123:0/4273534006 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f7d780305e0 con 0x7f7d88103e30 2024-09-19T04:49:44.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.972+0000 7f7d8fe05740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc883d50 0x5569cc884130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:44.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.972+0000 7f7d8fe05740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc884670 0x5569cc88e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:44.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.972+0000 7f7d8fe05740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc88ef30 0x5569cc891320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:44.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.972+0000 7f7d8fe05740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5569cc7ea3e0 con 0x5569cc884670 2024-09-19T04:49:44.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.972+0000 7f7d8fe05740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569cc791d80 con 0x5569cc883d50 2024-09-19T04:49:44.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.972+0000 7f7d8fe05740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5569cc7e63a0 con 0x5569cc88ef30 2024-09-19T04:49:44.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.972+0000 7f7d85e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc884670 0x5569cc88e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:44.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.972+0000 7f7d85400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc88ef30 0x5569cc891320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:44.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.972+0000 7f7d86800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc883d50 0x5569cc884130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:44.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.972+0000 7f7d85e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc884670 0x5569cc88e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46520/0 (socket says 172.21.15.123:46520) 2024-09-19T04:49:44.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.972+0000 7f7d85e00640 1 -- 172.21.15.123:0/3849125791 learned_addr learned my addr 172.21.15.123:0/3849125791 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:44.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.972+0000 7f7d85400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc88ef30 0x5569cc891320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:51870/0 (socket says 172.21.15.123:51870) 2024-09-19T04:49:44.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.973+0000 7f7d85400640 1 -- 172.21.15.123:0/3849125791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc883d50 msgr2=0x5569cc884130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.973+0000 7f7d85400640 1 --2- 172.21.15.123:0/3849125791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc883d50 0x5569cc884130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.973+0000 7f7d85400640 1 -- 172.21.15.123:0/3849125791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc884670 msgr2=0x5569cc88e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.973+0000 7f7d85400640 1 --2- 172.21.15.123:0/3849125791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc884670 0x5569cc88e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.973+0000 7f7d85400640 1 -- 172.21.15.123:0/3849125791 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569cc7b0ae0 con 0x5569cc88ef30 2024-09-19T04:49:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.973+0000 7f7d85400640 1 --2- 172.21.15.123:0/3849125791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc88ef30 0x5569cc891320 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7d74005fd0 tx=0x7f7d740113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3050bc51c7b51a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.974+0000 7f7d73400640 1 -- 172.21.15.123:0/3849125791 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74011e10 con 0x5569cc88ef30 2024-09-19T04:49:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.974+0000 7f7d73400640 1 -- 172.21.15.123:0/3849125791 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d74005500 con 0x5569cc88ef30 2024-09-19T04:49:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.974+0000 7f7d73400640 1 -- 172.21.15.123:0/3849125791 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d740127d0 con 0x5569cc88ef30 2024-09-19T04:49:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.974+0000 7f7d7ea00640 1 --2- 172.21.15.123:0/4273534006 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7d6c0a3430 0x7f7d6c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.974+0000 7f7d86800640 1 --2- 172.21.15.123:0/4273534006 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7d6c0a3430 0x7f7d6c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:44.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.975+0000 7f7d7ea00640 1 -- 172.21.15.123:0/4273534006 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f7d780efcb0 con 0x7f7d88103e30 2024-09-19T04:49:44.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.975+0000 7f7d8fe05740 1 -- 172.21.15.123:0/3849125791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc88ef30 msgr2=0x5569cc891320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:44.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.975+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/3849125791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc88ef30 0x5569cc891320 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7d74005fd0 tx=0x7f7d740113e0 comp rx=0 tx=0).stop 2024-09-19T04:49:44.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.975+0000 7f7d86800640 1 --2- 172.21.15.123:0/4273534006 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7d6c0a3430 0x7f7d6c0a58f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7d80005fd0 tx=0x7f7d80009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:44.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.976+0000 7f7d8fe05740 1 -- 172.21.15.123:0/3849125791 shutdown_connections 2024-09-19T04:49:44.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.976+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/3849125791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc88ef30 0x5569cc891320 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:44.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.976+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/3849125791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc884670 0x5569cc88e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:44.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.976+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/3849125791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc883d50 0x5569cc884130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:44.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.976+0000 7f7d8fe05740 1 -- 172.21.15.123:0/3849125791 >> 172.21.15.123:0/3849125791 conn(0x5569cc8802e0 msgr2=0x5569cc8806c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:44.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.976+0000 7f7d8fe05740 1 -- 172.21.15.123:0/3849125791 shutdown_connections 2024-09-19T04:49:44.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.976+0000 7f7d8fe05740 1 -- 172.21.15.123:0/3849125791 wait complete. 2024-09-19T04:49:44.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.977+0000 7f7d8fe05740 1 -- start start 2024-09-19T04:49:44.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.977+0000 7f7d8fe05740 1 Processor -- start 2024-09-19T04:49:44.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.977+0000 7f7d8fe05740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc883d50 0x5569cc88be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:44.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.977+0000 7f7d8fe05740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc884670 0x5569cc88c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:44.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.978+0000 7f7d8fe05740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc88ef30 0x5569cc88c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:44.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.978+0000 7f7d8fe05740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5569cc891ba0 con 0x5569cc88ef30 2024-09-19T04:49:44.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.978+0000 7f7d8fe05740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569cc7a50f0 con 0x5569cc883d50 2024-09-19T04:49:44.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.978+0000 7f7d8fe05740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5569cc891d20 con 0x5569cc884670 2024-09-19T04:49:44.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.978+0000 7f7d8fe05740 10 client.0 _ll_register_callbacks cb 0x5569cc7969e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:44.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.978+0000 7f7d85e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc883d50 0x5569cc88be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:44.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.978+0000 7f7d85400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc884670 0x5569cc88c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:44.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.978+0000 7f7d85400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc884670 0x5569cc88c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:51886/0 (socket says 172.21.15.123:51886) 2024-09-19T04:49:44.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.978+0000 7f7d85400640 1 -- 172.21.15.123:0/4126849192 learned_addr learned my addr 172.21.15.123:0/4126849192 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:44.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.978+0000 7f7d86800640 1 --2- 172.21.15.123:0/4126849192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc88ef30 0x5569cc88c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:44.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.978+0000 7f7d85e00640 1 -- 172.21.15.123:0/4126849192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc884670 msgr2=0x5569cc88c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:44.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.978+0000 7f7d85e00640 1 --2- 172.21.15.123:0/4126849192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc884670 0x5569cc88c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:44.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.978+0000 7f7d85e00640 1 -- 172.21.15.123:0/4126849192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc88ef30 msgr2=0x5569cc88c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:44.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.978+0000 7f7d85e00640 1 --2- 172.21.15.123:0/4126849192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc88ef30 0x5569cc88c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:44.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.978+0000 7f7d85e00640 1 -- 172.21.15.123:0/4126849192 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569cc55b3c0 con 0x5569cc883d50 2024-09-19T04:49:44.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.978+0000 7f7d85400640 1 --2- 172.21.15.123:0/4126849192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc884670 0x5569cc88c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:49:44.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.979+0000 7f7d85e00640 1 --2- 172.21.15.123:0/4126849192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc883d50 0x5569cc88be20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7d78030b80 tx=0x7f7d780f3350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3079dd8c5f9aed7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:44.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.979+0000 7f7d67e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:49:44.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.979+0000 7f7d67e00640 1 -- 172.21.15.123:0/4126849192 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d780bbbb0 con 0x5569cc883d50 2024-09-19T04:49:44.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.979+0000 7f7d67e00640 1 -- 172.21.15.123:0/4126849192 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f7d780ed650 con 0x5569cc883d50 2024-09-19T04:49:44.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.979+0000 7f7d8fe05740 1 -- 172.21.15.123:0/4126849192 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d74005500 con 0x5569cc883d50 2024-09-19T04:49:44.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.980+0000 7f7d67e00640 1 -- 172.21.15.123:0/4126849192 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7d780bb9d0 con 0x5569cc883d50 2024-09-19T04:49:44.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.980+0000 7f7d8fe05740 10 client.25868 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:44.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.980+0000 7f7d8fe05740 1 -- 172.21.15.123:0/4126849192 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5569cc8a6090 con 0x5569cc883d50 2024-09-19T04:49:44.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.980+0000 7f7d67e00640 1 -- 172.21.15.123:0/4126849192 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d74005500 con 0x5569cc883d50 2024-09-19T04:49:44.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.980+0000 7f7d67e00640 1 -- 172.21.15.123:0/4126849192 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f7d780d7020 con 0x5569cc883d50 2024-09-19T04:49:44.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.980+0000 7f7d8fe05740 10 client.25868 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:44.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.980+0000 7f7d8fe05740 10 client.25868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:49:44.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.980+0000 7f7d8fe05740 20 client.25868 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:44.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.980+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/4126849192 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5569cc8a7540 0x5569cc8a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:44.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.981+0000 7f7d8fe05740 4 client.25868 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:49:44.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.981+0000 7f7d8fe05740 1 -- 172.21.15.123:0/4126849192 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569cc7f7950 con 0x5569cc8a7540 2024-09-19T04:49:44.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.983+0000 7f7d85400640 1 --2- 172.21.15.123:0/4126849192 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5569cc8a7540 0x5569cc8a99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:44.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.983+0000 7f7d85400640 1 --2- 172.21.15.123:0/4126849192 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5569cc8a7540 0x5569cc8a99e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7f80e61bd136a4a server_cookie=eab0973f3474b814 in_seq=0 out_seq=0 2024-09-19T04:49:44.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.984+0000 7f7d67e00640 10 client.25868 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:44.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.984+0000 7f7d67e00640 1 -- 172.21.15.123:0/4126849192 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5569cc7f7950 con 0x5569cc8a7540 2024-09-19T04:49:44.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.984+0000 7f7d67e00640 10 client.25868 handle_command_reply: tid=0 2024-09-19T04:49:44.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.984+0000 7f7d8fe05740 1 client.25868 shutdown 2024-09-19T04:49:44.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.985+0000 7f7d8fe05740 1 -- 172.21.15.123:0/4126849192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc883d50 msgr2=0x5569cc88be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:44.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.985+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/4126849192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc883d50 0x5569cc88be20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7d78030b80 tx=0x7f7d780f3350 comp rx=0 tx=0).stop 2024-09-19T04:49:44.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.985+0000 7f7d8fe05740 1 -- 172.21.15.123:0/4126849192 shutdown_connections 2024-09-19T04:49:44.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.985+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/4126849192 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5569cc8a7540 0x5569cc8a99e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:44.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.985+0000 7f7d67e00640 0 client.25868 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:44.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.985+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/4126849192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc88ef30 0x5569cc88c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:44.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.985+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/4126849192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc884670 0x5569cc88c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:44.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.985+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/4126849192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc883d50 0x5569cc88be20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:44.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.985+0000 7f7d8fe05740 1 -- 172.21.15.123:0/4126849192 >> 172.21.15.123:0/4126849192 conn(0x5569cc8802e0 msgr2=0x5569cc886010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:44.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.985+0000 7f7d8fe05740 1 -- 172.21.15.123:0/4126849192 shutdown_connections 2024-09-19T04:49:44.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.985+0000 7f7d8fe05740 1 -- 172.21.15.123:0/4126849192 wait complete. 2024-09-19T04:49:44.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:44.985+0000 7f7d8fe05740 20 client.25868 trim_cache size 0 max 16384 2024-09-19T04:49:45.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.011+0000 7f7d8fe05740 1 Processor -- start 2024-09-19T04:49:45.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.011+0000 7f7d8fe05740 1 -- start start 2024-09-19T04:49:45.013 INFO:tasks.workunit.client.0.smithi123.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-09-19T04:49:45.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.011+0000 7f7d8fe05740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc883d50 0x5569cc8af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:45.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.011+0000 7f7d8fe05740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc884670 0x5569cc8af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:45.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.011+0000 7f7d8fe05740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc88ef30 0x5569cc8a26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:45.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.011+0000 7f7d8fe05740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881ab530 con 0x5569cc88ef30 2024-09-19T04:49:45.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.011+0000 7f7d8fe05740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881ad1c0 con 0x5569cc884670 2024-09-19T04:49:45.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.011+0000 7f7d8fe05740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881abc00 con 0x5569cc883d50 2024-09-19T04:49:45.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.012+0000 7f7d85400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc884670 0x5569cc8af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:45.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.012+0000 7f7d85e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc883d50 0x5569cc8af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:45.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.012+0000 7f7d85400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc884670 0x5569cc8af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:50240/0 (socket says 172.21.15.123:50240) 2024-09-19T04:49:45.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.012+0000 7f7d85400640 1 -- 172.21.15.123:0/176598934 learned_addr learned my addr 172.21.15.123:0/176598934 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:45.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.012+0000 7f7d86800640 1 --2- 172.21.15.123:0/176598934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc88ef30 0x5569cc8a26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:45.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.012+0000 7f7d85400640 1 -- 172.21.15.123:0/176598934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc883d50 msgr2=0x5569cc8af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:45.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.012+0000 7f7d85400640 1 --2- 172.21.15.123:0/176598934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc883d50 0x5569cc8af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:45.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.012+0000 7f7d85400640 1 -- 172.21.15.123:0/176598934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc88ef30 msgr2=0x5569cc8a26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:45.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.012+0000 7f7d85400640 1 --2- 172.21.15.123:0/176598934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc88ef30 0x5569cc8a26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:45.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.012+0000 7f7d85400640 1 -- 172.21.15.123:0/176598934 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569cc87fb70 con 0x5569cc884670 2024-09-19T04:49:45.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.012+0000 7f7d85e00640 1 --2- 172.21.15.123:0/176598934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc883d50 0x5569cc8af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:49:45.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.012+0000 7f7d85400640 1 --2- 172.21.15.123:0/176598934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc884670 0x5569cc8af830 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7d74005fd0 tx=0x7f7d74009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3910df452121d86d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:45.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.013+0000 7f7d72a00640 1 -- 172.21.15.123:0/176598934 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7401f040 con 0x5569cc884670 2024-09-19T04:49:45.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.013+0000 7f7d72a00640 1 -- 172.21.15.123:0/176598934 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d74009790 con 0x5569cc884670 2024-09-19T04:49:45.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.013+0000 7f7d72a00640 1 -- 172.21.15.123:0/176598934 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74005390 con 0x5569cc884670 2024-09-19T04:49:45.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.014+0000 7f7d8fe05740 1 -- 172.21.15.123:0/176598934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc884670 msgr2=0x5569cc8af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:45.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.014+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/176598934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc884670 0x5569cc8af830 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7d74005fd0 tx=0x7f7d74009260 comp rx=0 tx=0).stop 2024-09-19T04:49:45.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.014+0000 7f7d8fe05740 1 -- 172.21.15.123:0/176598934 shutdown_connections 2024-09-19T04:49:45.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.014+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/176598934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc88ef30 0x5569cc8a26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:45.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.014+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/176598934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc884670 0x5569cc8af830 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:45.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.014+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/176598934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc883d50 0x5569cc8af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:45.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.014+0000 7f7d8fe05740 1 -- 172.21.15.123:0/176598934 >> 172.21.15.123:0/176598934 conn(0x5569cc8802e0 msgr2=0x5569cc88af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:45.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.014+0000 7f7d8fe05740 1 -- 172.21.15.123:0/176598934 shutdown_connections 2024-09-19T04:49:45.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.014+0000 7f7d8fe05740 1 -- 172.21.15.123:0/176598934 wait complete. 2024-09-19T04:49:45.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.014+0000 7f7d8fe05740 1 -- start start 2024-09-19T04:49:45.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.015+0000 7f7d8fe05740 1 Processor -- start 2024-09-19T04:49:45.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.015+0000 7f7d8fe05740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc883d50 0x5569cc89fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:45.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.015+0000 7f7d8fe05740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc884670 0x5569cc8888d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:45.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.015+0000 7f7d8fe05740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc88ef30 0x5569cc888e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:45.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.015+0000 7f7d8fe05740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881aca30 con 0x5569cc884670 2024-09-19T04:49:45.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.015+0000 7f7d8fe05740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881ac360 con 0x5569cc88ef30 2024-09-19T04:49:45.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.015+0000 7f7d8fe05740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881a9510 con 0x5569cc883d50 2024-09-19T04:49:45.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.015+0000 7f7d8fe05740 10 client.0 _ll_register_callbacks cb 0x7f7d780ed480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:45.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.015+0000 7f7d86800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc88ef30 0x5569cc888e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:45.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.015+0000 7f7d85e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc883d50 0x5569cc89fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:45.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.015+0000 7f7d86800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc88ef30 0x5569cc888e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:50242/0 (socket says 172.21.15.123:50242) 2024-09-19T04:49:45.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.015+0000 7f7d86800640 1 -- 172.21.15.123:0/1145205391 learned_addr learned my addr 172.21.15.123:0/1145205391 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:45.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.015+0000 7f7d85e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc883d50 0x5569cc89fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:51906/0 (socket says 172.21.15.123:51906) 2024-09-19T04:49:45.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.016+0000 7f7d85400640 1 --2- 172.21.15.123:0/1145205391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc884670 0x5569cc8888d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:45.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.016+0000 7f7d86800640 1 -- 172.21.15.123:0/1145205391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc883d50 msgr2=0x5569cc89fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:45.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.016+0000 7f7d86800640 1 --2- 172.21.15.123:0/1145205391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc883d50 0x5569cc89fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:45.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.016+0000 7f7d86800640 1 -- 172.21.15.123:0/1145205391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc884670 msgr2=0x5569cc8888d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:45.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.016+0000 7f7d86800640 1 --2- 172.21.15.123:0/1145205391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc884670 0x5569cc8888d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:45.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.016+0000 7f7d86800640 1 -- 172.21.15.123:0/1145205391 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569cc8893d0 con 0x5569cc88ef30 2024-09-19T04:49:45.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.016+0000 7f7d85400640 1 --2- 172.21.15.123:0/1145205391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc884670 0x5569cc8888d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:49:45.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.016+0000 7f7d86800640 1 --2- 172.21.15.123:0/1145205391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc88ef30 0x5569cc888e10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7d80012730 tx=0x7f7d80004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45cf8a0c61a359ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:45.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.016+0000 7f7d67e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:49:45.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.017+0000 7f7d67e00640 1 -- 172.21.15.123:0/1145205391 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8001c8c0 con 0x5569cc88ef30 2024-09-19T04:49:45.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.017+0000 7f7d67e00640 1 -- 172.21.15.123:0/1145205391 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f7d8001ca60 con 0x5569cc88ef30 2024-09-19T04:49:45.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.017+0000 7f7d8fe05740 1 -- 172.21.15.123:0/1145205391 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d74009790 con 0x5569cc88ef30 2024-09-19T04:49:45.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.017+0000 7f7d67e00640 1 -- 172.21.15.123:0/1145205391 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7d8001cc60 con 0x5569cc88ef30 2024-09-19T04:49:45.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.017+0000 7f7d8fe05740 10 client.25874 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:45.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.017+0000 7f7d8fe05740 1 -- 172.21.15.123:0/1145205391 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5569cc9120c0 con 0x5569cc88ef30 2024-09-19T04:49:45.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.018+0000 7f7d67e00640 1 -- 172.21.15.123:0/1145205391 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d74009790 con 0x5569cc88ef30 2024-09-19T04:49:45.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.018+0000 7f7d67e00640 1 -- 172.21.15.123:0/1145205391 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f7d80026070 con 0x5569cc88ef30 2024-09-19T04:49:45.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.018+0000 7f7d8fe05740 10 client.25874 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:45.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.018+0000 7f7d8fe05740 10 client.25874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:49:45.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.018+0000 7f7d8fe05740 20 client.25874 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:45.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.018+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/1145205391 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5569cc912960 0x5569cc914d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:45.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.018+0000 7f7d8fe05740 4 client.25874 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:49:45.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.018+0000 7f7d8fe05740 1 -- 172.21.15.123:0/1145205391 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5569cc7f1e60 con 0x5569cc912960 2024-09-19T04:49:45.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.018+0000 7f7d85e00640 1 --2- 172.21.15.123:0/1145205391 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5569cc912960 0x5569cc914d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:45.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.021+0000 7f7d85e00640 1 --2- 172.21.15.123:0/1145205391 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5569cc912960 0x5569cc914d80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae0306675929901a server_cookie=189ef5e7f5c927f8 in_seq=0 out_seq=0 2024-09-19T04:49:45.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.021+0000 7f7d67e00640 10 client.25874 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:45.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.021+0000 7f7d67e00640 1 -- 172.21.15.123:0/1145205391 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5569cc7f1e60 con 0x5569cc912960 2024-09-19T04:49:45.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.021+0000 7f7d67e00640 10 client.25874 handle_command_reply: tid=0 2024-09-19T04:49:45.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.021+0000 7f7d8fe05740 1 client.25874 shutdown 2024-09-19T04:49:45.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.022+0000 7f7d8fe05740 1 -- 172.21.15.123:0/1145205391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc88ef30 msgr2=0x5569cc888e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:45.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.022+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/1145205391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc88ef30 0x5569cc888e10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7d80012730 tx=0x7f7d80004a40 comp rx=0 tx=0).stop 2024-09-19T04:49:45.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.022+0000 7f7d8fe05740 1 -- 172.21.15.123:0/1145205391 shutdown_connections 2024-09-19T04:49:45.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.022+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/1145205391 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5569cc912960 0x5569cc914d80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:45.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.022+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/1145205391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569cc88ef30 0x5569cc888e10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:45.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.022+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/1145205391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569cc884670 0x5569cc8888d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:45.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.022+0000 7f7d8fe05740 1 --2- 172.21.15.123:0/1145205391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569cc883d50 0x5569cc89fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:45.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.022+0000 7f7d8fe05740 1 -- 172.21.15.123:0/1145205391 >> 172.21.15.123:0/1145205391 conn(0x5569cc8802e0 msgr2=0x5569cc87eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:45.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.022+0000 7f7d67e00640 0 client.25874 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:45.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.022+0000 7f7d8fe05740 1 -- 172.21.15.123:0/1145205391 shutdown_connections 2024-09-19T04:49:45.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.023+0000 7f7d8fe05740 1 -- 172.21.15.123:0/1145205391 wait complete. 2024-09-19T04:49:45.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.023+0000 7f7d8fe05740 20 client.25874 trim_cache size 0 max 16384 2024-09-19T04:49:45.025 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:49:45.025 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-09-19T04:49:45.025 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-09-19T04:49:45.025 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:49:45.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.024+0000 7f7d87200640 1 -- 172.21.15.123:0/4273534006 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7d6c0a3430 msgr2=0x7f7d6c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:45.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.024+0000 7f7d87200640 1 --2- 172.21.15.123:0/4273534006 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7d6c0a3430 0x7f7d6c0a58f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7d80005fd0 tx=0x7f7d80009500 comp rx=0 tx=0).stop 2024-09-19T04:49:45.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.024+0000 7f7d87200640 1 -- 172.21.15.123:0/4273534006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d88103e30 msgr2=0x7f7d8819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:45.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.024+0000 7f7d87200640 1 --2- 172.21.15.123:0/4273534006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d88103e30 0x7f7d8819a4c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7d78035040 tx=0x7f7d78033b60 comp rx=0 tx=0).stop 2024-09-19T04:49:45.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.024+0000 7f7d87200640 1 -- 172.21.15.123:0/4273534006 shutdown_connections 2024-09-19T04:49:45.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.024+0000 7f7d87200640 1 --2- 172.21.15.123:0/4273534006 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7d6c0a3430 0x7f7d6c0a58f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:45.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.024+0000 7f7d87200640 1 --2- 172.21.15.123:0/4273534006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d88108f60 0x7f7d8819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:45.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.024+0000 7f7d87200640 1 --2- 172.21.15.123:0/4273534006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d881047e0 0x7f7d8819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:45.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.024+0000 7f7d87200640 1 --2- 172.21.15.123:0/4273534006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d88103e30 0x7f7d8819a4c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:45.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.024+0000 7f7d87200640 1 -- 172.21.15.123:0/4273534006 >> 172.21.15.123:0/4273534006 conn(0x7f7d880ff910 msgr2=0x7f7d88100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:45.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.025+0000 7f7d87200640 1 -- 172.21.15.123:0/4273534006 shutdown_connections 2024-09-19T04:49:45.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:45.025+0000 7f7d87200640 1 -- 172.21.15.123:0/4273534006 wait complete. 2024-09-19T04:49:45.443 INFO:tasks.workunit.client.0.smithi123.stdout:copying to largest ever: 0xffcdd 2024-09-19T04:49:46.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:45 smithi183 ceph-mon[35333]: pgmap v94: 97 pgs: 97 active+clean; 90 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.3 MiB/s wr, 836 op/s 2024-09-19T04:49:46.100 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:45 smithi123 ceph-mon[30511]: pgmap v94: 97 pgs: 97 active+clean; 90 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.3 MiB/s wr, 836 op/s 2024-09-19T04:49:46.146 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:49:46.146 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:49:46.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:45 smithi193 ceph-mon[35592]: pgmap v94: 97 pgs: 97 active+clean; 90 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.3 MiB/s wr, 836 op/s 2024-09-19T04:49:46.601 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:49:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.218+0000 7f32ec200640 1 -- 172.21.15.123:0/3848973086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32e4104980 msgr2=0x7f32e41089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.218+0000 7f32ec200640 1 --2- 172.21.15.123:0/3848973086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32e4104980 0x7f32e41089d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009860 tx=0x7f32d002a340 comp rx=0 tx=0).stop 2024-09-19T04:49:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.218+0000 7f32ec200640 1 -- 172.21.15.123:0/3848973086 shutdown_connections 2024-09-19T04:49:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.218+0000 7f32ec200640 1 --2- 172.21.15.123:0/3848973086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32e4109100 0x7f32e4110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.218+0000 7f32ec200640 1 --2- 172.21.15.123:0/3848973086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32e4104980 0x7f32e41089d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.218+0000 7f32ec200640 1 --2- 172.21.15.123:0/3848973086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e4103fd0 0x7f32e41043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.218+0000 7f32ec200640 1 -- 172.21.15.123:0/3848973086 >> 172.21.15.123:0/3848973086 conn(0x7f32e40ffa30 msgr2=0x7f32e4101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.218+0000 7f32ec200640 1 -- 172.21.15.123:0/3848973086 shutdown_connections 2024-09-19T04:49:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.218+0000 7f32ec200640 1 -- 172.21.15.123:0/3848973086 wait complete. 2024-09-19T04:49:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.219+0000 7f32ec200640 1 Processor -- start 2024-09-19T04:49:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.219+0000 7f32ec200640 1 -- start start 2024-09-19T04:49:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.220+0000 7f32ec200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32e4103fd0 0x7f32e4110b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.220+0000 7f32ec200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e4104980 0x7f32e410dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.220+0000 7f32ec200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32e4109100 0x7f32e410e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.220+0000 7f32ec200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4114120 con 0x7f32e4103fd0 2024-09-19T04:49:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.220+0000 7f32ec200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4113fa0 con 0x7f32e4104980 2024-09-19T04:49:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.220+0000 7f32ec200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41142a0 con 0x7f32e4109100 2024-09-19T04:49:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.220+0000 7f32e8e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e4104980 0x7f32e410dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.220+0000 7f32e8e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e4104980 0x7f32e410dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:50262/0 (socket says 172.21.15.123:50262) 2024-09-19T04:49:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.220+0000 7f32e8e00640 1 -- 172.21.15.123:0/2433669184 learned_addr learned my addr 172.21.15.123:0/2433669184 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.220+0000 7f32ea200640 1 --2- 172.21.15.123:0/2433669184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32e4109100 0x7f32e410e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.220+0000 7f32e8e00640 1 -- 172.21.15.123:0/2433669184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32e4109100 msgr2=0x7f32e410e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.220+0000 7f32e8e00640 1 --2- 172.21.15.123:0/2433669184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32e4109100 0x7f32e410e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.220+0000 7f32e8e00640 1 -- 172.21.15.123:0/2433669184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32e4103fd0 msgr2=0x7f32e4110b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.220+0000 7f32e8e00640 1 --2- 172.21.15.123:0/2433669184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32e4103fd0 0x7f32e4110b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.221+0000 7f32e8e00640 1 -- 172.21.15.123:0/2433669184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e410ea50 con 0x7f32e4104980 2024-09-19T04:49:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.221+0000 7f32ea200640 1 --2- 172.21.15.123:0/2433669184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32e4109100 0x7f32e410e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:49:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.221+0000 7f32e8e00640 1 --2- 172.21.15.123:0/2433669184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e4104980 0x7f32e410dc10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f32d0035040 tx=0x7f32d0033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.221+0000 7f32e2000640 1 -- 172.21.15.123:0/2433669184 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d003f020 con 0x7f32e4104980 2024-09-19T04:49:47.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.221+0000 7f32e2000640 1 -- 172.21.15.123:0/2433669184 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32d0032770 con 0x7f32e4104980 2024-09-19T04:49:47.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.221+0000 7f32e2000640 1 -- 172.21.15.123:0/2433669184 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0037040 con 0x7f32e4104980 2024-09-19T04:49:47.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.221+0000 7f32ec200640 1 -- 172.21.15.123:0/2433669184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e41a0700 con 0x7f32e4104980 2024-09-19T04:49:47.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.221+0000 7f32ec200640 1 -- 172.21.15.123:0/2433669184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e41a0b90 con 0x7f32e4104980 2024-09-19T04:49:47.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.222+0000 7f32ec200640 1 -- 172.21.15.123:0/2433669184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e41054b0 con 0x7f32e4104980 2024-09-19T04:49:47.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.225+0000 7f32e2000640 1 -- 172.21.15.123:0/2433669184 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f32d0042020 con 0x7f32e4104980 2024-09-19T04:49:47.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.226+0000 7f32e2000640 1 --2- 172.21.15.123:0/2433669184 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32cc0a33b0 0x7f32cc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:47.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.226+0000 7f32e2000640 1 -- 172.21.15.123:0/2433669184 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f32d00efe70 con 0x7f32e4104980 2024-09-19T04:49:47.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.227+0000 7f32e9800640 1 --2- 172.21.15.123:0/2433669184 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32cc0a33b0 0x7f32cc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:47.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.228+0000 7f32e9800640 1 --2- 172.21.15.123:0/2433669184 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32cc0a33b0 0x7f32cc0a5870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f32d80058c0 tx=0x7f32d8005850 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:47.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.228+0000 7f32e2000640 1 -- 172.21.15.123:0/2433669184 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f32d0049020 con 0x7f32e4104980 2024-09-19T04:49:47.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.505+0000 7f32ec200640 1 -- 172.21.15.123:0/2433669184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32e40778c0 con 0x7f32e4104980 2024-09-19T04:49:47.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.506+0000 7f32e2000640 1 -- 172.21.15.123:0/2433669184 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+12002 (secure 0 0 0) 0x7f32d00bc990 con 0x7f32e4104980 2024-09-19T04:49:47.508 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:49:47.508 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":31,"btime":"2024-09-19T04:48:34:620594+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:48:33.828912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24529,24505,24539,24500,24499]},"id":1}]} 2024-09-19T04:49:47.508 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 31 2024-09-19T04:49:47.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.508+0000 7f32ec200640 1 -- 172.21.15.123:0/2433669184 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32cc0a33b0 msgr2=0x7f32cc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:47.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.508+0000 7f32ec200640 1 --2- 172.21.15.123:0/2433669184 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32cc0a33b0 0x7f32cc0a5870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f32d80058c0 tx=0x7f32d8005850 comp rx=0 tx=0).stop 2024-09-19T04:49:47.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.508+0000 7f32ec200640 1 -- 172.21.15.123:0/2433669184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e4104980 msgr2=0x7f32e410dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:47.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.508+0000 7f32ec200640 1 --2- 172.21.15.123:0/2433669184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e4104980 0x7f32e410dc10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f32d0035040 tx=0x7f32d0033d90 comp rx=0 tx=0).stop 2024-09-19T04:49:47.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.509+0000 7f32ec200640 1 -- 172.21.15.123:0/2433669184 shutdown_connections 2024-09-19T04:49:47.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.509+0000 7f32ec200640 1 --2- 172.21.15.123:0/2433669184 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32cc0a33b0 0x7f32cc0a5870 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f32d80058c0 tx=0x7f32d8005850 comp rx=0 tx=0).stop 2024-09-19T04:49:47.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.509+0000 7f32ec200640 1 --2- 172.21.15.123:0/2433669184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32e4109100 0x7f32e410e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:47.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.509+0000 7f32ec200640 1 --2- 172.21.15.123:0/2433669184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e4104980 0x7f32e410dc10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:47.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.509+0000 7f32ec200640 1 --2- 172.21.15.123:0/2433669184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32e4103fd0 0x7f32e4110b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:47.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.509+0000 7f32ec200640 1 -- 172.21.15.123:0/2433669184 >> 172.21.15.123:0/2433669184 conn(0x7f32e40ffa30 msgr2=0x7f32e4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:47.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.509+0000 7f32ec200640 1 -- 172.21.15.123:0/2433669184 shutdown_connections 2024-09-19T04:49:47.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:47.509+0000 7f32ec200640 1 -- 172.21.15.123:0/2433669184 wait complete. 2024-09-19T04:49:47.566 INFO:tasks.workunit.client.0.smithi123.stdout:copying to largest ever: 0xffd97 2024-09-19T04:49:48.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:47 smithi183 ceph-mon[35333]: pgmap v95: 97 pgs: 97 active+clean; 90 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 524 op/s 2024-09-19T04:49:48.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:47 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2433669184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:49:48.176 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:47 smithi123 ceph-mon[30511]: pgmap v95: 97 pgs: 97 active+clean; 90 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 524 op/s 2024-09-19T04:49:48.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:47 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2433669184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:49:48.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:47 smithi193 ceph-mon[35592]: pgmap v95: 97 pgs: 97 active+clean; 90 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 524 op/s 2024-09-19T04:49:48.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:47 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2433669184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:49:48.528 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:49:48.988 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:49:50.045 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:49 smithi123 ceph-mon[30511]: pgmap v96: 97 pgs: 97 active+clean; 94 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 734 op/s 2024-09-19T04:49:50.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.070+0000 7fdbcf000640 1 -- 172.21.15.123:0/3205850520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbc8109120 msgr2=0x7fdbc8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:50.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.070+0000 7fdbcf000640 1 --2- 172.21.15.123:0/3205850520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbc8109120 0x7fdbc8110ca0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009be0 tx=0x7fdbc002a6d0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.070+0000 7fdbcf000640 1 -- 172.21.15.123:0/3205850520 shutdown_connections 2024-09-19T04:49:50.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.070+0000 7fdbcf000640 1 --2- 172.21.15.123:0/3205850520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbc8109120 0x7fdbc8110ca0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.070+0000 7fdbcf000640 1 --2- 172.21.15.123:0/3205850520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbc81049a0 0x7fdbc81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.070+0000 7fdbcf000640 1 --2- 172.21.15.123:0/3205850520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbc8103ff0 0x7fdbc81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.070+0000 7fdbcf000640 1 -- 172.21.15.123:0/3205850520 >> 172.21.15.123:0/3205850520 conn(0x7fdbc80ffab0 msgr2=0x7fdbc8101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:50.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.070+0000 7fdbcf000640 1 -- 172.21.15.123:0/3205850520 shutdown_connections 2024-09-19T04:49:50.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.070+0000 7fdbcf000640 1 -- 172.21.15.123:0/3205850520 wait complete. 2024-09-19T04:49:50.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.072+0000 7fdbcf000640 1 Processor -- start 2024-09-19T04:49:50.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.072+0000 7fdbcf000640 1 -- start start 2024-09-19T04:49:50.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.072+0000 7fdbcf000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbc8103ff0 0x7fdbc80739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.072+0000 7fdbcf000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbc81049a0 0x7fdbc806cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.072+0000 7fdbcf000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbc8109120 0x7fdbc806d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.072+0000 7fdbcf000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc8114130 con 0x7fdbc8103ff0 2024-09-19T04:49:50.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.072+0000 7fdbcf000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc8113fb0 con 0x7fdbc8109120 2024-09-19T04:49:50.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.072+0000 7fdbcf000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc81142b0 con 0x7fdbc81049a0 2024-09-19T04:49:50.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.072+0000 7fdbcd000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbc8109120 0x7fdbc806d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:50.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.072+0000 7fdbbfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbc81049a0 0x7fdbc806cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:50.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.072+0000 7fdbcd000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbc8109120 0x7fdbc806d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:50280/0 (socket says 172.21.15.123:50280) 2024-09-19T04:49:50.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.072+0000 7fdbcd000640 1 -- 172.21.15.123:0/2214553166 learned_addr learned my addr 172.21.15.123:0/2214553166 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:50.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.073+0000 7fdbc7e00640 1 --2- 172.21.15.123:0/2214553166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbc8103ff0 0x7fdbc80739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:50.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.073+0000 7fdbcd000640 1 -- 172.21.15.123:0/2214553166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbc81049a0 msgr2=0x7fdbc806cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:50.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.073+0000 7fdbcd000640 1 --2- 172.21.15.123:0/2214553166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbc81049a0 0x7fdbc806cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.073+0000 7fdbcd000640 1 -- 172.21.15.123:0/2214553166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbc8103ff0 msgr2=0x7fdbc80739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:50.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.073+0000 7fdbcd000640 1 --2- 172.21.15.123:0/2214553166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbc8103ff0 0x7fdbc80739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.073+0000 7fdbcd000640 1 -- 172.21.15.123:0/2214553166 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc806d5f0 con 0x7fdbc8109120 2024-09-19T04:49:50.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.073+0000 7fdbcd000640 1 --2- 172.21.15.123:0/2214553166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbc8109120 0x7fdbc806d000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009b30 tx=0x7fdbc00305b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:50.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.073+0000 7fdbc5600640 1 -- 172.21.15.123:0/2214553166 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0041020 con 0x7fdbc8109120 2024-09-19T04:49:50.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.074+0000 7fdbc5600640 1 -- 172.21.15.123:0/2214553166 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc0003e50 con 0x7fdbc8109120 2024-09-19T04:49:50.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.074+0000 7fdbcf000640 1 -- 172.21.15.123:0/2214553166 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc806d880 con 0x7fdbc8109120 2024-09-19T04:49:50.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.074+0000 7fdbc5600640 1 -- 172.21.15.123:0/2214553166 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0036040 con 0x7fdbc8109120 2024-09-19T04:49:50.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.074+0000 7fdbcf000640 1 -- 172.21.15.123:0/2214553166 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc8071840 con 0x7fdbc8109120 2024-09-19T04:49:50.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.077+0000 7fdbd1ab1740 1 Processor -- start 2024-09-19T04:49:50.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.077+0000 7fdbc5600640 1 -- 172.21.15.123:0/2214553166 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fdbc0044020 con 0x7fdbc8109120 2024-09-19T04:49:50.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.077+0000 7fdbd1ab1740 1 -- start start 2024-09-19T04:49:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.078+0000 7fdbd1ab1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78f480 0x55b15c78f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.078+0000 7fdbd1ab1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c78fda0 0x55b15c79a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.078+0000 7fdbd1ab1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c79a660 0x55b15c79ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.078+0000 7fdbd1ab1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b15c6ed3e0 con 0x55b15c78fda0 2024-09-19T04:49:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.078+0000 7fdbd1ab1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b15c694d80 con 0x55b15c79a660 2024-09-19T04:49:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.078+0000 7fdbd1ab1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b15c6e93a0 con 0x55b15c78f480 2024-09-19T04:49:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.078+0000 7fdbbfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78f480 0x55b15c78f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:50.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.078+0000 7fdbbfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78f480 0x55b15c78f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:51950/0 (socket says 172.21.15.123:51950) 2024-09-19T04:49:50.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.078+0000 7fdbbfe00640 1 -- 172.21.15.123:0/362774556 learned_addr learned my addr 172.21.15.123:0/362774556 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:50.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.078+0000 7fdbbfe00640 1 --2- 172.21.15.123:0/362774556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c79a660 0x55b15c79ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:50.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.079+0000 7fdbbfe00640 1 -- 172.21.15.123:0/362774556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c79a660 msgr2=0x55b15c79ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:50.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.079+0000 7fdbbfe00640 1 --2- 172.21.15.123:0/362774556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c79a660 0x55b15c79ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.079+0000 7fdbbfe00640 1 -- 172.21.15.123:0/362774556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c78fda0 msgr2=0x55b15c79a120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:49:50.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.079+0000 7fdbbfe00640 1 --2- 172.21.15.123:0/362774556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c78fda0 0x55b15c79a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.079+0000 7fdbbfe00640 1 -- 172.21.15.123:0/362774556 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b15c6b3ae0 con 0x55b15c78f480 2024-09-19T04:49:50.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.079+0000 7fdbbfe00640 1 --2- 172.21.15.123:0/362774556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78f480 0x55b15c78f860 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0012770 tx=0x7fdbb0009660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ff9ec68e239dcde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:50.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.079+0000 7fdbbd600640 1 -- 172.21.15.123:0/362774556 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0012cf0 con 0x55b15c78f480 2024-09-19T04:49:50.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.079+0000 7fdbbd600640 1 -- 172.21.15.123:0/362774556 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb00107d0 con 0x55b15c78f480 2024-09-19T04:49:50.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.079+0000 7fdbbd600640 1 -- 172.21.15.123:0/362774556 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0003850 con 0x55b15c78f480 2024-09-19T04:49:50.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.080+0000 7fdbc5600640 1 --2- 172.21.15.123:0/2214553166 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdbac0a3430 0x7fdbac0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.080+0000 7fdbc5600640 1 -- 172.21.15.123:0/2214553166 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fdbc00f08d0 con 0x7fdbc8109120 2024-09-19T04:49:50.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.080+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/362774556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78f480 msgr2=0x55b15c78f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:50.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.080+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/362774556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78f480 0x55b15c78f860 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0012770 tx=0x7fdbb0009660 comp rx=0 tx=0).stop 2024-09-19T04:49:50.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.081+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/362774556 shutdown_connections 2024-09-19T04:49:50.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.081+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/362774556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c79a660 0x55b15c79ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.081+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/362774556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c78fda0 0x55b15c79a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.081+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/362774556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78f480 0x55b15c78f860 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.081+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/362774556 >> 172.21.15.123:0/362774556 conn(0x55b15c78ba10 msgr2=0x55b15c78bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:50.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.080+0000 7fdbc7e00640 1 --2- 172.21.15.123:0/2214553166 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdbac0a3430 0x7fdbac0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:50.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.081+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/362774556 shutdown_connections 2024-09-19T04:49:50.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.081+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/362774556 wait complete. 2024-09-19T04:49:50.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.082+0000 7fdbc7e00640 1 --2- 172.21.15.123:0/2214553166 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdbac0a3430 0x7fdbac0a58f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8005fd0 tx=0x7fdbb8005850 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:50.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.082+0000 7fdbd1ab1740 1 -- start start 2024-09-19T04:49:50.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.082+0000 7fdbd1ab1740 1 Processor -- start 2024-09-19T04:49:50.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.083+0000 7fdbd1ab1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 0x55b15c7976f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.083+0000 7fdbd1ab1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c79a660 0x55b15c797c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.083+0000 7fdbd1ab1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 0x55b15c7a6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.083+0000 7fdbc7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 0x55b15c7976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:50.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.083+0000 7fdbc7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 0x55b15c7976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:51966/0 (socket says 172.21.15.123:51966) 2024-09-19T04:49:50.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.083+0000 7fdbbfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c79a660 0x55b15c797c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:50.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.083+0000 7fdbd1ab1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b15c79d230 con 0x55b15c79a660 2024-09-19T04:49:50.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.083+0000 7fdbbfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c79a660 0x55b15c797c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46558/0 (socket says 172.21.15.123:46558) 2024-09-19T04:49:50.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.083+0000 7fdbc7e00640 1 -- 172.21.15.123:0/3253297730 learned_addr learned my addr 172.21.15.123:0/3253297730 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:50.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.084+0000 7fdbcd000640 1 --2- 172.21.15.123:0/3253297730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 0x55b15c7a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:50.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.084+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/3253297730 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b15c6a80f0 con 0x55b15c798170 2024-09-19T04:49:50.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.084+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/3253297730 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b15c79d3b0 con 0x55b15c78fda0 2024-09-19T04:49:50.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.084+0000 7fdbd1ab1740 10 client.0 _ll_register_callbacks cb 0x55b15c68fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:50.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.084+0000 7fdbcd000640 1 -- 172.21.15.123:0/3253297730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 msgr2=0x55b15c7976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:50.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.084+0000 7fdbcd000640 1 --2- 172.21.15.123:0/3253297730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 0x55b15c7976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.084+0000 7fdbcd000640 1 -- 172.21.15.123:0/3253297730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c79a660 msgr2=0x55b15c797c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:50.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.084+0000 7fdbcd000640 1 --2- 172.21.15.123:0/3253297730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c79a660 0x55b15c797c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.084+0000 7fdbcd000640 1 -- 172.21.15.123:0/3253297730 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b15c45e3c0 con 0x55b15c798170 2024-09-19T04:49:50.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.084+0000 7fdbc7e00640 1 --2- 172.21.15.123:0/3253297730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 0x55b15c7976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:49:50.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.084+0000 7fdbcd000640 1 --2- 172.21.15.123:0/3253297730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 0x55b15c7a6730 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00ee0d0 tx=0x7fdbc00bc850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=261e20a4ef6a8918 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:50.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.084+0000 7fdbb6000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:49:50.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.085+0000 7fdbb6000640 1 -- 172.21.15.123:0/3253297730 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00bcc90 con 0x55b15c798170 2024-09-19T04:49:50.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.085+0000 7fdbb6000640 1 -- 172.21.15.123:0/3253297730 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fdbc0003970 con 0x55b15c798170 2024-09-19T04:49:50.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.085+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/3253297730 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbb00107d0 con 0x55b15c798170 2024-09-19T04:49:50.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.085+0000 7fdbb6000640 1 -- 172.21.15.123:0/3253297730 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdbc00c1040 con 0x55b15c798170 2024-09-19T04:49:50.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.085+0000 7fdbd1ab1740 10 client.25892 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:50.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.085+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/3253297730 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b15c7a9880 con 0x55b15c798170 2024-09-19T04:49:50.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.085+0000 7fdbb6000640 1 -- 172.21.15.123:0/3253297730 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb00107d0 con 0x55b15c798170 2024-09-19T04:49:50.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.086+0000 7fdbb6000640 1 -- 172.21.15.123:0/3253297730 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fdbc00c5b50 con 0x55b15c798170 2024-09-19T04:49:50.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.086+0000 7fdbd1ab1740 10 client.25892 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:50.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.086+0000 7fdbd1ab1740 10 client.25892 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:49:50.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.086+0000 7fdbd1ab1740 20 client.25892 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:50.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.086+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/3253297730 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b15c7aad30 0x55b15c7ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.086+0000 7fdbd1ab1740 4 client.25892 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:49:50.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.086+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/3253297730 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b15c6fa950 con 0x55b15c7aad30 2024-09-19T04:49:50.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.086+0000 7fdbc7e00640 1 --2- 172.21.15.123:0/3253297730 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b15c7aad30 0x55b15c7ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:50.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.087+0000 7fdbc7e00640 1 --2- 172.21.15.123:0/3253297730 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b15c7aad30 0x55b15c7ad1b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fa06c3eb3c39f72d server_cookie=e94be352c546aec9 in_seq=0 out_seq=0 2024-09-19T04:49:50.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.087+0000 7fdbb6000640 10 client.25892 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:49:50.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.088+0000 7fdbb6000640 1 -- 172.21.15.123:0/3253297730 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b15c6fa950 con 0x55b15c7aad30 2024-09-19T04:49:50.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.088+0000 7fdbb6000640 10 client.25892 handle_command_reply: tid=0 2024-09-19T04:49:50.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.088+0000 7fdbd1ab1740 1 client.25892 shutdown 2024-09-19T04:49:50.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.088+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/3253297730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 msgr2=0x55b15c7a6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:50.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.088+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/3253297730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 0x55b15c7a6730 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00ee0d0 tx=0x7fdbc00bc850 comp rx=0 tx=0).stop 2024-09-19T04:49:50.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.088+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/3253297730 shutdown_connections 2024-09-19T04:49:50.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.088+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/3253297730 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b15c7aad30 0x55b15c7ad1b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.088+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/3253297730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 0x55b15c7a6730 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.088+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/3253297730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c79a660 0x55b15c797c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.088+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/3253297730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 0x55b15c7976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.088+0000 7fdbb6000640 0 client.25892 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:49:50.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.088+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/3253297730 >> 172.21.15.123:0/3253297730 conn(0x55b15c78ba10 msgr2=0x55b15c782380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:50.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.089+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/3253297730 shutdown_connections 2024-09-19T04:49:50.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.089+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/3253297730 wait complete. 2024-09-19T04:49:50.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.089+0000 7fdbd1ab1740 20 client.25892 trim_cache size 0 max 16384 2024-09-19T04:49:50.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:49 smithi183 ceph-mon[35333]: pgmap v96: 97 pgs: 97 active+clean; 94 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 734 op/s 2024-09-19T04:49:50.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.105+0000 7fdbd1ab1740 1 Processor -- start 2024-09-19T04:49:50.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.105+0000 7fdbd1ab1740 1 -- start start 2024-09-19T04:49:50.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.105+0000 7fdbd1ab1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 0x55b15c791f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.106+0000 7fdbd1ab1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 0x55b15c78bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.106+0000 7fdbd1ab1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c79a660 0x55b15c7a6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.106+0000 7fdbd1ab1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdba400e6d0 con 0x55b15c79a660 2024-09-19T04:49:50.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.106+0000 7fdbd1ab1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdba4010360 con 0x55b15c798170 2024-09-19T04:49:50.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.106+0000 7fdbd1ab1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdba400eda0 con 0x55b15c78fda0 2024-09-19T04:49:50.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.106+0000 7fdbbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 0x55b15c78bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:50.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.106+0000 7fdbbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 0x55b15c78bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:50300/0 (socket says 172.21.15.123:50300) 2024-09-19T04:49:50.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.106+0000 7fdbbfe00640 1 -- 172.21.15.123:0/2970689426 learned_addr learned my addr 172.21.15.123:0/2970689426 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.106+0000 7fdbc7e00640 1 --2- 172.21.15.123:0/2970689426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 0x55b15c791f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.106+0000 7fdbbfe00640 1 -- 172.21.15.123:0/2970689426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 msgr2=0x55b15c791f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.106+0000 7fdbbfe00640 1 --2- 172.21.15.123:0/2970689426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 0x55b15c791f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.106+0000 7fdbbfe00640 1 -- 172.21.15.123:0/2970689426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c79a660 msgr2=0x55b15c7a6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.106+0000 7fdbbfe00640 1 --2- 172.21.15.123:0/2970689426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c79a660 0x55b15c7a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.106+0000 7fdbbfe00640 1 -- 172.21.15.123:0/2970689426 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b15c7966a0 con 0x55b15c798170 2024-09-19T04:49:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.107+0000 7fdbbfe00640 1 --2- 172.21.15.123:0/2970689426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 0x55b15c78bdf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0012770 tx=0x7fdbb0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3352c41f304945e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.107+0000 7fdbbcc00640 1 -- 172.21.15.123:0/2970689426 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0005140 con 0x55b15c798170 2024-09-19T04:49:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.107+0000 7fdbbcc00640 1 -- 172.21.15.123:0/2970689426 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb00107a0 con 0x55b15c798170 2024-09-19T04:49:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.107+0000 7fdbbcc00640 1 -- 172.21.15.123:0/2970689426 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb00039d0 con 0x55b15c798170 2024-09-19T04:49:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.108+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/2970689426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 msgr2=0x55b15c78bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.108+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/2970689426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 0x55b15c78bdf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0012770 tx=0x7fdbb0005f90 comp rx=0 tx=0).stop 2024-09-19T04:49:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.108+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/2970689426 shutdown_connections 2024-09-19T04:49:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.108+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/2970689426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c79a660 0x55b15c7a6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.108+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/2970689426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 0x55b15c78bdf0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.108+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/2970689426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 0x55b15c791f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.108+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/2970689426 >> 172.21.15.123:0/2970689426 conn(0x55b15c78ba10 msgr2=0x55b15c782380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.108+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/2970689426 shutdown_connections 2024-09-19T04:49:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.108+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/2970689426 wait complete. 2024-09-19T04:49:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.109+0000 7fdbd1ab1740 1 -- start start 2024-09-19T04:49:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.109+0000 7fdbd1ab1740 1 Processor -- start 2024-09-19T04:49:50.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.110+0000 7fdbd1ab1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 0x55b15c7a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.110+0000 7fdbd1ab1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 0x55b15c7a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.110+0000 7fdbd1ab1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c79a660 0x55b15c79dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.110+0000 7fdbd1ab1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdba400fbd0 con 0x55b15c79a660 2024-09-19T04:49:50.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.110+0000 7fdbd1ab1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdba400f500 con 0x55b15c798170 2024-09-19T04:49:50.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.111+0000 7fdbd1ab1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdba400c6b0 con 0x55b15c78fda0 2024-09-19T04:49:50.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.111+0000 7fdbc7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 0x55b15c7a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:50.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.111+0000 7fdbd1ab1740 10 client.0 _ll_register_callbacks cb 0x7fdbc0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:50.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.111+0000 7fdbc7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 0x55b15c7a2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:51976/0 (socket says 172.21.15.123:51976) 2024-09-19T04:49:50.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.111+0000 7fdbc7e00640 1 -- 172.21.15.123:0/731167478 learned_addr learned my addr 172.21.15.123:0/731167478 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:50.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.111+0000 7fdbbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 0x55b15c7a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:50.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.112+0000 7fdbcd000640 1 --2- 172.21.15.123:0/731167478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c79a660 0x55b15c79dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:50.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.112+0000 7fdbbfe00640 1 -- 172.21.15.123:0/731167478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 msgr2=0x55b15c7a2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:50.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.112+0000 7fdbbfe00640 1 --2- 172.21.15.123:0/731167478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 0x55b15c7a2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.112+0000 7fdbbfe00640 1 -- 172.21.15.123:0/731167478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c79a660 msgr2=0x55b15c79dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:50.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.112+0000 7fdbbfe00640 1 --2- 172.21.15.123:0/731167478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c79a660 0x55b15c79dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.112+0000 7fdbbfe00640 1 -- 172.21.15.123:0/731167478 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b15c79e810 con 0x55b15c798170 2024-09-19T04:49:50.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.113+0000 7fdbcd000640 1 --2- 172.21.15.123:0/731167478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c79a660 0x55b15c79dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:49:50.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.113+0000 7fdbbfe00640 1 --2- 172.21.15.123:0/731167478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 0x55b15c7a2f50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55b15c78ee10 tx=0x7fdbb00124e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7a9537bbcad5f6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:50.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.114+0000 7fdbb6000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:49:50.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.114+0000 7fdbb6000640 1 -- 172.21.15.123:0/731167478 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0005550 con 0x55b15c798170 2024-09-19T04:49:50.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.114+0000 7fdbb6000640 1 -- 172.21.15.123:0/731167478 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fdbb001f470 con 0x55b15c798170 2024-09-19T04:49:50.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.114+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/731167478 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbb00107a0 con 0x55b15c798170 2024-09-19T04:49:50.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.114+0000 7fdbb6000640 1 -- 172.21.15.123:0/731167478 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdbb0003770 con 0x55b15c798170 2024-09-19T04:49:50.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.114+0000 7fdbd1ab1740 10 client.25898 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:50.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.114+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/731167478 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b15c8156f0 con 0x55b15c798170 2024-09-19T04:49:50.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.115+0000 7fdbb6000640 1 -- 172.21.15.123:0/731167478 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb00107a0 con 0x55b15c798170 2024-09-19T04:49:50.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.115+0000 7fdbb6000640 1 -- 172.21.15.123:0/731167478 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fdbb0027700 con 0x55b15c798170 2024-09-19T04:49:50.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.115+0000 7fdbd1ab1740 10 client.25898 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:50.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.115+0000 7fdbd1ab1740 10 client.25898 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:49:50.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.115+0000 7fdbd1ab1740 20 client.25898 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:50.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.115+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/731167478 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b15c815b70 0x55b15c817f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:50.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.115+0000 7fdbd1ab1740 4 client.25898 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:49:50.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.115+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/731167478 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b15c6f4e60 con 0x55b15c815b70 2024-09-19T04:49:50.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.115+0000 7fdbc7e00640 1 --2- 172.21.15.123:0/731167478 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b15c815b70 0x55b15c817f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:50.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.116+0000 7fdbc7e00640 1 --2- 172.21.15.123:0/731167478 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b15c815b70 0x55b15c817f60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=42892cce6d95a864 server_cookie=d83048f5334e25b6 in_seq=0 out_seq=0 2024-09-19T04:49:50.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.116+0000 7fdbb6000640 10 client.25898 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:49:50.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.116+0000 7fdbb6000640 1 -- 172.21.15.123:0/731167478 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b15c6f4e60 con 0x55b15c815b70 2024-09-19T04:49:50.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.116+0000 7fdbb6000640 10 client.25898 handle_command_reply: tid=0 2024-09-19T04:49:50.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.116+0000 7fdbd1ab1740 1 client.25898 shutdown 2024-09-19T04:49:50.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.116+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/731167478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 msgr2=0x55b15c7a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:50.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.116+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/731167478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 0x55b15c7a2f50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55b15c78ee10 tx=0x7fdbb00124e0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.116+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/731167478 shutdown_connections 2024-09-19T04:49:50.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.116+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/731167478 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b15c815b70 0x55b15c817f60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.116+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/731167478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b15c79a660 0x55b15c79dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.116+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/731167478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b15c798170 0x55b15c7a2f50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.116+0000 7fdbd1ab1740 1 --2- 172.21.15.123:0/731167478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b15c78fda0 0x55b15c7a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.116+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/731167478 >> 172.21.15.123:0/731167478 conn(0x55b15c78ba10 msgr2=0x55b15c798f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:50.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.116+0000 7fdbb6000640 0 client.25898 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:49:50.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.117+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/731167478 shutdown_connections 2024-09-19T04:49:50.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.117+0000 7fdbd1ab1740 1 -- 172.21.15.123:0/731167478 wait complete. 2024-09-19T04:49:50.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.117+0000 7fdbd1ab1740 20 client.25898 trim_cache size 0 max 16384 2024-09-19T04:49:50.121 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:49:50.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.118+0000 7fdbcf000640 1 -- 172.21.15.123:0/2214553166 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdbac0a3430 msgr2=0x7fdbac0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:50.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.118+0000 7fdbcf000640 1 --2- 172.21.15.123:0/2214553166 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdbac0a3430 0x7fdbac0a58f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8005fd0 tx=0x7fdbb8005850 comp rx=0 tx=0).stop 2024-09-19T04:49:50.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.118+0000 7fdbcf000640 1 -- 172.21.15.123:0/2214553166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbc8109120 msgr2=0x7fdbc806d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:50.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.118+0000 7fdbcf000640 1 --2- 172.21.15.123:0/2214553166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbc8109120 0x7fdbc806d000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009b30 tx=0x7fdbc00305b0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.118+0000 7fdbcf000640 1 -- 172.21.15.123:0/2214553166 shutdown_connections 2024-09-19T04:49:50.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.118+0000 7fdbcf000640 1 --2- 172.21.15.123:0/2214553166 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdbac0a3430 0x7fdbac0a58f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.118+0000 7fdbcf000640 1 --2- 172.21.15.123:0/2214553166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbc8109120 0x7fdbc806d000 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.118+0000 7fdbcf000640 1 --2- 172.21.15.123:0/2214553166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbc81049a0 0x7fdbc806cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.118+0000 7fdbcf000640 1 --2- 172.21.15.123:0/2214553166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbc8103ff0 0x7fdbc80739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:50.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.118+0000 7fdbcf000640 1 -- 172.21.15.123:0/2214553166 >> 172.21.15.123:0/2214553166 conn(0x7fdbc80ffab0 msgr2=0x7fdbc8100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:50.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.119+0000 7fdbcf000640 1 -- 172.21.15.123:0/2214553166 shutdown_connections 2024-09-19T04:49:50.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:50.119+0000 7fdbcf000640 1 -- 172.21.15.123:0/2214553166 wait complete. 2024-09-19T04:49:50.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:49 smithi193 ceph-mon[35592]: pgmap v96: 97 pgs: 97 active+clean; 94 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 734 op/s 2024-09-19T04:49:50.989 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:49:51.438 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:49:51.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:51 smithi123 ceph-mon[30511]: pgmap v97: 97 pgs: 97 active+clean; 94 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 470 op/s 2024-09-19T04:49:52.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:51 smithi183 ceph-mon[35333]: pgmap v97: 97 pgs: 97 active+clean; 94 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 470 op/s 2024-09-19T04:49:52.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:51 smithi193 ceph-mon[35592]: pgmap v97: 97 pgs: 97 active+clean; 94 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 470 op/s 2024-09-19T04:49:52.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.415+0000 7fa5b8a00640 1 -- 172.21.15.123:0/2931660707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b4106f00 msgr2=0x7fa5b410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:52.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.415+0000 7fa5b8a00640 1 --2- 172.21.15.123:0/2931660707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b4106f00 0x7fa5b410ea80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009ca0 tx=0x7fa5ac02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.415+0000 7fa5b8a00640 1 -- 172.21.15.123:0/2931660707 shutdown_connections 2024-09-19T04:49:52.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.415+0000 7fa5b8a00640 1 --2- 172.21.15.123:0/2931660707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b4106f00 0x7fa5b410ea80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.415+0000 7fa5b8a00640 1 --2- 172.21.15.123:0/2931660707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5b4102780 0x7fa5b41067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.415+0000 7fa5b8a00640 1 --2- 172.21.15.123:0/2931660707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5b4101dd0 0x7fa5b41021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.415+0000 7fa5b8a00640 1 -- 172.21.15.123:0/2931660707 >> 172.21.15.123:0/2931660707 conn(0x7fa5b40fd8b0 msgr2=0x7fa5b40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:52.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.415+0000 7fa5b8a00640 1 -- 172.21.15.123:0/2931660707 shutdown_connections 2024-09-19T04:49:52.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.415+0000 7fa5b8a00640 1 -- 172.21.15.123:0/2931660707 wait complete. 2024-09-19T04:49:52.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.416+0000 7fa5b8a00640 1 Processor -- start 2024-09-19T04:49:52.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.416+0000 7fa5b8a00640 1 -- start start 2024-09-19T04:49:52.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.417+0000 7fa5b8a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b4101dd0 0x7fa5b40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.417+0000 7fa5b8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5b4102780 0x7fa5b4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.417+0000 7fa5b8a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5b4106f00 0x7fa5b4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.417+0000 7fa5b8a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b4111fb0 con 0x7fa5b4102780 2024-09-19T04:49:52.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.417+0000 7fa5b8a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b4111e30 con 0x7fa5b4106f00 2024-09-19T04:49:52.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.417+0000 7fa5b8a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b4112130 con 0x7fa5b4101dd0 2024-09-19T04:49:52.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.417+0000 7fa5b1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b4101dd0 0x7fa5b40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:52.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.417+0000 7fa5b1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b4101dd0 0x7fa5b40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49684/0 (socket says 172.21.15.123:49684) 2024-09-19T04:49:52.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.417+0000 7fa5b1e00640 1 -- 172.21.15.123:0/1744697197 learned_addr learned my addr 172.21.15.123:0/1744697197 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:52.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.417+0000 7fa5abe00640 1 --2- 172.21.15.123:0/1744697197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5b4102780 0x7fa5b4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:52.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.417+0000 7fa5b2800640 1 --2- 172.21.15.123:0/1744697197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5b4106f00 0x7fa5b4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:52.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.418+0000 7fa5b1e00640 1 -- 172.21.15.123:0/1744697197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5b4106f00 msgr2=0x7fa5b4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:52.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.418+0000 7fa5b1e00640 1 --2- 172.21.15.123:0/1744697197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5b4106f00 0x7fa5b4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.418+0000 7fa5b1e00640 1 -- 172.21.15.123:0/1744697197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5b4102780 msgr2=0x7fa5b4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.418+0000 7fa5b1e00640 1 --2- 172.21.15.123:0/1744697197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5b4102780 0x7fa5b4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.418+0000 7fa5b1e00640 1 -- 172.21.15.123:0/1744697197 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b41aae00 con 0x7fa5b4101dd0 2024-09-19T04:49:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.418+0000 7fa5b1e00640 1 --2- 172.21.15.123:0/1744697197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b4101dd0 0x7fa5b40744f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400e9f0 tx=0x7fa5a400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.418+0000 7fa5aaa00640 1 -- 172.21.15.123:0/1744697197 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a400cd80 con 0x7fa5b4101dd0 2024-09-19T04:49:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.418+0000 7fa5aaa00640 1 -- 172.21.15.123:0/1744697197 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a4010910 con 0x7fa5b4101dd0 2024-09-19T04:49:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.418+0000 7fa5b8a00640 1 -- 172.21.15.123:0/1744697197 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b41aafd0 con 0x7fa5b4101dd0 2024-09-19T04:49:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.418+0000 7fa5aaa00640 1 -- 172.21.15.123:0/1744697197 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4018990 con 0x7fa5b4101dd0 2024-09-19T04:49:52.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.418+0000 7fa5b8a00640 1 -- 172.21.15.123:0/1744697197 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b41ab500 con 0x7fa5b4101dd0 2024-09-19T04:49:52.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.423+0000 7fa5bb5d2740 1 Processor -- start 2024-09-19T04:49:52.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.423+0000 7fa5bb5d2740 1 -- start start 2024-09-19T04:49:52.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.423+0000 7fa5aaa00640 1 -- 172.21.15.123:0/1744697197 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa5a4018b50 con 0x7fa5b4101dd0 2024-09-19T04:49:52.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.423+0000 7fa5bb5d2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684ec70 0x56255684f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.423+0000 7fa5bb5d2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56255684f590 0x562556859910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.423+0000 7fa5bb5d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562556859e50 0x56255685c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.423+0000 7fa5bb5d2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5625567b53e0 con 0x56255684ec70 2024-09-19T04:49:52.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.423+0000 7fa5bb5d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56255675cd80 con 0x562556859e50 2024-09-19T04:49:52.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.423+0000 7fa5bb5d2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5625567b13a0 con 0x56255684f590 2024-09-19T04:49:52.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.424+0000 7fa5b2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684ec70 0x56255684f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:52.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.424+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562556859e50 0x56255685c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:52.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.424+0000 7fa5b1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56255684f590 0x562556859910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:52.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.424+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562556859e50 0x56255685c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:42132/0 (socket says 172.21.15.123:42132) 2024-09-19T04:49:52.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.424+0000 7fa5b2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684ec70 0x56255684f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42192/0 (socket says 172.21.15.123:42192) 2024-09-19T04:49:52.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.424+0000 7fa5abe00640 1 -- 172.21.15.123:0/2767262240 learned_addr learned my addr 172.21.15.123:0/2767262240 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:52.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.424+0000 7fa5abe00640 1 -- 172.21.15.123:0/2767262240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56255684f590 msgr2=0x562556859910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:52.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.424+0000 7fa5abe00640 1 --2- 172.21.15.123:0/2767262240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56255684f590 0x562556859910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.424+0000 7fa5abe00640 1 -- 172.21.15.123:0/2767262240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684ec70 msgr2=0x56255684f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:52.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.424+0000 7fa5abe00640 1 --2- 172.21.15.123:0/2767262240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684ec70 0x56255684f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.424+0000 7fa5abe00640 1 -- 172.21.15.123:0/2767262240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56255677bae0 con 0x562556859e50 2024-09-19T04:49:52.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.424+0000 7fa5b2800640 1 --2- 172.21.15.123:0/2767262240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684ec70 0x56255684f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:49:52.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.425+0000 7fa5abe00640 1 --2- 172.21.15.123:0/2767262240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562556859e50 0x56255685c240 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009ba0 tx=0x7fa59c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7203e726f496da1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:52.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.425+0000 7fa5aaa00640 1 --2- 172.21.15.123:0/1744697197 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa5980a3430 0x7fa5980a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.425+0000 7fa5a3000640 1 -- 172.21.15.123:0/2767262240 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c004230 con 0x562556859e50 2024-09-19T04:49:52.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.425+0000 7fa5a3000640 1 -- 172.21.15.123:0/2767262240 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa59c009e10 con 0x562556859e50 2024-09-19T04:49:52.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.425+0000 7fa5a3000640 1 -- 172.21.15.123:0/2767262240 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c004de0 con 0x562556859e50 2024-09-19T04:49:52.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.425+0000 7fa5aaa00640 1 -- 172.21.15.123:0/1744697197 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fa5a40cff90 con 0x7fa5b4101dd0 2024-09-19T04:49:52.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.425+0000 7fa5b2800640 1 --2- 172.21.15.123:0/1744697197 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa5980a3430 0x7fa5980a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:52.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.426+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/2767262240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562556859e50 msgr2=0x56255685c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:52.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.426+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/2767262240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562556859e50 0x56255685c240 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009ba0 tx=0x7fa59c011810 comp rx=0 tx=0).stop 2024-09-19T04:49:52.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.426+0000 7fa5b2800640 1 --2- 172.21.15.123:0/1744697197 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa5980a3430 0x7fa5980a58f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac006010 tx=0x7fa5ac005d80 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:52.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.426+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/2767262240 shutdown_connections 2024-09-19T04:49:52.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.426+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/2767262240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562556859e50 0x56255685c240 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.426+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/2767262240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56255684f590 0x562556859910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.426+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/2767262240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684ec70 0x56255684f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.426+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/2767262240 >> 172.21.15.123:0/2767262240 conn(0x56255684b200 msgr2=0x56255684b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:52.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.428+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/2767262240 shutdown_connections 2024-09-19T04:49:52.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.428+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/2767262240 wait complete. 2024-09-19T04:49:52.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.428+0000 7fa5bb5d2740 1 -- start start 2024-09-19T04:49:52.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.428+0000 7fa5bb5d2740 1 Processor -- start 2024-09-19T04:49:52.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.429+0000 7fa5bb5d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56255684ec70 0x562556856cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.429+0000 7fa5bb5d2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56255684f590 0x562556857210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.429+0000 7fa5bb5d2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562556859e50 0x562556857750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.429+0000 7fa5bb5d2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56255685ca20 con 0x562556859e50 2024-09-19T04:49:52.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.429+0000 7fa5bb5d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5625567700f0 con 0x56255684ec70 2024-09-19T04:49:52.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.429+0000 7fa5bb5d2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56255685cba0 con 0x56255684f590 2024-09-19T04:49:52.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.429+0000 7fa5bb5d2740 10 client.0 _ll_register_callbacks cb 0x562556757bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:52.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.429+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56255684f590 0x562556857210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:52.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.429+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56255684f590 0x562556857210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49708/0 (socket says 172.21.15.123:49708) 2024-09-19T04:49:52.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.429+0000 7fa5abe00640 1 -- 172.21.15.123:0/1926928801 learned_addr learned my addr 172.21.15.123:0/1926928801 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:52.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.429+0000 7fa5b1e00640 1 --2- 172.21.15.123:0/1926928801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56255684ec70 0x562556856cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:52.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.430+0000 7fa5abe00640 1 -- 172.21.15.123:0/1926928801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56255684ec70 msgr2=0x562556856cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:52.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.430+0000 7fa5abe00640 1 --2- 172.21.15.123:0/1926928801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56255684ec70 0x562556856cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.430+0000 7fa5abe00640 1 -- 172.21.15.123:0/1926928801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562556859e50 msgr2=0x562556857750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:52.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.430+0000 7fa5abe00640 1 --2- 172.21.15.123:0/1926928801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562556859e50 0x562556857750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.430+0000 7fa5abe00640 1 -- 172.21.15.123:0/1926928801 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625565263c0 con 0x56255684f590 2024-09-19T04:49:52.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.430+0000 7fa5abe00640 1 --2- 172.21.15.123:0/1926928801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56255684f590 0x562556857210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009640 tx=0x7fa59c009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b4191bd6e8a8413 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:52.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.430+0000 7fa597400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:49:52.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.430+0000 7fa597400640 1 -- 172.21.15.123:0/1926928801 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c027020 con 0x56255684f590 2024-09-19T04:49:52.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.430+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/1926928801 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa59c009e10 con 0x56255684f590 2024-09-19T04:49:52.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.430+0000 7fa597400640 1 -- 172.21.15.123:0/1926928801 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fa59c0037f0 con 0x56255684f590 2024-09-19T04:49:52.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.431+0000 7fa597400640 1 -- 172.21.15.123:0/1926928801 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa59c032050 con 0x56255684f590 2024-09-19T04:49:52.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.431+0000 7fa5bb5d2740 10 client.25927 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:52.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.431+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/1926928801 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562556868a50 con 0x56255684f590 2024-09-19T04:49:52.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.431+0000 7fa597400640 1 -- 172.21.15.123:0/1926928801 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa59c009e10 con 0x56255684f590 2024-09-19T04:49:52.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.431+0000 7fa597400640 1 -- 172.21.15.123:0/1926928801 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fa59c018070 con 0x56255684f590 2024-09-19T04:49:52.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.431+0000 7fa5bb5d2740 10 client.25927 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:52.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.431+0000 7fa5bb5d2740 10 client.25927 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:49:52.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.431+0000 7fa5bb5d2740 20 client.25927 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.431+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/1926928801 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x562556869e60 0x56255686c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.431+0000 7fa5bb5d2740 4 client.25927 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:49:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.431+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/1926928801 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625567c2950 con 0x562556869e60 2024-09-19T04:49:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.432+0000 7fa5b1e00640 1 --2- 172.21.15.123:0/1926928801 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x562556869e60 0x56255686c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.432+0000 7fa597400640 10 client.25927 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:49:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.432+0000 7fa5b1e00640 1 --2- 172.21.15.123:0/1926928801 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x562556869e60 0x56255686c250 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e07166b6685a8af7 server_cookie=87c6f5728f1ee16a in_seq=0 out_seq=0 2024-09-19T04:49:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.433+0000 7fa597400640 1 -- 172.21.15.123:0/1926928801 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625567c2950 con 0x562556869e60 2024-09-19T04:49:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.433+0000 7fa597400640 10 client.25927 handle_command_reply: tid=0 2024-09-19T04:49:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.433+0000 7fa5bb5d2740 1 client.25927 shutdown 2024-09-19T04:49:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.433+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/1926928801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56255684f590 msgr2=0x562556857210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.433+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/1926928801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56255684f590 0x562556857210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009640 tx=0x7fa59c009670 comp rx=0 tx=0).stop 2024-09-19T04:49:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.433+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/1926928801 shutdown_connections 2024-09-19T04:49:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.433+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/1926928801 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x562556869e60 0x56255686c250 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.434+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/1926928801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562556859e50 0x562556857750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.434+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/1926928801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56255684f590 0x562556857210 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.434+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/1926928801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56255684ec70 0x562556856cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.434+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/1926928801 >> 172.21.15.123:0/1926928801 conn(0x56255684b200 msgr2=0x562556850f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.434+0000 7fa597400640 0 client.25927 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:49:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.434+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/1926928801 shutdown_connections 2024-09-19T04:49:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.434+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/1926928801 wait complete. 2024-09-19T04:49:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.434+0000 7fa5bb5d2740 20 client.25927 trim_cache size 0 max 16384 2024-09-19T04:49:52.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.453+0000 7fa5bb5d2740 1 Processor -- start 2024-09-19T04:49:52.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.453+0000 7fa5bb5d2740 1 -- start start 2024-09-19T04:49:52.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.454+0000 7fa5bb5d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56255684ec70 0x562556858c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.454+0000 7fa5bb5d2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684f590 0x562556871a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.454+0000 7fa5bb5d2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562556859e50 0x562556871f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.454+0000 7fa5bb5d2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b41a4c50 con 0x56255684f590 2024-09-19T04:49:52.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.454+0000 7fa5bb5d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b41cf340 con 0x56255684ec70 2024-09-19T04:49:52.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.454+0000 7fa5bb5d2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b41cdd80 con 0x562556859e50 2024-09-19T04:49:52.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.454+0000 7fa5b1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56255684ec70 0x562556858c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:52.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.454+0000 7fa5b1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56255684ec70 0x562556858c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:42150/0 (socket says 172.21.15.123:42150) 2024-09-19T04:49:52.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.454+0000 7fa5b1e00640 1 -- 172.21.15.123:0/509505128 learned_addr learned my addr 172.21.15.123:0/509505128 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:52.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.454+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684f590 0x562556871a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:52.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.454+0000 7fa5b2800640 1 --2- 172.21.15.123:0/509505128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562556859e50 0x562556871f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:52.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.455+0000 7fa5b1e00640 1 -- 172.21.15.123:0/509505128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562556859e50 msgr2=0x562556871f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:52.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.455+0000 7fa5b1e00640 1 --2- 172.21.15.123:0/509505128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562556859e50 0x562556871f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.455+0000 7fa5b1e00640 1 -- 172.21.15.123:0/509505128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684f590 msgr2=0x562556871a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:52.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.455+0000 7fa5b1e00640 1 --2- 172.21.15.123:0/509505128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684f590 0x562556871a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.455+0000 7fa5b1e00640 1 -- 172.21.15.123:0/509505128 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56255684b030 con 0x56255684ec70 2024-09-19T04:49:52.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.455+0000 7fa5b1e00640 1 --2- 172.21.15.123:0/509505128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56255684ec70 0x562556858c40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400e9c0 tx=0x7fa5a40d24a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d21ddab903914c7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:52.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.455+0000 7fa5a2600640 1 -- 172.21.15.123:0/509505128 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40cde60 con 0x56255684ec70 2024-09-19T04:49:52.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.455+0000 7fa5a2600640 1 -- 172.21.15.123:0/509505128 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a409c350 con 0x56255684ec70 2024-09-19T04:49:52.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.455+0000 7fa5a2600640 1 -- 172.21.15.123:0/509505128 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4010ac0 con 0x56255684ec70 2024-09-19T04:49:52.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.456+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/509505128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56255684ec70 msgr2=0x562556858c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:52.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.456+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/509505128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56255684ec70 0x562556858c40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400e9c0 tx=0x7fa5a40d24a0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.456+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/509505128 shutdown_connections 2024-09-19T04:49:52.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.456+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/509505128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562556859e50 0x562556871f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.456+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/509505128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684f590 0x562556871a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.456+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/509505128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56255684ec70 0x562556858c40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.456+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/509505128 >> 172.21.15.123:0/509505128 conn(0x56255684b200 msgr2=0x562556858530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:52.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.456+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/509505128 shutdown_connections 2024-09-19T04:49:52.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.456+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/509505128 wait complete. 2024-09-19T04:49:52.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.457+0000 7fa5bb5d2740 1 -- start start 2024-09-19T04:49:52.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.457+0000 7fa5bb5d2740 1 Processor -- start 2024-09-19T04:49:52.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.458+0000 7fa5bb5d2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684ec70 0x562556853e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.458+0000 7fa5bb5d2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56255684f590 0x5625568543b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.458+0000 7fa5bb5d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562556859e50 0x5625568548f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.458+0000 7fa5bb5d2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b41cebb0 con 0x56255684ec70 2024-09-19T04:49:52.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.458+0000 7fa5bb5d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b41ce4e0 con 0x562556859e50 2024-09-19T04:49:52.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.458+0000 7fa5bb5d2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b41a2c30 con 0x56255684f590 2024-09-19T04:49:52.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.458+0000 7fa5bb5d2740 10 client.0 _ll_register_callbacks cb 0x7fa59c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:52.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.458+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56255684f590 0x5625568543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:52.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.458+0000 7fa5b1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684ec70 0x562556853e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:52.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.458+0000 7fa5b1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684ec70 0x562556853e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42220/0 (socket says 172.21.15.123:42220) 2024-09-19T04:49:52.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.458+0000 7fa5b1e00640 1 -- 172.21.15.123:0/1817648263 learned_addr learned my addr 172.21.15.123:0/1817648263 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:52.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.458+0000 7fa5b2800640 1 --2- 172.21.15.123:0/1817648263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562556859e50 0x5625568548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:52.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.459+0000 7fa5b1e00640 1 -- 172.21.15.123:0/1817648263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56255684f590 msgr2=0x5625568543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:52.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.459+0000 7fa5b1e00640 1 --2- 172.21.15.123:0/1817648263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56255684f590 0x5625568543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.459+0000 7fa5b1e00640 1 -- 172.21.15.123:0/1817648263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562556859e50 msgr2=0x5625568548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:52.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.459+0000 7fa5b1e00640 1 --2- 172.21.15.123:0/1817648263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562556859e50 0x5625568548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.459+0000 7fa5b1e00640 1 -- 172.21.15.123:0/1817648263 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562556851ae0 con 0x56255684ec70 2024-09-19T04:49:52.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.459+0000 7fa5b1e00640 1 --2- 172.21.15.123:0/1817648263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684ec70 0x562556853e70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40987b0 tx=0x7fa5a409c1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0e78d010031128 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:52.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.459+0000 7fa597400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:49:52.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.459+0000 7fa597400640 1 -- 172.21.15.123:0/1817648263 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40cde60 con 0x56255684ec70 2024-09-19T04:49:52.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.459+0000 7fa597400640 1 -- 172.21.15.123:0/1817648263 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fa5a40989c0 con 0x56255684ec70 2024-09-19T04:49:52.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.460+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/1817648263 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a409c350 con 0x56255684ec70 2024-09-19T04:49:52.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.460+0000 7fa597400640 1 -- 172.21.15.123:0/1817648263 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa5a409bcb0 con 0x56255684ec70 2024-09-19T04:49:52.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.460+0000 7fa5bb5d2740 10 client.15852 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:52.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.460+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/1817648263 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56255684be90 con 0x56255684ec70 2024-09-19T04:49:52.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.460+0000 7fa597400640 1 -- 172.21.15.123:0/1817648263 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a409c350 con 0x56255684ec70 2024-09-19T04:49:52.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.460+0000 7fa597400640 1 -- 172.21.15.123:0/1817648263 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fa5a40d20b0 con 0x56255684ec70 2024-09-19T04:49:52.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.460+0000 7fa5bb5d2740 10 client.15852 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:52.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.460+0000 7fa5bb5d2740 10 client.15852 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:49:52.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.460+0000 7fa5bb5d2740 20 client.15852 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:52.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.460+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/1817648263 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5625568d5110 0x5625568d7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:52.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.460+0000 7fa5bb5d2740 4 client.15852 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:49:52.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.460+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/1817648263 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625567bce60 con 0x5625568d5110 2024-09-19T04:49:52.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.461+0000 7fa5abe00640 1 --2- 172.21.15.123:0/1817648263 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5625568d5110 0x5625568d7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:52.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.462+0000 7fa5abe00640 1 --2- 172.21.15.123:0/1817648263 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5625568d5110 0x5625568d7530 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9099a4baebb7aeec server_cookie=228366eab7a665fc in_seq=0 out_seq=0 2024-09-19T04:49:52.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.462+0000 7fa597400640 10 client.15852 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:49:52.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.462+0000 7fa597400640 1 -- 172.21.15.123:0/1817648263 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625567bce60 con 0x5625568d5110 2024-09-19T04:49:52.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.462+0000 7fa597400640 10 client.15852 handle_command_reply: tid=0 2024-09-19T04:49:52.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.462+0000 7fa5bb5d2740 1 client.15852 shutdown 2024-09-19T04:49:52.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.462+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/1817648263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684ec70 msgr2=0x562556853e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:52.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.462+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/1817648263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684ec70 0x562556853e70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40987b0 tx=0x7fa5a409c1e0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.462+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/1817648263 shutdown_connections 2024-09-19T04:49:52.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.462+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/1817648263 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5625568d5110 0x5625568d7530 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.462+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/1817648263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562556859e50 0x5625568548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.462+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/1817648263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56255684f590 0x5625568543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.462+0000 7fa5bb5d2740 1 --2- 172.21.15.123:0/1817648263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255684ec70 0x562556853e70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.462+0000 7fa597400640 0 client.15852 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:49:52.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.462+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/1817648263 >> 172.21.15.123:0/1817648263 conn(0x56255684b200 msgr2=0x562556850ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:52.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.462+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/1817648263 shutdown_connections 2024-09-19T04:49:52.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.463+0000 7fa5bb5d2740 1 -- 172.21.15.123:0/1817648263 wait complete. 2024-09-19T04:49:52.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.463+0000 7fa5bb5d2740 20 client.15852 trim_cache size 0 max 16384 2024-09-19T04:49:52.467 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:49:52.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.463+0000 7fa5b8a00640 1 -- 172.21.15.123:0/1744697197 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa5980a3430 msgr2=0x7fa5980a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:52.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.463+0000 7fa5b8a00640 1 --2- 172.21.15.123:0/1744697197 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa5980a3430 0x7fa5980a58f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac006010 tx=0x7fa5ac005d80 comp rx=0 tx=0).stop 2024-09-19T04:49:52.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.463+0000 7fa5b8a00640 1 -- 172.21.15.123:0/1744697197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b4101dd0 msgr2=0x7fa5b40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:52.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.463+0000 7fa5b8a00640 1 --2- 172.21.15.123:0/1744697197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b4101dd0 0x7fa5b40744f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400e9f0 tx=0x7fa5a400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.464+0000 7fa5b8a00640 1 -- 172.21.15.123:0/1744697197 shutdown_connections 2024-09-19T04:49:52.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.464+0000 7fa5b8a00640 1 --2- 172.21.15.123:0/1744697197 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa5980a3430 0x7fa5980a58f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.464+0000 7fa5b8a00640 1 --2- 172.21.15.123:0/1744697197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5b4106f00 0x7fa5b4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.464+0000 7fa5b8a00640 1 --2- 172.21.15.123:0/1744697197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5b4102780 0x7fa5b4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.464+0000 7fa5b8a00640 1 --2- 172.21.15.123:0/1744697197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b4101dd0 0x7fa5b40744f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:52.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.464+0000 7fa5b8a00640 1 -- 172.21.15.123:0/1744697197 >> 172.21.15.123:0/1744697197 conn(0x7fa5b40fd8b0 msgr2=0x7fa5b40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:52.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.464+0000 7fa5b8a00640 1 -- 172.21.15.123:0/1744697197 shutdown_connections 2024-09-19T04:49:52.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:52.466+0000 7fa5b8a00640 1 -- 172.21.15.123:0/1744697197 wait complete. 2024-09-19T04:49:53.395 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:49:53.814 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:49:54.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:53 smithi183 ceph-mon[35333]: pgmap v98: 97 pgs: 97 active+clean; 94 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 470 op/s 2024-09-19T04:49:54.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:53 smithi123 ceph-mon[30511]: pgmap v98: 97 pgs: 97 active+clean; 94 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 470 op/s 2024-09-19T04:49:54.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:53 smithi193 ceph-mon[35592]: pgmap v98: 97 pgs: 97 active+clean; 94 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 470 op/s 2024-09-19T04:49:54.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.671+0000 7fbc9be00640 1 -- 172.21.15.123:0/704514481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc9c102360 msgr2=0x7fbc9c10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:54.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.671+0000 7fbc9be00640 1 --2- 172.21.15.123:0/704514481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc9c102360 0x7fbc9c10e890 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009be0 tx=0x7fbc9402a6c0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.671+0000 7fbc9be00640 1 -- 172.21.15.123:0/704514481 shutdown_connections 2024-09-19T04:49:54.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.671+0000 7fbc9be00640 1 --2- 172.21.15.123:0/704514481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc9c102360 0x7fbc9c10e890 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.671+0000 7fbc9be00640 1 --2- 172.21.15.123:0/704514481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9c1019c0 0x7fbc9c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.671+0000 7fbc9be00640 1 --2- 172.21.15.123:0/704514481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc9c1079c0 0x7fbc9c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.671+0000 7fbc9be00640 1 -- 172.21.15.123:0/704514481 >> 172.21.15.123:0/704514481 conn(0x7fbc9c0fd740 msgr2=0x7fbc9c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:54.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.671+0000 7fbc9be00640 1 -- 172.21.15.123:0/704514481 shutdown_connections 2024-09-19T04:49:54.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.671+0000 7fbc9be00640 1 -- 172.21.15.123:0/704514481 wait complete. 2024-09-19T04:49:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.672+0000 7fbc9be00640 1 Processor -- start 2024-09-19T04:49:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.672+0000 7fbc9be00640 1 -- start start 2024-09-19T04:49:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc9c1019c0 0x7fbc9c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9c102360 0x7fbc9c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc9c1079c0 0x7fbc9c19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c111dc0 con 0x7fbc9c102360 2024-09-19T04:49:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c111c40 con 0x7fbc9c1019c0 2024-09-19T04:49:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c111f40 con 0x7fbc9c1079c0 2024-09-19T04:49:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9c102360 0x7fbc9c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc9c1079c0 0x7fbc9c19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9c102360 0x7fbc9c198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42234/0 (socket says 172.21.15.123:42234) 2024-09-19T04:49:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc9c1019c0 0x7fbc9c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9a000640 1 -- 172.21.15.123:0/3402048811 learned_addr learned my addr 172.21.15.123:0/3402048811 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9b400640 1 -- 172.21.15.123:0/3402048811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc9c1019c0 msgr2=0x7fbc9c198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9b400640 1 --2- 172.21.15.123:0/3402048811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc9c1019c0 0x7fbc9c198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9b400640 1 -- 172.21.15.123:0/3402048811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9c102360 msgr2=0x7fbc9c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9b400640 1 --2- 172.21.15.123:0/3402048811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9c102360 0x7fbc9c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9b400640 1 -- 172.21.15.123:0/3402048811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9c19d1d0 con 0x7fbc9c1079c0 2024-09-19T04:49:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9a000640 1 --2- 172.21.15.123:0/3402048811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9c102360 0x7fbc9c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:49:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.673+0000 7fbc9aa00640 1 --2- 172.21.15.123:0/3402048811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc9c1019c0 0x7fbc9c198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:49:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.674+0000 7fbc9b400640 1 --2- 172.21.15.123:0/3402048811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc9c1079c0 0x7fbc9c19caf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009b30 tx=0x7fbc94030c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.674+0000 7fbc93400640 1 -- 172.21.15.123:0/3402048811 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc94041020 con 0x7fbc9c1079c0 2024-09-19T04:49:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.674+0000 7fbc9be00640 1 -- 172.21.15.123:0/3402048811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9c19d460 con 0x7fbc9c1079c0 2024-09-19T04:49:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.674+0000 7fbc93400640 1 -- 172.21.15.123:0/3402048811 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc94003e50 con 0x7fbc9c1079c0 2024-09-19T04:49:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.674+0000 7fbc93400640 1 -- 172.21.15.123:0/3402048811 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc94036040 con 0x7fbc9c1079c0 2024-09-19T04:49:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.674+0000 7fbc9be00640 1 -- 172.21.15.123:0/3402048811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc9c1a4da0 con 0x7fbc9c1079c0 2024-09-19T04:49:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.678+0000 7fbca4826740 1 Processor -- start 2024-09-19T04:49:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.679+0000 7fbc93400640 1 -- 172.21.15.123:0/3402048811 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fbc94044020 con 0x7fbc9c1079c0 2024-09-19T04:49:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.679+0000 7fbca4826740 1 -- start start 2024-09-19T04:49:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.679+0000 7fbca4826740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b6d30 0x55d6570b7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.679+0000 7fbca4826740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b7650 0x55d6570c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.679+0000 7fbca4826740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570c6480 0x55d6570c8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.679+0000 7fbca4826740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d65701d3e0 con 0x55d6570b6d30 2024-09-19T04:49:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.679+0000 7fbca4826740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d656fc4d80 con 0x55d6570b7650 2024-09-19T04:49:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.679+0000 7fbca4826740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d6570193a0 con 0x55d6570c6480 2024-09-19T04:49:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.680+0000 7fbc9a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b6d30 0x55d6570b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.680+0000 7fbc9aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b7650 0x55d6570c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.680+0000 7fbc9aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b7650 0x55d6570c5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:42198/0 (socket says 172.21.15.123:42198) 2024-09-19T04:49:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.680+0000 7fbc9aa00640 1 -- 172.21.15.123:0/3291157211 learned_addr learned my addr 172.21.15.123:0/3291157211 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.680+0000 7fbc9a000640 1 --2- 172.21.15.123:0/3291157211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570c6480 0x55d6570c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.680+0000 7fbc9aa00640 1 -- 172.21.15.123:0/3291157211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570c6480 msgr2=0x55d6570c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.680+0000 7fbc9aa00640 1 --2- 172.21.15.123:0/3291157211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570c6480 0x55d6570c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.680+0000 7fbc9aa00640 1 -- 172.21.15.123:0/3291157211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b6d30 msgr2=0x55d6570b7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.680+0000 7fbc9aa00640 1 --2- 172.21.15.123:0/3291157211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b6d30 0x55d6570b7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.680+0000 7fbc9aa00640 1 -- 172.21.15.123:0/3291157211 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d656fe3ae0 con 0x55d6570b7650 2024-09-19T04:49:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.680+0000 7fbc93400640 1 --2- 172.21.15.123:0/3402048811 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbc800a33b0 0x7fbc800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.680+0000 7fbc9aa00640 1 --2- 172.21.15.123:0/3291157211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b7650 0x55d6570c5f40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0114f0 tx=0x7fbc8c0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b7267affa2166ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.681+0000 7fbc93400640 1 -- 172.21.15.123:0/3402048811 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fbc940f0740 con 0x7fbc9c1079c0 2024-09-19T04:49:54.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.681+0000 7fbc87e00640 1 -- 172.21.15.123:0/3291157211 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c01e070 con 0x55d6570b7650 2024-09-19T04:49:54.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.681+0000 7fbc87e00640 1 -- 172.21.15.123:0/3291157211 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc8c00faa0 con 0x55d6570b7650 2024-09-19T04:49:54.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.681+0000 7fbc87e00640 1 -- 172.21.15.123:0/3291157211 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c019b90 con 0x55d6570b7650 2024-09-19T04:49:54.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.681+0000 7fbc9a000640 1 --2- 172.21.15.123:0/3402048811 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbc800a33b0 0x7fbc800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:54.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.682+0000 7fbca4826740 1 -- 172.21.15.123:0/3291157211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b7650 msgr2=0x55d6570c5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:54.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.682+0000 7fbc9a000640 1 --2- 172.21.15.123:0/3402048811 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbc800a33b0 0x7fbc800a5870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc880094b0 tx=0x7fbc88009680 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:54.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.682+0000 7fbca4826740 1 --2- 172.21.15.123:0/3291157211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b7650 0x55d6570c5f40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0114f0 tx=0x7fbc8c0119b0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.682+0000 7fbca4826740 1 -- 172.21.15.123:0/3291157211 shutdown_connections 2024-09-19T04:49:54.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.682+0000 7fbca4826740 1 --2- 172.21.15.123:0/3291157211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570c6480 0x55d6570c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.682+0000 7fbca4826740 1 --2- 172.21.15.123:0/3291157211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b7650 0x55d6570c5f40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.682+0000 7fbca4826740 1 --2- 172.21.15.123:0/3291157211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b6d30 0x55d6570b7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.682+0000 7fbca4826740 1 -- 172.21.15.123:0/3291157211 >> 172.21.15.123:0/3291157211 conn(0x55d6570b3420 msgr2=0x55d6570b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:54.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.682+0000 7fbca4826740 1 -- 172.21.15.123:0/3291157211 shutdown_connections 2024-09-19T04:49:54.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.682+0000 7fbca4826740 1 -- 172.21.15.123:0/3291157211 wait complete. 2024-09-19T04:49:54.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.682+0000 7fbca4826740 1 -- start start 2024-09-19T04:49:54.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.683+0000 7fbca4826740 1 Processor -- start 2024-09-19T04:49:54.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.683+0000 7fbca4826740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b6d30 0x55d6570c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.683+0000 7fbca4826740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b7650 0x55d6570c3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.683+0000 7fbca4826740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570c6480 0x55d6570c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.683+0000 7fbca4826740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d6570c9050 con 0x55d6570c6480 2024-09-19T04:49:54.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.683+0000 7fbca4826740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d656fd80f0 con 0x55d6570b7650 2024-09-19T04:49:54.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.683+0000 7fbca4826740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d6570c91d0 con 0x55d6570b6d30 2024-09-19T04:49:54.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.683+0000 7fbca4826740 10 client.0 _ll_register_callbacks cb 0x55d656fbfbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:54.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.683+0000 7fbc9aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b6d30 0x55d6570c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:54.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.683+0000 7fbc9a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b7650 0x55d6570c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:54.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.683+0000 7fbc9aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b6d30 0x55d6570c33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49760/0 (socket says 172.21.15.123:49760) 2024-09-19T04:49:54.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.683+0000 7fbc9aa00640 1 -- 172.21.15.123:0/3698990761 learned_addr learned my addr 172.21.15.123:0/3698990761 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:54.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.683+0000 7fbc9b400640 1 --2- 172.21.15.123:0/3698990761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570c6480 0x55d6570c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:54.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.684+0000 7fbc9aa00640 1 -- 172.21.15.123:0/3698990761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b7650 msgr2=0x55d6570c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:54.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.684+0000 7fbc9aa00640 1 --2- 172.21.15.123:0/3698990761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b7650 0x55d6570c3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.684+0000 7fbc9aa00640 1 -- 172.21.15.123:0/3698990761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570c6480 msgr2=0x55d6570c3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:54.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.684+0000 7fbc9aa00640 1 --2- 172.21.15.123:0/3698990761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570c6480 0x55d6570c3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.684+0000 7fbc9aa00640 1 -- 172.21.15.123:0/3698990761 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d656d8e3c0 con 0x55d6570b6d30 2024-09-19T04:49:54.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.684+0000 7fbc9a000640 1 --2- 172.21.15.123:0/3698990761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b7650 0x55d6570c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:49:54.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.684+0000 7fbc9aa00640 1 --2- 172.21.15.123:0/3698990761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b6d30 0x55d6570c33f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c01c040 tx=0x7fbc8c005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1c723764add531 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:54.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.684+0000 7fbc84c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:49:54.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.684+0000 7fbc84c00640 1 -- 172.21.15.123:0/3698990761 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c01e040 con 0x55d6570b6d30 2024-09-19T04:49:54.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.684+0000 7fbc84c00640 1 -- 172.21.15.123:0/3698990761 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fbc8c00f5c0 con 0x55d6570b6d30 2024-09-19T04:49:54.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.684+0000 7fbca4826740 1 -- 172.21.15.123:0/3698990761 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc8c00faa0 con 0x55d6570b6d30 2024-09-19T04:49:54.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.685+0000 7fbc84c00640 1 -- 172.21.15.123:0/3698990761 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbc8c00f7c0 con 0x55d6570b6d30 2024-09-19T04:49:54.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.685+0000 7fbca4826740 10 client.25945 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:54.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.685+0000 7fbca4826740 1 -- 172.21.15.123:0/3698990761 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d6570d5150 con 0x55d6570b6d30 2024-09-19T04:49:54.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.685+0000 7fbc84c00640 1 -- 172.21.15.123:0/3698990761 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc8c00faa0 con 0x55d6570b6d30 2024-09-19T04:49:54.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.685+0000 7fbc84c00640 1 -- 172.21.15.123:0/3698990761 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fbc8c021030 con 0x55d6570b6d30 2024-09-19T04:49:54.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.685+0000 7fbca4826740 10 client.25945 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:54.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.685+0000 7fbca4826740 10 client.25945 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:49:54.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.685+0000 7fbca4826740 20 client.25945 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:54.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.685+0000 7fbca4826740 1 --2- 172.21.15.123:0/3698990761 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55d6570d65b0 0x55d6570d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.685+0000 7fbca4826740 4 client.25945 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:49:54.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.685+0000 7fbca4826740 1 -- 172.21.15.123:0/3698990761 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d65702a950 con 0x55d6570d65b0 2024-09-19T04:49:54.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.686+0000 7fbc9a000640 1 --2- 172.21.15.123:0/3698990761 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55d6570d65b0 0x55d6570d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:54.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.686+0000 7fbc9a000640 1 --2- 172.21.15.123:0/3698990761 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55d6570d65b0 0x55d6570d89a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b28ab62d9754b5fb server_cookie=a3355415f77444f6 in_seq=0 out_seq=0 2024-09-19T04:49:54.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.686+0000 7fbc84c00640 10 client.25945 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:49:54.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.687+0000 7fbc84c00640 1 -- 172.21.15.123:0/3698990761 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d65702a950 con 0x55d6570d65b0 2024-09-19T04:49:54.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.687+0000 7fbc84c00640 10 client.25945 handle_command_reply: tid=0 2024-09-19T04:49:54.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.687+0000 7fbca4826740 1 client.25945 shutdown 2024-09-19T04:49:54.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.687+0000 7fbca4826740 1 -- 172.21.15.123:0/3698990761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b6d30 msgr2=0x55d6570c33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:54.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.687+0000 7fbca4826740 1 --2- 172.21.15.123:0/3698990761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b6d30 0x55d6570c33f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c01c040 tx=0x7fbc8c005780 comp rx=0 tx=0).stop 2024-09-19T04:49:54.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.687+0000 7fbca4826740 1 -- 172.21.15.123:0/3698990761 shutdown_connections 2024-09-19T04:49:54.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.687+0000 7fbca4826740 1 --2- 172.21.15.123:0/3698990761 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55d6570d65b0 0x55d6570d89a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.687+0000 7fbca4826740 1 --2- 172.21.15.123:0/3698990761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570c6480 0x55d6570c3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.687+0000 7fbca4826740 1 --2- 172.21.15.123:0/3698990761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b7650 0x55d6570c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.687+0000 7fbca4826740 1 --2- 172.21.15.123:0/3698990761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b6d30 0x55d6570c33f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.687+0000 7fbca4826740 1 -- 172.21.15.123:0/3698990761 >> 172.21.15.123:0/3698990761 conn(0x55d6570b3420 msgr2=0x55d6570b9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:54.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.687+0000 7fbc84c00640 0 client.25945 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:49:54.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.687+0000 7fbca4826740 1 -- 172.21.15.123:0/3698990761 shutdown_connections 2024-09-19T04:49:54.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.687+0000 7fbca4826740 1 -- 172.21.15.123:0/3698990761 wait complete. 2024-09-19T04:49:54.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.688+0000 7fbca4826740 20 client.25945 trim_cache size 0 max 16384 2024-09-19T04:49:54.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.702+0000 7fbca4826740 1 Processor -- start 2024-09-19T04:49:54.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.702+0000 7fbca4826740 1 -- start start 2024-09-19T04:49:54.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbca4826740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b3420 0x55d6570c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbca4826740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b6d30 0x55d6570de290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbca4826740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b7650 0x55d6570de7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbca4826740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1ab410 con 0x55d6570b7650 2024-09-19T04:49:54.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbca4826740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1ad0a0 con 0x55d6570b3420 2024-09-19T04:49:54.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbca4826740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1abae0 con 0x55d6570b6d30 2024-09-19T04:49:54.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbc9b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b7650 0x55d6570de7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:54.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbc9a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b6d30 0x55d6570de290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:54.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbc9aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b3420 0x55d6570c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:54.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbc9aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b3420 0x55d6570c1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:42224/0 (socket says 172.21.15.123:42224) 2024-09-19T04:49:54.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbc9b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b7650 0x55d6570de7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42256/0 (socket says 172.21.15.123:42256) 2024-09-19T04:49:54.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbc9a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b6d30 0x55d6570de290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49766/0 (socket says 172.21.15.123:49766) 2024-09-19T04:49:54.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbc9aa00640 1 -- 172.21.15.123:0/3210574354 learned_addr learned my addr 172.21.15.123:0/3210574354 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:54.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbc9aa00640 1 -- 172.21.15.123:0/3210574354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b6d30 msgr2=0x55d6570de290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:54.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbc9aa00640 1 --2- 172.21.15.123:0/3210574354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b6d30 0x55d6570de290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbc9aa00640 1 -- 172.21.15.123:0/3210574354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b7650 msgr2=0x55d6570de7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:54.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbc9aa00640 1 --2- 172.21.15.123:0/3210574354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b7650 0x55d6570de7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.703+0000 7fbc9aa00640 1 -- 172.21.15.123:0/3210574354 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6570c23a0 con 0x55d6570b3420 2024-09-19T04:49:54.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.704+0000 7fbc9b400640 1 --2- 172.21.15.123:0/3210574354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b7650 0x55d6570de7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:49:54.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.704+0000 7fbc9aa00640 1 --2- 172.21.15.123:0/3210574354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b3420 0x55d6570c1cc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c005650 tx=0x7fbc8c0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1051134a344ec3c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:54.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.704+0000 7fbc87400640 1 -- 172.21.15.123:0/3210574354 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c01e030 con 0x55d6570b3420 2024-09-19T04:49:54.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.704+0000 7fbc87400640 1 -- 172.21.15.123:0/3210574354 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc8c00f960 con 0x55d6570b3420 2024-09-19T04:49:54.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.704+0000 7fbc87400640 1 -- 172.21.15.123:0/3210574354 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c0039e0 con 0x55d6570b3420 2024-09-19T04:49:54.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.705+0000 7fbca4826740 1 -- 172.21.15.123:0/3210574354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b3420 msgr2=0x55d6570c1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:54.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.705+0000 7fbca4826740 1 --2- 172.21.15.123:0/3210574354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b3420 0x55d6570c1cc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c005650 tx=0x7fbc8c0119b0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.705+0000 7fbca4826740 1 -- 172.21.15.123:0/3210574354 shutdown_connections 2024-09-19T04:49:54.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.705+0000 7fbca4826740 1 --2- 172.21.15.123:0/3210574354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b7650 0x55d6570de7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.705+0000 7fbca4826740 1 --2- 172.21.15.123:0/3210574354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b6d30 0x55d6570de290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.705+0000 7fbca4826740 1 --2- 172.21.15.123:0/3210574354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b3420 0x55d6570c1cc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.705+0000 7fbca4826740 1 -- 172.21.15.123:0/3210574354 >> 172.21.15.123:0/3210574354 conn(0x55d6570bede0 msgr2=0x55d6570b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:54.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.705+0000 7fbca4826740 1 -- 172.21.15.123:0/3210574354 shutdown_connections 2024-09-19T04:49:54.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.705+0000 7fbca4826740 1 -- 172.21.15.123:0/3210574354 wait complete. 2024-09-19T04:49:54.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.705+0000 7fbca4826740 1 -- start start 2024-09-19T04:49:54.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.706+0000 7fbca4826740 1 Processor -- start 2024-09-19T04:49:54.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.706+0000 7fbca4826740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b3420 0x55d6570cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.706+0000 7fbca4826740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b6d30 0x55d6570c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.706+0000 7fbca4826740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b7650 0x55d6570ca270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.706+0000 7fbca4826740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1ac910 con 0x55d6570b3420 2024-09-19T04:49:54.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.706+0000 7fbca4826740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1ac240 con 0x55d6570b6d30 2024-09-19T04:49:54.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.706+0000 7fbca4826740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1a93f0 con 0x55d6570b7650 2024-09-19T04:49:54.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.707+0000 7fbca4826740 10 client.0 _ll_register_callbacks cb 0x7fbc8c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:54.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.707+0000 7fbc9aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b3420 0x55d6570cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:54.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.707+0000 7fbc9b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b7650 0x55d6570ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:54.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.707+0000 7fbc9aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b3420 0x55d6570cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42272/0 (socket says 172.21.15.123:42272) 2024-09-19T04:49:54.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.707+0000 7fbc9aa00640 1 -- 172.21.15.123:0/3450589944 learned_addr learned my addr 172.21.15.123:0/3450589944 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:54.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.707+0000 7fbc9b400640 1 -- 172.21.15.123:0/3450589944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b6d30 msgr2=0x55d6570c9d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:49:54.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.707+0000 7fbc9b400640 1 --2- 172.21.15.123:0/3450589944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b6d30 0x55d6570c9d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.707+0000 7fbc9b400640 1 -- 172.21.15.123:0/3450589944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b3420 msgr2=0x55d6570cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:54.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.707+0000 7fbc9b400640 1 --2- 172.21.15.123:0/3450589944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b3420 0x55d6570cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.707+0000 7fbc9b400640 1 -- 172.21.15.123:0/3450589944 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6570b7480 con 0x55d6570b7650 2024-09-19T04:49:54.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.707+0000 7fbc9aa00640 1 --2- 172.21.15.123:0/3450589944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b3420 0x55d6570cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:49:54.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.708+0000 7fbc9b400640 1 --2- 172.21.15.123:0/3450589944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b7650 0x55d6570ca270 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbc940ede90 tx=0x7fbc94005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c394bbe7a52bf2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:54.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.708+0000 7fbc84c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:49:54.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.708+0000 7fbc84c00640 1 -- 172.21.15.123:0/3450589944 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc940c0e00 con 0x55d6570b7650 2024-09-19T04:49:54.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.708+0000 7fbc84c00640 1 -- 172.21.15.123:0/3450589944 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fbc940ee140 con 0x55d6570b7650 2024-09-19T04:49:54.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.708+0000 7fbca4826740 1 -- 172.21.15.123:0/3450589944 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc8c00f960 con 0x55d6570b7650 2024-09-19T04:49:54.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.708+0000 7fbc84c00640 1 -- 172.21.15.123:0/3450589944 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbc940c56c0 con 0x55d6570b7650 2024-09-19T04:49:54.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.708+0000 7fbca4826740 10 client.25951 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:54.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.708+0000 7fbca4826740 1 -- 172.21.15.123:0/3450589944 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d657140ee0 con 0x55d6570b7650 2024-09-19T04:49:54.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.708+0000 7fbc84c00640 1 -- 172.21.15.123:0/3450589944 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc8c00f960 con 0x55d6570b7650 2024-09-19T04:49:54.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.709+0000 7fbc84c00640 1 -- 172.21.15.123:0/3450589944 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fbc940316d0 con 0x55d6570b7650 2024-09-19T04:49:54.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.709+0000 7fbca4826740 10 client.25951 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:54.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.709+0000 7fbca4826740 10 client.25951 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:49:54.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.709+0000 7fbca4826740 20 client.25951 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:54.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.709+0000 7fbca4826740 1 --2- 172.21.15.123:0/3450589944 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55d6570c6480 0x55d6571436c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:54.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.709+0000 7fbca4826740 4 client.25951 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:49:54.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.709+0000 7fbca4826740 1 -- 172.21.15.123:0/3450589944 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d657024e60 con 0x55d6570c6480 2024-09-19T04:49:54.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.709+0000 7fbc9aa00640 1 --2- 172.21.15.123:0/3450589944 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55d6570c6480 0x55d6571436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:54.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.710+0000 7fbc9aa00640 1 --2- 172.21.15.123:0/3450589944 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55d6570c6480 0x55d6571436c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=27161f1b9e50bbd7 server_cookie=d8210441debc6f43 in_seq=0 out_seq=0 2024-09-19T04:49:54.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.710+0000 7fbc84c00640 10 client.25951 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:49:54.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.710+0000 7fbc84c00640 1 -- 172.21.15.123:0/3450589944 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d657024e60 con 0x55d6570c6480 2024-09-19T04:49:54.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.710+0000 7fbc84c00640 10 client.25951 handle_command_reply: tid=0 2024-09-19T04:49:54.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.710+0000 7fbca4826740 1 client.25951 shutdown 2024-09-19T04:49:54.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.710+0000 7fbca4826740 1 -- 172.21.15.123:0/3450589944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b7650 msgr2=0x55d6570ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:54.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.710+0000 7fbca4826740 1 --2- 172.21.15.123:0/3450589944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b7650 0x55d6570ca270 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbc940ede90 tx=0x7fbc94005fd0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.710+0000 7fbca4826740 1 -- 172.21.15.123:0/3450589944 shutdown_connections 2024-09-19T04:49:54.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.710+0000 7fbca4826740 1 --2- 172.21.15.123:0/3450589944 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55d6570c6480 0x55d6571436c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.710+0000 7fbca4826740 1 --2- 172.21.15.123:0/3450589944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6570b7650 0x55d6570ca270 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.710+0000 7fbca4826740 1 --2- 172.21.15.123:0/3450589944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6570b6d30 0x55d6570c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.710+0000 7fbca4826740 1 --2- 172.21.15.123:0/3450589944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6570b3420 0x55d6570cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.710+0000 7fbca4826740 1 -- 172.21.15.123:0/3450589944 >> 172.21.15.123:0/3450589944 conn(0x55d6570bede0 msgr2=0x55d6570b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:54.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.710+0000 7fbc84c00640 0 client.25951 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:49:54.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.710+0000 7fbca4826740 1 -- 172.21.15.123:0/3450589944 shutdown_connections 2024-09-19T04:49:54.873 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:49:54.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.710+0000 7fbca4826740 1 -- 172.21.15.123:0/3450589944 wait complete. 2024-09-19T04:49:54.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.711+0000 7fbca4826740 20 client.25951 trim_cache size 0 max 16384 2024-09-19T04:49:54.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.711+0000 7fbc9be00640 1 -- 172.21.15.123:0/3402048811 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbc800a33b0 msgr2=0x7fbc800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:54.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.711+0000 7fbc9be00640 1 --2- 172.21.15.123:0/3402048811 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbc800a33b0 0x7fbc800a5870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc880094b0 tx=0x7fbc88009680 comp rx=0 tx=0).stop 2024-09-19T04:49:54.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.711+0000 7fbc9be00640 1 -- 172.21.15.123:0/3402048811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc9c1079c0 msgr2=0x7fbc9c19caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:54.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.711+0000 7fbc9be00640 1 --2- 172.21.15.123:0/3402048811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc9c1079c0 0x7fbc9c19caf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009b30 tx=0x7fbc94030c50 comp rx=0 tx=0).stop 2024-09-19T04:49:54.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.711+0000 7fbc9be00640 1 -- 172.21.15.123:0/3402048811 shutdown_connections 2024-09-19T04:49:54.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.711+0000 7fbc9be00640 1 --2- 172.21.15.123:0/3402048811 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbc800a33b0 0x7fbc800a5870 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.712+0000 7fbc9be00640 1 --2- 172.21.15.123:0/3402048811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc9c1079c0 0x7fbc9c19caf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.712+0000 7fbc9be00640 1 --2- 172.21.15.123:0/3402048811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9c102360 0x7fbc9c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.712+0000 7fbc9be00640 1 --2- 172.21.15.123:0/3402048811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc9c1019c0 0x7fbc9c198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:54.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.712+0000 7fbc9be00640 1 -- 172.21.15.123:0/3402048811 >> 172.21.15.123:0/3402048811 conn(0x7fbc9c0fd740 msgr2=0x7fbc9c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:54.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.712+0000 7fbc9be00640 1 -- 172.21.15.123:0/3402048811 shutdown_connections 2024-09-19T04:49:54.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:54.712+0000 7fbc9be00640 1 -- 172.21.15.123:0/3402048811 wait complete. 2024-09-19T04:49:55.795 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:49:56.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:55 smithi193 ceph-mon[35592]: pgmap v99: 97 pgs: 97 active+clean; 97 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.4 MiB/s wr, 665 op/s 2024-09-19T04:49:56.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:55 smithi183 ceph-mon[35333]: pgmap v99: 97 pgs: 97 active+clean; 97 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.4 MiB/s wr, 665 op/s 2024-09-19T04:49:56.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:55 smithi123 ceph-mon[30511]: pgmap v99: 97 pgs: 97 active+clean; 97 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.4 MiB/s wr, 665 op/s 2024-09-19T04:49:56.244 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:49:57.045 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:56 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:49:57.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:56 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:49:57.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.180+0000 7fb9a5000640 1 -- 172.21.15.123:0/3233008801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a0104010 msgr2=0x7fb9a01043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:57.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.180+0000 7fb9a5000640 1 --2- 172.21.15.123:0/3233008801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a0104010 0x7fb9a01043f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb990009900 tx=0x7fb99002a410 comp rx=0 tx=0).stop 2024-09-19T04:49:57.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.181+0000 7fb9a5000640 1 -- 172.21.15.123:0/3233008801 shutdown_connections 2024-09-19T04:49:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.181+0000 7fb9a5000640 1 --2- 172.21.15.123:0/3233008801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9a0109140 0x7fb9a0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.181+0000 7fb9a5000640 1 --2- 172.21.15.123:0/3233008801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9a01049c0 0x7fb9a0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.181+0000 7fb9a5000640 1 --2- 172.21.15.123:0/3233008801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a0104010 0x7fb9a01043f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.181+0000 7fb9a5000640 1 -- 172.21.15.123:0/3233008801 >> 172.21.15.123:0/3233008801 conn(0x7fb9a00ffa50 msgr2=0x7fb9a0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.181+0000 7fb9a5000640 1 -- 172.21.15.123:0/3233008801 shutdown_connections 2024-09-19T04:49:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.181+0000 7fb9a5000640 1 -- 172.21.15.123:0/3233008801 wait complete. 2024-09-19T04:49:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.182+0000 7fb9a5000640 1 Processor -- start 2024-09-19T04:49:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.182+0000 7fb9a5000640 1 -- start start 2024-09-19T04:49:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.182+0000 7fb9a5000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9a0104010 0x7fb9a019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.182+0000 7fb9a5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a01049c0 0x7fb9a019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.182+0000 7fb9a5000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9a0109140 0x7fb9a019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.182+0000 7fb9a5000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01141b0 con 0x7fb9a01049c0 2024-09-19T04:49:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.182+0000 7fb9a5000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0114030 con 0x7fb9a0104010 2024-09-19T04:49:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.182+0000 7fb9a5000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0114330 con 0x7fb9a0109140 2024-09-19T04:49:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.182+0000 7fb99de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9a0104010 0x7fb9a019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.182+0000 7fb99d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a01049c0 0x7fb9a019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.182+0000 7fb99de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9a0104010 0x7fb9a019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:42246/0 (socket says 172.21.15.123:42246) 2024-09-19T04:49:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.182+0000 7fb99de00640 1 -- 172.21.15.123:0/1013971131 learned_addr learned my addr 172.21.15.123:0/1013971131 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.182+0000 7fb99e800640 1 --2- 172.21.15.123:0/1013971131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9a0109140 0x7fb9a019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.183+0000 7fb99de00640 1 -- 172.21.15.123:0/1013971131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9a0109140 msgr2=0x7fb9a019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.183+0000 7fb99de00640 1 --2- 172.21.15.123:0/1013971131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9a0109140 0x7fb9a019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.183+0000 7fb99de00640 1 -- 172.21.15.123:0/1013971131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a01049c0 msgr2=0x7fb9a019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.183+0000 7fb99de00640 1 --2- 172.21.15.123:0/1013971131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a01049c0 0x7fb9a019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.183+0000 7fb99de00640 1 -- 172.21.15.123:0/1013971131 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a019f5e0 con 0x7fb9a0104010 2024-09-19T04:49:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.183+0000 7fb99de00640 1 --2- 172.21.15.123:0/1013971131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9a0104010 0x7fb9a019a630 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb990004770 tx=0x7fb990003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:57.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.183+0000 7fb996a00640 1 -- 172.21.15.123:0/1013971131 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99003f070 con 0x7fb9a0104010 2024-09-19T04:49:57.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.183+0000 7fb9a5000640 1 -- 172.21.15.123:0/1013971131 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a019f870 con 0x7fb9a0104010 2024-09-19T04:49:57.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.183+0000 7fb996a00640 1 -- 172.21.15.123:0/1013971131 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb990031ca0 con 0x7fb9a0104010 2024-09-19T04:49:57.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.183+0000 7fb996a00640 1 -- 172.21.15.123:0/1013971131 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990035030 con 0x7fb9a0104010 2024-09-19T04:49:57.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.183+0000 7fb9a5000640 1 -- 172.21.15.123:0/1013971131 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a01a71b0 con 0x7fb9a0104010 2024-09-19T04:49:57.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.187+0000 7fb9a7c02740 1 Processor -- start 2024-09-19T04:49:57.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.187+0000 7fb9a7c02740 1 -- start start 2024-09-19T04:49:57.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.187+0000 7fb996a00640 1 -- 172.21.15.123:0/1013971131 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb990042070 con 0x7fb9a0104010 2024-09-19T04:49:57.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.187+0000 7fb9a7c02740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e754c70 0x55735e755050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.187+0000 7fb9a7c02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 0x55735e75f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.187+0000 7fb9a7c02740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e75fe50 0x55735e762240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.187+0000 7fb9a7c02740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55735e6bb3e0 con 0x55735e75fe50 2024-09-19T04:49:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.187+0000 7fb9a7c02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55735e662d80 con 0x55735e755590 2024-09-19T04:49:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.187+0000 7fb9a7c02740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55735e6b73a0 con 0x55735e754c70 2024-09-19T04:49:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.188+0000 7fb99de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 0x55735e75f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.188+0000 7fb99de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 0x55735e75f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:42250/0 (socket says 172.21.15.123:42250) 2024-09-19T04:49:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.188+0000 7fb99de00640 1 -- 172.21.15.123:0/3544649156 learned_addr learned my addr 172.21.15.123:0/3544649156 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.188+0000 7fb99e800640 1 --2- 172.21.15.123:0/3544649156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e754c70 0x55735e755050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.188+0000 7fb99de00640 1 -- 172.21.15.123:0/3544649156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e754c70 msgr2=0x55735e755050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.188+0000 7fb99de00640 1 --2- 172.21.15.123:0/3544649156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e754c70 0x55735e755050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.188+0000 7fb99de00640 1 -- 172.21.15.123:0/3544649156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e75fe50 msgr2=0x55735e762240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:49:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.188+0000 7fb99de00640 1 --2- 172.21.15.123:0/3544649156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e75fe50 0x55735e762240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.188+0000 7fb99de00640 1 -- 172.21.15.123:0/3544649156 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55735e681ae0 con 0x55735e755590 2024-09-19T04:49:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.188+0000 7fb99e800640 1 --2- 172.21.15.123:0/3544649156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e754c70 0x55735e755050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:49:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.188+0000 7fb99de00640 1 --2- 172.21.15.123:0/3544649156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 0x55735e75f910 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb9900ed9f0 tx=0x7fb9900f8600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23f1bd6997ce652a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.189+0000 7fb996a00640 1 --2- 172.21.15.123:0/1013971131 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb9840a33b0 0x7fb9840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.189+0000 7fb98b400640 1 -- 172.21.15.123:0/3544649156 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9900b89f0 con 0x55735e755590 2024-09-19T04:49:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.189+0000 7fb98b400640 1 -- 172.21.15.123:0/3544649156 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9900b8640 con 0x55735e755590 2024-09-19T04:49:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.189+0000 7fb98b400640 1 -- 172.21.15.123:0/3544649156 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9900bbba0 con 0x55735e755590 2024-09-19T04:49:57.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.189+0000 7fb99e800640 1 --2- 172.21.15.123:0/1013971131 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb9840a33b0 0x7fb9840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:57.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.190+0000 7fb996a00640 1 -- 172.21.15.123:0/1013971131 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fb9900f02d0 con 0x7fb9a0104010 2024-09-19T04:49:57.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.190+0000 7fb9a7c02740 1 -- 172.21.15.123:0/3544649156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 msgr2=0x55735e75f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:57.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.190+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/3544649156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 0x55735e75f910 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb9900ed9f0 tx=0x7fb9900f8600 comp rx=0 tx=0).stop 2024-09-19T04:49:57.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.190+0000 7fb9a7c02740 1 -- 172.21.15.123:0/3544649156 shutdown_connections 2024-09-19T04:49:57.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.190+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/3544649156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e75fe50 0x55735e762240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.190+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/3544649156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 0x55735e75f910 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.190+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/3544649156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e754c70 0x55735e755050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.190+0000 7fb9a7c02740 1 -- 172.21.15.123:0/3544649156 >> 172.21.15.123:0/3544649156 conn(0x55735e751200 msgr2=0x55735e7515e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:57.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.190+0000 7fb99e800640 1 --2- 172.21.15.123:0/1013971131 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb9840a33b0 0x7fb9840a5870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb998005fd0 tx=0x7fb998009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:57.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.191+0000 7fb9a7c02740 1 -- 172.21.15.123:0/3544649156 shutdown_connections 2024-09-19T04:49:57.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.191+0000 7fb9a7c02740 1 -- 172.21.15.123:0/3544649156 wait complete. 2024-09-19T04:49:57.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.191+0000 7fb9a7c02740 1 -- start start 2024-09-19T04:49:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.191+0000 7fb9a7c02740 1 Processor -- start 2024-09-19T04:49:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.192+0000 7fb9a7c02740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e754c70 0x55735e75cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.192+0000 7fb9a7c02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 0x55735e75d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.192+0000 7fb9a7c02740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e75fe50 0x55735e75d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.192+0000 7fb9a7c02740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55735e7627e0 con 0x55735e754c70 2024-09-19T04:49:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.192+0000 7fb9a7c02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55735e6760f0 con 0x55735e755590 2024-09-19T04:49:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.192+0000 7fb9a7c02740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55735e762960 con 0x55735e75fe50 2024-09-19T04:49:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.192+0000 7fb9a7c02740 10 client.0 _ll_register_callbacks cb 0x55735e65dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.192+0000 7fb99e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e75fe50 0x55735e75d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.192+0000 7fb99e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e75fe50 0x55735e75d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49812/0 (socket says 172.21.15.123:49812) 2024-09-19T04:49:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.192+0000 7fb99e800640 1 -- 172.21.15.123:0/3361480451 learned_addr learned my addr 172.21.15.123:0/3361480451 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.192+0000 7fb99de00640 1 --2- 172.21.15.123:0/3361480451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e754c70 0x55735e75cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.192+0000 7fb99d400640 1 --2- 172.21.15.123:0/3361480451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 0x55735e75d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:57.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.193+0000 7fb99e800640 1 -- 172.21.15.123:0/3361480451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 msgr2=0x55735e75d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:57.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.193+0000 7fb99e800640 1 --2- 172.21.15.123:0/3361480451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 0x55735e75d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.193+0000 7fb99e800640 1 -- 172.21.15.123:0/3361480451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e754c70 msgr2=0x55735e75cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:57.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.193+0000 7fb99e800640 1 --2- 172.21.15.123:0/3361480451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e754c70 0x55735e75cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.193+0000 7fb99e800640 1 -- 172.21.15.123:0/3361480451 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55735e42c3c0 con 0x55735e75fe50 2024-09-19T04:49:57.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.193+0000 7fb99de00640 1 --2- 172.21.15.123:0/3361480451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e754c70 0x55735e75cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:49:57.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.193+0000 7fb99e800640 1 --2- 172.21.15.123:0/3361480451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e75fe50 0x55735e75d7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb998003f50 tx=0x7fb998004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcff59e7ccd58ab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:57.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.193+0000 7fb97fe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:49:57.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.193+0000 7fb97fe00640 1 -- 172.21.15.123:0/3361480451 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99800f870 con 0x55735e75fe50 2024-09-19T04:49:57.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.193+0000 7fb9a7c02740 1 -- 172.21.15.123:0/3361480451 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9900b8640 con 0x55735e75fe50 2024-09-19T04:49:57.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.193+0000 7fb97fe00640 1 -- 172.21.15.123:0/3361480451 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fb99800fa10 con 0x55735e75fe50 2024-09-19T04:49:57.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.194+0000 7fb97fe00640 1 -- 172.21.15.123:0/3361480451 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb99800fc10 con 0x55735e75fe50 2024-09-19T04:49:57.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.194+0000 7fb9a7c02740 10 client.25963 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:57.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.194+0000 7fb9a7c02740 1 -- 172.21.15.123:0/3361480451 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55735e76e820 con 0x55735e75fe50 2024-09-19T04:49:57.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.194+0000 7fb97fe00640 1 -- 172.21.15.123:0/3361480451 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9900b8640 con 0x55735e75fe50 2024-09-19T04:49:57.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.194+0000 7fb97fe00640 1 -- 172.21.15.123:0/3361480451 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fb998027020 con 0x55735e75fe50 2024-09-19T04:49:57.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.194+0000 7fb9a7c02740 10 client.25963 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:57.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.194+0000 7fb9a7c02740 10 client.25963 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:49:57.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.194+0000 7fb9a7c02740 20 client.25963 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:57.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.194+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/3361480451 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55735e76fc80 0x55735e7720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.194+0000 7fb9a7c02740 4 client.25963 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:49:57.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.194+0000 7fb9a7c02740 1 -- 172.21.15.123:0/3361480451 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55735e6c8950 con 0x55735e76fc80 2024-09-19T04:49:57.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.195+0000 7fb99de00640 1 --2- 172.21.15.123:0/3361480451 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55735e76fc80 0x55735e7720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:57.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.195+0000 7fb99de00640 1 --2- 172.21.15.123:0/3361480451 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55735e76fc80 0x55735e7720e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95777f785ac7e731 server_cookie=cf9480687ea5fd1f in_seq=0 out_seq=0 2024-09-19T04:49:57.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.195+0000 7fb97fe00640 10 client.25963 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:49:57.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.196+0000 7fb97fe00640 1 -- 172.21.15.123:0/3361480451 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55735e6c8950 con 0x55735e76fc80 2024-09-19T04:49:57.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.196+0000 7fb97fe00640 10 client.25963 handle_command_reply: tid=0 2024-09-19T04:49:57.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.196+0000 7fb9a7c02740 1 client.25963 shutdown 2024-09-19T04:49:57.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.196+0000 7fb9a7c02740 1 -- 172.21.15.123:0/3361480451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e75fe50 msgr2=0x55735e75d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:57.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.196+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/3361480451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e75fe50 0x55735e75d7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb998003f50 tx=0x7fb998004a40 comp rx=0 tx=0).stop 2024-09-19T04:49:57.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.196+0000 7fb9a7c02740 1 -- 172.21.15.123:0/3361480451 shutdown_connections 2024-09-19T04:49:57.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.196+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/3361480451 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55735e76fc80 0x55735e7720e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.196+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/3361480451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e75fe50 0x55735e75d7a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.196+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/3361480451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 0x55735e75d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.196+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/3361480451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e754c70 0x55735e75cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.196+0000 7fb9a7c02740 1 -- 172.21.15.123:0/3361480451 >> 172.21.15.123:0/3361480451 conn(0x55735e751200 msgr2=0x55735e750380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:57.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.196+0000 7fb97fe00640 0 client.25963 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:49:57.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.197+0000 7fb9a7c02740 1 -- 172.21.15.123:0/3361480451 shutdown_connections 2024-09-19T04:49:57.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.197+0000 7fb9a7c02740 1 -- 172.21.15.123:0/3361480451 wait complete. 2024-09-19T04:49:57.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.197+0000 7fb9a7c02740 20 client.25963 trim_cache size 0 max 16384 2024-09-19T04:49:57.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.213+0000 7fb9a7c02740 1 Processor -- start 2024-09-19T04:49:57.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.213+0000 7fb9a7c02740 1 -- start start 2024-09-19T04:49:57.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.214+0000 7fb9a7c02740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e754c70 0x55735e7779b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.214+0000 7fb9a7c02740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e755590 0x55735e777ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.214+0000 7fb9a7c02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e75fe50 0x55735e74fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.214+0000 7fb9a7c02740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01afa70 con 0x55735e755590 2024-09-19T04:49:57.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.214+0000 7fb9a7c02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01b1700 con 0x55735e75fe50 2024-09-19T04:49:57.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.214+0000 7fb9a7c02740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01b0140 con 0x55735e754c70 2024-09-19T04:49:57.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.214+0000 7fb99e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e75fe50 0x55735e74fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:57.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.214+0000 7fb99d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e755590 0x55735e777ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:57.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.214+0000 7fb99e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e75fe50 0x55735e74fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:42270/0 (socket says 172.21.15.123:42270) 2024-09-19T04:49:57.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.214+0000 7fb99e800640 1 -- 172.21.15.123:0/301668732 learned_addr learned my addr 172.21.15.123:0/301668732 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:57.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.214+0000 7fb99d400640 1 --2- 172.21.15.123:0/301668732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e755590 0x55735e777ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42304/0 (socket says 172.21.15.123:42304) 2024-09-19T04:49:57.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.214+0000 7fb99de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e754c70 0x55735e7779b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:57.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.214+0000 7fb99de00640 1 -- 172.21.15.123:0/301668732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e75fe50 msgr2=0x55735e74fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:57.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.214+0000 7fb99de00640 1 --2- 172.21.15.123:0/301668732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e75fe50 0x55735e74fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.214+0000 7fb99de00640 1 -- 172.21.15.123:0/301668732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e755590 msgr2=0x55735e777ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:57.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.214+0000 7fb99de00640 1 --2- 172.21.15.123:0/301668732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e755590 0x55735e777ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.214+0000 7fb99de00640 1 -- 172.21.15.123:0/301668732 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55735e754aa0 con 0x55735e754c70 2024-09-19T04:49:57.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.215+0000 7fb99de00640 1 --2- 172.21.15.123:0/301668732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e754c70 0x55735e7779b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb9900b8bb0 tx=0x7fb9900f3e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=576f962260166dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:57.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.215+0000 7fb98aa00640 1 -- 172.21.15.123:0/301668732 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990049050 con 0x55735e754c70 2024-09-19T04:49:57.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.215+0000 7fb98aa00640 1 -- 172.21.15.123:0/301668732 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9900f8150 con 0x55735e754c70 2024-09-19T04:49:57.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.215+0000 7fb98aa00640 1 -- 172.21.15.123:0/301668732 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9900bbed0 con 0x55735e754c70 2024-09-19T04:49:57.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.216+0000 7fb9a7c02740 1 -- 172.21.15.123:0/301668732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e754c70 msgr2=0x55735e7779b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.216+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/301668732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e754c70 0x55735e7779b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb9900b8bb0 tx=0x7fb9900f3e20 comp rx=0 tx=0).stop 2024-09-19T04:49:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.216+0000 7fb9a7c02740 1 -- 172.21.15.123:0/301668732 shutdown_connections 2024-09-19T04:49:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.216+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/301668732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e75fe50 0x55735e74fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.216+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/301668732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e755590 0x55735e777ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.216+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/301668732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e754c70 0x55735e7779b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.216+0000 7fb9a7c02740 1 -- 172.21.15.123:0/301668732 >> 172.21.15.123:0/301668732 conn(0x55735e751200 msgr2=0x55735e757ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.216+0000 7fb9a7c02740 1 -- 172.21.15.123:0/301668732 shutdown_connections 2024-09-19T04:49:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.216+0000 7fb9a7c02740 1 -- 172.21.15.123:0/301668732 wait complete. 2024-09-19T04:49:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.216+0000 7fb9a7c02740 1 -- start start 2024-09-19T04:49:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.217+0000 7fb9a7c02740 1 Processor -- start 2024-09-19T04:49:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.217+0000 7fb9a7c02740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e754c70 0x55735e763770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.217+0000 7fb9a7c02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 0x55735e763cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.217+0000 7fb9a7c02740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e75fe50 0x55735e7641f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.217+0000 7fb9a7c02740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01b0f70 con 0x55735e754c70 2024-09-19T04:49:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.217+0000 7fb9a7c02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01b08a0 con 0x55735e755590 2024-09-19T04:49:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.217+0000 7fb9a7c02740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01ada50 con 0x55735e75fe50 2024-09-19T04:49:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.217+0000 7fb9a7c02740 10 client.0 _ll_register_callbacks cb 0x7fb998009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.217+0000 7fb99d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 0x55735e763cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.217+0000 7fb99d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 0x55735e763cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:42276/0 (socket says 172.21.15.123:42276) 2024-09-19T04:49:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.217+0000 7fb99de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e754c70 0x55735e763770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.217+0000 7fb99d400640 1 -- 172.21.15.123:0/1944321394 learned_addr learned my addr 172.21.15.123:0/1944321394 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.218+0000 7fb99e800640 1 --2- 172.21.15.123:0/1944321394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e75fe50 0x55735e7641f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:57.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.218+0000 7fb99d400640 1 -- 172.21.15.123:0/1944321394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e75fe50 msgr2=0x55735e7641f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:57.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.218+0000 7fb99d400640 1 --2- 172.21.15.123:0/1944321394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e75fe50 0x55735e7641f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.218+0000 7fb99d400640 1 -- 172.21.15.123:0/1944321394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e754c70 msgr2=0x55735e763770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:57.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.218+0000 7fb99d400640 1 --2- 172.21.15.123:0/1944321394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e754c70 0x55735e763770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.218+0000 7fb99d400640 1 -- 172.21.15.123:0/1944321394 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55735e757ae0 con 0x55735e755590 2024-09-19T04:49:57.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.218+0000 7fb99de00640 1 --2- 172.21.15.123:0/1944321394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e754c70 0x55735e763770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:49:57.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.218+0000 7fb99d400640 1 --2- 172.21.15.123:0/1944321394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 0x55735e763cb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00ffe70 tx=0x7fb98c010b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2b6adae83b13409 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:57.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.218+0000 7fb97fe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:49:57.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.218+0000 7fb97fe00640 1 -- 172.21.15.123:0/1944321394 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c021060 con 0x55735e755590 2024-09-19T04:49:57.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.218+0000 7fb9a7c02740 1 -- 172.21.15.123:0/1944321394 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9900f8150 con 0x55735e755590 2024-09-19T04:49:57.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.218+0000 7fb97fe00640 1 -- 172.21.15.123:0/1944321394 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fb98c00fcb0 con 0x55735e755590 2024-09-19T04:49:57.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.219+0000 7fb97fe00640 1 -- 172.21.15.123:0/1944321394 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb98c010e50 con 0x55735e755590 2024-09-19T04:49:57.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.219+0000 7fb9a7c02740 10 client.25949 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:57.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.219+0000 7fb9a7c02740 1 -- 172.21.15.123:0/1944321394 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55735e74c770 con 0x55735e755590 2024-09-19T04:49:57.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.219+0000 7fb97fe00640 1 -- 172.21.15.123:0/1944321394 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9900f8150 con 0x55735e755590 2024-09-19T04:49:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.219+0000 7fb97fe00640 1 -- 172.21.15.123:0/1944321394 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fb98c01b070 con 0x55735e755590 2024-09-19T04:49:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.219+0000 7fb9a7c02740 10 client.25949 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.219+0000 7fb9a7c02740 10 client.25949 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:49:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.219+0000 7fb9a7c02740 20 client.25949 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.219+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/1944321394 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55735e7db0b0 0x55735e7dd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.219+0000 7fb9a7c02740 4 client.25949 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:49:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.219+0000 7fb9a7c02740 1 -- 172.21.15.123:0/1944321394 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55735e6c2e60 con 0x55735e7db0b0 2024-09-19T04:49:57.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.219+0000 7fb99de00640 1 --2- 172.21.15.123:0/1944321394 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55735e7db0b0 0x55735e7dd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:57.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.220+0000 7fb99de00640 1 --2- 172.21.15.123:0/1944321394 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55735e7db0b0 0x55735e7dd4d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2b087908c6630565 server_cookie=b64a2dbe26dc7cbe in_seq=0 out_seq=0 2024-09-19T04:49:57.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.220+0000 7fb97fe00640 10 client.25949 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:49:57.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.220+0000 7fb97fe00640 1 -- 172.21.15.123:0/1944321394 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55735e6c2e60 con 0x55735e7db0b0 2024-09-19T04:49:57.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.220+0000 7fb97fe00640 10 client.25949 handle_command_reply: tid=0 2024-09-19T04:49:57.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.220+0000 7fb9a7c02740 1 client.25949 shutdown 2024-09-19T04:49:57.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.220+0000 7fb9a7c02740 1 -- 172.21.15.123:0/1944321394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 msgr2=0x55735e763cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:57.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.220+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/1944321394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 0x55735e763cb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00ffe70 tx=0x7fb98c010b40 comp rx=0 tx=0).stop 2024-09-19T04:49:57.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.220+0000 7fb9a7c02740 1 -- 172.21.15.123:0/1944321394 shutdown_connections 2024-09-19T04:49:57.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.220+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/1944321394 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55735e7db0b0 0x55735e7dd4d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.220+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/1944321394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55735e75fe50 0x55735e7641f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.220+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/1944321394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55735e755590 0x55735e763cb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.220+0000 7fb9a7c02740 1 --2- 172.21.15.123:0/1944321394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55735e754c70 0x55735e763770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.220+0000 7fb9a7c02740 1 -- 172.21.15.123:0/1944321394 >> 172.21.15.123:0/1944321394 conn(0x55735e751200 msgr2=0x55735e74fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:57.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.220+0000 7fb97fe00640 0 client.25949 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:49:57.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.221+0000 7fb9a7c02740 1 -- 172.21.15.123:0/1944321394 shutdown_connections 2024-09-19T04:49:57.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.221+0000 7fb9a7c02740 1 -- 172.21.15.123:0/1944321394 wait complete. 2024-09-19T04:49:57.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.221+0000 7fb9a7c02740 20 client.25949 trim_cache size 0 max 16384 2024-09-19T04:49:57.227 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:49:57.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.222+0000 7fb9a5000640 1 -- 172.21.15.123:0/1013971131 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb9840a33b0 msgr2=0x7fb9840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:57.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.222+0000 7fb9a5000640 1 --2- 172.21.15.123:0/1013971131 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb9840a33b0 0x7fb9840a5870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb998005fd0 tx=0x7fb998009500 comp rx=0 tx=0).stop 2024-09-19T04:49:57.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.222+0000 7fb9a5000640 1 -- 172.21.15.123:0/1013971131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9a0104010 msgr2=0x7fb9a019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:57.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.222+0000 7fb9a5000640 1 --2- 172.21.15.123:0/1013971131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9a0104010 0x7fb9a019a630 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb990004770 tx=0x7fb990003c10 comp rx=0 tx=0).stop 2024-09-19T04:49:57.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.222+0000 7fb9a5000640 1 -- 172.21.15.123:0/1013971131 shutdown_connections 2024-09-19T04:49:57.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.222+0000 7fb9a5000640 1 --2- 172.21.15.123:0/1013971131 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb9840a33b0 0x7fb9840a5870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.222+0000 7fb9a5000640 1 --2- 172.21.15.123:0/1013971131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9a0109140 0x7fb9a019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.222+0000 7fb9a5000640 1 --2- 172.21.15.123:0/1013971131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a01049c0 0x7fb9a019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.222+0000 7fb9a5000640 1 --2- 172.21.15.123:0/1013971131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9a0104010 0x7fb9a019a630 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:57.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.222+0000 7fb9a5000640 1 -- 172.21.15.123:0/1013971131 >> 172.21.15.123:0/1013971131 conn(0x7fb9a00ffa50 msgr2=0x7fb9a0100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:57.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.222+0000 7fb9a5000640 1 -- 172.21.15.123:0/1013971131 shutdown_connections 2024-09-19T04:49:57.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:57.222+0000 7fb9a5000640 1 -- 172.21.15.123:0/1013971131 wait complete. 2024-09-19T04:49:57.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:56 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:49:57.306 INFO:tasks.workunit.client.0.smithi123.stdout:copying to largest ever: 0xffdb8 2024-09-19T04:49:57.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:57 smithi123 ceph-mon[30511]: pgmap v100: 97 pgs: 97 active+clean; 97 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 405 op/s 2024-09-19T04:49:57.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:57 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:49:57.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:57 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:49:57.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:57 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:49:58.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:57 smithi183 ceph-mon[35333]: pgmap v100: 97 pgs: 97 active+clean; 97 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 405 op/s 2024-09-19T04:49:58.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:57 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:49:58.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:57 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:49:58.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:57 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:49:58.129 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:49:58.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:57 smithi193 ceph-mon[35592]: pgmap v100: 97 pgs: 97 active+clean; 97 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 405 op/s 2024-09-19T04:49:58.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:57 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:49:58.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:57 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:49:58.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:57 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:49:58.555 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:49:59.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.184+0000 7f6ced200640 1 -- 172.21.15.123:0/1245433211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ce8109120 msgr2=0x7f6ce8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:59.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.184+0000 7f6ced200640 1 --2- 172.21.15.123:0/1245433211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ce8109120 0x7f6ce8110ca0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009be0 tx=0x7f6ce002a6c0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.185+0000 7f6ced200640 1 -- 172.21.15.123:0/1245433211 shutdown_connections 2024-09-19T04:49:59.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.185+0000 7f6ced200640 1 --2- 172.21.15.123:0/1245433211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ce8109120 0x7f6ce8110ca0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.185+0000 7f6ced200640 1 --2- 172.21.15.123:0/1245433211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce81049a0 0x7f6ce81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.185+0000 7f6ced200640 1 --2- 172.21.15.123:0/1245433211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ce8103ff0 0x7f6ce81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.185+0000 7f6ced200640 1 -- 172.21.15.123:0/1245433211 >> 172.21.15.123:0/1245433211 conn(0x7f6ce80ffa90 msgr2=0x7f6ce8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:59.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.186+0000 7f6ced200640 1 -- 172.21.15.123:0/1245433211 shutdown_connections 2024-09-19T04:49:59.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.186+0000 7f6ced200640 1 -- 172.21.15.123:0/1245433211 wait complete. 2024-09-19T04:49:59.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.187+0000 7f6ced200640 1 Processor -- start 2024-09-19T04:49:59.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.187+0000 7f6ced200640 1 -- start start 2024-09-19T04:49:59.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.188+0000 7f6ced200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ce8103ff0 0x7f6ce819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.188+0000 7f6ced200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce81049a0 0x7f6ce819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.188+0000 7f6ced200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ce8109120 0x7f6ce819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.188+0000 7f6ced200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce8114130 con 0x7f6ce81049a0 2024-09-19T04:49:59.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.188+0000 7f6ced200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce8113fb0 con 0x7f6ce8109120 2024-09-19T04:49:59.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.188+0000 7f6ced200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce81142b0 con 0x7f6ce8103ff0 2024-09-19T04:49:59.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.188+0000 7f6ce5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ce8103ff0 0x7f6ce819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:59.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.188+0000 7f6ce5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ce8103ff0 0x7f6ce819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49858/0 (socket says 172.21.15.123:49858) 2024-09-19T04:49:59.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.188+0000 7f6ce5e00640 1 -- 172.21.15.123:0/3966790165 learned_addr learned my addr 172.21.15.123:0/3966790165 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:59.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.188+0000 7f6cdfe00640 1 --2- 172.21.15.123:0/3966790165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce81049a0 0x7f6ce819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:59.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.188+0000 7f6ce6800640 1 --2- 172.21.15.123:0/3966790165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ce8109120 0x7f6ce819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:59.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.188+0000 7f6ce5e00640 1 -- 172.21.15.123:0/3966790165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ce8109120 msgr2=0x7f6ce819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:59.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.188+0000 7f6ce5e00640 1 --2- 172.21.15.123:0/3966790165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ce8109120 0x7f6ce819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.188+0000 7f6ce5e00640 1 -- 172.21.15.123:0/3966790165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce81049a0 msgr2=0x7f6ce819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:59.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.188+0000 7f6ce5e00640 1 --2- 172.21.15.123:0/3966790165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce81049a0 0x7f6ce819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.188+0000 7f6ce5e00640 1 -- 172.21.15.123:0/3966790165 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce819f540 con 0x7f6ce8103ff0 2024-09-19T04:49:59.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.189+0000 7f6ce5e00640 1 --2- 172.21.15.123:0/3966790165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ce8103ff0 0x7f6ce819a590 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800d950 tx=0x7f6cd800dd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:59.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.189+0000 7f6cdea00640 1 -- 172.21.15.123:0/3966790165 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8017070 con 0x7f6ce8103ff0 2024-09-19T04:49:59.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.189+0000 7f6cdea00640 1 -- 172.21.15.123:0/3966790165 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd800de80 con 0x7f6ce8103ff0 2024-09-19T04:49:59.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.189+0000 7f6ced200640 1 -- 172.21.15.123:0/3966790165 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce819f830 con 0x7f6ce8103ff0 2024-09-19T04:49:59.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.189+0000 7f6cdea00640 1 -- 172.21.15.123:0/3966790165 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd800f620 con 0x7f6ce8103ff0 2024-09-19T04:49:59.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.189+0000 7f6ced200640 1 -- 172.21.15.123:0/3966790165 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce81a7110 con 0x7f6ce8103ff0 2024-09-19T04:49:59.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.192+0000 7f6cdea00640 1 -- 172.21.15.123:0/3966790165 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6cd8009c30 con 0x7f6ce8103ff0 2024-09-19T04:49:59.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.193+0000 7f6cefdf0740 1 Processor -- start 2024-09-19T04:49:59.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.193+0000 7f6cefdf0740 1 -- start start 2024-09-19T04:49:59.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.194+0000 7f6cefdf0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddb4c70 0x5652cddb5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.194+0000 7f6cefdf0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb5590 0x5652cddbf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.194+0000 7f6cefdf0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddbfe50 0x5652cddc2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.194+0000 7f6cefdf0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5652cdd1b3e0 con 0x5652cddb5590 2024-09-19T04:49:59.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.194+0000 7f6cefdf0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5652cdcc2d80 con 0x5652cddbfe50 2024-09-19T04:49:59.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.194+0000 7f6cefdf0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5652cdd173a0 con 0x5652cddb4c70 2024-09-19T04:49:59.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.194+0000 7f6ce6800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddb4c70 0x5652cddb5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:59.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.194+0000 7f6ce6800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddb4c70 0x5652cddb5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49872/0 (socket says 172.21.15.123:49872) 2024-09-19T04:49:59.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.194+0000 7f6ce6800640 1 -- 172.21.15.123:0/146091622 learned_addr learned my addr 172.21.15.123:0/146091622 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:59.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.194+0000 7f6cdea00640 1 --2- 172.21.15.123:0/3966790165 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ccc0a33b0 0x7f6ccc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.194+0000 7f6cdfe00640 1 --2- 172.21.15.123:0/146091622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddbfe50 0x5652cddc2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:59.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.195+0000 7f6cdea00640 1 -- 172.21.15.123:0/3966790165 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f6cd80cf990 con 0x7f6ce8103ff0 2024-09-19T04:49:59.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.195+0000 7f6ce6800640 1 --2- 172.21.15.123:0/3966790165 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ccc0a33b0 0x7f6ccc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:59.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.195+0000 7f6ce6800640 1 -- 172.21.15.123:0/146091622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddbfe50 msgr2=0x5652cddc2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:59.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.195+0000 7f6ce6800640 1 --2- 172.21.15.123:0/146091622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddbfe50 0x5652cddc2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.195+0000 7f6ce6800640 1 -- 172.21.15.123:0/146091622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb5590 msgr2=0x5652cddbf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:59.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.195+0000 7f6ce6800640 1 --2- 172.21.15.123:0/146091622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb5590 0x5652cddbf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.195+0000 7f6ce6800640 1 -- 172.21.15.123:0/146091622 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652cdce1ae0 con 0x5652cddb4c70 2024-09-19T04:49:59.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.195+0000 7f6cdfe00640 1 --2- 172.21.15.123:0/146091622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddbfe50 0x5652cddc2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:49:59.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.196+0000 7f6ce6800640 1 --2- 172.21.15.123:0/146091622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddb4c70 0x5652cddb5050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6ce002acb0 tx=0x7f6ce00303a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70dc457bd73c45db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:59.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.196+0000 7f6ce6800640 1 --2- 172.21.15.123:0/3966790165 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ccc0a33b0 0x7f6ccc0a5870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0046000 tx=0x7f6ce0048000 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:59.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.196+0000 7f6cd7400640 1 -- 172.21.15.123:0/146091622 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0036040 con 0x5652cddb4c70 2024-09-19T04:49:59.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.196+0000 7f6cd7400640 1 -- 172.21.15.123:0/146091622 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ce0030710 con 0x5652cddb4c70 2024-09-19T04:49:59.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.196+0000 7f6cd7400640 1 -- 172.21.15.123:0/146091622 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce004a6d0 con 0x5652cddb4c70 2024-09-19T04:49:59.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.197+0000 7f6cefdf0740 1 -- 172.21.15.123:0/146091622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddb4c70 msgr2=0x5652cddb5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:59.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.197+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/146091622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddb4c70 0x5652cddb5050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6ce002acb0 tx=0x7f6ce00303a0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.197+0000 7f6cefdf0740 1 -- 172.21.15.123:0/146091622 shutdown_connections 2024-09-19T04:49:59.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.197+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/146091622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddbfe50 0x5652cddc2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.197+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/146091622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb5590 0x5652cddbf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.197+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/146091622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddb4c70 0x5652cddb5050 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.197+0000 7f6cefdf0740 1 -- 172.21.15.123:0/146091622 >> 172.21.15.123:0/146091622 conn(0x5652cddb1200 msgr2=0x5652cddb15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:59.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.197+0000 7f6cefdf0740 1 -- 172.21.15.123:0/146091622 shutdown_connections 2024-09-19T04:49:59.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.198+0000 7f6cefdf0740 1 -- 172.21.15.123:0/146091622 wait complete. 2024-09-19T04:49:59.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.198+0000 7f6cefdf0740 1 -- start start 2024-09-19T04:49:59.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.199+0000 7f6cefdf0740 1 Processor -- start 2024-09-19T04:49:59.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.199+0000 7f6cefdf0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddb4c70 0x5652cddbcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.199+0000 7f6cefdf0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb5590 0x5652cddbd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.199+0000 7f6cefdf0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddbfe50 0x5652cddbd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.199+0000 7f6cefdf0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5652cddc2a20 con 0x5652cddb5590 2024-09-19T04:49:59.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.199+0000 7f6cefdf0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5652cdcd60f0 con 0x5652cddb4c70 2024-09-19T04:49:59.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.199+0000 7f6cefdf0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5652cddc2ba0 con 0x5652cddbfe50 2024-09-19T04:49:59.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.199+0000 7f6cefdf0740 10 client.0 _ll_register_callbacks cb 0x5652cdcbdbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:59.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.200+0000 7f6ce6800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddbfe50 0x5652cddbd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:59.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.200+0000 7f6ce6800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddbfe50 0x5652cddbd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49888/0 (socket says 172.21.15.123:49888) 2024-09-19T04:49:59.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.200+0000 7f6ce6800640 1 -- 172.21.15.123:0/619411204 learned_addr learned my addr 172.21.15.123:0/619411204 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:59.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.200+0000 7f6cdfe00640 1 --2- 172.21.15.123:0/619411204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb5590 0x5652cddbd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:59.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.200+0000 7f6ce6800640 1 -- 172.21.15.123:0/619411204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddb4c70 msgr2=0x5652cddbcdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:49:59.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.200+0000 7f6ce6800640 1 --2- 172.21.15.123:0/619411204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddb4c70 0x5652cddbcdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.200+0000 7f6ce6800640 1 -- 172.21.15.123:0/619411204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb5590 msgr2=0x5652cddbd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:59.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.200+0000 7f6ce6800640 1 --2- 172.21.15.123:0/619411204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb5590 0x5652cddbd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.200+0000 7f6ce6800640 1 -- 172.21.15.123:0/619411204 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652cda8c3c0 con 0x5652cddbfe50 2024-09-19T04:49:59.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.200+0000 7f6cdfe00640 1 --2- 172.21.15.123:0/619411204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb5590 0x5652cddbd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:49:59.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.200+0000 7f6ce6800640 1 --2- 172.21.15.123:0/619411204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddbfe50 0x5652cddbd870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009340 tx=0x7f6ce0030a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ec4d7973f3949eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:59.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.200+0000 7f6ccbe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:49:59.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.201+0000 7f6ccbe00640 1 -- 172.21.15.123:0/619411204 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0056020 con 0x5652cddbfe50 2024-09-19T04:49:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.201+0000 7f6cefdf0740 1 -- 172.21.15.123:0/619411204 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ce0030710 con 0x5652cddbfe50 2024-09-19T04:49:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.201+0000 7f6ccbe00640 1 -- 172.21.15.123:0/619411204 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f6ce0004250 con 0x5652cddbfe50 2024-09-19T04:49:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.201+0000 7f6ccbe00640 1 -- 172.21.15.123:0/619411204 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6ce0061050 con 0x5652cddbfe50 2024-09-19T04:49:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.201+0000 7f6cefdf0740 10 client.25981 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.201+0000 7f6cefdf0740 1 -- 172.21.15.123:0/619411204 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5652cddc2a20 con 0x5652cddbfe50 2024-09-19T04:49:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.202+0000 7f6ccbe00640 1 -- 172.21.15.123:0/619411204 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ce0030710 con 0x5652cddbfe50 2024-09-19T04:49:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.202+0000 7f6ccbe00640 1 -- 172.21.15.123:0/619411204 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f6ce004a470 con 0x5652cddbfe50 2024-09-19T04:49:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.202+0000 7f6cefdf0740 10 client.25981 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.202+0000 7f6cefdf0740 10 client.25981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:49:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.202+0000 7f6cefdf0740 20 client.25981 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.202+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/619411204 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5652cddcfcd0 0x5652cddd20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.202+0000 7f6cefdf0740 4 client.25981 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:49:59.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.202+0000 7f6cefdf0740 1 -- 172.21.15.123:0/619411204 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652cdd28950 con 0x5652cddcfcd0 2024-09-19T04:49:59.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.203+0000 7f6ce5e00640 1 --2- 172.21.15.123:0/619411204 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5652cddcfcd0 0x5652cddd20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:59.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.203+0000 7f6ce5e00640 1 --2- 172.21.15.123:0/619411204 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5652cddcfcd0 0x5652cddd20c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bc533f4f098c88d server_cookie=77c4b502477e5d52 in_seq=0 out_seq=0 2024-09-19T04:49:59.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.203+0000 7f6ccbe00640 10 client.25981 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:59.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.204+0000 7f6ccbe00640 1 -- 172.21.15.123:0/619411204 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652cdd28950 con 0x5652cddcfcd0 2024-09-19T04:49:59.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.204+0000 7f6ccbe00640 10 client.25981 handle_command_reply: tid=0 2024-09-19T04:49:59.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.204+0000 7f6cefdf0740 1 client.25981 shutdown 2024-09-19T04:49:59.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.204+0000 7f6cefdf0740 1 -- 172.21.15.123:0/619411204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddbfe50 msgr2=0x5652cddbd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:59.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.204+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/619411204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddbfe50 0x5652cddbd870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009340 tx=0x7f6ce0030a50 comp rx=0 tx=0).stop 2024-09-19T04:49:59.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.204+0000 7f6cefdf0740 1 -- 172.21.15.123:0/619411204 shutdown_connections 2024-09-19T04:49:59.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.204+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/619411204 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5652cddcfcd0 0x5652cddd20c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.204+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/619411204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddbfe50 0x5652cddbd870 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.204+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/619411204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb5590 0x5652cddbd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.204+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/619411204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddb4c70 0x5652cddbcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.204+0000 7f6ccbe00640 0 client.25981 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:59.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.204+0000 7f6cefdf0740 1 -- 172.21.15.123:0/619411204 >> 172.21.15.123:0/619411204 conn(0x5652cddb1200 msgr2=0x5652cddb6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:59.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.205+0000 7f6cefdf0740 1 -- 172.21.15.123:0/619411204 shutdown_connections 2024-09-19T04:49:59.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.205+0000 7f6cefdf0740 1 -- 172.21.15.123:0/619411204 wait complete. 2024-09-19T04:49:59.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.205+0000 7f6cefdf0740 20 client.25981 trim_cache size 0 max 16384 2024-09-19T04:49:59.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.223+0000 7f6cefdf0740 1 Processor -- start 2024-09-19T04:49:59.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.223+0000 7f6cefdf0740 1 -- start start 2024-09-19T04:49:59.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.223+0000 7f6cefdf0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb4c70 0x5652cddbd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.223+0000 7f6cefdf0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddb5590 0x5652cddd7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.223+0000 7f6cefdf0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddbfe50 0x5652cddd7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.223+0000 7f6cefdf0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce81af8d0 con 0x5652cddb4c70 2024-09-19T04:49:59.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.223+0000 7f6cefdf0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce81b1560 con 0x5652cddb5590 2024-09-19T04:49:59.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.223+0000 7f6cefdf0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce81affa0 con 0x5652cddbfe50 2024-09-19T04:49:59.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.223+0000 7f6ce5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb4c70 0x5652cddbd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:59.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.224+0000 7f6ce5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb4c70 0x5652cddbd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42352/0 (socket says 172.21.15.123:42352) 2024-09-19T04:49:59.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.224+0000 7f6ce5e00640 1 -- 172.21.15.123:0/1795299606 learned_addr learned my addr 172.21.15.123:0/1795299606 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:59.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.224+0000 7f6ce6800640 1 --2- 172.21.15.123:0/1795299606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddbfe50 0x5652cddd7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:59.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.224+0000 7f6cdfe00640 1 --2- 172.21.15.123:0/1795299606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddb5590 0x5652cddd7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:59.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.224+0000 7f6ce5e00640 1 -- 172.21.15.123:0/1795299606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddbfe50 msgr2=0x5652cddd7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:59.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.224+0000 7f6ce5e00640 1 --2- 172.21.15.123:0/1795299606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddbfe50 0x5652cddd7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.224+0000 7f6ce5e00640 1 -- 172.21.15.123:0/1795299606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddb5590 msgr2=0x5652cddd7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:59.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.224+0000 7f6ce5e00640 1 --2- 172.21.15.123:0/1795299606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddb5590 0x5652cddd7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.224+0000 7f6ce5e00640 1 -- 172.21.15.123:0/1795299606 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652cddac770 con 0x5652cddb4c70 2024-09-19T04:49:59.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.224+0000 7f6ce5e00640 1 --2- 172.21.15.123:0/1795299606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb4c70 0x5652cddbd6e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6cd80058c0 tx=0x7f6cd8097850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0bcf2efc6f1cbdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:59.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.225+0000 7f6cd6a00640 1 -- 172.21.15.123:0/1795299606 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8020040 con 0x5652cddb4c70 2024-09-19T04:49:59.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.225+0000 7f6cd6a00640 1 -- 172.21.15.123:0/1795299606 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd809fff0 con 0x5652cddb4c70 2024-09-19T04:49:59.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.225+0000 7f6cd6a00640 1 -- 172.21.15.123:0/1795299606 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd809b090 con 0x5652cddb4c70 2024-09-19T04:49:59.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.225+0000 7f6cefdf0740 1 -- 172.21.15.123:0/1795299606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb4c70 msgr2=0x5652cddbd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:59.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.226+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/1795299606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb4c70 0x5652cddbd6e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6cd80058c0 tx=0x7f6cd8097850 comp rx=0 tx=0).stop 2024-09-19T04:49:59.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.226+0000 7f6cefdf0740 1 -- 172.21.15.123:0/1795299606 shutdown_connections 2024-09-19T04:49:59.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.226+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/1795299606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddbfe50 0x5652cddd7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.226+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/1795299606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddb5590 0x5652cddd7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.226+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/1795299606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb4c70 0x5652cddbd6e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.226+0000 7f6cefdf0740 1 -- 172.21.15.123:0/1795299606 >> 172.21.15.123:0/1795299606 conn(0x5652cddb1200 msgr2=0x5652cddbee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:59.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.226+0000 7f6cefdf0740 1 -- 172.21.15.123:0/1795299606 shutdown_connections 2024-09-19T04:49:59.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.226+0000 7f6cefdf0740 1 -- 172.21.15.123:0/1795299606 wait complete. 2024-09-19T04:49:59.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.226+0000 7f6cefdf0740 1 -- start start 2024-09-19T04:49:59.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.226+0000 7f6cefdf0740 1 Processor -- start 2024-09-19T04:49:59.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.227+0000 7f6cefdf0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb4c70 0x5652cddb9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.227+0000 7f6cefdf0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddb5590 0x5652cddba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.227+0000 7f6cefdf0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddbfe50 0x5652cddba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.227+0000 7f6cefdf0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce81b0dd0 con 0x5652cddb4c70 2024-09-19T04:49:59.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.227+0000 7f6cefdf0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce81b0700 con 0x5652cddb5590 2024-09-19T04:49:59.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.227+0000 7f6cefdf0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce81ad8b0 con 0x5652cddbfe50 2024-09-19T04:49:59.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.227+0000 7f6cefdf0740 10 client.0 _ll_register_callbacks cb 0x7f6cd00097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:49:59.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.227+0000 7f6ce5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb4c70 0x5652cddb9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:59.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.227+0000 7f6ce6800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddbfe50 0x5652cddba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:59.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.227+0000 7f6ce5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb4c70 0x5652cddb9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42360/0 (socket says 172.21.15.123:42360) 2024-09-19T04:49:59.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.227+0000 7f6ce5e00640 1 -- 172.21.15.123:0/938015222 learned_addr learned my addr 172.21.15.123:0/938015222 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:49:59.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.227+0000 7f6cdfe00640 1 --2- 172.21.15.123:0/938015222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddb5590 0x5652cddba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:59.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.227+0000 7f6ce5e00640 1 -- 172.21.15.123:0/938015222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddbfe50 msgr2=0x5652cddba8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:59.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.227+0000 7f6ce5e00640 1 --2- 172.21.15.123:0/938015222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddbfe50 0x5652cddba8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.228+0000 7f6ce5e00640 1 -- 172.21.15.123:0/938015222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddb5590 msgr2=0x5652cddba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:49:59.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.228+0000 7f6ce5e00640 1 --2- 172.21.15.123:0/938015222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddb5590 0x5652cddba3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.228+0000 7f6ce5e00640 1 -- 172.21.15.123:0/938015222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652cde39960 con 0x5652cddb4c70 2024-09-19T04:49:59.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.228+0000 7f6cdfe00640 1 --2- 172.21.15.123:0/938015222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddb5590 0x5652cddba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:49:59.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.228+0000 7f6ce5e00640 1 --2- 172.21.15.123:0/938015222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb4c70 0x5652cddb9e60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6cd809b940 tx=0x7f6cd809b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec62b6ced830e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:49:59.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.228+0000 7f6ccbe00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:49:59.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.228+0000 7f6ccbe00640 1 -- 172.21.15.123:0/938015222 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8020070 con 0x5652cddb4c70 2024-09-19T04:49:59.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.228+0000 7f6ccbe00640 1 -- 172.21.15.123:0/938015222 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f6cd80cd330 con 0x5652cddb4c70 2024-09-19T04:49:59.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.228+0000 7f6cefdf0740 1 -- 172.21.15.123:0/938015222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cd809fff0 con 0x5652cddb4c70 2024-09-19T04:49:59.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.228+0000 7f6ccbe00640 1 -- 172.21.15.123:0/938015222 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6cd809b090 con 0x5652cddb4c70 2024-09-19T04:49:59.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.228+0000 7f6cefdf0740 10 client.15900 fetch_fsmap learned FSMap version 31 2024-09-19T04:49:59.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.228+0000 7f6cefdf0740 1 -- 172.21.15.123:0/938015222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5652cde3ac20 con 0x5652cddb4c70 2024-09-19T04:49:59.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.229+0000 7f6ccbe00640 1 -- 172.21.15.123:0/938015222 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd809fff0 con 0x5652cddb4c70 2024-09-19T04:49:59.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.229+0000 7f6ccbe00640 1 -- 172.21.15.123:0/938015222 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f6cd8097b20 con 0x5652cddb4c70 2024-09-19T04:49:59.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.229+0000 7f6cefdf0740 10 client.15900 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:49:59.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.229+0000 7f6cefdf0740 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:49:59.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.229+0000 7f6cefdf0740 20 client.15900 populate_metadata read hostname 'smithi123' 2024-09-19T04:49:59.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.229+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/938015222 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5652cde3b410 0x5652cde3d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:49:59.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.229+0000 7f6cefdf0740 4 client.15900 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:49:59.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.229+0000 7f6cefdf0740 1 -- 172.21.15.123:0/938015222 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652cdd22e60 con 0x5652cde3b410 2024-09-19T04:49:59.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.229+0000 7f6cdfe00640 1 --2- 172.21.15.123:0/938015222 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5652cde3b410 0x5652cde3d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:49:59.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.230+0000 7f6cdfe00640 1 --2- 172.21.15.123:0/938015222 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5652cde3b410 0x5652cde3d830 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c905972177f732df server_cookie=106f31f726b628e8 in_seq=0 out_seq=0 2024-09-19T04:49:59.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.230+0000 7f6ccbe00640 10 client.15900 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:59.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.230+0000 7f6ccbe00640 1 -- 172.21.15.123:0/938015222 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652cdd22e60 con 0x5652cde3b410 2024-09-19T04:49:59.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.230+0000 7f6ccbe00640 10 client.15900 handle_command_reply: tid=0 2024-09-19T04:49:59.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.230+0000 7f6cefdf0740 1 client.15900 shutdown 2024-09-19T04:49:59.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.230+0000 7f6cefdf0740 1 -- 172.21.15.123:0/938015222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb4c70 msgr2=0x5652cddb9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:59.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.230+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/938015222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb4c70 0x5652cddb9e60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6cd809b940 tx=0x7f6cd809b5a0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.230+0000 7f6cefdf0740 1 -- 172.21.15.123:0/938015222 shutdown_connections 2024-09-19T04:49:59.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.230+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/938015222 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5652cde3b410 0x5652cde3d830 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.230+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/938015222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652cddbfe50 0x5652cddba8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.230+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/938015222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652cddb5590 0x5652cddba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.230+0000 7f6cefdf0740 1 --2- 172.21.15.123:0/938015222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652cddb4c70 0x5652cddb9e60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.230+0000 7f6ccbe00640 0 client.15900 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:49:59.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.230+0000 7f6cefdf0740 1 -- 172.21.15.123:0/938015222 >> 172.21.15.123:0/938015222 conn(0x5652cddb1200 msgr2=0x5652cddb66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:59.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.231+0000 7f6cefdf0740 1 -- 172.21.15.123:0/938015222 shutdown_connections 2024-09-19T04:49:59.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.231+0000 7f6cefdf0740 1 -- 172.21.15.123:0/938015222 wait complete. 2024-09-19T04:49:59.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.231+0000 7f6cefdf0740 20 client.15900 trim_cache size 0 max 16384 2024-09-19T04:49:59.237 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:49:59.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.232+0000 7f6ced200640 1 -- 172.21.15.123:0/3966790165 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ccc0a33b0 msgr2=0x7f6ccc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:59.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.232+0000 7f6ced200640 1 --2- 172.21.15.123:0/3966790165 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ccc0a33b0 0x7f6ccc0a5870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0046000 tx=0x7f6ce0048000 comp rx=0 tx=0).stop 2024-09-19T04:49:59.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.232+0000 7f6ced200640 1 -- 172.21.15.123:0/3966790165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ce8103ff0 msgr2=0x7f6ce819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:49:59.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.232+0000 7f6ced200640 1 --2- 172.21.15.123:0/3966790165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ce8103ff0 0x7f6ce819a590 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800d950 tx=0x7f6cd800dd50 comp rx=0 tx=0).stop 2024-09-19T04:49:59.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.232+0000 7f6ced200640 1 -- 172.21.15.123:0/3966790165 shutdown_connections 2024-09-19T04:49:59.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.232+0000 7f6ced200640 1 --2- 172.21.15.123:0/3966790165 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ccc0a33b0 0x7f6ccc0a5870 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.232+0000 7f6ced200640 1 --2- 172.21.15.123:0/3966790165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ce8109120 0x7f6ce819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.232+0000 7f6ced200640 1 --2- 172.21.15.123:0/3966790165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce81049a0 0x7f6ce819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.232+0000 7f6ced200640 1 --2- 172.21.15.123:0/3966790165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ce8103ff0 0x7f6ce819a590 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:49:59.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.232+0000 7f6ced200640 1 -- 172.21.15.123:0/3966790165 >> 172.21.15.123:0/3966790165 conn(0x7f6ce80ffa90 msgr2=0x7f6ce8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:49:59.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.232+0000 7f6ced200640 1 -- 172.21.15.123:0/3966790165 shutdown_connections 2024-09-19T04:49:59.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:49:59.232+0000 7f6ced200640 1 -- 172.21.15.123:0/3966790165 wait complete. 2024-09-19T04:49:59.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:49:59 smithi123 ceph-mon[30511]: pgmap v101: 97 pgs: 97 active+clean; 101 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 596 op/s 2024-09-19T04:50:00.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:49:59 smithi183 ceph-mon[35333]: pgmap v101: 97 pgs: 97 active+clean; 101 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 596 op/s 2024-09-19T04:50:00.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:49:59 smithi193 ceph-mon[35592]: pgmap v101: 97 pgs: 97 active+clean; 101 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 596 op/s 2024-09-19T04:50:01.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:00 smithi183 ceph-mon[35333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:50:01.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:00 smithi183 ceph-mon[35333]: overall HEALTH_OK 2024-09-19T04:50:01.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:00 smithi123 ceph-mon[30511]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:50:01.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:00 smithi123 ceph-mon[30511]: overall HEALTH_OK 2024-09-19T04:50:01.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:00 smithi193 ceph-mon[35592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:50:01.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:00 smithi193 ceph-mon[35592]: overall HEALTH_OK 2024-09-19T04:50:01.256 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:50:01.256 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:50:01.694 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:50:02.070 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:01 smithi123 ceph-mon[30511]: pgmap v102: 97 pgs: 97 active+clean; 101 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 386 op/s 2024-09-19T04:50:02.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:01 smithi183 ceph-mon[35333]: pgmap v102: 97 pgs: 97 active+clean; 101 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 386 op/s 2024-09-19T04:50:02.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:01 smithi193 ceph-mon[35592]: pgmap v102: 97 pgs: 97 active+clean; 101 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 386 op/s 2024-09-19T04:50:02.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.317+0000 7f4614000640 1 -- 172.21.15.123:0/3549097164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f460c074620 msgr2=0x7f460c074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:02.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.317+0000 7f4614000640 1 --2- 172.21.15.123:0/3549097164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f460c074620 0x7f460c074a00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4600009900 tx=0x7f460002a480 comp rx=0 tx=0).stop 2024-09-19T04:50:02.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.317+0000 7f4614000640 1 -- 172.21.15.123:0/3549097164 shutdown_connections 2024-09-19T04:50:02.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.317+0000 7f4614000640 1 --2- 172.21.15.123:0/3549097164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c10e3c0 0x7f460c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.317+0000 7f4614000640 1 --2- 172.21.15.123:0/3549097164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f460c074f40 0x7f460c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.317+0000 7f4614000640 1 --2- 172.21.15.123:0/3549097164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f460c074620 0x7f460c074a00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.317+0000 7f4614000640 1 -- 172.21.15.123:0/3549097164 >> 172.21.15.123:0/3549097164 conn(0x7f460c0fd8c0 msgr2=0x7f460c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:02.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.318+0000 7f4614000640 1 -- 172.21.15.123:0/3549097164 shutdown_connections 2024-09-19T04:50:02.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.318+0000 7f4614000640 1 -- 172.21.15.123:0/3549097164 wait complete. 2024-09-19T04:50:02.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.319+0000 7f4614000640 1 Processor -- start 2024-09-19T04:50:02.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.319+0000 7f4614000640 1 -- start start 2024-09-19T04:50:02.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.320+0000 7f4614000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f460c074620 0x7f460c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.320+0000 7f4614000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c074f40 0x7f460c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.320+0000 7f4614000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f460c10e3c0 0x7f460c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.320+0000 7f4614000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f460c114150 con 0x7f460c074620 2024-09-19T04:50:02.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.320+0000 7f4614000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f460c113fd0 con 0x7f460c074f40 2024-09-19T04:50:02.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.320+0000 7f4614000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1142d0 con 0x7f460c10e3c0 2024-09-19T04:50:02.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.320+0000 7f4612000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f460c10e3c0 0x7f460c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.320+0000 7f4611600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f460c074620 0x7f460c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.320+0000 7f4612000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f460c10e3c0 0x7f460c19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42670/0 (socket says 172.21.15.123:42670) 2024-09-19T04:50:02.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.320+0000 7f4612000640 1 -- 172.21.15.123:0/3424136996 learned_addr learned my addr 172.21.15.123:0/3424136996 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:02.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.320+0000 7f4610c00640 1 --2- 172.21.15.123:0/3424136996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c074f40 0x7f460c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.320+0000 7f4611600640 1 -- 172.21.15.123:0/3424136996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f460c10e3c0 msgr2=0x7f460c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:02.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.320+0000 7f4611600640 1 --2- 172.21.15.123:0/3424136996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f460c10e3c0 0x7f460c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.320+0000 7f4611600640 1 -- 172.21.15.123:0/3424136996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c074f40 msgr2=0x7f460c19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:02.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.320+0000 7f4611600640 1 --2- 172.21.15.123:0/3424136996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c074f40 0x7f460c19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.320+0000 7f4611600640 1 -- 172.21.15.123:0/3424136996 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460c19f660 con 0x7f460c074620 2024-09-19T04:50:02.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.320+0000 7f4610c00640 1 --2- 172.21.15.123:0/3424136996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c074f40 0x7f460c19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:02.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.321+0000 7f4611600640 1 --2- 172.21.15.123:0/3424136996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f460c074620 0x7f460c19a6b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4600005940 tx=0x7f46000040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:02.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.321+0000 7f460a000640 1 -- 172.21.15.123:0/3424136996 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460003f030 con 0x7f460c074620 2024-09-19T04:50:02.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.321+0000 7f460a000640 1 -- 172.21.15.123:0/3424136996 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46000044c0 con 0x7f460c074620 2024-09-19T04:50:02.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.321+0000 7f4614000640 1 -- 172.21.15.123:0/3424136996 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460c19f8f0 con 0x7f460c074620 2024-09-19T04:50:02.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.321+0000 7f4614000640 1 -- 172.21.15.123:0/3424136996 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f460c1a7230 con 0x7f460c074620 2024-09-19T04:50:02.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.321+0000 7f460a000640 1 -- 172.21.15.123:0/3424136996 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4600035070 con 0x7f460c074620 2024-09-19T04:50:02.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.325+0000 7f4616bdf740 1 Processor -- start 2024-09-19T04:50:02.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.325+0000 7f4616bdf740 1 -- start start 2024-09-19T04:50:02.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.326+0000 7f4616bdf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 0x56122fb6bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.326+0000 7f4616bdf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb6c3e0 0x56122fb76760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.326+0000 7f460a000640 1 -- 172.21.15.123:0/3424136996 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4600042030 con 0x7f460c074620 2024-09-19T04:50:02.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.326+0000 7f4616bdf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb76ca0 0x56122fb79090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.326+0000 7f4616bdf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56122fad2290 con 0x56122fb76ca0 2024-09-19T04:50:02.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.326+0000 7f4616bdf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56122fa79b40 con 0x56122fb6c3e0 2024-09-19T04:50:02.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.326+0000 7f4616bdf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56122face250 con 0x56122fb6bac0 2024-09-19T04:50:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.326+0000 7f4611600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb6c3e0 0x56122fb76760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.326+0000 7f4612000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 0x56122fb6bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.326+0000 7f4612000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 0x56122fb6bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42686/0 (socket says 172.21.15.123:42686) 2024-09-19T04:50:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.326+0000 7f4611600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb6c3e0 0x56122fb76760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:41518/0 (socket says 172.21.15.123:41518) 2024-09-19T04:50:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.326+0000 7f4612000640 1 -- 172.21.15.123:0/2473123456 learned_addr learned my addr 172.21.15.123:0/2473123456 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.326+0000 7f4610c00640 1 --2- 172.21.15.123:0/2473123456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb76ca0 0x56122fb79090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.327+0000 7f4611600640 1 -- 172.21.15.123:0/2473123456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 msgr2=0x56122fb6bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.327+0000 7f4611600640 1 --2- 172.21.15.123:0/2473123456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 0x56122fb6bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.327+0000 7f4611600640 1 -- 172.21.15.123:0/2473123456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb76ca0 msgr2=0x56122fb79090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.327+0000 7f4611600640 1 --2- 172.21.15.123:0/2473123456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb76ca0 0x56122fb79090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.327+0000 7f4611600640 1 -- 172.21.15.123:0/2473123456 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56122f8436a0 con 0x56122fb6c3e0 2024-09-19T04:50:02.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.327+0000 7f4610c00640 1 --2- 172.21.15.123:0/2473123456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb76ca0 0x56122fb79090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:50:02.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.327+0000 7f4611600640 1 --2- 172.21.15.123:0/2473123456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb6c3e0 0x56122fb76760 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f46000ed500 tx=0x7f46000f3f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4d20996ffa27570 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:02.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.327+0000 7f45fe600640 1 -- 172.21.15.123:0/2473123456 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46000f6be0 con 0x56122fb6c3e0 2024-09-19T04:50:02.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.327+0000 7f45fe600640 1 -- 172.21.15.123:0/2473123456 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46000f6d80 con 0x56122fb6c3e0 2024-09-19T04:50:02.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.327+0000 7f45fe600640 1 -- 172.21.15.123:0/2473123456 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4600100410 con 0x56122fb6c3e0 2024-09-19T04:50:02.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.328+0000 7f460a000640 1 --2- 172.21.15.123:0/3424136996 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f45f40a3500 0x7f45f40a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.328+0000 7f460a000640 1 -- 172.21.15.123:0/3424136996 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f46000efde0 con 0x7f460c074620 2024-09-19T04:50:02.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.328+0000 7f4616bdf740 1 -- 172.21.15.123:0/2473123456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb6c3e0 msgr2=0x56122fb76760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:02.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.328+0000 7f4616bdf740 1 --2- 172.21.15.123:0/2473123456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb6c3e0 0x56122fb76760 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f46000ed500 tx=0x7f46000f3f90 comp rx=0 tx=0).stop 2024-09-19T04:50:02.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.328+0000 7f4616bdf740 1 -- 172.21.15.123:0/2473123456 shutdown_connections 2024-09-19T04:50:02.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.328+0000 7f4616bdf740 1 --2- 172.21.15.123:0/2473123456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb76ca0 0x56122fb79090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.328+0000 7f4616bdf740 1 --2- 172.21.15.123:0/2473123456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb6c3e0 0x56122fb76760 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.328+0000 7f4616bdf740 1 --2- 172.21.15.123:0/2473123456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 0x56122fb6bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.328+0000 7f4616bdf740 1 -- 172.21.15.123:0/2473123456 >> 172.21.15.123:0/2473123456 conn(0x56122fb68050 msgr2=0x56122fb68430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:02.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.329+0000 7f4612000640 1 --2- 172.21.15.123:0/3424136996 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f45f40a3500 0x7f45f40a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.329+0000 7f4616bdf740 1 -- 172.21.15.123:0/2473123456 shutdown_connections 2024-09-19T04:50:02.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.329+0000 7f4616bdf740 1 -- 172.21.15.123:0/2473123456 wait complete. 2024-09-19T04:50:02.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.330+0000 7f4616bdf740 1 -- start start 2024-09-19T04:50:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.330+0000 7f4616bdf740 1 Processor -- start 2024-09-19T04:50:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.330+0000 7f4612000640 1 --2- 172.21.15.123:0/3424136996 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f45f40a3500 0x7f45f40a59c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4604005c40 tx=0x7f4604009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.330+0000 7f4616bdf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 0x56122fb76500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.330+0000 7f4616bdf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb6c3e0 0x56122fb73800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.330+0000 7f4616bdf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb76ca0 0x56122fb73d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.331+0000 7f4616bdf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56122fb79630 con 0x56122fb76ca0 2024-09-19T04:50:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.331+0000 7f4616bdf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56122fa8cfa0 con 0x56122fb6c3e0 2024-09-19T04:50:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.331+0000 7f4616bdf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56122fb797b0 con 0x56122fb6bac0 2024-09-19T04:50:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.331+0000 7f4616bdf740 10 client.0 _ll_register_callbacks cb 0x56122f7e0ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.331+0000 7f4610c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb6c3e0 0x56122fb73800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.331+0000 7f4610c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb6c3e0 0x56122fb73800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:41524/0 (socket says 172.21.15.123:41524) 2024-09-19T04:50:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.331+0000 7f4610c00640 1 -- 172.21.15.123:0/3938602549 learned_addr learned my addr 172.21.15.123:0/3938602549 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.331+0000 7f4611600640 1 --2- 172.21.15.123:0/3938602549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 0x56122fb76500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.331+0000 7f4612000640 1 --2- 172.21.15.123:0/3938602549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb76ca0 0x56122fb73d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.331+0000 7f4610c00640 1 -- 172.21.15.123:0/3938602549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 msgr2=0x56122fb76500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.331+0000 7f4610c00640 1 --2- 172.21.15.123:0/3938602549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 0x56122fb76500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.331+0000 7f4610c00640 1 -- 172.21.15.123:0/3938602549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb76ca0 msgr2=0x56122fb73d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.331+0000 7f4610c00640 1 --2- 172.21.15.123:0/3938602549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb76ca0 0x56122fb73d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.331+0000 7f4610c00640 1 -- 172.21.15.123:0/3938602549 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122fb74540 con 0x56122fb6c3e0 2024-09-19T04:50:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.331+0000 7f4611600640 1 --2- 172.21.15.123:0/3938602549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 0x56122fb76500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.332+0000 7f4610c00640 1 --2- 172.21.15.123:0/3938602549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb6c3e0 0x56122fb73800 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f45f8012d20 tx=0x7f45f800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53167e7487384c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.332+0000 7f45f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:50:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.332+0000 7f45f2a00640 1 -- 172.21.15.123:0/3938602549 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f80120d0 con 0x56122fb6c3e0 2024-09-19T04:50:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.332+0000 7f45f2a00640 1 -- 172.21.15.123:0/3938602549 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f45f8004020 con 0x56122fb6c3e0 2024-09-19T04:50:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.332+0000 7f4616bdf740 1 -- 172.21.15.123:0/3938602549 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46000f6d80 con 0x56122fb6c3e0 2024-09-19T04:50:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.332+0000 7f45f2a00640 1 -- 172.21.15.123:0/3938602549 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f45f8012270 con 0x56122fb6c3e0 2024-09-19T04:50:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.332+0000 7f4616bdf740 10 client.25976 fetch_fsmap learned FSMap version 32 2024-09-19T04:50:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.332+0000 7f4616bdf740 1 -- 172.21.15.123:0/3938602549 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56122fb8dde0 con 0x56122fb6c3e0 2024-09-19T04:50:02.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.333+0000 7f45f2a00640 1 -- 172.21.15.123:0/3938602549 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46000f6d80 con 0x56122fb6c3e0 2024-09-19T04:50:02.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.333+0000 7f45f2a00640 1 -- 172.21.15.123:0/3938602549 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f45f801c470 con 0x56122fb6c3e0 2024-09-19T04:50:02.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.333+0000 7f4616bdf740 10 client.25976 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T04:50:02.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.333+0000 7f4616bdf740 10 client.25976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:50:02.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.333+0000 7f4616bdf740 20 client.25976 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:02.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.333+0000 7f4616bdf740 1 --2- 172.21.15.123:0/3938602549 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56122fb8f1f0 0x56122fb91640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.333+0000 7f4616bdf740 4 client.25976 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:50:02.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.333+0000 7f4616bdf740 1 -- 172.21.15.123:0/3938602549 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56122fadf800 con 0x56122fb8f1f0 2024-09-19T04:50:02.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.333+0000 7f4611600640 1 --2- 172.21.15.123:0/3938602549 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56122fb8f1f0 0x56122fb91640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.334+0000 7f4611600640 1 --2- 172.21.15.123:0/3938602549 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56122fb8f1f0 0x56122fb91640 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d887b18e8718ad49 server_cookie=f82b1f684d8d5e67 in_seq=0 out_seq=0 2024-09-19T04:50:02.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.334+0000 7f45f2a00640 10 client.25976 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:02.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.335+0000 7f45f2a00640 1 -- 172.21.15.123:0/3938602549 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56122fadf800 con 0x56122fb8f1f0 2024-09-19T04:50:02.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.335+0000 7f45f2a00640 10 client.25976 handle_command_reply: tid=0 2024-09-19T04:50:02.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.335+0000 7f4616bdf740 1 client.25976 shutdown 2024-09-19T04:50:02.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.335+0000 7f4616bdf740 1 -- 172.21.15.123:0/3938602549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb6c3e0 msgr2=0x56122fb73800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:02.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.335+0000 7f4616bdf740 1 --2- 172.21.15.123:0/3938602549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb6c3e0 0x56122fb73800 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f45f8012d20 tx=0x7f45f800f790 comp rx=0 tx=0).stop 2024-09-19T04:50:02.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.336+0000 7f4616bdf740 1 -- 172.21.15.123:0/3938602549 shutdown_connections 2024-09-19T04:50:02.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.336+0000 7f4616bdf740 1 --2- 172.21.15.123:0/3938602549 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56122fb8f1f0 0x56122fb91640 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.336+0000 7f4616bdf740 1 --2- 172.21.15.123:0/3938602549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb76ca0 0x56122fb73d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.336+0000 7f4616bdf740 1 --2- 172.21.15.123:0/3938602549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb6c3e0 0x56122fb73800 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.336+0000 7f4616bdf740 1 --2- 172.21.15.123:0/3938602549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 0x56122fb76500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.336+0000 7f4616bdf740 1 -- 172.21.15.123:0/3938602549 >> 172.21.15.123:0/3938602549 conn(0x56122fb68050 msgr2=0x56122fb671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:02.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.336+0000 7f45f2a00640 0 client.25976 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:02.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.336+0000 7f4616bdf740 1 -- 172.21.15.123:0/3938602549 shutdown_connections 2024-09-19T04:50:02.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.336+0000 7f4616bdf740 1 -- 172.21.15.123:0/3938602549 wait complete. 2024-09-19T04:50:02.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.336+0000 7f4616bdf740 20 client.25976 trim_cache size 0 max 16384 2024-09-19T04:50:02.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.358+0000 7f4616bdf740 1 Processor -- start 2024-09-19T04:50:02.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.358+0000 7f4616bdf740 1 -- start start 2024-09-19T04:50:02.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.359+0000 7f4616bdf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb68050 0x56122fb75770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.359+0000 7f4616bdf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb6bac0 0x56122fb96f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.359+0000 7f4616bdf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6c3e0 0x56122fb97490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.359+0000 7f4616bdf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec00e6d0 con 0x56122fb6bac0 2024-09-19T04:50:02.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.359+0000 7f4616bdf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec0102d0 con 0x56122fb68050 2024-09-19T04:50:02.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.359+0000 7f4616bdf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec00eda0 con 0x56122fb6c3e0 2024-09-19T04:50:02.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.359+0000 7f4610c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb6bac0 0x56122fb96f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.359+0000 7f4610c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb6bac0 0x56122fb96f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60648/0 (socket says 172.21.15.123:60648) 2024-09-19T04:50:02.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.359+0000 7f4610c00640 1 -- 172.21.15.123:0/1505910700 learned_addr learned my addr 172.21.15.123:0/1505910700 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:02.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.359+0000 7f4612000640 1 --2- 172.21.15.123:0/1505910700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6c3e0 0x56122fb97490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.359+0000 7f4611600640 1 --2- 172.21.15.123:0/1505910700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb68050 0x56122fb75770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.360+0000 7f4611600640 1 -- 172.21.15.123:0/1505910700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6c3e0 msgr2=0x56122fb97490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:02.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.360+0000 7f4611600640 1 --2- 172.21.15.123:0/1505910700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6c3e0 0x56122fb97490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.360+0000 7f4611600640 1 -- 172.21.15.123:0/1505910700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb6bac0 msgr2=0x56122fb96f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.360+0000 7f4611600640 1 --2- 172.21.15.123:0/1505910700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb6bac0 0x56122fb96f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.360+0000 7f4611600640 1 -- 172.21.15.123:0/1505910700 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56122fb727b0 con 0x56122fb68050 2024-09-19T04:50:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.360+0000 7f4610c00640 1 --2- 172.21.15.123:0/1505910700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb6bac0 0x56122fb96f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.360+0000 7f4612000640 1 --2- 172.21.15.123:0/1505910700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6c3e0 0x56122fb97490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.360+0000 7f4611600640 1 --2- 172.21.15.123:0/1505910700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb68050 0x56122fb75770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f46000ed560 tx=0x7f46000f4e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72189d8713bb5b4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.360+0000 7f45fdc00640 1 -- 172.21.15.123:0/1505910700 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46000b8330 con 0x56122fb68050 2024-09-19T04:50:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.360+0000 7f45fdc00640 1 -- 172.21.15.123:0/1505910700 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46000bc790 con 0x56122fb68050 2024-09-19T04:50:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.360+0000 7f45fdc00640 1 -- 172.21.15.123:0/1505910700 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46000bb9f0 con 0x56122fb68050 2024-09-19T04:50:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.361+0000 7f4616bdf740 1 -- 172.21.15.123:0/1505910700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb68050 msgr2=0x56122fb75770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:02.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.361+0000 7f4616bdf740 1 --2- 172.21.15.123:0/1505910700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb68050 0x56122fb75770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f46000ed560 tx=0x7f46000f4e00 comp rx=0 tx=0).stop 2024-09-19T04:50:02.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.362+0000 7f4616bdf740 1 -- 172.21.15.123:0/1505910700 shutdown_connections 2024-09-19T04:50:02.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.362+0000 7f4616bdf740 1 --2- 172.21.15.123:0/1505910700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6c3e0 0x56122fb97490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.362+0000 7f4616bdf740 1 --2- 172.21.15.123:0/1505910700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb6bac0 0x56122fb96f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.362+0000 7f4616bdf740 1 --2- 172.21.15.123:0/1505910700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb68050 0x56122fb75770 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.362+0000 7f4616bdf740 1 -- 172.21.15.123:0/1505910700 >> 172.21.15.123:0/1505910700 conn(0x56122fb73ea0 msgr2=0x56122fb75060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:02.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.362+0000 7f4616bdf740 1 -- 172.21.15.123:0/1505910700 shutdown_connections 2024-09-19T04:50:02.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.362+0000 7f4616bdf740 1 -- 172.21.15.123:0/1505910700 wait complete. 2024-09-19T04:50:02.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.362+0000 7f4616bdf740 1 -- start start 2024-09-19T04:50:02.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.362+0000 7f4616bdf740 1 Processor -- start 2024-09-19T04:50:02.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.363+0000 7f4616bdf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb68050 0x56122fb87980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.363+0000 7f4616bdf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 0x56122fb83cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.363+0000 7f4616bdf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb6c3e0 0x56122fb84200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.363+0000 7f4616bdf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec00fbd0 con 0x56122fb6c3e0 2024-09-19T04:50:02.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.363+0000 7f4616bdf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec00f500 con 0x56122fb68050 2024-09-19T04:50:02.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.363+0000 7f4616bdf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec00c6b0 con 0x56122fb6bac0 2024-09-19T04:50:02.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.363+0000 7f4616bdf740 10 client.0 _ll_register_callbacks cb 0x7f45f8009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:02.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.363+0000 7f4610c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 0x56122fb83cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.364+0000 7f4611600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb68050 0x56122fb87980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.364+0000 7f4612000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb6c3e0 0x56122fb84200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.364+0000 7f4611600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb68050 0x56122fb87980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:41540/0 (socket says 172.21.15.123:41540) 2024-09-19T04:50:02.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.364+0000 7f4611600640 1 -- 172.21.15.123:0/2393144620 learned_addr learned my addr 172.21.15.123:0/2393144620 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:02.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.364+0000 7f4610c00640 1 -- 172.21.15.123:0/2393144620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb68050 msgr2=0x56122fb87980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:02.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.364+0000 7f4610c00640 1 --2- 172.21.15.123:0/2393144620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb68050 0x56122fb87980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.364+0000 7f4610c00640 1 -- 172.21.15.123:0/2393144620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb6c3e0 msgr2=0x56122fb84200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:02.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.364+0000 7f4610c00640 1 --2- 172.21.15.123:0/2393144620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb6c3e0 0x56122fb84200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.364+0000 7f4610c00640 1 -- 172.21.15.123:0/2393144620 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122fb84a30 con 0x56122fb6bac0 2024-09-19T04:50:02.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.364+0000 7f4612000640 1 --2- 172.21.15.123:0/2393144620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb6c3e0 0x56122fb84200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:02.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.364+0000 7f4610c00640 1 --2- 172.21.15.123:0/2393144620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 0x56122fb83cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f45f8000c00 tx=0x7f45f801cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97474197312be303 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:02.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.365+0000 7f45f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:50:02.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.365+0000 7f45f2a00640 1 -- 172.21.15.123:0/2393144620 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f800fc90 con 0x56122fb6bac0 2024-09-19T04:50:02.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.365+0000 7f45f2a00640 1 -- 172.21.15.123:0/2393144620 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f45f8005250 con 0x56122fb6bac0 2024-09-19T04:50:02.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.365+0000 7f4616bdf740 1 -- 172.21.15.123:0/2393144620 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46000bc790 con 0x56122fb6bac0 2024-09-19T04:50:02.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.365+0000 7f45f2a00640 1 -- 172.21.15.123:0/2393144620 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f45f8036050 con 0x56122fb6bac0 2024-09-19T04:50:02.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.365+0000 7f4616bdf740 10 client.26005 fetch_fsmap learned FSMap version 32 2024-09-19T04:50:02.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.365+0000 7f4616bdf740 1 -- 172.21.15.123:0/2393144620 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56122fbfae80 con 0x56122fb6bac0 2024-09-19T04:50:02.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.366+0000 7f45f2a00640 1 -- 172.21.15.123:0/2393144620 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46000bc790 con 0x56122fb6bac0 2024-09-19T04:50:02.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.366+0000 7f45f2a00640 1 -- 172.21.15.123:0/2393144620 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f45f8003cd0 con 0x56122fb6bac0 2024-09-19T04:50:02.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.366+0000 7f4616bdf740 10 client.26005 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T04:50:02.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.366+0000 7f4616bdf740 10 client.26005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:50:02.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.366+0000 7f4616bdf740 20 client.26005 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:02.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.366+0000 7f4616bdf740 1 --2- 172.21.15.123:0/2393144620 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56122fb76ca0 0x56122fbfd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:02.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.366+0000 7f4616bdf740 4 client.26005 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:50:02.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.366+0000 7f4616bdf740 1 -- 172.21.15.123:0/2393144620 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56122fad9d10 con 0x56122fb76ca0 2024-09-19T04:50:02.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.366+0000 7f4611600640 1 --2- 172.21.15.123:0/2393144620 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56122fb76ca0 0x56122fbfd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:02.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.367+0000 7f4611600640 1 --2- 172.21.15.123:0/2393144620 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56122fb76ca0 0x56122fbfd630 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5189e29f21103af server_cookie=c8d200910301bb65 in_seq=0 out_seq=0 2024-09-19T04:50:02.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.367+0000 7f45f2a00640 10 client.26005 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:02.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.368+0000 7f45f2a00640 1 -- 172.21.15.123:0/2393144620 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56122fad9d10 con 0x56122fb76ca0 2024-09-19T04:50:02.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.368+0000 7f45f2a00640 10 client.26005 handle_command_reply: tid=0 2024-09-19T04:50:02.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.368+0000 7f4616bdf740 1 client.26005 shutdown 2024-09-19T04:50:02.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.368+0000 7f4616bdf740 1 -- 172.21.15.123:0/2393144620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 msgr2=0x56122fb83cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:02.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.368+0000 7f4616bdf740 1 --2- 172.21.15.123:0/2393144620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 0x56122fb83cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f45f8000c00 tx=0x7f45f801cd60 comp rx=0 tx=0).stop 2024-09-19T04:50:02.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.368+0000 7f4616bdf740 1 -- 172.21.15.123:0/2393144620 shutdown_connections 2024-09-19T04:50:02.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.368+0000 7f4616bdf740 1 --2- 172.21.15.123:0/2393144620 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56122fb76ca0 0x56122fbfd630 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.368+0000 7f4616bdf740 1 --2- 172.21.15.123:0/2393144620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56122fb6c3e0 0x56122fb84200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.368+0000 7f4616bdf740 1 --2- 172.21.15.123:0/2393144620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56122fb6bac0 0x56122fb83cc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.368+0000 7f4616bdf740 1 --2- 172.21.15.123:0/2393144620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56122fb68050 0x56122fb87980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.368+0000 7f4616bdf740 1 -- 172.21.15.123:0/2393144620 >> 172.21.15.123:0/2393144620 conn(0x56122fb73ea0 msgr2=0x56122fb6dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:02.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.368+0000 7f45f2a00640 0 client.26005 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:02.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.369+0000 7f4616bdf740 1 -- 172.21.15.123:0/2393144620 shutdown_connections 2024-09-19T04:50:02.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.369+0000 7f4616bdf740 1 -- 172.21.15.123:0/2393144620 wait complete. 2024-09-19T04:50:02.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.369+0000 7f4616bdf740 20 client.26005 trim_cache size 0 max 16384 2024-09-19T04:50:02.374 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:50:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-09-19T04:50:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "bf83a384-d8dc-4e88-9f3c-a935743afe69", 2024-09-19T04:50:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-09-19T04:50:02.374 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:50:02.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.370+0000 7f4614000640 1 -- 172.21.15.123:0/3424136996 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f45f40a3500 msgr2=0x7f45f40a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:02.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.370+0000 7f4614000640 1 --2- 172.21.15.123:0/3424136996 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f45f40a3500 0x7f45f40a59c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4604005c40 tx=0x7f4604009500 comp rx=0 tx=0).stop 2024-09-19T04:50:02.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.370+0000 7f4614000640 1 -- 172.21.15.123:0/3424136996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f460c074620 msgr2=0x7f460c19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:02.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.370+0000 7f4614000640 1 --2- 172.21.15.123:0/3424136996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f460c074620 0x7f460c19a6b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4600005940 tx=0x7f46000040c0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.370+0000 7f4614000640 1 -- 172.21.15.123:0/3424136996 shutdown_connections 2024-09-19T04:50:02.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.370+0000 7f4614000640 1 --2- 172.21.15.123:0/3424136996 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f45f40a3500 0x7f45f40a59c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.370+0000 7f4614000640 1 --2- 172.21.15.123:0/3424136996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f460c10e3c0 0x7f460c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.370+0000 7f4614000640 1 --2- 172.21.15.123:0/3424136996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c074f40 0x7f460c19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.370+0000 7f4614000640 1 --2- 172.21.15.123:0/3424136996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f460c074620 0x7f460c19a6b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:02.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.370+0000 7f4614000640 1 -- 172.21.15.123:0/3424136996 >> 172.21.15.123:0/3424136996 conn(0x7f460c0fd8c0 msgr2=0x7f460c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:02.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.370+0000 7f4614000640 1 -- 172.21.15.123:0/3424136996 shutdown_connections 2024-09-19T04:50:02.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:02.370+0000 7f4614000640 1 -- 172.21.15.123:0/3424136996 wait complete. 2024-09-19T04:50:03.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:02 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:50:03.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:02 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:03.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:02 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:50:03.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:02 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:03.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:02 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:50:03.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:02 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:03.281 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:50:03.718 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:50:04.073 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:03 smithi123 ceph-mon[30511]: pgmap v103: 97 pgs: 97 active+clean; 101 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 386 op/s 2024-09-19T04:50:04.073 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:03 smithi123 ceph-mon[30511]: scrub queued for path: / 2024-09-19T04:50:04.073 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:03 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:04.073 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:03 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:50:04.073 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:03 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:50:04.073 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:03 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:04.073 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:03 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:50:04.073 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:03 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:04.073 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:03 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:50:04.073 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:03 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:04.099 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:03 smithi183 ceph-mon[35333]: pgmap v103: 97 pgs: 97 active+clean; 101 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 386 op/s 2024-09-19T04:50:04.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:03 smithi183 ceph-mon[35333]: scrub queued for path: / 2024-09-19T04:50:04.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:03 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:04.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:03 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:50:04.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:03 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:50:04.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:03 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:04.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:03 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:50:04.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:03 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:04.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:03 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:50:04.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:03 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:04.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:03 smithi193 ceph-mon[35592]: pgmap v103: 97 pgs: 97 active+clean; 101 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 386 op/s 2024-09-19T04:50:04.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:03 smithi193 ceph-mon[35592]: scrub queued for path: / 2024-09-19T04:50:04.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:03 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:04.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:03 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:50:04.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:03 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:50:04.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:03 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:04.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:03 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:50:04.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:03 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:04.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:03 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:50:04.241 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:03 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:04.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.337+0000 7fd2ca600640 1 -- 172.21.15.123:0/4058282795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2c41012e0 msgr2=0x7fd2c410dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:04.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.337+0000 7fd2ca600640 1 --2- 172.21.15.123:0/4058282795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2c41012e0 0x7fd2c410dfa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009900 tx=0x7fd2b002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.337+0000 7fd2ca600640 1 -- 172.21.15.123:0/4058282795 shutdown_connections 2024-09-19T04:50:04.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.337+0000 7fd2ca600640 1 --2- 172.21.15.123:0/4058282795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2c410e4e0 0x7fd2c41108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.337+0000 7fd2ca600640 1 --2- 172.21.15.123:0/4058282795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2c41012e0 0x7fd2c410dfa0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.337+0000 7fd2ca600640 1 --2- 172.21.15.123:0/4058282795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c4100930 0x7fd2c4100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.337+0000 7fd2ca600640 1 -- 172.21.15.123:0/4058282795 >> 172.21.15.123:0/4058282795 conn(0x7fd2c40fc7a0 msgr2=0x7fd2c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:04.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.338+0000 7fd2ca600640 1 -- 172.21.15.123:0/4058282795 shutdown_connections 2024-09-19T04:50:04.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.338+0000 7fd2ca600640 1 -- 172.21.15.123:0/4058282795 wait complete. 2024-09-19T04:50:04.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.339+0000 7fd2ca600640 1 Processor -- start 2024-09-19T04:50:04.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.339+0000 7fd2ca600640 1 -- start start 2024-09-19T04:50:04.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.339+0000 7fd2ca600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2c4100930 0x7fd2c41092e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.340+0000 7fd2ca600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c41012e0 0x7fd2c4109820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.340+0000 7fd2ca600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2c410e4e0 0x7fd2c4104890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.340+0000 7fd2ca600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41040d0 con 0x7fd2c41012e0 2024-09-19T04:50:04.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.340+0000 7fd2ca600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4103f50 con 0x7fd2c410e4e0 2024-09-19T04:50:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.340+0000 7fd2ca600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4104250 con 0x7fd2c4100930 2024-09-19T04:50:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.340+0000 7fd2c3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2c4100930 0x7fd2c41092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.340+0000 7fd2c3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2c4100930 0x7fd2c41092e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42734/0 (socket says 172.21.15.123:42734) 2024-09-19T04:50:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.340+0000 7fd2c3400640 1 -- 172.21.15.123:0/3404400956 learned_addr learned my addr 172.21.15.123:0/3404400956 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.340+0000 7fd2c3e00640 1 --2- 172.21.15.123:0/3404400956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2c410e4e0 0x7fd2c4104890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.340+0000 7fd2c3400640 1 -- 172.21.15.123:0/3404400956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2c410e4e0 msgr2=0x7fd2c4104890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.340+0000 7fd2c3400640 1 --2- 172.21.15.123:0/3404400956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2c410e4e0 0x7fd2c4104890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.340+0000 7fd2c3400640 1 -- 172.21.15.123:0/3404400956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c41012e0 msgr2=0x7fd2c4109820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.340+0000 7fd2c3400640 1 --2- 172.21.15.123:0/3404400956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c41012e0 0x7fd2c4109820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.340+0000 7fd2c3400640 1 -- 172.21.15.123:0/3404400956 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c4105150 con 0x7fd2c4100930 2024-09-19T04:50:04.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.341+0000 7fd2c3400640 1 --2- 172.21.15.123:0/3404400956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2c4100930 0x7fd2c41092e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400ea60 tx=0x7fd2b400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:04.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.341+0000 7fd2bbe00640 1 -- 172.21.15.123:0/3404400956 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b400cde0 con 0x7fd2c4100930 2024-09-19T04:50:04.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.341+0000 7fd2bbe00640 1 -- 172.21.15.123:0/3404400956 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2b40035f0 con 0x7fd2c4100930 2024-09-19T04:50:04.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.341+0000 7fd2bbe00640 1 -- 172.21.15.123:0/3404400956 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4010690 con 0x7fd2c4100930 2024-09-19T04:50:04.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.341+0000 7fd2ca600640 1 -- 172.21.15.123:0/3404400956 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c4105440 con 0x7fd2c4100930 2024-09-19T04:50:04.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.341+0000 7fd2ca600640 1 -- 172.21.15.123:0/3404400956 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c410d1c0 con 0x7fd2c4100930 2024-09-19T04:50:04.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.345+0000 7fd2cd0b1740 1 Processor -- start 2024-09-19T04:50:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.346+0000 7fd2cd0b1740 1 -- start start 2024-09-19T04:50:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.346+0000 7fd2cd0b1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7eb3b90 0x55a9c7eb3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.346+0000 7fd2bbe00640 1 -- 172.21.15.123:0/3404400956 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd2b40037a0 con 0x7fd2c4100930 2024-09-19T04:50:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.346+0000 7fd2cd0b1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7eb44b0 0x55a9c7ebe830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.346+0000 7fd2cd0b1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7ebed70 0x55a9c7ec1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.346+0000 7fd2cd0b1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a9c7e1a3e0 con 0x55a9c7ebed70 2024-09-19T04:50:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.346+0000 7fd2cd0b1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a9c7dc1d80 con 0x55a9c7eb3b90 2024-09-19T04:50:04.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.346+0000 7fd2cd0b1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a9c7e163a0 con 0x55a9c7eb44b0 2024-09-19T04:50:04.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.347+0000 7fd2c3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7eb3b90 0x55a9c7eb3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:04.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.347+0000 7fd2c3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7eb3b90 0x55a9c7eb3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:41570/0 (socket says 172.21.15.123:41570) 2024-09-19T04:50:04.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.347+0000 7fd2c3e00640 1 -- 172.21.15.123:0/164384567 learned_addr learned my addr 172.21.15.123:0/164384567 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:04.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.347+0000 7fd2c3e00640 1 -- 172.21.15.123:0/164384567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7eb44b0 msgr2=0x55a9c7ebe830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:04.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.347+0000 7fd2c3e00640 1 --2- 172.21.15.123:0/164384567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7eb44b0 0x55a9c7ebe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.347+0000 7fd2c3e00640 1 -- 172.21.15.123:0/164384567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7ebed70 msgr2=0x55a9c7ec1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:04.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.347+0000 7fd2c3e00640 1 --2- 172.21.15.123:0/164384567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7ebed70 0x55a9c7ec1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.347+0000 7fd2c3e00640 1 -- 172.21.15.123:0/164384567 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9c7de0ae0 con 0x55a9c7eb3b90 2024-09-19T04:50:04.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.347+0000 7fd2c3e00640 1 --2- 172.21.15.123:0/164384567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7eb3b90 0x55a9c7eb3f70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009e30 tx=0x7fd2bc010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c626ae0512487a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:04.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.348+0000 7fd2b8c00640 1 -- 172.21.15.123:0/164384567 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc016070 con 0x55a9c7eb3b90 2024-09-19T04:50:04.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.348+0000 7fd2b8c00640 1 -- 172.21.15.123:0/164384567 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2bc004d20 con 0x55a9c7eb3b90 2024-09-19T04:50:04.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.348+0000 7fd2b8c00640 1 -- 172.21.15.123:0/164384567 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc012740 con 0x55a9c7eb3b90 2024-09-19T04:50:04.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.348+0000 7fd2bbe00640 1 --2- 172.21.15.123:0/3404400956 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd2a80a3430 0x7fd2a80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.348+0000 7fd2bbe00640 1 -- 172.21.15.123:0/3404400956 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd2b40d0070 con 0x7fd2c4100930 2024-09-19T04:50:04.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.349+0000 7fd2c3400640 1 --2- 172.21.15.123:0/3404400956 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd2a80a3430 0x7fd2a80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:04.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.349+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/164384567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7eb3b90 msgr2=0x55a9c7eb3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:04.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.349+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/164384567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7eb3b90 0x55a9c7eb3f70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009e30 tx=0x7fd2bc010740 comp rx=0 tx=0).stop 2024-09-19T04:50:04.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.349+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/164384567 shutdown_connections 2024-09-19T04:50:04.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.349+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/164384567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7ebed70 0x55a9c7ec1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.349+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/164384567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7eb44b0 0x55a9c7ebe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.349+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/164384567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7eb3b90 0x55a9c7eb3f70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.349+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/164384567 >> 172.21.15.123:0/164384567 conn(0x55a9c7eb0120 msgr2=0x55a9c7eb0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:04.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.350+0000 7fd2c3400640 1 --2- 172.21.15.123:0/3404400956 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd2a80a3430 0x7fd2a80a58f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55a9c7eb0f80 tx=0x7fd2b4009520 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:04.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.350+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/164384567 shutdown_connections 2024-09-19T04:50:04.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.350+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/164384567 wait complete. 2024-09-19T04:50:04.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.351+0000 7fd2cd0b1740 1 -- start start 2024-09-19T04:50:04.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.351+0000 7fd2cd0b1740 1 Processor -- start 2024-09-19T04:50:04.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.352+0000 7fd2cd0b1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7eb3b90 0x55a9c7ebbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.352+0000 7fd2cd0b1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7eb44b0 0x55a9c7ebc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.352+0000 7fd2cd0b1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7ebed70 0x55a9c7ebc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.352+0000 7fd2cd0b1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a9c7ec19e0 con 0x55a9c7eb44b0 2024-09-19T04:50:04.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.352+0000 7fd2cd0b1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a9c7dd50f0 con 0x55a9c7ebed70 2024-09-19T04:50:04.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.352+0000 7fd2cd0b1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a9c7ec1b60 con 0x55a9c7eb3b90 2024-09-19T04:50:04.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.352+0000 7fd2cd0b1740 10 client.0 _ll_register_callbacks cb 0x55a9c7dc69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:04.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.352+0000 7fd2c3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7eb3b90 0x55a9c7ebbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:04.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.352+0000 7fd2c3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7eb3b90 0x55a9c7ebbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42750/0 (socket says 172.21.15.123:42750) 2024-09-19T04:50:04.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.352+0000 7fd2c3e00640 1 -- 172.21.15.123:0/2578516655 learned_addr learned my addr 172.21.15.123:0/2578516655 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:04.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.352+0000 7fd2c3e00640 1 --2- 172.21.15.123:0/2578516655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7ebed70 0x55a9c7ebc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:04.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.353+0000 7fd2c3e00640 1 -- 172.21.15.123:0/2578516655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7eb3b90 msgr2=0x55a9c7ebbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:04.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.353+0000 7fd2c3e00640 1 --2- 172.21.15.123:0/2578516655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7eb3b90 0x55a9c7ebbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.353+0000 7fd2c3e00640 1 -- 172.21.15.123:0/2578516655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7eb44b0 msgr2=0x55a9c7ebc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:04.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.353+0000 7fd2c3e00640 1 --2- 172.21.15.123:0/2578516655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7eb44b0 0x55a9c7ebc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.353+0000 7fd2c3e00640 1 -- 172.21.15.123:0/2578516655 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9c7b8b3c0 con 0x55a9c7ebed70 2024-09-19T04:50:04.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.353+0000 7fd2c3e00640 1 --2- 172.21.15.123:0/2578516655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7ebed70 0x55a9c7ebc790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc005fd0 tx=0x7fd2bc005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=966f3711e556870e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:04.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.353+0000 7fd2ad600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:50:04.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.353+0000 7fd2ad600640 1 -- 172.21.15.123:0/2578516655 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc016040 con 0x55a9c7ebed70 2024-09-19T04:50:04.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.353+0000 7fd2ad600640 1 -- 172.21.15.123:0/2578516655 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd2bc012c30 con 0x55a9c7ebed70 2024-09-19T04:50:04.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.353+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/2578516655 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2bc004d20 con 0x55a9c7ebed70 2024-09-19T04:50:04.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.354+0000 7fd2ad600640 1 -- 172.21.15.123:0/2578516655 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd2bc004ed0 con 0x55a9c7ebed70 2024-09-19T04:50:04.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.354+0000 7fd2cd0b1740 10 client.25994 fetch_fsmap learned FSMap version 32 2024-09-19T04:50:04.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.354+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/2578516655 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a9c7ecd730 con 0x55a9c7ebed70 2024-09-19T04:50:04.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.354+0000 7fd2ad600640 1 -- 172.21.15.123:0/2578516655 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2bc004d20 con 0x55a9c7ebed70 2024-09-19T04:50:04.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.354+0000 7fd2ad600640 1 -- 172.21.15.123:0/2578516655 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fd2bc028020 con 0x55a9c7ebed70 2024-09-19T04:50:04.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.354+0000 7fd2cd0b1740 10 client.25994 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T04:50:04.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.354+0000 7fd2cd0b1740 10 client.25994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:50:04.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.354+0000 7fd2cd0b1740 20 client.25994 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:04.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.354+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/2578516655 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a9c7eceb90 0x55a9c7ed0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.354+0000 7fd2cd0b1740 4 client.25994 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:50:04.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.354+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/2578516655 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9c7e27950 con 0x55a9c7eceb90 2024-09-19T04:50:04.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.355+0000 7fd2c2a00640 1 --2- 172.21.15.123:0/2578516655 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a9c7eceb90 0x55a9c7ed0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:04.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.356+0000 7fd2c2a00640 1 --2- 172.21.15.123:0/2578516655 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a9c7eceb90 0x55a9c7ed0f80 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c803ba7fb9868502 server_cookie=955ef2fe43ccddd4 in_seq=0 out_seq=0 2024-09-19T04:50:04.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.356+0000 7fd2ad600640 10 client.25994 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:04.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.356+0000 7fd2ad600640 1 -- 172.21.15.123:0/2578516655 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a9c7e27950 con 0x55a9c7eceb90 2024-09-19T04:50:04.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.356+0000 7fd2ad600640 10 client.25994 handle_command_reply: tid=0 2024-09-19T04:50:04.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.357+0000 7fd2cd0b1740 1 client.25994 shutdown 2024-09-19T04:50:04.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.357+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/2578516655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7ebed70 msgr2=0x55a9c7ebc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:04.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.357+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/2578516655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7ebed70 0x55a9c7ebc790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc005fd0 tx=0x7fd2bc005970 comp rx=0 tx=0).stop 2024-09-19T04:50:04.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.357+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/2578516655 shutdown_connections 2024-09-19T04:50:04.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.357+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/2578516655 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a9c7eceb90 0x55a9c7ed0f80 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.357+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/2578516655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7ebed70 0x55a9c7ebc790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.357+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/2578516655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7eb44b0 0x55a9c7ebc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.357+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/2578516655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7eb3b90 0x55a9c7ebbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.357+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/2578516655 >> 172.21.15.123:0/2578516655 conn(0x55a9c7eb0120 msgr2=0x55a9c7eb5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:04.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.357+0000 7fd2ad600640 0 client.25994 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:04.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.358+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/2578516655 shutdown_connections 2024-09-19T04:50:04.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.358+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/2578516655 wait complete. 2024-09-19T04:50:04.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.358+0000 7fd2cd0b1740 20 client.25994 trim_cache size 0 max 16384 2024-09-19T04:50:04.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.392+0000 7fd2cd0b1740 1 Processor -- start 2024-09-19T04:50:04.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.392+0000 7fd2cd0b1740 1 -- start start 2024-09-19T04:50:04.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.392+0000 7fd2cd0b1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7eb3b90 0x55a9c7eb65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.393+0000 7fd2cd0b1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7eb44b0 0x55a9c7ed6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.393+0000 7fd2cd0b1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7ebed70 0x55a9c7ed6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.393+0000 7fd2cd0b1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41bc860 con 0x55a9c7eb44b0 2024-09-19T04:50:04.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.393+0000 7fd2cd0b1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41be460 con 0x55a9c7ebed70 2024-09-19T04:50:04.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.393+0000 7fd2cd0b1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41bcf30 con 0x55a9c7eb3b90 2024-09-19T04:50:04.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.393+0000 7fd2c3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7eb3b90 0x55a9c7eb65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:04.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.393+0000 7fd2c2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7eb44b0 0x55a9c7ed6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:04.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.393+0000 7fd2c2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7eb44b0 0x55a9c7ed6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60694/0 (socket says 172.21.15.123:60694) 2024-09-19T04:50:04.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.393+0000 7fd2c3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7eb3b90 0x55a9c7eb65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42752/0 (socket says 172.21.15.123:42752) 2024-09-19T04:50:04.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.393+0000 7fd2c2a00640 1 -- 172.21.15.123:0/1282685715 learned_addr learned my addr 172.21.15.123:0/1282685715 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:04.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.393+0000 7fd2c3e00640 1 --2- 172.21.15.123:0/1282685715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7ebed70 0x55a9c7ed6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:04.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.393+0000 7fd2c3e00640 1 -- 172.21.15.123:0/1282685715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7ebed70 msgr2=0x55a9c7ed6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:04.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.393+0000 7fd2c3e00640 1 --2- 172.21.15.123:0/1282685715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7ebed70 0x55a9c7ed6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.393+0000 7fd2c3e00640 1 -- 172.21.15.123:0/1282685715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7eb44b0 msgr2=0x55a9c7ed6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:04.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.393+0000 7fd2c3e00640 1 --2- 172.21.15.123:0/1282685715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7eb44b0 0x55a9c7ed6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.393+0000 7fd2c3e00640 1 -- 172.21.15.123:0/1282685715 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9c7eb0db0 con 0x55a9c7eb3b90 2024-09-19T04:50:04.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.394+0000 7fd2c3e00640 1 --2- 172.21.15.123:0/1282685715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7eb3b90 0x55a9c7eb65f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc011010 tx=0x7fd2bc010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d132103496c6f3bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:04.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.394+0000 7fd2afe00640 1 -- 172.21.15.123:0/1282685715 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0049e0 con 0x55a9c7eb3b90 2024-09-19T04:50:04.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.394+0000 7fd2afe00640 1 -- 172.21.15.123:0/1282685715 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2bc01f040 con 0x55a9c7eb3b90 2024-09-19T04:50:04.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.394+0000 7fd2afe00640 1 -- 172.21.15.123:0/1282685715 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc005070 con 0x55a9c7eb3b90 2024-09-19T04:50:04.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.395+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/1282685715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7eb3b90 msgr2=0x55a9c7eb65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:04.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.395+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/1282685715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7eb3b90 0x55a9c7eb65f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc011010 tx=0x7fd2bc010740 comp rx=0 tx=0).stop 2024-09-19T04:50:04.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.395+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/1282685715 shutdown_connections 2024-09-19T04:50:04.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.395+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/1282685715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7ebed70 0x55a9c7ed6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.395+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/1282685715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7eb44b0 0x55a9c7ed6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.395+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/1282685715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7eb3b90 0x55a9c7eb65f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.395+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/1282685715 >> 172.21.15.123:0/1282685715 conn(0x55a9c7eb0120 msgr2=0x55a9c7ebae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:04.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.395+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/1282685715 shutdown_connections 2024-09-19T04:50:04.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.395+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/1282685715 wait complete. 2024-09-19T04:50:04.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.396+0000 7fd2cd0b1740 1 -- start start 2024-09-19T04:50:04.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.396+0000 7fd2cd0b1740 1 Processor -- start 2024-09-19T04:50:04.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.397+0000 7fd2cd0b1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7eb3b90 0x55a9c7ec2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.397+0000 7fd2cd0b1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7eb44b0 0x55a9c7ec2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.397+0000 7fd2cd0b1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7ebed70 0x55a9c7f375d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.397+0000 7fd2cd0b1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41bdd60 con 0x55a9c7eb3b90 2024-09-19T04:50:04.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.397+0000 7fd2cd0b1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41bd690 con 0x55a9c7eb44b0 2024-09-19T04:50:04.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.397+0000 7fd2cd0b1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41ba840 con 0x55a9c7ebed70 2024-09-19T04:50:04.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.397+0000 7fd2cd0b1740 10 client.0 _ll_register_callbacks cb 0x7fd2bc012760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:04.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.397+0000 7fd2c3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7eb3b90 0x55a9c7ec2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:04.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.397+0000 7fd2c2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7eb44b0 0x55a9c7ec2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:04.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.397+0000 7fd2c2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7eb44b0 0x55a9c7ec2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:41608/0 (socket says 172.21.15.123:41608) 2024-09-19T04:50:04.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.397+0000 7fd2c2a00640 1 -- 172.21.15.123:0/2532989081 learned_addr learned my addr 172.21.15.123:0/2532989081 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:04.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.398+0000 7fd2c2a00640 1 -- 172.21.15.123:0/2532989081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7ebed70 msgr2=0x55a9c7f375d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:04.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.398+0000 7fd2c2a00640 1 --2- 172.21.15.123:0/2532989081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7ebed70 0x55a9c7f375d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.398+0000 7fd2c2a00640 1 -- 172.21.15.123:0/2532989081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7eb3b90 msgr2=0x55a9c7ec2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:04.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.398+0000 7fd2c2a00640 1 --2- 172.21.15.123:0/2532989081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7eb3b90 0x55a9c7ec2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.398+0000 7fd2c2a00640 1 -- 172.21.15.123:0/2532989081 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9c7f37b10 con 0x55a9c7eb44b0 2024-09-19T04:50:04.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.398+0000 7fd2c3e00640 1 --2- 172.21.15.123:0/2532989081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7eb3b90 0x55a9c7ec2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:50:04.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.398+0000 7fd2c2a00640 1 --2- 172.21.15.123:0/2532989081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7eb44b0 0x55a9c7ec2690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009900 tx=0x7fd2b0005180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=298d27283dd7777b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:04.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.398+0000 7fd2ad600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:50:04.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.399+0000 7fd2ad600640 1 -- 172.21.15.123:0/2532989081 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0035040 con 0x55a9c7eb44b0 2024-09-19T04:50:04.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.399+0000 7fd2ad600640 1 -- 172.21.15.123:0/2532989081 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd2b00046c0 con 0x55a9c7eb44b0 2024-09-19T04:50:04.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.400+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/2532989081 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2bc01f040 con 0x55a9c7eb44b0 2024-09-19T04:50:04.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.400+0000 7fd2ad600640 1 -- 172.21.15.123:0/2532989081 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd2b00048c0 con 0x55a9c7eb44b0 2024-09-19T04:50:04.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.400+0000 7fd2cd0b1740 10 client.26000 fetch_fsmap learned FSMap version 32 2024-09-19T04:50:04.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.400+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/2532989081 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a9c7f38cf0 con 0x55a9c7eb44b0 2024-09-19T04:50:04.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.400+0000 7fd2ad600640 1 -- 172.21.15.123:0/2532989081 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2bc01f040 con 0x55a9c7eb44b0 2024-09-19T04:50:04.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.400+0000 7fd2ad600640 1 -- 172.21.15.123:0/2532989081 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fd2b004c030 con 0x55a9c7eb44b0 2024-09-19T04:50:04.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.400+0000 7fd2cd0b1740 10 client.26000 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T04:50:04.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.400+0000 7fd2cd0b1740 10 client.26000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:50:04.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.400+0000 7fd2cd0b1740 20 client.26000 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:04.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.400+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/2532989081 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a9c7f394d0 0x55a9c7f3b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:04.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.401+0000 7fd2cd0b1740 4 client.26000 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:50:04.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.401+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/2532989081 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a9c7e21e60 con 0x55a9c7f394d0 2024-09-19T04:50:04.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.401+0000 7fd2c3e00640 1 --2- 172.21.15.123:0/2532989081 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a9c7f394d0 0x55a9c7f3b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:04.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.401+0000 7fd2c3e00640 1 --2- 172.21.15.123:0/2532989081 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a9c7f394d0 0x55a9c7f3b8f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d829ef91f1c6304 server_cookie=44f91578e5fe2bad in_seq=0 out_seq=0 2024-09-19T04:50:04.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.401+0000 7fd2ad600640 10 client.26000 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:04.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.402+0000 7fd2ad600640 1 -- 172.21.15.123:0/2532989081 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a9c7e21e60 con 0x55a9c7f394d0 2024-09-19T04:50:04.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.402+0000 7fd2ad600640 10 client.26000 handle_command_reply: tid=0 2024-09-19T04:50:04.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.402+0000 7fd2cd0b1740 1 client.26000 shutdown 2024-09-19T04:50:04.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.402+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/2532989081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7eb44b0 msgr2=0x55a9c7ec2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:04.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.402+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/2532989081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7eb44b0 0x55a9c7ec2690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009900 tx=0x7fd2b0005180 comp rx=0 tx=0).stop 2024-09-19T04:50:04.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.402+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/2532989081 shutdown_connections 2024-09-19T04:50:04.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.402+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/2532989081 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a9c7f394d0 0x55a9c7f3b8f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.402+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/2532989081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9c7ebed70 0x55a9c7f375d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.402+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/2532989081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c7eb44b0 0x55a9c7ec2690 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.402+0000 7fd2cd0b1740 1 --2- 172.21.15.123:0/2532989081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9c7eb3b90 0x55a9c7ec2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.402+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/2532989081 >> 172.21.15.123:0/2532989081 conn(0x55a9c7eb0120 msgr2=0x55a9c7ebd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:04.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.402+0000 7fd2ad600640 0 client.26000 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:04.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.403+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/2532989081 shutdown_connections 2024-09-19T04:50:04.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.403+0000 7fd2cd0b1740 1 -- 172.21.15.123:0/2532989081 wait complete. 2024-09-19T04:50:04.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.403+0000 7fd2cd0b1740 20 client.26000 trim_cache size 0 max 16384 2024-09-19T04:50:04.406 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:50:04.406 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:50:04.406 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": { 2024-09-19T04:50:04.406 INFO:teuthology.orchestra.run.smithi123.stdout: "bf83a384-d8dc-4e88-9f3c-a935743afe69": { 2024-09-19T04:50:04.406 INFO:teuthology.orchestra.run.smithi123.stdout: "path": "/", 2024-09-19T04:50:04.406 INFO:teuthology.orchestra.run.smithi123.stdout: "tag": "bf83a384-d8dc-4e88-9f3c-a935743afe69", 2024-09-19T04:50:04.406 INFO:teuthology.orchestra.run.smithi123.stdout: "options": "recursive,force" 2024-09-19T04:50:04.407 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:50:04.407 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:50:04.407 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:50:04.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.404+0000 7fd2ca600640 1 -- 172.21.15.123:0/3404400956 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd2a80a3430 msgr2=0x7fd2a80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:04.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.404+0000 7fd2ca600640 1 --2- 172.21.15.123:0/3404400956 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd2a80a3430 0x7fd2a80a58f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55a9c7eb0f80 tx=0x7fd2b4009520 comp rx=0 tx=0).stop 2024-09-19T04:50:04.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.404+0000 7fd2ca600640 1 -- 172.21.15.123:0/3404400956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2c4100930 msgr2=0x7fd2c41092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:04.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.404+0000 7fd2ca600640 1 --2- 172.21.15.123:0/3404400956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2c4100930 0x7fd2c41092e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400ea60 tx=0x7fd2b400ef20 comp rx=0 tx=0).stop 2024-09-19T04:50:04.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.404+0000 7fd2ca600640 1 -- 172.21.15.123:0/3404400956 shutdown_connections 2024-09-19T04:50:04.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.404+0000 7fd2ca600640 1 --2- 172.21.15.123:0/3404400956 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd2a80a3430 0x7fd2a80a58f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.404+0000 7fd2ca600640 1 --2- 172.21.15.123:0/3404400956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2c410e4e0 0x7fd2c4104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.404+0000 7fd2ca600640 1 --2- 172.21.15.123:0/3404400956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c41012e0 0x7fd2c4109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.404+0000 7fd2ca600640 1 --2- 172.21.15.123:0/3404400956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2c4100930 0x7fd2c41092e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:04.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.404+0000 7fd2ca600640 1 -- 172.21.15.123:0/3404400956 >> 172.21.15.123:0/3404400956 conn(0x7fd2c40fc7a0 msgr2=0x7fd2c40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:04.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.407+0000 7fd2ca600640 1 -- 172.21.15.123:0/3404400956 shutdown_connections 2024-09-19T04:50:04.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:04.407+0000 7fd2ca600640 1 -- 172.21.15.123:0/3404400956 wait complete. 2024-09-19T04:50:05.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:04 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:05.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:04 smithi183 ceph-mon[35333]: pgmap v104: 97 pgs: 97 active+clean; 104 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 581 op/s 2024-09-19T04:50:05.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:04 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:05.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:04 smithi123 ceph-mon[30511]: pgmap v104: 97 pgs: 97 active+clean; 104 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 581 op/s 2024-09-19T04:50:05.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:04 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:05.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:04 smithi193 ceph-mon[35592]: pgmap v104: 97 pgs: 97 active+clean; 104 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 581 op/s 2024-09-19T04:50:05.390 INFO:tasks.cephfs.filesystem:scrub status for tag:bf83a384-d8dc-4e88-9f3c-a935743afe69 - {'path': '/', 'tag': 'bf83a384-d8dc-4e88-9f3c-a935743afe69', 'options': 'recursive,force'} 2024-09-19T04:50:07.144 INFO:tasks.workunit.client.0.smithi123.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-09-19T04:50:07.144 INFO:tasks.workunit.client.0.smithi123.stdout:All 50000 operations completed A-OK! 2024-09-19T04:50:07.144 INFO:tasks.workunit.client.0.smithi123.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-09-19T04:50:07.146 INFO:tasks.workunit.client.0.smithi123.stdout:Seed set to 1 2024-09-19T04:50:07.176 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:06 smithi123 ceph-mon[30511]: pgmap v105: 97 pgs: 97 active+clean; 104 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-09-19T04:50:07.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:06 smithi193 ceph-mon[35592]: pgmap v105: 97 pgs: 97 active+clean; 104 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-09-19T04:50:07.281 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-19T04:50:07.281 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-19T04:50:07.283 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-19T04:50:07.283 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-19T04:50:07.283 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support clone range, disabling! 2024-09-19T04:50:07.286 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-19T04:50:07.288 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support exchange range, disabling! 2024-09-19T04:50:07.289 INFO:tasks.workunit.client.0.smithi123.stdout:skipping zero length punch hole 2024-09-19T04:50:07.289 INFO:tasks.workunit.client.0.smithi123.stdout:skipping zero length punch hole 2024-09-19T04:50:07.289 INFO:tasks.workunit.client.0.smithi123.stdout:skipping zero length punch hole 2024-09-19T04:50:07.289 INFO:tasks.workunit.client.0.smithi123.stdout:skipping zero size read 2024-09-19T04:50:07.289 INFO:tasks.workunit.client.0.smithi123.stdout:skipping zero length punch hole 2024-09-19T04:50:07.289 INFO:tasks.workunit.client.0.smithi123.stdout:skipping zero size read 2024-09-19T04:50:07.290 INFO:tasks.workunit.client.0.smithi123.stdout:skipping zero length punch hole 2024-09-19T04:50:07.290 INFO:tasks.workunit.client.0.smithi123.stdout:truncating to largest ever: 0x4310e9 2024-09-19T04:50:07.297 INFO:tasks.workunit.client.0.smithi123.stdout:copying to largest ever: 0x954a45 2024-09-19T04:50:07.331 INFO:tasks.workunit.client.0.smithi123.stdout:truncating to largest ever: 0x9bc582 2024-09-19T04:50:07.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:06 smithi183 ceph-mon[35333]: pgmap v105: 97 pgs: 97 active+clean; 104 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-09-19T04:50:07.357 INFO:tasks.workunit.client.0.smithi123.stdout:truncating to largest ever: 0x9eecbc 2024-09-19T04:50:09.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:08 smithi123 ceph-mon[30511]: pgmap v106: 97 pgs: 97 active+clean; 111 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 561 op/s 2024-09-19T04:50:09.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:08 smithi193 ceph-mon[35592]: pgmap v106: 97 pgs: 97 active+clean; 111 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 561 op/s 2024-09-19T04:50:09.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:08 smithi183 ceph-mon[35333]: pgmap v106: 97 pgs: 97 active+clean; 111 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 561 op/s 2024-09-19T04:50:09.631 INFO:tasks.workunit.client.0.smithi123.stdout:copying to largest ever: 0x9fdd84 2024-09-19T04:50:11.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:10 smithi193 ceph-mon[35592]: pgmap v107: 97 pgs: 97 active+clean; 111 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 370 op/s 2024-09-19T04:50:11.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:10 smithi183 ceph-mon[35333]: pgmap v107: 97 pgs: 97 active+clean; 111 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 370 op/s 2024-09-19T04:50:11.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:10 smithi123 ceph-mon[30511]: pgmap v107: 97 pgs: 97 active+clean; 111 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 370 op/s 2024-09-19T04:50:13.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:12 smithi193 ceph-mon[35592]: pgmap v108: 97 pgs: 97 active+clean; 111 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 370 op/s 2024-09-19T04:50:13.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:12 smithi183 ceph-mon[35333]: pgmap v108: 97 pgs: 97 active+clean; 111 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 370 op/s 2024-09-19T04:50:13.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:12 smithi123 ceph-mon[30511]: pgmap v108: 97 pgs: 97 active+clean; 111 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 370 op/s 2024-09-19T04:50:15.225 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:14 smithi123 ceph-mon[30511]: pgmap v109: 97 pgs: 97 active+clean; 116 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.8 MiB/s wr, 691 op/s 2024-09-19T04:50:15.225 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:14 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:50:15.225 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:14 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:50:15.225 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:14 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:50:15.225 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:14 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:50:15.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:14 smithi193 ceph-mon[35592]: pgmap v109: 97 pgs: 97 active+clean; 116 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.8 MiB/s wr, 691 op/s 2024-09-19T04:50:15.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:14 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:50:15.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:14 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:50:15.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:14 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:50:15.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:14 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:50:15.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:14 smithi183 ceph-mon[35333]: pgmap v109: 97 pgs: 97 active+clean; 116 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.8 MiB/s wr, 691 op/s 2024-09-19T04:50:15.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:14 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:50:15.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:14 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:50:15.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:14 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:50:15.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:14 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:50:16.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:15 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:50:16.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:15 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:50:16.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:15 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:50:17.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:16 smithi193 ceph-mon[35592]: pgmap v110: 97 pgs: 97 active+clean; 116 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.8 MiB/s wr, 496 op/s 2024-09-19T04:50:17.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:16 smithi183 ceph-mon[35333]: pgmap v110: 97 pgs: 97 active+clean; 116 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.8 MiB/s wr, 496 op/s 2024-09-19T04:50:17.426 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:16 smithi123 ceph-mon[30511]: pgmap v110: 97 pgs: 97 active+clean; 116 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.8 MiB/s wr, 496 op/s 2024-09-19T04:50:19.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:18 smithi193 ceph-mon[35592]: pgmap v111: 97 pgs: 97 active+clean; 121 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.6 MiB/s wr, 801 op/s 2024-09-19T04:50:19.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:18 smithi183 ceph-mon[35333]: pgmap v111: 97 pgs: 97 active+clean; 121 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.6 MiB/s wr, 801 op/s 2024-09-19T04:50:19.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:18 smithi123 ceph-mon[30511]: pgmap v111: 97 pgs: 97 active+clean; 121 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.6 MiB/s wr, 801 op/s 2024-09-19T04:50:20.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:19 smithi193 ceph-mon[35592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:50:20.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:19 smithi183 ceph-mon[35333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:50:20.426 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:19 smithi123 ceph-mon[30511]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:50:21.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:20 smithi193 ceph-mon[35592]: pgmap v112: 97 pgs: 97 active+clean; 121 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.5 MiB/s wr, 625 op/s 2024-09-19T04:50:21.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:20 smithi183 ceph-mon[35333]: pgmap v112: 97 pgs: 97 active+clean; 121 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.5 MiB/s wr, 625 op/s 2024-09-19T04:50:21.393 INFO:tasks.workunit.client.0.smithi123.stdout:copying to largest ever: 0x9ff773 2024-09-19T04:50:21.426 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:20 smithi123 ceph-mon[30511]: pgmap v112: 97 pgs: 97 active+clean; 121 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.5 MiB/s wr, 625 op/s 2024-09-19T04:50:21.841 INFO:tasks.workunit.client.0.smithi123.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-09-19T04:50:23.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:22 smithi193 ceph-mon[35592]: pgmap v113: 97 pgs: 97 active+clean; 121 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.5 MiB/s wr, 625 op/s 2024-09-19T04:50:23.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:22 smithi183 ceph-mon[35333]: pgmap v113: 97 pgs: 97 active+clean; 121 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.5 MiB/s wr, 625 op/s 2024-09-19T04:50:23.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:22 smithi123 ceph-mon[30511]: pgmap v113: 97 pgs: 97 active+clean; 121 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.5 MiB/s wr, 625 op/s 2024-09-19T04:50:25.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:24 smithi193 ceph-mon[35592]: pgmap v114: 97 pgs: 97 active+clean; 119 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.1 MiB/s wr, 909 op/s 2024-09-19T04:50:25.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:24 smithi183 ceph-mon[35333]: pgmap v114: 97 pgs: 97 active+clean; 119 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.1 MiB/s wr, 909 op/s 2024-09-19T04:50:25.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:24 smithi123 ceph-mon[30511]: pgmap v114: 97 pgs: 97 active+clean; 119 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.1 MiB/s wr, 909 op/s 2024-09-19T04:50:27.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:26 smithi193 ceph-mon[35592]: pgmap v115: 97 pgs: 97 active+clean; 119 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.4 MiB/s wr, 587 op/s 2024-09-19T04:50:27.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:26 smithi183 ceph-mon[35333]: pgmap v115: 97 pgs: 97 active+clean; 119 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.4 MiB/s wr, 587 op/s 2024-09-19T04:50:27.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:26 smithi123 ceph-mon[30511]: pgmap v115: 97 pgs: 97 active+clean; 119 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.4 MiB/s wr, 587 op/s 2024-09-19T04:50:29.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:28 smithi193 ceph-mon[35592]: pgmap v116: 97 pgs: 97 active+clean; 129 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 822 op/s 2024-09-19T04:50:29.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:28 smithi183 ceph-mon[35333]: pgmap v116: 97 pgs: 97 active+clean; 129 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 822 op/s 2024-09-19T04:50:29.426 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:28 smithi123 ceph-mon[30511]: pgmap v116: 97 pgs: 97 active+clean; 129 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 822 op/s 2024-09-19T04:50:31.190 INFO:tasks.workunit.client.0.smithi123.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-09-19T04:50:31.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:30 smithi193 ceph-mon[35592]: pgmap v117: 97 pgs: 97 active+clean; 129 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.2 MiB/s wr, 517 op/s 2024-09-19T04:50:31.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:30 smithi183 ceph-mon[35333]: pgmap v117: 97 pgs: 97 active+clean; 129 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.2 MiB/s wr, 517 op/s 2024-09-19T04:50:31.426 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:30 smithi123 ceph-mon[30511]: pgmap v117: 97 pgs: 97 active+clean; 129 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.2 MiB/s wr, 517 op/s 2024-09-19T04:50:33.228 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:32 smithi123 ceph-mon[30511]: pgmap v118: 97 pgs: 97 active+clean; 129 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.2 MiB/s wr, 517 op/s 2024-09-19T04:50:33.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:32 smithi193 ceph-mon[35592]: pgmap v118: 97 pgs: 97 active+clean; 129 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.2 MiB/s wr, 517 op/s 2024-09-19T04:50:33.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:32 smithi183 ceph-mon[35333]: pgmap v118: 97 pgs: 97 active+clean; 129 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.2 MiB/s wr, 517 op/s 2024-09-19T04:50:35.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:34 smithi193 ceph-mon[35592]: pgmap v119: 97 pgs: 97 active+clean; 131 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 769 op/s 2024-09-19T04:50:35.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:34 smithi183 ceph-mon[35333]: pgmap v119: 97 pgs: 97 active+clean; 131 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 769 op/s 2024-09-19T04:50:35.392 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:50:35.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:34 smithi123 ceph-mon[30511]: pgmap v119: 97 pgs: 97 active+clean; 131 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 769 op/s 2024-09-19T04:50:35.839 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:50:36.060 INFO:tasks.workunit.client.0.smithi123.stdout:truncating to largest ever: 0x9ffb01 2024-09-19T04:50:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.462+0000 7fa17b400640 1 -- 172.21.15.123:0/3113912896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa174104010 msgr2=0x7fa1741043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.462+0000 7fa17b400640 1 --2- 172.21.15.123:0/3113912896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa174104010 0x7fa1741043f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa164009860 tx=0x7fa16402a340 comp rx=0 tx=0).stop 2024-09-19T04:50:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.462+0000 7fa17b400640 1 -- 172.21.15.123:0/3113912896 shutdown_connections 2024-09-19T04:50:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.462+0000 7fa17b400640 1 --2- 172.21.15.123:0/3113912896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa174109140 0x7fa174110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.462+0000 7fa17b400640 1 --2- 172.21.15.123:0/3113912896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1741049c0 0x7fa174108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.462+0000 7fa17b400640 1 --2- 172.21.15.123:0/3113912896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa174104010 0x7fa1741043f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.462+0000 7fa17b400640 1 -- 172.21.15.123:0/3113912896 >> 172.21.15.123:0/3113912896 conn(0x7fa1740ffa50 msgr2=0x7fa174101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.463+0000 7fa17b400640 1 -- 172.21.15.123:0/3113912896 shutdown_connections 2024-09-19T04:50:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.463+0000 7fa17b400640 1 -- 172.21.15.123:0/3113912896 wait complete. 2024-09-19T04:50:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.464+0000 7fa17b400640 1 Processor -- start 2024-09-19T04:50:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.464+0000 7fa17b400640 1 -- start start 2024-09-19T04:50:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.465+0000 7fa17b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa174104010 0x7fa17419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.465+0000 7fa17b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1741049c0 0x7fa17419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.465+0000 7fa17b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa174109140 0x7fa17419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.465+0000 7fa17b400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa174114170 con 0x7fa1741049c0 2024-09-19T04:50:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.465+0000 7fa17b400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa174113ff0 con 0x7fa174104010 2024-09-19T04:50:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.465+0000 7fa17b400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741142f0 con 0x7fa174109140 2024-09-19T04:50:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.465+0000 7fa179400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa174109140 0x7fa17419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.465+0000 7fa179400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa174109140 0x7fa17419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49628/0 (socket says 172.21.15.123:49628) 2024-09-19T04:50:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.465+0000 7fa179400640 1 -- 172.21.15.123:0/1578978504 learned_addr learned my addr 172.21.15.123:0/1578978504 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.466+0000 7fa179400640 1 -- 172.21.15.123:0/1578978504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa174104010 msgr2=0x7fa17419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.466+0000 7fa179400640 1 --2- 172.21.15.123:0/1578978504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa174104010 0x7fa17419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.466+0000 7fa179400640 1 -- 172.21.15.123:0/1578978504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1741049c0 msgr2=0x7fa17419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.466+0000 7fa179400640 1 --2- 172.21.15.123:0/1578978504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1741049c0 0x7fa17419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.466+0000 7fa179400640 1 -- 172.21.15.123:0/1578978504 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17419f600 con 0x7fa174109140 2024-09-19T04:50:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.466+0000 7fa179400640 1 --2- 172.21.15.123:0/1578978504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa174109140 0x7fa17419ef20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00eb10 tx=0x7fa16c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.466+0000 7fa172000640 1 -- 172.21.15.123:0/1578978504 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c004b20 con 0x7fa174109140 2024-09-19T04:50:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.466+0000 7fa172000640 1 -- 172.21.15.123:0/1578978504 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa16c004cc0 con 0x7fa174109140 2024-09-19T04:50:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.466+0000 7fa172000640 1 -- 172.21.15.123:0/1578978504 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c010790 con 0x7fa174109140 2024-09-19T04:50:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.466+0000 7fa17b400640 1 -- 172.21.15.123:0/1578978504 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17419f8f0 con 0x7fa174109140 2024-09-19T04:50:36.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.466+0000 7fa17b400640 1 -- 172.21.15.123:0/1578978504 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa174076630 con 0x7fa174109140 2024-09-19T04:50:36.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.470+0000 7fa17de13740 1 Processor -- start 2024-09-19T04:50:36.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.470+0000 7fa172000640 1 -- 172.21.15.123:0/1578978504 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa16c005020 con 0x7fa174109140 2024-09-19T04:50:36.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.471+0000 7fa17de13740 1 -- start start 2024-09-19T04:50:36.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.471+0000 7fa17de13740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cabb90 0x55b7b7cabf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.471+0000 7fa17de13740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cac4b0 0x55b7b7cb6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.471+0000 7fa17de13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cb9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.471+0000 7fa17de13740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b7b7c123e0 con 0x55b7b7cac4b0 2024-09-19T04:50:36.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.471+0000 7fa17de13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b7b7bb9d80 con 0x55b7b7cb6d70 2024-09-19T04:50:36.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.471+0000 7fa17de13740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b7b7c0e3a0 con 0x55b7b7cabb90 2024-09-19T04:50:36.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.471+0000 7fa16be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cb9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:36.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.472+0000 7fa16be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cabb90 0x55b7b7cabf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:36.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.472+0000 7fa16be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cabb90 0x55b7b7cabf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49640/0 (socket says 172.21.15.123:49640) 2024-09-19T04:50:36.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.472+0000 7fa16be00640 1 -- 172.21.15.123:0/1911734615 learned_addr learned my addr 172.21.15.123:0/1911734615 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:36.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.472+0000 7fa16be00640 1 -- 172.21.15.123:0/1911734615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cb6d70 msgr2=0x55b7b7cb9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:36.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.472+0000 7fa16be00640 1 --2- 172.21.15.123:0/1911734615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cb9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.472+0000 7fa16be00640 1 -- 172.21.15.123:0/1911734615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cac4b0 msgr2=0x55b7b7cb6830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:50:36.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.472+0000 7fa16be00640 1 --2- 172.21.15.123:0/1911734615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cac4b0 0x55b7b7cb6830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.472+0000 7fa16be00640 1 -- 172.21.15.123:0/1911734615 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7b7bd8ae0 con 0x55b7b7cabb90 2024-09-19T04:50:36.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.472+0000 7fa16be00640 1 --2- 172.21.15.123:0/1911734615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cabb90 0x55b7b7cabf70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0096c0 tx=0x7fa15c005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=148f1d552d61504f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:36.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.472+0000 7fa169c00640 1 -- 172.21.15.123:0/1911734615 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c013110 con 0x55b7b7cabb90 2024-09-19T04:50:36.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.473+0000 7fa169c00640 1 -- 172.21.15.123:0/1911734615 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa15c0132b0 con 0x55b7b7cabb90 2024-09-19T04:50:36.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.473+0000 7fa169c00640 1 -- 172.21.15.123:0/1911734615 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c0035f0 con 0x55b7b7cabb90 2024-09-19T04:50:36.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.473+0000 7fa172000640 1 --2- 172.21.15.123:0/1578978504 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa1580a3430 0x7fa1580a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.473+0000 7fa172000640 1 -- 172.21.15.123:0/1578978504 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fa16c0d02f0 con 0x7fa174109140 2024-09-19T04:50:36.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.474+0000 7fa17de13740 1 -- 172.21.15.123:0/1911734615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cabb90 msgr2=0x55b7b7cabf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:36.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.474+0000 7fa17de13740 1 --2- 172.21.15.123:0/1911734615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cabb90 0x55b7b7cabf70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0096c0 tx=0x7fa15c005a20 comp rx=0 tx=0).stop 2024-09-19T04:50:36.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.474+0000 7fa178a00640 1 --2- 172.21.15.123:0/1578978504 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa1580a3430 0x7fa1580a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:36.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.474+0000 7fa17de13740 1 -- 172.21.15.123:0/1911734615 shutdown_connections 2024-09-19T04:50:36.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.474+0000 7fa17de13740 1 --2- 172.21.15.123:0/1911734615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cb9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.474+0000 7fa17de13740 1 --2- 172.21.15.123:0/1911734615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cac4b0 0x55b7b7cb6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.474+0000 7fa17de13740 1 --2- 172.21.15.123:0/1911734615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cabb90 0x55b7b7cabf70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.474+0000 7fa17de13740 1 -- 172.21.15.123:0/1911734615 >> 172.21.15.123:0/1911734615 conn(0x55b7b7ca8120 msgr2=0x55b7b7ca8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:36.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.474+0000 7fa17de13740 1 -- 172.21.15.123:0/1911734615 shutdown_connections 2024-09-19T04:50:36.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.474+0000 7fa17de13740 1 -- 172.21.15.123:0/1911734615 wait complete. 2024-09-19T04:50:36.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.474+0000 7fa178a00640 1 --2- 172.21.15.123:0/1578978504 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa1580a3430 0x7fa1580a58f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa164009780 tx=0x7fa164030c50 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:36.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.475+0000 7fa17de13740 1 -- start start 2024-09-19T04:50:36.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.475+0000 7fa17de13740 1 Processor -- start 2024-09-19T04:50:36.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.475+0000 7fa17de13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cabb90 0x55b7b7cb3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.475+0000 7fa17de13740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cac4b0 0x55b7b7cb41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.475+0000 7fa17de13740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cb4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.476+0000 7fa17de13740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b7b7cca9f0 con 0x55b7b7cac4b0 2024-09-19T04:50:36.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.476+0000 7fa17de13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b7b7bcd0f0 con 0x55b7b7cabb90 2024-09-19T04:50:36.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.476+0000 7fa17de13740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b7b7ccab70 con 0x55b7b7cb6d70 2024-09-19T04:50:36.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.476+0000 7fa17de13740 10 client.0 _ll_register_callbacks cb 0x55b7b7bbe9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:36.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.476+0000 7fa178a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cabb90 0x55b7b7cb3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:36.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.476+0000 7fa178a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cabb90 0x55b7b7cb3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:60612/0 (socket says 172.21.15.123:60612) 2024-09-19T04:50:36.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.476+0000 7fa178a00640 1 -- 172.21.15.123:0/1923294127 learned_addr learned my addr 172.21.15.123:0/1923294127 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:36.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.476+0000 7fa179400640 1 --2- 172.21.15.123:0/1923294127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cb4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:36.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.476+0000 7fa179400640 1 -- 172.21.15.123:0/1923294127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cabb90 msgr2=0x55b7b7cb3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:36.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.476+0000 7fa16be00640 1 --2- 172.21.15.123:0/1923294127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cac4b0 0x55b7b7cb41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:36.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.476+0000 7fa179400640 1 --2- 172.21.15.123:0/1923294127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cabb90 0x55b7b7cb3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.476+0000 7fa179400640 1 -- 172.21.15.123:0/1923294127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cac4b0 msgr2=0x55b7b7cb41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:36.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.476+0000 7fa179400640 1 --2- 172.21.15.123:0/1923294127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cac4b0 0x55b7b7cb41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.476+0000 7fa179400640 1 -- 172.21.15.123:0/1923294127 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7b79833c0 con 0x55b7b7cb6d70 2024-09-19T04:50:36.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.476+0000 7fa16be00640 1 --2- 172.21.15.123:0/1923294127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cac4b0 0x55b7b7cb41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:50:36.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.477+0000 7fa179400640 1 --2- 172.21.15.123:0/1923294127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cb4700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0cd9e0 tx=0x7fa16c09cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60939fc97571b668 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:36.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.477+0000 7fa162000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:50:36.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.477+0000 7fa162000640 1 -- 172.21.15.123:0/1923294127 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c0b4020 con 0x55b7b7cb6d70 2024-09-19T04:50:36.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.477+0000 7fa162000640 1 -- 172.21.15.123:0/1923294127 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fa16c098c10 con 0x55b7b7cb6d70 2024-09-19T04:50:36.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.477+0000 7fa17de13740 1 -- 172.21.15.123:0/1923294127 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa15c0132b0 con 0x55b7b7cb6d70 2024-09-19T04:50:36.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.477+0000 7fa162000640 1 -- 172.21.15.123:0/1923294127 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa16c0ba050 con 0x55b7b7cb6d70 2024-09-19T04:50:36.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.477+0000 7fa17de13740 10 client.26029 fetch_fsmap learned FSMap version 33 2024-09-19T04:50:36.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.477+0000 7fa17de13740 1 -- 172.21.15.123:0/1923294127 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b7b7cd6730 con 0x55b7b7cb6d70 2024-09-19T04:50:36.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.478+0000 7fa162000640 1 -- 172.21.15.123:0/1923294127 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa16c07c020 con 0x55b7b7cb6d70 2024-09-19T04:50:36.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.478+0000 7fa162000640 1 -- 172.21.15.123:0/1923294127 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fa16c09bb10 con 0x55b7b7cb6d70 2024-09-19T04:50:36.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.478+0000 7fa17de13740 10 client.26029 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:50:36.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.478+0000 7fa17de13740 10 client.26029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:50:36.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.478+0000 7fa17de13740 20 client.26029 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:36.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.478+0000 7fa17de13740 1 --2- 172.21.15.123:0/1923294127 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b7b7cd7c40 0x55b7b7cda0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.478+0000 7fa17de13740 4 client.26029 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:50:36.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.478+0000 7fa17de13740 1 -- 172.21.15.123:0/1923294127 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7b7c1f950 con 0x55b7b7cd7c40 2024-09-19T04:50:36.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.478+0000 7fa178a00640 1 --2- 172.21.15.123:0/1923294127 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b7b7cd7c40 0x55b7b7cda0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:36.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.479+0000 7fa178a00640 1 --2- 172.21.15.123:0/1923294127 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b7b7cd7c40 0x55b7b7cda0b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fa9040663ec4892 server_cookie=1040cc325b39306c in_seq=0 out_seq=0 2024-09-19T04:50:36.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.479+0000 7fa162000640 10 client.26029 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:36.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.479+0000 7fa162000640 1 -- 172.21.15.123:0/1923294127 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7b7c1f950 con 0x55b7b7cd7c40 2024-09-19T04:50:36.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.479+0000 7fa162000640 10 client.26029 handle_command_reply: tid=0 2024-09-19T04:50:36.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.480+0000 7fa17de13740 1 client.26029 shutdown 2024-09-19T04:50:36.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.480+0000 7fa17de13740 1 -- 172.21.15.123:0/1923294127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cb6d70 msgr2=0x55b7b7cb4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:36.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.480+0000 7fa17de13740 1 --2- 172.21.15.123:0/1923294127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cb4700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0cd9e0 tx=0x7fa16c09cd60 comp rx=0 tx=0).stop 2024-09-19T04:50:36.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.480+0000 7fa17de13740 1 -- 172.21.15.123:0/1923294127 shutdown_connections 2024-09-19T04:50:36.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.480+0000 7fa17de13740 1 --2- 172.21.15.123:0/1923294127 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b7b7cd7c40 0x55b7b7cda0b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.480+0000 7fa17de13740 1 --2- 172.21.15.123:0/1923294127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cb4700 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.480+0000 7fa17de13740 1 --2- 172.21.15.123:0/1923294127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cac4b0 0x55b7b7cb41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.480+0000 7fa17de13740 1 --2- 172.21.15.123:0/1923294127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cabb90 0x55b7b7cb3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.480+0000 7fa162000640 0 client.26029 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:36.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.480+0000 7fa17de13740 1 -- 172.21.15.123:0/1923294127 >> 172.21.15.123:0/1923294127 conn(0x55b7b7ca8120 msgr2=0x55b7b7cade50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:36.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.480+0000 7fa17de13740 1 -- 172.21.15.123:0/1923294127 shutdown_connections 2024-09-19T04:50:36.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.480+0000 7fa17de13740 1 -- 172.21.15.123:0/1923294127 wait complete. 2024-09-19T04:50:36.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.480+0000 7fa17de13740 20 client.26029 trim_cache size 0 max 16384 2024-09-19T04:50:36.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.501+0000 7fa17de13740 1 Processor -- start 2024-09-19T04:50:36.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.501+0000 7fa17de13740 1 -- start start 2024-09-19T04:50:36.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.501+0000 7fa17de13740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cabb90 0x55b7b7cae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.501+0000 7fa17de13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cac4b0 0x55b7b7cb4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.501+0000 7fa17de13740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cdf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.501+0000 7fa17de13740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741ab700 con 0x55b7b7cb6d70 2024-09-19T04:50:36.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.501+0000 7fa17de13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741ad2d0 con 0x55b7b7cac4b0 2024-09-19T04:50:36.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.501+0000 7fa17de13740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741abdd0 con 0x55b7b7cabb90 2024-09-19T04:50:36.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.502+0000 7fa179400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cdf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:36.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.502+0000 7fa179400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cdf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54568/0 (socket says 172.21.15.123:54568) 2024-09-19T04:50:36.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.502+0000 7fa178a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cabb90 0x55b7b7cae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:36.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.502+0000 7fa179400640 1 -- 172.21.15.123:0/4142873154 learned_addr learned my addr 172.21.15.123:0/4142873154 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:36.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.502+0000 7fa16be00640 1 --2- 172.21.15.123:0/4142873154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cac4b0 0x55b7b7cb4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:36.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.502+0000 7fa178a00640 1 -- 172.21.15.123:0/4142873154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cac4b0 msgr2=0x55b7b7cb4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:36.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.502+0000 7fa178a00640 1 --2- 172.21.15.123:0/4142873154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cac4b0 0x55b7b7cb4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.502+0000 7fa178a00640 1 -- 172.21.15.123:0/4142873154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cb6d70 msgr2=0x55b7b7cdf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:36.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.502+0000 7fa178a00640 1 --2- 172.21.15.123:0/4142873154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cdf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.502+0000 7fa178a00640 1 -- 172.21.15.123:0/4142873154 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7b7cb21d0 con 0x55b7b7cabb90 2024-09-19T04:50:36.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.502+0000 7fa179400640 1 --2- 172.21.15.123:0/4142873154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cdf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:36.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.502+0000 7fa178a00640 1 --2- 172.21.15.123:0/4142873154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cabb90 0x55b7b7cae640 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55b7b7cda8c0 tx=0x7fa16403c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85fd11493b01af15 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:36.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.503+0000 7fa169200640 1 -- 172.21.15.123:0/4142873154 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164037040 con 0x55b7b7cabb90 2024-09-19T04:50:36.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.503+0000 7fa169200640 1 -- 172.21.15.123:0/4142873154 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa164004380 con 0x55b7b7cabb90 2024-09-19T04:50:36.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.503+0000 7fa169200640 1 -- 172.21.15.123:0/4142873154 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164003d10 con 0x55b7b7cabb90 2024-09-19T04:50:36.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.504+0000 7fa17de13740 1 -- 172.21.15.123:0/4142873154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cabb90 msgr2=0x55b7b7cae640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:36.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.504+0000 7fa17de13740 1 --2- 172.21.15.123:0/4142873154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cabb90 0x55b7b7cae640 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55b7b7cda8c0 tx=0x7fa16403c000 comp rx=0 tx=0).stop 2024-09-19T04:50:36.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.504+0000 7fa17de13740 1 -- 172.21.15.123:0/4142873154 shutdown_connections 2024-09-19T04:50:36.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.504+0000 7fa17de13740 1 --2- 172.21.15.123:0/4142873154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cdf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.504+0000 7fa17de13740 1 --2- 172.21.15.123:0/4142873154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cac4b0 0x55b7b7cb4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.504+0000 7fa17de13740 1 --2- 172.21.15.123:0/4142873154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cabb90 0x55b7b7cae640 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.504+0000 7fa17de13740 1 -- 172.21.15.123:0/4142873154 >> 172.21.15.123:0/4142873154 conn(0x55b7b7ca8120 msgr2=0x55b7b7cb5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:36.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.504+0000 7fa17de13740 1 -- 172.21.15.123:0/4142873154 shutdown_connections 2024-09-19T04:50:36.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.504+0000 7fa17de13740 1 -- 172.21.15.123:0/4142873154 wait complete. 2024-09-19T04:50:36.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.504+0000 7fa17de13740 1 -- start start 2024-09-19T04:50:36.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.505+0000 7fa17de13740 1 Processor -- start 2024-09-19T04:50:36.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.505+0000 7fa17de13740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cabb90 0x55b7b7cd06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.505+0000 7fa17de13740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cac4b0 0x55b7b7cb0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.505+0000 7fa17de13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cb10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.505+0000 7fa17de13740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741acbd0 con 0x55b7b7cabb90 2024-09-19T04:50:36.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.505+0000 7fa17de13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741ac500 con 0x55b7b7cb6d70 2024-09-19T04:50:36.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.505+0000 7fa17de13740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741a96e0 con 0x55b7b7cac4b0 2024-09-19T04:50:36.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.505+0000 7fa17de13740 10 client.0 _ll_register_callbacks cb 0x7fa16c09c440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:36.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.505+0000 7fa179400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cb10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:36.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.505+0000 7fa178a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cabb90 0x55b7b7cd06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:36.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.505+0000 7fa179400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cb10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:60634/0 (socket says 172.21.15.123:60634) 2024-09-19T04:50:36.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.505+0000 7fa179400640 1 -- 172.21.15.123:0/3170221956 learned_addr learned my addr 172.21.15.123:0/3170221956 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:36.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.505+0000 7fa16be00640 1 --2- 172.21.15.123:0/3170221956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cac4b0 0x55b7b7cb0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:36.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.506+0000 7fa179400640 1 -- 172.21.15.123:0/3170221956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cac4b0 msgr2=0x55b7b7cb0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:36.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.506+0000 7fa179400640 1 --2- 172.21.15.123:0/3170221956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cac4b0 0x55b7b7cb0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.506+0000 7fa179400640 1 -- 172.21.15.123:0/3170221956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cabb90 msgr2=0x55b7b7cd06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.506+0000 7fa179400640 1 --2- 172.21.15.123:0/3170221956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cabb90 0x55b7b7cd06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.506+0000 7fa179400640 1 -- 172.21.15.123:0/3170221956 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7b7cb1920 con 0x55b7b7cb6d70 2024-09-19T04:50:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.506+0000 7fa16be00640 1 --2- 172.21.15.123:0/3170221956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cac4b0 0x55b7b7cb0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:50:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.506+0000 7fa179400640 1 --2- 172.21.15.123:0/3170221956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cb10f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa16c09c530 tx=0x7fa16c005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d28e447b2a2e2505 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.506+0000 7fa162000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:50:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.506+0000 7fa162000640 1 -- 172.21.15.123:0/3170221956 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c0b4070 con 0x55b7b7cb6d70 2024-09-19T04:50:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.506+0000 7fa162000640 1 -- 172.21.15.123:0/3170221956 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fa16c07d030 con 0x55b7b7cb6d70 2024-09-19T04:50:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.506+0000 7fa17de13740 1 -- 172.21.15.123:0/3170221956 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa164004380 con 0x55b7b7cb6d70 2024-09-19T04:50:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.506+0000 7fa162000640 1 -- 172.21.15.123:0/3170221956 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa16c0a5210 con 0x55b7b7cb6d70 2024-09-19T04:50:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.507+0000 7fa17de13740 10 client.26015 fetch_fsmap learned FSMap version 33 2024-09-19T04:50:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.507+0000 7fa17de13740 1 -- 172.21.15.123:0/3170221956 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b7b7d42a90 con 0x55b7b7cb6d70 2024-09-19T04:50:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.507+0000 7fa162000640 1 -- 172.21.15.123:0/3170221956 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa164004380 con 0x55b7b7cb6d70 2024-09-19T04:50:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.507+0000 7fa162000640 1 -- 172.21.15.123:0/3170221956 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fa16c005400 con 0x55b7b7cb6d70 2024-09-19T04:50:36.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.507+0000 7fa17de13740 10 client.26015 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:50:36.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.507+0000 7fa17de13740 10 client.26015 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:50:36.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.507+0000 7fa17de13740 20 client.26015 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:36.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.507+0000 7fa17de13740 1 --2- 172.21.15.123:0/3170221956 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b7b7d42fd0 0x55b7b7d453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:36.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.507+0000 7fa17de13740 4 client.26015 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:50:36.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.507+0000 7fa17de13740 1 -- 172.21.15.123:0/3170221956 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b7b7c19e60 con 0x55b7b7d42fd0 2024-09-19T04:50:36.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.507+0000 7fa178a00640 1 --2- 172.21.15.123:0/3170221956 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b7b7d42fd0 0x55b7b7d453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:36.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.508+0000 7fa178a00640 1 --2- 172.21.15.123:0/3170221956 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b7b7d42fd0 0x55b7b7d453f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91923169638da5dc server_cookie=b4dc46f27ffd604e in_seq=0 out_seq=0 2024-09-19T04:50:36.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.508+0000 7fa162000640 10 client.26015 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:36.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.509+0000 7fa162000640 1 -- 172.21.15.123:0/3170221956 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b7b7c19e60 con 0x55b7b7d42fd0 2024-09-19T04:50:36.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.509+0000 7fa162000640 10 client.26015 handle_command_reply: tid=0 2024-09-19T04:50:36.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.509+0000 7fa17de13740 1 client.26015 shutdown 2024-09-19T04:50:36.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.509+0000 7fa17de13740 1 -- 172.21.15.123:0/3170221956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cb6d70 msgr2=0x55b7b7cb10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:36.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.509+0000 7fa17de13740 1 --2- 172.21.15.123:0/3170221956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cb10f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa16c09c530 tx=0x7fa16c005970 comp rx=0 tx=0).stop 2024-09-19T04:50:36.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.509+0000 7fa17de13740 1 -- 172.21.15.123:0/3170221956 shutdown_connections 2024-09-19T04:50:36.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.509+0000 7fa17de13740 1 --2- 172.21.15.123:0/3170221956 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b7b7d42fd0 0x55b7b7d453f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.509+0000 7fa17de13740 1 --2- 172.21.15.123:0/3170221956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7b7cb6d70 0x55b7b7cb10f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.509+0000 7fa17de13740 1 --2- 172.21.15.123:0/3170221956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7b7cac4b0 0x55b7b7cb0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.509+0000 7fa17de13740 1 --2- 172.21.15.123:0/3170221956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7b7cabb90 0x55b7b7cd06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.509+0000 7fa17de13740 1 -- 172.21.15.123:0/3170221956 >> 172.21.15.123:0/3170221956 conn(0x55b7b7ca8120 msgr2=0x55b7b7ca69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:36.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.509+0000 7fa162000640 0 client.26015 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:36.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.510+0000 7fa17de13740 1 -- 172.21.15.123:0/3170221956 shutdown_connections 2024-09-19T04:50:36.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.510+0000 7fa17de13740 1 -- 172.21.15.123:0/3170221956 wait complete. 2024-09-19T04:50:36.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.510+0000 7fa17de13740 20 client.26015 trim_cache size 0 max 16384 2024-09-19T04:50:36.514 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:50:36.514 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-09-19T04:50:36.514 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-09-19T04:50:36.514 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:50:36.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.510+0000 7fa17b400640 1 -- 172.21.15.123:0/1578978504 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa1580a3430 msgr2=0x7fa1580a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:36.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.510+0000 7fa17b400640 1 --2- 172.21.15.123:0/1578978504 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa1580a3430 0x7fa1580a58f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa164009780 tx=0x7fa164030c50 comp rx=0 tx=0).stop 2024-09-19T04:50:36.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.510+0000 7fa17b400640 1 -- 172.21.15.123:0/1578978504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa174109140 msgr2=0x7fa17419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:36.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.510+0000 7fa17b400640 1 --2- 172.21.15.123:0/1578978504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa174109140 0x7fa17419ef20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00eb10 tx=0x7fa16c00c2d0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.510+0000 7fa17b400640 1 -- 172.21.15.123:0/1578978504 shutdown_connections 2024-09-19T04:50:36.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.510+0000 7fa17b400640 1 --2- 172.21.15.123:0/1578978504 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa1580a3430 0x7fa1580a58f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.510+0000 7fa17b400640 1 --2- 172.21.15.123:0/1578978504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa174109140 0x7fa17419ef20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.510+0000 7fa17b400640 1 --2- 172.21.15.123:0/1578978504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1741049c0 0x7fa17419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.510+0000 7fa17b400640 1 --2- 172.21.15.123:0/1578978504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa174104010 0x7fa17419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:36.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.511+0000 7fa17b400640 1 -- 172.21.15.123:0/1578978504 >> 172.21.15.123:0/1578978504 conn(0x7fa1740ffa50 msgr2=0x7fa174100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:36.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.511+0000 7fa17b400640 1 -- 172.21.15.123:0/1578978504 shutdown_connections 2024-09-19T04:50:36.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:36.511+0000 7fa17b400640 1 -- 172.21.15.123:0/1578978504 wait complete. 2024-09-19T04:50:37.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:36 smithi123 ceph-mon[30511]: pgmap v120: 97 pgs: 97 active+clean; 131 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.1 MiB/s wr, 485 op/s 2024-09-19T04:50:37.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:36 smithi193 ceph-mon[35592]: pgmap v120: 97 pgs: 97 active+clean; 131 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.1 MiB/s wr, 485 op/s 2024-09-19T04:50:37.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:36 smithi183 ceph-mon[35333]: pgmap v120: 97 pgs: 97 active+clean; 131 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.1 MiB/s wr, 485 op/s 2024-09-19T04:50:37.432 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:50:37.432 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:50:37.911 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:50:38.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.517+0000 7fc4b3600640 1 -- 172.21.15.123:0/1359323161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ac102790 msgr2=0x7fc4ac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:38.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.517+0000 7fc4b3600640 1 --2- 172.21.15.123:0/1359323161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ac102790 0x7fc4ac1067e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc498009900 tx=0x7fc49802a410 comp rx=0 tx=0).stop 2024-09-19T04:50:38.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.517+0000 7fc4b3600640 1 -- 172.21.15.123:0/1359323161 shutdown_connections 2024-09-19T04:50:38.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.517+0000 7fc4b3600640 1 --2- 172.21.15.123:0/1359323161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4ac106f10 0x7fc4ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:38.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.517+0000 7fc4b3600640 1 --2- 172.21.15.123:0/1359323161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ac102790 0x7fc4ac1067e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:38.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.517+0000 7fc4b3600640 1 --2- 172.21.15.123:0/1359323161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4ac101de0 0x7fc4ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:38.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.517+0000 7fc4b3600640 1 -- 172.21.15.123:0/1359323161 >> 172.21.15.123:0/1359323161 conn(0x7fc4ac0fd8c0 msgr2=0x7fc4ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:38.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.517+0000 7fc4b3600640 1 -- 172.21.15.123:0/1359323161 shutdown_connections 2024-09-19T04:50:38.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.517+0000 7fc4b3600640 1 -- 172.21.15.123:0/1359323161 wait complete. 2024-09-19T04:50:38.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.518+0000 7fc4b3600640 1 Processor -- start 2024-09-19T04:50:38.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.519+0000 7fc4b3600640 1 -- start start 2024-09-19T04:50:38.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.519+0000 7fc4b3600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4ac101de0 0x7fc4ac198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:38.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.519+0000 7fc4b3600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ac102790 0x7fc4ac1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:38.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.519+0000 7fc4b3600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4ac106f10 0x7fc4ac19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:38.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.519+0000 7fc4b3600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac111fc0 con 0x7fc4ac106f10 2024-09-19T04:50:38.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.519+0000 7fc4b3600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac111e40 con 0x7fc4ac102790 2024-09-19T04:50:38.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.519+0000 7fc4b3600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac112140 con 0x7fc4ac101de0 2024-09-19T04:50:38.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.519+0000 7fc4b0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4ac101de0 0x7fc4ac198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:38.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.519+0000 7fc4abe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ac102790 0x7fc4ac1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:38.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.519+0000 7fc4b0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4ac101de0 0x7fc4ac198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49670/0 (socket says 172.21.15.123:49670) 2024-09-19T04:50:38.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.519+0000 7fc4b0c00640 1 -- 172.21.15.123:0/3175103557 learned_addr learned my addr 172.21.15.123:0/3175103557 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:38.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.519+0000 7fc4b1600640 1 --2- 172.21.15.123:0/3175103557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4ac106f10 0x7fc4ac19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:38.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.520+0000 7fc4b0c00640 1 -- 172.21.15.123:0/3175103557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ac102790 msgr2=0x7fc4ac1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:38.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.520+0000 7fc4b0c00640 1 --2- 172.21.15.123:0/3175103557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ac102790 0x7fc4ac1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:38.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.520+0000 7fc4b0c00640 1 -- 172.21.15.123:0/3175103557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4ac106f10 msgr2=0x7fc4ac19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:38.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.520+0000 7fc4b0c00640 1 --2- 172.21.15.123:0/3175103557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4ac106f10 0x7fc4ac19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:38.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.520+0000 7fc4b0c00640 1 -- 172.21.15.123:0/3175103557 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ac19d410 con 0x7fc4ac101de0 2024-09-19T04:50:38.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.520+0000 7fc4abe00640 1 --2- 172.21.15.123:0/3175103557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ac102790 0x7fc4ac1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:38.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.520+0000 7fc4b0c00640 1 --2- 172.21.15.123:0/3175103557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4ac101de0 0x7fc4ac198460 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000e9f0 tx=0x7fc4a000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:38.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.520+0000 7fc4a9600640 1 -- 172.21.15.123:0/3175103557 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a000cd80 con 0x7fc4ac101de0 2024-09-19T04:50:38.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.520+0000 7fc4a9600640 1 -- 172.21.15.123:0/3175103557 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4a0010910 con 0x7fc4ac101de0 2024-09-19T04:50:38.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.520+0000 7fc4b3600640 1 -- 172.21.15.123:0/3175103557 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ac19d700 con 0x7fc4ac101de0 2024-09-19T04:50:38.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.521+0000 7fc4b3600640 1 -- 172.21.15.123:0/3175103557 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ac1a4fe0 con 0x7fc4ac101de0 2024-09-19T04:50:38.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.521+0000 7fc4a9600640 1 -- 172.21.15.123:0/3175103557 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0018990 con 0x7fc4ac101de0 2024-09-19T04:50:38.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.522+0000 7fc4b3600640 1 -- 172.21.15.123:0/3175103557 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc48c005290 con 0x7fc4ac101de0 2024-09-19T04:50:38.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.525+0000 7fc4a9600640 1 -- 172.21.15.123:0/3175103557 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc4a0018b30 con 0x7fc4ac101de0 2024-09-19T04:50:38.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.528+0000 7fc4a9600640 1 --2- 172.21.15.123:0/3175103557 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc4940a3430 0x7fc4940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:38.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.528+0000 7fc4abe00640 1 --2- 172.21.15.123:0/3175103557 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc4940a3430 0x7fc4940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:38.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.528+0000 7fc4a9600640 1 -- 172.21.15.123:0/3175103557 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fc4a00cffe0 con 0x7fc4ac101de0 2024-09-19T04:50:38.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.529+0000 7fc4a9600640 1 -- 172.21.15.123:0/3175103557 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc4a00d04a0 con 0x7fc4ac101de0 2024-09-19T04:50:38.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.529+0000 7fc4abe00640 1 --2- 172.21.15.123:0/3175103557 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc4940a3430 0x7fc4940a58f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc498000c00 tx=0x7fc498031510 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:38.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.801+0000 7fc4b3600640 1 -- 172.21.15.123:0/3175103557 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc48c005580 con 0x7fc4ac101de0 2024-09-19T04:50:38.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.801+0000 7fc4a9600640 1 -- 172.21.15.123:0/3175103557 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11994 (secure 0 0 0) 0x7fc4a009c9d0 con 0x7fc4ac101de0 2024-09-19T04:50:38.804 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:50:38.804 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":33,"btime":"2024-09-19T04:50:18:942001+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:50:18.831090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24529,24505,24539,24500,24499]},"id":1}]} 2024-09-19T04:50:38.805 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 33 2024-09-19T04:50:38.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.805+0000 7fc4b3600640 1 -- 172.21.15.123:0/3175103557 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc4940a3430 msgr2=0x7fc4940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:38.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.805+0000 7fc4b3600640 1 --2- 172.21.15.123:0/3175103557 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc4940a3430 0x7fc4940a58f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc498000c00 tx=0x7fc498031510 comp rx=0 tx=0).stop 2024-09-19T04:50:38.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.805+0000 7fc4b3600640 1 -- 172.21.15.123:0/3175103557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4ac101de0 msgr2=0x7fc4ac198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:38.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.805+0000 7fc4b3600640 1 --2- 172.21.15.123:0/3175103557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4ac101de0 0x7fc4ac198460 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000e9f0 tx=0x7fc4a000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:50:38.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.805+0000 7fc4b3600640 1 -- 172.21.15.123:0/3175103557 shutdown_connections 2024-09-19T04:50:38.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.805+0000 7fc4b3600640 1 --2- 172.21.15.123:0/3175103557 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc4940a3430 0x7fc4940a58f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:38.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.805+0000 7fc4b3600640 1 --2- 172.21.15.123:0/3175103557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4ac106f10 0x7fc4ac19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:38.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.805+0000 7fc4b3600640 1 --2- 172.21.15.123:0/3175103557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ac102790 0x7fc4ac1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:38.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.805+0000 7fc4b3600640 1 --2- 172.21.15.123:0/3175103557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4ac101de0 0x7fc4ac198460 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:38.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.805+0000 7fc4b3600640 1 -- 172.21.15.123:0/3175103557 >> 172.21.15.123:0/3175103557 conn(0x7fc4ac0fd8c0 msgr2=0x7fc4ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:38.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.806+0000 7fc4b3600640 1 -- 172.21.15.123:0/3175103557 shutdown_connections 2024-09-19T04:50:38.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:38.806+0000 7fc4b3600640 1 -- 172.21.15.123:0/3175103557 wait complete. 2024-09-19T04:50:39.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:38 smithi123 ceph-mon[30511]: pgmap v121: 97 pgs: 97 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.5 MiB/s wr, 715 op/s 2024-09-19T04:50:39.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:38 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3175103557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:50:39.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:38 smithi193 ceph-mon[35592]: pgmap v121: 97 pgs: 97 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.5 MiB/s wr, 715 op/s 2024-09-19T04:50:39.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:38 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3175103557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:50:39.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:38 smithi183 ceph-mon[35333]: pgmap v121: 97 pgs: 97 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.5 MiB/s wr, 715 op/s 2024-09-19T04:50:39.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:38 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3175103557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:50:39.631 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:50:40.078 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:50:40.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:39 smithi193 ceph-mon[35592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:50:40.265 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:39 smithi123 ceph-mon[30511]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:50:40.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:39 smithi183 ceph-mon[35333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:50:40.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.695+0000 7fa1b6200640 1 -- 172.21.15.123:0/1275623441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b0106f10 msgr2=0x7fa1b010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:40.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.695+0000 7fa1b6200640 1 --2- 172.21.15.123:0/1275623441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b0106f10 0x7fa1b010ea90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009ca0 tx=0x7fa1a802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.695+0000 7fa1b6200640 1 -- 172.21.15.123:0/1275623441 shutdown_connections 2024-09-19T04:50:40.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.695+0000 7fa1b6200640 1 --2- 172.21.15.123:0/1275623441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b0106f10 0x7fa1b010ea90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.695+0000 7fa1b6200640 1 --2- 172.21.15.123:0/1275623441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1b0102790 0x7fa1b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.695+0000 7fa1b6200640 1 --2- 172.21.15.123:0/1275623441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1b0101de0 0x7fa1b01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.695+0000 7fa1b6200640 1 -- 172.21.15.123:0/1275623441 >> 172.21.15.123:0/1275623441 conn(0x7fa1b00fd8c0 msgr2=0x7fa1b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:40.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.695+0000 7fa1b6200640 1 -- 172.21.15.123:0/1275623441 shutdown_connections 2024-09-19T04:50:40.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.695+0000 7fa1b6200640 1 -- 172.21.15.123:0/1275623441 wait complete. 2024-09-19T04:50:40.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.696+0000 7fa1b6200640 1 Processor -- start 2024-09-19T04:50:40.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.697+0000 7fa1b6200640 1 -- start start 2024-09-19T04:50:40.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.697+0000 7fa1b6200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1b0101de0 0x7fa1b019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.697+0000 7fa1b6200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b0102790 0x7fa1b019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.697+0000 7fa1b6200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1b0106f10 0x7fa1b019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.697+0000 7fa1b6200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0111fc0 con 0x7fa1b0101de0 2024-09-19T04:50:40.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.697+0000 7fa1b6200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0111e40 con 0x7fa1b0106f10 2024-09-19T04:50:40.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.697+0000 7fa1b6200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0112140 con 0x7fa1b0102790 2024-09-19T04:50:40.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.697+0000 7fa1afe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1b0106f10 0x7fa1b019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:40.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.697+0000 7fa1aea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b0102790 0x7fa1b019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:40.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.697+0000 7fa1aea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b0102790 0x7fa1b019ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:49694/0 (socket says 172.21.15.123:49694) 2024-09-19T04:50:40.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.697+0000 7fa1aea00640 1 -- 172.21.15.123:0/673177653 learned_addr learned my addr 172.21.15.123:0/673177653 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:40.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.697+0000 7fa1afe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1b0106f10 0x7fa1b019eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:60672/0 (socket says 172.21.15.123:60672) 2024-09-19T04:50:40.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.698+0000 7fa1aea00640 1 -- 172.21.15.123:0/673177653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1b0106f10 msgr2=0x7fa1b019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:40.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.698+0000 7fa1aea00640 1 --2- 172.21.15.123:0/673177653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1b0106f10 0x7fa1b019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.698+0000 7fa1aea00640 1 -- 172.21.15.123:0/673177653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1b0101de0 msgr2=0x7fa1b019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:40.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.698+0000 7fa1aea00640 1 --2- 172.21.15.123:0/673177653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1b0101de0 0x7fa1b019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.698+0000 7fa1aea00640 1 -- 172.21.15.123:0/673177653 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b019f5a0 con 0x7fa1b0102790 2024-09-19T04:50:40.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.698+0000 7fa1afe00640 1 --2- 172.21.15.123:0/673177653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1b0106f10 0x7fa1b019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:50:40.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.699+0000 7fa1aea00640 1 --2- 172.21.15.123:0/673177653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b0102790 0x7fa1b019ab30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00ea60 tx=0x7fa19c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:40.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.699+0000 7fa1a7e00640 1 -- 172.21.15.123:0/673177653 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c00cde0 con 0x7fa1b0102790 2024-09-19T04:50:40.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.699+0000 7fa1a7e00640 1 -- 172.21.15.123:0/673177653 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa19c0035f0 con 0x7fa1b0102790 2024-09-19T04:50:40.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.699+0000 7fa1b6200640 1 -- 172.21.15.123:0/673177653 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b019f890 con 0x7fa1b0102790 2024-09-19T04:50:40.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.699+0000 7fa1a7e00640 1 -- 172.21.15.123:0/673177653 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c010670 con 0x7fa1b0102790 2024-09-19T04:50:40.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.699+0000 7fa1b6200640 1 -- 172.21.15.123:0/673177653 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b0108a40 con 0x7fa1b0102790 2024-09-19T04:50:40.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.703+0000 7fa1a7e00640 1 -- 172.21.15.123:0/673177653 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa19c010810 con 0x7fa1b0102790 2024-09-19T04:50:40.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.703+0000 7fa1b8e35740 1 Processor -- start 2024-09-19T04:50:40.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.703+0000 7fa1b8e35740 1 -- start start 2024-09-19T04:50:40.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.704+0000 7fa1b8e35740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bf3c70 0x563e60bf4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.704+0000 7fa1b8e35740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bf4590 0x563e60bfe910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.704+0000 7fa1b8e35740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bfee50 0x563e60c01240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.704+0000 7fa1b8e35740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563e60b5a3e0 con 0x563e60bf4590 2024-09-19T04:50:40.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.704+0000 7fa1b8e35740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563e60b01d80 con 0x563e60bfee50 2024-09-19T04:50:40.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.704+0000 7fa1b8e35740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563e60b563a0 con 0x563e60bf3c70 2024-09-19T04:50:40.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.704+0000 7fa1af400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bf4590 0x563e60bfe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:40.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.704+0000 7fa1af400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bf4590 0x563e60bfe910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54632/0 (socket says 172.21.15.123:54632) 2024-09-19T04:50:40.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.704+0000 7fa1af400640 1 -- 172.21.15.123:0/1098180705 learned_addr learned my addr 172.21.15.123:0/1098180705 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:40.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.704+0000 7fa1aea00640 1 --2- 172.21.15.123:0/1098180705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bfee50 0x563e60c01240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:40.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.705+0000 7fa1afe00640 1 --2- 172.21.15.123:0/1098180705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bf3c70 0x563e60bf4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:40.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.705+0000 7fa1af400640 1 -- 172.21.15.123:0/1098180705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bf3c70 msgr2=0x563e60bf4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:40.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.705+0000 7fa1af400640 1 --2- 172.21.15.123:0/1098180705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bf3c70 0x563e60bf4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.705+0000 7fa1af400640 1 -- 172.21.15.123:0/1098180705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bfee50 msgr2=0x563e60c01240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:40.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.705+0000 7fa1af400640 1 --2- 172.21.15.123:0/1098180705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bfee50 0x563e60c01240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.705+0000 7fa1af400640 1 -- 172.21.15.123:0/1098180705 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e60b20ae0 con 0x563e60bf4590 2024-09-19T04:50:40.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.705+0000 7fa1a7e00640 1 --2- 172.21.15.123:0/673177653 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa1940a3430 0x7fa1940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.705+0000 7fa1af400640 1 --2- 172.21.15.123:0/1098180705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bf4590 0x563e60bfe910 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009c40 tx=0x7fa1a000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92fdb3a2504476cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:40.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.705+0000 7fa1a7e00640 1 -- 172.21.15.123:0/673177653 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fa19c0d0100 con 0x7fa1b0102790 2024-09-19T04:50:40.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.706+0000 7fa1afe00640 1 --2- 172.21.15.123:0/673177653 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa1940a3430 0x7fa1940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:40.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.706+0000 7fa19be00640 1 -- 172.21.15.123:0/1098180705 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0005020 con 0x563e60bf4590 2024-09-19T04:50:40.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.706+0000 7fa19be00640 1 -- 172.21.15.123:0/1098180705 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a00047a0 con 0x563e60bf4590 2024-09-19T04:50:40.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.706+0000 7fa19be00640 1 -- 172.21.15.123:0/1098180705 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0010760 con 0x563e60bf4590 2024-09-19T04:50:40.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.707+0000 7fa1b8e35740 1 -- 172.21.15.123:0/1098180705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bf4590 msgr2=0x563e60bfe910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:40.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.707+0000 7fa1afe00640 1 --2- 172.21.15.123:0/673177653 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa1940a3430 0x7fa1940a58f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8004af0 tx=0x7fa1a8004a40 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:40.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.707+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/1098180705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bf4590 0x563e60bfe910 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009c40 tx=0x7fa1a000e720 comp rx=0 tx=0).stop 2024-09-19T04:50:40.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.707+0000 7fa1b8e35740 1 -- 172.21.15.123:0/1098180705 shutdown_connections 2024-09-19T04:50:40.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.707+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/1098180705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bfee50 0x563e60c01240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.707+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/1098180705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bf4590 0x563e60bfe910 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.707+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/1098180705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bf3c70 0x563e60bf4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.707+0000 7fa1b8e35740 1 -- 172.21.15.123:0/1098180705 >> 172.21.15.123:0/1098180705 conn(0x563e60bf0200 msgr2=0x563e60bf05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:40.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.707+0000 7fa1b8e35740 1 -- 172.21.15.123:0/1098180705 shutdown_connections 2024-09-19T04:50:40.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.707+0000 7fa1b8e35740 1 -- 172.21.15.123:0/1098180705 wait complete. 2024-09-19T04:50:40.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.707+0000 7fa1b8e35740 1 -- start start 2024-09-19T04:50:40.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.708+0000 7fa1b8e35740 1 Processor -- start 2024-09-19T04:50:40.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.708+0000 7fa1b8e35740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bf3c70 0x563e60bfbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.708+0000 7fa1b8e35740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf4590 0x563e60bfc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.708+0000 7fa1b8e35740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bfee50 0x563e60bfc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.708+0000 7fa1b8e35740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563e60c01a20 con 0x563e60bf3c70 2024-09-19T04:50:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.708+0000 7fa1b8e35740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563e60b150f0 con 0x563e60bf4590 2024-09-19T04:50:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.708+0000 7fa1b8e35740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563e60c01ba0 con 0x563e60bfee50 2024-09-19T04:50:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.708+0000 7fa1b8e35740 10 client.0 _ll_register_callbacks cb 0x563e60afcbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.708+0000 7fa1aea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf4590 0x563e60bfc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.708+0000 7fa1af400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bf3c70 0x563e60bfbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.708+0000 7fa1aea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf4590 0x563e60bfc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:60692/0 (socket says 172.21.15.123:60692) 2024-09-19T04:50:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.708+0000 7fa1aea00640 1 -- 172.21.15.123:0/3595506998 learned_addr learned my addr 172.21.15.123:0/3595506998 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.709+0000 7fa1afe00640 1 --2- 172.21.15.123:0/3595506998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bfee50 0x563e60bfc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.709+0000 7fa1aea00640 1 -- 172.21.15.123:0/3595506998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bfee50 msgr2=0x563e60bfc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:40.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.709+0000 7fa1aea00640 1 --2- 172.21.15.123:0/3595506998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bfee50 0x563e60bfc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.709+0000 7fa1aea00640 1 -- 172.21.15.123:0/3595506998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bf3c70 msgr2=0x563e60bfbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:40.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.709+0000 7fa1aea00640 1 --2- 172.21.15.123:0/3595506998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bf3c70 0x563e60bfbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.709+0000 7fa1aea00640 1 -- 172.21.15.123:0/3595506998 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e608cb3c0 con 0x563e60bf4590 2024-09-19T04:50:40.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.709+0000 7fa1af400640 1 --2- 172.21.15.123:0/3595506998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bf3c70 0x563e60bfbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:40.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.709+0000 7fa1aea00640 1 --2- 172.21.15.123:0/3595506998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf4590 0x563e60bfc240 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00c760 tx=0x7fa19c09c180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db20b38713556b48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:40.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.709+0000 7fa198c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:50:40.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.709+0000 7fa198c00640 1 -- 172.21.15.123:0/3595506998 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c09cab0 con 0x563e60bf4590 2024-09-19T04:50:40.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.709+0000 7fa198c00640 1 -- 172.21.15.123:0/3595506998 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fa19c098a20 con 0x563e60bf4590 2024-09-19T04:50:40.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.709+0000 7fa1b8e35740 1 -- 172.21.15.123:0/3595506998 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a00047a0 con 0x563e60bf4590 2024-09-19T04:50:40.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.710+0000 7fa198c00640 1 -- 172.21.15.123:0/3595506998 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa19c09c3b0 con 0x563e60bf4590 2024-09-19T04:50:40.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.710+0000 7fa1b8e35740 10 client.26033 fetch_fsmap learned FSMap version 34 2024-09-19T04:50:40.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.710+0000 7fa1b8e35740 1 -- 172.21.15.123:0/3595506998 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563e60c0da30 con 0x563e60bf4590 2024-09-19T04:50:40.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.710+0000 7fa198c00640 1 -- 172.21.15.123:0/3595506998 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a00047a0 con 0x563e60bf4590 2024-09-19T04:50:40.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.710+0000 7fa198c00640 1 -- 172.21.15.123:0/3595506998 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa19c0037a0 con 0x563e60bf4590 2024-09-19T04:50:40.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.710+0000 7fa1b8e35740 10 client.26033 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:50:40.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.710+0000 7fa1b8e35740 10 client.26033 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:50:40.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.710+0000 7fa1b8e35740 20 client.26033 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:40.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.711+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/3595506998 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x563e60c0eef0 0x563e60c11360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.711+0000 7fa1b8e35740 4 client.26033 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:50:40.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.711+0000 7fa1b8e35740 1 -- 172.21.15.123:0/3595506998 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e60b67950 con 0x563e60c0eef0 2024-09-19T04:50:40.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.711+0000 7fa1af400640 1 --2- 172.21.15.123:0/3595506998 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x563e60c0eef0 0x563e60c11360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:40.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.711+0000 7fa1af400640 1 --2- 172.21.15.123:0/3595506998 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x563e60c0eef0 0x563e60c11360 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de508ebe7c583318 server_cookie=4748aceacb7f4b00 in_seq=0 out_seq=0 2024-09-19T04:50:40.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.712+0000 7fa198c00640 10 client.26033 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:50:40.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.712+0000 7fa198c00640 1 -- 172.21.15.123:0/3595506998 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e60b67950 con 0x563e60c0eef0 2024-09-19T04:50:40.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.712+0000 7fa198c00640 10 client.26033 handle_command_reply: tid=0 2024-09-19T04:50:40.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.712+0000 7fa1b8e35740 1 client.26033 shutdown 2024-09-19T04:50:40.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.713+0000 7fa1b8e35740 1 -- 172.21.15.123:0/3595506998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf4590 msgr2=0x563e60bfc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:40.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.713+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/3595506998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf4590 0x563e60bfc240 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00c760 tx=0x7fa19c09c180 comp rx=0 tx=0).stop 2024-09-19T04:50:40.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.713+0000 7fa1b8e35740 1 -- 172.21.15.123:0/3595506998 shutdown_connections 2024-09-19T04:50:40.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.713+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/3595506998 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x563e60c0eef0 0x563e60c11360 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.713+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/3595506998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bfee50 0x563e60bfc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.713+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/3595506998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf4590 0x563e60bfc240 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.713+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/3595506998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bf3c70 0x563e60bfbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.713+0000 7fa1b8e35740 1 -- 172.21.15.123:0/3595506998 >> 172.21.15.123:0/3595506998 conn(0x563e60bf0200 msgr2=0x563e60bf5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:40.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.713+0000 7fa198c00640 0 client.26033 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:50:40.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.713+0000 7fa1b8e35740 1 -- 172.21.15.123:0/3595506998 shutdown_connections 2024-09-19T04:50:40.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.713+0000 7fa1b8e35740 1 -- 172.21.15.123:0/3595506998 wait complete. 2024-09-19T04:50:40.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.715+0000 7fa1b8e35740 20 client.26033 trim_cache size 0 max 16384 2024-09-19T04:50:40.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.732+0000 7fa1b8e35740 1 Processor -- start 2024-09-19T04:50:40.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.732+0000 7fa1b8e35740 1 -- start start 2024-09-19T04:50:40.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.732+0000 7fa1b8e35740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf3c70 0x563e60bfdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.732+0000 7fa1b8e35740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bf4590 0x563e60c16c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.732+0000 7fa1b8e35740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bfee50 0x563e60c17170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.732+0000 7fa1b8e35740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01b3fa0 con 0x563e60bfee50 2024-09-19T04:50:40.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.732+0000 7fa1b8e35740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01b5b40 con 0x563e60bf3c70 2024-09-19T04:50:40.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.732+0000 7fa1b8e35740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01b4670 con 0x563e60bf4590 2024-09-19T04:50:40.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.733+0000 7fa1af400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf3c70 0x563e60bfdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:40.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.733+0000 7fa1af400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf3c70 0x563e60bfdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:60706/0 (socket says 172.21.15.123:60706) 2024-09-19T04:50:40.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.733+0000 7fa1af400640 1 -- 172.21.15.123:0/2982960959 learned_addr learned my addr 172.21.15.123:0/2982960959 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:40.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.733+0000 7fa1aea00640 1 --2- 172.21.15.123:0/2982960959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bf4590 0x563e60c16c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:40.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.733+0000 7fa1afe00640 1 --2- 172.21.15.123:0/2982960959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bfee50 0x563e60c17170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:40.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.733+0000 7fa1af400640 1 -- 172.21.15.123:0/2982960959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bf4590 msgr2=0x563e60c16c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:40.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.733+0000 7fa1af400640 1 --2- 172.21.15.123:0/2982960959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bf4590 0x563e60c16c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.733+0000 7fa1af400640 1 -- 172.21.15.123:0/2982960959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bfee50 msgr2=0x563e60c17170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:40.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.733+0000 7fa1af400640 1 --2- 172.21.15.123:0/2982960959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bfee50 0x563e60c17170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.733+0000 7fa1af400640 1 -- 172.21.15.123:0/2982960959 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e60bf43c0 con 0x563e60bf3c70 2024-09-19T04:50:40.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.733+0000 7fa1afe00640 1 --2- 172.21.15.123:0/2982960959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bfee50 0x563e60c17170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:50:40.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.733+0000 7fa1af400640 1 --2- 172.21.15.123:0/2982960959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf3c70 0x563e60bfdc70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009c40 tx=0x7fa1a000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1bff2d87ac4b5c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:40.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.734+0000 7fa19b400640 1 -- 172.21.15.123:0/2982960959 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a001d040 con 0x563e60bf3c70 2024-09-19T04:50:40.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.734+0000 7fa19b400640 1 -- 172.21.15.123:0/2982960959 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a0028070 con 0x563e60bf3c70 2024-09-19T04:50:40.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.734+0000 7fa19b400640 1 -- 172.21.15.123:0/2982960959 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0004a00 con 0x563e60bf3c70 2024-09-19T04:50:40.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.735+0000 7fa1b8e35740 1 -- 172.21.15.123:0/2982960959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf3c70 msgr2=0x563e60bfdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:40.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.735+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/2982960959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf3c70 0x563e60bfdc70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009c40 tx=0x7fa1a000ecd0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.735+0000 7fa1b8e35740 1 -- 172.21.15.123:0/2982960959 shutdown_connections 2024-09-19T04:50:40.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.735+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/2982960959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bfee50 0x563e60c17170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.735+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/2982960959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bf4590 0x563e60c16c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.735+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/2982960959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf3c70 0x563e60bfdc70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.735+0000 7fa1b8e35740 1 -- 172.21.15.123:0/2982960959 >> 172.21.15.123:0/2982960959 conn(0x563e60bf0200 msgr2=0x563e60bfd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:40.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.735+0000 7fa1b8e35740 1 -- 172.21.15.123:0/2982960959 shutdown_connections 2024-09-19T04:50:40.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.735+0000 7fa1b8e35740 1 -- 172.21.15.123:0/2982960959 wait complete. 2024-09-19T04:50:40.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.736+0000 7fa1b8e35740 1 -- start start 2024-09-19T04:50:40.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.736+0000 7fa1b8e35740 1 Processor -- start 2024-09-19T04:50:40.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.736+0000 7fa1b8e35740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf3c70 0x563e60c02c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.736+0000 7fa1b8e35740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bf4590 0x563e60c03180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.736+0000 7fa1b8e35740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bfee50 0x563e60c036c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.736+0000 7fa1b8e35740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01b5440 con 0x563e60bfee50 2024-09-19T04:50:40.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.736+0000 7fa1b8e35740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01b4d70 con 0x563e60bf3c70 2024-09-19T04:50:40.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.736+0000 7fa1b8e35740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01b1fc0 con 0x563e60bf4590 2024-09-19T04:50:40.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.736+0000 7fa1b8e35740 10 client.0 _ll_register_callbacks cb 0x7fa19c009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:40.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.737+0000 7fa1af400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf3c70 0x563e60c02c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:40.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.737+0000 7fa1af400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf3c70 0x563e60c02c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:60710/0 (socket says 172.21.15.123:60710) 2024-09-19T04:50:40.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.737+0000 7fa1af400640 1 -- 172.21.15.123:0/400120162 learned_addr learned my addr 172.21.15.123:0/400120162 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:40.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.737+0000 7fa1aea00640 1 --2- 172.21.15.123:0/400120162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bf4590 0x563e60c03180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:40.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.737+0000 7fa1afe00640 1 --2- 172.21.15.123:0/400120162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bfee50 0x563e60c036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:40.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.737+0000 7fa1af400640 1 -- 172.21.15.123:0/400120162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bf4590 msgr2=0x563e60c03180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:40.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.737+0000 7fa1af400640 1 --2- 172.21.15.123:0/400120162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bf4590 0x563e60c03180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.737+0000 7fa1af400640 1 -- 172.21.15.123:0/400120162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bfee50 msgr2=0x563e60c036c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:40.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.737+0000 7fa1af400640 1 --2- 172.21.15.123:0/400120162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bfee50 0x563e60c036c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.737+0000 7fa1af400640 1 -- 172.21.15.123:0/400120162 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e60c78d00 con 0x563e60bf3c70 2024-09-19T04:50:40.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.737+0000 7fa1aea00640 1 --2- 172.21.15.123:0/400120162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bf4590 0x563e60c03180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:40.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.737+0000 7fa1af400640 1 --2- 172.21.15.123:0/400120162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf3c70 0x563e60c02c40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00092e0 tx=0x7fa1a00057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d68dd6a699e4293 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:40.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.737+0000 7fa198c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:50:40.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.738+0000 7fa198c00640 1 -- 172.21.15.123:0/400120162 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0005120 con 0x563e60bf3c70 2024-09-19T04:50:40.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.738+0000 7fa198c00640 1 -- 172.21.15.123:0/400120162 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fa1a00052c0 con 0x563e60bf3c70 2024-09-19T04:50:40.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.738+0000 7fa1b8e35740 1 -- 172.21.15.123:0/400120162 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a0028070 con 0x563e60bf3c70 2024-09-19T04:50:40.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.738+0000 7fa198c00640 1 -- 172.21.15.123:0/400120162 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa1a003d050 con 0x563e60bf3c70 2024-09-19T04:50:40.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.738+0000 7fa1b8e35740 10 client.26039 fetch_fsmap learned FSMap version 34 2024-09-19T04:50:40.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.738+0000 7fa1b8e35740 1 -- 172.21.15.123:0/400120162 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563e60c79fe0 con 0x563e60bf3c70 2024-09-19T04:50:40.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.739+0000 7fa198c00640 1 -- 172.21.15.123:0/400120162 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a0028070 con 0x563e60bf3c70 2024-09-19T04:50:40.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.739+0000 7fa198c00640 1 -- 172.21.15.123:0/400120162 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa1a00035f0 con 0x563e60bf3c70 2024-09-19T04:50:40.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.739+0000 7fa1b8e35740 10 client.26039 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:50:40.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.739+0000 7fa1b8e35740 10 client.26039 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:50:40.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.739+0000 7fa1b8e35740 20 client.26039 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:40.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.739+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/400120162 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x563e60c7a4a0 0x563e60c7c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:40.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.739+0000 7fa1b8e35740 4 client.26039 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:50:40.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.739+0000 7fa1b8e35740 1 -- 172.21.15.123:0/400120162 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e60b61e60 con 0x563e60c7a4a0 2024-09-19T04:50:40.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.740+0000 7fa1aea00640 1 --2- 172.21.15.123:0/400120162 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x563e60c7a4a0 0x563e60c7c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:40.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.740+0000 7fa1aea00640 1 --2- 172.21.15.123:0/400120162 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x563e60c7a4a0 0x563e60c7c890 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2543fb438f3ad589 server_cookie=c3b5c64d97085424 in_seq=0 out_seq=0 2024-09-19T04:50:40.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.740+0000 7fa198c00640 10 client.26039 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:50:40.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.740+0000 7fa198c00640 1 -- 172.21.15.123:0/400120162 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e60b61e60 con 0x563e60c7a4a0 2024-09-19T04:50:40.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.740+0000 7fa198c00640 10 client.26039 handle_command_reply: tid=0 2024-09-19T04:50:40.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.740+0000 7fa1b8e35740 1 client.26039 shutdown 2024-09-19T04:50:40.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.741+0000 7fa1b8e35740 1 -- 172.21.15.123:0/400120162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf3c70 msgr2=0x563e60c02c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:40.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.741+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/400120162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf3c70 0x563e60c02c40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00092e0 tx=0x7fa1a00057c0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.741+0000 7fa1b8e35740 1 -- 172.21.15.123:0/400120162 shutdown_connections 2024-09-19T04:50:40.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.741+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/400120162 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x563e60c7a4a0 0x563e60c7c890 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.741+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/400120162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e60bfee50 0x563e60c036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.741+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/400120162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e60bf4590 0x563e60c03180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.741+0000 7fa198c00640 0 client.26039 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:50:40.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.741+0000 7fa1b8e35740 1 --2- 172.21.15.123:0/400120162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e60bf3c70 0x563e60c02c40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.741+0000 7fa1b8e35740 1 -- 172.21.15.123:0/400120162 >> 172.21.15.123:0/400120162 conn(0x563e60bf0200 msgr2=0x563e60bfc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:40.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.741+0000 7fa1b8e35740 1 -- 172.21.15.123:0/400120162 shutdown_connections 2024-09-19T04:50:40.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.741+0000 7fa1b8e35740 1 -- 172.21.15.123:0/400120162 wait complete. 2024-09-19T04:50:40.746 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:50:40.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.741+0000 7fa1b8e35740 20 client.26039 trim_cache size 0 max 16384 2024-09-19T04:50:40.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.742+0000 7fa1b6200640 1 -- 172.21.15.123:0/673177653 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa1940a3430 msgr2=0x7fa1940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:40.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.742+0000 7fa1b6200640 1 --2- 172.21.15.123:0/673177653 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa1940a3430 0x7fa1940a58f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8004af0 tx=0x7fa1a8004a40 comp rx=0 tx=0).stop 2024-09-19T04:50:40.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.742+0000 7fa1b6200640 1 -- 172.21.15.123:0/673177653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b0102790 msgr2=0x7fa1b019ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:40.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.742+0000 7fa1b6200640 1 --2- 172.21.15.123:0/673177653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b0102790 0x7fa1b019ab30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00ea60 tx=0x7fa19c00ef20 comp rx=0 tx=0).stop 2024-09-19T04:50:40.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.742+0000 7fa1b6200640 1 -- 172.21.15.123:0/673177653 shutdown_connections 2024-09-19T04:50:40.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.742+0000 7fa1b6200640 1 --2- 172.21.15.123:0/673177653 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa1940a3430 0x7fa1940a58f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.742+0000 7fa1b6200640 1 --2- 172.21.15.123:0/673177653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1b0106f10 0x7fa1b019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.742+0000 7fa1b6200640 1 --2- 172.21.15.123:0/673177653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b0102790 0x7fa1b019ab30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.742+0000 7fa1b6200640 1 --2- 172.21.15.123:0/673177653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1b0101de0 0x7fa1b019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:40.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.742+0000 7fa1b6200640 1 -- 172.21.15.123:0/673177653 >> 172.21.15.123:0/673177653 conn(0x7fa1b00fd8c0 msgr2=0x7fa1b00fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:40.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.743+0000 7fa1b6200640 1 -- 172.21.15.123:0/673177653 shutdown_connections 2024-09-19T04:50:40.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:40.743+0000 7fa1b6200640 1 -- 172.21.15.123:0/673177653 wait complete. 2024-09-19T04:50:41.015 INFO:tasks.workunit.client.0.smithi123.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-09-19T04:50:41.270 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:40 smithi123 ceph-mon[30511]: pgmap v122: 97 pgs: 97 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 2.9 MiB/s wr, 480 op/s 2024-09-19T04:50:41.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:40 smithi183 ceph-mon[35333]: pgmap v122: 97 pgs: 97 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 2.9 MiB/s wr, 480 op/s 2024-09-19T04:50:41.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:40 smithi193 ceph-mon[35592]: pgmap v122: 97 pgs: 97 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 2.9 MiB/s wr, 480 op/s 2024-09-19T04:50:41.687 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:50:42.121 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:50:42.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.736+0000 7f49e3200640 1 -- 172.21.15.123:0/707156504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49dc108f60 msgr2=0x7f49dc110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:42.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.736+0000 7f49e3200640 1 --2- 172.21.15.123:0/707156504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49dc108f60 0x7f49dc110ae0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009ca0 tx=0x7f49d802a8b0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.736+0000 7f49e3200640 1 -- 172.21.15.123:0/707156504 shutdown_connections 2024-09-19T04:50:42.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.736+0000 7f49e3200640 1 --2- 172.21.15.123:0/707156504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49dc108f60 0x7f49dc110ae0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.736+0000 7f49e3200640 1 --2- 172.21.15.123:0/707156504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49dc1047e0 0x7f49dc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.736+0000 7f49e3200640 1 --2- 172.21.15.123:0/707156504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49dc103e30 0x7f49dc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.736+0000 7f49e3200640 1 -- 172.21.15.123:0/707156504 >> 172.21.15.123:0/707156504 conn(0x7f49dc0ff910 msgr2=0x7f49dc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:42.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.736+0000 7f49e3200640 1 -- 172.21.15.123:0/707156504 shutdown_connections 2024-09-19T04:50:42.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.736+0000 7f49e3200640 1 -- 172.21.15.123:0/707156504 wait complete. 2024-09-19T04:50:42.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.737+0000 7f49e3200640 1 Processor -- start 2024-09-19T04:50:42.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.738+0000 7f49e3200640 1 -- start start 2024-09-19T04:50:42.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.738+0000 7f49e3200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49dc103e30 0x7f49dc19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.738+0000 7f49e3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49dc1047e0 0x7f49dc19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.738+0000 7f49e3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49dc108f60 0x7f49dc19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.738+0000 7f49e3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc114010 con 0x7f49dc108f60 2024-09-19T04:50:42.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.738+0000 7f49e3200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc113e90 con 0x7f49dc1047e0 2024-09-19T04:50:42.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.738+0000 7f49e3200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc114190 con 0x7f49dc103e30 2024-09-19T04:50:42.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.738+0000 7f49e1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49dc103e30 0x7f49dc19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.738+0000 7f49e1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49dc103e30 0x7f49dc19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:59204/0 (socket says 172.21.15.123:59204) 2024-09-19T04:50:42.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.738+0000 7f49e1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49dc1047e0 0x7f49dc19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.738+0000 7f49e1e00640 1 -- 172.21.15.123:0/2677451267 learned_addr learned my addr 172.21.15.123:0/2677451267 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:42.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.739+0000 7f49e2800640 1 --2- 172.21.15.123:0/2677451267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49dc108f60 0x7f49dc19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.739+0000 7f49e1e00640 1 -- 172.21.15.123:0/2677451267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49dc1047e0 msgr2=0x7f49dc19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:42.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.739+0000 7f49e1e00640 1 --2- 172.21.15.123:0/2677451267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49dc1047e0 0x7f49dc19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.739+0000 7f49e1e00640 1 -- 172.21.15.123:0/2677451267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49dc108f60 msgr2=0x7f49dc19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:42.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.739+0000 7f49e1e00640 1 --2- 172.21.15.123:0/2677451267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49dc108f60 0x7f49dc19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.739+0000 7f49e1e00640 1 -- 172.21.15.123:0/2677451267 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49dc19f400 con 0x7f49dc103e30 2024-09-19T04:50:42.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.739+0000 7f49e1e00640 1 --2- 172.21.15.123:0/2677451267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49dc103e30 0x7f49dc19a450 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f49d000e9f0 tx=0x7f49d000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:42.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.739+0000 7f49d6a00640 1 -- 172.21.15.123:0/2677451267 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d000cd80 con 0x7f49dc103e30 2024-09-19T04:50:42.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.739+0000 7f49d6a00640 1 -- 172.21.15.123:0/2677451267 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d0010910 con 0x7f49dc103e30 2024-09-19T04:50:42.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.739+0000 7f49e3200640 1 -- 172.21.15.123:0/2677451267 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49dc19f6f0 con 0x7f49dc103e30 2024-09-19T04:50:42.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.740+0000 7f49e3200640 1 -- 172.21.15.123:0/2677451267 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49dc1a6fd0 con 0x7f49dc103e30 2024-09-19T04:50:42.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.740+0000 7f49d6a00640 1 -- 172.21.15.123:0/2677451267 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0018980 con 0x7f49dc103e30 2024-09-19T04:50:42.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.743+0000 7f49d6a00640 1 -- 172.21.15.123:0/2677451267 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f49d0010430 con 0x7f49dc103e30 2024-09-19T04:50:42.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.743+0000 7f49e71e4740 1 Processor -- start 2024-09-19T04:50:42.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.743+0000 7f49e71e4740 1 -- start start 2024-09-19T04:50:42.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.743+0000 7f49e71e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd116e40 0x555bcd117220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.743+0000 7f49e71e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd117760 0x555bcd121ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.744+0000 7f49e71e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd122020 0x555bcd124410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.744+0000 7f49e71e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555bcd07d3e0 con 0x555bcd117760 2024-09-19T04:50:42.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.744+0000 7f49e71e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555bcd024d80 con 0x555bcd116e40 2024-09-19T04:50:42.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.744+0000 7f49e71e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555bcd0793a0 con 0x555bcd122020 2024-09-19T04:50:42.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.744+0000 7f49e1400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd122020 0x555bcd124410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.744+0000 7f49e1400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd122020 0x555bcd124410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:59220/0 (socket says 172.21.15.123:59220) 2024-09-19T04:50:42.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.744+0000 7f49e1400640 1 -- 172.21.15.123:0/1784737060 learned_addr learned my addr 172.21.15.123:0/1784737060 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:42.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.744+0000 7f49e1e00640 1 --2- 172.21.15.123:0/1784737060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd117760 0x555bcd121ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.744+0000 7f49e2800640 1 --2- 172.21.15.123:0/1784737060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd116e40 0x555bcd117220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.744+0000 7f49e1400640 1 -- 172.21.15.123:0/1784737060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd116e40 msgr2=0x555bcd117220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:42.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.744+0000 7f49e1400640 1 --2- 172.21.15.123:0/1784737060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd116e40 0x555bcd117220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.744+0000 7f49e1400640 1 -- 172.21.15.123:0/1784737060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd117760 msgr2=0x555bcd121ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:42.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.744+0000 7f49e1400640 1 --2- 172.21.15.123:0/1784737060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd117760 0x555bcd121ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.744+0000 7f49e1400640 1 -- 172.21.15.123:0/1784737060 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bcd043ae0 con 0x555bcd122020 2024-09-19T04:50:42.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.745+0000 7f49d6a00640 1 --2- 172.21.15.123:0/2677451267 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f49c40a3430 0x7f49c40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.745+0000 7f49e2800640 1 --2- 172.21.15.123:0/1784737060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd116e40 0x555bcd117220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:50:42.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.745+0000 7f49e1400640 1 --2- 172.21.15.123:0/1784737060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd122020 0x555bcd124410 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009c20 tx=0x7f49cc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=417624b2e0ef3cd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:42.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.745+0000 7f49d6a00640 1 -- 172.21.15.123:0/2677451267 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f49d00cff90 con 0x7f49dc103e30 2024-09-19T04:50:42.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.745+0000 7f49e2800640 1 --2- 172.21.15.123:0/2677451267 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f49c40a3430 0x7f49c40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.745+0000 7f49cb000640 1 -- 172.21.15.123:0/1784737060 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc004230 con 0x555bcd122020 2024-09-19T04:50:42.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.745+0000 7f49cb000640 1 -- 172.21.15.123:0/1784737060 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49cc0043d0 con 0x555bcd122020 2024-09-19T04:50:42.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.745+0000 7f49cb000640 1 -- 172.21.15.123:0/1784737060 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc004db0 con 0x555bcd122020 2024-09-19T04:50:42.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.746+0000 7f49e71e4740 1 -- 172.21.15.123:0/1784737060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd122020 msgr2=0x555bcd124410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:42.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.746+0000 7f49e71e4740 1 --2- 172.21.15.123:0/1784737060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd122020 0x555bcd124410 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009c20 tx=0x7f49cc011810 comp rx=0 tx=0).stop 2024-09-19T04:50:42.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.746+0000 7f49e2800640 1 --2- 172.21.15.123:0/2677451267 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f49c40a3430 0x7f49c40a58f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f49d8004af0 tx=0x7f49d8004a40 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:42.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.746+0000 7f49e71e4740 1 -- 172.21.15.123:0/1784737060 shutdown_connections 2024-09-19T04:50:42.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.746+0000 7f49e71e4740 1 --2- 172.21.15.123:0/1784737060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd122020 0x555bcd124410 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.746+0000 7f49e71e4740 1 --2- 172.21.15.123:0/1784737060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd117760 0x555bcd121ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.746+0000 7f49e71e4740 1 --2- 172.21.15.123:0/1784737060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd116e40 0x555bcd117220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.746+0000 7f49e71e4740 1 -- 172.21.15.123:0/1784737060 >> 172.21.15.123:0/1784737060 conn(0x555bcd1133d0 msgr2=0x555bcd1137b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:42.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.746+0000 7f49e71e4740 1 -- 172.21.15.123:0/1784737060 shutdown_connections 2024-09-19T04:50:42.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.746+0000 7f49e71e4740 1 -- 172.21.15.123:0/1784737060 wait complete. 2024-09-19T04:50:42.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.747+0000 7f49e71e4740 1 -- start start 2024-09-19T04:50:42.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.747+0000 7f49e71e4740 1 Processor -- start 2024-09-19T04:50:42.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.748+0000 7f49e71e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd116e40 0x555bcd11eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.748+0000 7f49e71e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd117760 0x555bcd11f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.748+0000 7f49e71e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd122020 0x555bcd11f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.748+0000 7f49e71e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555bcd124bf0 con 0x555bcd116e40 2024-09-19T04:50:42.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.748+0000 7f49e71e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555bcd0380f0 con 0x555bcd122020 2024-09-19T04:50:42.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.748+0000 7f49e71e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555bcd124d70 con 0x555bcd117760 2024-09-19T04:50:42.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.748+0000 7f49e71e4740 10 client.0 _ll_register_callbacks cb 0x555bcd01fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:42.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.748+0000 7f49e1400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd117760 0x555bcd11f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.748+0000 7f49e1400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd117760 0x555bcd11f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:59228/0 (socket says 172.21.15.123:59228) 2024-09-19T04:50:42.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.748+0000 7f49e1400640 1 -- 172.21.15.123:0/1500367645 learned_addr learned my addr 172.21.15.123:0/1500367645 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:42.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.748+0000 7f49e1e00640 1 --2- 172.21.15.123:0/1500367645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd116e40 0x555bcd11eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.748+0000 7f49e2800640 1 --2- 172.21.15.123:0/1500367645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd122020 0x555bcd11f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.748+0000 7f49e1400640 1 -- 172.21.15.123:0/1500367645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd122020 msgr2=0x555bcd11f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:42.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.748+0000 7f49e1400640 1 --2- 172.21.15.123:0/1500367645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd122020 0x555bcd11f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.748+0000 7f49e1400640 1 -- 172.21.15.123:0/1500367645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd116e40 msgr2=0x555bcd11eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:42.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.749+0000 7f49e1400640 1 --2- 172.21.15.123:0/1500367645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd116e40 0x555bcd11eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.749+0000 7f49e1400640 1 -- 172.21.15.123:0/1500367645 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bccdee3c0 con 0x555bcd117760 2024-09-19T04:50:42.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.749+0000 7f49e1e00640 1 --2- 172.21.15.123:0/1500367645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd116e40 0x555bcd11eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:50:42.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.749+0000 7f49e1400640 1 --2- 172.21.15.123:0/1500367645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd117760 0x555bcd11f3f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f49cc011d10 tx=0x7f49cc019bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=681ebb75d2d10382 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:42.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.749+0000 7f49bf400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:50:42.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.749+0000 7f49bf400640 1 -- 172.21.15.123:0/1500367645 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc028020 con 0x555bcd117760 2024-09-19T04:50:42.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.749+0000 7f49bf400640 1 -- 172.21.15.123:0/1500367645 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f49cc024620 con 0x555bcd117760 2024-09-19T04:50:42.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.749+0000 7f49e71e4740 1 -- 172.21.15.123:0/1500367645 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49cc0043d0 con 0x555bcd117760 2024-09-19T04:50:42.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.750+0000 7f49bf400640 1 -- 172.21.15.123:0/1500367645 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f49cc033050 con 0x555bcd117760 2024-09-19T04:50:42.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.750+0000 7f49e71e4740 10 client.26071 fetch_fsmap learned FSMap version 34 2024-09-19T04:50:42.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.750+0000 7f49e71e4740 1 -- 172.21.15.123:0/1500367645 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555bcd130c50 con 0x555bcd117760 2024-09-19T04:50:42.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.750+0000 7f49bf400640 1 -- 172.21.15.123:0/1500367645 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49cc0043d0 con 0x555bcd117760 2024-09-19T04:50:42.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.750+0000 7f49bf400640 1 -- 172.21.15.123:0/1500367645 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f49cc018070 con 0x555bcd117760 2024-09-19T04:50:42.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.750+0000 7f49e71e4740 10 client.26071 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:50:42.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.750+0000 7f49e71e4740 10 client.26071 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:50:42.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.751+0000 7f49e71e4740 20 client.26071 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:42.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.751+0000 7f49e71e4740 1 --2- 172.21.15.123:0/1500367645 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x555bcd132110 0x555bcd134500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.751+0000 7f49e71e4740 4 client.26071 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:50:42.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.751+0000 7f49e71e4740 1 -- 172.21.15.123:0/1500367645 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bcd08a950 con 0x555bcd132110 2024-09-19T04:50:42.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.751+0000 7f49e1e00640 1 --2- 172.21.15.123:0/1500367645 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x555bcd132110 0x555bcd134500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.751+0000 7f49e1e00640 1 --2- 172.21.15.123:0/1500367645 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x555bcd132110 0x555bcd134500 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e95b72413fea0b4a server_cookie=6f50b47e79b639a in_seq=0 out_seq=0 2024-09-19T04:50:42.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.751+0000 7f49bf400640 10 client.26071 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:50:42.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.752+0000 7f49bf400640 1 -- 172.21.15.123:0/1500367645 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555bcd08a950 con 0x555bcd132110 2024-09-19T04:50:42.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.752+0000 7f49bf400640 10 client.26071 handle_command_reply: tid=0 2024-09-19T04:50:42.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.752+0000 7f49e71e4740 1 client.26071 shutdown 2024-09-19T04:50:42.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.753+0000 7f49e71e4740 1 -- 172.21.15.123:0/1500367645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd117760 msgr2=0x555bcd11f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:42.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.753+0000 7f49e71e4740 1 --2- 172.21.15.123:0/1500367645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd117760 0x555bcd11f3f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f49cc011d10 tx=0x7f49cc019bd0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.753+0000 7f49e71e4740 1 -- 172.21.15.123:0/1500367645 shutdown_connections 2024-09-19T04:50:42.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.753+0000 7f49e71e4740 1 --2- 172.21.15.123:0/1500367645 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x555bcd132110 0x555bcd134500 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.753+0000 7f49e71e4740 1 --2- 172.21.15.123:0/1500367645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd122020 0x555bcd11f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.753+0000 7f49e71e4740 1 --2- 172.21.15.123:0/1500367645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd117760 0x555bcd11f3f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.753+0000 7f49e71e4740 1 --2- 172.21.15.123:0/1500367645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd116e40 0x555bcd11eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.753+0000 7f49e71e4740 1 -- 172.21.15.123:0/1500367645 >> 172.21.15.123:0/1500367645 conn(0x555bcd1133d0 msgr2=0x555bcd119100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:42.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.753+0000 7f49bf400640 0 client.26071 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:50:42.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.753+0000 7f49e71e4740 1 -- 172.21.15.123:0/1500367645 shutdown_connections 2024-09-19T04:50:42.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.753+0000 7f49e71e4740 1 -- 172.21.15.123:0/1500367645 wait complete. 2024-09-19T04:50:42.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.753+0000 7f49e71e4740 20 client.26071 trim_cache size 0 max 16384 2024-09-19T04:50:42.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.768+0000 7f49e71e4740 1 Processor -- start 2024-09-19T04:50:42.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.768+0000 7f49e71e4740 1 -- start start 2024-09-19T04:50:42.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.768+0000 7f49e71e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd116e40 0x555bcd120e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.768+0000 7f49e71e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd117760 0x555bcd139d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.768+0000 7f49e71e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd122020 0x555bcd13a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.768+0000 7f49e71e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f49c000e6d0 con 0x555bcd117760 2024-09-19T04:50:42.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.768+0000 7f49e71e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f49c0010270 con 0x555bcd122020 2024-09-19T04:50:42.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.768+0000 7f49e71e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f49c000eda0 con 0x555bcd116e40 2024-09-19T04:50:42.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.768+0000 7f49e1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd117760 0x555bcd139d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.768+0000 7f49e2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd122020 0x555bcd13a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.769+0000 7f49e1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd117760 0x555bcd139d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53596/0 (socket says 172.21.15.123:53596) 2024-09-19T04:50:42.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.769+0000 7f49e2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd122020 0x555bcd13a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:36408/0 (socket says 172.21.15.123:36408) 2024-09-19T04:50:42.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.769+0000 7f49e1400640 1 -- 172.21.15.123:0/2743617179 learned_addr learned my addr 172.21.15.123:0/2743617179 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:42.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.769+0000 7f49e1e00640 1 --2- 172.21.15.123:0/2743617179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd116e40 0x555bcd120e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.769+0000 7f49e2800640 1 -- 172.21.15.123:0/2743617179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd116e40 msgr2=0x555bcd120e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:42.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.769+0000 7f49e2800640 1 --2- 172.21.15.123:0/2743617179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd116e40 0x555bcd120e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.769+0000 7f49e2800640 1 -- 172.21.15.123:0/2743617179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd117760 msgr2=0x555bcd139d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:42.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.769+0000 7f49e2800640 1 --2- 172.21.15.123:0/2743617179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd117760 0x555bcd139d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.769+0000 7f49e2800640 1 -- 172.21.15.123:0/2743617179 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bcd117590 con 0x555bcd122020 2024-09-19T04:50:42.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.769+0000 7f49e1e00640 1 --2- 172.21.15.123:0/2743617179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd116e40 0x555bcd120e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:42.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.769+0000 7f49e1400640 1 --2- 172.21.15.123:0/2743617179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd117760 0x555bcd139d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:42.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.770+0000 7f49e2800640 1 --2- 172.21.15.123:0/2743617179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd122020 0x555bcd13a290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f49d8006010 tx=0x7f49d8040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82b81c61ffc0abf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:42.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.770+0000 7f49ca600640 1 -- 172.21.15.123:0/2743617179 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8003a30 con 0x555bcd122020 2024-09-19T04:50:42.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.770+0000 7f49ca600640 1 -- 172.21.15.123:0/2743617179 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d8036040 con 0x555bcd122020 2024-09-19T04:50:42.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.770+0000 7f49ca600640 1 -- 172.21.15.123:0/2743617179 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8004570 con 0x555bcd122020 2024-09-19T04:50:42.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.771+0000 7f49e71e4740 1 -- 172.21.15.123:0/2743617179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd122020 msgr2=0x555bcd13a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:42.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.771+0000 7f49e71e4740 1 --2- 172.21.15.123:0/2743617179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd122020 0x555bcd13a290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f49d8006010 tx=0x7f49d8040000 comp rx=0 tx=0).stop 2024-09-19T04:50:42.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.771+0000 7f49e71e4740 1 -- 172.21.15.123:0/2743617179 shutdown_connections 2024-09-19T04:50:42.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.771+0000 7f49e71e4740 1 --2- 172.21.15.123:0/2743617179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd122020 0x555bcd13a290 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.771+0000 7f49e71e4740 1 --2- 172.21.15.123:0/2743617179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd117760 0x555bcd139d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.771+0000 7f49e71e4740 1 --2- 172.21.15.123:0/2743617179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd116e40 0x555bcd120e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.771+0000 7f49e71e4740 1 -- 172.21.15.123:0/2743617179 >> 172.21.15.123:0/2743617179 conn(0x555bcd1133d0 msgr2=0x555bcd120710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:42.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.771+0000 7f49e71e4740 1 -- 172.21.15.123:0/2743617179 shutdown_connections 2024-09-19T04:50:42.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.771+0000 7f49e71e4740 1 -- 172.21.15.123:0/2743617179 wait complete. 2024-09-19T04:50:42.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.772+0000 7f49e71e4740 1 -- start start 2024-09-19T04:50:42.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.772+0000 7f49e71e4740 1 Processor -- start 2024-09-19T04:50:42.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.772+0000 7f49e71e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd116e40 0x555bcd12a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.772+0000 7f49e71e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd117760 0x555bcd11bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.772+0000 7f49e71e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd122020 0x555bcd11c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.772+0000 7f49e71e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f49c000fb70 con 0x555bcd122020 2024-09-19T04:50:42.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.772+0000 7f49e71e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f49c000f4a0 con 0x555bcd117760 2024-09-19T04:50:42.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.772+0000 7f49e71e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f49c000c6b0 con 0x555bcd116e40 2024-09-19T04:50:42.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.772+0000 7f49e71e4740 10 client.0 _ll_register_callbacks cb 0x7f49cc009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:42.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.773+0000 7f49e1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd117760 0x555bcd11bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.773+0000 7f49e1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd116e40 0x555bcd12a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.773+0000 7f49e1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd117760 0x555bcd11bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:36424/0 (socket says 172.21.15.123:36424) 2024-09-19T04:50:42.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.773+0000 7f49e2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd122020 0x555bcd11c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.773+0000 7f49e1400640 1 -- 172.21.15.123:0/3109820857 learned_addr learned my addr 172.21.15.123:0/3109820857 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:42.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.773+0000 7f49e1400640 1 -- 172.21.15.123:0/3109820857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd116e40 msgr2=0x555bcd12a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:42.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.773+0000 7f49e1400640 1 --2- 172.21.15.123:0/3109820857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd116e40 0x555bcd12a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.773+0000 7f49e1400640 1 -- 172.21.15.123:0/3109820857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd122020 msgr2=0x555bcd11c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:42.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.773+0000 7f49e1400640 1 --2- 172.21.15.123:0/3109820857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd122020 0x555bcd11c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.773+0000 7f49e1400640 1 -- 172.21.15.123:0/3109820857 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bcd19bd20 con 0x555bcd117760 2024-09-19T04:50:42.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.773+0000 7f49e1e00640 1 --2- 172.21.15.123:0/3109820857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd116e40 0x555bcd12a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:42.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.774+0000 7f49e1400640 1 --2- 172.21.15.123:0/3109820857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd117760 0x555bcd11bd40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0197a0 tx=0x7f49cc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dccb46a3d7fbb0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:42.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.774+0000 7f49bf400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:50:42.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.774+0000 7f49bf400640 1 -- 172.21.15.123:0/3109820857 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc028070 con 0x555bcd117760 2024-09-19T04:50:42.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.774+0000 7f49e71e4740 1 -- 172.21.15.123:0/3109820857 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49d8036040 con 0x555bcd117760 2024-09-19T04:50:42.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.774+0000 7f49bf400640 1 -- 172.21.15.123:0/3109820857 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f49cc005270 con 0x555bcd117760 2024-09-19T04:50:42.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.774+0000 7f49bf400640 1 -- 172.21.15.123:0/3109820857 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f49cc004ff0 con 0x555bcd117760 2024-09-19T04:50:42.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.774+0000 7f49e71e4740 10 client.26057 fetch_fsmap learned FSMap version 34 2024-09-19T04:50:42.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.774+0000 7f49e71e4740 1 -- 172.21.15.123:0/3109820857 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555bcd19ce80 con 0x555bcd117760 2024-09-19T04:50:42.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.774+0000 7f49bf400640 1 -- 172.21.15.123:0/3109820857 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d8036040 con 0x555bcd117760 2024-09-19T04:50:42.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.775+0000 7f49bf400640 1 -- 172.21.15.123:0/3109820857 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f49cc018040 con 0x555bcd117760 2024-09-19T04:50:42.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.775+0000 7f49e71e4740 10 client.26057 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:50:42.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.775+0000 7f49e71e4740 10 client.26057 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:50:42.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.775+0000 7f49e71e4740 20 client.26057 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:42.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.775+0000 7f49e71e4740 1 --2- 172.21.15.123:0/3109820857 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x555bcd19d590 0x555bcd19f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:42.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.775+0000 7f49e71e4740 4 client.26057 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:50:42.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.775+0000 7f49e71e4740 1 -- 172.21.15.123:0/3109820857 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555bcd084e60 con 0x555bcd19d590 2024-09-19T04:50:42.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.775+0000 7f49e1e00640 1 --2- 172.21.15.123:0/3109820857 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x555bcd19d590 0x555bcd19f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:42.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.776+0000 7f49e1e00640 1 --2- 172.21.15.123:0/3109820857 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x555bcd19d590 0x555bcd19f9b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c432e97ec43693ce server_cookie=f608aa558db15c4 in_seq=0 out_seq=0 2024-09-19T04:50:42.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.776+0000 7f49bf400640 10 client.26057 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:50:42.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.776+0000 7f49bf400640 1 -- 172.21.15.123:0/3109820857 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555bcd084e60 con 0x555bcd19d590 2024-09-19T04:50:42.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.776+0000 7f49bf400640 10 client.26057 handle_command_reply: tid=0 2024-09-19T04:50:42.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.776+0000 7f49e71e4740 1 client.26057 shutdown 2024-09-19T04:50:42.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.776+0000 7f49e71e4740 1 -- 172.21.15.123:0/3109820857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd117760 msgr2=0x555bcd11bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:42.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.776+0000 7f49e71e4740 1 --2- 172.21.15.123:0/3109820857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd117760 0x555bcd11bd40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0197a0 tx=0x7f49cc005f90 comp rx=0 tx=0).stop 2024-09-19T04:50:42.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.776+0000 7f49e71e4740 1 -- 172.21.15.123:0/3109820857 shutdown_connections 2024-09-19T04:50:42.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.776+0000 7f49e71e4740 1 --2- 172.21.15.123:0/3109820857 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x555bcd19d590 0x555bcd19f9b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.776+0000 7f49e71e4740 1 --2- 172.21.15.123:0/3109820857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcd122020 0x555bcd11c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.776+0000 7f49e71e4740 1 --2- 172.21.15.123:0/3109820857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bcd117760 0x555bcd11bd40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.776+0000 7f49e71e4740 1 --2- 172.21.15.123:0/3109820857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bcd116e40 0x555bcd12a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.776+0000 7f49e71e4740 1 -- 172.21.15.123:0/3109820857 >> 172.21.15.123:0/3109820857 conn(0x555bcd1133d0 msgr2=0x555bcd119020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:42.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.776+0000 7f49bf400640 0 client.26057 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:50:42.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.777+0000 7f49e71e4740 1 -- 172.21.15.123:0/3109820857 shutdown_connections 2024-09-19T04:50:42.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.777+0000 7f49e71e4740 1 -- 172.21.15.123:0/3109820857 wait complete. 2024-09-19T04:50:42.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.777+0000 7f49e71e4740 20 client.26057 trim_cache size 0 max 16384 2024-09-19T04:50:42.782 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:50:42.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.777+0000 7f49e3200640 1 -- 172.21.15.123:0/2677451267 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f49c40a3430 msgr2=0x7f49c40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:42.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.777+0000 7f49e3200640 1 --2- 172.21.15.123:0/2677451267 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f49c40a3430 0x7f49c40a58f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f49d8004af0 tx=0x7f49d8004a40 comp rx=0 tx=0).stop 2024-09-19T04:50:42.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.777+0000 7f49e3200640 1 -- 172.21.15.123:0/2677451267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49dc103e30 msgr2=0x7f49dc19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:42.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.777+0000 7f49e3200640 1 --2- 172.21.15.123:0/2677451267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49dc103e30 0x7f49dc19a450 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f49d000e9f0 tx=0x7f49d000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.778+0000 7f49e3200640 1 -- 172.21.15.123:0/2677451267 shutdown_connections 2024-09-19T04:50:42.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.778+0000 7f49e3200640 1 --2- 172.21.15.123:0/2677451267 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f49c40a3430 0x7f49c40a58f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.778+0000 7f49e3200640 1 --2- 172.21.15.123:0/2677451267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49dc108f60 0x7f49dc19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.778+0000 7f49e3200640 1 --2- 172.21.15.123:0/2677451267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49dc1047e0 0x7f49dc19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.778+0000 7f49e3200640 1 --2- 172.21.15.123:0/2677451267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49dc103e30 0x7f49dc19a450 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:42.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.778+0000 7f49e3200640 1 -- 172.21.15.123:0/2677451267 >> 172.21.15.123:0/2677451267 conn(0x7f49dc0ff910 msgr2=0x7f49dc101d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:42.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.778+0000 7f49e3200640 1 -- 172.21.15.123:0/2677451267 shutdown_connections 2024-09-19T04:50:42.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:42.778+0000 7f49e3200640 1 -- 172.21.15.123:0/2677451267 wait complete. 2024-09-19T04:50:43.300 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:42 smithi123 ceph-mon[30511]: pgmap v123: 97 pgs: 97 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 2.9 MiB/s wr, 480 op/s 2024-09-19T04:50:43.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:42 smithi183 ceph-mon[35333]: pgmap v123: 97 pgs: 97 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 2.9 MiB/s wr, 480 op/s 2024-09-19T04:50:43.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:42 smithi193 ceph-mon[35592]: pgmap v123: 97 pgs: 97 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 2.9 MiB/s wr, 480 op/s 2024-09-19T04:50:43.809 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:50:44.230 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:50:44.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.816+0000 7efda8a00640 1 -- 172.21.15.123:0/1569630133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda41079d0 msgr2=0x7efda4107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:44.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.816+0000 7efda8a00640 1 --2- 172.21.15.123:0/1569630133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda41079d0 0x7efda4107db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efd94009900 tx=0x7efd9402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.816+0000 7efda8a00640 1 -- 172.21.15.123:0/1569630133 shutdown_connections 2024-09-19T04:50:44.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.816+0000 7efda8a00640 1 --2- 172.21.15.123:0/1569630133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efda4102370 0x7efda410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.816+0000 7efda8a00640 1 --2- 172.21.15.123:0/1569630133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efda41019d0 0x7efda4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.816+0000 7efda8a00640 1 --2- 172.21.15.123:0/1569630133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda41079d0 0x7efda4107db0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.816+0000 7efda8a00640 1 -- 172.21.15.123:0/1569630133 >> 172.21.15.123:0/1569630133 conn(0x7efda40fd710 msgr2=0x7efda40ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:44.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.816+0000 7efda8a00640 1 -- 172.21.15.123:0/1569630133 shutdown_connections 2024-09-19T04:50:44.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.816+0000 7efda8a00640 1 -- 172.21.15.123:0/1569630133 wait complete. 2024-09-19T04:50:44.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.817+0000 7efda8a00640 1 Processor -- start 2024-09-19T04:50:44.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.817+0000 7efda8a00640 1 -- start start 2024-09-19T04:50:44.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.817+0000 7efda8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efda41019d0 0x7efda419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.817+0000 7efda8a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda4102370 0x7efda419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.817+0000 7efda8a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efda41079d0 0x7efda419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.817+0000 7efda8a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efda4111dd0 con 0x7efda41019d0 2024-09-19T04:50:44.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.817+0000 7efda8a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efda4111c50 con 0x7efda4102370 2024-09-19T04:50:44.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.817+0000 7efda8a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efda4111f50 con 0x7efda41079d0 2024-09-19T04:50:44.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.818+0000 7efda2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efda41019d0 0x7efda419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.818+0000 7efda3200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efda41079d0 0x7efda419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.818+0000 7efda2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efda41019d0 0x7efda419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53612/0 (socket says 172.21.15.123:53612) 2024-09-19T04:50:44.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.818+0000 7efda2800640 1 -- 172.21.15.123:0/3900785163 learned_addr learned my addr 172.21.15.123:0/3900785163 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:44.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.818+0000 7efda1e00640 1 --2- 172.21.15.123:0/3900785163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda4102370 0x7efda419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.818+0000 7efda3200640 1 -- 172.21.15.123:0/3900785163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda4102370 msgr2=0x7efda419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:44.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.818+0000 7efda3200640 1 --2- 172.21.15.123:0/3900785163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda4102370 0x7efda419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.818+0000 7efda3200640 1 -- 172.21.15.123:0/3900785163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efda41019d0 msgr2=0x7efda419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:44.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.818+0000 7efda3200640 1 --2- 172.21.15.123:0/3900785163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efda41019d0 0x7efda419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.818+0000 7efda3200640 1 -- 172.21.15.123:0/3900785163 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda419f430 con 0x7efda41079d0 2024-09-19T04:50:44.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.819+0000 7efda3200640 1 --2- 172.21.15.123:0/3900785163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efda41079d0 0x7efda419ed50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009fd0 tx=0x7efd9c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:44.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.819+0000 7efd9b400640 1 -- 172.21.15.123:0/3900785163 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00ce20 con 0x7efda41079d0 2024-09-19T04:50:44.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.819+0000 7efd9b400640 1 -- 172.21.15.123:0/3900785163 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd9c005020 con 0x7efda41079d0 2024-09-19T04:50:44.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.819+0000 7efda8a00640 1 -- 172.21.15.123:0/3900785163 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda419f720 con 0x7efda41079d0 2024-09-19T04:50:44.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.819+0000 7efd9b400640 1 -- 172.21.15.123:0/3900785163 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c010720 con 0x7efda41079d0 2024-09-19T04:50:44.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.819+0000 7efda8a00640 1 -- 172.21.15.123:0/3900785163 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda41a7000 con 0x7efda41079d0 2024-09-19T04:50:44.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.822+0000 7efd9b400640 1 -- 172.21.15.123:0/3900785163 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7efd9c005540 con 0x7efda41079d0 2024-09-19T04:50:44.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.823+0000 7efdabed3740 1 Processor -- start 2024-09-19T04:50:44.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.823+0000 7efdabed3740 1 -- start start 2024-09-19T04:50:44.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.824+0000 7efdabed3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 0x55b393c7e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.824+0000 7efdabed3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c7e760 0x55b393c88ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.824+0000 7efdabed3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c89020 0x55b393c8b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.824+0000 7efdabed3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b393be43e0 con 0x55b393c7de40 2024-09-19T04:50:44.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.824+0000 7efdabed3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b393b8bd80 con 0x55b393c7e760 2024-09-19T04:50:44.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.824+0000 7efdabed3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b393be03a0 con 0x55b393c89020 2024-09-19T04:50:44.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.824+0000 7efda2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c7e760 0x55b393c88ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.824+0000 7efda1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 0x55b393c7e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.824+0000 7efda2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c7e760 0x55b393c88ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:36450/0 (socket says 172.21.15.123:36450) 2024-09-19T04:50:44.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.824+0000 7efda2800640 1 -- 172.21.15.123:0/946751662 learned_addr learned my addr 172.21.15.123:0/946751662 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:44.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.824+0000 7efda1e00640 1 --2- 172.21.15.123:0/946751662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c89020 0x55b393c8b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.824+0000 7efda2800640 1 -- 172.21.15.123:0/946751662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c89020 msgr2=0x55b393c8b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:44.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.824+0000 7efda2800640 1 --2- 172.21.15.123:0/946751662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c89020 0x55b393c8b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.824+0000 7efda2800640 1 -- 172.21.15.123:0/946751662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 msgr2=0x55b393c7e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:44.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.824+0000 7efda2800640 1 --2- 172.21.15.123:0/946751662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 0x55b393c7e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.824+0000 7efda2800640 1 -- 172.21.15.123:0/946751662 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b393baaae0 con 0x55b393c7e760 2024-09-19T04:50:44.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.824+0000 7efda1e00640 1 --2- 172.21.15.123:0/946751662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 0x55b393c7e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:50:44.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.825+0000 7efd9b400640 1 --2- 172.21.15.123:0/3900785163 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efd880a3430 0x7efd880a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.825+0000 7efda2800640 1 --2- 172.21.15.123:0/946751662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c7e760 0x55b393c88ae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd94009a20 tx=0x7efd94005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bde510071b6d48a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:44.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.825+0000 7efd9b400640 1 -- 172.21.15.123:0/3900785163 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7efd9c0d0240 con 0x7efda41079d0 2024-09-19T04:50:44.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.825+0000 7efd8f400640 1 -- 172.21.15.123:0/946751662 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94040070 con 0x55b393c7e760 2024-09-19T04:50:44.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.825+0000 7efd8f400640 1 -- 172.21.15.123:0/946751662 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd94004480 con 0x55b393c7e760 2024-09-19T04:50:44.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.825+0000 7efd8f400640 1 -- 172.21.15.123:0/946751662 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94035040 con 0x55b393c7e760 2024-09-19T04:50:44.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.825+0000 7efda1e00640 1 --2- 172.21.15.123:0/3900785163 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efd880a3430 0x7efd880a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.826+0000 7efdabed3740 1 -- 172.21.15.123:0/946751662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c7e760 msgr2=0x55b393c88ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:44.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.826+0000 7efdabed3740 1 --2- 172.21.15.123:0/946751662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c7e760 0x55b393c88ae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd94009a20 tx=0x7efd94005e10 comp rx=0 tx=0).stop 2024-09-19T04:50:44.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.826+0000 7efdabed3740 1 -- 172.21.15.123:0/946751662 shutdown_connections 2024-09-19T04:50:44.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.826+0000 7efda1e00640 1 --2- 172.21.15.123:0/3900785163 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efd880a3430 0x7efd880a58f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efd90009730 tx=0x7efd900096c0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:44.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.826+0000 7efdabed3740 1 --2- 172.21.15.123:0/946751662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c89020 0x55b393c8b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.826+0000 7efdabed3740 1 --2- 172.21.15.123:0/946751662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c7e760 0x55b393c88ae0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.826+0000 7efdabed3740 1 --2- 172.21.15.123:0/946751662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 0x55b393c7e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.826+0000 7efdabed3740 1 -- 172.21.15.123:0/946751662 >> 172.21.15.123:0/946751662 conn(0x55b393c7a3d0 msgr2=0x55b393c7a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:44.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.826+0000 7efdabed3740 1 -- 172.21.15.123:0/946751662 shutdown_connections 2024-09-19T04:50:44.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.826+0000 7efdabed3740 1 -- 172.21.15.123:0/946751662 wait complete. 2024-09-19T04:50:44.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.827+0000 7efdabed3740 1 -- start start 2024-09-19T04:50:44.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.827+0000 7efdabed3740 1 Processor -- start 2024-09-19T04:50:44.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efdabed3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 0x55b393c85ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efdabed3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c7e760 0x55b393c863e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efdabed3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c89020 0x55b393c86920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efdabed3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b393c8bbf0 con 0x55b393c7de40 2024-09-19T04:50:44.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efdabed3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b393b9f0f0 con 0x55b393c89020 2024-09-19T04:50:44.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efdabed3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b393c8bd70 con 0x55b393c7e760 2024-09-19T04:50:44.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efdabed3740 10 client.0 _ll_register_callbacks cb 0x55b393b86bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:44.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efda2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 0x55b393c85ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efda1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c7e760 0x55b393c863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efda2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 0x55b393c85ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53642/0 (socket says 172.21.15.123:53642) 2024-09-19T04:50:44.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efda2800640 1 -- 172.21.15.123:0/3056889340 learned_addr learned my addr 172.21.15.123:0/3056889340 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:44.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efda3200640 1 --2- 172.21.15.123:0/3056889340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c89020 0x55b393c86920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efda1e00640 1 -- 172.21.15.123:0/3056889340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c89020 msgr2=0x55b393c86920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:44.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efda1e00640 1 --2- 172.21.15.123:0/3056889340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c89020 0x55b393c86920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efda1e00640 1 -- 172.21.15.123:0/3056889340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 msgr2=0x55b393c85ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:44.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efda1e00640 1 --2- 172.21.15.123:0/3056889340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 0x55b393c85ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efda1e00640 1 -- 172.21.15.123:0/3056889340 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3939553c0 con 0x55b393c7e760 2024-09-19T04:50:44.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.828+0000 7efda3200640 1 --2- 172.21.15.123:0/3056889340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c89020 0x55b393c86920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:50:44.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.829+0000 7efda2800640 1 --2- 172.21.15.123:0/3056889340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 0x55b393c85ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:44.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.829+0000 7efda1e00640 1 --2- 172.21.15.123:0/3056889340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c7e760 0x55b393c863e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efda410eca0 tx=0x7efd90004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf1040bd0bd68074 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:44.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.829+0000 7efd83e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:50:44.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.829+0000 7efd83e00640 1 -- 172.21.15.123:0/3056889340 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90022850 con 0x55b393c7e760 2024-09-19T04:50:44.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.829+0000 7efdabed3740 1 -- 172.21.15.123:0/3056889340 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd94004480 con 0x55b393c7e760 2024-09-19T04:50:44.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.829+0000 7efd83e00640 1 -- 172.21.15.123:0/3056889340 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7efd900229f0 con 0x55b393c7e760 2024-09-19T04:50:44.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.829+0000 7efd83e00640 1 -- 172.21.15.123:0/3056889340 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efd90022bf0 con 0x55b393c7e760 2024-09-19T04:50:44.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.829+0000 7efdabed3740 10 client.26089 fetch_fsmap learned FSMap version 35 2024-09-19T04:50:44.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.829+0000 7efdabed3740 1 -- 172.21.15.123:0/3056889340 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b393c97c70 con 0x55b393c7e760 2024-09-19T04:50:44.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.830+0000 7efd83e00640 1 -- 172.21.15.123:0/3056889340 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd94004480 con 0x55b393c7e760 2024-09-19T04:50:44.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.830+0000 7efd83e00640 1 -- 172.21.15.123:0/3056889340 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7efd9001f430 con 0x55b393c7e760 2024-09-19T04:50:44.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.830+0000 7efdabed3740 10 client.26089 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T04:50:44.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.830+0000 7efdabed3740 10 client.26089 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:50:44.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.830+0000 7efdabed3740 20 client.26089 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:44.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.830+0000 7efdabed3740 1 --2- 172.21.15.123:0/3056889340 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55b393c99080 0x55b393c9b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.830+0000 7efdabed3740 4 client.26089 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:50:44.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.830+0000 7efdabed3740 1 -- 172.21.15.123:0/3056889340 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b393bf1950 con 0x55b393c99080 2024-09-19T04:50:44.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.830+0000 7efda2800640 1 --2- 172.21.15.123:0/3056889340 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55b393c99080 0x55b393c9b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.831+0000 7efda2800640 1 --2- 172.21.15.123:0/3056889340 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55b393c99080 0x55b393c9b4b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6779ac6f2ca6179 server_cookie=74ec3a70ae3277d8 in_seq=0 out_seq=0 2024-09-19T04:50:44.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.831+0000 7efd83e00640 10 client.26089 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:50:44.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.832+0000 7efd83e00640 1 -- 172.21.15.123:0/3056889340 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b393bf1950 con 0x55b393c99080 2024-09-19T04:50:44.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.832+0000 7efd83e00640 10 client.26089 handle_command_reply: tid=0 2024-09-19T04:50:44.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.832+0000 7efdabed3740 1 client.26089 shutdown 2024-09-19T04:50:44.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.832+0000 7efdabed3740 1 -- 172.21.15.123:0/3056889340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c7e760 msgr2=0x55b393c863e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:44.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.832+0000 7efdabed3740 1 --2- 172.21.15.123:0/3056889340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c7e760 0x55b393c863e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efda410eca0 tx=0x7efd90004730 comp rx=0 tx=0).stop 2024-09-19T04:50:44.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.832+0000 7efdabed3740 1 -- 172.21.15.123:0/3056889340 shutdown_connections 2024-09-19T04:50:44.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.832+0000 7efdabed3740 1 --2- 172.21.15.123:0/3056889340 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55b393c99080 0x55b393c9b4b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.832+0000 7efdabed3740 1 --2- 172.21.15.123:0/3056889340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c89020 0x55b393c86920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.832+0000 7efdabed3740 1 --2- 172.21.15.123:0/3056889340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c7e760 0x55b393c863e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.832+0000 7efdabed3740 1 --2- 172.21.15.123:0/3056889340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 0x55b393c85ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.832+0000 7efdabed3740 1 -- 172.21.15.123:0/3056889340 >> 172.21.15.123:0/3056889340 conn(0x55b393c7a3d0 msgr2=0x55b393c80100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:44.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.832+0000 7efd83e00640 0 client.26089 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:50:44.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.832+0000 7efdabed3740 1 -- 172.21.15.123:0/3056889340 shutdown_connections 2024-09-19T04:50:44.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.832+0000 7efdabed3740 1 -- 172.21.15.123:0/3056889340 wait complete. 2024-09-19T04:50:44.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.833+0000 7efdabed3740 20 client.26089 trim_cache size 0 max 16384 2024-09-19T04:50:44.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.848+0000 7efdabed3740 1 Processor -- start 2024-09-19T04:50:44.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.848+0000 7efdabed3740 1 -- start start 2024-09-19T04:50:44.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.848+0000 7efdabed3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c7de40 0x55b393ca0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.848+0000 7efdabed3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7e760 0x55b393ca1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.848+0000 7efdabed3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c89020 0x55b393c87700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.848+0000 7efdabed3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efda41af8a0 con 0x55b393c7e760 2024-09-19T04:50:44.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.848+0000 7efdabed3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efda41b1410 con 0x55b393c7de40 2024-09-19T04:50:44.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.848+0000 7efdabed3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efda41aff70 con 0x55b393c89020 2024-09-19T04:50:44.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.849+0000 7efda2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c7de40 0x55b393ca0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.849+0000 7efda2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c7de40 0x55b393ca0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:36472/0 (socket says 172.21.15.123:36472) 2024-09-19T04:50:44.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.849+0000 7efda2800640 1 -- 172.21.15.123:0/3768835352 learned_addr learned my addr 172.21.15.123:0/3768835352 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:44.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.849+0000 7efda3200640 1 --2- 172.21.15.123:0/3768835352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c89020 0x55b393c87700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.849+0000 7efda1e00640 1 --2- 172.21.15.123:0/3768835352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7e760 0x55b393ca1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.849+0000 7efda3200640 1 -- 172.21.15.123:0/3768835352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c7de40 msgr2=0x55b393ca0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:44.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.849+0000 7efda3200640 1 --2- 172.21.15.123:0/3768835352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c7de40 0x55b393ca0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.849+0000 7efda3200640 1 -- 172.21.15.123:0/3768835352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7e760 msgr2=0x55b393ca1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:44.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.849+0000 7efda3200640 1 --2- 172.21.15.123:0/3768835352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7e760 0x55b393ca1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.849+0000 7efda3200640 1 -- 172.21.15.123:0/3768835352 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b393c7af80 con 0x55b393c89020 2024-09-19T04:50:44.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.849+0000 7efda2800640 1 --2- 172.21.15.123:0/3768835352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c7de40 0x55b393ca0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:44.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.849+0000 7efda3200640 1 --2- 172.21.15.123:0/3768835352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c89020 0x55b393c87700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0cd960 tx=0x7efd9c09be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c074b6065ae505a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:44.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.850+0000 7efd8ea00640 1 -- 172.21.15.123:0/3768835352 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0b4070 con 0x55b393c89020 2024-09-19T04:50:44.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.850+0000 7efd8ea00640 1 -- 172.21.15.123:0/3768835352 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd9c0a08a0 con 0x55b393c89020 2024-09-19T04:50:44.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.850+0000 7efd8ea00640 1 -- 172.21.15.123:0/3768835352 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c09c750 con 0x55b393c89020 2024-09-19T04:50:44.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.850+0000 7efdabed3740 1 -- 172.21.15.123:0/3768835352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c89020 msgr2=0x55b393c87700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:44.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.850+0000 7efdabed3740 1 --2- 172.21.15.123:0/3768835352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c89020 0x55b393c87700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0cd960 tx=0x7efd9c09be50 comp rx=0 tx=0).stop 2024-09-19T04:50:44.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.851+0000 7efdabed3740 1 -- 172.21.15.123:0/3768835352 shutdown_connections 2024-09-19T04:50:44.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.851+0000 7efdabed3740 1 --2- 172.21.15.123:0/3768835352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c89020 0x55b393c87700 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.851+0000 7efdabed3740 1 --2- 172.21.15.123:0/3768835352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7e760 0x55b393ca1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.851+0000 7efdabed3740 1 --2- 172.21.15.123:0/3768835352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c7de40 0x55b393ca0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.851+0000 7efdabed3740 1 -- 172.21.15.123:0/3768835352 >> 172.21.15.123:0/3768835352 conn(0x55b393c7a3d0 msgr2=0x55b393c811c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:44.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.851+0000 7efdabed3740 1 -- 172.21.15.123:0/3768835352 shutdown_connections 2024-09-19T04:50:44.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.851+0000 7efdabed3740 1 -- 172.21.15.123:0/3768835352 wait complete. 2024-09-19T04:50:44.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.851+0000 7efdabed3740 1 -- start start 2024-09-19T04:50:44.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.851+0000 7efdabed3740 1 Processor -- start 2024-09-19T04:50:44.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.852+0000 7efdabed3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 0x55b393c91250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.852+0000 7efdabed3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c7e760 0x55b393c91790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.852+0000 7efdabed3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c89020 0x55b393c827f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.852+0000 7efdabed3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efda41b0d10 con 0x55b393c7de40 2024-09-19T04:50:44.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.852+0000 7efdabed3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efda41b0640 con 0x55b393c89020 2024-09-19T04:50:44.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.852+0000 7efdabed3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efda41ad880 con 0x55b393c7e760 2024-09-19T04:50:44.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.852+0000 7efdabed3740 10 client.0 _ll_register_callbacks cb 0x7efd90010f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:44.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.852+0000 7efda2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 0x55b393c91250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.852+0000 7efda1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c7e760 0x55b393c91790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.852+0000 7efda3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c89020 0x55b393c827f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.852+0000 7efda1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c7e760 0x55b393c91790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:59290/0 (socket says 172.21.15.123:59290) 2024-09-19T04:50:44.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.852+0000 7efda1e00640 1 -- 172.21.15.123:0/3768515538 learned_addr learned my addr 172.21.15.123:0/3768515538 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:44.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.852+0000 7efda2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 0x55b393c91250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53654/0 (socket says 172.21.15.123:53654) 2024-09-19T04:50:44.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.853+0000 7efda3200640 1 -- 172.21.15.123:0/3768515538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c7e760 msgr2=0x55b393c91790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:44.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.853+0000 7efda3200640 1 --2- 172.21.15.123:0/3768515538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c7e760 0x55b393c91790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.853+0000 7efda3200640 1 -- 172.21.15.123:0/3768515538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 msgr2=0x55b393c91250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:44.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.853+0000 7efda3200640 1 --2- 172.21.15.123:0/3768515538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 0x55b393c91250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.853+0000 7efda3200640 1 -- 172.21.15.123:0/3768515538 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b393c83020 con 0x55b393c89020 2024-09-19T04:50:44.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.853+0000 7efda2800640 1 --2- 172.21.15.123:0/3768515538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 0x55b393c91250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:44.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.853+0000 7efda3200640 1 --2- 172.21.15.123:0/3768515538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c89020 0x55b393c827f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0cd960 tx=0x7efd9c0054b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60996cd3ef0b5af7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:44.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.853+0000 7efd83e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:50:44.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.853+0000 7efd83e00640 1 -- 172.21.15.123:0/3768515538 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0b4070 con 0x55b393c89020 2024-09-19T04:50:44.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.853+0000 7efd83e00640 1 -- 172.21.15.123:0/3768515538 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7efd9c0b5260 con 0x55b393c89020 2024-09-19T04:50:44.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.853+0000 7efdabed3740 1 -- 172.21.15.123:0/3768515538 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd9c0a08a0 con 0x55b393c89020 2024-09-19T04:50:44.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.853+0000 7efd83e00640 1 -- 172.21.15.123:0/3768515538 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efd9c0b5460 con 0x55b393c89020 2024-09-19T04:50:44.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.853+0000 7efdabed3740 10 client.26075 fetch_fsmap learned FSMap version 35 2024-09-19T04:50:44.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.853+0000 7efdabed3740 1 -- 172.21.15.123:0/3768515538 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b393d03910 con 0x55b393c89020 2024-09-19T04:50:44.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.854+0000 7efd83e00640 1 -- 172.21.15.123:0/3768515538 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd9c0a08a0 con 0x55b393c89020 2024-09-19T04:50:44.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.854+0000 7efd83e00640 1 -- 172.21.15.123:0/3768515538 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7efd9c09b940 con 0x55b393c89020 2024-09-19T04:50:44.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.854+0000 7efdabed3740 10 client.26075 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T04:50:44.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.854+0000 7efdabed3740 10 client.26075 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:50:44.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.854+0000 7efdabed3740 20 client.26075 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:44.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.854+0000 7efdabed3740 1 --2- 172.21.15.123:0/3768515538 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55b393d04000 0x55b393d06420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:44.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.854+0000 7efdabed3740 4 client.26075 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:50:44.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.854+0000 7efdabed3740 1 -- 172.21.15.123:0/3768515538 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b393bebe60 con 0x55b393d04000 2024-09-19T04:50:44.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.854+0000 7efda2800640 1 --2- 172.21.15.123:0/3768515538 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55b393d04000 0x55b393d06420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:44.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.855+0000 7efda2800640 1 --2- 172.21.15.123:0/3768515538 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55b393d04000 0x55b393d06420 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41432d16bd7ed58b server_cookie=8e7f08e2b246d6ae in_seq=0 out_seq=0 2024-09-19T04:50:44.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.855+0000 7efd83e00640 10 client.26075 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:50:44.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.855+0000 7efd83e00640 1 -- 172.21.15.123:0/3768515538 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b393bebe60 con 0x55b393d04000 2024-09-19T04:50:44.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.855+0000 7efd83e00640 10 client.26075 handle_command_reply: tid=0 2024-09-19T04:50:44.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.855+0000 7efdabed3740 1 client.26075 shutdown 2024-09-19T04:50:44.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.855+0000 7efdabed3740 1 -- 172.21.15.123:0/3768515538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c89020 msgr2=0x55b393c827f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:44.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.855+0000 7efdabed3740 1 --2- 172.21.15.123:0/3768515538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c89020 0x55b393c827f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0cd960 tx=0x7efd9c0054b0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.855+0000 7efdabed3740 1 -- 172.21.15.123:0/3768515538 shutdown_connections 2024-09-19T04:50:44.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.855+0000 7efdabed3740 1 --2- 172.21.15.123:0/3768515538 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55b393d04000 0x55b393d06420 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.855+0000 7efdabed3740 1 --2- 172.21.15.123:0/3768515538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b393c89020 0x55b393c827f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.855+0000 7efdabed3740 1 --2- 172.21.15.123:0/3768515538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b393c7e760 0x55b393c91790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.855+0000 7efdabed3740 1 --2- 172.21.15.123:0/3768515538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b393c7de40 0x55b393c91250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.855+0000 7efdabed3740 1 -- 172.21.15.123:0/3768515538 >> 172.21.15.123:0/3768515538 conn(0x55b393c7a3d0 msgr2=0x55b393c7f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:44.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.855+0000 7efd83e00640 0 client.26075 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:50:44.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.856+0000 7efdabed3740 1 -- 172.21.15.123:0/3768515538 shutdown_connections 2024-09-19T04:50:44.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.856+0000 7efdabed3740 1 -- 172.21.15.123:0/3768515538 wait complete. 2024-09-19T04:50:44.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.856+0000 7efdabed3740 20 client.26075 trim_cache size 0 max 16384 2024-09-19T04:50:44.861 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:50:44.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.856+0000 7efda8a00640 1 -- 172.21.15.123:0/3900785163 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efd880a3430 msgr2=0x7efd880a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:44.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.856+0000 7efda8a00640 1 --2- 172.21.15.123:0/3900785163 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efd880a3430 0x7efd880a58f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efd90009730 tx=0x7efd900096c0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.857+0000 7efda8a00640 1 -- 172.21.15.123:0/3900785163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efda41079d0 msgr2=0x7efda419ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:44.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.857+0000 7efda8a00640 1 --2- 172.21.15.123:0/3900785163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efda41079d0 0x7efda419ed50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009fd0 tx=0x7efd9c00ef10 comp rx=0 tx=0).stop 2024-09-19T04:50:44.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.857+0000 7efda8a00640 1 -- 172.21.15.123:0/3900785163 shutdown_connections 2024-09-19T04:50:44.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.857+0000 7efda8a00640 1 --2- 172.21.15.123:0/3900785163 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efd880a3430 0x7efd880a58f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.857+0000 7efda8a00640 1 --2- 172.21.15.123:0/3900785163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efda41079d0 0x7efda419ed50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.857+0000 7efda8a00640 1 --2- 172.21.15.123:0/3900785163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda4102370 0x7efda419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.857+0000 7efda8a00640 1 --2- 172.21.15.123:0/3900785163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efda41019d0 0x7efda419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:44.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.857+0000 7efda8a00640 1 -- 172.21.15.123:0/3900785163 >> 172.21.15.123:0/3900785163 conn(0x7efda40fd710 msgr2=0x7efda40fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:44.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.857+0000 7efda8a00640 1 -- 172.21.15.123:0/3900785163 shutdown_connections 2024-09-19T04:50:44.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:44.857+0000 7efda8a00640 1 -- 172.21.15.123:0/3900785163 wait complete. 2024-09-19T04:50:45.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:45 smithi183 ceph-mon[35333]: pgmap v124: 97 pgs: 97 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.1 MiB/s wr, 699 op/s 2024-09-19T04:50:45.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:45 smithi183 ceph-mon[35333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:50:45.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:45 smithi123 ceph-mon[30511]: pgmap v124: 97 pgs: 97 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.1 MiB/s wr, 699 op/s 2024-09-19T04:50:45.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:45 smithi123 ceph-mon[30511]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:50:45.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:45 smithi193 ceph-mon[35592]: pgmap v124: 97 pgs: 97 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.1 MiB/s wr, 699 op/s 2024-09-19T04:50:45.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:45 smithi193 ceph-mon[35592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:50:46.036 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:50:46.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:46 smithi183 ceph-mon[35333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:50:46.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:46 smithi123 ceph-mon[30511]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:50:46.460 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:50:46.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:46 smithi193 ceph-mon[35592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:50:47.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.071+0000 7f4795600640 1 -- 172.21.15.123:0/3811163915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4790102790 msgr2=0x7f47901067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:47.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.071+0000 7f4795600640 1 --2- 172.21.15.123:0/3811163915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4790102790 0x7f47901067e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4778009900 tx=0x7f477802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.071+0000 7f4795600640 1 -- 172.21.15.123:0/3811163915 shutdown_connections 2024-09-19T04:50:47.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.071+0000 7f4795600640 1 --2- 172.21.15.123:0/3811163915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4790106f10 0x7f479010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.071+0000 7f4795600640 1 --2- 172.21.15.123:0/3811163915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4790102790 0x7f47901067e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.071+0000 7f4795600640 1 --2- 172.21.15.123:0/3811163915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4790101de0 0x7f47901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.071+0000 7f4795600640 1 -- 172.21.15.123:0/3811163915 >> 172.21.15.123:0/3811163915 conn(0x7f47900fd8c0 msgr2=0x7f47900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:47.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.071+0000 7f4795600640 1 -- 172.21.15.123:0/3811163915 shutdown_connections 2024-09-19T04:50:47.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.071+0000 7f4795600640 1 -- 172.21.15.123:0/3811163915 wait complete. 2024-09-19T04:50:47.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.072+0000 7f4795600640 1 Processor -- start 2024-09-19T04:50:47.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.073+0000 7f4795600640 1 -- start start 2024-09-19T04:50:47.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.073+0000 7f4795600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4790101de0 0x7f479019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.073+0000 7f4795600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4790102790 0x7f479019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.073+0000 7f4795600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4790106f10 0x7f479019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.073+0000 7f4795600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4790111fc0 con 0x7f4790101de0 2024-09-19T04:50:47.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.073+0000 7f4795600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4790111e40 con 0x7f4790102790 2024-09-19T04:50:47.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.073+0000 7f4795600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4790112140 con 0x7f4790106f10 2024-09-19T04:50:47.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.073+0000 7f478ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4790101de0 0x7f479019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.073+0000 7f478f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4790106f10 0x7f479019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.073+0000 7f4786000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4790102790 0x7f479019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.073+0000 7f478ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4790101de0 0x7f479019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53672/0 (socket says 172.21.15.123:53672) 2024-09-19T04:50:47.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.073+0000 7f478ea00640 1 -- 172.21.15.123:0/3955609213 learned_addr learned my addr 172.21.15.123:0/3955609213 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:47.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.074+0000 7f478f400640 1 -- 172.21.15.123:0/3955609213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4790102790 msgr2=0x7f479019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:47.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.074+0000 7f478f400640 1 --2- 172.21.15.123:0/3955609213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4790102790 0x7f479019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.074+0000 7f478f400640 1 -- 172.21.15.123:0/3955609213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4790101de0 msgr2=0x7f479019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:47.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.074+0000 7f478f400640 1 --2- 172.21.15.123:0/3955609213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4790101de0 0x7f479019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.074+0000 7f478f400640 1 -- 172.21.15.123:0/3955609213 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479019f630 con 0x7f4790106f10 2024-09-19T04:50:47.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.074+0000 7f478f400640 1 --2- 172.21.15.123:0/3955609213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4790106f10 0x7f479019ef50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f478800e970 tx=0x7f478800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:47.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.074+0000 7f4787e00640 1 -- 172.21.15.123:0/3955609213 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478800ccc0 con 0x7f4790106f10 2024-09-19T04:50:47.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.074+0000 7f4787e00640 1 -- 172.21.15.123:0/3955609213 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4788005020 con 0x7f4790106f10 2024-09-19T04:50:47.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.074+0000 7f4795600640 1 -- 172.21.15.123:0/3955609213 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479019f920 con 0x7f4790106f10 2024-09-19T04:50:47.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.075+0000 7f4787e00640 1 -- 172.21.15.123:0/3955609213 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788004a00 con 0x7f4790106f10 2024-09-19T04:50:47.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.075+0000 7f4795600640 1 -- 172.21.15.123:0/3955609213 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47901a7200 con 0x7f4790106f10 2024-09-19T04:50:47.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.078+0000 7f4787e00640 1 -- 172.21.15.123:0/3955609213 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f47880051d0 con 0x7f4790106f10 2024-09-19T04:50:47.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.078+0000 7f479807c740 1 Processor -- start 2024-09-19T04:50:47.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.078+0000 7f479807c740 1 -- start start 2024-09-19T04:50:47.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.078+0000 7f479807c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928388c70 0x55d928389050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.078+0000 7f479807c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928389590 0x55d928393910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.079+0000 7f479807c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928393e50 0x55d928396240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.079+0000 7f479807c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d9282ef3e0 con 0x55d928389590 2024-09-19T04:50:47.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.079+0000 7f479807c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d928296d80 con 0x55d928393e50 2024-09-19T04:50:47.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.079+0000 7f479807c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d9282eb3a0 con 0x55d928388c70 2024-09-19T04:50:47.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.079+0000 7f4786000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928388c70 0x55d928389050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.079+0000 7f4786000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928388c70 0x55d928389050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:59328/0 (socket says 172.21.15.123:59328) 2024-09-19T04:50:47.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.079+0000 7f4786000640 1 -- 172.21.15.123:0/244417285 learned_addr learned my addr 172.21.15.123:0/244417285 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:47.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.079+0000 7f478ea00640 1 --2- 172.21.15.123:0/244417285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928389590 0x55d928393910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.079+0000 7f4786000640 1 --2- 172.21.15.123:0/244417285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928393e50 0x55d928396240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.080+0000 7f478ea00640 1 -- 172.21.15.123:0/244417285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928388c70 msgr2=0x55d928389050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:47.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.080+0000 7f4787e00640 1 --2- 172.21.15.123:0/3955609213 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f47740a3430 0x7f47740a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.080+0000 7f478ea00640 1 --2- 172.21.15.123:0/244417285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928388c70 0x55d928389050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.080+0000 7f478ea00640 1 -- 172.21.15.123:0/244417285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928393e50 msgr2=0x55d928396240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:47.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.080+0000 7f478ea00640 1 --2- 172.21.15.123:0/244417285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928393e50 0x55d928396240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.080+0000 7f478ea00640 1 -- 172.21.15.123:0/244417285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9282b5ae0 con 0x55d928389590 2024-09-19T04:50:47.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.080+0000 7f4786000640 1 --2- 172.21.15.123:0/244417285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928388c70 0x55d928389050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:47.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.080+0000 7f4787e00640 1 -- 172.21.15.123:0/3955609213 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f47880cfbf0 con 0x7f4790106f10 2024-09-19T04:50:47.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.080+0000 7f478f400640 1 --2- 172.21.15.123:0/3955609213 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f47740a3430 0x7f47740a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.080+0000 7f478ea00640 1 --2- 172.21.15.123:0/244417285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928389590 0x55d928393910 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f47800114c0 tx=0x7f4780011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f1cab91aa39d84 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:47.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.081+0000 7f477f400640 1 -- 172.21.15.123:0/244417285 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47800110f0 con 0x55d928389590 2024-09-19T04:50:47.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.081+0000 7f477f400640 1 -- 172.21.15.123:0/244417285 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4780011290 con 0x55d928389590 2024-09-19T04:50:47.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.081+0000 7f477f400640 1 -- 172.21.15.123:0/244417285 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780005020 con 0x55d928389590 2024-09-19T04:50:47.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.082+0000 7f479807c740 1 -- 172.21.15.123:0/244417285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928389590 msgr2=0x55d928393910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:47.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.082+0000 7f479807c740 1 --2- 172.21.15.123:0/244417285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928389590 0x55d928393910 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f47800114c0 tx=0x7f4780011980 comp rx=0 tx=0).stop 2024-09-19T04:50:47.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.082+0000 7f478f400640 1 --2- 172.21.15.123:0/3955609213 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f47740a3430 0x7f47740a58f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f47880a45e0 tx=0x7f4788004590 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:47.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.082+0000 7f479807c740 1 -- 172.21.15.123:0/244417285 shutdown_connections 2024-09-19T04:50:47.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.082+0000 7f479807c740 1 --2- 172.21.15.123:0/244417285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928393e50 0x55d928396240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.082+0000 7f479807c740 1 --2- 172.21.15.123:0/244417285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928389590 0x55d928393910 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.082+0000 7f479807c740 1 --2- 172.21.15.123:0/244417285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928388c70 0x55d928389050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.082+0000 7f479807c740 1 -- 172.21.15.123:0/244417285 >> 172.21.15.123:0/244417285 conn(0x55d928385200 msgr2=0x55d9283855e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:47.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.082+0000 7f479807c740 1 -- 172.21.15.123:0/244417285 shutdown_connections 2024-09-19T04:50:47.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.082+0000 7f479807c740 1 -- 172.21.15.123:0/244417285 wait complete. 2024-09-19T04:50:47.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.082+0000 7f479807c740 1 -- start start 2024-09-19T04:50:47.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.083+0000 7f479807c740 1 Processor -- start 2024-09-19T04:50:47.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.083+0000 7f479807c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928388c70 0x55d928390cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.083+0000 7f479807c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928389590 0x55d9283911f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.083+0000 7f479807c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928393e50 0x55d928391730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.083+0000 7f479807c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d928396a20 con 0x55d928388c70 2024-09-19T04:50:47.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.083+0000 7f479807c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d9282aa0f0 con 0x55d928393e50 2024-09-19T04:50:47.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.083+0000 7f479807c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d928396ba0 con 0x55d928389590 2024-09-19T04:50:47.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.083+0000 7f479807c740 10 client.0 _ll_register_callbacks cb 0x55d928291bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:47.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.083+0000 7f4786000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928388c70 0x55d928390cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.083+0000 7f4786000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928388c70 0x55d928390cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53680/0 (socket says 172.21.15.123:53680) 2024-09-19T04:50:47.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.083+0000 7f4786000640 1 -- 172.21.15.123:0/3493644382 learned_addr learned my addr 172.21.15.123:0/3493644382 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:47.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.083+0000 7f4786000640 1 --2- 172.21.15.123:0/3493644382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928393e50 0x55d928391730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.084+0000 7f478ea00640 1 --2- 172.21.15.123:0/3493644382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928389590 0x55d9283911f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.084+0000 7f4786000640 1 -- 172.21.15.123:0/3493644382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928389590 msgr2=0x55d9283911f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:47.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.084+0000 7f4786000640 1 --2- 172.21.15.123:0/3493644382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928389590 0x55d9283911f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.084+0000 7f4786000640 1 -- 172.21.15.123:0/3493644382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928393e50 msgr2=0x55d928391730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:47.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.084+0000 7f4786000640 1 --2- 172.21.15.123:0/3493644382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928393e50 0x55d928391730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.084+0000 7f4786000640 1 -- 172.21.15.123:0/3493644382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9280603c0 con 0x55d928388c70 2024-09-19T04:50:47.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.084+0000 7f4786000640 1 --2- 172.21.15.123:0/3493644382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928388c70 0x55d928390cb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f477802a910 tx=0x7f4778005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfaea11864ed801a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:47.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.084+0000 7f4773e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:50:47.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.084+0000 7f4773e00640 1 -- 172.21.15.123:0/3493644382 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778035070 con 0x55d928388c70 2024-09-19T04:50:47.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.084+0000 7f479807c740 1 -- 172.21.15.123:0/3493644382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4780011290 con 0x55d928388c70 2024-09-19T04:50:47.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.084+0000 7f4773e00640 1 -- 172.21.15.123:0/3493644382 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f4778004af0 con 0x55d928388c70 2024-09-19T04:50:47.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.085+0000 7f4773e00640 1 -- 172.21.15.123:0/3493644382 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4778004cf0 con 0x55d928388c70 2024-09-19T04:50:47.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.085+0000 7f479807c740 10 client.16005 fetch_fsmap learned FSMap version 36 2024-09-19T04:50:47.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.085+0000 7f479807c740 1 -- 172.21.15.123:0/3493644382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d9283a2a30 con 0x55d928388c70 2024-09-19T04:50:47.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.085+0000 7f4773e00640 1 -- 172.21.15.123:0/3493644382 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4780011290 con 0x55d928388c70 2024-09-19T04:50:47.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.085+0000 7f4773e00640 1 -- 172.21.15.123:0/3493644382 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f47780036f0 con 0x55d928388c70 2024-09-19T04:50:47.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.085+0000 7f479807c740 10 client.16005 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:50:47.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.085+0000 7f479807c740 10 client.16005 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:50:47.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.085+0000 7f479807c740 20 client.16005 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:47.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.085+0000 7f479807c740 1 --2- 172.21.15.123:0/3493644382 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d9283a3e40 0x55d9283a6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.085+0000 7f479807c740 4 client.16005 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:50:47.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.085+0000 7f479807c740 1 -- 172.21.15.123:0/3493644382 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9282fc950 con 0x55d9283a3e40 2024-09-19T04:50:47.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.086+0000 7f478ea00640 1 --2- 172.21.15.123:0/3493644382 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d9283a3e40 0x55d9283a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.086+0000 7f478ea00640 1 --2- 172.21.15.123:0/3493644382 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d9283a3e40 0x55d9283a6230 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e94b91109b1721b5 server_cookie=cb662e6115486c2a in_seq=0 out_seq=0 2024-09-19T04:50:47.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.086+0000 7f4773e00640 10 client.16005 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:50:47.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.087+0000 7f4773e00640 1 -- 172.21.15.123:0/3493644382 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9282fc950 con 0x55d9283a3e40 2024-09-19T04:50:47.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.087+0000 7f4773e00640 10 client.16005 handle_command_reply: tid=0 2024-09-19T04:50:47.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.087+0000 7f479807c740 1 client.16005 shutdown 2024-09-19T04:50:47.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.087+0000 7f479807c740 1 -- 172.21.15.123:0/3493644382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928388c70 msgr2=0x55d928390cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:47.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.087+0000 7f479807c740 1 --2- 172.21.15.123:0/3493644382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928388c70 0x55d928390cb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f477802a910 tx=0x7f4778005d50 comp rx=0 tx=0).stop 2024-09-19T04:50:47.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.088+0000 7f479807c740 1 -- 172.21.15.123:0/3493644382 shutdown_connections 2024-09-19T04:50:47.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.088+0000 7f479807c740 1 --2- 172.21.15.123:0/3493644382 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d9283a3e40 0x55d9283a6230 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.088+0000 7f479807c740 1 --2- 172.21.15.123:0/3493644382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928393e50 0x55d928391730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.088+0000 7f479807c740 1 --2- 172.21.15.123:0/3493644382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928389590 0x55d9283911f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.088+0000 7f4773e00640 0 client.16005 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:50:47.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.088+0000 7f479807c740 1 --2- 172.21.15.123:0/3493644382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928388c70 0x55d928390cb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.088+0000 7f479807c740 1 -- 172.21.15.123:0/3493644382 >> 172.21.15.123:0/3493644382 conn(0x55d928385200 msgr2=0x55d928384380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:47.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.088+0000 7f479807c740 1 -- 172.21.15.123:0/3493644382 shutdown_connections 2024-09-19T04:50:47.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.088+0000 7f479807c740 1 -- 172.21.15.123:0/3493644382 wait complete. 2024-09-19T04:50:47.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.088+0000 7f479807c740 20 client.16005 trim_cache size 0 max 16384 2024-09-19T04:50:47.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.103+0000 7f479807c740 1 Processor -- start 2024-09-19T04:50:47.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.103+0000 7f479807c740 1 -- start start 2024-09-19T04:50:47.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.104+0000 7f479807c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928388c70 0x55d928392c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.104+0000 7f479807c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928389590 0x55d9283abae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.104+0000 7f479807c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928393e50 0x55d9283ac020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.104+0000 7f479807c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47901af960 con 0x55d928388c70 2024-09-19T04:50:47.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.104+0000 7f479807c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47901b14a0 con 0x55d928393e50 2024-09-19T04:50:47.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.104+0000 7f479807c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f47901b0030 con 0x55d928389590 2024-09-19T04:50:47.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.104+0000 7f478ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928389590 0x55d9283abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.104+0000 7f4786000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928388c70 0x55d928392c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.104+0000 7f478ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928389590 0x55d9283abae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:59358/0 (socket says 172.21.15.123:59358) 2024-09-19T04:50:47.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.104+0000 7f478ea00640 1 -- 172.21.15.123:0/2792552319 learned_addr learned my addr 172.21.15.123:0/2792552319 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:47.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.104+0000 7f4786000640 1 --2- 172.21.15.123:0/2792552319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928393e50 0x55d9283ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.105+0000 7f478ea00640 1 -- 172.21.15.123:0/2792552319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928393e50 msgr2=0x55d9283ac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:47.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.105+0000 7f478ea00640 1 --2- 172.21.15.123:0/2792552319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928393e50 0x55d9283ac020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.105+0000 7f478ea00640 1 -- 172.21.15.123:0/2792552319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928388c70 msgr2=0x55d928392c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:47.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.105+0000 7f478ea00640 1 --2- 172.21.15.123:0/2792552319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928388c70 0x55d928392c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.105+0000 7f478ea00640 1 -- 172.21.15.123:0/2792552319 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d928384ca0 con 0x55d928389590 2024-09-19T04:50:47.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.105+0000 7f4786000640 1 --2- 172.21.15.123:0/2792552319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928388c70 0x55d928392c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:50:47.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.105+0000 7f478ea00640 1 --2- 172.21.15.123:0/2792552319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928389590 0x55d9283abae0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4780009c80 tx=0x7f4780011280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e2f1b36548c2876 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:47.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.105+0000 7f477ea00640 1 -- 172.21.15.123:0/2792552319 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780034070 con 0x55d928389590 2024-09-19T04:50:47.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.105+0000 7f477ea00640 1 -- 172.21.15.123:0/2792552319 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f478001e040 con 0x55d928389590 2024-09-19T04:50:47.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.105+0000 7f477ea00640 1 -- 172.21.15.123:0/2792552319 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478001d040 con 0x55d928389590 2024-09-19T04:50:47.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.106+0000 7f479807c740 1 -- 172.21.15.123:0/2792552319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928389590 msgr2=0x55d9283abae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:47.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.106+0000 7f479807c740 1 --2- 172.21.15.123:0/2792552319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928389590 0x55d9283abae0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4780009c80 tx=0x7f4780011280 comp rx=0 tx=0).stop 2024-09-19T04:50:47.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.106+0000 7f479807c740 1 -- 172.21.15.123:0/2792552319 shutdown_connections 2024-09-19T04:50:47.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.106+0000 7f479807c740 1 --2- 172.21.15.123:0/2792552319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928393e50 0x55d9283ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.106+0000 7f479807c740 1 --2- 172.21.15.123:0/2792552319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928389590 0x55d9283abae0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.106+0000 7f479807c740 1 --2- 172.21.15.123:0/2792552319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928388c70 0x55d928392c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.106+0000 7f479807c740 1 -- 172.21.15.123:0/2792552319 >> 172.21.15.123:0/2792552319 conn(0x55d928385200 msgr2=0x55d928392510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:47.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.106+0000 7f479807c740 1 -- 172.21.15.123:0/2792552319 shutdown_connections 2024-09-19T04:50:47.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.106+0000 7f479807c740 1 -- 172.21.15.123:0/2792552319 wait complete. 2024-09-19T04:50:47.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.107+0000 7f479807c740 1 -- start start 2024-09-19T04:50:47.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.107+0000 7f479807c740 1 Processor -- start 2024-09-19T04:50:47.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.107+0000 7f479807c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928388c70 0x55d928397af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.107+0000 7f479807c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928389590 0x55d928398030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.107+0000 7f479807c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928393e50 0x55d928398570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.107+0000 7f479807c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47901b0dd0 con 0x55d928389590 2024-09-19T04:50:47.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.107+0000 7f479807c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47901b0700 con 0x55d928388c70 2024-09-19T04:50:47.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.107+0000 7f479807c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f47901ad940 con 0x55d928393e50 2024-09-19T04:50:47.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.107+0000 7f479807c740 10 client.0 _ll_register_callbacks cb 0x7f4778005240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:47.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.108+0000 7f478ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928389590 0x55d928398030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.108+0000 7f478ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928389590 0x55d928398030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53700/0 (socket says 172.21.15.123:53700) 2024-09-19T04:50:47.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.108+0000 7f478ea00640 1 -- 172.21.15.123:0/404648182 learned_addr learned my addr 172.21.15.123:0/404648182 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:47.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.108+0000 7f4786000640 1 --2- 172.21.15.123:0/404648182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928388c70 0x55d928397af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.108+0000 7f4786000640 1 --2- 172.21.15.123:0/404648182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928393e50 0x55d928398570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.108+0000 7f478ea00640 1 -- 172.21.15.123:0/404648182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928393e50 msgr2=0x55d928398570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:47.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.108+0000 7f478ea00640 1 --2- 172.21.15.123:0/404648182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928393e50 0x55d928398570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.108+0000 7f478ea00640 1 -- 172.21.15.123:0/404648182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928388c70 msgr2=0x55d928397af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:47.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.108+0000 7f478ea00640 1 --2- 172.21.15.123:0/404648182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928388c70 0x55d928397af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.108+0000 7f478ea00640 1 -- 172.21.15.123:0/404648182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d92840dab0 con 0x55d928389590 2024-09-19T04:50:47.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.108+0000 7f4786000640 1 --2- 172.21.15.123:0/404648182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928393e50 0x55d928398570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:50:47.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.109+0000 7f478ea00640 1 --2- 172.21.15.123:0/404648182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928389590 0x55d928398030 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4780009c30 tx=0x7f4780009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7edba42910b4cb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:47.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.109+0000 7f4773e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:50:47.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.109+0000 7f4773e00640 1 -- 172.21.15.123:0/404648182 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780034030 con 0x55d928389590 2024-09-19T04:50:47.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.109+0000 7f4773e00640 1 -- 172.21.15.123:0/404648182 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f4780035070 con 0x55d928389590 2024-09-19T04:50:47.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.109+0000 7f479807c740 1 -- 172.21.15.123:0/404648182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f478001e040 con 0x55d928389590 2024-09-19T04:50:47.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.109+0000 7f4773e00640 1 -- 172.21.15.123:0/404648182 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f478001d040 con 0x55d928389590 2024-09-19T04:50:47.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.110+0000 7f479807c740 10 client.16011 fetch_fsmap learned FSMap version 36 2024-09-19T04:50:47.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.110+0000 7f479807c740 1 -- 172.21.15.123:0/404648182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d92840ecf0 con 0x55d928389590 2024-09-19T04:50:47.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.110+0000 7f4773e00640 1 -- 172.21.15.123:0/404648182 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f478001e040 con 0x55d928389590 2024-09-19T04:50:47.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.110+0000 7f4773e00640 1 -- 172.21.15.123:0/404648182 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f478002e070 con 0x55d928389590 2024-09-19T04:50:47.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.110+0000 7f479807c740 10 client.16011 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:50:47.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.110+0000 7f479807c740 10 client.16011 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:50:47.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.110+0000 7f479807c740 20 client.16011 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:47.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.110+0000 7f479807c740 1 --2- 172.21.15.123:0/404648182 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d92840f170 0x55d928411560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:47.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.110+0000 7f479807c740 4 client.16011 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:50:47.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.110+0000 7f479807c740 1 -- 172.21.15.123:0/404648182 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d9282f6e60 con 0x55d92840f170 2024-09-19T04:50:47.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.110+0000 7f4786000640 1 --2- 172.21.15.123:0/404648182 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d92840f170 0x55d928411560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:47.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.111+0000 7f4786000640 1 --2- 172.21.15.123:0/404648182 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d92840f170 0x55d928411560 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=618f646013d1d011 server_cookie=95645dff8b1f4f23 in_seq=0 out_seq=0 2024-09-19T04:50:47.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.111+0000 7f4773e00640 10 client.16011 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:50:47.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.111+0000 7f4773e00640 1 -- 172.21.15.123:0/404648182 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d9282f6e60 con 0x55d92840f170 2024-09-19T04:50:47.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.111+0000 7f4773e00640 10 client.16011 handle_command_reply: tid=0 2024-09-19T04:50:47.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.111+0000 7f479807c740 1 client.16011 shutdown 2024-09-19T04:50:47.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.111+0000 7f479807c740 1 -- 172.21.15.123:0/404648182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928389590 msgr2=0x55d928398030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:47.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.111+0000 7f479807c740 1 --2- 172.21.15.123:0/404648182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928389590 0x55d928398030 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4780009c30 tx=0x7f4780009a40 comp rx=0 tx=0).stop 2024-09-19T04:50:47.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.111+0000 7f479807c740 1 -- 172.21.15.123:0/404648182 shutdown_connections 2024-09-19T04:50:47.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.111+0000 7f479807c740 1 --2- 172.21.15.123:0/404648182 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d92840f170 0x55d928411560 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.111+0000 7f479807c740 1 --2- 172.21.15.123:0/404648182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d928393e50 0x55d928398570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.111+0000 7f479807c740 1 --2- 172.21.15.123:0/404648182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d928389590 0x55d928398030 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.111+0000 7f479807c740 1 --2- 172.21.15.123:0/404648182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d928388c70 0x55d928397af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.111+0000 7f479807c740 1 -- 172.21.15.123:0/404648182 >> 172.21.15.123:0/404648182 conn(0x55d928385200 msgr2=0x55d928391490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:47.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.111+0000 7f4773e00640 0 client.16011 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:50:47.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.112+0000 7f479807c740 1 -- 172.21.15.123:0/404648182 shutdown_connections 2024-09-19T04:50:47.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.112+0000 7f479807c740 1 -- 172.21.15.123:0/404648182 wait complete. 2024-09-19T04:50:47.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.112+0000 7f479807c740 20 client.16011 trim_cache size 0 max 16384 2024-09-19T04:50:47.118 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:50:47.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.113+0000 7f4795600640 1 -- 172.21.15.123:0/3955609213 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f47740a3430 msgr2=0x7f47740a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:47.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.113+0000 7f4795600640 1 --2- 172.21.15.123:0/3955609213 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f47740a3430 0x7f47740a58f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f47880a45e0 tx=0x7f4788004590 comp rx=0 tx=0).stop 2024-09-19T04:50:47.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.113+0000 7f4795600640 1 -- 172.21.15.123:0/3955609213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4790106f10 msgr2=0x7f479019ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:47.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.113+0000 7f4795600640 1 --2- 172.21.15.123:0/3955609213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4790106f10 0x7f479019ef50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f478800e970 tx=0x7f478800ee30 comp rx=0 tx=0).stop 2024-09-19T04:50:47.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.113+0000 7f4795600640 1 -- 172.21.15.123:0/3955609213 shutdown_connections 2024-09-19T04:50:47.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.113+0000 7f4795600640 1 --2- 172.21.15.123:0/3955609213 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f47740a3430 0x7f47740a58f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.113+0000 7f4795600640 1 --2- 172.21.15.123:0/3955609213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4790106f10 0x7f479019ef50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.113+0000 7f4795600640 1 --2- 172.21.15.123:0/3955609213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4790102790 0x7f479019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.113+0000 7f4795600640 1 --2- 172.21.15.123:0/3955609213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4790101de0 0x7f479019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:47.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.113+0000 7f4795600640 1 -- 172.21.15.123:0/3955609213 >> 172.21.15.123:0/3955609213 conn(0x7f47900fd8c0 msgr2=0x7f47900ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:47.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.113+0000 7f4795600640 1 -- 172.21.15.123:0/3955609213 shutdown_connections 2024-09-19T04:50:47.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:47.113+0000 7f4795600640 1 -- 172.21.15.123:0/3955609213 wait complete. 2024-09-19T04:50:47.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:47 smithi183 ceph-mon[35333]: pgmap v125: 97 pgs: 97 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.6 MiB/s wr, 447 op/s 2024-09-19T04:50:47.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:47 smithi123 ceph-mon[30511]: pgmap v125: 97 pgs: 97 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.6 MiB/s wr, 447 op/s 2024-09-19T04:50:47.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:47 smithi193 ceph-mon[35592]: pgmap v125: 97 pgs: 97 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.6 MiB/s wr, 447 op/s 2024-09-19T04:50:48.063 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:50:48.486 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:50:49.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.079+0000 7fbb5aa00640 1 -- 172.21.15.123:0/3293486996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb541019d0 msgr2=0x7fbb54101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:49.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.079+0000 7fbb5aa00640 1 --2- 172.21.15.123:0/3293486996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb541019d0 0x7fbb54101e30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009900 tx=0x7fbb4002a400 comp rx=0 tx=0).stop 2024-09-19T04:50:49.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.079+0000 7fbb5aa00640 1 -- 172.21.15.123:0/3293486996 shutdown_connections 2024-09-19T04:50:49.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.079+0000 7fbb5aa00640 1 --2- 172.21.15.123:0/3293486996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb54102370 0x7fbb5410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.079+0000 7fbb5aa00640 1 --2- 172.21.15.123:0/3293486996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb541019d0 0x7fbb54101e30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.079+0000 7fbb5aa00640 1 --2- 172.21.15.123:0/3293486996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb541079d0 0x7fbb54107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.079+0000 7fbb5aa00640 1 -- 172.21.15.123:0/3293486996 >> 172.21.15.123:0/3293486996 conn(0x7fbb540fd710 msgr2=0x7fbb540ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:49.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.079+0000 7fbb5aa00640 1 -- 172.21.15.123:0/3293486996 shutdown_connections 2024-09-19T04:50:49.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.079+0000 7fbb5aa00640 1 -- 172.21.15.123:0/3293486996 wait complete. 2024-09-19T04:50:49.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.081+0000 7fbb5aa00640 1 Processor -- start 2024-09-19T04:50:49.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.081+0000 7fbb5aa00640 1 -- start start 2024-09-19T04:50:49.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.081+0000 7fbb5aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb541019d0 0x7fbb541982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.081+0000 7fbb5aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb54102370 0x7fbb541987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.081+0000 7fbb5aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb541079d0 0x7fbb5419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.081+0000 7fbb5aa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbb54111e20 con 0x7fbb54102370 2024-09-19T04:50:49.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.081+0000 7fbb5aa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb54111ca0 con 0x7fbb541019d0 2024-09-19T04:50:49.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.081+0000 7fbb5aa00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbb54111fa0 con 0x7fbb541079d0 2024-09-19T04:50:49.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.082+0000 7fbb59600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb541019d0 0x7fbb541982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.082+0000 7fbb59600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb541019d0 0x7fbb541982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:36520/0 (socket says 172.21.15.123:36520) 2024-09-19T04:50:49.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.082+0000 7fbb59600640 1 -- 172.21.15.123:0/1054234399 learned_addr learned my addr 172.21.15.123:0/1054234399 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:49.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.082+0000 7fbb5a000640 1 --2- 172.21.15.123:0/1054234399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb541079d0 0x7fbb5419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.082+0000 7fbb58c00640 1 --2- 172.21.15.123:0/1054234399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb54102370 0x7fbb541987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.082+0000 7fbb59600640 1 -- 172.21.15.123:0/1054234399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb541079d0 msgr2=0x7fbb5419cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:49.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.082+0000 7fbb59600640 1 --2- 172.21.15.123:0/1054234399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb541079d0 0x7fbb5419cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.082+0000 7fbb59600640 1 -- 172.21.15.123:0/1054234399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb54102370 msgr2=0x7fbb541987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:49.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.082+0000 7fbb59600640 1 --2- 172.21.15.123:0/1054234399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb54102370 0x7fbb541987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.082+0000 7fbb59600640 1 -- 172.21.15.123:0/1054234399 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5419d250 con 0x7fbb541019d0 2024-09-19T04:50:49.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.082+0000 7fbb59600640 1 --2- 172.21.15.123:0/1054234399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb541019d0 0x7fbb541982a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800e9f0 tx=0x7fbb4800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:49.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.083+0000 7fbb52000640 1 -- 172.21.15.123:0/1054234399 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4800cd80 con 0x7fbb541019d0 2024-09-19T04:50:49.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.083+0000 7fbb52000640 1 -- 172.21.15.123:0/1054234399 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb48005500 con 0x7fbb541019d0 2024-09-19T04:50:49.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.083+0000 7fbb5aa00640 1 -- 172.21.15.123:0/1054234399 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5419d540 con 0x7fbb541019d0 2024-09-19T04:50:49.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.083+0000 7fbb52000640 1 -- 172.21.15.123:0/1054234399 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb48010690 con 0x7fbb541019d0 2024-09-19T04:50:49.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.083+0000 7fbb5aa00640 1 -- 172.21.15.123:0/1054234399 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb541a4e20 con 0x7fbb541019d0 2024-09-19T04:50:49.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.086+0000 7fbb5ec40740 1 Processor -- start 2024-09-19T04:50:49.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.086+0000 7fbb52000640 1 -- 172.21.15.123:0/1054234399 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fbb480046c0 con 0x7fbb541019d0 2024-09-19T04:50:49.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.086+0000 7fbb5ec40740 1 -- start start 2024-09-19T04:50:49.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.086+0000 7fbb5ec40740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc05e40 0x55ed8fc06220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.086+0000 7fbb5ec40740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc06760 0x55ed8fc10ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.086+0000 7fbb5ec40740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc11020 0x55ed8fc13410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.086+0000 7fbb5ec40740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ed8fb6c3e0 con 0x55ed8fc05e40 2024-09-19T04:50:49.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.087+0000 7fbb5ec40740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed8fb13d80 con 0x55ed8fc06760 2024-09-19T04:50:49.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.087+0000 7fbb5ec40740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ed8fb683a0 con 0x55ed8fc11020 2024-09-19T04:50:49.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.087+0000 7fbb59600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc06760 0x55ed8fc10ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.087+0000 7fbb59600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc06760 0x55ed8fc10ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:36530/0 (socket says 172.21.15.123:36530) 2024-09-19T04:50:49.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.087+0000 7fbb59600640 1 -- 172.21.15.123:0/3277927616 learned_addr learned my addr 172.21.15.123:0/3277927616 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:49.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.087+0000 7fbb58c00640 1 --2- 172.21.15.123:0/3277927616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc11020 0x55ed8fc13410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.087+0000 7fbb5a000640 1 --2- 172.21.15.123:0/3277927616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc05e40 0x55ed8fc06220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.087+0000 7fbb59600640 1 -- 172.21.15.123:0/3277927616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc11020 msgr2=0x55ed8fc13410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:49.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.087+0000 7fbb59600640 1 --2- 172.21.15.123:0/3277927616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc11020 0x55ed8fc13410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.087+0000 7fbb59600640 1 -- 172.21.15.123:0/3277927616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc05e40 msgr2=0x55ed8fc06220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:49.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.087+0000 7fbb59600640 1 --2- 172.21.15.123:0/3277927616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc05e40 0x55ed8fc06220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.087+0000 7fbb59600640 1 -- 172.21.15.123:0/3277927616 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed8fb32ae0 con 0x55ed8fc06760 2024-09-19T04:50:49.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.087+0000 7fbb5a000640 1 --2- 172.21.15.123:0/3277927616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc05e40 0x55ed8fc06220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:50:49.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.088+0000 7fbb52000640 1 --2- 172.21.15.123:0/1054234399 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbb3c0a33b0 0x7fbb3c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.088+0000 7fbb59600640 1 --2- 172.21.15.123:0/3277927616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc06760 0x55ed8fc10ae0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbb480cd830 tx=0x7fbb480d82d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44c25421513a81aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:49.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.088+0000 7fbb52000640 1 -- 172.21.15.123:0/1054234399 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fbb480d0140 con 0x7fbb541019d0 2024-09-19T04:50:49.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.088+0000 7fbb5a000640 1 --2- 172.21.15.123:0/1054234399 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbb3c0a33b0 0x7fbb3c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.088+0000 7fbb46600640 1 -- 172.21.15.123:0/3277927616 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb480986f0 con 0x55ed8fc06760 2024-09-19T04:50:49.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.088+0000 7fbb46600640 1 -- 172.21.15.123:0/3277927616 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb4809cb50 con 0x55ed8fc06760 2024-09-19T04:50:49.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.088+0000 7fbb46600640 1 -- 172.21.15.123:0/3277927616 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4809bdb0 con 0x55ed8fc06760 2024-09-19T04:50:49.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.089+0000 7fbb5ec40740 1 -- 172.21.15.123:0/3277927616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc06760 msgr2=0x55ed8fc10ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:49.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.089+0000 7fbb5a000640 1 --2- 172.21.15.123:0/1054234399 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbb3c0a33b0 0x7fbb3c0a5870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c005fd0 tx=0x7fbb4c009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:49.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.089+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/3277927616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc06760 0x55ed8fc10ae0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbb480cd830 tx=0x7fbb480d82d0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.089+0000 7fbb5ec40740 1 -- 172.21.15.123:0/3277927616 shutdown_connections 2024-09-19T04:50:49.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.089+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/3277927616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc11020 0x55ed8fc13410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.089+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/3277927616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc06760 0x55ed8fc10ae0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.089+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/3277927616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc05e40 0x55ed8fc06220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.089+0000 7fbb5ec40740 1 -- 172.21.15.123:0/3277927616 >> 172.21.15.123:0/3277927616 conn(0x55ed8fc023d0 msgr2=0x55ed8fc027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:49.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.089+0000 7fbb5ec40740 1 -- 172.21.15.123:0/3277927616 shutdown_connections 2024-09-19T04:50:49.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.089+0000 7fbb5ec40740 1 -- 172.21.15.123:0/3277927616 wait complete. 2024-09-19T04:50:49.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.090+0000 7fbb5ec40740 1 -- start start 2024-09-19T04:50:49.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.090+0000 7fbb5ec40740 1 Processor -- start 2024-09-19T04:50:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.091+0000 7fbb5ec40740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc05e40 0x55ed8fc0df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.091+0000 7fbb5ec40740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc06760 0x55ed8fc0e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.091+0000 7fbb5ec40740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc11020 0x55ed8fc0e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.091+0000 7fbb5ec40740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ed8fc13bf0 con 0x55ed8fc05e40 2024-09-19T04:50:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.091+0000 7fbb5ec40740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed8fb270f0 con 0x55ed8fc06760 2024-09-19T04:50:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.091+0000 7fbb5ec40740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ed8fc13d70 con 0x55ed8fc11020 2024-09-19T04:50:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.091+0000 7fbb5ec40740 10 client.0 _ll_register_callbacks cb 0x55ed8fb0ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.091+0000 7fbb58c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc06760 0x55ed8fc0e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.091+0000 7fbb58c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc06760 0x55ed8fc0e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:36538/0 (socket says 172.21.15.123:36538) 2024-09-19T04:50:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.091+0000 7fbb58c00640 1 -- 172.21.15.123:0/1681011166 learned_addr learned my addr 172.21.15.123:0/1681011166 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.091+0000 7fbb59600640 1 --2- 172.21.15.123:0/1681011166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc05e40 0x55ed8fc0df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.091+0000 7fbb5a000640 1 --2- 172.21.15.123:0/1681011166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc11020 0x55ed8fc0e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.092+0000 7fbb58c00640 1 -- 172.21.15.123:0/1681011166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc11020 msgr2=0x55ed8fc0e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.092+0000 7fbb58c00640 1 --2- 172.21.15.123:0/1681011166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc11020 0x55ed8fc0e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.092+0000 7fbb58c00640 1 -- 172.21.15.123:0/1681011166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc05e40 msgr2=0x55ed8fc0df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.092+0000 7fbb58c00640 1 --2- 172.21.15.123:0/1681011166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc05e40 0x55ed8fc0df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.092+0000 7fbb58c00640 1 -- 172.21.15.123:0/1681011166 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8f8dd3c0 con 0x55ed8fc06760 2024-09-19T04:50:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.092+0000 7fbb58c00640 1 --2- 172.21.15.123:0/1681011166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc06760 0x55ed8fc0e4b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbb40033000 tx=0x7fbb40005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88d1fe2c01c6549b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.092+0000 7fbb3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:50:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.092+0000 7fbb3aa00640 1 -- 172.21.15.123:0/1681011166 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40035040 con 0x55ed8fc06760 2024-09-19T04:50:49.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.092+0000 7fbb3aa00640 1 -- 172.21.15.123:0/1681011166 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fbb400048c0 con 0x55ed8fc06760 2024-09-19T04:50:49.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.092+0000 7fbb5ec40740 1 -- 172.21.15.123:0/1681011166 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb4809cb50 con 0x55ed8fc06760 2024-09-19T04:50:49.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.093+0000 7fbb3aa00640 1 -- 172.21.15.123:0/1681011166 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbb40005d50 con 0x55ed8fc06760 2024-09-19T04:50:49.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.093+0000 7fbb5ec40740 10 client.26105 fetch_fsmap learned FSMap version 36 2024-09-19T04:50:49.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.093+0000 7fbb5ec40740 1 -- 172.21.15.123:0/1681011166 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ed8fc24010 con 0x55ed8fc06760 2024-09-19T04:50:49.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.093+0000 7fbb3aa00640 1 -- 172.21.15.123:0/1681011166 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb4809cb50 con 0x55ed8fc06760 2024-09-19T04:50:49.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.093+0000 7fbb3aa00640 1 -- 172.21.15.123:0/1681011166 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fbb40031540 con 0x55ed8fc06760 2024-09-19T04:50:49.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.093+0000 7fbb5ec40740 10 client.26105 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:50:49.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.093+0000 7fbb5ec40740 10 client.26105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:50:49.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.093+0000 7fbb5ec40740 20 client.26105 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:49.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.093+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/1681011166 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ed8fc254d0 0x55ed8fc278e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.093+0000 7fbb5ec40740 4 client.26105 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:50:49.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.093+0000 7fbb5ec40740 1 -- 172.21.15.123:0/1681011166 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed8fb79950 con 0x55ed8fc254d0 2024-09-19T04:50:49.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.094+0000 7fbb59600640 1 --2- 172.21.15.123:0/1681011166 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ed8fc254d0 0x55ed8fc278e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.094+0000 7fbb59600640 1 --2- 172.21.15.123:0/1681011166 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ed8fc254d0 0x55ed8fc278e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=523a3f71134e849b server_cookie=7e2003fad74c612c in_seq=0 out_seq=0 2024-09-19T04:50:49.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.094+0000 7fbb3aa00640 10 client.26105 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:49.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.095+0000 7fbb3aa00640 1 -- 172.21.15.123:0/1681011166 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed8fb79950 con 0x55ed8fc254d0 2024-09-19T04:50:49.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.095+0000 7fbb3aa00640 10 client.26105 handle_command_reply: tid=0 2024-09-19T04:50:49.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.095+0000 7fbb5ec40740 1 client.26105 shutdown 2024-09-19T04:50:49.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.095+0000 7fbb5ec40740 1 -- 172.21.15.123:0/1681011166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc06760 msgr2=0x55ed8fc0e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:49.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.095+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/1681011166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc06760 0x55ed8fc0e4b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbb40033000 tx=0x7fbb40005270 comp rx=0 tx=0).stop 2024-09-19T04:50:49.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.095+0000 7fbb5ec40740 1 -- 172.21.15.123:0/1681011166 shutdown_connections 2024-09-19T04:50:49.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.095+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/1681011166 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ed8fc254d0 0x55ed8fc278e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.095+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/1681011166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc11020 0x55ed8fc0e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.095+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/1681011166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc06760 0x55ed8fc0e4b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.095+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/1681011166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc05e40 0x55ed8fc0df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.095+0000 7fbb5ec40740 1 -- 172.21.15.123:0/1681011166 >> 172.21.15.123:0/1681011166 conn(0x55ed8fc023d0 msgr2=0x55ed8fc08100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:49.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.095+0000 7fbb3aa00640 0 client.26105 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:49.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.095+0000 7fbb5ec40740 1 -- 172.21.15.123:0/1681011166 shutdown_connections 2024-09-19T04:50:49.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.096+0000 7fbb5ec40740 1 -- 172.21.15.123:0/1681011166 wait complete. 2024-09-19T04:50:49.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.096+0000 7fbb5ec40740 20 client.26105 trim_cache size 0 max 16384 2024-09-19T04:50:49.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.110+0000 7fbb5ec40740 1 Processor -- start 2024-09-19T04:50:49.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.110+0000 7fbb5ec40740 1 -- start start 2024-09-19T04:50:49.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.110+0000 7fbb5ec40740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc05e40 0x55ed8fc088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.110+0000 7fbb5ec40740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc06760 0x55ed8fc0c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.110+0000 7fbb5ec40740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc11020 0x55ed8fc2d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.110+0000 7fbb5ec40740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbb541ab490 con 0x55ed8fc11020 2024-09-19T04:50:49.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.110+0000 7fbb5ec40740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb541acfd0 con 0x55ed8fc05e40 2024-09-19T04:50:49.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.110+0000 7fbb5ec40740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbb541abb60 con 0x55ed8fc06760 2024-09-19T04:50:49.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.110+0000 7fbb58c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc06760 0x55ed8fc0c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.110+0000 7fbb58c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc06760 0x55ed8fc0c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:59418/0 (socket says 172.21.15.123:59418) 2024-09-19T04:50:49.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.110+0000 7fbb58c00640 1 -- 172.21.15.123:0/2502347920 learned_addr learned my addr 172.21.15.123:0/2502347920 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:49.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.110+0000 7fbb59600640 1 --2- 172.21.15.123:0/2502347920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc05e40 0x55ed8fc088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.110+0000 7fbb5a000640 1 --2- 172.21.15.123:0/2502347920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc11020 0x55ed8fc2d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.111+0000 7fbb58c00640 1 -- 172.21.15.123:0/2502347920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc05e40 msgr2=0x55ed8fc088f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:49.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.111+0000 7fbb58c00640 1 --2- 172.21.15.123:0/2502347920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc05e40 0x55ed8fc088f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.111+0000 7fbb58c00640 1 -- 172.21.15.123:0/2502347920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc11020 msgr2=0x55ed8fc2d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:49.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.111+0000 7fbb58c00640 1 --2- 172.21.15.123:0/2502347920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc11020 0x55ed8fc2d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.111+0000 7fbb58c00640 1 -- 172.21.15.123:0/2502347920 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed8fc0cf20 con 0x55ed8fc06760 2024-09-19T04:50:49.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.111+0000 7fbb59600640 1 --2- 172.21.15.123:0/2502347920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc05e40 0x55ed8fc088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:49.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.111+0000 7fbb58c00640 1 --2- 172.21.15.123:0/2502347920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc06760 0x55ed8fc0c840 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbb40031990 tx=0x7fbb4002a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb954b641ebf0628 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:49.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.111+0000 7fbb45c00640 1 -- 172.21.15.123:0/2502347920 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40052070 con 0x55ed8fc06760 2024-09-19T04:50:49.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.111+0000 7fbb45c00640 1 -- 172.21.15.123:0/2502347920 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb40040070 con 0x55ed8fc06760 2024-09-19T04:50:49.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.111+0000 7fbb45c00640 1 -- 172.21.15.123:0/2502347920 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40035040 con 0x55ed8fc06760 2024-09-19T04:50:49.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.112+0000 7fbb5ec40740 1 -- 172.21.15.123:0/2502347920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc06760 msgr2=0x55ed8fc0c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:49.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.112+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/2502347920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc06760 0x55ed8fc0c840 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbb40031990 tx=0x7fbb4002a400 comp rx=0 tx=0).stop 2024-09-19T04:50:49.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.112+0000 7fbb5ec40740 1 -- 172.21.15.123:0/2502347920 shutdown_connections 2024-09-19T04:50:49.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.112+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/2502347920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc11020 0x55ed8fc2d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.112+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/2502347920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc06760 0x55ed8fc0c840 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.112+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/2502347920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc05e40 0x55ed8fc088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.112+0000 7fbb5ec40740 1 -- 172.21.15.123:0/2502347920 >> 172.21.15.123:0/2502347920 conn(0x55ed8fc023d0 msgr2=0x55ed8fc0fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:49.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.112+0000 7fbb5ec40740 1 -- 172.21.15.123:0/2502347920 shutdown_connections 2024-09-19T04:50:49.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.112+0000 7fbb5ec40740 1 -- 172.21.15.123:0/2502347920 wait complete. 2024-09-19T04:50:49.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.113+0000 7fbb5ec40740 1 -- start start 2024-09-19T04:50:49.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.113+0000 7fbb5ec40740 1 Processor -- start 2024-09-19T04:50:49.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.113+0000 7fbb5ec40740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc05e40 0x55ed8fc14de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.113+0000 7fbb5ec40740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc06760 0x55ed8fc15320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.113+0000 7fbb5ec40740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc11020 0x55ed8fc15860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.113+0000 7fbb5ec40740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbb541ac900 con 0x55ed8fc06760 2024-09-19T04:50:49.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.113+0000 7fbb5ec40740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb541ac230 con 0x55ed8fc11020 2024-09-19T04:50:49.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.113+0000 7fbb5ec40740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbb541a9470 con 0x55ed8fc05e40 2024-09-19T04:50:49.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.113+0000 7fbb5ec40740 10 client.0 _ll_register_callbacks cb 0x7fbb48005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:49.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.114+0000 7fbb5a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc11020 0x55ed8fc15860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.114+0000 7fbb59600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc05e40 0x55ed8fc14de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.114+0000 7fbb5a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc11020 0x55ed8fc15860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:36558/0 (socket says 172.21.15.123:36558) 2024-09-19T04:50:49.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.114+0000 7fbb5a000640 1 -- 172.21.15.123:0/1722366288 learned_addr learned my addr 172.21.15.123:0/1722366288 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:49.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.114+0000 7fbb58c00640 1 --2- 172.21.15.123:0/1722366288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc06760 0x55ed8fc15320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.114+0000 7fbb5a000640 1 -- 172.21.15.123:0/1722366288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc05e40 msgr2=0x55ed8fc14de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:49.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.114+0000 7fbb5a000640 1 --2- 172.21.15.123:0/1722366288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc05e40 0x55ed8fc14de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.114+0000 7fbb5a000640 1 -- 172.21.15.123:0/1722366288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc06760 msgr2=0x55ed8fc15320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:49.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.114+0000 7fbb5a000640 1 --2- 172.21.15.123:0/1722366288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc06760 0x55ed8fc15320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.114+0000 7fbb5a000640 1 -- 172.21.15.123:0/1722366288 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8fc8f250 con 0x55ed8fc11020 2024-09-19T04:50:49.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.114+0000 7fbb59600640 1 --2- 172.21.15.123:0/1722366288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc05e40 0x55ed8fc14de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:49.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.114+0000 7fbb5a000640 1 --2- 172.21.15.123:0/1722366288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc11020 0x55ed8fc15860 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c004310 tx=0x7fbb4c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=235222833d67aa65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:49.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.114+0000 7fbb3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:50:49.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.115+0000 7fbb3aa00640 1 -- 172.21.15.123:0/1722366288 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c011b30 con 0x55ed8fc11020 2024-09-19T04:50:49.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.115+0000 7fbb5ec40740 1 -- 172.21.15.123:0/1722366288 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb40040070 con 0x55ed8fc11020 2024-09-19T04:50:49.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.115+0000 7fbb3aa00640 1 -- 172.21.15.123:0/1722366288 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fbb4c011cd0 con 0x55ed8fc11020 2024-09-19T04:50:49.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.115+0000 7fbb3aa00640 1 -- 172.21.15.123:0/1722366288 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbb4c032430 con 0x55ed8fc11020 2024-09-19T04:50:49.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.115+0000 7fbb5ec40740 10 client.26111 fetch_fsmap learned FSMap version 36 2024-09-19T04:50:49.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.115+0000 7fbb5ec40740 1 -- 172.21.15.123:0/1722366288 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ed8fc90580 con 0x55ed8fc11020 2024-09-19T04:50:49.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.115+0000 7fbb3aa00640 1 -- 172.21.15.123:0/1722366288 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb40040070 con 0x55ed8fc11020 2024-09-19T04:50:49.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.115+0000 7fbb3aa00640 1 -- 172.21.15.123:0/1722366288 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fbb4c029410 con 0x55ed8fc11020 2024-09-19T04:50:49.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.115+0000 7fbb5ec40740 10 client.26111 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:50:49.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.115+0000 7fbb5ec40740 10 client.26111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:50:49.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.115+0000 7fbb5ec40740 20 client.26111 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:49.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.116+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/1722366288 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ed8fc90d00 0x55ed8fc93140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:49.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.116+0000 7fbb5ec40740 4 client.26111 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:50:49.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.116+0000 7fbb5ec40740 1 -- 172.21.15.123:0/1722366288 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed8fb73e60 con 0x55ed8fc90d00 2024-09-19T04:50:49.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.116+0000 7fbb59600640 1 --2- 172.21.15.123:0/1722366288 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ed8fc90d00 0x55ed8fc93140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:49.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.116+0000 7fbb3aa00640 10 client.26111 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:49.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.116+0000 7fbb59600640 1 --2- 172.21.15.123:0/1722366288 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ed8fc90d00 0x55ed8fc93140 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eacc49a068ca7c0 server_cookie=cbd3bc6744a08c56 in_seq=0 out_seq=0 2024-09-19T04:50:49.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.116+0000 7fbb3aa00640 1 -- 172.21.15.123:0/1722366288 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed8fb73e60 con 0x55ed8fc90d00 2024-09-19T04:50:49.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.116+0000 7fbb3aa00640 10 client.26111 handle_command_reply: tid=0 2024-09-19T04:50:49.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.117+0000 7fbb5ec40740 1 client.26111 shutdown 2024-09-19T04:50:49.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.117+0000 7fbb5ec40740 1 -- 172.21.15.123:0/1722366288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc11020 msgr2=0x55ed8fc15860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:49.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.117+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/1722366288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc11020 0x55ed8fc15860 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c004310 tx=0x7fbb4c004a40 comp rx=0 tx=0).stop 2024-09-19T04:50:49.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.117+0000 7fbb5ec40740 1 -- 172.21.15.123:0/1722366288 shutdown_connections 2024-09-19T04:50:49.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.117+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/1722366288 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ed8fc90d00 0x55ed8fc93140 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.117+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/1722366288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8fc11020 0x55ed8fc15860 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.117+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/1722366288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8fc06760 0x55ed8fc15320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.117+0000 7fbb5ec40740 1 --2- 172.21.15.123:0/1722366288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed8fc05e40 0x55ed8fc14de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.117+0000 7fbb3aa00640 0 client.26111 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:49.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.117+0000 7fbb5ec40740 1 -- 172.21.15.123:0/1722366288 >> 172.21.15.123:0/1722366288 conn(0x55ed8fc023d0 msgr2=0x55ed8fc0df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:49.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.117+0000 7fbb5ec40740 1 -- 172.21.15.123:0/1722366288 shutdown_connections 2024-09-19T04:50:49.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.117+0000 7fbb5ec40740 1 -- 172.21.15.123:0/1722366288 wait complete. 2024-09-19T04:50:49.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.117+0000 7fbb5ec40740 20 client.26111 trim_cache size 0 max 16384 2024-09-19T04:50:49.124 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:50:49.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.118+0000 7fbb5aa00640 1 -- 172.21.15.123:0/1054234399 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbb3c0a33b0 msgr2=0x7fbb3c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:49.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.118+0000 7fbb5aa00640 1 --2- 172.21.15.123:0/1054234399 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbb3c0a33b0 0x7fbb3c0a5870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c005fd0 tx=0x7fbb4c009500 comp rx=0 tx=0).stop 2024-09-19T04:50:49.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.118+0000 7fbb5aa00640 1 -- 172.21.15.123:0/1054234399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb541019d0 msgr2=0x7fbb541982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:49.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.118+0000 7fbb5aa00640 1 --2- 172.21.15.123:0/1054234399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb541019d0 0x7fbb541982a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800e9f0 tx=0x7fbb4800edf0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.118+0000 7fbb5aa00640 1 -- 172.21.15.123:0/1054234399 shutdown_connections 2024-09-19T04:50:49.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.118+0000 7fbb5aa00640 1 --2- 172.21.15.123:0/1054234399 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbb3c0a33b0 0x7fbb3c0a5870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.118+0000 7fbb5aa00640 1 --2- 172.21.15.123:0/1054234399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb541079d0 0x7fbb5419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.118+0000 7fbb5aa00640 1 --2- 172.21.15.123:0/1054234399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb54102370 0x7fbb541987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.118+0000 7fbb5aa00640 1 --2- 172.21.15.123:0/1054234399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb541019d0 0x7fbb541982a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:49.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.118+0000 7fbb5aa00640 1 -- 172.21.15.123:0/1054234399 >> 172.21.15.123:0/1054234399 conn(0x7fbb540fd710 msgr2=0x7fbb540fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:49.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.118+0000 7fbb5aa00640 1 -- 172.21.15.123:0/1054234399 shutdown_connections 2024-09-19T04:50:49.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:49.118+0000 7fbb5aa00640 1 -- 172.21.15.123:0/1054234399 wait complete. 2024-09-19T04:50:49.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:49 smithi183 ceph-mon[35333]: pgmap v126: 97 pgs: 97 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 659 op/s 2024-09-19T04:50:49.381 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:49 smithi123 ceph-mon[30511]: pgmap v126: 97 pgs: 97 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 659 op/s 2024-09-19T04:50:49.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:49 smithi193 ceph-mon[35592]: pgmap v126: 97 pgs: 97 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 659 op/s 2024-09-19T04:50:51.164 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:50:51.164 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:50:51.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:51 smithi183 ceph-mon[35333]: pgmap v127: 97 pgs: 97 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.5 MiB/s wr, 430 op/s 2024-09-19T04:50:51.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:51 smithi123 ceph-mon[30511]: pgmap v127: 97 pgs: 97 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.5 MiB/s wr, 430 op/s 2024-09-19T04:50:51.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:51 smithi193 ceph-mon[35592]: pgmap v127: 97 pgs: 97 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.5 MiB/s wr, 430 op/s 2024-09-19T04:50:51.575 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:50:51.776 INFO:tasks.workunit.client.0.smithi123.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-09-19T04:50:51.777 INFO:tasks.workunit.client.0.smithi123.stdout:All 50000 operations completed A-OK! 2024-09-19T04:50:51.778 INFO:tasks.workunit.client.0.smithi123.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-09-19T04:50:51.780 INFO:tasks.workunit.client.0.smithi123.stdout:Seed set to 1 2024-09-19T04:50:52.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.257+0000 7fcf5d000640 1 -- 172.21.15.123:0/2908823795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf5810e440 msgr2=0x7fcf58110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:52.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.257+0000 7fcf5d000640 1 --2- 172.21.15.123:0/2908823795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf5810e440 0x7fcf58110830 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009be0 tx=0x7fcf5002a750 comp rx=0 tx=0).stop 2024-09-19T04:50:52.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.257+0000 7fcf5d000640 1 -- 172.21.15.123:0/2908823795 shutdown_connections 2024-09-19T04:50:52.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.257+0000 7fcf5d000640 1 --2- 172.21.15.123:0/2908823795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf5810e440 0x7fcf58110830 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.257+0000 7fcf5d000640 1 --2- 172.21.15.123:0/2908823795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf58074e30 0x7fcf5810df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.257+0000 7fcf5d000640 1 --2- 172.21.15.123:0/2908823795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf58074510 0x7fcf580748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.257+0000 7fcf5d000640 1 -- 172.21.15.123:0/2908823795 >> 172.21.15.123:0/2908823795 conn(0x7fcf580fd8c0 msgr2=0x7fcf580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:52.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.257+0000 7fcf5d000640 1 -- 172.21.15.123:0/2908823795 shutdown_connections 2024-09-19T04:50:52.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.257+0000 7fcf5d000640 1 -- 172.21.15.123:0/2908823795 wait complete. 2024-09-19T04:50:52.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.258+0000 7fcf5d000640 1 Processor -- start 2024-09-19T04:50:52.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.258+0000 7fcf5d000640 1 -- start start 2024-09-19T04:50:52.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.259+0000 7fcf5d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf58074510 0x7fcf5819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.259+0000 7fcf5d000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf58074e30 0x7fcf5819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.259+0000 7fcf5d000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf5810e440 0x7fcf5819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.259+0000 7fcf5d000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcf58114150 con 0x7fcf58074510 2024-09-19T04:50:52.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.259+0000 7fcf5d000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf58113fd0 con 0x7fcf5810e440 2024-09-19T04:50:52.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.259+0000 7fcf5d000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581142d0 con 0x7fcf58074e30 2024-09-19T04:50:52.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.259+0000 7fcf55e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf58074510 0x7fcf5819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.259+0000 7fcf55e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf58074510 0x7fcf5819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46392/0 (socket says 172.21.15.123:46392) 2024-09-19T04:50:52.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.259+0000 7fcf55e00640 1 -- 172.21.15.123:0/587339060 learned_addr learned my addr 172.21.15.123:0/587339060 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:52.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.259+0000 7fcf4fe00640 1 --2- 172.21.15.123:0/587339060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf58074e30 0x7fcf5819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.259+0000 7fcf56800640 1 --2- 172.21.15.123:0/587339060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf5810e440 0x7fcf5819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.260+0000 7fcf55e00640 1 -- 172.21.15.123:0/587339060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf58074e30 msgr2=0x7fcf5819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:52.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.260+0000 7fcf55e00640 1 --2- 172.21.15.123:0/587339060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf58074e30 0x7fcf5819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.260+0000 7fcf55e00640 1 -- 172.21.15.123:0/587339060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf5810e440 msgr2=0x7fcf5819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:52.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.260+0000 7fcf55e00640 1 --2- 172.21.15.123:0/587339060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf5810e440 0x7fcf5819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.260+0000 7fcf55e00640 1 -- 172.21.15.123:0/587339060 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5819f5d0 con 0x7fcf58074510 2024-09-19T04:50:52.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.260+0000 7fcf4fe00640 1 --2- 172.21.15.123:0/587339060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf58074e30 0x7fcf5819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:52.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.260+0000 7fcf55e00640 1 --2- 172.21.15.123:0/587339060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf58074510 0x7fcf5819a620 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800e9f0 tx=0x7fcf4800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:52.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.260+0000 7fcf4ea00640 1 -- 172.21.15.123:0/587339060 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4800ccc0 con 0x7fcf58074510 2024-09-19T04:50:52.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.260+0000 7fcf4ea00640 1 -- 172.21.15.123:0/587339060 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf48010910 con 0x7fcf58074510 2024-09-19T04:50:52.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.260+0000 7fcf4ea00640 1 -- 172.21.15.123:0/587339060 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf480188a0 con 0x7fcf58074510 2024-09-19T04:50:52.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.260+0000 7fcf5d000640 1 -- 172.21.15.123:0/587339060 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5819f8c0 con 0x7fcf58074510 2024-09-19T04:50:52.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.260+0000 7fcf5d000640 1 -- 172.21.15.123:0/587339060 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf581a71a0 con 0x7fcf58074510 2024-09-19T04:50:52.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.264+0000 7fcf4ea00640 1 -- 172.21.15.123:0/587339060 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fcf48010ac0 con 0x7fcf58074510 2024-09-19T04:50:52.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.264+0000 7fcf5fbbc740 1 Processor -- start 2024-09-19T04:50:52.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.264+0000 7fcf5fbbc740 1 -- start start 2024-09-19T04:50:52.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.264+0000 7fcf5fbbc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e8ac0 0x55559e5e8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.264+0000 7fcf5fbbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e93e0 0x55559e5f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.264+0000 7fcf5fbbc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 0x55559e5f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.264+0000 7fcf5fbbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55559e54f290 con 0x55559e5e93e0 2024-09-19T04:50:52.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.264+0000 7fcf5fbbc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55559e4f6b40 con 0x55559e5e8ac0 2024-09-19T04:50:52.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.264+0000 7fcf5fbbc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55559e54b250 con 0x55559e5f3ca0 2024-09-19T04:50:52.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.265+0000 7fcf55e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e93e0 0x55559e5f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.265+0000 7fcf4fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 0x55559e5f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.265+0000 7fcf55e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e93e0 0x55559e5f3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46406/0 (socket says 172.21.15.123:46406) 2024-09-19T04:50:52.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.265+0000 7fcf55e00640 1 -- 172.21.15.123:0/4126365743 learned_addr learned my addr 172.21.15.123:0/4126365743 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:52.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.265+0000 7fcf56800640 1 --2- 172.21.15.123:0/4126365743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e8ac0 0x55559e5e8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.265+0000 7fcf55e00640 1 -- 172.21.15.123:0/4126365743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 msgr2=0x55559e5f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:52.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.265+0000 7fcf55e00640 1 --2- 172.21.15.123:0/4126365743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 0x55559e5f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.265+0000 7fcf55e00640 1 -- 172.21.15.123:0/4126365743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e8ac0 msgr2=0x55559e5e8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:52.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.265+0000 7fcf55e00640 1 --2- 172.21.15.123:0/4126365743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e8ac0 0x55559e5e8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.265+0000 7fcf4ea00640 1 --2- 172.21.15.123:0/587339060 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fcf3c0a3430 0x7fcf3c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.265+0000 7fcf55e00640 1 -- 172.21.15.123:0/4126365743 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55559e2c06a0 con 0x55559e5e93e0 2024-09-19T04:50:52.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.265+0000 7fcf4fe00640 1 --2- 172.21.15.123:0/4126365743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 0x55559e5f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:52.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.266+0000 7fcf4ea00640 1 -- 172.21.15.123:0/587339060 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fcf480d0100 con 0x7fcf58074510 2024-09-19T04:50:52.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.266+0000 7fcf55e00640 1 --2- 172.21.15.123:0/4126365743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e93e0 0x55559e5f3760 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcf480cd760 tx=0x7fcf48098650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973cdaf0dba30357 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:52.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.266+0000 7fcf4fe00640 1 --2- 172.21.15.123:0/587339060 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fcf3c0a3430 0x7fcf3c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.266+0000 7fcf47400640 1 -- 172.21.15.123:0/4126365743 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48026030 con 0x55559e5e93e0 2024-09-19T04:50:52.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.266+0000 7fcf47400640 1 -- 172.21.15.123:0/4126365743 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf480a5020 con 0x55559e5e93e0 2024-09-19T04:50:52.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.266+0000 7fcf47400640 1 -- 172.21.15.123:0/4126365743 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf480a4b90 con 0x55559e5e93e0 2024-09-19T04:50:52.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.267+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/4126365743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e93e0 msgr2=0x55559e5f3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:52.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.267+0000 7fcf4fe00640 1 --2- 172.21.15.123:0/587339060 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fcf3c0a3430 0x7fcf3c0a58f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009fd0 tx=0x7fcf40009260 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:52.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.267+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/4126365743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e93e0 0x55559e5f3760 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcf480cd760 tx=0x7fcf48098650 comp rx=0 tx=0).stop 2024-09-19T04:50:52.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.267+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/4126365743 shutdown_connections 2024-09-19T04:50:52.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.267+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/4126365743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 0x55559e5f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.267+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/4126365743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e93e0 0x55559e5f3760 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.267+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/4126365743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e8ac0 0x55559e5e8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.267+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/4126365743 >> 172.21.15.123:0/4126365743 conn(0x55559e5e5050 msgr2=0x55559e5e5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:52.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.267+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/4126365743 shutdown_connections 2024-09-19T04:50:52.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.267+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/4126365743 wait complete. 2024-09-19T04:50:52.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.268+0000 7fcf5fbbc740 1 -- start start 2024-09-19T04:50:52.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.268+0000 7fcf5fbbc740 1 Processor -- start 2024-09-19T04:50:52.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.268+0000 7fcf5fbbc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e8ac0 0x55559e5f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.268+0000 7fcf5fbbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e93e0 0x55559e5f0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.268+0000 7fcf5fbbc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 0x55559e5f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.269+0000 7fcf5fbbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55559e5f6910 con 0x55559e5e93e0 2024-09-19T04:50:52.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.269+0000 7fcf5fbbc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55559e509fa0 con 0x55559e5e8ac0 2024-09-19T04:50:52.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.269+0000 7fcf5fbbc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55559e5f6a90 con 0x55559e5f3ca0 2024-09-19T04:50:52.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.269+0000 7fcf5fbbc740 10 client.0 _ll_register_callbacks cb 0x55559e25dba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:52.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.269+0000 7fcf56800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 0x55559e5f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.269+0000 7fcf56800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 0x55559e5f0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58332/0 (socket says 172.21.15.123:58332) 2024-09-19T04:50:52.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.269+0000 7fcf56800640 1 -- 172.21.15.123:0/3006440270 learned_addr learned my addr 172.21.15.123:0/3006440270 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:52.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.269+0000 7fcf4fe00640 1 --2- 172.21.15.123:0/3006440270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e93e0 0x55559e5f0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.269+0000 7fcf55e00640 1 --2- 172.21.15.123:0/3006440270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e8ac0 0x55559e5f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.269+0000 7fcf56800640 1 -- 172.21.15.123:0/3006440270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e8ac0 msgr2=0x55559e5f36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:52.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.269+0000 7fcf56800640 1 --2- 172.21.15.123:0/3006440270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e8ac0 0x55559e5f36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.269+0000 7fcf56800640 1 -- 172.21.15.123:0/3006440270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e93e0 msgr2=0x55559e5f0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:52.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.269+0000 7fcf56800640 1 --2- 172.21.15.123:0/3006440270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e93e0 0x55559e5f0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.269+0000 7fcf56800640 1 -- 172.21.15.123:0/3006440270 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55559e5f17a0 con 0x55559e5f3ca0 2024-09-19T04:50:52.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.270+0000 7fcf56800640 1 --2- 172.21.15.123:0/3006440270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 0x55559e5f0f70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcf580754f0 tx=0x7fcf50030da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0e38446ee7808b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:52.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.270+0000 7fcf3be00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:50:52.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.270+0000 7fcf3be00640 1 -- 172.21.15.123:0/3006440270 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50036040 con 0x55559e5f3ca0 2024-09-19T04:50:52.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.270+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/3006440270 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55559e55c800 con 0x55559e5f3ca0 2024-09-19T04:50:52.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.270+0000 7fcf3be00640 1 -- 172.21.15.123:0/3006440270 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fcf500305a0 con 0x55559e5f3ca0 2024-09-19T04:50:52.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.270+0000 7fcf3be00640 1 -- 172.21.15.123:0/3006440270 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcf500307a0 con 0x55559e5f3ca0 2024-09-19T04:50:52.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.270+0000 7fcf5fbbc740 10 client.26143 fetch_fsmap learned FSMap version 36 2024-09-19T04:50:52.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.270+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/3006440270 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55559e60b150 con 0x55559e5f3ca0 2024-09-19T04:50:52.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.271+0000 7fcf3be00640 1 -- 172.21.15.123:0/3006440270 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55559e55c800 con 0x55559e5f3ca0 2024-09-19T04:50:52.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.271+0000 7fcf3be00640 1 -- 172.21.15.123:0/3006440270 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fcf500045e0 con 0x55559e5f3ca0 2024-09-19T04:50:52.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.271+0000 7fcf5fbbc740 10 client.26143 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:50:52.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.271+0000 7fcf5fbbc740 10 client.26143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:50:52.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.271+0000 7fcf5fbbc740 20 client.26143 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:52.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.271+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/3006440270 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55559e60c5b0 0x55559e60e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.271+0000 7fcf5fbbc740 4 client.26143 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:50:52.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.271+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/3006440270 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55559e556d10 con 0x55559e60c5b0 2024-09-19T04:50:52.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.271+0000 7fcf55e00640 1 --2- 172.21.15.123:0/3006440270 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55559e60c5b0 0x55559e60e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.272+0000 7fcf55e00640 1 --2- 172.21.15.123:0/3006440270 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55559e60c5b0 0x55559e60e9d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9875d8dd600c7075 server_cookie=ca7a8a31bc26a6fd in_seq=0 out_seq=0 2024-09-19T04:50:52.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.272+0000 7fcf3be00640 10 client.26143 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:52.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.273+0000 7fcf3be00640 1 -- 172.21.15.123:0/3006440270 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55559e556d10 con 0x55559e60c5b0 2024-09-19T04:50:52.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.273+0000 7fcf3be00640 10 client.26143 handle_command_reply: tid=0 2024-09-19T04:50:52.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.273+0000 7fcf5fbbc740 1 client.26143 shutdown 2024-09-19T04:50:52.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.273+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/3006440270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 msgr2=0x55559e5f0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:52.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.273+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/3006440270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 0x55559e5f0f70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcf580754f0 tx=0x7fcf50030da0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.273+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/3006440270 shutdown_connections 2024-09-19T04:50:52.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.273+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/3006440270 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55559e60c5b0 0x55559e60e9d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.273+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/3006440270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 0x55559e5f0f70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.273+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/3006440270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e93e0 0x55559e5f0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.273+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/3006440270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e8ac0 0x55559e5f36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.273+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/3006440270 >> 172.21.15.123:0/3006440270 conn(0x55559e5e5050 msgr2=0x55559e5ead80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:52.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.273+0000 7fcf3be00640 0 client.26143 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:52.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.273+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/3006440270 shutdown_connections 2024-09-19T04:50:52.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.273+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/3006440270 wait complete. 2024-09-19T04:50:52.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.273+0000 7fcf5fbbc740 20 client.26143 trim_cache size 0 max 16384 2024-09-19T04:50:52.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.291+0000 7fcf5fbbc740 1 Processor -- start 2024-09-19T04:50:52.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.291+0000 7fcf5fbbc740 1 -- start start 2024-09-19T04:50:52.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.292+0000 7fcf5fbbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e8ac0 0x55559e5f2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.292+0000 7fcf5fbbc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e93e0 0x55559e607b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.292+0000 7fcf5fbbc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 0x55559e608040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.292+0000 7fcf5fbbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581ab670 con 0x55559e5e8ac0 2024-09-19T04:50:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.292+0000 7fcf5fbbc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581ad1b0 con 0x55559e5e93e0 2024-09-19T04:50:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.292+0000 7fcf5fbbc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581abd40 con 0x55559e5f3ca0 2024-09-19T04:50:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.292+0000 7fcf55e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e8ac0 0x55559e5f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.292+0000 7fcf55e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e8ac0 0x55559e5f2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46412/0 (socket says 172.21.15.123:46412) 2024-09-19T04:50:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.292+0000 7fcf55e00640 1 -- 172.21.15.123:0/2780987412 learned_addr learned my addr 172.21.15.123:0/2780987412 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.292+0000 7fcf4fe00640 1 --2- 172.21.15.123:0/2780987412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e93e0 0x55559e607b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.292+0000 7fcf56800640 1 --2- 172.21.15.123:0/2780987412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 0x55559e608040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.292+0000 7fcf55e00640 1 -- 172.21.15.123:0/2780987412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 msgr2=0x55559e608040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.292+0000 7fcf55e00640 1 --2- 172.21.15.123:0/2780987412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 0x55559e608040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.292+0000 7fcf55e00640 1 -- 172.21.15.123:0/2780987412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e93e0 msgr2=0x55559e607b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.292+0000 7fcf55e00640 1 --2- 172.21.15.123:0/2780987412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e93e0 0x55559e607b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.292+0000 7fcf55e00640 1 -- 172.21.15.123:0/2780987412 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55559e5f6400 con 0x55559e5e8ac0 2024-09-19T04:50:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.292+0000 7fcf56800640 1 --2- 172.21.15.123:0/2780987412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 0x55559e608040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:50:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.293+0000 7fcf55e00640 1 --2- 172.21.15.123:0/2780987412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e8ac0 0x55559e5f2830 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcf4809c0b0 tx=0x7fcf480097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968fb55f0f5f42a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.293+0000 7fcf46a00640 1 -- 172.21.15.123:0/2780987412 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4809bef0 con 0x55559e5e8ac0 2024-09-19T04:50:52.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.293+0000 7fcf46a00640 1 -- 172.21.15.123:0/2780987412 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf480d36e0 con 0x55559e5e8ac0 2024-09-19T04:50:52.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.293+0000 7fcf46a00640 1 -- 172.21.15.123:0/2780987412 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf480d3a80 con 0x55559e5e8ac0 2024-09-19T04:50:52.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.294+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/2780987412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e8ac0 msgr2=0x55559e5f2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:52.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.294+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/2780987412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e8ac0 0x55559e5f2830 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcf4809c0b0 tx=0x7fcf480097a0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.294+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/2780987412 shutdown_connections 2024-09-19T04:50:52.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.294+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/2780987412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5f3ca0 0x55559e608040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.294+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/2780987412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e93e0 0x55559e607b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.294+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/2780987412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5e8ac0 0x55559e5f2830 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.294+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/2780987412 >> 172.21.15.123:0/2780987412 conn(0x55559e5e5050 msgr2=0x55559e6073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:52.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.294+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/2780987412 shutdown_connections 2024-09-19T04:50:52.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.294+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/2780987412 wait complete. 2024-09-19T04:50:52.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.294+0000 7fcf5fbbc740 1 -- start start 2024-09-19T04:50:52.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.295+0000 7fcf5fbbc740 1 Processor -- start 2024-09-19T04:50:52.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.295+0000 7fcf5fbbc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5e8ac0 0x55559e602af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.295+0000 7fcf5fbbc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e93e0 0x55559e603030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.295+0000 7fcf5fbbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5f3ca0 0x55559e676280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.295+0000 7fcf5fbbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581acae0 con 0x55559e5f3ca0 2024-09-19T04:50:52.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.295+0000 7fcf5fbbc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581ac410 con 0x55559e5e93e0 2024-09-19T04:50:52.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.295+0000 7fcf5fbbc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581a9650 con 0x55559e5e8ac0 2024-09-19T04:50:52.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.295+0000 7fcf5fbbc740 10 client.0 _ll_register_callbacks cb 0x7fcf48005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:52.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.295+0000 7fcf55e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5e8ac0 0x55559e602af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.295+0000 7fcf56800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5f3ca0 0x55559e676280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.295+0000 7fcf55e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5e8ac0 0x55559e602af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58354/0 (socket says 172.21.15.123:58354) 2024-09-19T04:50:52.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.295+0000 7fcf55e00640 1 -- 172.21.15.123:0/367125967 learned_addr learned my addr 172.21.15.123:0/367125967 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:52.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.296+0000 7fcf4fe00640 1 --2- 172.21.15.123:0/367125967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e93e0 0x55559e603030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.296+0000 7fcf55e00640 1 -- 172.21.15.123:0/367125967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e93e0 msgr2=0x55559e603030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:52.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.296+0000 7fcf55e00640 1 --2- 172.21.15.123:0/367125967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e93e0 0x55559e603030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.296+0000 7fcf55e00640 1 -- 172.21.15.123:0/367125967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5f3ca0 msgr2=0x55559e676280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:52.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.296+0000 7fcf55e00640 1 --2- 172.21.15.123:0/367125967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5f3ca0 0x55559e676280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.296+0000 7fcf55e00640 1 -- 172.21.15.123:0/367125967 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55559e6768c0 con 0x55559e5e8ac0 2024-09-19T04:50:52.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.296+0000 7fcf55e00640 1 --2- 172.21.15.123:0/367125967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5e8ac0 0x55559e602af0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcf48000c00 tx=0x7fcf48009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc227c8947b5ce8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:52.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.296+0000 7fcf3be00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:50:52.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.296+0000 7fcf3be00640 1 -- 172.21.15.123:0/367125967 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf480b5060 con 0x55559e5e8ac0 2024-09-19T04:50:52.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.296+0000 7fcf3be00640 1 -- 172.21.15.123:0/367125967 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fcf48098ae0 con 0x55559e5e8ac0 2024-09-19T04:50:52.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.296+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/367125967 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55559e4d2fb0 con 0x55559e5e8ac0 2024-09-19T04:50:52.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.297+0000 7fcf3be00640 1 -- 172.21.15.123:0/367125967 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcf4809c3b0 con 0x55559e5e8ac0 2024-09-19T04:50:52.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.297+0000 7fcf5fbbc740 10 client.26146 fetch_fsmap learned FSMap version 36 2024-09-19T04:50:52.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.297+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/367125967 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55559e677b30 con 0x55559e5e8ac0 2024-09-19T04:50:52.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.297+0000 7fcf3be00640 1 -- 172.21.15.123:0/367125967 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55559e4d2fb0 con 0x55559e5e8ac0 2024-09-19T04:50:52.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.297+0000 7fcf3be00640 1 -- 172.21.15.123:0/367125967 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fcf4809c610 con 0x55559e5e8ac0 2024-09-19T04:50:52.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.297+0000 7fcf5fbbc740 10 client.26146 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:50:52.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.297+0000 7fcf5fbbc740 10 client.26146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:50:52.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.297+0000 7fcf5fbbc740 20 client.26146 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:52.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.297+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/367125967 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55559e677db0 0x55559e67a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:52.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.297+0000 7fcf5fbbc740 4 client.26146 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:50:52.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.297+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/367125967 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55559e520950 con 0x55559e677db0 2024-09-19T04:50:52.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.298+0000 7fcf4fe00640 1 --2- 172.21.15.123:0/367125967 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55559e677db0 0x55559e67a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:52.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.298+0000 7fcf4fe00640 1 --2- 172.21.15.123:0/367125967 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55559e677db0 0x55559e67a1a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35dbbc4d82e0d172 server_cookie=ccafba7397386255 in_seq=0 out_seq=0 2024-09-19T04:50:52.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.298+0000 7fcf3be00640 10 client.26146 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:52.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.299+0000 7fcf3be00640 1 -- 172.21.15.123:0/367125967 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55559e520950 con 0x55559e677db0 2024-09-19T04:50:52.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.299+0000 7fcf3be00640 10 client.26146 handle_command_reply: tid=0 2024-09-19T04:50:52.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.299+0000 7fcf5fbbc740 1 client.26146 shutdown 2024-09-19T04:50:52.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.300+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/367125967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5e8ac0 msgr2=0x55559e602af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:52.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.300+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/367125967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5e8ac0 0x55559e602af0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcf48000c00 tx=0x7fcf48009700 comp rx=0 tx=0).stop 2024-09-19T04:50:52.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.300+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/367125967 shutdown_connections 2024-09-19T04:50:52.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.300+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/367125967 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55559e677db0 0x55559e67a1a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.300+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/367125967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55559e5f3ca0 0x55559e676280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.300+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/367125967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55559e5e93e0 0x55559e603030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.300+0000 7fcf3be00640 0 client.26146 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:52.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.300+0000 7fcf5fbbc740 1 --2- 172.21.15.123:0/367125967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55559e5e8ac0 0x55559e602af0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.300+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/367125967 >> 172.21.15.123:0/367125967 conn(0x55559e5e5050 msgr2=0x55559e607c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:52.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.300+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/367125967 shutdown_connections 2024-09-19T04:50:52.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.300+0000 7fcf5fbbc740 1 -- 172.21.15.123:0/367125967 wait complete. 2024-09-19T04:50:52.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.300+0000 7fcf5fbbc740 20 client.26146 trim_cache size 0 max 16384 2024-09-19T04:50:52.307 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:50:52.307 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-09-19T04:50:52.307 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "028c6058-35c8-40cb-a13c-9552d5e5555b", 2024-09-19T04:50:52.307 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-09-19T04:50:52.308 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:50:52.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.301+0000 7fcf5d000640 1 -- 172.21.15.123:0/587339060 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fcf3c0a3430 msgr2=0x7fcf3c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:52.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.301+0000 7fcf5d000640 1 --2- 172.21.15.123:0/587339060 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fcf3c0a3430 0x7fcf3c0a58f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009fd0 tx=0x7fcf40009260 comp rx=0 tx=0).stop 2024-09-19T04:50:52.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.301+0000 7fcf5d000640 1 -- 172.21.15.123:0/587339060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf58074510 msgr2=0x7fcf5819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.301+0000 7fcf5d000640 1 --2- 172.21.15.123:0/587339060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf58074510 0x7fcf5819a620 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800e9f0 tx=0x7fcf4800edf0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.301+0000 7fcf5d000640 1 -- 172.21.15.123:0/587339060 shutdown_connections 2024-09-19T04:50:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.301+0000 7fcf5d000640 1 --2- 172.21.15.123:0/587339060 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fcf3c0a3430 0x7fcf3c0a58f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.301+0000 7fcf5d000640 1 --2- 172.21.15.123:0/587339060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf5810e440 0x7fcf5819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.301+0000 7fcf5d000640 1 --2- 172.21.15.123:0/587339060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf58074e30 0x7fcf5819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.301+0000 7fcf5d000640 1 --2- 172.21.15.123:0/587339060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf58074510 0x7fcf5819a620 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.301+0000 7fcf5d000640 1 -- 172.21.15.123:0/587339060 >> 172.21.15.123:0/587339060 conn(0x7fcf580fd8c0 msgr2=0x7fcf580fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.301+0000 7fcf5d000640 1 -- 172.21.15.123:0/587339060 shutdown_connections 2024-09-19T04:50:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:52.301+0000 7fcf5d000640 1 -- 172.21.15.123:0/587339060 wait complete. 2024-09-19T04:50:53.149 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-19T04:50:53.149 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-19T04:50:53.151 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-19T04:50:53.151 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-19T04:50:53.151 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support clone range, disabling! 2024-09-19T04:50:53.154 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-19T04:50:53.156 INFO:tasks.workunit.client.0.smithi123.stdout:skipping zero size read 2024-09-19T04:50:53.157 INFO:tasks.workunit.client.0.smithi123.stderr:main: filesystem does not support exchange range, disabling! 2024-09-19T04:50:53.172 INFO:tasks.workunit.client.0.smithi123.stdout:truncating to largest ever: 0x1c37b90 2024-09-19T04:50:53.180 INFO:tasks.workunit.client.0.smithi123.stdout:truncating to largest ever: 0x5bd330d 2024-09-19T04:50:53.200 INFO:tasks.workunit.client.0.smithi123.stdout:copying to largest ever: 0x637bf99 2024-09-19T04:50:53.324 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:53 smithi123 ceph-mon[30511]: pgmap v128: 97 pgs: 97 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.5 MiB/s wr, 430 op/s 2024-09-19T04:50:53.324 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:53 smithi123 ceph-mon[30511]: scrub queued for path: / 2024-09-19T04:50:53.324 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:53 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:53.324 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:53 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:50:53.324 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:53 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:50:53.324 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:53 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:50:53.324 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:53 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:53.325 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:53 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:53.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:53 smithi183 ceph-mon[35333]: pgmap v128: 97 pgs: 97 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.5 MiB/s wr, 430 op/s 2024-09-19T04:50:53.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:53 smithi183 ceph-mon[35333]: scrub queued for path: / 2024-09-19T04:50:53.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:53 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:53.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:53 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:50:53.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:53 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:50:53.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:53 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:50:53.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:53 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:53.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:53 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:53.365 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:50:53.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:53 smithi193 ceph-mon[35592]: pgmap v128: 97 pgs: 97 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.5 MiB/s wr, 430 op/s 2024-09-19T04:50:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:53 smithi193 ceph-mon[35592]: scrub queued for path: / 2024-09-19T04:50:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:53 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:53 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:50:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:53 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:50:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:53 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:50:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:53 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:53.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:53 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:53.502 INFO:tasks.workunit.client.0.smithi123.stdout:copying to largest ever: 0x63d9b9c 2024-09-19T04:50:53.776 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:50:54.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:54 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:54.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:54 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:50:54.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:54 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:54.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:54 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:50:54.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:54 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:54.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:54 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:54.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:54 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:50:54.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:54 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:54.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:54 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:50:54.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:54 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:54.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.388+0000 7f1a85e00640 1 -- 172.21.15.123:0/3781114374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a80074ee0 msgr2=0x7f1a8010def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:54.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.388+0000 7f1a85e00640 1 --2- 172.21.15.123:0/3781114374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a80074ee0 0x7f1a8010def0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c009900 tx=0x7f1a6c02a480 comp rx=0 tx=0).stop 2024-09-19T04:50:54.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.389+0000 7f1a85e00640 1 -- 172.21.15.123:0/3781114374 shutdown_connections 2024-09-19T04:50:54.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.389+0000 7f1a85e00640 1 --2- 172.21.15.123:0/3781114374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a8010e430 0x7f1a80110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.389+0000 7f1a85e00640 1 --2- 172.21.15.123:0/3781114374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a80074ee0 0x7f1a8010def0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.389+0000 7f1a85e00640 1 --2- 172.21.15.123:0/3781114374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a800745c0 0x7f1a800749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.389+0000 7f1a85e00640 1 -- 172.21.15.123:0/3781114374 >> 172.21.15.123:0/3781114374 conn(0x7f1a800fd8b0 msgr2=0x7f1a800ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:54.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.389+0000 7f1a85e00640 1 -- 172.21.15.123:0/3781114374 shutdown_connections 2024-09-19T04:50:54.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.389+0000 7f1a85e00640 1 -- 172.21.15.123:0/3781114374 wait complete. 2024-09-19T04:50:54.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.390+0000 7f1a85e00640 1 Processor -- start 2024-09-19T04:50:54.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.390+0000 7f1a85e00640 1 -- start start 2024-09-19T04:50:54.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.391+0000 7f1a85e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a800745c0 0x7f1a8019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.391+0000 7f1a85e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a80074ee0 0x7f1a8019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.391+0000 7f1a85e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a8010e430 0x7f1a8019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.391+0000 7f1a85e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1a80114170 con 0x7f1a800745c0 2024-09-19T04:50:54.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.391+0000 7f1a85e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a80113ff0 con 0x7f1a80074ee0 2024-09-19T04:50:54.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.391+0000 7f1a85e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801142f0 con 0x7f1a8010e430 2024-09-19T04:50:54.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.391+0000 7f1a7e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a80074ee0 0x7f1a8019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:54.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.391+0000 7f1a7f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a8010e430 0x7f1a8019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:54.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.391+0000 7f1a7e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a80074ee0 0x7f1a8019ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:52700/0 (socket says 172.21.15.123:52700) 2024-09-19T04:50:54.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.391+0000 7f1a7ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a800745c0 0x7f1a8019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:54.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.391+0000 7f1a7e000640 1 -- 172.21.15.123:0/1668856565 learned_addr learned my addr 172.21.15.123:0/1668856565 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:54.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.392+0000 7f1a7f400640 1 -- 172.21.15.123:0/1668856565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a80074ee0 msgr2=0x7f1a8019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:54.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.392+0000 7f1a7f400640 1 --2- 172.21.15.123:0/1668856565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a80074ee0 0x7f1a8019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.392+0000 7f1a7f400640 1 -- 172.21.15.123:0/1668856565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a800745c0 msgr2=0x7f1a8019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:54.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.392+0000 7f1a7f400640 1 --2- 172.21.15.123:0/1668856565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a800745c0 0x7f1a8019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.392+0000 7f1a7f400640 1 -- 172.21.15.123:0/1668856565 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8019f680 con 0x7f1a8010e430 2024-09-19T04:50:54.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.392+0000 7f1a7e000640 1 --2- 172.21.15.123:0/1668856565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a80074ee0 0x7f1a8019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:54.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.392+0000 7f1a7f400640 1 --2- 172.21.15.123:0/1668856565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a8010e430 0x7f1a8019efa0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1a78005fd0 tx=0x7f1a7800edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:54.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.392+0000 7f1a84a00640 1 -- 172.21.15.123:0/1668856565 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78004d20 con 0x7f1a8010e430 2024-09-19T04:50:54.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.392+0000 7f1a85e00640 1 -- 172.21.15.123:0/1668856565 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a8019f970 con 0x7f1a8010e430 2024-09-19T04:50:54.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.392+0000 7f1a84a00640 1 -- 172.21.15.123:0/1668856565 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a78005020 con 0x7f1a8010e430 2024-09-19T04:50:54.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.392+0000 7f1a84a00640 1 -- 172.21.15.123:0/1668856565 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a780116d0 con 0x7f1a8010e430 2024-09-19T04:50:54.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.392+0000 7f1a85e00640 1 -- 172.21.15.123:0/1668856565 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a801a7250 con 0x7f1a8010e430 2024-09-19T04:50:54.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.395+0000 7f1a84a00640 1 -- 172.21.15.123:0/1668856565 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f1a78011870 con 0x7f1a8010e430 2024-09-19T04:50:54.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.396+0000 7f1a88a57740 1 Processor -- start 2024-09-19T04:50:54.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.396+0000 7f1a88a57740 1 -- start start 2024-09-19T04:50:54.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.397+0000 7f1a88a57740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 0x55b234021f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.397+0000 7f1a88a57740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2340224b0 0x55b23402c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.397+0000 7f1a88a57740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23402cd70 0x55b23402f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.397+0000 7f1a88a57740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b233f883e0 con 0x55b234021b90 2024-09-19T04:50:54.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.397+0000 7f1a88a57740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b233f2fd80 con 0x55b23402cd70 2024-09-19T04:50:54.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.397+0000 7f1a88a57740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b233f843a0 con 0x55b2340224b0 2024-09-19T04:50:54.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.397+0000 7f1a7ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2340224b0 0x55b23402c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:54.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.397+0000 7f1a7ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2340224b0 0x55b23402c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58386/0 (socket says 172.21.15.123:58386) 2024-09-19T04:50:54.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.397+0000 7f1a7ea00640 1 -- 172.21.15.123:0/1055548369 learned_addr learned my addr 172.21.15.123:0/1055548369 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:54.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.397+0000 7f1a7e000640 1 --2- 172.21.15.123:0/1055548369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 0x55b234021f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:54.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.397+0000 7f1a84a00640 1 --2- 172.21.15.123:0/1668856565 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1a640a33b0 0x7f1a640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.397+0000 7f1a84a00640 1 -- 172.21.15.123:0/1668856565 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f1a780cfd00 con 0x7f1a8010e430 2024-09-19T04:50:54.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.398+0000 7f1a7ea00640 1 -- 172.21.15.123:0/1055548369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23402cd70 msgr2=0x55b23402f160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:50:54.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.398+0000 7f1a7ea00640 1 --2- 172.21.15.123:0/1055548369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23402cd70 0x55b23402f160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.398+0000 7f1a7ea00640 1 -- 172.21.15.123:0/1055548369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 msgr2=0x55b234021f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:54.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.398+0000 7f1a7ea00640 1 --2- 172.21.15.123:0/1055548369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 0x55b234021f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.398+0000 7f1a7ea00640 1 -- 172.21.15.123:0/1055548369 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b233f4eae0 con 0x55b2340224b0 2024-09-19T04:50:54.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.398+0000 7f1a7e000640 1 --2- 172.21.15.123:0/1055548369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 0x55b234021f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:54.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.398+0000 7f1a7ea00640 1 --2- 172.21.15.123:0/1055548369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2340224b0 0x55b23402c830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009cf0 tx=0x7f1a70011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cf5d9158d281641 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:54.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.398+0000 7f1a7f400640 1 --2- 172.21.15.123:0/1668856565 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1a640a33b0 0x7f1a640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:54.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.398+0000 7f1a6be00640 1 -- 172.21.15.123:0/1055548369 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70004230 con 0x55b2340224b0 2024-09-19T04:50:54.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.398+0000 7f1a6be00640 1 -- 172.21.15.123:0/1055548369 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a700043d0 con 0x55b2340224b0 2024-09-19T04:50:54.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.398+0000 7f1a6be00640 1 -- 172.21.15.123:0/1055548369 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70019920 con 0x55b2340224b0 2024-09-19T04:50:54.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.399+0000 7f1a88a57740 1 -- 172.21.15.123:0/1055548369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2340224b0 msgr2=0x55b23402c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:54.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.399+0000 7f1a88a57740 1 --2- 172.21.15.123:0/1055548369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2340224b0 0x55b23402c830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009cf0 tx=0x7f1a70011810 comp rx=0 tx=0).stop 2024-09-19T04:50:54.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.399+0000 7f1a7f400640 1 --2- 172.21.15.123:0/1668856565 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1a640a33b0 0x7f1a640a5870 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1a780a46f0 tx=0x7f1a78005b20 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:54.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.399+0000 7f1a88a57740 1 -- 172.21.15.123:0/1055548369 shutdown_connections 2024-09-19T04:50:54.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.399+0000 7f1a88a57740 1 --2- 172.21.15.123:0/1055548369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23402cd70 0x55b23402f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.399+0000 7f1a88a57740 1 --2- 172.21.15.123:0/1055548369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2340224b0 0x55b23402c830 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.399+0000 7f1a88a57740 1 --2- 172.21.15.123:0/1055548369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 0x55b234021f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.399+0000 7f1a88a57740 1 -- 172.21.15.123:0/1055548369 >> 172.21.15.123:0/1055548369 conn(0x55b23401e120 msgr2=0x55b23401e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:54.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.399+0000 7f1a88a57740 1 -- 172.21.15.123:0/1055548369 shutdown_connections 2024-09-19T04:50:54.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.399+0000 7f1a88a57740 1 -- 172.21.15.123:0/1055548369 wait complete. 2024-09-19T04:50:54.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.400+0000 7f1a88a57740 1 -- start start 2024-09-19T04:50:54.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.400+0000 7f1a88a57740 1 Processor -- start 2024-09-19T04:50:54.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.401+0000 7f1a88a57740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 0x55b234029d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.401+0000 7f1a88a57740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2340224b0 0x55b23402a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.401+0000 7f1a88a57740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b23402cd70 0x55b23402a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.401+0000 7f1a88a57740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b23402f9e0 con 0x55b234021b90 2024-09-19T04:50:54.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.401+0000 7f1a88a57740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b233f430f0 con 0x55b2340224b0 2024-09-19T04:50:54.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.401+0000 7f1a88a57740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b23402fb60 con 0x55b23402cd70 2024-09-19T04:50:54.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.401+0000 7f1a88a57740 10 client.0 _ll_register_callbacks cb 0x55b233f349e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:54.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.401+0000 7f1a7e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 0x55b234029d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:54.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.401+0000 7f1a7e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 0x55b234029d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46458/0 (socket says 172.21.15.123:46458) 2024-09-19T04:50:54.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.401+0000 7f1a7e000640 1 -- 172.21.15.123:0/298560964 learned_addr learned my addr 172.21.15.123:0/298560964 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:54.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.402+0000 7f1a7ea00640 1 --2- 172.21.15.123:0/298560964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2340224b0 0x55b23402a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:54.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.402+0000 7f1a7e000640 1 --2- 172.21.15.123:0/298560964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b23402cd70 0x55b23402a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:54.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.402+0000 7f1a7e000640 1 -- 172.21.15.123:0/298560964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b23402cd70 msgr2=0x55b23402a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:54.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.402+0000 7f1a7e000640 1 --2- 172.21.15.123:0/298560964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b23402cd70 0x55b23402a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.402+0000 7f1a7e000640 1 -- 172.21.15.123:0/298560964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2340224b0 msgr2=0x55b23402a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:54.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.402+0000 7f1a7e000640 1 --2- 172.21.15.123:0/298560964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2340224b0 0x55b23402a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.402+0000 7f1a7e000640 1 -- 172.21.15.123:0/298560964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b233cf93c0 con 0x55b234021b90 2024-09-19T04:50:54.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.402+0000 7f1a7ea00640 1 --2- 172.21.15.123:0/298560964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2340224b0 0x55b23402a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:50:54.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.402+0000 7f1a7e000640 1 --2- 172.21.15.123:0/298560964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 0x55b234029d00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c006010 tx=0x7f1a6c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426fa82d5756eec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:54.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.402+0000 7f1a68c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:50:54.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.402+0000 7f1a68c00640 1 -- 172.21.15.123:0/298560964 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c035040 con 0x55b234021b90 2024-09-19T04:50:54.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.402+0000 7f1a68c00640 1 -- 172.21.15.123:0/298560964 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f1a6c005030 con 0x55b234021b90 2024-09-19T04:50:54.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.403+0000 7f1a88a57740 1 -- 172.21.15.123:0/298560964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a700043d0 con 0x55b234021b90 2024-09-19T04:50:54.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.403+0000 7f1a68c00640 1 -- 172.21.15.123:0/298560964 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1a6c005230 con 0x55b234021b90 2024-09-19T04:50:54.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.403+0000 7f1a88a57740 10 client.16056 fetch_fsmap learned FSMap version 36 2024-09-19T04:50:54.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.403+0000 7f1a88a57740 1 -- 172.21.15.123:0/298560964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b23403b730 con 0x55b234021b90 2024-09-19T04:50:54.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.403+0000 7f1a68c00640 1 -- 172.21.15.123:0/298560964 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a700043d0 con 0x55b234021b90 2024-09-19T04:50:54.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.404+0000 7f1a68c00640 1 -- 172.21.15.123:0/298560964 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f1a6c049060 con 0x55b234021b90 2024-09-19T04:50:54.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.404+0000 7f1a88a57740 10 client.16056 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:50:54.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.404+0000 7f1a88a57740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:50:54.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.404+0000 7f1a88a57740 20 client.16056 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:54.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.404+0000 7f1a88a57740 1 --2- 172.21.15.123:0/298560964 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b23403cbf0 0x55b23403f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.404+0000 7f1a88a57740 4 client.16056 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:50:54.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.404+0000 7f1a88a57740 1 -- 172.21.15.123:0/298560964 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b233f95950 con 0x55b23403cbf0 2024-09-19T04:50:54.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.404+0000 7f1a7ea00640 1 --2- 172.21.15.123:0/298560964 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b23403cbf0 0x55b23403f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:54.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.405+0000 7f1a7ea00640 1 --2- 172.21.15.123:0/298560964 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b23403cbf0 0x55b23403f020 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4342121a4086dce server_cookie=290b09a2831f9ddf in_seq=0 out_seq=0 2024-09-19T04:50:54.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.405+0000 7f1a68c00640 10 client.16056 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:54.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.406+0000 7f1a68c00640 1 -- 172.21.15.123:0/298560964 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b233f95950 con 0x55b23403cbf0 2024-09-19T04:50:54.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.406+0000 7f1a68c00640 10 client.16056 handle_command_reply: tid=0 2024-09-19T04:50:54.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.406+0000 7f1a88a57740 1 client.16056 shutdown 2024-09-19T04:50:54.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.406+0000 7f1a88a57740 1 -- 172.21.15.123:0/298560964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 msgr2=0x55b234029d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:54.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.406+0000 7f1a88a57740 1 --2- 172.21.15.123:0/298560964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 0x55b234029d00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c006010 tx=0x7f1a6c009680 comp rx=0 tx=0).stop 2024-09-19T04:50:54.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.406+0000 7f1a88a57740 1 -- 172.21.15.123:0/298560964 shutdown_connections 2024-09-19T04:50:54.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.406+0000 7f1a88a57740 1 --2- 172.21.15.123:0/298560964 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b23403cbf0 0x55b23403f020 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.406+0000 7f1a88a57740 1 --2- 172.21.15.123:0/298560964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b23402cd70 0x55b23402a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.406+0000 7f1a88a57740 1 --2- 172.21.15.123:0/298560964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2340224b0 0x55b23402a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.406+0000 7f1a68c00640 0 client.16056 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:54.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.406+0000 7f1a88a57740 1 --2- 172.21.15.123:0/298560964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 0x55b234029d00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.406+0000 7f1a88a57740 1 -- 172.21.15.123:0/298560964 >> 172.21.15.123:0/298560964 conn(0x55b23401e120 msgr2=0x55b234023e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:54.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.407+0000 7f1a88a57740 1 -- 172.21.15.123:0/298560964 shutdown_connections 2024-09-19T04:50:54.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.408+0000 7f1a88a57740 1 -- 172.21.15.123:0/298560964 wait complete. 2024-09-19T04:50:54.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.408+0000 7f1a88a57740 20 client.16056 trim_cache size 0 max 16384 2024-09-19T04:50:54.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.430+0000 7f1a88a57740 1 Processor -- start 2024-09-19T04:50:54.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.431+0000 7f1a88a57740 1 -- start start 2024-09-19T04:50:54.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.431+0000 7f1a88a57740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 0x55b234029620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.431+0000 7f1a88a57740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2340224b0 0x55b234044890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.431+0000 7f1a88a57740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b23402cd70 0x55b234044dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.431+0000 7f1a88a57740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801afaa0 con 0x55b234021b90 2024-09-19T04:50:54.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.431+0000 7f1a88a57740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801b15e0 con 0x55b2340224b0 2024-09-19T04:50:54.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.431+0000 7f1a88a57740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801b0170 con 0x55b23402cd70 2024-09-19T04:50:54.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.431+0000 7f1a7ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2340224b0 0x55b234044890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:54.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.431+0000 7f1a7e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 0x55b234029620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:54.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.431+0000 7f1a7ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2340224b0 0x55b234044890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:52740/0 (socket says 172.21.15.123:52740) 2024-09-19T04:50:54.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.431+0000 7f1a7ea00640 1 -- 172.21.15.123:0/3430888214 learned_addr learned my addr 172.21.15.123:0/3430888214 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:54.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.432+0000 7f1a7e000640 1 --2- 172.21.15.123:0/3430888214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b23402cd70 0x55b234044dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:54.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.432+0000 7f1a7ea00640 1 -- 172.21.15.123:0/3430888214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b23402cd70 msgr2=0x55b234044dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:54.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.432+0000 7f1a7ea00640 1 --2- 172.21.15.123:0/3430888214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b23402cd70 0x55b234044dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.432+0000 7f1a7ea00640 1 -- 172.21.15.123:0/3430888214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 msgr2=0x55b234029620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:54.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.432+0000 7f1a7ea00640 1 --2- 172.21.15.123:0/3430888214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 0x55b234029620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.432+0000 7f1a7ea00640 1 -- 172.21.15.123:0/3430888214 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b23401d950 con 0x55b2340224b0 2024-09-19T04:50:54.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.432+0000 7f1a7ea00640 1 --2- 172.21.15.123:0/3430888214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2340224b0 0x55b234044890 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1a70005fd0 tx=0x7f1a70011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31c906a5c283945a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:54.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.432+0000 7f1a6b400640 1 -- 172.21.15.123:0/3430888214 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a700052d0 con 0x55b2340224b0 2024-09-19T04:50:54.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.432+0000 7f1a6b400640 1 -- 172.21.15.123:0/3430888214 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a70005470 con 0x55b2340224b0 2024-09-19T04:50:54.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.432+0000 7f1a6b400640 1 -- 172.21.15.123:0/3430888214 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70019790 con 0x55b2340224b0 2024-09-19T04:50:54.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.433+0000 7f1a88a57740 1 -- 172.21.15.123:0/3430888214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2340224b0 msgr2=0x55b234044890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:54.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.433+0000 7f1a88a57740 1 --2- 172.21.15.123:0/3430888214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2340224b0 0x55b234044890 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1a70005fd0 tx=0x7f1a70011810 comp rx=0 tx=0).stop 2024-09-19T04:50:54.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.433+0000 7f1a88a57740 1 -- 172.21.15.123:0/3430888214 shutdown_connections 2024-09-19T04:50:54.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.433+0000 7f1a88a57740 1 --2- 172.21.15.123:0/3430888214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b23402cd70 0x55b234044dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.433+0000 7f1a88a57740 1 --2- 172.21.15.123:0/3430888214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2340224b0 0x55b234044890 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.433+0000 7f1a88a57740 1 --2- 172.21.15.123:0/3430888214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b234021b90 0x55b234029620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.433+0000 7f1a88a57740 1 -- 172.21.15.123:0/3430888214 >> 172.21.15.123:0/3430888214 conn(0x55b23401e120 msgr2=0x55b23401d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:54.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.433+0000 7f1a88a57740 1 -- 172.21.15.123:0/3430888214 shutdown_connections 2024-09-19T04:50:54.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.433+0000 7f1a88a57740 1 -- 172.21.15.123:0/3430888214 wait complete. 2024-09-19T04:50:54.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.434+0000 7f1a88a57740 1 -- start start 2024-09-19T04:50:54.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.434+0000 7f1a88a57740 1 Processor -- start 2024-09-19T04:50:54.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a88a57740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b234021b90 0x55b234035160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a88a57740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2340224b0 0x55b2340356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a88a57740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b23402cd70 0x55b234031f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a88a57740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801b0f10 con 0x55b2340224b0 2024-09-19T04:50:54.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a88a57740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801b0840 con 0x55b234021b90 2024-09-19T04:50:54.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a88a57740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801ada80 con 0x55b23402cd70 2024-09-19T04:50:54.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a88a57740 10 client.0 _ll_register_callbacks cb 0x7f1a6c0317f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:50:54.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a7ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2340224b0 0x55b2340356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:54.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a7e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b234021b90 0x55b234035160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:54.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a7e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b234021b90 0x55b234035160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:52742/0 (socket says 172.21.15.123:52742) 2024-09-19T04:50:54.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a7e000640 1 -- 172.21.15.123:0/3814226449 learned_addr learned my addr 172.21.15.123:0/3814226449 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:50:54.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a7e000640 1 --2- 172.21.15.123:0/3814226449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b23402cd70 0x55b234031f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:54.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a7ea00640 1 -- 172.21.15.123:0/3814226449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b23402cd70 msgr2=0x55b234031f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:54.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a7ea00640 1 --2- 172.21.15.123:0/3814226449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b23402cd70 0x55b234031f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a7ea00640 1 -- 172.21.15.123:0/3814226449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b234021b90 msgr2=0x55b234035160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:50:54.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a7ea00640 1 --2- 172.21.15.123:0/3814226449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b234021b90 0x55b234035160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a7ea00640 1 -- 172.21.15.123:0/3814226449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b234025960 con 0x55b2340224b0 2024-09-19T04:50:54.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.435+0000 7f1a7e000640 1 --2- 172.21.15.123:0/3814226449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b234021b90 0x55b234035160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:50:54.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.436+0000 7f1a7ea00640 1 --2- 172.21.15.123:0/3814226449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2340224b0 0x55b2340356a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1a7001b040 tx=0x7f1a70005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65df9eb060d99f9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:50:54.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.436+0000 7f1a68c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:50:54.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.436+0000 7f1a68c00640 1 -- 172.21.15.123:0/3814226449 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70005290 con 0x55b2340224b0 2024-09-19T04:50:54.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.436+0000 7f1a68c00640 1 -- 172.21.15.123:0/3814226449 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f1a700035f0 con 0x55b2340224b0 2024-09-19T04:50:54.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.436+0000 7f1a88a57740 1 -- 172.21.15.123:0/3814226449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a70005470 con 0x55b2340224b0 2024-09-19T04:50:54.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.436+0000 7f1a68c00640 1 -- 172.21.15.123:0/3814226449 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1a700037f0 con 0x55b2340224b0 2024-09-19T04:50:54.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.436+0000 7f1a88a57740 10 client.16062 fetch_fsmap learned FSMap version 36 2024-09-19T04:50:54.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.436+0000 7f1a88a57740 1 -- 172.21.15.123:0/3814226449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b2340a83a0 con 0x55b2340224b0 2024-09-19T04:50:54.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.437+0000 7f1a68c00640 1 -- 172.21.15.123:0/3814226449 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a70005470 con 0x55b2340224b0 2024-09-19T04:50:54.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.437+0000 7f1a68c00640 1 -- 172.21.15.123:0/3814226449 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f1a70019c80 con 0x55b2340224b0 2024-09-19T04:50:54.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.437+0000 7f1a88a57740 10 client.16062 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:50:54.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.437+0000 7f1a88a57740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:50:54.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.437+0000 7f1a88a57740 20 client.16062 populate_metadata read hostname 'smithi123' 2024-09-19T04:50:54.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.437+0000 7f1a88a57740 1 --2- 172.21.15.123:0/3814226449 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b2340a88e0 0x55b2340aad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:50:54.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.437+0000 7f1a88a57740 4 client.16062 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:50:54.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.437+0000 7f1a88a57740 1 -- 172.21.15.123:0/3814226449 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b233f8fe60 con 0x55b2340a88e0 2024-09-19T04:50:54.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.438+0000 7f1a7e000640 1 --2- 172.21.15.123:0/3814226449 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b2340a88e0 0x55b2340aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:50:54.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.439+0000 7f1a7e000640 1 --2- 172.21.15.123:0/3814226449 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b2340a88e0 0x55b2340aad00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31c3e9cda346743f server_cookie=c7b8b2003e7cd15a in_seq=0 out_seq=0 2024-09-19T04:50:54.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.439+0000 7f1a68c00640 10 client.16062 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:54.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.439+0000 7f1a68c00640 1 -- 172.21.15.123:0/3814226449 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b233f8fe60 con 0x55b2340a88e0 2024-09-19T04:50:54.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.440+0000 7f1a68c00640 10 client.16062 handle_command_reply: tid=0 2024-09-19T04:50:54.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.440+0000 7f1a88a57740 1 client.16062 shutdown 2024-09-19T04:50:54.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.440+0000 7f1a88a57740 1 -- 172.21.15.123:0/3814226449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2340224b0 msgr2=0x55b2340356a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:54.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.440+0000 7f1a88a57740 1 --2- 172.21.15.123:0/3814226449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2340224b0 0x55b2340356a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1a7001b040 tx=0x7f1a70005d30 comp rx=0 tx=0).stop 2024-09-19T04:50:54.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.440+0000 7f1a88a57740 1 -- 172.21.15.123:0/3814226449 shutdown_connections 2024-09-19T04:50:54.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.440+0000 7f1a88a57740 1 --2- 172.21.15.123:0/3814226449 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b2340a88e0 0x55b2340aad00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.440+0000 7f1a88a57740 1 --2- 172.21.15.123:0/3814226449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b23402cd70 0x55b234031f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.440+0000 7f1a68c00640 0 client.16062 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:50:54.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.440+0000 7f1a88a57740 1 --2- 172.21.15.123:0/3814226449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2340224b0 0x55b2340356a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.440+0000 7f1a88a57740 1 --2- 172.21.15.123:0/3814226449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b234021b90 0x55b234035160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.440+0000 7f1a88a57740 1 -- 172.21.15.123:0/3814226449 >> 172.21.15.123:0/3814226449 conn(0x55b23401e120 msgr2=0x55b2340235e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:54.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.440+0000 7f1a88a57740 1 -- 172.21.15.123:0/3814226449 shutdown_connections 2024-09-19T04:50:54.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.440+0000 7f1a88a57740 1 -- 172.21.15.123:0/3814226449 wait complete. 2024-09-19T04:50:54.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.441+0000 7f1a88a57740 20 client.16062 trim_cache size 0 max 16384 2024-09-19T04:50:54.445 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:50:54.445 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:50:54.445 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": { 2024-09-19T04:50:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "028c6058-35c8-40cb-a13c-9552d5e5555b": { 2024-09-19T04:50:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "path": "/", 2024-09-19T04:50:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "tag": "028c6058-35c8-40cb-a13c-9552d5e5555b", 2024-09-19T04:50:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "options": "recursive,force" 2024-09-19T04:50:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:50:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:50:54.446 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:50:54.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.441+0000 7f1a85e00640 1 -- 172.21.15.123:0/1668856565 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1a640a33b0 msgr2=0x7f1a640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:54.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.441+0000 7f1a85e00640 1 --2- 172.21.15.123:0/1668856565 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1a640a33b0 0x7f1a640a5870 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1a780a46f0 tx=0x7f1a78005b20 comp rx=0 tx=0).stop 2024-09-19T04:50:54.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.441+0000 7f1a85e00640 1 -- 172.21.15.123:0/1668856565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a8010e430 msgr2=0x7f1a8019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:50:54.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.441+0000 7f1a85e00640 1 --2- 172.21.15.123:0/1668856565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a8010e430 0x7f1a8019efa0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1a78005fd0 tx=0x7f1a7800edd0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.441+0000 7f1a85e00640 1 -- 172.21.15.123:0/1668856565 shutdown_connections 2024-09-19T04:50:54.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.441+0000 7f1a85e00640 1 --2- 172.21.15.123:0/1668856565 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1a640a33b0 0x7f1a640a5870 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.441+0000 7f1a85e00640 1 --2- 172.21.15.123:0/1668856565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a8010e430 0x7f1a8019efa0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.441+0000 7f1a85e00640 1 --2- 172.21.15.123:0/1668856565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a80074ee0 0x7f1a8019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.441+0000 7f1a85e00640 1 --2- 172.21.15.123:0/1668856565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a800745c0 0x7f1a8019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:50:54.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.441+0000 7f1a85e00640 1 -- 172.21.15.123:0/1668856565 >> 172.21.15.123:0/1668856565 conn(0x7f1a800fd8b0 msgr2=0x7f1a800fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:50:54.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.441+0000 7f1a85e00640 1 -- 172.21.15.123:0/1668856565 shutdown_connections 2024-09-19T04:50:54.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:50:54.442+0000 7f1a85e00640 1 -- 172.21.15.123:0/1668856565 wait complete. 2024-09-19T04:50:54.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:54 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:54.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:54 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:50:54.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:54 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:54.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:54 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:50:54.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:54 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:50:55.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:55 smithi183 ceph-mon[35333]: pgmap v129: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.5 MiB/s wr, 602 op/s 2024-09-19T04:50:55.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:55 smithi123 ceph-mon[30511]: pgmap v129: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.5 MiB/s wr, 602 op/s 2024-09-19T04:50:55.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:55 smithi193 ceph-mon[35592]: pgmap v129: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.5 MiB/s wr, 602 op/s 2024-09-19T04:50:56.061 INFO:tasks.cephfs.filesystem:scrub status for tag:028c6058-35c8-40cb-a13c-9552d5e5555b - {'path': '/', 'tag': '028c6058-35c8-40cb-a13c-9552d5e5555b', 'options': 'recursive,force'} 2024-09-19T04:50:57.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:57 smithi193 ceph-mon[35592]: pgmap v130: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.3 MiB/s wr, 384 op/s 2024-09-19T04:50:57.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:57 smithi183 ceph-mon[35333]: pgmap v130: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.3 MiB/s wr, 384 op/s 2024-09-19T04:50:57.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:57 smithi123 ceph-mon[30511]: pgmap v130: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.3 MiB/s wr, 384 op/s 2024-09-19T04:50:58.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:58 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:50:58.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:58 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:50:58.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:58 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:50:58.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:58 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:50:58.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:58 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:50:58.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:58 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:50:58.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:58 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:50:58.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:58 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:50:58.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:58 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:50:58.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:58 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:50:58.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:58 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:50:58.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:58 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:50:59.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:50:59 smithi183 ceph-mon[35333]: pgmap v131: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.9 MiB/s wr, 612 op/s 2024-09-19T04:50:59.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:50:59 smithi123 ceph-mon[30511]: pgmap v131: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.9 MiB/s wr, 612 op/s 2024-09-19T04:50:59.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:50:59 smithi193 ceph-mon[35592]: pgmap v131: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.9 MiB/s wr, 612 op/s 2024-09-19T04:51:01.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:01 smithi183 ceph-mon[35333]: pgmap v132: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.6 MiB/s wr, 400 op/s 2024-09-19T04:51:01.426 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:01 smithi123 ceph-mon[30511]: pgmap v132: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.6 MiB/s wr, 400 op/s 2024-09-19T04:51:01.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:01 smithi193 ceph-mon[35592]: pgmap v132: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.6 MiB/s wr, 400 op/s 2024-09-19T04:51:03.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:03 smithi183 ceph-mon[35333]: pgmap v133: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.6 MiB/s wr, 400 op/s 2024-09-19T04:51:03.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:03 smithi123 ceph-mon[30511]: pgmap v133: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.6 MiB/s wr, 400 op/s 2024-09-19T04:51:03.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:03 smithi193 ceph-mon[35592]: pgmap v133: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.6 MiB/s wr, 400 op/s 2024-09-19T04:51:05.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:05 smithi183 ceph-mon[35333]: pgmap v134: 97 pgs: 97 active+clean; 167 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.3 MiB/s wr, 706 op/s 2024-09-19T04:51:05.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:05 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:51:05.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:05 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:51:05.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:05 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:51:05.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:05 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:51:05.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:05 smithi123 ceph-mon[30511]: pgmap v134: 97 pgs: 97 active+clean; 167 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.3 MiB/s wr, 706 op/s 2024-09-19T04:51:05.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:05 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:51:05.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:05 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:51:05.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:05 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:51:05.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:05 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:51:05.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:05 smithi193 ceph-mon[35592]: pgmap v134: 97 pgs: 97 active+clean; 167 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.3 MiB/s wr, 706 op/s 2024-09-19T04:51:05.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:05 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:51:05.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:05 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:51:05.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:05 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:51:05.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:05 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:51:06.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:06 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:51:06.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:06 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:51:06.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:06 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:51:07.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:07 smithi183 ceph-mon[35333]: pgmap v135: 97 pgs: 97 active+clean; 167 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.4 MiB/s wr, 533 op/s 2024-09-19T04:51:07.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:07 smithi123 ceph-mon[30511]: pgmap v135: 97 pgs: 97 active+clean; 167 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.4 MiB/s wr, 533 op/s 2024-09-19T04:51:07.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:07 smithi193 ceph-mon[35592]: pgmap v135: 97 pgs: 97 active+clean; 167 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.4 MiB/s wr, 533 op/s 2024-09-19T04:51:09.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:09 smithi183 ceph-mon[35333]: pgmap v136: 97 pgs: 97 active+clean; 178 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.2 MiB/s wr, 815 op/s 2024-09-19T04:51:09.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:09 smithi123 ceph-mon[30511]: pgmap v136: 97 pgs: 97 active+clean; 178 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.2 MiB/s wr, 815 op/s 2024-09-19T04:51:09.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:09 smithi193 ceph-mon[35592]: pgmap v136: 97 pgs: 97 active+clean; 178 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.2 MiB/s wr, 815 op/s 2024-09-19T04:51:10.347 INFO:tasks.workunit.client.0.smithi123.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-09-19T04:51:11.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:11 smithi183 ceph-mon[35333]: pgmap v137: 97 pgs: 97 active+clean; 178 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.5 MiB/s wr, 587 op/s 2024-09-19T04:51:11.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:11 smithi123 ceph-mon[30511]: pgmap v137: 97 pgs: 97 active+clean; 178 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.5 MiB/s wr, 587 op/s 2024-09-19T04:51:11.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:11 smithi193 ceph-mon[35592]: pgmap v137: 97 pgs: 97 active+clean; 178 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.5 MiB/s wr, 587 op/s 2024-09-19T04:51:13.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:13 smithi183 ceph-mon[35333]: pgmap v138: 97 pgs: 97 active+clean; 178 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.5 MiB/s wr, 587 op/s 2024-09-19T04:51:13.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:13 smithi123 ceph-mon[30511]: pgmap v138: 97 pgs: 97 active+clean; 178 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.5 MiB/s wr, 587 op/s 2024-09-19T04:51:13.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:13 smithi193 ceph-mon[35592]: pgmap v138: 97 pgs: 97 active+clean; 178 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.5 MiB/s wr, 587 op/s 2024-09-19T04:51:15.320 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:15 smithi123 ceph-mon[30511]: pgmap v139: 97 pgs: 97 active+clean; 196 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.3 MiB/s wr, 869 op/s 2024-09-19T04:51:15.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:15 smithi183 ceph-mon[35333]: pgmap v139: 97 pgs: 97 active+clean; 196 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.3 MiB/s wr, 869 op/s 2024-09-19T04:51:15.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:15 smithi193 ceph-mon[35592]: pgmap v139: 97 pgs: 97 active+clean; 196 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.3 MiB/s wr, 869 op/s 2024-09-19T04:51:17.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:17 smithi183 ceph-mon[35333]: pgmap v140: 97 pgs: 97 active+clean; 196 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 563 op/s 2024-09-19T04:51:17.426 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:17 smithi123 ceph-mon[30511]: pgmap v140: 97 pgs: 97 active+clean; 196 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 563 op/s 2024-09-19T04:51:17.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:17 smithi193 ceph-mon[35592]: pgmap v140: 97 pgs: 97 active+clean; 196 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 563 op/s 2024-09-19T04:51:19.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:19 smithi183 ceph-mon[35333]: pgmap v141: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.3 MiB/s wr, 844 op/s 2024-09-19T04:51:19.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:19 smithi123 ceph-mon[30511]: pgmap v141: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.3 MiB/s wr, 844 op/s 2024-09-19T04:51:19.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:19 smithi193 ceph-mon[35592]: pgmap v141: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.3 MiB/s wr, 844 op/s 2024-09-19T04:51:20.684 INFO:tasks.workunit.client.0.smithi123.stdout:truncating to largest ever: 0x63e5d2a 2024-09-19T04:51:21.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:21 smithi183 ceph-mon[35333]: pgmap v142: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 562 op/s 2024-09-19T04:51:21.426 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:21 smithi123 ceph-mon[30511]: pgmap v142: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 562 op/s 2024-09-19T04:51:21.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:21 smithi193 ceph-mon[35592]: pgmap v142: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 562 op/s 2024-09-19T04:51:23.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:23 smithi183 ceph-mon[35333]: pgmap v143: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 562 op/s 2024-09-19T04:51:23.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:23 smithi123 ceph-mon[30511]: pgmap v143: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 562 op/s 2024-09-19T04:51:23.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:23 smithi193 ceph-mon[35592]: pgmap v143: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 562 op/s 2024-09-19T04:51:24.677 INFO:tasks.workunit.client.0.smithi123.stdout:truncating to largest ever: 0x63e9b52 2024-09-19T04:51:25.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:25 smithi183 ceph-mon[35333]: pgmap v144: 97 pgs: 97 active+clean; 199 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.3 MiB/s wr, 827 op/s 2024-09-19T04:51:25.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:25 smithi123 ceph-mon[30511]: pgmap v144: 97 pgs: 97 active+clean; 199 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.3 MiB/s wr, 827 op/s 2024-09-19T04:51:25.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:25 smithi193 ceph-mon[35592]: pgmap v144: 97 pgs: 97 active+clean; 199 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.3 MiB/s wr, 827 op/s 2024-09-19T04:51:26.063 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:51:26.489 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:51:27.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.099+0000 7fd51a800640 1 -- 172.21.15.123:0/937131376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd514101dd0 msgr2=0x7fd5141021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:27.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.099+0000 7fd51a800640 1 --2- 172.21.15.123:0/937131376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd514101dd0 0x7fd5141021b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd504009900 tx=0x7fd50402a400 comp rx=0 tx=0).stop 2024-09-19T04:51:27.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.099+0000 7fd51a800640 1 -- 172.21.15.123:0/937131376 shutdown_connections 2024-09-19T04:51:27.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.099+0000 7fd51a800640 1 --2- 172.21.15.123:0/937131376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd514106f00 0x7fd51410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.099+0000 7fd51a800640 1 --2- 172.21.15.123:0/937131376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd514102780 0x7fd5141067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.099+0000 7fd51a800640 1 --2- 172.21.15.123:0/937131376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd514101dd0 0x7fd5141021b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.099+0000 7fd51a800640 1 -- 172.21.15.123:0/937131376 >> 172.21.15.123:0/937131376 conn(0x7fd5140fd890 msgr2=0x7fd5140ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:27.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.099+0000 7fd51a800640 1 -- 172.21.15.123:0/937131376 shutdown_connections 2024-09-19T04:51:27.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.099+0000 7fd51a800640 1 -- 172.21.15.123:0/937131376 wait complete. 2024-09-19T04:51:27.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.101+0000 7fd51a800640 1 Processor -- start 2024-09-19T04:51:27.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.101+0000 7fd51a800640 1 -- start start 2024-09-19T04:51:27.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.101+0000 7fd51a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd514101dd0 0x7fd514198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.101+0000 7fd51a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd514102780 0x7fd514198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.101+0000 7fd51a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd514106f00 0x7fd51419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.101+0000 7fd51a800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd514111fb0 con 0x7fd514106f00 2024-09-19T04:51:27.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.101+0000 7fd51a800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd514111e30 con 0x7fd514101dd0 2024-09-19T04:51:27.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.101+0000 7fd51a800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd514112130 con 0x7fd514102780 2024-09-19T04:51:27.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.101+0000 7fd513400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd514101dd0 0x7fd514198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.101+0000 7fd512a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd514102780 0x7fd514198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.101+0000 7fd513e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd514106f00 0x7fd51419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.101+0000 7fd513400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd514101dd0 0x7fd514198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:42842/0 (socket says 172.21.15.123:42842) 2024-09-19T04:51:27.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.101+0000 7fd512a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd514102780 0x7fd514198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58736/0 (socket says 172.21.15.123:58736) 2024-09-19T04:51:27.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.101+0000 7fd513400640 1 -- 172.21.15.123:0/3748436159 learned_addr learned my addr 172.21.15.123:0/3748436159 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:27.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.102+0000 7fd512a00640 1 -- 172.21.15.123:0/3748436159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd514101dd0 msgr2=0x7fd514198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:27.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.102+0000 7fd512a00640 1 --2- 172.21.15.123:0/3748436159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd514101dd0 0x7fd514198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.102+0000 7fd512a00640 1 -- 172.21.15.123:0/3748436159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd514106f00 msgr2=0x7fd51419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:27.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.102+0000 7fd512a00640 1 --2- 172.21.15.123:0/3748436159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd514106f00 0x7fd51419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.102+0000 7fd512a00640 1 -- 172.21.15.123:0/3748436159 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51419d3e0 con 0x7fd514102780 2024-09-19T04:51:27.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.102+0000 7fd513400640 1 --2- 172.21.15.123:0/3748436159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd514101dd0 0x7fd514198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:27.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.102+0000 7fd513e00640 1 --2- 172.21.15.123:0/3748436159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd514106f00 0x7fd51419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:27.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.102+0000 7fd512a00640 1 --2- 172.21.15.123:0/3748436159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd514102780 0x7fd514198970 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd50000e9f0 tx=0x7fd50000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:27.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.102+0000 7fd50be00640 1 -- 172.21.15.123:0/3748436159 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50000ccc0 con 0x7fd514102780 2024-09-19T04:51:27.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.102+0000 7fd50be00640 1 -- 172.21.15.123:0/3748436159 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd500010910 con 0x7fd514102780 2024-09-19T04:51:27.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.102+0000 7fd50be00640 1 -- 172.21.15.123:0/3748436159 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500018960 con 0x7fd514102780 2024-09-19T04:51:27.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.102+0000 7fd51a800640 1 -- 172.21.15.123:0/3748436159 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51419d6d0 con 0x7fd514102780 2024-09-19T04:51:27.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.103+0000 7fd51a800640 1 -- 172.21.15.123:0/3748436159 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5141a4fb0 con 0x7fd514102780 2024-09-19T04:51:27.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.106+0000 7fd50be00640 1 -- 172.21.15.123:0/3748436159 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd500010ac0 con 0x7fd514102780 2024-09-19T04:51:27.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.106+0000 7fd51d423740 1 Processor -- start 2024-09-19T04:51:27.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.106+0000 7fd51d423740 1 -- start start 2024-09-19T04:51:27.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.106+0000 7fd51d423740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4ab90 0x561fe7c4af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.106+0000 7fd51d423740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4b4b0 0x561fe7c55830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.106+0000 7fd51d423740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 0x561fe7c58160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.106+0000 7fd51d423740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561fe7bb13e0 con 0x561fe7c4ab90 2024-09-19T04:51:27.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.106+0000 7fd51d423740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561fe7b58d80 con 0x561fe7c55d70 2024-09-19T04:51:27.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.106+0000 7fd51d423740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561fe7bad3a0 con 0x561fe7c4b4b0 2024-09-19T04:51:27.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.107+0000 7fd513400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4b4b0 0x561fe7c55830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.107+0000 7fd513400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4b4b0 0x561fe7c55830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58752/0 (socket says 172.21.15.123:58752) 2024-09-19T04:51:27.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.107+0000 7fd513400640 1 -- 172.21.15.123:0/3085126420 learned_addr learned my addr 172.21.15.123:0/3085126420 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:27.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.107+0000 7fd512a00640 1 --2- 172.21.15.123:0/3085126420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 0x561fe7c58160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.107+0000 7fd513e00640 1 --2- 172.21.15.123:0/3085126420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4ab90 0x561fe7c4af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.107+0000 7fd513400640 1 -- 172.21.15.123:0/3085126420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 msgr2=0x561fe7c58160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:27.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.107+0000 7fd513400640 1 --2- 172.21.15.123:0/3085126420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 0x561fe7c58160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.107+0000 7fd513400640 1 -- 172.21.15.123:0/3085126420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4ab90 msgr2=0x561fe7c4af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:27.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.107+0000 7fd513400640 1 --2- 172.21.15.123:0/3085126420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4ab90 0x561fe7c4af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.107+0000 7fd513400640 1 -- 172.21.15.123:0/3085126420 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe7b77ae0 con 0x561fe7c4b4b0 2024-09-19T04:51:27.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.107+0000 7fd512a00640 1 --2- 172.21.15.123:0/3085126420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 0x561fe7c58160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:27.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.107+0000 7fd50be00640 1 --2- 172.21.15.123:0/3748436159 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd4f80a3430 0x7fd4f80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.108+0000 7fd50be00640 1 -- 172.21.15.123:0/3748436159 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd5000cffe0 con 0x7fd514102780 2024-09-19T04:51:27.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.108+0000 7fd513400640 1 --2- 172.21.15.123:0/3085126420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4b4b0 0x561fe7c55830 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd504005c40 tx=0x7fd504005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80da1590d8157cfb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:27.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.108+0000 7fd513e00640 1 --2- 172.21.15.123:0/3748436159 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd4f80a3430 0x7fd4f80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.108+0000 7fd508c00640 1 -- 172.21.15.123:0/3085126420 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd504040070 con 0x561fe7c4b4b0 2024-09-19T04:51:27.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.108+0000 7fd508c00640 1 -- 172.21.15.123:0/3085126420 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5040047a0 con 0x561fe7c4b4b0 2024-09-19T04:51:27.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.108+0000 7fd508c00640 1 -- 172.21.15.123:0/3085126420 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd504035040 con 0x561fe7c4b4b0 2024-09-19T04:51:27.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.109+0000 7fd51d423740 1 -- 172.21.15.123:0/3085126420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4b4b0 msgr2=0x561fe7c55830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:27.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.109+0000 7fd513e00640 1 --2- 172.21.15.123:0/3748436159 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd4f80a3430 0x7fd4f80a58f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009fd0 tx=0x7fd50c009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:27.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.109+0000 7fd51d423740 1 --2- 172.21.15.123:0/3085126420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4b4b0 0x561fe7c55830 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd504005c40 tx=0x7fd504005dd0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.109+0000 7fd51d423740 1 -- 172.21.15.123:0/3085126420 shutdown_connections 2024-09-19T04:51:27.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.109+0000 7fd51d423740 1 --2- 172.21.15.123:0/3085126420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 0x561fe7c58160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.109+0000 7fd51d423740 1 --2- 172.21.15.123:0/3085126420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4b4b0 0x561fe7c55830 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.109+0000 7fd51d423740 1 --2- 172.21.15.123:0/3085126420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4ab90 0x561fe7c4af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.109+0000 7fd51d423740 1 -- 172.21.15.123:0/3085126420 >> 172.21.15.123:0/3085126420 conn(0x561fe7c47120 msgr2=0x561fe7c47500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:27.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.109+0000 7fd51d423740 1 -- 172.21.15.123:0/3085126420 shutdown_connections 2024-09-19T04:51:27.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.109+0000 7fd51d423740 1 -- 172.21.15.123:0/3085126420 wait complete. 2024-09-19T04:51:27.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.109+0000 7fd51d423740 1 -- start start 2024-09-19T04:51:27.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.110+0000 7fd51d423740 1 Processor -- start 2024-09-19T04:51:27.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.110+0000 7fd51d423740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4ab90 0x561fe7c52c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.110+0000 7fd51d423740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4b4b0 0x561fe7c53150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.110+0000 7fd51d423740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 0x561fe7c53690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.110+0000 7fd51d423740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561fe7c589e0 con 0x561fe7c4ab90 2024-09-19T04:51:27.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.110+0000 7fd51d423740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561fe7b6c0f0 con 0x561fe7c55d70 2024-09-19T04:51:27.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.110+0000 7fd51d423740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561fe7c58b60 con 0x561fe7c4b4b0 2024-09-19T04:51:27.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.110+0000 7fd51d423740 10 client.0 _ll_register_callbacks cb 0x561fe7b5d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:51:27.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.110+0000 7fd513e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 0x561fe7c53690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.110+0000 7fd513400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4ab90 0x561fe7c52c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.110+0000 7fd513e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 0x561fe7c53690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:42854/0 (socket says 172.21.15.123:42854) 2024-09-19T04:51:27.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.110+0000 7fd513e00640 1 -- 172.21.15.123:0/2807345560 learned_addr learned my addr 172.21.15.123:0/2807345560 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:27.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.111+0000 7fd512a00640 1 --2- 172.21.15.123:0/2807345560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4b4b0 0x561fe7c53150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.111+0000 7fd513e00640 1 -- 172.21.15.123:0/2807345560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4b4b0 msgr2=0x561fe7c53150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:27.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.111+0000 7fd513e00640 1 --2- 172.21.15.123:0/2807345560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4b4b0 0x561fe7c53150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.111+0000 7fd513e00640 1 -- 172.21.15.123:0/2807345560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4ab90 msgr2=0x561fe7c52c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:27.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.111+0000 7fd513e00640 1 --2- 172.21.15.123:0/2807345560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4ab90 0x561fe7c52c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.111+0000 7fd513e00640 1 -- 172.21.15.123:0/2807345560 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe79223c0 con 0x561fe7c55d70 2024-09-19T04:51:27.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.111+0000 7fd513400640 1 --2- 172.21.15.123:0/2807345560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4ab90 0x561fe7c52c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:27.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.111+0000 7fd513e00640 1 --2- 172.21.15.123:0/2807345560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 0x561fe7c53690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd50c003f40 tx=0x7fd50c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80e411c47706be68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:27.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.111+0000 7fd4fd600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:51:27.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.112+0000 7fd4fd600640 1 -- 172.21.15.123:0/2807345560 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c0045c0 con 0x561fe7c55d70 2024-09-19T04:51:27.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.112+0000 7fd51d423740 1 -- 172.21.15.123:0/2807345560 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5040047a0 con 0x561fe7c55d70 2024-09-19T04:51:27.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.112+0000 7fd4fd600640 1 -- 172.21.15.123:0/2807345560 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd50c004760 con 0x561fe7c55d70 2024-09-19T04:51:27.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.112+0000 7fd4fd600640 1 -- 172.21.15.123:0/2807345560 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd50c012020 con 0x561fe7c55d70 2024-09-19T04:51:27.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.112+0000 7fd51d423740 10 client.26156 fetch_fsmap learned FSMap version 36 2024-09-19T04:51:27.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.112+0000 7fd51d423740 1 -- 172.21.15.123:0/2807345560 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561fe7c646f0 con 0x561fe7c55d70 2024-09-19T04:51:27.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.112+0000 7fd4fd600640 1 -- 172.21.15.123:0/2807345560 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5040047a0 con 0x561fe7c55d70 2024-09-19T04:51:27.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.112+0000 7fd4fd600640 1 -- 172.21.15.123:0/2807345560 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fd50c027030 con 0x561fe7c55d70 2024-09-19T04:51:27.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.112+0000 7fd51d423740 10 client.26156 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:51:27.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.112+0000 7fd51d423740 10 client.26156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:51:27.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.112+0000 7fd51d423740 20 client.26156 populate_metadata read hostname 'smithi123' 2024-09-19T04:51:27.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.112+0000 7fd51d423740 1 --2- 172.21.15.123:0/2807345560 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x561fe7c65b00 0x561fe7c67ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.112+0000 7fd51d423740 4 client.26156 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:51:27.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.112+0000 7fd51d423740 1 -- 172.21.15.123:0/2807345560 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fe7bbe950 con 0x561fe7c65b00 2024-09-19T04:51:27.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.113+0000 7fd513400640 1 --2- 172.21.15.123:0/2807345560 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x561fe7c65b00 0x561fe7c67ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.114+0000 7fd513400640 1 --2- 172.21.15.123:0/2807345560 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x561fe7c65b00 0x561fe7c67ef0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e6f7eccfa1e4130 server_cookie=5ba65f8020f7c141 in_seq=0 out_seq=0 2024-09-19T04:51:27.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.114+0000 7fd4fd600640 10 client.26156 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:51:27.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.115+0000 7fd4fd600640 1 -- 172.21.15.123:0/2807345560 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fe7bbe950 con 0x561fe7c65b00 2024-09-19T04:51:27.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.115+0000 7fd4fd600640 10 client.26156 handle_command_reply: tid=0 2024-09-19T04:51:27.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.115+0000 7fd51d423740 1 client.26156 shutdown 2024-09-19T04:51:27.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.115+0000 7fd51d423740 1 -- 172.21.15.123:0/2807345560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 msgr2=0x561fe7c53690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:27.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.115+0000 7fd51d423740 1 --2- 172.21.15.123:0/2807345560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 0x561fe7c53690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd50c003f40 tx=0x7fd50c004ab0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.115+0000 7fd51d423740 1 -- 172.21.15.123:0/2807345560 shutdown_connections 2024-09-19T04:51:27.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.115+0000 7fd51d423740 1 --2- 172.21.15.123:0/2807345560 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x561fe7c65b00 0x561fe7c67ef0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.115+0000 7fd51d423740 1 --2- 172.21.15.123:0/2807345560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 0x561fe7c53690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.115+0000 7fd51d423740 1 --2- 172.21.15.123:0/2807345560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4b4b0 0x561fe7c53150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.115+0000 7fd51d423740 1 --2- 172.21.15.123:0/2807345560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4ab90 0x561fe7c52c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.115+0000 7fd51d423740 1 -- 172.21.15.123:0/2807345560 >> 172.21.15.123:0/2807345560 conn(0x561fe7c47120 msgr2=0x561fe7c4ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:27.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.115+0000 7fd4fd600640 0 client.26156 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:51:27.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.116+0000 7fd51d423740 1 -- 172.21.15.123:0/2807345560 shutdown_connections 2024-09-19T04:51:27.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.116+0000 7fd51d423740 1 -- 172.21.15.123:0/2807345560 wait complete. 2024-09-19T04:51:27.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.116+0000 7fd51d423740 20 client.26156 trim_cache size 0 max 16384 2024-09-19T04:51:27.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.145+0000 7fd51d423740 1 Processor -- start 2024-09-19T04:51:27.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.145+0000 7fd51d423740 1 -- start start 2024-09-19T04:51:27.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.145+0000 7fd51d423740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4ab90 0x561fe7c4d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.145+0000 7fd51d423740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4b4b0 0x561fe7c47500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.145+0000 7fd51d423740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 0x561fe7c460b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.145+0000 7fd51d423740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5141afa70 con 0x561fe7c4b4b0 2024-09-19T04:51:27.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.145+0000 7fd51d423740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd5141b15b0 con 0x561fe7c55d70 2024-09-19T04:51:27.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.145+0000 7fd51d423740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5141b0140 con 0x561fe7c4ab90 2024-09-19T04:51:27.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.145+0000 7fd512a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4b4b0 0x561fe7c47500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.146+0000 7fd513400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4ab90 0x561fe7c4d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.146+0000 7fd512a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4b4b0 0x561fe7c47500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49342/0 (socket says 172.21.15.123:49342) 2024-09-19T04:51:27.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.146+0000 7fd512a00640 1 -- 172.21.15.123:0/2383234455 learned_addr learned my addr 172.21.15.123:0/2383234455 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:27.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.146+0000 7fd513e00640 1 --2- 172.21.15.123:0/2383234455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 0x561fe7c460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.146+0000 7fd513400640 1 -- 172.21.15.123:0/2383234455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 msgr2=0x561fe7c460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:27.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.146+0000 7fd513400640 1 --2- 172.21.15.123:0/2383234455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 0x561fe7c460b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.146+0000 7fd513400640 1 -- 172.21.15.123:0/2383234455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4b4b0 msgr2=0x561fe7c47500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:27.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.146+0000 7fd513400640 1 --2- 172.21.15.123:0/2383234455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4b4b0 0x561fe7c47500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.146+0000 7fd513400640 1 -- 172.21.15.123:0/2383234455 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe7c51bc0 con 0x561fe7c4ab90 2024-09-19T04:51:27.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.146+0000 7fd512a00640 1 --2- 172.21.15.123:0/2383234455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4b4b0 0x561fe7c47500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:27.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.146+0000 7fd513e00640 1 --2- 172.21.15.123:0/2383234455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 0x561fe7c460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:27.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.146+0000 7fd513400640 1 --2- 172.21.15.123:0/2383234455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4ab90 0x561fe7c4d5f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd504031550 tx=0x7fd50402a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84155d411ce78ad1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:27.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.147+0000 7fd4ffe00640 1 -- 172.21.15.123:0/2383234455 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd504040040 con 0x561fe7c4ab90 2024-09-19T04:51:27.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.147+0000 7fd4ffe00640 1 -- 172.21.15.123:0/2383234455 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd504003c10 con 0x561fe7c4ab90 2024-09-19T04:51:27.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.147+0000 7fd4ffe00640 1 -- 172.21.15.123:0/2383234455 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd504035040 con 0x561fe7c4ab90 2024-09-19T04:51:27.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.148+0000 7fd51d423740 1 -- 172.21.15.123:0/2383234455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4ab90 msgr2=0x561fe7c4d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:27.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.148+0000 7fd51d423740 1 --2- 172.21.15.123:0/2383234455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4ab90 0x561fe7c4d5f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd504031550 tx=0x7fd50402a400 comp rx=0 tx=0).stop 2024-09-19T04:51:27.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.148+0000 7fd51d423740 1 -- 172.21.15.123:0/2383234455 shutdown_connections 2024-09-19T04:51:27.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.148+0000 7fd51d423740 1 --2- 172.21.15.123:0/2383234455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c55d70 0x561fe7c460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.148+0000 7fd51d423740 1 --2- 172.21.15.123:0/2383234455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4b4b0 0x561fe7c47500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.148+0000 7fd51d423740 1 --2- 172.21.15.123:0/2383234455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c4ab90 0x561fe7c4d5f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.148+0000 7fd51d423740 1 -- 172.21.15.123:0/2383234455 >> 172.21.15.123:0/2383234455 conn(0x561fe7c47120 msgr2=0x561fe7c459a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:27.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.148+0000 7fd51d423740 1 -- 172.21.15.123:0/2383234455 shutdown_connections 2024-09-19T04:51:27.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.148+0000 7fd51d423740 1 -- 172.21.15.123:0/2383234455 wait complete. 2024-09-19T04:51:27.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.148+0000 7fd51d423740 1 -- start start 2024-09-19T04:51:27.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.148+0000 7fd51d423740 1 Processor -- start 2024-09-19T04:51:27.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.149+0000 7fd51d423740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c4ab90 0x561fe7c59a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.149+0000 7fd51d423740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4b4b0 0x561fe7c59f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.149+0000 7fd51d423740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c55d70 0x561fe7c5a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.149+0000 7fd51d423740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5141b0ee0 con 0x561fe7c4b4b0 2024-09-19T04:51:27.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.149+0000 7fd51d423740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd5141b0810 con 0x561fe7c4ab90 2024-09-19T04:51:27.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.149+0000 7fd51d423740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5141ada50 con 0x561fe7c55d70 2024-09-19T04:51:27.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.149+0000 7fd51d423740 10 client.0 _ll_register_callbacks cb 0x7fd50c003e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:51:27.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.149+0000 7fd513400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c4ab90 0x561fe7c59a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.149+0000 7fd512a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4b4b0 0x561fe7c59f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.149+0000 7fd513400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c4ab90 0x561fe7c59a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:42880/0 (socket says 172.21.15.123:42880) 2024-09-19T04:51:27.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.149+0000 7fd512a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4b4b0 0x561fe7c59f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49356/0 (socket says 172.21.15.123:49356) 2024-09-19T04:51:27.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.149+0000 7fd513400640 1 -- 172.21.15.123:0/1938103109 learned_addr learned my addr 172.21.15.123:0/1938103109 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:27.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.149+0000 7fd513e00640 1 --2- 172.21.15.123:0/1938103109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c55d70 0x561fe7c5a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.150+0000 7fd513400640 1 -- 172.21.15.123:0/1938103109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c55d70 msgr2=0x561fe7c5a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:27.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.150+0000 7fd513400640 1 --2- 172.21.15.123:0/1938103109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c55d70 0x561fe7c5a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.150+0000 7fd513400640 1 -- 172.21.15.123:0/1938103109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4b4b0 msgr2=0x561fe7c59f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:27.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.150+0000 7fd513400640 1 --2- 172.21.15.123:0/1938103109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4b4b0 0x561fe7c59f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.150+0000 7fd513400640 1 -- 172.21.15.123:0/1938103109 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe7ccf790 con 0x561fe7c4ab90 2024-09-19T04:51:27.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.150+0000 7fd512a00640 1 --2- 172.21.15.123:0/1938103109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4b4b0 0x561fe7c59f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:51:27.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.150+0000 7fd513400640 1 --2- 172.21.15.123:0/1938103109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c4ab90 0x561fe7c59a20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd504005c40 tx=0x7fd504009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2b93c1bd350268d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:27.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.150+0000 7fd4fd600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:51:27.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.150+0000 7fd4fd600640 1 -- 172.21.15.123:0/1938103109 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd504060070 con 0x561fe7c4ab90 2024-09-19T04:51:27.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.150+0000 7fd51d423740 1 -- 172.21.15.123:0/1938103109 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd504003c10 con 0x561fe7c4ab90 2024-09-19T04:51:27.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.150+0000 7fd4fd600640 1 -- 172.21.15.123:0/1938103109 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd5040315b0 con 0x561fe7c4ab90 2024-09-19T04:51:27.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.151+0000 7fd4fd600640 1 -- 172.21.15.123:0/1938103109 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd50406c050 con 0x561fe7c4ab90 2024-09-19T04:51:27.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.151+0000 7fd51d423740 10 client.26162 fetch_fsmap learned FSMap version 36 2024-09-19T04:51:27.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.151+0000 7fd51d423740 1 -- 172.21.15.123:0/1938103109 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561fe7cd09a0 con 0x561fe7c4ab90 2024-09-19T04:51:27.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.151+0000 7fd4fd600640 1 -- 172.21.15.123:0/1938103109 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd504003c10 con 0x561fe7c4ab90 2024-09-19T04:51:27.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.151+0000 7fd4fd600640 1 -- 172.21.15.123:0/1938103109 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fd504050980 con 0x561fe7c4ab90 2024-09-19T04:51:27.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.151+0000 7fd51d423740 10 client.26162 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:51:27.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.151+0000 7fd51d423740 10 client.26162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:51:27.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.151+0000 7fd51d423740 20 client.26162 populate_metadata read hostname 'smithi123' 2024-09-19T04:51:27.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.151+0000 7fd51d423740 1 --2- 172.21.15.123:0/1938103109 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x561fe7cd0bd0 0x561fe7cd2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:27.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.151+0000 7fd51d423740 4 client.26162 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:51:27.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.151+0000 7fd51d423740 1 -- 172.21.15.123:0/1938103109 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561fe7bb8e60 con 0x561fe7cd0bd0 2024-09-19T04:51:27.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.152+0000 7fd512a00640 1 --2- 172.21.15.123:0/1938103109 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x561fe7cd0bd0 0x561fe7cd2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:27.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.152+0000 7fd512a00640 1 --2- 172.21.15.123:0/1938103109 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x561fe7cd0bd0 0x561fe7cd2fc0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b392167329ba1ba0 server_cookie=33bc6edee134e1d4 in_seq=0 out_seq=0 2024-09-19T04:51:27.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.152+0000 7fd4fd600640 10 client.26162 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:51:27.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.152+0000 7fd4fd600640 1 -- 172.21.15.123:0/1938103109 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561fe7bb8e60 con 0x561fe7cd0bd0 2024-09-19T04:51:27.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.152+0000 7fd4fd600640 10 client.26162 handle_command_reply: tid=0 2024-09-19T04:51:27.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.152+0000 7fd51d423740 1 client.26162 shutdown 2024-09-19T04:51:27.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.153+0000 7fd51d423740 1 -- 172.21.15.123:0/1938103109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c4ab90 msgr2=0x561fe7c59a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:27.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.153+0000 7fd51d423740 1 --2- 172.21.15.123:0/1938103109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c4ab90 0x561fe7c59a20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd504005c40 tx=0x7fd504009260 comp rx=0 tx=0).stop 2024-09-19T04:51:27.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.153+0000 7fd51d423740 1 -- 172.21.15.123:0/1938103109 shutdown_connections 2024-09-19T04:51:27.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.153+0000 7fd51d423740 1 --2- 172.21.15.123:0/1938103109 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x561fe7cd0bd0 0x561fe7cd2fc0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.153+0000 7fd51d423740 1 --2- 172.21.15.123:0/1938103109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fe7c55d70 0x561fe7c5a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.153+0000 7fd51d423740 1 --2- 172.21.15.123:0/1938103109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fe7c4b4b0 0x561fe7c59f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.153+0000 7fd51d423740 1 --2- 172.21.15.123:0/1938103109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fe7c4ab90 0x561fe7c59a20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.153+0000 7fd4fd600640 0 client.26162 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:51:27.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.153+0000 7fd51d423740 1 -- 172.21.15.123:0/1938103109 >> 172.21.15.123:0/1938103109 conn(0x561fe7c47120 msgr2=0x561fe7c533f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:27.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.153+0000 7fd51d423740 1 -- 172.21.15.123:0/1938103109 shutdown_connections 2024-09-19T04:51:27.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.153+0000 7fd51d423740 1 -- 172.21.15.123:0/1938103109 wait complete. 2024-09-19T04:51:27.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.153+0000 7fd51d423740 20 client.26162 trim_cache size 0 max 16384 2024-09-19T04:51:27.159 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:51:27.159 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-09-19T04:51:27.159 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-09-19T04:51:27.159 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:51:27.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.154+0000 7fd51a800640 1 -- 172.21.15.123:0/3748436159 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd4f80a3430 msgr2=0x7fd4f80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:27.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.154+0000 7fd51a800640 1 --2- 172.21.15.123:0/3748436159 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd4f80a3430 0x7fd4f80a58f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009fd0 tx=0x7fd50c009500 comp rx=0 tx=0).stop 2024-09-19T04:51:27.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.154+0000 7fd51a800640 1 -- 172.21.15.123:0/3748436159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd514102780 msgr2=0x7fd514198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:27.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.154+0000 7fd51a800640 1 --2- 172.21.15.123:0/3748436159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd514102780 0x7fd514198970 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd50000e9f0 tx=0x7fd50000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.154+0000 7fd51a800640 1 -- 172.21.15.123:0/3748436159 shutdown_connections 2024-09-19T04:51:27.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.154+0000 7fd51a800640 1 --2- 172.21.15.123:0/3748436159 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd4f80a3430 0x7fd4f80a58f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.154+0000 7fd51a800640 1 --2- 172.21.15.123:0/3748436159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd514106f00 0x7fd51419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.154+0000 7fd51a800640 1 --2- 172.21.15.123:0/3748436159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd514102780 0x7fd514198970 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.154+0000 7fd51a800640 1 --2- 172.21.15.123:0/3748436159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd514101dd0 0x7fd514198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:27.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.154+0000 7fd51a800640 1 -- 172.21.15.123:0/3748436159 >> 172.21.15.123:0/3748436159 conn(0x7fd5140fd890 msgr2=0x7fd5140fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:27.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.154+0000 7fd51a800640 1 -- 172.21.15.123:0/3748436159 shutdown_connections 2024-09-19T04:51:27.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:27.154+0000 7fd51a800640 1 -- 172.21.15.123:0/3748436159 wait complete. 2024-09-19T04:51:27.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:27 smithi183 ceph-mon[35333]: pgmap v145: 97 pgs: 97 active+clean; 199 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 545 op/s 2024-09-19T04:51:27.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:27 smithi123 ceph-mon[30511]: pgmap v145: 97 pgs: 97 active+clean; 199 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 545 op/s 2024-09-19T04:51:27.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:27 smithi193 ceph-mon[35592]: pgmap v145: 97 pgs: 97 active+clean; 199 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 545 op/s 2024-09-19T04:51:27.665 INFO:tasks.workunit.client.0.smithi123.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-09-19T04:51:28.317 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:51:28.318 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:51:28.756 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:51:29.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:29 smithi123 ceph-mon[30511]: pgmap v146: 97 pgs: 97 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.2 MiB/s wr, 820 op/s 2024-09-19T04:51:29.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:29 smithi183 ceph-mon[35333]: pgmap v146: 97 pgs: 97 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.2 MiB/s wr, 820 op/s 2024-09-19T04:51:29.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.376+0000 7f5179400640 1 -- 172.21.15.123:0/2602753893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51741047e0 msgr2=0x7f5174108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:29.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.376+0000 7f5179400640 1 --2- 172.21.15.123:0/2602753893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51741047e0 0x7f5174108830 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5160009900 tx=0x7f516002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:51:29.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.376+0000 7f5179400640 1 -- 172.21.15.123:0/2602753893 shutdown_connections 2024-09-19T04:51:29.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.376+0000 7f5179400640 1 --2- 172.21.15.123:0/2602753893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174108f60 0x7f5174110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:29.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.376+0000 7f5179400640 1 --2- 172.21.15.123:0/2602753893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51741047e0 0x7f5174108830 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:29.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.376+0000 7f5179400640 1 --2- 172.21.15.123:0/2602753893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5174103e30 0x7f5174104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:29.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.376+0000 7f5179400640 1 -- 172.21.15.123:0/2602753893 >> 172.21.15.123:0/2602753893 conn(0x7f51740ff910 msgr2=0x7f5174101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:29.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.376+0000 7f5179400640 1 -- 172.21.15.123:0/2602753893 shutdown_connections 2024-09-19T04:51:29.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.376+0000 7f5179400640 1 -- 172.21.15.123:0/2602753893 wait complete. 2024-09-19T04:51:29.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.377+0000 7f5179400640 1 Processor -- start 2024-09-19T04:51:29.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.377+0000 7f5179400640 1 -- start start 2024-09-19T04:51:29.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.378+0000 7f5179400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5174103e30 0x7f517419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:29.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.378+0000 7f5179400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51741047e0 0x7f517419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:29.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.378+0000 7f5179400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174108f60 0x7f517419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:29.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.378+0000 7f5179400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5174113fc0 con 0x7f5174108f60 2024-09-19T04:51:29.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.378+0000 7f5179400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5174113e40 con 0x7f51741047e0 2024-09-19T04:51:29.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.378+0000 7f5179400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5174114140 con 0x7f5174103e30 2024-09-19T04:51:29.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.378+0000 7f5173400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51741047e0 0x7f517419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:29.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.378+0000 7f5173400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51741047e0 0x7f517419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:42898/0 (socket says 172.21.15.123:42898) 2024-09-19T04:51:29.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.378+0000 7f5173400640 1 -- 172.21.15.123:0/1294897205 learned_addr learned my addr 172.21.15.123:0/1294897205 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:29.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.378+0000 7f5178a00640 1 --2- 172.21.15.123:0/1294897205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174108f60 0x7f517419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:29.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.378+0000 7f5173400640 1 -- 172.21.15.123:0/1294897205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5174103e30 msgr2=0x7f517419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:29.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.378+0000 7f5173400640 1 --2- 172.21.15.123:0/1294897205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5174103e30 0x7f517419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:29.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.378+0000 7f5173400640 1 -- 172.21.15.123:0/1294897205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174108f60 msgr2=0x7f517419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:29.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.378+0000 7f5173400640 1 --2- 172.21.15.123:0/1294897205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174108f60 0x7f517419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:29.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.379+0000 7f5173400640 1 -- 172.21.15.123:0/1294897205 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517419f430 con 0x7f51741047e0 2024-09-19T04:51:29.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.379+0000 7f5178a00640 1 --2- 172.21.15.123:0/1294897205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174108f60 0x7f517419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:29.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.379+0000 7f5173400640 1 --2- 172.21.15.123:0/1294897205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51741047e0 0x7f517419a9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5160006010 tx=0x7f51600042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:29.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.379+0000 7f5170c00640 1 -- 172.21.15.123:0/1294897205 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516003f030 con 0x7f51741047e0 2024-09-19T04:51:29.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.379+0000 7f5170c00640 1 -- 172.21.15.123:0/1294897205 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5160003ce0 con 0x7f51741047e0 2024-09-19T04:51:29.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.379+0000 7f5179400640 1 -- 172.21.15.123:0/1294897205 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f517419f6c0 con 0x7f51741047e0 2024-09-19T04:51:29.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.380+0000 7f5179400640 1 -- 172.21.15.123:0/1294897205 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51741a7000 con 0x7f51741047e0 2024-09-19T04:51:29.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.380+0000 7f5170c00640 1 -- 172.21.15.123:0/1294897205 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5160035070 con 0x7f51741047e0 2024-09-19T04:51:29.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.381+0000 7f5179400640 1 -- 172.21.15.123:0/1294897205 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5174105310 con 0x7f51741047e0 2024-09-19T04:51:29.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.383+0000 7f5170c00640 1 -- 172.21.15.123:0/1294897205 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5160042030 con 0x7f51741047e0 2024-09-19T04:51:29.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.384+0000 7f5170c00640 1 --2- 172.21.15.123:0/1294897205 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f51580a3430 0x7f51580a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:29.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.384+0000 7f5170c00640 1 -- 172.21.15.123:0/1294897205 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f51600f02b0 con 0x7f51741047e0 2024-09-19T04:51:29.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.385+0000 7f5173e00640 1 --2- 172.21.15.123:0/1294897205 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f51580a3430 0x7f51580a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:29.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.385+0000 7f5173e00640 1 --2- 172.21.15.123:0/1294897205 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f51580a3430 0x7f51580a58f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f51640058c0 tx=0x7f5164005850 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:29.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.388+0000 7f5170c00640 1 -- 172.21.15.123:0/1294897205 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f51600bcd20 con 0x7f51741047e0 2024-09-19T04:51:29.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:29 smithi193 ceph-mon[35592]: pgmap v146: 97 pgs: 97 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.2 MiB/s wr, 820 op/s 2024-09-19T04:51:29.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.623+0000 7f5179400640 1 -- 172.21.15.123:0/1294897205 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f517419b7e0 con 0x7f51741047e0 2024-09-19T04:51:29.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.624+0000 7f5170c00640 1 -- 172.21.15.123:0/1294897205 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11990 (secure 0 0 0) 0x7f517419b7e0 con 0x7f51741047e0 2024-09-19T04:51:29.626 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:51:29.626 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 36 2024-09-19T04:51:29.626 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":36,"btime":"2024-09-19T04:50:44:998923+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:50:44.262505+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24499,24500,24539,24505,24529]},"id":1}]} 2024-09-19T04:51:29.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.628+0000 7f5179400640 1 -- 172.21.15.123:0/1294897205 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f51580a3430 msgr2=0x7f51580a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:29.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.628+0000 7f5179400640 1 --2- 172.21.15.123:0/1294897205 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f51580a3430 0x7f51580a58f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f51640058c0 tx=0x7f5164005850 comp rx=0 tx=0).stop 2024-09-19T04:51:29.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.628+0000 7f5179400640 1 -- 172.21.15.123:0/1294897205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51741047e0 msgr2=0x7f517419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:29.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.628+0000 7f5179400640 1 --2- 172.21.15.123:0/1294897205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51741047e0 0x7f517419a9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5160006010 tx=0x7f51600042f0 comp rx=0 tx=0).stop 2024-09-19T04:51:29.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.629+0000 7f5179400640 1 -- 172.21.15.123:0/1294897205 shutdown_connections 2024-09-19T04:51:29.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.629+0000 7f5179400640 1 --2- 172.21.15.123:0/1294897205 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f51580a3430 0x7f51580a58f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:29.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.629+0000 7f5179400640 1 --2- 172.21.15.123:0/1294897205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174108f60 0x7f517419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:29.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.629+0000 7f5179400640 1 --2- 172.21.15.123:0/1294897205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51741047e0 0x7f517419a9c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:29.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.629+0000 7f5179400640 1 --2- 172.21.15.123:0/1294897205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5174103e30 0x7f517419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:29.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.629+0000 7f5179400640 1 -- 172.21.15.123:0/1294897205 >> 172.21.15.123:0/1294897205 conn(0x7f51740ff910 msgr2=0x7f5174100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:29.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.629+0000 7f5179400640 1 -- 172.21.15.123:0/1294897205 shutdown_connections 2024-09-19T04:51:29.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:29.629+0000 7f5179400640 1 -- 172.21.15.123:0/1294897205 wait complete. 2024-09-19T04:51:30.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:30 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1294897205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:51:30.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:30 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1294897205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:51:30.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:30 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1294897205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:51:30.629 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:51:30.667 INFO:tasks.workunit.client.0.smithi123.stdout:copying to largest ever: 0x63ed6b9 2024-09-19T04:51:31.064 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:51:31.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:31 smithi183 ceph-mon[35333]: pgmap v147: 97 pgs: 97 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 540 op/s 2024-09-19T04:51:31.373 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:31 smithi123 ceph-mon[30511]: pgmap v147: 97 pgs: 97 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 540 op/s 2024-09-19T04:51:31.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:31 smithi193 ceph-mon[35592]: pgmap v147: 97 pgs: 97 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 540 op/s 2024-09-19T04:51:31.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.651+0000 7f2868000640 1 -- 172.21.15.123:0/3142341925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2860106f00 msgr2=0x7f286010ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:31.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.651+0000 7f2868000640 1 --2- 172.21.15.123:0/3142341925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2860106f00 0x7f286010ea80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2858009ca0 tx=0x7f285802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.652+0000 7f2868000640 1 -- 172.21.15.123:0/3142341925 shutdown_connections 2024-09-19T04:51:31.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.652+0000 7f2868000640 1 --2- 172.21.15.123:0/3142341925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2860106f00 0x7f286010ea80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.652+0000 7f2868000640 1 --2- 172.21.15.123:0/3142341925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2860102780 0x7f28601067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.652+0000 7f2868000640 1 --2- 172.21.15.123:0/3142341925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2860101dd0 0x7f28601021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.652+0000 7f2868000640 1 -- 172.21.15.123:0/3142341925 >> 172.21.15.123:0/3142341925 conn(0x7f28600fd890 msgr2=0x7f28600ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:31.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.652+0000 7f2868000640 1 -- 172.21.15.123:0/3142341925 shutdown_connections 2024-09-19T04:51:31.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.652+0000 7f2868000640 1 -- 172.21.15.123:0/3142341925 wait complete. 2024-09-19T04:51:31.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.653+0000 7f2868000640 1 Processor -- start 2024-09-19T04:51:31.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.653+0000 7f2868000640 1 -- start start 2024-09-19T04:51:31.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.653+0000 7f2868000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2860101dd0 0x7f28600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.653+0000 7f2868000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2860102780 0x7f2860074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.653+0000 7f2868000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2860106f00 0x7f2860074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.653+0000 7f2868000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2860111fb0 con 0x7f2860101dd0 2024-09-19T04:51:31.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.653+0000 7f2868000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2860111e30 con 0x7f2860102780 2024-09-19T04:51:31.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.653+0000 7f2868000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2860112130 con 0x7f2860106f00 2024-09-19T04:51:31.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.654+0000 7f2865600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2860101dd0 0x7f28600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.654+0000 7f2864c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2860102780 0x7f2860074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.654+0000 7f2866000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2860106f00 0x7f2860074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.654+0000 7f2865600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2860101dd0 0x7f28600744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49386/0 (socket says 172.21.15.123:49386) 2024-09-19T04:51:31.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.654+0000 7f2865600640 1 -- 172.21.15.123:0/2379651169 learned_addr learned my addr 172.21.15.123:0/2379651169 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:31.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.654+0000 7f2864c00640 1 -- 172.21.15.123:0/2379651169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2860106f00 msgr2=0x7f2860074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:31.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.654+0000 7f2864c00640 1 --2- 172.21.15.123:0/2379651169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2860106f00 0x7f2860074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.654+0000 7f2864c00640 1 -- 172.21.15.123:0/2379651169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2860101dd0 msgr2=0x7f28600744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:31.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.654+0000 7f2864c00640 1 --2- 172.21.15.123:0/2379651169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2860101dd0 0x7f28600744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.654+0000 7f2864c00640 1 -- 172.21.15.123:0/2379651169 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28601a68f0 con 0x7f2860102780 2024-09-19T04:51:31.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.654+0000 7f2865600640 1 --2- 172.21.15.123:0/2379651169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2860101dd0 0x7f28600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:31.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.655+0000 7f2864c00640 1 --2- 172.21.15.123:0/2379651169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2860102780 0x7f2860074a30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f284c00e9f0 tx=0x7f284c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:31.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.655+0000 7f285e000640 1 -- 172.21.15.123:0/2379651169 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c00ccc0 con 0x7f2860102780 2024-09-19T04:51:31.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.655+0000 7f285e000640 1 -- 172.21.15.123:0/2379651169 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f284c010910 con 0x7f2860102780 2024-09-19T04:51:31.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.655+0000 7f2868000640 1 -- 172.21.15.123:0/2379651169 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28601a6be0 con 0x7f2860102780 2024-09-19T04:51:31.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.655+0000 7f285e000640 1 -- 172.21.15.123:0/2379651169 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c018960 con 0x7f2860102780 2024-09-19T04:51:31.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.655+0000 7f2868000640 1 -- 172.21.15.123:0/2379651169 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28601a7120 con 0x7f2860102780 2024-09-19T04:51:31.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.658+0000 7f285e000640 1 -- 172.21.15.123:0/2379651169 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f284c010ac0 con 0x7f2860102780 2024-09-19T04:51:31.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.658+0000 7f286ac5d740 1 Processor -- start 2024-09-19T04:51:31.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.658+0000 7f286ac5d740 1 -- start start 2024-09-19T04:51:31.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.659+0000 7f286ac5d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08e4c70 0x5613d08e5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.659+0000 7f286ac5d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e5590 0x5613d08ef910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.659+0000 7f286ac5d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08efe50 0x5613d08f2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.659+0000 7f286ac5d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5613d084b3e0 con 0x5613d08e4c70 2024-09-19T04:51:31.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.659+0000 7f286ac5d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5613d07f2d80 con 0x5613d08efe50 2024-09-19T04:51:31.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.659+0000 7f286ac5d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5613d08473a0 con 0x5613d08e5590 2024-09-19T04:51:31.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.659+0000 7f2865600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e5590 0x5613d08ef910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.659+0000 7f2865600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e5590 0x5613d08ef910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58836/0 (socket says 172.21.15.123:58836) 2024-09-19T04:51:31.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.659+0000 7f2865600640 1 -- 172.21.15.123:0/2504760229 learned_addr learned my addr 172.21.15.123:0/2504760229 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:31.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.659+0000 7f2866000640 1 --2- 172.21.15.123:0/2504760229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08e4c70 0x5613d08e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.659+0000 7f2864c00640 1 --2- 172.21.15.123:0/2504760229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08efe50 0x5613d08f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.659+0000 7f2865600640 1 -- 172.21.15.123:0/2504760229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08efe50 msgr2=0x5613d08f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.660+0000 7f2865600640 1 --2- 172.21.15.123:0/2504760229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08efe50 0x5613d08f2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.660+0000 7f2865600640 1 -- 172.21.15.123:0/2504760229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08e4c70 msgr2=0x5613d08e5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.660+0000 7f2865600640 1 --2- 172.21.15.123:0/2504760229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08e4c70 0x5613d08e5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.660+0000 7f2865600640 1 -- 172.21.15.123:0/2504760229 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d0811ae0 con 0x5613d08e5590 2024-09-19T04:51:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.660+0000 7f2866000640 1 --2- 172.21.15.123:0/2504760229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08e4c70 0x5613d08e5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:51:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.660+0000 7f2864c00640 1 --2- 172.21.15.123:0/2504760229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08efe50 0x5613d08f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:51:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.660+0000 7f2865600640 1 --2- 172.21.15.123:0/2504760229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e5590 0x5613d08ef910 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2854011650 tx=0x7f2854009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5930991ba6a483d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.660+0000 7f2852600640 1 -- 172.21.15.123:0/2504760229 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285401e070 con 0x5613d08e5590 2024-09-19T04:51:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.660+0000 7f2852600640 1 -- 172.21.15.123:0/2504760229 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f285400fba0 con 0x5613d08e5590 2024-09-19T04:51:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.660+0000 7f2852600640 1 -- 172.21.15.123:0/2504760229 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854019cd0 con 0x5613d08e5590 2024-09-19T04:51:31.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.660+0000 7f285e000640 1 --2- 172.21.15.123:0/2379651169 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f28480a33b0 0x7f28480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.661+0000 7f285e000640 1 -- 172.21.15.123:0/2379651169 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f284c0cff20 con 0x7f2860102780 2024-09-19T04:51:31.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.661+0000 7f2866000640 1 --2- 172.21.15.123:0/2379651169 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f28480a33b0 0x7f28480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.661+0000 7f286ac5d740 1 -- 172.21.15.123:0/2504760229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e5590 msgr2=0x5613d08ef910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:31.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.661+0000 7f286ac5d740 1 --2- 172.21.15.123:0/2504760229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e5590 0x5613d08ef910 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2854011650 tx=0x7f2854009f90 comp rx=0 tx=0).stop 2024-09-19T04:51:31.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.661+0000 7f286ac5d740 1 -- 172.21.15.123:0/2504760229 shutdown_connections 2024-09-19T04:51:31.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.661+0000 7f286ac5d740 1 --2- 172.21.15.123:0/2504760229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08efe50 0x5613d08f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.661+0000 7f286ac5d740 1 --2- 172.21.15.123:0/2504760229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e5590 0x5613d08ef910 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.661+0000 7f286ac5d740 1 --2- 172.21.15.123:0/2504760229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08e4c70 0x5613d08e5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.661+0000 7f286ac5d740 1 -- 172.21.15.123:0/2504760229 >> 172.21.15.123:0/2504760229 conn(0x5613d08e1200 msgr2=0x5613d08e15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:31.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.662+0000 7f2866000640 1 --2- 172.21.15.123:0/2379651169 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f28480a33b0 0x7f28480a5870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2858006010 tx=0x7f2858005d80 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:31.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.662+0000 7f286ac5d740 1 -- 172.21.15.123:0/2504760229 shutdown_connections 2024-09-19T04:51:31.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.662+0000 7f286ac5d740 1 -- 172.21.15.123:0/2504760229 wait complete. 2024-09-19T04:51:31.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.662+0000 7f286ac5d740 1 -- start start 2024-09-19T04:51:31.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.663+0000 7f286ac5d740 1 Processor -- start 2024-09-19T04:51:31.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.663+0000 7f286ac5d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e4c70 0x5613d08ecc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.663+0000 7f286ac5d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 0x5613d08ed140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.663+0000 7f286ac5d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08efe50 0x5613d08ed680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.663+0000 7f286ac5d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5613d08f2a20 con 0x5613d08efe50 2024-09-19T04:51:31.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.663+0000 7f286ac5d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5613d08060f0 con 0x5613d08e5590 2024-09-19T04:51:31.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.663+0000 7f286ac5d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5613d08f2ba0 con 0x5613d08e4c70 2024-09-19T04:51:31.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.663+0000 7f286ac5d740 10 client.0 _ll_register_callbacks cb 0x5613d07edbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:51:31.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.664+0000 7f2866000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08efe50 0x5613d08ed680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.664+0000 7f2866000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08efe50 0x5613d08ed680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49404/0 (socket says 172.21.15.123:49404) 2024-09-19T04:51:31.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.664+0000 7f2866000640 1 -- 172.21.15.123:0/1118345871 learned_addr learned my addr 172.21.15.123:0/1118345871 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:31.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.664+0000 7f2864c00640 1 --2- 172.21.15.123:0/1118345871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 0x5613d08ed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.664+0000 7f2865600640 1 --2- 172.21.15.123:0/1118345871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e4c70 0x5613d08ecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.664+0000 7f2864c00640 1 -- 172.21.15.123:0/1118345871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e4c70 msgr2=0x5613d08ecc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:31.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.664+0000 7f2864c00640 1 --2- 172.21.15.123:0/1118345871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e4c70 0x5613d08ecc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.664+0000 7f2864c00640 1 -- 172.21.15.123:0/1118345871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08efe50 msgr2=0x5613d08ed680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:31.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.664+0000 7f2864c00640 1 --2- 172.21.15.123:0/1118345871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08efe50 0x5613d08ed680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.664+0000 7f2864c00640 1 -- 172.21.15.123:0/1118345871 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d05bc3c0 con 0x5613d08e5590 2024-09-19T04:51:31.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.664+0000 7f2865600640 1 --2- 172.21.15.123:0/1118345871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e4c70 0x5613d08ecc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:31.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.664+0000 7f2864c00640 1 --2- 172.21.15.123:0/1118345871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 0x5613d08ed140 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f284c0cd640 tx=0x7f284c00cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35acdc6ff8c5c2bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:31.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.664+0000 7f2846a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:51:31.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.665+0000 7f2846a00640 1 -- 172.21.15.123:0/1118345871 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c0cdac0 con 0x5613d08e5590 2024-09-19T04:51:31.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.665+0000 7f2846a00640 1 -- 172.21.15.123:0/1118345871 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f284c0a0120 con 0x5613d08e5590 2024-09-19T04:51:31.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.665+0000 7f286ac5d740 1 -- 172.21.15.123:0/1118345871 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f285400fba0 con 0x5613d08e5590 2024-09-19T04:51:31.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.665+0000 7f2846a00640 1 -- 172.21.15.123:0/1118345871 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f284c0a4e40 con 0x5613d08e5590 2024-09-19T04:51:31.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.665+0000 7f286ac5d740 10 client.26177 fetch_fsmap learned FSMap version 36 2024-09-19T04:51:31.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.665+0000 7f286ac5d740 1 -- 172.21.15.123:0/1118345871 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5613d08fe970 con 0x5613d08e5590 2024-09-19T04:51:31.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.665+0000 7f2846a00640 1 -- 172.21.15.123:0/1118345871 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f285400fba0 con 0x5613d08e5590 2024-09-19T04:51:31.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.666+0000 7f2846a00640 1 -- 172.21.15.123:0/1118345871 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f284c09b760 con 0x5613d08e5590 2024-09-19T04:51:31.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.666+0000 7f286ac5d740 10 client.26177 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:51:31.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.666+0000 7f286ac5d740 10 client.26177 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:51:31.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.666+0000 7f286ac5d740 20 client.26177 populate_metadata read hostname 'smithi123' 2024-09-19T04:51:31.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.666+0000 7f286ac5d740 1 --2- 172.21.15.123:0/1118345871 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x5613d08ffd80 0x5613d0902240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.666+0000 7f286ac5d740 4 client.26177 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:51:31.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.666+0000 7f286ac5d740 1 -- 172.21.15.123:0/1118345871 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613d0858950 con 0x5613d08ffd80 2024-09-19T04:51:31.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.666+0000 7f2865600640 1 --2- 172.21.15.123:0/1118345871 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x5613d08ffd80 0x5613d0902240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.667+0000 7f2846a00640 10 client.26177 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:51:31.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.667+0000 7f2865600640 1 --2- 172.21.15.123:0/1118345871 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x5613d08ffd80 0x5613d0902240 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d25cd2114e5715d4 server_cookie=9b22c4e722d1cc18 in_seq=0 out_seq=0 2024-09-19T04:51:31.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.667+0000 7f2846a00640 1 -- 172.21.15.123:0/1118345871 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613d0858950 con 0x5613d08ffd80 2024-09-19T04:51:31.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.667+0000 7f2846a00640 10 client.26177 handle_command_reply: tid=0 2024-09-19T04:51:31.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.667+0000 7f286ac5d740 1 client.26177 shutdown 2024-09-19T04:51:31.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.668+0000 7f286ac5d740 1 -- 172.21.15.123:0/1118345871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 msgr2=0x5613d08ed140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:31.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.668+0000 7f286ac5d740 1 --2- 172.21.15.123:0/1118345871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 0x5613d08ed140 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f284c0cd640 tx=0x7f284c00cf50 comp rx=0 tx=0).stop 2024-09-19T04:51:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.668+0000 7f286ac5d740 1 -- 172.21.15.123:0/1118345871 shutdown_connections 2024-09-19T04:51:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.668+0000 7f286ac5d740 1 --2- 172.21.15.123:0/1118345871 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x5613d08ffd80 0x5613d0902240 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.668+0000 7f286ac5d740 1 --2- 172.21.15.123:0/1118345871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08efe50 0x5613d08ed680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.668+0000 7f286ac5d740 1 --2- 172.21.15.123:0/1118345871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 0x5613d08ed140 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.668+0000 7f286ac5d740 1 --2- 172.21.15.123:0/1118345871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e4c70 0x5613d08ecc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.668+0000 7f286ac5d740 1 -- 172.21.15.123:0/1118345871 >> 172.21.15.123:0/1118345871 conn(0x5613d08e1200 msgr2=0x5613d08e6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.668+0000 7f2846a00640 0 client.26177 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:51:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.668+0000 7f286ac5d740 1 -- 172.21.15.123:0/1118345871 shutdown_connections 2024-09-19T04:51:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.668+0000 7f286ac5d740 1 -- 172.21.15.123:0/1118345871 wait complete. 2024-09-19T04:51:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.668+0000 7f286ac5d740 20 client.26177 trim_cache size 0 max 16384 2024-09-19T04:51:31.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.682+0000 7f286ac5d740 1 Processor -- start 2024-09-19T04:51:31.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.682+0000 7f286ac5d740 1 -- start start 2024-09-19T04:51:31.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.683+0000 7f286ac5d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e4c70 0x5613d08eb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.683+0000 7f286ac5d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 0x5613d0907b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.683+0000 7f286ac5d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08efe50 0x5613d0908050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.683+0000 7f286ac5d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f284000e6d0 con 0x5613d08efe50 2024-09-19T04:51:31.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.683+0000 7f286ac5d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2840010210 con 0x5613d08e5590 2024-09-19T04:51:31.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.683+0000 7f286ac5d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f284000eda0 con 0x5613d08e4c70 2024-09-19T04:51:31.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.683+0000 7f2865600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e4c70 0x5613d08eb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.683+0000 7f2866000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08efe50 0x5613d0908050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.683+0000 7f2864c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 0x5613d0907b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.683+0000 7f2865600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e4c70 0x5613d08eb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:58852/0 (socket says 172.21.15.123:58852) 2024-09-19T04:51:31.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.683+0000 7f2865600640 1 -- 172.21.15.123:0/2278538421 learned_addr learned my addr 172.21.15.123:0/2278538421 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:31.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.684+0000 7f2865600640 1 -- 172.21.15.123:0/2278538421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 msgr2=0x5613d0907b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:31.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.684+0000 7f2865600640 1 --2- 172.21.15.123:0/2278538421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 0x5613d0907b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.684+0000 7f2865600640 1 -- 172.21.15.123:0/2278538421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08efe50 msgr2=0x5613d0908050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:31.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.684+0000 7f2865600640 1 --2- 172.21.15.123:0/2278538421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08efe50 0x5613d0908050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.684+0000 7f2865600640 1 -- 172.21.15.123:0/2278538421 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d08e1030 con 0x5613d08e4c70 2024-09-19T04:51:31.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.684+0000 7f2865600640 1 --2- 172.21.15.123:0/2278538421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e4c70 0x5613d08eb4d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f28540097f0 tx=0x7f2854005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=876a8472a693370b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:31.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.684+0000 7f2851c00640 1 -- 172.21.15.123:0/2278538421 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285401e040 con 0x5613d08e4c70 2024-09-19T04:51:31.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.684+0000 7f2851c00640 1 -- 172.21.15.123:0/2278538421 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28540055a0 con 0x5613d08e4c70 2024-09-19T04:51:31.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.684+0000 7f2851c00640 1 -- 172.21.15.123:0/2278538421 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854019c30 con 0x5613d08e4c70 2024-09-19T04:51:31.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.685+0000 7f286ac5d740 1 -- 172.21.15.123:0/2278538421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e4c70 msgr2=0x5613d08eb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:31.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.685+0000 7f286ac5d740 1 --2- 172.21.15.123:0/2278538421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e4c70 0x5613d08eb4d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f28540097f0 tx=0x7f2854005d30 comp rx=0 tx=0).stop 2024-09-19T04:51:31.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.685+0000 7f286ac5d740 1 -- 172.21.15.123:0/2278538421 shutdown_connections 2024-09-19T04:51:31.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.685+0000 7f286ac5d740 1 --2- 172.21.15.123:0/2278538421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08efe50 0x5613d0908050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.685+0000 7f286ac5d740 1 --2- 172.21.15.123:0/2278538421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 0x5613d0907b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.685+0000 7f286ac5d740 1 --2- 172.21.15.123:0/2278538421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08e4c70 0x5613d08eb4d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.685+0000 7f286ac5d740 1 -- 172.21.15.123:0/2278538421 >> 172.21.15.123:0/2278538421 conn(0x5613d08e1200 msgr2=0x5613d08eec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:31.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.685+0000 7f286ac5d740 1 -- 172.21.15.123:0/2278538421 shutdown_connections 2024-09-19T04:51:31.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.685+0000 7f286ac5d740 1 -- 172.21.15.123:0/2278538421 wait complete. 2024-09-19T04:51:31.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.686+0000 7f286ac5d740 1 -- start start 2024-09-19T04:51:31.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.686+0000 7f286ac5d740 1 Processor -- start 2024-09-19T04:51:31.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.686+0000 7f286ac5d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08e4c70 0x5613d08f8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.686+0000 7f286ac5d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 0x5613d08f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.686+0000 7f286ac5d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08efe50 0x5613d08f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.686+0000 7f286ac5d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f284000fb40 con 0x5613d08e4c70 2024-09-19T04:51:31.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.686+0000 7f286ac5d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f284000f470 con 0x5613d08e5590 2024-09-19T04:51:31.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.686+0000 7f286ac5d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f284000c6b0 con 0x5613d08efe50 2024-09-19T04:51:31.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.686+0000 7f286ac5d740 10 client.0 _ll_register_callbacks cb 0x7f284c098520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:51:31.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.686+0000 7f2864c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 0x5613d08f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.687+0000 7f2864c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 0x5613d08f86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:42952/0 (socket says 172.21.15.123:42952) 2024-09-19T04:51:31.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.687+0000 7f2864c00640 1 -- 172.21.15.123:0/332844416 learned_addr learned my addr 172.21.15.123:0/332844416 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:31.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.687+0000 7f2866000640 1 --2- 172.21.15.123:0/332844416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08efe50 0x5613d08f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.687+0000 7f2865600640 1 --2- 172.21.15.123:0/332844416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08e4c70 0x5613d08f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.687+0000 7f2864c00640 1 -- 172.21.15.123:0/332844416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08efe50 msgr2=0x5613d08f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:31.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.687+0000 7f2864c00640 1 --2- 172.21.15.123:0/332844416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08efe50 0x5613d08f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.687+0000 7f2864c00640 1 -- 172.21.15.123:0/332844416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08e4c70 msgr2=0x5613d08f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:31.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.687+0000 7f2864c00640 1 --2- 172.21.15.123:0/332844416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08e4c70 0x5613d08f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.687+0000 7f2864c00640 1 -- 172.21.15.123:0/332844416 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d08dc770 con 0x5613d08e5590 2024-09-19T04:51:31.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.687+0000 7f2864c00640 1 --2- 172.21.15.123:0/332844416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 0x5613d08f86b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f284c00c6a0 tx=0x7f284c00ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a26f8ea60a40e1cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:31.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.687+0000 7f2846a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:51:31.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.687+0000 7f2846a00640 1 -- 172.21.15.123:0/332844416 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c09bbe0 con 0x5613d08e5590 2024-09-19T04:51:31.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.687+0000 7f286ac5d740 1 -- 172.21.15.123:0/332844416 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28540055a0 con 0x5613d08e5590 2024-09-19T04:51:31.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.687+0000 7f2846a00640 1 -- 172.21.15.123:0/332844416 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f284c0a0320 con 0x5613d08e5590 2024-09-19T04:51:31.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.688+0000 7f2846a00640 1 -- 172.21.15.123:0/332844416 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f284c081050 con 0x5613d08e5590 2024-09-19T04:51:31.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.688+0000 7f286ac5d740 10 client.26183 fetch_fsmap learned FSMap version 36 2024-09-19T04:51:31.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.688+0000 7f286ac5d740 1 -- 172.21.15.123:0/332844416 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5613d096b4e0 con 0x5613d08e5590 2024-09-19T04:51:31.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.688+0000 7f2846a00640 1 -- 172.21.15.123:0/332844416 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28540055a0 con 0x5613d08e5590 2024-09-19T04:51:31.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.688+0000 7f2846a00640 1 -- 172.21.15.123:0/332844416 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f284c098060 con 0x5613d08e5590 2024-09-19T04:51:31.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.688+0000 7f286ac5d740 10 client.26183 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:51:31.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.688+0000 7f286ac5d740 10 client.26183 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:51:31.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.688+0000 7f286ac5d740 20 client.26183 populate_metadata read hostname 'smithi123' 2024-09-19T04:51:31.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.688+0000 7f286ac5d740 1 --2- 172.21.15.123:0/332844416 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x5613d096bc60 0x5613d096e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:31.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.688+0000 7f286ac5d740 4 client.26183 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:51:31.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.688+0000 7f286ac5d740 1 -- 172.21.15.123:0/332844416 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5613d0852e60 con 0x5613d096bc60 2024-09-19T04:51:31.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.688+0000 7f2865600640 1 --2- 172.21.15.123:0/332844416 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x5613d096bc60 0x5613d096e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:31.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.689+0000 7f2865600640 1 --2- 172.21.15.123:0/332844416 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x5613d096bc60 0x5613d096e080 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a06e85986e5217b0 server_cookie=75fc8013d089030d in_seq=0 out_seq=0 2024-09-19T04:51:31.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.689+0000 7f2846a00640 10 client.26183 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:51:31.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.689+0000 7f2846a00640 1 -- 172.21.15.123:0/332844416 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5613d0852e60 con 0x5613d096bc60 2024-09-19T04:51:31.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.689+0000 7f2846a00640 10 client.26183 handle_command_reply: tid=0 2024-09-19T04:51:31.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.689+0000 7f286ac5d740 1 client.26183 shutdown 2024-09-19T04:51:31.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.689+0000 7f286ac5d740 1 -- 172.21.15.123:0/332844416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 msgr2=0x5613d08f86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:31.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.689+0000 7f286ac5d740 1 --2- 172.21.15.123:0/332844416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 0x5613d08f86b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f284c00c6a0 tx=0x7f284c00ce60 comp rx=0 tx=0).stop 2024-09-19T04:51:31.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.689+0000 7f286ac5d740 1 -- 172.21.15.123:0/332844416 shutdown_connections 2024-09-19T04:51:31.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.689+0000 7f286ac5d740 1 --2- 172.21.15.123:0/332844416 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x5613d096bc60 0x5613d096e080 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.689+0000 7f286ac5d740 1 --2- 172.21.15.123:0/332844416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613d08efe50 0x5613d08f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.689+0000 7f286ac5d740 1 --2- 172.21.15.123:0/332844416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5613d08e5590 0x5613d08f86b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.689+0000 7f286ac5d740 1 --2- 172.21.15.123:0/332844416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613d08e4c70 0x5613d08f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.689+0000 7f2846a00640 0 client.26183 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:51:31.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.689+0000 7f286ac5d740 1 -- 172.21.15.123:0/332844416 >> 172.21.15.123:0/332844416 conn(0x5613d08e1200 msgr2=0x5613d08dfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:31.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.689+0000 7f286ac5d740 1 -- 172.21.15.123:0/332844416 shutdown_connections 2024-09-19T04:51:31.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.690+0000 7f286ac5d740 1 -- 172.21.15.123:0/332844416 wait complete. 2024-09-19T04:51:31.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.690+0000 7f286ac5d740 20 client.26183 trim_cache size 0 max 16384 2024-09-19T04:51:31.695 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:51:31.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.690+0000 7f2868000640 1 -- 172.21.15.123:0/2379651169 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f28480a33b0 msgr2=0x7f28480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:31.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.690+0000 7f2868000640 1 --2- 172.21.15.123:0/2379651169 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f28480a33b0 0x7f28480a5870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2858006010 tx=0x7f2858005d80 comp rx=0 tx=0).stop 2024-09-19T04:51:31.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.690+0000 7f2868000640 1 -- 172.21.15.123:0/2379651169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2860102780 msgr2=0x7f2860074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:31.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.690+0000 7f2868000640 1 --2- 172.21.15.123:0/2379651169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2860102780 0x7f2860074a30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f284c00e9f0 tx=0x7f284c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.690+0000 7f2868000640 1 -- 172.21.15.123:0/2379651169 shutdown_connections 2024-09-19T04:51:31.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.690+0000 7f2868000640 1 --2- 172.21.15.123:0/2379651169 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f28480a33b0 0x7f28480a5870 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.690+0000 7f2868000640 1 --2- 172.21.15.123:0/2379651169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2860106f00 0x7f2860074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.690+0000 7f2868000640 1 --2- 172.21.15.123:0/2379651169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2860102780 0x7f2860074a30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.690+0000 7f2868000640 1 --2- 172.21.15.123:0/2379651169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2860101dd0 0x7f28600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:31.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.690+0000 7f2868000640 1 -- 172.21.15.123:0/2379651169 >> 172.21.15.123:0/2379651169 conn(0x7f28600fd890 msgr2=0x7f28600fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:31.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.691+0000 7f2868000640 1 -- 172.21.15.123:0/2379651169 shutdown_connections 2024-09-19T04:51:31.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:31.691+0000 7f2868000640 1 -- 172.21.15.123:0/2379651169 wait complete. 2024-09-19T04:51:32.605 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:51:33.062 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:51:33.349 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:33 smithi183 ceph-mon[35333]: pgmap v148: 97 pgs: 97 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 540 op/s 2024-09-19T04:51:33.370 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:33 smithi123 ceph-mon[30511]: pgmap v148: 97 pgs: 97 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 540 op/s 2024-09-19T04:51:33.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:33 smithi193 ceph-mon[35592]: pgmap v148: 97 pgs: 97 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 540 op/s 2024-09-19T04:51:33.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.658+0000 7fdb65c00640 1 -- 172.21.15.123:0/117716161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb6010e4c0 msgr2=0x7fdb601108b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:33.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.658+0000 7fdb65c00640 1 --2- 172.21.15.123:0/117716161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb6010e4c0 0x7fdb601108b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdb58009ca0 tx=0x7fdb5802a8b0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.659+0000 7fdb65c00640 1 -- 172.21.15.123:0/117716161 shutdown_connections 2024-09-19T04:51:33.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.659+0000 7fdb65c00640 1 --2- 172.21.15.123:0/117716161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb6010e4c0 0x7fdb601108b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.659+0000 7fdb65c00640 1 --2- 172.21.15.123:0/117716161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb601012c0 0x7fdb6010df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.659+0000 7fdb65c00640 1 --2- 172.21.15.123:0/117716161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb60100910 0x7fdb60100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.659+0000 7fdb65c00640 1 -- 172.21.15.123:0/117716161 >> 172.21.15.123:0/117716161 conn(0x7fdb600fc780 msgr2=0x7fdb600feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:33.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.659+0000 7fdb65c00640 1 -- 172.21.15.123:0/117716161 shutdown_connections 2024-09-19T04:51:33.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.659+0000 7fdb65c00640 1 -- 172.21.15.123:0/117716161 wait complete. 2024-09-19T04:51:33.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.660+0000 7fdb65c00640 1 Processor -- start 2024-09-19T04:51:33.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.660+0000 7fdb65c00640 1 -- start start 2024-09-19T04:51:33.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.660+0000 7fdb65c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb60100910 0x7fdb6019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.660+0000 7fdb65c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb601012c0 0x7fdb6019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.660+0000 7fdb65c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb6010e4c0 0x7fdb6019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.660+0000 7fdb65c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb601141a0 con 0x7fdb60100910 2024-09-19T04:51:33.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.660+0000 7fdb65c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb60114020 con 0x7fdb6010e4c0 2024-09-19T04:51:33.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.660+0000 7fdb65c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb60114320 con 0x7fdb601012c0 2024-09-19T04:51:33.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.660+0000 7fdb5e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb601012c0 0x7fdb6019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.660+0000 7fdb5e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb601012c0 0x7fdb6019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53544/0 (socket says 172.21.15.123:53544) 2024-09-19T04:51:33.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.660+0000 7fdb5e000640 1 -- 172.21.15.123:0/2080400672 learned_addr learned my addr 172.21.15.123:0/2080400672 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:33.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.660+0000 7fdb5ea00640 1 --2- 172.21.15.123:0/2080400672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb60100910 0x7fdb6019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.660+0000 7fdb5f400640 1 --2- 172.21.15.123:0/2080400672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb6010e4c0 0x7fdb6019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.661+0000 7fdb5e000640 1 -- 172.21.15.123:0/2080400672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb6010e4c0 msgr2=0x7fdb6019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:33.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.661+0000 7fdb5e000640 1 --2- 172.21.15.123:0/2080400672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb6010e4c0 0x7fdb6019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.661+0000 7fdb5e000640 1 -- 172.21.15.123:0/2080400672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb60100910 msgr2=0x7fdb6019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:33.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.661+0000 7fdb5e000640 1 --2- 172.21.15.123:0/2080400672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb60100910 0x7fdb6019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.661+0000 7fdb5e000640 1 -- 172.21.15.123:0/2080400672 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6019f5f0 con 0x7fdb601012c0 2024-09-19T04:51:33.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.661+0000 7fdb5e000640 1 --2- 172.21.15.123:0/2080400672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb601012c0 0x7fdb6019ab80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00e9f0 tx=0x7fdb4c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:33.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.661+0000 7fdb57400640 1 -- 172.21.15.123:0/2080400672 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c00cd80 con 0x7fdb601012c0 2024-09-19T04:51:33.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.661+0000 7fdb65c00640 1 -- 172.21.15.123:0/2080400672 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb6019f8e0 con 0x7fdb601012c0 2024-09-19T04:51:33.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.661+0000 7fdb57400640 1 -- 172.21.15.123:0/2080400672 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb4c005500 con 0x7fdb601012c0 2024-09-19T04:51:33.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.662+0000 7fdb57400640 1 -- 172.21.15.123:0/2080400672 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c010690 con 0x7fdb601012c0 2024-09-19T04:51:33.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.662+0000 7fdb65c00640 1 -- 172.21.15.123:0/2080400672 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb601a71c0 con 0x7fdb601012c0 2024-09-19T04:51:33.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.665+0000 7fdb6887a740 1 Processor -- start 2024-09-19T04:51:33.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.666+0000 7fdb6887a740 1 -- start start 2024-09-19T04:51:33.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.666+0000 7fdb57400640 1 -- 172.21.15.123:0/2080400672 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fdb4c0046c0 con 0x7fdb601012c0 2024-09-19T04:51:33.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.666+0000 7fdb6887a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eacc70 0x5626a1ead050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.666+0000 7fdb6887a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1ead590 0x5626a1eb7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.666+0000 7fdb6887a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eb7e50 0x5626a1eba240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.666+0000 7fdb6887a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5626a1e133e0 con 0x5626a1ead590 2024-09-19T04:51:33.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.666+0000 7fdb6887a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5626a1dbad80 con 0x5626a1eacc70 2024-09-19T04:51:33.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.666+0000 7fdb6887a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5626a1e0f3a0 con 0x5626a1eb7e50 2024-09-19T04:51:33.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.666+0000 7fdb5f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eacc70 0x5626a1ead050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.666+0000 7fdb5f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eacc70 0x5626a1ead050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:37612/0 (socket says 172.21.15.123:37612) 2024-09-19T04:51:33.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.666+0000 7fdb5f400640 1 -- 172.21.15.123:0/3011477512 learned_addr learned my addr 172.21.15.123:0/3011477512 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:33.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.666+0000 7fdb5e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eb7e50 0x5626a1eba240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.667+0000 7fdb5ea00640 1 --2- 172.21.15.123:0/3011477512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1ead590 0x5626a1eb7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.667+0000 7fdb5f400640 1 -- 172.21.15.123:0/3011477512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eb7e50 msgr2=0x5626a1eba240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:33.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.667+0000 7fdb5f400640 1 --2- 172.21.15.123:0/3011477512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eb7e50 0x5626a1eba240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.667+0000 7fdb5f400640 1 -- 172.21.15.123:0/3011477512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1ead590 msgr2=0x5626a1eb7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:33.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.667+0000 7fdb5f400640 1 --2- 172.21.15.123:0/3011477512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1ead590 0x5626a1eb7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.667+0000 7fdb5f400640 1 -- 172.21.15.123:0/3011477512 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626a1dd9ae0 con 0x5626a1eacc70 2024-09-19T04:51:33.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.667+0000 7fdb5e000640 1 --2- 172.21.15.123:0/3011477512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eb7e50 0x5626a1eba240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:51:33.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.667+0000 7fdb5ea00640 1 --2- 172.21.15.123:0/3011477512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1ead590 0x5626a1eb7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:51:33.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.667+0000 7fdb5f400640 1 --2- 172.21.15.123:0/3011477512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eacc70 0x5626a1ead050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdb58009ca0 tx=0x7fdb58005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68c710932707b736 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:33.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.667+0000 7fdb4be00640 1 -- 172.21.15.123:0/3011477512 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58030760 con 0x5626a1eacc70 2024-09-19T04:51:33.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.668+0000 7fdb4be00640 1 -- 172.21.15.123:0/3011477512 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb58036030 con 0x5626a1eacc70 2024-09-19T04:51:33.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.668+0000 7fdb4be00640 1 -- 172.21.15.123:0/3011477512 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb580043e0 con 0x5626a1eacc70 2024-09-19T04:51:33.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.668+0000 7fdb57400640 1 --2- 172.21.15.123:0/2080400672 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb440a33b0 0x7fdb440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.668+0000 7fdb57400640 1 -- 172.21.15.123:0/2080400672 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fdb4c0d00d0 con 0x7fdb601012c0 2024-09-19T04:51:33.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.668+0000 7fdb5ea00640 1 --2- 172.21.15.123:0/2080400672 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb440a33b0 0x7fdb440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.668+0000 7fdb6887a740 1 -- 172.21.15.123:0/3011477512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eacc70 msgr2=0x5626a1ead050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:33.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.669+0000 7fdb6887a740 1 --2- 172.21.15.123:0/3011477512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eacc70 0x5626a1ead050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdb58009ca0 tx=0x7fdb58005f80 comp rx=0 tx=0).stop 2024-09-19T04:51:33.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.669+0000 7fdb6887a740 1 -- 172.21.15.123:0/3011477512 shutdown_connections 2024-09-19T04:51:33.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.669+0000 7fdb6887a740 1 --2- 172.21.15.123:0/3011477512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eb7e50 0x5626a1eba240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.669+0000 7fdb6887a740 1 --2- 172.21.15.123:0/3011477512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1ead590 0x5626a1eb7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.669+0000 7fdb6887a740 1 --2- 172.21.15.123:0/3011477512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eacc70 0x5626a1ead050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.669+0000 7fdb6887a740 1 -- 172.21.15.123:0/3011477512 >> 172.21.15.123:0/3011477512 conn(0x5626a1ea9200 msgr2=0x5626a1ea95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:33.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.669+0000 7fdb6887a740 1 -- 172.21.15.123:0/3011477512 shutdown_connections 2024-09-19T04:51:33.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.669+0000 7fdb5ea00640 1 --2- 172.21.15.123:0/2080400672 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb440a33b0 0x7fdb440a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x5626a1eaa060 tx=0x7fdb50005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:33.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.669+0000 7fdb6887a740 1 -- 172.21.15.123:0/3011477512 wait complete. 2024-09-19T04:51:33.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.670+0000 7fdb6887a740 1 -- start start 2024-09-19T04:51:33.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.670+0000 7fdb6887a740 1 Processor -- start 2024-09-19T04:51:33.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.671+0000 7fdb6887a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eacc70 0x5626a1eb4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.671+0000 7fdb6887a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1ead590 0x5626a1eb5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.671+0000 7fdb6887a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1eb7e50 0x5626a1eb5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.671+0000 7fdb6887a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5626a1ebaa20 con 0x5626a1eb7e50 2024-09-19T04:51:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.671+0000 7fdb6887a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5626a1dce0f0 con 0x5626a1eacc70 2024-09-19T04:51:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.671+0000 7fdb6887a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5626a1ebaba0 con 0x5626a1ead590 2024-09-19T04:51:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.671+0000 7fdb6887a740 10 client.0 _ll_register_callbacks cb 0x5626a1db5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:51:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.671+0000 7fdb5e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1ead590 0x5626a1eb5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.671+0000 7fdb5e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1ead590 0x5626a1eb5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53558/0 (socket says 172.21.15.123:53558) 2024-09-19T04:51:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.671+0000 7fdb5e000640 1 -- 172.21.15.123:0/1394708366 learned_addr learned my addr 172.21.15.123:0/1394708366 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.671+0000 7fdb5f400640 1 --2- 172.21.15.123:0/1394708366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1eb7e50 0x5626a1eb5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.671+0000 7fdb5ea00640 1 --2- 172.21.15.123:0/1394708366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eacc70 0x5626a1eb4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.671+0000 7fdb5e000640 1 -- 172.21.15.123:0/1394708366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eacc70 msgr2=0x5626a1eb4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:33.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.672+0000 7fdb5e000640 1 --2- 172.21.15.123:0/1394708366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eacc70 0x5626a1eb4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.672+0000 7fdb5e000640 1 -- 172.21.15.123:0/1394708366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1eb7e50 msgr2=0x5626a1eb5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:33.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.672+0000 7fdb5e000640 1 --2- 172.21.15.123:0/1394708366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1eb7e50 0x5626a1eb5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.672+0000 7fdb5e000640 1 -- 172.21.15.123:0/1394708366 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626a1b843c0 con 0x5626a1ead590 2024-09-19T04:51:33.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.672+0000 7fdb5f400640 1 --2- 172.21.15.123:0/1394708366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1eb7e50 0x5626a1eb5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:51:33.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.672+0000 7fdb5ea00640 1 --2- 172.21.15.123:0/1394708366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eacc70 0x5626a1eb4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:51:33.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.672+0000 7fdb5e000640 1 --2- 172.21.15.123:0/1394708366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1ead590 0x5626a1eb5240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c09c310 tx=0x7fdb4c004ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2baaa620f281a70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:33.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.672+0000 7fdb48c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:51:33.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.672+0000 7fdb48c00640 1 -- 172.21.15.123:0/1394708366 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c09bce0 con 0x5626a1ead590 2024-09-19T04:51:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.672+0000 7fdb48c00640 1 -- 172.21.15.123:0/1394708366 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fdb4c0a4b60 con 0x5626a1ead590 2024-09-19T04:51:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.672+0000 7fdb6887a740 1 -- 172.21.15.123:0/1394708366 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb58036030 con 0x5626a1ead590 2024-09-19T04:51:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.673+0000 7fdb48c00640 1 -- 172.21.15.123:0/1394708366 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdb4c0cdc80 con 0x5626a1ead590 2024-09-19T04:51:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.673+0000 7fdb6887a740 10 client.26215 fetch_fsmap learned FSMap version 36 2024-09-19T04:51:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.673+0000 7fdb6887a740 1 -- 172.21.15.123:0/1394708366 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5626a1ec6a50 con 0x5626a1ead590 2024-09-19T04:51:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.673+0000 7fdb48c00640 1 -- 172.21.15.123:0/1394708366 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb58036030 con 0x5626a1ead590 2024-09-19T04:51:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.673+0000 7fdb48c00640 1 -- 172.21.15.123:0/1394708366 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fdb4c0b5150 con 0x5626a1ead590 2024-09-19T04:51:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.673+0000 7fdb6887a740 10 client.26215 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:51:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.673+0000 7fdb6887a740 10 client.26215 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:51:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.673+0000 7fdb6887a740 20 client.26215 populate_metadata read hostname 'smithi123' 2024-09-19T04:51:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.673+0000 7fdb6887a740 1 --2- 172.21.15.123:0/1394708366 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5626a1ec7f10 0x5626a1eca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.673+0000 7fdb6887a740 4 client.26215 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:51:33.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.673+0000 7fdb6887a740 1 -- 172.21.15.123:0/1394708366 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626a1e20950 con 0x5626a1ec7f10 2024-09-19T04:51:33.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.674+0000 7fdb5ea00640 1 --2- 172.21.15.123:0/1394708366 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5626a1ec7f10 0x5626a1eca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.675+0000 7fdb48c00640 10 client.26215 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:51:33.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.675+0000 7fdb5ea00640 1 --2- 172.21.15.123:0/1394708366 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5626a1ec7f10 0x5626a1eca300 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a9daa912fdbc53e9 server_cookie=8e458778346dee2a in_seq=0 out_seq=0 2024-09-19T04:51:33.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.675+0000 7fdb48c00640 1 -- 172.21.15.123:0/1394708366 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626a1e20950 con 0x5626a1ec7f10 2024-09-19T04:51:33.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.675+0000 7fdb48c00640 10 client.26215 handle_command_reply: tid=0 2024-09-19T04:51:33.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.675+0000 7fdb6887a740 1 client.26215 shutdown 2024-09-19T04:51:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.676+0000 7fdb6887a740 1 -- 172.21.15.123:0/1394708366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1ead590 msgr2=0x5626a1eb5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.676+0000 7fdb6887a740 1 --2- 172.21.15.123:0/1394708366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1ead590 0x5626a1eb5240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c09c310 tx=0x7fdb4c004ba0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.676+0000 7fdb6887a740 1 -- 172.21.15.123:0/1394708366 shutdown_connections 2024-09-19T04:51:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.676+0000 7fdb6887a740 1 --2- 172.21.15.123:0/1394708366 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5626a1ec7f10 0x5626a1eca300 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.676+0000 7fdb6887a740 1 --2- 172.21.15.123:0/1394708366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1eb7e50 0x5626a1eb5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.676+0000 7fdb6887a740 1 --2- 172.21.15.123:0/1394708366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1ead590 0x5626a1eb5240 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.676+0000 7fdb6887a740 1 --2- 172.21.15.123:0/1394708366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eacc70 0x5626a1eb4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.676+0000 7fdb6887a740 1 -- 172.21.15.123:0/1394708366 >> 172.21.15.123:0/1394708366 conn(0x5626a1ea9200 msgr2=0x5626a1eaef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.676+0000 7fdb48c00640 0 client.26215 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:51:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.676+0000 7fdb6887a740 1 -- 172.21.15.123:0/1394708366 shutdown_connections 2024-09-19T04:51:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.676+0000 7fdb6887a740 1 -- 172.21.15.123:0/1394708366 wait complete. 2024-09-19T04:51:33.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.676+0000 7fdb6887a740 20 client.26215 trim_cache size 0 max 16384 2024-09-19T04:51:33.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.699+0000 7fdb6887a740 1 Processor -- start 2024-09-19T04:51:33.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.699+0000 7fdb6887a740 1 -- start start 2024-09-19T04:51:33.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.699+0000 7fdb6887a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eacc70 0x5626a1eb6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.699+0000 7fdb6887a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1ead590 0x5626a1ecfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.699+0000 7fdb6887a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eb7e50 0x5626a1ed0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.699+0000 7fdb6887a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb601af970 con 0x5626a1ead590 2024-09-19T04:51:33.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.699+0000 7fdb6887a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb601b14b0 con 0x5626a1eb7e50 2024-09-19T04:51:33.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.699+0000 7fdb6887a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb601b0040 con 0x5626a1eacc70 2024-09-19T04:51:33.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.700+0000 7fdb5ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eacc70 0x5626a1eb6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.700+0000 7fdb5f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eb7e50 0x5626a1ed0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.700+0000 7fdb5e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1ead590 0x5626a1ecfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.700+0000 7fdb5ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eacc70 0x5626a1eb6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53570/0 (socket says 172.21.15.123:53570) 2024-09-19T04:51:33.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.700+0000 7fdb5ea00640 1 -- 172.21.15.123:0/4183039993 learned_addr learned my addr 172.21.15.123:0/4183039993 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:33.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.700+0000 7fdb5f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eb7e50 0x5626a1ed0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:37622/0 (socket says 172.21.15.123:37622) 2024-09-19T04:51:33.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.700+0000 7fdb5ea00640 1 -- 172.21.15.123:0/4183039993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eb7e50 msgr2=0x5626a1ed0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:33.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.700+0000 7fdb5ea00640 1 --2- 172.21.15.123:0/4183039993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eb7e50 0x5626a1ed0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.700+0000 7fdb5ea00640 1 -- 172.21.15.123:0/4183039993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1ead590 msgr2=0x5626a1ecfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:33.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.700+0000 7fdb5ea00640 1 --2- 172.21.15.123:0/4183039993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1ead590 0x5626a1ecfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.700+0000 7fdb5ea00640 1 -- 172.21.15.123:0/4183039993 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626a1ea4770 con 0x5626a1eacc70 2024-09-19T04:51:33.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.700+0000 7fdb5e000640 1 --2- 172.21.15.123:0/4183039993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1ead590 0x5626a1ecfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:33.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.700+0000 7fdb5ea00640 1 --2- 172.21.15.123:0/4183039993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eacc70 0x5626a1eb6c70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009520 tx=0x7fdb50016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=868c9d82d941a4c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:33.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.701+0000 7fdb4b400640 1 -- 172.21.15.123:0/4183039993 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5001d510 con 0x5626a1eacc70 2024-09-19T04:51:33.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.701+0000 7fdb4b400640 1 -- 172.21.15.123:0/4183039993 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb5001db90 con 0x5626a1eacc70 2024-09-19T04:51:33.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.701+0000 7fdb4b400640 1 -- 172.21.15.123:0/4183039993 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb50028c00 con 0x5626a1eacc70 2024-09-19T04:51:33.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.702+0000 7fdb6887a740 1 -- 172.21.15.123:0/4183039993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eacc70 msgr2=0x5626a1eb6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:33.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.702+0000 7fdb6887a740 1 --2- 172.21.15.123:0/4183039993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eacc70 0x5626a1eb6c70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009520 tx=0x7fdb50016000 comp rx=0 tx=0).stop 2024-09-19T04:51:33.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.702+0000 7fdb6887a740 1 -- 172.21.15.123:0/4183039993 shutdown_connections 2024-09-19T04:51:33.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.702+0000 7fdb6887a740 1 --2- 172.21.15.123:0/4183039993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1eb7e50 0x5626a1ed0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.702+0000 7fdb6887a740 1 --2- 172.21.15.123:0/4183039993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1ead590 0x5626a1ecfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.702+0000 7fdb6887a740 1 --2- 172.21.15.123:0/4183039993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eacc70 0x5626a1eb6c70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.702+0000 7fdb6887a740 1 -- 172.21.15.123:0/4183039993 >> 172.21.15.123:0/4183039993 conn(0x5626a1ea9200 msgr2=0x5626a1eb6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:33.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.702+0000 7fdb6887a740 1 -- 172.21.15.123:0/4183039993 shutdown_connections 2024-09-19T04:51:33.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.702+0000 7fdb6887a740 1 -- 172.21.15.123:0/4183039993 wait complete. 2024-09-19T04:51:33.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.702+0000 7fdb6887a740 1 -- start start 2024-09-19T04:51:33.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.703+0000 7fdb6887a740 1 Processor -- start 2024-09-19T04:51:33.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.703+0000 7fdb6887a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1eacc70 0x5626a1eb1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.703+0000 7fdb6887a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1ead590 0x5626a1eb2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.703+0000 7fdb6887a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eb7e50 0x5626a1eb29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.703+0000 7fdb6887a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb601b0de0 con 0x5626a1eacc70 2024-09-19T04:51:33.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.703+0000 7fdb6887a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb601b0710 con 0x5626a1ead590 2024-09-19T04:51:33.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.703+0000 7fdb6887a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb601ad950 con 0x5626a1eb7e50 2024-09-19T04:51:33.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.703+0000 7fdb6887a740 10 client.0 _ll_register_callbacks cb 0x7fdb4c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:51:33.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.703+0000 7fdb5e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1ead590 0x5626a1eb2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.703+0000 7fdb5e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1ead590 0x5626a1eb2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:37634/0 (socket says 172.21.15.123:37634) 2024-09-19T04:51:33.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.703+0000 7fdb5e000640 1 -- 172.21.15.123:0/1236498292 learned_addr learned my addr 172.21.15.123:0/1236498292 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:33.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.703+0000 7fdb5ea00640 1 --2- 172.21.15.123:0/1236498292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1eacc70 0x5626a1eb1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.703+0000 7fdb5f400640 1 --2- 172.21.15.123:0/1236498292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eb7e50 0x5626a1eb29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.704+0000 7fdb5e000640 1 -- 172.21.15.123:0/1236498292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eb7e50 msgr2=0x5626a1eb29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:33.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.704+0000 7fdb5e000640 1 --2- 172.21.15.123:0/1236498292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eb7e50 0x5626a1eb29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.704+0000 7fdb5e000640 1 -- 172.21.15.123:0/1236498292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1eacc70 msgr2=0x5626a1eb1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:33.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.704+0000 7fdb5e000640 1 --2- 172.21.15.123:0/1236498292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1eacc70 0x5626a1eb1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.704+0000 7fdb5e000640 1 -- 172.21.15.123:0/1236498292 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626a1ea9e90 con 0x5626a1ead590 2024-09-19T04:51:33.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.704+0000 7fdb5e000640 1 --2- 172.21.15.123:0/1236498292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1ead590 0x5626a1eb2470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c098d00 tx=0x7fdb4c09c240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d7fe79fda8230f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:33.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.704+0000 7fdb48c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:51:33.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.704+0000 7fdb48c00640 1 -- 172.21.15.123:0/1236498292 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c09c380 con 0x5626a1ead590 2024-09-19T04:51:33.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.704+0000 7fdb48c00640 1 -- 172.21.15.123:0/1236498292 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fdb4c09ba80 con 0x5626a1ead590 2024-09-19T04:51:33.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.704+0000 7fdb6887a740 1 -- 172.21.15.123:0/1236498292 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb5001db90 con 0x5626a1ead590 2024-09-19T04:51:33.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.705+0000 7fdb48c00640 1 -- 172.21.15.123:0/1236498292 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdb4c0a0730 con 0x5626a1ead590 2024-09-19T04:51:33.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.705+0000 7fdb6887a740 10 client.26198 fetch_fsmap learned FSMap version 36 2024-09-19T04:51:33.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.705+0000 7fdb6887a740 1 -- 172.21.15.123:0/1236498292 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5626a1f32e10 con 0x5626a1ead590 2024-09-19T04:51:33.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.705+0000 7fdb48c00640 1 -- 172.21.15.123:0/1236498292 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb5001db90 con 0x5626a1ead590 2024-09-19T04:51:33.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.705+0000 7fdb48c00640 1 -- 172.21.15.123:0/1236498292 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fdb4c0a18c0 con 0x5626a1ead590 2024-09-19T04:51:33.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.705+0000 7fdb6887a740 10 client.26198 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:51:33.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.705+0000 7fdb6887a740 10 client.26198 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:51:33.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.705+0000 7fdb6887a740 20 client.26198 populate_metadata read hostname 'smithi123' 2024-09-19T04:51:33.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.705+0000 7fdb6887a740 1 --2- 172.21.15.123:0/1236498292 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5626a1f33580 0x5626a1f359a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:33.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.705+0000 7fdb6887a740 4 client.26198 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:51:33.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.705+0000 7fdb6887a740 1 -- 172.21.15.123:0/1236498292 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5626a1e1ae60 con 0x5626a1f33580 2024-09-19T04:51:33.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.706+0000 7fdb5ea00640 1 --2- 172.21.15.123:0/1236498292 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5626a1f33580 0x5626a1f359a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:33.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.706+0000 7fdb5ea00640 1 --2- 172.21.15.123:0/1236498292 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5626a1f33580 0x5626a1f359a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=929923b1bd8ac95f server_cookie=bb97c7c1651a900c in_seq=0 out_seq=0 2024-09-19T04:51:33.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.706+0000 7fdb48c00640 10 client.26198 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:51:33.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.706+0000 7fdb48c00640 1 -- 172.21.15.123:0/1236498292 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5626a1e1ae60 con 0x5626a1f33580 2024-09-19T04:51:33.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.706+0000 7fdb48c00640 10 client.26198 handle_command_reply: tid=0 2024-09-19T04:51:33.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.706+0000 7fdb6887a740 1 client.26198 shutdown 2024-09-19T04:51:33.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.707+0000 7fdb6887a740 1 -- 172.21.15.123:0/1236498292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1ead590 msgr2=0x5626a1eb2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:33.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.707+0000 7fdb6887a740 1 --2- 172.21.15.123:0/1236498292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1ead590 0x5626a1eb2470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c098d00 tx=0x7fdb4c09c240 comp rx=0 tx=0).stop 2024-09-19T04:51:33.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.707+0000 7fdb6887a740 1 -- 172.21.15.123:0/1236498292 shutdown_connections 2024-09-19T04:51:33.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.707+0000 7fdb6887a740 1 --2- 172.21.15.123:0/1236498292 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5626a1f33580 0x5626a1f359a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.707+0000 7fdb6887a740 1 --2- 172.21.15.123:0/1236498292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626a1eb7e50 0x5626a1eb29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.707+0000 7fdb6887a740 1 --2- 172.21.15.123:0/1236498292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a1ead590 0x5626a1eb2470 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.707+0000 7fdb6887a740 1 --2- 172.21.15.123:0/1236498292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626a1eacc70 0x5626a1eb1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.707+0000 7fdb6887a740 1 -- 172.21.15.123:0/1236498292 >> 172.21.15.123:0/1236498292 conn(0x5626a1ea9200 msgr2=0x5626a1eaef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:33.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.707+0000 7fdb48c00640 0 client.26198 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:51:33.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.707+0000 7fdb6887a740 1 -- 172.21.15.123:0/1236498292 shutdown_connections 2024-09-19T04:51:33.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.707+0000 7fdb6887a740 1 -- 172.21.15.123:0/1236498292 wait complete. 2024-09-19T04:51:33.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.707+0000 7fdb6887a740 20 client.26198 trim_cache size 0 max 16384 2024-09-19T04:51:33.712 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:51:33.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.708+0000 7fdb65c00640 1 -- 172.21.15.123:0/2080400672 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb440a33b0 msgr2=0x7fdb440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:33.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.708+0000 7fdb65c00640 1 --2- 172.21.15.123:0/2080400672 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb440a33b0 0x7fdb440a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x5626a1eaa060 tx=0x7fdb50005f90 comp rx=0 tx=0).stop 2024-09-19T04:51:33.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.708+0000 7fdb65c00640 1 -- 172.21.15.123:0/2080400672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb601012c0 msgr2=0x7fdb6019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:33.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.708+0000 7fdb65c00640 1 --2- 172.21.15.123:0/2080400672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb601012c0 0x7fdb6019ab80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00e9f0 tx=0x7fdb4c00edf0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.708+0000 7fdb65c00640 1 -- 172.21.15.123:0/2080400672 shutdown_connections 2024-09-19T04:51:33.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.708+0000 7fdb65c00640 1 --2- 172.21.15.123:0/2080400672 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb440a33b0 0x7fdb440a5870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.708+0000 7fdb65c00640 1 --2- 172.21.15.123:0/2080400672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb6010e4c0 0x7fdb6019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.708+0000 7fdb65c00640 1 --2- 172.21.15.123:0/2080400672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb601012c0 0x7fdb6019ab80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.708+0000 7fdb65c00640 1 --2- 172.21.15.123:0/2080400672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb60100910 0x7fdb6019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:33.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.708+0000 7fdb65c00640 1 -- 172.21.15.123:0/2080400672 >> 172.21.15.123:0/2080400672 conn(0x7fdb600fc780 msgr2=0x7fdb600feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:33.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.708+0000 7fdb65c00640 1 -- 172.21.15.123:0/2080400672 shutdown_connections 2024-09-19T04:51:33.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:33.708+0000 7fdb65c00640 1 -- 172.21.15.123:0/2080400672 wait complete. 2024-09-19T04:51:34.601 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:51:35.021 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:51:35.355 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:35 smithi123 ceph-mon[30511]: pgmap v149: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.3 MiB/s wr, 808 op/s 2024-09-19T04:51:35.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:35 smithi193 ceph-mon[35592]: pgmap v149: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.3 MiB/s wr, 808 op/s 2024-09-19T04:51:35.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:35 smithi183 ceph-mon[35333]: pgmap v149: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.3 MiB/s wr, 808 op/s 2024-09-19T04:51:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.598+0000 7fc0f7e00640 1 -- 172.21.15.123:0/2916823780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f81068d0 msgr2=0x7fc0f8106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.598+0000 7fc0f7e00640 1 --2- 172.21.15.123:0/2916823780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f81068d0 0x7fc0f8106cb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009930 tx=0x7fc0e802a420 comp rx=0 tx=0).stop 2024-09-19T04:51:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.598+0000 7fc0f7e00640 1 -- 172.21.15.123:0/2916823780 shutdown_connections 2024-09-19T04:51:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.598+0000 7fc0f7e00640 1 --2- 172.21.15.123:0/2916823780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f8101270 0x7fc0f810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.598+0000 7fc0f7e00640 1 --2- 172.21.15.123:0/2916823780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0f81008d0 0x7fc0f8100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.598+0000 7fc0f7e00640 1 --2- 172.21.15.123:0/2916823780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f81068d0 0x7fc0f8106cb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.598+0000 7fc0f7e00640 1 -- 172.21.15.123:0/2916823780 >> 172.21.15.123:0/2916823780 conn(0x7fc0f80fc610 msgr2=0x7fc0f80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.598+0000 7fc0f7e00640 1 -- 172.21.15.123:0/2916823780 shutdown_connections 2024-09-19T04:51:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.598+0000 7fc0f7e00640 1 -- 172.21.15.123:0/2916823780 wait complete. 2024-09-19T04:51:35.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.599+0000 7fc0f7e00640 1 Processor -- start 2024-09-19T04:51:35.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.600+0000 7fc0f7e00640 1 -- start start 2024-09-19T04:51:35.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.600+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f81008d0 0x7fc0f819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.600+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f8101270 0x7fc0f819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.600+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0f81068d0 0x7fc0f819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.600+0000 7fc0f7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f8110d20 con 0x7fc0f8101270 2024-09-19T04:51:35.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.600+0000 7fc0f7e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f8110ba0 con 0x7fc0f81008d0 2024-09-19T04:51:35.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.600+0000 7fc0f7e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f8110ea0 con 0x7fc0f81068d0 2024-09-19T04:51:35.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.600+0000 7fc0f6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f81008d0 0x7fc0f819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.600+0000 7fc0f7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0f81068d0 0x7fc0f819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.600+0000 7fc0f6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f81008d0 0x7fc0f819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:37638/0 (socket says 172.21.15.123:37638) 2024-09-19T04:51:35.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.600+0000 7fc0f6a00640 1 -- 172.21.15.123:0/3305842306 learned_addr learned my addr 172.21.15.123:0/3305842306 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:35.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.600+0000 7fc0f6000640 1 --2- 172.21.15.123:0/3305842306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f8101270 0x7fc0f819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.601+0000 7fc0f6a00640 1 -- 172.21.15.123:0/3305842306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0f81068d0 msgr2=0x7fc0f819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:35.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.601+0000 7fc0f6a00640 1 --2- 172.21.15.123:0/3305842306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0f81068d0 0x7fc0f819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.601+0000 7fc0f6a00640 1 -- 172.21.15.123:0/3305842306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f8101270 msgr2=0x7fc0f819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:35.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.601+0000 7fc0f6a00640 1 --2- 172.21.15.123:0/3305842306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f8101270 0x7fc0f819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.601+0000 7fc0f6a00640 1 -- 172.21.15.123:0/3305842306 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f819f480 con 0x7fc0f81008d0 2024-09-19T04:51:35.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.601+0000 7fc0f6a00640 1 --2- 172.21.15.123:0/3305842306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f81008d0 0x7fc0f819a4d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc0e802a940 tx=0x7fc0e8003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:35.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.601+0000 7fc0fca00640 1 -- 172.21.15.123:0/3305842306 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e803f070 con 0x7fc0f81008d0 2024-09-19T04:51:35.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.601+0000 7fc0fca00640 1 -- 172.21.15.123:0/3305842306 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e8031c90 con 0x7fc0f81008d0 2024-09-19T04:51:35.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.601+0000 7fc0fca00640 1 -- 172.21.15.123:0/3305842306 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8035030 con 0x7fc0f81008d0 2024-09-19T04:51:35.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.602+0000 7fc0f7e00640 1 -- 172.21.15.123:0/3305842306 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f819f710 con 0x7fc0f81008d0 2024-09-19T04:51:35.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.602+0000 7fc0f7e00640 1 -- 172.21.15.123:0/3305842306 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f819fba0 con 0x7fc0f81008d0 2024-09-19T04:51:35.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.605+0000 7fc0fca00640 1 -- 172.21.15.123:0/3305842306 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc0e8042070 con 0x7fc0f81008d0 2024-09-19T04:51:35.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.605+0000 7fc100862740 1 Processor -- start 2024-09-19T04:51:35.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.605+0000 7fc100862740 1 -- start start 2024-09-19T04:51:35.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.605+0000 7fc100862740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f2ee40 0x5555d5f2f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.605+0000 7fc100862740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2f760 0x5555d5f39ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.605+0000 7fc100862740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f3a020 0x5555d5f3c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.605+0000 7fc100862740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5555d5e953e0 con 0x5555d5f3a020 2024-09-19T04:51:35.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.605+0000 7fc100862740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5555d5e3cd80 con 0x5555d5f2ee40 2024-09-19T04:51:35.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.605+0000 7fc100862740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5555d5e913a0 con 0x5555d5f2f760 2024-09-19T04:51:35.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.606+0000 7fc0f6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2f760 0x5555d5f39ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.606+0000 7fc0f6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2f760 0x5555d5f39ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53606/0 (socket says 172.21.15.123:53606) 2024-09-19T04:51:35.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.606+0000 7fc0f6a00640 1 -- 172.21.15.123:0/181669241 learned_addr learned my addr 172.21.15.123:0/181669241 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:35.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.606+0000 7fc0f7400640 1 --2- 172.21.15.123:0/181669241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f2ee40 0x5555d5f2f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.606+0000 7fc0f6000640 1 --2- 172.21.15.123:0/181669241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f3a020 0x5555d5f3c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.606+0000 7fc0f6a00640 1 -- 172.21.15.123:0/181669241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f2ee40 msgr2=0x5555d5f2f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:35.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.606+0000 7fc0f6a00640 1 --2- 172.21.15.123:0/181669241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f2ee40 0x5555d5f2f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.606+0000 7fc0f6a00640 1 -- 172.21.15.123:0/181669241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f3a020 msgr2=0x5555d5f3c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:35.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.606+0000 7fc0f6a00640 1 --2- 172.21.15.123:0/181669241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f3a020 0x5555d5f3c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.606+0000 7fc0f6a00640 1 -- 172.21.15.123:0/181669241 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d5e5bae0 con 0x5555d5f2f760 2024-09-19T04:51:35.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.606+0000 7fc0f6000640 1 --2- 172.21.15.123:0/181669241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f3a020 0x5555d5f3c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:51:35.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.606+0000 7fc0fca00640 1 --2- 172.21.15.123:0/3305842306 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc0dc0a3430 0x7fc0dc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.606+0000 7fc0f6a00640 1 --2- 172.21.15.123:0/181669241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2f760 0x5555d5f39ae0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80ed960 tx=0x7fc0e80f8560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6f3300133df1bba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:35.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.607+0000 7fc0fca00640 1 -- 172.21.15.123:0/3305842306 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fc0e80f02d0 con 0x7fc0f81008d0 2024-09-19T04:51:35.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.607+0000 7fc0f7400640 1 --2- 172.21.15.123:0/3305842306 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc0dc0a3430 0x7fc0dc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.607+0000 7fc0e3e00640 1 -- 172.21.15.123:0/181669241 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e80b88d0 con 0x5555d5f2f760 2024-09-19T04:51:35.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.607+0000 7fc0e3e00640 1 -- 172.21.15.123:0/181669241 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e8031e40 con 0x5555d5f2f760 2024-09-19T04:51:35.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.607+0000 7fc0e3e00640 1 -- 172.21.15.123:0/181669241 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e80bbf90 con 0x5555d5f2f760 2024-09-19T04:51:35.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.608+0000 7fc100862740 1 -- 172.21.15.123:0/181669241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2f760 msgr2=0x5555d5f39ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:35.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.608+0000 7fc0f7400640 1 --2- 172.21.15.123:0/3305842306 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc0dc0a3430 0x7fc0dc0a58f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0005fd0 tx=0x7fc0f0009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:35.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.608+0000 7fc100862740 1 --2- 172.21.15.123:0/181669241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2f760 0x5555d5f39ae0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80ed960 tx=0x7fc0e80f8560 comp rx=0 tx=0).stop 2024-09-19T04:51:35.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.608+0000 7fc100862740 1 -- 172.21.15.123:0/181669241 shutdown_connections 2024-09-19T04:51:35.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.608+0000 7fc100862740 1 --2- 172.21.15.123:0/181669241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f3a020 0x5555d5f3c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.608+0000 7fc100862740 1 --2- 172.21.15.123:0/181669241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2f760 0x5555d5f39ae0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.608+0000 7fc100862740 1 --2- 172.21.15.123:0/181669241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f2ee40 0x5555d5f2f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.608+0000 7fc100862740 1 -- 172.21.15.123:0/181669241 >> 172.21.15.123:0/181669241 conn(0x5555d5f2b3d0 msgr2=0x5555d5f2b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:35.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.608+0000 7fc100862740 1 -- 172.21.15.123:0/181669241 shutdown_connections 2024-09-19T04:51:35.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.608+0000 7fc100862740 1 -- 172.21.15.123:0/181669241 wait complete. 2024-09-19T04:51:35.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.608+0000 7fc100862740 1 -- start start 2024-09-19T04:51:35.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.609+0000 7fc100862740 1 Processor -- start 2024-09-19T04:51:35.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.609+0000 7fc100862740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f2ee40 0x5555d5f36fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.609+0000 7fc100862740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2f760 0x5555d5f37520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.609+0000 7fc100862740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f3a020 0x5555d5f37a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.609+0000 7fc100862740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5555d5f3c9b0 con 0x5555d5f2ee40 2024-09-19T04:51:35.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.609+0000 7fc100862740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5555d5e500f0 con 0x5555d5f3a020 2024-09-19T04:51:35.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.609+0000 7fc100862740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5555d5f3cb30 con 0x5555d5f2f760 2024-09-19T04:51:35.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.609+0000 7fc100862740 10 client.0 _ll_register_callbacks cb 0x5555d5e37bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:51:35.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.609+0000 7fc0f6000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2f760 0x5555d5f37520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.609+0000 7fc0f6000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2f760 0x5555d5f37520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53612/0 (socket says 172.21.15.123:53612) 2024-09-19T04:51:35.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.609+0000 7fc0f6000640 1 -- 172.21.15.123:0/4010471877 learned_addr learned my addr 172.21.15.123:0/4010471877 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:35.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.609+0000 7fc0f7400640 1 --2- 172.21.15.123:0/4010471877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f3a020 0x5555d5f37a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.610+0000 7fc0f6a00640 1 --2- 172.21.15.123:0/4010471877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f2ee40 0x5555d5f36fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.610+0000 7fc0f6000640 1 -- 172.21.15.123:0/4010471877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f3a020 msgr2=0x5555d5f37a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:35.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.610+0000 7fc0f6000640 1 --2- 172.21.15.123:0/4010471877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f3a020 0x5555d5f37a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.610+0000 7fc0f6000640 1 -- 172.21.15.123:0/4010471877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f2ee40 msgr2=0x5555d5f36fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:35.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.610+0000 7fc0f6000640 1 --2- 172.21.15.123:0/4010471877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f2ee40 0x5555d5f36fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.610+0000 7fc0f6000640 1 -- 172.21.15.123:0/4010471877 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d5c063c0 con 0x5555d5f2f760 2024-09-19T04:51:35.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.610+0000 7fc0f6a00640 1 --2- 172.21.15.123:0/4010471877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f2ee40 0x5555d5f36fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:51:35.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.610+0000 7fc0f6000640 1 --2- 172.21.15.123:0/4010471877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2f760 0x5555d5f37520 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4013710 tx=0x7fc0e4013bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47d439755cc0cb18 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:35.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.610+0000 7fc0e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:51:35.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.610+0000 7fc0e0c00640 1 -- 172.21.15.123:0/4010471877 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4004460 con 0x5555d5f2f760 2024-09-19T04:51:35.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.610+0000 7fc100862740 1 -- 172.21.15.123:0/4010471877 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5555d5ea2950 con 0x5555d5f2f760 2024-09-19T04:51:35.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.610+0000 7fc0e0c00640 1 -- 172.21.15.123:0/4010471877 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fc0e40119c0 con 0x5555d5f2f760 2024-09-19T04:51:35.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.611+0000 7fc0e0c00640 1 -- 172.21.15.123:0/4010471877 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc0e4011bc0 con 0x5555d5f2f760 2024-09-19T04:51:35.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.611+0000 7fc100862740 10 client.26233 fetch_fsmap learned FSMap version 36 2024-09-19T04:51:35.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.611+0000 7fc100862740 1 -- 172.21.15.123:0/4010471877 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5555d5f48a40 con 0x5555d5f2f760 2024-09-19T04:51:35.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.611+0000 7fc0e0c00640 1 -- 172.21.15.123:0/4010471877 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5555d5ea2950 con 0x5555d5f2f760 2024-09-19T04:51:35.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.611+0000 7fc0e0c00640 1 -- 172.21.15.123:0/4010471877 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fc0e401a070 con 0x5555d5f2f760 2024-09-19T04:51:35.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.611+0000 7fc100862740 10 client.26233 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:51:35.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.611+0000 7fc100862740 10 client.26233 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:51:35.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.611+0000 7fc100862740 20 client.26233 populate_metadata read hostname 'smithi123' 2024-09-19T04:51:35.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.612+0000 7fc100862740 1 --2- 172.21.15.123:0/4010471877 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5555d5f49e50 0x5555d5f4c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.612+0000 7fc100862740 4 client.26233 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:51:35.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.612+0000 7fc100862740 1 -- 172.21.15.123:0/4010471877 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555d5e9ce60 con 0x5555d5f49e50 2024-09-19T04:51:35.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.612+0000 7fc0f6a00640 1 --2- 172.21.15.123:0/4010471877 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5555d5f49e50 0x5555d5f4c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.613+0000 7fc0f6a00640 1 --2- 172.21.15.123:0/4010471877 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5555d5f49e50 0x5555d5f4c2f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a20055f7002cf26 server_cookie=6bbed3ab4b7b6731 in_seq=0 out_seq=0 2024-09-19T04:51:35.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.613+0000 7fc0e0c00640 10 client.26233 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:51:35.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.614+0000 7fc0e0c00640 1 -- 172.21.15.123:0/4010471877 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555d5e9ce60 con 0x5555d5f49e50 2024-09-19T04:51:35.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.614+0000 7fc0e0c00640 10 client.26233 handle_command_reply: tid=0 2024-09-19T04:51:35.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.614+0000 7fc100862740 1 client.26233 shutdown 2024-09-19T04:51:35.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.615+0000 7fc100862740 1 -- 172.21.15.123:0/4010471877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2f760 msgr2=0x5555d5f37520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:35.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.615+0000 7fc100862740 1 --2- 172.21.15.123:0/4010471877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2f760 0x5555d5f37520 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4013710 tx=0x7fc0e4013bd0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.615+0000 7fc100862740 1 -- 172.21.15.123:0/4010471877 shutdown_connections 2024-09-19T04:51:35.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.615+0000 7fc100862740 1 --2- 172.21.15.123:0/4010471877 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5555d5f49e50 0x5555d5f4c2f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.615+0000 7fc100862740 1 --2- 172.21.15.123:0/4010471877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f3a020 0x5555d5f37a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.615+0000 7fc100862740 1 --2- 172.21.15.123:0/4010471877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2f760 0x5555d5f37520 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.615+0000 7fc100862740 1 --2- 172.21.15.123:0/4010471877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f2ee40 0x5555d5f36fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.615+0000 7fc0e0c00640 0 client.26233 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:51:35.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.615+0000 7fc100862740 1 -- 172.21.15.123:0/4010471877 >> 172.21.15.123:0/4010471877 conn(0x5555d5f2b3d0 msgr2=0x5555d5f310b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:35.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.615+0000 7fc100862740 1 -- 172.21.15.123:0/4010471877 shutdown_connections 2024-09-19T04:51:35.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.615+0000 7fc100862740 1 -- 172.21.15.123:0/4010471877 wait complete. 2024-09-19T04:51:35.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.615+0000 7fc100862740 20 client.26233 trim_cache size 0 max 16384 2024-09-19T04:51:35.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.629+0000 7fc100862740 1 Processor -- start 2024-09-19T04:51:35.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.630+0000 7fc100862740 1 -- start start 2024-09-19T04:51:35.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.630+0000 7fc100862740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2ee40 0x5555d5f51bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.630+0000 7fc100862740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f2f760 0x5555d5f52100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.630+0000 7fc100862740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f3a020 0x5555d5f34240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.630+0000 7fc100862740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f81af8c0 con 0x5555d5f2f760 2024-09-19T04:51:35.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.630+0000 7fc100862740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f81b1400 con 0x5555d5f3a020 2024-09-19T04:51:35.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.630+0000 7fc100862740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f81aff90 con 0x5555d5f2ee40 2024-09-19T04:51:35.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.630+0000 7fc0f6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2ee40 0x5555d5f51bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.630+0000 7fc0f7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f3a020 0x5555d5f34240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.630+0000 7fc0f6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2ee40 0x5555d5f51bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53626/0 (socket says 172.21.15.123:53626) 2024-09-19T04:51:35.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.630+0000 7fc0f6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f2f760 0x5555d5f52100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.630+0000 7fc0f6a00640 1 -- 172.21.15.123:0/3762999903 learned_addr learned my addr 172.21.15.123:0/3762999903 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:35.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.630+0000 7fc0f7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f3a020 0x5555d5f34240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:37682/0 (socket says 172.21.15.123:37682) 2024-09-19T04:51:35.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.631+0000 7fc0f6a00640 1 -- 172.21.15.123:0/3762999903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f3a020 msgr2=0x5555d5f34240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:35.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.631+0000 7fc0f6a00640 1 --2- 172.21.15.123:0/3762999903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f3a020 0x5555d5f34240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.631+0000 7fc0f6a00640 1 -- 172.21.15.123:0/3762999903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f2f760 msgr2=0x5555d5f52100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:35.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.631+0000 7fc0f6a00640 1 --2- 172.21.15.123:0/3762999903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f2f760 0x5555d5f52100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.631+0000 7fc0f6a00640 1 -- 172.21.15.123:0/3762999903 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d5f39950 con 0x5555d5f2ee40 2024-09-19T04:51:35.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.631+0000 7fc0f6a00640 1 --2- 172.21.15.123:0/3762999903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2ee40 0x5555d5f51bc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80b8a90 tx=0x7fc0e802a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74ea655f3da2ec47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:35.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.631+0000 7fc0e3400640 1 -- 172.21.15.123:0/3762999903 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e80d5070 con 0x5555d5f2ee40 2024-09-19T04:51:35.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.631+0000 7fc0e3400640 1 -- 172.21.15.123:0/3762999903 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e80c0930 con 0x5555d5f2ee40 2024-09-19T04:51:35.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.631+0000 7fc0e3400640 1 -- 172.21.15.123:0/3762999903 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8049040 con 0x5555d5f2ee40 2024-09-19T04:51:35.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.632+0000 7fc100862740 1 -- 172.21.15.123:0/3762999903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2ee40 msgr2=0x5555d5f51bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:35.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.632+0000 7fc100862740 1 --2- 172.21.15.123:0/3762999903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2ee40 0x5555d5f51bc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80b8a90 tx=0x7fc0e802a420 comp rx=0 tx=0).stop 2024-09-19T04:51:35.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.632+0000 7fc100862740 1 -- 172.21.15.123:0/3762999903 shutdown_connections 2024-09-19T04:51:35.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.632+0000 7fc100862740 1 --2- 172.21.15.123:0/3762999903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f3a020 0x5555d5f34240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.632+0000 7fc100862740 1 --2- 172.21.15.123:0/3762999903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f2f760 0x5555d5f52100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.632+0000 7fc100862740 1 --2- 172.21.15.123:0/3762999903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f2ee40 0x5555d5f51bc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.632+0000 7fc100862740 1 -- 172.21.15.123:0/3762999903 >> 172.21.15.123:0/3762999903 conn(0x5555d5f2b3d0 msgr2=0x5555d5f322c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:35.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.632+0000 7fc100862740 1 -- 172.21.15.123:0/3762999903 shutdown_connections 2024-09-19T04:51:35.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.632+0000 7fc100862740 1 -- 172.21.15.123:0/3762999903 wait complete. 2024-09-19T04:51:35.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.633+0000 7fc100862740 1 -- start start 2024-09-19T04:51:35.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.633+0000 7fc100862740 1 Processor -- start 2024-09-19T04:51:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc100862740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f2ee40 0x5555d5f42010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc100862740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f2f760 0x5555d5f42550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc100862740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f3a020 0x5555d5f3d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc100862740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f81b0d30 con 0x5555d5f2f760 2024-09-19T04:51:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc100862740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f81b0660 con 0x5555d5f2ee40 2024-09-19T04:51:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc100862740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f81ad8a0 con 0x5555d5f3a020 2024-09-19T04:51:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc100862740 10 client.0 _ll_register_callbacks cb 0x7fc0e4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:51:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc0f7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f3a020 0x5555d5f3d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc0f6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f2f760 0x5555d5f42550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc0f7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f3a020 0x5555d5f3d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53630/0 (socket says 172.21.15.123:53630) 2024-09-19T04:51:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc0f7400640 1 -- 172.21.15.123:0/2370333388 learned_addr learned my addr 172.21.15.123:0/2370333388 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc0f6a00640 1 --2- 172.21.15.123:0/2370333388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f2ee40 0x5555d5f42010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc0f7400640 1 -- 172.21.15.123:0/2370333388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f2ee40 msgr2=0x5555d5f42010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc0f7400640 1 --2- 172.21.15.123:0/2370333388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f2ee40 0x5555d5f42010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc0f7400640 1 -- 172.21.15.123:0/2370333388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f2f760 msgr2=0x5555d5f42550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc0f7400640 1 --2- 172.21.15.123:0/2370333388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f2f760 0x5555d5f42550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc0f7400640 1 -- 172.21.15.123:0/2370333388 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d5f3dc40 con 0x5555d5f3a020 2024-09-19T04:51:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.634+0000 7fc0f6000640 1 --2- 172.21.15.123:0/2370333388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f2f760 0x5555d5f42550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.635+0000 7fc0f7400640 1 --2- 172.21.15.123:0/2370333388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f3a020 0x5555d5f3d410 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0004250 tx=0x7fc0f0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=931087d3f84b636 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.635+0000 7fc0e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:51:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.635+0000 7fc0e0c00640 1 -- 172.21.15.123:0/2370333388 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f002e060 con 0x5555d5f3a020 2024-09-19T04:51:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.635+0000 7fc100862740 1 -- 172.21.15.123:0/2370333388 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0e80c0930 con 0x5555d5f3a020 2024-09-19T04:51:35.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.635+0000 7fc0e0c00640 1 -- 172.21.15.123:0/2370333388 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fc0f000fc10 con 0x5555d5f3a020 2024-09-19T04:51:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.635+0000 7fc0e0c00640 1 -- 172.21.15.123:0/2370333388 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc0f000fe10 con 0x5555d5f3a020 2024-09-19T04:51:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.635+0000 7fc100862740 10 client.26239 fetch_fsmap learned FSMap version 36 2024-09-19T04:51:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.635+0000 7fc100862740 1 -- 172.21.15.123:0/2370333388 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5555d5fb4840 con 0x5555d5f3a020 2024-09-19T04:51:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.635+0000 7fc0e0c00640 1 -- 172.21.15.123:0/2370333388 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e80c0930 con 0x5555d5f3a020 2024-09-19T04:51:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.636+0000 7fc0e0c00640 1 -- 172.21.15.123:0/2370333388 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fc0f001d5e0 con 0x5555d5f3a020 2024-09-19T04:51:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.636+0000 7fc100862740 10 client.26239 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:51:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.636+0000 7fc100862740 10 client.26239 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:51:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.636+0000 7fc100862740 20 client.26239 populate_metadata read hostname 'smithi123' 2024-09-19T04:51:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.636+0000 7fc100862740 1 --2- 172.21.15.123:0/2370333388 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5555d5fb4f70 0x5555d5fb7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.636+0000 7fc100862740 4 client.26239 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:51:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.636+0000 7fc100862740 1 -- 172.21.15.123:0/2370333388 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555d5e19010 con 0x5555d5fb4f70 2024-09-19T04:51:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.636+0000 7fc0f6a00640 1 --2- 172.21.15.123:0/2370333388 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5555d5fb4f70 0x5555d5fb7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.637+0000 7fc0f6a00640 1 --2- 172.21.15.123:0/2370333388 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5555d5fb4f70 0x5555d5fb7390 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a35a318b5f6f27ed server_cookie=3d6a3b241cf98859 in_seq=0 out_seq=0 2024-09-19T04:51:35.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.637+0000 7fc0e0c00640 10 client.26239 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:51:35.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.637+0000 7fc0e0c00640 1 -- 172.21.15.123:0/2370333388 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555d5e19010 con 0x5555d5fb4f70 2024-09-19T04:51:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.637+0000 7fc0e0c00640 10 client.26239 handle_command_reply: tid=0 2024-09-19T04:51:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.637+0000 7fc100862740 1 client.26239 shutdown 2024-09-19T04:51:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.637+0000 7fc100862740 1 -- 172.21.15.123:0/2370333388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f3a020 msgr2=0x5555d5f3d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.637+0000 7fc100862740 1 --2- 172.21.15.123:0/2370333388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f3a020 0x5555d5f3d410 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0004250 tx=0x7fc0f0004a40 comp rx=0 tx=0).stop 2024-09-19T04:51:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.637+0000 7fc100862740 1 -- 172.21.15.123:0/2370333388 shutdown_connections 2024-09-19T04:51:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.637+0000 7fc100862740 1 --2- 172.21.15.123:0/2370333388 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5555d5fb4f70 0x5555d5fb7390 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.637+0000 7fc100862740 1 --2- 172.21.15.123:0/2370333388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555d5f3a020 0x5555d5f3d410 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.637+0000 7fc100862740 1 --2- 172.21.15.123:0/2370333388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555d5f2f760 0x5555d5f42550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.637+0000 7fc100862740 1 --2- 172.21.15.123:0/2370333388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d5f2ee40 0x5555d5f42010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.637+0000 7fc0e0c00640 0 client.26239 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:51:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.637+0000 7fc100862740 1 -- 172.21.15.123:0/2370333388 >> 172.21.15.123:0/2370333388 conn(0x5555d5f2b3d0 msgr2=0x5555d5f38840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.637+0000 7fc100862740 1 -- 172.21.15.123:0/2370333388 shutdown_connections 2024-09-19T04:51:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.637+0000 7fc100862740 1 -- 172.21.15.123:0/2370333388 wait complete. 2024-09-19T04:51:35.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.638+0000 7fc100862740 20 client.26239 trim_cache size 0 max 16384 2024-09-19T04:51:35.645 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:51:35.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.639+0000 7fc0f7e00640 1 -- 172.21.15.123:0/3305842306 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc0dc0a3430 msgr2=0x7fc0dc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:35.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.639+0000 7fc0f7e00640 1 --2- 172.21.15.123:0/3305842306 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc0dc0a3430 0x7fc0dc0a58f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0005fd0 tx=0x7fc0f0009500 comp rx=0 tx=0).stop 2024-09-19T04:51:35.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.639+0000 7fc0f7e00640 1 -- 172.21.15.123:0/3305842306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f81008d0 msgr2=0x7fc0f819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:35.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.639+0000 7fc0f7e00640 1 --2- 172.21.15.123:0/3305842306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f81008d0 0x7fc0f819a4d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc0e802a940 tx=0x7fc0e8003b20 comp rx=0 tx=0).stop 2024-09-19T04:51:35.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.639+0000 7fc0f7e00640 1 -- 172.21.15.123:0/3305842306 shutdown_connections 2024-09-19T04:51:35.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.639+0000 7fc0f7e00640 1 --2- 172.21.15.123:0/3305842306 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc0dc0a3430 0x7fc0dc0a58f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.639+0000 7fc0f7e00640 1 --2- 172.21.15.123:0/3305842306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0f81068d0 0x7fc0f819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.639+0000 7fc0f7e00640 1 --2- 172.21.15.123:0/3305842306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f8101270 0x7fc0f819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.639+0000 7fc0f7e00640 1 --2- 172.21.15.123:0/3305842306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f81008d0 0x7fc0f819a4d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:35.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.639+0000 7fc0f7e00640 1 -- 172.21.15.123:0/3305842306 >> 172.21.15.123:0/3305842306 conn(0x7fc0f80fc610 msgr2=0x7fc0f8069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:35.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.639+0000 7fc0f7e00640 1 -- 172.21.15.123:0/3305842306 shutdown_connections 2024-09-19T04:51:35.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:35.640+0000 7fc0f7e00640 1 -- 172.21.15.123:0/3305842306 wait complete. 2024-09-19T04:51:36.483 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:51:36.909 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:51:37.369 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:37 smithi123 ceph-mon[30511]: pgmap v150: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 543 op/s 2024-09-19T04:51:37.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:37 smithi193 ceph-mon[35592]: pgmap v150: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 543 op/s 2024-09-19T04:51:37.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.498+0000 7f5fbac00640 1 -- 172.21.15.123:0/991422574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fb41008c0 msgr2=0x7f5fb4100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:37.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.498+0000 7f5fbac00640 1 --2- 172.21.15.123:0/991422574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fb41008c0 0x7f5fb4100ca0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009930 tx=0x7f5fa402a420 comp rx=0 tx=0).stop 2024-09-19T04:51:37.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.498+0000 7f5fbac00640 1 -- 172.21.15.123:0/991422574 shutdown_connections 2024-09-19T04:51:37.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.498+0000 7f5fbac00640 1 --2- 172.21.15.123:0/991422574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fb4105ec0 0x7f5fb4106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.498+0000 7f5fbac00640 1 --2- 172.21.15.123:0/991422574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fb41011e0 0x7f5fb4105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.498+0000 7f5fbac00640 1 --2- 172.21.15.123:0/991422574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fb41008c0 0x7f5fb4100ca0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.498+0000 7f5fbac00640 1 -- 172.21.15.123:0/991422574 >> 172.21.15.123:0/991422574 conn(0x7f5fb40fc7d0 msgr2=0x7f5fb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:37.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.498+0000 7f5fbac00640 1 -- 172.21.15.123:0/991422574 shutdown_connections 2024-09-19T04:51:37.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.499+0000 7f5fbac00640 1 -- 172.21.15.123:0/991422574 wait complete. 2024-09-19T04:51:37.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.501+0000 7f5fbac00640 1 Processor -- start 2024-09-19T04:51:37.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.501+0000 7f5fbac00640 1 -- start start 2024-09-19T04:51:37.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.501+0000 7f5fbac00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fb41008c0 0x7f5fb41a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.501+0000 7f5fbac00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fb41011e0 0x7f5fb41a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.501+0000 7f5fbac00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fb4105ec0 0x7f5fb419ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.501+0000 7f5fbac00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb4118660 con 0x7f5fb41011e0 2024-09-19T04:51:37.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.501+0000 7f5fbac00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41184e0 con 0x7f5fb4105ec0 2024-09-19T04:51:37.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.501+0000 7f5fbac00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41187e0 con 0x7f5fb41008c0 2024-09-19T04:51:37.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.502+0000 7f5fb3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fb41008c0 0x7f5fb41a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:37.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.502+0000 7f5fb3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fb41008c0 0x7f5fb41a4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53648/0 (socket says 172.21.15.123:53648) 2024-09-19T04:51:37.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.502+0000 7f5fb3e00640 1 -- 172.21.15.123:0/1509882555 learned_addr learned my addr 172.21.15.123:0/1509882555 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:37.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.502+0000 7f5fb8c00640 1 --2- 172.21.15.123:0/1509882555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fb4105ec0 0x7f5fb419ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:37.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.502+0000 7f5fb3400640 1 --2- 172.21.15.123:0/1509882555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fb41011e0 0x7f5fb41a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:37.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.502+0000 7f5fb3e00640 1 -- 172.21.15.123:0/1509882555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fb4105ec0 msgr2=0x7f5fb419ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:37.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.502+0000 7f5fb3e00640 1 --2- 172.21.15.123:0/1509882555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fb4105ec0 0x7f5fb419ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.502+0000 7f5fb3e00640 1 -- 172.21.15.123:0/1509882555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fb41011e0 msgr2=0x7f5fb41a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:37.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.502+0000 7f5fb3e00640 1 --2- 172.21.15.123:0/1509882555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fb41011e0 0x7f5fb41a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.502+0000 7f5fb3e00640 1 -- 172.21.15.123:0/1509882555 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb419f140 con 0x7f5fb41008c0 2024-09-19T04:51:37.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.502+0000 7f5fb8c00640 1 --2- 172.21.15.123:0/1509882555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fb4105ec0 0x7f5fb419ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:37.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.503+0000 7f5fb3e00640 1 --2- 172.21.15.123:0/1509882555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fb41008c0 0x7f5fb41a4850 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5fa402a940 tx=0x7f5fa4003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:37.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.503+0000 7f5fb0c00640 1 -- 172.21.15.123:0/1509882555 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa403f070 con 0x7f5fb41008c0 2024-09-19T04:51:37.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.503+0000 7f5fb0c00640 1 -- 172.21.15.123:0/1509882555 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa4031c90 con 0x7f5fb41008c0 2024-09-19T04:51:37.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.503+0000 7f5fbac00640 1 -- 172.21.15.123:0/1509882555 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb419f3d0 con 0x7f5fb41008c0 2024-09-19T04:51:37.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.503+0000 7f5fb0c00640 1 -- 172.21.15.123:0/1509882555 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4035030 con 0x7f5fb41008c0 2024-09-19T04:51:37.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.503+0000 7f5fbac00640 1 -- 172.21.15.123:0/1509882555 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb41ab720 con 0x7f5fb41008c0 2024-09-19T04:51:37.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.507+0000 7f5fbd5df740 1 Processor -- start 2024-09-19T04:51:37.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.507+0000 7f5fbd5df740 1 -- start start 2024-09-19T04:51:37.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.507+0000 7f5fbd5df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e32c70 0x555b61e33050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.507+0000 7f5fb0c00640 1 -- 172.21.15.123:0/1509882555 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5fa4042070 con 0x7f5fb41008c0 2024-09-19T04:51:37.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.508+0000 7f5fbd5df740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e33590 0x555b61e3d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.508+0000 7f5fbd5df740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 0x555b61e40240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.508+0000 7f5fbd5df740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555b61d993e0 con 0x555b61e33590 2024-09-19T04:51:37.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.508+0000 7f5fbd5df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555b61d40d80 con 0x555b61e32c70 2024-09-19T04:51:37.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.508+0000 7f5fbd5df740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555b61d953a0 con 0x555b61e3de50 2024-09-19T04:51:37.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.508+0000 7f5fb8c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e32c70 0x555b61e33050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:37.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.508+0000 7f5fb8c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e32c70 0x555b61e33050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:37730/0 (socket says 172.21.15.123:37730) 2024-09-19T04:51:37.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.508+0000 7f5fb8c00640 1 -- 172.21.15.123:0/1655833661 learned_addr learned my addr 172.21.15.123:0/1655833661 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:37.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.508+0000 7f5fb3400640 1 --2- 172.21.15.123:0/1655833661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 0x555b61e40240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:37.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.508+0000 7f5fb8c00640 1 -- 172.21.15.123:0/1655833661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 msgr2=0x555b61e40240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:37.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.508+0000 7f5fb8c00640 1 --2- 172.21.15.123:0/1655833661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 0x555b61e40240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.508+0000 7f5fb8c00640 1 -- 172.21.15.123:0/1655833661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e33590 msgr2=0x555b61e3d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:37.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.510+0000 7f5fb3e00640 1 --2- 172.21.15.123:0/1655833661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e33590 0x555b61e3d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:37.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.510+0000 7f5fb8c00640 1 --2- 172.21.15.123:0/1655833661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e33590 0x555b61e3d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.510+0000 7f5fb8c00640 1 -- 172.21.15.123:0/1655833661 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b61d5fae0 con 0x555b61e32c70 2024-09-19T04:51:37.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.510+0000 7f5fb3400640 1 --2- 172.21.15.123:0/1655833661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 0x555b61e40240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:51:37.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.510+0000 7f5fb3e00640 1 --2- 172.21.15.123:0/1655833661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e33590 0x555b61e3d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:51:37.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.510+0000 7f5fb0c00640 1 --2- 172.21.15.123:0/1509882555 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5f980a34b0 0x7f5f980a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.510+0000 7f5fb8c00640 1 --2- 172.21.15.123:0/1655833661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e32c70 0x555b61e33050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5fac011790 tx=0x7f5fac011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8de29679f6c017a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:37.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.510+0000 7f5fb0c00640 1 -- 172.21.15.123:0/1509882555 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f5fa40f02d0 con 0x7f5fb41008c0 2024-09-19T04:51:37.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.510+0000 7f5fa9200640 1 -- 172.21.15.123:0/1655833661 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac003c30 con 0x555b61e32c70 2024-09-19T04:51:37.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.510+0000 7f5fa9200640 1 -- 172.21.15.123:0/1655833661 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fac00fe80 con 0x555b61e32c70 2024-09-19T04:51:37.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.510+0000 7f5fa9200640 1 -- 172.21.15.123:0/1655833661 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac0193e0 con 0x555b61e32c70 2024-09-19T04:51:37.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.511+0000 7f5fb3e00640 1 --2- 172.21.15.123:0/1509882555 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5f980a34b0 0x7f5f980a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:37.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.511+0000 7f5fbd5df740 1 -- 172.21.15.123:0/1655833661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e32c70 msgr2=0x555b61e33050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:37.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.511+0000 7f5fb3e00640 1 --2- 172.21.15.123:0/1509882555 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5f980a34b0 0x7f5f980a5970 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x555b61e30060 tx=0x7f5fa4031490 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:37.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.511+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/1655833661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e32c70 0x555b61e33050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5fac011790 tx=0x7f5fac011c50 comp rx=0 tx=0).stop 2024-09-19T04:51:37.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.512+0000 7f5fbd5df740 1 -- 172.21.15.123:0/1655833661 shutdown_connections 2024-09-19T04:51:37.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.512+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/1655833661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 0x555b61e40240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.512+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/1655833661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e33590 0x555b61e3d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.512+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/1655833661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e32c70 0x555b61e33050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.512+0000 7f5fbd5df740 1 -- 172.21.15.123:0/1655833661 >> 172.21.15.123:0/1655833661 conn(0x555b61e2f200 msgr2=0x555b61e2f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:37.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.512+0000 7f5fbd5df740 1 -- 172.21.15.123:0/1655833661 shutdown_connections 2024-09-19T04:51:37.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.512+0000 7f5fbd5df740 1 -- 172.21.15.123:0/1655833661 wait complete. 2024-09-19T04:51:37.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.513+0000 7f5fbd5df740 1 -- start start 2024-09-19T04:51:37.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.513+0000 7f5fbd5df740 1 Processor -- start 2024-09-19T04:51:37.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.514+0000 7f5fbd5df740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e32c70 0x555b61e3ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.514+0000 7f5fbd5df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e33590 0x555b61e3b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.514+0000 7f5fbd5df740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 0x555b61e3b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.514+0000 7f5fbd5df740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555b61e40a20 con 0x555b61e32c70 2024-09-19T04:51:37.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.514+0000 7f5fbd5df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555b61d540f0 con 0x555b61e33590 2024-09-19T04:51:37.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.514+0000 7f5fbd5df740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555b61e40ba0 con 0x555b61e3de50 2024-09-19T04:51:37.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.514+0000 7f5fbd5df740 10 client.0 _ll_register_callbacks cb 0x555b61d3bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:51:37.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.514+0000 7f5fb8c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e32c70 0x555b61e3ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:37.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.514+0000 7f5fb3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e33590 0x555b61e3b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:37.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.514+0000 7f5fb3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e33590 0x555b61e3b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:37736/0 (socket says 172.21.15.123:37736) 2024-09-19T04:51:37.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.514+0000 7f5fb3400640 1 -- 172.21.15.123:0/1309165910 learned_addr learned my addr 172.21.15.123:0/1309165910 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:37.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.514+0000 7f5fb3400640 1 -- 172.21.15.123:0/1309165910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 msgr2=0x555b61e3b7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:51:37.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.514+0000 7f5fb3400640 1 --2- 172.21.15.123:0/1309165910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 0x555b61e3b7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.514+0000 7f5fb3400640 1 -- 172.21.15.123:0/1309165910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e32c70 msgr2=0x555b61e3ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:37.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.515+0000 7f5fb3400640 1 --2- 172.21.15.123:0/1309165910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e32c70 0x555b61e3ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.515+0000 7f5fb3400640 1 -- 172.21.15.123:0/1309165910 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b61b0a3c0 con 0x555b61e33590 2024-09-19T04:51:37.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.515+0000 7f5fb8c00640 1 --2- 172.21.15.123:0/1309165910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e32c70 0x555b61e3ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:51:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.515+0000 7f5f9d600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:51:37.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.515+0000 7f5fb3400640 1 --2- 172.21.15.123:0/1309165910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e33590 0x555b61e3b260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0012700 tx=0x7f5fa0012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=500db288a652cdab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:37.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.516+0000 7f5f9d600640 1 -- 172.21.15.123:0/1309165910 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0004050 con 0x555b61e33590 2024-09-19T04:51:37.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.516+0000 7f5f9d600640 1 -- 172.21.15.123:0/1309165910 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f5fa00041f0 con 0x555b61e33590 2024-09-19T04:51:37.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.516+0000 7f5fbd5df740 1 -- 172.21.15.123:0/1309165910 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555b61da6950 con 0x555b61e33590 2024-09-19T04:51:37.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.516+0000 7f5f9d600640 1 -- 172.21.15.123:0/1309165910 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5fa00043f0 con 0x555b61e33590 2024-09-19T04:51:37.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.516+0000 7f5fbd5df740 10 client.26228 fetch_fsmap learned FSMap version 36 2024-09-19T04:51:37.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.516+0000 7f5fbd5df740 1 -- 172.21.15.123:0/1309165910 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555b61e4ca10 con 0x555b61e33590 2024-09-19T04:51:37.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.516+0000 7f5f9d600640 1 -- 172.21.15.123:0/1309165910 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555b61da6950 con 0x555b61e33590 2024-09-19T04:51:37.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.517+0000 7f5f9d600640 1 -- 172.21.15.123:0/1309165910 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f5fa0003ca0 con 0x555b61e33590 2024-09-19T04:51:37.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.517+0000 7f5fbd5df740 10 client.26228 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:51:37.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.517+0000 7f5fbd5df740 10 client.26228 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:51:37.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.517+0000 7f5fbd5df740 20 client.26228 populate_metadata read hostname 'smithi123' 2024-09-19T04:51:37.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.517+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/1309165910 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555b61e4df20 0x555b61e503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.517+0000 7f5fbd5df740 4 client.26228 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:51:37.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.517+0000 7f5fbd5df740 1 -- 172.21.15.123:0/1309165910 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b61da0e60 con 0x555b61e4df20 2024-09-19T04:51:37.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.517+0000 7f5fb8c00640 1 --2- 172.21.15.123:0/1309165910 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555b61e4df20 0x555b61e503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:37.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.518+0000 7f5fb8c00640 1 --2- 172.21.15.123:0/1309165910 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555b61e4df20 0x555b61e503a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38fa7ff2a7e312fd server_cookie=b6563b0bd526fdb1 in_seq=0 out_seq=0 2024-09-19T04:51:37.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.518+0000 7f5f9d600640 10 client.26228 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:51:37.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.519+0000 7f5f9d600640 1 -- 172.21.15.123:0/1309165910 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b61da0e60 con 0x555b61e4df20 2024-09-19T04:51:37.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.519+0000 7f5f9d600640 10 client.26228 handle_command_reply: tid=0 2024-09-19T04:51:37.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.519+0000 7f5fbd5df740 1 client.26228 shutdown 2024-09-19T04:51:37.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.519+0000 7f5fbd5df740 1 -- 172.21.15.123:0/1309165910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e33590 msgr2=0x555b61e3b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:37.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.519+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/1309165910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e33590 0x555b61e3b260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0012700 tx=0x7f5fa0012bc0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.519+0000 7f5fbd5df740 1 -- 172.21.15.123:0/1309165910 shutdown_connections 2024-09-19T04:51:37.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.519+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/1309165910 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555b61e4df20 0x555b61e503a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.519+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/1309165910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 0x555b61e3b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.519+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/1309165910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e33590 0x555b61e3b260 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.519+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/1309165910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e32c70 0x555b61e3ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.519+0000 7f5f9d600640 0 client.26228 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:51:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.519+0000 7f5fbd5df740 1 -- 172.21.15.123:0/1309165910 >> 172.21.15.123:0/1309165910 conn(0x555b61e2f200 msgr2=0x555b61e34f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.519+0000 7f5fbd5df740 1 -- 172.21.15.123:0/1309165910 shutdown_connections 2024-09-19T04:51:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.519+0000 7f5fbd5df740 1 -- 172.21.15.123:0/1309165910 wait complete. 2024-09-19T04:51:37.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.520+0000 7f5fbd5df740 20 client.26228 trim_cache size 0 max 16384 2024-09-19T04:51:37.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.534+0000 7f5fbd5df740 1 Processor -- start 2024-09-19T04:51:37.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.534+0000 7f5fbd5df740 1 -- start start 2024-09-19T04:51:37.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.534+0000 7f5fbd5df740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e32c70 0x555b61e2e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.534+0000 7f5fbd5df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e33590 0x555b61e55c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.534+0000 7f5fbd5df740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 0x555b61e561b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.534+0000 7f5fbd5df740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41b3e50 con 0x555b61e32c70 2024-09-19T04:51:37.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.534+0000 7f5fbd5df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41b5990 con 0x555b61e33590 2024-09-19T04:51:37.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.534+0000 7f5fbd5df740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41b4520 con 0x555b61e3de50 2024-09-19T04:51:37.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.535+0000 7f5fb8c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e32c70 0x555b61e2e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:37.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.535+0000 7f5fb3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e33590 0x555b61e55c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:37.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.535+0000 7f5fb8c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e32c70 0x555b61e2e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59810/0 (socket says 172.21.15.123:59810) 2024-09-19T04:51:37.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.535+0000 7f5fb3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e33590 0x555b61e55c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:37752/0 (socket says 172.21.15.123:37752) 2024-09-19T04:51:37.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.535+0000 7f5fb8c00640 1 -- 172.21.15.123:0/3647870227 learned_addr learned my addr 172.21.15.123:0/3647870227 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:37.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.535+0000 7f5fb8c00640 1 --2- 172.21.15.123:0/3647870227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 0x555b61e561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:37.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.535+0000 7f5fb3400640 1 -- 172.21.15.123:0/3647870227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 msgr2=0x555b61e561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:37.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.535+0000 7f5fb3400640 1 --2- 172.21.15.123:0/3647870227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 0x555b61e561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.535+0000 7f5fb3400640 1 -- 172.21.15.123:0/3647870227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e32c70 msgr2=0x555b61e2e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:37.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.535+0000 7f5fb3400640 1 --2- 172.21.15.123:0/3647870227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e32c70 0x555b61e2e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.535+0000 7f5fb3400640 1 -- 172.21.15.123:0/3647870227 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b61e2a770 con 0x555b61e33590 2024-09-19T04:51:37.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.535+0000 7f5fb3400640 1 --2- 172.21.15.123:0/3647870227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e33590 0x555b61e55c70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0010500 tx=0x7f5fa00055a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b83b5b85a11554d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:37.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.536+0000 7f5f9fe00640 1 -- 172.21.15.123:0/3647870227 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0029040 con 0x555b61e33590 2024-09-19T04:51:37.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.536+0000 7f5f9fe00640 1 -- 172.21.15.123:0/3647870227 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa00106f0 con 0x555b61e33590 2024-09-19T04:51:37.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.536+0000 7f5f9fe00640 1 -- 172.21.15.123:0/3647870227 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa001c710 con 0x555b61e33590 2024-09-19T04:51:37.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.537+0000 7f5fbd5df740 1 -- 172.21.15.123:0/3647870227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e33590 msgr2=0x555b61e55c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:37.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.537+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/3647870227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e33590 0x555b61e55c70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0010500 tx=0x7f5fa00055a0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.537+0000 7f5fbd5df740 1 -- 172.21.15.123:0/3647870227 shutdown_connections 2024-09-19T04:51:37.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.537+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/3647870227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 0x555b61e561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.537+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/3647870227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e33590 0x555b61e55c70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.537+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/3647870227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e32c70 0x555b61e2e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.537+0000 7f5fbd5df740 1 -- 172.21.15.123:0/3647870227 >> 172.21.15.123:0/3647870227 conn(0x555b61e2f200 msgr2=0x555b61e2db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:37.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.537+0000 7f5fbd5df740 1 -- 172.21.15.123:0/3647870227 shutdown_connections 2024-09-19T04:51:37.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.537+0000 7f5fbd5df740 1 -- 172.21.15.123:0/3647870227 wait complete. 2024-09-19T04:51:37.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.538+0000 7f5fbd5df740 1 -- start start 2024-09-19T04:51:37.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.538+0000 7f5fbd5df740 1 Processor -- start 2024-09-19T04:51:37.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.539+0000 7f5fbd5df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e32c70 0x555b61e37da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.539+0000 7f5fbd5df740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e33590 0x555b61e382e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.539+0000 7f5fbd5df740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 0x555b61e38820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.539+0000 7f5fbd5df740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41b52c0 con 0x555b61e33590 2024-09-19T04:51:37.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.539+0000 7f5fbd5df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41b4bf0 con 0x555b61e32c70 2024-09-19T04:51:37.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.539+0000 7f5fbd5df740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41b1e30 con 0x555b61e3de50 2024-09-19T04:51:37.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.539+0000 7f5fbd5df740 10 client.0 _ll_register_callbacks cb 0x7f5fa0009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:51:37.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.539+0000 7f5fb3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e33590 0x555b61e382e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:37.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.539+0000 7f5fb3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e33590 0x555b61e382e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59822/0 (socket says 172.21.15.123:59822) 2024-09-19T04:51:37.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.539+0000 7f5fb3400640 1 -- 172.21.15.123:0/2380096603 learned_addr learned my addr 172.21.15.123:0/2380096603 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:37.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.540+0000 7f5fb3400640 1 -- 172.21.15.123:0/2380096603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 msgr2=0x555b61e38820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:51:37.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.540+0000 7f5fb8c00640 1 --2- 172.21.15.123:0/2380096603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 0x555b61e38820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:37.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.540+0000 7f5fb3400640 1 --2- 172.21.15.123:0/2380096603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 0x555b61e38820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.540+0000 7f5fb3400640 1 -- 172.21.15.123:0/2380096603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e32c70 msgr2=0x555b61e37da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:37.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.540+0000 7f5fb8c00640 1 --2- 172.21.15.123:0/2380096603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e32c70 0x555b61e37da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:37.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.540+0000 7f5fb3400640 1 --2- 172.21.15.123:0/2380096603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e32c70 0x555b61e37da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.541+0000 7f5fb3400640 1 -- 172.21.15.123:0/2380096603 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b61e2fe90 con 0x555b61e33590 2024-09-19T04:51:37.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.541+0000 7f5fb3400640 1 --2- 172.21.15.123:0/2380096603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e33590 0x555b61e382e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0012c00 tx=0x7f5fa000f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40f8107db84d824 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:37.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.541+0000 7f5f9d600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:51:37.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.542+0000 7f5f9d600640 1 -- 172.21.15.123:0/2380096603 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0029070 con 0x555b61e33590 2024-09-19T04:51:37.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.542+0000 7f5f9d600640 1 -- 172.21.15.123:0/2380096603 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f5fa0018040 con 0x555b61e33590 2024-09-19T04:51:37.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.542+0000 7f5fbd5df740 1 -- 172.21.15.123:0/2380096603 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fa00106f0 con 0x555b61e33590 2024-09-19T04:51:37.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.542+0000 7f5f9d600640 1 -- 172.21.15.123:0/2380096603 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5fa000fb60 con 0x555b61e33590 2024-09-19T04:51:37.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.542+0000 7f5fbd5df740 10 client.16140 fetch_fsmap learned FSMap version 36 2024-09-19T04:51:37.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.542+0000 7f5fbd5df740 1 -- 172.21.15.123:0/2380096603 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555b61eb8d60 con 0x555b61e33590 2024-09-19T04:51:37.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.542+0000 7f5f9d600640 1 -- 172.21.15.123:0/2380096603 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa00106f0 con 0x555b61e33590 2024-09-19T04:51:37.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.543+0000 7f5f9d600640 1 -- 172.21.15.123:0/2380096603 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f5fa002c070 con 0x555b61e33590 2024-09-19T04:51:37.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.543+0000 7f5fbd5df740 10 client.16140 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:51:37.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.543+0000 7f5fbd5df740 10 client.16140 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:51:37.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.543+0000 7f5fbd5df740 20 client.16140 populate_metadata read hostname 'smithi123' 2024-09-19T04:51:37.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.543+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/2380096603 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555b61eb9310 0x555b61ebb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:37.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.543+0000 7f5fbd5df740 4 client.16140 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:51:37.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.543+0000 7f5fbd5df740 1 -- 172.21.15.123:0/2380096603 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b61d1d010 con 0x555b61eb9310 2024-09-19T04:51:37.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.544+0000 7f5fb8c00640 1 --2- 172.21.15.123:0/2380096603 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555b61eb9310 0x555b61ebb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:37.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.544+0000 7f5fb8c00640 1 --2- 172.21.15.123:0/2380096603 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555b61eb9310 0x555b61ebb700 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa5dd765645d06ca server_cookie=c0fd225851c24be2 in_seq=0 out_seq=0 2024-09-19T04:51:37.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.544+0000 7f5f9d600640 10 client.16140 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:51:37.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.544+0000 7f5f9d600640 1 -- 172.21.15.123:0/2380096603 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b61d1d010 con 0x555b61eb9310 2024-09-19T04:51:37.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.544+0000 7f5f9d600640 10 client.16140 handle_command_reply: tid=0 2024-09-19T04:51:37.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.544+0000 7f5fbd5df740 1 client.16140 shutdown 2024-09-19T04:51:37.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.545+0000 7f5fbd5df740 1 -- 172.21.15.123:0/2380096603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e33590 msgr2=0x555b61e382e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:37.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.545+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/2380096603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e33590 0x555b61e382e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0012c00 tx=0x7f5fa000f810 comp rx=0 tx=0).stop 2024-09-19T04:51:37.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.545+0000 7f5fbd5df740 1 -- 172.21.15.123:0/2380096603 shutdown_connections 2024-09-19T04:51:37.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.545+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/2380096603 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555b61eb9310 0x555b61ebb700 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.545+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/2380096603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b61e3de50 0x555b61e38820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.545+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/2380096603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b61e33590 0x555b61e382e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.545+0000 7f5fbd5df740 1 --2- 172.21.15.123:0/2380096603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b61e32c70 0x555b61e37da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.545+0000 7f5fbd5df740 1 -- 172.21.15.123:0/2380096603 >> 172.21.15.123:0/2380096603 conn(0x555b61e2f200 msgr2=0x555b61e3cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:37.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.545+0000 7f5f9d600640 0 client.16140 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:51:37.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.545+0000 7f5fbd5df740 1 -- 172.21.15.123:0/2380096603 shutdown_connections 2024-09-19T04:51:37.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.545+0000 7f5fbd5df740 1 -- 172.21.15.123:0/2380096603 wait complete. 2024-09-19T04:51:37.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.545+0000 7f5fbd5df740 20 client.16140 trim_cache size 0 max 16384 2024-09-19T04:51:37.548 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:51:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.546+0000 7f5fbac00640 1 -- 172.21.15.123:0/1509882555 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5f980a34b0 msgr2=0x7f5f980a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.546+0000 7f5fbac00640 1 --2- 172.21.15.123:0/1509882555 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5f980a34b0 0x7f5f980a5970 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x555b61e30060 tx=0x7f5fa4031490 comp rx=0 tx=0).stop 2024-09-19T04:51:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.546+0000 7f5fbac00640 1 -- 172.21.15.123:0/1509882555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fb41008c0 msgr2=0x7f5fb41a4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.546+0000 7f5fbac00640 1 --2- 172.21.15.123:0/1509882555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fb41008c0 0x7f5fb41a4850 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5fa402a940 tx=0x7f5fa4003b20 comp rx=0 tx=0).stop 2024-09-19T04:51:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.546+0000 7f5fbac00640 1 -- 172.21.15.123:0/1509882555 shutdown_connections 2024-09-19T04:51:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.546+0000 7f5fbac00640 1 --2- 172.21.15.123:0/1509882555 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5f980a34b0 0x7f5f980a5970 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.546+0000 7f5fbac00640 1 --2- 172.21.15.123:0/1509882555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fb4105ec0 0x7f5fb419ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.546+0000 7f5fbac00640 1 --2- 172.21.15.123:0/1509882555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fb41011e0 0x7f5fb41a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.546+0000 7f5fbac00640 1 --2- 172.21.15.123:0/1509882555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fb41008c0 0x7f5fb41a4850 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.546+0000 7f5fbac00640 1 -- 172.21.15.123:0/1509882555 >> 172.21.15.123:0/1509882555 conn(0x7f5fb40fc7d0 msgr2=0x7f5fb4103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.546+0000 7f5fbac00640 1 -- 172.21.15.123:0/1509882555 shutdown_connections 2024-09-19T04:51:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:37.546+0000 7f5fbac00640 1 -- 172.21.15.123:0/1509882555 wait complete. 2024-09-19T04:51:37.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:37 smithi183 ceph-mon[35333]: pgmap v150: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 543 op/s 2024-09-19T04:51:38.510 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:51:38.941 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:51:39.389 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:39 smithi123 ceph-mon[30511]: pgmap v151: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 812 op/s 2024-09-19T04:51:39.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:39 smithi193 ceph-mon[35592]: pgmap v151: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 812 op/s 2024-09-19T04:51:39.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.516+0000 7fd5e2000640 1 -- 172.21.15.123:0/2671604817 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40319b0 con 0x7fd5e4102d90 2024-09-19T04:51:39.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.516+0000 7fd5e9800640 1 -- 172.21.15.123:0/2671604817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e4102d90 msgr2=0x7fd5e4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:39.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.516+0000 7fd5e9800640 1 --2- 172.21.15.123:0/2671604817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e4102d90 0x7fd5e4103170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009900 tx=0x7fd5d402a480 comp rx=0 tx=0).stop 2024-09-19T04:51:39.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.517+0000 7fd5e9800640 1 -- 172.21.15.123:0/2671604817 shutdown_connections 2024-09-19T04:51:39.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.517+0000 7fd5e9800640 1 --2- 172.21.15.123:0/2671604817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e410c170 0x7fd5e410e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.517+0000 7fd5e9800640 1 --2- 172.21.15.123:0/2671604817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5e41036b0 0x7fd5e410bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.517+0000 7fd5e9800640 1 --2- 172.21.15.123:0/2671604817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e4102d90 0x7fd5e4103170 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.517+0000 7fd5e9800640 1 -- 172.21.15.123:0/2671604817 >> 172.21.15.123:0/2671604817 conn(0x7fd5e40fc790 msgr2=0x7fd5e40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:39.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.517+0000 7fd5e9800640 1 -- 172.21.15.123:0/2671604817 shutdown_connections 2024-09-19T04:51:39.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.517+0000 7fd5e9800640 1 -- 172.21.15.123:0/2671604817 wait complete. 2024-09-19T04:51:39.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.518+0000 7fd5e9800640 1 Processor -- start 2024-09-19T04:51:39.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.518+0000 7fd5e9800640 1 -- start start 2024-09-19T04:51:39.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.518+0000 7fd5e9800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5e4102d90 0x7fd5e419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.518+0000 7fd5e9800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e41036b0 0x7fd5e419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.518+0000 7fd5e9800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e410c170 0x7fd5e4196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.518+0000 7fd5e9800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4100aa0 con 0x7fd5e4102d90 2024-09-19T04:51:39.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.518+0000 7fd5e9800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4100920 con 0x7fd5e410c170 2024-09-19T04:51:39.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.518+0000 7fd5e9800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4100c20 con 0x7fd5e41036b0 2024-09-19T04:51:39.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.519+0000 7fd5e3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e410c170 0x7fd5e4196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:39.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.519+0000 7fd5e2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5e4102d90 0x7fd5e419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:39.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.519+0000 7fd5e3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e410c170 0x7fd5e4196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:37780/0 (socket says 172.21.15.123:37780) 2024-09-19T04:51:39.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.519+0000 7fd5e3400640 1 -- 172.21.15.123:0/2658263772 learned_addr learned my addr 172.21.15.123:0/2658263772 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:39.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.519+0000 7fd5dbe00640 1 --2- 172.21.15.123:0/2658263772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e41036b0 0x7fd5e419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:39.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.519+0000 7fd5e2a00640 1 -- 172.21.15.123:0/2658263772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e41036b0 msgr2=0x7fd5e419b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:39.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.519+0000 7fd5e2a00640 1 --2- 172.21.15.123:0/2658263772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e41036b0 0x7fd5e419b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.519+0000 7fd5e2a00640 1 -- 172.21.15.123:0/2658263772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e410c170 msgr2=0x7fd5e4196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:39.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.519+0000 7fd5e2a00640 1 --2- 172.21.15.123:0/2658263772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e410c170 0x7fd5e4196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.519+0000 7fd5e2a00640 1 -- 172.21.15.123:0/2658263772 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e4197300 con 0x7fd5e4102d90 2024-09-19T04:51:39.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.519+0000 7fd5e3400640 1 --2- 172.21.15.123:0/2658263772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e410c170 0x7fd5e4196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:39.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.520+0000 7fd5e2a00640 1 --2- 172.21.15.123:0/2658263772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5e4102d90 0x7fd5e419af60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4005940 tx=0x7fd5d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:39.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.520+0000 7fd5db400640 1 -- 172.21.15.123:0/2658263772 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d403f030 con 0x7fd5e4102d90 2024-09-19T04:51:39.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.520+0000 7fd5db400640 1 -- 172.21.15.123:0/2658263772 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d4003ad0 con 0x7fd5e4102d90 2024-09-19T04:51:39.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.520+0000 7fd5e9800640 1 -- 172.21.15.123:0/2658263772 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e4197590 con 0x7fd5e4102d90 2024-09-19T04:51:39.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.520+0000 7fd5db400640 1 -- 172.21.15.123:0/2658263772 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4035070 con 0x7fd5e4102d90 2024-09-19T04:51:39.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.520+0000 7fd5e9800640 1 -- 172.21.15.123:0/2658263772 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e41a2c50 con 0x7fd5e4102d90 2024-09-19T04:51:39.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.523+0000 7fd5db400640 1 -- 172.21.15.123:0/2658263772 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd5d4042030 con 0x7fd5e4102d90 2024-09-19T04:51:39.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.524+0000 7fd5ec250740 1 Processor -- start 2024-09-19T04:51:39.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.524+0000 7fd5ec250740 1 -- start start 2024-09-19T04:51:39.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.524+0000 7fd5ec250740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 0x5577e3068050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.524+0000 7fd5ec250740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e3072910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.524+0000 7fd5ec250740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3072e50 0x5577e3075240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.524+0000 7fd5ec250740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5577e2fce3e0 con 0x5577e3072e50 2024-09-19T04:51:39.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.524+0000 7fd5ec250740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5577e2f75d80 con 0x5577e3068590 2024-09-19T04:51:39.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.524+0000 7fd5ec250740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5577e2fca3a0 con 0x5577e3067c70 2024-09-19T04:51:39.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.525+0000 7fd5e3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 0x5577e3068050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:39.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.525+0000 7fd5e3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 0x5577e3068050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53708/0 (socket says 172.21.15.123:53708) 2024-09-19T04:51:39.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.525+0000 7fd5e3400640 1 -- 172.21.15.123:0/178252854 learned_addr learned my addr 172.21.15.123:0/178252854 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:39.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.525+0000 7fd5e2a00640 1 --2- 172.21.15.123:0/178252854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e3072910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:39.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.525+0000 7fd5e3400640 1 -- 172.21.15.123:0/178252854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 msgr2=0x5577e3072910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:39.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.525+0000 7fd5e3400640 1 --2- 172.21.15.123:0/178252854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e3072910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.525+0000 7fd5e3400640 1 -- 172.21.15.123:0/178252854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3072e50 msgr2=0x5577e3075240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:39.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.525+0000 7fd5e3400640 1 --2- 172.21.15.123:0/178252854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3072e50 0x5577e3075240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.525+0000 7fd5e3400640 1 -- 172.21.15.123:0/178252854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e2f94ae0 con 0x5577e3067c70 2024-09-19T04:51:39.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.525+0000 7fd5db400640 1 --2- 172.21.15.123:0/2658263772 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd5c80a77c0 0x7fd5c80a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.525+0000 7fd5e3400640 1 --2- 172.21.15.123:0/178252854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 0x5577e3068050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0117b0 tx=0x7fd5dc011c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0bcc3418e7f317 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:39.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.526+0000 7fd5e2a00640 1 --2- 172.21.15.123:0/2658263772 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd5c80a77c0 0x7fd5c80a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:39.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.526+0000 7fd5d3e00640 1 -- 172.21.15.123:0/178252854 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc004d20 con 0x5577e3067c70 2024-09-19T04:51:39.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.526+0000 7fd5d3e00640 1 -- 172.21.15.123:0/178252854 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5dc00fe20 con 0x5577e3067c70 2024-09-19T04:51:39.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.526+0000 7fd5d3e00640 1 -- 172.21.15.123:0/178252854 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0058e0 con 0x5577e3067c70 2024-09-19T04:51:39.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.526+0000 7fd5db400640 1 -- 172.21.15.123:0/2658263772 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd5d40f0350 con 0x7fd5e4102d90 2024-09-19T04:51:39.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.527+0000 7fd5ec250740 1 -- 172.21.15.123:0/178252854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 msgr2=0x5577e3068050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:39.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.527+0000 7fd5ec250740 1 --2- 172.21.15.123:0/178252854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 0x5577e3068050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0117b0 tx=0x7fd5dc011c70 comp rx=0 tx=0).stop 2024-09-19T04:51:39.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.527+0000 7fd5ec250740 1 -- 172.21.15.123:0/178252854 shutdown_connections 2024-09-19T04:51:39.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.527+0000 7fd5ec250740 1 --2- 172.21.15.123:0/178252854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3072e50 0x5577e3075240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.527+0000 7fd5ec250740 1 --2- 172.21.15.123:0/178252854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e3072910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.527+0000 7fd5ec250740 1 --2- 172.21.15.123:0/178252854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 0x5577e3068050 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.527+0000 7fd5ec250740 1 -- 172.21.15.123:0/178252854 >> 172.21.15.123:0/178252854 conn(0x5577e3064200 msgr2=0x5577e30645e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:39.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.527+0000 7fd5ec250740 1 -- 172.21.15.123:0/178252854 shutdown_connections 2024-09-19T04:51:39.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.527+0000 7fd5e2a00640 1 --2- 172.21.15.123:0/2658263772 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd5c80a77c0 0x7fd5c80a9c80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5577e3065060 tx=0x7fd5d40f3e00 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:39.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.527+0000 7fd5ec250740 1 -- 172.21.15.123:0/178252854 wait complete. 2024-09-19T04:51:39.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.528+0000 7fd5ec250740 1 -- start start 2024-09-19T04:51:39.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.528+0000 7fd5ec250740 1 Processor -- start 2024-09-19T04:51:39.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.529+0000 7fd5ec250740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3067c70 0x5577e306fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.529+0000 7fd5ec250740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e30702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.529+0000 7fd5ec250740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3072e50 0x5577e3070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.529+0000 7fd5ec250740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5577e3075a20 con 0x5577e3067c70 2024-09-19T04:51:39.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.529+0000 7fd5ec250740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5577e2f890f0 con 0x5577e3068590 2024-09-19T04:51:39.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.529+0000 7fd5ec250740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5577e3075ba0 con 0x5577e3072e50 2024-09-19T04:51:39.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.529+0000 7fd5ec250740 10 client.0 _ll_register_callbacks cb 0x5577e2f70bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:51:39.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.529+0000 7fd5e3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3067c70 0x5577e306fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:39.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.529+0000 7fd5dbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e30702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:39.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.529+0000 7fd5e3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3067c70 0x5577e306fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59842/0 (socket says 172.21.15.123:59842) 2024-09-19T04:51:39.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.529+0000 7fd5e3400640 1 -- 172.21.15.123:0/908920082 learned_addr learned my addr 172.21.15.123:0/908920082 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:39.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.529+0000 7fd5e3400640 1 --2- 172.21.15.123:0/908920082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3072e50 0x5577e3070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:39.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.530+0000 7fd5dbe00640 1 -- 172.21.15.123:0/908920082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3072e50 msgr2=0x5577e3070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:39.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.530+0000 7fd5dbe00640 1 --2- 172.21.15.123:0/908920082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3072e50 0x5577e3070820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.530+0000 7fd5dbe00640 1 -- 172.21.15.123:0/908920082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3067c70 msgr2=0x5577e306fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:39.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.530+0000 7fd5dbe00640 1 --2- 172.21.15.123:0/908920082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3067c70 0x5577e306fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.530+0000 7fd5dbe00640 1 -- 172.21.15.123:0/908920082 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e2d3f3c0 con 0x5577e3068590 2024-09-19T04:51:39.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.530+0000 7fd5dbe00640 1 --2- 172.21.15.123:0/908920082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e30702e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd5e410bef0 tx=0x7fd5cc011920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=480920ec7e7dda5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:39.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.530+0000 7fd5d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:51:39.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.530+0000 7fd5d0c00640 1 -- 172.21.15.123:0/908920082 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc0035f0 con 0x5577e3068590 2024-09-19T04:51:39.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.530+0000 7fd5ec250740 1 -- 172.21.15.123:0/908920082 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5dc00fe20 con 0x5577e3068590 2024-09-19T04:51:39.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.530+0000 7fd5d0c00640 1 -- 172.21.15.123:0/908920082 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd5cc003790 con 0x5577e3068590 2024-09-19T04:51:39.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.530+0000 7fd5d0c00640 1 -- 172.21.15.123:0/908920082 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd5cc003990 con 0x5577e3068590 2024-09-19T04:51:39.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.530+0000 7fd5ec250740 10 client.26243 fetch_fsmap learned FSMap version 36 2024-09-19T04:51:39.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.530+0000 7fd5ec250740 1 -- 172.21.15.123:0/908920082 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5577e3081b50 con 0x5577e3068590 2024-09-19T04:51:39.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.531+0000 7fd5d0c00640 1 -- 172.21.15.123:0/908920082 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5dc00fe20 con 0x5577e3068590 2024-09-19T04:51:39.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.531+0000 7fd5d0c00640 1 -- 172.21.15.123:0/908920082 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fd5cc019430 con 0x5577e3068590 2024-09-19T04:51:39.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.531+0000 7fd5ec250740 10 client.26243 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:51:39.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.531+0000 7fd5ec250740 10 client.26243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:51:39.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.531+0000 7fd5ec250740 20 client.26243 populate_metadata read hostname 'smithi123' 2024-09-19T04:51:39.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.531+0000 7fd5ec250740 1 --2- 172.21.15.123:0/908920082 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5577e3083010 0x5577e3085400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.531+0000 7fd5ec250740 4 client.26243 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:51:39.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.531+0000 7fd5ec250740 1 -- 172.21.15.123:0/908920082 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577e2fdb950 con 0x5577e3083010 2024-09-19T04:51:39.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.531+0000 7fd5e3400640 1 --2- 172.21.15.123:0/908920082 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5577e3083010 0x5577e3085400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:39.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.532+0000 7fd5e3400640 1 --2- 172.21.15.123:0/908920082 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5577e3083010 0x5577e3085400 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f431ff762fa269 server_cookie=9e700c46a2ae4f96 in_seq=0 out_seq=0 2024-09-19T04:51:39.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.532+0000 7fd5d0c00640 10 client.26243 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:51:39.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.532+0000 7fd5d0c00640 1 -- 172.21.15.123:0/908920082 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577e2fdb950 con 0x5577e3083010 2024-09-19T04:51:39.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.532+0000 7fd5d0c00640 10 client.26243 handle_command_reply: tid=0 2024-09-19T04:51:39.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.532+0000 7fd5ec250740 1 client.26243 shutdown 2024-09-19T04:51:39.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.533+0000 7fd5ec250740 1 -- 172.21.15.123:0/908920082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 msgr2=0x5577e30702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:39.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.533+0000 7fd5ec250740 1 --2- 172.21.15.123:0/908920082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e30702e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd5e410bef0 tx=0x7fd5cc011920 comp rx=0 tx=0).stop 2024-09-19T04:51:39.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.533+0000 7fd5ec250740 1 -- 172.21.15.123:0/908920082 shutdown_connections 2024-09-19T04:51:39.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.533+0000 7fd5ec250740 1 --2- 172.21.15.123:0/908920082 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5577e3083010 0x5577e3085400 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.533+0000 7fd5ec250740 1 --2- 172.21.15.123:0/908920082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3072e50 0x5577e3070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.533+0000 7fd5ec250740 1 --2- 172.21.15.123:0/908920082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e30702e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.533+0000 7fd5ec250740 1 --2- 172.21.15.123:0/908920082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3067c70 0x5577e306fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.533+0000 7fd5ec250740 1 -- 172.21.15.123:0/908920082 >> 172.21.15.123:0/908920082 conn(0x5577e3064200 msgr2=0x5577e3063380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:39.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.533+0000 7fd5d0c00640 0 client.26243 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:51:39.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.533+0000 7fd5ec250740 1 -- 172.21.15.123:0/908920082 shutdown_connections 2024-09-19T04:51:39.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.533+0000 7fd5ec250740 1 -- 172.21.15.123:0/908920082 wait complete. 2024-09-19T04:51:39.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.533+0000 7fd5ec250740 20 client.26243 trim_cache size 0 max 16384 2024-09-19T04:51:39.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.548+0000 7fd5ec250740 1 Processor -- start 2024-09-19T04:51:39.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.548+0000 7fd5ec250740 1 -- start start 2024-09-19T04:51:39.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.548+0000 7fd5ec250740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 0x5577e308acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.549+0000 7fd5ec250740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e308b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.549+0000 7fd5ec250740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3072e50 0x5577e30696c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.549+0000 7fd5ec250740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e41ab620 con 0x5577e3072e50 2024-09-19T04:51:39.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.549+0000 7fd5ec250740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e41ad160 con 0x5577e3068590 2024-09-19T04:51:39.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.549+0000 7fd5ec250740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e41abcf0 con 0x5577e3067c70 2024-09-19T04:51:39.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.549+0000 7fd5e3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 0x5577e308acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:39.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.549+0000 7fd5e3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3072e50 0x5577e30696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:39.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.549+0000 7fd5e3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3072e50 0x5577e30696c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59854/0 (socket says 172.21.15.123:59854) 2024-09-19T04:51:39.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.549+0000 7fd5e3400640 1 -- 172.21.15.123:0/3550951618 learned_addr learned my addr 172.21.15.123:0/3550951618 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:39.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.549+0000 7fd5dbe00640 1 --2- 172.21.15.123:0/3550951618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e308b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.549+0000 7fd5dbe00640 1 -- 172.21.15.123:0/3550951618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 msgr2=0x5577e308acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.549+0000 7fd5dbe00640 1 --2- 172.21.15.123:0/3550951618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 0x5577e308acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.549+0000 7fd5dbe00640 1 -- 172.21.15.123:0/3550951618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3072e50 msgr2=0x5577e30696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.549+0000 7fd5dbe00640 1 --2- 172.21.15.123:0/3550951618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3072e50 0x5577e30696c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.549+0000 7fd5dbe00640 1 -- 172.21.15.123:0/3550951618 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e306e2f0 con 0x5577e3068590 2024-09-19T04:51:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.549+0000 7fd5e3400640 1 --2- 172.21.15.123:0/3550951618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3072e50 0x5577e30696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.550+0000 7fd5dbe00640 1 --2- 172.21.15.123:0/3550951618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e308b230 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc0199c0 tx=0x7fd5cc005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f889ae0e4be18cc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.550+0000 7fd5d3400640 1 -- 172.21.15.123:0/3550951618 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc000e60 con 0x5577e3068590 2024-09-19T04:51:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.550+0000 7fd5d3400640 1 -- 172.21.15.123:0/3550951618 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5cc0035f0 con 0x5577e3068590 2024-09-19T04:51:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.550+0000 7fd5d3400640 1 -- 172.21.15.123:0/3550951618 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc005070 con 0x5577e3068590 2024-09-19T04:51:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.551+0000 7fd5ec250740 1 -- 172.21.15.123:0/3550951618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 msgr2=0x5577e308b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.551+0000 7fd5ec250740 1 --2- 172.21.15.123:0/3550951618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e308b230 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc0199c0 tx=0x7fd5cc005f10 comp rx=0 tx=0).stop 2024-09-19T04:51:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.551+0000 7fd5ec250740 1 -- 172.21.15.123:0/3550951618 shutdown_connections 2024-09-19T04:51:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.551+0000 7fd5ec250740 1 --2- 172.21.15.123:0/3550951618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3072e50 0x5577e30696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.551+0000 7fd5ec250740 1 --2- 172.21.15.123:0/3550951618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e308b230 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.551+0000 7fd5ec250740 1 --2- 172.21.15.123:0/3550951618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 0x5577e308acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.551+0000 7fd5ec250740 1 -- 172.21.15.123:0/3550951618 >> 172.21.15.123:0/3550951618 conn(0x5577e3064200 msgr2=0x5577e306b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.551+0000 7fd5ec250740 1 -- 172.21.15.123:0/3550951618 shutdown_connections 2024-09-19T04:51:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.551+0000 7fd5ec250740 1 -- 172.21.15.123:0/3550951618 wait complete. 2024-09-19T04:51:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.551+0000 7fd5ec250740 1 -- start start 2024-09-19T04:51:39.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.552+0000 7fd5ec250740 1 Processor -- start 2024-09-19T04:51:39.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.552+0000 7fd5ec250740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 0x5577e307ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.552+0000 7fd5ec250740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e307b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.552+0000 7fd5ec250740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3072e50 0x5577e3077df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.552+0000 7fd5ec250740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e41aca90 con 0x5577e3072e50 2024-09-19T04:51:39.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.552+0000 7fd5ec250740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e41ac3c0 con 0x5577e3068590 2024-09-19T04:51:39.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.552+0000 7fd5ec250740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e41a9600 con 0x5577e3067c70 2024-09-19T04:51:39.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.552+0000 7fd5ec250740 10 client.0 _ll_register_callbacks cb 0x7fd5cc009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:51:39.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.552+0000 7fd5e3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 0x5577e307ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:39.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.553+0000 7fd5e3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 0x5577e307ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53734/0 (socket says 172.21.15.123:53734) 2024-09-19T04:51:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.553+0000 7fd5e3400640 1 -- 172.21.15.123:0/801058590 learned_addr learned my addr 172.21.15.123:0/801058590 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.553+0000 7fd5dbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e307b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.553+0000 7fd5e3400640 1 --2- 172.21.15.123:0/801058590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3072e50 0x5577e3077df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.553+0000 7fd5dbe00640 1 -- 172.21.15.123:0/801058590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 msgr2=0x5577e307ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.553+0000 7fd5dbe00640 1 --2- 172.21.15.123:0/801058590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 0x5577e307ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.553+0000 7fd5dbe00640 1 -- 172.21.15.123:0/801058590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3072e50 msgr2=0x5577e3077df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.553+0000 7fd5dbe00640 1 --2- 172.21.15.123:0/801058590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3072e50 0x5577e3077df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.553+0000 7fd5dbe00640 1 -- 172.21.15.123:0/801058590 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e307ba50 con 0x5577e3068590 2024-09-19T04:51:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.553+0000 7fd5dbe00640 1 --2- 172.21.15.123:0/801058590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e307b360 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5577e3070f10 tx=0x7fd5cc0045d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24b074f6e6dc9baf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.553+0000 7fd5d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:51:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.554+0000 7fd5d0c00640 1 -- 172.21.15.123:0/801058590 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc038060 con 0x5577e3068590 2024-09-19T04:51:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.554+0000 7fd5d0c00640 1 -- 172.21.15.123:0/801058590 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fd5cc019d40 con 0x5577e3068590 2024-09-19T04:51:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.554+0000 7fd5ec250740 1 -- 172.21.15.123:0/801058590 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5cc0035f0 con 0x5577e3068590 2024-09-19T04:51:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.554+0000 7fd5d0c00640 1 -- 172.21.15.123:0/801058590 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd5cc0037a0 con 0x5577e3068590 2024-09-19T04:51:39.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.554+0000 7fd5ec250740 10 client.26249 fetch_fsmap learned FSMap version 36 2024-09-19T04:51:39.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.554+0000 7fd5ec250740 1 -- 172.21.15.123:0/801058590 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5577e30ee2a0 con 0x5577e3068590 2024-09-19T04:51:39.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.555+0000 7fd5d0c00640 1 -- 172.21.15.123:0/801058590 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5cc0035f0 con 0x5577e3068590 2024-09-19T04:51:39.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.555+0000 7fd5d0c00640 1 -- 172.21.15.123:0/801058590 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fd5cc00fab0 con 0x5577e3068590 2024-09-19T04:51:39.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.555+0000 7fd5ec250740 10 client.26249 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:51:39.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.556+0000 7fd5ec250740 10 client.26249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:51:39.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.556+0000 7fd5ec250740 20 client.26249 populate_metadata read hostname 'smithi123' 2024-09-19T04:51:39.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.556+0000 7fd5ec250740 1 --2- 172.21.15.123:0/801058590 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5577e30ee860 0x5577e30f0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:39.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.556+0000 7fd5ec250740 4 client.26249 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:51:39.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.556+0000 7fd5ec250740 1 -- 172.21.15.123:0/801058590 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577e2fd5e60 con 0x5577e30ee860 2024-09-19T04:51:39.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.556+0000 7fd5e3400640 1 --2- 172.21.15.123:0/801058590 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5577e30ee860 0x5577e30f0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:39.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.557+0000 7fd5e3400640 1 --2- 172.21.15.123:0/801058590 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5577e30ee860 0x5577e30f0c50 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50481ed71ca24818 server_cookie=46355c8199c4508c in_seq=0 out_seq=0 2024-09-19T04:51:39.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.557+0000 7fd5d0c00640 10 client.26249 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:51:39.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.558+0000 7fd5d0c00640 1 -- 172.21.15.123:0/801058590 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577e2fd5e60 con 0x5577e30ee860 2024-09-19T04:51:39.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.558+0000 7fd5d0c00640 10 client.26249 handle_command_reply: tid=0 2024-09-19T04:51:39.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.558+0000 7fd5ec250740 1 client.26249 shutdown 2024-09-19T04:51:39.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.558+0000 7fd5ec250740 1 -- 172.21.15.123:0/801058590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 msgr2=0x5577e307b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:39.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.558+0000 7fd5ec250740 1 --2- 172.21.15.123:0/801058590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e307b360 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5577e3070f10 tx=0x7fd5cc0045d0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.560+0000 7fd5ec250740 1 -- 172.21.15.123:0/801058590 shutdown_connections 2024-09-19T04:51:39.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.560+0000 7fd5ec250740 1 --2- 172.21.15.123:0/801058590 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5577e30ee860 0x5577e30f0c50 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.560+0000 7fd5ec250740 1 --2- 172.21.15.123:0/801058590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577e3072e50 0x5577e3077df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.560+0000 7fd5ec250740 1 --2- 172.21.15.123:0/801058590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577e3068590 0x5577e307b360 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.560+0000 7fd5ec250740 1 --2- 172.21.15.123:0/801058590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5577e3067c70 0x5577e307ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.560+0000 7fd5ec250740 1 -- 172.21.15.123:0/801058590 >> 172.21.15.123:0/801058590 conn(0x5577e3064200 msgr2=0x5577e3071600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:39.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.560+0000 7fd5d0c00640 0 client.26249 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:51:39.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.560+0000 7fd5ec250740 1 -- 172.21.15.123:0/801058590 shutdown_connections 2024-09-19T04:51:39.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.560+0000 7fd5ec250740 1 -- 172.21.15.123:0/801058590 wait complete. 2024-09-19T04:51:39.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.561+0000 7fd5ec250740 20 client.26249 trim_cache size 0 max 16384 2024-09-19T04:51:39.563 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:51:39.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.561+0000 7fd5e9800640 1 -- 172.21.15.123:0/2658263772 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd5c80a77c0 msgr2=0x7fd5c80a9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:39.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.561+0000 7fd5e9800640 1 --2- 172.21.15.123:0/2658263772 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd5c80a77c0 0x7fd5c80a9c80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5577e3065060 tx=0x7fd5d40f3e00 comp rx=0 tx=0).stop 2024-09-19T04:51:39.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.561+0000 7fd5e9800640 1 -- 172.21.15.123:0/2658263772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5e4102d90 msgr2=0x7fd5e419af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:39.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.561+0000 7fd5e9800640 1 --2- 172.21.15.123:0/2658263772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5e4102d90 0x7fd5e419af60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4005940 tx=0x7fd5d4004270 comp rx=0 tx=0).stop 2024-09-19T04:51:39.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.562+0000 7fd5e9800640 1 -- 172.21.15.123:0/2658263772 shutdown_connections 2024-09-19T04:51:39.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.562+0000 7fd5e9800640 1 --2- 172.21.15.123:0/2658263772 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd5c80a77c0 0x7fd5c80a9c80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.562+0000 7fd5e9800640 1 --2- 172.21.15.123:0/2658263772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e410c170 0x7fd5e4196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.562+0000 7fd5e9800640 1 --2- 172.21.15.123:0/2658263772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e41036b0 0x7fd5e419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.562+0000 7fd5e9800640 1 --2- 172.21.15.123:0/2658263772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5e4102d90 0x7fd5e419af60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:39.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.562+0000 7fd5e9800640 1 -- 172.21.15.123:0/2658263772 >> 172.21.15.123:0/2658263772 conn(0x7fd5e40fc790 msgr2=0x7fd5e410c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:39.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.562+0000 7fd5e9800640 1 -- 172.21.15.123:0/2658263772 shutdown_connections 2024-09-19T04:51:39.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:39.562+0000 7fd5e9800640 1 -- 172.21.15.123:0/2658263772 wait complete. 2024-09-19T04:51:39.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:39 smithi183 ceph-mon[35333]: pgmap v151: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 812 op/s 2024-09-19T04:51:41.388 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:51:41.389 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:51:41.426 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:41 smithi123 ceph-mon[30511]: pgmap v152: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.5 MiB/s wr, 537 op/s 2024-09-19T04:51:41.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:41 smithi193 ceph-mon[35592]: pgmap v152: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.5 MiB/s wr, 537 op/s 2024-09-19T04:51:41.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:41 smithi183 ceph-mon[35333]: pgmap v152: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.5 MiB/s wr, 537 op/s 2024-09-19T04:51:41.812 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:51:42.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.377+0000 7fbea1a00640 1 -- 172.21.15.123:0/4051645667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c103ff0 msgr2=0x7fbe9c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:42.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.377+0000 7fbea1a00640 1 --2- 172.21.15.123:0/4051645667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c103ff0 0x7fbe9c1043d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009900 tx=0x7fbe8c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.377+0000 7fbea1a00640 1 -- 172.21.15.123:0/4051645667 shutdown_connections 2024-09-19T04:51:42.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.377+0000 7fbea1a00640 1 --2- 172.21.15.123:0/4051645667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe9c109120 0x7fbe9c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.377+0000 7fbea1a00640 1 --2- 172.21.15.123:0/4051645667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe9c1049a0 0x7fbe9c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.377+0000 7fbea1a00640 1 --2- 172.21.15.123:0/4051645667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c103ff0 0x7fbe9c1043d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.377+0000 7fbea1a00640 1 -- 172.21.15.123:0/4051645667 >> 172.21.15.123:0/4051645667 conn(0x7fbe9c0ffa90 msgr2=0x7fbe9c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:42.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.377+0000 7fbea1a00640 1 -- 172.21.15.123:0/4051645667 shutdown_connections 2024-09-19T04:51:42.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.377+0000 7fbea1a00640 1 -- 172.21.15.123:0/4051645667 wait complete. 2024-09-19T04:51:42.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.378+0000 7fbea1a00640 1 Processor -- start 2024-09-19T04:51:42.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.378+0000 7fbea1a00640 1 -- start start 2024-09-19T04:51:42.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.379+0000 7fbea1a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe9c103ff0 0x7fbe9c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.379+0000 7fbea1a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c1049a0 0x7fbe9c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.379+0000 7fbea1a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe9c109120 0x7fbe9c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.379+0000 7fbea1a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c114150 con 0x7fbe9c103ff0 2024-09-19T04:51:42.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.379+0000 7fbea1a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c113fd0 con 0x7fbe9c1049a0 2024-09-19T04:51:42.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.379+0000 7fbea1a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1142d0 con 0x7fbe9c109120 2024-09-19T04:51:42.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.379+0000 7fbe9a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c1049a0 0x7fbe9c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.379+0000 7fbe9b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe9c109120 0x7fbe9c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.379+0000 7fbe9aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe9c103ff0 0x7fbe9c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.379+0000 7fbe9b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe9c109120 0x7fbe9c19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53126/0 (socket says 172.21.15.123:53126) 2024-09-19T04:51:42.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.379+0000 7fbe9b400640 1 -- 172.21.15.123:0/2887094307 learned_addr learned my addr 172.21.15.123:0/2887094307 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:42.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.379+0000 7fbe9a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c1049a0 0x7fbe9c19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:41752/0 (socket says 172.21.15.123:41752) 2024-09-19T04:51:42.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.380+0000 7fbe9a000640 1 -- 172.21.15.123:0/2887094307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe9c109120 msgr2=0x7fbe9c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:42.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.380+0000 7fbe9a000640 1 --2- 172.21.15.123:0/2887094307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe9c109120 0x7fbe9c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.380+0000 7fbe9a000640 1 -- 172.21.15.123:0/2887094307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe9c103ff0 msgr2=0x7fbe9c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:42.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.380+0000 7fbe9a000640 1 --2- 172.21.15.123:0/2887094307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe9c103ff0 0x7fbe9c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.380+0000 7fbe9a000640 1 -- 172.21.15.123:0/2887094307 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c19f620 con 0x7fbe9c1049a0 2024-09-19T04:51:42.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.380+0000 7fbe9a000640 1 --2- 172.21.15.123:0/2887094307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c1049a0 0x7fbe9c19abb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800e9f0 tx=0x7fbe8800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:42.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.380+0000 7fbe93400640 1 -- 172.21.15.123:0/2887094307 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8800ccc0 con 0x7fbe9c1049a0 2024-09-19T04:51:42.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.380+0000 7fbe93400640 1 -- 172.21.15.123:0/2887094307 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe88010910 con 0x7fbe9c1049a0 2024-09-19T04:51:42.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.380+0000 7fbe93400640 1 -- 172.21.15.123:0/2887094307 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88018960 con 0x7fbe9c1049a0 2024-09-19T04:51:42.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.380+0000 7fbea1a00640 1 -- 172.21.15.123:0/2887094307 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9c19f910 con 0x7fbe9c1049a0 2024-09-19T04:51:42.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.381+0000 7fbea1a00640 1 -- 172.21.15.123:0/2887094307 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe9c1a71f0 con 0x7fbe9c1049a0 2024-09-19T04:51:42.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.385+0000 7fbea4436740 1 Processor -- start 2024-09-19T04:51:42.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.385+0000 7fbea4436740 1 -- start start 2024-09-19T04:51:42.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.385+0000 7fbe93400640 1 -- 172.21.15.123:0/2887094307 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fbe88010ac0 con 0x7fbe9c1049a0 2024-09-19T04:51:42.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.385+0000 7fbea4436740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdddac0 0x55dd3fdddea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.385+0000 7fbea4436740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fdde3e0 0x55dd3fde8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.385+0000 7fbea4436740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fde8ca0 0x55dd3fdeb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.385+0000 7fbea4436740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dd3fd44290 con 0x55dd3fdde3e0 2024-09-19T04:51:42.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.385+0000 7fbea4436740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dd3fcebb40 con 0x55dd3fdddac0 2024-09-19T04:51:42.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.385+0000 7fbea4436740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dd3fd40250 con 0x55dd3fde8ca0 2024-09-19T04:51:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.386+0000 7fbe9a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fde8ca0 0x55dd3fdeb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.386+0000 7fbe9b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdddac0 0x55dd3fdddea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.386+0000 7fbe9aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fdde3e0 0x55dd3fde8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.386+0000 7fbe9a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fde8ca0 0x55dd3fdeb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53140/0 (socket says 172.21.15.123:53140) 2024-09-19T04:51:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.386+0000 7fbe9a000640 1 -- 172.21.15.123:0/2685097184 learned_addr learned my addr 172.21.15.123:0/2685097184 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.386+0000 7fbe9a000640 1 -- 172.21.15.123:0/2685097184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdddac0 msgr2=0x55dd3fdddea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.386+0000 7fbe9a000640 1 --2- 172.21.15.123:0/2685097184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdddac0 0x55dd3fdddea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.386+0000 7fbe9a000640 1 -- 172.21.15.123:0/2685097184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fdde3e0 msgr2=0x55dd3fde8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.386+0000 7fbe9a000640 1 --2- 172.21.15.123:0/2685097184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fdde3e0 0x55dd3fde8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.386+0000 7fbe9a000640 1 -- 172.21.15.123:0/2685097184 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd3fab56a0 con 0x55dd3fde8ca0 2024-09-19T04:51:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.386+0000 7fbe9b400640 1 --2- 172.21.15.123:0/2685097184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdddac0 0x55dd3fdddea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.386+0000 7fbe9aa00640 1 --2- 172.21.15.123:0/2685097184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fdde3e0 0x55dd3fde8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.387+0000 7fbe9a000640 1 --2- 172.21.15.123:0/2685097184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fde8ca0 0x55dd3fdeb090 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbe880cd780 tx=0x7fbe880d8780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c02a765f46dc2a10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:42.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.387+0000 7fbe87e00640 1 -- 172.21.15.123:0/2685097184 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe880e3070 con 0x55dd3fde8ca0 2024-09-19T04:51:42.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.387+0000 7fbe87e00640 1 -- 172.21.15.123:0/2685097184 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe880d09f0 con 0x55dd3fde8ca0 2024-09-19T04:51:42.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.387+0000 7fbe93400640 1 --2- 172.21.15.123:0/2887094307 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbe800a3430 0x7fbe800a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.387+0000 7fbe87e00640 1 -- 172.21.15.123:0/2685097184 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe880deb50 con 0x55dd3fde8ca0 2024-09-19T04:51:42.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.387+0000 7fbe93400640 1 -- 172.21.15.123:0/2887094307 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fbe880d0090 con 0x7fbe9c1049a0 2024-09-19T04:51:42.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.388+0000 7fbe9b400640 1 --2- 172.21.15.123:0/2887094307 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbe800a3430 0x7fbe800a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.388+0000 7fbea4436740 1 -- 172.21.15.123:0/2685097184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fde8ca0 msgr2=0x55dd3fdeb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:42.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.388+0000 7fbea4436740 1 --2- 172.21.15.123:0/2685097184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fde8ca0 0x55dd3fdeb090 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbe880cd780 tx=0x7fbe880d8780 comp rx=0 tx=0).stop 2024-09-19T04:51:42.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.388+0000 7fbea4436740 1 -- 172.21.15.123:0/2685097184 shutdown_connections 2024-09-19T04:51:42.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.388+0000 7fbea4436740 1 --2- 172.21.15.123:0/2685097184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fde8ca0 0x55dd3fdeb090 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.388+0000 7fbea4436740 1 --2- 172.21.15.123:0/2685097184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fdde3e0 0x55dd3fde8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.388+0000 7fbea4436740 1 --2- 172.21.15.123:0/2685097184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdddac0 0x55dd3fdddea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.388+0000 7fbea4436740 1 -- 172.21.15.123:0/2685097184 >> 172.21.15.123:0/2685097184 conn(0x55dd3fdda050 msgr2=0x55dd3fdda430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:42.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.388+0000 7fbe9b400640 1 --2- 172.21.15.123:0/2887094307 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbe800a3430 0x7fbe800a58f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbe940049d0 tx=0x7fbe94009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:42.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.389+0000 7fbea4436740 1 -- 172.21.15.123:0/2685097184 shutdown_connections 2024-09-19T04:51:42.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.389+0000 7fbea4436740 1 -- 172.21.15.123:0/2685097184 wait complete. 2024-09-19T04:51:42.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.389+0000 7fbea4436740 1 -- start start 2024-09-19T04:51:42.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.389+0000 7fbea4436740 1 Processor -- start 2024-09-19T04:51:42.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.390+0000 7fbea4436740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fdddac0 0x55dd3fde8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.390+0000 7fbea4436740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdde3e0 0x55dd3fde58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.390+0000 7fbea4436740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fde8ca0 0x55dd3fde5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.390+0000 7fbea4436740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dd3fdeb910 con 0x55dd3fde8ca0 2024-09-19T04:51:42.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.390+0000 7fbea4436740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dd3fcfefa0 con 0x55dd3fdde3e0 2024-09-19T04:51:42.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.390+0000 7fbea4436740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dd3fdeba90 con 0x55dd3fdddac0 2024-09-19T04:51:42.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.390+0000 7fbea4436740 10 client.0 _ll_register_callbacks cb 0x55dd3fa52ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:51:42.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.390+0000 7fbe9a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdde3e0 0x55dd3fde58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.390+0000 7fbe9a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdde3e0 0x55dd3fde58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:41764/0 (socket says 172.21.15.123:41764) 2024-09-19T04:51:42.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.390+0000 7fbe9a000640 1 -- 172.21.15.123:0/660704461 learned_addr learned my addr 172.21.15.123:0/660704461 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:42.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.390+0000 7fbe9aa00640 1 --2- 172.21.15.123:0/660704461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fdddac0 0x55dd3fde8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.390+0000 7fbe9b400640 1 --2- 172.21.15.123:0/660704461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fde8ca0 0x55dd3fde5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.391+0000 7fbe9a000640 1 -- 172.21.15.123:0/660704461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fdddac0 msgr2=0x55dd3fde8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:42.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.391+0000 7fbe9a000640 1 --2- 172.21.15.123:0/660704461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fdddac0 0x55dd3fde8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.391+0000 7fbe9a000640 1 -- 172.21.15.123:0/660704461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fde8ca0 msgr2=0x55dd3fde5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:42.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.391+0000 7fbe9a000640 1 --2- 172.21.15.123:0/660704461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fde8ca0 0x55dd3fde5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.391+0000 7fbe9a000640 1 -- 172.21.15.123:0/660704461 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd3fde6680 con 0x55dd3fdde3e0 2024-09-19T04:51:42.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.391+0000 7fbe9aa00640 1 --2- 172.21.15.123:0/660704461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fdddac0 0x55dd3fde8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:42.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.391+0000 7fbe9a000640 1 --2- 172.21.15.123:0/660704461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdde3e0 0x55dd3fde58d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbe88098d70 tx=0x7fbe8800cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a693a751b567a37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:42.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.391+0000 7fbe84c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:51:42.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.391+0000 7fbe84c00640 1 -- 172.21.15.123:0/660704461 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88098640 con 0x55dd3fdde3e0 2024-09-19T04:51:42.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.391+0000 7fbe84c00640 1 -- 172.21.15.123:0/660704461 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fbe880cda90 con 0x55dd3fdde3e0 2024-09-19T04:51:42.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.391+0000 7fbea4436740 1 -- 172.21.15.123:0/660704461 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe880d09f0 con 0x55dd3fdde3e0 2024-09-19T04:51:42.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.391+0000 7fbe84c00640 1 -- 172.21.15.123:0/660704461 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbe8809bd00 con 0x55dd3fdde3e0 2024-09-19T04:51:42.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.391+0000 7fbea4436740 10 client.26261 fetch_fsmap learned FSMap version 36 2024-09-19T04:51:42.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.391+0000 7fbea4436740 1 -- 172.21.15.123:0/660704461 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55dd3fdeb910 con 0x55dd3fdde3e0 2024-09-19T04:51:42.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.392+0000 7fbe84c00640 1 -- 172.21.15.123:0/660704461 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe880d09f0 con 0x55dd3fdde3e0 2024-09-19T04:51:42.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.392+0000 7fbe84c00640 1 -- 172.21.15.123:0/660704461 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fbe8809c600 con 0x55dd3fdde3e0 2024-09-19T04:51:42.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.392+0000 7fbea4436740 10 client.26261 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:51:42.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.392+0000 7fbea4436740 10 client.26261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:51:42.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.392+0000 7fbea4436740 20 client.26261 populate_metadata read hostname 'smithi123' 2024-09-19T04:51:42.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.392+0000 7fbea4436740 1 --2- 172.21.15.123:0/660704461 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55dd3fdf8a10 0x55dd3fdfae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.392+0000 7fbea4436740 4 client.26261 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:51:42.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.392+0000 7fbea4436740 1 -- 172.21.15.123:0/660704461 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd3fd51800 con 0x55dd3fdf8a10 2024-09-19T04:51:42.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.393+0000 7fbe9aa00640 1 --2- 172.21.15.123:0/660704461 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55dd3fdf8a10 0x55dd3fdfae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.393+0000 7fbe9aa00640 1 --2- 172.21.15.123:0/660704461 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55dd3fdf8a10 0x55dd3fdfae00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=489969ef7d8f7ecc server_cookie=47109f320c47f73 in_seq=0 out_seq=0 2024-09-19T04:51:42.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.393+0000 7fbe84c00640 10 client.26261 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:51:42.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.394+0000 7fbe84c00640 1 -- 172.21.15.123:0/660704461 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dd3fd51800 con 0x55dd3fdf8a10 2024-09-19T04:51:42.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.394+0000 7fbe84c00640 10 client.26261 handle_command_reply: tid=0 2024-09-19T04:51:42.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.394+0000 7fbea4436740 1 client.26261 shutdown 2024-09-19T04:51:42.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.394+0000 7fbea4436740 1 -- 172.21.15.123:0/660704461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdde3e0 msgr2=0x55dd3fde58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:42.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.394+0000 7fbea4436740 1 --2- 172.21.15.123:0/660704461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdde3e0 0x55dd3fde58d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbe88098d70 tx=0x7fbe8800cf60 comp rx=0 tx=0).stop 2024-09-19T04:51:42.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.394+0000 7fbea4436740 1 -- 172.21.15.123:0/660704461 shutdown_connections 2024-09-19T04:51:42.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.394+0000 7fbea4436740 1 --2- 172.21.15.123:0/660704461 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55dd3fdf8a10 0x55dd3fdfae00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.394+0000 7fbea4436740 1 --2- 172.21.15.123:0/660704461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fde8ca0 0x55dd3fde5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.394+0000 7fbea4436740 1 --2- 172.21.15.123:0/660704461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdde3e0 0x55dd3fde58d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.394+0000 7fbea4436740 1 --2- 172.21.15.123:0/660704461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fdddac0 0x55dd3fde8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.394+0000 7fbea4436740 1 -- 172.21.15.123:0/660704461 >> 172.21.15.123:0/660704461 conn(0x55dd3fdda050 msgr2=0x55dd3fddfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:42.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.394+0000 7fbe84c00640 0 client.26261 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:51:42.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.394+0000 7fbea4436740 1 -- 172.21.15.123:0/660704461 shutdown_connections 2024-09-19T04:51:42.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.394+0000 7fbea4436740 1 -- 172.21.15.123:0/660704461 wait complete. 2024-09-19T04:51:42.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.394+0000 7fbea4436740 20 client.26261 trim_cache size 0 max 16384 2024-09-19T04:51:42.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.413+0000 7fbea4436740 1 Processor -- start 2024-09-19T04:51:42.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.413+0000 7fbea4436740 1 -- start start 2024-09-19T04:51:42.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.413+0000 7fbea4436740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fdddac0 0x55dd3fe00490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.413+0000 7fbea4436740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdde3e0 0x55dd3fe009d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.413+0000 7fbea4436740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fde8ca0 0x55dd3fddf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.413+0000 7fbea4436740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1afa60 con 0x55dd3fde8ca0 2024-09-19T04:51:42.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.413+0000 7fbea4436740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1b15a0 con 0x55dd3fdde3e0 2024-09-19T04:51:42.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.413+0000 7fbea4436740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1b0130 con 0x55dd3fdddac0 2024-09-19T04:51:42.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.414+0000 7fbe9b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fde8ca0 0x55dd3fddf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.414+0000 7fbe9a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdde3e0 0x55dd3fe009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.414+0000 7fbe9b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fde8ca0 0x55dd3fddf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51614/0 (socket says 172.21.15.123:51614) 2024-09-19T04:51:42.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.414+0000 7fbe9b400640 1 -- 172.21.15.123:0/2806184698 learned_addr learned my addr 172.21.15.123:0/2806184698 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:42.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.414+0000 7fbe9aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fdddac0 0x55dd3fe00490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.414+0000 7fbe9aa00640 1 -- 172.21.15.123:0/2806184698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdde3e0 msgr2=0x55dd3fe009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:42.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.414+0000 7fbe9aa00640 1 --2- 172.21.15.123:0/2806184698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdde3e0 0x55dd3fe009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.414+0000 7fbe9aa00640 1 -- 172.21.15.123:0/2806184698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fde8ca0 msgr2=0x55dd3fddf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:42.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.414+0000 7fbe9aa00640 1 --2- 172.21.15.123:0/2806184698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fde8ca0 0x55dd3fddf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.414+0000 7fbe9aa00640 1 -- 172.21.15.123:0/2806184698 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd3fde61a0 con 0x55dd3fdddac0 2024-09-19T04:51:42.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.414+0000 7fbe9a000640 1 --2- 172.21.15.123:0/2806184698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdde3e0 0x55dd3fe009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:42.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.414+0000 7fbe9aa00640 1 --2- 172.21.15.123:0/2806184698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fdddac0 0x55dd3fe00490 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c005940 tx=0x7fbe8c031a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91a0de1bea2ae904 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:42.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.415+0000 7fbe87400640 1 -- 172.21.15.123:0/2806184698 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c035040 con 0x55dd3fdddac0 2024-09-19T04:51:42.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.415+0000 7fbe87400640 1 -- 172.21.15.123:0/2806184698 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe8c005500 con 0x55dd3fdddac0 2024-09-19T04:51:42.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.415+0000 7fbe87400640 1 -- 172.21.15.123:0/2806184698 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c004ca0 con 0x55dd3fdddac0 2024-09-19T04:51:42.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.415+0000 7fbea4436740 1 -- 172.21.15.123:0/2806184698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fdddac0 msgr2=0x55dd3fe00490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:42.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.415+0000 7fbea4436740 1 --2- 172.21.15.123:0/2806184698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fdddac0 0x55dd3fe00490 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c005940 tx=0x7fbe8c031a10 comp rx=0 tx=0).stop 2024-09-19T04:51:42.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.415+0000 7fbea4436740 1 -- 172.21.15.123:0/2806184698 shutdown_connections 2024-09-19T04:51:42.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.415+0000 7fbea4436740 1 --2- 172.21.15.123:0/2806184698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fde8ca0 0x55dd3fddf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.415+0000 7fbea4436740 1 --2- 172.21.15.123:0/2806184698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdde3e0 0x55dd3fe009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.415+0000 7fbea4436740 1 --2- 172.21.15.123:0/2806184698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fdddac0 0x55dd3fe00490 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.415+0000 7fbea4436740 1 -- 172.21.15.123:0/2806184698 >> 172.21.15.123:0/2806184698 conn(0x55dd3fdda050 msgr2=0x55dd3fde7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:42.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.416+0000 7fbea4436740 1 -- 172.21.15.123:0/2806184698 shutdown_connections 2024-09-19T04:51:42.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.416+0000 7fbea4436740 1 -- 172.21.15.123:0/2806184698 wait complete. 2024-09-19T04:51:42.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.416+0000 7fbea4436740 1 -- start start 2024-09-19T04:51:42.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.416+0000 7fbea4436740 1 Processor -- start 2024-09-19T04:51:42.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.417+0000 7fbea4436740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdddac0 0x55dd3fdf1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.417+0000 7fbea4436740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fdde3e0 0x55dd3fdec590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.417+0000 7fbea4436740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fde8ca0 0x55dd3fdecad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.417+0000 7fbea4436740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1b0ed0 con 0x55dd3fdde3e0 2024-09-19T04:51:42.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.417+0000 7fbea4436740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1b0800 con 0x55dd3fdddac0 2024-09-19T04:51:42.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.417+0000 7fbea4436740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1ada40 con 0x55dd3fde8ca0 2024-09-19T04:51:42.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.417+0000 7fbea4436740 10 client.0 _ll_register_callbacks cb 0x7fbe880e1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:51:42.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.417+0000 7fbe9aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdddac0 0x55dd3fdf1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.417+0000 7fbe9aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdddac0 0x55dd3fdf1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:41776/0 (socket says 172.21.15.123:41776) 2024-09-19T04:51:42.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.417+0000 7fbe9aa00640 1 -- 172.21.15.123:0/2872109243 learned_addr learned my addr 172.21.15.123:0/2872109243 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:42.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.417+0000 7fbe9b400640 1 --2- 172.21.15.123:0/2872109243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fde8ca0 0x55dd3fdecad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.417+0000 7fbe9a000640 1 --2- 172.21.15.123:0/2872109243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fdde3e0 0x55dd3fdec590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.417+0000 7fbe9aa00640 1 -- 172.21.15.123:0/2872109243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fde8ca0 msgr2=0x55dd3fdecad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:42.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.417+0000 7fbe9aa00640 1 --2- 172.21.15.123:0/2872109243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fde8ca0 0x55dd3fdecad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.417+0000 7fbe9aa00640 1 -- 172.21.15.123:0/2872109243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fdde3e0 msgr2=0x55dd3fdec590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:42.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.417+0000 7fbe9aa00640 1 --2- 172.21.15.123:0/2872109243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fdde3e0 0x55dd3fdec590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.417+0000 7fbe9aa00640 1 -- 172.21.15.123:0/2872109243 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd3fded0c0 con 0x55dd3fdddac0 2024-09-19T04:51:42.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.418+0000 7fbe9a000640 1 --2- 172.21.15.123:0/2872109243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fdde3e0 0x55dd3fdec590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:51:42.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.418+0000 7fbe9aa00640 1 --2- 172.21.15.123:0/2872109243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdddac0 0x55dd3fdf1260 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c031450 tx=0x7fbe8c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=897458a2c4ff1f02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:42.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.418+0000 7fbe84c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:51:42.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.418+0000 7fbe84c00640 1 -- 172.21.15.123:0/2872109243 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c035070 con 0x55dd3fdddac0 2024-09-19T04:51:42.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.418+0000 7fbe84c00640 1 -- 172.21.15.123:0/2872109243 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fbe8c004a20 con 0x55dd3fdddac0 2024-09-19T04:51:42.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.418+0000 7fbea4436740 1 -- 172.21.15.123:0/2872109243 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe8c005500 con 0x55dd3fdddac0 2024-09-19T04:51:42.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.419+0000 7fbe84c00640 1 -- 172.21.15.123:0/2872109243 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbe8c004c60 con 0x55dd3fdddac0 2024-09-19T04:51:42.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.419+0000 7fbea4436740 10 client.26267 fetch_fsmap learned FSMap version 36 2024-09-19T04:51:42.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.419+0000 7fbea4436740 1 -- 172.21.15.123:0/2872109243 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55dd3fe632a0 con 0x55dd3fdddac0 2024-09-19T04:51:42.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.419+0000 7fbe84c00640 1 -- 172.21.15.123:0/2872109243 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe8c005500 con 0x55dd3fdddac0 2024-09-19T04:51:42.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.419+0000 7fbe84c00640 1 -- 172.21.15.123:0/2872109243 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fbe8c05b470 con 0x55dd3fdddac0 2024-09-19T04:51:42.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.419+0000 7fbea4436740 10 client.26267 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:51:42.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.419+0000 7fbea4436740 10 client.26267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:51:42.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.419+0000 7fbea4436740 20 client.26267 populate_metadata read hostname 'smithi123' 2024-09-19T04:51:42.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.419+0000 7fbea4436740 1 --2- 172.21.15.123:0/2872109243 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55dd3fe638c0 0x55dd3fe65ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:42.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.419+0000 7fbea4436740 4 client.26267 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:51:42.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.419+0000 7fbea4436740 1 -- 172.21.15.123:0/2872109243 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dd3fd4bd10 con 0x55dd3fe638c0 2024-09-19T04:51:42.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.419+0000 7fbe9a000640 1 --2- 172.21.15.123:0/2872109243 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55dd3fe638c0 0x55dd3fe65ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:42.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.420+0000 7fbe9a000640 1 --2- 172.21.15.123:0/2872109243 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55dd3fe638c0 0x55dd3fe65ce0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c61927f022b8b710 server_cookie=66ecb076deb95792 in_seq=0 out_seq=0 2024-09-19T04:51:42.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.420+0000 7fbe84c00640 10 client.26267 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:51:42.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.421+0000 7fbe84c00640 1 -- 172.21.15.123:0/2872109243 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dd3fd4bd10 con 0x55dd3fe638c0 2024-09-19T04:51:42.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.421+0000 7fbe84c00640 10 client.26267 handle_command_reply: tid=0 2024-09-19T04:51:42.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.421+0000 7fbea4436740 1 client.26267 shutdown 2024-09-19T04:51:42.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.421+0000 7fbea4436740 1 -- 172.21.15.123:0/2872109243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdddac0 msgr2=0x55dd3fdf1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:42.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.421+0000 7fbea4436740 1 --2- 172.21.15.123:0/2872109243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdddac0 0x55dd3fdf1260 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c031450 tx=0x7fbe8c000bc0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.421+0000 7fbea4436740 1 -- 172.21.15.123:0/2872109243 shutdown_connections 2024-09-19T04:51:42.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.421+0000 7fbea4436740 1 --2- 172.21.15.123:0/2872109243 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55dd3fe638c0 0x55dd3fe65ce0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.421+0000 7fbea4436740 1 --2- 172.21.15.123:0/2872109243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3fde8ca0 0x55dd3fdecad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.421+0000 7fbea4436740 1 --2- 172.21.15.123:0/2872109243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd3fdde3e0 0x55dd3fdec590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.421+0000 7fbea4436740 1 --2- 172.21.15.123:0/2872109243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3fdddac0 0x55dd3fdf1260 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.421+0000 7fbea4436740 1 -- 172.21.15.123:0/2872109243 >> 172.21.15.123:0/2872109243 conn(0x55dd3fdda050 msgr2=0x55dd3fde5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:42.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.421+0000 7fbe84c00640 0 client.26267 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:51:42.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.422+0000 7fbea4436740 1 -- 172.21.15.123:0/2872109243 shutdown_connections 2024-09-19T04:51:42.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.422+0000 7fbea4436740 1 -- 172.21.15.123:0/2872109243 wait complete. 2024-09-19T04:51:42.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.422+0000 7fbea4436740 20 client.26267 trim_cache size 0 max 16384 2024-09-19T04:51:42.428 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:51:42.428 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-09-19T04:51:42.428 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "d1ca1c94-4939-43d7-8a4d-6df4ba1e8d65", 2024-09-19T04:51:42.428 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-09-19T04:51:42.429 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:51:42.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.423+0000 7fbea1a00640 1 -- 172.21.15.123:0/2887094307 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbe800a3430 msgr2=0x7fbe800a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:42.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.423+0000 7fbea1a00640 1 --2- 172.21.15.123:0/2887094307 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbe800a3430 0x7fbe800a58f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbe940049d0 tx=0x7fbe94009500 comp rx=0 tx=0).stop 2024-09-19T04:51:42.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.423+0000 7fbea1a00640 1 -- 172.21.15.123:0/2887094307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c1049a0 msgr2=0x7fbe9c19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:42.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.423+0000 7fbea1a00640 1 --2- 172.21.15.123:0/2887094307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c1049a0 0x7fbe9c19abb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800e9f0 tx=0x7fbe8800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.423+0000 7fbea1a00640 1 -- 172.21.15.123:0/2887094307 shutdown_connections 2024-09-19T04:51:42.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.423+0000 7fbea1a00640 1 --2- 172.21.15.123:0/2887094307 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbe800a3430 0x7fbe800a58f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.423+0000 7fbea1a00640 1 --2- 172.21.15.123:0/2887094307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe9c109120 0x7fbe9c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.423+0000 7fbea1a00640 1 --2- 172.21.15.123:0/2887094307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c1049a0 0x7fbe9c19abb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.423+0000 7fbea1a00640 1 --2- 172.21.15.123:0/2887094307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe9c103ff0 0x7fbe9c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:42.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.423+0000 7fbea1a00640 1 -- 172.21.15.123:0/2887094307 >> 172.21.15.123:0/2887094307 conn(0x7fbe9c0ffa90 msgr2=0x7fbe9c100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:42.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.424+0000 7fbea1a00640 1 -- 172.21.15.123:0/2887094307 shutdown_connections 2024-09-19T04:51:42.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:42.424+0000 7fbea1a00640 1 -- 172.21.15.123:0/2887094307 wait complete. 2024-09-19T04:51:43.242 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:51:43.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:43 smithi123 ceph-mon[30511]: pgmap v153: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.5 MiB/s wr, 537 op/s 2024-09-19T04:51:43.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:43 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:51:43.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:43 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:51:43.428 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:43 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:43.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:43 smithi193 ceph-mon[35592]: pgmap v153: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.5 MiB/s wr, 537 op/s 2024-09-19T04:51:43.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:43 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:51:43.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:43 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:51:43.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:43 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:43.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:43 smithi183 ceph-mon[35333]: pgmap v153: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.5 MiB/s wr, 537 op/s 2024-09-19T04:51:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:43 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:51:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:43 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:51:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:43 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:43.724 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:51:44.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.284+0000 7fbaf3c00640 1 -- 172.21.15.123:0/3065592207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaec103fc0 msgr2=0x7fbaec1043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:44.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.284+0000 7fbaf3c00640 1 --2- 172.21.15.123:0/3065592207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaec103fc0 0x7fbaec1043a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009ca0 tx=0x7fbae402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.284+0000 7fbaf3c00640 1 -- 172.21.15.123:0/3065592207 shutdown_connections 2024-09-19T04:51:44.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.284+0000 7fbaf3c00640 1 --2- 172.21.15.123:0/3065592207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaec1090f0 0x7fbaec110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.284+0000 7fbaf3c00640 1 --2- 172.21.15.123:0/3065592207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaec104970 0x7fbaec1089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.284+0000 7fbaf3c00640 1 --2- 172.21.15.123:0/3065592207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaec103fc0 0x7fbaec1043a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.284+0000 7fbaf3c00640 1 -- 172.21.15.123:0/3065592207 >> 172.21.15.123:0/3065592207 conn(0x7fbaec0ffa40 msgr2=0x7fbaec101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:44.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.284+0000 7fbaf3c00640 1 -- 172.21.15.123:0/3065592207 shutdown_connections 2024-09-19T04:51:44.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.284+0000 7fbaf3c00640 1 -- 172.21.15.123:0/3065592207 wait complete. 2024-09-19T04:51:44.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.285+0000 7fbaf3c00640 1 Processor -- start 2024-09-19T04:51:44.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.285+0000 7fbaf3c00640 1 -- start start 2024-09-19T04:51:44.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.286+0000 7fbaf3c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaec103fc0 0x7fbaec19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.286+0000 7fbaf3c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaec104970 0x7fbaec19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.286+0000 7fbaf3c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaec1090f0 0x7fbaec19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.286+0000 7fbaf3c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec114140 con 0x7fbaec103fc0 2024-09-19T04:51:44.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.286+0000 7fbaf3c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec113fc0 con 0x7fbaec104970 2024-09-19T04:51:44.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.286+0000 7fbaf3c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1142c0 con 0x7fbaec1090f0 2024-09-19T04:51:44.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.286+0000 7fbaf1200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaec103fc0 0x7fbaec19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.286+0000 7fbaf1200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaec103fc0 0x7fbaec19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51656/0 (socket says 172.21.15.123:51656) 2024-09-19T04:51:44.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.286+0000 7fbaf1200640 1 -- 172.21.15.123:0/2200302869 learned_addr learned my addr 172.21.15.123:0/2200302869 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:44.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.286+0000 7fbaebe00640 1 --2- 172.21.15.123:0/2200302869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaec104970 0x7fbaec19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.286+0000 7fbaf1c00640 1 --2- 172.21.15.123:0/2200302869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaec1090f0 0x7fbaec19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.287+0000 7fbaebe00640 1 -- 172.21.15.123:0/2200302869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaec1090f0 msgr2=0x7fbaec19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:44.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.287+0000 7fbaebe00640 1 --2- 172.21.15.123:0/2200302869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaec1090f0 0x7fbaec19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.287+0000 7fbaebe00640 1 -- 172.21.15.123:0/2200302869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaec103fc0 msgr2=0x7fbaec19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:44.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.287+0000 7fbaebe00640 1 --2- 172.21.15.123:0/2200302869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaec103fc0 0x7fbaec19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.287+0000 7fbaebe00640 1 -- 172.21.15.123:0/2200302869 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec19f5b0 con 0x7fbaec104970 2024-09-19T04:51:44.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.287+0000 7fbaebe00640 1 --2- 172.21.15.123:0/2200302869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaec104970 0x7fbaec19ab40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbad800e9f0 tx=0x7fbad800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:44.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.287+0000 7fbae9600640 1 -- 172.21.15.123:0/2200302869 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad800ccc0 con 0x7fbaec104970 2024-09-19T04:51:44.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.287+0000 7fbae9600640 1 -- 172.21.15.123:0/2200302869 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbad8010910 con 0x7fbaec104970 2024-09-19T04:51:44.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.287+0000 7fbaf3c00640 1 -- 172.21.15.123:0/2200302869 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaec19f8a0 con 0x7fbaec104970 2024-09-19T04:51:44.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.287+0000 7fbae9600640 1 -- 172.21.15.123:0/2200302869 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8018800 con 0x7fbaec104970 2024-09-19T04:51:44.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.287+0000 7fbaf3c00640 1 -- 172.21.15.123:0/2200302869 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaec1a7180 con 0x7fbaec104970 2024-09-19T04:51:44.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.292+0000 7fbaf665f740 1 Processor -- start 2024-09-19T04:51:44.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.292+0000 7fbae9600640 1 -- 172.21.15.123:0/2200302869 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fbad8010ac0 con 0x7fbaec104970 2024-09-19T04:51:44.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.292+0000 7fbaf665f740 1 -- start start 2024-09-19T04:51:44.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.292+0000 7fbaf665f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b14b90 0x55f595b14f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.292+0000 7fbaf665f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b154b0 0x55f595b1f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.292+0000 7fbaf665f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b1fd70 0x55f595b22160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.292+0000 7fbaf665f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f595a7b3e0 con 0x55f595b14b90 2024-09-19T04:51:44.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.292+0000 7fbaf665f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f595a22d80 con 0x55f595b154b0 2024-09-19T04:51:44.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.292+0000 7fbaf665f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f595a773a0 con 0x55f595b1fd70 2024-09-19T04:51:44.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.293+0000 7fbaf1c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b14b90 0x55f595b14f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.293+0000 7fbaf1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b154b0 0x55f595b1f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.293+0000 7fbaf1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b154b0 0x55f595b1f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:41820/0 (socket says 172.21.15.123:41820) 2024-09-19T04:51:44.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.293+0000 7fbaf1200640 1 -- 172.21.15.123:0/1257516820 learned_addr learned my addr 172.21.15.123:0/1257516820 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:44.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.293+0000 7fbaebe00640 1 --2- 172.21.15.123:0/1257516820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b1fd70 0x55f595b22160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.293+0000 7fbaf1200640 1 -- 172.21.15.123:0/1257516820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b1fd70 msgr2=0x55f595b22160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:44.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.293+0000 7fbaf1200640 1 --2- 172.21.15.123:0/1257516820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b1fd70 0x55f595b22160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.293+0000 7fbaf1200640 1 -- 172.21.15.123:0/1257516820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b14b90 msgr2=0x55f595b14f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:44.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.293+0000 7fbaf1200640 1 --2- 172.21.15.123:0/1257516820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b14b90 0x55f595b14f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.293+0000 7fbaf1200640 1 -- 172.21.15.123:0/1257516820 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f595a41ae0 con 0x55f595b154b0 2024-09-19T04:51:44.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.293+0000 7fbaf1200640 1 --2- 172.21.15.123:0/1257516820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b154b0 0x55f595b1f830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbae40097b0 tx=0x7fbae402f310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9382ffbbacbd418d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:44.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.294+0000 7fbae9600640 1 --2- 172.21.15.123:0/2200302869 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbad00a33b0 0x7fbad00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.294+0000 7fbae2000640 1 -- 172.21.15.123:0/1257516820 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4031db0 con 0x55f595b154b0 2024-09-19T04:51:44.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.294+0000 7fbae2000640 1 -- 172.21.15.123:0/1257516820 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae4036030 con 0x55f595b154b0 2024-09-19T04:51:44.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.294+0000 7fbae2000640 1 -- 172.21.15.123:0/1257516820 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae40043c0 con 0x55f595b154b0 2024-09-19T04:51:44.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.294+0000 7fbae9600640 1 -- 172.21.15.123:0/2200302869 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fbad80cfff0 con 0x7fbaec104970 2024-09-19T04:51:44.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.294+0000 7fbaf1c00640 1 --2- 172.21.15.123:0/2200302869 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbad00a33b0 0x7fbad00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.295+0000 7fbaf665f740 1 -- 172.21.15.123:0/1257516820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b154b0 msgr2=0x55f595b1f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:44.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.295+0000 7fbaf665f740 1 --2- 172.21.15.123:0/1257516820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b154b0 0x55f595b1f830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbae40097b0 tx=0x7fbae402f310 comp rx=0 tx=0).stop 2024-09-19T04:51:44.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.295+0000 7fbaf665f740 1 -- 172.21.15.123:0/1257516820 shutdown_connections 2024-09-19T04:51:44.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.295+0000 7fbaf665f740 1 --2- 172.21.15.123:0/1257516820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b1fd70 0x55f595b22160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.295+0000 7fbaf665f740 1 --2- 172.21.15.123:0/1257516820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b154b0 0x55f595b1f830 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.295+0000 7fbaf665f740 1 --2- 172.21.15.123:0/1257516820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b14b90 0x55f595b14f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.295+0000 7fbaf665f740 1 -- 172.21.15.123:0/1257516820 >> 172.21.15.123:0/1257516820 conn(0x55f595b11120 msgr2=0x55f595b11500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:44.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.295+0000 7fbaf1c00640 1 --2- 172.21.15.123:0/2200302869 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbad00a33b0 0x7fbad00a5870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009d30 tx=0x7fbadc005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:44.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.295+0000 7fbaf665f740 1 -- 172.21.15.123:0/1257516820 shutdown_connections 2024-09-19T04:51:44.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.295+0000 7fbaf665f740 1 -- 172.21.15.123:0/1257516820 wait complete. 2024-09-19T04:51:44.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.295+0000 7fbaf665f740 1 -- start start 2024-09-19T04:51:44.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.296+0000 7fbaf665f740 1 Processor -- start 2024-09-19T04:51:44.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.296+0000 7fbaf665f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b14b90 0x55f595b1cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.296+0000 7fbaf665f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b154b0 0x55f595b1d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.296+0000 7fbaf665f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b1fd70 0x55f595b1d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.296+0000 7fbaf665f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f595b22860 con 0x55f595b14b90 2024-09-19T04:51:44.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.296+0000 7fbaf665f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f595a360f0 con 0x55f595b1fd70 2024-09-19T04:51:44.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.296+0000 7fbaf665f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f595b229e0 con 0x55f595b154b0 2024-09-19T04:51:44.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.296+0000 7fbaf665f740 10 client.0 _ll_register_callbacks cb 0x55f595a279e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:51:44.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.296+0000 7fbaf1c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b1fd70 0x55f595b1d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.296+0000 7fbaebe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b154b0 0x55f595b1d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.296+0000 7fbaf1c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b1fd70 0x55f595b1d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:41826/0 (socket says 172.21.15.123:41826) 2024-09-19T04:51:44.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.296+0000 7fbaf1c00640 1 -- 172.21.15.123:0/2499127646 learned_addr learned my addr 172.21.15.123:0/2499127646 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:44.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.297+0000 7fbaf1200640 1 --2- 172.21.15.123:0/2499127646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b14b90 0x55f595b1cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.297+0000 7fbaf1c00640 1 -- 172.21.15.123:0/2499127646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b154b0 msgr2=0x55f595b1d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:44.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.297+0000 7fbaf1c00640 1 --2- 172.21.15.123:0/2499127646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b154b0 0x55f595b1d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.297+0000 7fbaf1c00640 1 -- 172.21.15.123:0/2499127646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b14b90 msgr2=0x55f595b1cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:44.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.297+0000 7fbaf1c00640 1 --2- 172.21.15.123:0/2499127646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b14b90 0x55f595b1cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.297+0000 7fbaf1c00640 1 -- 172.21.15.123:0/2499127646 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5957ec3c0 con 0x55f595b1fd70 2024-09-19T04:51:44.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.297+0000 7fbaf1200640 1 --2- 172.21.15.123:0/2499127646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b14b90 0x55f595b1cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:51:44.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.297+0000 7fbaf1c00640 1 --2- 172.21.15.123:0/2499127646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b1fd70 0x55f595b1d6c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0093a0 tx=0x7fbadc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cfbcb0e39b283ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:44.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.297+0000 7fbad6a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:51:44.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.297+0000 7fbad6a00640 1 -- 172.21.15.123:0/2499127646 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc02e060 con 0x55f595b1fd70 2024-09-19T04:51:44.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.297+0000 7fbad6a00640 1 -- 172.21.15.123:0/2499127646 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fbadc018030 con 0x55f595b1fd70 2024-09-19T04:51:44.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.297+0000 7fbaf665f740 1 -- 172.21.15.123:0/2499127646 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbae4036030 con 0x55f595b1fd70 2024-09-19T04:51:44.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.298+0000 7fbad6a00640 1 -- 172.21.15.123:0/2499127646 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbadc009e20 con 0x55f595b1fd70 2024-09-19T04:51:44.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.298+0000 7fbaf665f740 10 client.26279 fetch_fsmap learned FSMap version 36 2024-09-19T04:51:44.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.298+0000 7fbaf665f740 1 -- 172.21.15.123:0/2499127646 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f595b2e550 con 0x55f595b1fd70 2024-09-19T04:51:44.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.298+0000 7fbad6a00640 1 -- 172.21.15.123:0/2499127646 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae4036030 con 0x55f595b1fd70 2024-09-19T04:51:44.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.298+0000 7fbad6a00640 1 -- 172.21.15.123:0/2499127646 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fbadc00f580 con 0x55f595b1fd70 2024-09-19T04:51:44.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.298+0000 7fbaf665f740 10 client.26279 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:51:44.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.298+0000 7fbaf665f740 10 client.26279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:51:44.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.298+0000 7fbaf665f740 20 client.26279 populate_metadata read hostname 'smithi123' 2024-09-19T04:51:44.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.298+0000 7fbaf665f740 1 --2- 172.21.15.123:0/2499127646 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55f595b2fa10 0x55f595b31e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.298+0000 7fbaf665f740 4 client.26279 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:51:44.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.298+0000 7fbaf665f740 1 -- 172.21.15.123:0/2499127646 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f595a88950 con 0x55f595b2fa10 2024-09-19T04:51:44.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.299+0000 7fbaf1200640 1 --2- 172.21.15.123:0/2499127646 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55f595b2fa10 0x55f595b31e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.299+0000 7fbaf1200640 1 --2- 172.21.15.123:0/2499127646 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55f595b2fa10 0x55f595b31e80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ede092faae3b032 server_cookie=d97718464ddc876f in_seq=0 out_seq=0 2024-09-19T04:51:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.299+0000 7fbad6a00640 10 client.26279 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:51:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.300+0000 7fbad6a00640 1 -- 172.21.15.123:0/2499127646 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f595a88950 con 0x55f595b2fa10 2024-09-19T04:51:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.300+0000 7fbad6a00640 10 client.26279 handle_command_reply: tid=0 2024-09-19T04:51:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.300+0000 7fbaf665f740 1 client.26279 shutdown 2024-09-19T04:51:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.300+0000 7fbaf665f740 1 -- 172.21.15.123:0/2499127646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b1fd70 msgr2=0x55f595b1d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.300+0000 7fbaf665f740 1 --2- 172.21.15.123:0/2499127646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b1fd70 0x55f595b1d6c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0093a0 tx=0x7fbadc004730 comp rx=0 tx=0).stop 2024-09-19T04:51:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.300+0000 7fbaf665f740 1 -- 172.21.15.123:0/2499127646 shutdown_connections 2024-09-19T04:51:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.300+0000 7fbaf665f740 1 --2- 172.21.15.123:0/2499127646 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55f595b2fa10 0x55f595b31e80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.300+0000 7fbaf665f740 1 --2- 172.21.15.123:0/2499127646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b1fd70 0x55f595b1d6c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.300+0000 7fbaf665f740 1 --2- 172.21.15.123:0/2499127646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b154b0 0x55f595b1d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.300+0000 7fbad6a00640 0 client.26279 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:51:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.300+0000 7fbaf665f740 1 --2- 172.21.15.123:0/2499127646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b14b90 0x55f595b1cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.300+0000 7fbaf665f740 1 -- 172.21.15.123:0/2499127646 >> 172.21.15.123:0/2499127646 conn(0x55f595b11120 msgr2=0x55f595b16e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:44.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.301+0000 7fbaf665f740 1 -- 172.21.15.123:0/2499127646 shutdown_connections 2024-09-19T04:51:44.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.301+0000 7fbaf665f740 1 -- 172.21.15.123:0/2499127646 wait complete. 2024-09-19T04:51:44.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.301+0000 7fbaf665f740 20 client.26279 trim_cache size 0 max 16384 2024-09-19T04:51:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.319+0000 7fbaf665f740 1 Processor -- start 2024-09-19T04:51:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.319+0000 7fbaf665f740 1 -- start start 2024-09-19T04:51:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.319+0000 7fbaf665f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b14b90 0x55f595b17640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.319+0000 7fbaf665f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b154b0 0x55f595b37750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.319+0000 7fbaf665f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b1fd70 0x55f595b37c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.319+0000 7fbaf665f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1af990 con 0x55f595b1fd70 2024-09-19T04:51:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.319+0000 7fbaf665f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1b14d0 con 0x55f595b154b0 2024-09-19T04:51:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.319+0000 7fbaf665f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1b0060 con 0x55f595b14b90 2024-09-19T04:51:44.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.320+0000 7fbaf1200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b14b90 0x55f595b17640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.320+0000 7fbaebe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b154b0 0x55f595b37750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.320+0000 7fbaf1200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b14b90 0x55f595b17640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53224/0 (socket says 172.21.15.123:53224) 2024-09-19T04:51:44.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.320+0000 7fbaebe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b154b0 0x55f595b37750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:41838/0 (socket says 172.21.15.123:41838) 2024-09-19T04:51:44.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.320+0000 7fbaf1200640 1 -- 172.21.15.123:0/3449485207 learned_addr learned my addr 172.21.15.123:0/3449485207 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:44.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.320+0000 7fbaf1c00640 1 --2- 172.21.15.123:0/3449485207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b1fd70 0x55f595b37c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.320+0000 7fbaebe00640 1 -- 172.21.15.123:0/3449485207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b14b90 msgr2=0x55f595b17640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:44.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.320+0000 7fbaebe00640 1 --2- 172.21.15.123:0/3449485207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b14b90 0x55f595b17640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.320+0000 7fbaebe00640 1 -- 172.21.15.123:0/3449485207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b1fd70 msgr2=0x55f595b37c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:44.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.320+0000 7fbaebe00640 1 --2- 172.21.15.123:0/3449485207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b1fd70 0x55f595b37c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.320+0000 7fbaebe00640 1 -- 172.21.15.123:0/3449485207 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f595b10bc0 con 0x55f595b154b0 2024-09-19T04:51:44.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.320+0000 7fbaf1200640 1 --2- 172.21.15.123:0/3449485207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b14b90 0x55f595b17640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:44.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.320+0000 7fbaebe00640 1 --2- 172.21.15.123:0/3449485207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b154b0 0x55f595b37750 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbad80cd770 tx=0x7fbad800ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b8a0b59f9b8b0b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:44.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.320+0000 7fbae1600640 1 -- 172.21.15.123:0/3449485207 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad809be40 con 0x55f595b154b0 2024-09-19T04:51:44.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.321+0000 7fbae1600640 1 -- 172.21.15.123:0/3449485207 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbad809ca00 con 0x55f595b154b0 2024-09-19T04:51:44.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.321+0000 7fbae1600640 1 -- 172.21.15.123:0/3449485207 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad809bc60 con 0x55f595b154b0 2024-09-19T04:51:44.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.321+0000 7fbaf665f740 1 -- 172.21.15.123:0/3449485207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b154b0 msgr2=0x55f595b37750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:44.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.321+0000 7fbaf665f740 1 --2- 172.21.15.123:0/3449485207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b154b0 0x55f595b37750 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbad80cd770 tx=0x7fbad800ce60 comp rx=0 tx=0).stop 2024-09-19T04:51:44.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.322+0000 7fbaf665f740 1 -- 172.21.15.123:0/3449485207 shutdown_connections 2024-09-19T04:51:44.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.322+0000 7fbaf665f740 1 --2- 172.21.15.123:0/3449485207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b1fd70 0x55f595b37c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.322+0000 7fbaf665f740 1 --2- 172.21.15.123:0/3449485207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b154b0 0x55f595b37750 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.322+0000 7fbaf665f740 1 --2- 172.21.15.123:0/3449485207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b14b90 0x55f595b17640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.322+0000 7fbaf665f740 1 -- 172.21.15.123:0/3449485207 >> 172.21.15.123:0/3449485207 conn(0x55f595b11120 msgr2=0x55f595b1b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:44.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.322+0000 7fbaf665f740 1 -- 172.21.15.123:0/3449485207 shutdown_connections 2024-09-19T04:51:44.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.322+0000 7fbaf665f740 1 -- 172.21.15.123:0/3449485207 wait complete. 2024-09-19T04:51:44.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.322+0000 7fbaf665f740 1 -- start start 2024-09-19T04:51:44.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.322+0000 7fbaf665f740 1 Processor -- start 2024-09-19T04:51:44.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.323+0000 7fbaf665f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b14b90 0x55f595b23c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.323+0000 7fbaf665f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b154b0 0x55f595b24140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.323+0000 7fbaf665f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b1fd70 0x55f595b24680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.323+0000 7fbaf665f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1b0e00 con 0x55f595b1fd70 2024-09-19T04:51:44.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.323+0000 7fbaf665f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1b0730 con 0x55f595b14b90 2024-09-19T04:51:44.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.323+0000 7fbaf665f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1ad970 con 0x55f595b154b0 2024-09-19T04:51:44.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.323+0000 7fbaf665f740 10 client.0 _ll_register_callbacks cb 0x7fbadc01e820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:51:44.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.323+0000 7fbaf1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b14b90 0x55f595b23c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.323+0000 7fbaebe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b154b0 0x55f595b24140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.323+0000 7fbaf1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b14b90 0x55f595b23c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:41848/0 (socket says 172.21.15.123:41848) 2024-09-19T04:51:44.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.323+0000 7fbaf1200640 1 -- 172.21.15.123:0/2213908041 learned_addr learned my addr 172.21.15.123:0/2213908041 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:51:44.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.323+0000 7fbaf1c00640 1 --2- 172.21.15.123:0/2213908041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b1fd70 0x55f595b24680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.324+0000 7fbaf1200640 1 -- 172.21.15.123:0/2213908041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b154b0 msgr2=0x55f595b24140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:44.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.324+0000 7fbaf1200640 1 --2- 172.21.15.123:0/2213908041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b154b0 0x55f595b24140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.324+0000 7fbaf1200640 1 -- 172.21.15.123:0/2213908041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b1fd70 msgr2=0x55f595b24680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:51:44.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.324+0000 7fbaf1200640 1 --2- 172.21.15.123:0/2213908041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b1fd70 0x55f595b24680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.324+0000 7fbaf1200640 1 -- 172.21.15.123:0/2213908041 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f595b99970 con 0x55f595b14b90 2024-09-19T04:51:44.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.324+0000 7fbaebe00640 1 --2- 172.21.15.123:0/2213908041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b154b0 0x55f595b24140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:51:44.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.324+0000 7fbaf1200640 1 --2- 172.21.15.123:0/2213908041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b14b90 0x55f595b23c00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbae4000f80 tx=0x7fbae402f430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=999266b44ee276bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:51:44.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.324+0000 7fbad6a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:51:44.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.324+0000 7fbad6a00640 1 -- 172.21.15.123:0/2213908041 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae404a040 con 0x55f595b14b90 2024-09-19T04:51:44.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.324+0000 7fbad6a00640 1 -- 172.21.15.123:0/2213908041 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fbae4031db0 con 0x55f595b14b90 2024-09-19T04:51:44.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.324+0000 7fbaf665f740 1 -- 172.21.15.123:0/2213908041 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbad809ca00 con 0x55f595b14b90 2024-09-19T04:51:44.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.325+0000 7fbad6a00640 1 -- 172.21.15.123:0/2213908041 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbae40314d0 con 0x55f595b14b90 2024-09-19T04:51:44.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.325+0000 7fbaf665f740 10 client.26285 fetch_fsmap learned FSMap version 36 2024-09-19T04:51:44.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.325+0000 7fbaf665f740 1 -- 172.21.15.123:0/2213908041 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f595b9ac80 con 0x55f595b14b90 2024-09-19T04:51:44.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.325+0000 7fbad6a00640 1 -- 172.21.15.123:0/2213908041 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbad809ca00 con 0x55f595b14b90 2024-09-19T04:51:44.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.325+0000 7fbad6a00640 1 -- 172.21.15.123:0/2213908041 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fbae4043070 con 0x55f595b14b90 2024-09-19T04:51:44.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.325+0000 7fbaf665f740 10 client.26285 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:51:44.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.325+0000 7fbaf665f740 10 client.26285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:51:44.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.325+0000 7fbaf665f740 20 client.26285 populate_metadata read hostname 'smithi123' 2024-09-19T04:51:44.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.325+0000 7fbaf665f740 1 --2- 172.21.15.123:0/2213908041 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55f595b9b0d0 0x55f595b9d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:51:44.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.325+0000 7fbaf665f740 4 client.26285 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:51:44.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.325+0000 7fbaf665f740 1 -- 172.21.15.123:0/2213908041 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f595a82e60 con 0x55f595b9b0d0 2024-09-19T04:51:44.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.326+0000 7fbaebe00640 1 --2- 172.21.15.123:0/2213908041 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55f595b9b0d0 0x55f595b9d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:51:44.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.326+0000 7fbaebe00640 1 --2- 172.21.15.123:0/2213908041 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55f595b9b0d0 0x55f595b9d4c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e452b7f45424910 server_cookie=6c903223ab58605d in_seq=0 out_seq=0 2024-09-19T04:51:44.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.326+0000 7fbad6a00640 10 client.26285 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:51:44.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.326+0000 7fbad6a00640 1 -- 172.21.15.123:0/2213908041 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f595a82e60 con 0x55f595b9b0d0 2024-09-19T04:51:44.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.326+0000 7fbad6a00640 10 client.26285 handle_command_reply: tid=0 2024-09-19T04:51:44.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.326+0000 7fbaf665f740 1 client.26285 shutdown 2024-09-19T04:51:44.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.327+0000 7fbaf665f740 1 -- 172.21.15.123:0/2213908041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b14b90 msgr2=0x55f595b23c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:44.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.327+0000 7fbaf665f740 1 --2- 172.21.15.123:0/2213908041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b14b90 0x55f595b23c00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbae4000f80 tx=0x7fbae402f430 comp rx=0 tx=0).stop 2024-09-19T04:51:44.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.327+0000 7fbaf665f740 1 -- 172.21.15.123:0/2213908041 shutdown_connections 2024-09-19T04:51:44.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.327+0000 7fbaf665f740 1 --2- 172.21.15.123:0/2213908041 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55f595b9b0d0 0x55f595b9d4c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.327+0000 7fbaf665f740 1 --2- 172.21.15.123:0/2213908041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f595b1fd70 0x55f595b24680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.327+0000 7fbaf665f740 1 --2- 172.21.15.123:0/2213908041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f595b154b0 0x55f595b24140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.327+0000 7fbaf665f740 1 --2- 172.21.15.123:0/2213908041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f595b14b90 0x55f595b23c00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.327+0000 7fbaf665f740 1 -- 172.21.15.123:0/2213908041 >> 172.21.15.123:0/2213908041 conn(0x55f595b11120 msgr2=0x55f595b1cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:44.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.327+0000 7fbad6a00640 0 client.26285 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:51:44.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.327+0000 7fbaf665f740 1 -- 172.21.15.123:0/2213908041 shutdown_connections 2024-09-19T04:51:44.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.327+0000 7fbaf665f740 1 -- 172.21.15.123:0/2213908041 wait complete. 2024-09-19T04:51:44.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.327+0000 7fbaf665f740 20 client.26285 trim_cache size 0 max 16384 2024-09-19T04:51:44.332 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:51:44.332 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:51:44.332 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": { 2024-09-19T04:51:44.332 INFO:teuthology.orchestra.run.smithi123.stdout: "d1ca1c94-4939-43d7-8a4d-6df4ba1e8d65": { 2024-09-19T04:51:44.333 INFO:teuthology.orchestra.run.smithi123.stdout: "path": "/", 2024-09-19T04:51:44.333 INFO:teuthology.orchestra.run.smithi123.stdout: "tag": "d1ca1c94-4939-43d7-8a4d-6df4ba1e8d65", 2024-09-19T04:51:44.333 INFO:teuthology.orchestra.run.smithi123.stdout: "options": "recursive,force" 2024-09-19T04:51:44.333 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:51:44.333 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:51:44.333 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:51:44.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.328+0000 7fbaf3c00640 1 -- 172.21.15.123:0/2200302869 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbad00a33b0 msgr2=0x7fbad00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:44.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.328+0000 7fbaf3c00640 1 --2- 172.21.15.123:0/2200302869 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbad00a33b0 0x7fbad00a5870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009d30 tx=0x7fbadc005f90 comp rx=0 tx=0).stop 2024-09-19T04:51:44.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.328+0000 7fbaf3c00640 1 -- 172.21.15.123:0/2200302869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaec104970 msgr2=0x7fbaec19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:51:44.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.328+0000 7fbaf3c00640 1 --2- 172.21.15.123:0/2200302869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaec104970 0x7fbaec19ab40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbad800e9f0 tx=0x7fbad800edf0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.328+0000 7fbaf3c00640 1 -- 172.21.15.123:0/2200302869 shutdown_connections 2024-09-19T04:51:44.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.328+0000 7fbaf3c00640 1 --2- 172.21.15.123:0/2200302869 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbad00a33b0 0x7fbad00a5870 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.328+0000 7fbaf3c00640 1 --2- 172.21.15.123:0/2200302869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaec1090f0 0x7fbaec19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.328+0000 7fbaf3c00640 1 --2- 172.21.15.123:0/2200302869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaec104970 0x7fbaec19ab40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.328+0000 7fbaf3c00640 1 --2- 172.21.15.123:0/2200302869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaec103fc0 0x7fbaec19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:51:44.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.328+0000 7fbaf3c00640 1 -- 172.21.15.123:0/2200302869 >> 172.21.15.123:0/2200302869 conn(0x7fbaec0ffa40 msgr2=0x7fbaec100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:51:44.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.328+0000 7fbaf3c00640 1 -- 172.21.15.123:0/2200302869 shutdown_connections 2024-09-19T04:51:44.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:51:44.328+0000 7fbaf3c00640 1 -- 172.21.15.123:0/2200302869 wait complete. 2024-09-19T04:51:44.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:44 smithi123 ceph-mon[30511]: scrub queued for path: / 2024-09-19T04:51:44.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:44 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:44.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:44 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:51:44.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:44 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:44.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:44 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:51:44.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:44 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:44.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:44 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:51:44.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:44 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:44.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:44 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:44.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:44 smithi193 ceph-mon[35592]: scrub queued for path: / 2024-09-19T04:51:44.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:44 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:44.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:44 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:51:44.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:44 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:44.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:44 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:51:44.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:44 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:44.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:44 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:51:44.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:44 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:44.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:44 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:44.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:44 smithi183 ceph-mon[35333]: scrub queued for path: / 2024-09-19T04:51:44.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:44 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:44.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:44 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:51:44.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:44 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:44.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:44 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:51:44.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:44 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:44.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:44 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:51:44.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:44 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:44.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:44 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:51:45.114 INFO:tasks.cephfs.filesystem:scrub status for tag:d1ca1c94-4939-43d7-8a4d-6df4ba1e8d65 - {'path': '/', 'tag': 'd1ca1c94-4939-43d7-8a4d-6df4ba1e8d65', 'options': 'recursive,force'} 2024-09-19T04:51:45.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:45 smithi123 ceph-mon[30511]: pgmap v154: 97 pgs: 97 active+clean; 185 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.3 MiB/s wr, 805 op/s 2024-09-19T04:51:45.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:45 smithi193 ceph-mon[35592]: pgmap v154: 97 pgs: 97 active+clean; 185 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.3 MiB/s wr, 805 op/s 2024-09-19T04:51:45.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:45 smithi183 ceph-mon[35333]: pgmap v154: 97 pgs: 97 active+clean; 185 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.3 MiB/s wr, 805 op/s 2024-09-19T04:51:46.621 INFO:tasks.workunit.client.0.smithi123.stdout:copying to largest ever: 0x63ef9ee 2024-09-19T04:51:47.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:47 smithi123 ceph-mon[30511]: pgmap v155: 97 pgs: 97 active+clean; 185 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.4 MiB/s wr, 538 op/s 2024-09-19T04:51:47.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:47 smithi193 ceph-mon[35592]: pgmap v155: 97 pgs: 97 active+clean; 185 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.4 MiB/s wr, 538 op/s 2024-09-19T04:51:47.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:47 smithi183 ceph-mon[35333]: pgmap v155: 97 pgs: 97 active+clean; 185 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.4 MiB/s wr, 538 op/s 2024-09-19T04:51:49.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:49 smithi123 ceph-mon[30511]: pgmap v156: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 802 op/s 2024-09-19T04:51:49.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:49 smithi193 ceph-mon[35592]: pgmap v156: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 802 op/s 2024-09-19T04:51:49.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:49 smithi183 ceph-mon[35333]: pgmap v156: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 802 op/s 2024-09-19T04:51:51.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:51 smithi123 ceph-mon[30511]: pgmap v157: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 533 op/s 2024-09-19T04:51:51.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:51 smithi193 ceph-mon[35592]: pgmap v157: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 533 op/s 2024-09-19T04:51:51.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:51 smithi183 ceph-mon[35333]: pgmap v157: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 533 op/s 2024-09-19T04:51:53.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:53 smithi123 ceph-mon[30511]: pgmap v158: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 533 op/s 2024-09-19T04:51:53.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:53 smithi193 ceph-mon[35592]: pgmap v158: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 533 op/s 2024-09-19T04:51:53.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:53 smithi183 ceph-mon[35333]: pgmap v158: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 533 op/s 2024-09-19T04:51:55.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:55 smithi123 ceph-mon[30511]: pgmap v159: 97 pgs: 97 active+clean; 180 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 808 op/s 2024-09-19T04:51:55.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:55 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:51:55.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:55 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:51:55.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:55 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:51:55.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:55 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:51:55.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:55 smithi193 ceph-mon[35592]: pgmap v159: 97 pgs: 97 active+clean; 180 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 808 op/s 2024-09-19T04:51:55.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:55 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:51:55.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:55 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:51:55.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:55 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:51:55.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:55 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:51:55.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:55 smithi183 ceph-mon[35333]: pgmap v159: 97 pgs: 97 active+clean; 180 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 808 op/s 2024-09-19T04:51:55.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:55 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:51:55.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:55 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:51:55.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:55 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:51:55.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:55 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:51:56.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:56 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:51:56.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:56 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:51:56.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:56 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:51:57.391 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:57 smithi193 ceph-mon[35592]: pgmap v160: 97 pgs: 97 active+clean; 180 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 539 op/s 2024-09-19T04:51:57.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:57 smithi123 ceph-mon[30511]: pgmap v160: 97 pgs: 97 active+clean; 180 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 539 op/s 2024-09-19T04:51:57.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:57 smithi183 ceph-mon[35333]: pgmap v160: 97 pgs: 97 active+clean; 180 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 539 op/s 2024-09-19T04:51:58.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:58 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:51:58.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:58 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:51:58.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:58 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:51:59.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:59 smithi123 ceph-mon[30511]: pgmap v161: 97 pgs: 97 active+clean; 192 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.2 MiB/s wr, 816 op/s 2024-09-19T04:51:59.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:59 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:51:59.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:59 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:51:59.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:51:59 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:51:59.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:59 smithi193 ceph-mon[35592]: pgmap v161: 97 pgs: 97 active+clean; 192 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.2 MiB/s wr, 816 op/s 2024-09-19T04:51:59.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:59 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:51:59.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:59 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:51:59.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:51:59 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:51:59.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:59 smithi183 ceph-mon[35333]: pgmap v161: 97 pgs: 97 active+clean; 192 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.2 MiB/s wr, 816 op/s 2024-09-19T04:51:59.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:59 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:51:59.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:59 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:51:59.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:51:59 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:52:01.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:01 smithi123 ceph-mon[30511]: pgmap v162: 97 pgs: 97 active+clean; 192 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.5 MiB/s wr, 551 op/s 2024-09-19T04:52:01.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:01 smithi193 ceph-mon[35592]: pgmap v162: 97 pgs: 97 active+clean; 192 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.5 MiB/s wr, 551 op/s 2024-09-19T04:52:01.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:01 smithi183 ceph-mon[35333]: pgmap v162: 97 pgs: 97 active+clean; 192 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.5 MiB/s wr, 551 op/s 2024-09-19T04:52:03.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:03 smithi123 ceph-mon[30511]: pgmap v163: 97 pgs: 97 active+clean; 192 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.5 MiB/s wr, 551 op/s 2024-09-19T04:52:03.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:03 smithi193 ceph-mon[35592]: pgmap v163: 97 pgs: 97 active+clean; 192 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.5 MiB/s wr, 551 op/s 2024-09-19T04:52:03.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:03 smithi183 ceph-mon[35333]: pgmap v163: 97 pgs: 97 active+clean; 192 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.5 MiB/s wr, 551 op/s 2024-09-19T04:52:05.428 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:05 smithi123 ceph-mon[30511]: pgmap v164: 97 pgs: 97 active+clean; 189 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 828 op/s 2024-09-19T04:52:05.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:05 smithi193 ceph-mon[35592]: pgmap v164: 97 pgs: 97 active+clean; 189 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 828 op/s 2024-09-19T04:52:05.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:05 smithi183 ceph-mon[35333]: pgmap v164: 97 pgs: 97 active+clean; 189 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 828 op/s 2024-09-19T04:52:07.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:07 smithi123 ceph-mon[30511]: pgmap v165: 97 pgs: 97 active+clean; 189 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 553 op/s 2024-09-19T04:52:07.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:07 smithi193 ceph-mon[35592]: pgmap v165: 97 pgs: 97 active+clean; 189 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 553 op/s 2024-09-19T04:52:07.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:07 smithi183 ceph-mon[35333]: pgmap v165: 97 pgs: 97 active+clean; 189 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 553 op/s 2024-09-19T04:52:09.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:09 smithi123 ceph-mon[30511]: pgmap v166: 97 pgs: 97 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.2 MiB/s wr, 813 op/s 2024-09-19T04:52:09.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:09 smithi193 ceph-mon[35592]: pgmap v166: 97 pgs: 97 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.2 MiB/s wr, 813 op/s 2024-09-19T04:52:09.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:09 smithi183 ceph-mon[35333]: pgmap v166: 97 pgs: 97 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.2 MiB/s wr, 813 op/s 2024-09-19T04:52:11.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:11 smithi123 ceph-mon[30511]: pgmap v167: 97 pgs: 97 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 536 op/s 2024-09-19T04:52:11.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:11 smithi193 ceph-mon[35592]: pgmap v167: 97 pgs: 97 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 536 op/s 2024-09-19T04:52:11.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:11 smithi183 ceph-mon[35333]: pgmap v167: 97 pgs: 97 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 536 op/s 2024-09-19T04:52:13.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:13 smithi123 ceph-mon[30511]: pgmap v168: 97 pgs: 97 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 536 op/s 2024-09-19T04:52:13.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:13 smithi193 ceph-mon[35592]: pgmap v168: 97 pgs: 97 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 536 op/s 2024-09-19T04:52:13.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:13 smithi183 ceph-mon[35333]: pgmap v168: 97 pgs: 97 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 536 op/s 2024-09-19T04:52:15.116 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:52:15.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:15 smithi123 ceph-mon[30511]: pgmap v169: 97 pgs: 97 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 794 op/s 2024-09-19T04:52:15.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:15 smithi193 ceph-mon[35592]: pgmap v169: 97 pgs: 97 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 794 op/s 2024-09-19T04:52:15.548 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:52:15.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:15 smithi183 ceph-mon[35333]: pgmap v169: 97 pgs: 97 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 794 op/s 2024-09-19T04:52:16.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.137+0000 7f4b05e00640 1 -- 172.21.15.123:0/1457804739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b00101de0 msgr2=0x7f4b001021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:16.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.137+0000 7f4b05e00640 1 --2- 172.21.15.123:0/1457804739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b00101de0 0x7f4b001021c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4af0009900 tx=0x7f4af002a410 comp rx=0 tx=0).stop 2024-09-19T04:52:16.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.137+0000 7f4b05e00640 1 -- 172.21.15.123:0/1457804739 shutdown_connections 2024-09-19T04:52:16.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.137+0000 7f4b05e00640 1 --2- 172.21.15.123:0/1457804739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b00106f10 0x7f4b0010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.137+0000 7f4b05e00640 1 --2- 172.21.15.123:0/1457804739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b00102790 0x7f4b001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.137+0000 7f4b05e00640 1 --2- 172.21.15.123:0/1457804739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b00101de0 0x7f4b001021c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.137+0000 7f4b05e00640 1 -- 172.21.15.123:0/1457804739 >> 172.21.15.123:0/1457804739 conn(0x7f4b000fd8c0 msgr2=0x7f4b000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:16.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.137+0000 7f4b05e00640 1 -- 172.21.15.123:0/1457804739 shutdown_connections 2024-09-19T04:52:16.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.137+0000 7f4b05e00640 1 -- 172.21.15.123:0/1457804739 wait complete. 2024-09-19T04:52:16.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.138+0000 7f4b05e00640 1 Processor -- start 2024-09-19T04:52:16.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.139+0000 7f4b05e00640 1 -- start start 2024-09-19T04:52:16.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.139+0000 7f4b05e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b00101de0 0x7f4b00198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.139+0000 7f4b05e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b00102790 0x7f4b001989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.139+0000 7f4b05e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b00106f10 0x7f4b0019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.139+0000 7f4b05e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b00111fc0 con 0x7f4b00101de0 2024-09-19T04:52:16.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.139+0000 7f4b05e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b00111e40 con 0x7f4b00102790 2024-09-19T04:52:16.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.139+0000 7f4b05e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4b00112140 con 0x7f4b00106f10 2024-09-19T04:52:16.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.139+0000 7f4af7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b00102790 0x7f4b001989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.139+0000 7f4afea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b00101de0 0x7f4b00198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.139+0000 7f4aff400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b00106f10 0x7f4b0019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.139+0000 7f4af7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b00102790 0x7f4b001989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:57276/0 (socket says 172.21.15.123:57276) 2024-09-19T04:52:16.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.139+0000 7f4af7e00640 1 -- 172.21.15.123:0/2620583037 learned_addr learned my addr 172.21.15.123:0/2620583037 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:16.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.140+0000 7f4af7e00640 1 -- 172.21.15.123:0/2620583037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b00106f10 msgr2=0x7f4b0019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:16.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.140+0000 7f4af7e00640 1 --2- 172.21.15.123:0/2620583037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b00106f10 0x7f4b0019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.140+0000 7f4af7e00640 1 -- 172.21.15.123:0/2620583037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b00101de0 msgr2=0x7f4b00198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:16.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.140+0000 7f4af7e00640 1 --2- 172.21.15.123:0/2620583037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b00101de0 0x7f4b00198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.140+0000 7f4af7e00640 1 -- 172.21.15.123:0/2620583037 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0019d410 con 0x7f4b00102790 2024-09-19T04:52:16.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.140+0000 7f4aff400640 1 --2- 172.21.15.123:0/2620583037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b00106f10 0x7f4b0019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:52:16.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.140+0000 7f4af7e00640 1 --2- 172.21.15.123:0/2620583037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b00102790 0x7f4b001989a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800e9f0 tx=0x7f4ae800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:16.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.140+0000 7f4b04a00640 1 -- 172.21.15.123:0/2620583037 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae800cd80 con 0x7f4b00102790 2024-09-19T04:52:16.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.140+0000 7f4b04a00640 1 -- 172.21.15.123:0/2620583037 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ae8005500 con 0x7f4b00102790 2024-09-19T04:52:16.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.140+0000 7f4b04a00640 1 -- 172.21.15.123:0/2620583037 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8010690 con 0x7f4b00102790 2024-09-19T04:52:16.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.140+0000 7f4b05e00640 1 -- 172.21.15.123:0/2620583037 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0019d700 con 0x7f4b00102790 2024-09-19T04:52:16.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.141+0000 7f4b05e00640 1 -- 172.21.15.123:0/2620583037 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b001a4fe0 con 0x7f4b00102790 2024-09-19T04:52:16.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.144+0000 7f4b04a00640 1 -- 172.21.15.123:0/2620583037 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4ae80046c0 con 0x7f4b00102790 2024-09-19T04:52:16.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.144+0000 7f4b088a2740 1 Processor -- start 2024-09-19T04:52:16.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.144+0000 7f4b088a2740 1 -- start start 2024-09-19T04:52:16.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.144+0000 7f4b088a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f6fa0 0x56189d9f7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4b088a2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f78c0 0x56189da01c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4b088a2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 0x56189da04570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4b088a2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56189d9593e0 con 0x56189d9f78c0 2024-09-19T04:52:16.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4b088a2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56189d900d80 con 0x56189da02180 2024-09-19T04:52:16.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4b088a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56189d9553a0 con 0x56189d9f6fa0 2024-09-19T04:52:16.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4aff400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f6fa0 0x56189d9f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4aff400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f6fa0 0x56189d9f7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:40658/0 (socket says 172.21.15.123:40658) 2024-09-19T04:52:16.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4aff400640 1 -- 172.21.15.123:0/129049604 learned_addr learned my addr 172.21.15.123:0/129049604 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:16.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4af7e00640 1 --2- 172.21.15.123:0/129049604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 0x56189da04570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4afea00640 1 --2- 172.21.15.123:0/129049604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f78c0 0x56189da01c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4b04a00640 1 --2- 172.21.15.123:0/2620583037 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f4ae40a3430 0x7f4ae40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4aff400640 1 -- 172.21.15.123:0/129049604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 msgr2=0x56189da04570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:16.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4aff400640 1 --2- 172.21.15.123:0/129049604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 0x56189da04570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4aff400640 1 -- 172.21.15.123:0/129049604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f78c0 msgr2=0x56189da01c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:16.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4aff400640 1 --2- 172.21.15.123:0/129049604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f78c0 0x56189da01c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4aff400640 1 -- 172.21.15.123:0/129049604 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56189d91fae0 con 0x56189d9f6fa0 2024-09-19T04:52:16.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4afea00640 1 --2- 172.21.15.123:0/129049604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f78c0 0x56189da01c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:52:16.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.145+0000 7f4b04a00640 1 -- 172.21.15.123:0/2620583037 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f4ae80d00d0 con 0x7f4b00102790 2024-09-19T04:52:16.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.146+0000 7f4aff400640 1 --2- 172.21.15.123:0/2620583037 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f4ae40a3430 0x7f4ae40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.146+0000 7f4aff400640 1 --2- 172.21.15.123:0/129049604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f6fa0 0x56189d9f7380 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4af8011790 tx=0x7f4af8011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6a1a7a76388a9fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:16.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.146+0000 7f4aefe00640 1 -- 172.21.15.123:0/129049604 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af800f9a0 con 0x56189d9f6fa0 2024-09-19T04:52:16.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.146+0000 7f4aefe00640 1 -- 172.21.15.123:0/129049604 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4af800fc80 con 0x56189d9f6fa0 2024-09-19T04:52:16.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.146+0000 7f4aefe00640 1 -- 172.21.15.123:0/129049604 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af801cbc0 con 0x56189d9f6fa0 2024-09-19T04:52:16.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.147+0000 7f4b088a2740 1 -- 172.21.15.123:0/129049604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f6fa0 msgr2=0x56189d9f7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:16.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.147+0000 7f4aff400640 1 --2- 172.21.15.123:0/2620583037 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f4ae40a3430 0x7f4ae40a58f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4af8020000 tx=0x7f4af8021000 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:16.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.147+0000 7f4b088a2740 1 --2- 172.21.15.123:0/129049604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f6fa0 0x56189d9f7380 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4af8011790 tx=0x7f4af8011c50 comp rx=0 tx=0).stop 2024-09-19T04:52:16.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.147+0000 7f4b088a2740 1 -- 172.21.15.123:0/129049604 shutdown_connections 2024-09-19T04:52:16.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.147+0000 7f4b088a2740 1 --2- 172.21.15.123:0/129049604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 0x56189da04570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.147+0000 7f4b088a2740 1 --2- 172.21.15.123:0/129049604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f78c0 0x56189da01c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.147+0000 7f4b088a2740 1 --2- 172.21.15.123:0/129049604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f6fa0 0x56189d9f7380 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.147+0000 7f4b088a2740 1 -- 172.21.15.123:0/129049604 >> 172.21.15.123:0/129049604 conn(0x56189d9f3530 msgr2=0x56189d9f3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:16.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.148+0000 7f4b088a2740 1 -- 172.21.15.123:0/129049604 shutdown_connections 2024-09-19T04:52:16.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.148+0000 7f4b088a2740 1 -- 172.21.15.123:0/129049604 wait complete. 2024-09-19T04:52:16.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.148+0000 7f4b088a2740 1 -- start start 2024-09-19T04:52:16.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.148+0000 7f4b088a2740 1 Processor -- start 2024-09-19T04:52:16.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.149+0000 7f4b088a2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f6fa0 0x56189d9ff090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.149+0000 7f4b088a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f78c0 0x56189d9ff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.149+0000 7f4b088a2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 0x56189d9ffb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.149+0000 7f4b088a2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56189da04df0 con 0x56189d9f6fa0 2024-09-19T04:52:16.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.149+0000 7f4b088a2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56189d9140f0 con 0x56189da02180 2024-09-19T04:52:16.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.149+0000 7f4b088a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56189da04f70 con 0x56189d9f78c0 2024-09-19T04:52:16.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.149+0000 7f4b088a2740 10 client.0 _ll_register_callbacks cb 0x56189d9059e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:52:16.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.149+0000 7f4af7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f78c0 0x56189d9ff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.149+0000 7f4afea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f6fa0 0x56189d9ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.149+0000 7f4af7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f78c0 0x56189d9ff5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:40664/0 (socket says 172.21.15.123:40664) 2024-09-19T04:52:16.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.149+0000 7f4af7e00640 1 -- 172.21.15.123:0/2358690398 learned_addr learned my addr 172.21.15.123:0/2358690398 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:16.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.149+0000 7f4aff400640 1 --2- 172.21.15.123:0/2358690398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 0x56189d9ffb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.150+0000 7f4afea00640 1 -- 172.21.15.123:0/2358690398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f78c0 msgr2=0x56189d9ff5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:16.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.150+0000 7f4afea00640 1 --2- 172.21.15.123:0/2358690398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f78c0 0x56189d9ff5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.150+0000 7f4afea00640 1 -- 172.21.15.123:0/2358690398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 msgr2=0x56189d9ffb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:16.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.150+0000 7f4afea00640 1 --2- 172.21.15.123:0/2358690398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 0x56189d9ffb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.150+0000 7f4afea00640 1 -- 172.21.15.123:0/2358690398 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56189d6ca3c0 con 0x56189d9f6fa0 2024-09-19T04:52:16.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.150+0000 7f4afea00640 1 --2- 172.21.15.123:0/2358690398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f6fa0 0x56189d9ff090 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4af002aa00 tx=0x7f4af0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c81ec87d99d87a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:16.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.150+0000 7f4aecc00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:52:16.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.151+0000 7f4aecc00640 1 -- 172.21.15.123:0/2358690398 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af0035040 con 0x56189d9f6fa0 2024-09-19T04:52:16.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.151+0000 7f4aecc00640 1 -- 172.21.15.123:0/2358690398 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f4af0005230 con 0x56189d9f6fa0 2024-09-19T04:52:16.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.151+0000 7f4b088a2740 1 -- 172.21.15.123:0/2358690398 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4af800fc80 con 0x56189d9f6fa0 2024-09-19T04:52:16.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.151+0000 7f4aecc00640 1 -- 172.21.15.123:0/2358690398 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4af0005430 con 0x56189d9f6fa0 2024-09-19T04:52:16.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.151+0000 7f4b088a2740 10 client.16197 fetch_fsmap learned FSMap version 36 2024-09-19T04:52:16.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.151+0000 7f4b088a2740 1 -- 172.21.15.123:0/2358690398 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56189d9eeab0 con 0x56189d9f6fa0 2024-09-19T04:52:16.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.154+0000 7f4aecc00640 1 -- 172.21.15.123:0/2358690398 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4af800fc80 con 0x56189d9f6fa0 2024-09-19T04:52:16.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.154+0000 7f4aecc00640 1 -- 172.21.15.123:0/2358690398 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f4af0003ab0 con 0x56189d9f6fa0 2024-09-19T04:52:16.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.154+0000 7f4b088a2740 10 client.16197 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:52:16.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.154+0000 7f4b088a2740 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:52:16.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.154+0000 7f4b088a2740 20 client.16197 populate_metadata read hostname 'smithi123' 2024-09-19T04:52:16.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.154+0000 7f4b088a2740 1 --2- 172.21.15.123:0/2358690398 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56189d9efec0 0x56189d9f2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.154+0000 7f4b088a2740 4 client.16197 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:52:16.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.154+0000 7f4b088a2740 1 -- 172.21.15.123:0/2358690398 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56189d966950 con 0x56189d9efec0 2024-09-19T04:52:16.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.155+0000 7f4af7e00640 1 --2- 172.21.15.123:0/2358690398 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56189d9efec0 0x56189d9f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.155+0000 7f4af7e00640 1 --2- 172.21.15.123:0/2358690398 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56189d9efec0 0x56189d9f2350 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1659506f29faa74f server_cookie=602f66c50f4c2eda in_seq=0 out_seq=0 2024-09-19T04:52:16.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.155+0000 7f4aecc00640 10 client.16197 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:52:16.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.156+0000 7f4aecc00640 1 -- 172.21.15.123:0/2358690398 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56189d966950 con 0x56189d9efec0 2024-09-19T04:52:16.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.156+0000 7f4aecc00640 10 client.16197 handle_command_reply: tid=0 2024-09-19T04:52:16.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.156+0000 7f4b088a2740 1 client.16197 shutdown 2024-09-19T04:52:16.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.156+0000 7f4b088a2740 1 -- 172.21.15.123:0/2358690398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f6fa0 msgr2=0x56189d9ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:16.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.156+0000 7f4b088a2740 1 --2- 172.21.15.123:0/2358690398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f6fa0 0x56189d9ff090 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4af002aa00 tx=0x7f4af0004270 comp rx=0 tx=0).stop 2024-09-19T04:52:16.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.156+0000 7f4b088a2740 1 -- 172.21.15.123:0/2358690398 shutdown_connections 2024-09-19T04:52:16.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.156+0000 7f4b088a2740 1 --2- 172.21.15.123:0/2358690398 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56189d9efec0 0x56189d9f2350 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.156+0000 7f4b088a2740 1 --2- 172.21.15.123:0/2358690398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 0x56189d9ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.156+0000 7f4b088a2740 1 --2- 172.21.15.123:0/2358690398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f78c0 0x56189d9ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.156+0000 7f4b088a2740 1 --2- 172.21.15.123:0/2358690398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f6fa0 0x56189d9ff090 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.156+0000 7f4b088a2740 1 -- 172.21.15.123:0/2358690398 >> 172.21.15.123:0/2358690398 conn(0x56189d9f3530 msgr2=0x56189d9ee1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:16.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.157+0000 7f4aecc00640 0 client.16197 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:52:16.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.157+0000 7f4b088a2740 1 -- 172.21.15.123:0/2358690398 shutdown_connections 2024-09-19T04:52:16.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.157+0000 7f4b088a2740 1 -- 172.21.15.123:0/2358690398 wait complete. 2024-09-19T04:52:16.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.157+0000 7f4b088a2740 20 client.16197 trim_cache size 0 max 16384 2024-09-19T04:52:16.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.180+0000 7f4b088a2740 1 Processor -- start 2024-09-19T04:52:16.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.180+0000 7f4b088a2740 1 -- start start 2024-09-19T04:52:16.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.181+0000 7f4b088a2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f6fa0 0x56189d9f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.181+0000 7f4b088a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f78c0 0x56189d9ee0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.181+0000 7f4b088a2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 0x56189d9fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.181+0000 7f4b088a2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001ab740 con 0x56189d9f6fa0 2024-09-19T04:52:16.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.181+0000 7f4b088a2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001ad280 con 0x56189da02180 2024-09-19T04:52:16.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.181+0000 7f4b088a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001abe10 con 0x56189d9f78c0 2024-09-19T04:52:16.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.181+0000 7f4af7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f78c0 0x56189d9ee0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.181+0000 7f4afea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f6fa0 0x56189d9f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.181+0000 7f4af7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f78c0 0x56189d9ee0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:40666/0 (socket says 172.21.15.123:40666) 2024-09-19T04:52:16.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.181+0000 7f4af7e00640 1 -- 172.21.15.123:0/1124635580 learned_addr learned my addr 172.21.15.123:0/1124635580 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:16.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.181+0000 7f4aff400640 1 --2- 172.21.15.123:0/1124635580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 0x56189d9fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.181+0000 7f4af7e00640 1 -- 172.21.15.123:0/1124635580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 msgr2=0x56189d9fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:16.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.181+0000 7f4af7e00640 1 --2- 172.21.15.123:0/1124635580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 0x56189d9fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.181+0000 7f4af7e00640 1 -- 172.21.15.123:0/1124635580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f6fa0 msgr2=0x56189d9f1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:16.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.181+0000 7f4af7e00640 1 --2- 172.21.15.123:0/1124635580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f6fa0 0x56189d9f1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.181+0000 7f4af7e00640 1 -- 172.21.15.123:0/1124635580 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56189d9fd5e0 con 0x56189d9f78c0 2024-09-19T04:52:16.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.182+0000 7f4af7e00640 1 --2- 172.21.15.123:0/1124635580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f78c0 0x56189d9ee0b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8098d50 tx=0x7f4ae80d2520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abcaf37603427f84 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:16.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.182+0000 7f4aef400640 1 -- 172.21.15.123:0/1124635580 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae809bce0 con 0x56189d9f78c0 2024-09-19T04:52:16.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.182+0000 7f4aef400640 1 -- 172.21.15.123:0/1124635580 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ae809ca80 con 0x56189d9f78c0 2024-09-19T04:52:16.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.182+0000 7f4aef400640 1 -- 172.21.15.123:0/1124635580 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae80cda70 con 0x56189d9f78c0 2024-09-19T04:52:16.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.183+0000 7f4b088a2740 1 -- 172.21.15.123:0/1124635580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f78c0 msgr2=0x56189d9ee0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:16.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.183+0000 7f4b088a2740 1 --2- 172.21.15.123:0/1124635580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f78c0 0x56189d9ee0b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8098d50 tx=0x7f4ae80d2520 comp rx=0 tx=0).stop 2024-09-19T04:52:16.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.183+0000 7f4b088a2740 1 -- 172.21.15.123:0/1124635580 shutdown_connections 2024-09-19T04:52:16.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.183+0000 7f4b088a2740 1 --2- 172.21.15.123:0/1124635580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 0x56189d9fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.183+0000 7f4b088a2740 1 --2- 172.21.15.123:0/1124635580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f78c0 0x56189d9ee0b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.183+0000 7f4b088a2740 1 --2- 172.21.15.123:0/1124635580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f6fa0 0x56189d9f1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.183+0000 7f4b088a2740 1 -- 172.21.15.123:0/1124635580 >> 172.21.15.123:0/1124635580 conn(0x56189d9f3530 msgr2=0x56189d9ed9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:16.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.183+0000 7f4b088a2740 1 -- 172.21.15.123:0/1124635580 shutdown_connections 2024-09-19T04:52:16.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.183+0000 7f4b088a2740 1 -- 172.21.15.123:0/1124635580 wait complete. 2024-09-19T04:52:16.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.184+0000 7f4b088a2740 1 -- start start 2024-09-19T04:52:16.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.184+0000 7f4b088a2740 1 Processor -- start 2024-09-19T04:52:16.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.185+0000 7f4b088a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f6fa0 0x56189da161d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.185+0000 7f4b088a2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f78c0 0x56189da13520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.185+0000 7f4b088a2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 0x56189da13a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.185+0000 7f4b088a2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001acbb0 con 0x56189d9f78c0 2024-09-19T04:52:16.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.185+0000 7f4b088a2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001ac4e0 con 0x56189da02180 2024-09-19T04:52:16.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.185+0000 7f4b088a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001a9720 con 0x56189d9f6fa0 2024-09-19T04:52:16.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.185+0000 7f4b088a2740 10 client.0 _ll_register_callbacks cb 0x7f4af0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:52:16.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.185+0000 7f4af7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f78c0 0x56189da13520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.185+0000 7f4afea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f6fa0 0x56189da161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.185+0000 7f4af7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f78c0 0x56189da13520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48284/0 (socket says 172.21.15.123:48284) 2024-09-19T04:52:16.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.185+0000 7f4af7e00640 1 -- 172.21.15.123:0/411442877 learned_addr learned my addr 172.21.15.123:0/411442877 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:16.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.185+0000 7f4aff400640 1 --2- 172.21.15.123:0/411442877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 0x56189da13a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.186+0000 7f4af7e00640 1 -- 172.21.15.123:0/411442877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f6fa0 msgr2=0x56189da161d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:16.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.186+0000 7f4af7e00640 1 --2- 172.21.15.123:0/411442877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f6fa0 0x56189da161d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.186+0000 7f4af7e00640 1 -- 172.21.15.123:0/411442877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 msgr2=0x56189da13a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:16.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.186+0000 7f4af7e00640 1 --2- 172.21.15.123:0/411442877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 0x56189da13a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.186+0000 7f4af7e00640 1 -- 172.21.15.123:0/411442877 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56189da14240 con 0x56189d9f78c0 2024-09-19T04:52:16.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.186+0000 7f4af7e00640 1 --2- 172.21.15.123:0/411442877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f78c0 0x56189da13520 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4ae809c080 tx=0x7f4ae800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ae9f1cc1e93dfb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:16.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.186+0000 7f4aecc00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:52:16.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.186+0000 7f4aecc00640 1 -- 172.21.15.123:0/411442877 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae80b4020 con 0x56189d9f78c0 2024-09-19T04:52:16.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.186+0000 7f4aecc00640 1 -- 172.21.15.123:0/411442877 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f4ae80989f0 con 0x56189d9f78c0 2024-09-19T04:52:16.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.186+0000 7f4b088a2740 1 -- 172.21.15.123:0/411442877 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ae809ca80 con 0x56189d9f78c0 2024-09-19T04:52:16.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.187+0000 7f4aecc00640 1 -- 172.21.15.123:0/411442877 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4ae80a4ff0 con 0x56189d9f78c0 2024-09-19T04:52:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.187+0000 7f4b088a2740 10 client.16203 fetch_fsmap learned FSMap version 36 2024-09-19T04:52:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.187+0000 7f4b088a2740 1 -- 172.21.15.123:0/411442877 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56189da78aa0 con 0x56189d9f78c0 2024-09-19T04:52:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.187+0000 7f4aecc00640 1 -- 172.21.15.123:0/411442877 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ae809ca80 con 0x56189d9f78c0 2024-09-19T04:52:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.187+0000 7f4aecc00640 1 -- 172.21.15.123:0/411442877 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f4ae80d2140 con 0x56189d9f78c0 2024-09-19T04:52:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.187+0000 7f4b088a2740 10 client.16203 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:52:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.187+0000 7f4b088a2740 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:52:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.187+0000 7f4b088a2740 20 client.16203 populate_metadata read hostname 'smithi123' 2024-09-19T04:52:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.187+0000 7f4b088a2740 1 --2- 172.21.15.123:0/411442877 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56189da79110 0x56189da7b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.187+0000 7f4b088a2740 4 client.16203 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:52:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.187+0000 7f4b088a2740 1 -- 172.21.15.123:0/411442877 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56189d960e60 con 0x56189da79110 2024-09-19T04:52:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.188+0000 7f4afea00640 1 --2- 172.21.15.123:0/411442877 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56189da79110 0x56189da7b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:16.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.189+0000 7f4afea00640 1 --2- 172.21.15.123:0/411442877 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56189da79110 0x56189da7b500 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f972c5f3d67ce68a server_cookie=17f8d87344b20621 in_seq=0 out_seq=0 2024-09-19T04:52:16.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.189+0000 7f4aecc00640 10 client.16203 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:52:16.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.189+0000 7f4aecc00640 1 -- 172.21.15.123:0/411442877 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56189d960e60 con 0x56189da79110 2024-09-19T04:52:16.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.189+0000 7f4aecc00640 10 client.16203 handle_command_reply: tid=0 2024-09-19T04:52:16.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.189+0000 7f4b088a2740 1 client.16203 shutdown 2024-09-19T04:52:16.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.189+0000 7f4b088a2740 1 -- 172.21.15.123:0/411442877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f78c0 msgr2=0x56189da13520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:16.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.189+0000 7f4b088a2740 1 --2- 172.21.15.123:0/411442877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f78c0 0x56189da13520 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4ae809c080 tx=0x7f4ae800c7d0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.189+0000 7f4b088a2740 1 -- 172.21.15.123:0/411442877 shutdown_connections 2024-09-19T04:52:16.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.189+0000 7f4b088a2740 1 --2- 172.21.15.123:0/411442877 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56189da79110 0x56189da7b500 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.189+0000 7f4b088a2740 1 --2- 172.21.15.123:0/411442877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56189da02180 0x56189da13a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.189+0000 7f4b088a2740 1 --2- 172.21.15.123:0/411442877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56189d9f78c0 0x56189da13520 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.189+0000 7f4b088a2740 1 --2- 172.21.15.123:0/411442877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56189d9f6fa0 0x56189da161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.189+0000 7f4b088a2740 1 -- 172.21.15.123:0/411442877 >> 172.21.15.123:0/411442877 conn(0x56189d9f3530 msgr2=0x56189d9ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.189+0000 7f4aecc00640 0 client.16203 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:52:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.190+0000 7f4b088a2740 1 -- 172.21.15.123:0/411442877 shutdown_connections 2024-09-19T04:52:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.190+0000 7f4b088a2740 1 -- 172.21.15.123:0/411442877 wait complete. 2024-09-19T04:52:16.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.190+0000 7f4b088a2740 20 client.16203 trim_cache size 0 max 16384 2024-09-19T04:52:16.194 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:52:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-09-19T04:52:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-09-19T04:52:16.195 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:52:16.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.190+0000 7f4b05e00640 1 -- 172.21.15.123:0/2620583037 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f4ae40a3430 msgr2=0x7f4ae40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:16.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.190+0000 7f4b05e00640 1 --2- 172.21.15.123:0/2620583037 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f4ae40a3430 0x7f4ae40a58f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4af8020000 tx=0x7f4af8021000 comp rx=0 tx=0).stop 2024-09-19T04:52:16.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.190+0000 7f4b05e00640 1 -- 172.21.15.123:0/2620583037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b00102790 msgr2=0x7f4b001989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:16.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.190+0000 7f4b05e00640 1 --2- 172.21.15.123:0/2620583037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b00102790 0x7f4b001989a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800e9f0 tx=0x7f4ae800edf0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.191+0000 7f4b05e00640 1 -- 172.21.15.123:0/2620583037 shutdown_connections 2024-09-19T04:52:16.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.191+0000 7f4b05e00640 1 --2- 172.21.15.123:0/2620583037 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f4ae40a3430 0x7f4ae40a58f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.191+0000 7f4b05e00640 1 --2- 172.21.15.123:0/2620583037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b00106f10 0x7f4b0019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.191+0000 7f4b05e00640 1 --2- 172.21.15.123:0/2620583037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b00102790 0x7f4b001989a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.191+0000 7f4b05e00640 1 --2- 172.21.15.123:0/2620583037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b00101de0 0x7f4b00198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:16.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.191+0000 7f4b05e00640 1 -- 172.21.15.123:0/2620583037 >> 172.21.15.123:0/2620583037 conn(0x7f4b000fd8c0 msgr2=0x7f4b000febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:16.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.191+0000 7f4b05e00640 1 -- 172.21.15.123:0/2620583037 shutdown_connections 2024-09-19T04:52:16.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:16.191+0000 7f4b05e00640 1 -- 172.21.15.123:0/2620583037 wait complete. 2024-09-19T04:52:17.101 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:52:17.101 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:52:17.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:17 smithi123 ceph-mon[30511]: pgmap v170: 97 pgs: 97 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.5 MiB/s wr, 517 op/s 2024-09-19T04:52:17.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:17 smithi193 ceph-mon[35592]: pgmap v170: 97 pgs: 97 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.5 MiB/s wr, 517 op/s 2024-09-19T04:52:17.522 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:52:17.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:17 smithi183 ceph-mon[35333]: pgmap v170: 97 pgs: 97 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.5 MiB/s wr, 517 op/s 2024-09-19T04:52:18.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.215+0000 7f9d96200640 1 -- 172.21.15.123:0/468274193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d90102d20 msgr2=0x7f9d90103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:18.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.215+0000 7f9d96200640 1 --2- 172.21.15.123:0/468274193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d90102d20 0x7f9d90103100 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9d80009900 tx=0x7f9d8002a480 comp rx=0 tx=0).stop 2024-09-19T04:52:18.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.215+0000 7f9d96200640 1 -- 172.21.15.123:0/468274193 shutdown_connections 2024-09-19T04:52:18.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.215+0000 7f9d96200640 1 --2- 172.21.15.123:0/468274193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d9010c150 0x7f9d9010e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:18.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.215+0000 7f9d96200640 1 --2- 172.21.15.123:0/468274193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d90103640 0x7f9d9010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:18.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.215+0000 7f9d96200640 1 --2- 172.21.15.123:0/468274193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d90102d20 0x7f9d90103100 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:18.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.215+0000 7f9d96200640 1 -- 172.21.15.123:0/468274193 >> 172.21.15.123:0/468274193 conn(0x7f9d900fc780 msgr2=0x7f9d900feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:18.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.215+0000 7f9d96200640 1 -- 172.21.15.123:0/468274193 shutdown_connections 2024-09-19T04:52:18.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.215+0000 7f9d96200640 1 -- 172.21.15.123:0/468274193 wait complete. 2024-09-19T04:52:18.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.216+0000 7f9d96200640 1 Processor -- start 2024-09-19T04:52:18.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.216+0000 7f9d96200640 1 -- start start 2024-09-19T04:52:18.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.217+0000 7f9d96200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d90102d20 0x7f9d9019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:18.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.217+0000 7f9d96200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d90103640 0x7f9d9019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:18.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.217+0000 7f9d96200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d9010c150 0x7f9d9019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:18.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.217+0000 7f9d96200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d90111fa0 con 0x7f9d90103640 2024-09-19T04:52:18.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.217+0000 7f9d96200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9d90111e20 con 0x7f9d9010c150 2024-09-19T04:52:18.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.217+0000 7f9d96200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9d90112120 con 0x7f9d90102d20 2024-09-19T04:52:18.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.217+0000 7f9d8f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d90102d20 0x7f9d9019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:18.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.217+0000 7f9d8f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d90102d20 0x7f9d9019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:40682/0 (socket says 172.21.15.123:40682) 2024-09-19T04:52:18.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.217+0000 7f9d8f400640 1 -- 172.21.15.123:0/1550008289 learned_addr learned my addr 172.21.15.123:0/1550008289 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:18.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.217+0000 7f9d8ea00640 1 --2- 172.21.15.123:0/1550008289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d90103640 0x7f9d9019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:18.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.217+0000 7f9d8fe00640 1 --2- 172.21.15.123:0/1550008289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d9010c150 0x7f9d9019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:18.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.218+0000 7f9d8f400640 1 -- 172.21.15.123:0/1550008289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d9010c150 msgr2=0x7f9d9019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:18.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.218+0000 7f9d8f400640 1 --2- 172.21.15.123:0/1550008289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d9010c150 0x7f9d9019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:18.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.218+0000 7f9d8f400640 1 -- 172.21.15.123:0/1550008289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d90103640 msgr2=0x7f9d9019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:18.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.218+0000 7f9d8f400640 1 --2- 172.21.15.123:0/1550008289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d90103640 0x7f9d9019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:18.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.218+0000 7f9d8f400640 1 -- 172.21.15.123:0/1550008289 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9019f630 con 0x7f9d90102d20 2024-09-19T04:52:18.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.218+0000 7f9d8f400640 1 --2- 172.21.15.123:0/1550008289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d90102d20 0x7f9d9019a680 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9d80004770 tx=0x7f9d80003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:18.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.218+0000 7f9d87e00640 1 -- 172.21.15.123:0/1550008289 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8003f070 con 0x7f9d90102d20 2024-09-19T04:52:18.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.218+0000 7f9d87e00640 1 -- 172.21.15.123:0/1550008289 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d80031ca0 con 0x7f9d90102d20 2024-09-19T04:52:18.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.218+0000 7f9d96200640 1 -- 172.21.15.123:0/1550008289 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9019f8c0 con 0x7f9d90102d20 2024-09-19T04:52:18.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.218+0000 7f9d87e00640 1 -- 172.21.15.123:0/1550008289 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80035030 con 0x7f9d90102d20 2024-09-19T04:52:18.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.218+0000 7f9d96200640 1 -- 172.21.15.123:0/1550008289 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d901a7200 con 0x7f9d90102d20 2024-09-19T04:52:18.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.220+0000 7f9d96200640 1 -- 172.21.15.123:0/1550008289 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d9006b070 con 0x7f9d90102d20 2024-09-19T04:52:18.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.221+0000 7f9d87e00640 1 -- 172.21.15.123:0/1550008289 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9d80042070 con 0x7f9d90102d20 2024-09-19T04:52:18.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.223+0000 7f9d87e00640 1 --2- 172.21.15.123:0/1550008289 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9d740a33b0 0x7f9d740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:18.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.223+0000 7f9d87e00640 1 -- 172.21.15.123:0/1550008289 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f9d800f0310 con 0x7f9d90102d20 2024-09-19T04:52:18.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.223+0000 7f9d8ea00640 1 --2- 172.21.15.123:0/1550008289 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9d740a33b0 0x7f9d740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:18.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.224+0000 7f9d8ea00640 1 --2- 172.21.15.123:0/1550008289 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9d740a33b0 0x7f9d740a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9d900febc0 tx=0x7f9d7c005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:18.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.227+0000 7f9d87e00640 1 -- 172.21.15.123:0/1550008289 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9d800bc730 con 0x7f9d90102d20 2024-09-19T04:52:18.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.497+0000 7f9d96200640 1 -- 172.21.15.123:0/1550008289 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d90103100 con 0x7f9d90102d20 2024-09-19T04:52:18.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.498+0000 7f9d87e00640 1 -- 172.21.15.123:0/1550008289 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11990 (secure 0 0 0) 0x7f9d800c0a30 con 0x7f9d90102d20 2024-09-19T04:52:18.499 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:52:18.500 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":36,"btime":"2024-09-19T04:50:44:998923+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:50:44.262505+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24499,24500,24539,24505,24529]},"id":1}]} 2024-09-19T04:52:18.500 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 36 2024-09-19T04:52:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.500+0000 7f9d96200640 1 -- 172.21.15.123:0/1550008289 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9d740a33b0 msgr2=0x7f9d740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.500+0000 7f9d96200640 1 --2- 172.21.15.123:0/1550008289 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9d740a33b0 0x7f9d740a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9d900febc0 tx=0x7f9d7c005f90 comp rx=0 tx=0).stop 2024-09-19T04:52:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.500+0000 7f9d96200640 1 -- 172.21.15.123:0/1550008289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d90102d20 msgr2=0x7f9d9019a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.500+0000 7f9d96200640 1 --2- 172.21.15.123:0/1550008289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d90102d20 0x7f9d9019a680 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9d80004770 tx=0x7f9d80003c10 comp rx=0 tx=0).stop 2024-09-19T04:52:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.500+0000 7f9d96200640 1 -- 172.21.15.123:0/1550008289 shutdown_connections 2024-09-19T04:52:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.500+0000 7f9d96200640 1 --2- 172.21.15.123:0/1550008289 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9d740a33b0 0x7f9d740a5870 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.500+0000 7f9d96200640 1 --2- 172.21.15.123:0/1550008289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d9010c150 0x7f9d9019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.500+0000 7f9d96200640 1 --2- 172.21.15.123:0/1550008289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d90103640 0x7f9d9019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.500+0000 7f9d96200640 1 --2- 172.21.15.123:0/1550008289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d90102d20 0x7f9d9019a680 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.500+0000 7f9d96200640 1 -- 172.21.15.123:0/1550008289 >> 172.21.15.123:0/1550008289 conn(0x7f9d900fc780 msgr2=0x7f9d901008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.500+0000 7f9d96200640 1 -- 172.21.15.123:0/1550008289 shutdown_connections 2024-09-19T04:52:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:18.500+0000 7f9d96200640 1 -- 172.21.15.123:0/1550008289 wait complete. 2024-09-19T04:52:19.370 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:52:19.426 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:19 smithi123 ceph-mon[30511]: pgmap v171: 97 pgs: 97 active+clean; 190 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.2 MiB/s wr, 773 op/s 2024-09-19T04:52:19.426 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:19 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1550008289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:52:19.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:19 smithi193 ceph-mon[35592]: pgmap v171: 97 pgs: 97 active+clean; 190 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.2 MiB/s wr, 773 op/s 2024-09-19T04:52:19.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:19 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1550008289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:52:19.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:19 smithi183 ceph-mon[35333]: pgmap v171: 97 pgs: 97 active+clean; 190 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.2 MiB/s wr, 773 op/s 2024-09-19T04:52:19.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:19 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1550008289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:52:19.810 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:52:20.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.812+0000 7f65cf200640 1 -- 172.21.15.123:0/2293982926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65d0074eb0 msgr2=0x7f65d01124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:20.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.812+0000 7f65cf200640 1 --2- 172.21.15.123:0/2293982926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65d0074eb0 0x7f65d01124a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009900 tx=0x7f65bc02a4a0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.813+0000 7f65cf200640 1 -- 172.21.15.123:0/2293982926 shutdown_connections 2024-09-19T04:52:20.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.813+0000 7f65cf200640 1 --2- 172.21.15.123:0/2293982926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d01129e0 0x7f65d0114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.813+0000 7f65cf200640 1 --2- 172.21.15.123:0/2293982926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65d0074eb0 0x7f65d01124a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.813+0000 7f65cf200640 1 --2- 172.21.15.123:0/2293982926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d0074590 0x7f65d0074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.813+0000 7f65cf200640 1 -- 172.21.15.123:0/2293982926 >> 172.21.15.123:0/2293982926 conn(0x7f65d00fd710 msgr2=0x7f65d00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:20.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.813+0000 7f65cf200640 1 -- 172.21.15.123:0/2293982926 shutdown_connections 2024-09-19T04:52:20.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.813+0000 7f65cf200640 1 -- 172.21.15.123:0/2293982926 wait complete. 2024-09-19T04:52:20.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.814+0000 7f65cf200640 1 Processor -- start 2024-09-19T04:52:20.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.814+0000 7f65cf200640 1 -- start start 2024-09-19T04:52:20.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.814+0000 7f65cf200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65d0074590 0x7f65d01a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.814+0000 7f65cf200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d0074eb0 0x7f65d01a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.814+0000 7f65cf200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d01129e0 0x7f65d019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.814+0000 7f65cf200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65d0118470 con 0x7f65d01129e0 2024-09-19T04:52:20.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.814+0000 7f65cf200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65d01182f0 con 0x7f65d0074590 2024-09-19T04:52:20.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.814+0000 7f65cf200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65d01185f0 con 0x7f65d0074eb0 2024-09-19T04:52:20.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.815+0000 7f65cde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65d0074590 0x7f65d01a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.815+0000 7f65cd400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d0074eb0 0x7f65d01a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.815+0000 7f65cde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65d0074590 0x7f65d01a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:57334/0 (socket says 172.21.15.123:57334) 2024-09-19T04:52:20.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.815+0000 7f65cde00640 1 -- 172.21.15.123:0/3529000902 learned_addr learned my addr 172.21.15.123:0/3529000902 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:20.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.815+0000 7f65ce800640 1 --2- 172.21.15.123:0/3529000902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d01129e0 0x7f65d019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.815+0000 7f65cde00640 1 -- 172.21.15.123:0/3529000902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d0074eb0 msgr2=0x7f65d01a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:20.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.815+0000 7f65cde00640 1 --2- 172.21.15.123:0/3529000902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d0074eb0 0x7f65d01a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.815+0000 7f65cde00640 1 -- 172.21.15.123:0/3529000902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d01129e0 msgr2=0x7f65d019e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:20.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.815+0000 7f65cde00640 1 --2- 172.21.15.123:0/3529000902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d01129e0 0x7f65d019e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.815+0000 7f65cde00640 1 -- 172.21.15.123:0/3529000902 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d019ef40 con 0x7f65d0074590 2024-09-19T04:52:20.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.815+0000 7f65cd400640 1 --2- 172.21.15.123:0/3529000902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d0074eb0 0x7f65d01a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:52:20.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.815+0000 7f65ce800640 1 --2- 172.21.15.123:0/3529000902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d01129e0 0x7f65d019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:52:20.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.816+0000 7f65cde00640 1 --2- 172.21.15.123:0/3529000902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65d0074590 0x7f65d01a4620 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f65c000e9f0 tx=0x7f65c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:20.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.816+0000 7f65c6a00640 1 -- 172.21.15.123:0/3529000902 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c000cd80 con 0x7f65d0074590 2024-09-19T04:52:20.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.816+0000 7f65c6a00640 1 -- 172.21.15.123:0/3529000902 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c0010910 con 0x7f65d0074590 2024-09-19T04:52:20.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.816+0000 7f65c6a00640 1 -- 172.21.15.123:0/3529000902 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0018990 con 0x7f65d0074590 2024-09-19T04:52:20.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.816+0000 7f65cf200640 1 -- 172.21.15.123:0/3529000902 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d019f230 con 0x7f65d0074590 2024-09-19T04:52:20.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.816+0000 7f65cf200640 1 -- 172.21.15.123:0/3529000902 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d01ab480 con 0x7f65d0074590 2024-09-19T04:52:20.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.819+0000 7f65d79c0740 1 Processor -- start 2024-09-19T04:52:20.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.819+0000 7f65c6a00640 1 -- 172.21.15.123:0/3529000902 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f65c0010ac0 con 0x7f65d0074590 2024-09-19T04:52:20.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.819+0000 7f65d79c0740 1 -- start start 2024-09-19T04:52:20.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.819+0000 7f65d79c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc692e40 0x55fbcc693220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.819+0000 7f65d79c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc693760 0x55fbcc69dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.819+0000 7f65d79c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc69e020 0x55fbcc6a0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.819+0000 7f65d79c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fbcc5f93e0 con 0x55fbcc69e020 2024-09-19T04:52:20.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.819+0000 7f65d79c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fbcc5a0d80 con 0x55fbcc693760 2024-09-19T04:52:20.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.819+0000 7f65d79c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fbcc5f53a0 con 0x55fbcc692e40 2024-09-19T04:52:20.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.820+0000 7f65cde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc693760 0x55fbcc69dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.820+0000 7f65ce800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc692e40 0x55fbcc693220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.820+0000 7f65ce800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc692e40 0x55fbcc693220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:40698/0 (socket says 172.21.15.123:40698) 2024-09-19T04:52:20.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.820+0000 7f65cd400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc69e020 0x55fbcc6a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.820+0000 7f65cde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc693760 0x55fbcc69dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:57340/0 (socket says 172.21.15.123:57340) 2024-09-19T04:52:20.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.820+0000 7f65ce800640 1 -- 172.21.15.123:0/3004428628 learned_addr learned my addr 172.21.15.123:0/3004428628 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:20.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.820+0000 7f65ce800640 1 -- 172.21.15.123:0/3004428628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc693760 msgr2=0x55fbcc69dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:20.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.820+0000 7f65ce800640 1 --2- 172.21.15.123:0/3004428628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc693760 0x55fbcc69dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.820+0000 7f65ce800640 1 -- 172.21.15.123:0/3004428628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc69e020 msgr2=0x55fbcc6a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:20.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.820+0000 7f65ce800640 1 --2- 172.21.15.123:0/3004428628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc69e020 0x55fbcc6a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.820+0000 7f65ce800640 1 -- 172.21.15.123:0/3004428628 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbcc5bfae0 con 0x55fbcc692e40 2024-09-19T04:52:20.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.821+0000 7f65ce800640 1 --2- 172.21.15.123:0/3004428628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc692e40 0x55fbcc693220 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f65c800a810 tx=0x7f65c8009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c30df7a687d09f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:20.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.821+0000 7f65c6a00640 1 --2- 172.21.15.123:0/3529000902 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f65b40a3430 0x7f65b40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.821+0000 7f65bb400640 1 -- 172.21.15.123:0/3004428628 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8018020 con 0x55fbcc692e40 2024-09-19T04:52:20.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.821+0000 7f65bb400640 1 -- 172.21.15.123:0/3004428628 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c8014400 con 0x55fbcc692e40 2024-09-19T04:52:20.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.821+0000 7f65bb400640 1 -- 172.21.15.123:0/3004428628 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c80135c0 con 0x55fbcc692e40 2024-09-19T04:52:20.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.821+0000 7f65c6a00640 1 -- 172.21.15.123:0/3529000902 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f65c00cff60 con 0x7f65d0074590 2024-09-19T04:52:20.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.821+0000 7f65cde00640 1 --2- 172.21.15.123:0/3529000902 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f65b40a3430 0x7f65b40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.822+0000 7f65d79c0740 1 -- 172.21.15.123:0/3004428628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc692e40 msgr2=0x55fbcc693220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:20.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.822+0000 7f65cde00640 1 --2- 172.21.15.123:0/3529000902 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f65b40a3430 0x7f65b40a58f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f65c00cd790 tx=0x7f65c00d3ab0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:20.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.822+0000 7f65d79c0740 1 --2- 172.21.15.123:0/3004428628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc692e40 0x55fbcc693220 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f65c800a810 tx=0x7f65c8009650 comp rx=0 tx=0).stop 2024-09-19T04:52:20.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.822+0000 7f65d79c0740 1 -- 172.21.15.123:0/3004428628 shutdown_connections 2024-09-19T04:52:20.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.822+0000 7f65d79c0740 1 --2- 172.21.15.123:0/3004428628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc69e020 0x55fbcc6a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.822+0000 7f65d79c0740 1 --2- 172.21.15.123:0/3004428628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc693760 0x55fbcc69dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.822+0000 7f65d79c0740 1 --2- 172.21.15.123:0/3004428628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc692e40 0x55fbcc693220 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.822+0000 7f65d79c0740 1 -- 172.21.15.123:0/3004428628 >> 172.21.15.123:0/3004428628 conn(0x55fbcc68f3d0 msgr2=0x55fbcc68f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:20.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.823+0000 7f65d79c0740 1 -- 172.21.15.123:0/3004428628 shutdown_connections 2024-09-19T04:52:20.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.823+0000 7f65d79c0740 1 -- 172.21.15.123:0/3004428628 wait complete. 2024-09-19T04:52:20.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.824+0000 7f65d79c0740 1 -- start start 2024-09-19T04:52:20.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.824+0000 7f65d79c0740 1 Processor -- start 2024-09-19T04:52:20.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.825+0000 7f65d79c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc692e40 0x55fbcc69aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.825+0000 7f65d79c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc693760 0x55fbcc69b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.825+0000 7f65d79c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc69e020 0x55fbcc69b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.825+0000 7f65d79c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fbcc6a0bf0 con 0x55fbcc693760 2024-09-19T04:52:20.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.825+0000 7f65d79c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fbcc5b40f0 con 0x55fbcc692e40 2024-09-19T04:52:20.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.825+0000 7f65d79c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fbcc6a0d70 con 0x55fbcc69e020 2024-09-19T04:52:20.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.825+0000 7f65d79c0740 10 client.0 _ll_register_callbacks cb 0x55fbcc59bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:52:20.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.825+0000 7f65ce800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc69e020 0x55fbcc69b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.825+0000 7f65ce800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc692e40 0x55fbcc69aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.825+0000 7f65ce800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc692e40 0x55fbcc69aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:57350/0 (socket says 172.21.15.123:57350) 2024-09-19T04:52:20.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.825+0000 7f65ce800640 1 -- 172.21.15.123:0/1888617262 learned_addr learned my addr 172.21.15.123:0/1888617262 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:20.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.826+0000 7f65cd400640 1 --2- 172.21.15.123:0/1888617262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc693760 0x55fbcc69b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.826+0000 7f65ce800640 1 -- 172.21.15.123:0/1888617262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc69e020 msgr2=0x55fbcc69b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:20.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.826+0000 7f65ce800640 1 --2- 172.21.15.123:0/1888617262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc69e020 0x55fbcc69b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.826+0000 7f65ce800640 1 -- 172.21.15.123:0/1888617262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc693760 msgr2=0x55fbcc69b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:20.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.826+0000 7f65ce800640 1 --2- 172.21.15.123:0/1888617262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc693760 0x55fbcc69b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.826+0000 7f65ce800640 1 -- 172.21.15.123:0/1888617262 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbcc36a3c0 con 0x55fbcc692e40 2024-09-19T04:52:20.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.826+0000 7f65cd400640 1 --2- 172.21.15.123:0/1888617262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc693760 0x55fbcc69b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:52:20.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.826+0000 7f65ce800640 1 --2- 172.21.15.123:0/1888617262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc692e40 0x55fbcc69aee0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f65c8014ab0 tx=0x7f65c8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0e85b0209c01dd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:20.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.826+0000 7f65afe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:52:20.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.826+0000 7f65afe00640 1 -- 172.21.15.123:0/1888617262 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8018070 con 0x55fbcc692e40 2024-09-19T04:52:20.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.826+0000 7f65afe00640 1 -- 172.21.15.123:0/1888617262 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f65c800ed70 con 0x55fbcc692e40 2024-09-19T04:52:20.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.826+0000 7f65d79c0740 1 -- 172.21.15.123:0/1888617262 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c8014400 con 0x55fbcc692e40 2024-09-19T04:52:20.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.827+0000 7f65afe00640 1 -- 172.21.15.123:0/1888617262 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f65c801d420 con 0x55fbcc692e40 2024-09-19T04:52:20.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.827+0000 7f65d79c0740 10 client.26321 fetch_fsmap learned FSMap version 36 2024-09-19T04:52:20.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.827+0000 7f65d79c0740 1 -- 172.21.15.123:0/1888617262 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fbcc6acc00 con 0x55fbcc692e40 2024-09-19T04:52:20.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.827+0000 7f65afe00640 1 -- 172.21.15.123:0/1888617262 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c8014400 con 0x55fbcc692e40 2024-09-19T04:52:20.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.827+0000 7f65afe00640 1 -- 172.21.15.123:0/1888617262 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f65c801c440 con 0x55fbcc692e40 2024-09-19T04:52:20.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.827+0000 7f65d79c0740 10 client.26321 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:52:20.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.827+0000 7f65d79c0740 10 client.26321 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:52:20.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.827+0000 7f65d79c0740 20 client.26321 populate_metadata read hostname 'smithi123' 2024-09-19T04:52:20.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.827+0000 7f65d79c0740 1 --2- 172.21.15.123:0/1888617262 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55fbcc6adfb0 0x55fbcc6b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.827+0000 7f65d79c0740 4 client.26321 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:52:20.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.827+0000 7f65d79c0740 1 -- 172.21.15.123:0/1888617262 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbcc606950 con 0x55fbcc6adfb0 2024-09-19T04:52:20.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.828+0000 7f65cd400640 1 --2- 172.21.15.123:0/1888617262 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55fbcc6adfb0 0x55fbcc6b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.829+0000 7f65afe00640 10 client.26321 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:52:20.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.829+0000 7f65cd400640 1 --2- 172.21.15.123:0/1888617262 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55fbcc6adfb0 0x55fbcc6b03a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=70c3f72960216a69 server_cookie=4253df7417983e2a in_seq=0 out_seq=0 2024-09-19T04:52:20.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.829+0000 7f65afe00640 1 -- 172.21.15.123:0/1888617262 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fbcc606950 con 0x55fbcc6adfb0 2024-09-19T04:52:20.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.829+0000 7f65afe00640 10 client.26321 handle_command_reply: tid=0 2024-09-19T04:52:20.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.830+0000 7f65d79c0740 1 client.26321 shutdown 2024-09-19T04:52:20.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.830+0000 7f65d79c0740 1 -- 172.21.15.123:0/1888617262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc692e40 msgr2=0x55fbcc69aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:20.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.830+0000 7f65d79c0740 1 --2- 172.21.15.123:0/1888617262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc692e40 0x55fbcc69aee0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f65c8014ab0 tx=0x7f65c8004a40 comp rx=0 tx=0).stop 2024-09-19T04:52:20.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.830+0000 7f65d79c0740 1 -- 172.21.15.123:0/1888617262 shutdown_connections 2024-09-19T04:52:20.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.830+0000 7f65d79c0740 1 --2- 172.21.15.123:0/1888617262 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55fbcc6adfb0 0x55fbcc6b03a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.830+0000 7f65d79c0740 1 --2- 172.21.15.123:0/1888617262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc69e020 0x55fbcc69b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.830+0000 7f65d79c0740 1 --2- 172.21.15.123:0/1888617262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc693760 0x55fbcc69b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.830+0000 7f65d79c0740 1 --2- 172.21.15.123:0/1888617262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc692e40 0x55fbcc69aee0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.830+0000 7f65afe00640 0 client.26321 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:52:20.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.830+0000 7f65d79c0740 1 -- 172.21.15.123:0/1888617262 >> 172.21.15.123:0/1888617262 conn(0x55fbcc68f3d0 msgr2=0x55fbcc6950b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:20.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.831+0000 7f65d79c0740 1 -- 172.21.15.123:0/1888617262 shutdown_connections 2024-09-19T04:52:20.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.831+0000 7f65d79c0740 1 -- 172.21.15.123:0/1888617262 wait complete. 2024-09-19T04:52:20.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.831+0000 7f65d79c0740 20 client.26321 trim_cache size 0 max 16384 2024-09-19T04:52:20.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.846+0000 7f65d79c0740 1 Processor -- start 2024-09-19T04:52:20.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.846+0000 7f65d79c0740 1 -- start start 2024-09-19T04:52:20.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.846+0000 7f65d79c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc692e40 0x55fbcc69b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.846+0000 7f65d79c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc693760 0x55fbcc68e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.846+0000 7f65d79c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc69e020 0x55fbcc6b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.846+0000 7f65d79c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65d01b3dc0 con 0x55fbcc693760 2024-09-19T04:52:20.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.846+0000 7f65d79c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65d01b5900 con 0x55fbcc69e020 2024-09-19T04:52:20.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.846+0000 7f65d79c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65d01b4490 con 0x55fbcc692e40 2024-09-19T04:52:20.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.847+0000 7f65ce800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc692e40 0x55fbcc69b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.847+0000 7f65ce800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc692e40 0x55fbcc69b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:40722/0 (socket says 172.21.15.123:40722) 2024-09-19T04:52:20.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.847+0000 7f65ce800640 1 -- 172.21.15.123:0/3151162162 learned_addr learned my addr 172.21.15.123:0/3151162162 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:20.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.847+0000 7f65cd400640 1 --2- 172.21.15.123:0/3151162162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc693760 0x55fbcc68e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.847+0000 7f65ce800640 1 --2- 172.21.15.123:0/3151162162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc69e020 0x55fbcc6b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.847+0000 7f65ce800640 1 -- 172.21.15.123:0/3151162162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc69e020 msgr2=0x55fbcc6b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:20.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.847+0000 7f65ce800640 1 --2- 172.21.15.123:0/3151162162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc69e020 0x55fbcc6b5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.847+0000 7f65ce800640 1 -- 172.21.15.123:0/3151162162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc693760 msgr2=0x55fbcc68e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:20.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.847+0000 7f65ce800640 1 --2- 172.21.15.123:0/3151162162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc693760 0x55fbcc68e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.847+0000 7f65ce800640 1 -- 172.21.15.123:0/3151162162 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbcc693590 con 0x55fbcc692e40 2024-09-19T04:52:20.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.847+0000 7f65ce800640 1 --2- 172.21.15.123:0/3151162162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc692e40 0x55fbcc69b7b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f65c8014a80 tx=0x7f65c8009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64a82b0726a60a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:20.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.848+0000 7f65baa00640 1 -- 172.21.15.123:0/3151162162 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8018040 con 0x55fbcc692e40 2024-09-19T04:52:20.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.848+0000 7f65baa00640 1 -- 172.21.15.123:0/3151162162 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c8023030 con 0x55fbcc692e40 2024-09-19T04:52:20.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.848+0000 7f65baa00640 1 -- 172.21.15.123:0/3151162162 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c801c3b0 con 0x55fbcc692e40 2024-09-19T04:52:20.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.849+0000 7f65d79c0740 1 -- 172.21.15.123:0/3151162162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc692e40 msgr2=0x55fbcc69b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:20.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.849+0000 7f65d79c0740 1 --2- 172.21.15.123:0/3151162162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc692e40 0x55fbcc69b7b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f65c8014a80 tx=0x7f65c8009650 comp rx=0 tx=0).stop 2024-09-19T04:52:20.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.849+0000 7f65d79c0740 1 -- 172.21.15.123:0/3151162162 shutdown_connections 2024-09-19T04:52:20.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.849+0000 7f65d79c0740 1 --2- 172.21.15.123:0/3151162162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc69e020 0x55fbcc6b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.849+0000 7f65d79c0740 1 --2- 172.21.15.123:0/3151162162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc693760 0x55fbcc68e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.849+0000 7f65d79c0740 1 --2- 172.21.15.123:0/3151162162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc692e40 0x55fbcc69b7b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.849+0000 7f65d79c0740 1 -- 172.21.15.123:0/3151162162 >> 172.21.15.123:0/3151162162 conn(0x55fbcc68f3d0 msgr2=0x55fbcc6a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:20.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.849+0000 7f65d79c0740 1 -- 172.21.15.123:0/3151162162 shutdown_connections 2024-09-19T04:52:20.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.849+0000 7f65d79c0740 1 -- 172.21.15.123:0/3151162162 wait complete. 2024-09-19T04:52:20.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.849+0000 7f65d79c0740 1 -- start start 2024-09-19T04:52:20.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.849+0000 7f65d79c0740 1 Processor -- start 2024-09-19T04:52:20.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.850+0000 7f65d79c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc692e40 0x55fbcc6a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.850+0000 7f65d79c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc693760 0x55fbcc6a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.850+0000 7f65d79c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc69e020 0x55fbcc6a2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.850+0000 7f65d79c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65d01b5230 con 0x55fbcc692e40 2024-09-19T04:52:20.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.850+0000 7f65d79c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65d01b4b60 con 0x55fbcc693760 2024-09-19T04:52:20.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.850+0000 7f65d79c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65d01b1da0 con 0x55fbcc69e020 2024-09-19T04:52:20.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.850+0000 7f65d79c0740 10 client.0 _ll_register_callbacks cb 0x7f65bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:52:20.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.850+0000 7f65ce800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc692e40 0x55fbcc6a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.850+0000 7f65ce800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc692e40 0x55fbcc6a67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48352/0 (socket says 172.21.15.123:48352) 2024-09-19T04:52:20.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.850+0000 7f65ce800640 1 -- 172.21.15.123:0/944874611 learned_addr learned my addr 172.21.15.123:0/944874611 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:20.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.850+0000 7f65cd400640 1 --2- 172.21.15.123:0/944874611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc693760 0x55fbcc6a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.850+0000 7f65ce800640 1 --2- 172.21.15.123:0/944874611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc69e020 0x55fbcc6a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.851+0000 7f65cd400640 1 -- 172.21.15.123:0/944874611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc69e020 msgr2=0x55fbcc6a2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:20.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.851+0000 7f65cd400640 1 --2- 172.21.15.123:0/944874611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc69e020 0x55fbcc6a2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.851+0000 7f65cd400640 1 -- 172.21.15.123:0/944874611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc692e40 msgr2=0x55fbcc6a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:20.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.851+0000 7f65cd400640 1 --2- 172.21.15.123:0/944874611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc692e40 0x55fbcc6a67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.851+0000 7f65cd400640 1 -- 172.21.15.123:0/944874611 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbcc6a2860 con 0x55fbcc693760 2024-09-19T04:52:20.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.851+0000 7f65ce800640 1 --2- 172.21.15.123:0/944874611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc692e40 0x55fbcc6a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:52:20.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.851+0000 7f65cd400640 1 --2- 172.21.15.123:0/944874611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc693760 0x55fbcc6a1ad0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f65bc02aa90 tx=0x7f65bc0051a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d33c8b730adf9168 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:20.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.851+0000 7f65afe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:52:20.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.851+0000 7f65afe00640 1 -- 172.21.15.123:0/944874611 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc035040 con 0x55fbcc693760 2024-09-19T04:52:20.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.851+0000 7f65afe00640 1 -- 172.21.15.123:0/944874611 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f65bc004760 con 0x55fbcc693760 2024-09-19T04:52:20.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.851+0000 7f65d79c0740 1 -- 172.21.15.123:0/944874611 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c8023030 con 0x55fbcc693760 2024-09-19T04:52:20.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.851+0000 7f65afe00640 1 -- 172.21.15.123:0/944874611 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f65bc053050 con 0x55fbcc693760 2024-09-19T04:52:20.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.851+0000 7f65d79c0740 10 client.26327 fetch_fsmap learned FSMap version 36 2024-09-19T04:52:20.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.851+0000 7f65d79c0740 1 -- 172.21.15.123:0/944874611 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fbcc718b80 con 0x55fbcc693760 2024-09-19T04:52:20.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.852+0000 7f65afe00640 1 -- 172.21.15.123:0/944874611 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c8023030 con 0x55fbcc693760 2024-09-19T04:52:20.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.852+0000 7f65afe00640 1 -- 172.21.15.123:0/944874611 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f65bc031b50 con 0x55fbcc693760 2024-09-19T04:52:20.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.852+0000 7f65d79c0740 10 client.26327 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:52:20.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.852+0000 7f65d79c0740 10 client.26327 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:52:20.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.852+0000 7f65d79c0740 20 client.26327 populate_metadata read hostname 'smithi123' 2024-09-19T04:52:20.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.852+0000 7f65d79c0740 1 --2- 172.21.15.123:0/944874611 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55fbcc719350 0x55fbcc71b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:20.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.852+0000 7f65d79c0740 4 client.26327 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:52:20.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.852+0000 7f65d79c0740 1 -- 172.21.15.123:0/944874611 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fbcc600e60 con 0x55fbcc719350 2024-09-19T04:52:20.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.852+0000 7f65ce800640 1 --2- 172.21.15.123:0/944874611 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55fbcc719350 0x55fbcc71b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:20.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.853+0000 7f65ce800640 1 --2- 172.21.15.123:0/944874611 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55fbcc719350 0x55fbcc71b770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5876536b1ee580ea server_cookie=c2b3c813475ce6b2 in_seq=0 out_seq=0 2024-09-19T04:52:20.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.853+0000 7f65afe00640 10 client.26327 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:52:20.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.853+0000 7f65afe00640 1 -- 172.21.15.123:0/944874611 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fbcc600e60 con 0x55fbcc719350 2024-09-19T04:52:20.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.853+0000 7f65afe00640 10 client.26327 handle_command_reply: tid=0 2024-09-19T04:52:20.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.853+0000 7f65d79c0740 1 client.26327 shutdown 2024-09-19T04:52:20.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.853+0000 7f65d79c0740 1 -- 172.21.15.123:0/944874611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc693760 msgr2=0x55fbcc6a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:20.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.853+0000 7f65d79c0740 1 --2- 172.21.15.123:0/944874611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc693760 0x55fbcc6a1ad0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f65bc02aa90 tx=0x7f65bc0051a0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.853+0000 7f65d79c0740 1 -- 172.21.15.123:0/944874611 shutdown_connections 2024-09-19T04:52:20.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.854+0000 7f65d79c0740 1 --2- 172.21.15.123:0/944874611 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55fbcc719350 0x55fbcc71b770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.854+0000 7f65d79c0740 1 --2- 172.21.15.123:0/944874611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fbcc69e020 0x55fbcc6a2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.854+0000 7f65d79c0740 1 --2- 172.21.15.123:0/944874611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fbcc693760 0x55fbcc6a1ad0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.854+0000 7f65d79c0740 1 --2- 172.21.15.123:0/944874611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fbcc692e40 0x55fbcc6a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.854+0000 7f65afe00640 0 client.26327 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:52:20.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.854+0000 7f65d79c0740 1 -- 172.21.15.123:0/944874611 >> 172.21.15.123:0/944874611 conn(0x55fbcc68f3d0 msgr2=0x55fbcc68e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:20.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.854+0000 7f65d79c0740 1 -- 172.21.15.123:0/944874611 shutdown_connections 2024-09-19T04:52:20.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.854+0000 7f65d79c0740 1 -- 172.21.15.123:0/944874611 wait complete. 2024-09-19T04:52:20.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.854+0000 7f65d79c0740 20 client.26327 trim_cache size 0 max 16384 2024-09-19T04:52:20.861 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:52:20.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.855+0000 7f65cf200640 1 -- 172.21.15.123:0/3529000902 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f65b40a3430 msgr2=0x7f65b40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:20.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.855+0000 7f65cf200640 1 --2- 172.21.15.123:0/3529000902 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f65b40a3430 0x7f65b40a58f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f65c00cd790 tx=0x7f65c00d3ab0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.855+0000 7f65cf200640 1 -- 172.21.15.123:0/3529000902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65d0074590 msgr2=0x7f65d01a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:20.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.855+0000 7f65cf200640 1 --2- 172.21.15.123:0/3529000902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65d0074590 0x7f65d01a4620 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f65c000e9f0 tx=0x7f65c000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.855+0000 7f65cf200640 1 -- 172.21.15.123:0/3529000902 shutdown_connections 2024-09-19T04:52:20.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.855+0000 7f65cf200640 1 --2- 172.21.15.123:0/3529000902 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f65b40a3430 0x7f65b40a58f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.855+0000 7f65cf200640 1 --2- 172.21.15.123:0/3529000902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d01129e0 0x7f65d019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.855+0000 7f65cf200640 1 --2- 172.21.15.123:0/3529000902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d0074eb0 0x7f65d01a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.855+0000 7f65cf200640 1 --2- 172.21.15.123:0/3529000902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65d0074590 0x7f65d01a4620 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:20.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.855+0000 7f65cf200640 1 -- 172.21.15.123:0/3529000902 >> 172.21.15.123:0/3529000902 conn(0x7f65d00fd710 msgr2=0x7f65d00fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:20.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.855+0000 7f65cf200640 1 -- 172.21.15.123:0/3529000902 shutdown_connections 2024-09-19T04:52:20.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:20.855+0000 7f65cf200640 1 -- 172.21.15.123:0/3529000902 wait complete. 2024-09-19T04:52:21.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:21 smithi193 ceph-mon[35592]: pgmap v172: 97 pgs: 97 active+clean; 190 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-09-19T04:52:21.553 INFO:tasks.workunit.client.0.smithi123.stdout:All 50000 operations completed A-OK! 2024-09-19T04:52:21.586 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T04:52:21.586 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T04:52:21.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:21 smithi183 ceph-mon[35333]: pgmap v172: 97 pgs: 97 active+clean; 190 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-09-19T04:52:21.635 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:21 smithi123 ceph-mon[30511]: pgmap v172: 97 pgs: 97 active+clean; 190 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-09-19T04:52:22.043 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:52:22.435 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:52:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.012+0000 7f5debe00640 1 -- 172.21.15.123:0/843490393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dec1079c0 msgr2=0x7f5dec107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.012+0000 7f5debe00640 1 --2- 172.21.15.123:0/843490393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dec1079c0 0x7f5dec107da0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009930 tx=0x7f5ddc02a440 comp rx=0 tx=0).stop 2024-09-19T04:52:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.012+0000 7f5debe00640 1 -- 172.21.15.123:0/843490393 shutdown_connections 2024-09-19T04:52:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.012+0000 7f5debe00640 1 --2- 172.21.15.123:0/843490393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec102360 0x7f5dec10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.012+0000 7f5debe00640 1 --2- 172.21.15.123:0/843490393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dec1019c0 0x7f5dec101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.012+0000 7f5debe00640 1 --2- 172.21.15.123:0/843490393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dec1079c0 0x7f5dec107da0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.012+0000 7f5debe00640 1 -- 172.21.15.123:0/843490393 >> 172.21.15.123:0/843490393 conn(0x7f5dec0fd740 msgr2=0x7f5dec0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.013+0000 7f5debe00640 1 -- 172.21.15.123:0/843490393 shutdown_connections 2024-09-19T04:52:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.013+0000 7f5debe00640 1 -- 172.21.15.123:0/843490393 wait complete. 2024-09-19T04:52:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.013+0000 7f5debe00640 1 Processor -- start 2024-09-19T04:52:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.014+0000 7f5debe00640 1 -- start start 2024-09-19T04:52:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.014+0000 7f5debe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dec1019c0 0x7f5dec1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.014+0000 7f5debe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dec102360 0x7f5dec198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.014+0000 7f5debe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec1079c0 0x7f5dec19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.014+0000 7f5debe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec111d90 con 0x7f5dec102360 2024-09-19T04:52:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.014+0000 7f5debe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec111c10 con 0x7f5dec1079c0 2024-09-19T04:52:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.014+0000 7f5debe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec111f10 con 0x7f5dec1019c0 2024-09-19T04:52:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.014+0000 7f5deaa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dec1019c0 0x7f5dec1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.014+0000 7f5deaa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dec1019c0 0x7f5dec1982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34456/0 (socket says 172.21.15.123:34456) 2024-09-19T04:52:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.014+0000 7f5deaa00640 1 -- 172.21.15.123:0/892440258 learned_addr learned my addr 172.21.15.123:0/892440258 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.014+0000 7f5dea000640 1 --2- 172.21.15.123:0/892440258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dec102360 0x7f5dec198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.014+0000 7f5deb400640 1 --2- 172.21.15.123:0/892440258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec1079c0 0x7f5dec19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.015+0000 7f5deaa00640 1 -- 172.21.15.123:0/892440258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec1079c0 msgr2=0x7f5dec19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.015+0000 7f5deaa00640 1 --2- 172.21.15.123:0/892440258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec1079c0 0x7f5dec19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.015+0000 7f5deaa00640 1 -- 172.21.15.123:0/892440258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dec102360 msgr2=0x7f5dec198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.015+0000 7f5deaa00640 1 --2- 172.21.15.123:0/892440258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dec102360 0x7f5dec198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.015+0000 7f5deaa00640 1 -- 172.21.15.123:0/892440258 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec19d2a0 con 0x7f5dec1019c0 2024-09-19T04:52:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.015+0000 7f5deaa00640 1 --2- 172.21.15.123:0/892440258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dec1019c0 0x7f5dec1982f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009230 tx=0x7f5ddc004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.015+0000 7f5de3400640 1 -- 172.21.15.123:0/892440258 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc03f030 con 0x7f5dec1019c0 2024-09-19T04:52:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.015+0000 7f5de3400640 1 -- 172.21.15.123:0/892440258 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ddc004440 con 0x7f5dec1019c0 2024-09-19T04:52:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.015+0000 7f5debe00640 1 -- 172.21.15.123:0/892440258 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec19d530 con 0x7f5dec1019c0 2024-09-19T04:52:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.015+0000 7f5de3400640 1 -- 172.21.15.123:0/892440258 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc035070 con 0x7f5dec1019c0 2024-09-19T04:52:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.015+0000 7f5debe00640 1 -- 172.21.15.123:0/892440258 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec1a4e70 con 0x7f5dec1019c0 2024-09-19T04:52:23.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.018+0000 7f5de3400640 1 -- 172.21.15.123:0/892440258 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5ddc042030 con 0x7f5dec1019c0 2024-09-19T04:52:23.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.018+0000 7f5df462a740 1 Processor -- start 2024-09-19T04:52:23.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.019+0000 7f5df462a740 1 -- start start 2024-09-19T04:52:23.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.019+0000 7f5df462a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6ba1e40 0x55e4e6ba2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.019+0000 7f5df462a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 0x55e4e6bacae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.019+0000 7f5df462a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6bad020 0x55e4e6baf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.019+0000 7f5df462a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e4e6b083e0 con 0x55e4e6ba1e40 2024-09-19T04:52:23.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.019+0000 7f5df462a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e4e6aafd80 con 0x55e4e6bad020 2024-09-19T04:52:23.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.019+0000 7f5df462a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e4e6b043a0 con 0x55e4e6ba2760 2024-09-19T04:52:23.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.019+0000 7f5deaa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 0x55e4e6bacae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.019+0000 7f5deaa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 0x55e4e6bacae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34466/0 (socket says 172.21.15.123:34466) 2024-09-19T04:52:23.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.019+0000 7f5deaa00640 1 -- 172.21.15.123:0/1484885062 learned_addr learned my addr 172.21.15.123:0/1484885062 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:23.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.019+0000 7f5deb400640 1 --2- 172.21.15.123:0/1484885062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6ba1e40 0x55e4e6ba2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.020+0000 7f5dea000640 1 --2- 172.21.15.123:0/1484885062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6bad020 0x55e4e6baf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.020+0000 7f5deaa00640 1 -- 172.21.15.123:0/1484885062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6bad020 msgr2=0x55e4e6baf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.020+0000 7f5deaa00640 1 --2- 172.21.15.123:0/1484885062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6bad020 0x55e4e6baf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.020+0000 7f5deaa00640 1 -- 172.21.15.123:0/1484885062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6ba1e40 msgr2=0x55e4e6ba2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.020+0000 7f5deaa00640 1 --2- 172.21.15.123:0/1484885062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6ba1e40 0x55e4e6ba2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.020+0000 7f5deaa00640 1 -- 172.21.15.123:0/1484885062 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4e6aceae0 con 0x55e4e6ba2760 2024-09-19T04:52:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.020+0000 7f5de3400640 1 --2- 172.21.15.123:0/892440258 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5dd00a3430 0x7f5dd00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.020+0000 7f5deaa00640 1 --2- 172.21.15.123:0/1484885062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 0x55e4e6bacae0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0ed5f0 tx=0x7f5ddc0f8220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af51d7c20f13651a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.020+0000 7f5de3400640 1 -- 172.21.15.123:0/892440258 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f5ddc0eff90 con 0x7f5dec1019c0 2024-09-19T04:52:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.020+0000 7f5deb400640 1 --2- 172.21.15.123:0/892440258 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5dd00a3430 0x7f5dd00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.020+0000 7f5dd7e00640 1 -- 172.21.15.123:0/1484885062 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc0b6e60 con 0x55e4e6ba2760 2024-09-19T04:52:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.021+0000 7f5dd7e00640 1 -- 172.21.15.123:0/1484885062 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ddc0c05f0 con 0x55e4e6ba2760 2024-09-19T04:52:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.021+0000 7f5dd7e00640 1 -- 172.21.15.123:0/1484885062 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc0bbc50 con 0x55e4e6ba2760 2024-09-19T04:52:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.021+0000 7f5df462a740 1 -- 172.21.15.123:0/1484885062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 msgr2=0x55e4e6bacae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:23.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.021+0000 7f5df462a740 1 --2- 172.21.15.123:0/1484885062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 0x55e4e6bacae0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0ed5f0 tx=0x7f5ddc0f8220 comp rx=0 tx=0).stop 2024-09-19T04:52:23.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.021+0000 7f5deb400640 1 --2- 172.21.15.123:0/892440258 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5dd00a3430 0x7f5dd00a58f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5de4005fd0 tx=0x7f5de4009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:23.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.021+0000 7f5df462a740 1 -- 172.21.15.123:0/1484885062 shutdown_connections 2024-09-19T04:52:23.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.021+0000 7f5df462a740 1 --2- 172.21.15.123:0/1484885062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6bad020 0x55e4e6baf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.021+0000 7f5df462a740 1 --2- 172.21.15.123:0/1484885062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 0x55e4e6bacae0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.021+0000 7f5df462a740 1 --2- 172.21.15.123:0/1484885062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6ba1e40 0x55e4e6ba2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.021+0000 7f5df462a740 1 -- 172.21.15.123:0/1484885062 >> 172.21.15.123:0/1484885062 conn(0x55e4e6b9e3d0 msgr2=0x55e4e6b9e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:23.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.021+0000 7f5df462a740 1 -- 172.21.15.123:0/1484885062 shutdown_connections 2024-09-19T04:52:23.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.022+0000 7f5df462a740 1 -- 172.21.15.123:0/1484885062 wait complete. 2024-09-19T04:52:23.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.022+0000 7f5df462a740 1 -- start start 2024-09-19T04:52:23.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.022+0000 7f5df462a740 1 Processor -- start 2024-09-19T04:52:23.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.023+0000 7f5df462a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6ba1e40 0x55e4e6ba9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.023+0000 7f5df462a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 0x55e4e6baa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.023+0000 7f5df462a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6bad020 0x55e4e6baa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.023+0000 7f5df462a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e4e6baf9b0 con 0x55e4e6bad020 2024-09-19T04:52:23.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.023+0000 7f5df462a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e4e6ac30f0 con 0x55e4e6ba1e40 2024-09-19T04:52:23.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.023+0000 7f5df462a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e4e6bafb30 con 0x55e4e6ba2760 2024-09-19T04:52:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.023+0000 7f5df462a740 10 client.0 _ll_register_callbacks cb 0x55e4e6aaabd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:52:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.023+0000 7f5dea000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 0x55e4e6baa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.023+0000 7f5deb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6bad020 0x55e4e6baa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.023+0000 7f5dea000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 0x55e4e6baa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34474/0 (socket says 172.21.15.123:34474) 2024-09-19T04:52:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.023+0000 7f5dea000640 1 -- 172.21.15.123:0/4075057002 learned_addr learned my addr 172.21.15.123:0/4075057002 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.023+0000 7f5deaa00640 1 --2- 172.21.15.123:0/4075057002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6ba1e40 0x55e4e6ba9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.024+0000 7f5deaa00640 1 -- 172.21.15.123:0/4075057002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 msgr2=0x55e4e6baa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.024+0000 7f5deaa00640 1 --2- 172.21.15.123:0/4075057002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 0x55e4e6baa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.024+0000 7f5deaa00640 1 -- 172.21.15.123:0/4075057002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6bad020 msgr2=0x55e4e6baa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.024+0000 7f5deaa00640 1 --2- 172.21.15.123:0/4075057002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6bad020 0x55e4e6baa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.024+0000 7f5deaa00640 1 -- 172.21.15.123:0/4075057002 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4e68793c0 con 0x55e4e6ba1e40 2024-09-19T04:52:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.024+0000 7f5deb400640 1 --2- 172.21.15.123:0/4075057002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6bad020 0x55e4e6baa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:52:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.024+0000 7f5deaa00640 1 --2- 172.21.15.123:0/4075057002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6ba1e40 0x55e4e6ba9e40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0ed5f0 tx=0x7f5ddc0b8b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b67962d0dcc682 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.024+0000 7f5dd4c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:52:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.024+0000 7f5dd4c00640 1 -- 172.21.15.123:0/4075057002 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc049070 con 0x55e4e6ba1e40 2024-09-19T04:52:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.024+0000 7f5dd4c00640 1 -- 172.21.15.123:0/4075057002 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f5ddc0ba5f0 con 0x55e4e6ba1e40 2024-09-19T04:52:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.024+0000 7f5df462a740 1 -- 172.21.15.123:0/4075057002 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ddc0c05f0 con 0x55e4e6ba1e40 2024-09-19T04:52:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.024+0000 7f5dd4c00640 1 -- 172.21.15.123:0/4075057002 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5ddc0b6e60 con 0x55e4e6ba1e40 2024-09-19T04:52:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.024+0000 7f5df462a740 10 client.26339 fetch_fsmap learned FSMap version 36 2024-09-19T04:52:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.024+0000 7f5df462a740 1 -- 172.21.15.123:0/4075057002 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e4e6baf9b0 con 0x55e4e6ba1e40 2024-09-19T04:52:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.025+0000 7f5dd4c00640 1 -- 172.21.15.123:0/4075057002 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ddc0c05f0 con 0x55e4e6ba1e40 2024-09-19T04:52:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.025+0000 7f5dd4c00640 1 -- 172.21.15.123:0/4075057002 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f5ddc0dc0c0 con 0x55e4e6ba1e40 2024-09-19T04:52:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.025+0000 7f5df462a740 10 client.26339 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:52:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.025+0000 7f5df462a740 10 client.26339 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:52:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.025+0000 7f5df462a740 20 client.26339 populate_metadata read hostname 'smithi123' 2024-09-19T04:52:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.025+0000 7f5df462a740 1 --2- 172.21.15.123:0/4075057002 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55e4e6bbcb20 0x55e4e6bbef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.025+0000 7f5df462a740 4 client.26339 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:52:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.025+0000 7f5df462a740 1 -- 172.21.15.123:0/4075057002 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4e6b15950 con 0x55e4e6bbcb20 2024-09-19T04:52:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.025+0000 7f5dea000640 1 --2- 172.21.15.123:0/4075057002 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55e4e6bbcb20 0x55e4e6bbef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.026+0000 7f5dea000640 1 --2- 172.21.15.123:0/4075057002 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55e4e6bbcb20 0x55e4e6bbef40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6644f1d0d7ad48ea server_cookie=e9ce8e8948a91090 in_seq=0 out_seq=0 2024-09-19T04:52:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.026+0000 7f5dd4c00640 10 client.26339 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:52:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.026+0000 7f5dd4c00640 1 -- 172.21.15.123:0/4075057002 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4e6b15950 con 0x55e4e6bbcb20 2024-09-19T04:52:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.026+0000 7f5dd4c00640 10 client.26339 handle_command_reply: tid=0 2024-09-19T04:52:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.027+0000 7f5df462a740 1 client.26339 shutdown 2024-09-19T04:52:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.027+0000 7f5df462a740 1 -- 172.21.15.123:0/4075057002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6ba1e40 msgr2=0x55e4e6ba9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.027+0000 7f5df462a740 1 --2- 172.21.15.123:0/4075057002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6ba1e40 0x55e4e6ba9e40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0ed5f0 tx=0x7f5ddc0b8b90 comp rx=0 tx=0).stop 2024-09-19T04:52:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.027+0000 7f5df462a740 1 -- 172.21.15.123:0/4075057002 shutdown_connections 2024-09-19T04:52:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.027+0000 7f5df462a740 1 --2- 172.21.15.123:0/4075057002 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55e4e6bbcb20 0x55e4e6bbef40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.027+0000 7f5df462a740 1 --2- 172.21.15.123:0/4075057002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6bad020 0x55e4e6baa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.027+0000 7f5df462a740 1 --2- 172.21.15.123:0/4075057002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 0x55e4e6baa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.027+0000 7f5df462a740 1 --2- 172.21.15.123:0/4075057002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6ba1e40 0x55e4e6ba9e40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.027+0000 7f5dd4c00640 0 client.26339 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:52:23.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.027+0000 7f5df462a740 1 -- 172.21.15.123:0/4075057002 >> 172.21.15.123:0/4075057002 conn(0x55e4e6b9e3d0 msgr2=0x55e4e6ba4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:23.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.027+0000 7f5df462a740 1 -- 172.21.15.123:0/4075057002 shutdown_connections 2024-09-19T04:52:23.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.027+0000 7f5df462a740 1 -- 172.21.15.123:0/4075057002 wait complete. 2024-09-19T04:52:23.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.027+0000 7f5df462a740 20 client.26339 trim_cache size 0 max 16384 2024-09-19T04:52:23.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.041+0000 7f5df462a740 1 Processor -- start 2024-09-19T04:52:23.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.041+0000 7f5df462a740 1 -- start start 2024-09-19T04:52:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.041+0000 7f5df462a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba1e40 0x55e4e6bc4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.041+0000 7f5df462a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6ba2760 0x55e4e6bc4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.041+0000 7f5df462a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6bad020 0x55e4e6ba9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.041+0000 7f5df462a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1ad6d0 con 0x55e4e6bad020 2024-09-19T04:52:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.041+0000 7f5df462a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1af210 con 0x55e4e6ba2760 2024-09-19T04:52:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.041+0000 7f5df462a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1adda0 con 0x55e4e6ba1e40 2024-09-19T04:52:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.041+0000 7f5deaa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba1e40 0x55e4e6bc4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.042+0000 7f5deaa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba1e40 0x55e4e6bc4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34488/0 (socket says 172.21.15.123:34488) 2024-09-19T04:52:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.042+0000 7f5deaa00640 1 -- 172.21.15.123:0/1405216306 learned_addr learned my addr 172.21.15.123:0/1405216306 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.042+0000 7f5deb400640 1 --2- 172.21.15.123:0/1405216306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6bad020 0x55e4e6ba9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.042+0000 7f5dea000640 1 --2- 172.21.15.123:0/1405216306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6ba2760 0x55e4e6bc4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.042+0000 7f5deaa00640 1 -- 172.21.15.123:0/1405216306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6ba2760 msgr2=0x55e4e6bc4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:23.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.042+0000 7f5deaa00640 1 --2- 172.21.15.123:0/1405216306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6ba2760 0x55e4e6bc4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.042+0000 7f5deaa00640 1 -- 172.21.15.123:0/1405216306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6bad020 msgr2=0x55e4e6ba9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:23.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.042+0000 7f5deaa00640 1 --2- 172.21.15.123:0/1405216306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6bad020 0x55e4e6ba9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.042+0000 7f5deaa00640 1 -- 172.21.15.123:0/1405216306 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4e6ba4cb0 con 0x55e4e6ba1e40 2024-09-19T04:52:23.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.042+0000 7f5deaa00640 1 --2- 172.21.15.123:0/1405216306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba1e40 0x55e4e6bc4810 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0ba5a0 tx=0x7f5ddc0bbf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fb769e7e638b406 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:23.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.042+0000 7f5dd7400640 1 -- 172.21.15.123:0/1405216306 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc049070 con 0x55e4e6ba1e40 2024-09-19T04:52:23.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.042+0000 7f5dd7400640 1 -- 172.21.15.123:0/1405216306 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ddc0d5230 con 0x55e4e6ba1e40 2024-09-19T04:52:23.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.042+0000 7f5dd7400640 1 -- 172.21.15.123:0/1405216306 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc0f3c70 con 0x55e4e6ba1e40 2024-09-19T04:52:23.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.043+0000 7f5df462a740 1 -- 172.21.15.123:0/1405216306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba1e40 msgr2=0x55e4e6bc4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:23.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.043+0000 7f5df462a740 1 --2- 172.21.15.123:0/1405216306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba1e40 0x55e4e6bc4810 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0ba5a0 tx=0x7f5ddc0bbf60 comp rx=0 tx=0).stop 2024-09-19T04:52:23.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.043+0000 7f5df462a740 1 -- 172.21.15.123:0/1405216306 shutdown_connections 2024-09-19T04:52:23.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.043+0000 7f5df462a740 1 --2- 172.21.15.123:0/1405216306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6bad020 0x55e4e6ba9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.043+0000 7f5df462a740 1 --2- 172.21.15.123:0/1405216306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6ba2760 0x55e4e6bc4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.043+0000 7f5df462a740 1 --2- 172.21.15.123:0/1405216306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba1e40 0x55e4e6bc4810 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.043+0000 7f5df462a740 1 -- 172.21.15.123:0/1405216306 >> 172.21.15.123:0/1405216306 conn(0x55e4e6b9e3d0 msgr2=0x55e4e6babec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:23.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.043+0000 7f5df462a740 1 -- 172.21.15.123:0/1405216306 shutdown_connections 2024-09-19T04:52:23.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.043+0000 7f5df462a740 1 -- 172.21.15.123:0/1405216306 wait complete. 2024-09-19T04:52:23.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.044+0000 7f5df462a740 1 -- start start 2024-09-19T04:52:23.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.044+0000 7f5df462a740 1 Processor -- start 2024-09-19T04:52:23.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.044+0000 7f5df462a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6ba1e40 0x55e4e6bb5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.044+0000 7f5df462a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 0x55e4e6bb5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.044+0000 7f5df462a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6bad020 0x55e4e6ba6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.044+0000 7f5df462a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1aeb40 con 0x55e4e6ba1e40 2024-09-19T04:52:23.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.044+0000 7f5df462a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1ae470 con 0x55e4e6bad020 2024-09-19T04:52:23.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.044+0000 7f5df462a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1ab6b0 con 0x55e4e6ba2760 2024-09-19T04:52:23.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.044+0000 7f5df462a740 10 client.0 _ll_register_callbacks cb 0x7f5de4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:52:23.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.045+0000 7f5dea000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 0x55e4e6bb5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.045+0000 7f5deaa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6ba1e40 0x55e4e6bb5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.045+0000 7f5dea000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 0x55e4e6bb5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34502/0 (socket says 172.21.15.123:34502) 2024-09-19T04:52:23.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.045+0000 7f5dea000640 1 -- 172.21.15.123:0/1821917632 learned_addr learned my addr 172.21.15.123:0/1821917632 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:23.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.045+0000 7f5deb400640 1 --2- 172.21.15.123:0/1821917632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6bad020 0x55e4e6ba6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.045+0000 7f5dea000640 1 -- 172.21.15.123:0/1821917632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6bad020 msgr2=0x55e4e6ba6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:23.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.045+0000 7f5dea000640 1 --2- 172.21.15.123:0/1821917632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6bad020 0x55e4e6ba6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.045+0000 7f5dea000640 1 -- 172.21.15.123:0/1821917632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6ba1e40 msgr2=0x55e4e6bb5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:23.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.045+0000 7f5dea000640 1 --2- 172.21.15.123:0/1821917632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6ba1e40 0x55e4e6bb5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.045+0000 7f5dea000640 1 -- 172.21.15.123:0/1821917632 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4e6b9dc80 con 0x55e4e6ba2760 2024-09-19T04:52:23.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.045+0000 7f5deaa00640 1 --2- 172.21.15.123:0/1821917632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6ba1e40 0x55e4e6bb5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:52:23.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.045+0000 7f5dea000640 1 --2- 172.21.15.123:0/1821917632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 0x55e4e6bb5570 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009e50 tx=0x7f5dd8010b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f32a6546fc3c0c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:23.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.045+0000 7f5dd4c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:52:23.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.045+0000 7f5dd4c00640 1 -- 172.21.15.123:0/1821917632 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8004230 con 0x55e4e6ba2760 2024-09-19T04:52:23.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.046+0000 7f5df462a740 1 -- 172.21.15.123:0/1821917632 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ddc0d5230 con 0x55e4e6ba2760 2024-09-19T04:52:23.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.046+0000 7f5dd4c00640 1 -- 172.21.15.123:0/1821917632 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f5dd80043d0 con 0x55e4e6ba2760 2024-09-19T04:52:23.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.046+0000 7f5dd4c00640 1 -- 172.21.15.123:0/1821917632 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5dd8005020 con 0x55e4e6ba2760 2024-09-19T04:52:23.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.046+0000 7f5df462a740 10 client.26362 fetch_fsmap learned FSMap version 36 2024-09-19T04:52:23.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.046+0000 7f5df462a740 1 -- 172.21.15.123:0/1821917632 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e4e6c2b740 con 0x55e4e6ba2760 2024-09-19T04:52:23.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.046+0000 7f5dd4c00640 1 -- 172.21.15.123:0/1821917632 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ddc0d5230 con 0x55e4e6ba2760 2024-09-19T04:52:23.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.046+0000 7f5dd4c00640 1 -- 172.21.15.123:0/1821917632 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f5dd8025030 con 0x55e4e6ba2760 2024-09-19T04:52:23.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.046+0000 7f5df462a740 10 client.26362 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:52:23.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.046+0000 7f5df462a740 10 client.26362 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:52:23.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.046+0000 7f5df462a740 20 client.26362 populate_metadata read hostname 'smithi123' 2024-09-19T04:52:23.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.046+0000 7f5df462a740 1 --2- 172.21.15.123:0/1821917632 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55e4e6c2be50 0x55e4e6c2e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:23.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.046+0000 7f5df462a740 4 client.26362 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:52:23.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.046+0000 7f5df462a740 1 -- 172.21.15.123:0/1821917632 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4e6b0fe60 con 0x55e4e6c2be50 2024-09-19T04:52:23.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.047+0000 7f5deaa00640 1 --2- 172.21.15.123:0/1821917632 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55e4e6c2be50 0x55e4e6c2e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:23.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.047+0000 7f5deaa00640 1 --2- 172.21.15.123:0/1821917632 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55e4e6c2be50 0x55e4e6c2e270 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88ee43ab0182686e server_cookie=806680bc1183b998 in_seq=0 out_seq=0 2024-09-19T04:52:23.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.047+0000 7f5dd4c00640 10 client.26362 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:52:23.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.047+0000 7f5dd4c00640 1 -- 172.21.15.123:0/1821917632 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4e6b0fe60 con 0x55e4e6c2be50 2024-09-19T04:52:23.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.047+0000 7f5dd4c00640 10 client.26362 handle_command_reply: tid=0 2024-09-19T04:52:23.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.047+0000 7f5df462a740 1 client.26362 shutdown 2024-09-19T04:52:23.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.047+0000 7f5df462a740 1 -- 172.21.15.123:0/1821917632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 msgr2=0x55e4e6bb5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:23.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.047+0000 7f5df462a740 1 --2- 172.21.15.123:0/1821917632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 0x55e4e6bb5570 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009e50 tx=0x7f5dd8010b10 comp rx=0 tx=0).stop 2024-09-19T04:52:23.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.048+0000 7f5df462a740 1 -- 172.21.15.123:0/1821917632 shutdown_connections 2024-09-19T04:52:23.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.048+0000 7f5df462a740 1 --2- 172.21.15.123:0/1821917632 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x55e4e6c2be50 0x55e4e6c2e270 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.048+0000 7f5df462a740 1 --2- 172.21.15.123:0/1821917632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e6bad020 0x55e4e6ba6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.048+0000 7f5df462a740 1 --2- 172.21.15.123:0/1821917632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4e6ba2760 0x55e4e6bb5570 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.048+0000 7f5dd4c00640 0 client.26362 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:52:23.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.048+0000 7f5df462a740 1 --2- 172.21.15.123:0/1821917632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4e6ba1e40 0x55e4e6bb5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.048+0000 7f5df462a740 1 -- 172.21.15.123:0/1821917632 >> 172.21.15.123:0/1821917632 conn(0x55e4e6b9e3d0 msgr2=0x55e4e6ba85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:23.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.048+0000 7f5df462a740 1 -- 172.21.15.123:0/1821917632 shutdown_connections 2024-09-19T04:52:23.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.048+0000 7f5df462a740 1 -- 172.21.15.123:0/1821917632 wait complete. 2024-09-19T04:52:23.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.048+0000 7f5df462a740 20 client.26362 trim_cache size 0 max 16384 2024-09-19T04:52:23.054 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:52:23.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.048+0000 7f5debe00640 1 -- 172.21.15.123:0/892440258 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5dd00a3430 msgr2=0x7f5dd00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:23.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.048+0000 7f5debe00640 1 --2- 172.21.15.123:0/892440258 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5dd00a3430 0x7f5dd00a58f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5de4005fd0 tx=0x7f5de4009500 comp rx=0 tx=0).stop 2024-09-19T04:52:23.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.048+0000 7f5debe00640 1 -- 172.21.15.123:0/892440258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dec1019c0 msgr2=0x7f5dec1982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:23.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.048+0000 7f5debe00640 1 --2- 172.21.15.123:0/892440258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dec1019c0 0x7f5dec1982f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009230 tx=0x7f5ddc004040 comp rx=0 tx=0).stop 2024-09-19T04:52:23.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.049+0000 7f5debe00640 1 -- 172.21.15.123:0/892440258 shutdown_connections 2024-09-19T04:52:23.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.049+0000 7f5debe00640 1 --2- 172.21.15.123:0/892440258 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5dd00a3430 0x7f5dd00a58f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.049+0000 7f5debe00640 1 --2- 172.21.15.123:0/892440258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec1079c0 0x7f5dec19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.049+0000 7f5debe00640 1 --2- 172.21.15.123:0/892440258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dec102360 0x7f5dec198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.049+0000 7f5debe00640 1 --2- 172.21.15.123:0/892440258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dec1019c0 0x7f5dec1982f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:23.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.049+0000 7f5debe00640 1 -- 172.21.15.123:0/892440258 >> 172.21.15.123:0/892440258 conn(0x7f5dec0fd740 msgr2=0x7f5dec0fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:23.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.049+0000 7f5debe00640 1 -- 172.21.15.123:0/892440258 shutdown_connections 2024-09-19T04:52:23.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:23.049+0000 7f5debe00640 1 -- 172.21.15.123:0/892440258 wait complete. 2024-09-19T04:52:23.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:23 smithi193 ceph-mon[35592]: pgmap v173: 97 pgs: 97 active+clean; 190 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-09-19T04:52:23.562 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:23 smithi123 ceph-mon[30511]: pgmap v173: 97 pgs: 97 active+clean; 190 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-09-19T04:52:23.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:23 smithi183 ceph-mon[35333]: pgmap v173: 97 pgs: 97 active+clean; 190 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-09-19T04:52:23.962 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:52:24.340 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:52:24.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.965+0000 7f1e3de00640 1 -- 172.21.15.123:0/2441333149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e38102790 msgr2=0x7f1e381067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:24.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.965+0000 7f1e3de00640 1 --2- 172.21.15.123:0/2441333149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e38102790 0x7f1e381067e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1e24009900 tx=0x7f1e2402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:52:24.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.966+0000 7f1e3de00640 1 -- 172.21.15.123:0/2441333149 shutdown_connections 2024-09-19T04:52:24.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.966+0000 7f1e3de00640 1 --2- 172.21.15.123:0/2441333149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e38106f10 0x7f1e3810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:24.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.966+0000 7f1e3de00640 1 --2- 172.21.15.123:0/2441333149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e38102790 0x7f1e381067e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:24.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.966+0000 7f1e3de00640 1 --2- 172.21.15.123:0/2441333149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e38101de0 0x7f1e381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:24.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.966+0000 7f1e3de00640 1 -- 172.21.15.123:0/2441333149 >> 172.21.15.123:0/2441333149 conn(0x7f1e380fd8c0 msgr2=0x7f1e380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:24.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.966+0000 7f1e3de00640 1 -- 172.21.15.123:0/2441333149 shutdown_connections 2024-09-19T04:52:24.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.966+0000 7f1e3de00640 1 -- 172.21.15.123:0/2441333149 wait complete. 2024-09-19T04:52:24.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.967+0000 7f1e3de00640 1 Processor -- start 2024-09-19T04:52:24.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.967+0000 7f1e3de00640 1 -- start start 2024-09-19T04:52:24.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.967+0000 7f1e3de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e38101de0 0x7f1e3819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:24.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.967+0000 7f1e3de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e38102790 0x7f1e3819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:24.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.967+0000 7f1e3de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e38106f10 0x7f1e3819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:24.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.967+0000 7f1e3de00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1e38111fc0 con 0x7f1e38106f10 2024-09-19T04:52:24.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.967+0000 7f1e3de00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e38111e40 con 0x7f1e38102790 2024-09-19T04:52:24.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.967+0000 7f1e3de00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1e38112140 con 0x7f1e38101de0 2024-09-19T04:52:24.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.967+0000 7f1e36a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e38101de0 0x7f1e3819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:24.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.967+0000 7f1e36a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e38101de0 0x7f1e3819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34514/0 (socket says 172.21.15.123:34514) 2024-09-19T04:52:24.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.967+0000 7f1e36a00640 1 -- 172.21.15.123:0/2843056860 learned_addr learned my addr 172.21.15.123:0/2843056860 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:24.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.968+0000 7f1e36000640 1 --2- 172.21.15.123:0/2843056860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e38102790 0x7f1e3819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:24.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.968+0000 7f1e37400640 1 --2- 172.21.15.123:0/2843056860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e38106f10 0x7f1e3819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:24.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.968+0000 7f1e36a00640 1 -- 172.21.15.123:0/2843056860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e38102790 msgr2=0x7f1e3819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:24.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.968+0000 7f1e36a00640 1 --2- 172.21.15.123:0/2843056860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e38102790 0x7f1e3819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:24.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.968+0000 7f1e36a00640 1 -- 172.21.15.123:0/2843056860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e38106f10 msgr2=0x7f1e3819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:24.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.968+0000 7f1e36a00640 1 --2- 172.21.15.123:0/2843056860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e38106f10 0x7f1e3819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:24.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.968+0000 7f1e36a00640 1 -- 172.21.15.123:0/2843056860 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e3819f640 con 0x7f1e38101de0 2024-09-19T04:52:24.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.968+0000 7f1e37400640 1 --2- 172.21.15.123:0/2843056860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e38106f10 0x7f1e3819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:52:24.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.968+0000 7f1e36a00640 1 --2- 172.21.15.123:0/2843056860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e38101de0 0x7f1e3819a690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800e9f0 tx=0x7f1e2800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:24.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.968+0000 7f1e3ca00640 1 -- 172.21.15.123:0/2843056860 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2800cd80 con 0x7f1e38101de0 2024-09-19T04:52:24.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.968+0000 7f1e3ca00640 1 -- 172.21.15.123:0/2843056860 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e28010910 con 0x7f1e38101de0 2024-09-19T04:52:24.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.968+0000 7f1e3de00640 1 -- 172.21.15.123:0/2843056860 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e3819f930 con 0x7f1e38101de0 2024-09-19T04:52:24.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.969+0000 7f1e3ca00640 1 -- 172.21.15.123:0/2843056860 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28018990 con 0x7f1e38101de0 2024-09-19T04:52:24.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.969+0000 7f1e3de00640 1 -- 172.21.15.123:0/2843056860 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e381a7210 con 0x7f1e38101de0 2024-09-19T04:52:24.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.972+0000 7f1e408bb740 1 Processor -- start 2024-09-19T04:52:24.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.972+0000 7f1e408bb740 1 -- start start 2024-09-19T04:52:24.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.972+0000 7f1e3ca00640 1 -- 172.21.15.123:0/2843056860 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f1e28018b30 con 0x7f1e38101de0 2024-09-19T04:52:24.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.972+0000 7f1e408bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777465c70 0x560777466050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:24.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.972+0000 7f1e408bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777466590 0x560777470910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:24.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.972+0000 7f1e408bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777470e50 0x560777473240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:24.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.972+0000 7f1e408bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5607773cc3e0 con 0x560777466590 2024-09-19T04:52:24.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.972+0000 7f1e408bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560777373d80 con 0x560777470e50 2024-09-19T04:52:24.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.972+0000 7f1e408bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5607773c83a0 con 0x560777465c70 2024-09-19T04:52:24.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.972+0000 7f1e37400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777465c70 0x560777466050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:24.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.972+0000 7f1e36a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777466590 0x560777470910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:24.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.972+0000 7f1e37400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777465c70 0x560777466050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34520/0 (socket says 172.21.15.123:34520) 2024-09-19T04:52:24.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.972+0000 7f1e37400640 1 -- 172.21.15.123:0/752793792 learned_addr learned my addr 172.21.15.123:0/752793792 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:24.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.973+0000 7f1e36000640 1 --2- 172.21.15.123:0/752793792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777470e50 0x560777473240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:24.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.973+0000 7f1e37400640 1 -- 172.21.15.123:0/752793792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777470e50 msgr2=0x560777473240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:24.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.973+0000 7f1e37400640 1 --2- 172.21.15.123:0/752793792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777470e50 0x560777473240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:24.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.973+0000 7f1e37400640 1 -- 172.21.15.123:0/752793792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777466590 msgr2=0x560777470910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:24.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.973+0000 7f1e37400640 1 --2- 172.21.15.123:0/752793792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777466590 0x560777470910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:24.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.973+0000 7f1e37400640 1 -- 172.21.15.123:0/752793792 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560777392ae0 con 0x560777465c70 2024-09-19T04:52:24.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.973+0000 7f1e36a00640 1 --2- 172.21.15.123:0/752793792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777466590 0x560777470910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:52:24.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.973+0000 7f1e37400640 1 --2- 172.21.15.123:0/752793792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777465c70 0x560777466050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1e30005f40 tx=0x7f1e30010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e894dbd8678b16ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:24.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.973+0000 7f1e23e00640 1 -- 172.21.15.123:0/752793792 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e30011700 con 0x560777465c70 2024-09-19T04:52:24.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.974+0000 7f1e23e00640 1 -- 172.21.15.123:0/752793792 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e30011d80 con 0x560777465c70 2024-09-19T04:52:24.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.974+0000 7f1e23e00640 1 -- 172.21.15.123:0/752793792 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3000e3c0 con 0x560777465c70 2024-09-19T04:52:24.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.974+0000 7f1e3ca00640 1 --2- 172.21.15.123:0/2843056860 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1e1c0a3430 0x7f1e1c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:24.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.974+0000 7f1e3ca00640 1 -- 172.21.15.123:0/2843056860 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f1e280cffc0 con 0x7f1e38101de0 2024-09-19T04:52:24.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.974+0000 7f1e36a00640 1 --2- 172.21.15.123:0/2843056860 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1e1c0a3430 0x7f1e1c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:24.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.975+0000 7f1e408bb740 1 -- 172.21.15.123:0/752793792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777465c70 msgr2=0x560777466050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:24.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.975+0000 7f1e408bb740 1 --2- 172.21.15.123:0/752793792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777465c70 0x560777466050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1e30005f40 tx=0x7f1e30010860 comp rx=0 tx=0).stop 2024-09-19T04:52:24.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.975+0000 7f1e36a00640 1 --2- 172.21.15.123:0/2843056860 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1e1c0a3430 0x7f1e1c0a58f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x560777463060 tx=0x7f1e280d3a70 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:24.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.975+0000 7f1e408bb740 1 -- 172.21.15.123:0/752793792 shutdown_connections 2024-09-19T04:52:24.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.975+0000 7f1e408bb740 1 --2- 172.21.15.123:0/752793792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777470e50 0x560777473240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:24.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.975+0000 7f1e408bb740 1 --2- 172.21.15.123:0/752793792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777466590 0x560777470910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:24.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.975+0000 7f1e408bb740 1 --2- 172.21.15.123:0/752793792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777465c70 0x560777466050 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:24.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.975+0000 7f1e408bb740 1 -- 172.21.15.123:0/752793792 >> 172.21.15.123:0/752793792 conn(0x560777462200 msgr2=0x5607774625e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:24.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.975+0000 7f1e408bb740 1 -- 172.21.15.123:0/752793792 shutdown_connections 2024-09-19T04:52:24.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.975+0000 7f1e408bb740 1 -- 172.21.15.123:0/752793792 wait complete. 2024-09-19T04:52:24.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.975+0000 7f1e408bb740 1 -- start start 2024-09-19T04:52:24.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.976+0000 7f1e408bb740 1 Processor -- start 2024-09-19T04:52:24.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.976+0000 7f1e408bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777465c70 0x56077746dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:24.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.976+0000 7f1e408bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777466590 0x56077746e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:24.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.976+0000 7f1e408bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777470e50 0x56077746e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:24.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.976+0000 7f1e408bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560777473a20 con 0x560777470e50 2024-09-19T04:52:24.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.976+0000 7f1e408bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5607773870f0 con 0x560777465c70 2024-09-19T04:52:24.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.976+0000 7f1e408bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560777473ba0 con 0x560777466590 2024-09-19T04:52:24.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.976+0000 7f1e408bb740 10 client.0 _ll_register_callbacks cb 0x56077736ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:52:24.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.976+0000 7f1e36000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777466590 0x56077746e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:24.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.976+0000 7f1e36000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777466590 0x56077746e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34530/0 (socket says 172.21.15.123:34530) 2024-09-19T04:52:24.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.976+0000 7f1e36000640 1 -- 172.21.15.123:0/2613377854 learned_addr learned my addr 172.21.15.123:0/2613377854 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:24.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.976+0000 7f1e37400640 1 --2- 172.21.15.123:0/2613377854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777465c70 0x56077746dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:24.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.977+0000 7f1e37400640 1 --2- 172.21.15.123:0/2613377854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777470e50 0x56077746e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:24.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.977+0000 7f1e36000640 1 -- 172.21.15.123:0/2613377854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777465c70 msgr2=0x56077746dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:24.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.977+0000 7f1e36000640 1 --2- 172.21.15.123:0/2613377854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777465c70 0x56077746dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:24.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.977+0000 7f1e36000640 1 -- 172.21.15.123:0/2613377854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777470e50 msgr2=0x56077746e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:24.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.977+0000 7f1e36000640 1 --2- 172.21.15.123:0/2613377854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777470e50 0x56077746e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:24.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.977+0000 7f1e36000640 1 -- 172.21.15.123:0/2613377854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56077713d3c0 con 0x560777466590 2024-09-19T04:52:24.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.977+0000 7f1e37400640 1 --2- 172.21.15.123:0/2613377854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777470e50 0x56077746e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:52:24.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.977+0000 7f1e36000640 1 --2- 172.21.15.123:0/2613377854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777466590 0x56077746e210 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1e24033000 tx=0x7f1e24004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37ab27582c9c657c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:24.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.977+0000 7f1e20c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:52:24.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.977+0000 7f1e20c00640 1 -- 172.21.15.123:0/2613377854 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e24035040 con 0x560777466590 2024-09-19T04:52:24.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.977+0000 7f1e20c00640 1 -- 172.21.15.123:0/2613377854 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f1e24005510 con 0x560777466590 2024-09-19T04:52:24.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.977+0000 7f1e408bb740 1 -- 172.21.15.123:0/2613377854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e30011d80 con 0x560777466590 2024-09-19T04:52:24.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.978+0000 7f1e20c00640 1 -- 172.21.15.123:0/2613377854 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1e240046c0 con 0x560777466590 2024-09-19T04:52:24.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.978+0000 7f1e408bb740 10 client.26374 fetch_fsmap learned FSMap version 36 2024-09-19T04:52:24.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.978+0000 7f1e408bb740 1 -- 172.21.15.123:0/2613377854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56077747fa80 con 0x560777466590 2024-09-19T04:52:24.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.978+0000 7f1e20c00640 1 -- 172.21.15.123:0/2613377854 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e30011d80 con 0x560777466590 2024-09-19T04:52:24.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.978+0000 7f1e20c00640 1 -- 172.21.15.123:0/2613377854 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f1e24031bb0 con 0x560777466590 2024-09-19T04:52:24.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.978+0000 7f1e408bb740 10 client.26374 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:52:24.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.978+0000 7f1e408bb740 10 client.26374 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:52:24.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.978+0000 7f1e408bb740 20 client.26374 populate_metadata read hostname 'smithi123' 2024-09-19T04:52:24.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.978+0000 7f1e408bb740 1 --2- 172.21.15.123:0/2613377854 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x560777480f00 0x560777483340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:24.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.978+0000 7f1e408bb740 4 client.26374 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:52:24.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.978+0000 7f1e408bb740 1 -- 172.21.15.123:0/2613377854 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607773d9950 con 0x560777480f00 2024-09-19T04:52:24.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.978+0000 7f1e37400640 1 --2- 172.21.15.123:0/2613377854 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x560777480f00 0x560777483340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:24.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.979+0000 7f1e37400640 1 --2- 172.21.15.123:0/2613377854 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x560777480f00 0x560777483340 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e2ac3f57aa49c3e server_cookie=7af9eb63efb3fd4b in_seq=0 out_seq=0 2024-09-19T04:52:24.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.979+0000 7f1e20c00640 10 client.26374 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:52:24.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.981+0000 7f1e20c00640 1 -- 172.21.15.123:0/2613377854 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607773d9950 con 0x560777480f00 2024-09-19T04:52:24.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.981+0000 7f1e20c00640 10 client.26374 handle_command_reply: tid=0 2024-09-19T04:52:24.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.981+0000 7f1e408bb740 1 client.26374 shutdown 2024-09-19T04:52:24.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.982+0000 7f1e408bb740 1 -- 172.21.15.123:0/2613377854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777466590 msgr2=0x56077746e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:24.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.982+0000 7f1e408bb740 1 --2- 172.21.15.123:0/2613377854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777466590 0x56077746e210 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1e24033000 tx=0x7f1e24004440 comp rx=0 tx=0).stop 2024-09-19T04:52:24.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.982+0000 7f1e408bb740 1 -- 172.21.15.123:0/2613377854 shutdown_connections 2024-09-19T04:52:24.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.982+0000 7f1e408bb740 1 --2- 172.21.15.123:0/2613377854 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x560777480f00 0x560777483340 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:24.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.982+0000 7f1e408bb740 1 --2- 172.21.15.123:0/2613377854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777470e50 0x56077746e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:24.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.982+0000 7f1e408bb740 1 --2- 172.21.15.123:0/2613377854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777466590 0x56077746e210 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:24.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.982+0000 7f1e408bb740 1 --2- 172.21.15.123:0/2613377854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777465c70 0x56077746dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:24.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.982+0000 7f1e20c00640 0 client.26374 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:52:24.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.982+0000 7f1e408bb740 1 -- 172.21.15.123:0/2613377854 >> 172.21.15.123:0/2613377854 conn(0x560777462200 msgr2=0x560777461380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:24.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.982+0000 7f1e408bb740 1 -- 172.21.15.123:0/2613377854 shutdown_connections 2024-09-19T04:52:24.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.982+0000 7f1e408bb740 1 -- 172.21.15.123:0/2613377854 wait complete. 2024-09-19T04:52:24.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.982+0000 7f1e408bb740 20 client.26374 trim_cache size 0 max 16384 2024-09-19T04:52:24.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.996+0000 7f1e408bb740 1 Processor -- start 2024-09-19T04:52:24.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.997+0000 7f1e408bb740 1 -- start start 2024-09-19T04:52:24.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.997+0000 7f1e408bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777465c70 0x560777488c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:24.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.997+0000 7f1e408bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777466590 0x5607774891a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:24.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.997+0000 7f1e408bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777470e50 0x560777460b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:24.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.997+0000 7f1e408bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1800e6d0 con 0x560777470e50 2024-09-19T04:52:24.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.997+0000 7f1e408bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e18010210 con 0x560777466590 2024-09-19T04:52:24.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.997+0000 7f1e408bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1800eda0 con 0x560777465c70 2024-09-19T04:52:24.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.997+0000 7f1e37400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777465c70 0x560777488c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:25.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.997+0000 7f1e36000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777466590 0x5607774891a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:25.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.997+0000 7f1e37400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777465c70 0x560777488c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34532/0 (socket says 172.21.15.123:34532) 2024-09-19T04:52:25.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.997+0000 7f1e37400640 1 -- 172.21.15.123:0/3984446696 learned_addr learned my addr 172.21.15.123:0/3984446696 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:25.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.997+0000 7f1e37400640 1 --2- 172.21.15.123:0/3984446696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777470e50 0x560777460b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:25.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.998+0000 7f1e36000640 1 -- 172.21.15.123:0/3984446696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777465c70 msgr2=0x560777488c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:25.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.998+0000 7f1e36000640 1 --2- 172.21.15.123:0/3984446696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777465c70 0x560777488c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:25.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.998+0000 7f1e36000640 1 -- 172.21.15.123:0/3984446696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777470e50 msgr2=0x560777460b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:25.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.998+0000 7f1e36000640 1 --2- 172.21.15.123:0/3984446696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777470e50 0x560777460b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:25.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.998+0000 7f1e36000640 1 -- 172.21.15.123:0/3984446696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560777462030 con 0x560777466590 2024-09-19T04:52:25.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.998+0000 7f1e36000640 1 --2- 172.21.15.123:0/3984446696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777466590 0x5607774891a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1e24003710 tx=0x7f1e2402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f8260b2132c0f8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:25.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.998+0000 7f1e23400640 1 -- 172.21.15.123:0/3984446696 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e24005510 con 0x560777466590 2024-09-19T04:52:25.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.998+0000 7f1e23400640 1 -- 172.21.15.123:0/3984446696 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e24041070 con 0x560777466590 2024-09-19T04:52:25.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.998+0000 7f1e23400640 1 -- 172.21.15.123:0/3984446696 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e24004db0 con 0x560777466590 2024-09-19T04:52:25.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.999+0000 7f1e408bb740 1 -- 172.21.15.123:0/3984446696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777466590 msgr2=0x5607774891a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:25.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.999+0000 7f1e408bb740 1 --2- 172.21.15.123:0/3984446696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777466590 0x5607774891a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1e24003710 tx=0x7f1e2402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:52:25.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.999+0000 7f1e408bb740 1 -- 172.21.15.123:0/3984446696 shutdown_connections 2024-09-19T04:52:25.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.999+0000 7f1e408bb740 1 --2- 172.21.15.123:0/3984446696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777470e50 0x560777460b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:25.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.999+0000 7f1e408bb740 1 --2- 172.21.15.123:0/3984446696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777466590 0x5607774891a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:25.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.999+0000 7f1e408bb740 1 --2- 172.21.15.123:0/3984446696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777465c70 0x560777488c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:25.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.999+0000 7f1e408bb740 1 -- 172.21.15.123:0/3984446696 >> 172.21.15.123:0/3984446696 conn(0x560777462200 msgr2=0x56077746cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:25.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.999+0000 7f1e408bb740 1 -- 172.21.15.123:0/3984446696 shutdown_connections 2024-09-19T04:52:25.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.999+0000 7f1e408bb740 1 -- 172.21.15.123:0/3984446696 wait complete. 2024-09-19T04:52:25.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.999+0000 7f1e408bb740 1 -- start start 2024-09-19T04:52:25.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:24.999+0000 7f1e408bb740 1 Processor -- start 2024-09-19T04:52:25.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.000+0000 7f1e408bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777465c70 0x560777478e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:25.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.000+0000 7f1e408bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777466590 0x560777479390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:25.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.000+0000 7f1e408bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777470e50 0x560777475e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:25.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.000+0000 7f1e408bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1800fb40 con 0x560777465c70 2024-09-19T04:52:25.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.000+0000 7f1e408bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1800f470 con 0x560777470e50 2024-09-19T04:52:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.000+0000 7f1e408bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1800c6b0 con 0x560777466590 2024-09-19T04:52:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.000+0000 7f1e408bb740 10 client.0 _ll_register_callbacks cb 0x7f1e24009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:52:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.000+0000 7f1e36000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777466590 0x560777479390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.000+0000 7f1e37400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777465c70 0x560777478e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.000+0000 7f1e36000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777466590 0x560777479390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34538/0 (socket says 172.21.15.123:34538) 2024-09-19T04:52:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.000+0000 7f1e36000640 1 -- 172.21.15.123:0/356344372 learned_addr learned my addr 172.21.15.123:0/356344372 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.001+0000 7f1e36000640 1 -- 172.21.15.123:0/356344372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777470e50 msgr2=0x560777475e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.001+0000 7f1e36000640 1 --2- 172.21.15.123:0/356344372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777470e50 0x560777475e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.001+0000 7f1e36000640 1 -- 172.21.15.123:0/356344372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777465c70 msgr2=0x560777478e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.001+0000 7f1e36000640 1 --2- 172.21.15.123:0/356344372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777465c70 0x560777478e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:25.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.001+0000 7f1e36000640 1 -- 172.21.15.123:0/356344372 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560777468ae0 con 0x560777466590 2024-09-19T04:52:25.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.001+0000 7f1e36000640 1 --2- 172.21.15.123:0/356344372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777466590 0x560777479390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1e24031550 tx=0x7f1e24003670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1851530c2501c1b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:25.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.001+0000 7f1e20c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:52:25.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.001+0000 7f1e20c00640 1 -- 172.21.15.123:0/356344372 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e24035040 con 0x560777466590 2024-09-19T04:52:25.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.001+0000 7f1e408bb740 1 -- 172.21.15.123:0/356344372 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e24041070 con 0x560777466590 2024-09-19T04:52:25.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.001+0000 7f1e20c00640 1 -- 172.21.15.123:0/356344372 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f1e24005520 con 0x560777466590 2024-09-19T04:52:25.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.001+0000 7f1e20c00640 1 -- 172.21.15.123:0/356344372 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1e2405d020 con 0x560777466590 2024-09-19T04:52:25.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.001+0000 7f1e408bb740 10 client.26380 fetch_fsmap learned FSMap version 36 2024-09-19T04:52:25.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.001+0000 7f1e408bb740 1 -- 172.21.15.123:0/356344372 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56077745d770 con 0x560777466590 2024-09-19T04:52:25.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.002+0000 7f1e20c00640 1 -- 172.21.15.123:0/356344372 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e24041070 con 0x560777466590 2024-09-19T04:52:25.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.002+0000 7f1e20c00640 1 -- 172.21.15.123:0/356344372 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f1e24031640 con 0x560777466590 2024-09-19T04:52:25.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.002+0000 7f1e408bb740 10 client.26380 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:52:25.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.002+0000 7f1e408bb740 10 client.26380 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:52:25.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.002+0000 7f1e408bb740 20 client.26380 populate_metadata read hostname 'smithi123' 2024-09-19T04:52:25.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.002+0000 7f1e408bb740 1 --2- 172.21.15.123:0/356344372 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5607774ec560 0x5607774ee980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:25.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.002+0000 7f1e408bb740 4 client.26380 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:52:25.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.002+0000 7f1e408bb740 1 -- 172.21.15.123:0/356344372 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607773d3e60 con 0x5607774ec560 2024-09-19T04:52:25.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.002+0000 7f1e37400640 1 --2- 172.21.15.123:0/356344372 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5607774ec560 0x5607774ee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:25.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.003+0000 7f1e37400640 1 --2- 172.21.15.123:0/356344372 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5607774ec560 0x5607774ee980 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=627e7a7f314012fc server_cookie=bb32cfc91d06f9f5 in_seq=0 out_seq=0 2024-09-19T04:52:25.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.003+0000 7f1e20c00640 10 client.26380 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:52:25.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.005+0000 7f1e20c00640 1 -- 172.21.15.123:0/356344372 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607773d3e60 con 0x5607774ec560 2024-09-19T04:52:25.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.005+0000 7f1e20c00640 10 client.26380 handle_command_reply: tid=0 2024-09-19T04:52:25.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.005+0000 7f1e408bb740 1 client.26380 shutdown 2024-09-19T04:52:25.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.005+0000 7f1e408bb740 1 -- 172.21.15.123:0/356344372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777466590 msgr2=0x560777479390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:25.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.005+0000 7f1e408bb740 1 --2- 172.21.15.123:0/356344372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777466590 0x560777479390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1e24031550 tx=0x7f1e24003670 comp rx=0 tx=0).stop 2024-09-19T04:52:25.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.005+0000 7f1e408bb740 1 -- 172.21.15.123:0/356344372 shutdown_connections 2024-09-19T04:52:25.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.005+0000 7f1e408bb740 1 --2- 172.21.15.123:0/356344372 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5607774ec560 0x5607774ee980 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:25.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.005+0000 7f1e408bb740 1 --2- 172.21.15.123:0/356344372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560777470e50 0x560777475e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:25.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.005+0000 7f1e408bb740 1 --2- 172.21.15.123:0/356344372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560777466590 0x560777479390 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:25.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.005+0000 7f1e408bb740 1 --2- 172.21.15.123:0/356344372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560777465c70 0x560777478e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:25.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.005+0000 7f1e20c00640 0 client.26380 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:52:25.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.005+0000 7f1e408bb740 1 -- 172.21.15.123:0/356344372 >> 172.21.15.123:0/356344372 conn(0x560777462200 msgr2=0x560777460b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:25.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.005+0000 7f1e408bb740 1 -- 172.21.15.123:0/356344372 shutdown_connections 2024-09-19T04:52:25.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.005+0000 7f1e408bb740 1 -- 172.21.15.123:0/356344372 wait complete. 2024-09-19T04:52:25.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.006+0000 7f1e408bb740 20 client.26380 trim_cache size 0 max 16384 2024-09-19T04:52:25.012 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:52:25.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.006+0000 7f1e3de00640 1 -- 172.21.15.123:0/2843056860 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1e1c0a3430 msgr2=0x7f1e1c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:25.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.006+0000 7f1e3de00640 1 --2- 172.21.15.123:0/2843056860 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1e1c0a3430 0x7f1e1c0a58f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x560777463060 tx=0x7f1e280d3a70 comp rx=0 tx=0).stop 2024-09-19T04:52:25.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.006+0000 7f1e3de00640 1 -- 172.21.15.123:0/2843056860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e38101de0 msgr2=0x7f1e3819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:25.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.006+0000 7f1e3de00640 1 --2- 172.21.15.123:0/2843056860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e38101de0 0x7f1e3819a690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800e9f0 tx=0x7f1e2800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:52:25.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.006+0000 7f1e3de00640 1 -- 172.21.15.123:0/2843056860 shutdown_connections 2024-09-19T04:52:25.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.006+0000 7f1e3de00640 1 --2- 172.21.15.123:0/2843056860 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1e1c0a3430 0x7f1e1c0a58f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:25.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.006+0000 7f1e3de00640 1 --2- 172.21.15.123:0/2843056860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e38106f10 0x7f1e3819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:25.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.006+0000 7f1e3de00640 1 --2- 172.21.15.123:0/2843056860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e38102790 0x7f1e3819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:25.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.006+0000 7f1e3de00640 1 --2- 172.21.15.123:0/2843056860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e38101de0 0x7f1e3819a690 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:25.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.006+0000 7f1e3de00640 1 -- 172.21.15.123:0/2843056860 >> 172.21.15.123:0/2843056860 conn(0x7f1e380fd8c0 msgr2=0x7f1e380ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:25.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.006+0000 7f1e3de00640 1 -- 172.21.15.123:0/2843056860 shutdown_connections 2024-09-19T04:52:25.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:25.006+0000 7f1e3de00640 1 -- 172.21.15.123:0/2843056860 wait complete. 2024-09-19T04:52:25.437 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:25 smithi123 ceph-mon[30511]: pgmap v174: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.3 MiB/s wr, 708 op/s 2024-09-19T04:52:25.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:25 smithi193 ceph-mon[35592]: pgmap v174: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.3 MiB/s wr, 708 op/s 2024-09-19T04:52:25.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:25 smithi183 ceph-mon[35333]: pgmap v174: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.3 MiB/s wr, 708 op/s 2024-09-19T04:52:25.810 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:52:26.208 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:52:26.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.787+0000 7f9cbe000640 1 -- 172.21.15.123:0/657997195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb8108f60 msgr2=0x7f9cb8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:26.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.787+0000 7f9cbe000640 1 --2- 172.21.15.123:0/657997195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb8108f60 0x7f9cb8110ae0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009ca0 tx=0x7f9cb002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.787+0000 7f9cbe000640 1 -- 172.21.15.123:0/657997195 shutdown_connections 2024-09-19T04:52:26.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.787+0000 7f9cbe000640 1 --2- 172.21.15.123:0/657997195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb8108f60 0x7f9cb8110ae0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.787+0000 7f9cbe000640 1 --2- 172.21.15.123:0/657997195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cb81047e0 0x7f9cb8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.787+0000 7f9cbe000640 1 --2- 172.21.15.123:0/657997195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb8103e30 0x7f9cb8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.787+0000 7f9cbe000640 1 -- 172.21.15.123:0/657997195 >> 172.21.15.123:0/657997195 conn(0x7f9cb80ff910 msgr2=0x7f9cb8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:26.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.787+0000 7f9cbe000640 1 -- 172.21.15.123:0/657997195 shutdown_connections 2024-09-19T04:52:26.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.787+0000 7f9cbe000640 1 -- 172.21.15.123:0/657997195 wait complete. 2024-09-19T04:52:26.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.788+0000 7f9cbe000640 1 Processor -- start 2024-09-19T04:52:26.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.788+0000 7f9cbe000640 1 -- start start 2024-09-19T04:52:26.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.788+0000 7f9cbe000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cb8103e30 0x7f9cb819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.788+0000 7f9cbe000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb81047e0 0x7f9cb819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.788+0000 7f9cbe000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb8108f60 0x7f9cb819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.788+0000 7f9cbe000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb8114010 con 0x7f9cb81047e0 2024-09-19T04:52:26.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.788+0000 7f9cbe000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb8113e90 con 0x7f9cb8108f60 2024-09-19T04:52:26.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.788+0000 7f9cbe000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb8114190 con 0x7f9cb8103e30 2024-09-19T04:52:26.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.789+0000 7f9cbcc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cb8103e30 0x7f9cb819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.789+0000 7f9cbcc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cb8103e30 0x7f9cb819a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34554/0 (socket says 172.21.15.123:34554) 2024-09-19T04:52:26.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.789+0000 7f9cbcc00640 1 -- 172.21.15.123:0/2190846644 learned_addr learned my addr 172.21.15.123:0/2190846644 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:26.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.789+0000 7f9cb7e00640 1 --2- 172.21.15.123:0/2190846644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb81047e0 0x7f9cb819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.789+0000 7f9cbd600640 1 --2- 172.21.15.123:0/2190846644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb8108f60 0x7f9cb819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.789+0000 7f9cbcc00640 1 -- 172.21.15.123:0/2190846644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb8108f60 msgr2=0x7f9cb819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:26.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.789+0000 7f9cbcc00640 1 --2- 172.21.15.123:0/2190846644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb8108f60 0x7f9cb819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.789+0000 7f9cbcc00640 1 -- 172.21.15.123:0/2190846644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb81047e0 msgr2=0x7f9cb819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:26.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.789+0000 7f9cbcc00640 1 --2- 172.21.15.123:0/2190846644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb81047e0 0x7f9cb819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.789+0000 7f9cbcc00640 1 -- 172.21.15.123:0/2190846644 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb819f3c0 con 0x7f9cb8103e30 2024-09-19T04:52:26.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.789+0000 7f9cbd600640 1 --2- 172.21.15.123:0/2190846644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb8108f60 0x7f9cb819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:52:26.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.789+0000 7f9cb7e00640 1 --2- 172.21.15.123:0/2190846644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb81047e0 0x7f9cb819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:52:26.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.789+0000 7f9cbcc00640 1 --2- 172.21.15.123:0/2190846644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cb8103e30 0x7f9cb819a410 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00ea60 tx=0x7f9cac00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:26.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.790+0000 7f9cb5600640 1 -- 172.21.15.123:0/2190846644 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac00cde0 con 0x7f9cb8103e30 2024-09-19T04:52:26.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.790+0000 7f9cbe000640 1 -- 172.21.15.123:0/2190846644 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb819f6b0 con 0x7f9cb8103e30 2024-09-19T04:52:26.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.790+0000 7f9cb5600640 1 -- 172.21.15.123:0/2190846644 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cac0035f0 con 0x7f9cb8103e30 2024-09-19T04:52:26.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.790+0000 7f9cb5600640 1 -- 172.21.15.123:0/2190846644 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac010670 con 0x7f9cb8103e30 2024-09-19T04:52:26.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.790+0000 7f9cbe000640 1 -- 172.21.15.123:0/2190846644 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb81a6f90 con 0x7f9cb8103e30 2024-09-19T04:52:26.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.793+0000 7f9cc205c740 1 Processor -- start 2024-09-19T04:52:26.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.793+0000 7f9cb5600640 1 -- 172.21.15.123:0/2190846644 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9cac0037a0 con 0x7f9cb8103e30 2024-09-19T04:52:26.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.793+0000 7f9cc205c740 1 -- start start 2024-09-19T04:52:26.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.793+0000 7f9cc205c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e198ae40 0x5641e198b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.793+0000 7f9cc205c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 0x5641e1995ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.793+0000 7f9cc205c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1996020 0x5641e1998410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.793+0000 7f9cc205c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5641e18f13e0 con 0x5641e198ae40 2024-09-19T04:52:26.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.793+0000 7f9cc205c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5641e1898d80 con 0x5641e198b760 2024-09-19T04:52:26.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.793+0000 7f9cc205c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5641e18ed3a0 con 0x5641e1996020 2024-09-19T04:52:26.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.794+0000 7f9cbcc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 0x5641e1995ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.794+0000 7f9cbd600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e198ae40 0x5641e198b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.794+0000 7f9cb7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1996020 0x5641e1998410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.794+0000 7f9cbcc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 0x5641e1995ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:56152/0 (socket says 172.21.15.123:56152) 2024-09-19T04:52:26.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.794+0000 7f9cbcc00640 1 -- 172.21.15.123:0/2016208598 learned_addr learned my addr 172.21.15.123:0/2016208598 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:26.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.794+0000 7f9cbcc00640 1 -- 172.21.15.123:0/2016208598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1996020 msgr2=0x5641e1998410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:26.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.794+0000 7f9cbcc00640 1 --2- 172.21.15.123:0/2016208598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1996020 0x5641e1998410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.794+0000 7f9cbcc00640 1 -- 172.21.15.123:0/2016208598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e198ae40 msgr2=0x5641e198b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:26.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.794+0000 7f9cbcc00640 1 --2- 172.21.15.123:0/2016208598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e198ae40 0x5641e198b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.794+0000 7f9cbcc00640 1 -- 172.21.15.123:0/2016208598 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641e18b7ae0 con 0x5641e198b760 2024-09-19T04:52:26.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.794+0000 7f9cbcc00640 1 --2- 172.21.15.123:0/2016208598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 0x5641e1995ae0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0cd710 tx=0x7f9cac0d8570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a46b3f3168590d01 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:26.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.795+0000 7f9ca9c00640 1 -- 172.21.15.123:0/2016208598 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac0e3070 con 0x5641e198b760 2024-09-19T04:52:26.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.795+0000 7f9cb5600640 1 --2- 172.21.15.123:0/2190846644 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9ca00a33b0 0x7f9ca00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.795+0000 7f9ca9c00640 1 -- 172.21.15.123:0/2016208598 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cac0d22d0 con 0x5641e198b760 2024-09-19T04:52:26.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.795+0000 7f9ca9c00640 1 -- 172.21.15.123:0/2016208598 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac09bea0 con 0x5641e198b760 2024-09-19T04:52:26.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.795+0000 7f9cb5600640 1 -- 172.21.15.123:0/2190846644 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f9cac0d00b0 con 0x7f9cb8103e30 2024-09-19T04:52:26.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.795+0000 7f9cbd600640 1 --2- 172.21.15.123:0/2190846644 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9ca00a33b0 0x7f9ca00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.795+0000 7f9cc205c740 1 -- 172.21.15.123:0/2016208598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 msgr2=0x5641e1995ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:26.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.795+0000 7f9cc205c740 1 --2- 172.21.15.123:0/2016208598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 0x5641e1995ae0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0cd710 tx=0x7f9cac0d8570 comp rx=0 tx=0).stop 2024-09-19T04:52:26.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.796+0000 7f9cc205c740 1 -- 172.21.15.123:0/2016208598 shutdown_connections 2024-09-19T04:52:26.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.796+0000 7f9cc205c740 1 --2- 172.21.15.123:0/2016208598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1996020 0x5641e1998410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.796+0000 7f9cc205c740 1 --2- 172.21.15.123:0/2016208598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 0x5641e1995ae0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.796+0000 7f9cc205c740 1 --2- 172.21.15.123:0/2016208598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e198ae40 0x5641e198b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.796+0000 7f9cc205c740 1 -- 172.21.15.123:0/2016208598 >> 172.21.15.123:0/2016208598 conn(0x5641e19873d0 msgr2=0x5641e19877b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:26.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.796+0000 7f9cbd600640 1 --2- 172.21.15.123:0/2190846644 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9ca00a33b0 0x7f9ca00a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0004af0 tx=0x7f9cb0004a40 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:26.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.796+0000 7f9cc205c740 1 -- 172.21.15.123:0/2016208598 shutdown_connections 2024-09-19T04:52:26.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.796+0000 7f9cc205c740 1 -- 172.21.15.123:0/2016208598 wait complete. 2024-09-19T04:52:26.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.796+0000 7f9cc205c740 1 -- start start 2024-09-19T04:52:26.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.797+0000 7f9cc205c740 1 Processor -- start 2024-09-19T04:52:26.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.797+0000 7f9cc205c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e198ae40 0x5641e1992f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.797+0000 7f9cc205c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 0x5641e1993490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.797+0000 7f9cc205c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1996020 0x5641e19939d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.797+0000 7f9cc205c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5641e1998bf0 con 0x5641e198ae40 2024-09-19T04:52:26.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.797+0000 7f9cc205c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5641e18ac0f0 con 0x5641e198b760 2024-09-19T04:52:26.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.797+0000 7f9cc205c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5641e1998d70 con 0x5641e1996020 2024-09-19T04:52:26.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.797+0000 7f9cc205c740 10 client.0 _ll_register_callbacks cb 0x5641e1893bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:52:26.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.797+0000 7f9cbd600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1996020 0x5641e19939d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.797+0000 7f9cb7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 0x5641e1993490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.797+0000 7f9cbcc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e198ae40 0x5641e1992f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.797+0000 7f9cbd600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1996020 0x5641e19939d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34572/0 (socket says 172.21.15.123:34572) 2024-09-19T04:52:26.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.797+0000 7f9cbd600640 1 -- 172.21.15.123:0/3325552177 learned_addr learned my addr 172.21.15.123:0/3325552177 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:26.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.798+0000 7f9cb7e00640 1 -- 172.21.15.123:0/3325552177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1996020 msgr2=0x5641e19939d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:26.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.798+0000 7f9cb7e00640 1 --2- 172.21.15.123:0/3325552177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1996020 0x5641e19939d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.798+0000 7f9cb7e00640 1 -- 172.21.15.123:0/3325552177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e198ae40 msgr2=0x5641e1992f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:26.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.798+0000 7f9cb7e00640 1 --2- 172.21.15.123:0/3325552177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e198ae40 0x5641e1992f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.798+0000 7f9cb7e00640 1 -- 172.21.15.123:0/3325552177 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641e16623c0 con 0x5641e198b760 2024-09-19T04:52:26.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.798+0000 7f9cb7e00640 1 --2- 172.21.15.123:0/3325552177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 0x5641e1993490 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9cb810a420 tx=0x7f9ca400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4395f33644ce3b88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:26.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.798+0000 7f9c9e000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:52:26.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.798+0000 7f9c9e000640 1 -- 172.21.15.123:0/3325552177 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4012110 con 0x5641e198b760 2024-09-19T04:52:26.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.798+0000 7f9c9e000640 1 -- 172.21.15.123:0/3325552177 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f9ca4004050 con 0x5641e198b760 2024-09-19T04:52:26.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.798+0000 7f9cc205c740 1 -- 172.21.15.123:0/3325552177 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cac0d22d0 con 0x5641e198b760 2024-09-19T04:52:26.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.799+0000 7f9c9e000640 1 -- 172.21.15.123:0/3325552177 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9ca40122b0 con 0x5641e198b760 2024-09-19T04:52:26.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.799+0000 7f9cc205c740 10 client.26372 fetch_fsmap learned FSMap version 36 2024-09-19T04:52:26.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.799+0000 7f9cc205c740 1 -- 172.21.15.123:0/3325552177 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5641e19a4ce0 con 0x5641e198b760 2024-09-19T04:52:26.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.799+0000 7f9c9e000640 1 -- 172.21.15.123:0/3325552177 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cac0d22d0 con 0x5641e198b760 2024-09-19T04:52:26.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.799+0000 7f9c9e000640 1 -- 172.21.15.123:0/3325552177 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f9ca40254c0 con 0x5641e198b760 2024-09-19T04:52:26.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.799+0000 7f9cc205c740 10 client.26372 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:52:26.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.799+0000 7f9cc205c740 10 client.26372 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:52:26.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.800+0000 7f9cc205c740 20 client.26372 populate_metadata read hostname 'smithi123' 2024-09-19T04:52:26.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.800+0000 7f9cc205c740 1 --2- 172.21.15.123:0/3325552177 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5641e19a60f0 0x5641e19a8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.800+0000 7f9cc205c740 4 client.26372 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:52:26.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.800+0000 7f9cc205c740 1 -- 172.21.15.123:0/3325552177 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641e18fe950 con 0x5641e19a60f0 2024-09-19T04:52:26.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.800+0000 7f9cbcc00640 1 --2- 172.21.15.123:0/3325552177 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5641e19a60f0 0x5641e19a8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.800+0000 7f9c9e000640 10 client.26372 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:52:26.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.800+0000 7f9cbcc00640 1 --2- 172.21.15.123:0/3325552177 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5641e19a60f0 0x5641e19a8520 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2dac7432738ad9c server_cookie=ebf665174af7c1d2 in_seq=0 out_seq=0 2024-09-19T04:52:26.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.801+0000 7f9c9e000640 1 -- 172.21.15.123:0/3325552177 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641e18fe950 con 0x5641e19a60f0 2024-09-19T04:52:26.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.801+0000 7f9c9e000640 10 client.26372 handle_command_reply: tid=0 2024-09-19T04:52:26.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.801+0000 7f9cc205c740 1 client.26372 shutdown 2024-09-19T04:52:26.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.802+0000 7f9cc205c740 1 -- 172.21.15.123:0/3325552177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 msgr2=0x5641e1993490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:26.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.802+0000 7f9cc205c740 1 --2- 172.21.15.123:0/3325552177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 0x5641e1993490 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9cb810a420 tx=0x7f9ca400f790 comp rx=0 tx=0).stop 2024-09-19T04:52:26.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.802+0000 7f9cc205c740 1 -- 172.21.15.123:0/3325552177 shutdown_connections 2024-09-19T04:52:26.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.802+0000 7f9cc205c740 1 --2- 172.21.15.123:0/3325552177 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5641e19a60f0 0x5641e19a8520 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.802+0000 7f9cc205c740 1 --2- 172.21.15.123:0/3325552177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1996020 0x5641e19939d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.802+0000 7f9cc205c740 1 --2- 172.21.15.123:0/3325552177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 0x5641e1993490 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.802+0000 7f9cc205c740 1 --2- 172.21.15.123:0/3325552177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e198ae40 0x5641e1992f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.802+0000 7f9cc205c740 1 -- 172.21.15.123:0/3325552177 >> 172.21.15.123:0/3325552177 conn(0x5641e19873d0 msgr2=0x5641e198d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:26.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.802+0000 7f9c9e000640 0 client.26372 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:52:26.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.802+0000 7f9cc205c740 1 -- 172.21.15.123:0/3325552177 shutdown_connections 2024-09-19T04:52:26.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.802+0000 7f9cc205c740 1 -- 172.21.15.123:0/3325552177 wait complete. 2024-09-19T04:52:26.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.802+0000 7f9cc205c740 20 client.26372 trim_cache size 0 max 16384 2024-09-19T04:52:26.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.817+0000 7f9cc205c740 1 Processor -- start 2024-09-19T04:52:26.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.817+0000 7f9cc205c740 1 -- start start 2024-09-19T04:52:26.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.817+0000 7f9cc205c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e198ae40 0x5641e1992810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.817+0000 7f9cc205c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 0x5641e19ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.818+0000 7f9cc205c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e1996020 0x5641e19ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.818+0000 7f9cc205c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb81af890 con 0x5641e1996020 2024-09-19T04:52:26.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.818+0000 7f9cc205c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb81b13d0 con 0x5641e198b760 2024-09-19T04:52:26.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.818+0000 7f9cc205c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb81aff60 con 0x5641e198ae40 2024-09-19T04:52:26.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.818+0000 7f9cbcc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e198ae40 0x5641e1992810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.818+0000 7f9cbcc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e198ae40 0x5641e1992810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34576/0 (socket says 172.21.15.123:34576) 2024-09-19T04:52:26.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.818+0000 7f9cbcc00640 1 -- 172.21.15.123:0/4008966768 learned_addr learned my addr 172.21.15.123:0/4008966768 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:26.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.818+0000 7f9cbd600640 1 --2- 172.21.15.123:0/4008966768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e1996020 0x5641e19ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.818+0000 7f9cb7e00640 1 --2- 172.21.15.123:0/4008966768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 0x5641e19ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.818+0000 7f9cbcc00640 1 -- 172.21.15.123:0/4008966768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 msgr2=0x5641e19ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:26.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.818+0000 7f9cbcc00640 1 --2- 172.21.15.123:0/4008966768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 0x5641e19ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.818+0000 7f9cbcc00640 1 -- 172.21.15.123:0/4008966768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e1996020 msgr2=0x5641e19ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:26.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.818+0000 7f9cbcc00640 1 --2- 172.21.15.123:0/4008966768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e1996020 0x5641e19ae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.818+0000 7f9cbcc00640 1 -- 172.21.15.123:0/4008966768 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641e1991f20 con 0x5641e198ae40 2024-09-19T04:52:26.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.818+0000 7f9cbcc00640 1 --2- 172.21.15.123:0/4008966768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e198ae40 0x5641e1992810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9cac09c060 tx=0x7f9cac0d0f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=977f4a90edf627db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:26.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.819+0000 7f9ca9200640 1 -- 172.21.15.123:0/4008966768 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac09c130 con 0x5641e198ae40 2024-09-19T04:52:26.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.819+0000 7f9ca9200640 1 -- 172.21.15.123:0/4008966768 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cac0d3930 con 0x5641e198ae40 2024-09-19T04:52:26.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.819+0000 7f9ca9200640 1 -- 172.21.15.123:0/4008966768 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac097ed0 con 0x5641e198ae40 2024-09-19T04:52:26.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.819+0000 7f9cc205c740 1 -- 172.21.15.123:0/4008966768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e198ae40 msgr2=0x5641e1992810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:26.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.819+0000 7f9cc205c740 1 --2- 172.21.15.123:0/4008966768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e198ae40 0x5641e1992810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9cac09c060 tx=0x7f9cac0d0f90 comp rx=0 tx=0).stop 2024-09-19T04:52:26.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.819+0000 7f9cc205c740 1 -- 172.21.15.123:0/4008966768 shutdown_connections 2024-09-19T04:52:26.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.819+0000 7f9cc205c740 1 --2- 172.21.15.123:0/4008966768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e1996020 0x5641e19ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.819+0000 7f9cc205c740 1 --2- 172.21.15.123:0/4008966768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e198b760 0x5641e19ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.819+0000 7f9cc205c740 1 --2- 172.21.15.123:0/4008966768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e198ae40 0x5641e1992810 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.819+0000 7f9cc205c740 1 -- 172.21.15.123:0/4008966768 >> 172.21.15.123:0/4008966768 conn(0x5641e19873d0 msgr2=0x5641e1994fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:26.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.819+0000 7f9cc205c740 1 -- 172.21.15.123:0/4008966768 shutdown_connections 2024-09-19T04:52:26.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.820+0000 7f9cc205c740 1 -- 172.21.15.123:0/4008966768 wait complete. 2024-09-19T04:52:26.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.820+0000 7f9cc205c740 1 -- start start 2024-09-19T04:52:26.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.820+0000 7f9cc205c740 1 Processor -- start 2024-09-19T04:52:26.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.820+0000 7f9cc205c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e198ae40 0x5641e199dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cc205c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e198b760 0x5641e199e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cc205c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e1996020 0x5641e199af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cc205c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb81b0d00 con 0x5641e198ae40 2024-09-19T04:52:26.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cc205c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb81b0630 con 0x5641e1996020 2024-09-19T04:52:26.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cc205c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb81ad870 con 0x5641e198b760 2024-09-19T04:52:26.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cc205c740 10 client.0 _ll_register_callbacks cb 0x7f9ca4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:52:26.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cb7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e198b760 0x5641e199e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cbcc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e198ae40 0x5641e199dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cb7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e198b760 0x5641e199e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34588/0 (socket says 172.21.15.123:34588) 2024-09-19T04:52:26.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cb7e00640 1 -- 172.21.15.123:0/2918383998 learned_addr learned my addr 172.21.15.123:0/2918383998 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:26.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cbd600640 1 --2- 172.21.15.123:0/2918383998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e1996020 0x5641e199af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cbcc00640 1 -- 172.21.15.123:0/2918383998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e198b760 msgr2=0x5641e199e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:26.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cbcc00640 1 --2- 172.21.15.123:0/2918383998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e198b760 0x5641e199e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cbcc00640 1 -- 172.21.15.123:0/2918383998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e1996020 msgr2=0x5641e199af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:26.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cbcc00640 1 --2- 172.21.15.123:0/2918383998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e1996020 0x5641e199af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cbcc00640 1 -- 172.21.15.123:0/2918383998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641e199b4d0 con 0x5641e198ae40 2024-09-19T04:52:26.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cb7e00640 1 --2- 172.21.15.123:0/2918383998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e198b760 0x5641e199e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:52:26.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.821+0000 7f9cbd600640 1 --2- 172.21.15.123:0/2918383998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e1996020 0x5641e199af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:52:26.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.822+0000 7f9cbcc00640 1 --2- 172.21.15.123:0/2918383998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e198ae40 0x5641e199dfc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9cac09c060 tx=0x7f9cac00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9a537e0957f623 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:26.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.822+0000 7f9c9e000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:52:26.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.822+0000 7f9c9e000640 1 -- 172.21.15.123:0/2918383998 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac0980b0 con 0x5641e198ae40 2024-09-19T04:52:26.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.822+0000 7f9c9e000640 1 -- 172.21.15.123:0/2918383998 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f9cac0b5260 con 0x5641e198ae40 2024-09-19T04:52:26.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.822+0000 7f9cc205c740 1 -- 172.21.15.123:0/2918383998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cac0d3930 con 0x5641e198ae40 2024-09-19T04:52:26.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.822+0000 7f9c9e000640 1 -- 172.21.15.123:0/2918383998 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9cac098430 con 0x5641e198ae40 2024-09-19T04:52:26.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.822+0000 7f9cc205c740 10 client.16275 fetch_fsmap learned FSMap version 36 2024-09-19T04:52:26.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.822+0000 7f9cc205c740 1 -- 172.21.15.123:0/2918383998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5641e1a19cd0 con 0x5641e198ae40 2024-09-19T04:52:26.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.822+0000 7f9c9e000640 1 -- 172.21.15.123:0/2918383998 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cac0d3930 con 0x5641e198ae40 2024-09-19T04:52:26.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.823+0000 7f9c9e000640 1 -- 172.21.15.123:0/2918383998 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f9cac0989d0 con 0x5641e198ae40 2024-09-19T04:52:26.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.823+0000 7f9cc205c740 10 client.16275 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:52:26.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.823+0000 7f9cc205c740 10 client.16275 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:52:26.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.823+0000 7f9cc205c740 20 client.16275 populate_metadata read hostname 'smithi123' 2024-09-19T04:52:26.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.823+0000 7f9cc205c740 1 --2- 172.21.15.123:0/2918383998 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5641e1a19f00 0x5641e1a1c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:26.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.823+0000 7f9cc205c740 4 client.16275 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:52:26.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.823+0000 7f9cc205c740 1 -- 172.21.15.123:0/2918383998 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641e18f8e60 con 0x5641e1a19f00 2024-09-19T04:52:26.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.823+0000 7f9cb7e00640 1 --2- 172.21.15.123:0/2918383998 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5641e1a19f00 0x5641e1a1c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:26.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.823+0000 7f9cb7e00640 1 --2- 172.21.15.123:0/2918383998 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5641e1a19f00 0x5641e1a1c2f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93f716ad3609b9f9 server_cookie=b975bfe15d3ca71c in_seq=0 out_seq=0 2024-09-19T04:52:26.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.823+0000 7f9c9e000640 10 client.16275 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:52:26.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.824+0000 7f9c9e000640 1 -- 172.21.15.123:0/2918383998 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641e18f8e60 con 0x5641e1a19f00 2024-09-19T04:52:26.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.824+0000 7f9c9e000640 10 client.16275 handle_command_reply: tid=0 2024-09-19T04:52:26.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.824+0000 7f9cc205c740 1 client.16275 shutdown 2024-09-19T04:52:26.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.824+0000 7f9cc205c740 1 -- 172.21.15.123:0/2918383998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e198ae40 msgr2=0x5641e199dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:26.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.824+0000 7f9cc205c740 1 --2- 172.21.15.123:0/2918383998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e198ae40 0x5641e199dfc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9cac09c060 tx=0x7f9cac00ef90 comp rx=0 tx=0).stop 2024-09-19T04:52:26.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.824+0000 7f9cc205c740 1 -- 172.21.15.123:0/2918383998 shutdown_connections 2024-09-19T04:52:26.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.824+0000 7f9cc205c740 1 --2- 172.21.15.123:0/2918383998 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5641e1a19f00 0x5641e1a1c2f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.824+0000 7f9cc205c740 1 --2- 172.21.15.123:0/2918383998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e1996020 0x5641e199af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.824+0000 7f9cc205c740 1 --2- 172.21.15.123:0/2918383998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e198b760 0x5641e199e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.824+0000 7f9cc205c740 1 --2- 172.21.15.123:0/2918383998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641e198ae40 0x5641e199dfc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.824+0000 7f9cc205c740 1 -- 172.21.15.123:0/2918383998 >> 172.21.15.123:0/2918383998 conn(0x5641e19873d0 msgr2=0x5641e1985c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:26.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.824+0000 7f9c9e000640 0 client.16275 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:52:26.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.824+0000 7f9cc205c740 1 -- 172.21.15.123:0/2918383998 shutdown_connections 2024-09-19T04:52:26.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.824+0000 7f9cc205c740 1 -- 172.21.15.123:0/2918383998 wait complete. 2024-09-19T04:52:26.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.825+0000 7f9cc205c740 20 client.16275 trim_cache size 0 max 16384 2024-09-19T04:52:26.830 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:52:26.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.825+0000 7f9cbe000640 1 -- 172.21.15.123:0/2190846644 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9ca00a33b0 msgr2=0x7f9ca00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:26.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.825+0000 7f9cbe000640 1 --2- 172.21.15.123:0/2190846644 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9ca00a33b0 0x7f9ca00a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0004af0 tx=0x7f9cb0004a40 comp rx=0 tx=0).stop 2024-09-19T04:52:26.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.825+0000 7f9cbe000640 1 -- 172.21.15.123:0/2190846644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cb8103e30 msgr2=0x7f9cb819a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:26.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.825+0000 7f9cbe000640 1 --2- 172.21.15.123:0/2190846644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cb8103e30 0x7f9cb819a410 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00ea60 tx=0x7f9cac00ef20 comp rx=0 tx=0).stop 2024-09-19T04:52:26.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.825+0000 7f9cbe000640 1 -- 172.21.15.123:0/2190846644 shutdown_connections 2024-09-19T04:52:26.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.825+0000 7f9cbe000640 1 --2- 172.21.15.123:0/2190846644 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9ca00a33b0 0x7f9ca00a5870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.825+0000 7f9cbe000640 1 --2- 172.21.15.123:0/2190846644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb8108f60 0x7f9cb819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.825+0000 7f9cbe000640 1 --2- 172.21.15.123:0/2190846644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb81047e0 0x7f9cb819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.825+0000 7f9cbe000640 1 --2- 172.21.15.123:0/2190846644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cb8103e30 0x7f9cb819a410 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:26.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.825+0000 7f9cbe000640 1 -- 172.21.15.123:0/2190846644 >> 172.21.15.123:0/2190846644 conn(0x7f9cb80ff910 msgr2=0x7f9cb8100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:26.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.825+0000 7f9cbe000640 1 -- 172.21.15.123:0/2190846644 shutdown_connections 2024-09-19T04:52:26.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:26.826+0000 7f9cbe000640 1 -- 172.21.15.123:0/2190846644 wait complete. 2024-09-19T04:52:27.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:27 smithi193 ceph-mon[35592]: pgmap v175: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.5 MiB/s wr, 451 op/s 2024-09-19T04:52:27.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:27 smithi183 ceph-mon[35333]: pgmap v175: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.5 MiB/s wr, 451 op/s 2024-09-19T04:52:27.644 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:27 smithi123 ceph-mon[30511]: pgmap v175: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.5 MiB/s wr, 451 op/s 2024-09-19T04:52:27.681 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:52:28.062 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:52:28.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:28 smithi193 ceph-mon[35592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:52:28.498 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:28 smithi123 ceph-mon[30511]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:52:28.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:28 smithi183 ceph-mon[35333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:52:28.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.617+0000 7f4696c00640 1 -- 172.21.15.123:0/1872970722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4690103e30 msgr2=0x7f4690104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:28.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.617+0000 7f4696c00640 1 --2- 172.21.15.123:0/1872970722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4690103e30 0x7f4690104210 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4684009900 tx=0x7f468402a410 comp rx=0 tx=0).stop 2024-09-19T04:52:28.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.617+0000 7f4696c00640 1 -- 172.21.15.123:0/1872970722 shutdown_connections 2024-09-19T04:52:28.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.617+0000 7f4696c00640 1 --2- 172.21.15.123:0/1872970722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4690108f60 0x7f4690110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.617+0000 7f4696c00640 1 --2- 172.21.15.123:0/1872970722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46901047e0 0x7f4690108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.617+0000 7f4696c00640 1 --2- 172.21.15.123:0/1872970722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4690103e30 0x7f4690104210 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.617+0000 7f4696c00640 1 -- 172.21.15.123:0/1872970722 >> 172.21.15.123:0/1872970722 conn(0x7f46900ff8d0 msgr2=0x7f4690101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:28.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.617+0000 7f4696c00640 1 -- 172.21.15.123:0/1872970722 shutdown_connections 2024-09-19T04:52:28.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.617+0000 7f4696c00640 1 -- 172.21.15.123:0/1872970722 wait complete. 2024-09-19T04:52:28.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.618+0000 7f4696c00640 1 Processor -- start 2024-09-19T04:52:28.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.618+0000 7f4696c00640 1 -- start start 2024-09-19T04:52:28.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.618+0000 7f4696c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4690103e30 0x7f469019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.618+0000 7f4696c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46901047e0 0x7f469019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.618+0000 7f4696c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4690108f60 0x7f469019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.618+0000 7f4696c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4690113fd0 con 0x7f4690103e30 2024-09-19T04:52:28.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.618+0000 7f4696c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4690113e50 con 0x7f46901047e0 2024-09-19T04:52:28.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.618+0000 7f4696c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4690114150 con 0x7f4690108f60 2024-09-19T04:52:28.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.619+0000 7f4696200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4690108f60 0x7f469019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.619+0000 7f4695800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4690103e30 0x7f469019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.619+0000 7f4694e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46901047e0 0x7f469019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.619+0000 7f4695800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4690103e30 0x7f469019a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46106/0 (socket says 172.21.15.123:46106) 2024-09-19T04:52:28.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.619+0000 7f4695800640 1 -- 172.21.15.123:0/2320672625 learned_addr learned my addr 172.21.15.123:0/2320672625 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:28.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.619+0000 7f4696200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4690108f60 0x7f469019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34602/0 (socket says 172.21.15.123:34602) 2024-09-19T04:52:28.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.619+0000 7f4696200640 1 -- 172.21.15.123:0/2320672625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46901047e0 msgr2=0x7f469019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:28.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.619+0000 7f4696200640 1 --2- 172.21.15.123:0/2320672625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46901047e0 0x7f469019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.619+0000 7f4696200640 1 -- 172.21.15.123:0/2320672625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4690103e30 msgr2=0x7f469019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:28.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.619+0000 7f4696200640 1 --2- 172.21.15.123:0/2320672625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4690103e30 0x7f469019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.619+0000 7f4696200640 1 -- 172.21.15.123:0/2320672625 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469019f420 con 0x7f4690108f60 2024-09-19T04:52:28.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.619+0000 7f4696200640 1 --2- 172.21.15.123:0/2320672625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4690108f60 0x7f469019ed40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4688009fd0 tx=0x7f468800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:28.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.620+0000 7f468e000640 1 -- 172.21.15.123:0/2320672625 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468800ce20 con 0x7f4690108f60 2024-09-19T04:52:28.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.620+0000 7f4696c00640 1 -- 172.21.15.123:0/2320672625 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469019f710 con 0x7f4690108f60 2024-09-19T04:52:28.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.620+0000 7f468e000640 1 -- 172.21.15.123:0/2320672625 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4688005020 con 0x7f4690108f60 2024-09-19T04:52:28.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.620+0000 7f468e000640 1 -- 172.21.15.123:0/2320672625 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688010720 con 0x7f4690108f60 2024-09-19T04:52:28.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.620+0000 7f4696c00640 1 -- 172.21.15.123:0/2320672625 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46901a6ff0 con 0x7f4690108f60 2024-09-19T04:52:28.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.623+0000 7f469ae23740 1 Processor -- start 2024-09-19T04:52:28.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.623+0000 7f468e000640 1 -- 172.21.15.123:0/2320672625 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4688005540 con 0x7f4690108f60 2024-09-19T04:52:28.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.623+0000 7f469ae23740 1 -- start start 2024-09-19T04:52:28.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.623+0000 7f469ae23740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bc7e40 0x55fad0bc8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.623+0000 7f469ae23740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bc8760 0x55fad0bd2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.623+0000 7f469ae23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bd3020 0x55fad0bd5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.623+0000 7f469ae23740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fad0b2e3e0 con 0x55fad0bc8760 2024-09-19T04:52:28.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.623+0000 7f469ae23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fad0ad5d80 con 0x55fad0bd3020 2024-09-19T04:52:28.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.623+0000 7f469ae23740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fad0b2a3a0 con 0x55fad0bc7e40 2024-09-19T04:52:28.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.624+0000 7f4694e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bc7e40 0x55fad0bc8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.624+0000 7f4694e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bc7e40 0x55fad0bc8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34612/0 (socket says 172.21.15.123:34612) 2024-09-19T04:52:28.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.624+0000 7f4694e00640 1 -- 172.21.15.123:0/2894425302 learned_addr learned my addr 172.21.15.123:0/2894425302 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:28.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.624+0000 7f4695800640 1 --2- 172.21.15.123:0/2894425302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bc8760 0x55fad0bd2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.624+0000 7f4694e00640 1 --2- 172.21.15.123:0/2894425302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bd3020 0x55fad0bd5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.624+0000 7f4695800640 1 -- 172.21.15.123:0/2894425302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bc7e40 msgr2=0x55fad0bc8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:28.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.624+0000 7f4695800640 1 --2- 172.21.15.123:0/2894425302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bc7e40 0x55fad0bc8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.624+0000 7f4695800640 1 -- 172.21.15.123:0/2894425302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bd3020 msgr2=0x55fad0bd5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:28.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.624+0000 7f4695800640 1 --2- 172.21.15.123:0/2894425302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bd3020 0x55fad0bd5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.624+0000 7f4695800640 1 -- 172.21.15.123:0/2894425302 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad0af4ae0 con 0x55fad0bc8760 2024-09-19T04:52:28.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.624+0000 7f468e000640 1 --2- 172.21.15.123:0/2320672625 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46780a33b0 0x7f46780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.625+0000 7f468e000640 1 -- 172.21.15.123:0/2320672625 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f46880d03c0 con 0x7f4690108f60 2024-09-19T04:52:28.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.625+0000 7f4695800640 1 --2- 172.21.15.123:0/2894425302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bc8760 0x55fad0bd2ae0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4684031550 tx=0x7f4684005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcdaf33a21c40d52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:28.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.625+0000 7f4694e00640 1 --2- 172.21.15.123:0/2320672625 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46780a33b0 0x7f46780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.625+0000 7f4682a00640 1 -- 172.21.15.123:0/2894425302 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684040070 con 0x55fad0bc8760 2024-09-19T04:52:28.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.625+0000 7f4682a00640 1 -- 172.21.15.123:0/2894425302 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4684004480 con 0x55fad0bc8760 2024-09-19T04:52:28.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.625+0000 7f4682a00640 1 -- 172.21.15.123:0/2894425302 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684035040 con 0x55fad0bc8760 2024-09-19T04:52:28.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.626+0000 7f469ae23740 1 -- 172.21.15.123:0/2894425302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bc8760 msgr2=0x55fad0bd2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:28.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.626+0000 7f469ae23740 1 --2- 172.21.15.123:0/2894425302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bc8760 0x55fad0bd2ae0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4684031550 tx=0x7f4684005c00 comp rx=0 tx=0).stop 2024-09-19T04:52:28.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.626+0000 7f4694e00640 1 --2- 172.21.15.123:0/2320672625 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46780a33b0 0x7f46780a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f467c005d00 tx=0x7f467c004730 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:28.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.626+0000 7f469ae23740 1 -- 172.21.15.123:0/2894425302 shutdown_connections 2024-09-19T04:52:28.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.626+0000 7f469ae23740 1 --2- 172.21.15.123:0/2894425302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bd3020 0x55fad0bd5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.626+0000 7f469ae23740 1 --2- 172.21.15.123:0/2894425302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bc8760 0x55fad0bd2ae0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.626+0000 7f469ae23740 1 --2- 172.21.15.123:0/2894425302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bc7e40 0x55fad0bc8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.626+0000 7f469ae23740 1 -- 172.21.15.123:0/2894425302 >> 172.21.15.123:0/2894425302 conn(0x55fad0bc43d0 msgr2=0x55fad0bc47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.626+0000 7f469ae23740 1 -- 172.21.15.123:0/2894425302 shutdown_connections 2024-09-19T04:52:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.626+0000 7f469ae23740 1 -- 172.21.15.123:0/2894425302 wait complete. 2024-09-19T04:52:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.626+0000 7f469ae23740 1 -- start start 2024-09-19T04:52:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.627+0000 7f469ae23740 1 Processor -- start 2024-09-19T04:52:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.627+0000 7f469ae23740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bc7e40 0x55fad0bcff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.627+0000 7f469ae23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bc8760 0x55fad0bd04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.627+0000 7f469ae23740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bd3020 0x55fad0bd0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.627+0000 7f469ae23740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fad0bd5bf0 con 0x55fad0bc7e40 2024-09-19T04:52:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.627+0000 7f469ae23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fad0ae90f0 con 0x55fad0bc8760 2024-09-19T04:52:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.627+0000 7f469ae23740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fad0bd5d70 con 0x55fad0bd3020 2024-09-19T04:52:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.627+0000 7f469ae23740 10 client.0 _ll_register_callbacks cb 0x55fad0ad0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:52:28.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.627+0000 7f4696200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bd3020 0x55fad0bd0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.628+0000 7f4694e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bc8760 0x55fad0bd04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.628+0000 7f4695800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bc7e40 0x55fad0bcff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.628+0000 7f4696200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bd3020 0x55fad0bd0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34628/0 (socket says 172.21.15.123:34628) 2024-09-19T04:52:28.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.628+0000 7f4696200640 1 -- 172.21.15.123:0/3908210833 learned_addr learned my addr 172.21.15.123:0/3908210833 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:28.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.628+0000 7f4696200640 1 -- 172.21.15.123:0/3908210833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bc8760 msgr2=0x55fad0bd04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:28.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.628+0000 7f4696200640 1 --2- 172.21.15.123:0/3908210833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bc8760 0x55fad0bd04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.628+0000 7f4696200640 1 -- 172.21.15.123:0/3908210833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bc7e40 msgr2=0x55fad0bcff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:28.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.628+0000 7f4696200640 1 --2- 172.21.15.123:0/3908210833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bc7e40 0x55fad0bcff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.628+0000 7f4696200640 1 -- 172.21.15.123:0/3908210833 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad089f3c0 con 0x55fad0bd3020 2024-09-19T04:52:28.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.628+0000 7f4696200640 1 --2- 172.21.15.123:0/3908210833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bd3020 0x55fad0bd0a10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f468809c600 tx=0x7f468809cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5b0118ebde459f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:28.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.628+0000 7f4677400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:52:28.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.628+0000 7f4677400640 1 -- 172.21.15.123:0/3908210833 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46880a52e0 con 0x55fad0bd3020 2024-09-19T04:52:28.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.628+0000 7f4677400640 1 -- 172.21.15.123:0/3908210833 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f4688098ce0 con 0x55fad0bd3020 2024-09-19T04:52:28.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.628+0000 7f469ae23740 1 -- 172.21.15.123:0/3908210833 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4684004480 con 0x55fad0bd3020 2024-09-19T04:52:28.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.629+0000 7f4677400640 1 -- 172.21.15.123:0/3908210833 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f46880a4e50 con 0x55fad0bd3020 2024-09-19T04:52:28.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.629+0000 7f469ae23740 10 client.26410 fetch_fsmap learned FSMap version 37 2024-09-19T04:52:28.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.629+0000 7f469ae23740 1 -- 172.21.15.123:0/3908210833 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fad0be1ce0 con 0x55fad0bd3020 2024-09-19T04:52:28.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.629+0000 7f4677400640 1 -- 172.21.15.123:0/3908210833 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4684004480 con 0x55fad0bd3020 2024-09-19T04:52:28.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.629+0000 7f4677400640 1 -- 172.21.15.123:0/3908210833 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f468807d020 con 0x55fad0bd3020 2024-09-19T04:52:28.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.629+0000 7f469ae23740 10 client.26410 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:52:28.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.629+0000 7f469ae23740 10 client.26410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:52:28.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.629+0000 7f469ae23740 20 client.26410 populate_metadata read hostname 'smithi123' 2024-09-19T04:52:28.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.629+0000 7f469ae23740 1 --2- 172.21.15.123:0/3908210833 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55fad0be30f0 0x55fad0be5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.629+0000 7f469ae23740 4 client.26410 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:52:28.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.629+0000 7f469ae23740 1 -- 172.21.15.123:0/3908210833 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fad0b3b950 con 0x55fad0be30f0 2024-09-19T04:52:28.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.630+0000 7f4695800640 1 --2- 172.21.15.123:0/3908210833 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55fad0be30f0 0x55fad0be5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.630+0000 7f4695800640 1 --2- 172.21.15.123:0/3908210833 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55fad0be30f0 0x55fad0be5540 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb7b0d162949ebb server_cookie=d06672c92a5d6f5b in_seq=0 out_seq=0 2024-09-19T04:52:28.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.630+0000 7f4677400640 10 client.26410 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:52:28.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.631+0000 7f4677400640 1 -- 172.21.15.123:0/3908210833 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fad0b3b950 con 0x55fad0be30f0 2024-09-19T04:52:28.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.631+0000 7f4677400640 10 client.26410 handle_command_reply: tid=0 2024-09-19T04:52:28.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.631+0000 7f469ae23740 1 client.26410 shutdown 2024-09-19T04:52:28.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.631+0000 7f469ae23740 1 -- 172.21.15.123:0/3908210833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bd3020 msgr2=0x55fad0bd0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:28.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.631+0000 7f469ae23740 1 --2- 172.21.15.123:0/3908210833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bd3020 0x55fad0bd0a10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f468809c600 tx=0x7f468809cd70 comp rx=0 tx=0).stop 2024-09-19T04:52:28.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.631+0000 7f469ae23740 1 -- 172.21.15.123:0/3908210833 shutdown_connections 2024-09-19T04:52:28.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.631+0000 7f469ae23740 1 --2- 172.21.15.123:0/3908210833 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55fad0be30f0 0x55fad0be5540 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.631+0000 7f469ae23740 1 --2- 172.21.15.123:0/3908210833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bd3020 0x55fad0bd0a10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.631+0000 7f469ae23740 1 --2- 172.21.15.123:0/3908210833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bc8760 0x55fad0bd04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.631+0000 7f4677400640 0 client.26410 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:52:28.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.631+0000 7f469ae23740 1 --2- 172.21.15.123:0/3908210833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bc7e40 0x55fad0bcff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.631+0000 7f469ae23740 1 -- 172.21.15.123:0/3908210833 >> 172.21.15.123:0/3908210833 conn(0x55fad0bc43d0 msgr2=0x55fad0bca0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:28.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.631+0000 7f469ae23740 1 -- 172.21.15.123:0/3908210833 shutdown_connections 2024-09-19T04:52:28.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.632+0000 7f469ae23740 1 -- 172.21.15.123:0/3908210833 wait complete. 2024-09-19T04:52:28.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.632+0000 7f469ae23740 20 client.26410 trim_cache size 0 max 16384 2024-09-19T04:52:28.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.645+0000 7f469ae23740 1 Processor -- start 2024-09-19T04:52:28.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.645+0000 7f469ae23740 1 -- start start 2024-09-19T04:52:28.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.645+0000 7f469ae23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bc7e40 0x55fad0bca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.645+0000 7f469ae23740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bc8760 0x55fad0bbf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.645+0000 7f469ae23740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bd3020 0x55fad0beae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.645+0000 7f469ae23740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46901af7e0 con 0x55fad0bc8760 2024-09-19T04:52:28.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.645+0000 7f469ae23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46901b1320 con 0x55fad0bc7e40 2024-09-19T04:52:28.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.645+0000 7f469ae23740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f46901afeb0 con 0x55fad0bd3020 2024-09-19T04:52:28.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.646+0000 7f4696200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bd3020 0x55fad0beae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.646+0000 7f4694e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bc8760 0x55fad0bbf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.646+0000 7f4696200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bd3020 0x55fad0beae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34644/0 (socket says 172.21.15.123:34644) 2024-09-19T04:52:28.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.646+0000 7f4696200640 1 -- 172.21.15.123:0/2976083651 learned_addr learned my addr 172.21.15.123:0/2976083651 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:28.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.646+0000 7f4695800640 1 --2- 172.21.15.123:0/2976083651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bc7e40 0x55fad0bca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.646+0000 7f4696200640 1 -- 172.21.15.123:0/2976083651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bc7e40 msgr2=0x55fad0bca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:28.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.646+0000 7f4696200640 1 --2- 172.21.15.123:0/2976083651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bc7e40 0x55fad0bca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.646+0000 7f4696200640 1 -- 172.21.15.123:0/2976083651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bc8760 msgr2=0x55fad0bbf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:28.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.646+0000 7f4696200640 1 --2- 172.21.15.123:0/2976083651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bc8760 0x55fad0bbf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.646+0000 7f4696200640 1 -- 172.21.15.123:0/2976083651 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad0bcef40 con 0x55fad0bd3020 2024-09-19T04:52:28.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.646+0000 7f4696200640 1 --2- 172.21.15.123:0/2976083651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bd3020 0x55fad0beae10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f46880a4db0 tx=0x7f468809ab80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=217528161f617dfa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.647+0000 7f4682000640 1 -- 172.21.15.123:0/2976083651 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688084030 con 0x55fad0bd3020 2024-09-19T04:52:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.647+0000 7f4682000640 1 -- 172.21.15.123:0/2976083651 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46880b5070 con 0x55fad0bd3020 2024-09-19T04:52:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.647+0000 7f4682000640 1 -- 172.21.15.123:0/2976083651 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688098ce0 con 0x55fad0bd3020 2024-09-19T04:52:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.647+0000 7f469ae23740 1 -- 172.21.15.123:0/2976083651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bd3020 msgr2=0x55fad0beae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.647+0000 7f469ae23740 1 --2- 172.21.15.123:0/2976083651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bd3020 0x55fad0beae10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f46880a4db0 tx=0x7f468809ab80 comp rx=0 tx=0).stop 2024-09-19T04:52:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.647+0000 7f469ae23740 1 -- 172.21.15.123:0/2976083651 shutdown_connections 2024-09-19T04:52:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.647+0000 7f469ae23740 1 --2- 172.21.15.123:0/2976083651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bd3020 0x55fad0beae10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.647+0000 7f469ae23740 1 --2- 172.21.15.123:0/2976083651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bc8760 0x55fad0bbf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.647+0000 7f469ae23740 1 --2- 172.21.15.123:0/2976083651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bc7e40 0x55fad0bca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.647+0000 7f469ae23740 1 -- 172.21.15.123:0/2976083651 >> 172.21.15.123:0/2976083651 conn(0x55fad0bc43d0 msgr2=0x55fad0bc34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.647+0000 7f469ae23740 1 -- 172.21.15.123:0/2976083651 shutdown_connections 2024-09-19T04:52:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.647+0000 7f469ae23740 1 -- 172.21.15.123:0/2976083651 wait complete. 2024-09-19T04:52:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.648+0000 7f469ae23740 1 -- start start 2024-09-19T04:52:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.648+0000 7f469ae23740 1 Processor -- start 2024-09-19T04:52:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.648+0000 7f469ae23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bc7e40 0x55fad0bd6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.648+0000 7f469ae23740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bc8760 0x55fad0bd7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.648+0000 7f469ae23740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bd3020 0x55fad0bd7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.648+0000 7f469ae23740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46901b0c50 con 0x55fad0bd3020 2024-09-19T04:52:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.648+0000 7f469ae23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46901b0580 con 0x55fad0bc7e40 2024-09-19T04:52:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.648+0000 7f469ae23740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f46901ad7c0 con 0x55fad0bc8760 2024-09-19T04:52:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.648+0000 7f469ae23740 10 client.0 _ll_register_callbacks cb 0x7f4688098250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:52:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.649+0000 7f4694e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bc8760 0x55fad0bd7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.649+0000 7f4694e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bc8760 0x55fad0bd7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34660/0 (socket says 172.21.15.123:34660) 2024-09-19T04:52:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.649+0000 7f4694e00640 1 -- 172.21.15.123:0/4082868869 learned_addr learned my addr 172.21.15.123:0/4082868869 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.649+0000 7f4696200640 1 --2- 172.21.15.123:0/4082868869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bd3020 0x55fad0bd7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.649+0000 7f4695800640 1 --2- 172.21.15.123:0/4082868869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bc7e40 0x55fad0bd6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.649+0000 7f4694e00640 1 -- 172.21.15.123:0/4082868869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bc7e40 msgr2=0x55fad0bd6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:28.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.649+0000 7f4694e00640 1 --2- 172.21.15.123:0/4082868869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bc7e40 0x55fad0bd6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.649+0000 7f4694e00640 1 -- 172.21.15.123:0/4082868869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bd3020 msgr2=0x55fad0bd7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:28.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.649+0000 7f4694e00640 1 --2- 172.21.15.123:0/4082868869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bd3020 0x55fad0bd7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.649+0000 7f4694e00640 1 -- 172.21.15.123:0/4082868869 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad0c4d080 con 0x55fad0bc8760 2024-09-19T04:52:28.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.649+0000 7f4694e00640 1 --2- 172.21.15.123:0/4082868869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bc8760 0x55fad0bd7520 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55fad0bcf820 tx=0x7f467c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f1e06edbc3be77f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:28.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.649+0000 7f4677400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:52:28.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.650+0000 7f4677400640 1 -- 172.21.15.123:0/4082868869 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c005220 con 0x55fad0bc8760 2024-09-19T04:52:28.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.650+0000 7f4677400640 1 -- 172.21.15.123:0/4082868869 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f467c0053c0 con 0x55fad0bc8760 2024-09-19T04:52:28.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.650+0000 7f469ae23740 1 -- 172.21.15.123:0/4082868869 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46880b5070 con 0x55fad0bc8760 2024-09-19T04:52:28.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.650+0000 7f4677400640 1 -- 172.21.15.123:0/4082868869 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f467c0055c0 con 0x55fad0bc8760 2024-09-19T04:52:28.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.650+0000 7f469ae23740 10 client.26416 fetch_fsmap learned FSMap version 37 2024-09-19T04:52:28.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.650+0000 7f469ae23740 1 -- 172.21.15.123:0/4082868869 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fad0c4e3c0 con 0x55fad0bc8760 2024-09-19T04:52:28.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.650+0000 7f4677400640 1 -- 172.21.15.123:0/4082868869 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46880b5070 con 0x55fad0bc8760 2024-09-19T04:52:28.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.650+0000 7f4677400640 1 -- 172.21.15.123:0/4082868869 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f467c029070 con 0x55fad0bc8760 2024-09-19T04:52:28.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.650+0000 7f469ae23740 10 client.26416 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:52:28.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.650+0000 7f469ae23740 10 client.26416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:52:28.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.650+0000 7f469ae23740 20 client.26416 populate_metadata read hostname 'smithi123' 2024-09-19T04:52:28.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.650+0000 7f469ae23740 1 --2- 172.21.15.123:0/4082868869 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55fad0c4ebc0 0x55fad0c50fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:28.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.650+0000 7f469ae23740 4 client.26416 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:52:28.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.650+0000 7f469ae23740 1 -- 172.21.15.123:0/4082868869 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fad0b35e60 con 0x55fad0c4ebc0 2024-09-19T04:52:28.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.651+0000 7f4695800640 1 --2- 172.21.15.123:0/4082868869 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55fad0c4ebc0 0x55fad0c50fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:28.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.651+0000 7f4695800640 1 --2- 172.21.15.123:0/4082868869 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55fad0c4ebc0 0x55fad0c50fe0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d582ccd1a8775f server_cookie=aab39a97c008f294 in_seq=0 out_seq=0 2024-09-19T04:52:28.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.651+0000 7f4677400640 10 client.26416 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:52:28.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.651+0000 7f4677400640 1 -- 172.21.15.123:0/4082868869 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fad0b35e60 con 0x55fad0c4ebc0 2024-09-19T04:52:28.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.652+0000 7f4677400640 10 client.26416 handle_command_reply: tid=0 2024-09-19T04:52:28.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.652+0000 7f469ae23740 1 client.26416 shutdown 2024-09-19T04:52:28.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.652+0000 7f469ae23740 1 -- 172.21.15.123:0/4082868869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bc8760 msgr2=0x55fad0bd7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:28.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.652+0000 7f469ae23740 1 --2- 172.21.15.123:0/4082868869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bc8760 0x55fad0bd7520 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55fad0bcf820 tx=0x7f467c0046c0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.652+0000 7f469ae23740 1 -- 172.21.15.123:0/4082868869 shutdown_connections 2024-09-19T04:52:28.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.652+0000 7f469ae23740 1 --2- 172.21.15.123:0/4082868869 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55fad0c4ebc0 0x55fad0c50fe0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.652+0000 7f469ae23740 1 --2- 172.21.15.123:0/4082868869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fad0bd3020 0x55fad0bd7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.652+0000 7f469ae23740 1 --2- 172.21.15.123:0/4082868869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad0bc8760 0x55fad0bd7520 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.652+0000 7f469ae23740 1 --2- 172.21.15.123:0/4082868869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fad0bc7e40 0x55fad0bd6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.652+0000 7f469ae23740 1 -- 172.21.15.123:0/4082868869 >> 172.21.15.123:0/4082868869 conn(0x55fad0bc43d0 msgr2=0x55fad0bcff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:28.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.652+0000 7f4677400640 0 client.26416 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:52:28.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.652+0000 7f469ae23740 1 -- 172.21.15.123:0/4082868869 shutdown_connections 2024-09-19T04:52:28.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.652+0000 7f469ae23740 1 -- 172.21.15.123:0/4082868869 wait complete. 2024-09-19T04:52:28.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.652+0000 7f469ae23740 20 client.26416 trim_cache size 0 max 16384 2024-09-19T04:52:28.657 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:52:28.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.653+0000 7f4696c00640 1 -- 172.21.15.123:0/2320672625 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46780a33b0 msgr2=0x7f46780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:28.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.653+0000 7f4696c00640 1 --2- 172.21.15.123:0/2320672625 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46780a33b0 0x7f46780a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f467c005d00 tx=0x7f467c004730 comp rx=0 tx=0).stop 2024-09-19T04:52:28.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.653+0000 7f4696c00640 1 -- 172.21.15.123:0/2320672625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4690108f60 msgr2=0x7f469019ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:28.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.653+0000 7f4696c00640 1 --2- 172.21.15.123:0/2320672625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4690108f60 0x7f469019ed40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4688009fd0 tx=0x7f468800ef10 comp rx=0 tx=0).stop 2024-09-19T04:52:28.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.653+0000 7f4696c00640 1 -- 172.21.15.123:0/2320672625 shutdown_connections 2024-09-19T04:52:28.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.653+0000 7f4696c00640 1 --2- 172.21.15.123:0/2320672625 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46780a33b0 0x7f46780a5870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.653+0000 7f4696c00640 1 --2- 172.21.15.123:0/2320672625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4690108f60 0x7f469019ed40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.653+0000 7f4696c00640 1 --2- 172.21.15.123:0/2320672625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46901047e0 0x7f469019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.653+0000 7f4696c00640 1 --2- 172.21.15.123:0/2320672625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4690103e30 0x7f469019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:28.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.653+0000 7f4696c00640 1 -- 172.21.15.123:0/2320672625 >> 172.21.15.123:0/2320672625 conn(0x7f46900ff8d0 msgr2=0x7f4690100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:28.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.653+0000 7f4696c00640 1 -- 172.21.15.123:0/2320672625 shutdown_connections 2024-09-19T04:52:28.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:28.653+0000 7f4696c00640 1 -- 172.21.15.123:0/2320672625 wait complete. 2024-09-19T04:52:29.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:29 smithi123 ceph-mon[30511]: pgmap v176: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 2.8 MiB/s wr, 613 op/s 2024-09-19T04:52:29.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:29 smithi193 ceph-mon[35592]: pgmap v176: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 2.8 MiB/s wr, 613 op/s 2024-09-19T04:52:29.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:29 smithi183 ceph-mon[35333]: pgmap v176: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 2.8 MiB/s wr, 613 op/s 2024-09-19T04:52:30.593 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:52:30.593 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:52:30.978 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:52:31.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:31 smithi123 ceph-mon[30511]: pgmap v177: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 356 op/s 2024-09-19T04:52:31.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:31 smithi193 ceph-mon[35592]: pgmap v177: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 356 op/s 2024-09-19T04:52:31.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.573+0000 7fa982400640 1 -- 172.21.15.123:0/2224344283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa97c1079c0 msgr2=0x7fa97c107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:31.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.573+0000 7fa982400640 1 --2- 172.21.15.123:0/2224344283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa97c1079c0 0x7fa97c107da0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa970009930 tx=0x7fa97002a420 comp rx=0 tx=0).stop 2024-09-19T04:52:31.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.573+0000 7fa982400640 1 -- 172.21.15.123:0/2224344283 shutdown_connections 2024-09-19T04:52:31.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.573+0000 7fa982400640 1 --2- 172.21.15.123:0/2224344283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa97c102360 0x7fa97c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.573+0000 7fa982400640 1 --2- 172.21.15.123:0/2224344283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa97c1019c0 0x7fa97c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.573+0000 7fa982400640 1 --2- 172.21.15.123:0/2224344283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa97c1079c0 0x7fa97c107da0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.573+0000 7fa982400640 1 -- 172.21.15.123:0/2224344283 >> 172.21.15.123:0/2224344283 conn(0x7fa97c0fd740 msgr2=0x7fa97c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:31.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.573+0000 7fa982400640 1 -- 172.21.15.123:0/2224344283 shutdown_connections 2024-09-19T04:52:31.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.573+0000 7fa982400640 1 -- 172.21.15.123:0/2224344283 wait complete. 2024-09-19T04:52:31.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.574+0000 7fa982400640 1 Processor -- start 2024-09-19T04:52:31.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.574+0000 7fa982400640 1 -- start start 2024-09-19T04:52:31.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.574+0000 7fa982400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa97c1019c0 0x7fa97c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.574+0000 7fa982400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa97c102360 0x7fa97c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.575+0000 7fa982400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa97c1079c0 0x7fa97c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.575+0000 7fa982400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c111e10 con 0x7fa97c102360 2024-09-19T04:52:31.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.575+0000 7fa982400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c111c90 con 0x7fa97c1019c0 2024-09-19T04:52:31.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.575+0000 7fa982400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c111f90 con 0x7fa97c1079c0 2024-09-19T04:52:31.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.575+0000 7fa981a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa97c1079c0 0x7fa97c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.575+0000 7fa981a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa97c1079c0 0x7fa97c19edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34674/0 (socket says 172.21.15.123:34674) 2024-09-19T04:52:31.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.575+0000 7fa97be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa97c102360 0x7fa97c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.575+0000 7fa981a00640 1 -- 172.21.15.123:0/204576244 learned_addr learned my addr 172.21.15.123:0/204576244 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:31.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.575+0000 7fa981000640 1 --2- 172.21.15.123:0/204576244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa97c1019c0 0x7fa97c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.575+0000 7fa981a00640 1 -- 172.21.15.123:0/204576244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa97c1019c0 msgr2=0x7fa97c19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:31.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.575+0000 7fa981a00640 1 --2- 172.21.15.123:0/204576244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa97c1019c0 0x7fa97c19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.575+0000 7fa981a00640 1 -- 172.21.15.123:0/204576244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa97c102360 msgr2=0x7fa97c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:31.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.575+0000 7fa981a00640 1 --2- 172.21.15.123:0/204576244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa97c102360 0x7fa97c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.575+0000 7fa981a00640 1 -- 172.21.15.123:0/204576244 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97c19f4a0 con 0x7fa97c1079c0 2024-09-19T04:52:31.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.576+0000 7fa981a00640 1 --2- 172.21.15.123:0/204576244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa97c1079c0 0x7fa97c19edc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa974009fd0 tx=0x7fa97400b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:31.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.576+0000 7fa979600640 1 -- 172.21.15.123:0/204576244 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974017070 con 0x7fa97c1079c0 2024-09-19T04:52:31.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.576+0000 7fa979600640 1 -- 172.21.15.123:0/204576244 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa974003e50 con 0x7fa97c1079c0 2024-09-19T04:52:31.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.576+0000 7fa982400640 1 -- 172.21.15.123:0/204576244 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97c19f790 con 0x7fa97c1079c0 2024-09-19T04:52:31.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.576+0000 7fa979600640 1 -- 172.21.15.123:0/204576244 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9740106c0 con 0x7fa97c1079c0 2024-09-19T04:52:31.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.576+0000 7fa982400640 1 -- 172.21.15.123:0/204576244 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa97c1a7070 con 0x7fa97c1079c0 2024-09-19T04:52:31.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.580+0000 7fa979600640 1 -- 172.21.15.123:0/204576244 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa974003970 con 0x7fa97c1079c0 2024-09-19T04:52:31.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.580+0000 7fa9865c7740 1 Processor -- start 2024-09-19T04:52:31.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.580+0000 7fa9865c7740 1 -- start start 2024-09-19T04:52:31.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.580+0000 7fa9865c7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 0x558f812dc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.580+0000 7fa9865c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812dc5d0 0x558f812e6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.581+0000 7fa9865c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812e6e90 0x558f812e9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.581+0000 7fa9865c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558f81242290 con 0x558f812e6e90 2024-09-19T04:52:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.581+0000 7fa9865c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558f811e9b40 con 0x558f812dc5d0 2024-09-19T04:52:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.581+0000 7fa9865c7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558f8123e250 con 0x558f812dbcb0 2024-09-19T04:52:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.581+0000 7fa981000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812dc5d0 0x558f812e6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.581+0000 7fa981000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812dc5d0 0x558f812e6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:56270/0 (socket says 172.21.15.123:56270) 2024-09-19T04:52:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.581+0000 7fa981000640 1 -- 172.21.15.123:0/2779543420 learned_addr learned my addr 172.21.15.123:0/2779543420 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.581+0000 7fa97be00640 1 --2- 172.21.15.123:0/2779543420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 0x558f812dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.581+0000 7fa97be00640 1 --2- 172.21.15.123:0/2779543420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812e6e90 0x558f812e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.581+0000 7fa981000640 1 -- 172.21.15.123:0/2779543420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 msgr2=0x558f812dc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.581+0000 7fa981000640 1 --2- 172.21.15.123:0/2779543420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 0x558f812dc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.581+0000 7fa981000640 1 -- 172.21.15.123:0/2779543420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812e6e90 msgr2=0x558f812e9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.581+0000 7fa981000640 1 --2- 172.21.15.123:0/2779543420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812e6e90 0x558f812e9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.581+0000 7fa981000640 1 -- 172.21.15.123:0/2779543420 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f80fb36a0 con 0x558f812dc5d0 2024-09-19T04:52:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.582+0000 7fa981000640 1 --2- 172.21.15.123:0/2779543420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812dc5d0 0x558f812e6950 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa970009930 tx=0x7fa970005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=686869339c067d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.582+0000 7fa979600640 1 --2- 172.21.15.123:0/204576244 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa9640a3430 0x7fa9640a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.582+0000 7fa979600640 1 -- 172.21.15.123:0/204576244 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fa9740cf940 con 0x7fa97c1079c0 2024-09-19T04:52:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.582+0000 7fa96e000640 1 -- 172.21.15.123:0/2779543420 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970035040 con 0x558f812dc5d0 2024-09-19T04:52:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.582+0000 7fa97be00640 1 --2- 172.21.15.123:0/204576244 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa9640a3430 0x7fa9640a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.582+0000 7fa96e000640 1 -- 172.21.15.123:0/2779543420 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa970004480 con 0x558f812dc5d0 2024-09-19T04:52:31.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.582+0000 7fa96e000640 1 -- 172.21.15.123:0/2779543420 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9700035f0 con 0x558f812dc5d0 2024-09-19T04:52:31.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.583+0000 7fa9865c7740 1 -- 172.21.15.123:0/2779543420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812dc5d0 msgr2=0x558f812e6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:31.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.583+0000 7fa9865c7740 1 --2- 172.21.15.123:0/2779543420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812dc5d0 0x558f812e6950 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa970009930 tx=0x7fa970005bf0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.583+0000 7fa97be00640 1 --2- 172.21.15.123:0/204576244 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa9640a3430 0x7fa9640a58f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x558f812d95c0 tx=0x7fa968004730 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:31.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.583+0000 7fa9865c7740 1 -- 172.21.15.123:0/2779543420 shutdown_connections 2024-09-19T04:52:31.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.583+0000 7fa9865c7740 1 --2- 172.21.15.123:0/2779543420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812e6e90 0x558f812e9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.583+0000 7fa9865c7740 1 --2- 172.21.15.123:0/2779543420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812dc5d0 0x558f812e6950 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.583+0000 7fa9865c7740 1 --2- 172.21.15.123:0/2779543420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 0x558f812dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.583+0000 7fa9865c7740 1 -- 172.21.15.123:0/2779543420 >> 172.21.15.123:0/2779543420 conn(0x558f812d8240 msgr2=0x558f812d8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:31.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.583+0000 7fa9865c7740 1 -- 172.21.15.123:0/2779543420 shutdown_connections 2024-09-19T04:52:31.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.583+0000 7fa9865c7740 1 -- 172.21.15.123:0/2779543420 wait complete. 2024-09-19T04:52:31.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.584+0000 7fa9865c7740 1 -- start start 2024-09-19T04:52:31.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.584+0000 7fa9865c7740 1 Processor -- start 2024-09-19T04:52:31.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.584+0000 7fa9865c7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 0x558f812e6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.584+0000 7fa9865c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812dc5d0 0x558f812e3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.584+0000 7fa9865c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812e6e90 0x558f812e3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.584+0000 7fa9865c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558f812e9b00 con 0x558f812dc5d0 2024-09-19T04:52:31.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.584+0000 7fa9865c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558f811fcfa0 con 0x558f812e6e90 2024-09-19T04:52:31.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.584+0000 7fa9865c7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558f812e9c80 con 0x558f812dbcb0 2024-09-19T04:52:31.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.585+0000 7fa9865c7740 10 client.0 _ll_register_callbacks cb 0x558f80f50ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:52:31.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.585+0000 7fa981000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 0x558f812e6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.585+0000 7fa981000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 0x558f812e6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34688/0 (socket says 172.21.15.123:34688) 2024-09-19T04:52:31.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.585+0000 7fa981000640 1 -- 172.21.15.123:0/3893174001 learned_addr learned my addr 172.21.15.123:0/3893174001 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:31.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.585+0000 7fa97be00640 1 --2- 172.21.15.123:0/3893174001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812dc5d0 0x558f812e3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.585+0000 7fa981a00640 1 --2- 172.21.15.123:0/3893174001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812e6e90 0x558f812e3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.585+0000 7fa981000640 1 -- 172.21.15.123:0/3893174001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812e6e90 msgr2=0x558f812e3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:31.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.585+0000 7fa981000640 1 --2- 172.21.15.123:0/3893174001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812e6e90 0x558f812e3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.585+0000 7fa981000640 1 -- 172.21.15.123:0/3893174001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812dc5d0 msgr2=0x558f812e3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:31.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.585+0000 7fa981000640 1 --2- 172.21.15.123:0/3893174001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812dc5d0 0x558f812e3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.585+0000 7fa981000640 1 -- 172.21.15.123:0/3893174001 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f812e47d0 con 0x558f812dbcb0 2024-09-19T04:52:31.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.585+0000 7fa97be00640 1 --2- 172.21.15.123:0/3893174001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812dc5d0 0x558f812e3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:52:31.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.585+0000 7fa981000640 1 --2- 172.21.15.123:0/3893174001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 0x558f812e6790 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa97002a940 tx=0x7fa97002a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f295851aa258d420 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:31.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.586+0000 7fa962a00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:52:31.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.586+0000 7fa962a00640 1 -- 172.21.15.123:0/3893174001 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970053060 con 0x558f812dbcb0 2024-09-19T04:52:31.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.586+0000 7fa962a00640 1 -- 172.21.15.123:0/3893174001 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fa970004280 con 0x558f812dbcb0 2024-09-19T04:52:31.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.586+0000 7fa9865c7740 1 -- 172.21.15.123:0/3893174001 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa970004480 con 0x558f812dbcb0 2024-09-19T04:52:31.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.586+0000 7fa962a00640 1 -- 172.21.15.123:0/3893174001 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa970035050 con 0x558f812dbcb0 2024-09-19T04:52:31.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.586+0000 7fa9865c7740 10 client.26428 fetch_fsmap learned FSMap version 37 2024-09-19T04:52:31.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.586+0000 7fa9865c7740 1 -- 172.21.15.123:0/3893174001 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558f812e9b00 con 0x558f812dbcb0 2024-09-19T04:52:31.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.586+0000 7fa962a00640 1 -- 172.21.15.123:0/3893174001 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa970004480 con 0x558f812dbcb0 2024-09-19T04:52:31.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.587+0000 7fa962a00640 1 -- 172.21.15.123:0/3893174001 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fa9700039b0 con 0x558f812dbcb0 2024-09-19T04:52:31.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.587+0000 7fa9865c7740 10 client.26428 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:52:31.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.587+0000 7fa9865c7740 10 client.26428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:52:31.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.587+0000 7fa9865c7740 20 client.26428 populate_metadata read hostname 'smithi123' 2024-09-19T04:52:31.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.587+0000 7fa9865c7740 1 --2- 172.21.15.123:0/3893174001 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x558f812f6b60 0x558f812f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.587+0000 7fa9865c7740 4 client.26428 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:52:31.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.587+0000 7fa9865c7740 1 -- 172.21.15.123:0/3893174001 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f8124f800 con 0x558f812f6b60 2024-09-19T04:52:31.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.587+0000 7fa97be00640 1 --2- 172.21.15.123:0/3893174001 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x558f812f6b60 0x558f812f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.588+0000 7fa962a00640 10 client.26428 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:52:31.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.588+0000 7fa97be00640 1 --2- 172.21.15.123:0/3893174001 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x558f812f6b60 0x558f812f8f50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70f672bec3f6a454 server_cookie=f442af073adb3b5b in_seq=0 out_seq=0 2024-09-19T04:52:31.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.588+0000 7fa962a00640 1 -- 172.21.15.123:0/3893174001 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f8124f800 con 0x558f812f6b60 2024-09-19T04:52:31.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.588+0000 7fa962a00640 10 client.26428 handle_command_reply: tid=0 2024-09-19T04:52:31.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.588+0000 7fa9865c7740 1 client.26428 shutdown 2024-09-19T04:52:31.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.589+0000 7fa9865c7740 1 -- 172.21.15.123:0/3893174001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 msgr2=0x558f812e6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:31.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.589+0000 7fa9865c7740 1 --2- 172.21.15.123:0/3893174001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 0x558f812e6790 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa97002a940 tx=0x7fa97002a420 comp rx=0 tx=0).stop 2024-09-19T04:52:31.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.589+0000 7fa9865c7740 1 -- 172.21.15.123:0/3893174001 shutdown_connections 2024-09-19T04:52:31.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.589+0000 7fa9865c7740 1 --2- 172.21.15.123:0/3893174001 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x558f812f6b60 0x558f812f8f50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.589+0000 7fa9865c7740 1 --2- 172.21.15.123:0/3893174001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812e6e90 0x558f812e3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.589+0000 7fa9865c7740 1 --2- 172.21.15.123:0/3893174001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812dc5d0 0x558f812e3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.589+0000 7fa9865c7740 1 --2- 172.21.15.123:0/3893174001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 0x558f812e6790 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.589+0000 7fa962a00640 0 client.26428 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:52:31.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.589+0000 7fa9865c7740 1 -- 172.21.15.123:0/3893174001 >> 172.21.15.123:0/3893174001 conn(0x558f812d8240 msgr2=0x558f812ddf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:31.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.589+0000 7fa9865c7740 1 -- 172.21.15.123:0/3893174001 shutdown_connections 2024-09-19T04:52:31.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.589+0000 7fa9865c7740 1 -- 172.21.15.123:0/3893174001 wait complete. 2024-09-19T04:52:31.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.589+0000 7fa9865c7740 20 client.26428 trim_cache size 0 max 16384 2024-09-19T04:52:31.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:31 smithi183 ceph-mon[35333]: pgmap v177: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 356 op/s 2024-09-19T04:52:31.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.607+0000 7fa9865c7740 1 Processor -- start 2024-09-19T04:52:31.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.607+0000 7fa9865c7740 1 -- start start 2024-09-19T04:52:31.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.607+0000 7fa9865c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812dbcb0 0x558f812df000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.607+0000 7fa9865c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812dc5d0 0x558f812fe6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.607+0000 7fa9865c7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812e6e90 0x558f812fec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.607+0000 7fa9865c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c1af7c0 con 0x558f812dc5d0 2024-09-19T04:52:31.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.607+0000 7fa9865c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c1b1300 con 0x558f812dbcb0 2024-09-19T04:52:31.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.607+0000 7fa9865c7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c1afe90 con 0x558f812e6e90 2024-09-19T04:52:31.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.607+0000 7fa97be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812dc5d0 0x558f812fe6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.607+0000 7fa981a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812e6e90 0x558f812fec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.607+0000 7fa97be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812dc5d0 0x558f812fe6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46172/0 (socket says 172.21.15.123:46172) 2024-09-19T04:52:31.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.607+0000 7fa97be00640 1 -- 172.21.15.123:0/113498331 learned_addr learned my addr 172.21.15.123:0/113498331 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:31.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.607+0000 7fa981000640 1 --2- 172.21.15.123:0/113498331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812dbcb0 0x558f812df000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.608+0000 7fa97be00640 1 -- 172.21.15.123:0/113498331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812e6e90 msgr2=0x558f812fec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:31.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.608+0000 7fa97be00640 1 --2- 172.21.15.123:0/113498331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812e6e90 0x558f812fec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.608+0000 7fa97be00640 1 -- 172.21.15.123:0/113498331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812dbcb0 msgr2=0x558f812df000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:31.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.608+0000 7fa97be00640 1 --2- 172.21.15.123:0/113498331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812dbcb0 0x558f812df000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.608+0000 7fa97be00640 1 -- 172.21.15.123:0/113498331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f812e1f90 con 0x558f812dc5d0 2024-09-19T04:52:31.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.608+0000 7fa981a00640 1 --2- 172.21.15.123:0/113498331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812e6e90 0x558f812fec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:52:31.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.608+0000 7fa97be00640 1 --2- 172.21.15.123:0/113498331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812dc5d0 0x558f812fe6c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x558f812e32d0 tx=0x7fa968019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918763e35942ab15 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:31.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.608+0000 7fa96d600640 1 -- 172.21.15.123:0/113498331 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96802d420 con 0x558f812dc5d0 2024-09-19T04:52:31.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.608+0000 7fa96d600640 1 -- 172.21.15.123:0/113498331 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa96802daa0 con 0x558f812dc5d0 2024-09-19T04:52:31.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.608+0000 7fa96d600640 1 -- 172.21.15.123:0/113498331 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968020440 con 0x558f812dc5d0 2024-09-19T04:52:31.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.609+0000 7fa9865c7740 1 -- 172.21.15.123:0/113498331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812dc5d0 msgr2=0x558f812fe6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:31.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.609+0000 7fa9865c7740 1 --2- 172.21.15.123:0/113498331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812dc5d0 0x558f812fe6c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x558f812e32d0 tx=0x7fa968019000 comp rx=0 tx=0).stop 2024-09-19T04:52:31.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.609+0000 7fa9865c7740 1 -- 172.21.15.123:0/113498331 shutdown_connections 2024-09-19T04:52:31.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.609+0000 7fa9865c7740 1 --2- 172.21.15.123:0/113498331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812e6e90 0x558f812fec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.609+0000 7fa9865c7740 1 --2- 172.21.15.123:0/113498331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812dc5d0 0x558f812fe6c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.609+0000 7fa9865c7740 1 --2- 172.21.15.123:0/113498331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812dbcb0 0x558f812df000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.609+0000 7fa9865c7740 1 -- 172.21.15.123:0/113498331 >> 172.21.15.123:0/113498331 conn(0x558f812d8240 msgr2=0x558f812de8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.609+0000 7fa9865c7740 1 -- 172.21.15.123:0/113498331 shutdown_connections 2024-09-19T04:52:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.609+0000 7fa9865c7740 1 -- 172.21.15.123:0/113498331 wait complete. 2024-09-19T04:52:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.610+0000 7fa9865c7740 1 -- start start 2024-09-19T04:52:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.610+0000 7fa9865c7740 1 Processor -- start 2024-09-19T04:52:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.610+0000 7fa9865c7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 0x558f812eab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.610+0000 7fa9865c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812dc5d0 0x558f812eb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.610+0000 7fa9865c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812e6e90 0x558f812eb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.610+0000 7fa9865c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c1b0c30 con 0x558f812e6e90 2024-09-19T04:52:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.610+0000 7fa9865c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c1b0560 con 0x558f812dc5d0 2024-09-19T04:52:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.610+0000 7fa9865c7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c1ad7a0 con 0x558f812dbcb0 2024-09-19T04:52:31.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.611+0000 7fa9865c7740 10 client.0 _ll_register_callbacks cb 0x558f812e49a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:52:31.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.611+0000 7fa981000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 0x558f812eab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.611+0000 7fa981000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 0x558f812eab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:34698/0 (socket says 172.21.15.123:34698) 2024-09-19T04:52:31.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.611+0000 7fa981000640 1 -- 172.21.15.123:0/447209630 learned_addr learned my addr 172.21.15.123:0/447209630 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:31.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.611+0000 7fa981a00640 1 --2- 172.21.15.123:0/447209630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812e6e90 0x558f812eb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.611+0000 7fa97be00640 1 --2- 172.21.15.123:0/447209630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812dc5d0 0x558f812eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.611+0000 7fa981000640 1 -- 172.21.15.123:0/447209630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812dc5d0 msgr2=0x558f812eb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:31.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.611+0000 7fa981000640 1 --2- 172.21.15.123:0/447209630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812dc5d0 0x558f812eb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.611+0000 7fa981000640 1 -- 172.21.15.123:0/447209630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812e6e90 msgr2=0x558f812eb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:31.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.611+0000 7fa981000640 1 --2- 172.21.15.123:0/447209630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812e6e90 0x558f812eb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.611+0000 7fa981000640 1 -- 172.21.15.123:0/447209630 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f81360690 con 0x558f812dbcb0 2024-09-19T04:52:31.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.611+0000 7fa981000640 1 --2- 172.21.15.123:0/447209630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 0x558f812eab40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa97002aa10 tx=0x7fa970004530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2bc0e37e5a68f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:31.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.611+0000 7fa962a00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:52:31.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.611+0000 7fa962a00640 1 -- 172.21.15.123:0/447209630 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970053040 con 0x558f812dbcb0 2024-09-19T04:52:31.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.611+0000 7fa9865c7740 1 -- 172.21.15.123:0/447209630 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa96802daa0 con 0x558f812dbcb0 2024-09-19T04:52:31.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.612+0000 7fa962a00640 1 -- 172.21.15.123:0/447209630 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fa970003800 con 0x558f812dbcb0 2024-09-19T04:52:31.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.612+0000 7fa962a00640 1 -- 172.21.15.123:0/447209630 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa970066050 con 0x558f812dbcb0 2024-09-19T04:52:31.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.612+0000 7fa9865c7740 10 client.26434 fetch_fsmap learned FSMap version 37 2024-09-19T04:52:31.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.612+0000 7fa9865c7740 1 -- 172.21.15.123:0/447209630 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558f8136a0e0 con 0x558f812dbcb0 2024-09-19T04:52:31.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.612+0000 7fa962a00640 1 -- 172.21.15.123:0/447209630 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa96802daa0 con 0x558f812dbcb0 2024-09-19T04:52:31.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.612+0000 7fa962a00640 1 -- 172.21.15.123:0/447209630 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fa970031850 con 0x558f812dbcb0 2024-09-19T04:52:31.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.612+0000 7fa9865c7740 10 client.26434 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:52:31.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.612+0000 7fa9865c7740 10 client.26434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:52:31.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.612+0000 7fa9865c7740 20 client.26434 populate_metadata read hostname 'smithi123' 2024-09-19T04:52:31.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.612+0000 7fa9865c7740 1 --2- 172.21.15.123:0/447209630 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x558f8136a6e0 0x558f8136cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:31.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.612+0000 7fa9865c7740 4 client.26434 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:52:31.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.612+0000 7fa9865c7740 1 -- 172.21.15.123:0/447209630 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558f81249d10 con 0x558f8136a6e0 2024-09-19T04:52:31.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.613+0000 7fa97be00640 1 --2- 172.21.15.123:0/447209630 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x558f8136a6e0 0x558f8136cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:31.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.613+0000 7fa97be00640 1 --2- 172.21.15.123:0/447209630 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x558f8136a6e0 0x558f8136cad0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e702580cf9fa11 server_cookie=54fea27b5ec187fb in_seq=0 out_seq=0 2024-09-19T04:52:31.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.613+0000 7fa962a00640 10 client.26434 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:52:31.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.614+0000 7fa962a00640 1 -- 172.21.15.123:0/447209630 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558f81249d10 con 0x558f8136a6e0 2024-09-19T04:52:31.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.614+0000 7fa962a00640 10 client.26434 handle_command_reply: tid=0 2024-09-19T04:52:31.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.614+0000 7fa9865c7740 1 client.26434 shutdown 2024-09-19T04:52:31.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.614+0000 7fa9865c7740 1 -- 172.21.15.123:0/447209630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 msgr2=0x558f812eab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:31.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.614+0000 7fa9865c7740 1 --2- 172.21.15.123:0/447209630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 0x558f812eab40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa97002aa10 tx=0x7fa970004530 comp rx=0 tx=0).stop 2024-09-19T04:52:31.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.614+0000 7fa9865c7740 1 -- 172.21.15.123:0/447209630 shutdown_connections 2024-09-19T04:52:31.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.614+0000 7fa9865c7740 1 --2- 172.21.15.123:0/447209630 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x558f8136a6e0 0x558f8136cad0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.614+0000 7fa9865c7740 1 --2- 172.21.15.123:0/447209630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f812e6e90 0x558f812eb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.614+0000 7fa9865c7740 1 --2- 172.21.15.123:0/447209630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f812dc5d0 0x558f812eb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.614+0000 7fa9865c7740 1 --2- 172.21.15.123:0/447209630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f812dbcb0 0x558f812eab40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.614+0000 7fa9865c7740 1 -- 172.21.15.123:0/447209630 >> 172.21.15.123:0/447209630 conn(0x558f812d8240 msgr2=0x558f812d7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:31.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.614+0000 7fa962a00640 0 client.26434 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:52:31.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.614+0000 7fa9865c7740 1 -- 172.21.15.123:0/447209630 shutdown_connections 2024-09-19T04:52:31.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.615+0000 7fa9865c7740 1 -- 172.21.15.123:0/447209630 wait complete. 2024-09-19T04:52:31.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.615+0000 7fa9865c7740 20 client.26434 trim_cache size 0 max 16384 2024-09-19T04:52:31.620 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:52:31.620 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-09-19T04:52:31.620 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "424c7811-e9b3-4fc2-adb8-55139b357402", 2024-09-19T04:52:31.620 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-09-19T04:52:31.620 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:52:31.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.615+0000 7fa982400640 1 -- 172.21.15.123:0/204576244 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa9640a3430 msgr2=0x7fa9640a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:31.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.615+0000 7fa982400640 1 --2- 172.21.15.123:0/204576244 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa9640a3430 0x7fa9640a58f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x558f812d95c0 tx=0x7fa968004730 comp rx=0 tx=0).stop 2024-09-19T04:52:31.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.615+0000 7fa982400640 1 -- 172.21.15.123:0/204576244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa97c1079c0 msgr2=0x7fa97c19edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:31.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.615+0000 7fa982400640 1 --2- 172.21.15.123:0/204576244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa97c1079c0 0x7fa97c19edc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa974009fd0 tx=0x7fa97400b260 comp rx=0 tx=0).stop 2024-09-19T04:52:31.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.615+0000 7fa982400640 1 -- 172.21.15.123:0/204576244 shutdown_connections 2024-09-19T04:52:31.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.615+0000 7fa982400640 1 --2- 172.21.15.123:0/204576244 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa9640a3430 0x7fa9640a58f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.615+0000 7fa982400640 1 --2- 172.21.15.123:0/204576244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa97c1079c0 0x7fa97c19edc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.615+0000 7fa982400640 1 --2- 172.21.15.123:0/204576244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa97c102360 0x7fa97c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.615+0000 7fa982400640 1 --2- 172.21.15.123:0/204576244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa97c1019c0 0x7fa97c19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:31.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.615+0000 7fa982400640 1 -- 172.21.15.123:0/204576244 >> 172.21.15.123:0/204576244 conn(0x7fa97c0fd740 msgr2=0x7fa97c10d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:31.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.616+0000 7fa982400640 1 -- 172.21.15.123:0/204576244 shutdown_connections 2024-09-19T04:52:31.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:31.616+0000 7fa982400640 1 -- 172.21.15.123:0/204576244 wait complete. 2024-09-19T04:52:32.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:32 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:52:32.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:32 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:52:32.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:32 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:52:32.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:32 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:52:32.602 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:32 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:52:32.602 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:32 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:52:32.667 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:52:33.072 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:52:33.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:33 smithi193 ceph-mon[35592]: scrub queued for path: / 2024-09-19T04:52:33.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:33 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:52:33.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:33 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:52:33.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:33 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:52:33.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:33 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:52:33.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:33 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:52:33.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:33 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:52:33.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:33 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:52:33.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:33 smithi193 ceph-mon[35592]: pgmap v178: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 356 op/s 2024-09-19T04:52:33.508 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:33 smithi123 ceph-mon[30511]: scrub queued for path: / 2024-09-19T04:52:33.509 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:33 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:52:33.509 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:33 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:52:33.509 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:33 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:52:33.509 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:33 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:52:33.509 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:33 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:52:33.510 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:33 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:52:33.510 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:33 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:52:33.510 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:33 smithi123 ceph-mon[30511]: pgmap v178: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 356 op/s 2024-09-19T04:52:33.550 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-09-19T04:52:33.550 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-19T04:52:33.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:33 smithi183 ceph-mon[35333]: scrub queued for path: / 2024-09-19T04:52:33.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:33 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:52:33.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:33 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:52:33.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:33 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:52:33.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:33 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:52:33.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:33 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:52:33.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:33 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:52:33.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:33 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:52:33.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:33 smithi183 ceph-mon[35333]: pgmap v178: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 356 op/s 2024-09-19T04:52:33.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.791+0000 7fbd53800640 1 -- 172.21.15.123:0/1650093518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd4c109120 msgr2=0x7fbd4c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:33.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.791+0000 7fbd53800640 1 --2- 172.21.15.123:0/1650093518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd4c109120 0x7fbd4c110ca0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009be0 tx=0x7fbd4402a6c0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.791+0000 7fbd53800640 1 -- 172.21.15.123:0/1650093518 shutdown_connections 2024-09-19T04:52:33.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.791+0000 7fbd53800640 1 --2- 172.21.15.123:0/1650093518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd4c109120 0x7fbd4c110ca0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.791+0000 7fbd53800640 1 --2- 172.21.15.123:0/1650093518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd4c1049a0 0x7fbd4c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.791+0000 7fbd53800640 1 --2- 172.21.15.123:0/1650093518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd4c103ff0 0x7fbd4c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.791+0000 7fbd53800640 1 -- 172.21.15.123:0/1650093518 >> 172.21.15.123:0/1650093518 conn(0x7fbd4c0ffa90 msgr2=0x7fbd4c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:33.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.791+0000 7fbd53800640 1 -- 172.21.15.123:0/1650093518 shutdown_connections 2024-09-19T04:52:33.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.791+0000 7fbd53800640 1 -- 172.21.15.123:0/1650093518 wait complete. 2024-09-19T04:52:33.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.792+0000 7fbd53800640 1 Processor -- start 2024-09-19T04:52:33.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.792+0000 7fbd53800640 1 -- start start 2024-09-19T04:52:33.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.793+0000 7fbd53800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd4c103ff0 0x7fbd4c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.793+0000 7fbd53800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd4c1049a0 0x7fbd4c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.793+0000 7fbd53800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd4c109120 0x7fbd4c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.793+0000 7fbd53800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c114130 con 0x7fbd4c103ff0 2024-09-19T04:52:33.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.793+0000 7fbd53800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c113fb0 con 0x7fbd4c109120 2024-09-19T04:52:33.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.793+0000 7fbd53800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c1142b0 con 0x7fbd4c1049a0 2024-09-19T04:52:33.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.793+0000 7fbd50e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd4c103ff0 0x7fbd4c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:33.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.793+0000 7fbd4be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd4c1049a0 0x7fbd4c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:33.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.793+0000 7fbd4be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd4c1049a0 0x7fbd4c19aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:57498/0 (socket says 172.21.15.123:57498) 2024-09-19T04:52:33.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.793+0000 7fbd4be00640 1 -- 172.21.15.123:0/4265430866 learned_addr learned my addr 172.21.15.123:0/4265430866 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:33.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.794+0000 7fbd51800640 1 --2- 172.21.15.123:0/4265430866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd4c109120 0x7fbd4c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:33.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.794+0000 7fbd50e00640 1 -- 172.21.15.123:0/4265430866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd4c1049a0 msgr2=0x7fbd4c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:33.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.794+0000 7fbd50e00640 1 --2- 172.21.15.123:0/4265430866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd4c1049a0 0x7fbd4c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.794+0000 7fbd50e00640 1 -- 172.21.15.123:0/4265430866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd4c109120 msgr2=0x7fbd4c19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:33.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.794+0000 7fbd50e00640 1 --2- 172.21.15.123:0/4265430866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd4c109120 0x7fbd4c19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.794+0000 7fbd50e00640 1 -- 172.21.15.123:0/4265430866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c19f540 con 0x7fbd4c103ff0 2024-09-19T04:52:33.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.794+0000 7fbd4be00640 1 --2- 172.21.15.123:0/4265430866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd4c1049a0 0x7fbd4c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:52:33.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.794+0000 7fbd50e00640 1 --2- 172.21.15.123:0/4265430866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd4c103ff0 0x7fbd4c19a590 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000e790 tx=0x7fbd4000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:33.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.795+0000 7fbd49600640 1 -- 172.21.15.123:0/4265430866 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4000eea0 con 0x7fbd4c103ff0 2024-09-19T04:52:33.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.795+0000 7fbd49600640 1 -- 172.21.15.123:0/4265430866 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd4000ccc0 con 0x7fbd4c103ff0 2024-09-19T04:52:33.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.795+0000 7fbd49600640 1 -- 172.21.15.123:0/4265430866 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40018780 con 0x7fbd4c103ff0 2024-09-19T04:52:33.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.795+0000 7fbd53800640 1 -- 172.21.15.123:0/4265430866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4c19f830 con 0x7fbd4c103ff0 2024-09-19T04:52:33.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.795+0000 7fbd53800640 1 -- 172.21.15.123:0/4265430866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4c1a7110 con 0x7fbd4c103ff0 2024-09-19T04:52:33.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.798+0000 7fbd562c6740 1 Processor -- start 2024-09-19T04:52:33.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.798+0000 7fbd562c6740 1 -- start start 2024-09-19T04:52:33.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.799+0000 7fbd562c6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb466b90 0x55a6cb466f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.799+0000 7fbd562c6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb4674b0 0x55a6cb471830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.799+0000 7fbd562c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb471d70 0x55a6cb474160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.799+0000 7fbd562c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a6cb3cd3e0 con 0x55a6cb471d70 2024-09-19T04:52:33.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.799+0000 7fbd562c6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a6cb374d80 con 0x55a6cb466b90 2024-09-19T04:52:33.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.799+0000 7fbd562c6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a6cb3c93a0 con 0x55a6cb4674b0 2024-09-19T04:52:33.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.799+0000 7fbd4be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb471d70 0x55a6cb474160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:33.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.799+0000 7fbd4be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb471d70 0x55a6cb474160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59374/0 (socket says 172.21.15.123:59374) 2024-09-19T04:52:33.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.799+0000 7fbd4be00640 1 -- 172.21.15.123:0/3928547362 learned_addr learned my addr 172.21.15.123:0/3928547362 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:33.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.799+0000 7fbd51800640 1 --2- 172.21.15.123:0/3928547362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb466b90 0x55a6cb466f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:33.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.799+0000 7fbd4be00640 1 -- 172.21.15.123:0/3928547362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb4674b0 msgr2=0x55a6cb471830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:52:33.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.799+0000 7fbd4be00640 1 --2- 172.21.15.123:0/3928547362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb4674b0 0x55a6cb471830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.799+0000 7fbd4be00640 1 -- 172.21.15.123:0/3928547362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb466b90 msgr2=0x55a6cb466f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:33.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.799+0000 7fbd4be00640 1 --2- 172.21.15.123:0/3928547362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb466b90 0x55a6cb466f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.799+0000 7fbd4be00640 1 -- 172.21.15.123:0/3928547362 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6cb393ae0 con 0x55a6cb471d70 2024-09-19T04:52:33.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.799+0000 7fbd49600640 1 -- 172.21.15.123:0/4265430866 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fbd40010430 con 0x7fbd4c103ff0 2024-09-19T04:52:33.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.800+0000 7fbd4be00640 1 --2- 172.21.15.123:0/3928547362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb471d70 0x55a6cb474160 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbd380115f0 tx=0x7fbd38011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a1d834ca6dad30 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:33.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.800+0000 7fbd3e000640 1 -- 172.21.15.123:0/3928547362 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3801e070 con 0x55a6cb471d70 2024-09-19T04:52:33.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.801+0000 7fbd3e000640 1 -- 172.21.15.123:0/3928547362 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd3800fbe0 con 0x55a6cb471d70 2024-09-19T04:52:33.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.801+0000 7fbd3e000640 1 -- 172.21.15.123:0/3928547362 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd38019c30 con 0x55a6cb471d70 2024-09-19T04:52:33.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.801+0000 7fbd49600640 1 --2- 172.21.15.123:0/4265430866 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbd340a3430 0x7fbd340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.801+0000 7fbd49600640 1 -- 172.21.15.123:0/4265430866 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fbd400cfe70 con 0x7fbd4c103ff0 2024-09-19T04:52:33.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.802+0000 7fbd562c6740 1 -- 172.21.15.123:0/3928547362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb471d70 msgr2=0x55a6cb474160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:33.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.802+0000 7fbd562c6740 1 --2- 172.21.15.123:0/3928547362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb471d70 0x55a6cb474160 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbd380115f0 tx=0x7fbd38011ab0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.802+0000 7fbd51800640 1 --2- 172.21.15.123:0/4265430866 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbd340a3430 0x7fbd340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:33.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.802+0000 7fbd562c6740 1 -- 172.21.15.123:0/3928547362 shutdown_connections 2024-09-19T04:52:33.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.802+0000 7fbd562c6740 1 --2- 172.21.15.123:0/3928547362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb471d70 0x55a6cb474160 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.802+0000 7fbd562c6740 1 --2- 172.21.15.123:0/3928547362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb4674b0 0x55a6cb471830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.802+0000 7fbd562c6740 1 --2- 172.21.15.123:0/3928547362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb466b90 0x55a6cb466f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.802+0000 7fbd562c6740 1 -- 172.21.15.123:0/3928547362 >> 172.21.15.123:0/3928547362 conn(0x55a6cb463120 msgr2=0x55a6cb463500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:33.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.802+0000 7fbd562c6740 1 -- 172.21.15.123:0/3928547362 shutdown_connections 2024-09-19T04:52:33.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.802+0000 7fbd562c6740 1 -- 172.21.15.123:0/3928547362 wait complete. 2024-09-19T04:52:33.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.803+0000 7fbd51800640 1 --2- 172.21.15.123:0/4265430866 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbd340a3430 0x7fbd340a58f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbd44004af0 tx=0x7fbd44005fd0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:33.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.803+0000 7fbd562c6740 1 -- start start 2024-09-19T04:52:33.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.803+0000 7fbd562c6740 1 Processor -- start 2024-09-19T04:52:33.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.803+0000 7fbd562c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb466b90 0x55a6cb46ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.803+0000 7fbd562c6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb4674b0 0x55a6cb46f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.803+0000 7fbd562c6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb471d70 0x55a6cb46f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.803+0000 7fbd562c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a6cb4749e0 con 0x55a6cb466b90 2024-09-19T04:52:33.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.803+0000 7fbd562c6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a6cb3880f0 con 0x55a6cb4674b0 2024-09-19T04:52:33.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.803+0000 7fbd562c6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a6cb474b60 con 0x55a6cb471d70 2024-09-19T04:52:33.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.803+0000 7fbd562c6740 10 client.0 _ll_register_callbacks cb 0x55a6cb3799e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:52:33.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.804+0000 7fbd50e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb466b90 0x55a6cb46ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:33.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.804+0000 7fbd50e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb466b90 0x55a6cb46ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59390/0 (socket says 172.21.15.123:59390) 2024-09-19T04:52:33.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.804+0000 7fbd50e00640 1 -- 172.21.15.123:0/1955926583 learned_addr learned my addr 172.21.15.123:0/1955926583 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:33.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.804+0000 7fbd4be00640 1 --2- 172.21.15.123:0/1955926583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb4674b0 0x55a6cb46f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:33.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.804+0000 7fbd51800640 1 --2- 172.21.15.123:0/1955926583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb471d70 0x55a6cb46f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:33.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.804+0000 7fbd50e00640 1 -- 172.21.15.123:0/1955926583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb471d70 msgr2=0x55a6cb46f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:33.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.804+0000 7fbd50e00640 1 --2- 172.21.15.123:0/1955926583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb471d70 0x55a6cb46f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.804+0000 7fbd50e00640 1 -- 172.21.15.123:0/1955926583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb4674b0 msgr2=0x55a6cb46f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:33.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.804+0000 7fbd50e00640 1 --2- 172.21.15.123:0/1955926583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb4674b0 0x55a6cb46f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.804+0000 7fbd50e00640 1 -- 172.21.15.123:0/1955926583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6cb13e3c0 con 0x55a6cb466b90 2024-09-19T04:52:33.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.804+0000 7fbd4be00640 1 --2- 172.21.15.123:0/1955926583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb4674b0 0x55a6cb46f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:52:33.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.805+0000 7fbd50e00640 1 --2- 172.21.15.123:0/1955926583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb466b90 0x55a6cb46ed00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbd40000c00 tx=0x7fbd4009bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a93130771808f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:33.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.805+0000 7fbd32a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:52:33.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.805+0000 7fbd32a00640 1 -- 172.21.15.123:0/1955926583 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd400a4d90 con 0x55a6cb466b90 2024-09-19T04:52:33.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.805+0000 7fbd562c6740 1 -- 172.21.15.123:0/1955926583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd3800fbe0 con 0x55a6cb466b90 2024-09-19T04:52:33.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.805+0000 7fbd32a00640 1 -- 172.21.15.123:0/1955926583 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fbd400a0170 con 0x55a6cb466b90 2024-09-19T04:52:33.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.805+0000 7fbd32a00640 1 -- 172.21.15.123:0/1955926583 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbd40098840 con 0x55a6cb466b90 2024-09-19T04:52:33.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.805+0000 7fbd562c6740 10 client.16323 fetch_fsmap learned FSMap version 37 2024-09-19T04:52:33.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.805+0000 7fbd562c6740 1 -- 172.21.15.123:0/1955926583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a6cb480730 con 0x55a6cb466b90 2024-09-19T04:52:33.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.806+0000 7fbd32a00640 1 -- 172.21.15.123:0/1955926583 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd3800fbe0 con 0x55a6cb466b90 2024-09-19T04:52:33.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.806+0000 7fbd32a00640 1 -- 172.21.15.123:0/1955926583 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fbd400cdce0 con 0x55a6cb466b90 2024-09-19T04:52:33.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.806+0000 7fbd562c6740 10 client.16323 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:52:33.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.806+0000 7fbd562c6740 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:52:33.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.806+0000 7fbd562c6740 20 client.16323 populate_metadata read hostname 'smithi123' 2024-09-19T04:52:33.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.806+0000 7fbd562c6740 1 --2- 172.21.15.123:0/1955926583 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a6cb481b40 0x55a6cb483f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.806+0000 7fbd562c6740 4 client.16323 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:52:33.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.806+0000 7fbd562c6740 1 -- 172.21.15.123:0/1955926583 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6cb3da950 con 0x55a6cb481b40 2024-09-19T04:52:33.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.806+0000 7fbd4be00640 1 --2- 172.21.15.123:0/1955926583 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a6cb481b40 0x55a6cb483f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:33.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.807+0000 7fbd32a00640 10 client.16323 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:52:33.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.807+0000 7fbd4be00640 1 --2- 172.21.15.123:0/1955926583 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a6cb481b40 0x55a6cb483f30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11714e1285b45c64 server_cookie=d8f37f0bd0694c7f in_seq=0 out_seq=0 2024-09-19T04:52:33.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.808+0000 7fbd32a00640 1 -- 172.21.15.123:0/1955926583 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6cb3da950 con 0x55a6cb481b40 2024-09-19T04:52:33.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.808+0000 7fbd32a00640 10 client.16323 handle_command_reply: tid=0 2024-09-19T04:52:33.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.808+0000 7fbd562c6740 1 client.16323 shutdown 2024-09-19T04:52:33.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.808+0000 7fbd562c6740 1 -- 172.21.15.123:0/1955926583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb466b90 msgr2=0x55a6cb46ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:33.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.808+0000 7fbd562c6740 1 --2- 172.21.15.123:0/1955926583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb466b90 0x55a6cb46ed00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbd40000c00 tx=0x7fbd4009bc60 comp rx=0 tx=0).stop 2024-09-19T04:52:33.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.808+0000 7fbd562c6740 1 -- 172.21.15.123:0/1955926583 shutdown_connections 2024-09-19T04:52:33.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.808+0000 7fbd562c6740 1 --2- 172.21.15.123:0/1955926583 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a6cb481b40 0x55a6cb483f30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.808+0000 7fbd562c6740 1 --2- 172.21.15.123:0/1955926583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb471d70 0x55a6cb46f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.808+0000 7fbd562c6740 1 --2- 172.21.15.123:0/1955926583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb4674b0 0x55a6cb46f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.808+0000 7fbd562c6740 1 --2- 172.21.15.123:0/1955926583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb466b90 0x55a6cb46ed00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.808+0000 7fbd562c6740 1 -- 172.21.15.123:0/1955926583 >> 172.21.15.123:0/1955926583 conn(0x55a6cb463120 msgr2=0x55a6cb468e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:33.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.808+0000 7fbd32a00640 0 client.16323 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:52:33.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.808+0000 7fbd562c6740 1 -- 172.21.15.123:0/1955926583 shutdown_connections 2024-09-19T04:52:33.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.809+0000 7fbd562c6740 1 -- 172.21.15.123:0/1955926583 wait complete. 2024-09-19T04:52:33.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.809+0000 7fbd562c6740 20 client.16323 trim_cache size 0 max 16384 2024-09-19T04:52:33.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.827+0000 7fbd562c6740 1 Processor -- start 2024-09-19T04:52:33.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.827+0000 7fbd562c6740 1 -- start start 2024-09-19T04:52:33.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.828+0000 7fbd562c6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb466b90 0x55a6cb468cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.828+0000 7fbd562c6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb4674b0 0x55a6cb4897b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.828+0000 7fbd562c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb471d70 0x55a6cb489cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.828+0000 7fbd562c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c1afa50 con 0x55a6cb471d70 2024-09-19T04:52:33.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.828+0000 7fbd562c6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c1b1590 con 0x55a6cb4674b0 2024-09-19T04:52:33.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.828+0000 7fbd562c6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c1b0120 con 0x55a6cb466b90 2024-09-19T04:52:33.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.828+0000 7fbd50e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb466b90 0x55a6cb468cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:33.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.828+0000 7fbd4be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb4674b0 0x55a6cb4897b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:33.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.828+0000 7fbd50e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb466b90 0x55a6cb468cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:57510/0 (socket says 172.21.15.123:57510) 2024-09-19T04:52:33.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.828+0000 7fbd4be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb4674b0 0x55a6cb4897b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:47178/0 (socket says 172.21.15.123:47178) 2024-09-19T04:52:33.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.828+0000 7fbd50e00640 1 -- 172.21.15.123:0/915546929 learned_addr learned my addr 172.21.15.123:0/915546929 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:33.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.828+0000 7fbd51800640 1 --2- 172.21.15.123:0/915546929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb471d70 0x55a6cb489cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:33.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.829+0000 7fbd50e00640 1 -- 172.21.15.123:0/915546929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb4674b0 msgr2=0x55a6cb4897b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:33.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.829+0000 7fbd50e00640 1 --2- 172.21.15.123:0/915546929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb4674b0 0x55a6cb4897b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.829+0000 7fbd50e00640 1 -- 172.21.15.123:0/915546929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb471d70 msgr2=0x55a6cb489cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:33.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.829+0000 7fbd50e00640 1 --2- 172.21.15.123:0/915546929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb471d70 0x55a6cb489cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.829+0000 7fbd50e00640 1 -- 172.21.15.123:0/915546929 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6cb4642b0 con 0x55a6cb466b90 2024-09-19T04:52:33.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.829+0000 7fbd51800640 1 --2- 172.21.15.123:0/915546929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb471d70 0x55a6cb489cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:52:33.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.829+0000 7fbd50e00640 1 --2- 172.21.15.123:0/915546929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb466b90 0x55a6cb468cf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbd40098ba0 tx=0x7fbd40010c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfa72ac6913ba54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:33.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.830+0000 7fbd3d600640 1 -- 172.21.15.123:0/915546929 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd400a0170 con 0x55a6cb466b90 2024-09-19T04:52:33.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.830+0000 7fbd3d600640 1 -- 172.21.15.123:0/915546929 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd400bc070 con 0x55a6cb466b90 2024-09-19T04:52:33.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.830+0000 7fbd3d600640 1 -- 172.21.15.123:0/915546929 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd400a1660 con 0x55a6cb466b90 2024-09-19T04:52:33.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.830+0000 7fbd562c6740 1 -- 172.21.15.123:0/915546929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb466b90 msgr2=0x55a6cb468cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:33.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.830+0000 7fbd562c6740 1 --2- 172.21.15.123:0/915546929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb466b90 0x55a6cb468cf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbd40098ba0 tx=0x7fbd40010c40 comp rx=0 tx=0).stop 2024-09-19T04:52:33.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.831+0000 7fbd562c6740 1 -- 172.21.15.123:0/915546929 shutdown_connections 2024-09-19T04:52:33.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.831+0000 7fbd562c6740 1 --2- 172.21.15.123:0/915546929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb471d70 0x55a6cb489cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.831+0000 7fbd562c6740 1 --2- 172.21.15.123:0/915546929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb4674b0 0x55a6cb4897b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.831+0000 7fbd562c6740 1 --2- 172.21.15.123:0/915546929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb466b90 0x55a6cb468cf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.831+0000 7fbd562c6740 1 -- 172.21.15.123:0/915546929 >> 172.21.15.123:0/915546929 conn(0x55a6cb463120 msgr2=0x55a6cb4685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:33.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.831+0000 7fbd562c6740 1 -- 172.21.15.123:0/915546929 shutdown_connections 2024-09-19T04:52:33.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.831+0000 7fbd562c6740 1 -- 172.21.15.123:0/915546929 wait complete. 2024-09-19T04:52:33.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.831+0000 7fbd562c6740 1 -- start start 2024-09-19T04:52:33.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.831+0000 7fbd562c6740 1 Processor -- start 2024-09-19T04:52:33.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.832+0000 7fbd562c6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb466b90 0x55a6cb46bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.832+0000 7fbd562c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb4674b0 0x55a6cb46c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.832+0000 7fbd562c6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb471d70 0x55a6cb46c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.832+0000 7fbd562c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c1b0ec0 con 0x55a6cb4674b0 2024-09-19T04:52:33.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.832+0000 7fbd562c6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c1b07f0 con 0x55a6cb466b90 2024-09-19T04:52:33.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.832+0000 7fbd562c6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c1ada30 con 0x55a6cb471d70 2024-09-19T04:52:33.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.832+0000 7fbd562c6740 10 client.0 _ll_register_callbacks cb 0x7fbd4009c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:52:33.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.832+0000 7fbd4be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb4674b0 0x55a6cb46c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:33.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.832+0000 7fbd4be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb4674b0 0x55a6cb46c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59408/0 (socket says 172.21.15.123:59408) 2024-09-19T04:52:33.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.832+0000 7fbd4be00640 1 -- 172.21.15.123:0/1488729519 learned_addr learned my addr 172.21.15.123:0/1488729519 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:52:33.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.832+0000 7fbd51800640 1 --2- 172.21.15.123:0/1488729519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb471d70 0x55a6cb46c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:33.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.832+0000 7fbd50e00640 1 --2- 172.21.15.123:0/1488729519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb466b90 0x55a6cb46bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:33.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.833+0000 7fbd50e00640 1 -- 172.21.15.123:0/1488729519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb471d70 msgr2=0x55a6cb46c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:33.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.833+0000 7fbd50e00640 1 --2- 172.21.15.123:0/1488729519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb471d70 0x55a6cb46c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.833+0000 7fbd50e00640 1 -- 172.21.15.123:0/1488729519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb4674b0 msgr2=0x55a6cb46c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:52:33.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.833+0000 7fbd50e00640 1 --2- 172.21.15.123:0/1488729519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb4674b0 0x55a6cb46c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.833+0000 7fbd50e00640 1 -- 172.21.15.123:0/1488729519 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6cb4eb980 con 0x55a6cb466b90 2024-09-19T04:52:33.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.833+0000 7fbd4be00640 1 --2- 172.21.15.123:0/1488729519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb4674b0 0x55a6cb46c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:52:33.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.833+0000 7fbd50e00640 1 --2- 172.21.15.123:0/1488729519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb466b90 0x55a6cb46bed0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbd40000c00 tx=0x7fbd400cd670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48366427b4fa7a96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:52:33.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.833+0000 7fbd32a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:52:33.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.833+0000 7fbd32a00640 1 -- 172.21.15.123:0/1488729519 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd400bc220 con 0x55a6cb466b90 2024-09-19T04:52:33.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.833+0000 7fbd562c6740 1 -- 172.21.15.123:0/1488729519 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd400bc070 con 0x55a6cb466b90 2024-09-19T04:52:33.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.833+0000 7fbd32a00640 1 -- 172.21.15.123:0/1488729519 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7fbd40010720 con 0x55a6cb466b90 2024-09-19T04:52:33.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.833+0000 7fbd32a00640 1 -- 172.21.15.123:0/1488729519 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbd400a4d90 con 0x55a6cb466b90 2024-09-19T04:52:33.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.833+0000 7fbd562c6740 10 client.26432 fetch_fsmap learned FSMap version 37 2024-09-19T04:52:33.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.833+0000 7fbd562c6740 1 -- 172.21.15.123:0/1488729519 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a6cb4ecc30 con 0x55a6cb466b90 2024-09-19T04:52:33.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.834+0000 7fbd32a00640 1 -- 172.21.15.123:0/1488729519 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd400bc070 con 0x55a6cb466b90 2024-09-19T04:52:33.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.834+0000 7fbd32a00640 1 -- 172.21.15.123:0/1488729519 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fbd400cdce0 con 0x55a6cb466b90 2024-09-19T04:52:33.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.834+0000 7fbd562c6740 10 client.26432 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:52:33.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.834+0000 7fbd562c6740 10 client.26432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:52:33.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.834+0000 7fbd562c6740 20 client.26432 populate_metadata read hostname 'smithi123' 2024-09-19T04:52:33.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.834+0000 7fbd562c6740 1 --2- 172.21.15.123:0/1488729519 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a6cb4ed0a0 0x55a6cb4ef4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:52:33.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.834+0000 7fbd562c6740 4 client.26432 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:52:33.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.834+0000 7fbd562c6740 1 -- 172.21.15.123:0/1488729519 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a6cb3d4e60 con 0x55a6cb4ed0a0 2024-09-19T04:52:33.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.834+0000 7fbd4be00640 1 --2- 172.21.15.123:0/1488729519 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a6cb4ed0a0 0x55a6cb4ef4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:52:33.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.835+0000 7fbd4be00640 1 --2- 172.21.15.123:0/1488729519 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a6cb4ed0a0 0x55a6cb4ef4c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8944e5076f1f6304 server_cookie=18db11ccc96f6768 in_seq=0 out_seq=0 2024-09-19T04:52:33.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.835+0000 7fbd32a00640 10 client.26432 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:52:33.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.835+0000 7fbd32a00640 1 -- 172.21.15.123:0/1488729519 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a6cb3d4e60 con 0x55a6cb4ed0a0 2024-09-19T04:52:33.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.835+0000 7fbd32a00640 10 client.26432 handle_command_reply: tid=0 2024-09-19T04:52:33.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.835+0000 7fbd562c6740 1 client.26432 shutdown 2024-09-19T04:52:33.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.835+0000 7fbd562c6740 1 -- 172.21.15.123:0/1488729519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb466b90 msgr2=0x55a6cb46bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:33.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.835+0000 7fbd562c6740 1 --2- 172.21.15.123:0/1488729519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb466b90 0x55a6cb46bed0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbd40000c00 tx=0x7fbd400cd670 comp rx=0 tx=0).stop 2024-09-19T04:52:33.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.835+0000 7fbd562c6740 1 -- 172.21.15.123:0/1488729519 shutdown_connections 2024-09-19T04:52:33.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.835+0000 7fbd562c6740 1 --2- 172.21.15.123:0/1488729519 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a6cb4ed0a0 0x55a6cb4ef4c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.835+0000 7fbd562c6740 1 --2- 172.21.15.123:0/1488729519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6cb471d70 0x55a6cb46c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.835+0000 7fbd562c6740 1 --2- 172.21.15.123:0/1488729519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a6cb4674b0 0x55a6cb46c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.835+0000 7fbd562c6740 1 --2- 172.21.15.123:0/1488729519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6cb466b90 0x55a6cb46bed0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.835+0000 7fbd562c6740 1 -- 172.21.15.123:0/1488729519 >> 172.21.15.123:0/1488729519 conn(0x55a6cb463120 msgr2=0x55a6cb468e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:33.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.835+0000 7fbd32a00640 0 client.26432 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:52:33.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.835+0000 7fbd562c6740 1 -- 172.21.15.123:0/1488729519 shutdown_connections 2024-09-19T04:52:33.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.835+0000 7fbd562c6740 1 -- 172.21.15.123:0/1488729519 wait complete. 2024-09-19T04:52:33.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.836+0000 7fbd562c6740 20 client.26432 trim_cache size 0 max 16384 2024-09-19T04:52:33.841 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:52:33.842 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:52:33.842 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": { 2024-09-19T04:52:33.842 INFO:teuthology.orchestra.run.smithi123.stdout: "424c7811-e9b3-4fc2-adb8-55139b357402": { 2024-09-19T04:52:33.842 INFO:teuthology.orchestra.run.smithi123.stdout: "path": "/", 2024-09-19T04:52:33.842 INFO:teuthology.orchestra.run.smithi123.stdout: "tag": "424c7811-e9b3-4fc2-adb8-55139b357402", 2024-09-19T04:52:33.842 INFO:teuthology.orchestra.run.smithi123.stdout: "options": "recursive,force" 2024-09-19T04:52:33.842 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:52:33.842 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:52:33.842 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:52:33.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.836+0000 7fbd53800640 1 -- 172.21.15.123:0/4265430866 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbd340a3430 msgr2=0x7fbd340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:33.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.836+0000 7fbd53800640 1 --2- 172.21.15.123:0/4265430866 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbd340a3430 0x7fbd340a58f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbd44004af0 tx=0x7fbd44005fd0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.836+0000 7fbd53800640 1 -- 172.21.15.123:0/4265430866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd4c103ff0 msgr2=0x7fbd4c19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:52:33.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.836+0000 7fbd53800640 1 --2- 172.21.15.123:0/4265430866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd4c103ff0 0x7fbd4c19a590 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000e790 tx=0x7fbd4000ec50 comp rx=0 tx=0).stop 2024-09-19T04:52:33.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.837+0000 7fbd53800640 1 -- 172.21.15.123:0/4265430866 shutdown_connections 2024-09-19T04:52:33.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.837+0000 7fbd53800640 1 --2- 172.21.15.123:0/4265430866 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbd340a3430 0x7fbd340a58f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.837+0000 7fbd53800640 1 --2- 172.21.15.123:0/4265430866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd4c109120 0x7fbd4c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.837+0000 7fbd53800640 1 --2- 172.21.15.123:0/4265430866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd4c1049a0 0x7fbd4c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.837+0000 7fbd53800640 1 --2- 172.21.15.123:0/4265430866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd4c103ff0 0x7fbd4c19a590 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:52:33.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.837+0000 7fbd53800640 1 -- 172.21.15.123:0/4265430866 >> 172.21.15.123:0/4265430866 conn(0x7fbd4c0ffa90 msgr2=0x7fbd4c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:52:33.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.837+0000 7fbd53800640 1 -- 172.21.15.123:0/4265430866 shutdown_connections 2024-09-19T04:52:33.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:52:33.837+0000 7fbd53800640 1 -- 172.21.15.123:0/4265430866 wait complete. 2024-09-19T04:52:34.750 INFO:tasks.cephfs.filesystem:scrub status for tag:424c7811-e9b3-4fc2-adb8-55139b357402 - {'path': '/', 'tag': '424c7811-e9b3-4fc2-adb8-55139b357402', 'options': 'recursive,force'} 2024-09-19T04:52:34.756 DEBUG:teuthology.parallel:result is None 2024-09-19T04:52:34.756 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T04:52:34.882 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T04:52:34.884 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-19T04:52:34.897 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-19T04:52:34.898 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T04:52:34.898 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T04:52:34.898 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-19T04:52:34.958 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-19T04:52:35.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:35 smithi123 ceph-mon[30511]: pgmap v179: 97 pgs: 97 active+clean; 242 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.5 MiB/s wr, 566 op/s 2024-09-19T04:52:35.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:35 smithi193 ceph-mon[35592]: pgmap v179: 97 pgs: 97 active+clean; 242 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.5 MiB/s wr, 566 op/s 2024-09-19T04:52:35.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:35 smithi183 ceph-mon[35333]: pgmap v179: 97 pgs: 97 active+clean; 242 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.5 MiB/s wr, 566 op/s 2024-09-19T04:52:36.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:36 smithi193 ceph-mon[35592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:52:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:36 smithi183 ceph-mon[35333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:52:36.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:36 smithi123 ceph-mon[30511]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:52:37.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:37 smithi193 ceph-mon[35592]: pgmap v180: 97 pgs: 97 active+clean; 242 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 371 op/s 2024-09-19T04:52:37.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:37 smithi193 ceph-mon[35592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:52:37.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:37 smithi183 ceph-mon[35333]: pgmap v180: 97 pgs: 97 active+clean; 242 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 371 op/s 2024-09-19T04:52:37.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:37 smithi183 ceph-mon[35333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:52:37.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:37 smithi123 ceph-mon[30511]: pgmap v180: 97 pgs: 97 active+clean; 242 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 371 op/s 2024-09-19T04:52:37.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:37 smithi123 ceph-mon[30511]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:52:39.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:39 smithi193 ceph-mon[35592]: pgmap v181: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 419 op/s 2024-09-19T04:52:39.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:39 smithi183 ceph-mon[35333]: pgmap v181: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 419 op/s 2024-09-19T04:52:39.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:39 smithi123 ceph-mon[30511]: pgmap v181: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 419 op/s 2024-09-19T04:52:40.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:40 smithi193 ceph-mon[35592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:52:40.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:40 smithi183 ceph-mon[35333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:52:40.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:40 smithi123 ceph-mon[30511]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:52:41.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:41 smithi183 ceph-mon[35333]: pgmap v182: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-09-19T04:52:41.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:41 smithi123 ceph-mon[30511]: pgmap v182: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-09-19T04:52:41.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:41 smithi193 ceph-mon[35592]: pgmap v182: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-09-19T04:52:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:43 smithi183 ceph-mon[35333]: pgmap v183: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-09-19T04:52:43.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:43 smithi123 ceph-mon[30511]: pgmap v183: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-09-19T04:52:43.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:43 smithi193 ceph-mon[35592]: pgmap v183: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-09-19T04:52:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:45 smithi183 ceph-mon[35333]: pgmap v184: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 264 op/s 2024-09-19T04:52:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:45 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:52:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:45 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:52:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:45 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:52:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:45 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:52:45.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:45 smithi123 ceph-mon[30511]: pgmap v184: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 264 op/s 2024-09-19T04:52:45.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:45 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:52:45.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:45 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:52:45.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:45 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:52:45.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:45 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:52:45.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:45 smithi193 ceph-mon[35592]: pgmap v184: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 264 op/s 2024-09-19T04:52:45.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:45 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:52:45.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:45 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:52:45.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:45 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:52:45.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:45 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:52:47.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:47 smithi183 ceph-mon[35333]: pgmap v185: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 157 KiB/s wr, 55 op/s 2024-09-19T04:52:47.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:47 smithi123 ceph-mon[30511]: pgmap v185: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 157 KiB/s wr, 55 op/s 2024-09-19T04:52:47.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:47 smithi193 ceph-mon[35592]: pgmap v185: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 157 KiB/s wr, 55 op/s 2024-09-19T04:52:49.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:49 smithi183 ceph-mon[35333]: pgmap v186: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 161 KiB/s wr, 62 op/s 2024-09-19T04:52:49.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:49 smithi123 ceph-mon[30511]: pgmap v186: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 161 KiB/s wr, 62 op/s 2024-09-19T04:52:49.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:49 smithi193 ceph-mon[35592]: pgmap v186: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 161 KiB/s wr, 62 op/s 2024-09-19T04:52:51.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:51 smithi183 ceph-mon[35333]: pgmap v187: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T04:52:51.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:51 smithi123 ceph-mon[30511]: pgmap v187: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T04:52:51.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:51 smithi193 ceph-mon[35592]: pgmap v187: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T04:52:53.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:53 smithi183 ceph-mon[35333]: pgmap v188: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T04:52:53.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:53 smithi123 ceph-mon[30511]: pgmap v188: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T04:52:53.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:53 smithi193 ceph-mon[35592]: pgmap v188: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T04:52:55.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:55 smithi183 ceph-mon[35333]: pgmap v189: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 19 op/s 2024-09-19T04:52:55.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:55 smithi123 ceph-mon[30511]: pgmap v189: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 19 op/s 2024-09-19T04:52:55.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:55 smithi193 ceph-mon[35592]: pgmap v189: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 19 op/s 2024-09-19T04:52:57.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:57 smithi193 ceph-mon[35592]: pgmap v190: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.5 KiB/s wr, 11 op/s 2024-09-19T04:52:57.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:57 smithi183 ceph-mon[35333]: pgmap v190: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.5 KiB/s wr, 11 op/s 2024-09-19T04:52:57.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:57 smithi123 ceph-mon[30511]: pgmap v190: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.5 KiB/s wr, 11 op/s 2024-09-19T04:52:59.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:59 smithi183 ceph-mon[35333]: pgmap v191: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 KiB/s wr, 15 op/s 2024-09-19T04:52:59.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:52:59 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:52:59.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:59 smithi123 ceph-mon[30511]: pgmap v191: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 KiB/s wr, 15 op/s 2024-09-19T04:52:59.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:52:59 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:52:59.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:59 smithi193 ceph-mon[35592]: pgmap v191: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 KiB/s wr, 15 op/s 2024-09-19T04:52:59.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:52:59 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:53:00.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:00 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:53:00.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:00 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:53:00.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:00 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:53:00.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:00 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:53:00.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:00 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:53:00.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:00 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:53:00.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:00 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:53:00.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:00 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:53:00.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:00 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:53:01.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:01 smithi183 ceph-mon[35333]: pgmap v192: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T04:53:01.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:01 smithi123 ceph-mon[30511]: pgmap v192: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T04:53:01.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:01 smithi193 ceph-mon[35592]: pgmap v192: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T04:53:03.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:03 smithi183 ceph-mon[35333]: pgmap v193: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T04:53:03.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:03 smithi123 ceph-mon[30511]: pgmap v193: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T04:53:03.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:03 smithi193 ceph-mon[35592]: pgmap v193: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T04:53:04.751 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:53:05.142 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:53:05.591 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:05 smithi123 ceph-mon[30511]: pgmap v194: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-09-19T04:53:05.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:05 smithi183 ceph-mon[35333]: pgmap v194: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-09-19T04:53:05.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.702+0000 7f1b3d000640 1 -- 172.21.15.123:0/3877072654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b38108f60 msgr2=0x7f1b38110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:05.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.702+0000 7f1b3d000640 1 --2- 172.21.15.123:0/3877072654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b38108f60 0x7f1b38110ae0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009ca0 tx=0x7f1b3002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.702+0000 7f1b3d000640 1 -- 172.21.15.123:0/3877072654 shutdown_connections 2024-09-19T04:53:05.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.702+0000 7f1b3d000640 1 --2- 172.21.15.123:0/3877072654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b38108f60 0x7f1b38110ae0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.702+0000 7f1b3d000640 1 --2- 172.21.15.123:0/3877072654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b381047e0 0x7f1b38108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.702+0000 7f1b3d000640 1 --2- 172.21.15.123:0/3877072654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b38103e30 0x7f1b38104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.702+0000 7f1b3d000640 1 -- 172.21.15.123:0/3877072654 >> 172.21.15.123:0/3877072654 conn(0x7f1b380ff8d0 msgr2=0x7f1b38101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:05.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.702+0000 7f1b3d000640 1 -- 172.21.15.123:0/3877072654 shutdown_connections 2024-09-19T04:53:05.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.702+0000 7f1b3d000640 1 -- 172.21.15.123:0/3877072654 wait complete. 2024-09-19T04:53:05.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.703+0000 7f1b3d000640 1 Processor -- start 2024-09-19T04:53:05.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.703+0000 7f1b3d000640 1 -- start start 2024-09-19T04:53:05.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b3d000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b38103e30 0x7f1b3819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b3d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b381047e0 0x7f1b3819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b3d000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b38108f60 0x7f1b3819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b3d000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38113fc0 con 0x7f1b381047e0 2024-09-19T04:53:05.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b3d000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38113e40 con 0x7f1b38108f60 2024-09-19T04:53:05.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b3d000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38114140 con 0x7f1b38103e30 2024-09-19T04:53:05.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b2ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b381047e0 0x7f1b3819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b37400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b38103e30 0x7f1b3819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b2ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b381047e0 0x7f1b3819a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57550/0 (socket says 172.21.15.123:57550) 2024-09-19T04:53:05.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b37400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b38103e30 0x7f1b3819a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42172/0 (socket says 172.21.15.123:42172) 2024-09-19T04:53:05.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b2ea00640 1 -- 172.21.15.123:0/552778194 learned_addr learned my addr 172.21.15.123:0/552778194 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:05.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b37e00640 1 --2- 172.21.15.123:0/552778194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b38108f60 0x7f1b3819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b37400640 1 -- 172.21.15.123:0/552778194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b38108f60 msgr2=0x7f1b3819ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:05.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b37400640 1 --2- 172.21.15.123:0/552778194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b38108f60 0x7f1b3819ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b37400640 1 -- 172.21.15.123:0/552778194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b381047e0 msgr2=0x7f1b3819a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:05.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b37400640 1 --2- 172.21.15.123:0/552778194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b381047e0 0x7f1b3819a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b37400640 1 -- 172.21.15.123:0/552778194 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3819f390 con 0x7f1b38103e30 2024-09-19T04:53:05.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.704+0000 7f1b2ea00640 1 --2- 172.21.15.123:0/552778194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b381047e0 0x7f1b3819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:05.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.705+0000 7f1b37400640 1 --2- 172.21.15.123:0/552778194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b38103e30 0x7f1b3819a3e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1b28005fd0 tx=0x7f1b2800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:05.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.705+0000 7f1b34c00640 1 -- 172.21.15.123:0/552778194 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28009c30 con 0x7f1b38103e30 2024-09-19T04:53:05.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.705+0000 7f1b3d000640 1 -- 172.21.15.123:0/552778194 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3819f680 con 0x7f1b38103e30 2024-09-19T04:53:05.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.705+0000 7f1b34c00640 1 -- 172.21.15.123:0/552778194 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b28009dd0 con 0x7f1b38103e30 2024-09-19T04:53:05.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.705+0000 7f1b34c00640 1 -- 172.21.15.123:0/552778194 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28010460 con 0x7f1b38103e30 2024-09-19T04:53:05.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.705+0000 7f1b3d000640 1 -- 172.21.15.123:0/552778194 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b381a6f60 con 0x7f1b38103e30 2024-09-19T04:53:05.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.708+0000 7f1b4127b740 1 Processor -- start 2024-09-19T04:53:05.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.708+0000 7f1b4127b740 1 -- start start 2024-09-19T04:53:05.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.708+0000 7f1b34c00640 1 -- 172.21.15.123:0/552778194 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f1b28003f50 con 0x7f1b38103e30 2024-09-19T04:53:05.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.708+0000 7f1b4127b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e7d50 0x5616551e8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.708+0000 7f1b4127b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551e8670 0x5616551f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.708+0000 7f1b4127b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551f2f30 0x5616551f5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.709+0000 7f1b4127b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56165514e3e0 con 0x5616551f2f30 2024-09-19T04:53:05.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.709+0000 7f1b4127b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5616550f5d80 con 0x5616551e7d50 2024-09-19T04:53:05.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.709+0000 7f1b4127b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56165514a3a0 con 0x5616551e8670 2024-09-19T04:53:05.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.709+0000 7f1b2ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551f2f30 0x5616551f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.709+0000 7f1b37e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e7d50 0x5616551e8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.709+0000 7f1b37400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551e8670 0x5616551f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.709+0000 7f1b37e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e7d50 0x5616551e8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:35908/0 (socket says 172.21.15.123:35908) 2024-09-19T04:53:05.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.709+0000 7f1b37e00640 1 -- 172.21.15.123:0/1137729036 learned_addr learned my addr 172.21.15.123:0/1137729036 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:05.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.709+0000 7f1b37e00640 1 -- 172.21.15.123:0/1137729036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551e8670 msgr2=0x5616551f29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:05.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.709+0000 7f1b37e00640 1 --2- 172.21.15.123:0/1137729036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551e8670 0x5616551f29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.709+0000 7f1b37e00640 1 -- 172.21.15.123:0/1137729036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551f2f30 msgr2=0x5616551f5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:05.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.709+0000 7f1b37e00640 1 --2- 172.21.15.123:0/1137729036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551f2f30 0x5616551f5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.709+0000 7f1b37e00640 1 -- 172.21.15.123:0/1137729036 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561655114ae0 con 0x5616551e7d50 2024-09-19T04:53:05.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.710+0000 7f1b37e00640 1 --2- 172.21.15.123:0/1137729036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e7d50 0x5616551e8130 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1b30004af0 tx=0x7f1b300098b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9db24bd4525a163a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:05.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.710+0000 7f1b2cc00640 1 -- 172.21.15.123:0/1137729036 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30003970 con 0x5616551e7d50 2024-09-19T04:53:05.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.710+0000 7f1b2cc00640 1 -- 172.21.15.123:0/1137729036 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b30036040 con 0x5616551e7d50 2024-09-19T04:53:05.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.710+0000 7f1b2cc00640 1 -- 172.21.15.123:0/1137729036 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30004330 con 0x5616551e7d50 2024-09-19T04:53:05.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.710+0000 7f1b34c00640 1 --2- 172.21.15.123:0/552778194 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1b1c0a33b0 0x7f1b1c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.710+0000 7f1b34c00640 1 -- 172.21.15.123:0/552778194 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f1b280cfbe0 con 0x7f1b38103e30 2024-09-19T04:53:05.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.710+0000 7f1b37400640 1 --2- 172.21.15.123:0/552778194 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1b1c0a33b0 0x7f1b1c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.711+0000 7f1b4127b740 1 -- 172.21.15.123:0/1137729036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e7d50 msgr2=0x5616551e8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:05.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.711+0000 7f1b4127b740 1 --2- 172.21.15.123:0/1137729036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e7d50 0x5616551e8130 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1b30004af0 tx=0x7f1b300098b0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.711+0000 7f1b4127b740 1 -- 172.21.15.123:0/1137729036 shutdown_connections 2024-09-19T04:53:05.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.711+0000 7f1b4127b740 1 --2- 172.21.15.123:0/1137729036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551f2f30 0x5616551f5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.711+0000 7f1b4127b740 1 --2- 172.21.15.123:0/1137729036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551e8670 0x5616551f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.711+0000 7f1b37400640 1 --2- 172.21.15.123:0/552778194 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1b1c0a33b0 0x7f1b1c0a5870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x5616551e5140 tx=0x7f1b280d3690 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:05.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.711+0000 7f1b4127b740 1 --2- 172.21.15.123:0/1137729036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e7d50 0x5616551e8130 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.711+0000 7f1b4127b740 1 -- 172.21.15.123:0/1137729036 >> 172.21.15.123:0/1137729036 conn(0x5616551e42e0 msgr2=0x5616551e46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:05.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.711+0000 7f1b4127b740 1 -- 172.21.15.123:0/1137729036 shutdown_connections 2024-09-19T04:53:05.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.711+0000 7f1b4127b740 1 -- 172.21.15.123:0/1137729036 wait complete. 2024-09-19T04:53:05.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.712+0000 7f1b4127b740 1 -- start start 2024-09-19T04:53:05.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.712+0000 7f1b4127b740 1 Processor -- start 2024-09-19T04:53:05.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.712+0000 7f1b4127b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e7d50 0x5616551efe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.712+0000 7f1b4127b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551e8670 0x5616551f0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.712+0000 7f1b4127b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551f2f30 0x5616551f08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.712+0000 7f1b4127b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5616551f5a20 con 0x5616551e8670 2024-09-19T04:53:05.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.712+0000 7f1b4127b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5616551090f0 con 0x5616551e7d50 2024-09-19T04:53:05.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.712+0000 7f1b4127b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5616551f5ba0 con 0x5616551f2f30 2024-09-19T04:53:05.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.712+0000 7f1b4127b740 10 client.0 _ll_register_callbacks cb 0x5616550fa9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:05.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.712+0000 7f1b2ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551e8670 0x5616551f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.713+0000 7f1b37e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551f2f30 0x5616551f08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.713+0000 7f1b2ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551e8670 0x5616551f0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57556/0 (socket says 172.21.15.123:57556) 2024-09-19T04:53:05.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.713+0000 7f1b37e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e7d50 0x5616551efe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.713+0000 7f1b2ea00640 1 -- 172.21.15.123:0/132755807 learned_addr learned my addr 172.21.15.123:0/132755807 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:05.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.713+0000 7f1b2ea00640 1 -- 172.21.15.123:0/132755807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551f2f30 msgr2=0x5616551f08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:05.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.713+0000 7f1b2ea00640 1 --2- 172.21.15.123:0/132755807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551f2f30 0x5616551f08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.713+0000 7f1b2ea00640 1 -- 172.21.15.123:0/132755807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e7d50 msgr2=0x5616551efe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:05.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.713+0000 7f1b2ea00640 1 --2- 172.21.15.123:0/132755807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e7d50 0x5616551efe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.713+0000 7f1b2ea00640 1 -- 172.21.15.123:0/132755807 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561654ebf3c0 con 0x5616551e8670 2024-09-19T04:53:05.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.713+0000 7f1b37e00640 1 --2- 172.21.15.123:0/132755807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e7d50 0x5616551efe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:05.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.713+0000 7f1b2ea00640 1 --2- 172.21.15.123:0/132755807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551e8670 0x5616551f0360 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1b200044a0 tx=0x7f1b200039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6bdc53fef00d6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:05.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.713+0000 7f1b25600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:53:05.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.714+0000 7f1b25600640 1 -- 172.21.15.123:0/132755807 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20011830 con 0x5616551e8670 2024-09-19T04:53:05.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.714+0000 7f1b25600640 1 -- 172.21.15.123:0/132755807 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f1b200119d0 con 0x5616551e8670 2024-09-19T04:53:05.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.714+0000 7f1b4127b740 1 -- 172.21.15.123:0/132755807 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b30036040 con 0x5616551e8670 2024-09-19T04:53:05.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.714+0000 7f1b25600640 1 -- 172.21.15.123:0/132755807 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1b20011bd0 con 0x5616551e8670 2024-09-19T04:53:05.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.714+0000 7f1b4127b740 10 client.16341 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:05.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.714+0000 7f1b4127b740 1 -- 172.21.15.123:0/132755807 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561655201730 con 0x5616551e8670 2024-09-19T04:53:05.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.714+0000 7f1b25600640 1 -- 172.21.15.123:0/132755807 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b30036040 con 0x5616551e8670 2024-09-19T04:53:05.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.714+0000 7f1b25600640 1 -- 172.21.15.123:0/132755807 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f1b20024450 con 0x5616551e8670 2024-09-19T04:53:05.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.714+0000 7f1b4127b740 10 client.16341 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:05.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.714+0000 7f1b4127b740 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:53:05.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.714+0000 7f1b4127b740 20 client.16341 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:05.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.714+0000 7f1b4127b740 1 --2- 172.21.15.123:0/132755807 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x561655202b40 0x561655204f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.714+0000 7f1b4127b740 4 client.16341 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:53:05.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.714+0000 7f1b4127b740 1 -- 172.21.15.123:0/132755807 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56165515b950 con 0x561655202b40 2024-09-19T04:53:05.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.715+0000 7f1b37e00640 1 --2- 172.21.15.123:0/132755807 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x561655202b40 0x561655204f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.715+0000 7f1b37e00640 1 --2- 172.21.15.123:0/132755807 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x561655202b40 0x561655204f30 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7abde73028346bab server_cookie=1e56a14e99966e9c in_seq=0 out_seq=0 2024-09-19T04:53:05.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.715+0000 7f1b25600640 10 client.16341 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:05.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.716+0000 7f1b25600640 1 -- 172.21.15.123:0/132755807 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56165515b950 con 0x561655202b40 2024-09-19T04:53:05.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.716+0000 7f1b25600640 10 client.16341 handle_command_reply: tid=0 2024-09-19T04:53:05.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.716+0000 7f1b4127b740 1 client.16341 shutdown 2024-09-19T04:53:05.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.716+0000 7f1b4127b740 1 -- 172.21.15.123:0/132755807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551e8670 msgr2=0x5616551f0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:05.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.716+0000 7f1b4127b740 1 --2- 172.21.15.123:0/132755807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551e8670 0x5616551f0360 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1b200044a0 tx=0x7f1b200039c0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.716+0000 7f1b4127b740 1 -- 172.21.15.123:0/132755807 shutdown_connections 2024-09-19T04:53:05.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.716+0000 7f1b4127b740 1 --2- 172.21.15.123:0/132755807 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x561655202b40 0x561655204f30 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.717+0000 7f1b4127b740 1 --2- 172.21.15.123:0/132755807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551f2f30 0x5616551f08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.717+0000 7f1b4127b740 1 --2- 172.21.15.123:0/132755807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551e8670 0x5616551f0360 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.717+0000 7f1b4127b740 1 --2- 172.21.15.123:0/132755807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e7d50 0x5616551efe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.717+0000 7f1b4127b740 1 -- 172.21.15.123:0/132755807 >> 172.21.15.123:0/132755807 conn(0x5616551e42e0 msgr2=0x5616551e9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:05.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.717+0000 7f1b25600640 0 client.16341 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:05.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.717+0000 7f1b4127b740 1 -- 172.21.15.123:0/132755807 shutdown_connections 2024-09-19T04:53:05.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.717+0000 7f1b4127b740 1 -- 172.21.15.123:0/132755807 wait complete. 2024-09-19T04:53:05.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.717+0000 7f1b4127b740 20 client.16341 trim_cache size 0 max 16384 2024-09-19T04:53:05.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.735+0000 7f1b4127b740 1 Processor -- start 2024-09-19T04:53:05.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.735+0000 7f1b4127b740 1 -- start start 2024-09-19T04:53:05.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.735+0000 7f1b4127b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551e7d50 0x56165520a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.735+0000 7f1b4127b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e8670 0x56165520ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.735+0000 7f1b4127b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551f2f30 0x5616551ed0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.735+0000 7f1b4127b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381af7a0 con 0x5616551f2f30 2024-09-19T04:53:05.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.735+0000 7f1b4127b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381b1370 con 0x5616551e8670 2024-09-19T04:53:05.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.735+0000 7f1b4127b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381afea0 con 0x5616551e7d50 2024-09-19T04:53:05.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.735+0000 7f1b37e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551f2f30 0x5616551ed0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.735+0000 7f1b37e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551e7d50 0x56165520a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.735+0000 7f1b37e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551e7d50 0x56165520a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42202/0 (socket says 172.21.15.123:42202) 2024-09-19T04:53:05.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.735+0000 7f1b37e00640 1 -- 172.21.15.123:0/3791260889 learned_addr learned my addr 172.21.15.123:0/3791260889 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:05.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.735+0000 7f1b2ea00640 1 --2- 172.21.15.123:0/3791260889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e8670 0x56165520ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.736+0000 7f1b37e00640 1 -- 172.21.15.123:0/3791260889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e8670 msgr2=0x56165520ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:05.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.736+0000 7f1b37e00640 1 --2- 172.21.15.123:0/3791260889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e8670 0x56165520ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.736+0000 7f1b37e00640 1 -- 172.21.15.123:0/3791260889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551f2f30 msgr2=0x5616551ed0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:05.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.736+0000 7f1b37e00640 1 --2- 172.21.15.123:0/3791260889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551f2f30 0x5616551ed0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.736+0000 7f1b37e00640 1 -- 172.21.15.123:0/3791260889 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616551e5390 con 0x5616551e7d50 2024-09-19T04:53:05.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.736+0000 7f1b37e00640 1 --2- 172.21.15.123:0/3791260889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551e7d50 0x56165520a840 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1b30004af0 tx=0x7f1b300098b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f27bebb98175dafc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:05.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.736+0000 7f1b27e00640 1 -- 172.21.15.123:0/3791260889 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3002f610 con 0x5616551e7d50 2024-09-19T04:53:05.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.736+0000 7f1b27e00640 1 -- 172.21.15.123:0/3791260889 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b30036070 con 0x5616551e7d50 2024-09-19T04:53:05.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.736+0000 7f1b27e00640 1 -- 172.21.15.123:0/3791260889 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30030940 con 0x5616551e7d50 2024-09-19T04:53:05.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.737+0000 7f1b4127b740 1 -- 172.21.15.123:0/3791260889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551e7d50 msgr2=0x56165520a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:05.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.737+0000 7f1b4127b740 1 --2- 172.21.15.123:0/3791260889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551e7d50 0x56165520a840 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1b30004af0 tx=0x7f1b300098b0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.737+0000 7f1b4127b740 1 -- 172.21.15.123:0/3791260889 shutdown_connections 2024-09-19T04:53:05.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.737+0000 7f1b4127b740 1 --2- 172.21.15.123:0/3791260889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551f2f30 0x5616551ed0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.737+0000 7f1b4127b740 1 --2- 172.21.15.123:0/3791260889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e8670 0x56165520ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.737+0000 7f1b4127b740 1 --2- 172.21.15.123:0/3791260889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551e7d50 0x56165520a840 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.737+0000 7f1b4127b740 1 -- 172.21.15.123:0/3791260889 >> 172.21.15.123:0/3791260889 conn(0x5616551e42e0 msgr2=0x5616551e9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:05.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.737+0000 7f1b4127b740 1 -- 172.21.15.123:0/3791260889 shutdown_connections 2024-09-19T04:53:05.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.737+0000 7f1b4127b740 1 -- 172.21.15.123:0/3791260889 wait complete. 2024-09-19T04:53:05.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.737+0000 7f1b4127b740 1 -- start start 2024-09-19T04:53:05.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.738+0000 7f1b4127b740 1 Processor -- start 2024-09-19T04:53:05.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.738+0000 7f1b4127b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551e7d50 0x5616551fadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.738+0000 7f1b4127b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e8670 0x5616551fb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.738+0000 7f1b4127b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551f2f30 0x5616551f7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.738+0000 7f1b4127b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381b0ca0 con 0x5616551e7d50 2024-09-19T04:53:05.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.738+0000 7f1b4127b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381b05a0 con 0x5616551e8670 2024-09-19T04:53:05.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.738+0000 7f1b4127b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381ad780 con 0x5616551f2f30 2024-09-19T04:53:05.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.738+0000 7f1b4127b740 10 client.0 _ll_register_callbacks cb 0x7f1b200040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:05.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.738+0000 7f1b37e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551e7d50 0x5616551fadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.738+0000 7f1b37e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551e7d50 0x5616551fadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57572/0 (socket says 172.21.15.123:57572) 2024-09-19T04:53:05.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.738+0000 7f1b37e00640 1 -- 172.21.15.123:0/1166154550 learned_addr learned my addr 172.21.15.123:0/1166154550 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:05.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.738+0000 7f1b2ea00640 1 --2- 172.21.15.123:0/1166154550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e8670 0x5616551fb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.738+0000 7f1b37e00640 1 --2- 172.21.15.123:0/1166154550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551f2f30 0x5616551f7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.739+0000 7f1b37e00640 1 -- 172.21.15.123:0/1166154550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551f2f30 msgr2=0x5616551f7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:05.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.739+0000 7f1b37e00640 1 --2- 172.21.15.123:0/1166154550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551f2f30 0x5616551f7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.739+0000 7f1b37e00640 1 -- 172.21.15.123:0/1166154550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e8670 msgr2=0x5616551fb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:05.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.739+0000 7f1b37e00640 1 --2- 172.21.15.123:0/1166154550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e8670 0x5616551fb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.739+0000 7f1b37e00640 1 -- 172.21.15.123:0/1166154550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616551fb970 con 0x5616551e7d50 2024-09-19T04:53:05.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.739+0000 7f1b2ea00640 1 --2- 172.21.15.123:0/1166154550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e8670 0x5616551fb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:05.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.739+0000 7f1b37e00640 1 --2- 172.21.15.123:0/1166154550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551e7d50 0x5616551fadf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1b30005e30 tx=0x7f1b3002f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f6d08bb40ffde0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:05.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.739+0000 7f1b25600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:53:05.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.739+0000 7f1b25600640 1 -- 172.21.15.123:0/1166154550 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30062060 con 0x5616551e7d50 2024-09-19T04:53:05.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.739+0000 7f1b25600640 1 -- 172.21.15.123:0/1166154550 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f1b3002f8a0 con 0x5616551e7d50 2024-09-19T04:53:05.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.739+0000 7f1b4127b740 1 -- 172.21.15.123:0/1166154550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b30036070 con 0x5616551e7d50 2024-09-19T04:53:05.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.739+0000 7f1b25600640 1 -- 172.21.15.123:0/1166154550 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1b3002faa0 con 0x5616551e7d50 2024-09-19T04:53:05.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.739+0000 7f1b4127b740 10 client.16347 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:05.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.739+0000 7f1b4127b740 1 -- 172.21.15.123:0/1166154550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56165526dd70 con 0x5616551e7d50 2024-09-19T04:53:05.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.740+0000 7f1b25600640 1 -- 172.21.15.123:0/1166154550 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b30036070 con 0x5616551e7d50 2024-09-19T04:53:05.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.740+0000 7f1b25600640 1 -- 172.21.15.123:0/1166154550 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f1b3005c070 con 0x5616551e7d50 2024-09-19T04:53:05.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.740+0000 7f1b4127b740 10 client.16347 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:05.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.740+0000 7f1b4127b740 10 client.16347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:53:05.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.740+0000 7f1b4127b740 20 client.16347 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:05.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.740+0000 7f1b4127b740 1 --2- 172.21.15.123:0/1166154550 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56165526e0b0 0x5616552704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:05.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.740+0000 7f1b4127b740 4 client.16347 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:53:05.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.740+0000 7f1b4127b740 1 -- 172.21.15.123:0/1166154550 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561655155e60 con 0x56165526e0b0 2024-09-19T04:53:05.746 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:05 smithi193 ceph-mon[35592]: pgmap v194: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-09-19T04:53:05.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.740+0000 7f1b2ea00640 1 --2- 172.21.15.123:0/1166154550 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56165526e0b0 0x5616552704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:05.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.741+0000 7f1b2ea00640 1 --2- 172.21.15.123:0/1166154550 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56165526e0b0 0x5616552704a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e358f4313caf6480 server_cookie=22912830c5871646 in_seq=0 out_seq=0 2024-09-19T04:53:05.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.741+0000 7f1b25600640 10 client.16347 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:05.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.741+0000 7f1b25600640 1 -- 172.21.15.123:0/1166154550 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561655155e60 con 0x56165526e0b0 2024-09-19T04:53:05.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.741+0000 7f1b25600640 10 client.16347 handle_command_reply: tid=0 2024-09-19T04:53:05.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.741+0000 7f1b4127b740 1 client.16347 shutdown 2024-09-19T04:53:05.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.741+0000 7f1b4127b740 1 -- 172.21.15.123:0/1166154550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551e7d50 msgr2=0x5616551fadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:05.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.741+0000 7f1b4127b740 1 --2- 172.21.15.123:0/1166154550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551e7d50 0x5616551fadf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1b30005e30 tx=0x7f1b3002f310 comp rx=0 tx=0).stop 2024-09-19T04:53:05.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.741+0000 7f1b4127b740 1 -- 172.21.15.123:0/1166154550 shutdown_connections 2024-09-19T04:53:05.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.741+0000 7f1b4127b740 1 --2- 172.21.15.123:0/1166154550 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56165526e0b0 0x5616552704a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b4127b740 1 --2- 172.21.15.123:0/1166154550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616551f2f30 0x5616551f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b4127b740 1 --2- 172.21.15.123:0/1166154550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616551e8670 0x5616551fb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b4127b740 1 --2- 172.21.15.123:0/1166154550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616551e7d50 0x5616551fadf0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b25600640 0 client.16347 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:05.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b4127b740 1 -- 172.21.15.123:0/1166154550 >> 172.21.15.123:0/1166154550 conn(0x5616551e42e0 msgr2=0x5616551e2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:05.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b4127b740 1 -- 172.21.15.123:0/1166154550 shutdown_connections 2024-09-19T04:53:05.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b4127b740 1 -- 172.21.15.123:0/1166154550 wait complete. 2024-09-19T04:53:05.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b4127b740 20 client.16347 trim_cache size 0 max 16384 2024-09-19T04:53:05.749 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:53:05.749 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-09-19T04:53:05.749 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-09-19T04:53:05.749 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:53:05.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b3d000640 1 -- 172.21.15.123:0/552778194 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1b1c0a33b0 msgr2=0x7f1b1c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:05.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b3d000640 1 --2- 172.21.15.123:0/552778194 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1b1c0a33b0 0x7f1b1c0a5870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x5616551e5140 tx=0x7f1b280d3690 comp rx=0 tx=0).stop 2024-09-19T04:53:05.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b3d000640 1 -- 172.21.15.123:0/552778194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b38103e30 msgr2=0x7f1b3819a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:05.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b3d000640 1 --2- 172.21.15.123:0/552778194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b38103e30 0x7f1b3819a3e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1b28005fd0 tx=0x7f1b2800dde0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b3d000640 1 -- 172.21.15.123:0/552778194 shutdown_connections 2024-09-19T04:53:05.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b3d000640 1 --2- 172.21.15.123:0/552778194 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1b1c0a33b0 0x7f1b1c0a5870 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b3d000640 1 --2- 172.21.15.123:0/552778194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b38108f60 0x7f1b3819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b3d000640 1 --2- 172.21.15.123:0/552778194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b381047e0 0x7f1b3819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b3d000640 1 --2- 172.21.15.123:0/552778194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b38103e30 0x7f1b3819a3e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:05.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.742+0000 7f1b3d000640 1 -- 172.21.15.123:0/552778194 >> 172.21.15.123:0/552778194 conn(0x7f1b380ff8d0 msgr2=0x7f1b38100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:05.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.743+0000 7f1b3d000640 1 -- 172.21.15.123:0/552778194 shutdown_connections 2024-09-19T04:53:05.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:05.743+0000 7f1b3d000640 1 -- 172.21.15.123:0/552778194 wait complete. 2024-09-19T04:53:06.653 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:53:06.653 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:53:07.024 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:53:07.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:07 smithi123 ceph-mon[30511]: pgmap v195: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T04:53:07.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.584+0000 7f1e40a00640 1 -- 172.21.15.123:0/2939568535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e3c1019c0 msgr2=0x7f1e3c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:07.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.584+0000 7f1e40a00640 1 --2- 172.21.15.123:0/2939568535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e3c1019c0 0x7f1e3c101e20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009890 tx=0x7f1e2802a360 comp rx=0 tx=0).stop 2024-09-19T04:53:07.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.584+0000 7f1e40a00640 1 -- 172.21.15.123:0/2939568535 shutdown_connections 2024-09-19T04:53:07.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.584+0000 7f1e40a00640 1 --2- 172.21.15.123:0/2939568535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e3c102360 0x7f1e3c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:07.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.584+0000 7f1e40a00640 1 --2- 172.21.15.123:0/2939568535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e3c1019c0 0x7f1e3c101e20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:07.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.584+0000 7f1e40a00640 1 --2- 172.21.15.123:0/2939568535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e3c1079c0 0x7f1e3c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:07.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.584+0000 7f1e40a00640 1 -- 172.21.15.123:0/2939568535 >> 172.21.15.123:0/2939568535 conn(0x7f1e3c0fd740 msgr2=0x7f1e3c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:07.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.584+0000 7f1e40a00640 1 -- 172.21.15.123:0/2939568535 shutdown_connections 2024-09-19T04:53:07.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.584+0000 7f1e40a00640 1 -- 172.21.15.123:0/2939568535 wait complete. 2024-09-19T04:53:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.586+0000 7f1e40a00640 1 Processor -- start 2024-09-19T04:53:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.586+0000 7f1e40a00640 1 -- start start 2024-09-19T04:53:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.586+0000 7f1e40a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e3c1019c0 0x7f1e3c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.586+0000 7f1e40a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e3c102360 0x7f1e3c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.586+0000 7f1e40a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e3c1079c0 0x7f1e3c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.586+0000 7f1e40a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c111dc0 con 0x7f1e3c1079c0 2024-09-19T04:53:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.586+0000 7f1e40a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c111c40 con 0x7f1e3c1019c0 2024-09-19T04:53:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.586+0000 7f1e40a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c111f40 con 0x7f1e3c102360 2024-09-19T04:53:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.586+0000 7f1e3b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e3c1079c0 0x7f1e3c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.587+0000 7f1e3a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e3c1019c0 0x7f1e3c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.587+0000 7f1e3a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e3c1019c0 0x7f1e3c1982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:35938/0 (socket says 172.21.15.123:35938) 2024-09-19T04:53:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.587+0000 7f1e3a800640 1 -- 172.21.15.123:0/3401998310 learned_addr learned my addr 172.21.15.123:0/3401998310 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.587+0000 7f1e39e00640 1 --2- 172.21.15.123:0/3401998310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e3c102360 0x7f1e3c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.587+0000 7f1e3a800640 1 -- 172.21.15.123:0/3401998310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e3c102360 msgr2=0x7f1e3c1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.587+0000 7f1e3a800640 1 --2- 172.21.15.123:0/3401998310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e3c102360 0x7f1e3c1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.587+0000 7f1e3a800640 1 -- 172.21.15.123:0/3401998310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e3c1079c0 msgr2=0x7f1e3c19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.587+0000 7f1e3a800640 1 --2- 172.21.15.123:0/3401998310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e3c1079c0 0x7f1e3c19cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.587+0000 7f1e3a800640 1 -- 172.21.15.123:0/3401998310 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e3c19d250 con 0x7f1e3c1019c0 2024-09-19T04:53:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.587+0000 7f1e3a800640 1 --2- 172.21.15.123:0/3401998310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e3c1019c0 0x7f1e3c1982a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00e9c0 tx=0x7f1e2c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:07.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.587+0000 7f1e33400640 1 -- 172.21.15.123:0/3401998310 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c00cde0 con 0x7f1e3c1019c0 2024-09-19T04:53:07.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.588+0000 7f1e40a00640 1 -- 172.21.15.123:0/3401998310 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e3c19d540 con 0x7f1e3c1019c0 2024-09-19T04:53:07.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.588+0000 7f1e33400640 1 -- 172.21.15.123:0/3401998310 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e2c0035f0 con 0x7f1e3c1019c0 2024-09-19T04:53:07.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.588+0000 7f1e33400640 1 -- 172.21.15.123:0/3401998310 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c010650 con 0x7f1e3c1019c0 2024-09-19T04:53:07.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.588+0000 7f1e40a00640 1 -- 172.21.15.123:0/3401998310 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e3c074420 con 0x7f1e3c1019c0 2024-09-19T04:53:07.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.589+0000 7f1e40a00640 1 -- 172.21.15.123:0/3401998310 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e3c101e20 con 0x7f1e3c1019c0 2024-09-19T04:53:07.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.591+0000 7f1e33400640 1 -- 172.21.15.123:0/3401998310 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f1e2c0037a0 con 0x7f1e3c1019c0 2024-09-19T04:53:07.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.592+0000 7f1e33400640 1 --2- 172.21.15.123:0/3401998310 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1e200a3430 0x7f1e200a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:07.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.592+0000 7f1e33400640 1 -- 172.21.15.123:0/3401998310 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8469+0+0 (secure 0 0 0) 0x7f1e2c0cffa0 con 0x7f1e3c1019c0 2024-09-19T04:53:07.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.592+0000 7f1e39e00640 1 --2- 172.21.15.123:0/3401998310 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1e200a3430 0x7f1e200a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:07.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.593+0000 7f1e39e00640 1 --2- 172.21.15.123:0/3401998310 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1e200a3430 0x7f1e200a58f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1e280097b0 tx=0x7f1e280058d0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:07.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.596+0000 7f1e33400640 1 -- 172.21.15.123:0/3401998310 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1e2c09c950 con 0x7f1e3c1019c0 2024-09-19T04:53:07.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:07 smithi183 ceph-mon[35333]: pgmap v195: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T04:53:07.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:07 smithi193 ceph-mon[35592]: pgmap v195: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T04:53:07.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.803+0000 7f1e40a00640 1 -- 172.21.15.123:0/3401998310 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e3c199600 con 0x7f1e3c1019c0 2024-09-19T04:53:07.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.804+0000 7f1e33400640 1 -- 172.21.15.123:0/3401998310 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11994 (secure 0 0 0) 0x7f1e2c09c250 con 0x7f1e3c1019c0 2024-09-19T04:53:07.806 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:53:07.806 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":40,"btime":"2024-09-19T04:52:39:230121+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:52:38.841872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24499,24500,24539,24505,24529]},"id":1}]} 2024-09-19T04:53:07.806 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 40 2024-09-19T04:53:07.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.806+0000 7f1e40a00640 1 -- 172.21.15.123:0/3401998310 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1e200a3430 msgr2=0x7f1e200a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:07.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.806+0000 7f1e40a00640 1 --2- 172.21.15.123:0/3401998310 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1e200a3430 0x7f1e200a58f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1e280097b0 tx=0x7f1e280058d0 comp rx=0 tx=0).stop 2024-09-19T04:53:07.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.806+0000 7f1e40a00640 1 -- 172.21.15.123:0/3401998310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e3c1019c0 msgr2=0x7f1e3c1982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:07.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.806+0000 7f1e40a00640 1 --2- 172.21.15.123:0/3401998310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e3c1019c0 0x7f1e3c1982a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00e9c0 tx=0x7f1e2c00ee80 comp rx=0 tx=0).stop 2024-09-19T04:53:07.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.806+0000 7f1e40a00640 1 -- 172.21.15.123:0/3401998310 shutdown_connections 2024-09-19T04:53:07.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.806+0000 7f1e40a00640 1 --2- 172.21.15.123:0/3401998310 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1e200a3430 0x7f1e200a58f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:07.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.806+0000 7f1e40a00640 1 --2- 172.21.15.123:0/3401998310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e3c1079c0 0x7f1e3c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:07.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.806+0000 7f1e40a00640 1 --2- 172.21.15.123:0/3401998310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e3c102360 0x7f1e3c1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:07.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.806+0000 7f1e40a00640 1 --2- 172.21.15.123:0/3401998310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e3c1019c0 0x7f1e3c1982a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:07.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.806+0000 7f1e40a00640 1 -- 172.21.15.123:0/3401998310 >> 172.21.15.123:0/3401998310 conn(0x7f1e3c0fd740 msgr2=0x7f1e3c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:07.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.807+0000 7f1e40a00640 1 -- 172.21.15.123:0/3401998310 shutdown_connections 2024-09-19T04:53:07.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:07.807+0000 7f1e40a00640 1 -- 172.21.15.123:0/3401998310 wait complete. 2024-09-19T04:53:08.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:08 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3401998310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:53:08.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:08 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3401998310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:53:08.653 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:53:08.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:08 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3401998310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:53:09.021 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:53:09.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:09 smithi123 ceph-mon[30511]: pgmap v196: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T04:53:09.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:09 smithi183 ceph-mon[35333]: pgmap v196: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T04:53:09.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.655+0000 7fdade800640 1 -- 172.21.15.123:0/1272925378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdad8102360 msgr2=0x7fdad810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:09.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.655+0000 7fdade800640 1 --2- 172.21.15.123:0/1272925378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdad8102360 0x7fdad810e890 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009930 tx=0x7fdad002a420 comp rx=0 tx=0).stop 2024-09-19T04:53:09.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.655+0000 7fdade800640 1 -- 172.21.15.123:0/1272925378 shutdown_connections 2024-09-19T04:53:09.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.655+0000 7fdade800640 1 --2- 172.21.15.123:0/1272925378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdad8102360 0x7fdad810e890 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.655+0000 7fdade800640 1 --2- 172.21.15.123:0/1272925378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdad81019c0 0x7fdad8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.655+0000 7fdade800640 1 --2- 172.21.15.123:0/1272925378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdad81079c0 0x7fdad8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.655+0000 7fdade800640 1 -- 172.21.15.123:0/1272925378 >> 172.21.15.123:0/1272925378 conn(0x7fdad80fd740 msgr2=0x7fdad80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:09.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.655+0000 7fdade800640 1 -- 172.21.15.123:0/1272925378 shutdown_connections 2024-09-19T04:53:09.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.655+0000 7fdade800640 1 -- 172.21.15.123:0/1272925378 wait complete. 2024-09-19T04:53:09.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.656+0000 7fdade800640 1 Processor -- start 2024-09-19T04:53:09.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.656+0000 7fdade800640 1 -- start start 2024-09-19T04:53:09.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.657+0000 7fdade800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdad81019c0 0x7fdad819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.657+0000 7fdade800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdad8102360 0x7fdad819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.657+0000 7fdade800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdad81079c0 0x7fdad819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.657+0000 7fdade800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdad8111e10 con 0x7fdad81019c0 2024-09-19T04:53:09.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.657+0000 7fdade800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdad8111c90 con 0x7fdad81079c0 2024-09-19T04:53:09.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.657+0000 7fdade800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdad8111f90 con 0x7fdad8102360 2024-09-19T04:53:09.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.657+0000 7fdadd400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdad81019c0 0x7fdad819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.657+0000 7fdadca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdad8102360 0x7fdad819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.657+0000 7fdadd400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdad81019c0 0x7fdad819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57624/0 (socket says 172.21.15.123:57624) 2024-09-19T04:53:09.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.657+0000 7fdadd400640 1 -- 172.21.15.123:0/3903848760 learned_addr learned my addr 172.21.15.123:0/3903848760 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:09.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.657+0000 7fdadde00640 1 --2- 172.21.15.123:0/3903848760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdad81079c0 0x7fdad819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.657+0000 7fdadca00640 1 -- 172.21.15.123:0/3903848760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdad81079c0 msgr2=0x7fdad819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:09.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.657+0000 7fdadca00640 1 --2- 172.21.15.123:0/3903848760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdad81079c0 0x7fdad819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.657+0000 7fdadca00640 1 -- 172.21.15.123:0/3903848760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdad81019c0 msgr2=0x7fdad819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:09.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.657+0000 7fdadca00640 1 --2- 172.21.15.123:0/3903848760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdad81019c0 0x7fdad819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.657+0000 7fdadca00640 1 -- 172.21.15.123:0/3903848760 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad819f430 con 0x7fdad8102360 2024-09-19T04:53:09.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.658+0000 7fdadca00640 1 --2- 172.21.15.123:0/3903848760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdad8102360 0x7fdad819a9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdac400e9f0 tx=0x7fdac400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:09.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.658+0000 7fdad6000640 1 -- 172.21.15.123:0/3903848760 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac400ccc0 con 0x7fdad8102360 2024-09-19T04:53:09.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.658+0000 7fdade800640 1 -- 172.21.15.123:0/3903848760 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad819f720 con 0x7fdad8102360 2024-09-19T04:53:09.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.658+0000 7fdad6000640 1 -- 172.21.15.123:0/3903848760 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac4005020 con 0x7fdad8102360 2024-09-19T04:53:09.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.658+0000 7fdad6000640 1 -- 172.21.15.123:0/3903848760 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac40049e0 con 0x7fdad8102360 2024-09-19T04:53:09.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.658+0000 7fdade800640 1 -- 172.21.15.123:0/3903848760 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad81a7000 con 0x7fdad8102360 2024-09-19T04:53:09.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.661+0000 7fdae29ca740 1 Processor -- start 2024-09-19T04:53:09.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.661+0000 7fdae29ca740 1 -- start start 2024-09-19T04:53:09.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.661+0000 7fdae29ca740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807ce40 0x55d63807d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.661+0000 7fdae29ca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d63807d760 0x55d638087ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.661+0000 7fdae29ca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d638088020 0x55d63808a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.661+0000 7fdae29ca740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d637fe33e0 con 0x55d63807ce40 2024-09-19T04:53:09.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.661+0000 7fdae29ca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d637f8ad80 con 0x55d638088020 2024-09-19T04:53:09.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.661+0000 7fdae29ca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d637fdf3a0 con 0x55d63807d760 2024-09-19T04:53:09.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.662+0000 7fdad6000640 1 -- 172.21.15.123:0/3903848760 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fdac4005560 con 0x7fdad8102360 2024-09-19T04:53:09.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.662+0000 7fdadde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807ce40 0x55d63807d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.662+0000 7fdadde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807ce40 0x55d63807d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57634/0 (socket says 172.21.15.123:57634) 2024-09-19T04:53:09.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.662+0000 7fdadde00640 1 -- 172.21.15.123:0/1087072553 learned_addr learned my addr 172.21.15.123:0/1087072553 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:09.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.662+0000 7fdadd400640 1 --2- 172.21.15.123:0/1087072553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d63807d760 0x55d638087ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.662+0000 7fdadca00640 1 --2- 172.21.15.123:0/1087072553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d638088020 0x55d63808a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.662+0000 7fdadde00640 1 -- 172.21.15.123:0/1087072553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d63807d760 msgr2=0x55d638087ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:09.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.662+0000 7fdadde00640 1 --2- 172.21.15.123:0/1087072553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d63807d760 0x55d638087ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.662+0000 7fdadde00640 1 -- 172.21.15.123:0/1087072553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d638088020 msgr2=0x55d63808a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:09.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.662+0000 7fdadde00640 1 --2- 172.21.15.123:0/1087072553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d638088020 0x55d63808a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.662+0000 7fdadde00640 1 -- 172.21.15.123:0/1087072553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d637fa9ae0 con 0x55d63807ce40 2024-09-19T04:53:09.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.662+0000 7fdadca00640 1 --2- 172.21.15.123:0/1087072553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d638088020 0x55d63808a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:53:09.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.663+0000 7fdadde00640 1 --2- 172.21.15.123:0/1087072553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807ce40 0x55d63807d220 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009930 tx=0x7fdad0005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfd27167631a6ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:09.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.663+0000 7fdaca600640 1 -- 172.21.15.123:0/1087072553 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0040070 con 0x55d63807ce40 2024-09-19T04:53:09.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.663+0000 7fdaca600640 1 -- 172.21.15.123:0/1087072553 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad0004340 con 0x55d63807ce40 2024-09-19T04:53:09.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.663+0000 7fdaca600640 1 -- 172.21.15.123:0/1087072553 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0035040 con 0x55d63807ce40 2024-09-19T04:53:09.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.663+0000 7fdad6000640 1 --2- 172.21.15.123:0/3903848760 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdac00a33b0 0x7fdac00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.663+0000 7fdad6000640 1 -- 172.21.15.123:0/3903848760 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7fdac40cfd20 con 0x7fdad8102360 2024-09-19T04:53:09.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.663+0000 7fdae29ca740 1 -- 172.21.15.123:0/1087072553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807ce40 msgr2=0x55d63807d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:09.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.663+0000 7fdae29ca740 1 --2- 172.21.15.123:0/1087072553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807ce40 0x55d63807d220 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009930 tx=0x7fdad0005e20 comp rx=0 tx=0).stop 2024-09-19T04:53:09.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.663+0000 7fdadd400640 1 --2- 172.21.15.123:0/3903848760 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdac00a33b0 0x7fdac00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.664+0000 7fdae29ca740 1 -- 172.21.15.123:0/1087072553 shutdown_connections 2024-09-19T04:53:09.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.664+0000 7fdae29ca740 1 --2- 172.21.15.123:0/1087072553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d638088020 0x55d63808a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.664+0000 7fdae29ca740 1 --2- 172.21.15.123:0/1087072553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d63807d760 0x55d638087ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.664+0000 7fdae29ca740 1 --2- 172.21.15.123:0/1087072553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807ce40 0x55d63807d220 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.664+0000 7fdae29ca740 1 -- 172.21.15.123:0/1087072553 >> 172.21.15.123:0/1087072553 conn(0x55d6380793d0 msgr2=0x55d6380797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:09.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.664+0000 7fdae29ca740 1 -- 172.21.15.123:0/1087072553 shutdown_connections 2024-09-19T04:53:09.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.664+0000 7fdae29ca740 1 -- 172.21.15.123:0/1087072553 wait complete. 2024-09-19T04:53:09.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.664+0000 7fdae29ca740 1 -- start start 2024-09-19T04:53:09.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.664+0000 7fdadd400640 1 --2- 172.21.15.123:0/3903848760 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdac00a33b0 0x7fdac00a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55d63807a230 tx=0x7fdacc009800 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:09.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.664+0000 7fdae29ca740 1 Processor -- start 2024-09-19T04:53:09.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.665+0000 7fdae29ca740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807ce40 0x55d638084f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.665+0000 7fdae29ca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63807d760 0x55d638085480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.665+0000 7fdae29ca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d638088020 0x55d6380859c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.665+0000 7fdae29ca740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d63808abf0 con 0x55d63807ce40 2024-09-19T04:53:09.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.665+0000 7fdae29ca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d637f9e0f0 con 0x55d63807d760 2024-09-19T04:53:09.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.665+0000 7fdae29ca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d63808ad70 con 0x55d638088020 2024-09-19T04:53:09.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.665+0000 7fdae29ca740 10 client.0 _ll_register_callbacks cb 0x55d637f85bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:09.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.665+0000 7fdadd400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807ce40 0x55d638084f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.665+0000 7fdadca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63807d760 0x55d638085480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.665+0000 7fdadd400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807ce40 0x55d638084f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57650/0 (socket says 172.21.15.123:57650) 2024-09-19T04:53:09.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.665+0000 7fdadd400640 1 -- 172.21.15.123:0/5218015 learned_addr learned my addr 172.21.15.123:0/5218015 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:09.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.665+0000 7fdadde00640 1 --2- 172.21.15.123:0/5218015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d638088020 0x55d6380859c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.666+0000 7fdadca00640 1 -- 172.21.15.123:0/5218015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d638088020 msgr2=0x55d6380859c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:09.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.666+0000 7fdadca00640 1 --2- 172.21.15.123:0/5218015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d638088020 0x55d6380859c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.666+0000 7fdadca00640 1 -- 172.21.15.123:0/5218015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807ce40 msgr2=0x55d638084f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:09.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.666+0000 7fdadca00640 1 --2- 172.21.15.123:0/5218015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807ce40 0x55d638084f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.666+0000 7fdadca00640 1 -- 172.21.15.123:0/5218015 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d637d543c0 con 0x55d63807d760 2024-09-19T04:53:09.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.666+0000 7fdadde00640 1 --2- 172.21.15.123:0/5218015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d638088020 0x55d6380859c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:09.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.666+0000 7fdadca00640 1 --2- 172.21.15.123:0/5218015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63807d760 0x55d638085480 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdac40cd3c0 tx=0x7fdac40051d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abd695997a978853 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:09.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.666+0000 7fdabea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:53:09.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.666+0000 7fdabea00640 1 -- 172.21.15.123:0/5218015 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac401c040 con 0x55d63807d760 2024-09-19T04:53:09.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.666+0000 7fdae29ca740 1 -- 172.21.15.123:0/5218015 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdad0004340 con 0x55d63807d760 2024-09-19T04:53:09.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.666+0000 7fdabea00640 1 -- 172.21.15.123:0/5218015 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7fdac409c3e0 con 0x55d63807d760 2024-09-19T04:53:09.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.667+0000 7fdabea00640 1 -- 172.21.15.123:0/5218015 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdac40cd880 con 0x55d63807d760 2024-09-19T04:53:09.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.667+0000 7fdae29ca740 10 client.26465 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:09.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.667+0000 7fdae29ca740 1 -- 172.21.15.123:0/5218015 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d638096cb0 con 0x55d63807d760 2024-09-19T04:53:09.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.667+0000 7fdabea00640 1 -- 172.21.15.123:0/5218015 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad0004340 con 0x55d63807d760 2024-09-19T04:53:09.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.667+0000 7fdabea00640 1 -- 172.21.15.123:0/5218015 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fdac40b9030 con 0x55d63807d760 2024-09-19T04:53:09.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.667+0000 7fdae29ca740 10 client.26465 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:09.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.667+0000 7fdae29ca740 10 client.26465 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:53:09.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.667+0000 7fdae29ca740 20 client.26465 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:09.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.667+0000 7fdae29ca740 1 --2- 172.21.15.123:0/5218015 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55d6380980c0 0x55d63809a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.667+0000 7fdae29ca740 4 client.26465 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:53:09.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.667+0000 7fdae29ca740 1 -- 172.21.15.123:0/5218015 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d637ff0950 con 0x55d6380980c0 2024-09-19T04:53:09.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.668+0000 7fdadd400640 1 --2- 172.21.15.123:0/5218015 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55d6380980c0 0x55d63809a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.668+0000 7fdadd400640 1 --2- 172.21.15.123:0/5218015 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55d6380980c0 0x55d63809a530 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a40641b69d7217b1 server_cookie=becaf3fc81878d62 in_seq=0 out_seq=0 2024-09-19T04:53:09.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.668+0000 7fdabea00640 10 client.26465 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:53:09.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.669+0000 7fdabea00640 1 -- 172.21.15.123:0/5218015 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d637ff0950 con 0x55d6380980c0 2024-09-19T04:53:09.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.669+0000 7fdabea00640 10 client.26465 handle_command_reply: tid=0 2024-09-19T04:53:09.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.669+0000 7fdae29ca740 1 client.26465 shutdown 2024-09-19T04:53:09.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.669+0000 7fdae29ca740 1 -- 172.21.15.123:0/5218015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63807d760 msgr2=0x55d638085480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:09.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.669+0000 7fdae29ca740 1 --2- 172.21.15.123:0/5218015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63807d760 0x55d638085480 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdac40cd3c0 tx=0x7fdac40051d0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.669+0000 7fdae29ca740 1 -- 172.21.15.123:0/5218015 shutdown_connections 2024-09-19T04:53:09.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.669+0000 7fdae29ca740 1 --2- 172.21.15.123:0/5218015 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55d6380980c0 0x55d63809a530 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.669+0000 7fdae29ca740 1 --2- 172.21.15.123:0/5218015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d638088020 0x55d6380859c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.669+0000 7fdae29ca740 1 --2- 172.21.15.123:0/5218015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63807d760 0x55d638085480 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.669+0000 7fdae29ca740 1 --2- 172.21.15.123:0/5218015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807ce40 0x55d638084f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.669+0000 7fdabea00640 0 client.26465 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:53:09.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.669+0000 7fdae29ca740 1 -- 172.21.15.123:0/5218015 >> 172.21.15.123:0/5218015 conn(0x55d6380793d0 msgr2=0x55d63807f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:09.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.669+0000 7fdae29ca740 1 -- 172.21.15.123:0/5218015 shutdown_connections 2024-09-19T04:53:09.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.670+0000 7fdae29ca740 1 -- 172.21.15.123:0/5218015 wait complete. 2024-09-19T04:53:09.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.670+0000 7fdae29ca740 20 client.26465 trim_cache size 0 max 16384 2024-09-19T04:53:09.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.683+0000 7fdae29ca740 1 Processor -- start 2024-09-19T04:53:09.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.683+0000 7fdae29ca740 1 -- start start 2024-09-19T04:53:09.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.683+0000 7fdae29ca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d63807ce40 0x55d63807f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.683+0000 7fdae29ca740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807d760 0x55d63809fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.684+0000 7fdae29ca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d638088020 0x55d6380a0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.684+0000 7fdae29ca740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81af9b0 con 0x55d63807d760 2024-09-19T04:53:09.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.684+0000 7fdae29ca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81b1580 con 0x55d638088020 2024-09-19T04:53:09.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.684+0000 7fdae29ca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81b00b0 con 0x55d63807ce40 2024-09-19T04:53:09.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.684+0000 7fdadca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807d760 0x55d63809fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.684+0000 7fdadca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807d760 0x55d63809fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57656/0 (socket says 172.21.15.123:57656) 2024-09-19T04:53:09.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.684+0000 7fdadca00640 1 -- 172.21.15.123:0/1095970814 learned_addr learned my addr 172.21.15.123:0/1095970814 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:09.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.684+0000 7fdadd400640 1 --2- 172.21.15.123:0/1095970814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d63807ce40 0x55d63807f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.684+0000 7fdadde00640 1 --2- 172.21.15.123:0/1095970814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d638088020 0x55d6380a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.684+0000 7fdadd400640 1 -- 172.21.15.123:0/1095970814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d638088020 msgr2=0x55d6380a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:09.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.684+0000 7fdadd400640 1 --2- 172.21.15.123:0/1095970814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d638088020 0x55d6380a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.684+0000 7fdadd400640 1 -- 172.21.15.123:0/1095970814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807d760 msgr2=0x55d63809fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:09.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.684+0000 7fdadd400640 1 --2- 172.21.15.123:0/1095970814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807d760 0x55d63809fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.684+0000 7fdadd400640 1 -- 172.21.15.123:0/1095970814 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d63809abb0 con 0x55d63807ce40 2024-09-19T04:53:09.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.684+0000 7fdadca00640 1 --2- 172.21.15.123:0/1095970814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807d760 0x55d63809fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:09.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.685+0000 7fdadd400640 1 --2- 172.21.15.123:0/1095970814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d63807ce40 0x55d63807f8a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55d63809ad80 tx=0x7fdacc019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58b2f9b1b983a509 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:09.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.685+0000 7fdac9c00640 1 -- 172.21.15.123:0/1095970814 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc010e20 con 0x55d63807ce40 2024-09-19T04:53:09.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.685+0000 7fdac9c00640 1 -- 172.21.15.123:0/1095970814 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdacc003e50 con 0x55d63807ce40 2024-09-19T04:53:09.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.685+0000 7fdac9c00640 1 -- 172.21.15.123:0/1095970814 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc004480 con 0x55d63807ce40 2024-09-19T04:53:09.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.686+0000 7fdae29ca740 1 -- 172.21.15.123:0/1095970814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d63807ce40 msgr2=0x55d63807f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:09.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.686+0000 7fdae29ca740 1 --2- 172.21.15.123:0/1095970814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d63807ce40 0x55d63807f8a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55d63809ad80 tx=0x7fdacc019000 comp rx=0 tx=0).stop 2024-09-19T04:53:09.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.686+0000 7fdae29ca740 1 -- 172.21.15.123:0/1095970814 shutdown_connections 2024-09-19T04:53:09.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.686+0000 7fdae29ca740 1 --2- 172.21.15.123:0/1095970814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d638088020 0x55d6380a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.686+0000 7fdae29ca740 1 --2- 172.21.15.123:0/1095970814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d63807d760 0x55d63809fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.686+0000 7fdae29ca740 1 --2- 172.21.15.123:0/1095970814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d63807ce40 0x55d63807f8a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.686+0000 7fdae29ca740 1 -- 172.21.15.123:0/1095970814 >> 172.21.15.123:0/1095970814 conn(0x55d6380793d0 msgr2=0x55d638086fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:09.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.686+0000 7fdae29ca740 1 -- 172.21.15.123:0/1095970814 shutdown_connections 2024-09-19T04:53:09.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.686+0000 7fdae29ca740 1 -- 172.21.15.123:0/1095970814 wait complete. 2024-09-19T04:53:09.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.686+0000 7fdae29ca740 1 -- start start 2024-09-19T04:53:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.686+0000 7fdae29ca740 1 Processor -- start 2024-09-19T04:53:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.687+0000 7fdae29ca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63807ce40 0x55d63808fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.687+0000 7fdae29ca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d63807d760 0x55d638090530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.687+0000 7fdae29ca740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d638088020 0x55d63808cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.687+0000 7fdae29ca740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81b0eb0 con 0x55d638088020 2024-09-19T04:53:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.687+0000 7fdae29ca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81b07b0 con 0x55d63807ce40 2024-09-19T04:53:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.687+0000 7fdae29ca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81ad9b0 con 0x55d63807d760 2024-09-19T04:53:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.687+0000 7fdae29ca740 10 client.0 _ll_register_callbacks cb 0x7fdac40053d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.687+0000 7fdadd400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63807ce40 0x55d63808fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.687+0000 7fdadd400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63807ce40 0x55d63808fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:35988/0 (socket says 172.21.15.123:35988) 2024-09-19T04:53:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.687+0000 7fdadca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d63807d760 0x55d638090530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.687+0000 7fdadd400640 1 -- 172.21.15.123:0/3114599318 learned_addr learned my addr 172.21.15.123:0/3114599318 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:09.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.687+0000 7fdadde00640 1 --2- 172.21.15.123:0/3114599318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d638088020 0x55d63808cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.688+0000 7fdadca00640 1 -- 172.21.15.123:0/3114599318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63807ce40 msgr2=0x55d63808fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:09.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.688+0000 7fdadca00640 1 --2- 172.21.15.123:0/3114599318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63807ce40 0x55d63808fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.688+0000 7fdadca00640 1 -- 172.21.15.123:0/3114599318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d638088020 msgr2=0x55d63808cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:09.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.688+0000 7fdadca00640 1 --2- 172.21.15.123:0/3114599318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d638088020 0x55d63808cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.688+0000 7fdadca00640 1 -- 172.21.15.123:0/3114599318 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d63807a060 con 0x55d63807d760 2024-09-19T04:53:09.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.688+0000 7fdadca00640 1 --2- 172.21.15.123:0/3114599318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d63807d760 0x55d638090530 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdac40bac40 tx=0x7fdac40bc130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fe10bd7001d258f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:09.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.688+0000 7fdabea00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:53:09.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.688+0000 7fdabea00640 1 -- 172.21.15.123:0/3114599318 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac401c070 con 0x55d63807d760 2024-09-19T04:53:09.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.688+0000 7fdabea00640 1 -- 172.21.15.123:0/3114599318 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7fdac407ad60 con 0x55d63807d760 2024-09-19T04:53:09.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.688+0000 7fdae29ca740 1 -- 172.21.15.123:0/3114599318 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdacc003e50 con 0x55d63807d760 2024-09-19T04:53:09.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.688+0000 7fdabea00640 1 -- 172.21.15.123:0/3114599318 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdac409bac0 con 0x55d63807d760 2024-09-19T04:53:09.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.688+0000 7fdae29ca740 10 client.26491 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:09.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.688+0000 7fdae29ca740 1 -- 172.21.15.123:0/3114599318 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d63810bce0 con 0x55d63807d760 2024-09-19T04:53:09.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.689+0000 7fdabea00640 1 -- 172.21.15.123:0/3114599318 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdacc003e50 con 0x55d63807d760 2024-09-19T04:53:09.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.689+0000 7fdabea00640 1 -- 172.21.15.123:0/3114599318 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fdac40b9040 con 0x55d63807d760 2024-09-19T04:53:09.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.689+0000 7fdae29ca740 10 client.26491 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:09.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.689+0000 7fdae29ca740 10 client.26491 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:53:09.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.689+0000 7fdae29ca740 20 client.26491 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:09.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.689+0000 7fdae29ca740 1 --2- 172.21.15.123:0/3114599318 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55d63810c4b0 0x55d63810e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:09.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.689+0000 7fdae29ca740 4 client.26491 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:53:09.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.689+0000 7fdae29ca740 1 -- 172.21.15.123:0/3114599318 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d637feae60 con 0x55d63810c4b0 2024-09-19T04:53:09.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.689+0000 7fdadd400640 1 --2- 172.21.15.123:0/3114599318 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55d63810c4b0 0x55d63810e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:09.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.690+0000 7fdadd400640 1 --2- 172.21.15.123:0/3114599318 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55d63810c4b0 0x55d63810e8d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=121a433e2c007e24 server_cookie=c27ea93489361eab in_seq=0 out_seq=0 2024-09-19T04:53:09.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.690+0000 7fdabea00640 10 client.26491 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:53:09.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.690+0000 7fdabea00640 1 -- 172.21.15.123:0/3114599318 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d637feae60 con 0x55d63810c4b0 2024-09-19T04:53:09.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.690+0000 7fdabea00640 10 client.26491 handle_command_reply: tid=0 2024-09-19T04:53:09.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.690+0000 7fdae29ca740 1 client.26491 shutdown 2024-09-19T04:53:09.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.690+0000 7fdae29ca740 1 -- 172.21.15.123:0/3114599318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d63807d760 msgr2=0x55d638090530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:09.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.690+0000 7fdae29ca740 1 --2- 172.21.15.123:0/3114599318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d63807d760 0x55d638090530 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdac40bac40 tx=0x7fdac40bc130 comp rx=0 tx=0).stop 2024-09-19T04:53:09.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.690+0000 7fdae29ca740 1 -- 172.21.15.123:0/3114599318 shutdown_connections 2024-09-19T04:53:09.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.690+0000 7fdae29ca740 1 --2- 172.21.15.123:0/3114599318 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55d63810c4b0 0x55d63810e8d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.690+0000 7fdae29ca740 1 --2- 172.21.15.123:0/3114599318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d638088020 0x55d63808cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.690+0000 7fdae29ca740 1 --2- 172.21.15.123:0/3114599318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d63807d760 0x55d638090530 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.690+0000 7fdabea00640 0 client.26491 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:53:09.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.690+0000 7fdae29ca740 1 --2- 172.21.15.123:0/3114599318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63807ce40 0x55d63808fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.690+0000 7fdae29ca740 1 -- 172.21.15.123:0/3114599318 >> 172.21.15.123:0/3114599318 conn(0x55d6380793d0 msgr2=0x55d638084f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:09.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.690+0000 7fdae29ca740 1 -- 172.21.15.123:0/3114599318 shutdown_connections 2024-09-19T04:53:09.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.691+0000 7fdae29ca740 1 -- 172.21.15.123:0/3114599318 wait complete. 2024-09-19T04:53:09.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.691+0000 7fdae29ca740 20 client.26491 trim_cache size 0 max 16384 2024-09-19T04:53:09.696 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:53:09.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.691+0000 7fdade800640 1 -- 172.21.15.123:0/3903848760 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdac00a33b0 msgr2=0x7fdac00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:09.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.691+0000 7fdade800640 1 --2- 172.21.15.123:0/3903848760 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdac00a33b0 0x7fdac00a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55d63807a230 tx=0x7fdacc009800 comp rx=0 tx=0).stop 2024-09-19T04:53:09.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.691+0000 7fdade800640 1 -- 172.21.15.123:0/3903848760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdad8102360 msgr2=0x7fdad819a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:09.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.691+0000 7fdade800640 1 --2- 172.21.15.123:0/3903848760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdad8102360 0x7fdad819a9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdac400e9f0 tx=0x7fdac400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.691+0000 7fdade800640 1 -- 172.21.15.123:0/3903848760 shutdown_connections 2024-09-19T04:53:09.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.691+0000 7fdade800640 1 --2- 172.21.15.123:0/3903848760 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdac00a33b0 0x7fdac00a5870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.691+0000 7fdade800640 1 --2- 172.21.15.123:0/3903848760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdad81079c0 0x7fdad819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.691+0000 7fdade800640 1 --2- 172.21.15.123:0/3903848760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdad8102360 0x7fdad819a9c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.691+0000 7fdade800640 1 --2- 172.21.15.123:0/3903848760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdad81019c0 0x7fdad819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:09.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.691+0000 7fdade800640 1 -- 172.21.15.123:0/3903848760 >> 172.21.15.123:0/3903848760 conn(0x7fdad80fd740 msgr2=0x7fdad80fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:09.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.691+0000 7fdade800640 1 -- 172.21.15.123:0/3903848760 shutdown_connections 2024-09-19T04:53:09.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:09.691+0000 7fdade800640 1 -- 172.21.15.123:0/3903848760 wait complete. 2024-09-19T04:53:09.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:09 smithi193 ceph-mon[35592]: pgmap v196: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T04:53:10.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:10 smithi123 ceph-mon[30511]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T04:53:10.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:10 smithi183 ceph-mon[35333]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T04:53:10.740 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:53:10.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:10 smithi193 ceph-mon[35592]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T04:53:11.113 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:53:11.502 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:11 smithi123 ceph-mon[30511]: pgmap v198: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T04:53:11.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:11 smithi183 ceph-mon[35333]: pgmap v198: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T04:53:11.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.712+0000 7f9a5fa00640 1 -- 172.21.15.123:0/753411543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a5810e430 msgr2=0x7f9a58110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:11.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.712+0000 7f9a5fa00640 1 --2- 172.21.15.123:0/753411543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a5810e430 0x7f9a58110820 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009ca0 tx=0x7f9a5002a950 comp rx=0 tx=0).stop 2024-09-19T04:53:11.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.712+0000 7f9a5fa00640 1 -- 172.21.15.123:0/753411543 shutdown_connections 2024-09-19T04:53:11.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.712+0000 7f9a5fa00640 1 --2- 172.21.15.123:0/753411543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a5810e430 0x7f9a58110820 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.712+0000 7f9a5fa00640 1 --2- 172.21.15.123:0/753411543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a58074ee0 0x7f9a5810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.712+0000 7f9a5fa00640 1 --2- 172.21.15.123:0/753411543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a580745c0 0x7f9a580749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.712+0000 7f9a5fa00640 1 -- 172.21.15.123:0/753411543 >> 172.21.15.123:0/753411543 conn(0x7f9a580fd8b0 msgr2=0x7f9a580ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:11.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.712+0000 7f9a5fa00640 1 -- 172.21.15.123:0/753411543 shutdown_connections 2024-09-19T04:53:11.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.712+0000 7f9a5fa00640 1 -- 172.21.15.123:0/753411543 wait complete. 2024-09-19T04:53:11.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.713+0000 7f9a5fa00640 1 Processor -- start 2024-09-19T04:53:11.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.713+0000 7f9a5fa00640 1 -- start start 2024-09-19T04:53:11.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.713+0000 7f9a5fa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a580745c0 0x7f9a5819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.713+0000 7f9a5fa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a58074ee0 0x7f9a5819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.713+0000 7f9a5fa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a5810e430 0x7f9a5819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.713+0000 7f9a5fa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9a58114180 con 0x7f9a58074ee0 2024-09-19T04:53:11.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.713+0000 7f9a5fa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9a58114000 con 0x7f9a5810e430 2024-09-19T04:53:11.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.713+0000 7f9a5fa00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a58114300 con 0x7f9a580745c0 2024-09-19T04:53:11.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.714+0000 7f9a57e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a58074ee0 0x7f9a5819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.714+0000 7f9a5d000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a580745c0 0x7f9a5819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.714+0000 7f9a57e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a58074ee0 0x7f9a5819ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57690/0 (socket says 172.21.15.123:57690) 2024-09-19T04:53:11.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.714+0000 7f9a57e00640 1 -- 172.21.15.123:0/344327958 learned_addr learned my addr 172.21.15.123:0/344327958 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:11.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.714+0000 7f9a5da00640 1 --2- 172.21.15.123:0/344327958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a5810e430 0x7f9a5819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.714+0000 7f9a57e00640 1 -- 172.21.15.123:0/344327958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a580745c0 msgr2=0x7f9a5819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:11.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.714+0000 7f9a57e00640 1 --2- 172.21.15.123:0/344327958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a580745c0 0x7f9a5819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.714+0000 7f9a57e00640 1 -- 172.21.15.123:0/344327958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a5810e430 msgr2=0x7f9a5819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.714+0000 7f9a57e00640 1 --2- 172.21.15.123:0/344327958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a5810e430 0x7f9a5819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.714+0000 7f9a57e00640 1 -- 172.21.15.123:0/344327958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5819f690 con 0x7f9a58074ee0 2024-09-19T04:53:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.715+0000 7f9a57e00640 1 --2- 172.21.15.123:0/344327958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a58074ee0 0x7f9a5819ac20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400e9f0 tx=0x7f9a4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.715+0000 7f9a55600640 1 -- 172.21.15.123:0/344327958 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4400cd80 con 0x7f9a58074ee0 2024-09-19T04:53:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.715+0000 7f9a5fa00640 1 -- 172.21.15.123:0/344327958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a5819f980 con 0x7f9a58074ee0 2024-09-19T04:53:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.715+0000 7f9a55600640 1 -- 172.21.15.123:0/344327958 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a44005500 con 0x7f9a58074ee0 2024-09-19T04:53:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.715+0000 7f9a55600640 1 -- 172.21.15.123:0/344327958 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44010690 con 0x7f9a58074ee0 2024-09-19T04:53:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.715+0000 7f9a5fa00640 1 -- 172.21.15.123:0/344327958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a581a7260 con 0x7f9a58074ee0 2024-09-19T04:53:11.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.718+0000 7f9a55600640 1 -- 172.21.15.123:0/344327958 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9a440046c0 con 0x7f9a58074ee0 2024-09-19T04:53:11.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.718+0000 7f9a624ba740 1 Processor -- start 2024-09-19T04:53:11.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.718+0000 7f9a624ba740 1 -- start start 2024-09-19T04:53:11.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.718+0000 7f9a624ba740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912cc70 0x55966912d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.718+0000 7f9a624ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55966912d590 0x559669137910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.718+0000 7f9a624ba740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559669137e50 0x55966913a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.719+0000 7f9a624ba740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5596690933e0 con 0x559669137e50 2024-09-19T04:53:11.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.719+0000 7f9a624ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55966903ad80 con 0x55966912d590 2024-09-19T04:53:11.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.719+0000 7f9a624ba740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55966908f3a0 con 0x55966912cc70 2024-09-19T04:53:11.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.719+0000 7f9a5da00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912cc70 0x55966912d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.719+0000 7f9a5da00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912cc70 0x55966912d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42328/0 (socket says 172.21.15.123:42328) 2024-09-19T04:53:11.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.719+0000 7f9a5da00640 1 -- 172.21.15.123:0/2115707707 learned_addr learned my addr 172.21.15.123:0/2115707707 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:11.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.719+0000 7f9a57e00640 1 --2- 172.21.15.123:0/2115707707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559669137e50 0x55966913a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.719+0000 7f9a5d000640 1 --2- 172.21.15.123:0/2115707707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55966912d590 0x559669137910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.719+0000 7f9a5da00640 1 -- 172.21.15.123:0/2115707707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55966912d590 msgr2=0x559669137910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:11.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.719+0000 7f9a5da00640 1 --2- 172.21.15.123:0/2115707707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55966912d590 0x559669137910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.719+0000 7f9a5da00640 1 -- 172.21.15.123:0/2115707707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559669137e50 msgr2=0x55966913a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:11.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.719+0000 7f9a5da00640 1 --2- 172.21.15.123:0/2115707707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559669137e50 0x55966913a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.719+0000 7f9a5da00640 1 -- 172.21.15.123:0/2115707707 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559669059ae0 con 0x55966912cc70 2024-09-19T04:53:11.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.720+0000 7f9a55600640 1 --2- 172.21.15.123:0/344327958 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9a400a33b0 0x7f9a400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.720+0000 7f9a5da00640 1 --2- 172.21.15.123:0/2115707707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912cc70 0x55966912d050 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009ca0 tx=0x7f9a500093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85c1fad36d6f1062 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:11.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.720+0000 7f9a55600640 1 -- 172.21.15.123:0/344327958 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7f9a440d0140 con 0x7f9a58074ee0 2024-09-19T04:53:11.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.720+0000 7f9a5d000640 1 --2- 172.21.15.123:0/344327958 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9a400a33b0 0x7f9a400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.720+0000 7f9a4a000640 1 -- 172.21.15.123:0/2115707707 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50048070 con 0x55966912cc70 2024-09-19T04:53:11.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.720+0000 7f9a4a000640 1 -- 172.21.15.123:0/2115707707 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a5002f940 con 0x55966912cc70 2024-09-19T04:53:11.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.720+0000 7f9a4a000640 1 -- 172.21.15.123:0/2115707707 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50034040 con 0x55966912cc70 2024-09-19T04:53:11.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.721+0000 7f9a624ba740 1 -- 172.21.15.123:0/2115707707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912cc70 msgr2=0x55966912d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:11.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.721+0000 7f9a624ba740 1 --2- 172.21.15.123:0/2115707707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912cc70 0x55966912d050 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009ca0 tx=0x7f9a500093e0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.721+0000 7f9a5d000640 1 --2- 172.21.15.123:0/344327958 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9a400a33b0 0x7f9a400a5870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55966912a060 tx=0x7f9a4c019040 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:11.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.721+0000 7f9a624ba740 1 -- 172.21.15.123:0/2115707707 shutdown_connections 2024-09-19T04:53:11.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.721+0000 7f9a624ba740 1 --2- 172.21.15.123:0/2115707707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559669137e50 0x55966913a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.721+0000 7f9a624ba740 1 --2- 172.21.15.123:0/2115707707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55966912d590 0x559669137910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.721+0000 7f9a624ba740 1 --2- 172.21.15.123:0/2115707707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912cc70 0x55966912d050 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.721+0000 7f9a624ba740 1 -- 172.21.15.123:0/2115707707 >> 172.21.15.123:0/2115707707 conn(0x559669129200 msgr2=0x5596691295e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:11.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.721+0000 7f9a624ba740 1 -- 172.21.15.123:0/2115707707 shutdown_connections 2024-09-19T04:53:11.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.721+0000 7f9a624ba740 1 -- 172.21.15.123:0/2115707707 wait complete. 2024-09-19T04:53:11.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.722+0000 7f9a624ba740 1 -- start start 2024-09-19T04:53:11.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.722+0000 7f9a624ba740 1 Processor -- start 2024-09-19T04:53:11.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.722+0000 7f9a624ba740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55966912cc70 0x559669134d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.722+0000 7f9a624ba740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912d590 0x559669135250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.722+0000 7f9a624ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559669137e50 0x559669135790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.722+0000 7f9a624ba740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55966913aa20 con 0x55966912cc70 2024-09-19T04:53:11.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.722+0000 7f9a624ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55966904e0f0 con 0x559669137e50 2024-09-19T04:53:11.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.722+0000 7f9a624ba740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55966913aba0 con 0x55966912d590 2024-09-19T04:53:11.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.722+0000 7f9a624ba740 10 client.0 _ll_register_callbacks cb 0x559669035bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:11.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.723+0000 7f9a5d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55966912cc70 0x559669134d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.723+0000 7f9a57e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912d590 0x559669135250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.723+0000 7f9a5da00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559669137e50 0x559669135790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.723+0000 7f9a57e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912d590 0x559669135250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42340/0 (socket says 172.21.15.123:42340) 2024-09-19T04:53:11.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.723+0000 7f9a57e00640 1 -- 172.21.15.123:0/754698748 learned_addr learned my addr 172.21.15.123:0/754698748 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:11.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.723+0000 7f9a5da00640 1 -- 172.21.15.123:0/754698748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912d590 msgr2=0x559669135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:11.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.723+0000 7f9a5da00640 1 --2- 172.21.15.123:0/754698748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912d590 0x559669135250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.723+0000 7f9a5da00640 1 -- 172.21.15.123:0/754698748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55966912cc70 msgr2=0x559669134d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:11.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.723+0000 7f9a5da00640 1 --2- 172.21.15.123:0/754698748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55966912cc70 0x559669134d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.723+0000 7f9a5da00640 1 -- 172.21.15.123:0/754698748 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559668e043c0 con 0x559669137e50 2024-09-19T04:53:11.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.724+0000 7f9a5da00640 1 --2- 172.21.15.123:0/754698748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559669137e50 0x559669135790 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9a50046040 tx=0x7f9a50003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff691c4f1de2b59b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:11.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.724+0000 7f9a3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:53:11.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.724+0000 7f9a3ea00640 1 -- 172.21.15.123:0/754698748 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50048030 con 0x559669137e50 2024-09-19T04:53:11.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.724+0000 7f9a3ea00640 1 -- 172.21.15.123:0/754698748 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7f9a50040040 con 0x559669137e50 2024-09-19T04:53:11.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.724+0000 7f9a624ba740 1 -- 172.21.15.123:0/754698748 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a5002f940 con 0x559669137e50 2024-09-19T04:53:11.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.724+0000 7f9a3ea00640 1 -- 172.21.15.123:0/754698748 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9a50034050 con 0x559669137e50 2024-09-19T04:53:11.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.724+0000 7f9a624ba740 10 client.26483 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:11.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.724+0000 7f9a624ba740 1 -- 172.21.15.123:0/754698748 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559669146a30 con 0x559669137e50 2024-09-19T04:53:11.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.725+0000 7f9a3ea00640 1 -- 172.21.15.123:0/754698748 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a5002f940 con 0x559669137e50 2024-09-19T04:53:11.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.725+0000 7f9a3ea00640 1 -- 172.21.15.123:0/754698748 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f9a5003a430 con 0x559669137e50 2024-09-19T04:53:11.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.725+0000 7f9a624ba740 10 client.26483 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:11.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.725+0000 7f9a624ba740 10 client.26483 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:53:11.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.725+0000 7f9a624ba740 20 client.26483 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:11.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.725+0000 7f9a624ba740 1 --2- 172.21.15.123:0/754698748 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x559669147d80 0x55966914a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.725+0000 7f9a624ba740 4 client.26483 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:53:11.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.725+0000 7f9a624ba740 1 -- 172.21.15.123:0/754698748 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596690a0950 con 0x559669147d80 2024-09-19T04:53:11.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.725+0000 7f9a5d000640 1 --2- 172.21.15.123:0/754698748 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x559669147d80 0x55966914a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.726+0000 7f9a5d000640 1 --2- 172.21.15.123:0/754698748 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x559669147d80 0x55966914a170 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=938a7ad902e6f09c server_cookie=2f4d6cd1cb119e86 in_seq=0 out_seq=0 2024-09-19T04:53:11.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.726+0000 7f9a3ea00640 10 client.26483 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:53:11.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.727+0000 7f9a3ea00640 1 -- 172.21.15.123:0/754698748 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596690a0950 con 0x559669147d80 2024-09-19T04:53:11.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.727+0000 7f9a3ea00640 10 client.26483 handle_command_reply: tid=0 2024-09-19T04:53:11.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.727+0000 7f9a624ba740 1 client.26483 shutdown 2024-09-19T04:53:11.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.727+0000 7f9a624ba740 1 -- 172.21.15.123:0/754698748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559669137e50 msgr2=0x559669135790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:11.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.727+0000 7f9a624ba740 1 --2- 172.21.15.123:0/754698748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559669137e50 0x559669135790 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9a50046040 tx=0x7f9a50003970 comp rx=0 tx=0).stop 2024-09-19T04:53:11.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.727+0000 7f9a624ba740 1 -- 172.21.15.123:0/754698748 shutdown_connections 2024-09-19T04:53:11.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.727+0000 7f9a624ba740 1 --2- 172.21.15.123:0/754698748 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x559669147d80 0x55966914a170 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.727+0000 7f9a624ba740 1 --2- 172.21.15.123:0/754698748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559669137e50 0x559669135790 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.727+0000 7f9a624ba740 1 --2- 172.21.15.123:0/754698748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912d590 0x559669135250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.727+0000 7f9a624ba740 1 --2- 172.21.15.123:0/754698748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55966912cc70 0x559669134d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.727+0000 7f9a3ea00640 0 client.26483 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:53:11.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.727+0000 7f9a624ba740 1 -- 172.21.15.123:0/754698748 >> 172.21.15.123:0/754698748 conn(0x559669129200 msgr2=0x559669128380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:11.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.727+0000 7f9a624ba740 1 -- 172.21.15.123:0/754698748 shutdown_connections 2024-09-19T04:53:11.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.727+0000 7f9a624ba740 1 -- 172.21.15.123:0/754698748 wait complete. 2024-09-19T04:53:11.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.728+0000 7f9a624ba740 20 client.26483 trim_cache size 0 max 16384 2024-09-19T04:53:11.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:11 smithi193 ceph-mon[35592]: pgmap v198: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T04:53:11.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.741+0000 7f9a624ba740 1 Processor -- start 2024-09-19T04:53:11.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.741+0000 7f9a624ba740 1 -- start start 2024-09-19T04:53:11.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.741+0000 7f9a624ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55966912cc70 0x55966912f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.741+0000 7f9a624ba740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912d590 0x559669136570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.741+0000 7f9a624ba740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559669137e50 0x55966914f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.741+0000 7f9a624ba740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581ab5b0 con 0x559669137e50 2024-09-19T04:53:11.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.741+0000 7f9a624ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581ad180 con 0x55966912cc70 2024-09-19T04:53:11.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.741+0000 7f9a624ba740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581abcb0 con 0x55966912d590 2024-09-19T04:53:11.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.742+0000 7f9a5da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559669137e50 0x55966914f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.742+0000 7f9a5da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559669137e50 0x55966914f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57712/0 (socket says 172.21.15.123:57712) 2024-09-19T04:53:11.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.742+0000 7f9a5da00640 1 -- 172.21.15.123:0/3900000666 learned_addr learned my addr 172.21.15.123:0/3900000666 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:11.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.742+0000 7f9a5d000640 1 --2- 172.21.15.123:0/3900000666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55966912cc70 0x55966912f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.742+0000 7f9a57e00640 1 --2- 172.21.15.123:0/3900000666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912d590 0x559669136570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.742+0000 7f9a5da00640 1 -- 172.21.15.123:0/3900000666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912d590 msgr2=0x559669136570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:11.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.742+0000 7f9a5da00640 1 --2- 172.21.15.123:0/3900000666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912d590 0x559669136570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.742+0000 7f9a5da00640 1 -- 172.21.15.123:0/3900000666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55966912cc70 msgr2=0x55966912f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:11.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.742+0000 7f9a5da00640 1 --2- 172.21.15.123:0/3900000666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55966912cc70 0x55966912f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.742+0000 7f9a5da00640 1 -- 172.21.15.123:0/3900000666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55966912a2b0 con 0x559669137e50 2024-09-19T04:53:11.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.742+0000 7f9a5da00640 1 --2- 172.21.15.123:0/3900000666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559669137e50 0x55966914f710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9a5003a820 tx=0x7f9a50003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba293010e383640f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:11.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.742+0000 7f9a49600640 1 -- 172.21.15.123:0/3900000666 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50048070 con 0x559669137e50 2024-09-19T04:53:11.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.742+0000 7f9a49600640 1 -- 172.21.15.123:0/3900000666 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a5004a070 con 0x559669137e50 2024-09-19T04:53:11.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.742+0000 7f9a49600640 1 -- 172.21.15.123:0/3900000666 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50030b30 con 0x559669137e50 2024-09-19T04:53:11.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.743+0000 7f9a624ba740 1 -- 172.21.15.123:0/3900000666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559669137e50 msgr2=0x55966914f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:11.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.743+0000 7f9a624ba740 1 --2- 172.21.15.123:0/3900000666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559669137e50 0x55966914f710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9a5003a820 tx=0x7f9a50003b60 comp rx=0 tx=0).stop 2024-09-19T04:53:11.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.743+0000 7f9a624ba740 1 -- 172.21.15.123:0/3900000666 shutdown_connections 2024-09-19T04:53:11.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.743+0000 7f9a624ba740 1 --2- 172.21.15.123:0/3900000666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559669137e50 0x55966914f710 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.743+0000 7f9a624ba740 1 --2- 172.21.15.123:0/3900000666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912d590 0x559669136570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.743+0000 7f9a624ba740 1 --2- 172.21.15.123:0/3900000666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55966912cc70 0x55966912f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.743+0000 7f9a624ba740 1 -- 172.21.15.123:0/3900000666 >> 172.21.15.123:0/3900000666 conn(0x559669129200 msgr2=0x559669133cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:11.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.743+0000 7f9a624ba740 1 -- 172.21.15.123:0/3900000666 shutdown_connections 2024-09-19T04:53:11.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.743+0000 7f9a624ba740 1 -- 172.21.15.123:0/3900000666 wait complete. 2024-09-19T04:53:11.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.744+0000 7f9a624ba740 1 -- start start 2024-09-19T04:53:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.744+0000 7f9a624ba740 1 Processor -- start 2024-09-19T04:53:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.744+0000 7f9a624ba740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55966912cc70 0x5596691404d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.744+0000 7f9a624ba740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912d590 0x559669131a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.744+0000 7f9a624ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559669137e50 0x559669131fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.744+0000 7f9a624ba740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581acab0 con 0x55966912cc70 2024-09-19T04:53:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.744+0000 7f9a624ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581ac3b0 con 0x559669137e50 2024-09-19T04:53:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.744+0000 7f9a624ba740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581a95b0 con 0x55966912d590 2024-09-19T04:53:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.744+0000 7f9a624ba740 10 client.0 _ll_register_callbacks cb 0x7f9a50046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.745+0000 7f9a5d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55966912cc70 0x5596691404d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.745+0000 7f9a5d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55966912cc70 0x5596691404d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57720/0 (socket says 172.21.15.123:57720) 2024-09-19T04:53:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.745+0000 7f9a5d000640 1 -- 172.21.15.123:0/1131634129 learned_addr learned my addr 172.21.15.123:0/1131634129 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.745+0000 7f9a57e00640 1 --2- 172.21.15.123:0/1131634129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912d590 0x559669131a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.745+0000 7f9a5da00640 1 --2- 172.21.15.123:0/1131634129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559669137e50 0x559669131fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.745+0000 7f9a5d000640 1 -- 172.21.15.123:0/1131634129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912d590 msgr2=0x559669131a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:11.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.745+0000 7f9a5d000640 1 --2- 172.21.15.123:0/1131634129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912d590 0x559669131a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.745+0000 7f9a5d000640 1 -- 172.21.15.123:0/1131634129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559669137e50 msgr2=0x559669131fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:11.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.745+0000 7f9a5d000640 1 --2- 172.21.15.123:0/1131634129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559669137e50 0x559669131fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.745+0000 7f9a5d000640 1 -- 172.21.15.123:0/1131634129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559669124770 con 0x55966912cc70 2024-09-19T04:53:11.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.745+0000 7f9a5da00640 1 --2- 172.21.15.123:0/1131634129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559669137e50 0x559669131fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:11.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.745+0000 7f9a5d000640 1 --2- 172.21.15.123:0/1131634129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55966912cc70 0x5596691404d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55966914a6b0 tx=0x7f9a4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0537d1c912a7162 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:11.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.745+0000 7f9a3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:53:11.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.746+0000 7f9a3ea00640 1 -- 172.21.15.123:0/1131634129 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c03d030 con 0x55966912cc70 2024-09-19T04:53:11.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.746+0000 7f9a3ea00640 1 -- 172.21.15.123:0/1131634129 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7f9a4c0318c0 con 0x55966912cc70 2024-09-19T04:53:11.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.746+0000 7f9a624ba740 1 -- 172.21.15.123:0/1131634129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a5004a070 con 0x55966912cc70 2024-09-19T04:53:11.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.746+0000 7f9a3ea00640 1 -- 172.21.15.123:0/1131634129 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9a4c047050 con 0x55966912cc70 2024-09-19T04:53:11.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.746+0000 7f9a624ba740 10 client.16389 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:11.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.746+0000 7f9a624ba740 1 -- 172.21.15.123:0/1131634129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5596691b24e0 con 0x55966912cc70 2024-09-19T04:53:11.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.746+0000 7f9a3ea00640 1 -- 172.21.15.123:0/1131634129 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a5004a070 con 0x55966912cc70 2024-09-19T04:53:11.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.746+0000 7f9a3ea00640 1 -- 172.21.15.123:0/1131634129 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f9a4c0048a0 con 0x55966912cc70 2024-09-19T04:53:11.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.746+0000 7f9a624ba740 10 client.16389 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:11.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.746+0000 7f9a624ba740 10 client.16389 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:53:11.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.747+0000 7f9a624ba740 20 client.16389 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:11.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.747+0000 7f9a624ba740 1 --2- 172.21.15.123:0/1131634129 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5596691b2cc0 0x5596691b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:11.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.747+0000 7f9a624ba740 4 client.16389 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:53:11.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.747+0000 7f9a624ba740 1 -- 172.21.15.123:0/1131634129 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55966909ae60 con 0x5596691b2cc0 2024-09-19T04:53:11.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.747+0000 7f9a57e00640 1 --2- 172.21.15.123:0/1131634129 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5596691b2cc0 0x5596691b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:11.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.748+0000 7f9a57e00640 1 --2- 172.21.15.123:0/1131634129 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5596691b2cc0 0x5596691b50e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef5a51f5e8105ac3 server_cookie=2c51f50a4015a3a6 in_seq=0 out_seq=0 2024-09-19T04:53:11.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.748+0000 7f9a3ea00640 10 client.16389 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:53:11.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.748+0000 7f9a3ea00640 1 -- 172.21.15.123:0/1131634129 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55966909ae60 con 0x5596691b2cc0 2024-09-19T04:53:11.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.748+0000 7f9a3ea00640 10 client.16389 handle_command_reply: tid=0 2024-09-19T04:53:11.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.748+0000 7f9a624ba740 1 client.16389 shutdown 2024-09-19T04:53:11.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.748+0000 7f9a624ba740 1 -- 172.21.15.123:0/1131634129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55966912cc70 msgr2=0x5596691404d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:11.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.748+0000 7f9a624ba740 1 --2- 172.21.15.123:0/1131634129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55966912cc70 0x5596691404d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55966914a6b0 tx=0x7f9a4c004730 comp rx=0 tx=0).stop 2024-09-19T04:53:11.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.748+0000 7f9a624ba740 1 -- 172.21.15.123:0/1131634129 shutdown_connections 2024-09-19T04:53:11.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.748+0000 7f9a624ba740 1 --2- 172.21.15.123:0/1131634129 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5596691b2cc0 0x5596691b50e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.748+0000 7f9a624ba740 1 --2- 172.21.15.123:0/1131634129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559669137e50 0x559669131fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.748+0000 7f9a3ea00640 0 client.16389 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:53:11.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.748+0000 7f9a624ba740 1 --2- 172.21.15.123:0/1131634129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55966912d590 0x559669131a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.748+0000 7f9a624ba740 1 --2- 172.21.15.123:0/1131634129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55966912cc70 0x5596691404d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.748+0000 7f9a624ba740 1 -- 172.21.15.123:0/1131634129 >> 172.21.15.123:0/1131634129 conn(0x559669129200 msgr2=0x5596691334b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:11.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.748+0000 7f9a624ba740 1 -- 172.21.15.123:0/1131634129 shutdown_connections 2024-09-19T04:53:11.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.748+0000 7f9a624ba740 1 -- 172.21.15.123:0/1131634129 wait complete. 2024-09-19T04:53:11.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.748+0000 7f9a624ba740 20 client.16389 trim_cache size 0 max 16384 2024-09-19T04:53:11.757 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:53:11.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.749+0000 7f9a5fa00640 1 -- 172.21.15.123:0/344327958 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9a400a33b0 msgr2=0x7f9a400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:11.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.749+0000 7f9a5fa00640 1 --2- 172.21.15.123:0/344327958 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9a400a33b0 0x7f9a400a5870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55966912a060 tx=0x7f9a4c019040 comp rx=0 tx=0).stop 2024-09-19T04:53:11.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.749+0000 7f9a5fa00640 1 -- 172.21.15.123:0/344327958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a58074ee0 msgr2=0x7f9a5819ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:11.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.749+0000 7f9a5fa00640 1 --2- 172.21.15.123:0/344327958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a58074ee0 0x7f9a5819ac20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400e9f0 tx=0x7f9a4400edf0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.749+0000 7f9a5fa00640 1 -- 172.21.15.123:0/344327958 shutdown_connections 2024-09-19T04:53:11.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.749+0000 7f9a5fa00640 1 --2- 172.21.15.123:0/344327958 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9a400a33b0 0x7f9a400a5870 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.749+0000 7f9a5fa00640 1 --2- 172.21.15.123:0/344327958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a5810e430 0x7f9a5819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.749+0000 7f9a5fa00640 1 --2- 172.21.15.123:0/344327958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a58074ee0 0x7f9a5819ac20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.749+0000 7f9a5fa00640 1 --2- 172.21.15.123:0/344327958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a580745c0 0x7f9a5819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:11.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.749+0000 7f9a5fa00640 1 -- 172.21.15.123:0/344327958 >> 172.21.15.123:0/344327958 conn(0x7f9a580fd8b0 msgr2=0x7f9a580fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:11.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.749+0000 7f9a5fa00640 1 -- 172.21.15.123:0/344327958 shutdown_connections 2024-09-19T04:53:11.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:11.749+0000 7f9a5fa00640 1 -- 172.21.15.123:0/344327958 wait complete. 2024-09-19T04:53:12.666 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:53:13.038 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:53:13.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:13 smithi123 ceph-mon[30511]: pgmap v199: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T04:53:13.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:13 smithi183 ceph-mon[35333]: pgmap v199: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T04:53:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.606+0000 7fc4d2800640 1 -- 172.21.15.123:0/78394115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4cc102b90 msgr2=0x7fc4cc102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.606+0000 7fc4d2800640 1 --2- 172.21.15.123:0/78394115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4cc102b90 0x7fc4cc102f70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009ca0 tx=0x7fc4c002a950 comp rx=0 tx=0).stop 2024-09-19T04:53:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.606+0000 7fc4d2800640 1 -- 172.21.15.123:0/78394115 shutdown_connections 2024-09-19T04:53:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.606+0000 7fc4d2800640 1 --2- 172.21.15.123:0/78394115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4cc10bfc0 0x7fc4cc10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.606+0000 7fc4d2800640 1 --2- 172.21.15.123:0/78394115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4cc1034b0 0x7fc4cc10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.606+0000 7fc4d2800640 1 --2- 172.21.15.123:0/78394115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4cc102b90 0x7fc4cc102f70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.606+0000 7fc4d2800640 1 -- 172.21.15.123:0/78394115 >> 172.21.15.123:0/78394115 conn(0x7fc4cc0fc610 msgr2=0x7fc4cc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.606+0000 7fc4d2800640 1 -- 172.21.15.123:0/78394115 shutdown_connections 2024-09-19T04:53:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.606+0000 7fc4d2800640 1 -- 172.21.15.123:0/78394115 wait complete. 2024-09-19T04:53:13.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.607+0000 7fc4d2800640 1 Processor -- start 2024-09-19T04:53:13.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.607+0000 7fc4d2800640 1 -- start start 2024-09-19T04:53:13.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.607+0000 7fc4d2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4cc102b90 0x7fc4cc19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.608+0000 7fc4d2800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4cc1034b0 0x7fc4cc19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.608+0000 7fc4d2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4cc10bfc0 0x7fc4cc19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.608+0000 7fc4d2800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc111dc0 con 0x7fc4cc10bfc0 2024-09-19T04:53:13.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.608+0000 7fc4d2800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc111c40 con 0x7fc4cc102b90 2024-09-19T04:53:13.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.608+0000 7fc4d2800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc111f40 con 0x7fc4cc1034b0 2024-09-19T04:53:13.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.608+0000 7fc4d1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4cc102b90 0x7fc4cc19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.608+0000 7fc4d0a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4cc1034b0 0x7fc4cc19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.608+0000 7fc4d1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4cc10bfc0 0x7fc4cc19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.608+0000 7fc4d1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4cc102b90 0x7fc4cc19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:33768/0 (socket says 172.21.15.123:33768) 2024-09-19T04:53:13.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.608+0000 7fc4d1400640 1 -- 172.21.15.123:0/2076007373 learned_addr learned my addr 172.21.15.123:0/2076007373 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:13.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.608+0000 7fc4d1400640 1 -- 172.21.15.123:0/2076007373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4cc1034b0 msgr2=0x7fc4cc19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:13.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.608+0000 7fc4d1400640 1 --2- 172.21.15.123:0/2076007373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4cc1034b0 0x7fc4cc19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.608+0000 7fc4d1400640 1 -- 172.21.15.123:0/2076007373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4cc10bfc0 msgr2=0x7fc4cc19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:13.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.608+0000 7fc4d1400640 1 --2- 172.21.15.123:0/2076007373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4cc10bfc0 0x7fc4cc19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.608+0000 7fc4d1400640 1 -- 172.21.15.123:0/2076007373 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4cc19f490 con 0x7fc4cc102b90 2024-09-19T04:53:13.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.608+0000 7fc4d0a00640 1 --2- 172.21.15.123:0/2076007373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4cc1034b0 0x7fc4cc19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:13.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.609+0000 7fc4d1400640 1 --2- 172.21.15.123:0/2076007373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4cc102b90 0x7fc4cc19a4e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0005cc0 tx=0x7fc4c002f420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:13.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.609+0000 7fc4ca000640 1 -- 172.21.15.123:0/2076007373 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0034040 con 0x7fc4cc102b90 2024-09-19T04:53:13.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.609+0000 7fc4d2800640 1 -- 172.21.15.123:0/2076007373 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4cc19f720 con 0x7fc4cc102b90 2024-09-19T04:53:13.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.609+0000 7fc4ca000640 1 -- 172.21.15.123:0/2076007373 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c0030920 con 0x7fc4cc102b90 2024-09-19T04:53:13.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.609+0000 7fc4ca000640 1 -- 172.21.15.123:0/2076007373 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00046e0 con 0x7fc4cc102b90 2024-09-19T04:53:13.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.609+0000 7fc4d2800640 1 -- 172.21.15.123:0/2076007373 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4cc1a7060 con 0x7fc4cc102b90 2024-09-19T04:53:13.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.612+0000 7fc4d6845740 1 Processor -- start 2024-09-19T04:53:13.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.612+0000 7fc4d6845740 1 -- start start 2024-09-19T04:53:13.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.612+0000 7fc4ca000640 1 -- 172.21.15.123:0/2076007373 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc4c0037020 con 0x7fc4cc102b90 2024-09-19T04:53:13.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.612+0000 7fc4d6845740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc693e40 0x55d8dc694220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.612+0000 7fc4d6845740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc694760 0x55d8dc69eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.612+0000 7fc4d6845740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc69f020 0x55d8dc6a1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.612+0000 7fc4d6845740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d8dc5fa3e0 con 0x55d8dc69f020 2024-09-19T04:53:13.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.612+0000 7fc4d6845740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d8dc5a1d80 con 0x55d8dc693e40 2024-09-19T04:53:13.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.612+0000 7fc4d6845740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d8dc5f63a0 con 0x55d8dc694760 2024-09-19T04:53:13.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.613+0000 7fc4d0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc69f020 0x55d8dc6a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.613+0000 7fc4d0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc69f020 0x55d8dc6a1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36634/0 (socket says 172.21.15.123:36634) 2024-09-19T04:53:13.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.613+0000 7fc4d0a00640 1 -- 172.21.15.123:0/1408550102 learned_addr learned my addr 172.21.15.123:0/1408550102 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:13.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.613+0000 7fc4d1e00640 1 --2- 172.21.15.123:0/1408550102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc693e40 0x55d8dc694220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.613+0000 7fc4d1400640 1 --2- 172.21.15.123:0/1408550102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc694760 0x55d8dc69eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.613+0000 7fc4d0a00640 1 -- 172.21.15.123:0/1408550102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc694760 msgr2=0x55d8dc69eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:13.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.613+0000 7fc4d0a00640 1 --2- 172.21.15.123:0/1408550102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc694760 0x55d8dc69eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.613+0000 7fc4d0a00640 1 -- 172.21.15.123:0/1408550102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc693e40 msgr2=0x55d8dc694220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:13.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.613+0000 7fc4d0a00640 1 --2- 172.21.15.123:0/1408550102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc693e40 0x55d8dc694220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.613+0000 7fc4d0a00640 1 -- 172.21.15.123:0/1408550102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8dc5c0ae0 con 0x55d8dc69f020 2024-09-19T04:53:13.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.614+0000 7fc4d0a00640 1 --2- 172.21.15.123:0/1408550102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc69f020 0x55d8dc6a1410 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80114c0 tx=0x7fc4b8011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ad1235685c11af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:13.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.614+0000 7fc4ca000640 1 --2- 172.21.15.123:0/2076007373 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc4b40a33b0 0x7fc4b40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.614+0000 7fc4ca000640 1 -- 172.21.15.123:0/2076007373 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7fc4c00f0ac0 con 0x7fc4cc102b90 2024-09-19T04:53:13.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.614+0000 7fc4be600640 1 -- 172.21.15.123:0/1408550102 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8004030 con 0x55d8dc69f020 2024-09-19T04:53:13.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.614+0000 7fc4be600640 1 -- 172.21.15.123:0/1408550102 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b80041d0 con 0x55d8dc69f020 2024-09-19T04:53:13.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.614+0000 7fc4be600640 1 -- 172.21.15.123:0/1408550102 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b800f9d0 con 0x55d8dc69f020 2024-09-19T04:53:13.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.614+0000 7fc4d1e00640 1 --2- 172.21.15.123:0/2076007373 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc4b40a33b0 0x7fc4b40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.615+0000 7fc4d6845740 1 -- 172.21.15.123:0/1408550102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc69f020 msgr2=0x55d8dc6a1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:13.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.615+0000 7fc4d1e00640 1 --2- 172.21.15.123:0/2076007373 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc4b40a33b0 0x7fc4b40a5870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009d20 tx=0x7fc4c4005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:13.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.615+0000 7fc4d6845740 1 --2- 172.21.15.123:0/1408550102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc69f020 0x55d8dc6a1410 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80114c0 tx=0x7fc4b8011980 comp rx=0 tx=0).stop 2024-09-19T04:53:13.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.615+0000 7fc4d6845740 1 -- 172.21.15.123:0/1408550102 shutdown_connections 2024-09-19T04:53:13.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.615+0000 7fc4d6845740 1 --2- 172.21.15.123:0/1408550102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc69f020 0x55d8dc6a1410 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.615+0000 7fc4d6845740 1 --2- 172.21.15.123:0/1408550102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc694760 0x55d8dc69eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.615+0000 7fc4d6845740 1 --2- 172.21.15.123:0/1408550102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc693e40 0x55d8dc694220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.615+0000 7fc4d6845740 1 -- 172.21.15.123:0/1408550102 >> 172.21.15.123:0/1408550102 conn(0x55d8dc6903d0 msgr2=0x55d8dc6907b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:13.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.615+0000 7fc4d6845740 1 -- 172.21.15.123:0/1408550102 shutdown_connections 2024-09-19T04:53:13.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.615+0000 7fc4d6845740 1 -- 172.21.15.123:0/1408550102 wait complete. 2024-09-19T04:53:13.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.616+0000 7fc4d6845740 1 -- start start 2024-09-19T04:53:13.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.616+0000 7fc4d6845740 1 Processor -- start 2024-09-19T04:53:13.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.616+0000 7fc4d6845740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc693e40 0x55d8dc69bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.616+0000 7fc4d6845740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc694760 0x55d8dc69c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.616+0000 7fc4d6845740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc69f020 0x55d8dc69c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.616+0000 7fc4d6845740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d8dc6a1bf0 con 0x55d8dc693e40 2024-09-19T04:53:13.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.616+0000 7fc4d6845740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d8dc5b50f0 con 0x55d8dc694760 2024-09-19T04:53:13.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.616+0000 7fc4d6845740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d8dc6a1d70 con 0x55d8dc69f020 2024-09-19T04:53:13.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.616+0000 7fc4d6845740 10 client.0 _ll_register_callbacks cb 0x55d8dc59cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:13.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.617+0000 7fc4d1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc693e40 0x55d8dc69bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.617+0000 7fc4d1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc693e40 0x55d8dc69bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36642/0 (socket says 172.21.15.123:36642) 2024-09-19T04:53:13.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.617+0000 7fc4d1400640 1 -- 172.21.15.123:0/2444580018 learned_addr learned my addr 172.21.15.123:0/2444580018 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:13.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.617+0000 7fc4d0a00640 1 --2- 172.21.15.123:0/2444580018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc694760 0x55d8dc69c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.617+0000 7fc4d1e00640 1 --2- 172.21.15.123:0/2444580018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc69f020 0x55d8dc69c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.617+0000 7fc4d1400640 1 -- 172.21.15.123:0/2444580018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc69f020 msgr2=0x55d8dc69c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:13.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.617+0000 7fc4d1400640 1 --2- 172.21.15.123:0/2444580018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc69f020 0x55d8dc69c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.617+0000 7fc4d1400640 1 -- 172.21.15.123:0/2444580018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc694760 msgr2=0x55d8dc69c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:13.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.617+0000 7fc4d1400640 1 --2- 172.21.15.123:0/2444580018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc694760 0x55d8dc69c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.617+0000 7fc4d1400640 1 -- 172.21.15.123:0/2444580018 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8dc36b3c0 con 0x55d8dc693e40 2024-09-19T04:53:13.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.617+0000 7fc4d1400640 1 --2- 172.21.15.123:0/2444580018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc693e40 0x55d8dc69bf60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0003fd0 tx=0x7fc4c002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38920ce816d33063 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:13.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.617+0000 7fc4b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:53:13.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.617+0000 7fc4b2a00640 1 -- 172.21.15.123:0/2444580018 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00bc970 con 0x55d8dc693e40 2024-09-19T04:53:13.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.617+0000 7fc4b2a00640 1 -- 172.21.15.123:0/2444580018 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7fc4c00c5610 con 0x55d8dc693e40 2024-09-19T04:53:13.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.618+0000 7fc4d6845740 1 -- 172.21.15.123:0/2444580018 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4b80041d0 con 0x55d8dc693e40 2024-09-19T04:53:13.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.618+0000 7fc4b2a00640 1 -- 172.21.15.123:0/2444580018 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc4c00bc790 con 0x55d8dc693e40 2024-09-19T04:53:13.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.618+0000 7fc4d6845740 10 client.16401 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:13.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.618+0000 7fc4d6845740 1 -- 172.21.15.123:0/2444580018 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d8dc6a1bf0 con 0x55d8dc693e40 2024-09-19T04:53:13.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.618+0000 7fc4b2a00640 1 -- 172.21.15.123:0/2444580018 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b80041d0 con 0x55d8dc693e40 2024-09-19T04:53:13.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.618+0000 7fc4b2a00640 1 -- 172.21.15.123:0/2444580018 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fc4c004d030 con 0x55d8dc693e40 2024-09-19T04:53:13.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.618+0000 7fc4d6845740 10 client.16401 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:13.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.618+0000 7fc4d6845740 10 client.16401 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:53:13.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.618+0000 7fc4d6845740 20 client.16401 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:13.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.618+0000 7fc4d6845740 1 --2- 172.21.15.123:0/2444580018 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55d8dc6af080 0x55d8dc6b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.618+0000 7fc4d6845740 4 client.16401 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:53:13.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.618+0000 7fc4d6845740 1 -- 172.21.15.123:0/2444580018 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8dc607950 con 0x55d8dc6af080 2024-09-19T04:53:13.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.619+0000 7fc4d0a00640 1 --2- 172.21.15.123:0/2444580018 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55d8dc6af080 0x55d8dc6b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.619+0000 7fc4d0a00640 1 --2- 172.21.15.123:0/2444580018 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55d8dc6af080 0x55d8dc6b14d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c619561f3a681da6 server_cookie=bb097275e6504527 in_seq=0 out_seq=0 2024-09-19T04:53:13.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.619+0000 7fc4b2a00640 10 client.16401 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:53:13.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.620+0000 7fc4b2a00640 1 -- 172.21.15.123:0/2444580018 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8dc607950 con 0x55d8dc6af080 2024-09-19T04:53:13.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.620+0000 7fc4b2a00640 10 client.16401 handle_command_reply: tid=0 2024-09-19T04:53:13.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.620+0000 7fc4d6845740 1 client.16401 shutdown 2024-09-19T04:53:13.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.620+0000 7fc4d6845740 1 -- 172.21.15.123:0/2444580018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc693e40 msgr2=0x55d8dc69bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:13.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.620+0000 7fc4d6845740 1 --2- 172.21.15.123:0/2444580018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc693e40 0x55d8dc69bf60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0003fd0 tx=0x7fc4c002a950 comp rx=0 tx=0).stop 2024-09-19T04:53:13.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.620+0000 7fc4d6845740 1 -- 172.21.15.123:0/2444580018 shutdown_connections 2024-09-19T04:53:13.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.620+0000 7fc4d6845740 1 --2- 172.21.15.123:0/2444580018 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55d8dc6af080 0x55d8dc6b14d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.621+0000 7fc4d6845740 1 --2- 172.21.15.123:0/2444580018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc69f020 0x55d8dc69c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.621+0000 7fc4d6845740 1 --2- 172.21.15.123:0/2444580018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc694760 0x55d8dc69c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.621+0000 7fc4b2a00640 0 client.16401 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:53:13.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.621+0000 7fc4d6845740 1 --2- 172.21.15.123:0/2444580018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc693e40 0x55d8dc69bf60 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.621+0000 7fc4d6845740 1 -- 172.21.15.123:0/2444580018 >> 172.21.15.123:0/2444580018 conn(0x55d8dc6903d0 msgr2=0x55d8dc6960b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:13.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.621+0000 7fc4d6845740 1 -- 172.21.15.123:0/2444580018 shutdown_connections 2024-09-19T04:53:13.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.621+0000 7fc4d6845740 1 -- 172.21.15.123:0/2444580018 wait complete. 2024-09-19T04:53:13.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.621+0000 7fc4d6845740 20 client.16401 trim_cache size 0 max 16384 2024-09-19T04:53:13.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.634+0000 7fc4d6845740 1 Processor -- start 2024-09-19T04:53:13.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.634+0000 7fc4d6845740 1 -- start start 2024-09-19T04:53:13.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d6845740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc693e40 0x55d8dc6968a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d6845740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc694760 0x55d8dc68f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d6845740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc69f020 0x55d8dc6b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d6845740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1af930 con 0x55d8dc694760 2024-09-19T04:53:13.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d6845740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1b1500 con 0x55d8dc693e40 2024-09-19T04:53:13.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d6845740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1b0030 con 0x55d8dc69f020 2024-09-19T04:53:13.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc694760 0x55d8dc68f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc694760 0x55d8dc68f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36656/0 (socket says 172.21.15.123:36656) 2024-09-19T04:53:13.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d0a00640 1 -- 172.21.15.123:0/1714444202 learned_addr learned my addr 172.21.15.123:0/1714444202 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:13.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d1400640 1 --2- 172.21.15.123:0/1714444202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc693e40 0x55d8dc6968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d1e00640 1 --2- 172.21.15.123:0/1714444202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc69f020 0x55d8dc6b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d1400640 1 -- 172.21.15.123:0/1714444202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc69f020 msgr2=0x55d8dc6b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:13.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d1400640 1 --2- 172.21.15.123:0/1714444202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc69f020 0x55d8dc6b6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d1400640 1 -- 172.21.15.123:0/1714444202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc694760 msgr2=0x55d8dc68f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:13.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d1400640 1 --2- 172.21.15.123:0/1714444202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc694760 0x55d8dc68f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d1400640 1 -- 172.21.15.123:0/1714444202 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8dc6900a0 con 0x55d8dc693e40 2024-09-19T04:53:13.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d0a00640 1 --2- 172.21.15.123:0/1714444202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc694760 0x55d8dc68f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:13.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.635+0000 7fc4d1e00640 1 --2- 172.21.15.123:0/1714444202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc69f020 0x55d8dc6b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:53:13.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.636+0000 7fc4d1400640 1 --2- 172.21.15.123:0/1714444202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc693e40 0x55d8dc6968a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc4c002ae70 tx=0x7fc4c0004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1545cb1c5d1ca336 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:13.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.636+0000 7fc4bdc00640 1 -- 172.21.15.123:0/1714444202 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00bc970 con 0x55d8dc693e40 2024-09-19T04:53:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.636+0000 7fc4bdc00640 1 -- 172.21.15.123:0/1714444202 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c00da050 con 0x55d8dc693e40 2024-09-19T04:53:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.636+0000 7fc4bdc00640 1 -- 172.21.15.123:0/1714444202 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c002c040 con 0x55d8dc693e40 2024-09-19T04:53:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.636+0000 7fc4d6845740 1 -- 172.21.15.123:0/1714444202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc693e40 msgr2=0x55d8dc6968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.636+0000 7fc4d6845740 1 --2- 172.21.15.123:0/1714444202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc693e40 0x55d8dc6968a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc4c002ae70 tx=0x7fc4c0004040 comp rx=0 tx=0).stop 2024-09-19T04:53:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.637+0000 7fc4d6845740 1 -- 172.21.15.123:0/1714444202 shutdown_connections 2024-09-19T04:53:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.637+0000 7fc4d6845740 1 --2- 172.21.15.123:0/1714444202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc69f020 0x55d8dc6b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.637+0000 7fc4d6845740 1 --2- 172.21.15.123:0/1714444202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc694760 0x55d8dc68f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.637+0000 7fc4d6845740 1 --2- 172.21.15.123:0/1714444202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc693e40 0x55d8dc6968a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.637+0000 7fc4d6845740 1 -- 172.21.15.123:0/1714444202 >> 172.21.15.123:0/1714444202 conn(0x55d8dc6903d0 msgr2=0x55d8dc68ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.637+0000 7fc4d6845740 1 -- 172.21.15.123:0/1714444202 shutdown_connections 2024-09-19T04:53:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.637+0000 7fc4d6845740 1 -- 172.21.15.123:0/1714444202 wait complete. 2024-09-19T04:53:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.637+0000 7fc4d6845740 1 -- start start 2024-09-19T04:53:13.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.637+0000 7fc4d6845740 1 Processor -- start 2024-09-19T04:53:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d6845740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc693e40 0x55d8dc6a7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d6845740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc694760 0x55d8dc698d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d6845740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc69f020 0x55d8dc699250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d6845740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1b0e30 con 0x55d8dc694760 2024-09-19T04:53:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d6845740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1b0730 con 0x55d8dc69f020 2024-09-19T04:53:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d6845740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1ad930 con 0x55d8dc693e40 2024-09-19T04:53:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d6845740 10 client.0 _ll_register_callbacks cb 0x7fc4c0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc694760 0x55d8dc698d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d1400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc693e40 0x55d8dc6a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc69f020 0x55d8dc699250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d1400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc693e40 0x55d8dc6a7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42212/0 (socket says 172.21.15.123:42212) 2024-09-19T04:53:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d1400640 1 -- 172.21.15.123:0/3432858479 learned_addr learned my addr 172.21.15.123:0/3432858479 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc69f020 0x55d8dc699250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:33794/0 (socket says 172.21.15.123:33794) 2024-09-19T04:53:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d1400640 1 -- 172.21.15.123:0/3432858479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc69f020 msgr2=0x55d8dc699250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d1400640 1 --2- 172.21.15.123:0/3432858479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc69f020 0x55d8dc699250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d1400640 1 -- 172.21.15.123:0/3432858479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc694760 msgr2=0x55d8dc698d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d1400640 1 --2- 172.21.15.123:0/3432858479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc694760 0x55d8dc698d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.638+0000 7fc4d1400640 1 -- 172.21.15.123:0/3432858479 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8dc699a80 con 0x55d8dc693e40 2024-09-19T04:53:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.639+0000 7fc4d1400640 1 --2- 172.21.15.123:0/3432858479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc693e40 0x55d8dc6a7780 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00b9290 tx=0x7fc4c002a950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92dfdefb03d347c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.639+0000 7fc4b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:53:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.639+0000 7fc4b2a00640 1 -- 172.21.15.123:0/3432858479 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00a0070 con 0x55d8dc693e40 2024-09-19T04:53:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.639+0000 7fc4d6845740 1 -- 172.21.15.123:0/3432858479 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4c00da050 con 0x55d8dc693e40 2024-09-19T04:53:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.639+0000 7fc4b2a00640 1 -- 172.21.15.123:0/3432858479 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7fc4c0030640 con 0x55d8dc693e40 2024-09-19T04:53:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.639+0000 7fc4b2a00640 1 -- 172.21.15.123:0/3432858479 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc4c00ca050 con 0x55d8dc693e40 2024-09-19T04:53:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.639+0000 7fc4d6845740 10 client.26527 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.639+0000 7fc4d6845740 1 -- 172.21.15.123:0/3432858479 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d8dc7225b0 con 0x55d8dc693e40 2024-09-19T04:53:13.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.639+0000 7fc4b2a00640 1 -- 172.21.15.123:0/3432858479 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c00da050 con 0x55d8dc693e40 2024-09-19T04:53:13.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.640+0000 7fc4b2a00640 1 -- 172.21.15.123:0/3432858479 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fc4c004d070 con 0x55d8dc693e40 2024-09-19T04:53:13.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.640+0000 7fc4d6845740 10 client.26527 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:13.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.640+0000 7fc4d6845740 10 client.26527 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:53:13.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.640+0000 7fc4d6845740 20 client.26527 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:13.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.640+0000 7fc4d6845740 1 --2- 172.21.15.123:0/3432858479 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55d8dc722a80 0x55d8dc724e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:13.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.640+0000 7fc4d6845740 4 client.26527 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:53:13.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.640+0000 7fc4d6845740 1 -- 172.21.15.123:0/3432858479 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8dc601e60 con 0x55d8dc722a80 2024-09-19T04:53:13.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.640+0000 7fc4d0a00640 1 --2- 172.21.15.123:0/3432858479 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55d8dc722a80 0x55d8dc724e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:13.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.640+0000 7fc4d0a00640 1 --2- 172.21.15.123:0/3432858479 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55d8dc722a80 0x55d8dc724e70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a22f8a86b5a391ee server_cookie=86757b557c14bcae in_seq=0 out_seq=0 2024-09-19T04:53:13.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.640+0000 7fc4b2a00640 10 client.26527 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:53:13.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.641+0000 7fc4b2a00640 1 -- 172.21.15.123:0/3432858479 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8dc601e60 con 0x55d8dc722a80 2024-09-19T04:53:13.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.641+0000 7fc4b2a00640 10 client.26527 handle_command_reply: tid=0 2024-09-19T04:53:13.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.641+0000 7fc4d6845740 1 client.26527 shutdown 2024-09-19T04:53:13.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.641+0000 7fc4d6845740 1 -- 172.21.15.123:0/3432858479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc693e40 msgr2=0x55d8dc6a7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:13.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.641+0000 7fc4d6845740 1 --2- 172.21.15.123:0/3432858479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc693e40 0x55d8dc6a7780 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00b9290 tx=0x7fc4c002a950 comp rx=0 tx=0).stop 2024-09-19T04:53:13.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.641+0000 7fc4d6845740 1 -- 172.21.15.123:0/3432858479 shutdown_connections 2024-09-19T04:53:13.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.641+0000 7fc4d6845740 1 --2- 172.21.15.123:0/3432858479 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55d8dc722a80 0x55d8dc724e70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.641+0000 7fc4d6845740 1 --2- 172.21.15.123:0/3432858479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8dc69f020 0x55d8dc699250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.641+0000 7fc4d6845740 1 --2- 172.21.15.123:0/3432858479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8dc694760 0x55d8dc698d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.641+0000 7fc4d6845740 1 --2- 172.21.15.123:0/3432858479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8dc693e40 0x55d8dc6a7780 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.641+0000 7fc4b2a00640 0 client.26527 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:53:13.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.641+0000 7fc4d6845740 1 -- 172.21.15.123:0/3432858479 >> 172.21.15.123:0/3432858479 conn(0x55d8dc6903d0 msgr2=0x55d8dc68f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:13.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.641+0000 7fc4d6845740 1 -- 172.21.15.123:0/3432858479 shutdown_connections 2024-09-19T04:53:13.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.641+0000 7fc4d6845740 1 -- 172.21.15.123:0/3432858479 wait complete. 2024-09-19T04:53:13.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.641+0000 7fc4d6845740 20 client.26527 trim_cache size 0 max 16384 2024-09-19T04:53:13.648 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:53:13.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.642+0000 7fc4d2800640 1 -- 172.21.15.123:0/2076007373 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc4b40a33b0 msgr2=0x7fc4b40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:13.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.642+0000 7fc4d2800640 1 --2- 172.21.15.123:0/2076007373 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc4b40a33b0 0x7fc4b40a5870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009d20 tx=0x7fc4c4005f90 comp rx=0 tx=0).stop 2024-09-19T04:53:13.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.642+0000 7fc4d2800640 1 -- 172.21.15.123:0/2076007373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4cc102b90 msgr2=0x7fc4cc19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:13.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.642+0000 7fc4d2800640 1 --2- 172.21.15.123:0/2076007373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4cc102b90 0x7fc4cc19a4e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0005cc0 tx=0x7fc4c002f420 comp rx=0 tx=0).stop 2024-09-19T04:53:13.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.642+0000 7fc4d2800640 1 -- 172.21.15.123:0/2076007373 shutdown_connections 2024-09-19T04:53:13.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.642+0000 7fc4d2800640 1 --2- 172.21.15.123:0/2076007373 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc4b40a33b0 0x7fc4b40a5870 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.642+0000 7fc4d2800640 1 --2- 172.21.15.123:0/2076007373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4cc10bfc0 0x7fc4cc19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.642+0000 7fc4d2800640 1 --2- 172.21.15.123:0/2076007373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4cc1034b0 0x7fc4cc19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.642+0000 7fc4d2800640 1 --2- 172.21.15.123:0/2076007373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4cc102b90 0x7fc4cc19a4e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:13.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.642+0000 7fc4d2800640 1 -- 172.21.15.123:0/2076007373 >> 172.21.15.123:0/2076007373 conn(0x7fc4cc0fc610 msgr2=0x7fc4cc0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:13.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.642+0000 7fc4d2800640 1 -- 172.21.15.123:0/2076007373 shutdown_connections 2024-09-19T04:53:13.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:13.642+0000 7fc4d2800640 1 -- 172.21.15.123:0/2076007373 wait complete. 2024-09-19T04:53:13.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:13 smithi193 ceph-mon[35592]: pgmap v199: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T04:53:14.532 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:53:14.901 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:53:15.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.470+0000 7fc48b600640 1 -- 172.21.15.123:0/3937365546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc484077040 msgr2=0x7fc4841124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:15.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.470+0000 7fc48b600640 1 --2- 172.21.15.123:0/3937365546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc484077040 0x7fc4841124f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc470009900 tx=0x7fc47002a400 comp rx=0 tx=0).stop 2024-09-19T04:53:15.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.470+0000 7fc48b600640 1 -- 172.21.15.123:0/3937365546 shutdown_connections 2024-09-19T04:53:15.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.470+0000 7fc48b600640 1 --2- 172.21.15.123:0/3937365546 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc484112a30 0x7fc484114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.470+0000 7fc48b600640 1 --2- 172.21.15.123:0/3937365546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc484077040 0x7fc4841124f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.470+0000 7fc48b600640 1 --2- 172.21.15.123:0/3937365546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc484076720 0x7fc484076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.470+0000 7fc48b600640 1 -- 172.21.15.123:0/3937365546 >> 172.21.15.123:0/3937365546 conn(0x7fc4840ffa90 msgr2=0x7fc484101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:15.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.470+0000 7fc48b600640 1 -- 172.21.15.123:0/3937365546 shutdown_connections 2024-09-19T04:53:15.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.470+0000 7fc48b600640 1 -- 172.21.15.123:0/3937365546 wait complete. 2024-09-19T04:53:15.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.471+0000 7fc48b600640 1 Processor -- start 2024-09-19T04:53:15.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.471+0000 7fc48b600640 1 -- start start 2024-09-19T04:53:15.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.472+0000 7fc48b600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc484076720 0x7fc4841a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.472+0000 7fc48b600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc484077040 0x7fc4841a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.472+0000 7fc48b600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc484112a30 0x7fc48419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.472+0000 7fc48b600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841185c0 con 0x7fc484076720 2024-09-19T04:53:15.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.472+0000 7fc48b600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc484118440 con 0x7fc484112a30 2024-09-19T04:53:15.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.472+0000 7fc48b600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc484118740 con 0x7fc484077040 2024-09-19T04:53:15.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.472+0000 7fc488c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc484076720 0x7fc4841a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.472+0000 7fc488c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc484076720 0x7fc4841a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36682/0 (socket says 172.21.15.123:36682) 2024-09-19T04:53:15.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.472+0000 7fc488c00640 1 -- 172.21.15.123:0/777688427 learned_addr learned my addr 172.21.15.123:0/777688427 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:15.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.472+0000 7fc483e00640 1 --2- 172.21.15.123:0/777688427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc484077040 0x7fc4841a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.472+0000 7fc489600640 1 --2- 172.21.15.123:0/777688427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc484112a30 0x7fc48419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.472+0000 7fc488c00640 1 -- 172.21.15.123:0/777688427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc484077040 msgr2=0x7fc4841a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:15.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.472+0000 7fc488c00640 1 --2- 172.21.15.123:0/777688427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc484077040 0x7fc4841a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.472+0000 7fc488c00640 1 -- 172.21.15.123:0/777688427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc484112a30 msgr2=0x7fc48419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:15.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.472+0000 7fc488c00640 1 --2- 172.21.15.123:0/777688427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc484112a30 0x7fc48419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.472+0000 7fc488c00640 1 -- 172.21.15.123:0/777688427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48419f000 con 0x7fc484076720 2024-09-19T04:53:15.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.473+0000 7fc488c00640 1 --2- 172.21.15.123:0/777688427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc484076720 0x7fc4841a4710 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc47800e9f0 tx=0x7fc47800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:15.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.473+0000 7fc481600640 1 -- 172.21.15.123:0/777688427 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47800cd80 con 0x7fc484076720 2024-09-19T04:53:15.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.473+0000 7fc481600640 1 -- 172.21.15.123:0/777688427 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc478005500 con 0x7fc484076720 2024-09-19T04:53:15.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.473+0000 7fc48b600640 1 -- 172.21.15.123:0/777688427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48419f2f0 con 0x7fc484076720 2024-09-19T04:53:15.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.473+0000 7fc481600640 1 -- 172.21.15.123:0/777688427 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478010690 con 0x7fc484076720 2024-09-19T04:53:15.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.473+0000 7fc48b600640 1 -- 172.21.15.123:0/777688427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4841ab5e0 con 0x7fc484076720 2024-09-19T04:53:15.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.476+0000 7fc48dff8740 1 Processor -- start 2024-09-19T04:53:15.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.476+0000 7fc481600640 1 -- 172.21.15.123:0/777688427 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc4780046c0 con 0x7fc484076720 2024-09-19T04:53:15.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.477+0000 7fc48dff8740 1 -- start start 2024-09-19T04:53:15.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.477+0000 7fc48dff8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132e3c70 0x55b0132e4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.477+0000 7fc48dff8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b0132ee910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.477+0000 7fc48dff8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 0x55b0132f1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.477+0000 7fc48dff8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b01324a3e0 con 0x55b0132eee50 2024-09-19T04:53:15.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.477+0000 7fc48dff8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b0131f1d80 con 0x55b0132e3c70 2024-09-19T04:53:15.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.477+0000 7fc48dff8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b0132463a0 con 0x55b0132e4590 2024-09-19T04:53:15.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.477+0000 7fc489600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132e3c70 0x55b0132e4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.477+0000 7fc483e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 0x55b0132f1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.477+0000 7fc483e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 0x55b0132f1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36696/0 (socket says 172.21.15.123:36696) 2024-09-19T04:53:15.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.477+0000 7fc488c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b0132ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.477+0000 7fc483e00640 1 -- 172.21.15.123:0/749889247 learned_addr learned my addr 172.21.15.123:0/749889247 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:15.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.477+0000 7fc489600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132e3c70 0x55b0132e4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:33822/0 (socket says 172.21.15.123:33822) 2024-09-19T04:53:15.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.478+0000 7fc483e00640 1 -- 172.21.15.123:0/749889247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 msgr2=0x55b0132ee910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.478+0000 7fc483e00640 1 --2- 172.21.15.123:0/749889247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b0132ee910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.478+0000 7fc483e00640 1 -- 172.21.15.123:0/749889247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132e3c70 msgr2=0x55b0132e4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.478+0000 7fc483e00640 1 --2- 172.21.15.123:0/749889247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132e3c70 0x55b0132e4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.478+0000 7fc483e00640 1 -- 172.21.15.123:0/749889247 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b013210ae0 con 0x55b0132eee50 2024-09-19T04:53:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.478+0000 7fc483e00640 1 --2- 172.21.15.123:0/749889247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 0x55b0132f1240 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc470009900 tx=0x7fc470005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7532b071d0f3176a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.478+0000 7fc475c00640 1 -- 172.21.15.123:0/749889247 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470040070 con 0x55b0132eee50 2024-09-19T04:53:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.479+0000 7fc475c00640 1 -- 172.21.15.123:0/749889247 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc470031870 con 0x55b0132eee50 2024-09-19T04:53:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.479+0000 7fc475c00640 1 -- 172.21.15.123:0/749889247 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470035040 con 0x55b0132eee50 2024-09-19T04:53:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.479+0000 7fc481600640 1 --2- 172.21.15.123:0/777688427 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc46c0a3430 0x7fc46c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.479+0000 7fc481600640 1 -- 172.21.15.123:0/777688427 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7fc4780d0120 con 0x7fc484076720 2024-09-19T04:53:15.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.479+0000 7fc489600640 1 --2- 172.21.15.123:0/777688427 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc46c0a3430 0x7fc46c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.479+0000 7fc48dff8740 1 -- 172.21.15.123:0/749889247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 msgr2=0x55b0132f1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:15.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.479+0000 7fc48dff8740 1 --2- 172.21.15.123:0/749889247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 0x55b0132f1240 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc470009900 tx=0x7fc470005be0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.480+0000 7fc48dff8740 1 -- 172.21.15.123:0/749889247 shutdown_connections 2024-09-19T04:53:15.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.480+0000 7fc48dff8740 1 --2- 172.21.15.123:0/749889247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 0x55b0132f1240 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.480+0000 7fc48dff8740 1 --2- 172.21.15.123:0/749889247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b0132ee910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.480+0000 7fc48dff8740 1 --2- 172.21.15.123:0/749889247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132e3c70 0x55b0132e4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.480+0000 7fc48dff8740 1 -- 172.21.15.123:0/749889247 >> 172.21.15.123:0/749889247 conn(0x55b0132e0200 msgr2=0x55b0132e05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:15.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.480+0000 7fc489600640 1 --2- 172.21.15.123:0/777688427 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc46c0a3430 0x7fc46c0a58f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc48406b460 tx=0x7fc47c009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:15.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.480+0000 7fc48dff8740 1 -- 172.21.15.123:0/749889247 shutdown_connections 2024-09-19T04:53:15.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.480+0000 7fc48dff8740 1 -- 172.21.15.123:0/749889247 wait complete. 2024-09-19T04:53:15.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.480+0000 7fc48dff8740 1 -- start start 2024-09-19T04:53:15.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.480+0000 7fc48dff8740 1 Processor -- start 2024-09-19T04:53:15.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.481+0000 7fc48dff8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132e3c70 0x55b0132ebce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.481+0000 7fc48dff8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b0132ec220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.481+0000 7fc48dff8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132eee50 0x55b0132ec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.481+0000 7fc48dff8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b0132f1a20 con 0x55b0132e3c70 2024-09-19T04:53:15.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.481+0000 7fc48dff8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b0132050f0 con 0x55b0132eee50 2024-09-19T04:53:15.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.481+0000 7fc48dff8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b0132f1ba0 con 0x55b0132e4590 2024-09-19T04:53:15.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.481+0000 7fc48dff8740 10 client.0 _ll_register_callbacks cb 0x55b0131ecbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:15.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.481+0000 7fc488c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132e3c70 0x55b0132ebce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.481+0000 7fc488c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132e3c70 0x55b0132ebce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36702/0 (socket says 172.21.15.123:36702) 2024-09-19T04:53:15.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.481+0000 7fc488c00640 1 -- 172.21.15.123:0/2512256010 learned_addr learned my addr 172.21.15.123:0/2512256010 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:15.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.481+0000 7fc489600640 1 --2- 172.21.15.123:0/2512256010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132eee50 0x55b0132ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.481+0000 7fc483e00640 1 --2- 172.21.15.123:0/2512256010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b0132ec220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.482+0000 7fc488c00640 1 -- 172.21.15.123:0/2512256010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 msgr2=0x55b0132ec220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:15.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.482+0000 7fc488c00640 1 --2- 172.21.15.123:0/2512256010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b0132ec220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.482+0000 7fc488c00640 1 -- 172.21.15.123:0/2512256010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132eee50 msgr2=0x55b0132ec760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:15.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.482+0000 7fc488c00640 1 --2- 172.21.15.123:0/2512256010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132eee50 0x55b0132ec760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.482+0000 7fc488c00640 1 -- 172.21.15.123:0/2512256010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b012fbb3c0 con 0x55b0132e3c70 2024-09-19T04:53:15.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.482+0000 7fc488c00640 1 --2- 172.21.15.123:0/2512256010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132e3c70 0x55b0132ebce0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc47800c6a0 tx=0x7fc478005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb63a760a97fc59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:15.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.482+0000 7fc46a000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:53:15.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.482+0000 7fc46a000640 1 -- 172.21.15.123:0/2512256010 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47809bce0 con 0x55b0132e3c70 2024-09-19T04:53:15.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.482+0000 7fc46a000640 1 -- 172.21.15.123:0/2512256010 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7fc4780d2110 con 0x55b0132e3c70 2024-09-19T04:53:15.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.482+0000 7fc48dff8740 1 -- 172.21.15.123:0/2512256010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc470031870 con 0x55b0132e3c70 2024-09-19T04:53:15.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.482+0000 7fc46a000640 1 -- 172.21.15.123:0/2512256010 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc4780cdc70 con 0x55b0132e3c70 2024-09-19T04:53:15.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.482+0000 7fc48dff8740 10 client.16419 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:15.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.482+0000 7fc48dff8740 1 -- 172.21.15.123:0/2512256010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b0132fda20 con 0x55b0132e3c70 2024-09-19T04:53:15.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.483+0000 7fc46a000640 1 -- 172.21.15.123:0/2512256010 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc470031870 con 0x55b0132e3c70 2024-09-19T04:53:15.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.483+0000 7fc46a000640 1 -- 172.21.15.123:0/2512256010 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fc4780b5140 con 0x55b0132e3c70 2024-09-19T04:53:15.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.483+0000 7fc48dff8740 10 client.16419 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:15.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.483+0000 7fc48dff8740 10 client.16419 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:53:15.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.483+0000 7fc48dff8740 20 client.16419 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:15.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.483+0000 7fc48dff8740 1 --2- 172.21.15.123:0/2512256010 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55b0132fee30 0x55b013301240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.483+0000 7fc48dff8740 4 client.16419 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:53:15.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.483+0000 7fc48dff8740 1 -- 172.21.15.123:0/2512256010 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b013257950 con 0x55b0132fee30 2024-09-19T04:53:15.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.483+0000 7fc483e00640 1 --2- 172.21.15.123:0/2512256010 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55b0132fee30 0x55b013301240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.484+0000 7fc483e00640 1 --2- 172.21.15.123:0/2512256010 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55b0132fee30 0x55b013301240 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eaae31f279ed4a75 server_cookie=9aeecc3afb8ba485 in_seq=0 out_seq=0 2024-09-19T04:53:15.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.484+0000 7fc46a000640 10 client.16419 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:53:15.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.485+0000 7fc46a000640 1 -- 172.21.15.123:0/2512256010 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b013257950 con 0x55b0132fee30 2024-09-19T04:53:15.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.485+0000 7fc46a000640 10 client.16419 handle_command_reply: tid=0 2024-09-19T04:53:15.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.485+0000 7fc48dff8740 1 client.16419 shutdown 2024-09-19T04:53:15.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.485+0000 7fc48dff8740 1 -- 172.21.15.123:0/2512256010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132e3c70 msgr2=0x55b0132ebce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:15.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.485+0000 7fc48dff8740 1 --2- 172.21.15.123:0/2512256010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132e3c70 0x55b0132ebce0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc47800c6a0 tx=0x7fc478005730 comp rx=0 tx=0).stop 2024-09-19T04:53:15.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.485+0000 7fc48dff8740 1 -- 172.21.15.123:0/2512256010 shutdown_connections 2024-09-19T04:53:15.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.485+0000 7fc48dff8740 1 --2- 172.21.15.123:0/2512256010 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55b0132fee30 0x55b013301240 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.485+0000 7fc48dff8740 1 --2- 172.21.15.123:0/2512256010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132eee50 0x55b0132ec760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.485+0000 7fc48dff8740 1 --2- 172.21.15.123:0/2512256010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b0132ec220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.485+0000 7fc48dff8740 1 --2- 172.21.15.123:0/2512256010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132e3c70 0x55b0132ebce0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.485+0000 7fc46a000640 0 client.16419 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:53:15.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.485+0000 7fc48dff8740 1 -- 172.21.15.123:0/2512256010 >> 172.21.15.123:0/2512256010 conn(0x55b0132e0200 msgr2=0x55b0132df380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:15.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.485+0000 7fc48dff8740 1 -- 172.21.15.123:0/2512256010 shutdown_connections 2024-09-19T04:53:15.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.486+0000 7fc48dff8740 1 -- 172.21.15.123:0/2512256010 wait complete. 2024-09-19T04:53:15.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.486+0000 7fc48dff8740 20 client.16419 trim_cache size 0 max 16384 2024-09-19T04:53:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.499+0000 7fc48dff8740 1 Processor -- start 2024-09-19T04:53:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.499+0000 7fc48dff8740 1 -- start start 2024-09-19T04:53:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.499+0000 7fc48dff8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132e3c70 0x55b013306b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.499+0000 7fc48dff8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b013307050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.499+0000 7fc48dff8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 0x55b0132e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.499+0000 7fc48dff8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841b3f70 con 0x55b0132eee50 2024-09-19T04:53:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.500+0000 7fc48dff8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841b5b40 con 0x55b0132e3c70 2024-09-19T04:53:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.500+0000 7fc48dff8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841b4670 con 0x55b0132e4590 2024-09-19T04:53:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.500+0000 7fc489600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 0x55b0132e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.500+0000 7fc489600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 0x55b0132e8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36710/0 (socket says 172.21.15.123:36710) 2024-09-19T04:53:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.500+0000 7fc489600640 1 -- 172.21.15.123:0/1841856091 learned_addr learned my addr 172.21.15.123:0/1841856091 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.500+0000 7fc488c00640 1 --2- 172.21.15.123:0/1841856091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132e3c70 0x55b013306b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.500+0000 7fc483e00640 1 --2- 172.21.15.123:0/1841856091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b013307050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.500+0000 7fc483e00640 1 -- 172.21.15.123:0/1841856091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132e3c70 msgr2=0x55b013306b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.500+0000 7fc483e00640 1 --2- 172.21.15.123:0/1841856091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132e3c70 0x55b013306b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.500+0000 7fc483e00640 1 -- 172.21.15.123:0/1841856091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 msgr2=0x55b0132e8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.500+0000 7fc483e00640 1 --2- 172.21.15.123:0/1841856091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 0x55b0132e8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.500+0000 7fc483e00640 1 -- 172.21.15.123:0/1841856091 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0132e3aa0 con 0x55b0132e4590 2024-09-19T04:53:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.500+0000 7fc489600640 1 --2- 172.21.15.123:0/1841856091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 0x55b0132e8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.501+0000 7fc483e00640 1 --2- 172.21.15.123:0/1841856091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b013307050 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc470006010 tx=0x7fc470047040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=add6e5f337cc90bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.501+0000 7fc475200640 1 -- 172.21.15.123:0/1841856091 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470040040 con 0x55b0132e4590 2024-09-19T04:53:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.501+0000 7fc475200640 1 -- 172.21.15.123:0/1841856091 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc470031870 con 0x55b0132e4590 2024-09-19T04:53:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.501+0000 7fc475200640 1 -- 172.21.15.123:0/1841856091 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470035040 con 0x55b0132e4590 2024-09-19T04:53:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.502+0000 7fc48dff8740 1 -- 172.21.15.123:0/1841856091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 msgr2=0x55b013307050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.502+0000 7fc48dff8740 1 --2- 172.21.15.123:0/1841856091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b013307050 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc470006010 tx=0x7fc470047040 comp rx=0 tx=0).stop 2024-09-19T04:53:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.502+0000 7fc48dff8740 1 -- 172.21.15.123:0/1841856091 shutdown_connections 2024-09-19T04:53:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.502+0000 7fc48dff8740 1 --2- 172.21.15.123:0/1841856091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 0x55b0132e8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.502+0000 7fc48dff8740 1 --2- 172.21.15.123:0/1841856091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b013307050 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.502+0000 7fc48dff8740 1 --2- 172.21.15.123:0/1841856091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132e3c70 0x55b013306b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.502+0000 7fc48dff8740 1 -- 172.21.15.123:0/1841856091 >> 172.21.15.123:0/1841856091 conn(0x55b0132e0200 msgr2=0x55b0132edd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.502+0000 7fc48dff8740 1 -- 172.21.15.123:0/1841856091 shutdown_connections 2024-09-19T04:53:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.502+0000 7fc48dff8740 1 -- 172.21.15.123:0/1841856091 wait complete. 2024-09-19T04:53:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.502+0000 7fc48dff8740 1 -- start start 2024-09-19T04:53:15.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.502+0000 7fc48dff8740 1 Processor -- start 2024-09-19T04:53:15.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.503+0000 7fc48dff8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132e3c70 0x55b0132f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.503+0000 7fc48dff8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b0132f7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.503+0000 7fc48dff8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 0x55b0132f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.503+0000 7fc48dff8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841b5470 con 0x55b0132eee50 2024-09-19T04:53:15.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.503+0000 7fc48dff8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841b4d70 con 0x55b0132e3c70 2024-09-19T04:53:15.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.503+0000 7fc48dff8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841b1f70 con 0x55b0132e4590 2024-09-19T04:53:15.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.503+0000 7fc48dff8740 10 client.0 _ll_register_callbacks cb 0x7fc4780d38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:15.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.503+0000 7fc483e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b0132f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.503+0000 7fc489600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 0x55b0132f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.503+0000 7fc483e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b0132f7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42276/0 (socket says 172.21.15.123:42276) 2024-09-19T04:53:15.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.503+0000 7fc483e00640 1 -- 172.21.15.123:0/1070445293 learned_addr learned my addr 172.21.15.123:0/1070445293 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:15.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.503+0000 7fc488c00640 1 --2- 172.21.15.123:0/1070445293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132e3c70 0x55b0132f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.503+0000 7fc483e00640 1 -- 172.21.15.123:0/1070445293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132e3c70 msgr2=0x55b0132f6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:15.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.503+0000 7fc483e00640 1 --2- 172.21.15.123:0/1070445293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132e3c70 0x55b0132f6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.503+0000 7fc483e00640 1 -- 172.21.15.123:0/1070445293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 msgr2=0x55b0132f3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:15.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.503+0000 7fc483e00640 1 --2- 172.21.15.123:0/1070445293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 0x55b0132f3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.503+0000 7fc483e00640 1 -- 172.21.15.123:0/1070445293 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0132db770 con 0x55b0132e4590 2024-09-19T04:53:15.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.504+0000 7fc483e00640 1 --2- 172.21.15.123:0/1070445293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b0132f7360 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc47002a920 tx=0x7fc470005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d925075eb7a1148 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:15.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.504+0000 7fc46a000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:53:15.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.504+0000 7fc46a000640 1 -- 172.21.15.123:0/1070445293 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470040040 con 0x55b0132e4590 2024-09-19T04:53:15.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.504+0000 7fc48dff8740 1 -- 172.21.15.123:0/1070445293 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc470031870 con 0x55b0132e4590 2024-09-19T04:53:15.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.504+0000 7fc46a000640 1 -- 172.21.15.123:0/1070445293 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7fc470004750 con 0x55b0132e4590 2024-09-19T04:53:15.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.504+0000 7fc46a000640 1 -- 172.21.15.123:0/1070445293 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc470009520 con 0x55b0132e4590 2024-09-19T04:53:15.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.504+0000 7fc48dff8740 10 client.26545 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:15.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.504+0000 7fc48dff8740 1 -- 172.21.15.123:0/1070445293 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b01336a120 con 0x55b0132e4590 2024-09-19T04:53:15.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.505+0000 7fc46a000640 1 -- 172.21.15.123:0/1070445293 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc470031870 con 0x55b0132e4590 2024-09-19T04:53:15.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.505+0000 7fc46a000640 1 -- 172.21.15.123:0/1070445293 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fc47003c750 con 0x55b0132e4590 2024-09-19T04:53:15.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.505+0000 7fc48dff8740 10 client.26545 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:15.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.505+0000 7fc48dff8740 10 client.26545 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:53:15.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.505+0000 7fc48dff8740 20 client.26545 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:15.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.505+0000 7fc48dff8740 1 --2- 172.21.15.123:0/1070445293 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55b01336a350 0x55b01336c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:15.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.505+0000 7fc48dff8740 4 client.26545 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:53:15.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.505+0000 7fc48dff8740 1 -- 172.21.15.123:0/1070445293 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b013251e60 con 0x55b01336a350 2024-09-19T04:53:15.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.505+0000 7fc488c00640 1 --2- 172.21.15.123:0/1070445293 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55b01336a350 0x55b01336c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:15.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.506+0000 7fc488c00640 1 --2- 172.21.15.123:0/1070445293 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55b01336a350 0x55b01336c740 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8acb3af99def1288 server_cookie=eb640ac2bc70674f in_seq=0 out_seq=0 2024-09-19T04:53:15.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.506+0000 7fc46a000640 10 client.26545 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:53:15.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.506+0000 7fc46a000640 1 -- 172.21.15.123:0/1070445293 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b013251e60 con 0x55b01336a350 2024-09-19T04:53:15.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.506+0000 7fc46a000640 10 client.26545 handle_command_reply: tid=0 2024-09-19T04:53:15.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.506+0000 7fc48dff8740 1 client.26545 shutdown 2024-09-19T04:53:15.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.506+0000 7fc48dff8740 1 -- 172.21.15.123:0/1070445293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 msgr2=0x55b0132f7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:15.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.506+0000 7fc48dff8740 1 --2- 172.21.15.123:0/1070445293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b0132f7360 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc47002a920 tx=0x7fc470005850 comp rx=0 tx=0).stop 2024-09-19T04:53:15.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.506+0000 7fc48dff8740 1 -- 172.21.15.123:0/1070445293 shutdown_connections 2024-09-19T04:53:15.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.506+0000 7fc48dff8740 1 --2- 172.21.15.123:0/1070445293 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55b01336a350 0x55b01336c740 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.506+0000 7fc48dff8740 1 --2- 172.21.15.123:0/1070445293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0132eee50 0x55b0132f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.506+0000 7fc48dff8740 1 --2- 172.21.15.123:0/1070445293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0132e4590 0x55b0132f7360 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.506+0000 7fc46a000640 0 client.26545 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:53:15.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.506+0000 7fc48dff8740 1 --2- 172.21.15.123:0/1070445293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0132e3c70 0x55b0132f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.506+0000 7fc48dff8740 1 -- 172.21.15.123:0/1070445293 >> 172.21.15.123:0/1070445293 conn(0x55b0132e0200 msgr2=0x55b0132e56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:15.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.506+0000 7fc48dff8740 1 -- 172.21.15.123:0/1070445293 shutdown_connections 2024-09-19T04:53:15.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.506+0000 7fc48dff8740 1 -- 172.21.15.123:0/1070445293 wait complete. 2024-09-19T04:53:15.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.506+0000 7fc48dff8740 20 client.26545 trim_cache size 0 max 16384 2024-09-19T04:53:15.520 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:53:15.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.507+0000 7fc48b600640 1 -- 172.21.15.123:0/777688427 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc46c0a3430 msgr2=0x7fc46c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:15.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.507+0000 7fc48b600640 1 --2- 172.21.15.123:0/777688427 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc46c0a3430 0x7fc46c0a58f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc48406b460 tx=0x7fc47c009500 comp rx=0 tx=0).stop 2024-09-19T04:53:15.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.507+0000 7fc48b600640 1 -- 172.21.15.123:0/777688427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc484076720 msgr2=0x7fc4841a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:15.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.507+0000 7fc48b600640 1 --2- 172.21.15.123:0/777688427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc484076720 0x7fc4841a4710 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc47800e9f0 tx=0x7fc47800edf0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.507+0000 7fc48b600640 1 -- 172.21.15.123:0/777688427 shutdown_connections 2024-09-19T04:53:15.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.507+0000 7fc48b600640 1 --2- 172.21.15.123:0/777688427 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc46c0a3430 0x7fc46c0a58f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.507+0000 7fc48b600640 1 --2- 172.21.15.123:0/777688427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc484112a30 0x7fc48419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.507+0000 7fc48b600640 1 --2- 172.21.15.123:0/777688427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc484077040 0x7fc4841a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.507+0000 7fc48b600640 1 --2- 172.21.15.123:0/777688427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc484076720 0x7fc4841a4710 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:15.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.507+0000 7fc48b600640 1 -- 172.21.15.123:0/777688427 >> 172.21.15.123:0/777688427 conn(0x7fc4840ffa90 msgr2=0x7fc484101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:15.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.507+0000 7fc48b600640 1 -- 172.21.15.123:0/777688427 shutdown_connections 2024-09-19T04:53:15.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:15.507+0000 7fc48b600640 1 -- 172.21.15.123:0/777688427 wait complete. 2024-09-19T04:53:15.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:15 smithi183 ceph-mon[35333]: pgmap v200: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:53:15.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:15 smithi123 ceph-mon[30511]: pgmap v200: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:53:15.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:15 smithi193 ceph-mon[35592]: pgmap v200: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:53:16.439 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:53:16.807 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:53:17.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.381+0000 7fb0ef200640 1 -- 172.21.15.123:0/3270310742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0f0077040 msgr2=0x7fb0f0112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:17.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.381+0000 7fb0ef200640 1 --2- 172.21.15.123:0/3270310742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0f0077040 0x7fb0f0112330 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009900 tx=0x7fb0dc02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.381+0000 7fb0ef200640 1 -- 172.21.15.123:0/3270310742 shutdown_connections 2024-09-19T04:53:17.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.381+0000 7fb0ef200640 1 --2- 172.21.15.123:0/3270310742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f0112870 0x7fb0f0114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.381+0000 7fb0ef200640 1 --2- 172.21.15.123:0/3270310742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0f0077040 0x7fb0f0112330 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.381+0000 7fb0ef200640 1 --2- 172.21.15.123:0/3270310742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0f0076720 0x7fb0f0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.381+0000 7fb0ef200640 1 -- 172.21.15.123:0/3270310742 >> 172.21.15.123:0/3270310742 conn(0x7fb0f00ff8d0 msgr2=0x7fb0f0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:17.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.381+0000 7fb0ef200640 1 -- 172.21.15.123:0/3270310742 shutdown_connections 2024-09-19T04:53:17.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.382+0000 7fb0ef200640 1 -- 172.21.15.123:0/3270310742 wait complete. 2024-09-19T04:53:17.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.382+0000 7fb0ef200640 1 Processor -- start 2024-09-19T04:53:17.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.383+0000 7fb0ef200640 1 -- start start 2024-09-19T04:53:17.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.383+0000 7fb0ef200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f0076720 0x7fb0f01a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.383+0000 7fb0ef200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0f0077040 0x7fb0f01a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.383+0000 7fb0ef200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0f0112870 0x7fb0f019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.383+0000 7fb0ef200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f0118430 con 0x7fb0f0112870 2024-09-19T04:53:17.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.383+0000 7fb0ef200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f01182b0 con 0x7fb0f0077040 2024-09-19T04:53:17.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.383+0000 7fb0ef200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f01185b0 con 0x7fb0f0076720 2024-09-19T04:53:17.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.383+0000 7fb0ee800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0f0112870 0x7fb0f019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.383+0000 7fb0ede00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f0076720 0x7fb0f01a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.383+0000 7fb0ee800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0f0112870 0x7fb0f019e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36742/0 (socket says 172.21.15.123:36742) 2024-09-19T04:53:17.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.383+0000 7fb0ee800640 1 -- 172.21.15.123:0/921585769 learned_addr learned my addr 172.21.15.123:0/921585769 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:17.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.383+0000 7fb0ed400640 1 --2- 172.21.15.123:0/921585769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0f0077040 0x7fb0f01a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.384+0000 7fb0ede00640 1 -- 172.21.15.123:0/921585769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0f0077040 msgr2=0x7fb0f01a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:17.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.384+0000 7fb0ede00640 1 --2- 172.21.15.123:0/921585769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0f0077040 0x7fb0f01a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.384+0000 7fb0ede00640 1 -- 172.21.15.123:0/921585769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0f0112870 msgr2=0x7fb0f019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:17.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.384+0000 7fb0ede00640 1 --2- 172.21.15.123:0/921585769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0f0112870 0x7fb0f019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.384+0000 7fb0ede00640 1 -- 172.21.15.123:0/921585769 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f019efc0 con 0x7fb0f0076720 2024-09-19T04:53:17.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.384+0000 7fb0ee800640 1 --2- 172.21.15.123:0/921585769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0f0112870 0x7fb0f019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:17.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.384+0000 7fb0ede00640 1 --2- 172.21.15.123:0/921585769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f0076720 0x7fb0f01a4680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000e9f0 tx=0x7fb0e000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:17.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.384+0000 7fb0e6a00640 1 -- 172.21.15.123:0/921585769 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000ccc0 con 0x7fb0f0076720 2024-09-19T04:53:17.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.384+0000 7fb0e6a00640 1 -- 172.21.15.123:0/921585769 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0e0010910 con 0x7fb0f0076720 2024-09-19T04:53:17.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.385+0000 7fb0ef200640 1 -- 172.21.15.123:0/921585769 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f019f2b0 con 0x7fb0f0076720 2024-09-19T04:53:17.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.385+0000 7fb0e6a00640 1 -- 172.21.15.123:0/921585769 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0018960 con 0x7fb0f0076720 2024-09-19T04:53:17.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.385+0000 7fb0ef200640 1 -- 172.21.15.123:0/921585769 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f01ab600 con 0x7fb0f0076720 2024-09-19T04:53:17.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.388+0000 7fb0f7839740 1 Processor -- start 2024-09-19T04:53:17.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.388+0000 7fb0f7839740 1 -- start start 2024-09-19T04:53:17.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.388+0000 7fb0e6a00640 1 -- 172.21.15.123:0/921585769 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb0e0010ac0 con 0x7fb0f0076720 2024-09-19T04:53:17.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.388+0000 7fb0f7839740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7d6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.389+0000 7fb0f7839740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7d6760 0x55cb2c7e0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.389+0000 7fb0f7839740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7e1020 0x55cb2c7e3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.389+0000 7fb0f7839740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cb2c73c3e0 con 0x55cb2c7d6760 2024-09-19T04:53:17.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.389+0000 7fb0f7839740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cb2c6e3d80 con 0x55cb2c7d5e40 2024-09-19T04:53:17.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.389+0000 7fb0f7839740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cb2c7383a0 con 0x55cb2c7e1020 2024-09-19T04:53:17.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.389+0000 7fb0ee800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7d6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.389+0000 7fb0ed400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7e1020 0x55cb2c7e3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.389+0000 7fb0ee800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7d6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:33892/0 (socket says 172.21.15.123:33892) 2024-09-19T04:53:17.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.389+0000 7fb0ee800640 1 -- 172.21.15.123:0/4120384272 learned_addr learned my addr 172.21.15.123:0/4120384272 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:17.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.389+0000 7fb0ede00640 1 --2- 172.21.15.123:0/4120384272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7d6760 0x55cb2c7e0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.389+0000 7fb0ee800640 1 -- 172.21.15.123:0/4120384272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7e1020 msgr2=0x55cb2c7e3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:17.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.389+0000 7fb0ee800640 1 --2- 172.21.15.123:0/4120384272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7e1020 0x55cb2c7e3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.389+0000 7fb0ee800640 1 -- 172.21.15.123:0/4120384272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7d6760 msgr2=0x55cb2c7e0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:17.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.389+0000 7fb0ee800640 1 --2- 172.21.15.123:0/4120384272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7d6760 0x55cb2c7e0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.389+0000 7fb0ee800640 1 -- 172.21.15.123:0/4120384272 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb2c702ae0 con 0x55cb2c7d5e40 2024-09-19T04:53:17.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.390+0000 7fb0ede00640 1 --2- 172.21.15.123:0/4120384272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7d6760 0x55cb2c7e0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:53:17.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.390+0000 7fb0ee800640 1 --2- 172.21.15.123:0/4120384272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7d6220 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8011780 tx=0x7fb0e8011c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50f27162761b879a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:17.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.390+0000 7fb0e6a00640 1 --2- 172.21.15.123:0/921585769 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb0d40a33b0 0x7fb0d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.390+0000 7fb0db400640 1 -- 172.21.15.123:0/4120384272 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e800fae0 con 0x55cb2c7d5e40 2024-09-19T04:53:17.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.390+0000 7fb0db400640 1 -- 172.21.15.123:0/4120384272 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0e8011e90 con 0x55cb2c7d5e40 2024-09-19T04:53:17.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.390+0000 7fb0db400640 1 -- 172.21.15.123:0/4120384272 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e801a650 con 0x55cb2c7d5e40 2024-09-19T04:53:17.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.390+0000 7fb0e6a00640 1 -- 172.21.15.123:0/921585769 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb0e00d0110 con 0x7fb0f0076720 2024-09-19T04:53:17.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.390+0000 7fb0ede00640 1 --2- 172.21.15.123:0/921585769 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb0d40a33b0 0x7fb0d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.391+0000 7fb0f7839740 1 -- 172.21.15.123:0/4120384272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d5e40 msgr2=0x55cb2c7d6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:17.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.391+0000 7fb0f7839740 1 --2- 172.21.15.123:0/4120384272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7d6220 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8011780 tx=0x7fb0e8011c40 comp rx=0 tx=0).stop 2024-09-19T04:53:17.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.391+0000 7fb0ede00640 1 --2- 172.21.15.123:0/921585769 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb0d40a33b0 0x7fb0d40a5870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55cb2c7d3230 tx=0x7fb0e0009700 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:17.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.391+0000 7fb0f7839740 1 -- 172.21.15.123:0/4120384272 shutdown_connections 2024-09-19T04:53:17.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.391+0000 7fb0f7839740 1 --2- 172.21.15.123:0/4120384272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7e1020 0x55cb2c7e3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.391+0000 7fb0f7839740 1 --2- 172.21.15.123:0/4120384272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7d6760 0x55cb2c7e0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.391+0000 7fb0f7839740 1 --2- 172.21.15.123:0/4120384272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7d6220 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.391+0000 7fb0f7839740 1 -- 172.21.15.123:0/4120384272 >> 172.21.15.123:0/4120384272 conn(0x55cb2c7d23d0 msgr2=0x55cb2c7d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:17.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.391+0000 7fb0f7839740 1 -- 172.21.15.123:0/4120384272 shutdown_connections 2024-09-19T04:53:17.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.391+0000 7fb0f7839740 1 -- 172.21.15.123:0/4120384272 wait complete. 2024-09-19T04:53:17.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.392+0000 7fb0f7839740 1 -- start start 2024-09-19T04:53:17.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.392+0000 7fb0f7839740 1 Processor -- start 2024-09-19T04:53:17.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0f7839740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7ddf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0f7839740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d6760 0x55cb2c7de480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0f7839740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 0x55cb2c7de9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0f7839740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cb2c7e3bf0 con 0x55cb2c7e1020 2024-09-19T04:53:17.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0f7839740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cb2c6f70f0 con 0x55cb2c7d5e40 2024-09-19T04:53:17.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0f7839740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cb2c7e3d70 con 0x55cb2c7d6760 2024-09-19T04:53:17.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0f7839740 10 client.0 _ll_register_callbacks cb 0x55cb2c6debd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:17.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0ed400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d6760 0x55cb2c7de480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0ee800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 0x55cb2c7de9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0ed400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d6760 0x55cb2c7de480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42294/0 (socket says 172.21.15.123:42294) 2024-09-19T04:53:17.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0ed400640 1 -- 172.21.15.123:0/1789783566 learned_addr learned my addr 172.21.15.123:0/1789783566 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:17.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0ee800640 1 --2- 172.21.15.123:0/1789783566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7ddf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0ed400640 1 -- 172.21.15.123:0/1789783566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d5e40 msgr2=0x55cb2c7ddf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:17.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0ed400640 1 --2- 172.21.15.123:0/1789783566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7ddf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0ed400640 1 -- 172.21.15.123:0/1789783566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 msgr2=0x55cb2c7de9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:17.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0ed400640 1 --2- 172.21.15.123:0/1789783566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 0x55cb2c7de9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0ed400640 1 -- 172.21.15.123:0/1789783566 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb2c4ad3c0 con 0x55cb2c7d6760 2024-09-19T04:53:17.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.393+0000 7fb0ee800640 1 --2- 172.21.15.123:0/1789783566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 0x55cb2c7de9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:53:17.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.394+0000 7fb0ed400640 1 --2- 172.21.15.123:0/1789783566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d6760 0x55cb2c7de480 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc033000 tx=0x7fb0dc004420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e60bf1fa9d5179 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:17.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.394+0000 7fb0cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:53:17.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.394+0000 7fb0cfe00640 1 -- 172.21.15.123:0/1789783566 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc035040 con 0x55cb2c7d6760 2024-09-19T04:53:17.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.394+0000 7fb0f7839740 1 -- 172.21.15.123:0/1789783566 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0e8011e90 con 0x55cb2c7d6760 2024-09-19T04:53:17.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.394+0000 7fb0cfe00640 1 -- 172.21.15.123:0/1789783566 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb0dc005450 con 0x55cb2c7d6760 2024-09-19T04:53:17.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.394+0000 7fb0cfe00640 1 -- 172.21.15.123:0/1789783566 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb0dc005650 con 0x55cb2c7d6760 2024-09-19T04:53:17.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.394+0000 7fb0f7839740 10 client.26557 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:17.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.394+0000 7fb0f7839740 1 -- 172.21.15.123:0/1789783566 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55cb2c7efcb0 con 0x55cb2c7d6760 2024-09-19T04:53:17.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.395+0000 7fb0cfe00640 1 -- 172.21.15.123:0/1789783566 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0e8011e90 con 0x55cb2c7d6760 2024-09-19T04:53:17.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.395+0000 7fb0cfe00640 1 -- 172.21.15.123:0/1789783566 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fb0dc031b40 con 0x55cb2c7d6760 2024-09-19T04:53:17.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.395+0000 7fb0f7839740 10 client.26557 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:17.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.395+0000 7fb0f7839740 10 client.26557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:53:17.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.395+0000 7fb0f7839740 20 client.26557 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:17.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.395+0000 7fb0f7839740 1 --2- 172.21.15.123:0/1789783566 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55cb2c7f10c0 0x55cb2c7f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.395+0000 7fb0f7839740 4 client.26557 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:53:17.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.395+0000 7fb0f7839740 1 -- 172.21.15.123:0/1789783566 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb2c749950 con 0x55cb2c7f10c0 2024-09-19T04:53:17.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.396+0000 7fb0ee800640 1 --2- 172.21.15.123:0/1789783566 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55cb2c7f10c0 0x55cb2c7f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.396+0000 7fb0ee800640 1 --2- 172.21.15.123:0/1789783566 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55cb2c7f10c0 0x55cb2c7f34e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd3acce409db5952 server_cookie=b98d4c4a70e4f88e in_seq=0 out_seq=0 2024-09-19T04:53:17.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.396+0000 7fb0cfe00640 10 client.26557 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:17.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.397+0000 7fb0cfe00640 1 -- 172.21.15.123:0/1789783566 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cb2c749950 con 0x55cb2c7f10c0 2024-09-19T04:53:17.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.397+0000 7fb0cfe00640 10 client.26557 handle_command_reply: tid=0 2024-09-19T04:53:17.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.397+0000 7fb0f7839740 1 client.26557 shutdown 2024-09-19T04:53:17.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.397+0000 7fb0f7839740 1 -- 172.21.15.123:0/1789783566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d6760 msgr2=0x55cb2c7de480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:17.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.397+0000 7fb0f7839740 1 --2- 172.21.15.123:0/1789783566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d6760 0x55cb2c7de480 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc033000 tx=0x7fb0dc004420 comp rx=0 tx=0).stop 2024-09-19T04:53:17.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.397+0000 7fb0f7839740 1 -- 172.21.15.123:0/1789783566 shutdown_connections 2024-09-19T04:53:17.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.397+0000 7fb0f7839740 1 --2- 172.21.15.123:0/1789783566 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55cb2c7f10c0 0x55cb2c7f34e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.397+0000 7fb0f7839740 1 --2- 172.21.15.123:0/1789783566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 0x55cb2c7de9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.397+0000 7fb0f7839740 1 --2- 172.21.15.123:0/1789783566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d6760 0x55cb2c7de480 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.397+0000 7fb0cfe00640 0 client.26557 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:17.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.397+0000 7fb0f7839740 1 --2- 172.21.15.123:0/1789783566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7ddf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.397+0000 7fb0f7839740 1 -- 172.21.15.123:0/1789783566 >> 172.21.15.123:0/1789783566 conn(0x55cb2c7d23d0 msgr2=0x55cb2c7d8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:17.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.397+0000 7fb0f7839740 1 -- 172.21.15.123:0/1789783566 shutdown_connections 2024-09-19T04:53:17.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.397+0000 7fb0f7839740 1 -- 172.21.15.123:0/1789783566 wait complete. 2024-09-19T04:53:17.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.397+0000 7fb0f7839740 20 client.26557 trim_cache size 0 max 16384 2024-09-19T04:53:17.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.411+0000 7fb0f7839740 1 Processor -- start 2024-09-19T04:53:17.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.411+0000 7fb0f7839740 1 -- start start 2024-09-19T04:53:17.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.411+0000 7fb0f7839740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.411+0000 7fb0f7839740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d6760 0x55cb2c7f92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.411+0000 7fb0f7839740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 0x55cb2c7ddf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.411+0000 7fb0f7839740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f01b3df0 con 0x55cb2c7e1020 2024-09-19T04:53:17.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.411+0000 7fb0f7839740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f01b59c0 con 0x55cb2c7d5e40 2024-09-19T04:53:17.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.411+0000 7fb0f7839740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f01b44f0 con 0x55cb2c7d6760 2024-09-19T04:53:17.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.412+0000 7fb0ee800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 0x55cb2c7ddf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.412+0000 7fb0ee800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 0x55cb2c7ddf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36760/0 (socket says 172.21.15.123:36760) 2024-09-19T04:53:17.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.412+0000 7fb0ee800640 1 -- 172.21.15.123:0/1053882537 learned_addr learned my addr 172.21.15.123:0/1053882537 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:17.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.412+0000 7fb0ed400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d6760 0x55cb2c7f92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.412+0000 7fb0ee800640 1 --2- 172.21.15.123:0/1053882537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.412+0000 7fb0ed400640 1 -- 172.21.15.123:0/1053882537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d5e40 msgr2=0x55cb2c7f8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:17.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.412+0000 7fb0ed400640 1 --2- 172.21.15.123:0/1053882537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7f8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.412+0000 7fb0ed400640 1 -- 172.21.15.123:0/1053882537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 msgr2=0x55cb2c7ddf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:17.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.412+0000 7fb0ed400640 1 --2- 172.21.15.123:0/1053882537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 0x55cb2c7ddf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.412+0000 7fb0ed400640 1 -- 172.21.15.123:0/1053882537 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb2c7d1e70 con 0x55cb2c7d6760 2024-09-19T04:53:17.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.412+0000 7fb0ee800640 1 --2- 172.21.15.123:0/1053882537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 0x55cb2c7ddf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:17.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.412+0000 7fb0ed400640 1 --2- 172.21.15.123:0/1053882537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d6760 0x55cb2c7f92f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc000c00 tx=0x7fb0dc005570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=648b9bc0010ff4e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:17.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.413+0000 7fb0daa00640 1 -- 172.21.15.123:0/1053882537 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc035070 con 0x55cb2c7d6760 2024-09-19T04:53:17.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.413+0000 7fb0daa00640 1 -- 172.21.15.123:0/1053882537 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0dc041070 con 0x55cb2c7d6760 2024-09-19T04:53:17.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.413+0000 7fb0daa00640 1 -- 172.21.15.123:0/1053882537 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc053070 con 0x55cb2c7d6760 2024-09-19T04:53:17.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.413+0000 7fb0f7839740 1 -- 172.21.15.123:0/1053882537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d6760 msgr2=0x55cb2c7f92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:17.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.413+0000 7fb0f7839740 1 --2- 172.21.15.123:0/1053882537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d6760 0x55cb2c7f92f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc000c00 tx=0x7fb0dc005570 comp rx=0 tx=0).stop 2024-09-19T04:53:17.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.413+0000 7fb0f7839740 1 -- 172.21.15.123:0/1053882537 shutdown_connections 2024-09-19T04:53:17.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.413+0000 7fb0f7839740 1 --2- 172.21.15.123:0/1053882537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 0x55cb2c7ddf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.413+0000 7fb0f7839740 1 --2- 172.21.15.123:0/1053882537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d6760 0x55cb2c7f92f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.413+0000 7fb0f7839740 1 --2- 172.21.15.123:0/1053882537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7f8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.413+0000 7fb0f7839740 1 -- 172.21.15.123:0/1053882537 >> 172.21.15.123:0/1053882537 conn(0x55cb2c7d23d0 msgr2=0x55cb2c7dd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.414+0000 7fb0f7839740 1 -- 172.21.15.123:0/1053882537 shutdown_connections 2024-09-19T04:53:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.414+0000 7fb0f7839740 1 -- 172.21.15.123:0/1053882537 wait complete. 2024-09-19T04:53:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.414+0000 7fb0f7839740 1 -- start start 2024-09-19T04:53:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.414+0000 7fb0f7839740 1 Processor -- start 2024-09-19T04:53:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0f7839740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7e9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0f7839740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d6760 0x55cb2c7e47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0f7839740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 0x55cb2c7e4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0f7839740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f01b52f0 con 0x55cb2c7e1020 2024-09-19T04:53:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0f7839740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f01b4bf0 con 0x55cb2c7d6760 2024-09-19T04:53:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0f7839740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f01b1df0 con 0x55cb2c7d5e40 2024-09-19T04:53:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0f7839740 10 client.0 _ll_register_callbacks cb 0x7fb0dc009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0ee800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 0x55cb2c7e4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0ed400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d6760 0x55cb2c7e47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0ee800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 0x55cb2c7e4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36770/0 (socket says 172.21.15.123:36770) 2024-09-19T04:53:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0ee800640 1 -- 172.21.15.123:0/2416254396 learned_addr learned my addr 172.21.15.123:0/2416254396 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0ee800640 1 --2- 172.21.15.123:0/2416254396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7e9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0ed400640 1 -- 172.21.15.123:0/2416254396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d5e40 msgr2=0x55cb2c7e9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0ed400640 1 --2- 172.21.15.123:0/2416254396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7e9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0ed400640 1 -- 172.21.15.123:0/2416254396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 msgr2=0x55cb2c7e4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0ed400640 1 --2- 172.21.15.123:0/2416254396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 0x55cb2c7e4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0ed400640 1 -- 172.21.15.123:0/2416254396 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb2c7e5240 con 0x55cb2c7d6760 2024-09-19T04:53:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.415+0000 7fb0ee800640 1 --2- 172.21.15.123:0/2416254396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 0x55cb2c7e4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.416+0000 7fb0ed400640 1 --2- 172.21.15.123:0/2416254396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d6760 0x55cb2c7e47c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009850 tx=0x7fb0dc031480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32483d6b80507a73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.416+0000 7fb0cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:53:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.416+0000 7fb0cfe00640 1 -- 172.21.15.123:0/2416254396 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc05b020 con 0x55cb2c7d6760 2024-09-19T04:53:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.416+0000 7fb0f7839740 1 -- 172.21.15.123:0/2416254396 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0dc041070 con 0x55cb2c7d6760 2024-09-19T04:53:17.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.416+0000 7fb0cfe00640 1 -- 172.21.15.123:0/2416254396 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb0dc0048c0 con 0x55cb2c7d6760 2024-09-19T04:53:17.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.416+0000 7fb0cfe00640 1 -- 172.21.15.123:0/2416254396 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb0dc066050 con 0x55cb2c7d6760 2024-09-19T04:53:17.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.416+0000 7fb0f7839740 10 client.26543 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:17.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.416+0000 7fb0f7839740 1 -- 172.21.15.123:0/2416254396 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55cb2c85bab0 con 0x55cb2c7d6760 2024-09-19T04:53:17.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.417+0000 7fb0cfe00640 1 -- 172.21.15.123:0/2416254396 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0dc041070 con 0x55cb2c7d6760 2024-09-19T04:53:17.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.417+0000 7fb0cfe00640 1 -- 172.21.15.123:0/2416254396 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fb0dc046470 con 0x55cb2c7d6760 2024-09-19T04:53:17.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.417+0000 7fb0f7839740 10 client.26543 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:17.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.417+0000 7fb0f7839740 10 client.26543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:53:17.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.417+0000 7fb0f7839740 20 client.26543 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:17.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.417+0000 7fb0f7839740 1 --2- 172.21.15.123:0/2416254396 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55cb2c85c140 0x55cb2c85e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:17.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.417+0000 7fb0f7839740 4 client.26543 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:53:17.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.417+0000 7fb0f7839740 1 -- 172.21.15.123:0/2416254396 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb2c743e60 con 0x55cb2c85c140 2024-09-19T04:53:17.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.417+0000 7fb0ee800640 1 --2- 172.21.15.123:0/2416254396 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55cb2c85c140 0x55cb2c85e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:17.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.418+0000 7fb0ee800640 1 --2- 172.21.15.123:0/2416254396 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55cb2c85c140 0x55cb2c85e560 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c8ab34139d2b45d server_cookie=932e7ffe56f29d95 in_seq=0 out_seq=0 2024-09-19T04:53:17.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.418+0000 7fb0cfe00640 10 client.26543 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:17.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.418+0000 7fb0cfe00640 1 -- 172.21.15.123:0/2416254396 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb2c743e60 con 0x55cb2c85c140 2024-09-19T04:53:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.418+0000 7fb0cfe00640 10 client.26543 handle_command_reply: tid=0 2024-09-19T04:53:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.418+0000 7fb0f7839740 1 client.26543 shutdown 2024-09-19T04:53:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.418+0000 7fb0f7839740 1 -- 172.21.15.123:0/2416254396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d6760 msgr2=0x55cb2c7e47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.418+0000 7fb0f7839740 1 --2- 172.21.15.123:0/2416254396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d6760 0x55cb2c7e47c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009850 tx=0x7fb0dc031480 comp rx=0 tx=0).stop 2024-09-19T04:53:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.418+0000 7fb0f7839740 1 -- 172.21.15.123:0/2416254396 shutdown_connections 2024-09-19T04:53:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.418+0000 7fb0f7839740 1 --2- 172.21.15.123:0/2416254396 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55cb2c85c140 0x55cb2c85e560 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.418+0000 7fb0f7839740 1 --2- 172.21.15.123:0/2416254396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb2c7e1020 0x55cb2c7e4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.418+0000 7fb0f7839740 1 --2- 172.21.15.123:0/2416254396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb2c7d6760 0x55cb2c7e47c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.418+0000 7fb0f7839740 1 --2- 172.21.15.123:0/2416254396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb2c7d5e40 0x55cb2c7e9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.418+0000 7fb0cfe00640 0 client.26543 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.418+0000 7fb0f7839740 1 -- 172.21.15.123:0/2416254396 >> 172.21.15.123:0/2416254396 conn(0x55cb2c7d23d0 msgr2=0x55cb2c7ddf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.419+0000 7fb0f7839740 1 -- 172.21.15.123:0/2416254396 shutdown_connections 2024-09-19T04:53:17.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.419+0000 7fb0f7839740 1 -- 172.21.15.123:0/2416254396 wait complete. 2024-09-19T04:53:17.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.419+0000 7fb0f7839740 20 client.26543 trim_cache size 0 max 16384 2024-09-19T04:53:17.425 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:53:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.419+0000 7fb0ef200640 1 -- 172.21.15.123:0/921585769 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb0d40a33b0 msgr2=0x7fb0d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.419+0000 7fb0ef200640 1 --2- 172.21.15.123:0/921585769 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb0d40a33b0 0x7fb0d40a5870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55cb2c7d3230 tx=0x7fb0e0009700 comp rx=0 tx=0).stop 2024-09-19T04:53:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.419+0000 7fb0ef200640 1 -- 172.21.15.123:0/921585769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f0076720 msgr2=0x7fb0f01a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.419+0000 7fb0ef200640 1 --2- 172.21.15.123:0/921585769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f0076720 0x7fb0f01a4680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000e9f0 tx=0x7fb0e000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.420+0000 7fb0ef200640 1 -- 172.21.15.123:0/921585769 shutdown_connections 2024-09-19T04:53:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.420+0000 7fb0ef200640 1 --2- 172.21.15.123:0/921585769 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb0d40a33b0 0x7fb0d40a5870 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.420+0000 7fb0ef200640 1 --2- 172.21.15.123:0/921585769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0f0112870 0x7fb0f019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.420+0000 7fb0ef200640 1 --2- 172.21.15.123:0/921585769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0f0077040 0x7fb0f01a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.420+0000 7fb0ef200640 1 --2- 172.21.15.123:0/921585769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f0076720 0x7fb0f01a4680 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.420+0000 7fb0ef200640 1 -- 172.21.15.123:0/921585769 >> 172.21.15.123:0/921585769 conn(0x7fb0f00ff8d0 msgr2=0x7fb0f0101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.420+0000 7fb0ef200640 1 -- 172.21.15.123:0/921585769 shutdown_connections 2024-09-19T04:53:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:17.420+0000 7fb0ef200640 1 -- 172.21.15.123:0/921585769 wait complete. 2024-09-19T04:53:17.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:17 smithi183 ceph-mon[35333]: pgmap v201: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:53:17.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:17 smithi183 ceph-mon[35333]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T04:53:17.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:17 smithi123 ceph-mon[30511]: pgmap v201: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:53:17.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:17 smithi123 ceph-mon[30511]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T04:53:17.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:17 smithi193 ceph-mon[35592]: pgmap v201: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:53:17.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:17 smithi193 ceph-mon[35592]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T04:53:19.161 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:53:19.161 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:53:19.537 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:53:19.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:19 smithi183 ceph-mon[35333]: pgmap v203: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 237 B/s wr, 12 op/s 2024-09-19T04:53:19.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:19 smithi123 ceph-mon[30511]: pgmap v203: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 237 B/s wr, 12 op/s 2024-09-19T04:53:19.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:19 smithi193 ceph-mon[35592]: pgmap v203: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 237 B/s wr, 12 op/s 2024-09-19T04:53:20.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.087+0000 7f7c38e00640 1 -- 172.21.15.123:0/1000226755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c340745c0 msgr2=0x7f7c340749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:20.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.087+0000 7f7c38e00640 1 --2- 172.21.15.123:0/1000226755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c340745c0 0x7f7c340749a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009ca0 tx=0x7f7c2402a950 comp rx=0 tx=0).stop 2024-09-19T04:53:20.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.087+0000 7f7c38e00640 1 -- 172.21.15.123:0/1000226755 shutdown_connections 2024-09-19T04:53:20.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.087+0000 7f7c38e00640 1 --2- 172.21.15.123:0/1000226755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c3410e420 0x7f7c34110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.087+0000 7f7c38e00640 1 --2- 172.21.15.123:0/1000226755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c34074ee0 0x7f7c3410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.087+0000 7f7c38e00640 1 --2- 172.21.15.123:0/1000226755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c340745c0 0x7f7c340749a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.087+0000 7f7c38e00640 1 -- 172.21.15.123:0/1000226755 >> 172.21.15.123:0/1000226755 conn(0x7f7c340fd8c0 msgr2=0x7f7c340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:20.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.087+0000 7f7c38e00640 1 -- 172.21.15.123:0/1000226755 shutdown_connections 2024-09-19T04:53:20.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.087+0000 7f7c38e00640 1 -- 172.21.15.123:0/1000226755 wait complete. 2024-09-19T04:53:20.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.088+0000 7f7c38e00640 1 Processor -- start 2024-09-19T04:53:20.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.088+0000 7f7c38e00640 1 -- start start 2024-09-19T04:53:20.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.088+0000 7f7c38e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c340745c0 0x7f7c3419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.088+0000 7f7c38e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c34074ee0 0x7f7c3419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.088+0000 7f7c38e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c3410e420 0x7f7c3419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.088+0000 7f7c38e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341141e0 con 0x7f7c3410e420 2024-09-19T04:53:20.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.088+0000 7f7c38e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34114060 con 0x7f7c340745c0 2024-09-19T04:53:20.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.088+0000 7f7c38e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34114360 con 0x7f7c34074ee0 2024-09-19T04:53:20.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.088+0000 7f7c32800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c3410e420 0x7f7c3419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.088+0000 7f7c31e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c340745c0 0x7f7c3419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.088+0000 7f7c31e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c340745c0 0x7f7c3419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:33934/0 (socket says 172.21.15.123:33934) 2024-09-19T04:53:20.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.089+0000 7f7c31e00640 1 -- 172.21.15.123:0/4083210495 learned_addr learned my addr 172.21.15.123:0/4083210495 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:20.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.088+0000 7f7c32800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c3410e420 0x7f7c3419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36796/0 (socket says 172.21.15.123:36796) 2024-09-19T04:53:20.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.089+0000 7f7c31400640 1 --2- 172.21.15.123:0/4083210495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c34074ee0 0x7f7c3419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.089+0000 7f7c31e00640 1 -- 172.21.15.123:0/4083210495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c34074ee0 msgr2=0x7f7c3419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:20.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.089+0000 7f7c31e00640 1 --2- 172.21.15.123:0/4083210495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c34074ee0 0x7f7c3419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.089+0000 7f7c31e00640 1 -- 172.21.15.123:0/4083210495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c3410e420 msgr2=0x7f7c3419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:20.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.089+0000 7f7c31e00640 1 --2- 172.21.15.123:0/4083210495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c3410e420 0x7f7c3419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.089+0000 7f7c31e00640 1 -- 172.21.15.123:0/4083210495 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3419f670 con 0x7f7c340745c0 2024-09-19T04:53:20.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.089+0000 7f7c31e00640 1 --2- 172.21.15.123:0/4083210495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c340745c0 0x7f7c3419a6c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7c24005e30 tx=0x7f7c24003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:20.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.090+0000 7f7c2aa00640 1 -- 172.21.15.123:0/4083210495 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c24034040 con 0x7f7c340745c0 2024-09-19T04:53:20.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.090+0000 7f7c38e00640 1 -- 172.21.15.123:0/4083210495 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3419f900 con 0x7f7c340745c0 2024-09-19T04:53:20.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.090+0000 7f7c38e00640 1 -- 172.21.15.123:0/4083210495 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c341a7240 con 0x7f7c340745c0 2024-09-19T04:53:20.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.090+0000 7f7c2aa00640 1 -- 172.21.15.123:0/4083210495 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c2402fe20 con 0x7f7c340745c0 2024-09-19T04:53:20.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.090+0000 7f7c2aa00640 1 -- 172.21.15.123:0/4083210495 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c24004830 con 0x7f7c340745c0 2024-09-19T04:53:20.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.093+0000 7f7c3ba3d740 1 Processor -- start 2024-09-19T04:53:20.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.093+0000 7f7c3ba3d740 1 -- start start 2024-09-19T04:53:20.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.093+0000 7f7c2aa00640 1 -- 172.21.15.123:0/4083210495 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f7c24037020 con 0x7f7c340745c0 2024-09-19T04:53:20.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.093+0000 7f7c3ba3d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef5ac0 0x562dcaef5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.093+0000 7f7c3ba3d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef63e0 0x562dcaf00760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.093+0000 7f7c3ba3d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaf00ca0 0x562dcaf03090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.093+0000 7f7c3ba3d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562dcae5c290 con 0x562dcaef5ac0 2024-09-19T04:53:20.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.093+0000 7f7c3ba3d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562dcae03b40 con 0x562dcaef63e0 2024-09-19T04:53:20.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.093+0000 7f7c3ba3d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562dcae58250 con 0x562dcaf00ca0 2024-09-19T04:53:20.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.094+0000 7f7c32800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef5ac0 0x562dcaef5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.094+0000 7f7c32800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef5ac0 0x562dcaef5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36810/0 (socket says 172.21.15.123:36810) 2024-09-19T04:53:20.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.094+0000 7f7c32800640 1 -- 172.21.15.123:0/477056644 learned_addr learned my addr 172.21.15.123:0/477056644 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:20.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.094+0000 7f7c31400640 1 --2- 172.21.15.123:0/477056644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaf00ca0 0x562dcaf03090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.094+0000 7f7c31e00640 1 --2- 172.21.15.123:0/477056644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef63e0 0x562dcaf00760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.094+0000 7f7c32800640 1 -- 172.21.15.123:0/477056644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaf00ca0 msgr2=0x562dcaf03090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:20.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.094+0000 7f7c32800640 1 --2- 172.21.15.123:0/477056644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaf00ca0 0x562dcaf03090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.094+0000 7f7c32800640 1 -- 172.21.15.123:0/477056644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef63e0 msgr2=0x562dcaf00760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:20.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.094+0000 7f7c32800640 1 --2- 172.21.15.123:0/477056644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef63e0 0x562dcaf00760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.094+0000 7f7c32800640 1 -- 172.21.15.123:0/477056644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dcabcd6a0 con 0x562dcaef5ac0 2024-09-19T04:53:20.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.094+0000 7f7c31e00640 1 --2- 172.21.15.123:0/477056644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef63e0 0x562dcaf00760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:53:20.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.094+0000 7f7c32800640 1 --2- 172.21.15.123:0/477056644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef5ac0 0x562dcaef5ea0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0045e0 tx=0x7f7c2c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324c43fb10d5ce72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:20.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.095+0000 7f7c1f400640 1 -- 172.21.15.123:0/477056644 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c003af0 con 0x562dcaef5ac0 2024-09-19T04:53:20.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.095+0000 7f7c1f400640 1 -- 172.21.15.123:0/477056644 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c2c015070 con 0x562dcaef5ac0 2024-09-19T04:53:20.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.095+0000 7f7c1f400640 1 -- 172.21.15.123:0/477056644 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c0116a0 con 0x562dcaef5ac0 2024-09-19T04:53:20.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.095+0000 7f7c2aa00640 1 --2- 172.21.15.123:0/4083210495 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7c180a33b0 0x7f7c180a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.095+0000 7f7c2aa00640 1 -- 172.21.15.123:0/4083210495 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f7c240f0ae0 con 0x7f7c340745c0 2024-09-19T04:53:20.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.095+0000 7f7c31e00640 1 --2- 172.21.15.123:0/4083210495 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7c180a33b0 0x7f7c180a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.095+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/477056644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef5ac0 msgr2=0x562dcaef5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:20.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.095+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/477056644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef5ac0 0x562dcaef5ea0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0045e0 tx=0x7f7c2c004610 comp rx=0 tx=0).stop 2024-09-19T04:53:20.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.096+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/477056644 shutdown_connections 2024-09-19T04:53:20.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.096+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/477056644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaf00ca0 0x562dcaf03090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.096+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/477056644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef63e0 0x562dcaf00760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.096+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/477056644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef5ac0 0x562dcaef5ea0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.096+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/477056644 >> 172.21.15.123:0/477056644 conn(0x562dcaef2050 msgr2=0x562dcaef2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:20.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.096+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/477056644 shutdown_connections 2024-09-19T04:53:20.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.096+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/477056644 wait complete. 2024-09-19T04:53:20.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.096+0000 7f7c31e00640 1 --2- 172.21.15.123:0/4083210495 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7c180a33b0 0x7f7c180a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x562dcaef2eb0 tx=0x7f7c240095c0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:20.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.096+0000 7f7c3ba3d740 1 -- start start 2024-09-19T04:53:20.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.097+0000 7f7c3ba3d740 1 Processor -- start 2024-09-19T04:53:20.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.097+0000 7f7c3ba3d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef5ac0 0x562dcaf00600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.097+0000 7f7c3ba3d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaefd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.097+0000 7f7c3ba3d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaf00ca0 0x562dcaefde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.097+0000 7f7c3ba3d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562dcaf03910 con 0x562dcaef63e0 2024-09-19T04:53:20.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.097+0000 7f7c3ba3d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562dcae16fa0 con 0x562dcaef5ac0 2024-09-19T04:53:20.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.097+0000 7f7c3ba3d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562dcaf03a90 con 0x562dcaf00ca0 2024-09-19T04:53:20.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.097+0000 7f7c3ba3d740 10 client.0 _ll_register_callbacks cb 0x562dcab6aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:20.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.097+0000 7f7c31400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaefd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.097+0000 7f7c31400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaefd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36824/0 (socket says 172.21.15.123:36824) 2024-09-19T04:53:20.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.097+0000 7f7c32800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef5ac0 0x562dcaf00600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.097+0000 7f7c31400640 1 -- 172.21.15.123:0/1323766380 learned_addr learned my addr 172.21.15.123:0/1323766380 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:20.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.097+0000 7f7c32800640 1 --2- 172.21.15.123:0/1323766380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaf00ca0 0x562dcaefde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.098+0000 7f7c31400640 1 -- 172.21.15.123:0/1323766380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaf00ca0 msgr2=0x562dcaefde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:20.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.098+0000 7f7c31400640 1 --2- 172.21.15.123:0/1323766380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaf00ca0 0x562dcaefde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.098+0000 7f7c31400640 1 -- 172.21.15.123:0/1323766380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef5ac0 msgr2=0x562dcaf00600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:20.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.098+0000 7f7c31400640 1 --2- 172.21.15.123:0/1323766380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef5ac0 0x562dcaf00600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.098+0000 7f7c31400640 1 -- 172.21.15.123:0/1323766380 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dcaefe640 con 0x562dcaef63e0 2024-09-19T04:53:20.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.098+0000 7f7c32800640 1 --2- 172.21.15.123:0/1323766380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef5ac0 0x562dcaf00600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:20.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.098+0000 7f7c31400640 1 --2- 172.21.15.123:0/1323766380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaefd8b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7c20012700 tx=0x7f7c20012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173760600c535537 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:20.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.098+0000 7f7c13e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:53:20.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.098+0000 7f7c13e00640 1 -- 172.21.15.123:0/1323766380 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20004050 con 0x562dcaef63e0 2024-09-19T04:53:20.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.098+0000 7f7c13e00640 1 -- 172.21.15.123:0/1323766380 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f7c200041f0 con 0x562dcaef63e0 2024-09-19T04:53:20.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.098+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/1323766380 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c2c015070 con 0x562dcaef63e0 2024-09-19T04:53:20.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.099+0000 7f7c13e00640 1 -- 172.21.15.123:0/1323766380 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7c200043f0 con 0x562dcaef63e0 2024-09-19T04:53:20.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.099+0000 7f7c3ba3d740 10 client.16455 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:20.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.099+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/1323766380 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562dcaf0f910 con 0x562dcaef63e0 2024-09-19T04:53:20.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.099+0000 7f7c13e00640 1 -- 172.21.15.123:0/1323766380 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c2c015070 con 0x562dcaef63e0 2024-09-19T04:53:20.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.099+0000 7f7c13e00640 1 -- 172.21.15.123:0/1323766380 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f7c20003bc0 con 0x562dcaef63e0 2024-09-19T04:53:20.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.099+0000 7f7c3ba3d740 10 client.16455 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:20.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.099+0000 7f7c3ba3d740 10 client.16455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:53:20.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.099+0000 7f7c3ba3d740 20 client.16455 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:20.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.099+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/1323766380 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562dcaf10d70 0x562dcaf13190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.099+0000 7f7c3ba3d740 4 client.16455 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:53:20.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.099+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/1323766380 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dcae69800 con 0x562dcaf10d70 2024-09-19T04:53:20.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.100+0000 7f7c32800640 1 --2- 172.21.15.123:0/1323766380 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562dcaf10d70 0x562dcaf13190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.100+0000 7f7c32800640 1 --2- 172.21.15.123:0/1323766380 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562dcaf10d70 0x562dcaf13190 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16f49c11ce92ac67 server_cookie=710fecebd4d8fc70 in_seq=0 out_seq=0 2024-09-19T04:53:20.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.100+0000 7f7c13e00640 10 client.16455 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:20.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.101+0000 7f7c13e00640 1 -- 172.21.15.123:0/1323766380 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562dcae69800 con 0x562dcaf10d70 2024-09-19T04:53:20.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.101+0000 7f7c13e00640 10 client.16455 handle_command_reply: tid=0 2024-09-19T04:53:20.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.101+0000 7f7c3ba3d740 1 client.16455 shutdown 2024-09-19T04:53:20.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.101+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/1323766380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 msgr2=0x562dcaefd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:20.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.101+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/1323766380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaefd8b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7c20012700 tx=0x7f7c20012bc0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.101+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/1323766380 shutdown_connections 2024-09-19T04:53:20.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.101+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/1323766380 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562dcaf10d70 0x562dcaf13190 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.101+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/1323766380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaf00ca0 0x562dcaefde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.101+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/1323766380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaefd8b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.101+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/1323766380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef5ac0 0x562dcaf00600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.101+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/1323766380 >> 172.21.15.123:0/1323766380 conn(0x562dcaef2050 msgr2=0x562dcaef7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:20.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.102+0000 7f7c13e00640 0 client.16455 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:20.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.102+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/1323766380 shutdown_connections 2024-09-19T04:53:20.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.102+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/1323766380 wait complete. 2024-09-19T04:53:20.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.102+0000 7f7c3ba3d740 20 client.16455 trim_cache size 0 max 16384 2024-09-19T04:53:20.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.119+0000 7f7c3ba3d740 1 Processor -- start 2024-09-19T04:53:20.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.120+0000 7f7c3ba3d740 1 -- start start 2024-09-19T04:53:20.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.120+0000 7f7c3ba3d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaef5ac0 0x562dcaefc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.120+0000 7f7c3ba3d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaeff820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.120+0000 7f7c3ba3d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaf00ca0 0x562dcaf18a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.120+0000 7f7c3ba3d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341afac0 con 0x562dcaef63e0 2024-09-19T04:53:20.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.120+0000 7f7c3ba3d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341b1690 con 0x562dcaf00ca0 2024-09-19T04:53:20.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.120+0000 7f7c3ba3d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341b01c0 con 0x562dcaef5ac0 2024-09-19T04:53:20.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.120+0000 7f7c31400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaeff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.120+0000 7f7c31400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaeff820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36836/0 (socket says 172.21.15.123:36836) 2024-09-19T04:53:20.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.120+0000 7f7c32800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaef5ac0 0x562dcaefc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.120+0000 7f7c31400640 1 -- 172.21.15.123:0/1153673004 learned_addr learned my addr 172.21.15.123:0/1153673004 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:20.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.120+0000 7f7c32800640 1 --2- 172.21.15.123:0/1153673004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaf00ca0 0x562dcaf18a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.121+0000 7f7c31400640 1 -- 172.21.15.123:0/1153673004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaef5ac0 msgr2=0x562dcaefc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:20.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.121+0000 7f7c31400640 1 --2- 172.21.15.123:0/1153673004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaef5ac0 0x562dcaefc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.121+0000 7f7c31400640 1 -- 172.21.15.123:0/1153673004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaf00ca0 msgr2=0x562dcaf18a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:20.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.121+0000 7f7c31400640 1 --2- 172.21.15.123:0/1153673004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaf00ca0 0x562dcaf18a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.121+0000 7f7c31400640 1 -- 172.21.15.123:0/1153673004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dcaefbe00 con 0x562dcaef63e0 2024-09-19T04:53:20.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.121+0000 7f7c32800640 1 --2- 172.21.15.123:0/1153673004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaef5ac0 0x562dcaefc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:20.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.121+0000 7f7c31400640 1 --2- 172.21.15.123:0/1153673004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaeff820 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7c20000c00 tx=0x7f7c20025900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20add3bf659a9707 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:20.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.121+0000 7f7c1ea00640 1 -- 172.21.15.123:0/1153673004 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20003fe0 con 0x562dcaef63e0 2024-09-19T04:53:20.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.121+0000 7f7c1ea00640 1 -- 172.21.15.123:0/1153673004 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c20025e50 con 0x562dcaef63e0 2024-09-19T04:53:20.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.121+0000 7f7c1ea00640 1 -- 172.21.15.123:0/1153673004 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2000fde0 con 0x562dcaef63e0 2024-09-19T04:53:20.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.122+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/1153673004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 msgr2=0x562dcaeff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:20.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.122+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/1153673004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaeff820 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7c20000c00 tx=0x7f7c20025900 comp rx=0 tx=0).stop 2024-09-19T04:53:20.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.122+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/1153673004 shutdown_connections 2024-09-19T04:53:20.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.122+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/1153673004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaf00ca0 0x562dcaf18a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.122+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/1153673004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaeff820 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.122+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/1153673004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaef5ac0 0x562dcaefc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.122+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/1153673004 >> 172.21.15.123:0/1153673004 conn(0x562dcaef2050 msgr2=0x562dcaeff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:20.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.122+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/1153673004 shutdown_connections 2024-09-19T04:53:20.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.122+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/1153673004 wait complete. 2024-09-19T04:53:20.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.123+0000 7f7c3ba3d740 1 -- start start 2024-09-19T04:53:20.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.123+0000 7f7c3ba3d740 1 Processor -- start 2024-09-19T04:53:20.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.123+0000 7f7c3ba3d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef5ac0 0x562dcaf093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.123+0000 7f7c3ba3d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaf06770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.123+0000 7f7c3ba3d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaf00ca0 0x562dcaf06cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.123+0000 7f7c3ba3d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341b0fc0 con 0x562dcaef63e0 2024-09-19T04:53:20.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.123+0000 7f7c3ba3d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341b08c0 con 0x562dcaef5ac0 2024-09-19T04:53:20.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.123+0000 7f7c3ba3d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341adac0 con 0x562dcaf00ca0 2024-09-19T04:53:20.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.123+0000 7f7c3ba3d740 10 client.0 _ll_register_callbacks cb 0x562dcaefe810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:20.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.123+0000 7f7c31400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaf06770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.123+0000 7f7c32800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef5ac0 0x562dcaf093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.123+0000 7f7c31400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaf06770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36844/0 (socket says 172.21.15.123:36844) 2024-09-19T04:53:20.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.123+0000 7f7c32800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef5ac0 0x562dcaf093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:33954/0 (socket says 172.21.15.123:33954) 2024-09-19T04:53:20.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.123+0000 7f7c31400640 1 -- 172.21.15.123:0/299630961 learned_addr learned my addr 172.21.15.123:0/299630961 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:20.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.124+0000 7f7c32800640 1 --2- 172.21.15.123:0/299630961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaf00ca0 0x562dcaf06cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.124+0000 7f7c32800640 1 -- 172.21.15.123:0/299630961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaf00ca0 msgr2=0x562dcaf06cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:20.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.124+0000 7f7c32800640 1 --2- 172.21.15.123:0/299630961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaf00ca0 0x562dcaf06cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.124+0000 7f7c32800640 1 -- 172.21.15.123:0/299630961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 msgr2=0x562dcaf06770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:20.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.124+0000 7f7c32800640 1 --2- 172.21.15.123:0/299630961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaf06770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.124+0000 7f7c32800640 1 -- 172.21.15.123:0/299630961 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dcaf07490 con 0x562dcaef5ac0 2024-09-19T04:53:20.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.124+0000 7f7c31400640 1 --2- 172.21.15.123:0/299630961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaf06770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:20.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.124+0000 7f7c32800640 1 --2- 172.21.15.123:0/299630961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef5ac0 0x562dcaf093d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c01ae30 tx=0x7f7c20009d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dae9062685a9142 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:20.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.124+0000 7f7c13e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:53:20.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.124+0000 7f7c13e00640 1 -- 172.21.15.123:0/299630961 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c023040 con 0x562dcaef5ac0 2024-09-19T04:53:20.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.124+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/299630961 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c20025e50 con 0x562dcaef5ac0 2024-09-19T04:53:20.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.124+0000 7f7c13e00640 1 -- 172.21.15.123:0/299630961 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f7c2c003d00 con 0x562dcaef5ac0 2024-09-19T04:53:20.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.125+0000 7f7c13e00640 1 -- 172.21.15.123:0/299630961 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7c2c005030 con 0x562dcaef5ac0 2024-09-19T04:53:20.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.125+0000 7f7c3ba3d740 10 client.26561 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.125+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/299630961 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562dcaf7b9d0 con 0x562dcaef5ac0 2024-09-19T04:53:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.125+0000 7f7c13e00640 1 -- 172.21.15.123:0/299630961 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c20025e50 con 0x562dcaef5ac0 2024-09-19T04:53:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.125+0000 7f7c13e00640 1 -- 172.21.15.123:0/299630961 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f7c2c004cc0 con 0x562dcaef5ac0 2024-09-19T04:53:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.125+0000 7f7c3ba3d740 10 client.26561 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.125+0000 7f7c3ba3d740 10 client.26561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:53:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.125+0000 7f7c3ba3d740 20 client.26561 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:20.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.125+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/299630961 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562dcaf7be60 0x562dcaf7e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.125+0000 7f7c3ba3d740 4 client.26561 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:53:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.125+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/299630961 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562dcae63d10 con 0x562dcaf7be60 2024-09-19T04:53:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.125+0000 7f7c31400640 1 --2- 172.21.15.123:0/299630961 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562dcaf7be60 0x562dcaf7e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.126+0000 7f7c31400640 1 --2- 172.21.15.123:0/299630961 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562dcaf7be60 0x562dcaf7e250 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d91c4a723ea5fa server_cookie=6d9bc34c63f8524a in_seq=0 out_seq=0 2024-09-19T04:53:20.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.126+0000 7f7c13e00640 10 client.26561 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:20.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.127+0000 7f7c13e00640 1 -- 172.21.15.123:0/299630961 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562dcae63d10 con 0x562dcaf7be60 2024-09-19T04:53:20.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.127+0000 7f7c13e00640 10 client.26561 handle_command_reply: tid=0 2024-09-19T04:53:20.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.127+0000 7f7c3ba3d740 1 client.26561 shutdown 2024-09-19T04:53:20.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.127+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/299630961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef5ac0 msgr2=0x562dcaf093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:20.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.127+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/299630961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef5ac0 0x562dcaf093d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c01ae30 tx=0x7f7c20009d20 comp rx=0 tx=0).stop 2024-09-19T04:53:20.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.127+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/299630961 shutdown_connections 2024-09-19T04:53:20.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.127+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/299630961 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562dcaf7be60 0x562dcaf7e250 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.127+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/299630961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562dcaf00ca0 0x562dcaf06cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.127+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/299630961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562dcaef63e0 0x562dcaf06770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.127+0000 7f7c3ba3d740 1 --2- 172.21.15.123:0/299630961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562dcaef5ac0 0x562dcaf093d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.127+0000 7f7c13e00640 0 client.26561 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:20.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.127+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/299630961 >> 172.21.15.123:0/299630961 conn(0x562dcaef2050 msgr2=0x562dcaef11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:20.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.128+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/299630961 shutdown_connections 2024-09-19T04:53:20.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.128+0000 7f7c3ba3d740 1 -- 172.21.15.123:0/299630961 wait complete. 2024-09-19T04:53:20.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.128+0000 7f7c3ba3d740 20 client.26561 trim_cache size 0 max 16384 2024-09-19T04:53:20.140 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:53:20.140 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-09-19T04:53:20.140 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "5b8bb3ed-4dc6-4789-abab-6e8356e27c21", 2024-09-19T04:53:20.141 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-09-19T04:53:20.141 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:53:20.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.128+0000 7f7c38e00640 1 -- 172.21.15.123:0/4083210495 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7c180a33b0 msgr2=0x7f7c180a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:20.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.128+0000 7f7c38e00640 1 --2- 172.21.15.123:0/4083210495 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7c180a33b0 0x7f7c180a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x562dcaef2eb0 tx=0x7f7c240095c0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.128+0000 7f7c38e00640 1 -- 172.21.15.123:0/4083210495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c340745c0 msgr2=0x7f7c3419a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:20.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.128+0000 7f7c38e00640 1 --2- 172.21.15.123:0/4083210495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c340745c0 0x7f7c3419a6c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7c24005e30 tx=0x7f7c24003ce0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.128+0000 7f7c38e00640 1 -- 172.21.15.123:0/4083210495 shutdown_connections 2024-09-19T04:53:20.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.128+0000 7f7c38e00640 1 --2- 172.21.15.123:0/4083210495 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7c180a33b0 0x7f7c180a5870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.128+0000 7f7c38e00640 1 --2- 172.21.15.123:0/4083210495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c3410e420 0x7f7c3419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.128+0000 7f7c38e00640 1 --2- 172.21.15.123:0/4083210495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c34074ee0 0x7f7c3419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.128+0000 7f7c38e00640 1 --2- 172.21.15.123:0/4083210495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c340745c0 0x7f7c3419a6c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:20.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.128+0000 7f7c38e00640 1 -- 172.21.15.123:0/4083210495 >> 172.21.15.123:0/4083210495 conn(0x7f7c340fd8c0 msgr2=0x7f7c340fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:20.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.129+0000 7f7c38e00640 1 -- 172.21.15.123:0/4083210495 shutdown_connections 2024-09-19T04:53:20.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:20.129+0000 7f7c38e00640 1 -- 172.21.15.123:0/4083210495 wait complete. 2024-09-19T04:53:20.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:20 smithi183 ceph-mon[35333]: scrub queued for path: / 2024-09-19T04:53:20.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:20 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:53:20.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:20 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:53:20.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:20 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:53:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:20 smithi123 ceph-mon[30511]: scrub queued for path: / 2024-09-19T04:53:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:20 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:53:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:20 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:53:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:20 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:53:20.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:20 smithi193 ceph-mon[35592]: scrub queued for path: / 2024-09-19T04:53:20.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:20 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:53:20.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:20 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:53:20.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:20 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:53:21.054 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:53:21.423 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:53:21.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:21 smithi183 ceph-mon[35333]: pgmap v204: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:53:21.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:21 smithi123 ceph-mon[30511]: pgmap v204: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:53:21.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:21 smithi193 ceph-mon[35592]: pgmap v204: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:53:21.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.961+0000 7f0493600640 1 -- 172.21.15.123:0/2256138588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f048c1049a0 msgr2=0x7f048c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:21.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.962+0000 7f0493600640 1 --2- 172.21.15.123:0/2256138588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f048c1049a0 0x7f048c1089f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0474009900 tx=0x7f047402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.962+0000 7f0493600640 1 -- 172.21.15.123:0/2256138588 shutdown_connections 2024-09-19T04:53:21.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.962+0000 7f0493600640 1 --2- 172.21.15.123:0/2256138588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f048c109120 0x7f048c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.962+0000 7f0493600640 1 --2- 172.21.15.123:0/2256138588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f048c1049a0 0x7f048c1089f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.962+0000 7f0493600640 1 --2- 172.21.15.123:0/2256138588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f048c103ff0 0x7f048c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.962+0000 7f0493600640 1 -- 172.21.15.123:0/2256138588 >> 172.21.15.123:0/2256138588 conn(0x7f048c0ffa90 msgr2=0x7f048c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:21.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.962+0000 7f0493600640 1 -- 172.21.15.123:0/2256138588 shutdown_connections 2024-09-19T04:53:21.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.962+0000 7f0493600640 1 -- 172.21.15.123:0/2256138588 wait complete. 2024-09-19T04:53:21.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.963+0000 7f0493600640 1 Processor -- start 2024-09-19T04:53:21.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.963+0000 7f0493600640 1 -- start start 2024-09-19T04:53:21.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.963+0000 7f0493600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f048c103ff0 0x7f048c076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:21.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.963+0000 7f0493600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f048c1049a0 0x7f048c076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:21.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.963+0000 7f0493600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f048c109120 0x7f048c077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:21.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.963+0000 7f0493600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f048c114190 con 0x7f048c109120 2024-09-19T04:53:21.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.963+0000 7f0493600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f048c114010 con 0x7f048c1049a0 2024-09-19T04:53:21.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.963+0000 7f0493600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f048c114310 con 0x7f048c103ff0 2024-09-19T04:53:21.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.963+0000 7f0491600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f048c109120 0x7f048c077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:21.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.964+0000 7f0491600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f048c109120 0x7f048c077180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60416/0 (socket says 172.21.15.123:60416) 2024-09-19T04:53:21.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.964+0000 7f0491600640 1 -- 172.21.15.123:0/620646217 learned_addr learned my addr 172.21.15.123:0/620646217 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:21.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.964+0000 7f0483e00640 1 --2- 172.21.15.123:0/620646217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f048c1049a0 0x7f048c076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:21.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.964+0000 7f0490c00640 1 --2- 172.21.15.123:0/620646217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f048c103ff0 0x7f048c076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:21.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.964+0000 7f0491600640 1 -- 172.21.15.123:0/620646217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f048c103ff0 msgr2=0x7f048c076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:21.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.964+0000 7f0491600640 1 --2- 172.21.15.123:0/620646217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f048c103ff0 0x7f048c076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.964+0000 7f0491600640 1 -- 172.21.15.123:0/620646217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f048c1049a0 msgr2=0x7f048c076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:21.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.964+0000 7f0491600640 1 --2- 172.21.15.123:0/620646217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f048c1049a0 0x7f048c076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.964+0000 7f0491600640 1 -- 172.21.15.123:0/620646217 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048c1aacd0 con 0x7f048c109120 2024-09-19T04:53:21.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.964+0000 7f0491600640 1 --2- 172.21.15.123:0/620646217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f048c109120 0x7f048c077180 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f048400db20 tx=0x7f048400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:21.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.964+0000 7f048a000640 1 -- 172.21.15.123:0/620646217 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484017070 con 0x7f048c109120 2024-09-19T04:53:21.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.964+0000 7f048a000640 1 -- 172.21.15.123:0/620646217 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0484003e50 con 0x7f048c109120 2024-09-19T04:53:21.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.964+0000 7f0493600640 1 -- 172.21.15.123:0/620646217 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f048c1aafc0 con 0x7f048c109120 2024-09-19T04:53:21.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.965+0000 7f048a000640 1 -- 172.21.15.123:0/620646217 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484004820 con 0x7f048c109120 2024-09-19T04:53:21.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.965+0000 7f0493600640 1 -- 172.21.15.123:0/620646217 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f048c1ab480 con 0x7f048c109120 2024-09-19T04:53:21.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.968+0000 7f049603c740 1 Processor -- start 2024-09-19T04:53:21.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.968+0000 7f048a000640 1 -- 172.21.15.123:0/620646217 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f04840040b0 con 0x7f048c109120 2024-09-19T04:53:21.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.968+0000 7f049603c740 1 -- start start 2024-09-19T04:53:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.968+0000 7f049603c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 0x556d402c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.968+0000 7f049603c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 0x556d402d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.968+0000 7f049603c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402d1d70 0x556d402d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.968+0000 7f049603c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556d4022d3e0 con 0x556d402d1d70 2024-09-19T04:53:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.968+0000 7f049603c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556d401d4d80 con 0x556d402c74b0 2024-09-19T04:53:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.968+0000 7f049603c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556d402293a0 con 0x556d402c6b90 2024-09-19T04:53:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.969+0000 7f0490c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 0x556d402d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.969+0000 7f0490c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 0x556d402d1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:52038/0 (socket says 172.21.15.123:52038) 2024-09-19T04:53:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.969+0000 7f0490c00640 1 -- 172.21.15.123:0/3776920213 learned_addr learned my addr 172.21.15.123:0/3776920213 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.969+0000 7f0483e00640 1 --2- 172.21.15.123:0/3776920213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 0x556d402c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.969+0000 7f0483e00640 1 --2- 172.21.15.123:0/3776920213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402d1d70 0x556d402d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.969+0000 7f0490c00640 1 -- 172.21.15.123:0/3776920213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 msgr2=0x556d402c6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.969+0000 7f0490c00640 1 --2- 172.21.15.123:0/3776920213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 0x556d402c6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.969+0000 7f0490c00640 1 -- 172.21.15.123:0/3776920213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402d1d70 msgr2=0x556d402d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:21.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.969+0000 7f0490c00640 1 --2- 172.21.15.123:0/3776920213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402d1d70 0x556d402d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.969+0000 7f0490c00640 1 -- 172.21.15.123:0/3776920213 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d401f3ae0 con 0x556d402c74b0 2024-09-19T04:53:21.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.969+0000 7f0483e00640 1 --2- 172.21.15.123:0/3776920213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402d1d70 0x556d402d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:53:21.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.969+0000 7f0490c00640 1 --2- 172.21.15.123:0/3776920213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 0x556d402d1830 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f047c009d20 tx=0x7f047c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38caf480c5c96dd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:21.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.969+0000 7f048a000640 1 --2- 172.21.15.123:0/620646217 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f04700a33b0 0x7f04700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:21.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.970+0000 7f0481c00640 1 -- 172.21.15.123:0/3776920213 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c01e070 con 0x556d402c74b0 2024-09-19T04:53:21.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.970+0000 7f048a000640 1 -- 172.21.15.123:0/620646217 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f04840cf9d0 con 0x7f048c109120 2024-09-19T04:53:21.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.970+0000 7f0483e00640 1 --2- 172.21.15.123:0/620646217 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f04700a33b0 0x7f04700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:21.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.970+0000 7f0481c00640 1 -- 172.21.15.123:0/3776920213 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f047c019b10 con 0x556d402c74b0 2024-09-19T04:53:21.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.970+0000 7f0481c00640 1 -- 172.21.15.123:0/3776920213 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c00eb40 con 0x556d402c74b0 2024-09-19T04:53:21.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.971+0000 7f049603c740 1 -- 172.21.15.123:0/3776920213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 msgr2=0x556d402d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:21.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.971+0000 7f0483e00640 1 --2- 172.21.15.123:0/620646217 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f04700a33b0 0x7f04700a5870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0474000c00 tx=0x7f047402a3f0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:21.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.971+0000 7f049603c740 1 --2- 172.21.15.123:0/3776920213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 0x556d402d1830 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f047c009d20 tx=0x7f047c011810 comp rx=0 tx=0).stop 2024-09-19T04:53:21.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.971+0000 7f049603c740 1 -- 172.21.15.123:0/3776920213 shutdown_connections 2024-09-19T04:53:21.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.971+0000 7f049603c740 1 --2- 172.21.15.123:0/3776920213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402d1d70 0x556d402d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.971+0000 7f049603c740 1 --2- 172.21.15.123:0/3776920213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 0x556d402d1830 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.971+0000 7f049603c740 1 --2- 172.21.15.123:0/3776920213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 0x556d402c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.971+0000 7f049603c740 1 -- 172.21.15.123:0/3776920213 >> 172.21.15.123:0/3776920213 conn(0x556d402c3120 msgr2=0x556d402c3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:21.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.971+0000 7f049603c740 1 -- 172.21.15.123:0/3776920213 shutdown_connections 2024-09-19T04:53:21.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.971+0000 7f049603c740 1 -- 172.21.15.123:0/3776920213 wait complete. 2024-09-19T04:53:21.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.971+0000 7f049603c740 1 -- start start 2024-09-19T04:53:21.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.972+0000 7f049603c740 1 Processor -- start 2024-09-19T04:53:21.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.972+0000 7f049603c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402c6b90 0x556d402cec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:21.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.972+0000 7f049603c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 0x556d402cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:21.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.972+0000 7f049603c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402d1d70 0x556d402cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:21.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.972+0000 7f049603c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556d402d49e0 con 0x556d402c6b90 2024-09-19T04:53:21.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.972+0000 7f049603c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556d401e80f0 con 0x556d402c74b0 2024-09-19T04:53:21.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.972+0000 7f049603c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556d402d4b60 con 0x556d402d1d70 2024-09-19T04:53:21.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.972+0000 7f049603c740 10 client.0 _ll_register_callbacks cb 0x556d401d99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:21.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.972+0000 7f0490c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402c6b90 0x556d402cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:21.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.972+0000 7f0490c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402c6b90 0x556d402cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60428/0 (socket says 172.21.15.123:60428) 2024-09-19T04:53:21.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.972+0000 7f0490c00640 1 -- 172.21.15.123:0/771190100 learned_addr learned my addr 172.21.15.123:0/771190100 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:21.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.973+0000 7f0491600640 1 --2- 172.21.15.123:0/771190100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402d1d70 0x556d402cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:21.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.973+0000 7f0483e00640 1 --2- 172.21.15.123:0/771190100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 0x556d402cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:21.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.973+0000 7f0491600640 1 -- 172.21.15.123:0/771190100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 msgr2=0x556d402cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:21.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.973+0000 7f0491600640 1 --2- 172.21.15.123:0/771190100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 0x556d402cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.973+0000 7f0491600640 1 -- 172.21.15.123:0/771190100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402c6b90 msgr2=0x556d402cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:21.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.973+0000 7f0491600640 1 --2- 172.21.15.123:0/771190100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402c6b90 0x556d402cec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.973+0000 7f0491600640 1 -- 172.21.15.123:0/771190100 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d3ff9e3c0 con 0x556d402d1d70 2024-09-19T04:53:21.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.973+0000 7f0490c00640 1 --2- 172.21.15.123:0/771190100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402c6b90 0x556d402cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:21.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.973+0000 7f0491600640 1 --2- 172.21.15.123:0/771190100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402d1d70 0x556d402cf680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f04840cd0c0 tx=0x7f048409b720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=801d4c8406a1f91d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:21.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.973+0000 7f047a000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:53:21.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.973+0000 7f047a000640 1 -- 172.21.15.123:0/771190100 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04840b1060 con 0x556d402d1d70 2024-09-19T04:53:21.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.974+0000 7f047a000640 1 -- 172.21.15.123:0/771190100 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f048409b150 con 0x556d402d1d70 2024-09-19T04:53:21.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.974+0000 7f049603c740 1 -- 172.21.15.123:0/771190100 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f047c019b10 con 0x556d402d1d70 2024-09-19T04:53:21.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.974+0000 7f047a000640 1 -- 172.21.15.123:0/771190100 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0484020040 con 0x556d402d1d70 2024-09-19T04:53:21.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.974+0000 7f049603c740 10 client.26593 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:21.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.974+0000 7f049603c740 1 -- 172.21.15.123:0/771190100 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556d402e0670 con 0x556d402d1d70 2024-09-19T04:53:21.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.974+0000 7f047a000640 1 -- 172.21.15.123:0/771190100 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f047c019b10 con 0x556d402d1d70 2024-09-19T04:53:21.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.974+0000 7f047a000640 1 -- 172.21.15.123:0/771190100 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f0484097c50 con 0x556d402d1d70 2024-09-19T04:53:21.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.974+0000 7f049603c740 10 client.26593 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:21.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.974+0000 7f049603c740 10 client.26593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:53:21.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.974+0000 7f049603c740 20 client.26593 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:21.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.974+0000 7f049603c740 1 --2- 172.21.15.123:0/771190100 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556d402e1b30 0x556d402e3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:21.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.974+0000 7f049603c740 4 client.26593 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:53:21.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.974+0000 7f049603c740 1 -- 172.21.15.123:0/771190100 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d4023a950 con 0x556d402e1b30 2024-09-19T04:53:21.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.975+0000 7f0490c00640 1 --2- 172.21.15.123:0/771190100 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556d402e1b30 0x556d402e3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:21.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.975+0000 7f0490c00640 1 --2- 172.21.15.123:0/771190100 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556d402e1b30 0x556d402e3fa0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46cf26c978bba1fa server_cookie=5175ac344323fed5 in_seq=0 out_seq=0 2024-09-19T04:53:21.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.975+0000 7f047a000640 10 client.26593 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:21.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.976+0000 7f047a000640 1 -- 172.21.15.123:0/771190100 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d4023a950 con 0x556d402e1b30 2024-09-19T04:53:21.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.976+0000 7f047a000640 10 client.26593 handle_command_reply: tid=0 2024-09-19T04:53:21.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.976+0000 7f049603c740 1 client.26593 shutdown 2024-09-19T04:53:21.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.976+0000 7f049603c740 1 -- 172.21.15.123:0/771190100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402d1d70 msgr2=0x556d402cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:21.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.976+0000 7f049603c740 1 --2- 172.21.15.123:0/771190100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402d1d70 0x556d402cf680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f04840cd0c0 tx=0x7f048409b720 comp rx=0 tx=0).stop 2024-09-19T04:53:21.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.976+0000 7f049603c740 1 -- 172.21.15.123:0/771190100 shutdown_connections 2024-09-19T04:53:21.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.977+0000 7f049603c740 1 --2- 172.21.15.123:0/771190100 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556d402e1b30 0x556d402e3fa0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.977+0000 7f049603c740 1 --2- 172.21.15.123:0/771190100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402d1d70 0x556d402cf680 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.977+0000 7f049603c740 1 --2- 172.21.15.123:0/771190100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 0x556d402cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.977+0000 7f047a000640 0 client.26593 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:21.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.977+0000 7f049603c740 1 --2- 172.21.15.123:0/771190100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402c6b90 0x556d402cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.977+0000 7f049603c740 1 -- 172.21.15.123:0/771190100 >> 172.21.15.123:0/771190100 conn(0x556d402c3120 msgr2=0x556d402c8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:21.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.977+0000 7f049603c740 1 -- 172.21.15.123:0/771190100 shutdown_connections 2024-09-19T04:53:21.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.977+0000 7f049603c740 1 -- 172.21.15.123:0/771190100 wait complete. 2024-09-19T04:53:21.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.977+0000 7f049603c740 20 client.26593 trim_cache size 0 max 16384 2024-09-19T04:53:21.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.994+0000 7f049603c740 1 Processor -- start 2024-09-19T04:53:21.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.995+0000 7f049603c740 1 -- start start 2024-09-19T04:53:21.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.995+0000 7f049603c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 0x556d402bde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:21.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.995+0000 7f049603c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402c74b0 0x556d402d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:21.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.995+0000 7f049603c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402d1d70 0x556d402e9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:21.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.995+0000 7f049603c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f048c1b3d70 con 0x556d402c74b0 2024-09-19T04:53:21.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.995+0000 7f049603c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f048c1b5940 con 0x556d402d1d70 2024-09-19T04:53:21.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.995+0000 7f049603c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f048c1b4470 con 0x556d402c6b90 2024-09-19T04:53:21.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.995+0000 7f0490c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 0x556d402bde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:21.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.995+0000 7f0483e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402c74b0 0x556d402d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:21.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.995+0000 7f0490c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 0x556d402bde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:48026/0 (socket says 172.21.15.123:48026) 2024-09-19T04:53:21.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.995+0000 7f0490c00640 1 -- 172.21.15.123:0/2493766910 learned_addr learned my addr 172.21.15.123:0/2493766910 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:21.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.995+0000 7f0491600640 1 --2- 172.21.15.123:0/2493766910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402d1d70 0x556d402e9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:21.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.995+0000 7f0490c00640 1 -- 172.21.15.123:0/2493766910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402d1d70 msgr2=0x556d402e9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:21.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.995+0000 7f0490c00640 1 --2- 172.21.15.123:0/2493766910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402d1d70 0x556d402e9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:21.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.995+0000 7f0490c00640 1 -- 172.21.15.123:0/2493766910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402c74b0 msgr2=0x556d402d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:21.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.995+0000 7f0490c00640 1 --2- 172.21.15.123:0/2493766910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402c74b0 0x556d402d0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:22.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.995+0000 7f0490c00640 1 -- 172.21.15.123:0/2493766910 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d402cd170 con 0x556d402c6b90 2024-09-19T04:53:22.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.996+0000 7f0490c00640 1 --2- 172.21.15.123:0/2493766910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 0x556d402bde50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f047c0058c0 tx=0x7f047c00e580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f325b52e94a0f67c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:22.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.996+0000 7f0481200640 1 -- 172.21.15.123:0/2493766910 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c02b070 con 0x556d402c6b90 2024-09-19T04:53:22.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.996+0000 7f0481200640 1 -- 172.21.15.123:0/2493766910 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f047c01f040 con 0x556d402c6b90 2024-09-19T04:53:22.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.996+0000 7f0481200640 1 -- 172.21.15.123:0/2493766910 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c01e040 con 0x556d402c6b90 2024-09-19T04:53:22.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.997+0000 7f049603c740 1 -- 172.21.15.123:0/2493766910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 msgr2=0x556d402bde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:22.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.997+0000 7f049603c740 1 --2- 172.21.15.123:0/2493766910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 0x556d402bde50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f047c0058c0 tx=0x7f047c00e580 comp rx=0 tx=0).stop 2024-09-19T04:53:22.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.997+0000 7f049603c740 1 -- 172.21.15.123:0/2493766910 shutdown_connections 2024-09-19T04:53:22.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.997+0000 7f049603c740 1 --2- 172.21.15.123:0/2493766910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402d1d70 0x556d402e9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:22.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.997+0000 7f049603c740 1 --2- 172.21.15.123:0/2493766910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402c74b0 0x556d402d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:22.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.997+0000 7f049603c740 1 --2- 172.21.15.123:0/2493766910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 0x556d402bde50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:22.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.997+0000 7f049603c740 1 -- 172.21.15.123:0/2493766910 >> 172.21.15.123:0/2493766910 conn(0x556d402c3120 msgr2=0x556d402d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:22.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.997+0000 7f049603c740 1 -- 172.21.15.123:0/2493766910 shutdown_connections 2024-09-19T04:53:22.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.997+0000 7f049603c740 1 -- 172.21.15.123:0/2493766910 wait complete. 2024-09-19T04:53:22.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.997+0000 7f049603c740 1 -- start start 2024-09-19T04:53:22.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.997+0000 7f049603c740 1 Processor -- start 2024-09-19T04:53:22.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f049603c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 0x556d402cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:22.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f049603c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 0x556d402cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:22.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f049603c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402d1d70 0x556d402cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:22.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f049603c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f048c1b5270 con 0x556d402d1d70 2024-09-19T04:53:22.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f049603c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f048c1b4b70 con 0x556d402c74b0 2024-09-19T04:53:22.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f049603c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f048c1b1d70 con 0x556d402c6b90 2024-09-19T04:53:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f049603c740 10 client.0 _ll_register_callbacks cb 0x7f048409b840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f0490c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 0x556d402cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f0490c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 0x556d402cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:48040/0 (socket says 172.21.15.123:48040) 2024-09-19T04:53:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f0490c00640 1 -- 172.21.15.123:0/2048289688 learned_addr learned my addr 172.21.15.123:0/2048289688 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f0491600640 1 --2- 172.21.15.123:0/2048289688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402d1d70 0x556d402cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f0483e00640 1 --2- 172.21.15.123:0/2048289688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 0x556d402cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f0490c00640 1 -- 172.21.15.123:0/2048289688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 msgr2=0x556d402cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f0490c00640 1 --2- 172.21.15.123:0/2048289688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 0x556d402cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f0490c00640 1 -- 172.21.15.123:0/2048289688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402d1d70 msgr2=0x556d402cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f0490c00640 1 --2- 172.21.15.123:0/2048289688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402d1d70 0x556d402cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f0490c00640 1 -- 172.21.15.123:0/2048289688 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d4034ba30 con 0x556d402c6b90 2024-09-19T04:53:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.998+0000 7f0483e00640 1 --2- 172.21.15.123:0/2048289688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 0x556d402cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:53:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.999+0000 7f0490c00640 1 --2- 172.21.15.123:0/2048289688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 0x556d402cbdd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f047c009c40 tx=0x7f047c00ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6391d98e6aba7ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.999+0000 7f047a000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:53:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.999+0000 7f047a000640 1 -- 172.21.15.123:0/2048289688 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c035060 con 0x556d402c6b90 2024-09-19T04:53:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.999+0000 7f047a000640 1 -- 172.21.15.123:0/2048289688 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f047c00e9f0 con 0x556d402c6b90 2024-09-19T04:53:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.999+0000 7f049603c740 1 -- 172.21.15.123:0/2048289688 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f047c01f040 con 0x556d402c6b90 2024-09-19T04:53:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.999+0000 7f047a000640 1 -- 172.21.15.123:0/2048289688 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f047c01e050 con 0x556d402c6b90 2024-09-19T04:53:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.999+0000 7f049603c740 10 client.26599 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:21.999+0000 7f049603c740 1 -- 172.21.15.123:0/2048289688 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556d4034ccc0 con 0x556d402c6b90 2024-09-19T04:53:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.000+0000 7f047a000640 1 -- 172.21.15.123:0/2048289688 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f047c01f040 con 0x556d402c6b90 2024-09-19T04:53:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.000+0000 7f047a000640 1 -- 172.21.15.123:0/2048289688 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f047c025040 con 0x556d402c6b90 2024-09-19T04:53:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.000+0000 7f049603c740 10 client.26599 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.000+0000 7f049603c740 10 client.26599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:53:22.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.000+0000 7f049603c740 20 client.26599 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:22.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.000+0000 7f049603c740 1 --2- 172.21.15.123:0/2048289688 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556d4034d170 0x556d4034f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:22.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.000+0000 7f049603c740 4 client.26599 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:53:22.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.000+0000 7f049603c740 1 -- 172.21.15.123:0/2048289688 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556d40234e60 con 0x556d4034d170 2024-09-19T04:53:22.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.001+0000 7f0483e00640 1 --2- 172.21.15.123:0/2048289688 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556d4034d170 0x556d4034f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:22.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.001+0000 7f0483e00640 1 --2- 172.21.15.123:0/2048289688 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556d4034d170 0x556d4034f590 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f44758ea9f237ffd server_cookie=24f63d9730b2bbdc in_seq=0 out_seq=0 2024-09-19T04:53:22.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.001+0000 7f047a000640 10 client.26599 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:22.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.001+0000 7f047a000640 1 -- 172.21.15.123:0/2048289688 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556d40234e60 con 0x556d4034d170 2024-09-19T04:53:22.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.001+0000 7f047a000640 10 client.26599 handle_command_reply: tid=0 2024-09-19T04:53:22.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.001+0000 7f049603c740 1 client.26599 shutdown 2024-09-19T04:53:22.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.001+0000 7f049603c740 1 -- 172.21.15.123:0/2048289688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 msgr2=0x556d402cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:22.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.001+0000 7f049603c740 1 --2- 172.21.15.123:0/2048289688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 0x556d402cbdd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f047c009c40 tx=0x7f047c00ecf0 comp rx=0 tx=0).stop 2024-09-19T04:53:22.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.002+0000 7f049603c740 1 -- 172.21.15.123:0/2048289688 shutdown_connections 2024-09-19T04:53:22.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.002+0000 7f049603c740 1 --2- 172.21.15.123:0/2048289688 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556d4034d170 0x556d4034f590 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:22.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.002+0000 7f049603c740 1 --2- 172.21.15.123:0/2048289688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d402d1d70 0x556d402cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:22.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.002+0000 7f049603c740 1 --2- 172.21.15.123:0/2048289688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d402c74b0 0x556d402cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:22.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.002+0000 7f047a000640 0 client.26599 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:22.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.002+0000 7f049603c740 1 --2- 172.21.15.123:0/2048289688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d402c6b90 0x556d402cbdd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:22.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.002+0000 7f049603c740 1 -- 172.21.15.123:0/2048289688 >> 172.21.15.123:0/2048289688 conn(0x556d402c3120 msgr2=0x556d402d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:22.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.002+0000 7f049603c740 1 -- 172.21.15.123:0/2048289688 shutdown_connections 2024-09-19T04:53:22.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.002+0000 7f049603c740 1 -- 172.21.15.123:0/2048289688 wait complete. 2024-09-19T04:53:22.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.002+0000 7f049603c740 20 client.26599 trim_cache size 0 max 16384 2024-09-19T04:53:22.011 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:53:22.011 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:53:22.011 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": { 2024-09-19T04:53:22.011 INFO:teuthology.orchestra.run.smithi123.stdout: "5b8bb3ed-4dc6-4789-abab-6e8356e27c21": { 2024-09-19T04:53:22.011 INFO:teuthology.orchestra.run.smithi123.stdout: "path": "/", 2024-09-19T04:53:22.011 INFO:teuthology.orchestra.run.smithi123.stdout: "tag": "5b8bb3ed-4dc6-4789-abab-6e8356e27c21", 2024-09-19T04:53:22.011 INFO:teuthology.orchestra.run.smithi123.stdout: "options": "recursive,force" 2024-09-19T04:53:22.011 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:53:22.011 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:53:22.012 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:53:22.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.003+0000 7f0493600640 1 -- 172.21.15.123:0/620646217 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f04700a33b0 msgr2=0x7f04700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:22.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.003+0000 7f0493600640 1 --2- 172.21.15.123:0/620646217 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f04700a33b0 0x7f04700a5870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0474000c00 tx=0x7f047402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:53:22.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.003+0000 7f0493600640 1 -- 172.21.15.123:0/620646217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f048c109120 msgr2=0x7f048c077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:22.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.003+0000 7f0493600640 1 --2- 172.21.15.123:0/620646217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f048c109120 0x7f048c077180 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f048400db20 tx=0x7f048400b260 comp rx=0 tx=0).stop 2024-09-19T04:53:22.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.003+0000 7f0493600640 1 -- 172.21.15.123:0/620646217 shutdown_connections 2024-09-19T04:53:22.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.003+0000 7f0493600640 1 --2- 172.21.15.123:0/620646217 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f04700a33b0 0x7f04700a5870 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:22.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.003+0000 7f0493600640 1 --2- 172.21.15.123:0/620646217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f048c109120 0x7f048c077180 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:22.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.003+0000 7f0493600640 1 --2- 172.21.15.123:0/620646217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f048c1049a0 0x7f048c076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:22.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.003+0000 7f0493600640 1 --2- 172.21.15.123:0/620646217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f048c103ff0 0x7f048c076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:22.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.003+0000 7f0493600640 1 -- 172.21.15.123:0/620646217 >> 172.21.15.123:0/620646217 conn(0x7f048c0ffa90 msgr2=0x7f048c100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:22.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.003+0000 7f0493600640 1 -- 172.21.15.123:0/620646217 shutdown_connections 2024-09-19T04:53:22.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:22.003+0000 7f0493600640 1 -- 172.21.15.123:0/620646217 wait complete. 2024-09-19T04:53:22.878 INFO:tasks.cephfs.filesystem:scrub status for tag:5b8bb3ed-4dc6-4789-abab-6e8356e27c21 - {'path': '/', 'tag': '5b8bb3ed-4dc6-4789-abab-6e8356e27c21', 'options': 'recursive,force'} 2024-09-19T04:53:23.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:23 smithi183 ceph-mon[35333]: pgmap v205: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:53:23.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:23 smithi123 ceph-mon[30511]: pgmap v205: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:53:23.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:23 smithi193 ceph-mon[35592]: pgmap v205: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:53:24.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:24 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:53:24.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:24 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:53:24.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:24 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:53:25.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:25 smithi183 ceph-mon[35333]: pgmap v206: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:53:25.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:25 smithi123 ceph-mon[30511]: pgmap v206: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:53:25.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:25 smithi193 ceph-mon[35592]: pgmap v206: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:53:27.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:27 smithi123 ceph-mon[30511]: pgmap v207: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:53:27.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:27 smithi193 ceph-mon[35592]: pgmap v207: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:53:27.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:27 smithi183 ceph-mon[35333]: pgmap v207: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:53:29.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:29 smithi123 ceph-mon[30511]: pgmap v208: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 94 B/s wr, 14 op/s 2024-09-19T04:53:29.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:29 smithi193 ceph-mon[35592]: pgmap v208: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 94 B/s wr, 14 op/s 2024-09-19T04:53:29.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:29 smithi183 ceph-mon[35333]: pgmap v208: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 94 B/s wr, 14 op/s 2024-09-19T04:53:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:31 smithi123 ceph-mon[30511]: pgmap v209: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:53:31.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:31 smithi193 ceph-mon[35592]: pgmap v209: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:53:31.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:31 smithi183 ceph-mon[35333]: pgmap v209: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:53:33.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:33 smithi123 ceph-mon[30511]: pgmap v210: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:53:33.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:33 smithi193 ceph-mon[35592]: pgmap v210: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:53:33.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:33 smithi183 ceph-mon[35333]: pgmap v210: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:53:35.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:35 smithi123 ceph-mon[30511]: pgmap v211: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:53:35.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:35 smithi193 ceph-mon[35592]: pgmap v211: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:53:35.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:35 smithi183 ceph-mon[35333]: pgmap v211: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:53:37.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:37 smithi123 ceph-mon[30511]: pgmap v212: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:37.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:37 smithi193 ceph-mon[35592]: pgmap v212: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:37.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:37 smithi183 ceph-mon[35333]: pgmap v212: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:39.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:39 smithi123 ceph-mon[30511]: pgmap v213: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:53:39.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:39 smithi193 ceph-mon[35592]: pgmap v213: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:53:39.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:39 smithi183 ceph-mon[35333]: pgmap v213: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:53:41.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:41 smithi123 ceph-mon[30511]: pgmap v214: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:41.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:41 smithi193 ceph-mon[35592]: pgmap v214: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:41.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:41 smithi183 ceph-mon[35333]: pgmap v214: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:43.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:43 smithi123 ceph-mon[30511]: pgmap v215: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:43.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:43 smithi193 ceph-mon[35592]: pgmap v215: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:43.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:43 smithi183 ceph-mon[35333]: pgmap v215: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:45.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:45 smithi123 ceph-mon[30511]: pgmap v216: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:53:45.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:45 smithi193 ceph-mon[35592]: pgmap v216: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:53:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:45 smithi183 ceph-mon[35333]: pgmap v216: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:53:47.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:47 smithi123 ceph-mon[30511]: pgmap v217: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:47.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:47 smithi193 ceph-mon[35592]: pgmap v217: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:47 smithi183 ceph-mon[35333]: pgmap v217: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:49.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:49 smithi123 ceph-mon[30511]: pgmap v218: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:53:49.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:49 smithi193 ceph-mon[35592]: pgmap v218: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:53:49.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:49 smithi183 ceph-mon[35333]: pgmap v218: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:53:51.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:51 smithi123 ceph-mon[30511]: pgmap v219: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:51.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:51 smithi193 ceph-mon[35592]: pgmap v219: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:51.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:51 smithi183 ceph-mon[35333]: pgmap v219: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:52.880 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:53:53.260 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:53:53.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:53 smithi123 ceph-mon[30511]: pgmap v220: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:53.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:53 smithi193 ceph-mon[35592]: pgmap v220: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:53.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.805+0000 7fb9d5c00640 1 -- 172.21.15.123:0/3273110145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d0074eb0 msgr2=0x7fb9d0112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:53.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.805+0000 7fb9d5c00640 1 --2- 172.21.15.123:0/3273110145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d0074eb0 0x7fb9d0112650 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009900 tx=0x7fb9b802a480 comp rx=0 tx=0).stop 2024-09-19T04:53:53.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.805+0000 7fb9d5c00640 1 -- 172.21.15.123:0/3273110145 shutdown_connections 2024-09-19T04:53:53.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.805+0000 7fb9d5c00640 1 --2- 172.21.15.123:0/3273110145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9d0112b90 0x7fb9d0114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.805+0000 7fb9d5c00640 1 --2- 172.21.15.123:0/3273110145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d0074eb0 0x7fb9d0112650 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.805+0000 7fb9d5c00640 1 --2- 172.21.15.123:0/3273110145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9d0074590 0x7fb9d0074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.805+0000 7fb9d5c00640 1 -- 172.21.15.123:0/3273110145 >> 172.21.15.123:0/3273110145 conn(0x7fb9d00fd8c0 msgr2=0x7fb9d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:53.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.805+0000 7fb9d5c00640 1 -- 172.21.15.123:0/3273110145 shutdown_connections 2024-09-19T04:53:53.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.805+0000 7fb9d5c00640 1 -- 172.21.15.123:0/3273110145 wait complete. 2024-09-19T04:53:53.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.806+0000 7fb9d5c00640 1 Processor -- start 2024-09-19T04:53:53.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.806+0000 7fb9d5c00640 1 -- start start 2024-09-19T04:53:53.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.806+0000 7fb9d5c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9d0074590 0x7fb9d019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.806+0000 7fb9d5c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9d0074eb0 0x7fb9d019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.806+0000 7fb9d5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d0112b90 0x7fb9d01a3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.806+0000 7fb9d5c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d0118560 con 0x7fb9d0112b90 2024-09-19T04:53:53.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.806+0000 7fb9d5c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d01183e0 con 0x7fb9d0074eb0 2024-09-19T04:53:53.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.806+0000 7fb9d5c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d01186e0 con 0x7fb9d0074590 2024-09-19T04:53:53.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.807+0000 7fb9cf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d0112b90 0x7fb9d01a3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.807+0000 7fb9c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9d0074eb0 0x7fb9d019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.807+0000 7fb9c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9d0074eb0 0x7fb9d019f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:46964/0 (socket says 172.21.15.123:46964) 2024-09-19T04:53:53.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.807+0000 7fb9cf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d0112b90 0x7fb9d01a3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48760/0 (socket says 172.21.15.123:48760) 2024-09-19T04:53:53.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.807+0000 7fb9cf400640 1 -- 172.21.15.123:0/3944830860 learned_addr learned my addr 172.21.15.123:0/3944830860 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:53.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.807+0000 7fb9cea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9d0074590 0x7fb9d019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.807+0000 7fb9c7e00640 1 -- 172.21.15.123:0/3944830860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9d0074590 msgr2=0x7fb9d019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:53.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.807+0000 7fb9c7e00640 1 --2- 172.21.15.123:0/3944830860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9d0074590 0x7fb9d019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.807+0000 7fb9c7e00640 1 -- 172.21.15.123:0/3944830860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d0112b90 msgr2=0x7fb9d01a3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:53.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.807+0000 7fb9c7e00640 1 --2- 172.21.15.123:0/3944830860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d0112b90 0x7fb9d01a3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.807+0000 7fb9c7e00640 1 -- 172.21.15.123:0/3944830860 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d01a3b50 con 0x7fb9d0074eb0 2024-09-19T04:53:53.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.807+0000 7fb9c7e00640 1 --2- 172.21.15.123:0/3944830860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9d0074eb0 0x7fb9d019f060 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8005940 tx=0x7fb9b80040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:53.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.808+0000 7fb9c7400640 1 -- 172.21.15.123:0/3944830860 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b803f030 con 0x7fb9d0074eb0 2024-09-19T04:53:53.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.808+0000 7fb9d5c00640 1 -- 172.21.15.123:0/3944830860 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d01a3de0 con 0x7fb9d0074eb0 2024-09-19T04:53:53.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.808+0000 7fb9c7400640 1 -- 172.21.15.123:0/3944830860 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b80044c0 con 0x7fb9d0074eb0 2024-09-19T04:53:53.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.808+0000 7fb9c7400640 1 -- 172.21.15.123:0/3944830860 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8035070 con 0x7fb9d0074eb0 2024-09-19T04:53:53.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.808+0000 7fb9d5c00640 1 -- 172.21.15.123:0/3944830860 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d01ab6d0 con 0x7fb9d0074eb0 2024-09-19T04:53:53.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.811+0000 7fb9c7400640 1 -- 172.21.15.123:0/3944830860 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb9b8042030 con 0x7fb9d0074eb0 2024-09-19T04:53:53.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.811+0000 7fb9d866b740 1 Processor -- start 2024-09-19T04:53:53.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.811+0000 7fb9d866b740 1 -- start start 2024-09-19T04:53:53.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.811+0000 7fb9d866b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de89fb90 0x5593de89ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.811+0000 7fb9d866b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8aa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.811+0000 7fb9d866b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8aad70 0x5593de8ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.811+0000 7fb9d866b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5593de8063e0 con 0x5593de8a04b0 2024-09-19T04:53:53.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.811+0000 7fb9d866b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5593de7add80 con 0x5593de8aad70 2024-09-19T04:53:53.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.811+0000 7fb9d866b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5593de8023a0 con 0x5593de89fb90 2024-09-19T04:53:53.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.811+0000 7fb9cea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8aa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.812+0000 7fb9c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8aad70 0x5593de8ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.812+0000 7fb9cea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8aa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48770/0 (socket says 172.21.15.123:48770) 2024-09-19T04:53:53.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.812+0000 7fb9cea00640 1 -- 172.21.15.123:0/2781495763 learned_addr learned my addr 172.21.15.123:0/2781495763 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:53.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.812+0000 7fb9cf400640 1 --2- 172.21.15.123:0/2781495763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de89fb90 0x5593de89ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.812+0000 7fb9cea00640 1 -- 172.21.15.123:0/2781495763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de89fb90 msgr2=0x5593de89ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:53.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.812+0000 7fb9cea00640 1 --2- 172.21.15.123:0/2781495763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de89fb90 0x5593de89ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.812+0000 7fb9cea00640 1 -- 172.21.15.123:0/2781495763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8aad70 msgr2=0x5593de8ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:53.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.812+0000 7fb9cea00640 1 --2- 172.21.15.123:0/2781495763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8aad70 0x5593de8ad160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.812+0000 7fb9cea00640 1 -- 172.21.15.123:0/2781495763 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593de7ccae0 con 0x5593de8a04b0 2024-09-19T04:53:53.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.812+0000 7fb9cea00640 1 --2- 172.21.15.123:0/2781495763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8aa830 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009cf0 tx=0x7fb9c0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb7202200626aaa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:53.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.813+0000 7fb9bfe00640 1 -- 172.21.15.123:0/2781495763 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0004230 con 0x5593de8a04b0 2024-09-19T04:53:53.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.813+0000 7fb9c7400640 1 --2- 172.21.15.123:0/3944830860 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb9b40a33b0 0x7fb9b40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.813+0000 7fb9bfe00640 1 -- 172.21.15.123:0/2781495763 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c00043d0 con 0x5593de8a04b0 2024-09-19T04:53:53.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.813+0000 7fb9bfe00640 1 -- 172.21.15.123:0/2781495763 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c00198c0 con 0x5593de8a04b0 2024-09-19T04:53:53.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.813+0000 7fb9c7400640 1 -- 172.21.15.123:0/3944830860 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb9b80efcf0 con 0x7fb9d0074eb0 2024-09-19T04:53:53.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.813+0000 7fb9cf400640 1 --2- 172.21.15.123:0/3944830860 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb9b40a33b0 0x7fb9b40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.814+0000 7fb9d866b740 1 -- 172.21.15.123:0/2781495763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 msgr2=0x5593de8aa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:53.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.814+0000 7fb9d866b740 1 --2- 172.21.15.123:0/2781495763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8aa830 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009cf0 tx=0x7fb9c0011810 comp rx=0 tx=0).stop 2024-09-19T04:53:53.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.814+0000 7fb9cf400640 1 --2- 172.21.15.123:0/3944830860 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb9b40a33b0 0x7fb9b40a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8011630 tx=0x7fb9c8009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:53.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.814+0000 7fb9d866b740 1 -- 172.21.15.123:0/2781495763 shutdown_connections 2024-09-19T04:53:53.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.814+0000 7fb9d866b740 1 --2- 172.21.15.123:0/2781495763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8aad70 0x5593de8ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.814+0000 7fb9d866b740 1 --2- 172.21.15.123:0/2781495763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8aa830 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.814+0000 7fb9d866b740 1 --2- 172.21.15.123:0/2781495763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de89fb90 0x5593de89ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.814+0000 7fb9d866b740 1 -- 172.21.15.123:0/2781495763 >> 172.21.15.123:0/2781495763 conn(0x5593de89c120 msgr2=0x5593de89c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:53.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.814+0000 7fb9d866b740 1 -- 172.21.15.123:0/2781495763 shutdown_connections 2024-09-19T04:53:53.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.814+0000 7fb9d866b740 1 -- 172.21.15.123:0/2781495763 wait complete. 2024-09-19T04:53:53.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.814+0000 7fb9d866b740 1 -- start start 2024-09-19T04:53:53.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.815+0000 7fb9d866b740 1 Processor -- start 2024-09-19T04:53:53.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.815+0000 7fb9d866b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de89fb90 0x5593de8a7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.815+0000 7fb9d866b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8a04b0 0x5593de8a80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.815+0000 7fb9d866b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de8aad70 0x5593de8a8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.815+0000 7fb9d866b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5593de8ad9e0 con 0x5593de89fb90 2024-09-19T04:53:53.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.815+0000 7fb9d866b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5593de7c10f0 con 0x5593de8a04b0 2024-09-19T04:53:53.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.815+0000 7fb9d866b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5593de8adb60 con 0x5593de8aad70 2024-09-19T04:53:53.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.815+0000 7fb9d866b740 10 client.0 _ll_register_callbacks cb 0x5593de7b29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:53.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.815+0000 7fb9cea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de89fb90 0x5593de8a7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.815+0000 7fb9c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8a04b0 0x5593de8a80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.815+0000 7fb9cf400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de8aad70 0x5593de8a8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.815+0000 7fb9cea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de89fb90 0x5593de8a7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48778/0 (socket says 172.21.15.123:48778) 2024-09-19T04:53:53.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.815+0000 7fb9c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8a04b0 0x5593de8a80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:46970/0 (socket says 172.21.15.123:46970) 2024-09-19T04:53:53.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.815+0000 7fb9cea00640 1 -- 172.21.15.123:0/1304081513 learned_addr learned my addr 172.21.15.123:0/1304081513 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:53.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.816+0000 7fb9c7e00640 1 -- 172.21.15.123:0/1304081513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de8aad70 msgr2=0x5593de8a8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:53.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.816+0000 7fb9c7e00640 1 --2- 172.21.15.123:0/1304081513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de8aad70 0x5593de8a8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.816+0000 7fb9c7e00640 1 -- 172.21.15.123:0/1304081513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de89fb90 msgr2=0x5593de8a7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:53.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.816+0000 7fb9c7e00640 1 --2- 172.21.15.123:0/1304081513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de89fb90 0x5593de8a7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.816+0000 7fb9c7e00640 1 -- 172.21.15.123:0/1304081513 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593de5773c0 con 0x5593de8a04b0 2024-09-19T04:53:53.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.816+0000 7fb9cea00640 1 --2- 172.21.15.123:0/1304081513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de89fb90 0x5593de8a7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:53.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.816+0000 7fb9c7e00640 1 --2- 172.21.15.123:0/1304081513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8a04b0 0x5593de8a80f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80ed3e0 tx=0x7fb9b80b7ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7a37a319ccc668c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:53.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.816+0000 7fb9bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:53:53.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.816+0000 7fb9bcc00640 1 -- 172.21.15.123:0/1304081513 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8049070 con 0x5593de8a04b0 2024-09-19T04:53:53.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.816+0000 7fb9bcc00640 1 -- 172.21.15.123:0/1304081513 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb9b80ed660 con 0x5593de8a04b0 2024-09-19T04:53:53.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.816+0000 7fb9d866b740 1 -- 172.21.15.123:0/1304081513 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9c00043d0 con 0x5593de8a04b0 2024-09-19T04:53:53.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.817+0000 7fb9bcc00640 1 -- 172.21.15.123:0/1304081513 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb9b80bbb60 con 0x5593de8a04b0 2024-09-19T04:53:53.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.817+0000 7fb9d866b740 10 client.26591 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:53.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.817+0000 7fb9d866b740 1 -- 172.21.15.123:0/1304081513 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5593de8b9610 con 0x5593de8a04b0 2024-09-19T04:53:53.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.817+0000 7fb9bcc00640 1 -- 172.21.15.123:0/1304081513 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c00043d0 con 0x5593de8a04b0 2024-09-19T04:53:53.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.817+0000 7fb9bcc00640 1 -- 172.21.15.123:0/1304081513 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fb9b80bf050 con 0x5593de8a04b0 2024-09-19T04:53:53.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.817+0000 7fb9d866b740 10 client.26591 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:53.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.817+0000 7fb9d866b740 10 client.26591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:53:53.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.817+0000 7fb9d866b740 20 client.26591 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:53.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.817+0000 7fb9d866b740 1 --2- 172.21.15.123:0/1304081513 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5593de8ba9b0 0x5593de8bce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.817+0000 7fb9d866b740 4 client.26591 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:53:53.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.817+0000 7fb9d866b740 1 -- 172.21.15.123:0/1304081513 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593de813950 con 0x5593de8ba9b0 2024-09-19T04:53:53.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.818+0000 7fb9cea00640 1 --2- 172.21.15.123:0/1304081513 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5593de8ba9b0 0x5593de8bce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.818+0000 7fb9cea00640 1 --2- 172.21.15.123:0/1304081513 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5593de8ba9b0 0x5593de8bce40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57509ca57d775880 server_cookie=69b20dfcaaa1f526 in_seq=0 out_seq=0 2024-09-19T04:53:53.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.818+0000 7fb9bcc00640 10 client.26591 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:53.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.819+0000 7fb9bcc00640 1 -- 172.21.15.123:0/1304081513 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5593de813950 con 0x5593de8ba9b0 2024-09-19T04:53:53.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.819+0000 7fb9bcc00640 10 client.26591 handle_command_reply: tid=0 2024-09-19T04:53:53.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.819+0000 7fb9d866b740 1 client.26591 shutdown 2024-09-19T04:53:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.819+0000 7fb9d866b740 1 -- 172.21.15.123:0/1304081513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8a04b0 msgr2=0x5593de8a80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.819+0000 7fb9d866b740 1 --2- 172.21.15.123:0/1304081513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8a04b0 0x5593de8a80f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80ed3e0 tx=0x7fb9b80b7ba0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.820+0000 7fb9d866b740 1 -- 172.21.15.123:0/1304081513 shutdown_connections 2024-09-19T04:53:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.820+0000 7fb9d866b740 1 --2- 172.21.15.123:0/1304081513 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5593de8ba9b0 0x5593de8bce40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.820+0000 7fb9d866b740 1 --2- 172.21.15.123:0/1304081513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de8aad70 0x5593de8a8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.820+0000 7fb9d866b740 1 --2- 172.21.15.123:0/1304081513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8a04b0 0x5593de8a80f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.820+0000 7fb9d866b740 1 --2- 172.21.15.123:0/1304081513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de89fb90 0x5593de8a7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.820+0000 7fb9d866b740 1 -- 172.21.15.123:0/1304081513 >> 172.21.15.123:0/1304081513 conn(0x5593de89c120 msgr2=0x5593de8a1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.820+0000 7fb9bcc00640 0 client.26591 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.820+0000 7fb9d866b740 1 -- 172.21.15.123:0/1304081513 shutdown_connections 2024-09-19T04:53:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.820+0000 7fb9d866b740 1 -- 172.21.15.123:0/1304081513 wait complete. 2024-09-19T04:53:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.820+0000 7fb9d866b740 20 client.26591 trim_cache size 0 max 16384 2024-09-19T04:53:53.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.837+0000 7fb9d866b740 1 Processor -- start 2024-09-19T04:53:53.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.837+0000 7fb9d866b740 1 -- start start 2024-09-19T04:53:53.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.838+0000 7fb9d866b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de89fb90 0x5593de8bc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.838+0000 7fb9d866b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8a9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.838+0000 7fb9d866b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8aad70 0x5593de8c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.838+0000 7fb9d866b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d01b3f40 con 0x5593de8a04b0 2024-09-19T04:53:53.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.838+0000 7fb9d866b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d01b5b10 con 0x5593de8aad70 2024-09-19T04:53:53.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.838+0000 7fb9d866b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d01b4640 con 0x5593de89fb90 2024-09-19T04:53:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.838+0000 7fb9c7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8a9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.838+0000 7fb9c7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8a9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48790/0 (socket says 172.21.15.123:48790) 2024-09-19T04:53:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.838+0000 7fb9c7e00640 1 -- 172.21.15.123:0/4243604271 learned_addr learned my addr 172.21.15.123:0/4243604271 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.838+0000 7fb9cea00640 1 --2- 172.21.15.123:0/4243604271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de89fb90 0x5593de8bc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.838+0000 7fb9cf400640 1 --2- 172.21.15.123:0/4243604271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8aad70 0x5593de8c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.838+0000 7fb9c7e00640 1 -- 172.21.15.123:0/4243604271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de89fb90 msgr2=0x5593de8bc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.838+0000 7fb9c7e00640 1 --2- 172.21.15.123:0/4243604271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de89fb90 0x5593de8bc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.838+0000 7fb9c7e00640 1 -- 172.21.15.123:0/4243604271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8aad70 msgr2=0x5593de8c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.838+0000 7fb9c7e00640 1 --2- 172.21.15.123:0/4243604271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8aad70 0x5593de8c26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.838+0000 7fb9c7e00640 1 -- 172.21.15.123:0/4243604271 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593de8aa520 con 0x5593de8a04b0 2024-09-19T04:53:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.839+0000 7fb9c7e00640 1 --2- 172.21.15.123:0/4243604271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8a9b20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80bbfb0 tx=0x7fb9b802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df2843a81c5f1d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.839+0000 7fb9bf400640 1 -- 172.21.15.123:0/4243604271 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b80d6070 con 0x5593de8a04b0 2024-09-19T04:53:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.839+0000 7fb9bf400640 1 -- 172.21.15.123:0/4243604271 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b80f2070 con 0x5593de8a04b0 2024-09-19T04:53:53.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.839+0000 7fb9bf400640 1 -- 172.21.15.123:0/4243604271 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b80d4070 con 0x5593de8a04b0 2024-09-19T04:53:53.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.840+0000 7fb9d866b740 1 -- 172.21.15.123:0/4243604271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 msgr2=0x5593de8a9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:53.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.840+0000 7fb9d866b740 1 --2- 172.21.15.123:0/4243604271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8a9b20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80bbfb0 tx=0x7fb9b802a480 comp rx=0 tx=0).stop 2024-09-19T04:53:53.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.840+0000 7fb9d866b740 1 -- 172.21.15.123:0/4243604271 shutdown_connections 2024-09-19T04:53:53.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.840+0000 7fb9d866b740 1 --2- 172.21.15.123:0/4243604271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8aad70 0x5593de8c26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.840+0000 7fb9d866b740 1 --2- 172.21.15.123:0/4243604271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8a9b20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.840+0000 7fb9d866b740 1 --2- 172.21.15.123:0/4243604271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de89fb90 0x5593de8bc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.840+0000 7fb9d866b740 1 -- 172.21.15.123:0/4243604271 >> 172.21.15.123:0/4243604271 conn(0x5593de89c120 msgr2=0x5593de8a9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:53.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.840+0000 7fb9d866b740 1 -- 172.21.15.123:0/4243604271 shutdown_connections 2024-09-19T04:53:53.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.840+0000 7fb9d866b740 1 -- 172.21.15.123:0/4243604271 wait complete. 2024-09-19T04:53:53.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.840+0000 7fb9d866b740 1 -- start start 2024-09-19T04:53:53.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.841+0000 7fb9d866b740 1 Processor -- start 2024-09-19T04:53:53.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.841+0000 7fb9d866b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de89fb90 0x5593de8aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.841+0000 7fb9d866b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.841+0000 7fb9d866b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8aad70 0x5593de8af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.841+0000 7fb9d866b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d01b5440 con 0x5593de8a04b0 2024-09-19T04:53:53.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.841+0000 7fb9d866b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d01b4d40 con 0x5593de8aad70 2024-09-19T04:53:53.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.841+0000 7fb9d866b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d01b1f40 con 0x5593de89fb90 2024-09-19T04:53:53.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.841+0000 7fb9d866b740 10 client.0 _ll_register_callbacks cb 0x7fb9b8005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:53.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.841+0000 7fb9cea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de89fb90 0x5593de8aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.841+0000 7fb9c7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.841+0000 7fb9cf400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8aad70 0x5593de8af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.841+0000 7fb9cea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de89fb90 0x5593de8aeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:44798/0 (socket says 172.21.15.123:44798) 2024-09-19T04:53:53.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.841+0000 7fb9c7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8af1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48798/0 (socket says 172.21.15.123:48798) 2024-09-19T04:53:53.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.841+0000 7fb9cea00640 1 -- 172.21.15.123:0/312492274 learned_addr learned my addr 172.21.15.123:0/312492274 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:53.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.842+0000 7fb9c7e00640 1 -- 172.21.15.123:0/312492274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de89fb90 msgr2=0x5593de8aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:53.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.842+0000 7fb9c7e00640 1 --2- 172.21.15.123:0/312492274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de89fb90 0x5593de8aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.842+0000 7fb9c7e00640 1 -- 172.21.15.123:0/312492274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8aad70 msgr2=0x5593de8af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:53.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.842+0000 7fb9c7e00640 1 --2- 172.21.15.123:0/312492274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8aad70 0x5593de8af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.842+0000 7fb9c7e00640 1 -- 172.21.15.123:0/312492274 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593de924820 con 0x5593de8a04b0 2024-09-19T04:53:53.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.842+0000 7fb9cea00640 1 --2- 172.21.15.123:0/312492274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de89fb90 0x5593de8aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:53.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.842+0000 7fb9cf400640 1 --2- 172.21.15.123:0/312492274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8aad70 0x5593de8af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:53.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.842+0000 7fb9c7e00640 1 --2- 172.21.15.123:0/312492274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8af1e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80c46b0 tx=0x7fb9b80bc130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b150d36f9da4a83 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:53.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.842+0000 7fb9bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:53:53.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.842+0000 7fb9bcc00640 1 -- 172.21.15.123:0/312492274 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b80d6070 con 0x5593de8a04b0 2024-09-19T04:53:53.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.842+0000 7fb9bcc00640 1 -- 172.21.15.123:0/312492274 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb9b80c4950 con 0x5593de8a04b0 2024-09-19T04:53:53.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.842+0000 7fb9d866b740 1 -- 172.21.15.123:0/312492274 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9b80f2070 con 0x5593de8a04b0 2024-09-19T04:53:53.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.842+0000 7fb9bcc00640 1 -- 172.21.15.123:0/312492274 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb9b80d4040 con 0x5593de8a04b0 2024-09-19T04:53:53.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.843+0000 7fb9d866b740 10 client.16497 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:53.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.843+0000 7fb9d866b740 1 -- 172.21.15.123:0/312492274 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5593de924820 con 0x5593de8a04b0 2024-09-19T04:53:53.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.843+0000 7fb9bcc00640 1 -- 172.21.15.123:0/312492274 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b80f2070 con 0x5593de8a04b0 2024-09-19T04:53:53.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.843+0000 7fb9bcc00640 1 -- 172.21.15.123:0/312492274 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fb9b80bf070 con 0x5593de8a04b0 2024-09-19T04:53:53.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.843+0000 7fb9d866b740 10 client.16497 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:53.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.843+0000 7fb9d866b740 10 client.16497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:53:53.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.843+0000 7fb9d866b740 20 client.16497 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:53.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.843+0000 7fb9d866b740 1 --2- 172.21.15.123:0/312492274 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5593de925e00 0x5593de9281f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:53.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.843+0000 7fb9d866b740 4 client.16497 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:53:53.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.843+0000 7fb9d866b740 1 -- 172.21.15.123:0/312492274 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5593de80de60 con 0x5593de925e00 2024-09-19T04:53:53.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.844+0000 7fb9cea00640 1 --2- 172.21.15.123:0/312492274 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5593de925e00 0x5593de9281f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:53.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.844+0000 7fb9cea00640 1 --2- 172.21.15.123:0/312492274 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5593de925e00 0x5593de9281f0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a2a4cdb358fed09 server_cookie=2a7b66efed594040 in_seq=0 out_seq=0 2024-09-19T04:53:53.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.844+0000 7fb9bcc00640 10 client.16497 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:53.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.844+0000 7fb9bcc00640 1 -- 172.21.15.123:0/312492274 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5593de80de60 con 0x5593de925e00 2024-09-19T04:53:53.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.844+0000 7fb9bcc00640 10 client.16497 handle_command_reply: tid=0 2024-09-19T04:53:53.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.844+0000 7fb9d866b740 1 client.16497 shutdown 2024-09-19T04:53:53.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.845+0000 7fb9d866b740 1 -- 172.21.15.123:0/312492274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 msgr2=0x5593de8af1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:53.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.845+0000 7fb9d866b740 1 --2- 172.21.15.123:0/312492274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8af1e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80c46b0 tx=0x7fb9b80bc130 comp rx=0 tx=0).stop 2024-09-19T04:53:53.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.845+0000 7fb9d866b740 1 -- 172.21.15.123:0/312492274 shutdown_connections 2024-09-19T04:53:53.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.845+0000 7fb9d866b740 1 --2- 172.21.15.123:0/312492274 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5593de925e00 0x5593de9281f0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.845+0000 7fb9d866b740 1 --2- 172.21.15.123:0/312492274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5593de8aad70 0x5593de8af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.845+0000 7fb9d866b740 1 --2- 172.21.15.123:0/312492274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593de8a04b0 0x5593de8af1e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.845+0000 7fb9d866b740 1 --2- 172.21.15.123:0/312492274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5593de89fb90 0x5593de8aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.845+0000 7fb9bcc00640 0 client.16497 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:53:53.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.845+0000 7fb9d866b740 1 -- 172.21.15.123:0/312492274 >> 172.21.15.123:0/312492274 conn(0x5593de89c120 msgr2=0x5593de8a8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:53.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.845+0000 7fb9d866b740 1 -- 172.21.15.123:0/312492274 shutdown_connections 2024-09-19T04:53:53.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.845+0000 7fb9d866b740 1 -- 172.21.15.123:0/312492274 wait complete. 2024-09-19T04:53:53.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.845+0000 7fb9d866b740 20 client.16497 trim_cache size 0 max 16384 2024-09-19T04:53:53.850 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:53:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-09-19T04:53:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-09-19T04:53:53.851 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:53:53.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.846+0000 7fb9d5c00640 1 -- 172.21.15.123:0/3944830860 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb9b40a33b0 msgr2=0x7fb9b40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:53.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.846+0000 7fb9d5c00640 1 --2- 172.21.15.123:0/3944830860 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb9b40a33b0 0x7fb9b40a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8011630 tx=0x7fb9c8009500 comp rx=0 tx=0).stop 2024-09-19T04:53:53.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.846+0000 7fb9d5c00640 1 -- 172.21.15.123:0/3944830860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9d0074eb0 msgr2=0x7fb9d019f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:53.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.846+0000 7fb9d5c00640 1 --2- 172.21.15.123:0/3944830860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9d0074eb0 0x7fb9d019f060 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8005940 tx=0x7fb9b80040c0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.846+0000 7fb9d5c00640 1 -- 172.21.15.123:0/3944830860 shutdown_connections 2024-09-19T04:53:53.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.846+0000 7fb9d5c00640 1 --2- 172.21.15.123:0/3944830860 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb9b40a33b0 0x7fb9b40a5870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.846+0000 7fb9d5c00640 1 --2- 172.21.15.123:0/3944830860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d0112b90 0x7fb9d01a3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.846+0000 7fb9d5c00640 1 --2- 172.21.15.123:0/3944830860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9d0074eb0 0x7fb9d019f060 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.846+0000 7fb9d5c00640 1 --2- 172.21.15.123:0/3944830860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9d0074590 0x7fb9d019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:53.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.846+0000 7fb9d5c00640 1 -- 172.21.15.123:0/3944830860 >> 172.21.15.123:0/3944830860 conn(0x7fb9d00fd8c0 msgr2=0x7fb9d00ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:53.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.846+0000 7fb9d5c00640 1 -- 172.21.15.123:0/3944830860 shutdown_connections 2024-09-19T04:53:53.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:53.846+0000 7fb9d5c00640 1 -- 172.21.15.123:0/3944830860 wait complete. 2024-09-19T04:53:53.853 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:53 smithi183 ceph-mon[35333]: pgmap v220: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:54.694 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:53:54.695 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:53:55.064 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:53:55.528 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:55 smithi123 ceph-mon[30511]: pgmap v221: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:53:55.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.640+0000 7f385a200640 1 -- 172.21.15.123:0/229324902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3854102360 msgr2=0x7f385410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:55.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.640+0000 7f385a200640 1 --2- 172.21.15.123:0/229324902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3854102360 0x7f385410e890 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f384c009cd0 tx=0x7f384c02a8f0 comp rx=0 tx=0).stop 2024-09-19T04:53:55.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.640+0000 7f385a200640 1 -- 172.21.15.123:0/229324902 shutdown_connections 2024-09-19T04:53:55.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.640+0000 7f385a200640 1 --2- 172.21.15.123:0/229324902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3854102360 0x7f385410e890 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:55.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.640+0000 7f385a200640 1 --2- 172.21.15.123:0/229324902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38541019c0 0x7f3854101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:55.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.640+0000 7f385a200640 1 --2- 172.21.15.123:0/229324902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38541079c0 0x7f3854107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:55.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.640+0000 7f385a200640 1 -- 172.21.15.123:0/229324902 >> 172.21.15.123:0/229324902 conn(0x7f38540fd740 msgr2=0x7f38540ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:55.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.640+0000 7f385a200640 1 -- 172.21.15.123:0/229324902 shutdown_connections 2024-09-19T04:53:55.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.640+0000 7f385a200640 1 -- 172.21.15.123:0/229324902 wait complete. 2024-09-19T04:53:55.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.641+0000 7f385a200640 1 Processor -- start 2024-09-19T04:53:55.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.642+0000 7f385a200640 1 -- start start 2024-09-19T04:53:55.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.642+0000 7f385a200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38541019c0 0x7f3854198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:55.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.642+0000 7f385a200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3854102360 0x7f38541987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:55.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.642+0000 7f385a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38541079c0 0x7f385419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:55.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.642+0000 7f385a200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3854111e10 con 0x7f38541079c0 2024-09-19T04:53:55.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.642+0000 7f385a200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3854111c90 con 0x7f3854102360 2024-09-19T04:53:55.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.642+0000 7f385a200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3854111f90 con 0x7f38541019c0 2024-09-19T04:53:55.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.642+0000 7f3858e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38541019c0 0x7f3854198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:55.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.643+0000 7f3858e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38541019c0 0x7f3854198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:44810/0 (socket says 172.21.15.123:44810) 2024-09-19T04:53:55.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.643+0000 7f3858e00640 1 -- 172.21.15.123:0/2044068957 learned_addr learned my addr 172.21.15.123:0/2044068957 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:55.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.643+0000 7f3859800640 1 --2- 172.21.15.123:0/2044068957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38541079c0 0x7f385419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:55.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.643+0000 7f3853e00640 1 --2- 172.21.15.123:0/2044068957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3854102360 0x7f38541987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:55.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.643+0000 7f3858e00640 1 -- 172.21.15.123:0/2044068957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3854102360 msgr2=0x7f38541987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:55.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.643+0000 7f3858e00640 1 --2- 172.21.15.123:0/2044068957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3854102360 0x7f38541987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:55.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.643+0000 7f3858e00640 1 -- 172.21.15.123:0/2044068957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38541079c0 msgr2=0x7f385419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:55.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.643+0000 7f3858e00640 1 --2- 172.21.15.123:0/2044068957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38541079c0 0x7f385419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:55.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.643+0000 7f3858e00640 1 -- 172.21.15.123:0/2044068957 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385419d240 con 0x7f38541019c0 2024-09-19T04:53:55.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.643+0000 7f3853e00640 1 --2- 172.21.15.123:0/2044068957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3854102360 0x7f38541987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:53:55.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.643+0000 7f3858e00640 1 --2- 172.21.15.123:0/2044068957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38541019c0 0x7f3854198290 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f384800e9f0 tx=0x7f384800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:55.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.644+0000 7f3851600640 1 -- 172.21.15.123:0/2044068957 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384800cd80 con 0x7f38541019c0 2024-09-19T04:53:55.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.644+0000 7f3851600640 1 -- 172.21.15.123:0/2044068957 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3848010910 con 0x7f38541019c0 2024-09-19T04:53:55.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.644+0000 7f385a200640 1 -- 172.21.15.123:0/2044068957 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385419d530 con 0x7f38541019c0 2024-09-19T04:53:55.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.644+0000 7f3851600640 1 -- 172.21.15.123:0/2044068957 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848018990 con 0x7f38541019c0 2024-09-19T04:53:55.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.644+0000 7f385a200640 1 -- 172.21.15.123:0/2044068957 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3854074420 con 0x7f38541019c0 2024-09-19T04:53:55.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.645+0000 7f385a200640 1 -- 172.21.15.123:0/2044068957 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3854101e20 con 0x7f38541019c0 2024-09-19T04:53:55.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.647+0000 7f3851600640 1 -- 172.21.15.123:0/2044068957 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3848010ac0 con 0x7f38541019c0 2024-09-19T04:53:55.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.648+0000 7f3851600640 1 --2- 172.21.15.123:0/2044068957 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f383c0a3430 0x7f383c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:55.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.648+0000 7f3851600640 1 -- 172.21.15.123:0/2044068957 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f38480d0470 con 0x7f38541019c0 2024-09-19T04:53:55.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.648+0000 7f3853e00640 1 --2- 172.21.15.123:0/2044068957 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f383c0a3430 0x7f383c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:55.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.649+0000 7f3853e00640 1 --2- 172.21.15.123:0/2044068957 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f383c0a3430 0x7f383c0a58f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f38540ffc70 tx=0x7f3840005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:55.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.650+0000 7f3851600640 1 -- 172.21.15.123:0/2044068957 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f384809cdf0 con 0x7f38541019c0 2024-09-19T04:53:55.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:55 smithi193 ceph-mon[35592]: pgmap v221: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:53:55.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:55 smithi183 ceph-mon[35333]: pgmap v221: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:53:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.869+0000 7f385a200640 1 -- 172.21.15.123:0/2044068957 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3854074680 con 0x7f38541019c0 2024-09-19T04:53:55.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.870+0000 7f3851600640 1 -- 172.21.15.123:0/2044068957 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11994 (secure 0 0 0) 0x7f384809c6f0 con 0x7f38541019c0 2024-09-19T04:53:55.872 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:53:55.872 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":40,"btime":"2024-09-19T04:52:39:230121+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:52:38.841872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24499,24500,24539,24505,24529]},"id":1}]} 2024-09-19T04:53:55.872 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 40 2024-09-19T04:53:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.872+0000 7f385a200640 1 -- 172.21.15.123:0/2044068957 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f383c0a3430 msgr2=0x7f383c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.872+0000 7f385a200640 1 --2- 172.21.15.123:0/2044068957 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f383c0a3430 0x7f383c0a58f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f38540ffc70 tx=0x7f3840005f90 comp rx=0 tx=0).stop 2024-09-19T04:53:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.872+0000 7f385a200640 1 -- 172.21.15.123:0/2044068957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38541019c0 msgr2=0x7f3854198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.872+0000 7f385a200640 1 --2- 172.21.15.123:0/2044068957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38541019c0 0x7f3854198290 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f384800e9f0 tx=0x7f384800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:53:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.872+0000 7f385a200640 1 -- 172.21.15.123:0/2044068957 shutdown_connections 2024-09-19T04:53:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.872+0000 7f385a200640 1 --2- 172.21.15.123:0/2044068957 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f383c0a3430 0x7f383c0a58f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:55.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.872+0000 7f385a200640 1 --2- 172.21.15.123:0/2044068957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38541079c0 0x7f385419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:55.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.872+0000 7f385a200640 1 --2- 172.21.15.123:0/2044068957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3854102360 0x7f38541987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:55.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.872+0000 7f385a200640 1 --2- 172.21.15.123:0/2044068957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38541019c0 0x7f3854198290 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:55.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.872+0000 7f385a200640 1 -- 172.21.15.123:0/2044068957 >> 172.21.15.123:0/2044068957 conn(0x7f38540fd740 msgr2=0x7f38540fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:55.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.873+0000 7f385a200640 1 -- 172.21.15.123:0/2044068957 shutdown_connections 2024-09-19T04:53:55.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:55.873+0000 7f385a200640 1 -- 172.21.15.123:0/2044068957 wait complete. 2024-09-19T04:53:56.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:56 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2044068957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:53:56.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:56 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2044068957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:53:56.773 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:53:56.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:56 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2044068957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:53:57.150 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:53:57.638 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:57 smithi123 ceph-mon[30511]: pgmap v222: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:57.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:57 smithi193 ceph-mon[35592]: pgmap v222: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:57.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.748+0000 7f3410200640 1 -- 172.21.15.123:0/3881430438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3408102790 msgr2=0x7f34081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:57.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.748+0000 7f3410200640 1 --2- 172.21.15.123:0/3881430438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3408102790 0x7f34081067e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009900 tx=0x7f33f402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.749+0000 7f3410200640 1 -- 172.21.15.123:0/3881430438 shutdown_connections 2024-09-19T04:53:57.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.749+0000 7f3410200640 1 --2- 172.21.15.123:0/3881430438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3408106f10 0x7f340810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.749+0000 7f3410200640 1 --2- 172.21.15.123:0/3881430438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3408102790 0x7f34081067e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.749+0000 7f3410200640 1 --2- 172.21.15.123:0/3881430438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3408101de0 0x7f34081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.749+0000 7f3410200640 1 -- 172.21.15.123:0/3881430438 >> 172.21.15.123:0/3881430438 conn(0x7f34080fd8c0 msgr2=0x7f34080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:57.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.749+0000 7f3410200640 1 -- 172.21.15.123:0/3881430438 shutdown_connections 2024-09-19T04:53:57.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.749+0000 7f3410200640 1 -- 172.21.15.123:0/3881430438 wait complete. 2024-09-19T04:53:57.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:57 smithi183 ceph-mon[35333]: pgmap v222: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:53:57.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.749+0000 7f3410200640 1 Processor -- start 2024-09-19T04:53:57.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.750+0000 7f3410200640 1 -- start start 2024-09-19T04:53:57.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.750+0000 7f3410200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3408101de0 0x7f3408198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.750+0000 7f3410200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3408102790 0x7f3408198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.750+0000 7f3410200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3408106f10 0x7f340819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.750+0000 7f3410200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3408111fc0 con 0x7f3408106f10 2024-09-19T04:53:57.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.750+0000 7f3410200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3408111e40 con 0x7f3408101de0 2024-09-19T04:53:57.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.750+0000 7f3410200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3408112140 con 0x7f3408102790 2024-09-19T04:53:57.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.750+0000 7f340ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3408102790 0x7f3408198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.750+0000 7f340e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3408106f10 0x7f340819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.750+0000 7f340d800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3408101de0 0x7f3408198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.750+0000 7f340ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3408102790 0x7f3408198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:44824/0 (socket says 172.21.15.123:44824) 2024-09-19T04:53:57.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.750+0000 7f340ce00640 1 -- 172.21.15.123:0/1321422023 learned_addr learned my addr 172.21.15.123:0/1321422023 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:57.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.751+0000 7f340d800640 1 -- 172.21.15.123:0/1321422023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3408102790 msgr2=0x7f3408198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:57.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.751+0000 7f340d800640 1 --2- 172.21.15.123:0/1321422023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3408102790 0x7f3408198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.751+0000 7f340d800640 1 -- 172.21.15.123:0/1321422023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3408106f10 msgr2=0x7f340819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:57.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.751+0000 7f340d800640 1 --2- 172.21.15.123:0/1321422023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3408106f10 0x7f340819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.751+0000 7f340d800640 1 -- 172.21.15.123:0/1321422023 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340819d3b0 con 0x7f3408101de0 2024-09-19T04:53:57.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.751+0000 7f340e200640 1 --2- 172.21.15.123:0/1321422023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3408106f10 0x7f340819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:57.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.751+0000 7f340d800640 1 --2- 172.21.15.123:0/1321422023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3408101de0 0x7f3408198400 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00e9f0 tx=0x7f33fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:57.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.751+0000 7f3406000640 1 -- 172.21.15.123:0/1321422023 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc00ccc0 con 0x7f3408101de0 2024-09-19T04:53:57.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.751+0000 7f3406000640 1 -- 172.21.15.123:0/1321422023 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33fc010910 con 0x7f3408101de0 2024-09-19T04:53:57.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.751+0000 7f3410200640 1 -- 172.21.15.123:0/1321422023 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340819d6a0 con 0x7f3408101de0 2024-09-19T04:53:57.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.751+0000 7f3406000640 1 -- 172.21.15.123:0/1321422023 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc018960 con 0x7f3408101de0 2024-09-19T04:53:57.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.752+0000 7f3410200640 1 -- 172.21.15.123:0/1321422023 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34081a4f80 con 0x7f3408101de0 2024-09-19T04:53:57.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.755+0000 7f3406000640 1 -- 172.21.15.123:0/1321422023 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f33fc010ac0 con 0x7f3408101de0 2024-09-19T04:53:57.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.755+0000 7f3412bbd740 1 Processor -- start 2024-09-19T04:53:57.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.755+0000 7f3412bbd740 1 -- start start 2024-09-19T04:53:57.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.755+0000 7f3412bbd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c3fb70 0x562f11c3ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.755+0000 7f3412bbd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c40490 0x562f11c4ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.755+0000 7f3412bbd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c4f260 0x562f11c51650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.755+0000 7f3412bbd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562f11ba63e0 con 0x562f11c3fb70 2024-09-19T04:53:57.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.755+0000 7f3412bbd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562f11b4dd80 con 0x562f11c40490 2024-09-19T04:53:57.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.755+0000 7f3412bbd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562f11ba23a0 con 0x562f11c4f260 2024-09-19T04:53:57.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.755+0000 7f340e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c3fb70 0x562f11c3ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.755+0000 7f340d800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c40490 0x562f11c4ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.755+0000 7f340ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c4f260 0x562f11c51650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.755+0000 7f340e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c3fb70 0x562f11c3ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48838/0 (socket says 172.21.15.123:48838) 2024-09-19T04:53:57.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.755+0000 7f340e200640 1 -- 172.21.15.123:0/210308196 learned_addr learned my addr 172.21.15.123:0/210308196 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:57.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.756+0000 7f340ce00640 1 -- 172.21.15.123:0/210308196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c40490 msgr2=0x562f11c4ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:57.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.756+0000 7f340ce00640 1 --2- 172.21.15.123:0/210308196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c40490 0x562f11c4ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.756+0000 7f340ce00640 1 -- 172.21.15.123:0/210308196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c3fb70 msgr2=0x562f11c3ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:57.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.756+0000 7f340ce00640 1 --2- 172.21.15.123:0/210308196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c3fb70 0x562f11c3ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.756+0000 7f340ce00640 1 -- 172.21.15.123:0/210308196 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f11b6cae0 con 0x562f11c4f260 2024-09-19T04:53:57.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.756+0000 7f3406000640 1 --2- 172.21.15.123:0/1321422023 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f33f00a33b0 0x7f33f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.756+0000 7f340ce00640 1 --2- 172.21.15.123:0/210308196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c4f260 0x562f11c51650 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f33f4000c00 tx=0x7f33f4005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b3682fb22c9a14d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:57.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.756+0000 7f3406000640 1 -- 172.21.15.123:0/1321422023 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f33fc0d00f0 con 0x7f3408101de0 2024-09-19T04:53:57.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.756+0000 7f33faa00640 1 -- 172.21.15.123:0/210308196 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4040070 con 0x562f11c4f260 2024-09-19T04:53:57.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.757+0000 7f340e200640 1 --2- 172.21.15.123:0/1321422023 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f33f00a33b0 0x7f33f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.757+0000 7f33faa00640 1 -- 172.21.15.123:0/210308196 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f4005500 con 0x562f11c4f260 2024-09-19T04:53:57.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.757+0000 7f33faa00640 1 -- 172.21.15.123:0/210308196 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4035040 con 0x562f11c4f260 2024-09-19T04:53:57.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.758+0000 7f3412bbd740 1 -- 172.21.15.123:0/210308196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c4f260 msgr2=0x562f11c51650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:57.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.758+0000 7f340e200640 1 --2- 172.21.15.123:0/1321422023 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f33f00a33b0 0x7f33f00a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3400009fd0 tx=0x7f3400009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:57.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.758+0000 7f3412bbd740 1 --2- 172.21.15.123:0/210308196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c4f260 0x562f11c51650 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f33f4000c00 tx=0x7f33f4005c00 comp rx=0 tx=0).stop 2024-09-19T04:53:57.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.758+0000 7f3412bbd740 1 -- 172.21.15.123:0/210308196 shutdown_connections 2024-09-19T04:53:57.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.758+0000 7f3412bbd740 1 --2- 172.21.15.123:0/210308196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c4f260 0x562f11c51650 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.758+0000 7f3412bbd740 1 --2- 172.21.15.123:0/210308196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c40490 0x562f11c4ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.758+0000 7f3412bbd740 1 --2- 172.21.15.123:0/210308196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c3fb70 0x562f11c3ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.758+0000 7f3412bbd740 1 -- 172.21.15.123:0/210308196 >> 172.21.15.123:0/210308196 conn(0x562f11c3c200 msgr2=0x562f11c3c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:57.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.758+0000 7f3412bbd740 1 -- 172.21.15.123:0/210308196 shutdown_connections 2024-09-19T04:53:57.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.758+0000 7f3412bbd740 1 -- 172.21.15.123:0/210308196 wait complete. 2024-09-19T04:53:57.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.758+0000 7f3412bbd740 1 -- start start 2024-09-19T04:53:57.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.758+0000 7f3412bbd740 1 Processor -- start 2024-09-19T04:53:57.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.759+0000 7f3412bbd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c3fb70 0x562f11c4c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.759+0000 7f3412bbd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c40490 0x562f11c4c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.759+0000 7f3412bbd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c4f260 0x562f11c4cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.759+0000 7f3412bbd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562f11c51e30 con 0x562f11c3fb70 2024-09-19T04:53:57.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.759+0000 7f3412bbd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562f11b610f0 con 0x562f11c4f260 2024-09-19T04:53:57.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.759+0000 7f3412bbd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562f11c51fb0 con 0x562f11c40490 2024-09-19T04:53:57.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.759+0000 7f3412bbd740 10 client.0 _ll_register_callbacks cb 0x562f11b48bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:57.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.759+0000 7f340d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c3fb70 0x562f11c4c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.759+0000 7f340d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c3fb70 0x562f11c4c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48852/0 (socket says 172.21.15.123:48852) 2024-09-19T04:53:57.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.759+0000 7f340d800640 1 -- 172.21.15.123:0/3479700229 learned_addr learned my addr 172.21.15.123:0/3479700229 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:57.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.759+0000 7f340e200640 1 --2- 172.21.15.123:0/3479700229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c4f260 0x562f11c4cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.759+0000 7f340ce00640 1 --2- 172.21.15.123:0/3479700229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c40490 0x562f11c4c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.760+0000 7f340d800640 1 -- 172.21.15.123:0/3479700229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c40490 msgr2=0x562f11c4c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:57.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.760+0000 7f340d800640 1 --2- 172.21.15.123:0/3479700229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c40490 0x562f11c4c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.760+0000 7f340d800640 1 -- 172.21.15.123:0/3479700229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c4f260 msgr2=0x562f11c4cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:57.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.760+0000 7f340d800640 1 --2- 172.21.15.123:0/3479700229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c4f260 0x562f11c4cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.760+0000 7f340d800640 1 -- 172.21.15.123:0/3479700229 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f119173c0 con 0x562f11c3fb70 2024-09-19T04:53:57.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.760+0000 7f340ce00640 1 --2- 172.21.15.123:0/3479700229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c40490 0x562f11c4c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:57.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.760+0000 7f340d800640 1 --2- 172.21.15.123:0/3479700229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c3fb70 0x562f11c4c130 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00c6a0 tx=0x7f33fc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d75e9599a9386a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:57.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.760+0000 7f33ef400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:53:57.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.760+0000 7f33ef400640 1 -- 172.21.15.123:0/3479700229 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc09bf60 con 0x562f11c3fb70 2024-09-19T04:53:57.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.760+0000 7f33ef400640 1 -- 172.21.15.123:0/3479700229 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f33fc0d2070 con 0x562f11c3fb70 2024-09-19T04:53:57.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.760+0000 7f3412bbd740 1 -- 172.21.15.123:0/3479700229 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33f4005500 con 0x562f11c3fb70 2024-09-19T04:53:57.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.760+0000 7f33ef400640 1 -- 172.21.15.123:0/3479700229 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f33fc09bd80 con 0x562f11c3fb70 2024-09-19T04:53:57.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.761+0000 7f3412bbd740 10 client.16515 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:57.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.761+0000 7f3412bbd740 1 -- 172.21.15.123:0/3479700229 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562f11c5dec0 con 0x562f11c3fb70 2024-09-19T04:53:57.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.761+0000 7f33ef400640 1 -- 172.21.15.123:0/3479700229 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f4005500 con 0x562f11c3fb70 2024-09-19T04:53:57.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.761+0000 7f33ef400640 1 -- 172.21.15.123:0/3479700229 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f33fc0b51f0 con 0x562f11c3fb70 2024-09-19T04:53:57.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.761+0000 7f3412bbd740 10 client.16515 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:57.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.761+0000 7f3412bbd740 10 client.16515 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:53:57.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.761+0000 7f3412bbd740 20 client.16515 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:57.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.761+0000 7f3412bbd740 1 --2- 172.21.15.123:0/3479700229 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x562f11c5f380 0x562f11c61790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.761+0000 7f3412bbd740 4 client.16515 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:53:57.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.761+0000 7f3412bbd740 1 -- 172.21.15.123:0/3479700229 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f11bb3950 con 0x562f11c5f380 2024-09-19T04:53:57.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.762+0000 7f340ce00640 1 --2- 172.21.15.123:0/3479700229 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x562f11c5f380 0x562f11c61790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.762+0000 7f340ce00640 1 --2- 172.21.15.123:0/3479700229 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x562f11c5f380 0x562f11c61790 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7c87bb6874246db0 server_cookie=a0ea5ee8f2531af4 in_seq=0 out_seq=0 2024-09-19T04:53:57.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.762+0000 7f33ef400640 10 client.16515 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:53:57.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.763+0000 7f33ef400640 1 -- 172.21.15.123:0/3479700229 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f11bb3950 con 0x562f11c5f380 2024-09-19T04:53:57.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.763+0000 7f33ef400640 10 client.16515 handle_command_reply: tid=0 2024-09-19T04:53:57.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.763+0000 7f3412bbd740 1 client.16515 shutdown 2024-09-19T04:53:57.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.763+0000 7f3412bbd740 1 -- 172.21.15.123:0/3479700229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c3fb70 msgr2=0x562f11c4c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:57.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.763+0000 7f3412bbd740 1 --2- 172.21.15.123:0/3479700229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c3fb70 0x562f11c4c130 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00c6a0 tx=0x7f33fc00cf60 comp rx=0 tx=0).stop 2024-09-19T04:53:57.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.763+0000 7f3412bbd740 1 -- 172.21.15.123:0/3479700229 shutdown_connections 2024-09-19T04:53:57.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.763+0000 7f3412bbd740 1 --2- 172.21.15.123:0/3479700229 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x562f11c5f380 0x562f11c61790 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.763+0000 7f3412bbd740 1 --2- 172.21.15.123:0/3479700229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c4f260 0x562f11c4cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.763+0000 7f3412bbd740 1 --2- 172.21.15.123:0/3479700229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c40490 0x562f11c4c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.763+0000 7f3412bbd740 1 --2- 172.21.15.123:0/3479700229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c3fb70 0x562f11c4c130 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.763+0000 7f3412bbd740 1 -- 172.21.15.123:0/3479700229 >> 172.21.15.123:0/3479700229 conn(0x562f11c3c200 msgr2=0x562f11c41f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:57.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.763+0000 7f33ef400640 0 client.16515 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:53:57.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.764+0000 7f3412bbd740 1 -- 172.21.15.123:0/3479700229 shutdown_connections 2024-09-19T04:53:57.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.764+0000 7f3412bbd740 1 -- 172.21.15.123:0/3479700229 wait complete. 2024-09-19T04:53:57.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.764+0000 7f3412bbd740 20 client.16515 trim_cache size 0 max 16384 2024-09-19T04:53:57.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.777+0000 7f3412bbd740 1 Processor -- start 2024-09-19T04:53:57.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.777+0000 7f3412bbd740 1 -- start start 2024-09-19T04:53:57.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.778+0000 7f3412bbd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c3fb70 0x562f11c4e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.778+0000 7f3412bbd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c40490 0x562f11c67060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.778+0000 7f3412bbd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c4f260 0x562f11c675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.778+0000 7f3412bbd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f34081ab6c0 con 0x562f11c4f260 2024-09-19T04:53:57.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.778+0000 7f3412bbd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34081ad290 con 0x562f11c40490 2024-09-19T04:53:57.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.778+0000 7f3412bbd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f34081abdc0 con 0x562f11c3fb70 2024-09-19T04:53:57.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.778+0000 7f340e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c4f260 0x562f11c675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.778+0000 7f340e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c4f260 0x562f11c675a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48856/0 (socket says 172.21.15.123:48856) 2024-09-19T04:53:57.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.778+0000 7f340e200640 1 -- 172.21.15.123:0/878020039 learned_addr learned my addr 172.21.15.123:0/878020039 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:57.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.778+0000 7f340ce00640 1 --2- 172.21.15.123:0/878020039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c40490 0x562f11c67060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.778+0000 7f340d800640 1 --2- 172.21.15.123:0/878020039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c3fb70 0x562f11c4e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.778+0000 7f340e200640 1 -- 172.21.15.123:0/878020039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c3fb70 msgr2=0x562f11c4e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:57.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.778+0000 7f340e200640 1 --2- 172.21.15.123:0/878020039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c3fb70 0x562f11c4e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.778+0000 7f340e200640 1 -- 172.21.15.123:0/878020039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c40490 msgr2=0x562f11c67060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:57.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.778+0000 7f340e200640 1 --2- 172.21.15.123:0/878020039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c40490 0x562f11c67060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.778+0000 7f340e200640 1 -- 172.21.15.123:0/878020039 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f11c3c030 con 0x562f11c4f260 2024-09-19T04:53:57.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.778+0000 7f340ce00640 1 --2- 172.21.15.123:0/878020039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c40490 0x562f11c67060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:57.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.779+0000 7f340e200640 1 --2- 172.21.15.123:0/878020039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c4f260 0x562f11c675a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3400012520 tx=0x7f340001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d3a56524d1d085 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:57.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.779+0000 7f33fa000640 1 -- 172.21.15.123:0/878020039 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34000119c0 con 0x562f11c4f260 2024-09-19T04:53:57.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.779+0000 7f33fa000640 1 -- 172.21.15.123:0/878020039 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3400003e50 con 0x562f11c4f260 2024-09-19T04:53:57.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.779+0000 7f33fa000640 1 -- 172.21.15.123:0/878020039 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400004730 con 0x562f11c4f260 2024-09-19T04:53:57.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.780+0000 7f3412bbd740 1 -- 172.21.15.123:0/878020039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c4f260 msgr2=0x562f11c675a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:57.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.780+0000 7f3412bbd740 1 --2- 172.21.15.123:0/878020039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c4f260 0x562f11c675a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3400012520 tx=0x7f340001a000 comp rx=0 tx=0).stop 2024-09-19T04:53:57.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.780+0000 7f3412bbd740 1 -- 172.21.15.123:0/878020039 shutdown_connections 2024-09-19T04:53:57.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.780+0000 7f3412bbd740 1 --2- 172.21.15.123:0/878020039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c4f260 0x562f11c675a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.780+0000 7f3412bbd740 1 --2- 172.21.15.123:0/878020039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c40490 0x562f11c67060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.780+0000 7f3412bbd740 1 --2- 172.21.15.123:0/878020039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c3fb70 0x562f11c4e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.780+0000 7f3412bbd740 1 -- 172.21.15.123:0/878020039 >> 172.21.15.123:0/878020039 conn(0x562f11c3c200 msgr2=0x562f11c4d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:57.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.780+0000 7f3412bbd740 1 -- 172.21.15.123:0/878020039 shutdown_connections 2024-09-19T04:53:57.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.780+0000 7f3412bbd740 1 -- 172.21.15.123:0/878020039 wait complete. 2024-09-19T04:53:57.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.780+0000 7f3412bbd740 1 -- start start 2024-09-19T04:53:57.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.780+0000 7f3412bbd740 1 Processor -- start 2024-09-19T04:53:57.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.781+0000 7f3412bbd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c3fb70 0x562f11c53420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.781+0000 7f3412bbd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c40490 0x562f11c53960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.781+0000 7f3412bbd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c4f260 0x562f11cc9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.781+0000 7f3412bbd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f34081acbc0 con 0x562f11c40490 2024-09-19T04:53:57.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.781+0000 7f3412bbd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34081ac4c0 con 0x562f11c3fb70 2024-09-19T04:53:57.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.781+0000 7f3412bbd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f34081a96c0 con 0x562f11c4f260 2024-09-19T04:53:57.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.781+0000 7f3412bbd740 10 client.0 _ll_register_callbacks cb 0x7f33fc0d3890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:57.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.781+0000 7f340ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c40490 0x562f11c53960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.781+0000 7f340ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c40490 0x562f11c53960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48870/0 (socket says 172.21.15.123:48870) 2024-09-19T04:53:57.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.781+0000 7f340ce00640 1 -- 172.21.15.123:0/323479967 learned_addr learned my addr 172.21.15.123:0/323479967 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:57.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.781+0000 7f340d800640 1 --2- 172.21.15.123:0/323479967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c3fb70 0x562f11c53420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.781+0000 7f340e200640 1 --2- 172.21.15.123:0/323479967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c4f260 0x562f11cc9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.782+0000 7f340d800640 1 -- 172.21.15.123:0/323479967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c4f260 msgr2=0x562f11cc9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:57.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.782+0000 7f340d800640 1 --2- 172.21.15.123:0/323479967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c4f260 0x562f11cc9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.782+0000 7f340d800640 1 -- 172.21.15.123:0/323479967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c40490 msgr2=0x562f11c53960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:57.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.782+0000 7f340d800640 1 --2- 172.21.15.123:0/323479967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c40490 0x562f11c53960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.782+0000 7f340d800640 1 -- 172.21.15.123:0/323479967 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f11c37770 con 0x562f11c3fb70 2024-09-19T04:53:57.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.782+0000 7f340ce00640 1 --2- 172.21.15.123:0/323479967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c40490 0x562f11c53960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:53:57.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.782+0000 7f340d800640 1 --2- 172.21.15.123:0/323479967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c3fb70 0x562f11c53420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f33fc098880 tx=0x7f33fc09b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a3f11e23c642aad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:57.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.782+0000 7f33ef400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:53:57.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.782+0000 7f33ef400640 1 -- 172.21.15.123:0/323479967 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc09bf60 con 0x562f11c3fb70 2024-09-19T04:53:57.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.782+0000 7f33ef400640 1 -- 172.21.15.123:0/323479967 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f33fc010630 con 0x562f11c3fb70 2024-09-19T04:53:57.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.782+0000 7f3412bbd740 1 -- 172.21.15.123:0/323479967 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3400003e50 con 0x562f11c3fb70 2024-09-19T04:53:57.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.782+0000 7f33ef400640 1 -- 172.21.15.123:0/323479967 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f33fc09c420 con 0x562f11c3fb70 2024-09-19T04:53:57.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.782+0000 7f3412bbd740 10 client.26621 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:57.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.782+0000 7f3412bbd740 1 -- 172.21.15.123:0/323479967 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562f11cca4f0 con 0x562f11c3fb70 2024-09-19T04:53:57.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.783+0000 7f33ef400640 1 -- 172.21.15.123:0/323479967 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3400003e50 con 0x562f11c3fb70 2024-09-19T04:53:57.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.783+0000 7f33ef400640 1 -- 172.21.15.123:0/323479967 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f33fc098170 con 0x562f11c3fb70 2024-09-19T04:53:57.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.783+0000 7f3412bbd740 10 client.26621 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:57.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.783+0000 7f3412bbd740 10 client.26621 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:53:57.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.783+0000 7f3412bbd740 20 client.26621 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:57.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.783+0000 7f3412bbd740 1 --2- 172.21.15.123:0/323479967 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x562f11ccaab0 0x562f11ccced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:57.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.783+0000 7f3412bbd740 4 client.26621 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:53:57.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.783+0000 7f3412bbd740 1 -- 172.21.15.123:0/323479967 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f11bade60 con 0x562f11ccaab0 2024-09-19T04:53:57.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.783+0000 7f340ce00640 1 --2- 172.21.15.123:0/323479967 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x562f11ccaab0 0x562f11ccced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:57.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.784+0000 7f340ce00640 1 --2- 172.21.15.123:0/323479967 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x562f11ccaab0 0x562f11ccced0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=30177b94eb07acb3 server_cookie=63178c073177bce9 in_seq=0 out_seq=0 2024-09-19T04:53:57.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.784+0000 7f33ef400640 10 client.26621 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:53:57.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.784+0000 7f33ef400640 1 -- 172.21.15.123:0/323479967 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f11bade60 con 0x562f11ccaab0 2024-09-19T04:53:57.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.784+0000 7f33ef400640 10 client.26621 handle_command_reply: tid=0 2024-09-19T04:53:57.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.784+0000 7f3412bbd740 1 client.26621 shutdown 2024-09-19T04:53:57.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.784+0000 7f3412bbd740 1 -- 172.21.15.123:0/323479967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c3fb70 msgr2=0x562f11c53420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:57.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.784+0000 7f3412bbd740 1 --2- 172.21.15.123:0/323479967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c3fb70 0x562f11c53420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f33fc098880 tx=0x7f33fc09b8c0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.784+0000 7f3412bbd740 1 -- 172.21.15.123:0/323479967 shutdown_connections 2024-09-19T04:53:57.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.784+0000 7f3412bbd740 1 --2- 172.21.15.123:0/323479967 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x562f11ccaab0 0x562f11ccced0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.784+0000 7f3412bbd740 1 --2- 172.21.15.123:0/323479967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f11c4f260 0x562f11cc9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.784+0000 7f3412bbd740 1 --2- 172.21.15.123:0/323479967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f11c40490 0x562f11c53960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.784+0000 7f33ef400640 0 client.26621 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:53:57.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.784+0000 7f3412bbd740 1 --2- 172.21.15.123:0/323479967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562f11c3fb70 0x562f11c53420 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.784+0000 7f3412bbd740 1 -- 172.21.15.123:0/323479967 >> 172.21.15.123:0/323479967 conn(0x562f11c3c200 msgr2=0x562f11c4c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:57.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.784+0000 7f3412bbd740 1 -- 172.21.15.123:0/323479967 shutdown_connections 2024-09-19T04:53:57.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.784+0000 7f3412bbd740 1 -- 172.21.15.123:0/323479967 wait complete. 2024-09-19T04:53:57.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.785+0000 7f3412bbd740 20 client.26621 trim_cache size 0 max 16384 2024-09-19T04:53:57.790 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:53:57.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.785+0000 7f3410200640 1 -- 172.21.15.123:0/1321422023 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f33f00a33b0 msgr2=0x7f33f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:57.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.785+0000 7f3410200640 1 --2- 172.21.15.123:0/1321422023 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f33f00a33b0 0x7f33f00a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3400009fd0 tx=0x7f3400009500 comp rx=0 tx=0).stop 2024-09-19T04:53:57.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.785+0000 7f3410200640 1 -- 172.21.15.123:0/1321422023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3408101de0 msgr2=0x7f3408198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:57.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.785+0000 7f3410200640 1 --2- 172.21.15.123:0/1321422023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3408101de0 0x7f3408198400 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00e9f0 tx=0x7f33fc00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.785+0000 7f3410200640 1 -- 172.21.15.123:0/1321422023 shutdown_connections 2024-09-19T04:53:57.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.785+0000 7f3410200640 1 --2- 172.21.15.123:0/1321422023 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f33f00a33b0 0x7f33f00a5870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.785+0000 7f3410200640 1 --2- 172.21.15.123:0/1321422023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3408106f10 0x7f340819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.785+0000 7f3410200640 1 --2- 172.21.15.123:0/1321422023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3408102790 0x7f3408198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.785+0000 7f3410200640 1 --2- 172.21.15.123:0/1321422023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3408101de0 0x7f3408198400 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:57.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.785+0000 7f3410200640 1 -- 172.21.15.123:0/1321422023 >> 172.21.15.123:0/1321422023 conn(0x7f34080fd8c0 msgr2=0x7f34080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:57.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.786+0000 7f3410200640 1 -- 172.21.15.123:0/1321422023 shutdown_connections 2024-09-19T04:53:57.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:57.786+0000 7f3410200640 1 -- 172.21.15.123:0/1321422023 wait complete. 2024-09-19T04:53:58.692 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:53:59.065 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:53:59.497 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:53:59 smithi123 ceph-mon[30511]: pgmap v223: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:53:59.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.608+0000 7f32a8e00640 1 -- 172.21.15.123:0/3066619321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a410c170 msgr2=0x7f32a410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:59.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.608+0000 7f32a8e00640 1 --2- 172.21.15.123:0/3066619321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a410c170 0x7f32a410e610 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f329c009ca0 tx=0x7f329c02a950 comp rx=0 tx=0).stop 2024-09-19T04:53:59.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.608+0000 7f32a8e00640 1 -- 172.21.15.123:0/3066619321 shutdown_connections 2024-09-19T04:53:59.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.608+0000 7f32a8e00640 1 --2- 172.21.15.123:0/3066619321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a410c170 0x7f32a410e610 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.608+0000 7f32a8e00640 1 --2- 172.21.15.123:0/3066619321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32a4103660 0x7f32a410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.608+0000 7f32a8e00640 1 --2- 172.21.15.123:0/3066619321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32a4102d40 0x7f32a4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.608+0000 7f32a8e00640 1 -- 172.21.15.123:0/3066619321 >> 172.21.15.123:0/3066619321 conn(0x7f32a40fc7a0 msgr2=0x7f32a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.608+0000 7f32a8e00640 1 -- 172.21.15.123:0/3066619321 shutdown_connections 2024-09-19T04:53:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.608+0000 7f32a8e00640 1 -- 172.21.15.123:0/3066619321 wait complete. 2024-09-19T04:53:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.609+0000 7f32a8e00640 1 Processor -- start 2024-09-19T04:53:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.609+0000 7f32a8e00640 1 -- start start 2024-09-19T04:53:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.609+0000 7f32a8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a4102d40 0x7f32a419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.609+0000 7f32a8e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32a4103660 0x7f32a419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.609+0000 7f32a8e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32a410c170 0x7f32a419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.609+0000 7f32a8e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32a4111f50 con 0x7f32a4102d40 2024-09-19T04:53:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.609+0000 7f32a8e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32a4111dd0 con 0x7f32a410c170 2024-09-19T04:53:59.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.609+0000 7f32a8e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41120d0 con 0x7f32a4103660 2024-09-19T04:53:59.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.610+0000 7f32a1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a4102d40 0x7f32a419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.610+0000 7f32a1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a4102d40 0x7f32a419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48888/0 (socket says 172.21.15.123:48888) 2024-09-19T04:53:59.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.610+0000 7f32a1e00640 1 -- 172.21.15.123:0/1812894043 learned_addr learned my addr 172.21.15.123:0/1812894043 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:59.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.610+0000 7f3299400640 1 --2- 172.21.15.123:0/1812894043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32a4103660 0x7f32a419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.610+0000 7f32a2800640 1 --2- 172.21.15.123:0/1812894043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32a410c170 0x7f32a419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.610+0000 7f32a1e00640 1 -- 172.21.15.123:0/1812894043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32a4103660 msgr2=0x7f32a419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:59.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.610+0000 7f32a1e00640 1 --2- 172.21.15.123:0/1812894043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32a4103660 0x7f32a419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.610+0000 7f32a1e00640 1 -- 172.21.15.123:0/1812894043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32a410c170 msgr2=0x7f32a419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:59.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.610+0000 7f32a1e00640 1 --2- 172.21.15.123:0/1812894043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32a410c170 0x7f32a419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.610+0000 7f32a1e00640 1 -- 172.21.15.123:0/1812894043 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a419f570 con 0x7f32a4102d40 2024-09-19T04:53:59.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.610+0000 7f32a2800640 1 --2- 172.21.15.123:0/1812894043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32a410c170 0x7f32a419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:53:59.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.610+0000 7f32a1e00640 1 --2- 172.21.15.123:0/1812894043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a4102d40 0x7f32a419a5c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f329400e9f0 tx=0x7f329400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:59.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.611+0000 7f329b400640 1 -- 172.21.15.123:0/1812894043 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329400cd80 con 0x7f32a4102d40 2024-09-19T04:53:59.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.611+0000 7f329b400640 1 -- 172.21.15.123:0/1812894043 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3294010910 con 0x7f32a4102d40 2024-09-19T04:53:59.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.611+0000 7f32a8e00640 1 -- 172.21.15.123:0/1812894043 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a419f860 con 0x7f32a4102d40 2024-09-19T04:53:59.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.611+0000 7f329b400640 1 -- 172.21.15.123:0/1812894043 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294018990 con 0x7f32a4102d40 2024-09-19T04:53:59.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.611+0000 7f32a8e00640 1 -- 172.21.15.123:0/1812894043 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a41a7140 con 0x7f32a4102d40 2024-09-19T04:53:59.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.615+0000 7f32ab89a740 1 Processor -- start 2024-09-19T04:53:59.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.615+0000 7f32ab89a740 1 -- start start 2024-09-19T04:53:59.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.615+0000 7f329b400640 1 -- 172.21.15.123:0/1812894043 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3294010ac0 con 0x7f32a4102d40 2024-09-19T04:53:59.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.615+0000 7f32ab89a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d273c70 0x560a2d274050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.615+0000 7f32ab89a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d274590 0x560a2d27e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.615+0000 7f32ab89a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d27ee50 0x560a2d281240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.615+0000 7f32ab89a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560a2d1da3e0 con 0x560a2d27ee50 2024-09-19T04:53:59.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.615+0000 7f32ab89a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560a2d181d80 con 0x560a2d273c70 2024-09-19T04:53:59.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.615+0000 7f32ab89a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560a2d1d63a0 con 0x560a2d274590 2024-09-19T04:53:59.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.616+0000 7f32a2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d273c70 0x560a2d274050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.616+0000 7f32a2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d273c70 0x560a2d274050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:47128/0 (socket says 172.21.15.123:47128) 2024-09-19T04:53:59.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.616+0000 7f3299400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d27ee50 0x560a2d281240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.616+0000 7f32a2800640 1 -- 172.21.15.123:0/2623138720 learned_addr learned my addr 172.21.15.123:0/2623138720 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:59.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.616+0000 7f32a1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d274590 0x560a2d27e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.616+0000 7f32a2800640 1 -- 172.21.15.123:0/2623138720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d274590 msgr2=0x560a2d27e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:59.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.616+0000 7f32a2800640 1 --2- 172.21.15.123:0/2623138720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d274590 0x560a2d27e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.616+0000 7f32a2800640 1 -- 172.21.15.123:0/2623138720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d27ee50 msgr2=0x560a2d281240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:59.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.616+0000 7f32a2800640 1 --2- 172.21.15.123:0/2623138720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d27ee50 0x560a2d281240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.616+0000 7f32a2800640 1 -- 172.21.15.123:0/2623138720 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a2d1a0ae0 con 0x560a2d273c70 2024-09-19T04:53:59.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.616+0000 7f32a2800640 1 --2- 172.21.15.123:0/2623138720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d273c70 0x560a2d274050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f329c009830 tx=0x7f329c009860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb4f5db4e1e8165b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:59.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.617+0000 7f3293400640 1 -- 172.21.15.123:0/2623138720 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c034040 con 0x560a2d273c70 2024-09-19T04:53:59.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.617+0000 7f3293400640 1 -- 172.21.15.123:0/2623138720 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329c02fbc0 con 0x560a2d273c70 2024-09-19T04:53:59.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.617+0000 7f3293400640 1 -- 172.21.15.123:0/2623138720 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c0044c0 con 0x560a2d273c70 2024-09-19T04:53:59.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.617+0000 7f329b400640 1 --2- 172.21.15.123:0/1812894043 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32880a33b0 0x7f32880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.617+0000 7f329b400640 1 -- 172.21.15.123:0/1812894043 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f32940d0140 con 0x7f32a4102d40 2024-09-19T04:53:59.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.617+0000 7f32a1e00640 1 --2- 172.21.15.123:0/1812894043 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32880a33b0 0x7f32880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.618+0000 7f32ab89a740 1 -- 172.21.15.123:0/2623138720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d273c70 msgr2=0x560a2d274050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:59.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.618+0000 7f32ab89a740 1 --2- 172.21.15.123:0/2623138720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d273c70 0x560a2d274050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f329c009830 tx=0x7f329c009860 comp rx=0 tx=0).stop 2024-09-19T04:53:59.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.618+0000 7f32ab89a740 1 -- 172.21.15.123:0/2623138720 shutdown_connections 2024-09-19T04:53:59.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.618+0000 7f32ab89a740 1 --2- 172.21.15.123:0/2623138720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d27ee50 0x560a2d281240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.618+0000 7f32ab89a740 1 --2- 172.21.15.123:0/2623138720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d274590 0x560a2d27e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.618+0000 7f32ab89a740 1 --2- 172.21.15.123:0/2623138720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d273c70 0x560a2d274050 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.618+0000 7f32ab89a740 1 -- 172.21.15.123:0/2623138720 >> 172.21.15.123:0/2623138720 conn(0x560a2d270200 msgr2=0x560a2d2705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:59.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.618+0000 7f32ab89a740 1 -- 172.21.15.123:0/2623138720 shutdown_connections 2024-09-19T04:53:59.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.618+0000 7f32a1e00640 1 --2- 172.21.15.123:0/1812894043 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32880a33b0 0x7f32880a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x560a2d271060 tx=0x7f32940d3bf0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:59.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.618+0000 7f32ab89a740 1 -- 172.21.15.123:0/2623138720 wait complete. 2024-09-19T04:53:59.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.618+0000 7f32ab89a740 1 -- start start 2024-09-19T04:53:59.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.619+0000 7f32ab89a740 1 Processor -- start 2024-09-19T04:53:59.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.619+0000 7f32ab89a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d273c70 0x560a2d27bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.619+0000 7f32ab89a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d274590 0x560a2d27c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.619+0000 7f32ab89a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d27ee50 0x560a2d27c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.619+0000 7f32ab89a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560a2d281a20 con 0x560a2d274590 2024-09-19T04:53:59.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.619+0000 7f32ab89a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560a2d1950f0 con 0x560a2d27ee50 2024-09-19T04:53:59.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.619+0000 7f32ab89a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560a2d281ba0 con 0x560a2d273c70 2024-09-19T04:53:59.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.619+0000 7f32ab89a740 10 client.0 _ll_register_callbacks cb 0x560a2d17cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:59.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.619+0000 7f3299400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d274590 0x560a2d27c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.619+0000 7f3299400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d274590 0x560a2d27c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48910/0 (socket says 172.21.15.123:48910) 2024-09-19T04:53:59.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.619+0000 7f3299400640 1 -- 172.21.15.123:0/2955385275 learned_addr learned my addr 172.21.15.123:0/2955385275 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:59.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.620+0000 7f32a2800640 1 --2- 172.21.15.123:0/2955385275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d273c70 0x560a2d27bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.620+0000 7f32a2800640 1 --2- 172.21.15.123:0/2955385275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d27ee50 0x560a2d27c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.620+0000 7f3299400640 1 -- 172.21.15.123:0/2955385275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d273c70 msgr2=0x560a2d27bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.620+0000 7f3299400640 1 --2- 172.21.15.123:0/2955385275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d273c70 0x560a2d27bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.620+0000 7f3299400640 1 -- 172.21.15.123:0/2955385275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d27ee50 msgr2=0x560a2d27c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.620+0000 7f3299400640 1 --2- 172.21.15.123:0/2955385275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d27ee50 0x560a2d27c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.620+0000 7f3299400640 1 -- 172.21.15.123:0/2955385275 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a2cf4b3c0 con 0x560a2d274590 2024-09-19T04:53:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.620+0000 7f32a2800640 1 --2- 172.21.15.123:0/2955385275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d273c70 0x560a2d27bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:53:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.620+0000 7f3299400640 1 --2- 172.21.15.123:0/2955385275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d274590 0x560a2d27c310 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f328c012700 tx=0x7f328c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3a2a58387ea55c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.620+0000 7f3287e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:53:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.620+0000 7f3287e00640 1 -- 172.21.15.123:0/2955385275 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c00f920 con 0x560a2d274590 2024-09-19T04:53:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.620+0000 7f3287e00640 1 -- 172.21.15.123:0/2955385275 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f328c00fac0 con 0x560a2d274590 2024-09-19T04:53:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.620+0000 7f32ab89a740 1 -- 172.21.15.123:0/2955385275 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329c02fbc0 con 0x560a2d274590 2024-09-19T04:53:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.621+0000 7f3287e00640 1 -- 172.21.15.123:0/2955385275 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f328c00fcc0 con 0x560a2d274590 2024-09-19T04:53:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.621+0000 7f32ab89a740 10 client.16533 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.621+0000 7f32ab89a740 1 -- 172.21.15.123:0/2955385275 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560a2d28db40 con 0x560a2d274590 2024-09-19T04:53:59.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.621+0000 7f3287e00640 1 -- 172.21.15.123:0/2955385275 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329c02fbc0 con 0x560a2d274590 2024-09-19T04:53:59.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.621+0000 7f3287e00640 1 -- 172.21.15.123:0/2955385275 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f328c01c590 con 0x560a2d274590 2024-09-19T04:53:59.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.621+0000 7f32ab89a740 10 client.16533 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:59.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.621+0000 7f32ab89a740 10 client.16533 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:53:59.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.621+0000 7f32ab89a740 20 client.16533 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:59.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.621+0000 7f32ab89a740 1 --2- 172.21.15.123:0/2955385275 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x560a2d28efa0 0x560a2d291390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.622+0000 7f32ab89a740 4 client.16533 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:53:59.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.622+0000 7f32ab89a740 1 -- 172.21.15.123:0/2955385275 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a2d1e7950 con 0x560a2d28efa0 2024-09-19T04:53:59.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.622+0000 7f32a2800640 1 --2- 172.21.15.123:0/2955385275 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x560a2d28efa0 0x560a2d291390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.623+0000 7f32a2800640 1 --2- 172.21.15.123:0/2955385275 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x560a2d28efa0 0x560a2d291390 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7f38b017c880f9ae server_cookie=4e762b1ea63a6965 in_seq=0 out_seq=0 2024-09-19T04:53:59.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.623+0000 7f3287e00640 10 client.16533 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:53:59.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.623+0000 7f3287e00640 1 -- 172.21.15.123:0/2955385275 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a2d1e7950 con 0x560a2d28efa0 2024-09-19T04:53:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.623+0000 7f3287e00640 10 client.16533 handle_command_reply: tid=0 2024-09-19T04:53:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.623+0000 7f32ab89a740 1 client.16533 shutdown 2024-09-19T04:53:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.624+0000 7f32ab89a740 1 -- 172.21.15.123:0/2955385275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d274590 msgr2=0x560a2d27c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.624+0000 7f32ab89a740 1 --2- 172.21.15.123:0/2955385275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d274590 0x560a2d27c310 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f328c012700 tx=0x7f328c009f90 comp rx=0 tx=0).stop 2024-09-19T04:53:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.624+0000 7f32ab89a740 1 -- 172.21.15.123:0/2955385275 shutdown_connections 2024-09-19T04:53:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.624+0000 7f32ab89a740 1 --2- 172.21.15.123:0/2955385275 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x560a2d28efa0 0x560a2d291390 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.624+0000 7f32ab89a740 1 --2- 172.21.15.123:0/2955385275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d27ee50 0x560a2d27c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.624+0000 7f32ab89a740 1 --2- 172.21.15.123:0/2955385275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d274590 0x560a2d27c310 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.624+0000 7f32ab89a740 1 --2- 172.21.15.123:0/2955385275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d273c70 0x560a2d27bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.624+0000 7f32ab89a740 1 -- 172.21.15.123:0/2955385275 >> 172.21.15.123:0/2955385275 conn(0x560a2d270200 msgr2=0x560a2d275ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.624+0000 7f3287e00640 0 client.16533 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:53:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.624+0000 7f32ab89a740 1 -- 172.21.15.123:0/2955385275 shutdown_connections 2024-09-19T04:53:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.624+0000 7f32ab89a740 1 -- 172.21.15.123:0/2955385275 wait complete. 2024-09-19T04:53:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.624+0000 7f32ab89a740 20 client.16533 trim_cache size 0 max 16384 2024-09-19T04:53:59.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.638+0000 7f32ab89a740 1 Processor -- start 2024-09-19T04:53:59.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.638+0000 7f32ab89a740 1 -- start start 2024-09-19T04:53:59.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.638+0000 7f32ab89a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d273c70 0x560a2d296c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.638+0000 7f32ab89a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d274590 0x560a2d297190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.638+0000 7f32ab89a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d27ee50 0x560a2d26eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.638+0000 7f32ab89a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41afbc0 con 0x560a2d274590 2024-09-19T04:53:59.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.638+0000 7f32ab89a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41b1790 con 0x560a2d273c70 2024-09-19T04:53:59.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.638+0000 7f32ab89a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41b02c0 con 0x560a2d27ee50 2024-09-19T04:53:59.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.638+0000 7f3299400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d274590 0x560a2d297190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.638+0000 7f32a2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d273c70 0x560a2d296c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.638+0000 7f3299400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d274590 0x560a2d297190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48912/0 (socket says 172.21.15.123:48912) 2024-09-19T04:53:59.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.638+0000 7f3299400640 1 -- 172.21.15.123:0/4276955705 learned_addr learned my addr 172.21.15.123:0/4276955705 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:59.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.638+0000 7f32a2800640 1 --2- 172.21.15.123:0/4276955705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d27ee50 0x560a2d26eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.639+0000 7f3299400640 1 -- 172.21.15.123:0/4276955705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d27ee50 msgr2=0x560a2d26eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:59.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.639+0000 7f3299400640 1 --2- 172.21.15.123:0/4276955705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d27ee50 0x560a2d26eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.639+0000 7f3299400640 1 -- 172.21.15.123:0/4276955705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d273c70 msgr2=0x560a2d296c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:59.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.639+0000 7f3299400640 1 --2- 172.21.15.123:0/4276955705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d273c70 0x560a2d296c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.639+0000 7f3299400640 1 -- 172.21.15.123:0/4276955705 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a2d28b7c0 con 0x560a2d274590 2024-09-19T04:53:59.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.639+0000 7f3299400640 1 --2- 172.21.15.123:0/4276955705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d274590 0x560a2d297190 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f328c009560 tx=0x7f328c009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf9d84cefe84b9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:59.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.639+0000 7f3292a00640 1 -- 172.21.15.123:0/4276955705 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c01cb50 con 0x560a2d274590 2024-09-19T04:53:59.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.639+0000 7f3292a00640 1 -- 172.21.15.123:0/4276955705 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f328c01ccf0 con 0x560a2d274590 2024-09-19T04:53:59.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.639+0000 7f3292a00640 1 -- 172.21.15.123:0/4276955705 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c0035f0 con 0x560a2d274590 2024-09-19T04:53:59.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.640+0000 7f32ab89a740 1 -- 172.21.15.123:0/4276955705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d274590 msgr2=0x560a2d297190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:59.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.640+0000 7f32ab89a740 1 --2- 172.21.15.123:0/4276955705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d274590 0x560a2d297190 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f328c009560 tx=0x7f328c009590 comp rx=0 tx=0).stop 2024-09-19T04:53:59.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.640+0000 7f32ab89a740 1 -- 172.21.15.123:0/4276955705 shutdown_connections 2024-09-19T04:53:59.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.640+0000 7f32ab89a740 1 --2- 172.21.15.123:0/4276955705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d27ee50 0x560a2d26eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.640+0000 7f32ab89a740 1 --2- 172.21.15.123:0/4276955705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d274590 0x560a2d297190 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.640+0000 7f32ab89a740 1 --2- 172.21.15.123:0/4276955705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d273c70 0x560a2d296c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.640+0000 7f32ab89a740 1 -- 172.21.15.123:0/4276955705 >> 172.21.15.123:0/4276955705 conn(0x560a2d270200 msgr2=0x560a2d2770d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:59.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.640+0000 7f32ab89a740 1 -- 172.21.15.123:0/4276955705 shutdown_connections 2024-09-19T04:53:59.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.640+0000 7f32ab89a740 1 -- 172.21.15.123:0/4276955705 wait complete. 2024-09-19T04:53:59.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.641+0000 7f32ab89a740 1 -- start start 2024-09-19T04:53:59.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.641+0000 7f32ab89a740 1 Processor -- start 2024-09-19T04:53:59.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.641+0000 7f32ab89a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d273c70 0x560a2d287410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.641+0000 7f32ab89a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d274590 0x560a2d278a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.641+0000 7f32ab89a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d27ee50 0x560a2d278f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.641+0000 7f32ab89a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41b10c0 con 0x560a2d27ee50 2024-09-19T04:53:59.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.641+0000 7f32ab89a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41b09c0 con 0x560a2d274590 2024-09-19T04:53:59.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.641+0000 7f32ab89a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41adbc0 con 0x560a2d273c70 2024-09-19T04:53:59.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.641+0000 7f32ab89a740 10 client.0 _ll_register_callbacks cb 0x7f328c009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:53:59.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.642+0000 7f32a2800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d273c70 0x560a2d287410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.642+0000 7f32a2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d27ee50 0x560a2d278f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.642+0000 7f3299400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d274590 0x560a2d278a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.642+0000 7f32a2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d27ee50 0x560a2d278f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48916/0 (socket says 172.21.15.123:48916) 2024-09-19T04:53:59.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.642+0000 7f32a2800640 1 -- 172.21.15.123:0/3104333442 learned_addr learned my addr 172.21.15.123:0/3104333442 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:53:59.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.642+0000 7f32a2800640 1 -- 172.21.15.123:0/3104333442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d273c70 msgr2=0x560a2d287410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:59.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.642+0000 7f32a2800640 1 --2- 172.21.15.123:0/3104333442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d273c70 0x560a2d287410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.642+0000 7f32a2800640 1 -- 172.21.15.123:0/3104333442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d274590 msgr2=0x560a2d278a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:53:59.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.642+0000 7f32a2800640 1 --2- 172.21.15.123:0/3104333442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d274590 0x560a2d278a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.642+0000 7f32a2800640 1 -- 172.21.15.123:0/3104333442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a2d279650 con 0x560a2d27ee50 2024-09-19T04:53:59.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.642+0000 7f32a2800640 1 --2- 172.21.15.123:0/3104333442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d27ee50 0x560a2d278f60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f329c009ca0 tx=0x7f329c005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7342d4ac84593db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:53:59.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.642+0000 7f3287e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:53:59.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.643+0000 7f3287e00640 1 -- 172.21.15.123:0/3104333442 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c003b70 con 0x560a2d27ee50 2024-09-19T04:53:59.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.643+0000 7f3287e00640 1 -- 172.21.15.123:0/3104333442 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f329c003d10 con 0x560a2d27ee50 2024-09-19T04:53:59.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.643+0000 7f32ab89a740 1 -- 172.21.15.123:0/3104333442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560a2d1e1e60 con 0x560a2d27ee50 2024-09-19T04:53:59.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.643+0000 7f3287e00640 1 -- 172.21.15.123:0/3104333442 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f329c02f730 con 0x560a2d27ee50 2024-09-19T04:53:59.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.643+0000 7f32ab89a740 10 client.16539 fetch_fsmap learned FSMap version 40 2024-09-19T04:53:59.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.643+0000 7f32ab89a740 1 -- 172.21.15.123:0/3104333442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560a2d3022f0 con 0x560a2d27ee50 2024-09-19T04:53:59.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.643+0000 7f3287e00640 1 -- 172.21.15.123:0/3104333442 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560a2d1e1e60 con 0x560a2d27ee50 2024-09-19T04:53:59.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.643+0000 7f3287e00640 1 -- 172.21.15.123:0/3104333442 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f329c042020 con 0x560a2d27ee50 2024-09-19T04:53:59.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.643+0000 7f32ab89a740 10 client.16539 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:53:59.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.643+0000 7f32ab89a740 10 client.16539 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:53:59.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.643+0000 7f32ab89a740 20 client.16539 populate_metadata read hostname 'smithi123' 2024-09-19T04:53:59.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.644+0000 7f32ab89a740 1 --2- 172.21.15.123:0/3104333442 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x560a2d302570 0x560a2d304960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:53:59.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.644+0000 7f32ab89a740 4 client.16539 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:53:59.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.644+0000 7f32ab89a740 1 -- 172.21.15.123:0/3104333442 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a2d15e010 con 0x560a2d302570 2024-09-19T04:53:59.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.644+0000 7f3299400640 1 --2- 172.21.15.123:0/3104333442 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x560a2d302570 0x560a2d304960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:53:59.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.644+0000 7f3299400640 1 --2- 172.21.15.123:0/3104333442 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x560a2d302570 0x560a2d304960 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=af54225f3beb4d7e server_cookie=26a91bc9b61acfb3 in_seq=0 out_seq=0 2024-09-19T04:53:59.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.644+0000 7f3287e00640 10 client.16539 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:53:59.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.645+0000 7f3287e00640 1 -- 172.21.15.123:0/3104333442 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a2d15e010 con 0x560a2d302570 2024-09-19T04:53:59.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.645+0000 7f3287e00640 10 client.16539 handle_command_reply: tid=0 2024-09-19T04:53:59.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.645+0000 7f32ab89a740 1 client.16539 shutdown 2024-09-19T04:53:59.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.645+0000 7f32ab89a740 1 -- 172.21.15.123:0/3104333442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d27ee50 msgr2=0x560a2d278f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:59.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.645+0000 7f32ab89a740 1 --2- 172.21.15.123:0/3104333442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d27ee50 0x560a2d278f60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f329c009ca0 tx=0x7f329c005d80 comp rx=0 tx=0).stop 2024-09-19T04:53:59.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.645+0000 7f32ab89a740 1 -- 172.21.15.123:0/3104333442 shutdown_connections 2024-09-19T04:53:59.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.645+0000 7f32ab89a740 1 --2- 172.21.15.123:0/3104333442 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x560a2d302570 0x560a2d304960 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.645+0000 7f32ab89a740 1 --2- 172.21.15.123:0/3104333442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a2d27ee50 0x560a2d278f60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.645+0000 7f32ab89a740 1 --2- 172.21.15.123:0/3104333442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a2d274590 0x560a2d278a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.645+0000 7f32ab89a740 1 --2- 172.21.15.123:0/3104333442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a2d273c70 0x560a2d287410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.645+0000 7f32ab89a740 1 -- 172.21.15.123:0/3104333442 >> 172.21.15.123:0/3104333442 conn(0x560a2d270200 msgr2=0x560a2d26eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:59.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.645+0000 7f3287e00640 0 client.16539 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:53:59.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.645+0000 7f32ab89a740 1 -- 172.21.15.123:0/3104333442 shutdown_connections 2024-09-19T04:53:59.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.645+0000 7f32ab89a740 1 -- 172.21.15.123:0/3104333442 wait complete. 2024-09-19T04:53:59.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.645+0000 7f32ab89a740 20 client.16539 trim_cache size 0 max 16384 2024-09-19T04:53:59.651 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:53:59.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.646+0000 7f32a8e00640 1 -- 172.21.15.123:0/1812894043 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32880a33b0 msgr2=0x7f32880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:59.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.646+0000 7f32a8e00640 1 --2- 172.21.15.123:0/1812894043 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32880a33b0 0x7f32880a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x560a2d271060 tx=0x7f32940d3bf0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.646+0000 7f32a8e00640 1 -- 172.21.15.123:0/1812894043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a4102d40 msgr2=0x7f32a419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:53:59.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.646+0000 7f32a8e00640 1 --2- 172.21.15.123:0/1812894043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a4102d40 0x7f32a419a5c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f329400e9f0 tx=0x7f329400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.646+0000 7f32a8e00640 1 -- 172.21.15.123:0/1812894043 shutdown_connections 2024-09-19T04:53:59.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.646+0000 7f32a8e00640 1 --2- 172.21.15.123:0/1812894043 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32880a33b0 0x7f32880a5870 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.646+0000 7f32a8e00640 1 --2- 172.21.15.123:0/1812894043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32a410c170 0x7f32a419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.646+0000 7f32a8e00640 1 --2- 172.21.15.123:0/1812894043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32a4103660 0x7f32a419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.646+0000 7f32a8e00640 1 --2- 172.21.15.123:0/1812894043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a4102d40 0x7f32a419a5c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:53:59.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.646+0000 7f32a8e00640 1 -- 172.21.15.123:0/1812894043 >> 172.21.15.123:0/1812894043 conn(0x7f32a40fc7a0 msgr2=0x7f32a40fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:53:59.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.646+0000 7f32a8e00640 1 -- 172.21.15.123:0/1812894043 shutdown_connections 2024-09-19T04:53:59.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:53:59.646+0000 7f32a8e00640 1 -- 172.21.15.123:0/1812894043 wait complete. 2024-09-19T04:53:59.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:53:59 smithi193 ceph-mon[35592]: pgmap v223: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:53:59.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:53:59 smithi183 ceph-mon[35333]: pgmap v223: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:54:00.506 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:54:00.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:00 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:54:00.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:00 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:54:00.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:00 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:54:00.922 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:01.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.501+0000 7feb27600640 1 -- 172.21.15.123:0/4275710599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb20106f10 msgr2=0x7feb2010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:01.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.501+0000 7feb27600640 1 --2- 172.21.15.123:0/4275710599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb20106f10 0x7feb2010ea90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7feb18009ca0 tx=0x7feb1802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.501+0000 7feb27600640 1 -- 172.21.15.123:0/4275710599 shutdown_connections 2024-09-19T04:54:01.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.501+0000 7feb27600640 1 --2- 172.21.15.123:0/4275710599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb20106f10 0x7feb2010ea90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.501+0000 7feb27600640 1 --2- 172.21.15.123:0/4275710599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb20102790 0x7feb201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.501+0000 7feb27600640 1 --2- 172.21.15.123:0/4275710599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb20101de0 0x7feb201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.501+0000 7feb27600640 1 -- 172.21.15.123:0/4275710599 >> 172.21.15.123:0/4275710599 conn(0x7feb200fd8c0 msgr2=0x7feb200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:01.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.501+0000 7feb27600640 1 -- 172.21.15.123:0/4275710599 shutdown_connections 2024-09-19T04:54:01.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.501+0000 7feb27600640 1 -- 172.21.15.123:0/4275710599 wait complete. 2024-09-19T04:54:01.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.502+0000 7feb27600640 1 Processor -- start 2024-09-19T04:54:01.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.502+0000 7feb27600640 1 -- start start 2024-09-19T04:54:01.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.502+0000 7feb27600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb20101de0 0x7feb201983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.502+0000 7feb27600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb20102790 0x7feb201988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.502+0000 7feb27600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb20106f10 0x7feb2019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.502+0000 7feb27600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb20111f70 con 0x7feb20101de0 2024-09-19T04:54:01.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.502+0000 7feb27600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feb20111df0 con 0x7feb20102790 2024-09-19T04:54:01.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.502+0000 7feb27600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb201120f0 con 0x7feb20106f10 2024-09-19T04:54:01.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.503+0000 7feb24c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb20101de0 0x7feb201983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.503+0000 7feb24c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb20101de0 0x7feb201983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48932/0 (socket says 172.21.15.123:48932) 2024-09-19T04:54:01.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.503+0000 7feb24c00640 1 -- 172.21.15.123:0/3132008202 learned_addr learned my addr 172.21.15.123:0/3132008202 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:01.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.503+0000 7feb17e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb20102790 0x7feb201988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.503+0000 7feb25600640 1 --2- 172.21.15.123:0/3132008202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb20106f10 0x7feb2019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.503+0000 7feb24c00640 1 -- 172.21.15.123:0/3132008202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb20106f10 msgr2=0x7feb2019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:01.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.503+0000 7feb24c00640 1 --2- 172.21.15.123:0/3132008202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb20106f10 0x7feb2019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.503+0000 7feb24c00640 1 -- 172.21.15.123:0/3132008202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb20102790 msgr2=0x7feb201988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:01.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.503+0000 7feb24c00640 1 --2- 172.21.15.123:0/3132008202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb20102790 0x7feb201988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.503+0000 7feb24c00640 1 -- 172.21.15.123:0/3132008202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2019d360 con 0x7feb20101de0 2024-09-19T04:54:01.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.503+0000 7feb24c00640 1 --2- 172.21.15.123:0/3132008202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb20101de0 0x7feb201983b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7feb1000e9f0 tx=0x7feb1000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:01.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.503+0000 7feb1e000640 1 -- 172.21.15.123:0/3132008202 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1000cd80 con 0x7feb20101de0 2024-09-19T04:54:01.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.503+0000 7feb1e000640 1 -- 172.21.15.123:0/3132008202 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb10005500 con 0x7feb20101de0 2024-09-19T04:54:01.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.503+0000 7feb1e000640 1 -- 172.21.15.123:0/3132008202 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10010690 con 0x7feb20101de0 2024-09-19T04:54:01.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.503+0000 7feb27600640 1 -- 172.21.15.123:0/3132008202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2019d650 con 0x7feb20101de0 2024-09-19T04:54:01.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.504+0000 7feb27600640 1 -- 172.21.15.123:0/3132008202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb201a4f30 con 0x7feb20101de0 2024-09-19T04:54:01.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.506+0000 7feb2a214740 1 Processor -- start 2024-09-19T04:54:01.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.507+0000 7feb1e000640 1 -- 172.21.15.123:0/3132008202 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7feb100046c0 con 0x7feb20101de0 2024-09-19T04:54:01.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.507+0000 7feb2a214740 1 -- start start 2024-09-19T04:54:01.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.507+0000 7feb2a214740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841312bb70 0x55841312bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.507+0000 7feb2a214740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312c490 0x55841313ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.507+0000 7feb2a214740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841313b260 0x55841313d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.507+0000 7feb2a214740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5584130923e0 con 0x55841312c490 2024-09-19T04:54:01.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.507+0000 7feb2a214740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558413039d80 con 0x55841313b260 2024-09-19T04:54:01.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.507+0000 7feb2a214740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55841308e3a0 con 0x55841312bb70 2024-09-19T04:54:01.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.507+0000 7feb24c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312c490 0x55841313ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.507+0000 7feb17e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841313b260 0x55841313d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.507+0000 7feb17e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841313b260 0x55841313d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:47170/0 (socket says 172.21.15.123:47170) 2024-09-19T04:54:01.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.507+0000 7feb17e00640 1 -- 172.21.15.123:0/1480552858 learned_addr learned my addr 172.21.15.123:0/1480552858 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:01.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.507+0000 7feb25600640 1 --2- 172.21.15.123:0/1480552858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841312bb70 0x55841312bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.508+0000 7feb17e00640 1 -- 172.21.15.123:0/1480552858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841312bb70 msgr2=0x55841312bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:01.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.508+0000 7feb17e00640 1 --2- 172.21.15.123:0/1480552858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841312bb70 0x55841312bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.508+0000 7feb17e00640 1 -- 172.21.15.123:0/1480552858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312c490 msgr2=0x55841313ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:01.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.508+0000 7feb17e00640 1 --2- 172.21.15.123:0/1480552858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312c490 0x55841313ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.508+0000 7feb17e00640 1 -- 172.21.15.123:0/1480552858 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558413058ae0 con 0x55841313b260 2024-09-19T04:54:01.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.508+0000 7feb17e00640 1 --2- 172.21.15.123:0/1480552858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841313b260 0x55841313d650 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7feb0800bc10 tx=0x7feb080107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6c654b5b2b11bd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:01.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.508+0000 7feb1e000640 1 --2- 172.21.15.123:0/3132008202 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7feb040a33b0 0x7feb040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.508+0000 7feb15c00640 1 -- 172.21.15.123:0/1480552858 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08005020 con 0x55841313b260 2024-09-19T04:54:01.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.508+0000 7feb15c00640 1 -- 172.21.15.123:0/1480552858 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb080047a0 con 0x55841313b260 2024-09-19T04:54:01.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.508+0000 7feb15c00640 1 -- 172.21.15.123:0/1480552858 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08011740 con 0x55841313b260 2024-09-19T04:54:01.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.509+0000 7feb1e000640 1 -- 172.21.15.123:0/3132008202 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7feb100d0250 con 0x7feb20101de0 2024-09-19T04:54:01.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.509+0000 7feb25600640 1 --2- 172.21.15.123:0/3132008202 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7feb040a33b0 0x7feb040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.509+0000 7feb2a214740 1 -- 172.21.15.123:0/1480552858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841313b260 msgr2=0x55841313d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:01.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.509+0000 7feb2a214740 1 --2- 172.21.15.123:0/1480552858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841313b260 0x55841313d650 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7feb0800bc10 tx=0x7feb080107a0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.509+0000 7feb25600640 1 --2- 172.21.15.123:0/3132008202 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7feb040a33b0 0x7feb040a5870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feb18005e30 tx=0x7feb18005c30 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:01.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.509+0000 7feb2a214740 1 -- 172.21.15.123:0/1480552858 shutdown_connections 2024-09-19T04:54:01.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.509+0000 7feb2a214740 1 --2- 172.21.15.123:0/1480552858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841313b260 0x55841313d650 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.509+0000 7feb2a214740 1 --2- 172.21.15.123:0/1480552858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312c490 0x55841313ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.509+0000 7feb2a214740 1 --2- 172.21.15.123:0/1480552858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841312bb70 0x55841312bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.509+0000 7feb2a214740 1 -- 172.21.15.123:0/1480552858 >> 172.21.15.123:0/1480552858 conn(0x558413128200 msgr2=0x5584131285e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:01.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.510+0000 7feb2a214740 1 -- 172.21.15.123:0/1480552858 shutdown_connections 2024-09-19T04:54:01.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.510+0000 7feb2a214740 1 -- 172.21.15.123:0/1480552858 wait complete. 2024-09-19T04:54:01.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.510+0000 7feb2a214740 1 -- start start 2024-09-19T04:54:01.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.510+0000 7feb2a214740 1 Processor -- start 2024-09-19T04:54:01.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.511+0000 7feb2a214740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312bb70 0x558413138140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.511+0000 7feb2a214740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312c490 0x558413138680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.511+0000 7feb2a214740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841313b260 0x558413138bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.511+0000 7feb2a214740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55841313de90 con 0x55841312bb70 2024-09-19T04:54:01.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.511+0000 7feb2a214740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55841304d0f0 con 0x55841312c490 2024-09-19T04:54:01.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.511+0000 7feb2a214740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55841313e010 con 0x55841313b260 2024-09-19T04:54:01.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.511+0000 7feb2a214740 10 client.0 _ll_register_callbacks cb 0x558413034bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:01.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.511+0000 7feb24c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312bb70 0x558413138140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.511+0000 7feb24c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312bb70 0x558413138140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48942/0 (socket says 172.21.15.123:48942) 2024-09-19T04:54:01.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.511+0000 7feb24c00640 1 -- 172.21.15.123:0/3307687365 learned_addr learned my addr 172.21.15.123:0/3307687365 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:01.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.511+0000 7feb25600640 1 --2- 172.21.15.123:0/3307687365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841313b260 0x558413138bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.511+0000 7feb17e00640 1 --2- 172.21.15.123:0/3307687365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312c490 0x558413138680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.511+0000 7feb24c00640 1 -- 172.21.15.123:0/3307687365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841313b260 msgr2=0x558413138bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:01.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.511+0000 7feb24c00640 1 --2- 172.21.15.123:0/3307687365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841313b260 0x558413138bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.511+0000 7feb24c00640 1 -- 172.21.15.123:0/3307687365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312c490 msgr2=0x558413138680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:01.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.511+0000 7feb24c00640 1 --2- 172.21.15.123:0/3307687365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312c490 0x558413138680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.511+0000 7feb24c00640 1 -- 172.21.15.123:0/3307687365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558412e033c0 con 0x55841312bb70 2024-09-19T04:54:01.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.512+0000 7feb24c00640 1 --2- 172.21.15.123:0/3307687365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312bb70 0x558413138140 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7feb100cd9a0 tx=0x7feb100980b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb8e7fb402b9206 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:01.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.512+0000 7feb0e000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:01.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.512+0000 7feb0e000640 1 -- 172.21.15.123:0/3307687365 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1009c2a0 con 0x55841312bb70 2024-09-19T04:54:01.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.512+0000 7feb0e000640 1 -- 172.21.15.123:0/3307687365 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7feb100a4eb0 con 0x55841312bb70 2024-09-19T04:54:01.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.512+0000 7feb2a214740 1 -- 172.21.15.123:0/3307687365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb080047a0 con 0x55841312bb70 2024-09-19T04:54:01.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.512+0000 7feb0e000640 1 -- 172.21.15.123:0/3307687365 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7feb100cddd0 con 0x55841312bb70 2024-09-19T04:54:01.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.512+0000 7feb2a214740 10 client.16551 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:01.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.512+0000 7feb2a214740 1 -- 172.21.15.123:0/3307687365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558413149f10 con 0x55841312bb70 2024-09-19T04:54:01.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.512+0000 7feb0e000640 1 -- 172.21.15.123:0/3307687365 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb080047a0 con 0x55841312bb70 2024-09-19T04:54:01.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.513+0000 7feb0e000640 1 -- 172.21.15.123:0/3307687365 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7feb100b5340 con 0x55841312bb70 2024-09-19T04:54:01.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.513+0000 7feb2a214740 10 client.16551 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:01.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.513+0000 7feb2a214740 10 client.16551 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:54:01.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.513+0000 7feb2a214740 20 client.16551 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:01.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.513+0000 7feb2a214740 1 --2- 172.21.15.123:0/3307687365 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55841314b370 0x55841314d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.513+0000 7feb2a214740 4 client.16551 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:01.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.513+0000 7feb2a214740 1 -- 172.21.15.123:0/3307687365 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55841309f950 con 0x55841314b370 2024-09-19T04:54:01.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.513+0000 7feb17e00640 1 --2- 172.21.15.123:0/3307687365 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55841314b370 0x55841314d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.514+0000 7feb17e00640 1 --2- 172.21.15.123:0/3307687365 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55841314b370 0x55841314d7c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52032e5cd5a6589b server_cookie=daad6875f29804e8 in_seq=0 out_seq=0 2024-09-19T04:54:01.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.514+0000 7feb0e000640 10 client.16551 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:54:01.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.515+0000 7feb0e000640 1 -- 172.21.15.123:0/3307687365 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55841309f950 con 0x55841314b370 2024-09-19T04:54:01.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.515+0000 7feb0e000640 10 client.16551 handle_command_reply: tid=0 2024-09-19T04:54:01.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.515+0000 7feb2a214740 1 client.16551 shutdown 2024-09-19T04:54:01.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.515+0000 7feb2a214740 1 -- 172.21.15.123:0/3307687365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312bb70 msgr2=0x558413138140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:01.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.515+0000 7feb2a214740 1 --2- 172.21.15.123:0/3307687365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312bb70 0x558413138140 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7feb100cd9a0 tx=0x7feb100980b0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.515+0000 7feb2a214740 1 -- 172.21.15.123:0/3307687365 shutdown_connections 2024-09-19T04:54:01.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.515+0000 7feb2a214740 1 --2- 172.21.15.123:0/3307687365 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55841314b370 0x55841314d7c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.515+0000 7feb2a214740 1 --2- 172.21.15.123:0/3307687365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841313b260 0x558413138bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.515+0000 7feb2a214740 1 --2- 172.21.15.123:0/3307687365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312c490 0x558413138680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.515+0000 7feb2a214740 1 --2- 172.21.15.123:0/3307687365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312bb70 0x558413138140 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.515+0000 7feb2a214740 1 -- 172.21.15.123:0/3307687365 >> 172.21.15.123:0/3307687365 conn(0x558413128200 msgr2=0x55841312df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:01.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.515+0000 7feb0e000640 0 client.16551 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:54:01.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.515+0000 7feb2a214740 1 -- 172.21.15.123:0/3307687365 shutdown_connections 2024-09-19T04:54:01.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.515+0000 7feb2a214740 1 -- 172.21.15.123:0/3307687365 wait complete. 2024-09-19T04:54:01.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.515+0000 7feb2a214740 20 client.16551 trim_cache size 0 max 16384 2024-09-19T04:54:01.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.529+0000 7feb2a214740 1 Processor -- start 2024-09-19T04:54:01.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.529+0000 7feb2a214740 1 -- start start 2024-09-19T04:54:01.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.529+0000 7feb2a214740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312bb70 0x558413136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.529+0000 7feb2a214740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312c490 0x558413153090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.529+0000 7feb2a214740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841313b260 0x5584131535d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.529+0000 7feb2a214740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb201ab6b0 con 0x55841312bb70 2024-09-19T04:54:01.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.529+0000 7feb2a214740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feb201ad280 con 0x55841312c490 2024-09-19T04:54:01.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.529+0000 7feb2a214740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb201abdb0 con 0x55841313b260 2024-09-19T04:54:01.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.529+0000 7feb24c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312bb70 0x558413136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.529+0000 7feb24c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312bb70 0x558413136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48950/0 (socket says 172.21.15.123:48950) 2024-09-19T04:54:01.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.529+0000 7feb24c00640 1 -- 172.21.15.123:0/2310446988 learned_addr learned my addr 172.21.15.123:0/2310446988 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:01.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.530+0000 7feb17e00640 1 --2- 172.21.15.123:0/2310446988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312c490 0x558413153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.530+0000 7feb25600640 1 --2- 172.21.15.123:0/2310446988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841313b260 0x5584131535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.530+0000 7feb24c00640 1 -- 172.21.15.123:0/2310446988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841313b260 msgr2=0x5584131535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:01.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.530+0000 7feb24c00640 1 --2- 172.21.15.123:0/2310446988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841313b260 0x5584131535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.530+0000 7feb24c00640 1 -- 172.21.15.123:0/2310446988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312c490 msgr2=0x558413153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:01.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.530+0000 7feb24c00640 1 --2- 172.21.15.123:0/2310446988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312c490 0x558413153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.530+0000 7feb24c00640 1 -- 172.21.15.123:0/2310446988 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584131370f0 con 0x55841312bb70 2024-09-19T04:54:01.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.530+0000 7feb17e00640 1 --2- 172.21.15.123:0/2310446988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312c490 0x558413153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:01.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.530+0000 7feb24c00640 1 --2- 172.21.15.123:0/2310446988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312bb70 0x558413136a10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7feb10098ea0 tx=0x7feb1009bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a52c936f8eb918 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:01.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.530+0000 7feb15200640 1 -- 172.21.15.123:0/2310446988 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1009be30 con 0x55841312bb70 2024-09-19T04:54:01.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.530+0000 7feb15200640 1 -- 172.21.15.123:0/2310446988 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb100b54b0 con 0x55841312bb70 2024-09-19T04:54:01.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.530+0000 7feb15200640 1 -- 172.21.15.123:0/2310446988 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1009c010 con 0x55841312bb70 2024-09-19T04:54:01.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.531+0000 7feb2a214740 1 -- 172.21.15.123:0/2310446988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312bb70 msgr2=0x558413136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:01.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.531+0000 7feb2a214740 1 --2- 172.21.15.123:0/2310446988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312bb70 0x558413136a10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7feb10098ea0 tx=0x7feb1009bbc0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.531+0000 7feb2a214740 1 -- 172.21.15.123:0/2310446988 shutdown_connections 2024-09-19T04:54:01.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.531+0000 7feb2a214740 1 --2- 172.21.15.123:0/2310446988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841313b260 0x5584131535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.531+0000 7feb2a214740 1 --2- 172.21.15.123:0/2310446988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312c490 0x558413153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.531+0000 7feb2a214740 1 --2- 172.21.15.123:0/2310446988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312bb70 0x558413136a10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.531+0000 7feb2a214740 1 -- 172.21.15.123:0/2310446988 >> 172.21.15.123:0/2310446988 conn(0x558413128200 msgr2=0x55841313a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:01.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.531+0000 7feb2a214740 1 -- 172.21.15.123:0/2310446988 shutdown_connections 2024-09-19T04:54:01.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.531+0000 7feb2a214740 1 -- 172.21.15.123:0/2310446988 wait complete. 2024-09-19T04:54:01.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.532+0000 7feb2a214740 1 -- start start 2024-09-19T04:54:01.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.532+0000 7feb2a214740 1 Processor -- start 2024-09-19T04:54:01.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.532+0000 7feb2a214740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312bb70 0x55841313f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.532+0000 7feb2a214740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312c490 0x55841313f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.532+0000 7feb2a214740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841313b260 0x55841313fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.532+0000 7feb2a214740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb201acbb0 con 0x55841312c490 2024-09-19T04:54:01.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.532+0000 7feb2a214740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feb201ac4b0 con 0x55841312bb70 2024-09-19T04:54:01.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.532+0000 7feb2a214740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb201a96b0 con 0x55841313b260 2024-09-19T04:54:01.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.532+0000 7feb2a214740 10 client.0 _ll_register_callbacks cb 0x7feb1000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:01.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.533+0000 7feb24c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312bb70 0x55841313f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.533+0000 7feb17e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312c490 0x55841313f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.533+0000 7feb24c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312bb70 0x55841313f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:47202/0 (socket says 172.21.15.123:47202) 2024-09-19T04:54:01.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.533+0000 7feb24c00640 1 -- 172.21.15.123:0/3206293626 learned_addr learned my addr 172.21.15.123:0/3206293626 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:01.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.533+0000 7feb25600640 1 --2- 172.21.15.123:0/3206293626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841313b260 0x55841313fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.533+0000 7feb24c00640 1 -- 172.21.15.123:0/3206293626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841313b260 msgr2=0x55841313fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:01.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.533+0000 7feb24c00640 1 --2- 172.21.15.123:0/3206293626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841313b260 0x55841313fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.533+0000 7feb24c00640 1 -- 172.21.15.123:0/3206293626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312c490 msgr2=0x55841313f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:01.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.533+0000 7feb24c00640 1 --2- 172.21.15.123:0/3206293626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312c490 0x55841313f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.533+0000 7feb24c00640 1 -- 172.21.15.123:0/3206293626 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584131b51c0 con 0x55841312bb70 2024-09-19T04:54:01.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.533+0000 7feb24c00640 1 --2- 172.21.15.123:0/3206293626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312bb70 0x55841313f120 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7feb1009a780 tx=0x7feb1009c570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa404c489da1e248 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:01.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.533+0000 7feb0e000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:54:01.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.533+0000 7feb0e000640 1 -- 172.21.15.123:0/3206293626 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1009c010 con 0x55841312bb70 2024-09-19T04:54:01.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.533+0000 7feb2a214740 1 -- 172.21.15.123:0/3206293626 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558413099e60 con 0x55841312bb70 2024-09-19T04:54:01.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.533+0000 7feb0e000640 1 -- 172.21.15.123:0/3206293626 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7feb1009b5a0 con 0x55841312bb70 2024-09-19T04:54:01.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.534+0000 7feb0e000640 1 -- 172.21.15.123:0/3206293626 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7feb1009c2a0 con 0x55841312bb70 2024-09-19T04:54:01.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.534+0000 7feb2a214740 10 client.26657 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:01.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.534+0000 7feb2a214740 1 -- 172.21.15.123:0/3206293626 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5584131b6460 con 0x55841312bb70 2024-09-19T04:54:01.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.534+0000 7feb0e000640 1 -- 172.21.15.123:0/3206293626 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558413099e60 con 0x55841312bb70 2024-09-19T04:54:01.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.534+0000 7feb0e000640 1 -- 172.21.15.123:0/3206293626 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7feb10067260 con 0x55841312bb70 2024-09-19T04:54:01.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.534+0000 7feb2a214740 10 client.26657 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:01.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.534+0000 7feb2a214740 10 client.26657 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:54:01.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.534+0000 7feb2a214740 20 client.26657 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:01.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.534+0000 7feb2a214740 1 --2- 172.21.15.123:0/3206293626 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5584131b6890 0x5584131b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:01.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.534+0000 7feb2a214740 4 client.26657 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:01.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.534+0000 7feb2a214740 1 -- 172.21.15.123:0/3206293626 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558413016010 con 0x5584131b6890 2024-09-19T04:54:01.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.534+0000 7feb17e00640 1 --2- 172.21.15.123:0/3206293626 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5584131b6890 0x5584131b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:01.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.535+0000 7feb17e00640 1 --2- 172.21.15.123:0/3206293626 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5584131b6890 0x5584131b8c80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a009a274b0ed4486 server_cookie=14590dd6f3d35bac in_seq=0 out_seq=0 2024-09-19T04:54:01.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.535+0000 7feb0e000640 10 client.26657 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:54:01.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.535+0000 7feb0e000640 1 -- 172.21.15.123:0/3206293626 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558413016010 con 0x5584131b6890 2024-09-19T04:54:01.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.535+0000 7feb0e000640 10 client.26657 handle_command_reply: tid=0 2024-09-19T04:54:01.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.535+0000 7feb2a214740 1 client.26657 shutdown 2024-09-19T04:54:01.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.535+0000 7feb2a214740 1 -- 172.21.15.123:0/3206293626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312bb70 msgr2=0x55841313f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:01.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.535+0000 7feb2a214740 1 --2- 172.21.15.123:0/3206293626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312bb70 0x55841313f120 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7feb1009a780 tx=0x7feb1009c570 comp rx=0 tx=0).stop 2024-09-19T04:54:01.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.535+0000 7feb2a214740 1 -- 172.21.15.123:0/3206293626 shutdown_connections 2024-09-19T04:54:01.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.535+0000 7feb2a214740 1 --2- 172.21.15.123:0/3206293626 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5584131b6890 0x5584131b8c80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.535+0000 7feb2a214740 1 --2- 172.21.15.123:0/3206293626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55841313b260 0x55841313fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.535+0000 7feb2a214740 1 --2- 172.21.15.123:0/3206293626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55841312c490 0x55841313f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.535+0000 7feb0e000640 0 client.26657 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:54:01.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.535+0000 7feb2a214740 1 --2- 172.21.15.123:0/3206293626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841312bb70 0x55841313f120 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.535+0000 7feb2a214740 1 -- 172.21.15.123:0/3206293626 >> 172.21.15.123:0/3206293626 conn(0x558413128200 msgr2=0x55841312df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:01.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.536+0000 7feb2a214740 1 -- 172.21.15.123:0/3206293626 shutdown_connections 2024-09-19T04:54:01.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.536+0000 7feb2a214740 1 -- 172.21.15.123:0/3206293626 wait complete. 2024-09-19T04:54:01.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.536+0000 7feb2a214740 20 client.26657 trim_cache size 0 max 16384 2024-09-19T04:54:01.544 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:01.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.536+0000 7feb27600640 1 -- 172.21.15.123:0/3132008202 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7feb040a33b0 msgr2=0x7feb040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:01.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.536+0000 7feb27600640 1 --2- 172.21.15.123:0/3132008202 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7feb040a33b0 0x7feb040a5870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feb18005e30 tx=0x7feb18005c30 comp rx=0 tx=0).stop 2024-09-19T04:54:01.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.536+0000 7feb27600640 1 -- 172.21.15.123:0/3132008202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb20101de0 msgr2=0x7feb201983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:01.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.536+0000 7feb27600640 1 --2- 172.21.15.123:0/3132008202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb20101de0 0x7feb201983b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7feb1000e9f0 tx=0x7feb1000edf0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.537+0000 7feb27600640 1 -- 172.21.15.123:0/3132008202 shutdown_connections 2024-09-19T04:54:01.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.537+0000 7feb27600640 1 --2- 172.21.15.123:0/3132008202 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7feb040a33b0 0x7feb040a5870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.537+0000 7feb27600640 1 --2- 172.21.15.123:0/3132008202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb20106f10 0x7feb2019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.537+0000 7feb27600640 1 --2- 172.21.15.123:0/3132008202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb20102790 0x7feb201988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.537+0000 7feb27600640 1 --2- 172.21.15.123:0/3132008202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb20101de0 0x7feb201983b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:01.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.537+0000 7feb27600640 1 -- 172.21.15.123:0/3132008202 >> 172.21.15.123:0/3132008202 conn(0x7feb200fd8c0 msgr2=0x7feb200feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:01.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.537+0000 7feb27600640 1 -- 172.21.15.123:0/3132008202 shutdown_connections 2024-09-19T04:54:01.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:01.537+0000 7feb27600640 1 -- 172.21.15.123:0/3132008202 wait complete. 2024-09-19T04:54:01.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:01 smithi123 ceph-mon[30511]: pgmap v224: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:54:01.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:01 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:54:01.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:01 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:54:01.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:01 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:54:01.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:01 smithi193 ceph-mon[35592]: pgmap v224: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:54:01.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:01 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:54:01.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:01 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:54:01.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:01 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:54:01.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:01 smithi183 ceph-mon[35333]: pgmap v224: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:54:01.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:01 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:54:01.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:01 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:54:01.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:01 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:54:02.432 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:54:02.800 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:03.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.369+0000 7f508be00640 1 -- 172.21.15.123:0/501819964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f508c10e270 msgr2=0x7f508c110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:03.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.369+0000 7f508be00640 1 --2- 172.21.15.123:0/501819964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f508c10e270 0x7f508c110660 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5084009ca0 tx=0x7f508402a950 comp rx=0 tx=0).stop 2024-09-19T04:54:03.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.369+0000 7f508be00640 1 -- 172.21.15.123:0/501819964 shutdown_connections 2024-09-19T04:54:03.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.369+0000 7f508be00640 1 --2- 172.21.15.123:0/501819964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f508c10e270 0x7f508c110660 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.369+0000 7f508be00640 1 --2- 172.21.15.123:0/501819964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f508c074ee0 0x7f508c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.369+0000 7f508be00640 1 --2- 172.21.15.123:0/501819964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f508c0745c0 0x7f508c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.369+0000 7f508be00640 1 -- 172.21.15.123:0/501819964 >> 172.21.15.123:0/501819964 conn(0x7f508c0fd710 msgr2=0x7f508c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:03.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.369+0000 7f508be00640 1 -- 172.21.15.123:0/501819964 shutdown_connections 2024-09-19T04:54:03.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.369+0000 7f508be00640 1 -- 172.21.15.123:0/501819964 wait complete. 2024-09-19T04:54:03.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.370+0000 7f508be00640 1 Processor -- start 2024-09-19T04:54:03.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.370+0000 7f508be00640 1 -- start start 2024-09-19T04:54:03.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.370+0000 7f508be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f508c0745c0 0x7f508c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.370+0000 7f508be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f508c074ee0 0x7f508c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.370+0000 7f508be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f508c10e270 0x7f508c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.370+0000 7f508be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f508c113fd0 con 0x7f508c10e270 2024-09-19T04:54:03.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.370+0000 7f508be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f508c113e50 con 0x7f508c074ee0 2024-09-19T04:54:03.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.370+0000 7f508be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f508c114150 con 0x7f508c0745c0 2024-09-19T04:54:03.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.370+0000 7f508b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f508c10e270 0x7f508c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.371+0000 7f508b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f508c10e270 0x7f508c19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54030/0 (socket says 172.21.15.123:54030) 2024-09-19T04:54:03.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.371+0000 7f508b400640 1 -- 172.21.15.123:0/2299760277 learned_addr learned my addr 172.21.15.123:0/2299760277 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:03.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.371+0000 7f508a000640 1 --2- 172.21.15.123:0/2299760277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f508c074ee0 0x7f508c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.371+0000 7f508aa00640 1 --2- 172.21.15.123:0/2299760277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f508c0745c0 0x7f508c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.371+0000 7f508b400640 1 -- 172.21.15.123:0/2299760277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f508c0745c0 msgr2=0x7f508c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:03.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.371+0000 7f508b400640 1 --2- 172.21.15.123:0/2299760277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f508c0745c0 0x7f508c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.371+0000 7f508b400640 1 -- 172.21.15.123:0/2299760277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f508c074ee0 msgr2=0x7f508c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:03.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.371+0000 7f508b400640 1 --2- 172.21.15.123:0/2299760277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f508c074ee0 0x7f508c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.371+0000 7f508b400640 1 -- 172.21.15.123:0/2299760277 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508c19f460 con 0x7f508c10e270 2024-09-19T04:54:03.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.371+0000 7f508a000640 1 --2- 172.21.15.123:0/2299760277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f508c074ee0 0x7f508c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:03.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.371+0000 7f508b400640 1 --2- 172.21.15.123:0/2299760277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f508c10e270 0x7f508c19ed80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5084005e30 tx=0x7f5084003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:03.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.372+0000 7f5090a00640 1 -- 172.21.15.123:0/2299760277 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084034040 con 0x7f508c10e270 2024-09-19T04:54:03.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.372+0000 7f5090a00640 1 -- 172.21.15.123:0/2299760277 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f508402fe20 con 0x7f508c10e270 2024-09-19T04:54:03.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.372+0000 7f5090a00640 1 -- 172.21.15.123:0/2299760277 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50840046e0 con 0x7f508c10e270 2024-09-19T04:54:03.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.372+0000 7f508be00640 1 -- 172.21.15.123:0/2299760277 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508c19f6f0 con 0x7f508c10e270 2024-09-19T04:54:03.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.372+0000 7f508be00640 1 -- 172.21.15.123:0/2299760277 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508c19fb80 con 0x7f508c10e270 2024-09-19T04:54:03.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.375+0000 7f5090a00640 1 -- 172.21.15.123:0/2299760277 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5084037020 con 0x7f508c10e270 2024-09-19T04:54:03.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.375+0000 7f5094a44740 1 Processor -- start 2024-09-19T04:54:03.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.375+0000 7f5094a44740 1 -- start start 2024-09-19T04:54:03.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.375+0000 7f5094a44740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e90e40 0x55d4b5e91220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.375+0000 7f5094a44740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e91760 0x55d4b5e9bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.375+0000 7f5094a44740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e9c020 0x55d4b5e9e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.375+0000 7f5094a44740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d4b5df73e0 con 0x55d4b5e9c020 2024-09-19T04:54:03.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.375+0000 7f5094a44740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d4b5d9ed80 con 0x55d4b5e91760 2024-09-19T04:54:03.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.375+0000 7f5094a44740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d4b5df33a0 con 0x55d4b5e90e40 2024-09-19T04:54:03.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.376+0000 7f508a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e90e40 0x55d4b5e91220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.376+0000 7f508aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e91760 0x55d4b5e9bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.376+0000 7f508a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e9c020 0x55d4b5e9e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.376+0000 7f508a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e9c020 0x55d4b5e9e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54032/0 (socket says 172.21.15.123:54032) 2024-09-19T04:54:03.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.376+0000 7f508aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e91760 0x55d4b5e9bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:49502/0 (socket says 172.21.15.123:49502) 2024-09-19T04:54:03.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.376+0000 7f508a000640 1 -- 172.21.15.123:0/3536589153 learned_addr learned my addr 172.21.15.123:0/3536589153 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:03.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.376+0000 7f508aa00640 1 -- 172.21.15.123:0/3536589153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e90e40 msgr2=0x55d4b5e91220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:03.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.376+0000 7f508aa00640 1 --2- 172.21.15.123:0/3536589153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e90e40 0x55d4b5e91220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.376+0000 7f508aa00640 1 -- 172.21.15.123:0/3536589153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e9c020 msgr2=0x55d4b5e9e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:03.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.376+0000 7f508aa00640 1 --2- 172.21.15.123:0/3536589153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e9c020 0x55d4b5e9e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.376+0000 7f508aa00640 1 -- 172.21.15.123:0/3536589153 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4b5dbdae0 con 0x55d4b5e91760 2024-09-19T04:54:03.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.376+0000 7f508a000640 1 --2- 172.21.15.123:0/3536589153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e9c020 0x55d4b5e9e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:03.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.377+0000 7f5090a00640 1 --2- 172.21.15.123:0/2299760277 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f50700a3430 0x7f50700a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.377+0000 7f508aa00640 1 --2- 172.21.15.123:0/3536589153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e91760 0x55d4b5e9bae0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f507c00bbf0 tx=0x7f507c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87ff24eb5fa743f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:03.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.377+0000 7f5090a00640 1 -- 172.21.15.123:0/2299760277 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f50840f0940 con 0x7f508c10e270 2024-09-19T04:54:03.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.377+0000 7f508a000640 1 --2- 172.21.15.123:0/2299760277 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f50700a3430 0x7f50700a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.377+0000 7f5077e00640 1 -- 172.21.15.123:0/3536589153 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c004030 con 0x55d4b5e91760 2024-09-19T04:54:03.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.377+0000 7f5077e00640 1 -- 172.21.15.123:0/3536589153 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f507c0041d0 con 0x55d4b5e91760 2024-09-19T04:54:03.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.377+0000 7f5077e00640 1 -- 172.21.15.123:0/3536589153 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c0116a0 con 0x55d4b5e91760 2024-09-19T04:54:03.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.378+0000 7f5094a44740 1 -- 172.21.15.123:0/3536589153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e91760 msgr2=0x55d4b5e9bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:03.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.378+0000 7f508a000640 1 --2- 172.21.15.123:0/2299760277 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f50700a3430 0x7f50700a58f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55d4b5e8e750 tx=0x7f5078004730 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:03.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.378+0000 7f5094a44740 1 --2- 172.21.15.123:0/3536589153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e91760 0x55d4b5e9bae0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f507c00bbf0 tx=0x7f507c0107a0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.378+0000 7f5094a44740 1 -- 172.21.15.123:0/3536589153 shutdown_connections 2024-09-19T04:54:03.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.378+0000 7f5094a44740 1 --2- 172.21.15.123:0/3536589153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e9c020 0x55d4b5e9e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.378+0000 7f5094a44740 1 --2- 172.21.15.123:0/3536589153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e91760 0x55d4b5e9bae0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.378+0000 7f5094a44740 1 --2- 172.21.15.123:0/3536589153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e90e40 0x55d4b5e91220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.378+0000 7f5094a44740 1 -- 172.21.15.123:0/3536589153 >> 172.21.15.123:0/3536589153 conn(0x55d4b5e8d3d0 msgr2=0x55d4b5e8d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:03.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.378+0000 7f5094a44740 1 -- 172.21.15.123:0/3536589153 shutdown_connections 2024-09-19T04:54:03.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.378+0000 7f5094a44740 1 -- 172.21.15.123:0/3536589153 wait complete. 2024-09-19T04:54:03.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.379+0000 7f5094a44740 1 -- start start 2024-09-19T04:54:03.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.379+0000 7f5094a44740 1 Processor -- start 2024-09-19T04:54:03.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.380+0000 7f5094a44740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e90e40 0x55d4b5e98e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.380+0000 7f5094a44740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e91760 0x55d4b5e993b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.380+0000 7f5094a44740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e9c020 0x55d4b5e998f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.380+0000 7f5094a44740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d4b5e9ebf0 con 0x55d4b5e91760 2024-09-19T04:54:03.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.380+0000 7f5094a44740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d4b5db20f0 con 0x55d4b5e90e40 2024-09-19T04:54:03.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.380+0000 7f5094a44740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d4b5e9ed70 con 0x55d4b5e9c020 2024-09-19T04:54:03.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.380+0000 7f5094a44740 10 client.0 _ll_register_callbacks cb 0x55d4b5d99bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:03.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.380+0000 7f508a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e91760 0x55d4b5e993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.380+0000 7f508b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e9c020 0x55d4b5e998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.380+0000 7f508a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e91760 0x55d4b5e993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54046/0 (socket says 172.21.15.123:54046) 2024-09-19T04:54:03.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.380+0000 7f508aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e90e40 0x55d4b5e98e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.380+0000 7f508a000640 1 -- 172.21.15.123:0/2913621636 learned_addr learned my addr 172.21.15.123:0/2913621636 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:03.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.380+0000 7f508b400640 1 -- 172.21.15.123:0/2913621636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e90e40 msgr2=0x55d4b5e98e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:03.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.380+0000 7f508b400640 1 --2- 172.21.15.123:0/2913621636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e90e40 0x55d4b5e98e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.380+0000 7f508b400640 1 -- 172.21.15.123:0/2913621636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e91760 msgr2=0x55d4b5e993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:03.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.380+0000 7f508b400640 1 --2- 172.21.15.123:0/2913621636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e91760 0x55d4b5e993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.380+0000 7f508b400640 1 -- 172.21.15.123:0/2913621636 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4b5b683c0 con 0x55d4b5e9c020 2024-09-19T04:54:03.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.381+0000 7f508b400640 1 --2- 172.21.15.123:0/2913621636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e9c020 0x55d4b5e998f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f50840ee140 tx=0x7f50840bcb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39ba7f301eff5409 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:03.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.381+0000 7f5074c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:54:03.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.381+0000 7f5074c00640 1 -- 172.21.15.123:0/2913621636 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50840bccd0 con 0x55d4b5e9c020 2024-09-19T04:54:03.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.381+0000 7f5074c00640 1 -- 172.21.15.123:0/2913621636 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f50840b9340 con 0x55d4b5e9c020 2024-09-19T04:54:03.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.381+0000 7f5094a44740 1 -- 172.21.15.123:0/2913621636 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f507c0041d0 con 0x55d4b5e9c020 2024-09-19T04:54:03.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.381+0000 7f5074c00640 1 -- 172.21.15.123:0/2913621636 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f50840c1080 con 0x55d4b5e9c020 2024-09-19T04:54:03.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.381+0000 7f5094a44740 10 client.26689 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:03.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.381+0000 7f5094a44740 1 -- 172.21.15.123:0/2913621636 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d4b5e9ec50 con 0x55d4b5e9c020 2024-09-19T04:54:03.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.381+0000 7f5074c00640 1 -- 172.21.15.123:0/2913621636 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f507c0041d0 con 0x55d4b5e9c020 2024-09-19T04:54:03.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.382+0000 7f5074c00640 1 -- 172.21.15.123:0/2913621636 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f508404d030 con 0x55d4b5e9c020 2024-09-19T04:54:03.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.382+0000 7f5094a44740 10 client.26689 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:03.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.382+0000 7f5094a44740 10 client.26689 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:54:03.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.382+0000 7f5094a44740 20 client.26689 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:03.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.382+0000 7f5094a44740 1 --2- 172.21.15.123:0/2913621636 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d4b5eac070 0x55d4b5eae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.382+0000 7f5094a44740 4 client.26689 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:03.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.382+0000 7f5094a44740 1 -- 172.21.15.123:0/2913621636 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4b5e04950 con 0x55d4b5eac070 2024-09-19T04:54:03.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.382+0000 7f508aa00640 1 --2- 172.21.15.123:0/2913621636 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d4b5eac070 0x55d4b5eae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.383+0000 7f508aa00640 1 --2- 172.21.15.123:0/2913621636 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d4b5eac070 0x55d4b5eae4f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d6260bca5f0b7430 server_cookie=a5e0527f3d118d0c in_seq=0 out_seq=0 2024-09-19T04:54:03.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.383+0000 7f5074c00640 10 client.26689 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:54:03.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.383+0000 7f5074c00640 1 -- 172.21.15.123:0/2913621636 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d4b5e04950 con 0x55d4b5eac070 2024-09-19T04:54:03.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.383+0000 7f5074c00640 10 client.26689 handle_command_reply: tid=0 2024-09-19T04:54:03.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.383+0000 7f5094a44740 1 client.26689 shutdown 2024-09-19T04:54:03.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.384+0000 7f5094a44740 1 -- 172.21.15.123:0/2913621636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e9c020 msgr2=0x55d4b5e998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:03.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.384+0000 7f5094a44740 1 --2- 172.21.15.123:0/2913621636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e9c020 0x55d4b5e998f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f50840ee140 tx=0x7f50840bcb70 comp rx=0 tx=0).stop 2024-09-19T04:54:03.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.384+0000 7f5094a44740 1 -- 172.21.15.123:0/2913621636 shutdown_connections 2024-09-19T04:54:03.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.384+0000 7f5094a44740 1 --2- 172.21.15.123:0/2913621636 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d4b5eac070 0x55d4b5eae4f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.384+0000 7f5094a44740 1 --2- 172.21.15.123:0/2913621636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e9c020 0x55d4b5e998f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.384+0000 7f5094a44740 1 --2- 172.21.15.123:0/2913621636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e91760 0x55d4b5e993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.384+0000 7f5094a44740 1 --2- 172.21.15.123:0/2913621636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e90e40 0x55d4b5e98e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.384+0000 7f5074c00640 0 client.26689 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:54:03.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.384+0000 7f5094a44740 1 -- 172.21.15.123:0/2913621636 >> 172.21.15.123:0/2913621636 conn(0x55d4b5e8d3d0 msgr2=0x55d4b5e930b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:03.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.384+0000 7f5094a44740 1 -- 172.21.15.123:0/2913621636 shutdown_connections 2024-09-19T04:54:03.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.384+0000 7f5094a44740 1 -- 172.21.15.123:0/2913621636 wait complete. 2024-09-19T04:54:03.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.384+0000 7f5094a44740 20 client.26689 trim_cache size 0 max 16384 2024-09-19T04:54:03.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.398+0000 7f5094a44740 1 Processor -- start 2024-09-19T04:54:03.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.398+0000 7f5094a44740 1 -- start start 2024-09-19T04:54:03.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.398+0000 7f5094a44740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e90e40 0x55d4b5e88300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.398+0000 7f5094a44740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e91760 0x55d4b5eb3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.398+0000 7f5094a44740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e9c020 0x55d4b5eb4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.398+0000 7f5094a44740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f508c1af880 con 0x55d4b5e90e40 2024-09-19T04:54:03.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.398+0000 7f5094a44740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f508c1b1450 con 0x55d4b5e91760 2024-09-19T04:54:03.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.398+0000 7f5094a44740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f508c1aff80 con 0x55d4b5e9c020 2024-09-19T04:54:03.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.398+0000 7f508aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e90e40 0x55d4b5e88300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.398+0000 7f508aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e90e40 0x55d4b5e88300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54050/0 (socket says 172.21.15.123:54050) 2024-09-19T04:54:03.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.398+0000 7f508aa00640 1 -- 172.21.15.123:0/1899527314 learned_addr learned my addr 172.21.15.123:0/1899527314 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:03.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.398+0000 7f508a000640 1 --2- 172.21.15.123:0/1899527314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e91760 0x55d4b5eb3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.398+0000 7f508b400640 1 --2- 172.21.15.123:0/1899527314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e9c020 0x55d4b5eb4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.399+0000 7f508aa00640 1 -- 172.21.15.123:0/1899527314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e9c020 msgr2=0x55d4b5eb4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:03.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.399+0000 7f508aa00640 1 --2- 172.21.15.123:0/1899527314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e9c020 0x55d4b5eb4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.399+0000 7f508aa00640 1 -- 172.21.15.123:0/1899527314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e91760 msgr2=0x55d4b5eb3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:03.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.399+0000 7f508aa00640 1 --2- 172.21.15.123:0/1899527314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e91760 0x55d4b5eb3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.399+0000 7f508aa00640 1 -- 172.21.15.123:0/1899527314 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4b5e97e20 con 0x55d4b5e90e40 2024-09-19T04:54:03.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.399+0000 7f508aa00640 1 --2- 172.21.15.123:0/1899527314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e90e40 0x55d4b5e88300 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f507c00bbf0 tx=0x7f507c003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b88b780eb6c6c2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:03.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.399+0000 7f5077400640 1 -- 172.21.15.123:0/1899527314 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c02c020 con 0x55d4b5e90e40 2024-09-19T04:54:03.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.399+0000 7f5077400640 1 -- 172.21.15.123:0/1899527314 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f507c003ad0 con 0x55d4b5e90e40 2024-09-19T04:54:03.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.399+0000 7f5077400640 1 -- 172.21.15.123:0/1899527314 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c029690 con 0x55d4b5e90e40 2024-09-19T04:54:03.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.400+0000 7f5094a44740 1 -- 172.21.15.123:0/1899527314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e90e40 msgr2=0x55d4b5e88300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:03.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.400+0000 7f5094a44740 1 --2- 172.21.15.123:0/1899527314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e90e40 0x55d4b5e88300 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f507c00bbf0 tx=0x7f507c003d60 comp rx=0 tx=0).stop 2024-09-19T04:54:03.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.400+0000 7f5094a44740 1 -- 172.21.15.123:0/1899527314 shutdown_connections 2024-09-19T04:54:03.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.400+0000 7f5094a44740 1 --2- 172.21.15.123:0/1899527314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e9c020 0x55d4b5eb4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.400+0000 7f5094a44740 1 --2- 172.21.15.123:0/1899527314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e91760 0x55d4b5eb3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.400+0000 7f5094a44740 1 --2- 172.21.15.123:0/1899527314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e90e40 0x55d4b5e88300 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.400+0000 7f5094a44740 1 -- 172.21.15.123:0/1899527314 >> 172.21.15.123:0/1899527314 conn(0x55d4b5e8d3d0 msgr2=0x55d4b5e8c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:03.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.400+0000 7f5094a44740 1 -- 172.21.15.123:0/1899527314 shutdown_connections 2024-09-19T04:54:03.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.400+0000 7f5094a44740 1 -- 172.21.15.123:0/1899527314 wait complete. 2024-09-19T04:54:03.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.400+0000 7f5094a44740 1 -- start start 2024-09-19T04:54:03.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.401+0000 7f5094a44740 1 Processor -- start 2024-09-19T04:54:03.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.401+0000 7f5094a44740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e90e40 0x55d4b5e95f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.401+0000 7f5094a44740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e91760 0x55d4b5e96470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.401+0000 7f5094a44740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e9c020 0x55d4b5e969b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.401+0000 7f5094a44740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f508c1b0d80 con 0x55d4b5e90e40 2024-09-19T04:54:03.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.401+0000 7f5094a44740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f508c1b0680 con 0x55d4b5e91760 2024-09-19T04:54:03.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.401+0000 7f5094a44740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f508c1ad880 con 0x55d4b5e9c020 2024-09-19T04:54:03.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.401+0000 7f5094a44740 10 client.0 _ll_register_callbacks cb 0x7f50840095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:03.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.401+0000 7f508aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e90e40 0x55d4b5e95f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.401+0000 7f508aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e90e40 0x55d4b5e95f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54058/0 (socket says 172.21.15.123:54058) 2024-09-19T04:54:03.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.401+0000 7f508aa00640 1 -- 172.21.15.123:0/3868858236 learned_addr learned my addr 172.21.15.123:0/3868858236 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:03.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.401+0000 7f508b400640 1 --2- 172.21.15.123:0/3868858236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e9c020 0x55d4b5e969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.401+0000 7f508a000640 1 --2- 172.21.15.123:0/3868858236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e91760 0x55d4b5e96470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.402+0000 7f508aa00640 1 -- 172.21.15.123:0/3868858236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e9c020 msgr2=0x55d4b5e969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:03.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.402+0000 7f508aa00640 1 --2- 172.21.15.123:0/3868858236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e9c020 0x55d4b5e969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.402+0000 7f508aa00640 1 -- 172.21.15.123:0/3868858236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e91760 msgr2=0x55d4b5e96470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:03.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.402+0000 7f508aa00640 1 --2- 172.21.15.123:0/3868858236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e91760 0x55d4b5e96470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.402+0000 7f508aa00640 1 -- 172.21.15.123:0/3868858236 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4b5f15dd0 con 0x55d4b5e90e40 2024-09-19T04:54:03.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.402+0000 7f508aa00640 1 --2- 172.21.15.123:0/3868858236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e90e40 0x55d4b5e95f30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f507c003d30 tx=0x7f507c005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2405c7b15f72be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:03.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.402+0000 7f5074c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:03.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.402+0000 7f5074c00640 1 -- 172.21.15.123:0/3868858236 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c0035f0 con 0x55d4b5e90e40 2024-09-19T04:54:03.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.402+0000 7f5074c00640 1 -- 172.21.15.123:0/3868858236 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f507c003790 con 0x55d4b5e90e40 2024-09-19T04:54:03.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.402+0000 7f5094a44740 1 -- 172.21.15.123:0/3868858236 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f507c003ad0 con 0x55d4b5e90e40 2024-09-19T04:54:03.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.403+0000 7f5074c00640 1 -- 172.21.15.123:0/3868858236 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f507c040050 con 0x55d4b5e90e40 2024-09-19T04:54:03.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.403+0000 7f5094a44740 10 client.16575 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:03.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.403+0000 7f5094a44740 1 -- 172.21.15.123:0/3868858236 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d4b5f170b0 con 0x55d4b5e90e40 2024-09-19T04:54:03.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.403+0000 7f5074c00640 1 -- 172.21.15.123:0/3868858236 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f507c003ad0 con 0x55d4b5e90e40 2024-09-19T04:54:03.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.403+0000 7f5074c00640 1 -- 172.21.15.123:0/3868858236 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f507c0274d0 con 0x55d4b5e90e40 2024-09-19T04:54:03.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.403+0000 7f5094a44740 10 client.16575 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:03.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.403+0000 7f5094a44740 10 client.16575 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:54:03.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.403+0000 7f5094a44740 20 client.16575 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:03.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.403+0000 7f5094a44740 1 --2- 172.21.15.123:0/3868858236 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d4b5f17620 0x55d4b5f19a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:03.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.403+0000 7f5094a44740 4 client.16575 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:03.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.403+0000 7f5094a44740 1 -- 172.21.15.123:0/3868858236 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4b5dfee60 con 0x55d4b5f17620 2024-09-19T04:54:03.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.404+0000 7f508a000640 1 --2- 172.21.15.123:0/3868858236 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d4b5f17620 0x55d4b5f19a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:03.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.404+0000 7f508a000640 1 --2- 172.21.15.123:0/3868858236 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d4b5f17620 0x55d4b5f19a40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b062d6069019f37f server_cookie=658250a706f5cb2 in_seq=0 out_seq=0 2024-09-19T04:54:03.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.404+0000 7f5074c00640 10 client.16575 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:54:03.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.404+0000 7f5074c00640 1 -- 172.21.15.123:0/3868858236 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4b5dfee60 con 0x55d4b5f17620 2024-09-19T04:54:03.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.404+0000 7f5074c00640 10 client.16575 handle_command_reply: tid=0 2024-09-19T04:54:03.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.404+0000 7f5094a44740 1 client.16575 shutdown 2024-09-19T04:54:03.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.405+0000 7f5094a44740 1 -- 172.21.15.123:0/3868858236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e90e40 msgr2=0x55d4b5e95f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:03.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.405+0000 7f5094a44740 1 --2- 172.21.15.123:0/3868858236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e90e40 0x55d4b5e95f30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f507c003d30 tx=0x7f507c005750 comp rx=0 tx=0).stop 2024-09-19T04:54:03.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.405+0000 7f5094a44740 1 -- 172.21.15.123:0/3868858236 shutdown_connections 2024-09-19T04:54:03.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.405+0000 7f5094a44740 1 --2- 172.21.15.123:0/3868858236 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55d4b5f17620 0x55d4b5f19a40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.405+0000 7f5094a44740 1 --2- 172.21.15.123:0/3868858236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4b5e9c020 0x55d4b5e969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.405+0000 7f5074c00640 0 client.16575 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:54:03.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.405+0000 7f5094a44740 1 --2- 172.21.15.123:0/3868858236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b5e91760 0x55d4b5e96470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.405+0000 7f5094a44740 1 --2- 172.21.15.123:0/3868858236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4b5e90e40 0x55d4b5e95f30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.405+0000 7f5094a44740 1 -- 172.21.15.123:0/3868858236 >> 172.21.15.123:0/3868858236 conn(0x55d4b5e8d3d0 msgr2=0x55d4b5e9a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:03.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.405+0000 7f5094a44740 1 -- 172.21.15.123:0/3868858236 shutdown_connections 2024-09-19T04:54:03.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.405+0000 7f5094a44740 1 -- 172.21.15.123:0/3868858236 wait complete. 2024-09-19T04:54:03.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.405+0000 7f5094a44740 20 client.16575 trim_cache size 0 max 16384 2024-09-19T04:54:03.411 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:03.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.406+0000 7f508be00640 1 -- 172.21.15.123:0/2299760277 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f50700a3430 msgr2=0x7f50700a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:03.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.406+0000 7f508be00640 1 --2- 172.21.15.123:0/2299760277 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f50700a3430 0x7f50700a58f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55d4b5e8e750 tx=0x7f5078004730 comp rx=0 tx=0).stop 2024-09-19T04:54:03.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.406+0000 7f508be00640 1 -- 172.21.15.123:0/2299760277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f508c10e270 msgr2=0x7f508c19ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:03.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.406+0000 7f508be00640 1 --2- 172.21.15.123:0/2299760277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f508c10e270 0x7f508c19ed80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5084005e30 tx=0x7f5084003c10 comp rx=0 tx=0).stop 2024-09-19T04:54:03.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.406+0000 7f508be00640 1 -- 172.21.15.123:0/2299760277 shutdown_connections 2024-09-19T04:54:03.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.406+0000 7f508be00640 1 --2- 172.21.15.123:0/2299760277 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f50700a3430 0x7f50700a58f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.406+0000 7f508be00640 1 --2- 172.21.15.123:0/2299760277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f508c10e270 0x7f508c19ed80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.406+0000 7f508be00640 1 --2- 172.21.15.123:0/2299760277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f508c074ee0 0x7f508c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.406+0000 7f508be00640 1 --2- 172.21.15.123:0/2299760277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f508c0745c0 0x7f508c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:03.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.406+0000 7f508be00640 1 -- 172.21.15.123:0/2299760277 >> 172.21.15.123:0/2299760277 conn(0x7f508c0fd710 msgr2=0x7f508c0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:03.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.406+0000 7f508be00640 1 -- 172.21.15.123:0/2299760277 shutdown_connections 2024-09-19T04:54:03.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:03.406+0000 7f508be00640 1 -- 172.21.15.123:0/2299760277 wait complete. 2024-09-19T04:54:03.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:03 smithi123 ceph-mon[30511]: pgmap v225: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:54:03.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:03 smithi193 ceph-mon[35592]: pgmap v225: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:54:03.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:03 smithi183 ceph-mon[35333]: pgmap v225: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:54:04.234 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:54:04.607 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:05.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.186+0000 7f7f6dc00640 1 -- 172.21.15.123:0/165844868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f68102d30 msgr2=0x7f7f68103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:05.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.186+0000 7f7f6dc00640 1 --2- 172.21.15.123:0/165844868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f68102d30 0x7f7f68103110 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009900 tx=0x7f7f5802a480 comp rx=0 tx=0).stop 2024-09-19T04:54:05.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.186+0000 7f7f6dc00640 1 -- 172.21.15.123:0/165844868 shutdown_connections 2024-09-19T04:54:05.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.186+0000 7f7f6dc00640 1 --2- 172.21.15.123:0/165844868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f6810c160 0x7f7f6810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.186+0000 7f7f6dc00640 1 --2- 172.21.15.123:0/165844868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f68103650 0x7f7f6810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.186+0000 7f7f6dc00640 1 --2- 172.21.15.123:0/165844868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f68102d30 0x7f7f68103110 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.186+0000 7f7f6dc00640 1 -- 172.21.15.123:0/165844868 >> 172.21.15.123:0/165844868 conn(0x7f7f680fc7d0 msgr2=0x7f7f680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:05.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.186+0000 7f7f6dc00640 1 -- 172.21.15.123:0/165844868 shutdown_connections 2024-09-19T04:54:05.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.186+0000 7f7f6dc00640 1 -- 172.21.15.123:0/165844868 wait complete. 2024-09-19T04:54:05.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.187+0000 7f7f6dc00640 1 Processor -- start 2024-09-19T04:54:05.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.187+0000 7f7f6dc00640 1 -- start start 2024-09-19T04:54:05.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.188+0000 7f7f6dc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f68102d30 0x7f7f6819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.188+0000 7f7f6dc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f68103650 0x7f7f6819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.188+0000 7f7f6dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f6810c160 0x7f7f6819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.188+0000 7f7f6dc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f68111fb0 con 0x7f7f6810c160 2024-09-19T04:54:05.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.188+0000 7f7f6dc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7f68111e30 con 0x7f7f68102d30 2024-09-19T04:54:05.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.188+0000 7f7f6dc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7f68112130 con 0x7f7f68103650 2024-09-19T04:54:05.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.188+0000 7f7f67400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f6810c160 0x7f7f6819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.188+0000 7f7f67400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f6810c160 0x7f7f6819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54074/0 (socket says 172.21.15.123:54074) 2024-09-19T04:54:05.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.188+0000 7f7f67400640 1 -- 172.21.15.123:0/780566358 learned_addr learned my addr 172.21.15.123:0/780566358 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:05.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.188+0000 7f7f66a00640 1 --2- 172.21.15.123:0/780566358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f68102d30 0x7f7f6819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.188+0000 7f7f5e000640 1 --2- 172.21.15.123:0/780566358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f68103650 0x7f7f6819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.188+0000 7f7f67400640 1 -- 172.21.15.123:0/780566358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f68103650 msgr2=0x7f7f6819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:05.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.188+0000 7f7f67400640 1 --2- 172.21.15.123:0/780566358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f68103650 0x7f7f6819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.188+0000 7f7f67400640 1 -- 172.21.15.123:0/780566358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f68102d30 msgr2=0x7f7f6819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:05.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.188+0000 7f7f67400640 1 --2- 172.21.15.123:0/780566358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f68102d30 0x7f7f6819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.188+0000 7f7f67400640 1 -- 172.21.15.123:0/780566358 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6819f600 con 0x7f7f6810c160 2024-09-19T04:54:05.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.189+0000 7f7f67400640 1 --2- 172.21.15.123:0/780566358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f6810c160 0x7f7f6819ef20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000eaa0 tx=0x7f7f6000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:05.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.189+0000 7f7f5fe00640 1 -- 172.21.15.123:0/780566358 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6000cdb0 con 0x7f7f6810c160 2024-09-19T04:54:05.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.189+0000 7f7f5fe00640 1 -- 172.21.15.123:0/780566358 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f60005020 con 0x7f7f6810c160 2024-09-19T04:54:05.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.189+0000 7f7f5fe00640 1 -- 172.21.15.123:0/780566358 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f600116b0 con 0x7f7f6810c160 2024-09-19T04:54:05.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.189+0000 7f7f6dc00640 1 -- 172.21.15.123:0/780566358 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6819f8f0 con 0x7f7f6810c160 2024-09-19T04:54:05.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.189+0000 7f7f6dc00640 1 -- 172.21.15.123:0/780566358 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f681a71d0 con 0x7f7f6810c160 2024-09-19T04:54:05.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.193+0000 7f7f705f4740 1 Processor -- start 2024-09-19T04:54:05.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.193+0000 7f7f5fe00640 1 -- 172.21.15.123:0/780566358 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f7f60005560 con 0x7f7f6810c160 2024-09-19T04:54:05.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.193+0000 7f7f705f4740 1 -- start start 2024-09-19T04:54:05.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.193+0000 7f7f705f4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e1081c70 0x5610e1082050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.193+0000 7f7f705f4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1082590 0x5610e108c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.193+0000 7f7f705f4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e108ce50 0x5610e108f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.193+0000 7f7f705f4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5610e0fe83e0 con 0x5610e1081c70 2024-09-19T04:54:05.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.193+0000 7f7f705f4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5610e0f8fd80 con 0x5610e108ce50 2024-09-19T04:54:05.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.193+0000 7f7f705f4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5610e0fe43a0 con 0x5610e1082590 2024-09-19T04:54:05.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.194+0000 7f7f66a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1082590 0x5610e108c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.194+0000 7f7f66a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1082590 0x5610e108c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:46136/0 (socket says 172.21.15.123:46136) 2024-09-19T04:54:05.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.194+0000 7f7f66a00640 1 -- 172.21.15.123:0/1672352433 learned_addr learned my addr 172.21.15.123:0/1672352433 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:05.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.194+0000 7f7f5e000640 1 --2- 172.21.15.123:0/1672352433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e1081c70 0x5610e1082050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.194+0000 7f7f5e000640 1 --2- 172.21.15.123:0/1672352433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e108ce50 0x5610e108f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.194+0000 7f7f66a00640 1 -- 172.21.15.123:0/1672352433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e108ce50 msgr2=0x5610e108f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:05.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.194+0000 7f7f66a00640 1 --2- 172.21.15.123:0/1672352433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e108ce50 0x5610e108f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.194+0000 7f7f66a00640 1 -- 172.21.15.123:0/1672352433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e1081c70 msgr2=0x5610e1082050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:05.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.194+0000 7f7f66a00640 1 --2- 172.21.15.123:0/1672352433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e1081c70 0x5610e1082050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.194+0000 7f7f66a00640 1 -- 172.21.15.123:0/1672352433 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610e0faeae0 con 0x5610e1082590 2024-09-19T04:54:05.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.194+0000 7f7f5fe00640 1 --2- 172.21.15.123:0/780566358 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7f4c0a33b0 0x7f7f4c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.195+0000 7f7f5fe00640 1 -- 172.21.15.123:0/780566358 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f7f600cff70 con 0x7f7f6810c160 2024-09-19T04:54:05.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.195+0000 7f7f66a00640 1 --2- 172.21.15.123:0/1672352433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1082590 0x5610e108c910 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009900 tx=0x7f7f58004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=265a05e5a70acf7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:05.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.195+0000 7f7f5e000640 1 --2- 172.21.15.123:0/780566358 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7f4c0a33b0 0x7f7f4c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.195+0000 7f7f57e00640 1 -- 172.21.15.123:0/1672352433 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58035040 con 0x5610e1082590 2024-09-19T04:54:05.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.195+0000 7f7f57e00640 1 -- 172.21.15.123:0/1672352433 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f58005520 con 0x5610e1082590 2024-09-19T04:54:05.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.195+0000 7f7f57e00640 1 -- 172.21.15.123:0/1672352433 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f580037f0 con 0x5610e1082590 2024-09-19T04:54:05.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.196+0000 7f7f5e000640 1 --2- 172.21.15.123:0/780566358 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7f4c0a33b0 0x7f7f4c0a5870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009fd0 tx=0x7f7f50015040 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:05.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.196+0000 7f7f705f4740 1 -- 172.21.15.123:0/1672352433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1082590 msgr2=0x5610e108c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:05.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.196+0000 7f7f705f4740 1 --2- 172.21.15.123:0/1672352433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1082590 0x5610e108c910 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009900 tx=0x7f7f58004040 comp rx=0 tx=0).stop 2024-09-19T04:54:05.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.196+0000 7f7f705f4740 1 -- 172.21.15.123:0/1672352433 shutdown_connections 2024-09-19T04:54:05.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.196+0000 7f7f705f4740 1 --2- 172.21.15.123:0/1672352433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e108ce50 0x5610e108f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.196+0000 7f7f705f4740 1 --2- 172.21.15.123:0/1672352433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1082590 0x5610e108c910 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.196+0000 7f7f705f4740 1 --2- 172.21.15.123:0/1672352433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e1081c70 0x5610e1082050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.196+0000 7f7f705f4740 1 -- 172.21.15.123:0/1672352433 >> 172.21.15.123:0/1672352433 conn(0x5610e107e200 msgr2=0x5610e107e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:05.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.196+0000 7f7f705f4740 1 -- 172.21.15.123:0/1672352433 shutdown_connections 2024-09-19T04:54:05.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.196+0000 7f7f705f4740 1 -- 172.21.15.123:0/1672352433 wait complete. 2024-09-19T04:54:05.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.197+0000 7f7f705f4740 1 -- start start 2024-09-19T04:54:05.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.197+0000 7f7f705f4740 1 Processor -- start 2024-09-19T04:54:05.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.197+0000 7f7f705f4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1081c70 0x5610e1089ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.197+0000 7f7f705f4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1082590 0x5610e108a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.197+0000 7f7f705f4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 0x5610e108a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.197+0000 7f7f705f4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5610e108fa20 con 0x5610e108ce50 2024-09-19T04:54:05.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.197+0000 7f7f705f4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5610e0fa30f0 con 0x5610e1081c70 2024-09-19T04:54:05.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.197+0000 7f7f705f4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5610e108fba0 con 0x5610e1082590 2024-09-19T04:54:05.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.197+0000 7f7f705f4740 10 client.0 _ll_register_callbacks cb 0x5610e0f8abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:05.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.198+0000 7f7f66a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1081c70 0x5610e1089ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.198+0000 7f7f66a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1081c70 0x5610e1089ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:49556/0 (socket says 172.21.15.123:49556) 2024-09-19T04:54:05.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.198+0000 7f7f66a00640 1 -- 172.21.15.123:0/1345469131 learned_addr learned my addr 172.21.15.123:0/1345469131 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:05.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.198+0000 7f7f5e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1082590 0x5610e108a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.198+0000 7f7f67400640 1 --2- 172.21.15.123:0/1345469131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 0x5610e108a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.198+0000 7f7f66a00640 1 -- 172.21.15.123:0/1345469131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1082590 msgr2=0x5610e108a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:05.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.198+0000 7f7f66a00640 1 --2- 172.21.15.123:0/1345469131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1082590 0x5610e108a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.198+0000 7f7f66a00640 1 -- 172.21.15.123:0/1345469131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 msgr2=0x5610e108a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:05.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.198+0000 7f7f66a00640 1 --2- 172.21.15.123:0/1345469131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 0x5610e108a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.198+0000 7f7f66a00640 1 -- 172.21.15.123:0/1345469131 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610e0d593c0 con 0x5610e1081c70 2024-09-19T04:54:05.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.198+0000 7f7f67400640 1 --2- 172.21.15.123:0/1345469131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 0x5610e108a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:54:05.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.198+0000 7f7f5e000640 1 --2- 172.21.15.123:0/1345469131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1082590 0x5610e108a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:05.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.199+0000 7f7f66a00640 1 --2- 172.21.15.123:0/1345469131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1081c70 0x5610e1089ce0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7f58004500 tx=0x7f7f5802a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=127a4239e73df99 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:05.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.199+0000 7f7f54c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:54:05.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.199+0000 7f7f54c00640 1 -- 172.21.15.123:0/1345469131 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58035050 con 0x5610e1081c70 2024-09-19T04:54:05.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.199+0000 7f7f705f4740 1 -- 172.21.15.123:0/1345469131 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f58005520 con 0x5610e1081c70 2024-09-19T04:54:05.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.199+0000 7f7f54c00640 1 -- 172.21.15.123:0/1345469131 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f7f58004e60 con 0x5610e1081c70 2024-09-19T04:54:05.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.199+0000 7f7f54c00640 1 -- 172.21.15.123:0/1345469131 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7f58009520 con 0x5610e1081c70 2024-09-19T04:54:05.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.199+0000 7f7f705f4740 10 client.26687 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:05.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.199+0000 7f7f705f4740 1 -- 172.21.15.123:0/1345469131 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5610e109b9f0 con 0x5610e1081c70 2024-09-19T04:54:05.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.200+0000 7f7f54c00640 1 -- 172.21.15.123:0/1345469131 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f58005520 con 0x5610e1081c70 2024-09-19T04:54:05.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.200+0000 7f7f54c00640 1 -- 172.21.15.123:0/1345469131 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f7f58031bb0 con 0x5610e1081c70 2024-09-19T04:54:05.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.200+0000 7f7f705f4740 10 client.26687 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:05.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.200+0000 7f7f705f4740 10 client.26687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:05.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.200+0000 7f7f705f4740 20 client.26687 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:05.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.200+0000 7f7f705f4740 1 --2- 172.21.15.123:0/1345469131 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5610e109cd40 0x5610e109f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.200+0000 7f7f705f4740 4 client.26687 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:05.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.200+0000 7f7f705f4740 1 -- 172.21.15.123:0/1345469131 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610e0ff5950 con 0x5610e109cd40 2024-09-19T04:54:05.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.200+0000 7f7f5e000640 1 --2- 172.21.15.123:0/1345469131 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5610e109cd40 0x5610e109f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.201+0000 7f7f5e000640 1 --2- 172.21.15.123:0/1345469131 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5610e109cd40 0x5610e109f130 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50944dde2eb8756c server_cookie=4224b146e5539c52 in_seq=0 out_seq=0 2024-09-19T04:54:05.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.201+0000 7f7f54c00640 10 client.26687 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:05.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.202+0000 7f7f54c00640 1 -- 172.21.15.123:0/1345469131 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610e0ff5950 con 0x5610e109cd40 2024-09-19T04:54:05.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.202+0000 7f7f54c00640 10 client.26687 handle_command_reply: tid=0 2024-09-19T04:54:05.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.202+0000 7f7f705f4740 1 client.26687 shutdown 2024-09-19T04:54:05.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.202+0000 7f7f705f4740 1 -- 172.21.15.123:0/1345469131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1081c70 msgr2=0x5610e1089ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:05.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.202+0000 7f7f705f4740 1 --2- 172.21.15.123:0/1345469131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1081c70 0x5610e1089ce0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7f58004500 tx=0x7f7f5802a480 comp rx=0 tx=0).stop 2024-09-19T04:54:05.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.202+0000 7f7f705f4740 1 -- 172.21.15.123:0/1345469131 shutdown_connections 2024-09-19T04:54:05.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.202+0000 7f7f705f4740 1 --2- 172.21.15.123:0/1345469131 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5610e109cd40 0x5610e109f130 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.202+0000 7f7f705f4740 1 --2- 172.21.15.123:0/1345469131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 0x5610e108a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.202+0000 7f7f705f4740 1 --2- 172.21.15.123:0/1345469131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1082590 0x5610e108a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.202+0000 7f7f54c00640 0 client.26687 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:05.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.202+0000 7f7f705f4740 1 --2- 172.21.15.123:0/1345469131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1081c70 0x5610e1089ce0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.202+0000 7f7f705f4740 1 -- 172.21.15.123:0/1345469131 >> 172.21.15.123:0/1345469131 conn(0x5610e107e200 msgr2=0x5610e1083f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:05.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.202+0000 7f7f705f4740 1 -- 172.21.15.123:0/1345469131 shutdown_connections 2024-09-19T04:54:05.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.202+0000 7f7f705f4740 1 -- 172.21.15.123:0/1345469131 wait complete. 2024-09-19T04:54:05.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.203+0000 7f7f705f4740 20 client.26687 trim_cache size 0 max 16384 2024-09-19T04:54:05.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.216+0000 7f7f705f4740 1 Processor -- start 2024-09-19T04:54:05.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.216+0000 7f7f705f4740 1 -- start start 2024-09-19T04:54:05.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.216+0000 7f7f705f4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1081c70 0x5610e108c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.217+0000 7f7f705f4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1082590 0x5610e10a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.217+0000 7f7f705f4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 0x5610e10a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.217+0000 7f7f705f4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f681afae0 con 0x5610e108ce50 2024-09-19T04:54:05.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.217+0000 7f7f705f4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7f681b16b0 con 0x5610e1082590 2024-09-19T04:54:05.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.217+0000 7f7f705f4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7f681b01e0 con 0x5610e1081c70 2024-09-19T04:54:05.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.217+0000 7f7f66a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1081c70 0x5610e108c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.217+0000 7f7f66a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1081c70 0x5610e108c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:46156/0 (socket says 172.21.15.123:46156) 2024-09-19T04:54:05.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.217+0000 7f7f66a00640 1 -- 172.21.15.123:0/1318695545 learned_addr learned my addr 172.21.15.123:0/1318695545 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:05.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.217+0000 7f7f67400640 1 --2- 172.21.15.123:0/1318695545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 0x5610e10a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.217+0000 7f7f5e000640 1 --2- 172.21.15.123:0/1318695545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1082590 0x5610e10a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.217+0000 7f7f66a00640 1 -- 172.21.15.123:0/1318695545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1082590 msgr2=0x5610e10a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:05.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.217+0000 7f7f66a00640 1 --2- 172.21.15.123:0/1318695545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1082590 0x5610e10a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.217+0000 7f7f66a00640 1 -- 172.21.15.123:0/1318695545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 msgr2=0x5610e10a4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:05.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.217+0000 7f7f66a00640 1 --2- 172.21.15.123:0/1318695545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 0x5610e10a4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.217+0000 7f7f66a00640 1 -- 172.21.15.123:0/1318695545 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610e107e030 con 0x5610e1081c70 2024-09-19T04:54:05.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.217+0000 7f7f5e000640 1 --2- 172.21.15.123:0/1318695545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1082590 0x5610e10a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:54:05.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.218+0000 7f7f66a00640 1 --2- 172.21.15.123:0/1318695545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1081c70 0x5610e108c010 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7f580039f0 tx=0x7f7f58003a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1d79fd53f223e6c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:05.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.218+0000 7f7f57400640 1 -- 172.21.15.123:0/1318695545 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58035040 con 0x5610e1081c70 2024-09-19T04:54:05.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.218+0000 7f7f57400640 1 -- 172.21.15.123:0/1318695545 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f580315b0 con 0x5610e1081c70 2024-09-19T04:54:05.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.218+0000 7f7f57400640 1 -- 172.21.15.123:0/1318695545 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5804e6d0 con 0x5610e1081c70 2024-09-19T04:54:05.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.219+0000 7f7f705f4740 1 -- 172.21.15.123:0/1318695545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1081c70 msgr2=0x5610e108c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:05.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.219+0000 7f7f705f4740 1 --2- 172.21.15.123:0/1318695545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1081c70 0x5610e108c010 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7f580039f0 tx=0x7f7f58003a20 comp rx=0 tx=0).stop 2024-09-19T04:54:05.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.219+0000 7f7f705f4740 1 -- 172.21.15.123:0/1318695545 shutdown_connections 2024-09-19T04:54:05.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.219+0000 7f7f705f4740 1 --2- 172.21.15.123:0/1318695545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 0x5610e10a4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.219+0000 7f7f705f4740 1 --2- 172.21.15.123:0/1318695545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1082590 0x5610e10a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.219+0000 7f7f705f4740 1 --2- 172.21.15.123:0/1318695545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1081c70 0x5610e108c010 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.219+0000 7f7f705f4740 1 -- 172.21.15.123:0/1318695545 >> 172.21.15.123:0/1318695545 conn(0x5610e107e200 msgr2=0x5610e107d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:05.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.219+0000 7f7f705f4740 1 -- 172.21.15.123:0/1318695545 shutdown_connections 2024-09-19T04:54:05.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.219+0000 7f7f705f4740 1 -- 172.21.15.123:0/1318695545 wait complete. 2024-09-19T04:54:05.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.219+0000 7f7f705f4740 1 -- start start 2024-09-19T04:54:05.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.219+0000 7f7f705f4740 1 Processor -- start 2024-09-19T04:54:05.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.220+0000 7f7f705f4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1081c70 0x5610e1095700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.220+0000 7f7f705f4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1082590 0x5610e1090a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.220+0000 7f7f705f4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 0x5610e10911c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.220+0000 7f7f705f4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f681b0fe0 con 0x5610e108ce50 2024-09-19T04:54:05.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.220+0000 7f7f705f4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7f681b08e0 con 0x5610e1082590 2024-09-19T04:54:05.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.220+0000 7f7f705f4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7f681adae0 con 0x5610e1081c70 2024-09-19T04:54:05.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.220+0000 7f7f705f4740 10 client.0 _ll_register_callbacks cb 0x7f7f6806b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:05.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.220+0000 7f7f67400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 0x5610e10911c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.220+0000 7f7f5e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1082590 0x5610e1090a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.220+0000 7f7f67400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 0x5610e10911c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54126/0 (socket says 172.21.15.123:54126) 2024-09-19T04:54:05.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.220+0000 7f7f67400640 1 -- 172.21.15.123:0/3962582783 learned_addr learned my addr 172.21.15.123:0/3962582783 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:05.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.220+0000 7f7f66a00640 1 --2- 172.21.15.123:0/3962582783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1081c70 0x5610e1095700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.221+0000 7f7f5e000640 1 -- 172.21.15.123:0/3962582783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1081c70 msgr2=0x5610e1095700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:05.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.221+0000 7f7f5e000640 1 --2- 172.21.15.123:0/3962582783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1081c70 0x5610e1095700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.221+0000 7f7f5e000640 1 -- 172.21.15.123:0/3962582783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 msgr2=0x5610e10911c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:05.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.221+0000 7f7f5e000640 1 --2- 172.21.15.123:0/3962582783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 0x5610e10911c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.221+0000 7f7f5e000640 1 -- 172.21.15.123:0/3962582783 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610e1084ae0 con 0x5610e1082590 2024-09-19T04:54:05.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.221+0000 7f7f67400640 1 --2- 172.21.15.123:0/3962582783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 0x5610e10911c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:05.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.221+0000 7f7f66a00640 1 --2- 172.21.15.123:0/3962582783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1081c70 0x5610e1095700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:05.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.221+0000 7f7f5e000640 1 --2- 172.21.15.123:0/3962582783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1082590 0x5610e1090a30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5610e108b320 tx=0x7f7f50005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=231258b606b36de2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:05.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.221+0000 7f7f54c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:54:05.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.221+0000 7f7f54c00640 1 -- 172.21.15.123:0/3962582783 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50032060 con 0x5610e1082590 2024-09-19T04:54:05.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.221+0000 7f7f705f4740 1 -- 172.21.15.123:0/3962582783 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f580315b0 con 0x5610e1082590 2024-09-19T04:54:05.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.221+0000 7f7f54c00640 1 -- 172.21.15.123:0/3962582783 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f7f50004440 con 0x5610e1082590 2024-09-19T04:54:05.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.222+0000 7f7f54c00640 1 -- 172.21.15.123:0/3962582783 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7f500237f0 con 0x5610e1082590 2024-09-19T04:54:05.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.222+0000 7f7f705f4740 10 client.26693 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:05.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.222+0000 7f7f705f4740 1 -- 172.21.15.123:0/3962582783 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5610e11077e0 con 0x5610e1082590 2024-09-19T04:54:05.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.222+0000 7f7f54c00640 1 -- 172.21.15.123:0/3962582783 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f580315b0 con 0x5610e1082590 2024-09-19T04:54:05.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.222+0000 7f7f54c00640 1 -- 172.21.15.123:0/3962582783 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f7f5002c250 con 0x5610e1082590 2024-09-19T04:54:05.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.222+0000 7f7f705f4740 10 client.26693 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:05.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.222+0000 7f7f705f4740 10 client.26693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:05.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.222+0000 7f7f705f4740 20 client.26693 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:05.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.222+0000 7f7f705f4740 1 --2- 172.21.15.123:0/3962582783 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5610e1107f10 0x5610e110a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:05.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.222+0000 7f7f705f4740 4 client.26693 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:05.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.222+0000 7f7f705f4740 1 -- 172.21.15.123:0/3962582783 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610e0fefe60 con 0x5610e1107f10 2024-09-19T04:54:05.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.223+0000 7f7f66a00640 1 --2- 172.21.15.123:0/3962582783 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5610e1107f10 0x5610e110a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:05.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.223+0000 7f7f66a00640 1 --2- 172.21.15.123:0/3962582783 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5610e1107f10 0x5610e110a330 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73b2dbdff838edac server_cookie=5504eb2f97c6a809 in_seq=0 out_seq=0 2024-09-19T04:54:05.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.223+0000 7f7f54c00640 10 client.26693 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:05.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.223+0000 7f7f54c00640 1 -- 172.21.15.123:0/3962582783 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610e0fefe60 con 0x5610e1107f10 2024-09-19T04:54:05.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.223+0000 7f7f54c00640 10 client.26693 handle_command_reply: tid=0 2024-09-19T04:54:05.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.223+0000 7f7f705f4740 1 client.26693 shutdown 2024-09-19T04:54:05.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.224+0000 7f7f705f4740 1 -- 172.21.15.123:0/3962582783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1082590 msgr2=0x5610e1090a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:05.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.224+0000 7f7f705f4740 1 --2- 172.21.15.123:0/3962582783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1082590 0x5610e1090a30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5610e108b320 tx=0x7f7f50005f90 comp rx=0 tx=0).stop 2024-09-19T04:54:05.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.224+0000 7f7f705f4740 1 -- 172.21.15.123:0/3962582783 shutdown_connections 2024-09-19T04:54:05.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.224+0000 7f7f705f4740 1 --2- 172.21.15.123:0/3962582783 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5610e1107f10 0x5610e110a330 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.224+0000 7f7f705f4740 1 --2- 172.21.15.123:0/3962582783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610e108ce50 0x5610e10911c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.224+0000 7f7f705f4740 1 --2- 172.21.15.123:0/3962582783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610e1082590 0x5610e1090a30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.224+0000 7f7f705f4740 1 --2- 172.21.15.123:0/3962582783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610e1081c70 0x5610e1095700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.224+0000 7f7f54c00640 0 client.26693 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:05.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.224+0000 7f7f705f4740 1 -- 172.21.15.123:0/3962582783 >> 172.21.15.123:0/3962582783 conn(0x5610e107e200 msgr2=0x5610e107d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:05.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.224+0000 7f7f705f4740 1 -- 172.21.15.123:0/3962582783 shutdown_connections 2024-09-19T04:54:05.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.224+0000 7f7f705f4740 1 -- 172.21.15.123:0/3962582783 wait complete. 2024-09-19T04:54:05.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.224+0000 7f7f705f4740 20 client.26693 trim_cache size 0 max 16384 2024-09-19T04:54:05.230 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:05.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.225+0000 7f7f6dc00640 1 -- 172.21.15.123:0/780566358 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7f4c0a33b0 msgr2=0x7f7f4c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:05.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.225+0000 7f7f6dc00640 1 --2- 172.21.15.123:0/780566358 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7f4c0a33b0 0x7f7f4c0a5870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009fd0 tx=0x7f7f50015040 comp rx=0 tx=0).stop 2024-09-19T04:54:05.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.225+0000 7f7f6dc00640 1 -- 172.21.15.123:0/780566358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f6810c160 msgr2=0x7f7f6819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:05.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.225+0000 7f7f6dc00640 1 --2- 172.21.15.123:0/780566358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f6810c160 0x7f7f6819ef20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000eaa0 tx=0x7f7f6000ef60 comp rx=0 tx=0).stop 2024-09-19T04:54:05.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.225+0000 7f7f6dc00640 1 -- 172.21.15.123:0/780566358 shutdown_connections 2024-09-19T04:54:05.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.225+0000 7f7f6dc00640 1 --2- 172.21.15.123:0/780566358 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f7f4c0a33b0 0x7f7f4c0a5870 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.225+0000 7f7f6dc00640 1 --2- 172.21.15.123:0/780566358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f6810c160 0x7f7f6819ef20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.225+0000 7f7f6dc00640 1 --2- 172.21.15.123:0/780566358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f68103650 0x7f7f6819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.225+0000 7f7f6dc00640 1 --2- 172.21.15.123:0/780566358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f68102d30 0x7f7f6819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:05.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.225+0000 7f7f6dc00640 1 -- 172.21.15.123:0/780566358 >> 172.21.15.123:0/780566358 conn(0x7f7f680fc7d0 msgr2=0x7f7f6810d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:05.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.225+0000 7f7f6dc00640 1 -- 172.21.15.123:0/780566358 shutdown_connections 2024-09-19T04:54:05.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:05.225+0000 7f7f6dc00640 1 -- 172.21.15.123:0/780566358 wait complete. 2024-09-19T04:54:05.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:05 smithi193 ceph-mon[35592]: pgmap v226: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T04:54:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:05 smithi183 ceph-mon[35333]: pgmap v226: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T04:54:05.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:05 smithi123 ceph-mon[30511]: pgmap v226: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T04:54:07.132 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:54:07.133 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:54:07.507 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:07.645 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:07 smithi123 ceph-mon[30511]: pgmap v227: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T04:54:07.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:07 smithi193 ceph-mon[35592]: pgmap v227: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T04:54:07.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:07 smithi183 ceph-mon[35333]: pgmap v227: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T04:54:08.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.064+0000 7f6d78a00640 1 -- 172.21.15.123:0/3946057028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d741079d0 msgr2=0x7f6d74107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:08.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.064+0000 7f6d78a00640 1 --2- 172.21.15.123:0/3946057028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d741079d0 0x7f6d74107db0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009930 tx=0x7f6d6402a440 comp rx=0 tx=0).stop 2024-09-19T04:54:08.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.065+0000 7f6d78a00640 1 -- 172.21.15.123:0/3946057028 shutdown_connections 2024-09-19T04:54:08.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.065+0000 7f6d78a00640 1 --2- 172.21.15.123:0/3946057028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d74102370 0x7f6d7410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.065+0000 7f6d78a00640 1 --2- 172.21.15.123:0/3946057028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d741019d0 0x7f6d74101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.065+0000 7f6d78a00640 1 --2- 172.21.15.123:0/3946057028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d741079d0 0x7f6d74107db0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.065+0000 7f6d78a00640 1 -- 172.21.15.123:0/3946057028 >> 172.21.15.123:0/3946057028 conn(0x7f6d740fd710 msgr2=0x7f6d740ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:08.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.065+0000 7f6d78a00640 1 -- 172.21.15.123:0/3946057028 shutdown_connections 2024-09-19T04:54:08.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.065+0000 7f6d78a00640 1 -- 172.21.15.123:0/3946057028 wait complete. 2024-09-19T04:54:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.066+0000 7f6d78a00640 1 Processor -- start 2024-09-19T04:54:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.066+0000 7f6d78a00640 1 -- start start 2024-09-19T04:54:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.066+0000 7f6d78a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d741019d0 0x7f6d7419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.066+0000 7f6d78a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d74102370 0x7f6d7419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.066+0000 7f6d78a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d741079d0 0x7f6d7419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.066+0000 7f6d78a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6d74111e20 con 0x7f6d741019d0 2024-09-19T04:54:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.066+0000 7f6d78a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d74111ca0 con 0x7f6d741079d0 2024-09-19T04:54:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.066+0000 7f6d78a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6d74111fa0 con 0x7f6d74102370 2024-09-19T04:54:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.066+0000 7f6d73400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d741019d0 0x7f6d7419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.067+0000 7f6d73400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d741019d0 0x7f6d7419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54138/0 (socket says 172.21.15.123:54138) 2024-09-19T04:54:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.067+0000 7f6d73400640 1 -- 172.21.15.123:0/3667548640 learned_addr learned my addr 172.21.15.123:0/3667548640 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.067+0000 7f6d6be00640 1 --2- 172.21.15.123:0/3667548640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d74102370 0x7f6d7419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.067+0000 7f6d73e00640 1 --2- 172.21.15.123:0/3667548640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d741079d0 0x7f6d7419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.067+0000 7f6d6be00640 1 -- 172.21.15.123:0/3667548640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d741079d0 msgr2=0x7f6d7419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:08.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.067+0000 7f6d6be00640 1 --2- 172.21.15.123:0/3667548640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d741079d0 0x7f6d7419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.067+0000 7f6d6be00640 1 -- 172.21.15.123:0/3667548640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d741019d0 msgr2=0x7f6d7419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:08.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.067+0000 7f6d6be00640 1 --2- 172.21.15.123:0/3667548640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d741019d0 0x7f6d7419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.067+0000 7f6d6be00640 1 -- 172.21.15.123:0/3667548640 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7419f440 con 0x7f6d74102370 2024-09-19T04:54:08.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.067+0000 7f6d73400640 1 --2- 172.21.15.123:0/3667548640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d741019d0 0x7f6d7419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:08.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.067+0000 7f6d6be00640 1 --2- 172.21.15.123:0/3667548640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d74102370 0x7f6d7419a9d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c005fd0 tx=0x7f6d5c00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:08.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.068+0000 7f6d70c00640 1 -- 172.21.15.123:0/3667548640 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c009c30 con 0x7f6d74102370 2024-09-19T04:54:08.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.068+0000 7f6d78a00640 1 -- 172.21.15.123:0/3667548640 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7419f730 con 0x7f6d74102370 2024-09-19T04:54:08.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.068+0000 7f6d70c00640 1 -- 172.21.15.123:0/3667548640 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d5c009dd0 con 0x7f6d74102370 2024-09-19T04:54:08.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.068+0000 7f6d70c00640 1 -- 172.21.15.123:0/3667548640 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c010460 con 0x7f6d74102370 2024-09-19T04:54:08.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.068+0000 7f6d78a00640 1 -- 172.21.15.123:0/3667548640 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d741a7010 con 0x7f6d74102370 2024-09-19T04:54:08.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.071+0000 7f6d7cc04740 1 Processor -- start 2024-09-19T04:54:08.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.071+0000 7f6d7cc04740 1 -- start start 2024-09-19T04:54:08.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.071+0000 7f6d70c00640 1 -- 172.21.15.123:0/3667548640 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6d5c003f50 con 0x7f6d74102370 2024-09-19T04:54:08.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.071+0000 7f6d7cc04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d37acb0 0x559c1d37b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.071+0000 7f6d7cc04740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d37b5d0 0x559c1d385950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.071+0000 7f6d7cc04740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d385e90 0x559c1d388280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.071+0000 7f6d7cc04740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559c1d2e1290 con 0x559c1d385e90 2024-09-19T04:54:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.071+0000 7f6d7cc04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559c1d288b40 con 0x559c1d37acb0 2024-09-19T04:54:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.071+0000 7f6d7cc04740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559c1d2dd250 con 0x559c1d37b5d0 2024-09-19T04:54:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.071+0000 7f6d73e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d37acb0 0x559c1d37b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.071+0000 7f6d6be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d385e90 0x559c1d388280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.072+0000 7f6d73e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d37acb0 0x559c1d37b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:49608/0 (socket says 172.21.15.123:49608) 2024-09-19T04:54:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.072+0000 7f6d73e00640 1 -- 172.21.15.123:0/1643940034 learned_addr learned my addr 172.21.15.123:0/1643940034 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.072+0000 7f6d73400640 1 --2- 172.21.15.123:0/1643940034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d37b5d0 0x559c1d385950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.072+0000 7f6d73e00640 1 -- 172.21.15.123:0/1643940034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d37b5d0 msgr2=0x559c1d385950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.072+0000 7f6d73e00640 1 --2- 172.21.15.123:0/1643940034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d37b5d0 0x559c1d385950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.072+0000 7f6d73e00640 1 -- 172.21.15.123:0/1643940034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d385e90 msgr2=0x559c1d388280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.072+0000 7f6d73e00640 1 --2- 172.21.15.123:0/1643940034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d385e90 0x559c1d388280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.072+0000 7f6d73e00640 1 -- 172.21.15.123:0/1643940034 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c1d0526a0 con 0x559c1d37acb0 2024-09-19T04:54:08.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.072+0000 7f6d73e00640 1 --2- 172.21.15.123:0/1643940034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d37acb0 0x559c1d37b090 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009f70 tx=0x7f6d6c010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4429689b6480da51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:08.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.072+0000 7f6d63e00640 1 -- 172.21.15.123:0/1643940034 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c016070 con 0x559c1d37acb0 2024-09-19T04:54:08.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.073+0000 7f6d63e00640 1 -- 172.21.15.123:0/1643940034 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6c004d20 con 0x559c1d37acb0 2024-09-19T04:54:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.073+0000 7f6d63e00640 1 -- 172.21.15.123:0/1643940034 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c012950 con 0x559c1d37acb0 2024-09-19T04:54:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.073+0000 7f6d70c00640 1 --2- 172.21.15.123:0/3667548640 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6d580a33b0 0x7f6d580a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.073+0000 7f6d7cc04740 1 -- 172.21.15.123:0/1643940034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d37acb0 msgr2=0x559c1d37b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.073+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/1643940034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d37acb0 0x559c1d37b090 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009f70 tx=0x7f6d6c010800 comp rx=0 tx=0).stop 2024-09-19T04:54:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.073+0000 7f6d70c00640 1 -- 172.21.15.123:0/3667548640 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f6d5c0cfbb0 con 0x7f6d74102370 2024-09-19T04:54:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.073+0000 7f6d73400640 1 --2- 172.21.15.123:0/3667548640 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6d580a33b0 0x7f6d580a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.073+0000 7f6d7cc04740 1 -- 172.21.15.123:0/1643940034 shutdown_connections 2024-09-19T04:54:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.073+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/1643940034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d385e90 0x559c1d388280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.073+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/1643940034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d37b5d0 0x559c1d385950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.073+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/1643940034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d37acb0 0x559c1d37b090 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.073+0000 7f6d7cc04740 1 -- 172.21.15.123:0/1643940034 >> 172.21.15.123:0/1643940034 conn(0x559c1d377240 msgr2=0x559c1d377620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.074+0000 7f6d7cc04740 1 -- 172.21.15.123:0/1643940034 shutdown_connections 2024-09-19T04:54:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.074+0000 7f6d7cc04740 1 -- 172.21.15.123:0/1643940034 wait complete. 2024-09-19T04:54:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.074+0000 7f6d7cc04740 1 -- start start 2024-09-19T04:54:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.074+0000 7f6d73400640 1 --2- 172.21.15.123:0/3667548640 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6d580a33b0 0x7f6d580a5870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x559c1d3780a0 tx=0x7f6d64031510 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.074+0000 7f6d7cc04740 1 Processor -- start 2024-09-19T04:54:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.075+0000 7f6d7cc04740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37acb0 0x559c1d3857a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.075+0000 7f6d7cc04740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d37b5d0 0x559c1d382a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.075+0000 7f6d7cc04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d385e90 0x559c1d382f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.075+0000 7f6d7cc04740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559c1d388b00 con 0x559c1d37acb0 2024-09-19T04:54:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.075+0000 7f6d7cc04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559c1d29bfa0 con 0x559c1d385e90 2024-09-19T04:54:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.075+0000 7f6d7cc04740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559c1d388c80 con 0x559c1d37b5d0 2024-09-19T04:54:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.075+0000 7f6d7cc04740 10 client.0 _ll_register_callbacks cb 0x559c1cfefba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.075+0000 7f6d73400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37acb0 0x559c1d3857a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.075+0000 7f6d73400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37acb0 0x559c1d3857a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54156/0 (socket says 172.21.15.123:54156) 2024-09-19T04:54:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.075+0000 7f6d73e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d385e90 0x559c1d382f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.075+0000 7f6d73400640 1 -- 172.21.15.123:0/3325750450 learned_addr learned my addr 172.21.15.123:0/3325750450 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.075+0000 7f6d6be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d37b5d0 0x559c1d382a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.075+0000 7f6d73400640 1 -- 172.21.15.123:0/3325750450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d37b5d0 msgr2=0x559c1d382a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:08.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.075+0000 7f6d73400640 1 --2- 172.21.15.123:0/3325750450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d37b5d0 0x559c1d382a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.075+0000 7f6d73400640 1 -- 172.21.15.123:0/3325750450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d385e90 msgr2=0x559c1d382f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:08.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.075+0000 7f6d73400640 1 --2- 172.21.15.123:0/3325750450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d385e90 0x559c1d382f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.075+0000 7f6d73400640 1 -- 172.21.15.123:0/3325750450 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c1d383790 con 0x559c1d37acb0 2024-09-19T04:54:08.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.076+0000 7f6d73e00640 1 --2- 172.21.15.123:0/3325750450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d385e90 0x559c1d382f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.076+0000 7f6d6be00640 1 --2- 172.21.15.123:0/3325750450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d37b5d0 0x559c1d382a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.076+0000 7f6d73400640 1 --2- 172.21.15.123:0/3325750450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37acb0 0x559c1d3857a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6d64005610 tx=0x7f6d6402a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6fbf3ee52b392b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.076+0000 7f6d60c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.076+0000 7f6d60c00640 1 -- 172.21.15.123:0/3325750450 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64035040 con 0x559c1d37acb0 2024-09-19T04:54:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.076+0000 7f6d7cc04740 1 -- 172.21.15.123:0/3325750450 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6c004d20 con 0x559c1d37acb0 2024-09-19T04:54:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.076+0000 7f6d60c00640 1 -- 172.21.15.123:0/3325750450 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f6d640317b0 con 0x559c1d37acb0 2024-09-19T04:54:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.076+0000 7f6d60c00640 1 -- 172.21.15.123:0/3325750450 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6d640319b0 con 0x559c1d37acb0 2024-09-19T04:54:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.076+0000 7f6d7cc04740 10 client.16605 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.076+0000 7f6d7cc04740 1 -- 172.21.15.123:0/3325750450 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559c1d394a30 con 0x559c1d37acb0 2024-09-19T04:54:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.077+0000 7f6d60c00640 1 -- 172.21.15.123:0/3325750450 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6c004d20 con 0x559c1d37acb0 2024-09-19T04:54:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.077+0000 7f6d60c00640 1 -- 172.21.15.123:0/3325750450 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f6d64044030 con 0x559c1d37acb0 2024-09-19T04:54:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.077+0000 7f6d7cc04740 10 client.16605 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.077+0000 7f6d7cc04740 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.077+0000 7f6d7cc04740 20 client.16605 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.077+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/3325750450 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x559c1d395ef0 0x559c1d398300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.077+0000 7f6d7cc04740 4 client.16605 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.077+0000 7f6d7cc04740 1 -- 172.21.15.123:0/3325750450 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c1d2ee800 con 0x559c1d395ef0 2024-09-19T04:54:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.078+0000 7f6d6be00640 1 --2- 172.21.15.123:0/3325750450 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x559c1d395ef0 0x559c1d398300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.078+0000 7f6d6be00640 1 --2- 172.21.15.123:0/3325750450 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x559c1d395ef0 0x559c1d398300 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b16dc3ad74df1bd7 server_cookie=8fd0cf2d6c628ab7 in_seq=0 out_seq=0 2024-09-19T04:54:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.078+0000 7f6d60c00640 10 client.16605 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.079+0000 7f6d60c00640 1 -- 172.21.15.123:0/3325750450 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c1d2ee800 con 0x559c1d395ef0 2024-09-19T04:54:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.079+0000 7f6d60c00640 10 client.16605 handle_command_reply: tid=0 2024-09-19T04:54:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.079+0000 7f6d7cc04740 1 client.16605 shutdown 2024-09-19T04:54:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.079+0000 7f6d7cc04740 1 -- 172.21.15.123:0/3325750450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37acb0 msgr2=0x559c1d3857a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.079+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/3325750450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37acb0 0x559c1d3857a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6d64005610 tx=0x7f6d6402a440 comp rx=0 tx=0).stop 2024-09-19T04:54:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.079+0000 7f6d7cc04740 1 -- 172.21.15.123:0/3325750450 shutdown_connections 2024-09-19T04:54:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.079+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/3325750450 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x559c1d395ef0 0x559c1d398300 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.079+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/3325750450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d385e90 0x559c1d382f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.079+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/3325750450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d37b5d0 0x559c1d382a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.079+0000 7f6d60c00640 0 client.16605 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.079+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/3325750450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37acb0 0x559c1d3857a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.079+0000 7f6d7cc04740 1 -- 172.21.15.123:0/3325750450 >> 172.21.15.123:0/3325750450 conn(0x559c1d377240 msgr2=0x559c1d37cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.079+0000 7f6d7cc04740 1 -- 172.21.15.123:0/3325750450 shutdown_connections 2024-09-19T04:54:08.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.079+0000 7f6d7cc04740 1 -- 172.21.15.123:0/3325750450 wait complete. 2024-09-19T04:54:08.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.079+0000 7f6d7cc04740 20 client.16605 trim_cache size 0 max 16384 2024-09-19T04:54:08.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.097+0000 7f6d7cc04740 1 Processor -- start 2024-09-19T04:54:08.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.097+0000 7f6d7cc04740 1 -- start start 2024-09-19T04:54:08.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.097+0000 7f6d7cc04740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37acb0 0x559c1d3849c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.097+0000 7f6d7cc04740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d37b5d0 0x559c1d39dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.097+0000 7f6d7cc04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d385e90 0x559c1d39e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.097+0000 7f6d7cc04740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741af820 con 0x559c1d37acb0 2024-09-19T04:54:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.097+0000 7f6d7cc04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741b13f0 con 0x559c1d385e90 2024-09-19T04:54:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.097+0000 7f6d7cc04740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741aff20 con 0x559c1d37b5d0 2024-09-19T04:54:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.098+0000 7f6d73400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37acb0 0x559c1d3849c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.098+0000 7f6d73400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37acb0 0x559c1d3849c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54172/0 (socket says 172.21.15.123:54172) 2024-09-19T04:54:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.098+0000 7f6d73400640 1 -- 172.21.15.123:0/3820147517 learned_addr learned my addr 172.21.15.123:0/3820147517 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.098+0000 7f6d6be00640 1 --2- 172.21.15.123:0/3820147517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d37b5d0 0x559c1d39dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.098+0000 7f6d73e00640 1 --2- 172.21.15.123:0/3820147517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d385e90 0x559c1d39e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.098+0000 7f6d73400640 1 -- 172.21.15.123:0/3820147517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d37b5d0 msgr2=0x559c1d39dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.098+0000 7f6d73400640 1 --2- 172.21.15.123:0/3820147517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d37b5d0 0x559c1d39dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.098+0000 7f6d73400640 1 -- 172.21.15.123:0/3820147517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d385e90 msgr2=0x559c1d39e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.098+0000 7f6d73400640 1 --2- 172.21.15.123:0/3820147517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d385e90 0x559c1d39e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.098+0000 7f6d73400640 1 -- 172.21.15.123:0/3820147517 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c1d381a00 con 0x559c1d37acb0 2024-09-19T04:54:08.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.098+0000 7f6d73400640 1 --2- 172.21.15.123:0/3820147517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37acb0 0x559c1d3849c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6d64042040 tx=0x7f6d640515f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0ce61f8980bbfb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:08.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.099+0000 7f6d63400640 1 -- 172.21.15.123:0/3820147517 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6405e020 con 0x559c1d37acb0 2024-09-19T04:54:08.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.099+0000 7f6d63400640 1 -- 172.21.15.123:0/3820147517 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6404c070 con 0x559c1d37acb0 2024-09-19T04:54:08.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.099+0000 7f6d63400640 1 -- 172.21.15.123:0/3820147517 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64031db0 con 0x559c1d37acb0 2024-09-19T04:54:08.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.099+0000 7f6d7cc04740 1 -- 172.21.15.123:0/3820147517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37acb0 msgr2=0x559c1d3849c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:08.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.099+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/3820147517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37acb0 0x559c1d3849c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6d64042040 tx=0x7f6d640515f0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.099+0000 7f6d7cc04740 1 -- 172.21.15.123:0/3820147517 shutdown_connections 2024-09-19T04:54:08.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.099+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/3820147517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d385e90 0x559c1d39e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.099+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/3820147517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d37b5d0 0x559c1d39dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.099+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/3820147517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37acb0 0x559c1d3849c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.099+0000 7f6d7cc04740 1 -- 172.21.15.123:0/3820147517 >> 172.21.15.123:0/3820147517 conn(0x559c1d377240 msgr2=0x559c1d3842b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:08.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.100+0000 7f6d7cc04740 1 -- 172.21.15.123:0/3820147517 shutdown_connections 2024-09-19T04:54:08.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.100+0000 7f6d7cc04740 1 -- 172.21.15.123:0/3820147517 wait complete. 2024-09-19T04:54:08.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.100+0000 7f6d7cc04740 1 -- start start 2024-09-19T04:54:08.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.100+0000 7f6d7cc04740 1 Processor -- start 2024-09-19T04:54:08.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d7cc04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d37acb0 0x559c1d38bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d7cc04740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37b5d0 0x559c1d38c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d7cc04740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d385e90 0x559c1d38c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d7cc04740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741b0d20 con 0x559c1d37b5d0 2024-09-19T04:54:08.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d7cc04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741b0620 con 0x559c1d37acb0 2024-09-19T04:54:08.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d7cc04740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741ad820 con 0x559c1d385e90 2024-09-19T04:54:08.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d7cc04740 10 client.0 _ll_register_callbacks cb 0x7f6d64009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:08.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d6be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37b5d0 0x559c1d38c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d6be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37b5d0 0x559c1d38c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54188/0 (socket says 172.21.15.123:54188) 2024-09-19T04:54:08.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d6be00640 1 -- 172.21.15.123:0/318955460 learned_addr learned my addr 172.21.15.123:0/318955460 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:08.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d73e00640 1 --2- 172.21.15.123:0/318955460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d385e90 0x559c1d38c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d73400640 1 --2- 172.21.15.123:0/318955460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d37acb0 0x559c1d38bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d6be00640 1 -- 172.21.15.123:0/318955460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d385e90 msgr2=0x559c1d38c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:08.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d6be00640 1 --2- 172.21.15.123:0/318955460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d385e90 0x559c1d38c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d6be00640 1 -- 172.21.15.123:0/318955460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d37acb0 msgr2=0x559c1d38bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:08.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d6be00640 1 --2- 172.21.15.123:0/318955460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d37acb0 0x559c1d38bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d6be00640 1 -- 172.21.15.123:0/318955460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c1d3ffce0 con 0x559c1d37b5d0 2024-09-19T04:54:08.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.101+0000 7f6d73400640 1 --2- 172.21.15.123:0/318955460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d37acb0 0x559c1d38bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:54:08.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.102+0000 7f6d6be00640 1 --2- 172.21.15.123:0/318955460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37b5d0 0x559c1d38c2a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0cd3b0 tx=0x7f6d5c0d3210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ab2551ef937c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:08.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.102+0000 7f6d60c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:08.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.102+0000 7f6d60c00640 1 -- 172.21.15.123:0/318955460 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c020040 con 0x559c1d37b5d0 2024-09-19T04:54:08.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.102+0000 7f6d60c00640 1 -- 172.21.15.123:0/318955460 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f6d5c0cd580 con 0x559c1d37b5d0 2024-09-19T04:54:08.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.102+0000 7f6d7cc04740 1 -- 172.21.15.123:0/318955460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6404c070 con 0x559c1d37b5d0 2024-09-19T04:54:08.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.102+0000 7f6d60c00640 1 -- 172.21.15.123:0/318955460 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6d5c09bf40 con 0x559c1d37b5d0 2024-09-19T04:54:08.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.102+0000 7f6d7cc04740 10 client.16611 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:08.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.102+0000 7f6d7cc04740 1 -- 172.21.15.123:0/318955460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559c1d401000 con 0x559c1d37b5d0 2024-09-19T04:54:08.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.102+0000 7f6d60c00640 1 -- 172.21.15.123:0/318955460 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6404c070 con 0x559c1d37b5d0 2024-09-19T04:54:08.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.102+0000 7f6d60c00640 1 -- 172.21.15.123:0/318955460 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f6d5c09c230 con 0x559c1d37b5d0 2024-09-19T04:54:08.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.102+0000 7f6d7cc04740 10 client.16611 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:08.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.103+0000 7f6d7cc04740 10 client.16611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:08.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.103+0000 7f6d7cc04740 20 client.16611 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:08.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.103+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/318955460 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x559c1d401730 0x559c1d403b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:08.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.103+0000 7f6d7cc04740 4 client.16611 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:54:08.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.103+0000 7f6d7cc04740 1 -- 172.21.15.123:0/318955460 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559c1d2e8d10 con 0x559c1d401730 2024-09-19T04:54:08.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.103+0000 7f6d73400640 1 --2- 172.21.15.123:0/318955460 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x559c1d401730 0x559c1d403b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:08.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.104+0000 7f6d73400640 1 --2- 172.21.15.123:0/318955460 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x559c1d401730 0x559c1d403b50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d85501e1888c5e13 server_cookie=1be686aa878c8b0e in_seq=0 out_seq=0 2024-09-19T04:54:08.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.104+0000 7f6d60c00640 10 client.16611 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:08.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.105+0000 7f6d60c00640 1 -- 172.21.15.123:0/318955460 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559c1d2e8d10 con 0x559c1d401730 2024-09-19T04:54:08.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.105+0000 7f6d60c00640 10 client.16611 handle_command_reply: tid=0 2024-09-19T04:54:08.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.105+0000 7f6d7cc04740 1 client.16611 shutdown 2024-09-19T04:54:08.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.105+0000 7f6d7cc04740 1 -- 172.21.15.123:0/318955460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37b5d0 msgr2=0x559c1d38c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:08.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.105+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/318955460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37b5d0 0x559c1d38c2a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0cd3b0 tx=0x7f6d5c0d3210 comp rx=0 tx=0).stop 2024-09-19T04:54:08.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.105+0000 7f6d7cc04740 1 -- 172.21.15.123:0/318955460 shutdown_connections 2024-09-19T04:54:08.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.105+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/318955460 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x559c1d401730 0x559c1d403b50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.105+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/318955460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c1d385e90 0x559c1d38c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.105+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/318955460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c1d37b5d0 0x559c1d38c2a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.105+0000 7f6d7cc04740 1 --2- 172.21.15.123:0/318955460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c1d37acb0 0x559c1d38bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.105+0000 7f6d60c00640 0 client.16611 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:08.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.105+0000 7f6d7cc04740 1 -- 172.21.15.123:0/318955460 >> 172.21.15.123:0/318955460 conn(0x559c1d377240 msgr2=0x559c1d376270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:08.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.105+0000 7f6d7cc04740 1 -- 172.21.15.123:0/318955460 shutdown_connections 2024-09-19T04:54:08.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.105+0000 7f6d7cc04740 1 -- 172.21.15.123:0/318955460 wait complete. 2024-09-19T04:54:08.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.105+0000 7f6d7cc04740 20 client.16611 trim_cache size 0 max 16384 2024-09-19T04:54:08.110 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:54:08.110 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-09-19T04:54:08.110 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "342efb1b-1929-40d6-8f33-53609080f43f", 2024-09-19T04:54:08.110 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-09-19T04:54:08.110 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:54:08.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.106+0000 7f6d78a00640 1 -- 172.21.15.123:0/3667548640 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6d580a33b0 msgr2=0x7f6d580a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:08.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.106+0000 7f6d78a00640 1 --2- 172.21.15.123:0/3667548640 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6d580a33b0 0x7f6d580a5870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x559c1d3780a0 tx=0x7f6d64031510 comp rx=0 tx=0).stop 2024-09-19T04:54:08.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.106+0000 7f6d78a00640 1 -- 172.21.15.123:0/3667548640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d74102370 msgr2=0x7f6d7419a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:08.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.106+0000 7f6d78a00640 1 --2- 172.21.15.123:0/3667548640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d74102370 0x7f6d7419a9d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c005fd0 tx=0x7f6d5c00dde0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.106+0000 7f6d78a00640 1 -- 172.21.15.123:0/3667548640 shutdown_connections 2024-09-19T04:54:08.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.106+0000 7f6d78a00640 1 --2- 172.21.15.123:0/3667548640 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6d580a33b0 0x7f6d580a5870 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.106+0000 7f6d78a00640 1 --2- 172.21.15.123:0/3667548640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d741079d0 0x7f6d7419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.106+0000 7f6d78a00640 1 --2- 172.21.15.123:0/3667548640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d74102370 0x7f6d7419a9d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.106+0000 7f6d78a00640 1 --2- 172.21.15.123:0/3667548640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d741019d0 0x7f6d7419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:08.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.106+0000 7f6d78a00640 1 -- 172.21.15.123:0/3667548640 >> 172.21.15.123:0/3667548640 conn(0x7f6d740fd710 msgr2=0x7f6d740fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:08.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.106+0000 7f6d78a00640 1 -- 172.21.15.123:0/3667548640 shutdown_connections 2024-09-19T04:54:08.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:08.106+0000 7f6d78a00640 1 -- 172.21.15.123:0/3667548640 wait complete. 2024-09-19T04:54:08.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:08 smithi193 ceph-mon[35592]: scrub queued for path: / 2024-09-19T04:54:08.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:08 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:08.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:08 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:54:08.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:08 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:08.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:08 smithi183 ceph-mon[35333]: scrub queued for path: / 2024-09-19T04:54:08.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:08 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:08.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:08 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:54:08.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:08 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:08.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:08 smithi123 ceph-mon[30511]: scrub queued for path: / 2024-09-19T04:54:08.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:08 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:08.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:08 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:54:08.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:08 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:09.020 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:54:09.387 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:09.694 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:09 smithi123 ceph-mon[30511]: pgmap v228: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T04:54:09.694 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:09 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:54:09.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:09 smithi193 ceph-mon[35592]: pgmap v228: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T04:54:09.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:09 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:54:09.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:09 smithi183 ceph-mon[35333]: pgmap v228: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T04:54:09.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:09 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:54:09.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.958+0000 7f0f54e00640 1 -- 172.21.15.123:0/1723579656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f5010c180 msgr2=0x7f0f5010e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:09.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.958+0000 7f0f54e00640 1 --2- 172.21.15.123:0/1723579656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f5010c180 0x7f0f5010e620 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009ca0 tx=0x7f0f4802a950 comp rx=0 tx=0).stop 2024-09-19T04:54:09.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.958+0000 7f0f54e00640 1 -- 172.21.15.123:0/1723579656 shutdown_connections 2024-09-19T04:54:09.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.958+0000 7f0f54e00640 1 --2- 172.21.15.123:0/1723579656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f5010c180 0x7f0f5010e620 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.958+0000 7f0f54e00640 1 --2- 172.21.15.123:0/1723579656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f50103670 0x7f0f5010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.958+0000 7f0f54e00640 1 --2- 172.21.15.123:0/1723579656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f50102d50 0x7f0f50103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.958+0000 7f0f54e00640 1 -- 172.21.15.123:0/1723579656 >> 172.21.15.123:0/1723579656 conn(0x7f0f500fc7d0 msgr2=0x7f0f500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:09.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.958+0000 7f0f54e00640 1 -- 172.21.15.123:0/1723579656 shutdown_connections 2024-09-19T04:54:09.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.958+0000 7f0f54e00640 1 -- 172.21.15.123:0/1723579656 wait complete. 2024-09-19T04:54:09.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.959+0000 7f0f54e00640 1 Processor -- start 2024-09-19T04:54:09.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.959+0000 7f0f54e00640 1 -- start start 2024-09-19T04:54:09.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.959+0000 7f0f54e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f50102d50 0x7f0f5019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:09.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.960+0000 7f0f54e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f50103670 0x7f0f5019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:09.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.960+0000 7f0f54e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f5010c180 0x7f0f5019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:09.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.960+0000 7f0f54e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0f50111f80 con 0x7f0f50103670 2024-09-19T04:54:09.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.960+0000 7f0f54e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f50111e00 con 0x7f0f50102d50 2024-09-19T04:54:09.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.960+0000 7f0f54e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0f50112100 con 0x7f0f5010c180 2024-09-19T04:54:09.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.960+0000 7f0f4d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f50103670 0x7f0f5019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:09.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.960+0000 7f0f4e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f5010c180 0x7f0f5019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:09.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.960+0000 7f0f4d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f50103670 0x7f0f5019ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54202/0 (socket says 172.21.15.123:54202) 2024-09-19T04:54:09.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.960+0000 7f0f4d400640 1 -- 172.21.15.123:0/3133746827 learned_addr learned my addr 172.21.15.123:0/3133746827 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:09.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.960+0000 7f0f4de00640 1 --2- 172.21.15.123:0/3133746827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f50102d50 0x7f0f5019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:09.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.960+0000 7f0f4e800640 1 -- 172.21.15.123:0/3133746827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f50102d50 msgr2=0x7f0f5019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:09.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.960+0000 7f0f4e800640 1 --2- 172.21.15.123:0/3133746827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f50102d50 0x7f0f5019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.960+0000 7f0f4e800640 1 -- 172.21.15.123:0/3133746827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f50103670 msgr2=0x7f0f5019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:09.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.960+0000 7f0f4e800640 1 --2- 172.21.15.123:0/3133746827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f50103670 0x7f0f5019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.960+0000 7f0f4e800640 1 -- 172.21.15.123:0/3133746827 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5019f5a0 con 0x7f0f5010c180 2024-09-19T04:54:09.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.960+0000 7f0f4d400640 1 --2- 172.21.15.123:0/3133746827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f50103670 0x7f0f5019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:09.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.961+0000 7f0f4e800640 1 --2- 172.21.15.123:0/3133746827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f5010c180 0x7f0f5019eec0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0f48005cc0 tx=0x7f0f4802f420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:09.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.961+0000 7f0f46a00640 1 -- 172.21.15.123:0/3133746827 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48034040 con 0x7f0f5010c180 2024-09-19T04:54:09.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.961+0000 7f0f54e00640 1 -- 172.21.15.123:0/3133746827 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5019f830 con 0x7f0f5010c180 2024-09-19T04:54:09.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.961+0000 7f0f46a00640 1 -- 172.21.15.123:0/3133746827 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f48030920 con 0x7f0f5010c180 2024-09-19T04:54:09.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.961+0000 7f0f46a00640 1 -- 172.21.15.123:0/3133746827 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48004610 con 0x7f0f5010c180 2024-09-19T04:54:09.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.961+0000 7f0f54e00640 1 -- 172.21.15.123:0/3133746827 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f501a7170 con 0x7f0f5010c180 2024-09-19T04:54:09.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.964+0000 7f0f57a13740 1 Processor -- start 2024-09-19T04:54:09.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.965+0000 7f0f46a00640 1 -- 172.21.15.123:0/3133746827 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f0f48037020 con 0x7f0f5010c180 2024-09-19T04:54:09.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.965+0000 7f0f57a13740 1 -- start start 2024-09-19T04:54:09.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.965+0000 7f0f57a13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a2793b90 0x55d7a2793f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:09.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.965+0000 7f0f57a13740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 0x55d7a279e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:09.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.965+0000 7f0f57a13740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a279ed70 0x55d7a27a1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:09.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.965+0000 7f0f57a13740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d7a26fa3e0 con 0x55d7a27944b0 2024-09-19T04:54:09.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.965+0000 7f0f57a13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d7a26a1d80 con 0x55d7a2793b90 2024-09-19T04:54:09.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.965+0000 7f0f57a13740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d7a26f63a0 con 0x55d7a279ed70 2024-09-19T04:54:09.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.965+0000 7f0f4de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 0x55d7a279e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:09.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.965+0000 7f0f4d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a2793b90 0x55d7a2793f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:09.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.965+0000 7f0f4d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a2793b90 0x55d7a2793f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:49660/0 (socket says 172.21.15.123:49660) 2024-09-19T04:54:09.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.965+0000 7f0f4d400640 1 -- 172.21.15.123:0/247464017 learned_addr learned my addr 172.21.15.123:0/247464017 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:09.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.966+0000 7f0f4d400640 1 --2- 172.21.15.123:0/247464017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a279ed70 0x55d7a27a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:09.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.966+0000 7f0f4de00640 1 -- 172.21.15.123:0/247464017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a279ed70 msgr2=0x55d7a27a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:09.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.966+0000 7f0f4de00640 1 --2- 172.21.15.123:0/247464017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a279ed70 0x55d7a27a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.966+0000 7f0f4de00640 1 -- 172.21.15.123:0/247464017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a2793b90 msgr2=0x55d7a2793f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:09.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.966+0000 7f0f4de00640 1 --2- 172.21.15.123:0/247464017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a2793b90 0x55d7a2793f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.966+0000 7f0f4de00640 1 -- 172.21.15.123:0/247464017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a26c0ae0 con 0x55d7a27944b0 2024-09-19T04:54:09.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.966+0000 7f0f4de00640 1 --2- 172.21.15.123:0/247464017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 0x55d7a279e830 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0f38005fd0 tx=0x7f0f380107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc54ad1ce5134377 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:09.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.966+0000 7f0f46a00640 1 --2- 172.21.15.123:0/3133746827 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0f340a3430 0x7f0f340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:09.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.967+0000 7f0f46a00640 1 -- 172.21.15.123:0/3133746827 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8557+0+0 (secure 0 0 0) 0x7f0f480f0a40 con 0x7f0f5010c180 2024-09-19T04:54:09.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.967+0000 7f0f3f400640 1 -- 172.21.15.123:0/247464017 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3801c480 con 0x55d7a27944b0 2024-09-19T04:54:09.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.967+0000 7f0f3f400640 1 -- 172.21.15.123:0/247464017 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f3801cb00 con 0x55d7a27944b0 2024-09-19T04:54:09.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.967+0000 7f0f3f400640 1 -- 172.21.15.123:0/247464017 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38011bb0 con 0x55d7a27944b0 2024-09-19T04:54:09.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.967+0000 7f0f4d400640 1 --2- 172.21.15.123:0/3133746827 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0f340a3430 0x7f0f340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:09.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.967+0000 7f0f57a13740 1 -- 172.21.15.123:0/247464017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 msgr2=0x55d7a279e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:09.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.967+0000 7f0f57a13740 1 --2- 172.21.15.123:0/247464017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 0x55d7a279e830 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0f38005fd0 tx=0x7f0f380107a0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.967+0000 7f0f4d400640 1 --2- 172.21.15.123:0/3133746827 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0f340a3430 0x7f0f340a58f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009830 tx=0x7f0f400097c0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:09.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.967+0000 7f0f57a13740 1 -- 172.21.15.123:0/247464017 shutdown_connections 2024-09-19T04:54:09.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.967+0000 7f0f57a13740 1 --2- 172.21.15.123:0/247464017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a279ed70 0x55d7a27a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.967+0000 7f0f57a13740 1 --2- 172.21.15.123:0/247464017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 0x55d7a279e830 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.968+0000 7f0f57a13740 1 --2- 172.21.15.123:0/247464017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a2793b90 0x55d7a2793f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.968+0000 7f0f57a13740 1 -- 172.21.15.123:0/247464017 >> 172.21.15.123:0/247464017 conn(0x55d7a2790120 msgr2=0x55d7a2790500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:09.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.968+0000 7f0f57a13740 1 -- 172.21.15.123:0/247464017 shutdown_connections 2024-09-19T04:54:09.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.968+0000 7f0f57a13740 1 -- 172.21.15.123:0/247464017 wait complete. 2024-09-19T04:54:09.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.968+0000 7f0f57a13740 1 -- start start 2024-09-19T04:54:09.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.968+0000 7f0f57a13740 1 Processor -- start 2024-09-19T04:54:09.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.969+0000 7f0f57a13740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a2793b90 0x55d7a279bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:09.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.969+0000 7f0f57a13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a27944b0 0x55d7a279c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:09.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.969+0000 7f0f57a13740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a279ed70 0x55d7a279c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:09.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.969+0000 7f0f57a13740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d7a27a19e0 con 0x55d7a279ed70 2024-09-19T04:54:09.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.969+0000 7f0f57a13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d7a26b50f0 con 0x55d7a27944b0 2024-09-19T04:54:09.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.969+0000 7f0f57a13740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d7a27a1b60 con 0x55d7a2793b90 2024-09-19T04:54:09.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.969+0000 7f0f57a13740 10 client.0 _ll_register_callbacks cb 0x55d7a26a69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:09.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.969+0000 7f0f4d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a27944b0 0x55d7a279c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:09.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.969+0000 7f0f4d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a27944b0 0x55d7a279c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:49674/0 (socket says 172.21.15.123:49674) 2024-09-19T04:54:09.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.969+0000 7f0f4d400640 1 -- 172.21.15.123:0/1591089885 learned_addr learned my addr 172.21.15.123:0/1591089885 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:09.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.969+0000 7f0f4de00640 1 --2- 172.21.15.123:0/1591089885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a2793b90 0x55d7a279bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:09.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.969+0000 7f0f4e800640 1 --2- 172.21.15.123:0/1591089885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a279ed70 0x55d7a279c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:09.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.969+0000 7f0f4d400640 1 -- 172.21.15.123:0/1591089885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a2793b90 msgr2=0x55d7a279bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:09.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.969+0000 7f0f4d400640 1 --2- 172.21.15.123:0/1591089885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a2793b90 0x55d7a279bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.969+0000 7f0f4d400640 1 -- 172.21.15.123:0/1591089885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a279ed70 msgr2=0x55d7a279c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:09.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.969+0000 7f0f4d400640 1 --2- 172.21.15.123:0/1591089885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a279ed70 0x55d7a279c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.970+0000 7f0f4d400640 1 -- 172.21.15.123:0/1591089885 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a246b3c0 con 0x55d7a27944b0 2024-09-19T04:54:09.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.970+0000 7f0f4d400640 1 --2- 172.21.15.123:0/1591089885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a27944b0 0x55d7a279c0e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0f40013410 tx=0x7f0f40005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0bb138731146c15 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:09.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.970+0000 7f0f33e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:54:09.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.970+0000 7f0f33e00640 1 -- 172.21.15.123:0/1591089885 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4002f060 con 0x55d7a27944b0 2024-09-19T04:54:09.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.970+0000 7f0f33e00640 1 -- 172.21.15.123:0/1591089885 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8557+0+0 (secure 0 0 0) 0x7f0f4001c030 con 0x55d7a27944b0 2024-09-19T04:54:09.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.970+0000 7f0f57a13740 1 -- 172.21.15.123:0/1591089885 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f3801cb00 con 0x55d7a27944b0 2024-09-19T04:54:09.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.970+0000 7f0f33e00640 1 -- 172.21.15.123:0/1591089885 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0f40020650 con 0x55d7a27944b0 2024-09-19T04:54:09.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.970+0000 7f0f57a13740 10 client.26723 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:09.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.970+0000 7f0f57a13740 1 -- 172.21.15.123:0/1591089885 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d7a27ad670 con 0x55d7a27944b0 2024-09-19T04:54:09.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.971+0000 7f0f33e00640 1 -- 172.21.15.123:0/1591089885 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f3801cb00 con 0x55d7a27944b0 2024-09-19T04:54:09.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.971+0000 7f0f33e00640 1 -- 172.21.15.123:0/1591089885 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f0f4002a780 con 0x55d7a27944b0 2024-09-19T04:54:09.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.971+0000 7f0f57a13740 10 client.26723 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:09.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.971+0000 7f0f57a13740 10 client.26723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:09.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.971+0000 7f0f57a13740 20 client.26723 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:09.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.971+0000 7f0f57a13740 1 --2- 172.21.15.123:0/1591089885 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55d7a27aeb80 0x55d7a27b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:09.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.971+0000 7f0f57a13740 4 client.26723 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:09.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.971+0000 7f0f57a13740 1 -- 172.21.15.123:0/1591089885 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7a2707950 con 0x55d7a27aeb80 2024-09-19T04:54:09.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.971+0000 7f0f4de00640 1 --2- 172.21.15.123:0/1591089885 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55d7a27aeb80 0x55d7a27b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:09.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.972+0000 7f0f4de00640 1 --2- 172.21.15.123:0/1591089885 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55d7a27aeb80 0x55d7a27b0fa0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b11fada1e904d9b4 server_cookie=234f934bed62a97 in_seq=0 out_seq=0 2024-09-19T04:54:09.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.972+0000 7f0f33e00640 10 client.26723 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:09.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.973+0000 7f0f33e00640 1 -- 172.21.15.123:0/1591089885 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7a2707950 con 0x55d7a27aeb80 2024-09-19T04:54:09.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.973+0000 7f0f33e00640 10 client.26723 handle_command_reply: tid=0 2024-09-19T04:54:09.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.973+0000 7f0f57a13740 1 client.26723 shutdown 2024-09-19T04:54:09.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.973+0000 7f0f57a13740 1 -- 172.21.15.123:0/1591089885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a27944b0 msgr2=0x55d7a279c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:09.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.973+0000 7f0f57a13740 1 --2- 172.21.15.123:0/1591089885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a27944b0 0x55d7a279c0e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0f40013410 tx=0x7f0f40005f90 comp rx=0 tx=0).stop 2024-09-19T04:54:09.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.973+0000 7f0f57a13740 1 -- 172.21.15.123:0/1591089885 shutdown_connections 2024-09-19T04:54:09.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.973+0000 7f0f57a13740 1 --2- 172.21.15.123:0/1591089885 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55d7a27aeb80 0x55d7a27b0fa0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.973+0000 7f0f57a13740 1 --2- 172.21.15.123:0/1591089885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a279ed70 0x55d7a279c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.973+0000 7f0f57a13740 1 --2- 172.21.15.123:0/1591089885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a27944b0 0x55d7a279c0e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.973+0000 7f0f57a13740 1 --2- 172.21.15.123:0/1591089885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a2793b90 0x55d7a279bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.973+0000 7f0f57a13740 1 -- 172.21.15.123:0/1591089885 >> 172.21.15.123:0/1591089885 conn(0x55d7a2790120 msgr2=0x55d7a2795e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:09.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.973+0000 7f0f33e00640 0 client.26723 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:09.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.973+0000 7f0f57a13740 1 -- 172.21.15.123:0/1591089885 shutdown_connections 2024-09-19T04:54:09.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.973+0000 7f0f57a13740 1 -- 172.21.15.123:0/1591089885 wait complete. 2024-09-19T04:54:09.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.973+0000 7f0f57a13740 20 client.26723 trim_cache size 0 max 16384 2024-09-19T04:54:09.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.991+0000 7f0f57a13740 1 Processor -- start 2024-09-19T04:54:09.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.991+0000 7f0f57a13740 1 -- start start 2024-09-19T04:54:09.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.991+0000 7f0f57a13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a2793b90 0x55d7a2790500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:09.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.991+0000 7f0f57a13740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 0x55d7a279db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:09.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.991+0000 7f0f57a13740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a279ed70 0x55d7a27b6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:09.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.991+0000 7f0f57a13740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0f501afaf0 con 0x55d7a27944b0 2024-09-19T04:54:09.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.991+0000 7f0f57a13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f501b16c0 con 0x55d7a2793b90 2024-09-19T04:54:09.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.991+0000 7f0f57a13740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0f501b01f0 con 0x55d7a279ed70 2024-09-19T04:54:09.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.992+0000 7f0f4d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 0x55d7a279db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:09.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.992+0000 7f0f4d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 0x55d7a279db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54234/0 (socket says 172.21.15.123:54234) 2024-09-19T04:54:09.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.992+0000 7f0f4d400640 1 -- 172.21.15.123:0/2248747595 learned_addr learned my addr 172.21.15.123:0/2248747595 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:09.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.992+0000 7f0f4e800640 1 --2- 172.21.15.123:0/2248747595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a279ed70 0x55d7a27b6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:09.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.992+0000 7f0f4de00640 1 --2- 172.21.15.123:0/2248747595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a2793b90 0x55d7a2790500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:09.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.992+0000 7f0f4d400640 1 -- 172.21.15.123:0/2248747595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a279ed70 msgr2=0x55d7a27b6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:09.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.992+0000 7f0f4d400640 1 --2- 172.21.15.123:0/2248747595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a279ed70 0x55d7a27b6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.992+0000 7f0f4d400640 1 -- 172.21.15.123:0/2248747595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a2793b90 msgr2=0x55d7a2790500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:09.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.992+0000 7f0f4d400640 1 --2- 172.21.15.123:0/2248747595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a2793b90 0x55d7a2790500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.992+0000 7f0f4d400640 1 -- 172.21.15.123:0/2248747595 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a2796a00 con 0x55d7a27944b0 2024-09-19T04:54:09.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.992+0000 7f0f4d400640 1 --2- 172.21.15.123:0/2248747595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 0x55d7a279db10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0f4001d040 tx=0x7f0f40013440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3f2a59e6f00452 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:09.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.992+0000 7f0f3ea00640 1 -- 172.21.15.123:0/2248747595 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4002a410 con 0x55d7a27944b0 2024-09-19T04:54:09.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.992+0000 7f0f3ea00640 1 -- 172.21.15.123:0/2248747595 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f40005030 con 0x55d7a27944b0 2024-09-19T04:54:09.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.992+0000 7f0f3ea00640 1 -- 172.21.15.123:0/2248747595 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f400207f0 con 0x55d7a27944b0 2024-09-19T04:54:09.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.993+0000 7f0f57a13740 1 -- 172.21.15.123:0/2248747595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 msgr2=0x55d7a279db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:09.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.993+0000 7f0f57a13740 1 --2- 172.21.15.123:0/2248747595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 0x55d7a279db10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0f4001d040 tx=0x7f0f40013440 comp rx=0 tx=0).stop 2024-09-19T04:54:09.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.993+0000 7f0f57a13740 1 -- 172.21.15.123:0/2248747595 shutdown_connections 2024-09-19T04:54:09.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.993+0000 7f0f57a13740 1 --2- 172.21.15.123:0/2248747595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a279ed70 0x55d7a27b6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.993+0000 7f0f57a13740 1 --2- 172.21.15.123:0/2248747595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 0x55d7a279db10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.993+0000 7f0f57a13740 1 --2- 172.21.15.123:0/2248747595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a2793b90 0x55d7a2790500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.993+0000 7f0f57a13740 1 -- 172.21.15.123:0/2248747595 >> 172.21.15.123:0/2248747595 conn(0x55d7a2790120 msgr2=0x55d7a279d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:09.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.993+0000 7f0f57a13740 1 -- 172.21.15.123:0/2248747595 shutdown_connections 2024-09-19T04:54:09.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.993+0000 7f0f57a13740 1 -- 172.21.15.123:0/2248747595 wait complete. 2024-09-19T04:54:09.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.994+0000 7f0f57a13740 1 -- start start 2024-09-19T04:54:09.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.994+0000 7f0f57a13740 1 Processor -- start 2024-09-19T04:54:09.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.994+0000 7f0f57a13740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a2793b90 0x55d7a27a73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:09.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.994+0000 7f0f57a13740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 0x55d7a2798770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:09.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.994+0000 7f0f57a13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a279ed70 0x55d7a2798cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:09.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.994+0000 7f0f57a13740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0f501b0ff0 con 0x55d7a27944b0 2024-09-19T04:54:09.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.994+0000 7f0f57a13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f501b08f0 con 0x55d7a279ed70 2024-09-19T04:54:09.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.994+0000 7f0f57a13740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0f501adaf0 con 0x55d7a2793b90 2024-09-19T04:54:09.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.994+0000 7f0f57a13740 10 client.0 _ll_register_callbacks cb 0x7f0f40005c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:09.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.995+0000 7f0f4d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 0x55d7a2798770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:09.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.995+0000 7f0f4de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a2793b90 0x55d7a27a73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:09.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.995+0000 7f0f4e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a279ed70 0x55d7a2798cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:09.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.995+0000 7f0f4d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 0x55d7a2798770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54244/0 (socket says 172.21.15.123:54244) 2024-09-19T04:54:09.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.995+0000 7f0f4d400640 1 -- 172.21.15.123:0/1693215975 learned_addr learned my addr 172.21.15.123:0/1693215975 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:09.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.995+0000 7f0f4de00640 1 -- 172.21.15.123:0/1693215975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a279ed70 msgr2=0x55d7a2798cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:09.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.995+0000 7f0f4de00640 1 --2- 172.21.15.123:0/1693215975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a279ed70 0x55d7a2798cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.995+0000 7f0f4de00640 1 -- 172.21.15.123:0/1693215975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 msgr2=0x55d7a2798770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:09.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.995+0000 7f0f4de00640 1 --2- 172.21.15.123:0/1693215975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 0x55d7a2798770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:09.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.995+0000 7f0f4de00640 1 -- 172.21.15.123:0/1693215975 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a27993a0 con 0x55d7a2793b90 2024-09-19T04:54:09.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.995+0000 7f0f4e800640 1 --2- 172.21.15.123:0/1693215975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a279ed70 0x55d7a2798cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:09.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.995+0000 7f0f4d400640 1 --2- 172.21.15.123:0/1693215975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 0x55d7a2798770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.995+0000 7f0f4de00640 1 --2- 172.21.15.123:0/1693215975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a2793b90 0x55d7a27a73b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0f38010ca0 tx=0x7f0f3800b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd0fb8a0b2099741 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.995+0000 7f0f33e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:54:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.996+0000 7f0f33e00640 1 -- 172.21.15.123:0/1693215975 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3802c030 con 0x55d7a2793b90 2024-09-19T04:54:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.996+0000 7f0f33e00640 1 -- 172.21.15.123:0/1693215975 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8557+0+0 (secure 0 0 0) 0x7f0f38004020 con 0x55d7a2793b90 2024-09-19T04:54:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.996+0000 7f0f57a13740 1 -- 172.21.15.123:0/1693215975 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f40005030 con 0x55d7a2793b90 2024-09-19T04:54:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.996+0000 7f0f33e00640 1 -- 172.21.15.123:0/1693215975 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0f38038050 con 0x55d7a2793b90 2024-09-19T04:54:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.996+0000 7f0f57a13740 10 client.26749 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.996+0000 7f0f57a13740 1 -- 172.21.15.123:0/1693215975 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d7a2819750 con 0x55d7a2793b90 2024-09-19T04:54:10.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.996+0000 7f0f33e00640 1 -- 172.21.15.123:0/1693215975 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f40005030 con 0x55d7a2793b90 2024-09-19T04:54:10.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.997+0000 7f0f33e00640 1 -- 172.21.15.123:0/1693215975 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f0f38011800 con 0x55d7a2793b90 2024-09-19T04:54:10.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.997+0000 7f0f57a13740 10 client.26749 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:10.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.997+0000 7f0f57a13740 10 client.26749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:10.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.997+0000 7f0f57a13740 20 client.26749 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:10.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.997+0000 7f0f57a13740 1 --2- 172.21.15.123:0/1693215975 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55d7a2819bf0 0x55d7a281bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:10.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.997+0000 7f0f57a13740 4 client.26749 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:54:10.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.997+0000 7f0f57a13740 1 -- 172.21.15.123:0/1693215975 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7a2701e60 con 0x55d7a2819bf0 2024-09-19T04:54:10.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.997+0000 7f0f4d400640 1 --2- 172.21.15.123:0/1693215975 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55d7a2819bf0 0x55d7a281bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:10.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.998+0000 7f0f4d400640 1 --2- 172.21.15.123:0/1693215975 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55d7a2819bf0 0x55d7a281bfe0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17828838177061f4 server_cookie=455d6fbd5d315161 in_seq=0 out_seq=0 2024-09-19T04:54:10.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.998+0000 7f0f33e00640 10 client.26749 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:10.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.998+0000 7f0f33e00640 1 -- 172.21.15.123:0/1693215975 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d7a2701e60 con 0x55d7a2819bf0 2024-09-19T04:54:10.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.998+0000 7f0f33e00640 10 client.26749 handle_command_reply: tid=0 2024-09-19T04:54:10.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.998+0000 7f0f57a13740 1 client.26749 shutdown 2024-09-19T04:54:10.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.998+0000 7f0f57a13740 1 -- 172.21.15.123:0/1693215975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a2793b90 msgr2=0x55d7a27a73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:10.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.998+0000 7f0f57a13740 1 --2- 172.21.15.123:0/1693215975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a2793b90 0x55d7a27a73b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0f38010ca0 tx=0x7f0f3800b880 comp rx=0 tx=0).stop 2024-09-19T04:54:10.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.998+0000 7f0f57a13740 1 -- 172.21.15.123:0/1693215975 shutdown_connections 2024-09-19T04:54:10.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.998+0000 7f0f57a13740 1 --2- 172.21.15.123:0/1693215975 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55d7a2819bf0 0x55d7a281bfe0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:10.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.998+0000 7f0f57a13740 1 --2- 172.21.15.123:0/1693215975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7a279ed70 0x55d7a2798cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:10.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.998+0000 7f0f33e00640 0 client.26749 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:10.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.998+0000 7f0f57a13740 1 --2- 172.21.15.123:0/1693215975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7a27944b0 0x55d7a2798770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:10.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.998+0000 7f0f57a13740 1 --2- 172.21.15.123:0/1693215975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a2793b90 0x55d7a27a73b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:10.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.998+0000 7f0f57a13740 1 -- 172.21.15.123:0/1693215975 >> 172.21.15.123:0/1693215975 conn(0x55d7a2790120 msgr2=0x55d7a278f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:10.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.998+0000 7f0f57a13740 1 -- 172.21.15.123:0/1693215975 shutdown_connections 2024-09-19T04:54:10.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.998+0000 7f0f57a13740 1 -- 172.21.15.123:0/1693215975 wait complete. 2024-09-19T04:54:10.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.998+0000 7f0f57a13740 20 client.26749 trim_cache size 0 max 16384 2024-09-19T04:54:10.004 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:54:10.004 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-09-19T04:54:10.004 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-09-19T04:54:10.004 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:54:10.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.999+0000 7f0f54e00640 1 -- 172.21.15.123:0/3133746827 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0f340a3430 msgr2=0x7f0f340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:10.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.999+0000 7f0f54e00640 1 --2- 172.21.15.123:0/3133746827 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0f340a3430 0x7f0f340a58f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009830 tx=0x7f0f400097c0 comp rx=0 tx=0).stop 2024-09-19T04:54:10.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.999+0000 7f0f54e00640 1 -- 172.21.15.123:0/3133746827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f5010c180 msgr2=0x7f0f5019eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:10.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.999+0000 7f0f54e00640 1 --2- 172.21.15.123:0/3133746827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f5010c180 0x7f0f5019eec0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0f48005cc0 tx=0x7f0f4802f420 comp rx=0 tx=0).stop 2024-09-19T04:54:10.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.999+0000 7f0f54e00640 1 -- 172.21.15.123:0/3133746827 shutdown_connections 2024-09-19T04:54:10.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.999+0000 7f0f54e00640 1 --2- 172.21.15.123:0/3133746827 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0f340a3430 0x7f0f340a58f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:10.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.999+0000 7f0f54e00640 1 --2- 172.21.15.123:0/3133746827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f5010c180 0x7f0f5019eec0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:10.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.999+0000 7f0f54e00640 1 --2- 172.21.15.123:0/3133746827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f50103670 0x7f0f5019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:10.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.999+0000 7f0f54e00640 1 --2- 172.21.15.123:0/3133746827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f50102d50 0x7f0f5019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:10.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.999+0000 7f0f54e00640 1 -- 172.21.15.123:0/3133746827 >> 172.21.15.123:0/3133746827 conn(0x7f0f500fc7d0 msgr2=0x7f0f500fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:10.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.999+0000 7f0f54e00640 1 -- 172.21.15.123:0/3133746827 shutdown_connections 2024-09-19T04:54:10.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:09.999+0000 7f0f54e00640 1 -- 172.21.15.123:0/3133746827 wait complete. 2024-09-19T04:54:10.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:10 smithi123 ceph-mon[30511]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T04:54:10.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:10 smithi193 ceph-mon[35592]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T04:54:10.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:10 smithi183 ceph-mon[35333]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T04:54:10.882 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:54:10.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:54:11.251 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:11.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:11 smithi123 ceph-mon[30511]: pgmap v230: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T04:54:11.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:11 smithi193 ceph-mon[35592]: pgmap v230: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T04:54:11.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.814+0000 7fc63a000640 1 -- 172.21.15.123:0/504057994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc634077040 msgr2=0x7fc6341124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:11.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.814+0000 7fc63a000640 1 --2- 172.21.15.123:0/504057994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc634077040 0x7fc6341124f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc620009900 tx=0x7fc62002a410 comp rx=0 tx=0).stop 2024-09-19T04:54:11.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.814+0000 7fc63a000640 1 -- 172.21.15.123:0/504057994 shutdown_connections 2024-09-19T04:54:11.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.814+0000 7fc63a000640 1 --2- 172.21.15.123:0/504057994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc634112a30 0x7fc634114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:11.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.814+0000 7fc63a000640 1 --2- 172.21.15.123:0/504057994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc634077040 0x7fc6341124f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:11.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.814+0000 7fc63a000640 1 --2- 172.21.15.123:0/504057994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc634076720 0x7fc634076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:11.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.814+0000 7fc63a000640 1 -- 172.21.15.123:0/504057994 >> 172.21.15.123:0/504057994 conn(0x7fc6340ffa90 msgr2=0x7fc634101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:11.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.814+0000 7fc63a000640 1 -- 172.21.15.123:0/504057994 shutdown_connections 2024-09-19T04:54:11.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.814+0000 7fc63a000640 1 -- 172.21.15.123:0/504057994 wait complete. 2024-09-19T04:54:11.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.815+0000 7fc63a000640 1 Processor -- start 2024-09-19T04:54:11.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.815+0000 7fc63a000640 1 -- start start 2024-09-19T04:54:11.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.815+0000 7fc63a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc634076720 0x7fc6341a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:11.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.815+0000 7fc63a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc634077040 0x7fc6341a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:11.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.815+0000 7fc63a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc634112a30 0x7fc63419e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:11.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.815+0000 7fc63a000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341185c0 con 0x7fc634112a30 2024-09-19T04:54:11.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.815+0000 7fc63a000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc634118440 con 0x7fc634077040 2024-09-19T04:54:11.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.815+0000 7fc63a000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc634118740 con 0x7fc634076720 2024-09-19T04:54:11.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.816+0000 7fc633e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc634112a30 0x7fc63419e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:11.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.816+0000 7fc633400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc634076720 0x7fc6341a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:11.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.816+0000 7fc633e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc634112a30 0x7fc63419e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54252/0 (socket says 172.21.15.123:54252) 2024-09-19T04:54:11.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.816+0000 7fc633e00640 1 -- 172.21.15.123:0/918522132 learned_addr learned my addr 172.21.15.123:0/918522132 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:11.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.816+0000 7fc632a00640 1 --2- 172.21.15.123:0/918522132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc634077040 0x7fc6341a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:11.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.816+0000 7fc633e00640 1 -- 172.21.15.123:0/918522132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc634076720 msgr2=0x7fc6341a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:11.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.816+0000 7fc633e00640 1 --2- 172.21.15.123:0/918522132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc634076720 0x7fc6341a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:11.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.816+0000 7fc633e00640 1 -- 172.21.15.123:0/918522132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc634077040 msgr2=0x7fc6341a4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:11.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.816+0000 7fc633e00640 1 --2- 172.21.15.123:0/918522132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc634077040 0x7fc6341a4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:11.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.816+0000 7fc633e00640 1 -- 172.21.15.123:0/918522132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63419f100 con 0x7fc634112a30 2024-09-19T04:54:11.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.817+0000 7fc633e00640 1 --2- 172.21.15.123:0/918522132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc634112a30 0x7fc63419e9d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00dac0 tx=0x7fc62c00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:11.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.817+0000 7fc62be00640 1 -- 172.21.15.123:0/918522132 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c017030 con 0x7fc634112a30 2024-09-19T04:54:11.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.817+0000 7fc62be00640 1 -- 172.21.15.123:0/918522132 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62c003e50 con 0x7fc634112a30 2024-09-19T04:54:11.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.817+0000 7fc63a000640 1 -- 172.21.15.123:0/918522132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63419f3f0 con 0x7fc634112a30 2024-09-19T04:54:11.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.817+0000 7fc62be00640 1 -- 172.21.15.123:0/918522132 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c010460 con 0x7fc634112a30 2024-09-19T04:54:11.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.817+0000 7fc63a000640 1 -- 172.21.15.123:0/918522132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6341ab6e0 con 0x7fc634112a30 2024-09-19T04:54:11.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.818+0000 7fc63a000640 1 -- 172.21.15.123:0/918522132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc634076b00 con 0x7fc634112a30 2024-09-19T04:54:11.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.820+0000 7fc62be00640 1 -- 172.21.15.123:0/918522132 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc62c01a070 con 0x7fc634112a30 2024-09-19T04:54:11.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.821+0000 7fc62be00640 1 --2- 172.21.15.123:0/918522132 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc6180a3430 0x7fc6180a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:11.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.821+0000 7fc62be00640 1 -- 172.21.15.123:0/918522132 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8557+0+0 (secure 0 0 0) 0x7fc62c0cfa00 con 0x7fc634112a30 2024-09-19T04:54:11.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.821+0000 7fc633400640 1 --2- 172.21.15.123:0/918522132 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc6180a3430 0x7fc6180a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:11.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.822+0000 7fc633400640 1 --2- 172.21.15.123:0/918522132 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc6180a3430 0x7fc6180a58f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc63419fb10 tx=0x7fc624005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:11.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:11.826+0000 7fc62be00640 1 -- 172.21.15.123:0/918522132 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc62c09c360 con 0x7fc634112a30 2024-09-19T04:54:11.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:11 smithi183 ceph-mon[35333]: pgmap v230: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T04:54:12.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:12.037+0000 7fc63a000640 1 -- 172.21.15.123:0/918522132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc63410dbc0 con 0x7fc634112a30 2024-09-19T04:54:12.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:12.038+0000 7fc62be00640 1 -- 172.21.15.123:0/918522132 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11994 (secure 0 0 0) 0x7fc62c09bc60 con 0x7fc634112a30 2024-09-19T04:54:12.040 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:54:12.040 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":40,"btime":"2024-09-19T04:52:39:230121+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:52:38.841872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24499,24500,24539,24505,24529]},"id":1}]} 2024-09-19T04:54:12.041 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 40 2024-09-19T04:54:12.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:12.041+0000 7fc63a000640 1 -- 172.21.15.123:0/918522132 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc6180a3430 msgr2=0x7fc6180a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:12.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:12.041+0000 7fc63a000640 1 --2- 172.21.15.123:0/918522132 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc6180a3430 0x7fc6180a58f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc63419fb10 tx=0x7fc624005f90 comp rx=0 tx=0).stop 2024-09-19T04:54:12.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:12.041+0000 7fc63a000640 1 -- 172.21.15.123:0/918522132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc634112a30 msgr2=0x7fc63419e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:12.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:12.041+0000 7fc63a000640 1 --2- 172.21.15.123:0/918522132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc634112a30 0x7fc63419e9d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00dac0 tx=0x7fc62c00df80 comp rx=0 tx=0).stop 2024-09-19T04:54:12.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:12.041+0000 7fc63a000640 1 -- 172.21.15.123:0/918522132 shutdown_connections 2024-09-19T04:54:12.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:12.041+0000 7fc63a000640 1 --2- 172.21.15.123:0/918522132 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc6180a3430 0x7fc6180a58f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:12.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:12.041+0000 7fc63a000640 1 --2- 172.21.15.123:0/918522132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc634112a30 0x7fc63419e9d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:12.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:12.041+0000 7fc63a000640 1 --2- 172.21.15.123:0/918522132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc634077040 0x7fc6341a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:12.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:12.041+0000 7fc63a000640 1 --2- 172.21.15.123:0/918522132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc634076720 0x7fc6341a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:12.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:12.041+0000 7fc63a000640 1 -- 172.21.15.123:0/918522132 >> 172.21.15.123:0/918522132 conn(0x7fc6340ffa90 msgr2=0x7fc634101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:12.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:12.041+0000 7fc63a000640 1 -- 172.21.15.123:0/918522132 shutdown_connections 2024-09-19T04:54:12.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:12.041+0000 7fc63a000640 1 -- 172.21.15.123:0/918522132 wait complete. 2024-09-19T04:54:12.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:12 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/918522132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:54:12.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:12 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/918522132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:54:12.878 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:12 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/918522132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:54:12.918 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:54:13.288 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:13.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:13 smithi123 ceph-mon[30511]: pgmap v231: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T04:54:13.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:13 smithi193 ceph-mon[35592]: pgmap v231: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T04:54:13.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.827+0000 7f984ea00640 1 -- 172.21.15.123:0/3247223536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98481049a0 msgr2=0x7f98481089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:13.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.827+0000 7f984ea00640 1 --2- 172.21.15.123:0/3247223536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98481049a0 0x7f98481089f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9834009900 tx=0x7f983402a410 comp rx=0 tx=0).stop 2024-09-19T04:54:13.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.827+0000 7f984ea00640 1 -- 172.21.15.123:0/3247223536 shutdown_connections 2024-09-19T04:54:13.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.827+0000 7f984ea00640 1 --2- 172.21.15.123:0/3247223536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9848109120 0x7f9848110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.827+0000 7f984ea00640 1 --2- 172.21.15.123:0/3247223536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98481049a0 0x7f98481089f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.827+0000 7f984ea00640 1 --2- 172.21.15.123:0/3247223536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9848103ff0 0x7f98481043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.827+0000 7f984ea00640 1 -- 172.21.15.123:0/3247223536 >> 172.21.15.123:0/3247223536 conn(0x7f98480ffa90 msgr2=0x7f9848101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:13.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.827+0000 7f984ea00640 1 -- 172.21.15.123:0/3247223536 shutdown_connections 2024-09-19T04:54:13.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.827+0000 7f984ea00640 1 -- 172.21.15.123:0/3247223536 wait complete. 2024-09-19T04:54:13.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.828+0000 7f984ea00640 1 Processor -- start 2024-09-19T04:54:13.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.828+0000 7f984ea00640 1 -- start start 2024-09-19T04:54:13.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.828+0000 7f984ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9848103ff0 0x7f984819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.828+0000 7f984ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98481049a0 0x7f984819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.829+0000 7f984ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9848109120 0x7f984819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.829+0000 7f984ea00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9848114190 con 0x7f9848109120 2024-09-19T04:54:13.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.829+0000 7f984ea00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9848114010 con 0x7f98481049a0 2024-09-19T04:54:13.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.829+0000 7f984ea00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9848114310 con 0x7f9848103ff0 2024-09-19T04:54:13.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.829+0000 7f9847e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9848103ff0 0x7f984819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.829+0000 7f9847e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9848103ff0 0x7f984819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:60070/0 (socket says 172.21.15.123:60070) 2024-09-19T04:54:13.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.829+0000 7f984ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9848109120 0x7f984819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.829+0000 7f9847e00640 1 -- 172.21.15.123:0/3918073713 learned_addr learned my addr 172.21.15.123:0/3918073713 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:13.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.829+0000 7f9847400640 1 --2- 172.21.15.123:0/3918073713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98481049a0 0x7f984819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.829+0000 7f984ca00640 1 -- 172.21.15.123:0/3918073713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9848103ff0 msgr2=0x7f984819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:13.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.829+0000 7f984ca00640 1 --2- 172.21.15.123:0/3918073713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9848103ff0 0x7f984819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.829+0000 7f984ca00640 1 -- 172.21.15.123:0/3918073713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98481049a0 msgr2=0x7f984819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:13.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.829+0000 7f984ca00640 1 --2- 172.21.15.123:0/3918073713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98481049a0 0x7f984819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.829+0000 7f984ca00640 1 -- 172.21.15.123:0/3918073713 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984819f5a0 con 0x7f9848109120 2024-09-19T04:54:13.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.829+0000 7f9847e00640 1 --2- 172.21.15.123:0/3918073713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9848103ff0 0x7f984819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:13.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.830+0000 7f984ca00640 1 --2- 172.21.15.123:0/3918073713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9848109120 0x7f984819eec0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9840009fd0 tx=0x7f984000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:13.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.830+0000 7f9844c00640 1 -- 172.21.15.123:0/3918073713 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984000ce20 con 0x7f9848109120 2024-09-19T04:54:13.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.830+0000 7f9844c00640 1 -- 172.21.15.123:0/3918073713 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9840005020 con 0x7f9848109120 2024-09-19T04:54:13.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.830+0000 7f984ea00640 1 -- 172.21.15.123:0/3918073713 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984819f890 con 0x7f9848109120 2024-09-19T04:54:13.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.830+0000 7f9844c00640 1 -- 172.21.15.123:0/3918073713 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840010720 con 0x7f9848109120 2024-09-19T04:54:13.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.830+0000 7f984ea00640 1 -- 172.21.15.123:0/3918073713 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98481a7170 con 0x7f9848109120 2024-09-19T04:54:13.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.834+0000 7f9851402740 1 Processor -- start 2024-09-19T04:54:13.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.834+0000 7f9844c00640 1 -- 172.21.15.123:0/3918073713 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9840005540 con 0x7f9848109120 2024-09-19T04:54:13.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.835+0000 7f9851402740 1 -- start start 2024-09-19T04:54:13.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.835+0000 7f9851402740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d82ec70 0x556f0d82f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.835+0000 7f9851402740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82f590 0x556f0d839910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.835+0000 7f9851402740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d839e50 0x556f0d83c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.835+0000 7f9851402740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556f0d7953e0 con 0x556f0d82f590 2024-09-19T04:54:13.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.835+0000 7f9851402740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556f0d73cd80 con 0x556f0d839e50 2024-09-19T04:54:13.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.835+0000 7f9851402740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556f0d7913a0 con 0x556f0d82ec70 2024-09-19T04:54:13.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.835+0000 7f9847e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82f590 0x556f0d839910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.835+0000 7f9847e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82f590 0x556f0d839910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53372/0 (socket says 172.21.15.123:53372) 2024-09-19T04:54:13.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.835+0000 7f9847e00640 1 -- 172.21.15.123:0/3977978987 learned_addr learned my addr 172.21.15.123:0/3977978987 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:13.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.835+0000 7f9847400640 1 --2- 172.21.15.123:0/3977978987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d82ec70 0x556f0d82f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.835+0000 7f9847400640 1 --2- 172.21.15.123:0/3977978987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d839e50 0x556f0d83c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.836+0000 7f9847e00640 1 -- 172.21.15.123:0/3977978987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d82ec70 msgr2=0x556f0d82f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:13.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.836+0000 7f9847e00640 1 --2- 172.21.15.123:0/3977978987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d82ec70 0x556f0d82f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.836+0000 7f9847e00640 1 -- 172.21.15.123:0/3977978987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d839e50 msgr2=0x556f0d83c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:13.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.836+0000 7f9847e00640 1 --2- 172.21.15.123:0/3977978987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d839e50 0x556f0d83c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.836+0000 7f9847e00640 1 -- 172.21.15.123:0/3977978987 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f0d75bae0 con 0x556f0d82f590 2024-09-19T04:54:13.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.836+0000 7f9847400640 1 --2- 172.21.15.123:0/3977978987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d839e50 0x556f0d83c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:54:13.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.836+0000 7f9847e00640 1 --2- 172.21.15.123:0/3977978987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82f590 0x556f0d839910 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9838011650 tx=0x7f9838009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da69df45500336e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:13.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.836+0000 7f983d200640 1 -- 172.21.15.123:0/3977978987 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983801e070 con 0x556f0d82f590 2024-09-19T04:54:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.836+0000 7f983d200640 1 -- 172.21.15.123:0/3977978987 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983800fba0 con 0x556f0d82f590 2024-09-19T04:54:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.836+0000 7f9844c00640 1 --2- 172.21.15.123:0/3918073713 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f982c0a3430 0x7f982c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.836+0000 7f983d200640 1 -- 172.21.15.123:0/3977978987 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838019cd0 con 0x556f0d82f590 2024-09-19T04:54:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.837+0000 7f9844c00640 1 -- 172.21.15.123:0/3918073713 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8557+0+0 (secure 0 0 0) 0x7f98400d0260 con 0x7f9848109120 2024-09-19T04:54:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.837+0000 7f9847400640 1 --2- 172.21.15.123:0/3918073713 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f982c0a3430 0x7f982c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.837+0000 7f9851402740 1 -- 172.21.15.123:0/3977978987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82f590 msgr2=0x556f0d839910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.837+0000 7f9851402740 1 --2- 172.21.15.123:0/3977978987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82f590 0x556f0d839910 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9838011650 tx=0x7f9838009f90 comp rx=0 tx=0).stop 2024-09-19T04:54:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.837+0000 7f9847400640 1 --2- 172.21.15.123:0/3918073713 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f982c0a3430 0x7f982c0a58f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9834000c00 tx=0x7f9834009260 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.837+0000 7f9851402740 1 -- 172.21.15.123:0/3977978987 shutdown_connections 2024-09-19T04:54:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.837+0000 7f9851402740 1 --2- 172.21.15.123:0/3977978987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d839e50 0x556f0d83c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.837+0000 7f9851402740 1 --2- 172.21.15.123:0/3977978987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82f590 0x556f0d839910 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.837+0000 7f9851402740 1 --2- 172.21.15.123:0/3977978987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d82ec70 0x556f0d82f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.837+0000 7f9851402740 1 -- 172.21.15.123:0/3977978987 >> 172.21.15.123:0/3977978987 conn(0x556f0d82b200 msgr2=0x556f0d82b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.837+0000 7f9851402740 1 -- 172.21.15.123:0/3977978987 shutdown_connections 2024-09-19T04:54:13.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.838+0000 7f9851402740 1 -- 172.21.15.123:0/3977978987 wait complete. 2024-09-19T04:54:13.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.838+0000 7f9851402740 1 -- start start 2024-09-19T04:54:13.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.838+0000 7f9851402740 1 Processor -- start 2024-09-19T04:54:13.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.839+0000 7f9851402740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82ec70 0x556f0d836d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.839+0000 7f9851402740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d82f590 0x556f0d837280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.839+0000 7f9851402740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d839e50 0x556f0d8377c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.839+0000 7f9851402740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556f0d83ca20 con 0x556f0d82ec70 2024-09-19T04:54:13.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.839+0000 7f9851402740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556f0d7500f0 con 0x556f0d82f590 2024-09-19T04:54:13.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.839+0000 7f9851402740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556f0d83cba0 con 0x556f0d839e50 2024-09-19T04:54:13.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.839+0000 7f9851402740 10 client.0 _ll_register_callbacks cb 0x556f0d737bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:13.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.839+0000 7f9847e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82ec70 0x556f0d836d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.839+0000 7f9847400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d82f590 0x556f0d837280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.839+0000 7f984ca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d839e50 0x556f0d8377c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.839+0000 7f984ca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d839e50 0x556f0d8377c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:60076/0 (socket says 172.21.15.123:60076) 2024-09-19T04:54:13.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.839+0000 7f9847e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82ec70 0x556f0d836d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53382/0 (socket says 172.21.15.123:53382) 2024-09-19T04:54:13.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.839+0000 7f984ca00640 1 -- 172.21.15.123:0/4283032816 learned_addr learned my addr 172.21.15.123:0/4283032816 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:13.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.839+0000 7f9847e00640 1 -- 172.21.15.123:0/4283032816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d839e50 msgr2=0x556f0d8377c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:13.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.839+0000 7f9847e00640 1 --2- 172.21.15.123:0/4283032816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d839e50 0x556f0d8377c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.839+0000 7f9847e00640 1 -- 172.21.15.123:0/4283032816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d82f590 msgr2=0x556f0d837280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:13.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.839+0000 7f9847e00640 1 --2- 172.21.15.123:0/4283032816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d82f590 0x556f0d837280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.840+0000 7f9847e00640 1 -- 172.21.15.123:0/4283032816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f0d5063c0 con 0x556f0d82ec70 2024-09-19T04:54:13.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.840+0000 7f9847e00640 1 --2- 172.21.15.123:0/4283032816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82ec70 0x556f0d836d40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9838019810 tx=0x7f983800fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83057d8faed90beb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:13.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.840+0000 7f9831600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:13.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.840+0000 7f9831600640 1 -- 172.21.15.123:0/4283032816 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983801e040 con 0x556f0d82ec70 2024-09-19T04:54:13.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.840+0000 7f9831600640 1 -- 172.21.15.123:0/4283032816 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8557+0+0 (secure 0 0 0) 0x7f983800fd50 con 0x556f0d82ec70 2024-09-19T04:54:13.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.840+0000 7f9851402740 1 -- 172.21.15.123:0/4283032816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f983800fba0 con 0x556f0d82ec70 2024-09-19T04:54:13.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.840+0000 7f9831600640 1 -- 172.21.15.123:0/4283032816 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f983802f050 con 0x556f0d82ec70 2024-09-19T04:54:13.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.840+0000 7f9851402740 10 client.16647 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:13.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.840+0000 7f9851402740 1 -- 172.21.15.123:0/4283032816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556f0d848a60 con 0x556f0d82ec70 2024-09-19T04:54:13.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.841+0000 7f9831600640 1 -- 172.21.15.123:0/4283032816 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983800fba0 con 0x556f0d82ec70 2024-09-19T04:54:13.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.841+0000 7f9831600640 1 -- 172.21.15.123:0/4283032816 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f983802a030 con 0x556f0d82ec70 2024-09-19T04:54:13.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.841+0000 7f9851402740 10 client.16647 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:13.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.841+0000 7f9851402740 10 client.16647 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:54:13.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.841+0000 7f9851402740 20 client.16647 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:13.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.841+0000 7f9851402740 1 --2- 172.21.15.123:0/4283032816 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556f0d849ec0 0x556f0d84c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.841+0000 7f9851402740 4 client.16647 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:13.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.841+0000 7f9851402740 1 -- 172.21.15.123:0/4283032816 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f0d7a2950 con 0x556f0d849ec0 2024-09-19T04:54:13.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.841+0000 7f9847400640 1 --2- 172.21.15.123:0/4283032816 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556f0d849ec0 0x556f0d84c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.842+0000 7f9847400640 1 --2- 172.21.15.123:0/4283032816 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556f0d849ec0 0x556f0d84c2b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2faf979ba35719a9 server_cookie=46b6708ccc55bd3e in_seq=0 out_seq=0 2024-09-19T04:54:13.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.842+0000 7f9831600640 10 client.16647 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:54:13.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.843+0000 7f9831600640 1 -- 172.21.15.123:0/4283032816 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f0d7a2950 con 0x556f0d849ec0 2024-09-19T04:54:13.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.843+0000 7f9831600640 10 client.16647 handle_command_reply: tid=0 2024-09-19T04:54:13.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.843+0000 7f9851402740 1 client.16647 shutdown 2024-09-19T04:54:13.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.843+0000 7f9851402740 1 -- 172.21.15.123:0/4283032816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82ec70 msgr2=0x556f0d836d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:13.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.843+0000 7f9851402740 1 --2- 172.21.15.123:0/4283032816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82ec70 0x556f0d836d40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9838019810 tx=0x7f983800fb30 comp rx=0 tx=0).stop 2024-09-19T04:54:13.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.843+0000 7f9851402740 1 -- 172.21.15.123:0/4283032816 shutdown_connections 2024-09-19T04:54:13.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.843+0000 7f9851402740 1 --2- 172.21.15.123:0/4283032816 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556f0d849ec0 0x556f0d84c2b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.843+0000 7f9851402740 1 --2- 172.21.15.123:0/4283032816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d839e50 0x556f0d8377c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.843+0000 7f9851402740 1 --2- 172.21.15.123:0/4283032816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d82f590 0x556f0d837280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.843+0000 7f9851402740 1 --2- 172.21.15.123:0/4283032816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82ec70 0x556f0d836d40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.843+0000 7f9851402740 1 -- 172.21.15.123:0/4283032816 >> 172.21.15.123:0/4283032816 conn(0x556f0d82b200 msgr2=0x556f0d830ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:13.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.843+0000 7f9831600640 0 client.16647 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:54:13.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.843+0000 7f9851402740 1 -- 172.21.15.123:0/4283032816 shutdown_connections 2024-09-19T04:54:13.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.843+0000 7f9851402740 1 -- 172.21.15.123:0/4283032816 wait complete. 2024-09-19T04:54:13.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.844+0000 7f9851402740 20 client.16647 trim_cache size 0 max 16384 2024-09-19T04:54:13.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:13 smithi183 ceph-mon[35333]: pgmap v231: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T04:54:13.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.857+0000 7f9851402740 1 Processor -- start 2024-09-19T04:54:13.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.857+0000 7f9851402740 1 -- start start 2024-09-19T04:54:13.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.857+0000 7f9851402740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82ec70 0x556f0d851a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.857+0000 7f9851402740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d82f590 0x556f0d851fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.857+0000 7f9851402740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d839e50 0x556f0d8354e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.857+0000 7f9851402740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98481afae0 con 0x556f0d82ec70 2024-09-19T04:54:13.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.857+0000 7f9851402740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98481b16b0 con 0x556f0d839e50 2024-09-19T04:54:13.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.857+0000 7f9851402740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f98481b01e0 con 0x556f0d82f590 2024-09-19T04:54:13.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.858+0000 7f9847e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82ec70 0x556f0d851a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.858+0000 7f9847e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82ec70 0x556f0d851a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53392/0 (socket says 172.21.15.123:53392) 2024-09-19T04:54:13.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.858+0000 7f9847e00640 1 -- 172.21.15.123:0/288783153 learned_addr learned my addr 172.21.15.123:0/288783153 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:13.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.858+0000 7f984ca00640 1 --2- 172.21.15.123:0/288783153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d839e50 0x556f0d8354e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.858+0000 7f9847400640 1 --2- 172.21.15.123:0/288783153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d82f590 0x556f0d851fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.858+0000 7f9847e00640 1 -- 172.21.15.123:0/288783153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d82f590 msgr2=0x556f0d851fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:13.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.858+0000 7f9847e00640 1 --2- 172.21.15.123:0/288783153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d82f590 0x556f0d851fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.858+0000 7f9847e00640 1 -- 172.21.15.123:0/288783153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d839e50 msgr2=0x556f0d8354e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:13.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.858+0000 7f9847e00640 1 --2- 172.21.15.123:0/288783153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d839e50 0x556f0d8354e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.858+0000 7f9847e00640 1 -- 172.21.15.123:0/288783153 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f0d82b030 con 0x556f0d82ec70 2024-09-19T04:54:13.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.858+0000 7f9847e00640 1 --2- 172.21.15.123:0/288783153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82ec70 0x556f0d851a70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f98380097f0 tx=0x7f98380039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5b8080db36dea8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:13.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.859+0000 7f9833e00640 1 -- 172.21.15.123:0/288783153 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838015070 con 0x556f0d82ec70 2024-09-19T04:54:13.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.859+0000 7f9833e00640 1 -- 172.21.15.123:0/288783153 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9838019610 con 0x556f0d82ec70 2024-09-19T04:54:13.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.859+0000 7f9833e00640 1 -- 172.21.15.123:0/288783153 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983801e070 con 0x556f0d82ec70 2024-09-19T04:54:13.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.859+0000 7f9851402740 1 -- 172.21.15.123:0/288783153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82ec70 msgr2=0x556f0d851a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:13.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.859+0000 7f9851402740 1 --2- 172.21.15.123:0/288783153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82ec70 0x556f0d851a70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f98380097f0 tx=0x7f98380039c0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.859+0000 7f9851402740 1 -- 172.21.15.123:0/288783153 shutdown_connections 2024-09-19T04:54:13.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.859+0000 7f9851402740 1 --2- 172.21.15.123:0/288783153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d839e50 0x556f0d8354e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.859+0000 7f9851402740 1 --2- 172.21.15.123:0/288783153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d82f590 0x556f0d851fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.859+0000 7f9851402740 1 --2- 172.21.15.123:0/288783153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d82ec70 0x556f0d851a70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.859+0000 7f9851402740 1 -- 172.21.15.123:0/288783153 >> 172.21.15.123:0/288783153 conn(0x556f0d82b200 msgr2=0x556f0d838dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:13.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.859+0000 7f9851402740 1 -- 172.21.15.123:0/288783153 shutdown_connections 2024-09-19T04:54:13.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.859+0000 7f9851402740 1 -- 172.21.15.123:0/288783153 wait complete. 2024-09-19T04:54:13.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.860+0000 7f9851402740 1 -- start start 2024-09-19T04:54:13.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.860+0000 7f9851402740 1 Processor -- start 2024-09-19T04:54:13.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.860+0000 7f9851402740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d82ec70 0x556f0d841fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.860+0000 7f9851402740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d82f590 0x556f0d842510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.860+0000 7f9851402740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d839e50 0x556f0d8335a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.860+0000 7f9851402740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98481b0fe0 con 0x556f0d839e50 2024-09-19T04:54:13.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.860+0000 7f9851402740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98481b08e0 con 0x556f0d82ec70 2024-09-19T04:54:13.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.860+0000 7f9851402740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f98481adae0 con 0x556f0d82f590 2024-09-19T04:54:13.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.860+0000 7f9851402740 10 client.0 _ll_register_callbacks cb 0x7f983801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:13.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.861+0000 7f9847400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d82f590 0x556f0d842510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.861+0000 7f984ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d839e50 0x556f0d8335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.861+0000 7f9847400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d82f590 0x556f0d842510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:60092/0 (socket says 172.21.15.123:60092) 2024-09-19T04:54:13.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.861+0000 7f9847400640 1 -- 172.21.15.123:0/3840963626 learned_addr learned my addr 172.21.15.123:0/3840963626 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:13.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.861+0000 7f9847e00640 1 --2- 172.21.15.123:0/3840963626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d82ec70 0x556f0d841fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.861+0000 7f9847400640 1 -- 172.21.15.123:0/3840963626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d82ec70 msgr2=0x556f0d841fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:13.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.861+0000 7f9847400640 1 --2- 172.21.15.123:0/3840963626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d82ec70 0x556f0d841fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.861+0000 7f9847400640 1 -- 172.21.15.123:0/3840963626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d839e50 msgr2=0x556f0d8335a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:13.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.861+0000 7f9847400640 1 --2- 172.21.15.123:0/3840963626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d839e50 0x556f0d8335a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.861+0000 7f9847400640 1 -- 172.21.15.123:0/3840963626 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f0d826770 con 0x556f0d82f590 2024-09-19T04:54:13.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.861+0000 7f9847400640 1 --2- 172.21.15.123:0/3840963626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d82f590 0x556f0d842510 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9834009230 tx=0x7f9834005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a08393f83ac35f86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:13.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.861+0000 7f9831600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:54:13.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.862+0000 7f9831600640 1 -- 172.21.15.123:0/3840963626 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98340329d0 con 0x556f0d82f590 2024-09-19T04:54:13.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.862+0000 7f9831600640 1 -- 172.21.15.123:0/3840963626 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8557+0+0 (secure 0 0 0) 0x7f9834032b70 con 0x556f0d82f590 2024-09-19T04:54:13.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.862+0000 7f9851402740 1 -- 172.21.15.123:0/3840963626 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9838019610 con 0x556f0d82f590 2024-09-19T04:54:13.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.862+0000 7f9831600640 1 -- 172.21.15.123:0/3840963626 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9834032d70 con 0x556f0d82f590 2024-09-19T04:54:13.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.862+0000 7f9851402740 10 client.26773 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:13.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.862+0000 7f9851402740 1 -- 172.21.15.123:0/3840963626 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556f0d8b42f0 con 0x556f0d82f590 2024-09-19T04:54:13.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.862+0000 7f9831600640 1 -- 172.21.15.123:0/3840963626 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9838019610 con 0x556f0d82f590 2024-09-19T04:54:13.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.863+0000 7f9831600640 1 -- 172.21.15.123:0/3840963626 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f98340544c0 con 0x556f0d82f590 2024-09-19T04:54:13.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.863+0000 7f9851402740 10 client.26773 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:13.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.863+0000 7f9851402740 10 client.26773 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:54:13.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.863+0000 7f9851402740 20 client.26773 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:13.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.863+0000 7f9851402740 1 --2- 172.21.15.123:0/3840963626 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556f0d8b4a80 0x556f0d8b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:13.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.863+0000 7f9851402740 4 client.26773 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:13.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.863+0000 7f9851402740 1 -- 172.21.15.123:0/3840963626 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f0d79ce60 con 0x556f0d8b4a80 2024-09-19T04:54:13.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.863+0000 7f9847e00640 1 --2- 172.21.15.123:0/3840963626 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556f0d8b4a80 0x556f0d8b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:13.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.863+0000 7f9847e00640 1 --2- 172.21.15.123:0/3840963626 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556f0d8b4a80 0x556f0d8b6ea0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c783259c482b103b server_cookie=df90f380cc84dc1a in_seq=0 out_seq=0 2024-09-19T04:54:13.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.863+0000 7f9831600640 10 client.26773 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:54:13.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.864+0000 7f9831600640 1 -- 172.21.15.123:0/3840963626 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f0d79ce60 con 0x556f0d8b4a80 2024-09-19T04:54:13.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.864+0000 7f9831600640 10 client.26773 handle_command_reply: tid=0 2024-09-19T04:54:13.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.864+0000 7f9851402740 1 client.26773 shutdown 2024-09-19T04:54:13.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.864+0000 7f9851402740 1 -- 172.21.15.123:0/3840963626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d82f590 msgr2=0x556f0d842510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:13.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.864+0000 7f9851402740 1 --2- 172.21.15.123:0/3840963626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d82f590 0x556f0d842510 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9834009230 tx=0x7f9834005b50 comp rx=0 tx=0).stop 2024-09-19T04:54:13.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.864+0000 7f9851402740 1 -- 172.21.15.123:0/3840963626 shutdown_connections 2024-09-19T04:54:13.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.864+0000 7f9851402740 1 --2- 172.21.15.123:0/3840963626 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x556f0d8b4a80 0x556f0d8b6ea0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.864+0000 7f9851402740 1 --2- 172.21.15.123:0/3840963626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f0d839e50 0x556f0d8335a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.864+0000 7f9851402740 1 --2- 172.21.15.123:0/3840963626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f0d82f590 0x556f0d842510 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.864+0000 7f9851402740 1 --2- 172.21.15.123:0/3840963626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f0d82ec70 0x556f0d841fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.864+0000 7f9831600640 0 client.26773 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:54:13.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.864+0000 7f9851402740 1 -- 172.21.15.123:0/3840963626 >> 172.21.15.123:0/3840963626 conn(0x556f0d82b200 msgr2=0x556f0d829b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:13.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.864+0000 7f9851402740 1 -- 172.21.15.123:0/3840963626 shutdown_connections 2024-09-19T04:54:13.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.864+0000 7f9851402740 1 -- 172.21.15.123:0/3840963626 wait complete. 2024-09-19T04:54:13.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.864+0000 7f9851402740 20 client.26773 trim_cache size 0 max 16384 2024-09-19T04:54:13.869 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:13.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.865+0000 7f984ea00640 1 -- 172.21.15.123:0/3918073713 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f982c0a3430 msgr2=0x7f982c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:13.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.865+0000 7f984ea00640 1 --2- 172.21.15.123:0/3918073713 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f982c0a3430 0x7f982c0a58f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9834000c00 tx=0x7f9834009260 comp rx=0 tx=0).stop 2024-09-19T04:54:13.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.865+0000 7f984ea00640 1 -- 172.21.15.123:0/3918073713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9848109120 msgr2=0x7f984819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:13.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.865+0000 7f984ea00640 1 --2- 172.21.15.123:0/3918073713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9848109120 0x7f984819eec0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9840009fd0 tx=0x7f984000ef10 comp rx=0 tx=0).stop 2024-09-19T04:54:13.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.865+0000 7f984ea00640 1 -- 172.21.15.123:0/3918073713 shutdown_connections 2024-09-19T04:54:13.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.865+0000 7f984ea00640 1 --2- 172.21.15.123:0/3918073713 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f982c0a3430 0x7f982c0a58f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.865+0000 7f984ea00640 1 --2- 172.21.15.123:0/3918073713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9848109120 0x7f984819eec0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.865+0000 7f984ea00640 1 --2- 172.21.15.123:0/3918073713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98481049a0 0x7f984819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.865+0000 7f984ea00640 1 --2- 172.21.15.123:0/3918073713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9848103ff0 0x7f984819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:13.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.865+0000 7f984ea00640 1 -- 172.21.15.123:0/3918073713 >> 172.21.15.123:0/3918073713 conn(0x7f98480ffa90 msgr2=0x7f9848101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:13.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.865+0000 7f984ea00640 1 -- 172.21.15.123:0/3918073713 shutdown_connections 2024-09-19T04:54:13.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:13.865+0000 7f984ea00640 1 -- 172.21.15.123:0/3918073713 wait complete. 2024-09-19T04:54:14.671 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:54:15.053 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:15.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.637+0000 7f1917800640 1 -- 172.21.15.123:0/1377422187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1910109140 msgr2=0x7f1910110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:15.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.637+0000 7f1917800640 1 --2- 172.21.15.123:0/1377422187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1910109140 0x7f1910110cc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1908009ca0 tx=0x7f190802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.637+0000 7f1917800640 1 -- 172.21.15.123:0/1377422187 shutdown_connections 2024-09-19T04:54:15.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.637+0000 7f1917800640 1 --2- 172.21.15.123:0/1377422187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1910109140 0x7f1910110cc0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.637+0000 7f1917800640 1 --2- 172.21.15.123:0/1377422187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19101049c0 0x7f1910108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.637+0000 7f1917800640 1 --2- 172.21.15.123:0/1377422187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1910104010 0x7f19101043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.637+0000 7f1917800640 1 -- 172.21.15.123:0/1377422187 >> 172.21.15.123:0/1377422187 conn(0x7f19100ffa50 msgr2=0x7f1910101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.637+0000 7f1917800640 1 -- 172.21.15.123:0/1377422187 shutdown_connections 2024-09-19T04:54:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.637+0000 7f1917800640 1 -- 172.21.15.123:0/1377422187 wait complete. 2024-09-19T04:54:15.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.639+0000 7f1917800640 1 Processor -- start 2024-09-19T04:54:15.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.639+0000 7f1917800640 1 -- start start 2024-09-19T04:54:15.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.639+0000 7f1917800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1910104010 0x7f191019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.639+0000 7f1917800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19101049c0 0x7f191019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.639+0000 7f1917800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1910109140 0x7f191019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.639+0000 7f1917800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1910114150 con 0x7f1910109140 2024-09-19T04:54:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.639+0000 7f1917800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1910113fd0 con 0x7f1910104010 2024-09-19T04:54:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.639+0000 7f1917800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f19101142d0 con 0x7f19101049c0 2024-09-19T04:54:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.639+0000 7f1915800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1910109140 0x7f191019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.640+0000 7f1914e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1910104010 0x7f191019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.640+0000 7f1915800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1910109140 0x7f191019ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53422/0 (socket says 172.21.15.123:53422) 2024-09-19T04:54:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.640+0000 7f190fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19101049c0 0x7f191019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.640+0000 7f1915800640 1 -- 172.21.15.123:0/204147881 learned_addr learned my addr 172.21.15.123:0/204147881 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.640+0000 7f1915800640 1 -- 172.21.15.123:0/204147881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19101049c0 msgr2=0x7f191019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.640+0000 7f1915800640 1 --2- 172.21.15.123:0/204147881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19101049c0 0x7f191019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.640+0000 7f1915800640 1 -- 172.21.15.123:0/204147881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1910104010 msgr2=0x7f191019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:15.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.640+0000 7f1915800640 1 --2- 172.21.15.123:0/204147881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1910104010 0x7f191019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.640+0000 7f1915800640 1 -- 172.21.15.123:0/204147881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191019f640 con 0x7f1910109140 2024-09-19T04:54:15.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.640+0000 7f1915800640 1 --2- 172.21.15.123:0/204147881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1910109140 0x7f191019ef60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1908031a20 tx=0x7f1908031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:15.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.641+0000 7f190d600640 1 -- 172.21.15.123:0/204147881 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908003c30 con 0x7f1910109140 2024-09-19T04:54:15.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.641+0000 7f190d600640 1 -- 172.21.15.123:0/204147881 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1908003dd0 con 0x7f1910109140 2024-09-19T04:54:15.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.641+0000 7f190d600640 1 -- 172.21.15.123:0/204147881 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190802fc40 con 0x7f1910109140 2024-09-19T04:54:15.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.641+0000 7f1917800640 1 -- 172.21.15.123:0/204147881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191019f8d0 con 0x7f1910109140 2024-09-19T04:54:15.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.641+0000 7f1917800640 1 -- 172.21.15.123:0/204147881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19101a7210 con 0x7f1910109140 2024-09-19T04:54:15.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.644+0000 7f190d600640 1 -- 172.21.15.123:0/204147881 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f1908004050 con 0x7f1910109140 2024-09-19T04:54:15.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.644+0000 7f191a424740 1 Processor -- start 2024-09-19T04:54:15.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.644+0000 7f191a424740 1 -- start start 2024-09-19T04:54:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.644+0000 7f191a424740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cc8c70 0x563809cc9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.645+0000 7f191a424740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.645+0000 7f191a424740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cd3e50 0x563809cd6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.645+0000 7f191a424740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563809c2f3e0 con 0x563809cc8c70 2024-09-19T04:54:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.645+0000 7f191a424740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563809bd6d80 con 0x563809cd3e50 2024-09-19T04:54:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.645+0000 7f191a424740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563809c2b3a0 con 0x563809cc9590 2024-09-19T04:54:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.645+0000 7f1914e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.645+0000 7f190fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cc8c70 0x563809cc9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.645+0000 7f1914e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:60118/0 (socket says 172.21.15.123:60118) 2024-09-19T04:54:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.645+0000 7f1914e00640 1 -- 172.21.15.123:0/526443588 learned_addr learned my addr 172.21.15.123:0/526443588 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.645+0000 7f1914e00640 1 -- 172.21.15.123:0/526443588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cd3e50 msgr2=0x563809cd6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:15.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.645+0000 7f1914e00640 1 --2- 172.21.15.123:0/526443588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cd3e50 0x563809cd6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.645+0000 7f1914e00640 1 -- 172.21.15.123:0/526443588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cc8c70 msgr2=0x563809cc9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:15.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.645+0000 7f1914e00640 1 --2- 172.21.15.123:0/526443588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cc8c70 0x563809cc9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.645+0000 7f1914e00640 1 -- 172.21.15.123:0/526443588 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563809bf5ae0 con 0x563809cc9590 2024-09-19T04:54:15.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.646+0000 7f190d600640 1 --2- 172.21.15.123:0/204147881 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f18f80a3430 0x7f18f80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.646+0000 7f1914e00640 1 --2- 172.21.15.123:0/526443588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd3910 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1904009cf0 tx=0x7f1904011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7831c4f036554da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:15.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.646+0000 7f1902000640 1 -- 172.21.15.123:0/526443588 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190401e070 con 0x563809cc9590 2024-09-19T04:54:15.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.646+0000 7f1902000640 1 -- 172.21.15.123:0/526443588 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1904019b10 con 0x563809cc9590 2024-09-19T04:54:15.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.646+0000 7f1902000640 1 -- 172.21.15.123:0/526443588 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190400ead0 con 0x563809cc9590 2024-09-19T04:54:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.646+0000 7f190d600640 1 -- 172.21.15.123:0/204147881 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8557+0+0 (secure 0 0 0) 0x7f19080f0d70 con 0x7f1910109140 2024-09-19T04:54:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.646+0000 7f190fe00640 1 --2- 172.21.15.123:0/204147881 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f18f80a3430 0x7f18f80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.647+0000 7f191a424740 1 -- 172.21.15.123:0/526443588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 msgr2=0x563809cd3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.647+0000 7f190fe00640 1 --2- 172.21.15.123:0/204147881 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f18f80a3430 0x7f18f80a58f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x563809cc6580 tx=0x7f18fc00b650 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.647+0000 7f191a424740 1 --2- 172.21.15.123:0/526443588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd3910 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1904009cf0 tx=0x7f1904011810 comp rx=0 tx=0).stop 2024-09-19T04:54:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.647+0000 7f191a424740 1 -- 172.21.15.123:0/526443588 shutdown_connections 2024-09-19T04:54:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.647+0000 7f191a424740 1 --2- 172.21.15.123:0/526443588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cd3e50 0x563809cd6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.647+0000 7f191a424740 1 --2- 172.21.15.123:0/526443588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd3910 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.647+0000 7f191a424740 1 --2- 172.21.15.123:0/526443588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cc8c70 0x563809cc9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.647+0000 7f191a424740 1 -- 172.21.15.123:0/526443588 >> 172.21.15.123:0/526443588 conn(0x563809cc5200 msgr2=0x563809cc55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.647+0000 7f191a424740 1 -- 172.21.15.123:0/526443588 shutdown_connections 2024-09-19T04:54:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.647+0000 7f191a424740 1 -- 172.21.15.123:0/526443588 wait complete. 2024-09-19T04:54:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.648+0000 7f191a424740 1 -- start start 2024-09-19T04:54:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.648+0000 7f191a424740 1 Processor -- start 2024-09-19T04:54:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.648+0000 7f191a424740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cc8c70 0x563809cd0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.648+0000 7f191a424740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.648+0000 7f191a424740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cd3e50 0x563809cd17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.648+0000 7f191a424740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563809cd6a20 con 0x563809cd3e50 2024-09-19T04:54:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.648+0000 7f191a424740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563809bea0f0 con 0x563809cc8c70 2024-09-19T04:54:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.648+0000 7f191a424740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563809cd6ba0 con 0x563809cc9590 2024-09-19T04:54:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.648+0000 7f191a424740 10 client.0 _ll_register_callbacks cb 0x563809bd1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.649+0000 7f1915800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cd3e50 0x563809cd17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.649+0000 7f190fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.649+0000 7f1915800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cd3e50 0x563809cd17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53444/0 (socket says 172.21.15.123:53444) 2024-09-19T04:54:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.649+0000 7f190fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:60124/0 (socket says 172.21.15.123:60124) 2024-09-19T04:54:15.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.649+0000 7f1915800640 1 -- 172.21.15.123:0/751135707 learned_addr learned my addr 172.21.15.123:0/751135707 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:15.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.649+0000 7f1914e00640 1 --2- 172.21.15.123:0/751135707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cc8c70 0x563809cd0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.649+0000 7f190fe00640 1 -- 172.21.15.123:0/751135707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cc8c70 msgr2=0x563809cd0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:15.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.649+0000 7f190fe00640 1 --2- 172.21.15.123:0/751135707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cc8c70 0x563809cd0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.649+0000 7f190fe00640 1 -- 172.21.15.123:0/751135707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cd3e50 msgr2=0x563809cd17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:15.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.649+0000 7f190fe00640 1 --2- 172.21.15.123:0/751135707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cd3e50 0x563809cd17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.649+0000 7f190fe00640 1 -- 172.21.15.123:0/751135707 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638099a03c0 con 0x563809cc9590 2024-09-19T04:54:15.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.649+0000 7f1914e00640 1 --2- 172.21.15.123:0/751135707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cc8c70 0x563809cd0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:15.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.649+0000 7f1915800640 1 --2- 172.21.15.123:0/751135707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cd3e50 0x563809cd17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:15.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.649+0000 7f190fe00640 1 --2- 172.21.15.123:0/751135707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd1270 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00b6c0 tx=0x7f18fc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e74aa0b40bed3c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:15.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.649+0000 7f18f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:54:15.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.650+0000 7f18f6a00640 1 -- 172.21.15.123:0/751135707 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc01db60 con 0x563809cc9590 2024-09-19T04:54:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.650+0000 7f18f6a00640 1 -- 172.21.15.123:0/751135707 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8557+0+0 (secure 0 0 0) 0x7f18fc01dd00 con 0x563809cc9590 2024-09-19T04:54:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.650+0000 7f191a424740 1 -- 172.21.15.123:0/751135707 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1904019b10 con 0x563809cc9590 2024-09-19T04:54:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.650+0000 7f18f6a00640 1 -- 172.21.15.123:0/751135707 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f18fc01b460 con 0x563809cc9590 2024-09-19T04:54:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.650+0000 7f191a424740 10 client.26785 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.650+0000 7f191a424740 1 -- 172.21.15.123:0/751135707 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563809ce2b10 con 0x563809cc9590 2024-09-19T04:54:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.650+0000 7f18f6a00640 1 -- 172.21.15.123:0/751135707 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1904019b10 con 0x563809cc9590 2024-09-19T04:54:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.651+0000 7f18f6a00640 1 -- 172.21.15.123:0/751135707 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f18fc024070 con 0x563809cc9590 2024-09-19T04:54:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.651+0000 7f191a424740 10 client.26785 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.651+0000 7f191a424740 10 client.26785 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:54:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.651+0000 7f191a424740 20 client.26785 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.651+0000 7f191a424740 1 --2- 172.21.15.123:0/751135707 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x563809ce4070 0x563809ce6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.651+0000 7f191a424740 4 client.26785 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:15.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.651+0000 7f191a424740 1 -- 172.21.15.123:0/751135707 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563809c3c950 con 0x563809ce4070 2024-09-19T04:54:15.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.651+0000 7f1914e00640 1 --2- 172.21.15.123:0/751135707 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x563809ce4070 0x563809ce6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.652+0000 7f1914e00640 1 --2- 172.21.15.123:0/751135707 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x563809ce4070 0x563809ce6480 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a72c757b92fae0de server_cookie=8efc6b747ff60b63 in_seq=0 out_seq=0 2024-09-19T04:54:15.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.652+0000 7f18f6a00640 10 client.26785 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:54:15.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.653+0000 7f18f6a00640 1 -- 172.21.15.123:0/751135707 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563809c3c950 con 0x563809ce4070 2024-09-19T04:54:15.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.653+0000 7f18f6a00640 10 client.26785 handle_command_reply: tid=0 2024-09-19T04:54:15.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.653+0000 7f191a424740 1 client.26785 shutdown 2024-09-19T04:54:15.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.653+0000 7f191a424740 1 -- 172.21.15.123:0/751135707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 msgr2=0x563809cd1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:15.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.653+0000 7f191a424740 1 --2- 172.21.15.123:0/751135707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd1270 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00b6c0 tx=0x7f18fc0046c0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.653+0000 7f191a424740 1 -- 172.21.15.123:0/751135707 shutdown_connections 2024-09-19T04:54:15.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.653+0000 7f191a424740 1 --2- 172.21.15.123:0/751135707 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x563809ce4070 0x563809ce6480 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.653+0000 7f191a424740 1 --2- 172.21.15.123:0/751135707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cd3e50 0x563809cd17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.653+0000 7f191a424740 1 --2- 172.21.15.123:0/751135707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd1270 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.653+0000 7f191a424740 1 --2- 172.21.15.123:0/751135707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cc8c70 0x563809cd0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.653+0000 7f18f6a00640 0 client.26785 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:54:15.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.653+0000 7f191a424740 1 -- 172.21.15.123:0/751135707 >> 172.21.15.123:0/751135707 conn(0x563809cc5200 msgr2=0x563809cc4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:15.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.653+0000 7f191a424740 1 -- 172.21.15.123:0/751135707 shutdown_connections 2024-09-19T04:54:15.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.653+0000 7f191a424740 1 -- 172.21.15.123:0/751135707 wait complete. 2024-09-19T04:54:15.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.653+0000 7f191a424740 20 client.26785 trim_cache size 0 max 16384 2024-09-19T04:54:15.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.667+0000 7f191a424740 1 Processor -- start 2024-09-19T04:54:15.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.667+0000 7f191a424740 1 -- start start 2024-09-19T04:54:15.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.667+0000 7f191a424740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc8c70 0x563809cebd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.667+0000 7f191a424740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cc9590 0x563809cec2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.667+0000 7f191a424740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cd3e50 0x563809cc3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.667+0000 7f191a424740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19101afad0 con 0x563809cc9590 2024-09-19T04:54:15.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.667+0000 7f191a424740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f19101b16a0 con 0x563809cd3e50 2024-09-19T04:54:15.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.667+0000 7f191a424740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f19101b01d0 con 0x563809cc8c70 2024-09-19T04:54:15.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.668+0000 7f190fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cc9590 0x563809cec2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.668+0000 7f1915800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cd3e50 0x563809cc3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.668+0000 7f190fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cc9590 0x563809cec2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53458/0 (socket says 172.21.15.123:53458) 2024-09-19T04:54:15.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.668+0000 7f190fe00640 1 -- 172.21.15.123:0/1311450698 learned_addr learned my addr 172.21.15.123:0/1311450698 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:15.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.668+0000 7f1914e00640 1 --2- 172.21.15.123:0/1311450698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc8c70 0x563809cebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.668+0000 7f190fe00640 1 -- 172.21.15.123:0/1311450698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc8c70 msgr2=0x563809cebd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:15.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.668+0000 7f190fe00640 1 --2- 172.21.15.123:0/1311450698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc8c70 0x563809cebd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.668+0000 7f190fe00640 1 -- 172.21.15.123:0/1311450698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cd3e50 msgr2=0x563809cc3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:15.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.668+0000 7f190fe00640 1 --2- 172.21.15.123:0/1311450698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cd3e50 0x563809cc3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.668+0000 7f190fe00640 1 -- 172.21.15.123:0/1311450698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563809cc0770 con 0x563809cc9590 2024-09-19T04:54:15.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.668+0000 7f1915800640 1 --2- 172.21.15.123:0/1311450698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cd3e50 0x563809cc3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:15.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.668+0000 7f190fe00640 1 --2- 172.21.15.123:0/1311450698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cc9590 0x563809cec2d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f18fc017040 tx=0x7f18fc00bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f911bb1a1423f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:15.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.668+0000 7f1901600640 1 -- 172.21.15.123:0/1311450698 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc010460 con 0x563809cc9590 2024-09-19T04:54:15.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.668+0000 7f1901600640 1 -- 172.21.15.123:0/1311450698 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18fc012e50 con 0x563809cc9590 2024-09-19T04:54:15.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.668+0000 7f1901600640 1 -- 172.21.15.123:0/1311450698 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc01b8f0 con 0x563809cc9590 2024-09-19T04:54:15.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.669+0000 7f191a424740 1 -- 172.21.15.123:0/1311450698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cc9590 msgr2=0x563809cec2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:15.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.669+0000 7f191a424740 1 --2- 172.21.15.123:0/1311450698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cc9590 0x563809cec2d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f18fc017040 tx=0x7f18fc00bcd0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.669+0000 7f191a424740 1 -- 172.21.15.123:0/1311450698 shutdown_connections 2024-09-19T04:54:15.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.669+0000 7f191a424740 1 --2- 172.21.15.123:0/1311450698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cd3e50 0x563809cc3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.669+0000 7f191a424740 1 --2- 172.21.15.123:0/1311450698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cc9590 0x563809cec2d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.669+0000 7f191a424740 1 --2- 172.21.15.123:0/1311450698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc8c70 0x563809cebd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.669+0000 7f191a424740 1 -- 172.21.15.123:0/1311450698 >> 172.21.15.123:0/1311450698 conn(0x563809cc5200 msgr2=0x563809ccfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:15.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.669+0000 7f191a424740 1 -- 172.21.15.123:0/1311450698 shutdown_connections 2024-09-19T04:54:15.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.669+0000 7f191a424740 1 -- 172.21.15.123:0/1311450698 wait complete. 2024-09-19T04:54:15.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.670+0000 7f191a424740 1 -- start start 2024-09-19T04:54:15.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.670+0000 7f191a424740 1 Processor -- start 2024-09-19T04:54:15.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.670+0000 7f191a424740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cc8c70 0x563809cdc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.670+0000 7f191a424740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.670+0000 7f191a424740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cd3e50 0x563809cd7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.670+0000 7f191a424740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19101b0fd0 con 0x563809cd3e50 2024-09-19T04:54:15.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.670+0000 7f191a424740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f19101b08d0 con 0x563809cc8c70 2024-09-19T04:54:15.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.670+0000 7f191a424740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f19101adad0 con 0x563809cc9590 2024-09-19T04:54:15.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.670+0000 7f191a424740 10 client.0 _ll_register_callbacks cb 0x7f18fc00bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:15.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.671+0000 7f190fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.671+0000 7f1914e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cc8c70 0x563809cdc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.671+0000 7f190fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:60134/0 (socket says 172.21.15.123:60134) 2024-09-19T04:54:15.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.671+0000 7f190fe00640 1 -- 172.21.15.123:0/3637795595 learned_addr learned my addr 172.21.15.123:0/3637795595 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:15.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.671+0000 7f1915800640 1 --2- 172.21.15.123:0/3637795595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cd3e50 0x563809cd7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.671+0000 7f190fe00640 1 -- 172.21.15.123:0/3637795595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cc8c70 msgr2=0x563809cdc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:15.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.671+0000 7f190fe00640 1 --2- 172.21.15.123:0/3637795595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cc8c70 0x563809cdc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.671+0000 7f190fe00640 1 -- 172.21.15.123:0/3637795595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cd3e50 msgr2=0x563809cd7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:15.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.671+0000 7f190fe00640 1 --2- 172.21.15.123:0/3637795595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cd3e50 0x563809cd7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.671+0000 7f190fe00640 1 -- 172.21.15.123:0/3637795595 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563809cc5e90 con 0x563809cc9590 2024-09-19T04:54:15.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.671+0000 7f1915800640 1 --2- 172.21.15.123:0/3637795595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cd3e50 0x563809cd7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:54:15.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.671+0000 7f190fe00640 1 --2- 172.21.15.123:0/3637795595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd7680 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f18fc005d00 tx=0x7f18fc0266f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6abf4b53bed93660 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:15.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.671+0000 7f18f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:54:15.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.671+0000 7f18f6a00640 1 -- 172.21.15.123:0/3637795595 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc043060 con 0x563809cc9590 2024-09-19T04:54:15.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.672+0000 7f191a424740 1 -- 172.21.15.123:0/3637795595 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18fc012e50 con 0x563809cc9590 2024-09-19T04:54:15.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.672+0000 7f18f6a00640 1 -- 172.21.15.123:0/3637795595 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8557+0+0 (secure 0 0 0) 0x7f18fc010e40 con 0x563809cc9590 2024-09-19T04:54:15.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.672+0000 7f18f6a00640 1 -- 172.21.15.123:0/3637795595 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f18fc01b670 con 0x563809cc9590 2024-09-19T04:54:15.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.672+0000 7f191a424740 10 client.26791 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:15.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.672+0000 7f191a424740 1 -- 172.21.15.123:0/3637795595 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563809d4e9a0 con 0x563809cc9590 2024-09-19T04:54:15.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.672+0000 7f18f6a00640 1 -- 172.21.15.123:0/3637795595 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18fc012e50 con 0x563809cc9590 2024-09-19T04:54:15.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.672+0000 7f18f6a00640 1 -- 172.21.15.123:0/3637795595 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f18fc024040 con 0x563809cc9590 2024-09-19T04:54:15.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.672+0000 7f191a424740 10 client.26791 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:15.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.672+0000 7f191a424740 10 client.26791 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:54:15.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.672+0000 7f191a424740 20 client.26791 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:15.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.672+0000 7f191a424740 1 --2- 172.21.15.123:0/3637795595 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x563809d4f010 0x563809d51430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.672+0000 7f191a424740 4 client.26791 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:15.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.672+0000 7f191a424740 1 -- 172.21.15.123:0/3637795595 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563809c36e60 con 0x563809d4f010 2024-09-19T04:54:15.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.673+0000 7f1914e00640 1 --2- 172.21.15.123:0/3637795595 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x563809d4f010 0x563809d51430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.673+0000 7f1914e00640 1 --2- 172.21.15.123:0/3637795595 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x563809d4f010 0x563809d51430 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a8baff9976dc2923 server_cookie=d8a947b040a17940 in_seq=0 out_seq=0 2024-09-19T04:54:15.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.673+0000 7f18f6a00640 10 client.26791 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:54:15.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.673+0000 7f18f6a00640 1 -- 172.21.15.123:0/3637795595 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563809c36e60 con 0x563809d4f010 2024-09-19T04:54:15.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.673+0000 7f18f6a00640 10 client.26791 handle_command_reply: tid=0 2024-09-19T04:54:15.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.673+0000 7f191a424740 1 client.26791 shutdown 2024-09-19T04:54:15.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.673+0000 7f191a424740 1 -- 172.21.15.123:0/3637795595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 msgr2=0x563809cd7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:15.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.673+0000 7f191a424740 1 --2- 172.21.15.123:0/3637795595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd7680 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f18fc005d00 tx=0x7f18fc0266f0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.674+0000 7f191a424740 1 -- 172.21.15.123:0/3637795595 shutdown_connections 2024-09-19T04:54:15.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.674+0000 7f191a424740 1 --2- 172.21.15.123:0/3637795595 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x563809d4f010 0x563809d51430 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.674+0000 7f191a424740 1 --2- 172.21.15.123:0/3637795595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563809cd3e50 0x563809cd7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.674+0000 7f191a424740 1 --2- 172.21.15.123:0/3637795595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563809cc9590 0x563809cd7680 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.674+0000 7f191a424740 1 --2- 172.21.15.123:0/3637795595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563809cc8c70 0x563809cdc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.674+0000 7f18f6a00640 0 client.26791 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:54:15.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.674+0000 7f191a424740 1 -- 172.21.15.123:0/3637795595 >> 172.21.15.123:0/3637795595 conn(0x563809cc5200 msgr2=0x563809cd0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:15.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.674+0000 7f191a424740 1 -- 172.21.15.123:0/3637795595 shutdown_connections 2024-09-19T04:54:15.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.674+0000 7f191a424740 1 -- 172.21.15.123:0/3637795595 wait complete. 2024-09-19T04:54:15.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.674+0000 7f191a424740 20 client.26791 trim_cache size 0 max 16384 2024-09-19T04:54:15.681 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:15.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.674+0000 7f1917800640 1 -- 172.21.15.123:0/204147881 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f18f80a3430 msgr2=0x7f18f80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:15.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.674+0000 7f1917800640 1 --2- 172.21.15.123:0/204147881 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f18f80a3430 0x7f18f80a58f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x563809cc6580 tx=0x7f18fc00b650 comp rx=0 tx=0).stop 2024-09-19T04:54:15.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.675+0000 7f1917800640 1 -- 172.21.15.123:0/204147881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1910109140 msgr2=0x7f191019ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:15.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.675+0000 7f1917800640 1 --2- 172.21.15.123:0/204147881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1910109140 0x7f191019ef60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1908031a20 tx=0x7f1908031a50 comp rx=0 tx=0).stop 2024-09-19T04:54:15.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.675+0000 7f1917800640 1 -- 172.21.15.123:0/204147881 shutdown_connections 2024-09-19T04:54:15.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.675+0000 7f1917800640 1 --2- 172.21.15.123:0/204147881 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f18f80a3430 0x7f18f80a58f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.675+0000 7f1917800640 1 --2- 172.21.15.123:0/204147881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1910109140 0x7f191019ef60 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.675+0000 7f1917800640 1 --2- 172.21.15.123:0/204147881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19101049c0 0x7f191019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.675+0000 7f1917800640 1 --2- 172.21.15.123:0/204147881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1910104010 0x7f191019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.675+0000 7f1917800640 1 -- 172.21.15.123:0/204147881 >> 172.21.15.123:0/204147881 conn(0x7f19100ffa50 msgr2=0x7f1910100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:15.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.675+0000 7f1917800640 1 -- 172.21.15.123:0/204147881 shutdown_connections 2024-09-19T04:54:15.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:15.675+0000 7f1917800640 1 -- 172.21.15.123:0/204147881 wait complete. 2024-09-19T04:54:15.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:15 smithi193 ceph-mon[35592]: pgmap v232: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:54:15.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:15 smithi183 ceph-mon[35333]: pgmap v232: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:54:15.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:15 smithi123 ceph-mon[30511]: pgmap v232: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:54:16.461 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:54:16.828 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.441+0000 7fdb49600640 1 -- 172.21.15.123:0/1233401822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb441079d0 msgr2=0x7fdb44107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.441+0000 7fdb49600640 1 --2- 172.21.15.123:0/1233401822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb441079d0 0x7fdb44107db0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdb34009930 tx=0x7fdb3402a420 comp rx=0 tx=0).stop 2024-09-19T04:54:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.441+0000 7fdb49600640 1 -- 172.21.15.123:0/1233401822 shutdown_connections 2024-09-19T04:54:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.441+0000 7fdb49600640 1 --2- 172.21.15.123:0/1233401822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb44102370 0x7fdb4410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.441+0000 7fdb49600640 1 --2- 172.21.15.123:0/1233401822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb441019d0 0x7fdb44101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.441+0000 7fdb49600640 1 --2- 172.21.15.123:0/1233401822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb441079d0 0x7fdb44107db0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.441+0000 7fdb49600640 1 -- 172.21.15.123:0/1233401822 >> 172.21.15.123:0/1233401822 conn(0x7fdb440fd710 msgr2=0x7fdb440ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.441+0000 7fdb49600640 1 -- 172.21.15.123:0/1233401822 shutdown_connections 2024-09-19T04:54:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.441+0000 7fdb49600640 1 -- 172.21.15.123:0/1233401822 wait complete. 2024-09-19T04:54:17.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.442+0000 7fdb49600640 1 Processor -- start 2024-09-19T04:54:17.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.442+0000 7fdb49600640 1 -- start start 2024-09-19T04:54:17.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb49600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb441019d0 0x7fdb4419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb49600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb44102370 0x7fdb4419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb49600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb441079d0 0x7fdb4419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb49600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb44111de0 con 0x7fdb441019d0 2024-09-19T04:54:17.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb49600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb44111c60 con 0x7fdb441079d0 2024-09-19T04:54:17.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb49600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb44111f60 con 0x7fdb44102370 2024-09-19T04:54:17.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb43e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb441019d0 0x7fdb4419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb43e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb441019d0 0x7fdb4419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53472/0 (socket says 172.21.15.123:53472) 2024-09-19T04:54:17.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb43e00640 1 -- 172.21.15.123:0/3285170057 learned_addr learned my addr 172.21.15.123:0/3285170057 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:17.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb43400640 1 --2- 172.21.15.123:0/3285170057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb44102370 0x7fdb4419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb48c00640 1 --2- 172.21.15.123:0/3285170057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb441079d0 0x7fdb4419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb43e00640 1 -- 172.21.15.123:0/3285170057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb44102370 msgr2=0x7fdb4419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:17.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb43e00640 1 --2- 172.21.15.123:0/3285170057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb44102370 0x7fdb4419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb43e00640 1 -- 172.21.15.123:0/3285170057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb441079d0 msgr2=0x7fdb4419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:17.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb43e00640 1 --2- 172.21.15.123:0/3285170057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb441079d0 0x7fdb4419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb43e00640 1 -- 172.21.15.123:0/3285170057 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4419f460 con 0x7fdb441019d0 2024-09-19T04:54:17.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb43400640 1 --2- 172.21.15.123:0/3285170057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb44102370 0x7fdb4419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:17.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.443+0000 7fdb48c00640 1 --2- 172.21.15.123:0/3285170057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb441079d0 0x7fdb4419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:17.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.444+0000 7fdb43e00640 1 --2- 172.21.15.123:0/3285170057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb441019d0 0x7fdb4419a4b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdb3402a940 tx=0x7fdb34003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:17.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.444+0000 7fdb40c00640 1 -- 172.21.15.123:0/3285170057 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3403f070 con 0x7fdb441019d0 2024-09-19T04:54:17.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.444+0000 7fdb40c00640 1 -- 172.21.15.123:0/3285170057 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb34031c90 con 0x7fdb441019d0 2024-09-19T04:54:17.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.444+0000 7fdb49600640 1 -- 172.21.15.123:0/3285170057 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4419f6f0 con 0x7fdb441019d0 2024-09-19T04:54:17.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.444+0000 7fdb40c00640 1 -- 172.21.15.123:0/3285170057 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34035030 con 0x7fdb441019d0 2024-09-19T04:54:17.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.444+0000 7fdb49600640 1 -- 172.21.15.123:0/3285170057 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb441a7030 con 0x7fdb441019d0 2024-09-19T04:54:17.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.448+0000 7fdb4d5fd740 1 Processor -- start 2024-09-19T04:54:17.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.449+0000 7fdb40c00640 1 -- 172.21.15.123:0/3285170057 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fdb34042070 con 0x7fdb441019d0 2024-09-19T04:54:17.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.449+0000 7fdb4d5fd740 1 -- start start 2024-09-19T04:54:17.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.449+0000 7fdb4d5fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13294e40 0x55fd13295220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.449+0000 7fdb4d5fd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 0x55fd1329fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.449+0000 7fdb4d5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 0x55fd132a2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.449+0000 7fdb4d5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fd131fb3e0 con 0x55fd132a0020 2024-09-19T04:54:17.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.449+0000 7fdb4d5fd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fd131a2d80 con 0x55fd13295760 2024-09-19T04:54:17.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.449+0000 7fdb4d5fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fd131f73a0 con 0x55fd13294e40 2024-09-19T04:54:17.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.449+0000 7fdb43400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 0x55fd132a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.449+0000 7fdb43400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 0x55fd132a2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53488/0 (socket says 172.21.15.123:53488) 2024-09-19T04:54:17.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.449+0000 7fdb43400640 1 -- 172.21.15.123:0/3996864437 learned_addr learned my addr 172.21.15.123:0/3996864437 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:17.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.449+0000 7fdb48c00640 1 --2- 172.21.15.123:0/3996864437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13294e40 0x55fd13295220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.449+0000 7fdb43e00640 1 --2- 172.21.15.123:0/3996864437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 0x55fd1329fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.450+0000 7fdb48c00640 1 -- 172.21.15.123:0/3996864437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 msgr2=0x55fd1329fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:17.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.450+0000 7fdb48c00640 1 --2- 172.21.15.123:0/3996864437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 0x55fd1329fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.450+0000 7fdb48c00640 1 -- 172.21.15.123:0/3996864437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 msgr2=0x55fd132a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:17.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.450+0000 7fdb48c00640 1 --2- 172.21.15.123:0/3996864437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 0x55fd132a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.450+0000 7fdb48c00640 1 -- 172.21.15.123:0/3996864437 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd131c1ae0 con 0x55fd13294e40 2024-09-19T04:54:17.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.450+0000 7fdb43400640 1 --2- 172.21.15.123:0/3996864437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 0x55fd132a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:17.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.450+0000 7fdb48c00640 1 --2- 172.21.15.123:0/3996864437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13294e40 0x55fd13295220 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c010770 tx=0x7fdb3c010c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ac2084906562a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:17.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.450+0000 7fdb39200640 1 -- 172.21.15.123:0/3996864437 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c003970 con 0x55fd13294e40 2024-09-19T04:54:17.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.451+0000 7fdb39200640 1 -- 172.21.15.123:0/3996864437 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb3c00e930 con 0x55fd13294e40 2024-09-19T04:54:17.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.451+0000 7fdb39200640 1 -- 172.21.15.123:0/3996864437 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c004610 con 0x55fd13294e40 2024-09-19T04:54:17.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.451+0000 7fdb40c00640 1 --2- 172.21.15.123:0/3285170057 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb280a3430 0x7fdb280a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.451+0000 7fdb40c00640 1 -- 172.21.15.123:0/3285170057 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fdb340f03d0 con 0x7fdb441019d0 2024-09-19T04:54:17.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.451+0000 7fdb43e00640 1 --2- 172.21.15.123:0/3285170057 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb280a3430 0x7fdb280a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.451+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/3996864437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13294e40 msgr2=0x55fd13295220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:17.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.451+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/3996864437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13294e40 0x55fd13295220 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c010770 tx=0x7fdb3c010c30 comp rx=0 tx=0).stop 2024-09-19T04:54:17.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.451+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/3996864437 shutdown_connections 2024-09-19T04:54:17.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.451+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/3996864437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 0x55fd132a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.451+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/3996864437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 0x55fd1329fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.451+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/3996864437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13294e40 0x55fd13295220 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.451+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/3996864437 >> 172.21.15.123:0/3996864437 conn(0x55fd132913d0 msgr2=0x55fd132917b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:17.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.452+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/3996864437 shutdown_connections 2024-09-19T04:54:17.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.452+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/3996864437 wait complete. 2024-09-19T04:54:17.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.452+0000 7fdb43e00640 1 --2- 172.21.15.123:0/3285170057 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb280a3430 0x7fdb280a58f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55fd13292230 tx=0x7fdb340f2820 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:17.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.452+0000 7fdb4d5fd740 1 -- start start 2024-09-19T04:54:17.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.452+0000 7fdb4d5fd740 1 Processor -- start 2024-09-19T04:54:17.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.453+0000 7fdb4d5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd13294e40 0x55fd1329cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.453+0000 7fdb4d5fd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 0x55fd1329d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.453+0000 7fdb4d5fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd132a0020 0x55fd1329d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.453+0000 7fdb4d5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fd132a2bf0 con 0x55fd13294e40 2024-09-19T04:54:17.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.453+0000 7fdb4d5fd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fd131b60f0 con 0x55fd13295760 2024-09-19T04:54:17.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.453+0000 7fdb4d5fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fd132a2d70 con 0x55fd132a0020 2024-09-19T04:54:17.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.453+0000 7fdb4d5fd740 10 client.0 _ll_register_callbacks cb 0x55fd1319dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:17.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.453+0000 7fdb48c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd13294e40 0x55fd1329cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.453+0000 7fdb48c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd13294e40 0x55fd1329cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53494/0 (socket says 172.21.15.123:53494) 2024-09-19T04:54:17.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.453+0000 7fdb43400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 0x55fd1329d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.453+0000 7fdb48c00640 1 -- 172.21.15.123:0/859031747 learned_addr learned my addr 172.21.15.123:0/859031747 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:17.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.453+0000 7fdb48c00640 1 --2- 172.21.15.123:0/859031747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd132a0020 0x55fd1329d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.454+0000 7fdb43400640 1 -- 172.21.15.123:0/859031747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd132a0020 msgr2=0x55fd1329d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:17.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.454+0000 7fdb43400640 1 --2- 172.21.15.123:0/859031747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd132a0020 0x55fd1329d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.454+0000 7fdb43400640 1 -- 172.21.15.123:0/859031747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd13294e40 msgr2=0x55fd1329cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:17.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.454+0000 7fdb43400640 1 --2- 172.21.15.123:0/859031747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd13294e40 0x55fd1329cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.454+0000 7fdb43400640 1 -- 172.21.15.123:0/859031747 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd12f6c3c0 con 0x55fd13295760 2024-09-19T04:54:17.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.454+0000 7fdb43400640 1 --2- 172.21.15.123:0/859031747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 0x55fd1329d480 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdb30012d20 tx=0x7fdb3000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa55566a07e74d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:17.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.454+0000 7fdb2d600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:54:17.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.454+0000 7fdb2d600640 1 -- 172.21.15.123:0/859031747 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb300120d0 con 0x55fd13295760 2024-09-19T04:54:17.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.454+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/859031747 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb3c00e930 con 0x55fd13295760 2024-09-19T04:54:17.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.454+0000 7fdb2d600640 1 -- 172.21.15.123:0/859031747 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fdb300047a0 con 0x55fd13295760 2024-09-19T04:54:17.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.454+0000 7fdb2d600640 1 -- 172.21.15.123:0/859031747 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdb30012270 con 0x55fd13295760 2024-09-19T04:54:17.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.455+0000 7fdb4d5fd740 10 client.26780 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:17.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.455+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/859031747 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fd132b7520 con 0x55fd13295760 2024-09-19T04:54:17.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.455+0000 7fdb2d600640 1 -- 172.21.15.123:0/859031747 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb3c00e930 con 0x55fd13295760 2024-09-19T04:54:17.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.455+0000 7fdb2d600640 1 -- 172.21.15.123:0/859031747 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fdb30010430 con 0x55fd13295760 2024-09-19T04:54:17.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.455+0000 7fdb4d5fd740 10 client.26780 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:17.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.455+0000 7fdb4d5fd740 10 client.26780 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:54:17.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.455+0000 7fdb4d5fd740 20 client.26780 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:17.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.455+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/859031747 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55fd132b8930 0x55fd132bad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.455+0000 7fdb4d5fd740 4 client.26780 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:17.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.455+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/859031747 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd13208950 con 0x55fd132b8930 2024-09-19T04:54:17.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.456+0000 7fdb48c00640 1 --2- 172.21.15.123:0/859031747 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55fd132b8930 0x55fd132bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.456+0000 7fdb48c00640 1 --2- 172.21.15.123:0/859031747 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55fd132b8930 0x55fd132bad90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14555c310eda0fac server_cookie=993602d46d11f7a0 in_seq=0 out_seq=0 2024-09-19T04:54:17.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.456+0000 7fdb2d600640 10 client.26780 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:54:17.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.457+0000 7fdb2d600640 1 -- 172.21.15.123:0/859031747 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd13208950 con 0x55fd132b8930 2024-09-19T04:54:17.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.457+0000 7fdb2d600640 10 client.26780 handle_command_reply: tid=0 2024-09-19T04:54:17.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.457+0000 7fdb4d5fd740 1 client.26780 shutdown 2024-09-19T04:54:17.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.457+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/859031747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 msgr2=0x55fd1329d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:17.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.457+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/859031747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 0x55fd1329d480 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdb30012d20 tx=0x7fdb3000f790 comp rx=0 tx=0).stop 2024-09-19T04:54:17.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.458+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/859031747 shutdown_connections 2024-09-19T04:54:17.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.458+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/859031747 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55fd132b8930 0x55fd132bad90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.458+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/859031747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd132a0020 0x55fd1329d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.458+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/859031747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 0x55fd1329d480 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.458+0000 7fdb2d600640 0 client.26780 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:54:17.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.458+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/859031747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd13294e40 0x55fd1329cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.458+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/859031747 >> 172.21.15.123:0/859031747 conn(0x55fd132913d0 msgr2=0x55fd132970b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:17.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.458+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/859031747 shutdown_connections 2024-09-19T04:54:17.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.458+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/859031747 wait complete. 2024-09-19T04:54:17.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.458+0000 7fdb4d5fd740 20 client.26780 trim_cache size 0 max 16384 2024-09-19T04:54:17.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.472+0000 7fdb4d5fd740 1 Processor -- start 2024-09-19T04:54:17.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.472+0000 7fdb4d5fd740 1 -- start start 2024-09-19T04:54:17.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.472+0000 7fdb4d5fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13294e40 0x55fd132978a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.472+0000 7fdb4d5fd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 0x55fd1329eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.472+0000 7fdb4d5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 0x55fd132c06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.472+0000 7fdb4d5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb441ab4f0 con 0x55fd132a0020 2024-09-19T04:54:17.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.472+0000 7fdb4d5fd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb441ad0c0 con 0x55fd13295760 2024-09-19T04:54:17.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.472+0000 7fdb4d5fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb441abbf0 con 0x55fd13294e40 2024-09-19T04:54:17.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.473+0000 7fdb43400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 0x55fd1329eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.473+0000 7fdb43400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 0x55fd1329eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:56454/0 (socket says 172.21.15.123:56454) 2024-09-19T04:54:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.473+0000 7fdb48c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13294e40 0x55fd132978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.473+0000 7fdb43400640 1 -- 172.21.15.123:0/980297561 learned_addr learned my addr 172.21.15.123:0/980297561 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.473+0000 7fdb48c00640 1 --2- 172.21.15.123:0/980297561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 0x55fd132c06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.473+0000 7fdb43400640 1 -- 172.21.15.123:0/980297561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13294e40 msgr2=0x55fd132978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.473+0000 7fdb43400640 1 --2- 172.21.15.123:0/980297561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13294e40 0x55fd132978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.473+0000 7fdb43400640 1 -- 172.21.15.123:0/980297561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 msgr2=0x55fd132c06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.473+0000 7fdb43400640 1 --2- 172.21.15.123:0/980297561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 0x55fd132c06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.473+0000 7fdb43400640 1 -- 172.21.15.123:0/980297561 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd132bb410 con 0x55fd13295760 2024-09-19T04:54:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.473+0000 7fdb48c00640 1 --2- 172.21.15.123:0/980297561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 0x55fd132c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:54:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.473+0000 7fdb43400640 1 --2- 172.21.15.123:0/980297561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 0x55fd1329eeb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdb30000c00 tx=0x7fdb3000fc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bbb6fa04e4f7f1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.474+0000 7fdb2fe00640 1 -- 172.21.15.123:0/980297561 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30029070 con 0x55fd13295760 2024-09-19T04:54:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.474+0000 7fdb2fe00640 1 -- 172.21.15.123:0/980297561 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb30010b80 con 0x55fd13295760 2024-09-19T04:54:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.474+0000 7fdb2fe00640 1 -- 172.21.15.123:0/980297561 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30003850 con 0x55fd13295760 2024-09-19T04:54:17.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.474+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/980297561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 msgr2=0x55fd1329eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:17.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.474+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/980297561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 0x55fd1329eeb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdb30000c00 tx=0x7fdb3000fc90 comp rx=0 tx=0).stop 2024-09-19T04:54:17.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.475+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/980297561 shutdown_connections 2024-09-19T04:54:17.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.475+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/980297561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 0x55fd132c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.475+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/980297561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13295760 0x55fd1329eeb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.475+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/980297561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13294e40 0x55fd132978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.475+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/980297561 >> 172.21.15.123:0/980297561 conn(0x55fd132913d0 msgr2=0x55fd1329e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:17.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.475+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/980297561 shutdown_connections 2024-09-19T04:54:17.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.475+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/980297561 wait complete. 2024-09-19T04:54:17.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.475+0000 7fdb4d5fd740 1 -- start start 2024-09-19T04:54:17.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.475+0000 7fdb4d5fd740 1 Processor -- start 2024-09-19T04:54:17.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.476+0000 7fdb4d5fd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13294e40 0x55fd132ac990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.476+0000 7fdb4d5fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13295760 0x55fd132aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.476+0000 7fdb4d5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 0x55fd13322670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.476+0000 7fdb4d5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb441ac9f0 con 0x55fd132a0020 2024-09-19T04:54:17.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.476+0000 7fdb4d5fd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb441ac2f0 con 0x55fd13294e40 2024-09-19T04:54:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.476+0000 7fdb4d5fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb441a9530 con 0x55fd13295760 2024-09-19T04:54:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.476+0000 7fdb4d5fd740 10 client.0 _ll_register_callbacks cb 0x7fdb30009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.476+0000 7fdb43400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13295760 0x55fd132aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.476+0000 7fdb48c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13294e40 0x55fd132ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.476+0000 7fdb48c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 0x55fd13322670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.476+0000 7fdb43400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13295760 0x55fd132aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:60188/0 (socket says 172.21.15.123:60188) 2024-09-19T04:54:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.476+0000 7fdb43400640 1 -- 172.21.15.123:0/2899796487 learned_addr learned my addr 172.21.15.123:0/2899796487 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.476+0000 7fdb43400640 1 -- 172.21.15.123:0/2899796487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13294e40 msgr2=0x55fd132ac990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.476+0000 7fdb43400640 1 --2- 172.21.15.123:0/2899796487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13294e40 0x55fd132ac990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.476+0000 7fdb43400640 1 -- 172.21.15.123:0/2899796487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 msgr2=0x55fd13322670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.476+0000 7fdb43400640 1 --2- 172.21.15.123:0/2899796487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 0x55fd13322670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.476+0000 7fdb43400640 1 -- 172.21.15.123:0/2899796487 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd13322bb0 con 0x55fd13295760 2024-09-19T04:54:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.477+0000 7fdb43400640 1 --2- 172.21.15.123:0/2899796487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13295760 0x55fd132aced0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdb30010970 tx=0x7fdb3001d6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25edeec5b7f6bfcb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.477+0000 7fdb2d600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:54:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.477+0000 7fdb2d600640 1 -- 172.21.15.123:0/2899796487 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30032030 con 0x55fd13295760 2024-09-19T04:54:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.477+0000 7fdb2d600640 1 -- 172.21.15.123:0/2899796487 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fdb30018070 con 0x55fd13295760 2024-09-19T04:54:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.477+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/2899796487 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb30010b80 con 0x55fd13295760 2024-09-19T04:54:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.477+0000 7fdb2d600640 1 -- 172.21.15.123:0/2899796487 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdb30029040 con 0x55fd13295760 2024-09-19T04:54:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.477+0000 7fdb4d5fd740 10 client.26809 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.477+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/2899796487 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fd13323d40 con 0x55fd13295760 2024-09-19T04:54:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.478+0000 7fdb2d600640 1 -- 172.21.15.123:0/2899796487 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb30010b80 con 0x55fd13295760 2024-09-19T04:54:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.478+0000 7fdb2d600640 1 -- 172.21.15.123:0/2899796487 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fdb30003680 con 0x55fd13295760 2024-09-19T04:54:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.478+0000 7fdb4d5fd740 10 client.26809 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.478+0000 7fdb4d5fd740 10 client.26809 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:54:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.478+0000 7fdb4d5fd740 20 client.26809 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.478+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/2899796487 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55fd133243d0 0x55fd133267f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.478+0000 7fdb4d5fd740 4 client.26809 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:17.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.478+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/2899796487 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd13202e60 con 0x55fd133243d0 2024-09-19T04:54:17.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.478+0000 7fdb48c00640 1 --2- 172.21.15.123:0/2899796487 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55fd133243d0 0x55fd133267f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.479+0000 7fdb48c00640 1 --2- 172.21.15.123:0/2899796487 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55fd133243d0 0x55fd133267f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f77cf6387c21e5d4 server_cookie=e4dde550347ff605 in_seq=0 out_seq=0 2024-09-19T04:54:17.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.479+0000 7fdb2d600640 10 client.26809 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:54:17.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.479+0000 7fdb2d600640 1 -- 172.21.15.123:0/2899796487 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd13202e60 con 0x55fd133243d0 2024-09-19T04:54:17.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.479+0000 7fdb2d600640 10 client.26809 handle_command_reply: tid=0 2024-09-19T04:54:17.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.479+0000 7fdb4d5fd740 1 client.26809 shutdown 2024-09-19T04:54:17.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.479+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/2899796487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13295760 msgr2=0x55fd132aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:17.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.479+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/2899796487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13295760 0x55fd132aced0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdb30010970 tx=0x7fdb3001d6b0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.479+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/2899796487 shutdown_connections 2024-09-19T04:54:17.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.479+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/2899796487 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55fd133243d0 0x55fd133267f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.479+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/2899796487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fd132a0020 0x55fd13322670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.479+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/2899796487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd13295760 0x55fd132aced0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.479+0000 7fdb4d5fd740 1 --2- 172.21.15.123:0/2899796487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd13294e40 0x55fd132ac990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.479+0000 7fdb2d600640 0 client.26809 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:54:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.479+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/2899796487 >> 172.21.15.123:0/2899796487 conn(0x55fd132913d0 msgr2=0x55fd1329efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.480+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/2899796487 shutdown_connections 2024-09-19T04:54:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.480+0000 7fdb4d5fd740 1 -- 172.21.15.123:0/2899796487 wait complete. 2024-09-19T04:54:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.480+0000 7fdb4d5fd740 20 client.26809 trim_cache size 0 max 16384 2024-09-19T04:54:17.485 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:17.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.480+0000 7fdb49600640 1 -- 172.21.15.123:0/3285170057 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb280a3430 msgr2=0x7fdb280a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:17.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.480+0000 7fdb49600640 1 --2- 172.21.15.123:0/3285170057 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb280a3430 0x7fdb280a58f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55fd13292230 tx=0x7fdb340f2820 comp rx=0 tx=0).stop 2024-09-19T04:54:17.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.480+0000 7fdb49600640 1 -- 172.21.15.123:0/3285170057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb441019d0 msgr2=0x7fdb4419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:17.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.480+0000 7fdb49600640 1 --2- 172.21.15.123:0/3285170057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb441019d0 0x7fdb4419a4b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdb3402a940 tx=0x7fdb34003b90 comp rx=0 tx=0).stop 2024-09-19T04:54:17.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.480+0000 7fdb49600640 1 -- 172.21.15.123:0/3285170057 shutdown_connections 2024-09-19T04:54:17.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.480+0000 7fdb49600640 1 --2- 172.21.15.123:0/3285170057 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fdb280a3430 0x7fdb280a58f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.480+0000 7fdb49600640 1 --2- 172.21.15.123:0/3285170057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb441079d0 0x7fdb4419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.480+0000 7fdb49600640 1 --2- 172.21.15.123:0/3285170057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb44102370 0x7fdb4419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.480+0000 7fdb49600640 1 --2- 172.21.15.123:0/3285170057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb441019d0 0x7fdb4419a4b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.480+0000 7fdb49600640 1 -- 172.21.15.123:0/3285170057 >> 172.21.15.123:0/3285170057 conn(0x7fdb440fd710 msgr2=0x7fdb4410d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:17.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.481+0000 7fdb49600640 1 -- 172.21.15.123:0/3285170057 shutdown_connections 2024-09-19T04:54:17.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:17.481+0000 7fdb49600640 1 -- 172.21.15.123:0/3285170057 wait complete. 2024-09-19T04:54:17.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:17 smithi123 ceph-mon[30511]: pgmap v233: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:54:17.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:17 smithi123 ceph-mon[30511]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T04:54:17.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:17 smithi193 ceph-mon[35592]: pgmap v233: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:54:17.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:17 smithi193 ceph-mon[35592]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T04:54:17.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:17 smithi183 ceph-mon[35333]: pgmap v233: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:54:17.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:17 smithi183 ceph-mon[35333]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T04:54:18.210 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:54:18.579 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:19.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.153+0000 7f0d1c200640 1 -- 172.21.15.123:0/3006918094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d14102790 msgr2=0x7f0d141067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:19.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.153+0000 7f0d1c200640 1 --2- 172.21.15.123:0/3006918094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d14102790 0x7f0d141067e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009900 tx=0x7f0d0002a410 comp rx=0 tx=0).stop 2024-09-19T04:54:19.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.154+0000 7f0d1c200640 1 -- 172.21.15.123:0/3006918094 shutdown_connections 2024-09-19T04:54:19.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.154+0000 7f0d1c200640 1 --2- 172.21.15.123:0/3006918094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d14106f10 0x7f0d1410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.154+0000 7f0d1c200640 1 --2- 172.21.15.123:0/3006918094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d14102790 0x7f0d141067e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.154+0000 7f0d1c200640 1 --2- 172.21.15.123:0/3006918094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d14101de0 0x7f0d141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.154+0000 7f0d1c200640 1 -- 172.21.15.123:0/3006918094 >> 172.21.15.123:0/3006918094 conn(0x7f0d140fd8c0 msgr2=0x7f0d140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:19.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.154+0000 7f0d1c200640 1 -- 172.21.15.123:0/3006918094 shutdown_connections 2024-09-19T04:54:19.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.154+0000 7f0d1c200640 1 -- 172.21.15.123:0/3006918094 wait complete. 2024-09-19T04:54:19.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.155+0000 7f0d1c200640 1 Processor -- start 2024-09-19T04:54:19.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.155+0000 7f0d1c200640 1 -- start start 2024-09-19T04:54:19.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.155+0000 7f0d1c200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d14101de0 0x7f0d14198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.155+0000 7f0d1c200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d14102790 0x7f0d14198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.155+0000 7f0d1c200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d14106f10 0x7f0d1419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.155+0000 7f0d1c200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d14111f70 con 0x7f0d14102790 2024-09-19T04:54:19.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.155+0000 7f0d1c200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d14111df0 con 0x7f0d14101de0 2024-09-19T04:54:19.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.155+0000 7f0d1c200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141120f0 con 0x7f0d14106f10 2024-09-19T04:54:19.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.156+0000 7f0d18e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d14102790 0x7f0d14198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.156+0000 7f0d19800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d14101de0 0x7f0d14198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.156+0000 7f0d18e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d14102790 0x7f0d14198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53532/0 (socket says 172.21.15.123:53532) 2024-09-19T04:54:19.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.156+0000 7f0d19800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d14101de0 0x7f0d14198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:56492/0 (socket says 172.21.15.123:56492) 2024-09-19T04:54:19.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.156+0000 7f0d18e00640 1 -- 172.21.15.123:0/3548653802 learned_addr learned my addr 172.21.15.123:0/3548653802 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:19.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.156+0000 7f0d1a200640 1 --2- 172.21.15.123:0/3548653802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d14106f10 0x7f0d1419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.156+0000 7f0d19800640 1 -- 172.21.15.123:0/3548653802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d14106f10 msgr2=0x7f0d1419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:19.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.156+0000 7f0d19800640 1 --2- 172.21.15.123:0/3548653802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d14106f10 0x7f0d1419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.156+0000 7f0d19800640 1 -- 172.21.15.123:0/3548653802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d14102790 msgr2=0x7f0d14198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:19.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.156+0000 7f0d19800640 1 --2- 172.21.15.123:0/3548653802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d14102790 0x7f0d14198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.156+0000 7f0d19800640 1 -- 172.21.15.123:0/3548653802 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1419d400 con 0x7f0d14101de0 2024-09-19T04:54:19.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.156+0000 7f0d1a200640 1 --2- 172.21.15.123:0/3548653802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d14106f10 0x7f0d1419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:54:19.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.156+0000 7f0d19800640 1 --2- 172.21.15.123:0/3548653802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d14101de0 0x7f0d14198450 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800e9f0 tx=0x7f0d0800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:19.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.157+0000 7f0d12000640 1 -- 172.21.15.123:0/3548653802 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0800ccc0 con 0x7f0d14101de0 2024-09-19T04:54:19.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.157+0000 7f0d1c200640 1 -- 172.21.15.123:0/3548653802 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1419d6f0 con 0x7f0d14101de0 2024-09-19T04:54:19.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.157+0000 7f0d12000640 1 -- 172.21.15.123:0/3548653802 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d08010910 con 0x7f0d14101de0 2024-09-19T04:54:19.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.157+0000 7f0d1c200640 1 -- 172.21.15.123:0/3548653802 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d141a4fd0 con 0x7f0d14101de0 2024-09-19T04:54:19.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.157+0000 7f0d12000640 1 -- 172.21.15.123:0/3548653802 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d08018880 con 0x7f0d14101de0 2024-09-19T04:54:19.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.160+0000 7f0d1ee7b740 1 Processor -- start 2024-09-19T04:54:19.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.160+0000 7f0d1ee7b740 1 -- start start 2024-09-19T04:54:19.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.160+0000 7f0d12000640 1 -- 172.21.15.123:0/3548653802 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f0d08010ac0 con 0x7f0d14101de0 2024-09-19T04:54:19.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.160+0000 7f0d1ee7b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ec5c70 0x561d03ec6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.160+0000 7f0d1ee7b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec6590 0x561d03ed0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.160+0000 7f0d1ee7b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ed0e50 0x561d03ed3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.160+0000 7f0d1ee7b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561d03e2c3e0 con 0x561d03ec5c70 2024-09-19T04:54:19.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.160+0000 7f0d1ee7b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561d03dd3d80 con 0x561d03ec6590 2024-09-19T04:54:19.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.160+0000 7f0d1ee7b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561d03e283a0 con 0x561d03ed0e50 2024-09-19T04:54:19.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.160+0000 7f0d1a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ec5c70 0x561d03ec6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.160+0000 7f0d19800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec6590 0x561d03ed0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.160+0000 7f0d1a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ec5c70 0x561d03ec6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53540/0 (socket says 172.21.15.123:53540) 2024-09-19T04:54:19.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.161+0000 7f0d1a200640 1 -- 172.21.15.123:0/1425100691 learned_addr learned my addr 172.21.15.123:0/1425100691 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:19.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.161+0000 7f0d18e00640 1 --2- 172.21.15.123:0/1425100691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ed0e50 0x561d03ed3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.161+0000 7f0d18e00640 1 -- 172.21.15.123:0/1425100691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec6590 msgr2=0x561d03ed0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:19.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.161+0000 7f0d18e00640 1 --2- 172.21.15.123:0/1425100691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec6590 0x561d03ed0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.161+0000 7f0d18e00640 1 -- 172.21.15.123:0/1425100691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ec5c70 msgr2=0x561d03ec6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:19.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.161+0000 7f0d18e00640 1 --2- 172.21.15.123:0/1425100691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ec5c70 0x561d03ec6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.161+0000 7f0d18e00640 1 -- 172.21.15.123:0/1425100691 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d03df2ae0 con 0x561d03ed0e50 2024-09-19T04:54:19.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.161+0000 7f0d18e00640 1 --2- 172.21.15.123:0/1425100691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ed0e50 0x561d03ed3240 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009230 tx=0x7f0d00005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5094479d2b8363a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:19.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.162+0000 7f0d06a00640 1 -- 172.21.15.123:0/1425100691 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00040070 con 0x561d03ed0e50 2024-09-19T04:54:19.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.162+0000 7f0d06a00640 1 -- 172.21.15.123:0/1425100691 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d00004030 con 0x561d03ed0e50 2024-09-19T04:54:19.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.162+0000 7f0d06a00640 1 -- 172.21.15.123:0/1425100691 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00035040 con 0x561d03ed0e50 2024-09-19T04:54:19.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.162+0000 7f0d12000640 1 --2- 172.21.15.123:0/3548653802 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0cfc0a3430 0x7f0cfc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.162+0000 7f0d12000640 1 -- 172.21.15.123:0/3548653802 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f0d080d0120 con 0x7f0d14101de0 2024-09-19T04:54:19.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.162+0000 7f0d1a200640 1 --2- 172.21.15.123:0/3548653802 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0cfc0a3430 0x7f0cfc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.162+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/1425100691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ed0e50 msgr2=0x561d03ed3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:19.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.162+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/1425100691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ed0e50 0x561d03ed3240 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009230 tx=0x7f0d00005dd0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.162+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/1425100691 shutdown_connections 2024-09-19T04:54:19.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.162+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/1425100691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ed0e50 0x561d03ed3240 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.162+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/1425100691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec6590 0x561d03ed0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.162+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/1425100691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ec5c70 0x561d03ec6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.162+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/1425100691 >> 172.21.15.123:0/1425100691 conn(0x561d03ec2200 msgr2=0x561d03ec25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:19.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.163+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/1425100691 shutdown_connections 2024-09-19T04:54:19.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.163+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/1425100691 wait complete. 2024-09-19T04:54:19.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.163+0000 7f0d1a200640 1 --2- 172.21.15.123:0/3548653802 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0cfc0a3430 0x7f0cfc0a58f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009fd0 tx=0x7f0d0c009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:19.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.163+0000 7f0d1ee7b740 1 -- start start 2024-09-19T04:54:19.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.163+0000 7f0d1ee7b740 1 Processor -- start 2024-09-19T04:54:19.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.164+0000 7f0d1ee7b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec5c70 0x561d03ed52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.164+0000 7f0d1ee7b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec6590 0x561d03ed57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.164+0000 7f0d1ee7b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 0x561d03ed9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.164+0000 7f0d1ee7b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561d03ed3a20 con 0x561d03ed0e50 2024-09-19T04:54:19.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.164+0000 7f0d1ee7b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561d03de70f0 con 0x561d03ec5c70 2024-09-19T04:54:19.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.164+0000 7f0d1ee7b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561d03ed3ba0 con 0x561d03ec6590 2024-09-19T04:54:19.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.164+0000 7f0d1ee7b740 10 client.0 _ll_register_callbacks cb 0x561d03dcebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:19.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.164+0000 7f0d1a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 0x561d03ed9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.164+0000 7f0d18e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec6590 0x561d03ed57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.164+0000 7f0d19800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec5c70 0x561d03ed52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.164+0000 7f0d18e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec6590 0x561d03ed57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:60202/0 (socket says 172.21.15.123:60202) 2024-09-19T04:54:19.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.164+0000 7f0d18e00640 1 -- 172.21.15.123:0/2689227902 learned_addr learned my addr 172.21.15.123:0/2689227902 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:19.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.165+0000 7f0d1a200640 1 -- 172.21.15.123:0/2689227902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec6590 msgr2=0x561d03ed57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:19.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.165+0000 7f0d1a200640 1 --2- 172.21.15.123:0/2689227902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec6590 0x561d03ed57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.165+0000 7f0d1a200640 1 -- 172.21.15.123:0/2689227902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec5c70 msgr2=0x561d03ed52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:19.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.165+0000 7f0d1a200640 1 --2- 172.21.15.123:0/2689227902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec5c70 0x561d03ed52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.165+0000 7f0d1a200640 1 -- 172.21.15.123:0/2689227902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d03b9d3c0 con 0x561d03ed0e50 2024-09-19T04:54:19.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.165+0000 7f0d18e00640 1 --2- 172.21.15.123:0/2689227902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec6590 0x561d03ed57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:19.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.165+0000 7f0d1a200640 1 --2- 172.21.15.123:0/2689227902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 0x561d03ed9950 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c005c40 tx=0x7f0d0c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449dde0e1e44a707 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:19.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.165+0000 7f0cfb400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:19.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.165+0000 7f0cfb400640 1 -- 172.21.15.123:0/2689227902 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c02c060 con 0x561d03ed0e50 2024-09-19T04:54:19.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.165+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2689227902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d00004030 con 0x561d03ed0e50 2024-09-19T04:54:19.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.165+0000 7f0cfb400640 1 -- 172.21.15.123:0/2689227902 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f0d0c017030 con 0x561d03ed0e50 2024-09-19T04:54:19.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.166+0000 7f0cfb400640 1 -- 172.21.15.123:0/2689227902 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0d0c01fd40 con 0x561d03ed0e50 2024-09-19T04:54:19.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.166+0000 7f0d1ee7b740 10 client.16701 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:19.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.166+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2689227902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561d03ee3e50 con 0x561d03ed0e50 2024-09-19T04:54:19.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.166+0000 7f0cfb400640 1 -- 172.21.15.123:0/2689227902 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d00004030 con 0x561d03ed0e50 2024-09-19T04:54:19.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.166+0000 7f0cfb400640 1 -- 172.21.15.123:0/2689227902 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f0d0c01c3f0 con 0x561d03ed0e50 2024-09-19T04:54:19.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.166+0000 7f0d1ee7b740 10 client.16701 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:19.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.166+0000 7f0d1ee7b740 10 client.16701 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:54:19.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.166+0000 7f0d1ee7b740 20 client.16701 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:19.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.166+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/2689227902 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x561d03ee5310 0x561d03ee7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.166+0000 7f0d1ee7b740 4 client.16701 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:19.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.166+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2689227902 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d03e39950 con 0x561d03ee5310 2024-09-19T04:54:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.167+0000 7f0d19800640 1 --2- 172.21.15.123:0/2689227902 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x561d03ee5310 0x561d03ee7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.167+0000 7f0d19800640 1 --2- 172.21.15.123:0/2689227902 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x561d03ee5310 0x561d03ee7790 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ded4f415627903c6 server_cookie=794300add6b900ed in_seq=0 out_seq=0 2024-09-19T04:54:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.167+0000 7f0cfb400640 10 client.16701 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:54:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.168+0000 7f0cfb400640 1 -- 172.21.15.123:0/2689227902 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d03e39950 con 0x561d03ee5310 2024-09-19T04:54:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.168+0000 7f0cfb400640 10 client.16701 handle_command_reply: tid=0 2024-09-19T04:54:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.168+0000 7f0d1ee7b740 1 client.16701 shutdown 2024-09-19T04:54:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.169+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2689227902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 msgr2=0x561d03ed9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.169+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/2689227902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 0x561d03ed9950 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c005c40 tx=0x7f0d0c004ab0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.169+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2689227902 shutdown_connections 2024-09-19T04:54:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.169+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/2689227902 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x561d03ee5310 0x561d03ee7790 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.169+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/2689227902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 0x561d03ed9950 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.169+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/2689227902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec6590 0x561d03ed57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.169+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/2689227902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec5c70 0x561d03ed52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.169+0000 7f0cfb400640 0 client.16701 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:54:19.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.169+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2689227902 >> 172.21.15.123:0/2689227902 conn(0x561d03ec2200 msgr2=0x561d03ec7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:19.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.169+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2689227902 shutdown_connections 2024-09-19T04:54:19.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.169+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2689227902 wait complete. 2024-09-19T04:54:19.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.169+0000 7f0d1ee7b740 20 client.16701 trim_cache size 0 max 16384 2024-09-19T04:54:19.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.182+0000 7f0d1ee7b740 1 Processor -- start 2024-09-19T04:54:19.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.182+0000 7f0d1ee7b740 1 -- start start 2024-09-19T04:54:19.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.183+0000 7f0d1ee7b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec5c70 0x561d03ed9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.183+0000 7f0d1ee7b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec6590 0x561d03ec25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.183+0000 7f0d1ee7b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 0x561d03ec2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.183+0000 7f0d1ee7b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141ab650 con 0x561d03ed0e50 2024-09-19T04:54:19.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.183+0000 7f0d1ee7b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141ad220 con 0x561d03ec5c70 2024-09-19T04:54:19.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.183+0000 7f0d1ee7b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141abd50 con 0x561d03ec6590 2024-09-19T04:54:19.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.183+0000 7f0d1a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 0x561d03ec2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.183+0000 7f0d18e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec6590 0x561d03ec25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.183+0000 7f0d1a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 0x561d03ec2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53562/0 (socket says 172.21.15.123:53562) 2024-09-19T04:54:19.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.183+0000 7f0d1a200640 1 -- 172.21.15.123:0/2143923800 learned_addr learned my addr 172.21.15.123:0/2143923800 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:19.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.183+0000 7f0d19800640 1 --2- 172.21.15.123:0/2143923800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec5c70 0x561d03ed9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.183+0000 7f0d18e00640 1 -- 172.21.15.123:0/2143923800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec5c70 msgr2=0x561d03ed9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:19.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.183+0000 7f0d18e00640 1 --2- 172.21.15.123:0/2143923800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec5c70 0x561d03ed9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.183+0000 7f0d18e00640 1 -- 172.21.15.123:0/2143923800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 msgr2=0x561d03ec2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:19.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.183+0000 7f0d18e00640 1 --2- 172.21.15.123:0/2143923800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 0x561d03ec2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.184+0000 7f0d18e00640 1 -- 172.21.15.123:0/2143923800 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d03ec2030 con 0x561d03ec6590 2024-09-19T04:54:19.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.184+0000 7f0d1a200640 1 --2- 172.21.15.123:0/2143923800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 0x561d03ec2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.184+0000 7f0d18e00640 1 --2- 172.21.15.123:0/2143923800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec6590 0x561d03ec25e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0d0002a450 tx=0x7f0d0002ab10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac9ea6f7954fa84c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.184+0000 7f0d06000640 1 -- 172.21.15.123:0/2143923800 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00052070 con 0x561d03ec6590 2024-09-19T04:54:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.184+0000 7f0d06000640 1 -- 172.21.15.123:0/2143923800 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d000054e0 con 0x561d03ec6590 2024-09-19T04:54:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.184+0000 7f0d06000640 1 -- 172.21.15.123:0/2143923800 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00040050 con 0x561d03ec6590 2024-09-19T04:54:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.185+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2143923800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec6590 msgr2=0x561d03ec25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.185+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/2143923800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec6590 0x561d03ec25e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0d0002a450 tx=0x7f0d0002ab10 comp rx=0 tx=0).stop 2024-09-19T04:54:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.185+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2143923800 shutdown_connections 2024-09-19T04:54:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.185+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/2143923800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 0x561d03ec2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.185+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/2143923800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec6590 0x561d03ec25e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.185+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/2143923800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec5c70 0x561d03ed9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.185+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2143923800 >> 172.21.15.123:0/2143923800 conn(0x561d03ec2200 msgr2=0x561d03ed3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.185+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2143923800 shutdown_connections 2024-09-19T04:54:19.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.185+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2143923800 wait complete. 2024-09-19T04:54:19.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.185+0000 7f0d1ee7b740 1 -- start start 2024-09-19T04:54:19.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.186+0000 7f0d1ee7b740 1 Processor -- start 2024-09-19T04:54:19.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.186+0000 7f0d1ee7b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec5c70 0x561d03eca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.186+0000 7f0d1ee7b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec6590 0x561d03ecad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.186+0000 7f0d1ee7b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 0x561d03ecb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.186+0000 7f0d1ee7b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141acb50 con 0x561d03ed0e50 2024-09-19T04:54:19.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.186+0000 7f0d1ee7b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141ac450 con 0x561d03ec6590 2024-09-19T04:54:19.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.186+0000 7f0d1ee7b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141a9650 con 0x561d03ec5c70 2024-09-19T04:54:19.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.186+0000 7f0d1ee7b740 10 client.0 _ll_register_callbacks cb 0x7f0d0c009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:19.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.186+0000 7f0d18e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec6590 0x561d03ecad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.186+0000 7f0d1a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 0x561d03ecb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.186+0000 7f0d18e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec6590 0x561d03ecad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:56526/0 (socket says 172.21.15.123:56526) 2024-09-19T04:54:19.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.186+0000 7f0d1a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 0x561d03ecb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53572/0 (socket says 172.21.15.123:53572) 2024-09-19T04:54:19.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.186+0000 7f0d18e00640 1 -- 172.21.15.123:0/2896829681 learned_addr learned my addr 172.21.15.123:0/2896829681 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:19.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.186+0000 7f0d19800640 1 --2- 172.21.15.123:0/2896829681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec5c70 0x561d03eca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.187+0000 7f0d1a200640 1 -- 172.21.15.123:0/2896829681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec5c70 msgr2=0x561d03eca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:19.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.187+0000 7f0d1a200640 1 --2- 172.21.15.123:0/2896829681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec5c70 0x561d03eca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.187+0000 7f0d1a200640 1 -- 172.21.15.123:0/2896829681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec6590 msgr2=0x561d03ecad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:19.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.187+0000 7f0d1a200640 1 --2- 172.21.15.123:0/2896829681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec6590 0x561d03ecad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.187+0000 7f0d1a200640 1 -- 172.21.15.123:0/2896829681 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d03eccef0 con 0x561d03ed0e50 2024-09-19T04:54:19.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.187+0000 7f0d19800640 1 --2- 172.21.15.123:0/2896829681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec5c70 0x561d03eca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:19.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.187+0000 7f0d18e00640 1 --2- 172.21.15.123:0/2896829681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec6590 0x561d03ecad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:19.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.187+0000 7f0d1a200640 1 --2- 172.21.15.123:0/2896829681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 0x561d03ecb2b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c000f80 tx=0x7f0d0c01c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e27c69f7238600 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:19.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.187+0000 7f0cfb400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:19.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.187+0000 7f0cfb400640 1 -- 172.21.15.123:0/2896829681 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c03e060 con 0x561d03ed0e50 2024-09-19T04:54:19.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.187+0000 7f0cfb400640 1 -- 172.21.15.123:0/2896829681 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f0d0c028860 con 0x561d03ed0e50 2024-09-19T04:54:19.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.187+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2896829681 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d000054e0 con 0x561d03ed0e50 2024-09-19T04:54:19.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.188+0000 7f0cfb400640 1 -- 172.21.15.123:0/2896829681 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0d0c01f800 con 0x561d03ed0e50 2024-09-19T04:54:19.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.188+0000 7f0d1ee7b740 10 client.16707 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:19.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.188+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2896829681 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561d03ed5b50 con 0x561d03ed0e50 2024-09-19T04:54:19.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.188+0000 7f0cfb400640 1 -- 172.21.15.123:0/2896829681 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d000054e0 con 0x561d03ed0e50 2024-09-19T04:54:19.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.188+0000 7f0cfb400640 1 -- 172.21.15.123:0/2896829681 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f0d0c03a980 con 0x561d03ed0e50 2024-09-19T04:54:19.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.188+0000 7f0d1ee7b740 10 client.16707 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:19.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.188+0000 7f0d1ee7b740 10 client.16707 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:54:19.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.188+0000 7f0d1ee7b740 20 client.16707 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:19.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.188+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/2896829681 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x561d03f54f40 0x561d03f57360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:19.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.188+0000 7f0d1ee7b740 4 client.16707 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:19.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.188+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2896829681 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d03e33e60 con 0x561d03f54f40 2024-09-19T04:54:19.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.189+0000 7f0d19800640 1 --2- 172.21.15.123:0/2896829681 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x561d03f54f40 0x561d03f57360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:19.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.189+0000 7f0d19800640 1 --2- 172.21.15.123:0/2896829681 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x561d03f54f40 0x561d03f57360 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de12aef9c0f3f68c server_cookie=180dedab94a5d066 in_seq=0 out_seq=0 2024-09-19T04:54:19.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.189+0000 7f0cfb400640 10 client.16707 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:54:19.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.189+0000 7f0cfb400640 1 -- 172.21.15.123:0/2896829681 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d03e33e60 con 0x561d03f54f40 2024-09-19T04:54:19.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.189+0000 7f0cfb400640 10 client.16707 handle_command_reply: tid=0 2024-09-19T04:54:19.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.189+0000 7f0d1ee7b740 1 client.16707 shutdown 2024-09-19T04:54:19.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.189+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2896829681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 msgr2=0x561d03ecb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:19.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.189+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/2896829681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 0x561d03ecb2b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c000f80 tx=0x7f0d0c01c6a0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.190+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2896829681 shutdown_connections 2024-09-19T04:54:19.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.190+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/2896829681 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x561d03f54f40 0x561d03f57360 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.190+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/2896829681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d03ed0e50 0x561d03ecb2b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.190+0000 7f0cfb400640 0 client.16707 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:54:19.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.190+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/2896829681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d03ec6590 0x561d03ecad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.190+0000 7f0d1ee7b740 1 --2- 172.21.15.123:0/2896829681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d03ec5c70 0x561d03eca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.190+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2896829681 >> 172.21.15.123:0/2896829681 conn(0x561d03ec2200 msgr2=0x561d03ecd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:19.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.190+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2896829681 shutdown_connections 2024-09-19T04:54:19.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.190+0000 7f0d1ee7b740 1 -- 172.21.15.123:0/2896829681 wait complete. 2024-09-19T04:54:19.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.190+0000 7f0d1ee7b740 20 client.16707 trim_cache size 0 max 16384 2024-09-19T04:54:19.196 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:19.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.191+0000 7f0d1c200640 1 -- 172.21.15.123:0/3548653802 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0cfc0a3430 msgr2=0x7f0cfc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.191+0000 7f0d1c200640 1 --2- 172.21.15.123:0/3548653802 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0cfc0a3430 0x7f0cfc0a58f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009fd0 tx=0x7f0d0c009500 comp rx=0 tx=0).stop 2024-09-19T04:54:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.191+0000 7f0d1c200640 1 -- 172.21.15.123:0/3548653802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d14101de0 msgr2=0x7f0d14198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.191+0000 7f0d1c200640 1 --2- 172.21.15.123:0/3548653802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d14101de0 0x7f0d14198450 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800e9f0 tx=0x7f0d0800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.191+0000 7f0d1c200640 1 -- 172.21.15.123:0/3548653802 shutdown_connections 2024-09-19T04:54:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.191+0000 7f0d1c200640 1 --2- 172.21.15.123:0/3548653802 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f0cfc0a3430 0x7f0cfc0a58f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.191+0000 7f0d1c200640 1 --2- 172.21.15.123:0/3548653802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d14106f10 0x7f0d1419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.191+0000 7f0d1c200640 1 --2- 172.21.15.123:0/3548653802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d14102790 0x7f0d14198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.191+0000 7f0d1c200640 1 --2- 172.21.15.123:0/3548653802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d14101de0 0x7f0d14198450 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.191+0000 7f0d1c200640 1 -- 172.21.15.123:0/3548653802 >> 172.21.15.123:0/3548653802 conn(0x7f0d140fd8c0 msgr2=0x7f0d140fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.191+0000 7f0d1c200640 1 -- 172.21.15.123:0/3548653802 shutdown_connections 2024-09-19T04:54:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:19.191+0000 7f0d1c200640 1 -- 172.21.15.123:0/3548653802 wait complete. 2024-09-19T04:54:19.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:19 smithi193 ceph-mon[35592]: pgmap v235: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 241 B/s wr, 12 op/s 2024-09-19T04:54:19.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:19 smithi183 ceph-mon[35333]: pgmap v235: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 241 B/s wr, 12 op/s 2024-09-19T04:54:19.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:19 smithi123 ceph-mon[30511]: pgmap v235: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 241 B/s wr, 12 op/s 2024-09-19T04:54:19.973 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:54:20.340 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:20.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.923+0000 7f311be00640 1 -- 172.21.15.123:0/2431298127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f311c1079c0 msgr2=0x7f311c107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:20.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.923+0000 7f311be00640 1 --2- 172.21.15.123:0/2431298127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f311c1079c0 0x7f311c107da0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f310c009930 tx=0x7f310c02a420 comp rx=0 tx=0).stop 2024-09-19T04:54:20.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.923+0000 7f311be00640 1 -- 172.21.15.123:0/2431298127 shutdown_connections 2024-09-19T04:54:20.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.923+0000 7f311be00640 1 --2- 172.21.15.123:0/2431298127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f311c102360 0x7f311c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.923+0000 7f311be00640 1 --2- 172.21.15.123:0/2431298127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f311c1019c0 0x7f311c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.923+0000 7f311be00640 1 --2- 172.21.15.123:0/2431298127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f311c1079c0 0x7f311c107da0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.923+0000 7f311be00640 1 -- 172.21.15.123:0/2431298127 >> 172.21.15.123:0/2431298127 conn(0x7f311c0fd740 msgr2=0x7f311c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:20.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.923+0000 7f311be00640 1 -- 172.21.15.123:0/2431298127 shutdown_connections 2024-09-19T04:54:20.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.923+0000 7f311be00640 1 -- 172.21.15.123:0/2431298127 wait complete. 2024-09-19T04:54:20.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.924+0000 7f311be00640 1 Processor -- start 2024-09-19T04:54:20.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.924+0000 7f311be00640 1 -- start start 2024-09-19T04:54:20.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.925+0000 7f311be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f311c1019c0 0x7f311c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.925+0000 7f311be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f311c102360 0x7f311c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.925+0000 7f311be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f311c1079c0 0x7f311c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.925+0000 7f311be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f311c111dc0 con 0x7f311c102360 2024-09-19T04:54:20.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.925+0000 7f311be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f311c111c40 con 0x7f311c1019c0 2024-09-19T04:54:20.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.925+0000 7f311be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f311c111f40 con 0x7f311c1079c0 2024-09-19T04:54:20.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.925+0000 7f3113e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f311c102360 0x7f311c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.925+0000 7f3113e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f311c102360 0x7f311c198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53598/0 (socket says 172.21.15.123:53598) 2024-09-19T04:54:20.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.925+0000 7f3113e00640 1 -- 172.21.15.123:0/627832461 learned_addr learned my addr 172.21.15.123:0/627832461 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:20.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.925+0000 7f311b400640 1 --2- 172.21.15.123:0/627832461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f311c1079c0 0x7f311c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.925+0000 7f311aa00640 1 --2- 172.21.15.123:0/627832461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f311c1019c0 0x7f311c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.926+0000 7f3113e00640 1 -- 172.21.15.123:0/627832461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f311c1079c0 msgr2=0x7f311c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:20.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.926+0000 7f3113e00640 1 --2- 172.21.15.123:0/627832461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f311c1079c0 0x7f311c19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.926+0000 7f3113e00640 1 -- 172.21.15.123:0/627832461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f311c1019c0 msgr2=0x7f311c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:20.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.926+0000 7f3113e00640 1 --2- 172.21.15.123:0/627832461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f311c1019c0 0x7f311c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.926+0000 7f3113e00640 1 -- 172.21.15.123:0/627832461 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311c19d2b0 con 0x7f311c102360 2024-09-19T04:54:20.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.926+0000 7f3113e00640 1 --2- 172.21.15.123:0/627832461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f311c102360 0x7f311c198840 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f310400e9f0 tx=0x7f310400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:20.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.926+0000 7f3120a00640 1 -- 172.21.15.123:0/627832461 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310400cd80 con 0x7f311c102360 2024-09-19T04:54:20.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.926+0000 7f3120a00640 1 -- 172.21.15.123:0/627832461 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3104005500 con 0x7f311c102360 2024-09-19T04:54:20.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.926+0000 7f311be00640 1 -- 172.21.15.123:0/627832461 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f311c19d5a0 con 0x7f311c102360 2024-09-19T04:54:20.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.926+0000 7f3120a00640 1 -- 172.21.15.123:0/627832461 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104010690 con 0x7f311c102360 2024-09-19T04:54:20.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.926+0000 7f311be00640 1 -- 172.21.15.123:0/627832461 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f311c1a4e80 con 0x7f311c102360 2024-09-19T04:54:20.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.929+0000 7f3120a00640 1 -- 172.21.15.123:0/627832461 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f31040046c0 con 0x7f311c102360 2024-09-19T04:54:20.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.929+0000 7f31249bd740 1 Processor -- start 2024-09-19T04:54:20.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.930+0000 7f31249bd740 1 -- start start 2024-09-19T04:54:20.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.930+0000 7f31249bd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 0x55b12a9b9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.930+0000 7f31249bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9b9660 0x55b12a9c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.930+0000 7f31249bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9c8430 0x55b12a9ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.930+0000 7f31249bd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b12a91f3e0 con 0x55b12a9b8d40 2024-09-19T04:54:20.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.930+0000 7f31249bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b12a8c6d80 con 0x55b12a9b9660 2024-09-19T04:54:20.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.930+0000 7f31249bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b12a91b3a0 con 0x55b12a9c8430 2024-09-19T04:54:20.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.930+0000 7f311b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 0x55b12a9b9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.930+0000 7f311aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9b9660 0x55b12a9c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.930+0000 7f311b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 0x55b12a9b9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53610/0 (socket says 172.21.15.123:53610) 2024-09-19T04:54:20.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.930+0000 7f311b400640 1 -- 172.21.15.123:0/2858659755 learned_addr learned my addr 172.21.15.123:0/2858659755 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:20.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.931+0000 7f3113e00640 1 --2- 172.21.15.123:0/2858659755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9c8430 0x55b12a9ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.931+0000 7f3120a00640 1 --2- 172.21.15.123:0/627832461 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31000a33b0 0x7f31000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.931+0000 7f3120a00640 1 -- 172.21.15.123:0/627832461 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f31040d0160 con 0x7f311c102360 2024-09-19T04:54:20.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.931+0000 7f311aa00640 1 -- 172.21.15.123:0/2858659755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9c8430 msgr2=0x55b12a9ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:20.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.931+0000 7f311aa00640 1 --2- 172.21.15.123:0/2858659755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9c8430 0x55b12a9ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.931+0000 7f311aa00640 1 -- 172.21.15.123:0/2858659755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 msgr2=0x55b12a9b9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:20.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.931+0000 7f311aa00640 1 --2- 172.21.15.123:0/2858659755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 0x55b12a9b9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.931+0000 7f311aa00640 1 -- 172.21.15.123:0/2858659755 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b12a8e5ae0 con 0x55b12a9b9660 2024-09-19T04:54:20.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.931+0000 7f311b400640 1 --2- 172.21.15.123:0/627832461 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31000a33b0 0x7f31000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.931+0000 7f311aa00640 1 --2- 172.21.15.123:0/2858659755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9b9660 0x55b12a9c7ef0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f310c005940 tx=0x7f310c005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ece367ded7e9bac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:20.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.932+0000 7f310be00640 1 -- 172.21.15.123:0/2858659755 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c040070 con 0x55b12a9b9660 2024-09-19T04:54:20.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.932+0000 7f310be00640 1 -- 172.21.15.123:0/2858659755 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f310c031870 con 0x55b12a9b9660 2024-09-19T04:54:20.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.932+0000 7f310be00640 1 -- 172.21.15.123:0/2858659755 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c035040 con 0x55b12a9b9660 2024-09-19T04:54:20.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.932+0000 7f31249bd740 1 -- 172.21.15.123:0/2858659755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9b9660 msgr2=0x55b12a9c7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:20.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.932+0000 7f311b400640 1 --2- 172.21.15.123:0/627832461 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31000a33b0 0x7f31000a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3114010610 tx=0x7f3114004a40 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:20.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.932+0000 7f31249bd740 1 --2- 172.21.15.123:0/2858659755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9b9660 0x55b12a9c7ef0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f310c005940 tx=0x7f310c005bd0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.933+0000 7f31249bd740 1 -- 172.21.15.123:0/2858659755 shutdown_connections 2024-09-19T04:54:20.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.933+0000 7f31249bd740 1 --2- 172.21.15.123:0/2858659755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9c8430 0x55b12a9ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.933+0000 7f31249bd740 1 --2- 172.21.15.123:0/2858659755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9b9660 0x55b12a9c7ef0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.933+0000 7f31249bd740 1 --2- 172.21.15.123:0/2858659755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 0x55b12a9b9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.933+0000 7f31249bd740 1 -- 172.21.15.123:0/2858659755 >> 172.21.15.123:0/2858659755 conn(0x55b12a9b53d0 msgr2=0x55b12a9b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:20.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.933+0000 7f31249bd740 1 -- 172.21.15.123:0/2858659755 shutdown_connections 2024-09-19T04:54:20.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.933+0000 7f31249bd740 1 -- 172.21.15.123:0/2858659755 wait complete. 2024-09-19T04:54:20.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.933+0000 7f31249bd740 1 -- start start 2024-09-19T04:54:20.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.933+0000 7f31249bd740 1 Processor -- start 2024-09-19T04:54:20.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.934+0000 7f31249bd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 0x55b12a9c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.934+0000 7f31249bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9b9660 0x55b12a9c5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.934+0000 7f31249bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9c8430 0x55b12a9c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.934+0000 7f31249bd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b12a9cb000 con 0x55b12a9b8d40 2024-09-19T04:54:20.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.934+0000 7f31249bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b12a8da0f0 con 0x55b12a9b9660 2024-09-19T04:54:20.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.934+0000 7f31249bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b12a9cb180 con 0x55b12a9c8430 2024-09-19T04:54:20.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.934+0000 7f31249bd740 10 client.0 _ll_register_callbacks cb 0x55b12a8c1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:20.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.934+0000 7f311aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 0x55b12a9c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.934+0000 7f311aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 0x55b12a9c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53618/0 (socket says 172.21.15.123:53618) 2024-09-19T04:54:20.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.934+0000 7f311aa00640 1 -- 172.21.15.123:0/3627352900 learned_addr learned my addr 172.21.15.123:0/3627352900 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:20.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.934+0000 7f3113e00640 1 --2- 172.21.15.123:0/3627352900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9b9660 0x55b12a9c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.934+0000 7f311b400640 1 --2- 172.21.15.123:0/3627352900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9c8430 0x55b12a9c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.935+0000 7f311aa00640 1 -- 172.21.15.123:0/3627352900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9c8430 msgr2=0x55b12a9c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:20.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.935+0000 7f311aa00640 1 --2- 172.21.15.123:0/3627352900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9c8430 0x55b12a9c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.935+0000 7f311aa00640 1 -- 172.21.15.123:0/3627352900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9b9660 msgr2=0x55b12a9c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:20.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.935+0000 7f311aa00640 1 --2- 172.21.15.123:0/3627352900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9b9660 0x55b12a9c5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.935+0000 7f311aa00640 1 -- 172.21.15.123:0/3627352900 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b12a6903c0 con 0x55b12a9b8d40 2024-09-19T04:54:20.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.935+0000 7f311aa00640 1 --2- 172.21.15.123:0/3627352900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 0x55b12a9c5330 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f310c005300 tx=0x7f310c005330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f380e0d15671535 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:20.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.935+0000 7f3108c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:20.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.935+0000 7f3108c00640 1 -- 172.21.15.123:0/3627352900 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c04c020 con 0x55b12a9b8d40 2024-09-19T04:54:20.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.935+0000 7f31249bd740 1 -- 172.21.15.123:0/3627352900 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f310c031870 con 0x55b12a9b8d40 2024-09-19T04:54:20.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.935+0000 7f3108c00640 1 -- 172.21.15.123:0/3627352900 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f310c004240 con 0x55b12a9b8d40 2024-09-19T04:54:20.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.936+0000 7f3108c00640 1 -- 172.21.15.123:0/3627352900 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f310c057050 con 0x55b12a9b8d40 2024-09-19T04:54:20.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.936+0000 7f31249bd740 10 client.16719 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:20.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.936+0000 7f31249bd740 1 -- 172.21.15.123:0/3627352900 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b12a9d7090 con 0x55b12a9b8d40 2024-09-19T04:54:20.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.936+0000 7f3108c00640 1 -- 172.21.15.123:0/3627352900 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f310c031870 con 0x55b12a9b8d40 2024-09-19T04:54:20.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.936+0000 7f3108c00640 1 -- 172.21.15.123:0/3627352900 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f310c003680 con 0x55b12a9b8d40 2024-09-19T04:54:20.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.936+0000 7f31249bd740 10 client.16719 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:20.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.936+0000 7f31249bd740 10 client.16719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:20.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.936+0000 7f31249bd740 20 client.16719 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:20.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.936+0000 7f31249bd740 1 --2- 172.21.15.123:0/3627352900 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b12a9d83e0 0x55b12a9da7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.936+0000 7f31249bd740 4 client.16719 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:20.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.936+0000 7f31249bd740 1 -- 172.21.15.123:0/3627352900 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b12a92c950 con 0x55b12a9d83e0 2024-09-19T04:54:20.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.937+0000 7f3113e00640 1 --2- 172.21.15.123:0/3627352900 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b12a9d83e0 0x55b12a9da7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.937+0000 7f3113e00640 1 --2- 172.21.15.123:0/3627352900 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b12a9d83e0 0x55b12a9da7d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78c64822e03c873f server_cookie=893aa9f32feed18c in_seq=0 out_seq=0 2024-09-19T04:54:20.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.937+0000 7f3108c00640 10 client.16719 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:20.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.938+0000 7f3108c00640 1 -- 172.21.15.123:0/3627352900 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b12a92c950 con 0x55b12a9d83e0 2024-09-19T04:54:20.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.938+0000 7f3108c00640 10 client.16719 handle_command_reply: tid=0 2024-09-19T04:54:20.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.938+0000 7f31249bd740 1 client.16719 shutdown 2024-09-19T04:54:20.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.938+0000 7f31249bd740 1 -- 172.21.15.123:0/3627352900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 msgr2=0x55b12a9c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:20.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.938+0000 7f31249bd740 1 --2- 172.21.15.123:0/3627352900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 0x55b12a9c5330 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f310c005300 tx=0x7f310c005330 comp rx=0 tx=0).stop 2024-09-19T04:54:20.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.938+0000 7f31249bd740 1 -- 172.21.15.123:0/3627352900 shutdown_connections 2024-09-19T04:54:20.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.938+0000 7f31249bd740 1 --2- 172.21.15.123:0/3627352900 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b12a9d83e0 0x55b12a9da7d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.938+0000 7f31249bd740 1 --2- 172.21.15.123:0/3627352900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9c8430 0x55b12a9c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.938+0000 7f31249bd740 1 --2- 172.21.15.123:0/3627352900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9b9660 0x55b12a9c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.938+0000 7f31249bd740 1 --2- 172.21.15.123:0/3627352900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 0x55b12a9c5330 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.938+0000 7f31249bd740 1 -- 172.21.15.123:0/3627352900 >> 172.21.15.123:0/3627352900 conn(0x55b12a9b53d0 msgr2=0x55b12a9bb130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:20.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.938+0000 7f3108c00640 0 client.16719 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:20.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.938+0000 7f31249bd740 1 -- 172.21.15.123:0/3627352900 shutdown_connections 2024-09-19T04:54:20.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.938+0000 7f31249bd740 1 -- 172.21.15.123:0/3627352900 wait complete. 2024-09-19T04:54:20.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.939+0000 7f31249bd740 20 client.16719 trim_cache size 0 max 16384 2024-09-19T04:54:20.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.952+0000 7f31249bd740 1 Processor -- start 2024-09-19T04:54:20.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.952+0000 7f31249bd740 1 -- start start 2024-09-19T04:54:20.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.952+0000 7f31249bd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 0x55b12a9dfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.952+0000 7f31249bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9b9660 0x55b12a9e0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.952+0000 7f31249bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9c8430 0x55b12a9ba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.952+0000 7f31249bd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1ab560 con 0x55b12a9b8d40 2024-09-19T04:54:20.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.952+0000 7f31249bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1ad130 con 0x55b12a9b9660 2024-09-19T04:54:20.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.952+0000 7f31249bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1abc60 con 0x55b12a9c8430 2024-09-19T04:54:20.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.953+0000 7f311aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 0x55b12a9dfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.953+0000 7f311aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 0x55b12a9dfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53622/0 (socket says 172.21.15.123:53622) 2024-09-19T04:54:20.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.953+0000 7f311aa00640 1 -- 172.21.15.123:0/649161577 learned_addr learned my addr 172.21.15.123:0/649161577 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:20.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.953+0000 7f3113e00640 1 --2- 172.21.15.123:0/649161577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9b9660 0x55b12a9e0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.953+0000 7f311b400640 1 --2- 172.21.15.123:0/649161577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9c8430 0x55b12a9ba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.953+0000 7f311aa00640 1 -- 172.21.15.123:0/649161577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9c8430 msgr2=0x55b12a9ba870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:20.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.953+0000 7f311aa00640 1 --2- 172.21.15.123:0/649161577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9c8430 0x55b12a9ba870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.953+0000 7f311aa00640 1 -- 172.21.15.123:0/649161577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9b9660 msgr2=0x55b12a9e0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:20.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.953+0000 7f311aa00640 1 --2- 172.21.15.123:0/649161577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9b9660 0x55b12a9e0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.953+0000 7f311aa00640 1 -- 172.21.15.123:0/649161577 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b12a9b4c80 con 0x55b12a9b8d40 2024-09-19T04:54:20.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.953+0000 7f311aa00640 1 --2- 172.21.15.123:0/649161577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 0x55b12a9dfdd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f310c001eb0 tx=0x7f310c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e183f85cb840d9b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:20.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.953+0000 7f310b400640 1 -- 172.21.15.123:0/649161577 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c04c040 con 0x55b12a9b8d40 2024-09-19T04:54:20.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.954+0000 7f310b400640 1 -- 172.21.15.123:0/649161577 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f310c005850 con 0x55b12a9b8d40 2024-09-19T04:54:20.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.954+0000 7f310b400640 1 -- 172.21.15.123:0/649161577 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c040070 con 0x55b12a9b8d40 2024-09-19T04:54:20.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.954+0000 7f31249bd740 1 -- 172.21.15.123:0/649161577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 msgr2=0x55b12a9dfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:20.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.954+0000 7f31249bd740 1 --2- 172.21.15.123:0/649161577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 0x55b12a9dfdd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f310c001eb0 tx=0x7f310c000bc0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.954+0000 7f31249bd740 1 -- 172.21.15.123:0/649161577 shutdown_connections 2024-09-19T04:54:20.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.954+0000 7f31249bd740 1 --2- 172.21.15.123:0/649161577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9c8430 0x55b12a9ba870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.954+0000 7f31249bd740 1 --2- 172.21.15.123:0/649161577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9b9660 0x55b12a9e0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.954+0000 7f31249bd740 1 --2- 172.21.15.123:0/649161577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9b8d40 0x55b12a9dfdd0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.954+0000 7f31249bd740 1 -- 172.21.15.123:0/649161577 >> 172.21.15.123:0/649161577 conn(0x55b12a9b53d0 msgr2=0x55b12a9c73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:20.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.954+0000 7f31249bd740 1 -- 172.21.15.123:0/649161577 shutdown_connections 2024-09-19T04:54:20.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.954+0000 7f31249bd740 1 -- 172.21.15.123:0/649161577 wait complete. 2024-09-19T04:54:20.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.955+0000 7f31249bd740 1 -- start start 2024-09-19T04:54:20.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.955+0000 7f31249bd740 1 Processor -- start 2024-09-19T04:54:20.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.955+0000 7f31249bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9b9660 0x55b12a9c23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.955+0000 7f31249bd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9c8430 0x55b12a9c2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.955+0000 7f31249bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9c2e50 0x55b12aa41da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.955+0000 7f31249bd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1aca60 con 0x55b12a9c8430 2024-09-19T04:54:20.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.955+0000 7f31249bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1ac360 con 0x55b12a9c2e50 2024-09-19T04:54:20.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.955+0000 7f31249bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1a9560 con 0x55b12a9b9660 2024-09-19T04:54:20.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.955+0000 7f31249bd740 10 client.0 _ll_register_callbacks cb 0x7f3114009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:20.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.956+0000 7f3113e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9c8430 0x55b12a9c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.956+0000 7f3113e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9c8430 0x55b12a9c2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53638/0 (socket says 172.21.15.123:53638) 2024-09-19T04:54:20.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.956+0000 7f3113e00640 1 -- 172.21.15.123:0/1600461530 learned_addr learned my addr 172.21.15.123:0/1600461530 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:20.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.956+0000 7f311aa00640 1 --2- 172.21.15.123:0/1600461530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9b9660 0x55b12a9c23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.956+0000 7f311b400640 1 --2- 172.21.15.123:0/1600461530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9c2e50 0x55b12aa41da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.956+0000 7f3113e00640 1 -- 172.21.15.123:0/1600461530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9b9660 msgr2=0x55b12a9c23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:20.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.956+0000 7f3113e00640 1 --2- 172.21.15.123:0/1600461530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9b9660 0x55b12a9c23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.956+0000 7f3113e00640 1 -- 172.21.15.123:0/1600461530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9c2e50 msgr2=0x55b12aa41da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:20.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.956+0000 7f3113e00640 1 --2- 172.21.15.123:0/1600461530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9c2e50 0x55b12aa41da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.956+0000 7f3113e00640 1 -- 172.21.15.123:0/1600461530 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b12aa42310 con 0x55b12a9c8430 2024-09-19T04:54:20.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.956+0000 7f3113e00640 1 --2- 172.21.15.123:0/1600461530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9c8430 0x55b12a9c2910 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f31040cd8b0 tx=0x7f3104098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c6f1cde13efdb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:20.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.956+0000 7f3108c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:20.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.957+0000 7f3108c00640 1 -- 172.21.15.123:0/1600461530 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310409bd40 con 0x55b12a9c8430 2024-09-19T04:54:20.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.957+0000 7f3108c00640 1 -- 172.21.15.123:0/1600461530 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f3104004f40 con 0x55b12a9c8430 2024-09-19T04:54:20.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.957+0000 7f31249bd740 1 -- 172.21.15.123:0/1600461530 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f310c005850 con 0x55b12a9c8430 2024-09-19T04:54:20.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.957+0000 7f3108c00640 1 -- 172.21.15.123:0/1600461530 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f310409cae0 con 0x55b12a9c8430 2024-09-19T04:54:20.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.957+0000 7f31249bd740 10 client.16725 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:20.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.957+0000 7f31249bd740 1 -- 172.21.15.123:0/1600461530 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b12aa435f0 con 0x55b12a9c8430 2024-09-19T04:54:20.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.957+0000 7f3108c00640 1 -- 172.21.15.123:0/1600461530 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f310c005850 con 0x55b12a9c8430 2024-09-19T04:54:20.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.957+0000 7f3108c00640 1 -- 172.21.15.123:0/1600461530 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f31040cdad0 con 0x55b12a9c8430 2024-09-19T04:54:20.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.957+0000 7f31249bd740 10 client.16725 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:20.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.957+0000 7f31249bd740 10 client.16725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:20.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.957+0000 7f31249bd740 20 client.16725 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:20.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.957+0000 7f31249bd740 1 --2- 172.21.15.123:0/1600461530 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b12aa43d10 0x55b12aa46100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:20.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.957+0000 7f31249bd740 4 client.16725 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:20.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.957+0000 7f31249bd740 1 -- 172.21.15.123:0/1600461530 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b12a926e60 con 0x55b12aa43d10 2024-09-19T04:54:20.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.958+0000 7f311aa00640 1 --2- 172.21.15.123:0/1600461530 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b12aa43d10 0x55b12aa46100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.958+0000 7f311aa00640 1 --2- 172.21.15.123:0/1600461530 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b12aa43d10 0x55b12aa46100 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc4a728426220832 server_cookie=13d21a99be58d480 in_seq=0 out_seq=0 2024-09-19T04:54:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.958+0000 7f3108c00640 10 client.16725 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.958+0000 7f3108c00640 1 -- 172.21.15.123:0/1600461530 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b12a926e60 con 0x55b12aa43d10 2024-09-19T04:54:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.958+0000 7f3108c00640 10 client.16725 handle_command_reply: tid=0 2024-09-19T04:54:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.959+0000 7f31249bd740 1 client.16725 shutdown 2024-09-19T04:54:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.959+0000 7f31249bd740 1 -- 172.21.15.123:0/1600461530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9c8430 msgr2=0x55b12a9c2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.959+0000 7f31249bd740 1 --2- 172.21.15.123:0/1600461530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9c8430 0x55b12a9c2910 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f31040cd8b0 tx=0x7f3104098040 comp rx=0 tx=0).stop 2024-09-19T04:54:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.959+0000 7f31249bd740 1 -- 172.21.15.123:0/1600461530 shutdown_connections 2024-09-19T04:54:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.959+0000 7f31249bd740 1 --2- 172.21.15.123:0/1600461530 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55b12aa43d10 0x55b12aa46100 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.959+0000 7f31249bd740 1 --2- 172.21.15.123:0/1600461530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b12a9c2e50 0x55b12aa41da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.959+0000 7f31249bd740 1 --2- 172.21.15.123:0/1600461530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b12a9c8430 0x55b12a9c2910 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.959+0000 7f31249bd740 1 --2- 172.21.15.123:0/1600461530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b12a9b9660 0x55b12a9c23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.959+0000 7f31249bd740 1 -- 172.21.15.123:0/1600461530 >> 172.21.15.123:0/1600461530 conn(0x55b12a9b53d0 msgr2=0x55b12a9b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:20.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.959+0000 7f3108c00640 0 client.16725 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:20.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.959+0000 7f31249bd740 1 -- 172.21.15.123:0/1600461530 shutdown_connections 2024-09-19T04:54:20.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.959+0000 7f31249bd740 1 -- 172.21.15.123:0/1600461530 wait complete. 2024-09-19T04:54:20.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.959+0000 7f31249bd740 20 client.16725 trim_cache size 0 max 16384 2024-09-19T04:54:20.965 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:20.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.960+0000 7f311be00640 1 -- 172.21.15.123:0/627832461 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31000a33b0 msgr2=0x7f31000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:20.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.960+0000 7f311be00640 1 --2- 172.21.15.123:0/627832461 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31000a33b0 0x7f31000a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3114010610 tx=0x7f3114004a40 comp rx=0 tx=0).stop 2024-09-19T04:54:20.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.960+0000 7f311be00640 1 -- 172.21.15.123:0/627832461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f311c102360 msgr2=0x7f311c198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:20.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.960+0000 7f311be00640 1 --2- 172.21.15.123:0/627832461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f311c102360 0x7f311c198840 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f310400e9f0 tx=0x7f310400edf0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.960+0000 7f311be00640 1 -- 172.21.15.123:0/627832461 shutdown_connections 2024-09-19T04:54:20.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.960+0000 7f311be00640 1 --2- 172.21.15.123:0/627832461 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f31000a33b0 0x7f31000a5870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.960+0000 7f311be00640 1 --2- 172.21.15.123:0/627832461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f311c1079c0 0x7f311c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.960+0000 7f311be00640 1 --2- 172.21.15.123:0/627832461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f311c102360 0x7f311c198840 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.960+0000 7f311be00640 1 --2- 172.21.15.123:0/627832461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f311c1019c0 0x7f311c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:20.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.960+0000 7f311be00640 1 -- 172.21.15.123:0/627832461 >> 172.21.15.123:0/627832461 conn(0x7f311c0fd740 msgr2=0x7f311c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:20.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.960+0000 7f311be00640 1 -- 172.21.15.123:0/627832461 shutdown_connections 2024-09-19T04:54:20.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:20.960+0000 7f311be00640 1 -- 172.21.15.123:0/627832461 wait complete. 2024-09-19T04:54:21.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:21 smithi123 ceph-mon[30511]: pgmap v236: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:54:21.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:21 smithi193 ceph-mon[35592]: pgmap v236: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:54:21.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:21 smithi183 ceph-mon[35333]: pgmap v236: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:54:22.814 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:54:22.815 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:54:23.192 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:23.622 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:23 smithi123 ceph-mon[30511]: pgmap v237: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:54:23.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:23 smithi193 ceph-mon[35592]: pgmap v237: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:54:23.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.733+0000 7fb7c7200640 1 -- 172.21.15.123:0/1995990099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c8102b90 msgr2=0x7fb7c8102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:23.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.733+0000 7fb7c7200640 1 --2- 172.21.15.123:0/1995990099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c8102b90 0x7fb7c8102f70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009900 tx=0x7fb7b802a480 comp rx=0 tx=0).stop 2024-09-19T04:54:23.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.733+0000 7fb7c7200640 1 -- 172.21.15.123:0/1995990099 shutdown_connections 2024-09-19T04:54:23.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.733+0000 7fb7c7200640 1 --2- 172.21.15.123:0/1995990099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7c810bfc0 0x7fb7c810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.733+0000 7fb7c7200640 1 --2- 172.21.15.123:0/1995990099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c81034b0 0x7fb7c810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.733+0000 7fb7c7200640 1 --2- 172.21.15.123:0/1995990099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c8102b90 0x7fb7c8102f70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.733+0000 7fb7c7200640 1 -- 172.21.15.123:0/1995990099 >> 172.21.15.123:0/1995990099 conn(0x7fb7c80fc610 msgr2=0x7fb7c80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:23.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.733+0000 7fb7c7200640 1 -- 172.21.15.123:0/1995990099 shutdown_connections 2024-09-19T04:54:23.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.733+0000 7fb7c7200640 1 -- 172.21.15.123:0/1995990099 wait complete. 2024-09-19T04:54:23.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.734+0000 7fb7c7200640 1 Processor -- start 2024-09-19T04:54:23.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.734+0000 7fb7c7200640 1 -- start start 2024-09-19T04:54:23.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.735+0000 7fb7c7200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c8102b90 0x7fb7c81007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.735+0000 7fb7c7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c81034b0 0x7fb7c8100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.735+0000 7fb7c7200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7c810bfc0 0x7fb7c8101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.735+0000 7fb7c7200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c8111dc0 con 0x7fb7c81034b0 2024-09-19T04:54:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.735+0000 7fb7c7200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c8111c40 con 0x7fb7c8102b90 2024-09-19T04:54:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.735+0000 7fb7c7200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c8111f40 con 0x7fb7c810bfc0 2024-09-19T04:54:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.735+0000 7fb7bfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c81034b0 0x7fb7c8100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.735+0000 7fb7c5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c8102b90 0x7fb7c81007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.735+0000 7fb7bfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c81034b0 0x7fb7c8100d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44652/0 (socket says 172.21.15.123:44652) 2024-09-19T04:54:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.735+0000 7fb7c6800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7c810bfc0 0x7fb7c8101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.735+0000 7fb7bfe00640 1 -- 172.21.15.123:0/1891369757 learned_addr learned my addr 172.21.15.123:0/1891369757 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.736+0000 7fb7bfe00640 1 -- 172.21.15.123:0/1891369757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7c810bfc0 msgr2=0x7fb7c8101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.736+0000 7fb7bfe00640 1 --2- 172.21.15.123:0/1891369757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7c810bfc0 0x7fb7c8101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.736+0000 7fb7bfe00640 1 -- 172.21.15.123:0/1891369757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c8102b90 msgr2=0x7fb7c81007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.736+0000 7fb7bfe00640 1 --2- 172.21.15.123:0/1891369757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c8102b90 0x7fb7c81007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.736+0000 7fb7bfe00640 1 -- 172.21.15.123:0/1891369757 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c81a6730 con 0x7fb7c81034b0 2024-09-19T04:54:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.736+0000 7fb7bfe00640 1 --2- 172.21.15.123:0/1891369757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c81034b0 0x7fb7c8100d30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000e9f0 tx=0x7fb7b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.736+0000 7fb7bea00640 1 -- 172.21.15.123:0/1891369757 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000cd80 con 0x7fb7c81034b0 2024-09-19T04:54:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.736+0000 7fb7bea00640 1 -- 172.21.15.123:0/1891369757 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b00035f0 con 0x7fb7c81034b0 2024-09-19T04:54:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.736+0000 7fb7c7200640 1 -- 172.21.15.123:0/1891369757 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c81a6a20 con 0x7fb7c81034b0 2024-09-19T04:54:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.736+0000 7fb7bea00640 1 -- 172.21.15.123:0/1891369757 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0010670 con 0x7fb7c81034b0 2024-09-19T04:54:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.736+0000 7fb7c7200640 1 -- 172.21.15.123:0/1891369757 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c81a6fb0 con 0x7fb7c81034b0 2024-09-19T04:54:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.739+0000 7fb7cf661740 1 Processor -- start 2024-09-19T04:54:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.739+0000 7fb7bea00640 1 -- 172.21.15.123:0/1891369757 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb7b00037a0 con 0x7fb7c81034b0 2024-09-19T04:54:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.739+0000 7fb7cf661740 1 -- start start 2024-09-19T04:54:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.740+0000 7fb7cf661740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188648cb0 0x565188649090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.740+0000 7fb7cf661740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651886495d0 0x565188653950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.740+0000 7fb7cf661740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188653e90 0x565188656280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.740+0000 7fb7cf661740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5651885af290 con 0x5651886495d0 2024-09-19T04:54:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.740+0000 7fb7cf661740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565188556b40 con 0x565188648cb0 2024-09-19T04:54:23.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.740+0000 7fb7cf661740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5651885ab250 con 0x565188653e90 2024-09-19T04:54:23.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.740+0000 7fb7c5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651886495d0 0x565188653950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.740+0000 7fb7c5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651886495d0 0x565188653950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44658/0 (socket says 172.21.15.123:44658) 2024-09-19T04:54:23.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.740+0000 7fb7c5e00640 1 -- 172.21.15.123:0/3538901757 learned_addr learned my addr 172.21.15.123:0/3538901757 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:23.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.740+0000 7fb7c6800640 1 --2- 172.21.15.123:0/3538901757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188648cb0 0x565188649090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.740+0000 7fb7bfe00640 1 --2- 172.21.15.123:0/3538901757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188653e90 0x565188656280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.740+0000 7fb7c5e00640 1 -- 172.21.15.123:0/3538901757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188653e90 msgr2=0x565188656280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:23.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.740+0000 7fb7c5e00640 1 --2- 172.21.15.123:0/3538901757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188653e90 0x565188656280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.741+0000 7fb7c5e00640 1 -- 172.21.15.123:0/3538901757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188648cb0 msgr2=0x565188649090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:23.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.741+0000 7fb7c5e00640 1 --2- 172.21.15.123:0/3538901757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188648cb0 0x565188649090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.741+0000 7fb7c5e00640 1 -- 172.21.15.123:0/3538901757 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651883206a0 con 0x5651886495d0 2024-09-19T04:54:23.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.741+0000 7fb7c5e00640 1 --2- 172.21.15.123:0/3538901757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651886495d0 0x565188653950 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009900 tx=0x7fb7b8005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57b0a5cebe5360 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:23.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.741+0000 7fb7bea00640 1 --2- 172.21.15.123:0/1891369757 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb7ac0a3430 0x7fb7ac0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.741+0000 7fb7b7000640 1 -- 172.21.15.123:0/3538901757 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8040070 con 0x5651886495d0 2024-09-19T04:54:23.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.741+0000 7fb7b7000640 1 -- 172.21.15.123:0/3538901757 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b8031870 con 0x5651886495d0 2024-09-19T04:54:23.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.741+0000 7fb7b7000640 1 -- 172.21.15.123:0/3538901757 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8035040 con 0x5651886495d0 2024-09-19T04:54:23.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.741+0000 7fb7bea00640 1 -- 172.21.15.123:0/1891369757 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fb7b00cff90 con 0x7fb7c81034b0 2024-09-19T04:54:23.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.741+0000 7fb7c6800640 1 --2- 172.21.15.123:0/1891369757 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb7ac0a3430 0x7fb7ac0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.742+0000 7fb7cf661740 1 -- 172.21.15.123:0/3538901757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651886495d0 msgr2=0x565188653950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:23.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.742+0000 7fb7cf661740 1 --2- 172.21.15.123:0/3538901757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651886495d0 0x565188653950 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009900 tx=0x7fb7b8005be0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.742+0000 7fb7c6800640 1 --2- 172.21.15.123:0/1891369757 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb7ac0a3430 0x7fb7ac0a58f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0005fd0 tx=0x7fb7c0009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:23.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.742+0000 7fb7cf661740 1 -- 172.21.15.123:0/3538901757 shutdown_connections 2024-09-19T04:54:23.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.742+0000 7fb7cf661740 1 --2- 172.21.15.123:0/3538901757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188653e90 0x565188656280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.742+0000 7fb7cf661740 1 --2- 172.21.15.123:0/3538901757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651886495d0 0x565188653950 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.742+0000 7fb7cf661740 1 --2- 172.21.15.123:0/3538901757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188648cb0 0x565188649090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.742+0000 7fb7cf661740 1 -- 172.21.15.123:0/3538901757 >> 172.21.15.123:0/3538901757 conn(0x565188645240 msgr2=0x565188645620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:23.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.742+0000 7fb7cf661740 1 -- 172.21.15.123:0/3538901757 shutdown_connections 2024-09-19T04:54:23.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.742+0000 7fb7cf661740 1 -- 172.21.15.123:0/3538901757 wait complete. 2024-09-19T04:54:23.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.743+0000 7fb7cf661740 1 -- start start 2024-09-19T04:54:23.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.743+0000 7fb7cf661740 1 Processor -- start 2024-09-19T04:54:23.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7cf661740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188648cb0 0x5651886536d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7cf661740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651886495d0 0x565188650980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7cf661740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565188653e90 0x565188650ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7cf661740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x565188656b00 con 0x565188653e90 2024-09-19T04:54:23.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7cf661740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565188569fa0 con 0x565188648cb0 2024-09-19T04:54:23.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7cf661740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x565188656c80 con 0x5651886495d0 2024-09-19T04:54:23.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7cf661740 10 client.0 _ll_register_callbacks cb 0x5651882bdba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:23.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7c5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188648cb0 0x5651886536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7c6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565188653e90 0x565188650ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7bfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651886495d0 0x565188650980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7c5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188648cb0 0x5651886536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:38648/0 (socket says 172.21.15.123:38648) 2024-09-19T04:54:23.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7c5e00640 1 -- 172.21.15.123:0/2250000696 learned_addr learned my addr 172.21.15.123:0/2250000696 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:23.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7c6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565188653e90 0x565188650ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44672/0 (socket says 172.21.15.123:44672) 2024-09-19T04:54:23.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7bfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651886495d0 0x565188650980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:39402/0 (socket says 172.21.15.123:39402) 2024-09-19T04:54:23.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7c5e00640 1 -- 172.21.15.123:0/2250000696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651886495d0 msgr2=0x565188650980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:23.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7c5e00640 1 --2- 172.21.15.123:0/2250000696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651886495d0 0x565188650980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7c5e00640 1 -- 172.21.15.123:0/2250000696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565188653e90 msgr2=0x565188650ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:23.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7c5e00640 1 --2- 172.21.15.123:0/2250000696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565188653e90 0x565188650ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7c5e00640 1 -- 172.21.15.123:0/2250000696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651886516c0 con 0x565188648cb0 2024-09-19T04:54:23.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7c6800640 1 --2- 172.21.15.123:0/2250000696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565188653e90 0x565188650ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:23.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.744+0000 7fb7bfe00640 1 --2- 172.21.15.123:0/2250000696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651886495d0 0x565188650980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:23.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.745+0000 7fb7c5e00640 1 --2- 172.21.15.123:0/2250000696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188648cb0 0x5651886536d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8006010 tx=0x7fb7b802a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=632cc0dc91a2c69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:23.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.745+0000 7fb7ab400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:54:23.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.745+0000 7fb7ab400640 1 -- 172.21.15.123:0/2250000696 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b804c020 con 0x565188648cb0 2024-09-19T04:54:23.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.745+0000 7fb7cf661740 1 -- 172.21.15.123:0/2250000696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7b8031870 con 0x565188648cb0 2024-09-19T04:54:23.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.745+0000 7fb7ab400640 1 -- 172.21.15.123:0/2250000696 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fb7b8004960 con 0x565188648cb0 2024-09-19T04:54:23.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.745+0000 7fb7ab400640 1 -- 172.21.15.123:0/2250000696 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb7b8057050 con 0x565188648cb0 2024-09-19T04:54:23.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.745+0000 7fb7cf661740 10 client.26834 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:23.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.745+0000 7fb7cf661740 1 -- 172.21.15.123:0/2250000696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x565188662950 con 0x565188648cb0 2024-09-19T04:54:23.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.745+0000 7fb7ab400640 1 -- 172.21.15.123:0/2250000696 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b8031870 con 0x565188648cb0 2024-09-19T04:54:23.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.746+0000 7fb7ab400640 1 -- 172.21.15.123:0/2250000696 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fb7b803cab0 con 0x565188648cb0 2024-09-19T04:54:23.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.746+0000 7fb7cf661740 10 client.26834 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:23.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.746+0000 7fb7cf661740 10 client.26834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:23.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.746+0000 7fb7cf661740 20 client.26834 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:23.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.746+0000 7fb7cf661740 1 --2- 172.21.15.123:0/2250000696 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x565188663ca0 0x565188666090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.746+0000 7fb7cf661740 4 client.26834 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:23.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.746+0000 7fb7cf661740 1 -- 172.21.15.123:0/2250000696 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651885bc800 con 0x565188663ca0 2024-09-19T04:54:23.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.746+0000 7fb7bfe00640 1 --2- 172.21.15.123:0/2250000696 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x565188663ca0 0x565188666090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.747+0000 7fb7bfe00640 1 --2- 172.21.15.123:0/2250000696 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x565188663ca0 0x565188666090 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e37260974680ec16 server_cookie=fa63e008fc873093 in_seq=0 out_seq=0 2024-09-19T04:54:23.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.747+0000 7fb7ab400640 10 client.26834 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:23.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.747+0000 7fb7ab400640 1 -- 172.21.15.123:0/2250000696 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651885bc800 con 0x565188663ca0 2024-09-19T04:54:23.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.747+0000 7fb7ab400640 10 client.26834 handle_command_reply: tid=0 2024-09-19T04:54:23.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.747+0000 7fb7cf661740 1 client.26834 shutdown 2024-09-19T04:54:23.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.747+0000 7fb7cf661740 1 -- 172.21.15.123:0/2250000696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188648cb0 msgr2=0x5651886536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:23.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.747+0000 7fb7cf661740 1 --2- 172.21.15.123:0/2250000696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188648cb0 0x5651886536d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8006010 tx=0x7fb7b802a480 comp rx=0 tx=0).stop 2024-09-19T04:54:23.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.748+0000 7fb7cf661740 1 -- 172.21.15.123:0/2250000696 shutdown_connections 2024-09-19T04:54:23.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.748+0000 7fb7cf661740 1 --2- 172.21.15.123:0/2250000696 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x565188663ca0 0x565188666090 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.748+0000 7fb7cf661740 1 --2- 172.21.15.123:0/2250000696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565188653e90 0x565188650ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.748+0000 7fb7cf661740 1 --2- 172.21.15.123:0/2250000696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651886495d0 0x565188650980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.748+0000 7fb7ab400640 0 client.26834 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:23.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.748+0000 7fb7cf661740 1 --2- 172.21.15.123:0/2250000696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188648cb0 0x5651886536d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.748+0000 7fb7cf661740 1 -- 172.21.15.123:0/2250000696 >> 172.21.15.123:0/2250000696 conn(0x565188645240 msgr2=0x56518864af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:23.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.748+0000 7fb7cf661740 1 -- 172.21.15.123:0/2250000696 shutdown_connections 2024-09-19T04:54:23.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.748+0000 7fb7cf661740 1 -- 172.21.15.123:0/2250000696 wait complete. 2024-09-19T04:54:23.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.748+0000 7fb7cf661740 20 client.26834 trim_cache size 0 max 16384 2024-09-19T04:54:23.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.766+0000 7fb7cf661740 1 Processor -- start 2024-09-19T04:54:23.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.766+0000 7fb7cf661740 1 -- start start 2024-09-19T04:54:23.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.766+0000 7fb7cf661740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188648cb0 0x5651886531f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.766+0000 7fb7cf661740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651886495d0 0x56518866b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.766+0000 7fb7cf661740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565188653e90 0x56518866bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.767+0000 7fb7cf661740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c81af8f0 con 0x565188653e90 2024-09-19T04:54:23.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.767+0000 7fb7cf661740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c81b14c0 con 0x5651886495d0 2024-09-19T04:54:23.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.767+0000 7fb7cf661740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c81afff0 con 0x565188648cb0 2024-09-19T04:54:23.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.767+0000 7fb7c6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565188653e90 0x56518866bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.767+0000 7fb7c6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565188653e90 0x56518866bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44684/0 (socket says 172.21.15.123:44684) 2024-09-19T04:54:23.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.767+0000 7fb7c5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188648cb0 0x5651886531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.767+0000 7fb7c6800640 1 -- 172.21.15.123:0/1183031046 learned_addr learned my addr 172.21.15.123:0/1183031046 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:23.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.767+0000 7fb7bfe00640 1 --2- 172.21.15.123:0/1183031046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651886495d0 0x56518866b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.767+0000 7fb7c5e00640 1 -- 172.21.15.123:0/1183031046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651886495d0 msgr2=0x56518866b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:23.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.767+0000 7fb7c5e00640 1 --2- 172.21.15.123:0/1183031046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651886495d0 0x56518866b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.767+0000 7fb7c5e00640 1 -- 172.21.15.123:0/1183031046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565188653e90 msgr2=0x56518866bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:23.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.767+0000 7fb7c5e00640 1 --2- 172.21.15.123:0/1183031046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565188653e90 0x56518866bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.767+0000 7fb7c5e00640 1 -- 172.21.15.123:0/1183031046 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565188649400 con 0x565188648cb0 2024-09-19T04:54:23.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.767+0000 7fb7c6800640 1 --2- 172.21.15.123:0/1183031046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565188653e90 0x56518866bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:23.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.768+0000 7fb7c5e00640 1 --2- 172.21.15.123:0/1183031046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188648cb0 0x5651886531f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8005250 tx=0x7fb7b8031ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a93ac3307ce0f84c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:23.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.768+0000 7fb7b6600640 1 -- 172.21.15.123:0/1183031046 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b804c070 con 0x565188648cb0 2024-09-19T04:54:23.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.768+0000 7fb7b6600640 1 -- 172.21.15.123:0/1183031046 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b8005850 con 0x565188648cb0 2024-09-19T04:54:23.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.768+0000 7fb7b6600640 1 -- 172.21.15.123:0/1183031046 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8040070 con 0x565188648cb0 2024-09-19T04:54:23.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.768+0000 7fb7cf661740 1 -- 172.21.15.123:0/1183031046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188648cb0 msgr2=0x5651886531f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:23.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.768+0000 7fb7cf661740 1 --2- 172.21.15.123:0/1183031046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188648cb0 0x5651886531f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8005250 tx=0x7fb7b8031ae0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.769+0000 7fb7cf661740 1 -- 172.21.15.123:0/1183031046 shutdown_connections 2024-09-19T04:54:23.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.769+0000 7fb7cf661740 1 --2- 172.21.15.123:0/1183031046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565188653e90 0x56518866bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.769+0000 7fb7cf661740 1 --2- 172.21.15.123:0/1183031046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651886495d0 0x56518866b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.769+0000 7fb7cf661740 1 --2- 172.21.15.123:0/1183031046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188648cb0 0x5651886531f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.769+0000 7fb7cf661740 1 -- 172.21.15.123:0/1183031046 >> 172.21.15.123:0/1183031046 conn(0x565188645240 msgr2=0x56518864fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:23.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.769+0000 7fb7cf661740 1 -- 172.21.15.123:0/1183031046 shutdown_connections 2024-09-19T04:54:23.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.769+0000 7fb7cf661740 1 -- 172.21.15.123:0/1183031046 wait complete. 2024-09-19T04:54:23.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.769+0000 7fb7cf661740 1 -- start start 2024-09-19T04:54:23.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.769+0000 7fb7cf661740 1 Processor -- start 2024-09-19T04:54:23.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7cf661740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188648cb0 0x565188657fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7cf661740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651886495d0 0x5651886584e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7cf661740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188653e90 0x5651886cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7cf661740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c81b0df0 con 0x5651886495d0 2024-09-19T04:54:23.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7cf661740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c81b06f0 con 0x565188653e90 2024-09-19T04:54:23.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7cf661740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c81ad8f0 con 0x565188648cb0 2024-09-19T04:54:23.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7cf661740 10 client.0 _ll_register_callbacks cb 0x7fb7c0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:23.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7c5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188648cb0 0x565188657fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7c5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188648cb0 0x565188657fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:39432/0 (socket says 172.21.15.123:39432) 2024-09-19T04:54:23.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7c5e00640 1 -- 172.21.15.123:0/217073382 learned_addr learned my addr 172.21.15.123:0/217073382 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:23.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7bfe00640 1 --2- 172.21.15.123:0/217073382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651886495d0 0x5651886584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7c6800640 1 --2- 172.21.15.123:0/217073382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188653e90 0x5651886cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7c5e00640 1 -- 172.21.15.123:0/217073382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188653e90 msgr2=0x5651886cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:23.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7c5e00640 1 --2- 172.21.15.123:0/217073382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188653e90 0x5651886cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7c5e00640 1 -- 172.21.15.123:0/217073382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651886495d0 msgr2=0x5651886584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:23.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7c5e00640 1 --2- 172.21.15.123:0/217073382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651886495d0 0x5651886584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7c5e00640 1 -- 172.21.15.123:0/217073382 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565188658dc0 con 0x565188648cb0 2024-09-19T04:54:23.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.770+0000 7fb7c6800640 1 --2- 172.21.15.123:0/217073382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188653e90 0x5651886cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:54:23.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.771+0000 7fb7c5e00640 1 --2- 172.21.15.123:0/217073382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188648cb0 0x565188657fa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80057a0 tx=0x7fb7b8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b66c20b410005857 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:23.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.771+0000 7fb7ab400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:54:23.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.771+0000 7fb7ab400640 1 -- 172.21.15.123:0/217073382 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8040070 con 0x565188648cb0 2024-09-19T04:54:23.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.771+0000 7fb7cf661740 1 -- 172.21.15.123:0/217073382 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7b8005850 con 0x565188648cb0 2024-09-19T04:54:23.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.771+0000 7fb7ab400640 1 -- 172.21.15.123:0/217073382 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fb7b8004b60 con 0x565188648cb0 2024-09-19T04:54:23.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.771+0000 7fb7ab400640 1 -- 172.21.15.123:0/217073382 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb7b8048070 con 0x565188648cb0 2024-09-19T04:54:23.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.771+0000 7fb7cf661740 10 client.26863 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:23.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.771+0000 7fb7cf661740 1 -- 172.21.15.123:0/217073382 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5651886cec90 con 0x565188648cb0 2024-09-19T04:54:23.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.771+0000 7fb7ab400640 1 -- 172.21.15.123:0/217073382 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b8005850 con 0x565188648cb0 2024-09-19T04:54:23.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.771+0000 7fb7ab400640 1 -- 172.21.15.123:0/217073382 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fb7b803c4d0 con 0x565188648cb0 2024-09-19T04:54:23.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.772+0000 7fb7cf661740 10 client.26863 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:23.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.772+0000 7fb7cf661740 10 client.26863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:23.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.772+0000 7fb7cf661740 20 client.26863 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:23.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.772+0000 7fb7cf661740 1 --2- 172.21.15.123:0/217073382 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5651886cf240 0x5651886d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:23.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.772+0000 7fb7cf661740 4 client.26863 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:54:23.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.772+0000 7fb7cf661740 1 -- 172.21.15.123:0/217073382 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5651885b6d10 con 0x5651886cf240 2024-09-19T04:54:23.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.772+0000 7fb7bfe00640 1 --2- 172.21.15.123:0/217073382 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5651886cf240 0x5651886d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:23.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.772+0000 7fb7bfe00640 1 --2- 172.21.15.123:0/217073382 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5651886cf240 0x5651886d1630 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28ba7c7d339ca035 server_cookie=f0c25f388eed38ed in_seq=0 out_seq=0 2024-09-19T04:54:23.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.772+0000 7fb7ab400640 10 client.26863 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:23.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.774+0000 7fb7ab400640 1 -- 172.21.15.123:0/217073382 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5651885b6d10 con 0x5651886cf240 2024-09-19T04:54:23.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.774+0000 7fb7ab400640 10 client.26863 handle_command_reply: tid=0 2024-09-19T04:54:23.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.774+0000 7fb7cf661740 1 client.26863 shutdown 2024-09-19T04:54:23.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.774+0000 7fb7cf661740 1 -- 172.21.15.123:0/217073382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188648cb0 msgr2=0x565188657fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:23.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.774+0000 7fb7cf661740 1 --2- 172.21.15.123:0/217073382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188648cb0 0x565188657fa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80057a0 tx=0x7fb7b8009780 comp rx=0 tx=0).stop 2024-09-19T04:54:23.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.774+0000 7fb7cf661740 1 -- 172.21.15.123:0/217073382 shutdown_connections 2024-09-19T04:54:23.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.774+0000 7fb7cf661740 1 --2- 172.21.15.123:0/217073382 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5651886cf240 0x5651886d1630 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.774+0000 7fb7cf661740 1 --2- 172.21.15.123:0/217073382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565188653e90 0x5651886cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.774+0000 7fb7cf661740 1 --2- 172.21.15.123:0/217073382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651886495d0 0x5651886584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.774+0000 7fb7cf661740 1 --2- 172.21.15.123:0/217073382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565188648cb0 0x565188657fa0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.774+0000 7fb7ab400640 0 client.26863 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:23.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.774+0000 7fb7cf661740 1 -- 172.21.15.123:0/217073382 >> 172.21.15.123:0/217073382 conn(0x565188645240 msgr2=0x565188643a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:23.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.774+0000 7fb7cf661740 1 -- 172.21.15.123:0/217073382 shutdown_connections 2024-09-19T04:54:23.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.774+0000 7fb7cf661740 1 -- 172.21.15.123:0/217073382 wait complete. 2024-09-19T04:54:23.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.774+0000 7fb7cf661740 20 client.26863 trim_cache size 0 max 16384 2024-09-19T04:54:23.786 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:54:23.786 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-09-19T04:54:23.786 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "e551e353-767f-402c-8cad-fa4bc882656c", 2024-09-19T04:54:23.787 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-09-19T04:54:23.787 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:54:23.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.775+0000 7fb7c7200640 1 -- 172.21.15.123:0/1891369757 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb7ac0a3430 msgr2=0x7fb7ac0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:23.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.775+0000 7fb7c7200640 1 --2- 172.21.15.123:0/1891369757 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb7ac0a3430 0x7fb7ac0a58f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0005fd0 tx=0x7fb7c0009500 comp rx=0 tx=0).stop 2024-09-19T04:54:23.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.775+0000 7fb7c7200640 1 -- 172.21.15.123:0/1891369757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c81034b0 msgr2=0x7fb7c8100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:23.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.775+0000 7fb7c7200640 1 --2- 172.21.15.123:0/1891369757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c81034b0 0x7fb7c8100d30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000e9f0 tx=0x7fb7b000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.775+0000 7fb7c7200640 1 -- 172.21.15.123:0/1891369757 shutdown_connections 2024-09-19T04:54:23.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.775+0000 7fb7c7200640 1 --2- 172.21.15.123:0/1891369757 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb7ac0a3430 0x7fb7ac0a58f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.775+0000 7fb7c7200640 1 --2- 172.21.15.123:0/1891369757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7c810bfc0 0x7fb7c8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.775+0000 7fb7c7200640 1 --2- 172.21.15.123:0/1891369757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c81034b0 0x7fb7c8100d30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.775+0000 7fb7c7200640 1 --2- 172.21.15.123:0/1891369757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c8102b90 0x7fb7c81007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:23.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.775+0000 7fb7c7200640 1 -- 172.21.15.123:0/1891369757 >> 172.21.15.123:0/1891369757 conn(0x7fb7c80fc610 msgr2=0x7fb7c80fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:23.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.775+0000 7fb7c7200640 1 -- 172.21.15.123:0/1891369757 shutdown_connections 2024-09-19T04:54:23.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:23.775+0000 7fb7c7200640 1 -- 172.21.15.123:0/1891369757 wait complete. 2024-09-19T04:54:23.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:23 smithi183 ceph-mon[35333]: pgmap v237: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:54:24.524 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:54:24.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:24 smithi193 ceph-mon[35592]: scrub queued for path: / 2024-09-19T04:54:24.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:24 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:24.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:24 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:54:24.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:24 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:24.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:24 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:54:24.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:24 smithi183 ceph-mon[35333]: scrub queued for path: / 2024-09-19T04:54:24.975 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:24 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:24.975 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:24 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:54:24.975 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:24 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:24.975 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:24 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:54:24.976 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:24.976 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:24 smithi123 ceph-mon[30511]: scrub queued for path: / 2024-09-19T04:54:24.976 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:24 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:24.976 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:24 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:54:24.976 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:24 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:24.976 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:24 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:54:25.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.453+0000 7f9501400640 1 -- 172.21.15.123:0/2053970744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94fc074620 msgr2=0x7f94fc074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:25.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.453+0000 7f9501400640 1 --2- 172.21.15.123:0/2053970744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94fc074620 0x7f94fc074a00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009900 tx=0x7f94ec02a480 comp rx=0 tx=0).stop 2024-09-19T04:54:25.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.453+0000 7f9501400640 1 -- 172.21.15.123:0/2053970744 shutdown_connections 2024-09-19T04:54:25.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.453+0000 7f9501400640 1 --2- 172.21.15.123:0/2053970744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94fc10e210 0x7f94fc110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.453+0000 7f9501400640 1 --2- 172.21.15.123:0/2053970744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94fc074f40 0x7f94fc10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.453+0000 7f9501400640 1 --2- 172.21.15.123:0/2053970744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94fc074620 0x7f94fc074a00 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.453+0000 7f9501400640 1 -- 172.21.15.123:0/2053970744 >> 172.21.15.123:0/2053970744 conn(0x7f94fc0fd710 msgr2=0x7f94fc0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:25.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.453+0000 7f9501400640 1 -- 172.21.15.123:0/2053970744 shutdown_connections 2024-09-19T04:54:25.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.453+0000 7f9501400640 1 -- 172.21.15.123:0/2053970744 wait complete. 2024-09-19T04:54:25.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.454+0000 7f9501400640 1 Processor -- start 2024-09-19T04:54:25.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.454+0000 7f9501400640 1 -- start start 2024-09-19T04:54:25.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.454+0000 7f9501400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94fc074620 0x7f94fc101930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.454+0000 7f9501400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94fc074f40 0x7f94fc101e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.454+0000 7f9501400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94fc10e210 0x7f94fc1023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.454+0000 7f9501400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc113fc0 con 0x7f94fc074620 2024-09-19T04:54:25.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.454+0000 7f9501400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc113e40 con 0x7f94fc10e210 2024-09-19T04:54:25.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.454+0000 7f9501400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc114140 con 0x7f94fc074f40 2024-09-19T04:54:25.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.455+0000 7f94fb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94fc074f40 0x7f94fc101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.455+0000 7f94fb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94fc074f40 0x7f94fc101e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:39456/0 (socket says 172.21.15.123:39456) 2024-09-19T04:54:25.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.455+0000 7f94fb400640 1 -- 172.21.15.123:0/999035765 learned_addr learned my addr 172.21.15.123:0/999035765 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:25.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.455+0000 7f94fbe00640 1 --2- 172.21.15.123:0/999035765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94fc074620 0x7f94fc101930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.455+0000 7f9500a00640 1 --2- 172.21.15.123:0/999035765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94fc10e210 0x7f94fc1023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.455+0000 7f94fb400640 1 -- 172.21.15.123:0/999035765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94fc10e210 msgr2=0x7f94fc1023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:25.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.455+0000 7f94fb400640 1 --2- 172.21.15.123:0/999035765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94fc10e210 0x7f94fc1023b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.455+0000 7f94fb400640 1 -- 172.21.15.123:0/999035765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94fc074620 msgr2=0x7f94fc101930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:25.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.455+0000 7f94fb400640 1 --2- 172.21.15.123:0/999035765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94fc074620 0x7f94fc101930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.455+0000 7f94fb400640 1 -- 172.21.15.123:0/999035765 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94fc1aab80 con 0x7f94fc074f40 2024-09-19T04:54:25.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.455+0000 7f9500a00640 1 --2- 172.21.15.123:0/999035765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94fc10e210 0x7f94fc1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:54:25.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.455+0000 7f94fb400640 1 --2- 172.21.15.123:0/999035765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94fc074f40 0x7f94fc101e70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f94e800ea20 tx=0x7f94e800eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:25.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.456+0000 7f94f8c00640 1 -- 172.21.15.123:0/999035765 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e800cd80 con 0x7f94fc074f40 2024-09-19T04:54:25.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.456+0000 7f94f8c00640 1 -- 172.21.15.123:0/999035765 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e8010910 con 0x7f94fc074f40 2024-09-19T04:54:25.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.456+0000 7f9501400640 1 -- 172.21.15.123:0/999035765 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94fc1aae70 con 0x7f94fc074f40 2024-09-19T04:54:25.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.456+0000 7f94f8c00640 1 -- 172.21.15.123:0/999035765 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e8018920 con 0x7f94fc074f40 2024-09-19T04:54:25.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.456+0000 7f9501400640 1 -- 172.21.15.123:0/999035765 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94fc1ab320 con 0x7f94fc074f40 2024-09-19T04:54:25.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.460+0000 7f950540e740 1 Processor -- start 2024-09-19T04:54:25.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.460+0000 7f950540e740 1 -- start start 2024-09-19T04:54:25.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.461+0000 7f94f8c00640 1 -- 172.21.15.123:0/999035765 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f94e8010ac0 con 0x7f94fc074f40 2024-09-19T04:54:25.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.461+0000 7f950540e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 0x5600bb87c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.461+0000 7f950540e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb87c670 0x5600bb8869f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.461+0000 7f950540e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb886f30 0x5600bb889320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.461+0000 7f950540e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5600bb7e23e0 con 0x5600bb87bd50 2024-09-19T04:54:25.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.461+0000 7f950540e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5600bb789d80 con 0x5600bb886f30 2024-09-19T04:54:25.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.461+0000 7f950540e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5600bb7de3a0 con 0x5600bb87c670 2024-09-19T04:54:25.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.461+0000 7f9500a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 0x5600bb87c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.461+0000 7f9500a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 0x5600bb87c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44738/0 (socket says 172.21.15.123:44738) 2024-09-19T04:54:25.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.461+0000 7f9500a00640 1 -- 172.21.15.123:0/68772856 learned_addr learned my addr 172.21.15.123:0/68772856 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:25.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.461+0000 7f94fbe00640 1 --2- 172.21.15.123:0/68772856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb87c670 0x5600bb8869f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.461+0000 7f94fb400640 1 --2- 172.21.15.123:0/68772856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb886f30 0x5600bb889320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.462+0000 7f9500a00640 1 -- 172.21.15.123:0/68772856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb87c670 msgr2=0x5600bb8869f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:25.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.462+0000 7f9500a00640 1 --2- 172.21.15.123:0/68772856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb87c670 0x5600bb8869f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.462+0000 7f9500a00640 1 -- 172.21.15.123:0/68772856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb886f30 msgr2=0x5600bb889320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:25.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.462+0000 7f9500a00640 1 --2- 172.21.15.123:0/68772856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb886f30 0x5600bb889320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.462+0000 7f9500a00640 1 -- 172.21.15.123:0/68772856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600bb7a8ae0 con 0x5600bb87bd50 2024-09-19T04:54:25.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.462+0000 7f94fb400640 1 --2- 172.21.15.123:0/68772856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb886f30 0x5600bb889320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:54:25.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.462+0000 7f9500a00640 1 --2- 172.21.15.123:0/68772856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 0x5600bb87c130 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009fd0 tx=0x7f94f400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41aa422c1b14eeac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:25.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.462+0000 7f94f1200640 1 -- 172.21.15.123:0/68772856 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4015070 con 0x5600bb87bd50 2024-09-19T04:54:25.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.463+0000 7f94f1200640 1 -- 172.21.15.123:0/68772856 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94f4011be0 con 0x5600bb87bd50 2024-09-19T04:54:25.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.463+0000 7f94f1200640 1 -- 172.21.15.123:0/68772856 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4010c40 con 0x5600bb87bd50 2024-09-19T04:54:25.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.463+0000 7f94f8c00640 1 --2- 172.21.15.123:0/999035765 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94e00a33b0 0x7f94e00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.463+0000 7f94f8c00640 1 -- 172.21.15.123:0/999035765 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f94e80d0120 con 0x7f94fc074f40 2024-09-19T04:54:25.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.463+0000 7f94fbe00640 1 --2- 172.21.15.123:0/999035765 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94e00a33b0 0x7f94e00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.463+0000 7f950540e740 1 -- 172.21.15.123:0/68772856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 msgr2=0x5600bb87c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:25.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.463+0000 7f950540e740 1 --2- 172.21.15.123:0/68772856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 0x5600bb87c130 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009fd0 tx=0x7f94f400f7f0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.464+0000 7f950540e740 1 -- 172.21.15.123:0/68772856 shutdown_connections 2024-09-19T04:54:25.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.464+0000 7f950540e740 1 --2- 172.21.15.123:0/68772856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb886f30 0x5600bb889320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.464+0000 7f950540e740 1 --2- 172.21.15.123:0/68772856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb87c670 0x5600bb8869f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.464+0000 7f950540e740 1 --2- 172.21.15.123:0/68772856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 0x5600bb87c130 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.464+0000 7f950540e740 1 -- 172.21.15.123:0/68772856 >> 172.21.15.123:0/68772856 conn(0x5600bb8782e0 msgr2=0x5600bb8786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:25.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.464+0000 7f950540e740 1 -- 172.21.15.123:0/68772856 shutdown_connections 2024-09-19T04:54:25.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.464+0000 7f950540e740 1 -- 172.21.15.123:0/68772856 wait complete. 2024-09-19T04:54:25.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.464+0000 7f94fbe00640 1 --2- 172.21.15.123:0/999035765 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94e00a33b0 0x7f94e00a5870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009230 tx=0x7f94ec0058d0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:25.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.464+0000 7f950540e740 1 -- start start 2024-09-19T04:54:25.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.465+0000 7f950540e740 1 Processor -- start 2024-09-19T04:54:25.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.465+0000 7f950540e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 0x5600bb883df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.465+0000 7f950540e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb87c670 0x5600bb884330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.465+0000 7f950540e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb886f30 0x5600bb884870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.465+0000 7f950540e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5600bb889ba0 con 0x5600bb87bd50 2024-09-19T04:54:25.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.465+0000 7f950540e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5600bb79d0f0 con 0x5600bb87c670 2024-09-19T04:54:25.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.465+0000 7f950540e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5600bb889d20 con 0x5600bb886f30 2024-09-19T04:54:25.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.465+0000 7f950540e740 10 client.0 _ll_register_callbacks cb 0x5600bb78e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:25.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.465+0000 7f94fbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 0x5600bb883df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.465+0000 7f94fbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 0x5600bb883df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44746/0 (socket says 172.21.15.123:44746) 2024-09-19T04:54:25.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.465+0000 7f94fbe00640 1 -- 172.21.15.123:0/1036205295 learned_addr learned my addr 172.21.15.123:0/1036205295 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:25.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.465+0000 7f94fb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb87c670 0x5600bb884330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.466+0000 7f9500a00640 1 --2- 172.21.15.123:0/1036205295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb886f30 0x5600bb884870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.466+0000 7f94fb400640 1 -- 172.21.15.123:0/1036205295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb886f30 msgr2=0x5600bb884870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:25.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.466+0000 7f94fb400640 1 --2- 172.21.15.123:0/1036205295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb886f30 0x5600bb884870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.466+0000 7f94fb400640 1 -- 172.21.15.123:0/1036205295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 msgr2=0x5600bb883df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:25.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.466+0000 7f94fb400640 1 --2- 172.21.15.123:0/1036205295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 0x5600bb883df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.466+0000 7f94fb400640 1 -- 172.21.15.123:0/1036205295 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600bb5533c0 con 0x5600bb87c670 2024-09-19T04:54:25.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.466+0000 7f94fb400640 1 --2- 172.21.15.123:0/1036205295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb87c670 0x5600bb884330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f94e80cd870 tx=0x7f94e809c1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fff6ffbbe246f909 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:25.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.466+0000 7f94e5600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:54:25.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.466+0000 7f94e5600640 1 -- 172.21.15.123:0/1036205295 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e80d3d40 con 0x5600bb87c670 2024-09-19T04:54:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.467+0000 7f950540e740 1 -- 172.21.15.123:0/1036205295 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94f4011be0 con 0x5600bb87c670 2024-09-19T04:54:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.467+0000 7f94e5600640 1 -- 172.21.15.123:0/1036205295 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f94e809b850 con 0x5600bb87c670 2024-09-19T04:54:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.467+0000 7f94e5600640 1 -- 172.21.15.123:0/1036205295 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f94e809cb00 con 0x5600bb87c670 2024-09-19T04:54:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.467+0000 7f950540e740 10 client.26849 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.467+0000 7f950540e740 1 -- 172.21.15.123:0/1036205295 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5600bb8958d0 con 0x5600bb87c670 2024-09-19T04:54:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.467+0000 7f94e5600640 1 -- 172.21.15.123:0/1036205295 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94f4011be0 con 0x5600bb87c670 2024-09-19T04:54:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.467+0000 7f94e5600640 1 -- 172.21.15.123:0/1036205295 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f94e8010430 con 0x5600bb87c670 2024-09-19T04:54:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.467+0000 7f950540e740 10 client.26849 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.467+0000 7f950540e740 10 client.26849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.467+0000 7f950540e740 20 client.26849 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.467+0000 7f950540e740 1 --2- 172.21.15.123:0/1036205295 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5600bb896c20 0x5600bb899010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.467+0000 7f950540e740 4 client.26849 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:25.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.467+0000 7f950540e740 1 -- 172.21.15.123:0/1036205295 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600bb7ef950 con 0x5600bb896c20 2024-09-19T04:54:25.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.468+0000 7f94fbe00640 1 --2- 172.21.15.123:0/1036205295 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5600bb896c20 0x5600bb899010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.468+0000 7f94fbe00640 1 --2- 172.21.15.123:0/1036205295 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5600bb896c20 0x5600bb899010 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d167decaaf27610 server_cookie=77fda5fbfb43b22b in_seq=0 out_seq=0 2024-09-19T04:54:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.468+0000 7f94e5600640 10 client.26849 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.469+0000 7f94e5600640 1 -- 172.21.15.123:0/1036205295 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600bb7ef950 con 0x5600bb896c20 2024-09-19T04:54:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.469+0000 7f94e5600640 10 client.26849 handle_command_reply: tid=0 2024-09-19T04:54:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.469+0000 7f950540e740 1 client.26849 shutdown 2024-09-19T04:54:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.469+0000 7f950540e740 1 -- 172.21.15.123:0/1036205295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb87c670 msgr2=0x5600bb884330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.469+0000 7f950540e740 1 --2- 172.21.15.123:0/1036205295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb87c670 0x5600bb884330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f94e80cd870 tx=0x7f94e809c1d0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.470+0000 7f950540e740 1 -- 172.21.15.123:0/1036205295 shutdown_connections 2024-09-19T04:54:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.470+0000 7f950540e740 1 --2- 172.21.15.123:0/1036205295 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5600bb896c20 0x5600bb899010 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.470+0000 7f950540e740 1 --2- 172.21.15.123:0/1036205295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb886f30 0x5600bb884870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.470+0000 7f950540e740 1 --2- 172.21.15.123:0/1036205295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb87c670 0x5600bb884330 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.470+0000 7f950540e740 1 --2- 172.21.15.123:0/1036205295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 0x5600bb883df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.470+0000 7f94e5600640 0 client.26849 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:25.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.470+0000 7f950540e740 1 -- 172.21.15.123:0/1036205295 >> 172.21.15.123:0/1036205295 conn(0x5600bb8782e0 msgr2=0x5600bb8773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:25.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.470+0000 7f950540e740 1 -- 172.21.15.123:0/1036205295 shutdown_connections 2024-09-19T04:54:25.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.470+0000 7f950540e740 1 -- 172.21.15.123:0/1036205295 wait complete. 2024-09-19T04:54:25.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.470+0000 7f950540e740 20 client.26849 trim_cache size 0 max 16384 2024-09-19T04:54:25.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.488+0000 7f950540e740 1 Processor -- start 2024-09-19T04:54:25.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.488+0000 7f950540e740 1 -- start start 2024-09-19T04:54:25.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.489+0000 7f950540e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb87bd50 0x5600bb87dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.489+0000 7f950540e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87c670 0x5600bb885d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.489+0000 7f950540e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb886f30 0x5600bb89e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.489+0000 7f950540e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc1b3df0 con 0x5600bb87c670 2024-09-19T04:54:25.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.489+0000 7f950540e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc1b59c0 con 0x5600bb886f30 2024-09-19T04:54:25.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.489+0000 7f950540e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc1b44f0 con 0x5600bb87bd50 2024-09-19T04:54:25.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.489+0000 7f94fb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87c670 0x5600bb885d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.489+0000 7f94fbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb87bd50 0x5600bb87dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.489+0000 7f94fb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87c670 0x5600bb885d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44756/0 (socket says 172.21.15.123:44756) 2024-09-19T04:54:25.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.489+0000 7f94fbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb87bd50 0x5600bb87dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:39468/0 (socket says 172.21.15.123:39468) 2024-09-19T04:54:25.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.489+0000 7f94fb400640 1 -- 172.21.15.123:0/595849005 learned_addr learned my addr 172.21.15.123:0/595849005 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:25.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.489+0000 7f9500a00640 1 --2- 172.21.15.123:0/595849005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb886f30 0x5600bb89e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.489+0000 7f94fbe00640 1 -- 172.21.15.123:0/595849005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb886f30 msgr2=0x5600bb89e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:25.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.489+0000 7f94fbe00640 1 --2- 172.21.15.123:0/595849005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb886f30 0x5600bb89e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.489+0000 7f94fbe00640 1 -- 172.21.15.123:0/595849005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87c670 msgr2=0x5600bb885d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:25.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.489+0000 7f94fbe00640 1 --2- 172.21.15.123:0/595849005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87c670 0x5600bb885d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.489+0000 7f94fbe00640 1 -- 172.21.15.123:0/595849005 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600bb87bb40 con 0x5600bb87bd50 2024-09-19T04:54:25.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.490+0000 7f94fb400640 1 --2- 172.21.15.123:0/595849005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87c670 0x5600bb885d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:25.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.490+0000 7f94fbe00640 1 --2- 172.21.15.123:0/595849005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb87bd50 0x5600bb87dfc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x5600bb8997d0 tx=0x7f94ec03f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61d0f48133865cde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:25.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.490+0000 7f94e7e00640 1 -- 172.21.15.123:0/595849005 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec035040 con 0x5600bb87bd50 2024-09-19T04:54:25.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.490+0000 7f94e7e00640 1 -- 172.21.15.123:0/595849005 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94ec0543a0 con 0x5600bb87bd50 2024-09-19T04:54:25.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.490+0000 7f94e7e00640 1 -- 172.21.15.123:0/595849005 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec03d6b0 con 0x5600bb87bd50 2024-09-19T04:54:25.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.491+0000 7f950540e740 1 -- 172.21.15.123:0/595849005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb87bd50 msgr2=0x5600bb87dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:25.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.491+0000 7f950540e740 1 --2- 172.21.15.123:0/595849005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb87bd50 0x5600bb87dfc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x5600bb8997d0 tx=0x7f94ec03f000 comp rx=0 tx=0).stop 2024-09-19T04:54:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.491+0000 7f950540e740 1 -- 172.21.15.123:0/595849005 shutdown_connections 2024-09-19T04:54:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.491+0000 7f950540e740 1 --2- 172.21.15.123:0/595849005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb886f30 0x5600bb89e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.491+0000 7f950540e740 1 --2- 172.21.15.123:0/595849005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87c670 0x5600bb885d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.491+0000 7f950540e740 1 --2- 172.21.15.123:0/595849005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb87bd50 0x5600bb87dfc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.491+0000 7f950540e740 1 -- 172.21.15.123:0/595849005 >> 172.21.15.123:0/595849005 conn(0x5600bb8782e0 msgr2=0x5600bb885650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.491+0000 7f950540e740 1 -- 172.21.15.123:0/595849005 shutdown_connections 2024-09-19T04:54:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.491+0000 7f950540e740 1 -- 172.21.15.123:0/595849005 wait complete. 2024-09-19T04:54:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.492+0000 7f950540e740 1 -- start start 2024-09-19T04:54:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.492+0000 7f950540e740 1 Processor -- start 2024-09-19T04:54:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.492+0000 7f950540e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 0x5600bb88b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.492+0000 7f950540e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb87c670 0x5600bb88b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.492+0000 7f950540e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb886f30 0x5600bb900860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.492+0000 7f950540e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc1b52f0 con 0x5600bb87bd50 2024-09-19T04:54:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.492+0000 7f950540e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc1b4bf0 con 0x5600bb886f30 2024-09-19T04:54:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.492+0000 7f950540e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc1b1df0 con 0x5600bb87c670 2024-09-19T04:54:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.492+0000 7f950540e740 10 client.0 _ll_register_callbacks cb 0x7f94e80d3980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.493+0000 7f94fbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 0x5600bb88b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.493+0000 7f94fbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 0x5600bb88b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44764/0 (socket says 172.21.15.123:44764) 2024-09-19T04:54:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.493+0000 7f94fbe00640 1 -- 172.21.15.123:0/1769828161 learned_addr learned my addr 172.21.15.123:0/1769828161 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.493+0000 7f9500a00640 1 --2- 172.21.15.123:0/1769828161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb886f30 0x5600bb900860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.493+0000 7f94fb400640 1 --2- 172.21.15.123:0/1769828161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb87c670 0x5600bb88b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.493+0000 7f94fbe00640 1 -- 172.21.15.123:0/1769828161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb87c670 msgr2=0x5600bb88b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.493+0000 7f94fbe00640 1 --2- 172.21.15.123:0/1769828161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb87c670 0x5600bb88b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.493+0000 7f94fbe00640 1 -- 172.21.15.123:0/1769828161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb886f30 msgr2=0x5600bb900860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.493+0000 7f94fbe00640 1 --2- 172.21.15.123:0/1769828161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb886f30 0x5600bb900860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.493+0000 7f94fbe00640 1 -- 172.21.15.123:0/1769828161 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600bb900da0 con 0x5600bb87bd50 2024-09-19T04:54:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.493+0000 7f9500a00640 1 --2- 172.21.15.123:0/1769828161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb886f30 0x5600bb900860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:54:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.493+0000 7f94fbe00640 1 --2- 172.21.15.123:0/1769828161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 0x5600bb88b3c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0096c0 tx=0x7f94ec0314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef425b7c1e12520 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.493+0000 7f94e5600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.494+0000 7f94e5600640 1 -- 172.21.15.123:0/1769828161 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec062060 con 0x5600bb87bd50 2024-09-19T04:54:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.494+0000 7f950540e740 1 -- 172.21.15.123:0/1769828161 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94ec0543a0 con 0x5600bb87bd50 2024-09-19T04:54:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.494+0000 7f94e5600640 1 -- 172.21.15.123:0/1769828161 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f94ec003bd0 con 0x5600bb87bd50 2024-09-19T04:54:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.494+0000 7f94e5600640 1 -- 172.21.15.123:0/1769828161 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f94ec003dd0 con 0x5600bb87bd50 2024-09-19T04:54:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.494+0000 7f950540e740 10 client.16761 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.494+0000 7f950540e740 1 -- 172.21.15.123:0/1769828161 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5600bb902010 con 0x5600bb87bd50 2024-09-19T04:54:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.494+0000 7f94e5600640 1 -- 172.21.15.123:0/1769828161 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94ec0543a0 con 0x5600bb87bd50 2024-09-19T04:54:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.494+0000 7f94e5600640 1 -- 172.21.15.123:0/1769828161 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f94ec05d050 con 0x5600bb87bd50 2024-09-19T04:54:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.495+0000 7f950540e740 10 client.16761 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.495+0000 7f950540e740 10 client.16761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:25.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.495+0000 7f950540e740 20 client.16761 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:25.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.495+0000 7f950540e740 1 --2- 172.21.15.123:0/1769828161 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5600bb9026d0 0x5600bb904ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:25.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.495+0000 7f950540e740 4 client.16761 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:54:25.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.495+0000 7f950540e740 1 -- 172.21.15.123:0/1769828161 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5600bb7e9e60 con 0x5600bb9026d0 2024-09-19T04:54:25.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.495+0000 7f94fb400640 1 --2- 172.21.15.123:0/1769828161 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5600bb9026d0 0x5600bb904ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:25.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.496+0000 7f94fb400640 1 --2- 172.21.15.123:0/1769828161 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5600bb9026d0 0x5600bb904ac0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef57532bded1565c server_cookie=d5209ddecc3a2a91 in_seq=0 out_seq=0 2024-09-19T04:54:25.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.496+0000 7f94e5600640 10 client.16761 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:25.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.496+0000 7f94e5600640 1 -- 172.21.15.123:0/1769828161 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5600bb7e9e60 con 0x5600bb9026d0 2024-09-19T04:54:25.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.496+0000 7f94e5600640 10 client.16761 handle_command_reply: tid=0 2024-09-19T04:54:25.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.496+0000 7f950540e740 1 client.16761 shutdown 2024-09-19T04:54:25.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.496+0000 7f950540e740 1 -- 172.21.15.123:0/1769828161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 msgr2=0x5600bb88b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:25.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.496+0000 7f950540e740 1 --2- 172.21.15.123:0/1769828161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 0x5600bb88b3c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0096c0 tx=0x7f94ec0314e0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.496+0000 7f950540e740 1 -- 172.21.15.123:0/1769828161 shutdown_connections 2024-09-19T04:54:25.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.496+0000 7f950540e740 1 --2- 172.21.15.123:0/1769828161 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5600bb9026d0 0x5600bb904ac0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.496+0000 7f950540e740 1 --2- 172.21.15.123:0/1769828161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600bb886f30 0x5600bb900860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.496+0000 7f950540e740 1 --2- 172.21.15.123:0/1769828161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600bb87c670 0x5600bb88b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.496+0000 7f950540e740 1 --2- 172.21.15.123:0/1769828161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600bb87bd50 0x5600bb88b3c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.496+0000 7f94e5600640 0 client.16761 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:25.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.496+0000 7f950540e740 1 -- 172.21.15.123:0/1769828161 >> 172.21.15.123:0/1769828161 conn(0x5600bb8782e0 msgr2=0x5600bb885e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:25.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.496+0000 7f950540e740 1 -- 172.21.15.123:0/1769828161 shutdown_connections 2024-09-19T04:54:25.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.497+0000 7f950540e740 1 -- 172.21.15.123:0/1769828161 wait complete. 2024-09-19T04:54:25.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.497+0000 7f950540e740 20 client.16761 trim_cache size 0 max 16384 2024-09-19T04:54:25.503 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:54:25.503 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-09-19T04:54:25.503 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-09-19T04:54:25.503 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:54:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.497+0000 7f9501400640 1 -- 172.21.15.123:0/999035765 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94e00a33b0 msgr2=0x7f94e00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.497+0000 7f9501400640 1 --2- 172.21.15.123:0/999035765 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94e00a33b0 0x7f94e00a5870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009230 tx=0x7f94ec0058d0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.497+0000 7f9501400640 1 -- 172.21.15.123:0/999035765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94fc074f40 msgr2=0x7f94fc101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.497+0000 7f9501400640 1 --2- 172.21.15.123:0/999035765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94fc074f40 0x7f94fc101e70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f94e800ea20 tx=0x7f94e800eee0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.498+0000 7f9501400640 1 -- 172.21.15.123:0/999035765 shutdown_connections 2024-09-19T04:54:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.498+0000 7f9501400640 1 --2- 172.21.15.123:0/999035765 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f94e00a33b0 0x7f94e00a5870 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.498+0000 7f9501400640 1 --2- 172.21.15.123:0/999035765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94fc10e210 0x7f94fc1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.498+0000 7f9501400640 1 --2- 172.21.15.123:0/999035765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94fc074f40 0x7f94fc101e70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.498+0000 7f9501400640 1 --2- 172.21.15.123:0/999035765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94fc074620 0x7f94fc101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.498+0000 7f9501400640 1 -- 172.21.15.123:0/999035765 >> 172.21.15.123:0/999035765 conn(0x7f94fc0fd710 msgr2=0x7f94fc0feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.498+0000 7f9501400640 1 -- 172.21.15.123:0/999035765 shutdown_connections 2024-09-19T04:54:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:25.498+0000 7f9501400640 1 -- 172.21.15.123:0/999035765 wait complete. 2024-09-19T04:54:25.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:25 smithi123 ceph-mon[30511]: pgmap v238: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:54:25.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:25 smithi193 ceph-mon[35592]: pgmap v238: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:54:25.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:25 smithi183 ceph-mon[35333]: pgmap v238: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:54:26.301 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:54:26.301 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:54:26.693 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:27.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.248+0000 7fe4fbc00640 1 -- 172.21.15.123:0/2874655812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4f4101df0 msgr2=0x7fe4f41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:27.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.248+0000 7fe4fbc00640 1 --2- 172.21.15.123:0/2874655812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4f4101df0 0x7fe4f41021d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009900 tx=0x7fe4e802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:54:27.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.249+0000 7fe4fbc00640 1 -- 172.21.15.123:0/2874655812 shutdown_connections 2024-09-19T04:54:27.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.249+0000 7fe4fbc00640 1 --2- 172.21.15.123:0/2874655812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4f4106f20 0x7fe4f410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:27.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.249+0000 7fe4fbc00640 1 --2- 172.21.15.123:0/2874655812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4f41027a0 0x7fe4f41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:27.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.249+0000 7fe4fbc00640 1 --2- 172.21.15.123:0/2874655812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4f4101df0 0x7fe4f41021d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:27.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.249+0000 7fe4fbc00640 1 -- 172.21.15.123:0/2874655812 >> 172.21.15.123:0/2874655812 conn(0x7fe4f40fd8b0 msgr2=0x7fe4f40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:27.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.249+0000 7fe4fbc00640 1 -- 172.21.15.123:0/2874655812 shutdown_connections 2024-09-19T04:54:27.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.249+0000 7fe4fbc00640 1 -- 172.21.15.123:0/2874655812 wait complete. 2024-09-19T04:54:27.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.250+0000 7fe4fbc00640 1 Processor -- start 2024-09-19T04:54:27.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.250+0000 7fe4fbc00640 1 -- start start 2024-09-19T04:54:27.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.250+0000 7fe4fbc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4f4101df0 0x7fe4f41983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:27.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.250+0000 7fe4fbc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4f41027a0 0x7fe4f4198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:27.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.250+0000 7fe4fbc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4f4106f20 0x7fe4f419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:27.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.250+0000 7fe4fbc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f4111fd0 con 0x7fe4f4101df0 2024-09-19T04:54:27.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.250+0000 7fe4fbc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f4111e50 con 0x7fe4f4106f20 2024-09-19T04:54:27.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.250+0000 7fe4fbc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f4112150 con 0x7fe4f41027a0 2024-09-19T04:54:27.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.250+0000 7fe4f9200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4f4101df0 0x7fe4f41983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:27.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.250+0000 7fe4f9200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4f4101df0 0x7fe4f41983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44784/0 (socket says 172.21.15.123:44784) 2024-09-19T04:54:27.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.250+0000 7fe4f9200640 1 -- 172.21.15.123:0/1630517544 learned_addr learned my addr 172.21.15.123:0/1630517544 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:27.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.251+0000 7fe4f3e00640 1 --2- 172.21.15.123:0/1630517544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4f41027a0 0x7fe4f4198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:27.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.251+0000 7fe4f9c00640 1 --2- 172.21.15.123:0/1630517544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4f4106f20 0x7fe4f419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:27.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.251+0000 7fe4f9200640 1 -- 172.21.15.123:0/1630517544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4f41027a0 msgr2=0x7fe4f4198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:27.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.251+0000 7fe4f9200640 1 --2- 172.21.15.123:0/1630517544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4f41027a0 0x7fe4f4198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:27.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.251+0000 7fe4f9200640 1 -- 172.21.15.123:0/1630517544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4f4106f20 msgr2=0x7fe4f419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:27.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.251+0000 7fe4f9200640 1 --2- 172.21.15.123:0/1630517544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4f4106f20 0x7fe4f419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:27.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.251+0000 7fe4f9200640 1 -- 172.21.15.123:0/1630517544 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f419d3a0 con 0x7fe4f4101df0 2024-09-19T04:54:27.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.251+0000 7fe4f9200640 1 --2- 172.21.15.123:0/1630517544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4f4101df0 0x7fe4f41983f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8004770 tx=0x7fe4e8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:27.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.252+0000 7fe4f1600640 1 -- 172.21.15.123:0/1630517544 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e803f070 con 0x7fe4f4101df0 2024-09-19T04:54:27.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.252+0000 7fe4f1600640 1 -- 172.21.15.123:0/1630517544 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e8031ca0 con 0x7fe4f4101df0 2024-09-19T04:54:27.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.252+0000 7fe4fbc00640 1 -- 172.21.15.123:0/1630517544 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f419d630 con 0x7fe4f4101df0 2024-09-19T04:54:27.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.252+0000 7fe4f1600640 1 -- 172.21.15.123:0/1630517544 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8035030 con 0x7fe4f4101df0 2024-09-19T04:54:27.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.252+0000 7fe4fbc00640 1 -- 172.21.15.123:0/1630517544 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f41a4f70 con 0x7fe4f4101df0 2024-09-19T04:54:27.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.253+0000 7fe4fbc00640 1 -- 172.21.15.123:0/1630517544 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f41032d0 con 0x7fe4f4101df0 2024-09-19T04:54:27.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.255+0000 7fe4f1600640 1 -- 172.21.15.123:0/1630517544 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe4e8042070 con 0x7fe4f4101df0 2024-09-19T04:54:27.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.256+0000 7fe4f1600640 1 --2- 172.21.15.123:0/1630517544 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe4dc0a33b0 0x7fe4dc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:27.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.256+0000 7fe4f1600640 1 -- 172.21.15.123:0/1630517544 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fe4e80f0340 con 0x7fe4f4101df0 2024-09-19T04:54:27.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.256+0000 7fe4f3e00640 1 --2- 172.21.15.123:0/1630517544 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe4dc0a33b0 0x7fe4dc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:27.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.257+0000 7fe4f3e00640 1 --2- 172.21.15.123:0/1630517544 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe4dc0a33b0 0x7fe4dc0a5870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40fdd40 tx=0x7fe4e0005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:27.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.260+0000 7fe4f1600640 1 -- 172.21.15.123:0/1630517544 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe4e80bc730 con 0x7fe4f4101df0 2024-09-19T04:54:27.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.472+0000 7fe4fbc00640 1 -- 172.21.15.123:0/1630517544 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4f410a100 con 0x7fe4f4101df0 2024-09-19T04:54:27.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.473+0000 7fe4f1600640 1 -- 172.21.15.123:0/1630517544 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11994 (secure 0 0 0) 0x7fe4e80c0a30 con 0x7fe4f4101df0 2024-09-19T04:54:27.475 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:54:27.475 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":40,"btime":"2024-09-19T04:52:39:230121+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:52:38.841872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24499,24500,24539,24505,24529]},"id":1}]} 2024-09-19T04:54:27.475 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 40 2024-09-19T04:54:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.475+0000 7fe4fbc00640 1 -- 172.21.15.123:0/1630517544 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe4dc0a33b0 msgr2=0x7fe4dc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.475+0000 7fe4fbc00640 1 --2- 172.21.15.123:0/1630517544 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe4dc0a33b0 0x7fe4dc0a5870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40fdd40 tx=0x7fe4e0005f90 comp rx=0 tx=0).stop 2024-09-19T04:54:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.475+0000 7fe4fbc00640 1 -- 172.21.15.123:0/1630517544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4f4101df0 msgr2=0x7fe4f41983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.475+0000 7fe4fbc00640 1 --2- 172.21.15.123:0/1630517544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4f4101df0 0x7fe4f41983f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8004770 tx=0x7fe4e8003c10 comp rx=0 tx=0).stop 2024-09-19T04:54:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.475+0000 7fe4fbc00640 1 -- 172.21.15.123:0/1630517544 shutdown_connections 2024-09-19T04:54:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.475+0000 7fe4fbc00640 1 --2- 172.21.15.123:0/1630517544 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe4dc0a33b0 0x7fe4dc0a5870 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.475+0000 7fe4fbc00640 1 --2- 172.21.15.123:0/1630517544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4f4106f20 0x7fe4f419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.475+0000 7fe4fbc00640 1 --2- 172.21.15.123:0/1630517544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4f41027a0 0x7fe4f4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.475+0000 7fe4fbc00640 1 --2- 172.21.15.123:0/1630517544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4f4101df0 0x7fe4f41983f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.475+0000 7fe4fbc00640 1 -- 172.21.15.123:0/1630517544 >> 172.21.15.123:0/1630517544 conn(0x7fe4f40fd8b0 msgr2=0x7fe4f40feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.476+0000 7fe4fbc00640 1 -- 172.21.15.123:0/1630517544 shutdown_connections 2024-09-19T04:54:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:27.476+0000 7fe4fbc00640 1 -- 172.21.15.123:0/1630517544 wait complete. 2024-09-19T04:54:27.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:27 smithi193 ceph-mon[35592]: pgmap v239: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:54:27.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:27 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1630517544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:54:27.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:27 smithi183 ceph-mon[35333]: pgmap v239: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:54:27.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:27 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1630517544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:54:27.861 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:27 smithi123 ceph-mon[30511]: pgmap v239: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:54:27.861 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:27 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1630517544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:54:28.262 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:54:28.635 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:29.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.192+0000 7fb015e00640 1 -- 172.21.15.123:0/3445960945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb010112b90 msgr2=0x7fb010114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:29.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.192+0000 7fb015e00640 1 --2- 172.21.15.123:0/3445960945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb010112b90 0x7fb010114f80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb008009900 tx=0x7fb00802a480 comp rx=0 tx=0).stop 2024-09-19T04:54:29.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.193+0000 7fb015e00640 1 -- 172.21.15.123:0/3445960945 shutdown_connections 2024-09-19T04:54:29.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.193+0000 7fb015e00640 1 --2- 172.21.15.123:0/3445960945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb010112b90 0x7fb010114f80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.193+0000 7fb015e00640 1 --2- 172.21.15.123:0/3445960945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb010074eb0 0x7fb010112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.193+0000 7fb015e00640 1 --2- 172.21.15.123:0/3445960945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb010074590 0x7fb010074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.193+0000 7fb015e00640 1 -- 172.21.15.123:0/3445960945 >> 172.21.15.123:0/3445960945 conn(0x7fb0100fd8c0 msgr2=0x7fb0100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:29.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.193+0000 7fb015e00640 1 -- 172.21.15.123:0/3445960945 shutdown_connections 2024-09-19T04:54:29.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.193+0000 7fb015e00640 1 -- 172.21.15.123:0/3445960945 wait complete. 2024-09-19T04:54:29.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.194+0000 7fb015e00640 1 Processor -- start 2024-09-19T04:54:29.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.194+0000 7fb015e00640 1 -- start start 2024-09-19T04:54:29.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb015e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb010074590 0x7fb01019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb015e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb010074eb0 0x7fb01019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb015e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb010112b90 0x7fb0101a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb015e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101185c0 con 0x7fb010112b90 2024-09-19T04:54:29.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb015e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb010118440 con 0x7fb010074590 2024-09-19T04:54:29.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb015e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb010118740 con 0x7fb010074eb0 2024-09-19T04:54:29.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb00e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb010074eb0 0x7fb01019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb00ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb010074590 0x7fb01019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb00e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb010074eb0 0x7fb01019f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:39514/0 (socket says 172.21.15.123:39514) 2024-09-19T04:54:29.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb00e000640 1 -- 172.21.15.123:0/3879574458 learned_addr learned my addr 172.21.15.123:0/3879574458 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:29.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb00ea00640 1 --2- 172.21.15.123:0/3879574458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb010074590 0x7fb01019eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:38754/0 (socket says 172.21.15.123:38754) 2024-09-19T04:54:29.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb00f400640 1 --2- 172.21.15.123:0/3879574458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb010112b90 0x7fb0101a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb00ea00640 1 -- 172.21.15.123:0/3879574458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb010074eb0 msgr2=0x7fb01019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:29.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb00ea00640 1 --2- 172.21.15.123:0/3879574458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb010074eb0 0x7fb01019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb00ea00640 1 -- 172.21.15.123:0/3879574458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb010112b90 msgr2=0x7fb0101a3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb00ea00640 1 --2- 172.21.15.123:0/3879574458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb010112b90 0x7fb0101a3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb00ea00640 1 -- 172.21.15.123:0/3879574458 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0101a3b90 con 0x7fb010074590 2024-09-19T04:54:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.195+0000 7fb00e000640 1 --2- 172.21.15.123:0/3879574458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb010074eb0 0x7fb01019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.196+0000 7fb00ea00640 1 --2- 172.21.15.123:0/3879574458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb010074590 0x7fb01019eb60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb00000eb00 tx=0x7fb00000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:29.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.196+0000 7fb014a00640 1 -- 172.21.15.123:0/3879574458 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000010430 con 0x7fb010074590 2024-09-19T04:54:29.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.196+0000 7fb015e00640 1 -- 172.21.15.123:0/3879574458 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0101a3e80 con 0x7fb010074590 2024-09-19T04:54:29.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.196+0000 7fb014a00640 1 -- 172.21.15.123:0/3879574458 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb000010ab0 con 0x7fb010074590 2024-09-19T04:54:29.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.196+0000 7fb014a00640 1 -- 172.21.15.123:0/3879574458 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000017bb0 con 0x7fb010074590 2024-09-19T04:54:29.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.196+0000 7fb015e00640 1 -- 172.21.15.123:0/3879574458 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0101ab710 con 0x7fb010074590 2024-09-19T04:54:29.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.199+0000 7fb018a64740 1 Processor -- start 2024-09-19T04:54:29.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.199+0000 7fb018a64740 1 -- start start 2024-09-19T04:54:29.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.200+0000 7fb018a64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c8f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.200+0000 7fb018a64740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8f590 0x564e41c99910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.200+0000 7fb018a64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c99e50 0x564e41c9c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.200+0000 7fb014a00640 1 -- 172.21.15.123:0/3879574458 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb000010c60 con 0x7fb010074590 2024-09-19T04:54:29.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.200+0000 7fb018a64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564e41bf53e0 con 0x564e41c8ec70 2024-09-19T04:54:29.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.200+0000 7fb018a64740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564e41b9cd80 con 0x564e41c8f590 2024-09-19T04:54:29.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.200+0000 7fb018a64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564e41bf13a0 con 0x564e41c99e50 2024-09-19T04:54:29.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.200+0000 7fb00f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c8f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.200+0000 7fb00f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c8f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44822/0 (socket says 172.21.15.123:44822) 2024-09-19T04:54:29.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.200+0000 7fb00f400640 1 -- 172.21.15.123:0/2838828005 learned_addr learned my addr 172.21.15.123:0/2838828005 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:29.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.200+0000 7fb00ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8f590 0x564e41c99910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.200+0000 7fb00e000640 1 --2- 172.21.15.123:0/2838828005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c99e50 0x564e41c9c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.200+0000 7fb00f400640 1 -- 172.21.15.123:0/2838828005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c99e50 msgr2=0x564e41c9c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:29.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.200+0000 7fb00f400640 1 --2- 172.21.15.123:0/2838828005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c99e50 0x564e41c9c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.200+0000 7fb00f400640 1 -- 172.21.15.123:0/2838828005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8f590 msgr2=0x564e41c99910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:29.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.200+0000 7fb00f400640 1 --2- 172.21.15.123:0/2838828005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8f590 0x564e41c99910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.200+0000 7fb00f400640 1 -- 172.21.15.123:0/2838828005 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e41bbbae0 con 0x564e41c8ec70 2024-09-19T04:54:29.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.201+0000 7fb00e000640 1 --2- 172.21.15.123:0/2838828005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c99e50 0x564e41c9c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:29.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.201+0000 7fb00f400640 1 --2- 172.21.15.123:0/2838828005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c8f050 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb008006010 tx=0x7fb008005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ad915c558f1499 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:29.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.201+0000 7faffbe00640 1 -- 172.21.15.123:0/2838828005 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008040070 con 0x564e41c8ec70 2024-09-19T04:54:29.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.201+0000 7faffbe00640 1 -- 172.21.15.123:0/2838828005 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb008004300 con 0x564e41c8ec70 2024-09-19T04:54:29.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.201+0000 7faffbe00640 1 -- 172.21.15.123:0/2838828005 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008035040 con 0x564e41c8ec70 2024-09-19T04:54:29.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.201+0000 7fb014a00640 1 --2- 172.21.15.123:0/3879574458 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7faff40a3430 0x7faff40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.202+0000 7fb014a00640 1 -- 172.21.15.123:0/3879574458 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fb0000d0670 con 0x7fb010074590 2024-09-19T04:54:29.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.202+0000 7fb00ea00640 1 --2- 172.21.15.123:0/3879574458 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7faff40a3430 0x7faff40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.202+0000 7fb018a64740 1 -- 172.21.15.123:0/2838828005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 msgr2=0x564e41c8f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:29.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.202+0000 7fb018a64740 1 --2- 172.21.15.123:0/2838828005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c8f050 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb008006010 tx=0x7fb008005be0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.202+0000 7fb018a64740 1 -- 172.21.15.123:0/2838828005 shutdown_connections 2024-09-19T04:54:29.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.202+0000 7fb018a64740 1 --2- 172.21.15.123:0/2838828005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c99e50 0x564e41c9c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.202+0000 7fb018a64740 1 --2- 172.21.15.123:0/2838828005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8f590 0x564e41c99910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.202+0000 7fb018a64740 1 --2- 172.21.15.123:0/2838828005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c8f050 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.202+0000 7fb018a64740 1 -- 172.21.15.123:0/2838828005 >> 172.21.15.123:0/2838828005 conn(0x564e41c8b200 msgr2=0x564e41c8b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:29.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.202+0000 7fb018a64740 1 -- 172.21.15.123:0/2838828005 shutdown_connections 2024-09-19T04:54:29.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.202+0000 7fb018a64740 1 -- 172.21.15.123:0/2838828005 wait complete. 2024-09-19T04:54:29.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.202+0000 7fb00ea00640 1 --2- 172.21.15.123:0/3879574458 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7faff40a3430 0x7faff40a58f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x564e41c8c060 tx=0x7fb0000d6530 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:29.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.203+0000 7fb018a64740 1 -- start start 2024-09-19T04:54:29.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.203+0000 7fb018a64740 1 Processor -- start 2024-09-19T04:54:29.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.203+0000 7fb018a64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c96d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.203+0000 7fb018a64740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8f590 0x564e41c97260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.203+0000 7fb018a64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c99e50 0x564e41c977a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.203+0000 7fb018a64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564e41c9ca20 con 0x564e41c8ec70 2024-09-19T04:54:29.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.203+0000 7fb018a64740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564e41bb00f0 con 0x564e41c8f590 2024-09-19T04:54:29.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.203+0000 7fb018a64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564e41c9cba0 con 0x564e41c99e50 2024-09-19T04:54:29.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.204+0000 7fb018a64740 10 client.0 _ll_register_callbacks cb 0x564e41b97bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:29.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.204+0000 7fb00f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c96d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.204+0000 7fb00f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c96d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44826/0 (socket says 172.21.15.123:44826) 2024-09-19T04:54:29.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.204+0000 7fb00f400640 1 -- 172.21.15.123:0/2330172535 learned_addr learned my addr 172.21.15.123:0/2330172535 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:29.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.204+0000 7fb00e000640 1 --2- 172.21.15.123:0/2330172535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8f590 0x564e41c97260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.204+0000 7fb00f400640 1 --2- 172.21.15.123:0/2330172535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c99e50 0x564e41c977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.204+0000 7fb00e000640 1 -- 172.21.15.123:0/2330172535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c99e50 msgr2=0x564e41c977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:29.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.204+0000 7fb00e000640 1 --2- 172.21.15.123:0/2330172535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c99e50 0x564e41c977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.204+0000 7fb00e000640 1 -- 172.21.15.123:0/2330172535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 msgr2=0x564e41c96d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:29.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.204+0000 7fb00e000640 1 --2- 172.21.15.123:0/2330172535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c96d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.204+0000 7fb00e000640 1 -- 172.21.15.123:0/2330172535 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e419663c0 con 0x564e41c8f590 2024-09-19T04:54:29.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.204+0000 7fb00f400640 1 --2- 172.21.15.123:0/2330172535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c96d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:29.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.205+0000 7fb00e000640 1 --2- 172.21.15.123:0/2330172535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8f590 0x564e41c97260 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x564e41c8c580 tx=0x7faffc00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85a2c0dd69bf6d26 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:29.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.205+0000 7faff8c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:54:29.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.205+0000 7faff8c00640 1 -- 172.21.15.123:0/2330172535 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc0120d0 con 0x564e41c8f590 2024-09-19T04:54:29.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.205+0000 7faff8c00640 1 -- 172.21.15.123:0/2330172535 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7faffc005020 con 0x564e41c8f590 2024-09-19T04:54:29.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.205+0000 7fb018a64740 1 -- 172.21.15.123:0/2330172535 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb008004300 con 0x564e41c8f590 2024-09-19T04:54:29.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.205+0000 7faff8c00640 1 -- 172.21.15.123:0/2330172535 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7faffc012270 con 0x564e41c8f590 2024-09-19T04:54:29.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.205+0000 7fb018a64740 10 client.26873 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:29.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.205+0000 7fb018a64740 1 -- 172.21.15.123:0/2330172535 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564e41ca8aa0 con 0x564e41c8f590 2024-09-19T04:54:29.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.206+0000 7faff8c00640 1 -- 172.21.15.123:0/2330172535 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb008004300 con 0x564e41c8f590 2024-09-19T04:54:29.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.206+0000 7faff8c00640 1 -- 172.21.15.123:0/2330172535 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7faffc025480 con 0x564e41c8f590 2024-09-19T04:54:29.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.206+0000 7fb018a64740 10 client.26873 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:29.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.206+0000 7fb018a64740 10 client.26873 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:54:29.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.206+0000 7fb018a64740 20 client.26873 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:29.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.206+0000 7fb018a64740 1 --2- 172.21.15.123:0/2330172535 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x564e41ca9eb0 0x564e41cac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.206+0000 7fb018a64740 4 client.26873 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:29.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.206+0000 7fb018a64740 1 -- 172.21.15.123:0/2330172535 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e41c02950 con 0x564e41ca9eb0 2024-09-19T04:54:29.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.206+0000 7fb00f400640 1 --2- 172.21.15.123:0/2330172535 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x564e41ca9eb0 0x564e41cac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.207+0000 7fb00f400640 1 --2- 172.21.15.123:0/2330172535 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x564e41ca9eb0 0x564e41cac2c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a57cbef7abca90fb server_cookie=a333fa58c6245bad in_seq=0 out_seq=0 2024-09-19T04:54:29.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.207+0000 7faff8c00640 10 client.26873 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:54:29.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.208+0000 7faff8c00640 1 -- 172.21.15.123:0/2330172535 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e41c02950 con 0x564e41ca9eb0 2024-09-19T04:54:29.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.208+0000 7faff8c00640 10 client.26873 handle_command_reply: tid=0 2024-09-19T04:54:29.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.208+0000 7fb018a64740 1 client.26873 shutdown 2024-09-19T04:54:29.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.208+0000 7fb018a64740 1 -- 172.21.15.123:0/2330172535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8f590 msgr2=0x564e41c97260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:29.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.208+0000 7fb018a64740 1 --2- 172.21.15.123:0/2330172535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8f590 0x564e41c97260 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x564e41c8c580 tx=0x7faffc00f790 comp rx=0 tx=0).stop 2024-09-19T04:54:29.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.208+0000 7fb018a64740 1 -- 172.21.15.123:0/2330172535 shutdown_connections 2024-09-19T04:54:29.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.208+0000 7fb018a64740 1 --2- 172.21.15.123:0/2330172535 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x564e41ca9eb0 0x564e41cac2c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.208+0000 7fb018a64740 1 --2- 172.21.15.123:0/2330172535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c99e50 0x564e41c977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.208+0000 7fb018a64740 1 --2- 172.21.15.123:0/2330172535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8f590 0x564e41c97260 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.208+0000 7fb018a64740 1 --2- 172.21.15.123:0/2330172535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c96d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.208+0000 7faff8c00640 0 client.26873 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:54:29.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.208+0000 7fb018a64740 1 -- 172.21.15.123:0/2330172535 >> 172.21.15.123:0/2330172535 conn(0x564e41c8b200 msgr2=0x564e41c90f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:29.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.208+0000 7fb018a64740 1 -- 172.21.15.123:0/2330172535 shutdown_connections 2024-09-19T04:54:29.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.208+0000 7fb018a64740 1 -- 172.21.15.123:0/2330172535 wait complete. 2024-09-19T04:54:29.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.208+0000 7fb018a64740 20 client.26873 trim_cache size 0 max 16384 2024-09-19T04:54:29.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.222+0000 7fb018a64740 1 Processor -- start 2024-09-19T04:54:29.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.222+0000 7fb018a64740 1 -- start start 2024-09-19T04:54:29.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.222+0000 7fb018a64740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8b200 0x564e41c916d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.222+0000 7fb018a64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c98c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.222+0000 7fb018a64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c8f590 0x564e41cb1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.222+0000 7fb018a64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101b3f90 con 0x564e41c8ec70 2024-09-19T04:54:29.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.222+0000 7fb018a64740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101b5b60 con 0x564e41c8b200 2024-09-19T04:54:29.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.222+0000 7fb018a64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101b4690 con 0x564e41c8f590 2024-09-19T04:54:29.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.222+0000 7fb00e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c98c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.223+0000 7fb00e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c98c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44840/0 (socket says 172.21.15.123:44840) 2024-09-19T04:54:29.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.223+0000 7fb00e000640 1 -- 172.21.15.123:0/3366141309 learned_addr learned my addr 172.21.15.123:0/3366141309 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:29.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.223+0000 7fb00f400640 1 --2- 172.21.15.123:0/3366141309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8b200 0x564e41c916d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.223+0000 7fb00f400640 1 --2- 172.21.15.123:0/3366141309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c8f590 0x564e41cb1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.223+0000 7fb00e000640 1 -- 172.21.15.123:0/3366141309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c8f590 msgr2=0x564e41cb1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:29.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.223+0000 7fb00e000640 1 --2- 172.21.15.123:0/3366141309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c8f590 0x564e41cb1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.223+0000 7fb00e000640 1 -- 172.21.15.123:0/3366141309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8b200 msgr2=0x564e41c916d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:29.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.223+0000 7fb00e000640 1 --2- 172.21.15.123:0/3366141309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8b200 0x564e41c916d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.223+0000 7fb00e000640 1 -- 172.21.15.123:0/3366141309 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e41c8be90 con 0x564e41c8ec70 2024-09-19T04:54:29.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.223+0000 7fb00f400640 1 --2- 172.21.15.123:0/3366141309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8b200 0x564e41c916d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:29.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.223+0000 7fb00e000640 1 --2- 172.21.15.123:0/3366141309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c98c90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faffc025a40 tx=0x7faffc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdef0bfcba3fc4be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:29.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.223+0000 7faffb400640 1 -- 172.21.15.123:0/3366141309 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc000e60 con 0x564e41c8ec70 2024-09-19T04:54:29.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.224+0000 7faffb400640 1 -- 172.21.15.123:0/3366141309 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faffc02a030 con 0x564e41c8ec70 2024-09-19T04:54:29.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.224+0000 7faffb400640 1 -- 172.21.15.123:0/3366141309 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc004680 con 0x564e41c8ec70 2024-09-19T04:54:29.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.224+0000 7fb018a64740 1 -- 172.21.15.123:0/3366141309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 msgr2=0x564e41c98c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:29.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.224+0000 7fb018a64740 1 --2- 172.21.15.123:0/3366141309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c98c90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faffc025a40 tx=0x7faffc009c30 comp rx=0 tx=0).stop 2024-09-19T04:54:29.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.224+0000 7fb018a64740 1 -- 172.21.15.123:0/3366141309 shutdown_connections 2024-09-19T04:54:29.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.224+0000 7fb018a64740 1 --2- 172.21.15.123:0/3366141309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c8f590 0x564e41cb1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.224+0000 7fb018a64740 1 --2- 172.21.15.123:0/3366141309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8ec70 0x564e41c98c90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.224+0000 7fb018a64740 1 --2- 172.21.15.123:0/3366141309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8b200 0x564e41c916d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.224+0000 7fb018a64740 1 -- 172.21.15.123:0/3366141309 >> 172.21.15.123:0/3366141309 conn(0x564e41c92710 msgr2=0x564e41c98580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:29.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.224+0000 7fb018a64740 1 -- 172.21.15.123:0/3366141309 shutdown_connections 2024-09-19T04:54:29.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.224+0000 7fb018a64740 1 -- 172.21.15.123:0/3366141309 wait complete. 2024-09-19T04:54:29.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.225+0000 7fb018a64740 1 -- start start 2024-09-19T04:54:29.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.225+0000 7fb018a64740 1 Processor -- start 2024-09-19T04:54:29.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.225+0000 7fb018a64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8b200 0x564e41ca2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.225+0000 7fb018a64740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8ec70 0x564e41c9d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.225+0000 7fb018a64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c8f590 0x564e41c9ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.225+0000 7fb018a64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101b5490 con 0x564e41c8b200 2024-09-19T04:54:29.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.225+0000 7fb018a64740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101b4d90 con 0x564e41c8ec70 2024-09-19T04:54:29.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.225+0000 7fb018a64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101b1f90 con 0x564e41c8f590 2024-09-19T04:54:29.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.226+0000 7fb018a64740 10 client.0 _ll_register_callbacks cb 0x7faffc009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:29.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.226+0000 7fb00f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8b200 0x564e41ca2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.226+0000 7fb00f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8b200 0x564e41ca2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44850/0 (socket says 172.21.15.123:44850) 2024-09-19T04:54:29.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.226+0000 7fb00f400640 1 -- 172.21.15.123:0/1781864376 learned_addr learned my addr 172.21.15.123:0/1781864376 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:29.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.226+0000 7fb00e000640 1 --2- 172.21.15.123:0/1781864376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8ec70 0x564e41c9d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.226+0000 7fb00f400640 1 --2- 172.21.15.123:0/1781864376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c8f590 0x564e41c9ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.226+0000 7fb00f400640 1 -- 172.21.15.123:0/1781864376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c8f590 msgr2=0x564e41c9ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:29.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.226+0000 7fb00f400640 1 --2- 172.21.15.123:0/1781864376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c8f590 0x564e41c9ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.226+0000 7fb00f400640 1 -- 172.21.15.123:0/1781864376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8ec70 msgr2=0x564e41c9d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:29.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.226+0000 7fb00f400640 1 --2- 172.21.15.123:0/1781864376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8ec70 0x564e41c9d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.226+0000 7fb00f400640 1 -- 172.21.15.123:0/1781864376 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e41c9e620 con 0x564e41c8b200 2024-09-19T04:54:29.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.226+0000 7fb00e000640 1 --2- 172.21.15.123:0/1781864376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8ec70 0x564e41c9d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:29.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.227+0000 7fb00f400640 1 --2- 172.21.15.123:0/1781864376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8b200 0x564e41ca2560 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb0080096c0 tx=0x7fb008031650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577333523a916da6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:29.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.227+0000 7faff8c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:29.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.227+0000 7faff8c00640 1 -- 172.21.15.123:0/1781864376 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00805a070 con 0x564e41c8b200 2024-09-19T04:54:29.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.227+0000 7faff8c00640 1 -- 172.21.15.123:0/1781864376 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fb008055e80 con 0x564e41c8b200 2024-09-19T04:54:29.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.227+0000 7fb018a64740 1 -- 172.21.15.123:0/1781864376 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faffc02a030 con 0x564e41c8b200 2024-09-19T04:54:29.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.227+0000 7faff8c00640 1 -- 172.21.15.123:0/1781864376 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb008065050 con 0x564e41c8b200 2024-09-19T04:54:29.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.227+0000 7fb018a64740 10 client.16785 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:29.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.227+0000 7fb018a64740 1 -- 172.21.15.123:0/1781864376 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564e41d14b40 con 0x564e41c8b200 2024-09-19T04:54:29.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.228+0000 7faff8c00640 1 -- 172.21.15.123:0/1781864376 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faffc02a030 con 0x564e41c8b200 2024-09-19T04:54:29.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.228+0000 7faff8c00640 1 -- 172.21.15.123:0/1781864376 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fb008004880 con 0x564e41c8b200 2024-09-19T04:54:29.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.228+0000 7fb018a64740 10 client.16785 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:29.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.228+0000 7fb018a64740 10 client.16785 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:54:29.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.228+0000 7fb018a64740 20 client.16785 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:29.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.228+0000 7fb018a64740 1 --2- 172.21.15.123:0/1781864376 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x564e41c99e50 0x564e41d16d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:29.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.228+0000 7fb018a64740 4 client.16785 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:29.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.228+0000 7fb018a64740 1 -- 172.21.15.123:0/1781864376 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e41bfce60 con 0x564e41c99e50 2024-09-19T04:54:29.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.228+0000 7fb00e000640 1 --2- 172.21.15.123:0/1781864376 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x564e41c99e50 0x564e41d16d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:29.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.229+0000 7fb00e000640 1 --2- 172.21.15.123:0/1781864376 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x564e41c99e50 0x564e41d16d20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=78a466ce0022e58a server_cookie=66b60ade58306fc5 in_seq=0 out_seq=0 2024-09-19T04:54:29.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.229+0000 7faff8c00640 10 client.16785 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:54:29.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.229+0000 7faff8c00640 1 -- 172.21.15.123:0/1781864376 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e41bfce60 con 0x564e41c99e50 2024-09-19T04:54:29.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.229+0000 7faff8c00640 10 client.16785 handle_command_reply: tid=0 2024-09-19T04:54:29.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.229+0000 7fb018a64740 1 client.16785 shutdown 2024-09-19T04:54:29.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.229+0000 7fb018a64740 1 -- 172.21.15.123:0/1781864376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8b200 msgr2=0x564e41ca2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:29.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.229+0000 7fb018a64740 1 --2- 172.21.15.123:0/1781864376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8b200 0x564e41ca2560 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb0080096c0 tx=0x7fb008031650 comp rx=0 tx=0).stop 2024-09-19T04:54:29.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.229+0000 7fb018a64740 1 -- 172.21.15.123:0/1781864376 shutdown_connections 2024-09-19T04:54:29.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.229+0000 7fb018a64740 1 --2- 172.21.15.123:0/1781864376 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x564e41c99e50 0x564e41d16d20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.229+0000 7fb018a64740 1 --2- 172.21.15.123:0/1781864376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564e41c8f590 0x564e41c9ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.229+0000 7fb018a64740 1 --2- 172.21.15.123:0/1781864376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e41c8ec70 0x564e41c9d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.229+0000 7fb018a64740 1 --2- 172.21.15.123:0/1781864376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e41c8b200 0x564e41ca2560 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.229+0000 7fb018a64740 1 -- 172.21.15.123:0/1781864376 >> 172.21.15.123:0/1781864376 conn(0x564e41c92710 msgr2=0x564e41c8a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:29.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.229+0000 7faff8c00640 0 client.16785 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:54:29.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.230+0000 7fb018a64740 1 -- 172.21.15.123:0/1781864376 shutdown_connections 2024-09-19T04:54:29.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.230+0000 7fb018a64740 1 -- 172.21.15.123:0/1781864376 wait complete. 2024-09-19T04:54:29.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.230+0000 7fb018a64740 20 client.16785 trim_cache size 0 max 16384 2024-09-19T04:54:29.240 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:29.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.230+0000 7fb015e00640 1 -- 172.21.15.123:0/3879574458 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7faff40a3430 msgr2=0x7faff40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:29.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.230+0000 7fb015e00640 1 --2- 172.21.15.123:0/3879574458 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7faff40a3430 0x7faff40a58f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x564e41c8c060 tx=0x7fb0000d6530 comp rx=0 tx=0).stop 2024-09-19T04:54:29.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.230+0000 7fb015e00640 1 -- 172.21.15.123:0/3879574458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb010074590 msgr2=0x7fb01019eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:29.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.230+0000 7fb015e00640 1 --2- 172.21.15.123:0/3879574458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb010074590 0x7fb01019eb60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb00000eb00 tx=0x7fb00000c2d0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.231+0000 7fb015e00640 1 -- 172.21.15.123:0/3879574458 shutdown_connections 2024-09-19T04:54:29.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.231+0000 7fb015e00640 1 --2- 172.21.15.123:0/3879574458 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7faff40a3430 0x7faff40a58f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.231+0000 7fb015e00640 1 --2- 172.21.15.123:0/3879574458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb010112b90 0x7fb0101a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.231+0000 7fb015e00640 1 --2- 172.21.15.123:0/3879574458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb010074eb0 0x7fb01019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.231+0000 7fb015e00640 1 --2- 172.21.15.123:0/3879574458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb010074590 0x7fb01019eb60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:29.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.231+0000 7fb015e00640 1 -- 172.21.15.123:0/3879574458 >> 172.21.15.123:0/3879574458 conn(0x7fb0100fd8c0 msgr2=0x7fb0100ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:29.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.231+0000 7fb015e00640 1 -- 172.21.15.123:0/3879574458 shutdown_connections 2024-09-19T04:54:29.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:29.231+0000 7fb015e00640 1 -- 172.21.15.123:0/3879574458 wait complete. 2024-09-19T04:54:29.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:29 smithi193 ceph-mon[35592]: pgmap v240: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 94 B/s wr, 14 op/s 2024-09-19T04:54:29.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:29 smithi183 ceph-mon[35333]: pgmap v240: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 94 B/s wr, 14 op/s 2024-09-19T04:54:29.923 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:29 smithi123 ceph-mon[30511]: pgmap v240: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 94 B/s wr, 14 op/s 2024-09-19T04:54:29.993 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:54:30.362 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:30.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.935+0000 7f67b2a00640 1 -- 172.21.15.123:0/3940639977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67ac1079d0 msgr2=0x7f67ac107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:30.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.935+0000 7f67b2a00640 1 --2- 172.21.15.123:0/3940639977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67ac1079d0 0x7f67ac107db0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f67a0009930 tx=0x7f67a002a420 comp rx=0 tx=0).stop 2024-09-19T04:54:30.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.935+0000 7f67b2a00640 1 -- 172.21.15.123:0/3940639977 shutdown_connections 2024-09-19T04:54:30.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.935+0000 7f67b2a00640 1 --2- 172.21.15.123:0/3940639977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67ac102370 0x7f67ac10e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.935+0000 7f67b2a00640 1 --2- 172.21.15.123:0/3940639977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67ac1019d0 0x7f67ac101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.935+0000 7f67b2a00640 1 --2- 172.21.15.123:0/3940639977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67ac1079d0 0x7f67ac107db0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.935+0000 7f67b2a00640 1 -- 172.21.15.123:0/3940639977 >> 172.21.15.123:0/3940639977 conn(0x7f67ac0fd710 msgr2=0x7f67ac0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:30.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.935+0000 7f67b2a00640 1 -- 172.21.15.123:0/3940639977 shutdown_connections 2024-09-19T04:54:30.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.935+0000 7f67b2a00640 1 -- 172.21.15.123:0/3940639977 wait complete. 2024-09-19T04:54:30.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.936+0000 7f67b2a00640 1 Processor -- start 2024-09-19T04:54:30.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.936+0000 7f67b2a00640 1 -- start start 2024-09-19T04:54:30.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.936+0000 7f67b2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67ac1019d0 0x7f67ac19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.936+0000 7f67b2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67ac102370 0x7f67ac19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.937+0000 7f67b2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67ac1079d0 0x7f67ac19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.937+0000 7f67b2a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac111e20 con 0x7f67ac102370 2024-09-19T04:54:30.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.937+0000 7f67b2a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac111ca0 con 0x7f67ac1079d0 2024-09-19T04:54:30.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.937+0000 7f67b2a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac111fa0 con 0x7f67ac1019d0 2024-09-19T04:54:30.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.937+0000 7f67b1600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67ac1019d0 0x7f67ac19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.937+0000 7f67b0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67ac102370 0x7f67ac19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.937+0000 7f67b1600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67ac1019d0 0x7f67ac19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:39560/0 (socket says 172.21.15.123:39560) 2024-09-19T04:54:30.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.937+0000 7f67b1600640 1 -- 172.21.15.123:0/2023104396 learned_addr learned my addr 172.21.15.123:0/2023104396 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:30.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.937+0000 7f67b2000640 1 --2- 172.21.15.123:0/2023104396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67ac1079d0 0x7f67ac19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.937+0000 7f67b2000640 1 -- 172.21.15.123:0/2023104396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67ac1019d0 msgr2=0x7f67ac19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:30.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.937+0000 7f67b2000640 1 --2- 172.21.15.123:0/2023104396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67ac1019d0 0x7f67ac19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.937+0000 7f67b2000640 1 -- 172.21.15.123:0/2023104396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67ac102370 msgr2=0x7f67ac19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:30.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.937+0000 7f67b2000640 1 --2- 172.21.15.123:0/2023104396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67ac102370 0x7f67ac19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.937+0000 7f67b2000640 1 -- 172.21.15.123:0/2023104396 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ac19f470 con 0x7f67ac1079d0 2024-09-19T04:54:30.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.937+0000 7f67b1600640 1 --2- 172.21.15.123:0/2023104396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67ac1019d0 0x7f67ac19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:30.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.937+0000 7f67b0c00640 1 --2- 172.21.15.123:0/2023104396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67ac102370 0x7f67ac19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:30.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.938+0000 7f67b2000640 1 --2- 172.21.15.123:0/2023104396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67ac1079d0 0x7f67ac19ed90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009fd0 tx=0x7f67a400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:30.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.938+0000 7f67aa000640 1 -- 172.21.15.123:0/2023104396 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a400ce20 con 0x7f67ac1079d0 2024-09-19T04:54:30.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.938+0000 7f67b2a00640 1 -- 172.21.15.123:0/2023104396 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ac19f760 con 0x7f67ac1079d0 2024-09-19T04:54:30.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.938+0000 7f67aa000640 1 -- 172.21.15.123:0/2023104396 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a4010430 con 0x7f67ac1079d0 2024-09-19T04:54:30.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.938+0000 7f67b2a00640 1 -- 172.21.15.123:0/2023104396 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ac1a7040 con 0x7f67ac1079d0 2024-09-19T04:54:30.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.938+0000 7f67aa000640 1 -- 172.21.15.123:0/2023104396 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a400f620 con 0x7f67ac1079d0 2024-09-19T04:54:30.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.941+0000 7f67b6aca740 1 Processor -- start 2024-09-19T04:54:30.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.941+0000 7f67aa000640 1 -- 172.21.15.123:0/2023104396 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f67a4010950 con 0x7f67ac1079d0 2024-09-19T04:54:30.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.941+0000 7f67b6aca740 1 -- start start 2024-09-19T04:54:30.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.942+0000 7f67b6aca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be79220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.942+0000 7f67b6aca740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be79760 0x56116be83ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.942+0000 7f67b6aca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be84020 0x56116be86410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.942+0000 7f67b6aca740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56116bddf3e0 con 0x56116be79760 2024-09-19T04:54:30.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.942+0000 7f67b6aca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56116bd86d80 con 0x56116be84020 2024-09-19T04:54:30.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.942+0000 7f67b6aca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56116bddb3a0 con 0x56116be78e40 2024-09-19T04:54:30.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.942+0000 7f67b1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be79760 0x56116be83ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.942+0000 7f67b0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be79220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.942+0000 7f67b0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be79220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:39570/0 (socket says 172.21.15.123:39570) 2024-09-19T04:54:30.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.942+0000 7f67b0c00640 1 -- 172.21.15.123:0/2865344694 learned_addr learned my addr 172.21.15.123:0/2865344694 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:30.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.942+0000 7f67b0c00640 1 --2- 172.21.15.123:0/2865344694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be84020 0x56116be86410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.943+0000 7f67b0c00640 1 -- 172.21.15.123:0/2865344694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be84020 msgr2=0x56116be86410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:30.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.943+0000 7f67b0c00640 1 --2- 172.21.15.123:0/2865344694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be84020 0x56116be86410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.943+0000 7f67b0c00640 1 -- 172.21.15.123:0/2865344694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be79760 msgr2=0x56116be83ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:30.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.943+0000 7f67b0c00640 1 --2- 172.21.15.123:0/2865344694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be79760 0x56116be83ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.943+0000 7f67b0c00640 1 -- 172.21.15.123:0/2865344694 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56116bda5ae0 con 0x56116be78e40 2024-09-19T04:54:30.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.943+0000 7f67b0c00640 1 --2- 172.21.15.123:0/2865344694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be79220 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6798005880 tx=0x7f679800bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e5d10d806871fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:30.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.943+0000 7f679e600640 1 -- 172.21.15.123:0/2865344694 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798004220 con 0x56116be78e40 2024-09-19T04:54:30.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.943+0000 7f679e600640 1 -- 172.21.15.123:0/2865344694 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6798010dd0 con 0x56116be78e40 2024-09-19T04:54:30.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.943+0000 7f679e600640 1 -- 172.21.15.123:0/2865344694 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798003de0 con 0x56116be78e40 2024-09-19T04:54:30.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.943+0000 7f67aa000640 1 --2- 172.21.15.123:0/2023104396 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f67940a3430 0x7f67940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.944+0000 7f67aa000640 1 -- 172.21.15.123:0/2023104396 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f67a40cfff0 con 0x7f67ac1079d0 2024-09-19T04:54:30.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.944+0000 7f67b1600640 1 --2- 172.21.15.123:0/2023104396 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f67940a3430 0x7f67940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.944+0000 7f67b6aca740 1 -- 172.21.15.123:0/2865344694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 msgr2=0x56116be79220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:30.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.944+0000 7f67b6aca740 1 --2- 172.21.15.123:0/2865344694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be79220 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6798005880 tx=0x7f679800bc40 comp rx=0 tx=0).stop 2024-09-19T04:54:30.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.944+0000 7f67b6aca740 1 -- 172.21.15.123:0/2865344694 shutdown_connections 2024-09-19T04:54:30.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.944+0000 7f67b6aca740 1 --2- 172.21.15.123:0/2865344694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be84020 0x56116be86410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.944+0000 7f67b6aca740 1 --2- 172.21.15.123:0/2865344694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be79760 0x56116be83ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.944+0000 7f67b6aca740 1 --2- 172.21.15.123:0/2865344694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be79220 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.944+0000 7f67b6aca740 1 -- 172.21.15.123:0/2865344694 >> 172.21.15.123:0/2865344694 conn(0x56116be753d0 msgr2=0x56116be757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:30.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.944+0000 7f67b6aca740 1 -- 172.21.15.123:0/2865344694 shutdown_connections 2024-09-19T04:54:30.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.944+0000 7f67b6aca740 1 -- 172.21.15.123:0/2865344694 wait complete. 2024-09-19T04:54:30.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.944+0000 7f67b1600640 1 --2- 172.21.15.123:0/2023104396 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f67940a3430 0x7f67940a58f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x56116be76230 tx=0x7f67a0031510 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:30.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.945+0000 7f67b6aca740 1 -- start start 2024-09-19T04:54:30.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.945+0000 7f67b6aca740 1 Processor -- start 2024-09-19T04:54:30.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b6aca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be80ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b6aca740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be79760 0x56116be81430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b6aca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be84020 0x56116be81970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b6aca740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56116be86bf0 con 0x56116be79760 2024-09-19T04:54:30.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b6aca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56116bd9a0f0 con 0x56116be84020 2024-09-19T04:54:30.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b6aca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56116be86d70 con 0x56116be78e40 2024-09-19T04:54:30.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b6aca740 10 client.0 _ll_register_callbacks cb 0x56116bd81bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:30.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b1600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be80ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b1600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be80ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:39578/0 (socket says 172.21.15.123:39578) 2024-09-19T04:54:30.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b1600640 1 -- 172.21.15.123:0/902934493 learned_addr learned my addr 172.21.15.123:0/902934493 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:30.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b0c00640 1 --2- 172.21.15.123:0/902934493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be79760 0x56116be81430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b2000640 1 --2- 172.21.15.123:0/902934493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be84020 0x56116be81970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b1600640 1 -- 172.21.15.123:0/902934493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be84020 msgr2=0x56116be81970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:30.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b1600640 1 --2- 172.21.15.123:0/902934493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be84020 0x56116be81970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b1600640 1 -- 172.21.15.123:0/902934493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be79760 msgr2=0x56116be81430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:30.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b1600640 1 --2- 172.21.15.123:0/902934493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be79760 0x56116be81430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b1600640 1 -- 172.21.15.123:0/902934493 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56116bb503c0 con 0x56116be78e40 2024-09-19T04:54:30.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.946+0000 7f67b0c00640 1 --2- 172.21.15.123:0/902934493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be79760 0x56116be81430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:30.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.947+0000 7f67b1600640 1 --2- 172.21.15.123:0/902934493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be80ef0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f67a0005650 tx=0x7f67a0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5016f6f0e72c9f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:30.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.947+0000 7f6792a00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:54:30.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.947+0000 7f6792a00640 1 -- 172.21.15.123:0/902934493 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0035040 con 0x56116be78e40 2024-09-19T04:54:30.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.947+0000 7f67b6aca740 1 -- 172.21.15.123:0/902934493 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6798010dd0 con 0x56116be78e40 2024-09-19T04:54:30.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.947+0000 7f6792a00640 1 -- 172.21.15.123:0/902934493 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f67a00317b0 con 0x56116be78e40 2024-09-19T04:54:30.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.947+0000 7f6792a00640 1 -- 172.21.15.123:0/902934493 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f67a00319b0 con 0x56116be78e40 2024-09-19T04:54:30.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.948+0000 7f67b6aca740 10 client.26917 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:30.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.948+0000 7f67b6aca740 1 -- 172.21.15.123:0/902934493 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56116be92c90 con 0x56116be78e40 2024-09-19T04:54:30.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.948+0000 7f6792a00640 1 -- 172.21.15.123:0/902934493 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6798010dd0 con 0x56116be78e40 2024-09-19T04:54:30.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.948+0000 7f6792a00640 1 -- 172.21.15.123:0/902934493 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f67a0044030 con 0x56116be78e40 2024-09-19T04:54:30.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.948+0000 7f67b6aca740 10 client.26917 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:30.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.948+0000 7f67b6aca740 10 client.26917 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:54:30.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.948+0000 7f67b6aca740 20 client.26917 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:30.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.948+0000 7f67b6aca740 1 --2- 172.21.15.123:0/902934493 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x56116be940a0 0x56116be96530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.948+0000 7f67b6aca740 4 client.26917 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:30.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.948+0000 7f67b6aca740 1 -- 172.21.15.123:0/902934493 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56116bdec950 con 0x56116be940a0 2024-09-19T04:54:30.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.949+0000 7f67b0c00640 1 --2- 172.21.15.123:0/902934493 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x56116be940a0 0x56116be96530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.949+0000 7f67b0c00640 1 --2- 172.21.15.123:0/902934493 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x56116be940a0 0x56116be96530 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5c588060fb48245c server_cookie=eb64be28987174cc in_seq=0 out_seq=0 2024-09-19T04:54:30.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.949+0000 7f6792a00640 10 client.26917 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:54:30.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.950+0000 7f6792a00640 1 -- 172.21.15.123:0/902934493 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56116bdec950 con 0x56116be940a0 2024-09-19T04:54:30.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.950+0000 7f6792a00640 10 client.26917 handle_command_reply: tid=0 2024-09-19T04:54:30.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.950+0000 7f67b6aca740 1 client.26917 shutdown 2024-09-19T04:54:30.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.950+0000 7f67b6aca740 1 -- 172.21.15.123:0/902934493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 msgr2=0x56116be80ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:30.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.950+0000 7f67b6aca740 1 --2- 172.21.15.123:0/902934493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be80ef0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f67a0005650 tx=0x7f67a0009260 comp rx=0 tx=0).stop 2024-09-19T04:54:30.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.950+0000 7f67b6aca740 1 -- 172.21.15.123:0/902934493 shutdown_connections 2024-09-19T04:54:30.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.950+0000 7f67b6aca740 1 --2- 172.21.15.123:0/902934493 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x56116be940a0 0x56116be96530 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.950+0000 7f67b6aca740 1 --2- 172.21.15.123:0/902934493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be84020 0x56116be81970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.950+0000 7f67b6aca740 1 --2- 172.21.15.123:0/902934493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be79760 0x56116be81430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.950+0000 7f67b6aca740 1 --2- 172.21.15.123:0/902934493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be80ef0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.950+0000 7f67b6aca740 1 -- 172.21.15.123:0/902934493 >> 172.21.15.123:0/902934493 conn(0x56116be753d0 msgr2=0x56116be7b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:30.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.950+0000 7f6792a00640 0 client.26917 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:54:30.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.951+0000 7f67b6aca740 1 -- 172.21.15.123:0/902934493 shutdown_connections 2024-09-19T04:54:30.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.951+0000 7f67b6aca740 1 -- 172.21.15.123:0/902934493 wait complete. 2024-09-19T04:54:30.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.951+0000 7f67b6aca740 20 client.26917 trim_cache size 0 max 16384 2024-09-19T04:54:30.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.964+0000 7f67b6aca740 1 Processor -- start 2024-09-19T04:54:30.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.964+0000 7f67b6aca740 1 -- start start 2024-09-19T04:54:30.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.965+0000 7f67b6aca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be78e40 0x56116be7b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.965+0000 7f67b6aca740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be79760 0x56116be9be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.965+0000 7f67b6aca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be84020 0x56116be9c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.965+0000 7f67b6aca740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac1af890 con 0x56116be79760 2024-09-19T04:54:30.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.965+0000 7f67b6aca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac1b1460 con 0x56116be78e40 2024-09-19T04:54:30.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.965+0000 7f67b6aca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac1aff90 con 0x56116be84020 2024-09-19T04:54:30.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.965+0000 7f67b0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be79760 0x56116be9be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.965+0000 7f67b2000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be84020 0x56116be9c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.965+0000 7f67b0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be79760 0x56116be9be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44888/0 (socket says 172.21.15.123:44888) 2024-09-19T04:54:30.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.965+0000 7f67b0c00640 1 -- 172.21.15.123:0/2609271491 learned_addr learned my addr 172.21.15.123:0/2609271491 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:30.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.965+0000 7f67b1600640 1 --2- 172.21.15.123:0/2609271491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be78e40 0x56116be7b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.965+0000 7f67b2000640 1 -- 172.21.15.123:0/2609271491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be78e40 msgr2=0x56116be7b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:30.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.965+0000 7f67b2000640 1 --2- 172.21.15.123:0/2609271491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be78e40 0x56116be7b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.965+0000 7f67b2000640 1 -- 172.21.15.123:0/2609271491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be79760 msgr2=0x56116be9be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:30.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.965+0000 7f67b2000640 1 --2- 172.21.15.123:0/2609271491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be79760 0x56116be9be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.965+0000 7f67b2000640 1 -- 172.21.15.123:0/2609271491 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56116be75200 con 0x56116be84020 2024-09-19T04:54:30.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.966+0000 7f67b2000640 1 --2- 172.21.15.123:0/2609271491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be84020 0x56116be9c340 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f67a40cd6e0 tx=0x7f67a40a0620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deca47f1694cd553 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:30.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.966+0000 7f679dc00640 1 -- 172.21.15.123:0/2609271491 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a40b5270 con 0x56116be84020 2024-09-19T04:54:30.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.966+0000 7f679dc00640 1 -- 172.21.15.123:0/2609271491 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a40105e0 con 0x56116be84020 2024-09-19T04:54:30.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.966+0000 7f679dc00640 1 -- 172.21.15.123:0/2609271491 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a40a0410 con 0x56116be84020 2024-09-19T04:54:30.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.966+0000 7f67b6aca740 1 -- 172.21.15.123:0/2609271491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be84020 msgr2=0x56116be9c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:30.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.966+0000 7f67b6aca740 1 --2- 172.21.15.123:0/2609271491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be84020 0x56116be9c340 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f67a40cd6e0 tx=0x7f67a40a0620 comp rx=0 tx=0).stop 2024-09-19T04:54:30.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.966+0000 7f67b6aca740 1 -- 172.21.15.123:0/2609271491 shutdown_connections 2024-09-19T04:54:30.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.966+0000 7f67b6aca740 1 --2- 172.21.15.123:0/2609271491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be84020 0x56116be9c340 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.966+0000 7f67b6aca740 1 --2- 172.21.15.123:0/2609271491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be79760 0x56116be9be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.967+0000 7f67b6aca740 1 --2- 172.21.15.123:0/2609271491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be78e40 0x56116be7b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.967+0000 7f67b6aca740 1 -- 172.21.15.123:0/2609271491 >> 172.21.15.123:0/2609271491 conn(0x56116be753d0 msgr2=0x56116be7ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:30.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.967+0000 7f67b6aca740 1 -- 172.21.15.123:0/2609271491 shutdown_connections 2024-09-19T04:54:30.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.967+0000 7f67b6aca740 1 -- 172.21.15.123:0/2609271491 wait complete. 2024-09-19T04:54:30.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.967+0000 7f67b6aca740 1 -- start start 2024-09-19T04:54:30.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.967+0000 7f67b6aca740 1 Processor -- start 2024-09-19T04:54:30.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.968+0000 7f67b6aca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be8c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.968+0000 7f67b6aca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be79760 0x56116be87920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.968+0000 7f67b6aca740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be84020 0x56116be87e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.968+0000 7f67b6aca740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac1b0d90 con 0x56116be84020 2024-09-19T04:54:30.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.968+0000 7f67b6aca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac1b0690 con 0x56116be79760 2024-09-19T04:54:30.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.968+0000 7f67b6aca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac1ad890 con 0x56116be78e40 2024-09-19T04:54:30.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.968+0000 7f67b6aca740 10 client.0 _ll_register_callbacks cb 0x7f67a00050c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:30.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.968+0000 7f67b1600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be8c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.968+0000 7f67b2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be84020 0x56116be87e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.968+0000 7f67b1600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be8c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:39598/0 (socket says 172.21.15.123:39598) 2024-09-19T04:54:30.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.968+0000 7f67b1600640 1 -- 172.21.15.123:0/777673595 learned_addr learned my addr 172.21.15.123:0/777673595 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:30.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.968+0000 7f67b0c00640 1 --2- 172.21.15.123:0/777673595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be79760 0x56116be87920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.969+0000 7f67b1600640 1 -- 172.21.15.123:0/777673595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be79760 msgr2=0x56116be87920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:30.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.969+0000 7f67b1600640 1 --2- 172.21.15.123:0/777673595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be79760 0x56116be87920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.969+0000 7f67b1600640 1 -- 172.21.15.123:0/777673595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be84020 msgr2=0x56116be87e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:30.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.969+0000 7f67b1600640 1 --2- 172.21.15.123:0/777673595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be84020 0x56116be87e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.969+0000 7f67b1600640 1 -- 172.21.15.123:0/777673595 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56116be74c80 con 0x56116be78e40 2024-09-19T04:54:30.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.969+0000 7f67b0c00640 1 --2- 172.21.15.123:0/777673595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be79760 0x56116be87920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:30.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.969+0000 7f67b2000640 1 --2- 172.21.15.123:0/777673595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be84020 0x56116be87e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:30.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.969+0000 7f67b1600640 1 --2- 172.21.15.123:0/777673595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be8c5f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f67a00051f0 tx=0x7f67a0004ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9f42283ce3ea427 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:30.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.969+0000 7f6792a00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:54:30.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.969+0000 7f6792a00640 1 -- 172.21.15.123:0/777673595 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a005e030 con 0x56116be78e40 2024-09-19T04:54:30.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.969+0000 7f6792a00640 1 -- 172.21.15.123:0/777673595 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f67a003da40 con 0x56116be78e40 2024-09-19T04:54:30.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.969+0000 7f67b6aca740 1 -- 172.21.15.123:0/777673595 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67a40105e0 con 0x56116be78e40 2024-09-19T04:54:30.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.969+0000 7f6792a00640 1 -- 172.21.15.123:0/777673595 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f67a003dc40 con 0x56116be78e40 2024-09-19T04:54:30.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.970+0000 7f67b6aca740 10 client.26923 fetch_fsmap learned FSMap version 40 2024-09-19T04:54:30.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.970+0000 7f67b6aca740 1 -- 172.21.15.123:0/777673595 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56116be76480 con 0x56116be78e40 2024-09-19T04:54:30.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.970+0000 7f6792a00640 1 -- 172.21.15.123:0/777673595 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a40105e0 con 0x56116be78e40 2024-09-19T04:54:30.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.970+0000 7f6792a00640 1 -- 172.21.15.123:0/777673595 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f67a0044040 con 0x56116be78e40 2024-09-19T04:54:30.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.970+0000 7f67b6aca740 10 client.26923 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T04:54:30.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.970+0000 7f67b6aca740 10 client.26923 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:54:30.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.970+0000 7f67b6aca740 20 client.26923 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:30.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.970+0000 7f67b6aca740 1 --2- 172.21.15.123:0/777673595 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x56116beff020 0x56116bf01440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:30.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.970+0000 7f67b6aca740 4 client.26923 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:30.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.970+0000 7f67b6aca740 1 -- 172.21.15.123:0/777673595 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56116bde6e60 con 0x56116beff020 2024-09-19T04:54:30.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.970+0000 7f67b0c00640 1 --2- 172.21.15.123:0/777673595 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x56116beff020 0x56116bf01440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:30.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.971+0000 7f67b0c00640 1 --2- 172.21.15.123:0/777673595 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x56116beff020 0x56116bf01440 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3fb331391bfd6bc3 server_cookie=9ceb7383582e1b04 in_seq=0 out_seq=0 2024-09-19T04:54:30.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.971+0000 7f6792a00640 10 client.26923 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:54:30.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.971+0000 7f6792a00640 1 -- 172.21.15.123:0/777673595 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56116bde6e60 con 0x56116beff020 2024-09-19T04:54:30.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.971+0000 7f6792a00640 10 client.26923 handle_command_reply: tid=0 2024-09-19T04:54:30.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.971+0000 7f67b6aca740 1 client.26923 shutdown 2024-09-19T04:54:30.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.971+0000 7f67b6aca740 1 -- 172.21.15.123:0/777673595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 msgr2=0x56116be8c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:30.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.971+0000 7f67b6aca740 1 --2- 172.21.15.123:0/777673595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be8c5f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f67a00051f0 tx=0x7f67a0004ba0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.971+0000 7f67b6aca740 1 -- 172.21.15.123:0/777673595 shutdown_connections 2024-09-19T04:54:30.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.971+0000 7f67b6aca740 1 --2- 172.21.15.123:0/777673595 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x56116beff020 0x56116bf01440 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.971+0000 7f67b6aca740 1 --2- 172.21.15.123:0/777673595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56116be84020 0x56116be87e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.971+0000 7f67b6aca740 1 --2- 172.21.15.123:0/777673595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56116be79760 0x56116be87920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.971+0000 7f67b6aca740 1 --2- 172.21.15.123:0/777673595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56116be78e40 0x56116be8c5f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.971+0000 7f6792a00640 0 client.26923 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:54:30.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.971+0000 7f67b6aca740 1 -- 172.21.15.123:0/777673595 >> 172.21.15.123:0/777673595 conn(0x56116be753d0 msgr2=0x56116be73c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:30.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.972+0000 7f67b6aca740 1 -- 172.21.15.123:0/777673595 shutdown_connections 2024-09-19T04:54:30.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.972+0000 7f67b6aca740 1 -- 172.21.15.123:0/777673595 wait complete. 2024-09-19T04:54:30.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.972+0000 7f67b6aca740 20 client.26923 trim_cache size 0 max 16384 2024-09-19T04:54:30.978 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:30.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.972+0000 7f67b2a00640 1 -- 172.21.15.123:0/2023104396 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f67940a3430 msgr2=0x7f67940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:30.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.972+0000 7f67b2a00640 1 --2- 172.21.15.123:0/2023104396 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f67940a3430 0x7f67940a58f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x56116be76230 tx=0x7f67a0031510 comp rx=0 tx=0).stop 2024-09-19T04:54:30.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.972+0000 7f67b2a00640 1 -- 172.21.15.123:0/2023104396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67ac1079d0 msgr2=0x7f67ac19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:30.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.972+0000 7f67b2a00640 1 --2- 172.21.15.123:0/2023104396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67ac1079d0 0x7f67ac19ed90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009fd0 tx=0x7f67a400ef10 comp rx=0 tx=0).stop 2024-09-19T04:54:30.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.973+0000 7f67b2a00640 1 -- 172.21.15.123:0/2023104396 shutdown_connections 2024-09-19T04:54:30.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.973+0000 7f67b2a00640 1 --2- 172.21.15.123:0/2023104396 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f67940a3430 0x7f67940a58f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.973+0000 7f67b2a00640 1 --2- 172.21.15.123:0/2023104396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67ac1079d0 0x7f67ac19ed90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.973+0000 7f67b2a00640 1 --2- 172.21.15.123:0/2023104396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67ac102370 0x7f67ac19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.973+0000 7f67b2a00640 1 --2- 172.21.15.123:0/2023104396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67ac1019d0 0x7f67ac19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:30.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.973+0000 7f67b2a00640 1 -- 172.21.15.123:0/2023104396 >> 172.21.15.123:0/2023104396 conn(0x7f67ac0fd710 msgr2=0x7f67ac0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:30.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.973+0000 7f67b2a00640 1 -- 172.21.15.123:0/2023104396 shutdown_connections 2024-09-19T04:54:30.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:30.973+0000 7f67b2a00640 1 -- 172.21.15.123:0/2023104396 wait complete. 2024-09-19T04:54:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:31 smithi123 ceph-mon[30511]: pgmap v241: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:31.818 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:54:31.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:31 smithi183 ceph-mon[35333]: pgmap v241: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:31.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:31 smithi193 ceph-mon[35592]: pgmap v241: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:32.190 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:32.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.787+0000 7f1698000640 1 -- 172.21.15.123:0/991702168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1690109120 msgr2=0x7f1690110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:32.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.787+0000 7f1698000640 1 --2- 172.21.15.123:0/991702168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1690109120 0x7f1690110ca0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1688009ca0 tx=0x7f168802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.787+0000 7f1698000640 1 -- 172.21.15.123:0/991702168 shutdown_connections 2024-09-19T04:54:32.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.787+0000 7f1698000640 1 --2- 172.21.15.123:0/991702168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1690109120 0x7f1690110ca0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.787+0000 7f1698000640 1 --2- 172.21.15.123:0/991702168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16901049a0 0x7f16901089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.787+0000 7f1698000640 1 --2- 172.21.15.123:0/991702168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1690103ff0 0x7f16901043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.787+0000 7f1698000640 1 -- 172.21.15.123:0/991702168 >> 172.21.15.123:0/991702168 conn(0x7f16900ffa90 msgr2=0x7f1690101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:32.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.787+0000 7f1698000640 1 -- 172.21.15.123:0/991702168 shutdown_connections 2024-09-19T04:54:32.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.787+0000 7f1698000640 1 -- 172.21.15.123:0/991702168 wait complete. 2024-09-19T04:54:32.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.788+0000 7f1698000640 1 Processor -- start 2024-09-19T04:54:32.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.788+0000 7f1698000640 1 -- start start 2024-09-19T04:54:32.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.788+0000 7f1698000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1690103ff0 0x7f169019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.788+0000 7f1698000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16901049a0 0x7f169019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.788+0000 7f1698000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1690109120 0x7f169019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.788+0000 7f1698000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1690114180 con 0x7f1690103ff0 2024-09-19T04:54:32.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.788+0000 7f1698000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1690114000 con 0x7f16901049a0 2024-09-19T04:54:32.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.788+0000 7f1698000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1690114300 con 0x7f1690109120 2024-09-19T04:54:32.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.789+0000 7f1695600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1690103ff0 0x7f169019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.789+0000 7f1694c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16901049a0 0x7f169019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.789+0000 7f1695600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1690103ff0 0x7f169019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35792/0 (socket says 172.21.15.123:35792) 2024-09-19T04:54:32.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.789+0000 7f1695600640 1 -- 172.21.15.123:0/3520844620 learned_addr learned my addr 172.21.15.123:0/3520844620 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:32.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.789+0000 7f1696000640 1 --2- 172.21.15.123:0/3520844620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1690109120 0x7f169019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.789+0000 7f1694c00640 1 -- 172.21.15.123:0/3520844620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1690109120 msgr2=0x7f169019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:32.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.789+0000 7f1694c00640 1 --2- 172.21.15.123:0/3520844620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1690109120 0x7f169019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.789+0000 7f1694c00640 1 -- 172.21.15.123:0/3520844620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1690103ff0 msgr2=0x7f169019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:32.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.789+0000 7f1694c00640 1 --2- 172.21.15.123:0/3520844620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1690103ff0 0x7f169019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.789+0000 7f1694c00640 1 -- 172.21.15.123:0/3520844620 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169019f550 con 0x7f16901049a0 2024-09-19T04:54:32.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.789+0000 7f1695600640 1 --2- 172.21.15.123:0/3520844620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1690103ff0 0x7f169019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:32.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.789+0000 7f1694c00640 1 --2- 172.21.15.123:0/3520844620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16901049a0 0x7f169019aae0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f167c00e9f0 tx=0x7f167c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:32.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.790+0000 7f168e000640 1 -- 172.21.15.123:0/3520844620 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c00ccc0 con 0x7f16901049a0 2024-09-19T04:54:32.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.790+0000 7f168e000640 1 -- 172.21.15.123:0/3520844620 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f167c010910 con 0x7f16901049a0 2024-09-19T04:54:32.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.790+0000 7f1698000640 1 -- 172.21.15.123:0/3520844620 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169019f840 con 0x7f16901049a0 2024-09-19T04:54:32.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.790+0000 7f168e000640 1 -- 172.21.15.123:0/3520844620 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c018960 con 0x7f16901049a0 2024-09-19T04:54:32.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.790+0000 7f1698000640 1 -- 172.21.15.123:0/3520844620 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16901a7120 con 0x7f16901049a0 2024-09-19T04:54:32.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.793+0000 7f169abba740 1 Processor -- start 2024-09-19T04:54:32.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.793+0000 7f169abba740 1 -- start start 2024-09-19T04:54:32.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.793+0000 7f168e000640 1 -- 172.21.15.123:0/3520844620 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f167c010ac0 con 0x7f16901049a0 2024-09-19T04:54:32.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.793+0000 7f169abba740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa7c70 0x55cde6aa8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.793+0000 7f169abba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa8590 0x55cde6ab2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.793+0000 7f169abba740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6ab5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.794+0000 7f169abba740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cde6a0e3e0 con 0x55cde6ab2e50 2024-09-19T04:54:32.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.794+0000 7f169abba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cde69b5d80 con 0x55cde6aa8590 2024-09-19T04:54:32.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.794+0000 7f169abba740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cde6a0a3a0 con 0x55cde6aa7c70 2024-09-19T04:54:32.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.794+0000 7f1696000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa7c70 0x55cde6aa8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.794+0000 7f1694c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6ab5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.794+0000 7f1696000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa7c70 0x55cde6aa8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:35674/0 (socket says 172.21.15.123:35674) 2024-09-19T04:54:32.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.794+0000 7f1696000640 1 -- 172.21.15.123:0/88956197 learned_addr learned my addr 172.21.15.123:0/88956197 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:32.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.794+0000 7f1695600640 1 --2- 172.21.15.123:0/88956197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa8590 0x55cde6ab2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.794+0000 7f1696000640 1 -- 172.21.15.123:0/88956197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa8590 msgr2=0x55cde6ab2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:32.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.794+0000 7f1696000640 1 --2- 172.21.15.123:0/88956197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa8590 0x55cde6ab2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.794+0000 7f1696000640 1 -- 172.21.15.123:0/88956197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 msgr2=0x55cde6ab5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:32.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.794+0000 7f1696000640 1 --2- 172.21.15.123:0/88956197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6ab5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.794+0000 7f1696000640 1 -- 172.21.15.123:0/88956197 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cde69d4ae0 con 0x55cde6aa7c70 2024-09-19T04:54:32.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.795+0000 7f1696000640 1 --2- 172.21.15.123:0/88956197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa7c70 0x55cde6aa8050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1688009ca0 tx=0x7f16880093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cbca9d5197219d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:32.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.795+0000 7f168e000640 1 --2- 172.21.15.123:0/3520844620 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f16780a33b0 0x7f16780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.795+0000 7f168e000640 1 -- 172.21.15.123:0/3520844620 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f167c0d0040 con 0x7f16901049a0 2024-09-19T04:54:32.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.795+0000 7f1682600640 1 -- 172.21.15.123:0/88956197 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688031db0 con 0x55cde6aa7c70 2024-09-19T04:54:32.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.795+0000 7f1695600640 1 --2- 172.21.15.123:0/3520844620 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f16780a33b0 0x7f16780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.795+0000 7f1682600640 1 -- 172.21.15.123:0/88956197 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1688036030 con 0x55cde6aa7c70 2024-09-19T04:54:32.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.795+0000 7f1682600640 1 -- 172.21.15.123:0/88956197 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16880042a0 con 0x55cde6aa7c70 2024-09-19T04:54:32.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.796+0000 7f169abba740 1 -- 172.21.15.123:0/88956197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa7c70 msgr2=0x55cde6aa8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:32.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.796+0000 7f169abba740 1 --2- 172.21.15.123:0/88956197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa7c70 0x55cde6aa8050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1688009ca0 tx=0x7f16880093e0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.796+0000 7f169abba740 1 -- 172.21.15.123:0/88956197 shutdown_connections 2024-09-19T04:54:32.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.796+0000 7f169abba740 1 --2- 172.21.15.123:0/88956197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6ab5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.796+0000 7f1695600640 1 --2- 172.21.15.123:0/3520844620 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f16780a33b0 0x7f16780a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1684009620 tx=0x7f1684005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:32.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.796+0000 7f169abba740 1 --2- 172.21.15.123:0/88956197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa8590 0x55cde6ab2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.796+0000 7f169abba740 1 --2- 172.21.15.123:0/88956197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa7c70 0x55cde6aa8050 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.796+0000 7f169abba740 1 -- 172.21.15.123:0/88956197 >> 172.21.15.123:0/88956197 conn(0x55cde6aa4200 msgr2=0x55cde6aa45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:32.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.796+0000 7f169abba740 1 -- 172.21.15.123:0/88956197 shutdown_connections 2024-09-19T04:54:32.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.796+0000 7f169abba740 1 -- 172.21.15.123:0/88956197 wait complete. 2024-09-19T04:54:32.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.796+0000 7f169abba740 1 -- start start 2024-09-19T04:54:32.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.797+0000 7f169abba740 1 Processor -- start 2024-09-19T04:54:32.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.797+0000 7f169abba740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa7c70 0x55cde6aafd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.797+0000 7f169abba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa8590 0x55cde6ab0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.797+0000 7f169abba740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6ab07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.797+0000 7f169abba740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cde6ab58a0 con 0x55cde6ab2e50 2024-09-19T04:54:32.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.797+0000 7f169abba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cde69c90f0 con 0x55cde6aa8590 2024-09-19T04:54:32.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.797+0000 7f169abba740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cde6ab5a20 con 0x55cde6aa7c70 2024-09-19T04:54:32.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.797+0000 7f169abba740 10 client.0 _ll_register_callbacks cb 0x55cde69b0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:32.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.797+0000 7f1694c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa8590 0x55cde6ab0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.797+0000 7f1694c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa8590 0x55cde6ab0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:54746/0 (socket says 172.21.15.123:54746) 2024-09-19T04:54:32.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.797+0000 7f1696000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6ab07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.797+0000 7f1694c00640 1 -- 172.21.15.123:0/252121534 learned_addr learned my addr 172.21.15.123:0/252121534 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:32.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.797+0000 7f1695600640 1 --2- 172.21.15.123:0/252121534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa7c70 0x55cde6aafd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.798+0000 7f1694c00640 1 -- 172.21.15.123:0/252121534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa7c70 msgr2=0x55cde6aafd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:32.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.798+0000 7f1694c00640 1 --2- 172.21.15.123:0/252121534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa7c70 0x55cde6aafd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.798+0000 7f1694c00640 1 -- 172.21.15.123:0/252121534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 msgr2=0x55cde6ab07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:32.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.798+0000 7f1694c00640 1 --2- 172.21.15.123:0/252121534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6ab07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.798+0000 7f1694c00640 1 -- 172.21.15.123:0/252121534 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cde677f3c0 con 0x55cde6aa8590 2024-09-19T04:54:32.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.798+0000 7f1694c00640 1 --2- 172.21.15.123:0/252121534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa8590 0x55cde6ab0270 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f167c00e9c0 tx=0x7f167c0d3780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1148da3d46ef24d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:32.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.798+0000 7f1676a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:54:32.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.798+0000 7f1676a00640 1 -- 172.21.15.123:0/252121534 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c09c1a0 con 0x55cde6aa8590 2024-09-19T04:54:32.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.798+0000 7f1676a00640 1 -- 172.21.15.123:0/252121534 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f167c0d2110 con 0x55cde6aa8590 2024-09-19T04:54:32.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.798+0000 7f169abba740 1 -- 172.21.15.123:0/252121534 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1688036030 con 0x55cde6aa8590 2024-09-19T04:54:32.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.799+0000 7f1676a00640 1 -- 172.21.15.123:0/252121534 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f167c09bd30 con 0x55cde6aa8590 2024-09-19T04:54:32.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.799+0000 7f169abba740 10 client.26909 fetch_fsmap learned FSMap version 41 2024-09-19T04:54:32.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.799+0000 7f169abba740 1 -- 172.21.15.123:0/252121534 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55cde6ac1930 con 0x55cde6aa8590 2024-09-19T04:54:32.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.799+0000 7f1676a00640 1 -- 172.21.15.123:0/252121534 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1688036030 con 0x55cde6aa8590 2024-09-19T04:54:32.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.799+0000 7f1676a00640 1 -- 172.21.15.123:0/252121534 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f167c09b820 con 0x55cde6aa8590 2024-09-19T04:54:32.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.799+0000 7f169abba740 10 client.26909 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T04:54:32.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.799+0000 7f169abba740 10 client.26909 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:54:32.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.799+0000 7f169abba740 20 client.26909 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:32.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.799+0000 7f169abba740 1 --2- 172.21.15.123:0/252121534 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55cde6ac2d90 0x55cde6ac51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.799+0000 7f169abba740 4 client.26909 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:32.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.799+0000 7f169abba740 1 -- 172.21.15.123:0/252121534 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cde6a1b950 con 0x55cde6ac2d90 2024-09-19T04:54:32.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.800+0000 7f1695600640 1 --2- 172.21.15.123:0/252121534 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55cde6ac2d90 0x55cde6ac51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.800+0000 7f1695600640 1 --2- 172.21.15.123:0/252121534 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55cde6ac2d90 0x55cde6ac51b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7bf9bf6aa354412 server_cookie=c33a2dc0b43e13e3 in_seq=0 out_seq=0 2024-09-19T04:54:32.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.800+0000 7f1676a00640 10 client.26909 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:54:32.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.801+0000 7f1676a00640 1 -- 172.21.15.123:0/252121534 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cde6a1b950 con 0x55cde6ac2d90 2024-09-19T04:54:32.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.801+0000 7f1676a00640 10 client.26909 handle_command_reply: tid=0 2024-09-19T04:54:32.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.801+0000 7f169abba740 1 client.26909 shutdown 2024-09-19T04:54:32.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.802+0000 7f169abba740 1 -- 172.21.15.123:0/252121534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa8590 msgr2=0x55cde6ab0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:32.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.802+0000 7f169abba740 1 --2- 172.21.15.123:0/252121534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa8590 0x55cde6ab0270 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f167c00e9c0 tx=0x7f167c0d3780 comp rx=0 tx=0).stop 2024-09-19T04:54:32.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.802+0000 7f169abba740 1 -- 172.21.15.123:0/252121534 shutdown_connections 2024-09-19T04:54:32.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.802+0000 7f169abba740 1 --2- 172.21.15.123:0/252121534 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55cde6ac2d90 0x55cde6ac51b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.802+0000 7f169abba740 1 --2- 172.21.15.123:0/252121534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6ab07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.802+0000 7f169abba740 1 --2- 172.21.15.123:0/252121534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa8590 0x55cde6ab0270 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.802+0000 7f169abba740 1 --2- 172.21.15.123:0/252121534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa7c70 0x55cde6aafd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.802+0000 7f1676a00640 0 client.26909 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:54:32.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.802+0000 7f169abba740 1 -- 172.21.15.123:0/252121534 >> 172.21.15.123:0/252121534 conn(0x55cde6aa4200 msgr2=0x55cde6aa3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:32.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.802+0000 7f169abba740 1 -- 172.21.15.123:0/252121534 shutdown_connections 2024-09-19T04:54:32.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.802+0000 7f169abba740 1 -- 172.21.15.123:0/252121534 wait complete. 2024-09-19T04:54:32.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.802+0000 7f169abba740 20 client.26909 trim_cache size 0 max 16384 2024-09-19T04:54:32.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.815+0000 7f169abba740 1 Processor -- start 2024-09-19T04:54:32.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.816+0000 7f169abba740 1 -- start start 2024-09-19T04:54:32.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.816+0000 7f169abba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa7c70 0x55cde6acaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.816+0000 7f169abba740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa8590 0x55cde6acafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.816+0000 7f169abba740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6aa2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.816+0000 7f169abba740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f16901af9b0 con 0x55cde6ab2e50 2024-09-19T04:54:32.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.816+0000 7f169abba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f16901b1580 con 0x55cde6aa7c70 2024-09-19T04:54:32.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.816+0000 7f169abba740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16901b00b0 con 0x55cde6aa8590 2024-09-19T04:54:32.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.816+0000 7f1696000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6aa2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.816+0000 7f1694c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa8590 0x55cde6acafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.816+0000 7f1696000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6aa2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35810/0 (socket says 172.21.15.123:35810) 2024-09-19T04:54:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.816+0000 7f1694c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa8590 0x55cde6acafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:35696/0 (socket says 172.21.15.123:35696) 2024-09-19T04:54:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.816+0000 7f1696000640 1 -- 172.21.15.123:0/3047634457 learned_addr learned my addr 172.21.15.123:0/3047634457 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.816+0000 7f1695600640 1 --2- 172.21.15.123:0/3047634457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa7c70 0x55cde6acaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.817+0000 7f1696000640 1 -- 172.21.15.123:0/3047634457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa8590 msgr2=0x55cde6acafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.817+0000 7f1696000640 1 --2- 172.21.15.123:0/3047634457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa8590 0x55cde6acafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.817+0000 7f1696000640 1 -- 172.21.15.123:0/3047634457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa7c70 msgr2=0x55cde6acaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.817+0000 7f1696000640 1 --2- 172.21.15.123:0/3047634457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa7c70 0x55cde6acaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.817+0000 7f1696000640 1 -- 172.21.15.123:0/3047634457 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cde6a9f770 con 0x55cde6ab2e50 2024-09-19T04:54:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.817+0000 7f1694c00640 1 --2- 172.21.15.123:0/3047634457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa8590 0x55cde6acafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.817+0000 7f1695600640 1 --2- 172.21.15.123:0/3047634457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa7c70 0x55cde6acaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:32.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.817+0000 7f1696000640 1 --2- 172.21.15.123:0/3047634457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6aa2b60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1688006010 tx=0x7f16880319d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376c7b5608796502 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:32.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.817+0000 7f1681c00640 1 -- 172.21.15.123:0/3047634457 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688055070 con 0x55cde6ab2e50 2024-09-19T04:54:32.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.817+0000 7f1681c00640 1 -- 172.21.15.123:0/3047634457 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1688036040 con 0x55cde6ab2e50 2024-09-19T04:54:32.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.817+0000 7f1681c00640 1 -- 172.21.15.123:0/3047634457 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168804a040 con 0x55cde6ab2e50 2024-09-19T04:54:32.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.818+0000 7f169abba740 1 -- 172.21.15.123:0/3047634457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 msgr2=0x55cde6aa2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:32.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.818+0000 7f169abba740 1 --2- 172.21.15.123:0/3047634457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6aa2b60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1688006010 tx=0x7f16880319d0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.818+0000 7f169abba740 1 -- 172.21.15.123:0/3047634457 shutdown_connections 2024-09-19T04:54:32.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.818+0000 7f169abba740 1 --2- 172.21.15.123:0/3047634457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6aa2b60 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.818+0000 7f169abba740 1 --2- 172.21.15.123:0/3047634457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa8590 0x55cde6acafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.818+0000 7f169abba740 1 --2- 172.21.15.123:0/3047634457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa7c70 0x55cde6acaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.818+0000 7f169abba740 1 -- 172.21.15.123:0/3047634457 >> 172.21.15.123:0/3047634457 conn(0x55cde6aa4200 msgr2=0x55cde6aaee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:32.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.818+0000 7f169abba740 1 -- 172.21.15.123:0/3047634457 shutdown_connections 2024-09-19T04:54:32.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.818+0000 7f169abba740 1 -- 172.21.15.123:0/3047634457 wait complete. 2024-09-19T04:54:32.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.819+0000 7f169abba740 1 -- start start 2024-09-19T04:54:32.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.819+0000 7f169abba740 1 Processor -- start 2024-09-19T04:54:32.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.819+0000 7f169abba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa7c70 0x55cde6abb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.819+0000 7f169abba740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa8590 0x55cde6ab6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.819+0000 7f169abba740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6ab6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.819+0000 7f169abba740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f16901b0eb0 con 0x55cde6ab2e50 2024-09-19T04:54:32.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.819+0000 7f169abba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f16901b07b0 con 0x55cde6aa7c70 2024-09-19T04:54:32.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.819+0000 7f169abba740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16901ad9b0 con 0x55cde6aa8590 2024-09-19T04:54:32.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.819+0000 7f169abba740 10 client.0 _ll_register_callbacks cb 0x7f167c0d3880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:32.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.820+0000 7f1696000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6ab6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.820+0000 7f1696000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6ab6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35820/0 (socket says 172.21.15.123:35820) 2024-09-19T04:54:32.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.820+0000 7f1696000640 1 -- 172.21.15.123:0/3226619091 learned_addr learned my addr 172.21.15.123:0/3226619091 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:32.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.820+0000 7f1695600640 1 --2- 172.21.15.123:0/3226619091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa7c70 0x55cde6abb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.820+0000 7f1694c00640 1 --2- 172.21.15.123:0/3226619091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa8590 0x55cde6ab6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.820+0000 7f1695600640 1 -- 172.21.15.123:0/3226619091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa8590 msgr2=0x55cde6ab6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:32.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.820+0000 7f1695600640 1 --2- 172.21.15.123:0/3226619091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa8590 0x55cde6ab6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.820+0000 7f1695600640 1 -- 172.21.15.123:0/3226619091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 msgr2=0x55cde6ab6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:32.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.820+0000 7f1695600640 1 --2- 172.21.15.123:0/3226619091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6ab6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.820+0000 7f1695600640 1 -- 172.21.15.123:0/3226619091 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cde6ab73e0 con 0x55cde6aa7c70 2024-09-19T04:54:32.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.820+0000 7f1696000640 1 --2- 172.21.15.123:0/3226619091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6ab6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:32.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.820+0000 7f1694c00640 1 --2- 172.21.15.123:0/3226619091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa8590 0x55cde6ab6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:32.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.821+0000 7f1695600640 1 --2- 172.21.15.123:0/3226619091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa7c70 0x55cde6abb340 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f167c00cf10 tx=0x7f16840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2dbbe1405cbe206 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:32.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.821+0000 7f1676a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:54:32.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.821+0000 7f1676a00640 1 -- 172.21.15.123:0/3226619091 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168400fdc0 con 0x55cde6aa7c70 2024-09-19T04:54:32.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.821+0000 7f1676a00640 1 -- 172.21.15.123:0/3226619091 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f1684004020 con 0x55cde6aa7c70 2024-09-19T04:54:32.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.821+0000 7f169abba740 1 -- 172.21.15.123:0/3226619091 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1688036040 con 0x55cde6aa7c70 2024-09-19T04:54:32.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.821+0000 7f1676a00640 1 -- 172.21.15.123:0/3226619091 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1684004220 con 0x55cde6aa7c70 2024-09-19T04:54:32.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.821+0000 7f169abba740 10 client.26915 fetch_fsmap learned FSMap version 41 2024-09-19T04:54:32.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.821+0000 7f169abba740 1 -- 172.21.15.123:0/3226619091 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55cde6b2da40 con 0x55cde6aa7c70 2024-09-19T04:54:32.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.822+0000 7f1676a00640 1 -- 172.21.15.123:0/3226619091 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1688036040 con 0x55cde6aa7c70 2024-09-19T04:54:32.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.822+0000 7f1676a00640 1 -- 172.21.15.123:0/3226619091 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f168402b540 con 0x55cde6aa7c70 2024-09-19T04:54:32.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.822+0000 7f169abba740 10 client.26915 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T04:54:32.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.822+0000 7f169abba740 10 client.26915 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:54:32.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.822+0000 7f169abba740 20 client.26915 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:32.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.822+0000 7f169abba740 1 --2- 172.21.15.123:0/3226619091 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55cde6b2e160 0x55cde6b30580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.822+0000 7f169abba740 4 client.26915 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:32.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.822+0000 7f169abba740 1 -- 172.21.15.123:0/3226619091 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cde6a15e60 con 0x55cde6b2e160 2024-09-19T04:54:32.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.822+0000 7f1694c00640 1 --2- 172.21.15.123:0/3226619091 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55cde6b2e160 0x55cde6b30580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.823+0000 7f1694c00640 1 --2- 172.21.15.123:0/3226619091 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55cde6b2e160 0x55cde6b30580 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf571bf682809dec server_cookie=eec3c58632ae71c8 in_seq=0 out_seq=0 2024-09-19T04:54:32.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.823+0000 7f1676a00640 10 client.26915 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:54:32.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.823+0000 7f1676a00640 1 -- 172.21.15.123:0/3226619091 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cde6a15e60 con 0x55cde6b2e160 2024-09-19T04:54:32.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.823+0000 7f1676a00640 10 client.26915 handle_command_reply: tid=0 2024-09-19T04:54:32.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.823+0000 7f169abba740 1 client.26915 shutdown 2024-09-19T04:54:32.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.823+0000 7f169abba740 1 -- 172.21.15.123:0/3226619091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa7c70 msgr2=0x55cde6abb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:32.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.823+0000 7f169abba740 1 --2- 172.21.15.123:0/3226619091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa7c70 0x55cde6abb340 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f167c00cf10 tx=0x7f16840046c0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.823+0000 7f169abba740 1 -- 172.21.15.123:0/3226619091 shutdown_connections 2024-09-19T04:54:32.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.823+0000 7f169abba740 1 --2- 172.21.15.123:0/3226619091 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55cde6b2e160 0x55cde6b30580 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.823+0000 7f169abba740 1 --2- 172.21.15.123:0/3226619091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cde6ab2e50 0x55cde6ab6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.823+0000 7f169abba740 1 --2- 172.21.15.123:0/3226619091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cde6aa8590 0x55cde6ab6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.823+0000 7f169abba740 1 --2- 172.21.15.123:0/3226619091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cde6aa7c70 0x55cde6abb340 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.823+0000 7f1676a00640 0 client.26915 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:54:32.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.823+0000 7f169abba740 1 -- 172.21.15.123:0/3226619091 >> 172.21.15.123:0/3226619091 conn(0x55cde6aa4200 msgr2=0x55cde6aa2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:32.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.823+0000 7f169abba740 1 -- 172.21.15.123:0/3226619091 shutdown_connections 2024-09-19T04:54:32.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.824+0000 7f169abba740 1 -- 172.21.15.123:0/3226619091 wait complete. 2024-09-19T04:54:32.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.824+0000 7f169abba740 20 client.26915 trim_cache size 0 max 16384 2024-09-19T04:54:32.829 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:32.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.824+0000 7f1698000640 1 -- 172.21.15.123:0/3520844620 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f16780a33b0 msgr2=0x7f16780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:32.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.824+0000 7f1698000640 1 --2- 172.21.15.123:0/3520844620 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f16780a33b0 0x7f16780a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1684009620 tx=0x7f1684005f90 comp rx=0 tx=0).stop 2024-09-19T04:54:32.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.824+0000 7f1698000640 1 -- 172.21.15.123:0/3520844620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16901049a0 msgr2=0x7f169019aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:32.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.824+0000 7f1698000640 1 --2- 172.21.15.123:0/3520844620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16901049a0 0x7f169019aae0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f167c00e9f0 tx=0x7f167c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.824+0000 7f1698000640 1 -- 172.21.15.123:0/3520844620 shutdown_connections 2024-09-19T04:54:32.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.824+0000 7f1698000640 1 --2- 172.21.15.123:0/3520844620 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f16780a33b0 0x7f16780a5870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.824+0000 7f1698000640 1 --2- 172.21.15.123:0/3520844620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1690109120 0x7f169019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.824+0000 7f1698000640 1 --2- 172.21.15.123:0/3520844620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16901049a0 0x7f169019aae0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.824+0000 7f1698000640 1 --2- 172.21.15.123:0/3520844620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1690103ff0 0x7f169019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.824+0000 7f1698000640 1 -- 172.21.15.123:0/3520844620 >> 172.21.15.123:0/3520844620 conn(0x7f16900ffa90 msgr2=0x7f16901005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:32.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.824+0000 7f1698000640 1 -- 172.21.15.123:0/3520844620 shutdown_connections 2024-09-19T04:54:32.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:32.824+0000 7f1698000640 1 -- 172.21.15.123:0/3520844620 wait complete. 2024-09-19T04:54:33.632 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:33 smithi123 ceph-mon[30511]: pgmap v242: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:33.632 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:33 smithi123 ceph-mon[30511]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:54:33.668 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:54:33.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:33 smithi183 ceph-mon[35333]: pgmap v242: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:33.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:33 smithi183 ceph-mon[35333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:54:33.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:33 smithi193 ceph-mon[35592]: pgmap v242: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:33.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:33 smithi193 ceph-mon[35592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:54:34.038 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:34.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.695+0000 7f9bb0600640 1 -- 172.21.15.123:0/2951184705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ba8101de0 msgr2=0x7f9ba81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:34.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.695+0000 7f9bb0600640 1 --2- 172.21.15.123:0/2951184705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ba8101de0 0x7f9ba81021c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009900 tx=0x7f9b9c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.695+0000 7f9bb0600640 1 -- 172.21.15.123:0/2951184705 shutdown_connections 2024-09-19T04:54:34.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.695+0000 7f9bb0600640 1 --2- 172.21.15.123:0/2951184705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ba8106f10 0x7f9ba810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.695+0000 7f9bb0600640 1 --2- 172.21.15.123:0/2951184705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ba8102790 0x7f9ba81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.695+0000 7f9bb0600640 1 --2- 172.21.15.123:0/2951184705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ba8101de0 0x7f9ba81021c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.695+0000 7f9bb0600640 1 -- 172.21.15.123:0/2951184705 >> 172.21.15.123:0/2951184705 conn(0x7f9ba80fd8c0 msgr2=0x7f9ba80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:34.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.695+0000 7f9bb0600640 1 -- 172.21.15.123:0/2951184705 shutdown_connections 2024-09-19T04:54:34.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.695+0000 7f9bb0600640 1 -- 172.21.15.123:0/2951184705 wait complete. 2024-09-19T04:54:34.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.696+0000 7f9bb0600640 1 Processor -- start 2024-09-19T04:54:34.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.697+0000 7f9bb0600640 1 -- start start 2024-09-19T04:54:34.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.697+0000 7f9bb0600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ba8101de0 0x7f9ba81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.697+0000 7f9bb0600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ba8102790 0x7f9ba8198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.697+0000 7f9bb0600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ba8106f10 0x7f9ba819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.697+0000 7f9bb0600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8111f80 con 0x7f9ba8106f10 2024-09-19T04:54:34.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.697+0000 7f9bb0600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8111e00 con 0x7f9ba8102790 2024-09-19T04:54:34.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.697+0000 7f9bb0600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8112100 con 0x7f9ba8101de0 2024-09-19T04:54:34.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.697+0000 7f9badc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ba8101de0 0x7f9ba81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.697+0000 7f9bae600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ba8106f10 0x7f9ba819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.697+0000 7f9badc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ba8101de0 0x7f9ba81984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:35708/0 (socket says 172.21.15.123:35708) 2024-09-19T04:54:34.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.697+0000 7f9badc00640 1 -- 172.21.15.123:0/1265601956 learned_addr learned my addr 172.21.15.123:0/1265601956 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:34.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.697+0000 7f9bad200640 1 --2- 172.21.15.123:0/1265601956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ba8102790 0x7f9ba8198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.698+0000 7f9badc00640 1 -- 172.21.15.123:0/1265601956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ba8102790 msgr2=0x7f9ba8198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:34.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.698+0000 7f9badc00640 1 --2- 172.21.15.123:0/1265601956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ba8102790 0x7f9ba8198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.698+0000 7f9badc00640 1 -- 172.21.15.123:0/1265601956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ba8106f10 msgr2=0x7f9ba819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:34.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.698+0000 7f9badc00640 1 --2- 172.21.15.123:0/1265601956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ba8106f10 0x7f9ba819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.698+0000 7f9badc00640 1 -- 172.21.15.123:0/1265601956 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba819d490 con 0x7f9ba8101de0 2024-09-19T04:54:34.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.698+0000 7f9badc00640 1 --2- 172.21.15.123:0/1265601956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ba8101de0 0x7f9ba81984e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c004770 tx=0x7f9b9c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:34.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.699+0000 7f9ba6000640 1 -- 172.21.15.123:0/1265601956 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c03f070 con 0x7f9ba8101de0 2024-09-19T04:54:34.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.699+0000 7f9bb0600640 1 -- 172.21.15.123:0/1265601956 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba819d720 con 0x7f9ba8101de0 2024-09-19T04:54:34.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.699+0000 7f9ba6000640 1 -- 172.21.15.123:0/1265601956 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b9c031ca0 con 0x7f9ba8101de0 2024-09-19T04:54:34.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.699+0000 7f9ba6000640 1 -- 172.21.15.123:0/1265601956 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c035030 con 0x7f9ba8101de0 2024-09-19T04:54:34.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.699+0000 7f9bb0600640 1 -- 172.21.15.123:0/1265601956 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba81a5060 con 0x7f9ba8101de0 2024-09-19T04:54:34.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.702+0000 7f9bb31f9740 1 Processor -- start 2024-09-19T04:54:34.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.702+0000 7f9bb31f9740 1 -- start start 2024-09-19T04:54:34.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.702+0000 7f9ba6000640 1 -- 172.21.15.123:0/1265601956 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9b9c042070 con 0x7f9ba8101de0 2024-09-19T04:54:34.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.702+0000 7f9bb31f9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363dc70 0x56259363e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.702+0000 7f9bb31f9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56259363e590 0x562593648910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.702+0000 7f9bb31f9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562593648e50 0x56259364b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.702+0000 7f9bb31f9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5625935a43e0 con 0x56259363dc70 2024-09-19T04:54:34.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.702+0000 7f9bb31f9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56259354bd80 con 0x562593648e50 2024-09-19T04:54:34.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.702+0000 7f9bb31f9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5625935a03a0 con 0x56259363e590 2024-09-19T04:54:34.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.703+0000 7f9bae600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363dc70 0x56259363e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.703+0000 7f9bae600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363dc70 0x56259363e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35850/0 (socket says 172.21.15.123:35850) 2024-09-19T04:54:34.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.703+0000 7f9bae600640 1 -- 172.21.15.123:0/1909605391 learned_addr learned my addr 172.21.15.123:0/1909605391 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:34.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.703+0000 7f9badc00640 1 --2- 172.21.15.123:0/1909605391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56259363e590 0x562593648910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.703+0000 7f9bad200640 1 --2- 172.21.15.123:0/1909605391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562593648e50 0x56259364b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.703+0000 7f9bae600640 1 -- 172.21.15.123:0/1909605391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56259363e590 msgr2=0x562593648910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:34.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.703+0000 7f9bae600640 1 --2- 172.21.15.123:0/1909605391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56259363e590 0x562593648910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.703+0000 7f9bae600640 1 -- 172.21.15.123:0/1909605391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562593648e50 msgr2=0x56259364b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:34.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.703+0000 7f9bae600640 1 --2- 172.21.15.123:0/1909605391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562593648e50 0x56259364b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.703+0000 7f9bae600640 1 -- 172.21.15.123:0/1909605391 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56259356aae0 con 0x56259363dc70 2024-09-19T04:54:34.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.703+0000 7f9bae600640 1 --2- 172.21.15.123:0/1909605391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363dc70 0x56259363e050 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009f90 tx=0x7f9ba0010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70be72468b88d7c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:34.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.704+0000 7f9b9aa00640 1 -- 172.21.15.123:0/1909605391 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0011700 con 0x56259363dc70 2024-09-19T04:54:34.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.704+0000 7f9b9aa00640 1 -- 172.21.15.123:0/1909605391 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ba0011d80 con 0x56259363dc70 2024-09-19T04:54:34.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.704+0000 7f9b9aa00640 1 -- 172.21.15.123:0/1909605391 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0018380 con 0x56259363dc70 2024-09-19T04:54:34.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.704+0000 7f9bb31f9740 1 -- 172.21.15.123:0/1909605391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363dc70 msgr2=0x56259363e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:34.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.704+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/1909605391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363dc70 0x56259363e050 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009f90 tx=0x7f9ba0010860 comp rx=0 tx=0).stop 2024-09-19T04:54:34.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.704+0000 7f9ba6000640 1 --2- 172.21.15.123:0/1265601956 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9b900a33b0 0x7f9b900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.705+0000 7f9ba6000640 1 -- 172.21.15.123:0/1265601956 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f9b9c0f04f0 con 0x7f9ba8101de0 2024-09-19T04:54:34.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.705+0000 7f9bb31f9740 1 -- 172.21.15.123:0/1909605391 shutdown_connections 2024-09-19T04:54:34.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.705+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/1909605391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562593648e50 0x56259364b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.705+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/1909605391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56259363e590 0x562593648910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.705+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/1909605391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363dc70 0x56259363e050 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.705+0000 7f9bb31f9740 1 -- 172.21.15.123:0/1909605391 >> 172.21.15.123:0/1909605391 conn(0x56259363a200 msgr2=0x56259363a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:34.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.705+0000 7f9bae600640 1 --2- 172.21.15.123:0/1265601956 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9b900a33b0 0x7f9b900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.705+0000 7f9bb31f9740 1 -- 172.21.15.123:0/1909605391 shutdown_connections 2024-09-19T04:54:34.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.705+0000 7f9bb31f9740 1 -- 172.21.15.123:0/1909605391 wait complete. 2024-09-19T04:54:34.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.705+0000 7f9bae600640 1 --2- 172.21.15.123:0/1265601956 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9b900a33b0 0x7f9b900a5870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0010d60 tx=0x7f9ba0010fd0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:34.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.705+0000 7f9bb31f9740 1 -- start start 2024-09-19T04:54:34.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.706+0000 7f9bb31f9740 1 Processor -- start 2024-09-19T04:54:34.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.706+0000 7f9bb31f9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56259363dc70 0x562593645d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.706+0000 7f9bb31f9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56259363e590 0x5625936462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.706+0000 7f9bb31f9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562593648e50 0x5625936467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.706+0000 7f9bb31f9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56259364ba20 con 0x562593648e50 2024-09-19T04:54:34.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.706+0000 7f9bb31f9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56259355f0f0 con 0x56259363e590 2024-09-19T04:54:34.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.706+0000 7f9bb31f9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56259364bba0 con 0x56259363dc70 2024-09-19T04:54:34.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.706+0000 7f9bb31f9740 10 client.0 _ll_register_callbacks cb 0x562593546bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:34.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.706+0000 7f9bae600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562593648e50 0x5625936467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.707+0000 7f9bae600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562593648e50 0x5625936467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35862/0 (socket says 172.21.15.123:35862) 2024-09-19T04:54:34.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.707+0000 7f9bae600640 1 -- 172.21.15.123:0/854371163 learned_addr learned my addr 172.21.15.123:0/854371163 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:34.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.707+0000 7f9badc00640 1 --2- 172.21.15.123:0/854371163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56259363dc70 0x562593645d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.707+0000 7f9bad200640 1 --2- 172.21.15.123:0/854371163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56259363e590 0x5625936462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.707+0000 7f9bae600640 1 -- 172.21.15.123:0/854371163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56259363dc70 msgr2=0x562593645d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:34.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.707+0000 7f9bae600640 1 --2- 172.21.15.123:0/854371163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56259363dc70 0x562593645d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.707+0000 7f9bae600640 1 -- 172.21.15.123:0/854371163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56259363e590 msgr2=0x5625936462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:34.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.707+0000 7f9bae600640 1 --2- 172.21.15.123:0/854371163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56259363e590 0x5625936462a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.707+0000 7f9bae600640 1 -- 172.21.15.123:0/854371163 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625933153c0 con 0x562593648e50 2024-09-19T04:54:34.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.707+0000 7f9badc00640 1 --2- 172.21.15.123:0/854371163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56259363dc70 0x562593645d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:34.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.707+0000 7f9bae600640 1 --2- 172.21.15.123:0/854371163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562593648e50 0x5625936467e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009f90 tx=0x7f9ba0019a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848125a5e39ae5cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:34.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.707+0000 7f9b8f400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:34.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.708+0000 7f9b8f400640 1 -- 172.21.15.123:0/854371163 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0003b70 con 0x562593648e50 2024-09-19T04:54:34.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.708+0000 7f9b8f400640 1 -- 172.21.15.123:0/854371163 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f9ba0003d10 con 0x562593648e50 2024-09-19T04:54:34.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.708+0000 7f9bb31f9740 1 -- 172.21.15.123:0/854371163 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ba0011d80 con 0x562593648e50 2024-09-19T04:54:34.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.708+0000 7f9b8f400640 1 -- 172.21.15.123:0/854371163 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9ba0011500 con 0x562593648e50 2024-09-19T04:54:34.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.708+0000 7f9bb31f9740 10 client.16833 fetch_fsmap learned FSMap version 41 2024-09-19T04:54:34.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.708+0000 7f9bb31f9740 1 -- 172.21.15.123:0/854371163 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562593657a90 con 0x562593648e50 2024-09-19T04:54:34.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.708+0000 7f9b8f400640 1 -- 172.21.15.123:0/854371163 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ba0011d80 con 0x562593648e50 2024-09-19T04:54:34.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.708+0000 7f9b8f400640 1 -- 172.21.15.123:0/854371163 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f9ba0004510 con 0x562593648e50 2024-09-19T04:54:34.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.708+0000 7f9bb31f9740 10 client.16833 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T04:54:34.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.708+0000 7f9bb31f9740 10 client.16833 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:54:34.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.708+0000 7f9bb31f9740 20 client.16833 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:34.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.708+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/854371163 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x562593658de0 0x56259365b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.709+0000 7f9bb31f9740 4 client.16833 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:34.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.709+0000 7f9bb31f9740 1 -- 172.21.15.123:0/854371163 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625935b1950 con 0x562593658de0 2024-09-19T04:54:34.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.709+0000 7f9badc00640 1 --2- 172.21.15.123:0/854371163 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x562593658de0 0x56259365b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.709+0000 7f9badc00640 1 --2- 172.21.15.123:0/854371163 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x562593658de0 0x56259365b1d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2358f7e4becae97 server_cookie=e6b56365824f6aa0 in_seq=0 out_seq=0 2024-09-19T04:54:34.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.709+0000 7f9b8f400640 10 client.16833 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:54:34.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.710+0000 7f9b8f400640 1 -- 172.21.15.123:0/854371163 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625935b1950 con 0x562593658de0 2024-09-19T04:54:34.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.710+0000 7f9b8f400640 10 client.16833 handle_command_reply: tid=0 2024-09-19T04:54:34.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.710+0000 7f9bb31f9740 1 client.16833 shutdown 2024-09-19T04:54:34.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.711+0000 7f9bb31f9740 1 -- 172.21.15.123:0/854371163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562593648e50 msgr2=0x5625936467e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:34.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.711+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/854371163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562593648e50 0x5625936467e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009f90 tx=0x7f9ba0019a30 comp rx=0 tx=0).stop 2024-09-19T04:54:34.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.711+0000 7f9bb31f9740 1 -- 172.21.15.123:0/854371163 shutdown_connections 2024-09-19T04:54:34.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.711+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/854371163 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x562593658de0 0x56259365b1d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.711+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/854371163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562593648e50 0x5625936467e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.711+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/854371163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56259363e590 0x5625936462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.711+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/854371163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56259363dc70 0x562593645d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.711+0000 7f9bb31f9740 1 -- 172.21.15.123:0/854371163 >> 172.21.15.123:0/854371163 conn(0x56259363a200 msgr2=0x562593639380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:34.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.711+0000 7f9b8f400640 0 client.16833 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:54:34.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.711+0000 7f9bb31f9740 1 -- 172.21.15.123:0/854371163 shutdown_connections 2024-09-19T04:54:34.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.711+0000 7f9bb31f9740 1 -- 172.21.15.123:0/854371163 wait complete. 2024-09-19T04:54:34.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.711+0000 7f9bb31f9740 20 client.16833 trim_cache size 0 max 16384 2024-09-19T04:54:34.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.725+0000 7f9bb31f9740 1 Processor -- start 2024-09-19T04:54:34.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.725+0000 7f9bb31f9740 1 -- start start 2024-09-19T04:54:34.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.725+0000 7f9bb31f9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56259363dc70 0x5625936406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.725+0000 7f9bb31f9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363e590 0x562593660a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.725+0000 7f9bb31f9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562593648e50 0x562593660f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.725+0000 7f9bb31f9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba81ad910 con 0x56259363e590 2024-09-19T04:54:34.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.725+0000 7f9bb31f9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba81af4e0 con 0x562593648e50 2024-09-19T04:54:34.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.725+0000 7f9bb31f9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba81ae010 con 0x56259363dc70 2024-09-19T04:54:34.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.725+0000 7f9bad200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363e590 0x562593660a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.725+0000 7f9bad200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363e590 0x562593660a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35866/0 (socket says 172.21.15.123:35866) 2024-09-19T04:54:34.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.725+0000 7f9bad200640 1 -- 172.21.15.123:0/723072844 learned_addr learned my addr 172.21.15.123:0/723072844 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:34.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.725+0000 7f9badc00640 1 --2- 172.21.15.123:0/723072844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56259363dc70 0x5625936406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.725+0000 7f9bae600640 1 --2- 172.21.15.123:0/723072844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562593648e50 0x562593660f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.726+0000 7f9bad200640 1 -- 172.21.15.123:0/723072844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56259363dc70 msgr2=0x5625936406d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:34.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.726+0000 7f9bad200640 1 --2- 172.21.15.123:0/723072844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56259363dc70 0x5625936406d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.726+0000 7f9bad200640 1 -- 172.21.15.123:0/723072844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562593648e50 msgr2=0x562593660f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:34.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.726+0000 7f9bad200640 1 --2- 172.21.15.123:0/723072844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562593648e50 0x562593660f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.726+0000 7f9bad200640 1 -- 172.21.15.123:0/723072844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56259363daa0 con 0x56259363e590 2024-09-19T04:54:34.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.726+0000 7f9bad200640 1 --2- 172.21.15.123:0/723072844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363e590 0x562593660a50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9b94005fd0 tx=0x7f9b94011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de12f627e57513d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:34.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.726+0000 7f9b9a000640 1 -- 172.21.15.123:0/723072844 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94004020 con 0x56259363e590 2024-09-19T04:54:34.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.726+0000 7f9b9a000640 1 -- 172.21.15.123:0/723072844 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b94005500 con 0x56259363e590 2024-09-19T04:54:34.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.726+0000 7f9b9a000640 1 -- 172.21.15.123:0/723072844 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94004bd0 con 0x56259363e590 2024-09-19T04:54:34.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.727+0000 7f9bb31f9740 1 -- 172.21.15.123:0/723072844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363e590 msgr2=0x562593660a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:34.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.727+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/723072844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363e590 0x562593660a50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9b94005fd0 tx=0x7f9b94011810 comp rx=0 tx=0).stop 2024-09-19T04:54:34.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.727+0000 7f9bb31f9740 1 -- 172.21.15.123:0/723072844 shutdown_connections 2024-09-19T04:54:34.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.727+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/723072844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562593648e50 0x562593660f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.727+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/723072844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363e590 0x562593660a50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.727+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/723072844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56259363dc70 0x5625936406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.727+0000 7f9bb31f9740 1 -- 172.21.15.123:0/723072844 >> 172.21.15.123:0/723072844 conn(0x56259363a200 msgr2=0x562593644e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:34.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.727+0000 7f9bb31f9740 1 -- 172.21.15.123:0/723072844 shutdown_connections 2024-09-19T04:54:34.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.727+0000 7f9bb31f9740 1 -- 172.21.15.123:0/723072844 wait complete. 2024-09-19T04:54:34.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.727+0000 7f9bb31f9740 1 -- start start 2024-09-19T04:54:34.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.728+0000 7f9bb31f9740 1 Processor -- start 2024-09-19T04:54:34.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.728+0000 7f9bb31f9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363dc70 0x562593651640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.728+0000 7f9bb31f9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56259363e590 0x56259364bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.728+0000 7f9bb31f9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562593648e50 0x56259364c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.728+0000 7f9bb31f9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba81aee10 con 0x56259363dc70 2024-09-19T04:54:34.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.728+0000 7f9bb31f9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba81ae710 con 0x56259363e590 2024-09-19T04:54:34.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.728+0000 7f9bb31f9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba81ab910 con 0x562593648e50 2024-09-19T04:54:34.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.728+0000 7f9bb31f9740 10 client.0 _ll_register_callbacks cb 0x7f9b9400b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:34.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.728+0000 7f9badc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363dc70 0x562593651640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.728+0000 7f9badc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363dc70 0x562593651640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35874/0 (socket says 172.21.15.123:35874) 2024-09-19T04:54:34.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.728+0000 7f9badc00640 1 -- 172.21.15.123:0/1037390063 learned_addr learned my addr 172.21.15.123:0/1037390063 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:34.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.729+0000 7f9bae600640 1 --2- 172.21.15.123:0/1037390063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562593648e50 0x56259364c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.729+0000 7f9bad200640 1 --2- 172.21.15.123:0/1037390063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56259363e590 0x56259364bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.729+0000 7f9badc00640 1 -- 172.21.15.123:0/1037390063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562593648e50 msgr2=0x56259364c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:34.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.729+0000 7f9badc00640 1 --2- 172.21.15.123:0/1037390063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562593648e50 0x56259364c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.729+0000 7f9badc00640 1 -- 172.21.15.123:0/1037390063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56259363e590 msgr2=0x56259364bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:34.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.729+0000 7f9badc00640 1 --2- 172.21.15.123:0/1037390063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56259363e590 0x56259364bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.729+0000 7f9badc00640 1 -- 172.21.15.123:0/1037390063 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625936c1a10 con 0x56259363dc70 2024-09-19T04:54:34.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.729+0000 7f9bad200640 1 --2- 172.21.15.123:0/1037390063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56259363e590 0x56259364bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:54:34.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.729+0000 7f9badc00640 1 --2- 172.21.15.123:0/1037390063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363dc70 0x562593651640 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0b8c80 tx=0x7f9b9c0f6060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3490b9e440f662c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:34.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.729+0000 7f9b8f400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:34.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.729+0000 7f9b8f400640 1 -- 172.21.15.123:0/1037390063 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c049040 con 0x56259363dc70 2024-09-19T04:54:34.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.729+0000 7f9b8f400640 1 -- 172.21.15.123:0/1037390063 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f9b9c03b5f0 con 0x56259363dc70 2024-09-19T04:54:34.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.729+0000 7f9bb31f9740 1 -- 172.21.15.123:0/1037390063 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b94005500 con 0x56259363dc70 2024-09-19T04:54:34.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.730+0000 7f9b8f400640 1 -- 172.21.15.123:0/1037390063 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9b9c0b8ac0 con 0x56259363dc70 2024-09-19T04:54:34.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.730+0000 7f9bb31f9740 10 client.16839 fetch_fsmap learned FSMap version 41 2024-09-19T04:54:34.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.730+0000 7f9bb31f9740 1 -- 172.21.15.123:0/1037390063 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5625936c2bd0 con 0x56259363dc70 2024-09-19T04:54:34.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.730+0000 7f9b8f400640 1 -- 172.21.15.123:0/1037390063 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b94005500 con 0x56259363dc70 2024-09-19T04:54:34.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.730+0000 7f9b8f400640 1 -- 172.21.15.123:0/1037390063 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f9b9c0ede60 con 0x56259363dc70 2024-09-19T04:54:34.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.730+0000 7f9bb31f9740 10 client.16839 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T04:54:34.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.730+0000 7f9bb31f9740 10 client.16839 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:54:34.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.730+0000 7f9bb31f9740 20 client.16839 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:34.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.730+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/1037390063 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5625936c3320 0x5625936c5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.730+0000 7f9bb31f9740 4 client.16839 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:34.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.730+0000 7f9bb31f9740 1 -- 172.21.15.123:0/1037390063 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625935abe60 con 0x5625936c3320 2024-09-19T04:54:34.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.730+0000 7f9bad200640 1 --2- 172.21.15.123:0/1037390063 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5625936c3320 0x5625936c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.731+0000 7f9bad200640 1 --2- 172.21.15.123:0/1037390063 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5625936c3320 0x5625936c5740 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d70cbc27ae136a3 server_cookie=ce5d431ce050a18d in_seq=0 out_seq=0 2024-09-19T04:54:34.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.731+0000 7f9b8f400640 10 client.16839 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:54:34.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.731+0000 7f9b8f400640 1 -- 172.21.15.123:0/1037390063 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625935abe60 con 0x5625936c3320 2024-09-19T04:54:34.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.731+0000 7f9b8f400640 10 client.16839 handle_command_reply: tid=0 2024-09-19T04:54:34.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.731+0000 7f9bb31f9740 1 client.16839 shutdown 2024-09-19T04:54:34.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.731+0000 7f9bb31f9740 1 -- 172.21.15.123:0/1037390063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363dc70 msgr2=0x562593651640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:34.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.731+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/1037390063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363dc70 0x562593651640 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0b8c80 tx=0x7f9b9c0f6060 comp rx=0 tx=0).stop 2024-09-19T04:54:34.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.731+0000 7f9bb31f9740 1 -- 172.21.15.123:0/1037390063 shutdown_connections 2024-09-19T04:54:34.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.731+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/1037390063 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x5625936c3320 0x5625936c5740 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.732+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/1037390063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562593648e50 0x56259364c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.732+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/1037390063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56259363e590 0x56259364bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.732+0000 7f9bb31f9740 1 --2- 172.21.15.123:0/1037390063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56259363dc70 0x562593651640 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.732+0000 7f9bb31f9740 1 -- 172.21.15.123:0/1037390063 >> 172.21.15.123:0/1037390063 conn(0x56259363a200 msgr2=0x562593638b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:34.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.732+0000 7f9b8f400640 0 client.16839 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:54:34.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.732+0000 7f9bb31f9740 1 -- 172.21.15.123:0/1037390063 shutdown_connections 2024-09-19T04:54:34.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.732+0000 7f9bb31f9740 1 -- 172.21.15.123:0/1037390063 wait complete. 2024-09-19T04:54:34.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.732+0000 7f9bb31f9740 20 client.16839 trim_cache size 0 max 16384 2024-09-19T04:54:34.739 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:34.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.732+0000 7f9bb0600640 1 -- 172.21.15.123:0/1265601956 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9b900a33b0 msgr2=0x7f9b900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:34.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.732+0000 7f9bb0600640 1 --2- 172.21.15.123:0/1265601956 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9b900a33b0 0x7f9b900a5870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0010d60 tx=0x7f9ba0010fd0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.733+0000 7f9bb0600640 1 -- 172.21.15.123:0/1265601956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ba8101de0 msgr2=0x7f9ba81984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:34.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.733+0000 7f9bb0600640 1 --2- 172.21.15.123:0/1265601956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ba8101de0 0x7f9ba81984e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c004770 tx=0x7f9b9c003c10 comp rx=0 tx=0).stop 2024-09-19T04:54:34.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.733+0000 7f9bb0600640 1 -- 172.21.15.123:0/1265601956 shutdown_connections 2024-09-19T04:54:34.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.733+0000 7f9bb0600640 1 --2- 172.21.15.123:0/1265601956 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9b900a33b0 0x7f9b900a5870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.733+0000 7f9bb0600640 1 --2- 172.21.15.123:0/1265601956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ba8106f10 0x7f9ba819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.733+0000 7f9bb0600640 1 --2- 172.21.15.123:0/1265601956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ba8102790 0x7f9ba8198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.733+0000 7f9bb0600640 1 --2- 172.21.15.123:0/1265601956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ba8101de0 0x7f9ba81984e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.733+0000 7f9bb0600640 1 -- 172.21.15.123:0/1265601956 >> 172.21.15.123:0/1265601956 conn(0x7f9ba80fd8c0 msgr2=0x7f9ba80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:34.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.733+0000 7f9bb0600640 1 -- 172.21.15.123:0/1265601956 shutdown_connections 2024-09-19T04:54:34.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:34.733+0000 7f9bb0600640 1 -- 172.21.15.123:0/1265601956 wait complete. 2024-09-19T04:54:35.513 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:54:35.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:35 smithi183 ceph-mon[35333]: pgmap v243: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:35.883 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:35.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:35 smithi123 ceph-mon[30511]: pgmap v243: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:35.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:35 smithi193 ceph-mon[35592]: pgmap v243: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:36.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.426+0000 7fad71600640 1 -- 172.21.15.123:0/2827906890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad6c074590 msgr2=0x7fad6c074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:36.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.426+0000 7fad71600640 1 --2- 172.21.15.123:0/2827906890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad6c074590 0x7fad6c074970 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009900 tx=0x7fad5c02a480 comp rx=0 tx=0).stop 2024-09-19T04:54:36.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.426+0000 7fad71600640 1 -- 172.21.15.123:0/2827906890 shutdown_connections 2024-09-19T04:54:36.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.426+0000 7fad71600640 1 --2- 172.21.15.123:0/2827906890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad6c1129e0 0x7fad6c114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.426+0000 7fad71600640 1 --2- 172.21.15.123:0/2827906890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad6c074eb0 0x7fad6c1124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.426+0000 7fad71600640 1 --2- 172.21.15.123:0/2827906890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad6c074590 0x7fad6c074970 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.426+0000 7fad71600640 1 -- 172.21.15.123:0/2827906890 >> 172.21.15.123:0/2827906890 conn(0x7fad6c0fd710 msgr2=0x7fad6c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:36.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.426+0000 7fad71600640 1 -- 172.21.15.123:0/2827906890 shutdown_connections 2024-09-19T04:54:36.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.426+0000 7fad71600640 1 -- 172.21.15.123:0/2827906890 wait complete. 2024-09-19T04:54:36.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.427+0000 7fad71600640 1 Processor -- start 2024-09-19T04:54:36.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.427+0000 7fad71600640 1 -- start start 2024-09-19T04:54:36.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad71600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad6c074590 0x7fad6c1a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad71600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad6c074eb0 0x7fad6c1a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad71600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad6c1129e0 0x7fad6c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad71600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1183c0 con 0x7fad6c1129e0 2024-09-19T04:54:36.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad71600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c118240 con 0x7fad6c074eb0 2024-09-19T04:54:36.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad71600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c118540 con 0x7fad6c074590 2024-09-19T04:54:36.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad63e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad6c074eb0 0x7fad6c1a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad70c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad6c1129e0 0x7fad6c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad6be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad6c074590 0x7fad6c1a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad6be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad6c074590 0x7fad6c1a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:35764/0 (socket says 172.21.15.123:35764) 2024-09-19T04:54:36.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad63e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad6c074eb0 0x7fad6c1a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:54830/0 (socket says 172.21.15.123:54830) 2024-09-19T04:54:36.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad6be00640 1 -- 172.21.15.123:0/2673174216 learned_addr learned my addr 172.21.15.123:0/2673174216 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:36.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad63e00640 1 -- 172.21.15.123:0/2673174216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad6c074590 msgr2=0x7fad6c1a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:36.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad63e00640 1 --2- 172.21.15.123:0/2673174216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad6c074590 0x7fad6c1a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad63e00640 1 -- 172.21.15.123:0/2673174216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad6c1129e0 msgr2=0x7fad6c19e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:36.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad63e00640 1 --2- 172.21.15.123:0/2673174216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad6c1129e0 0x7fad6c19e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad63e00640 1 -- 172.21.15.123:0/2673174216 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c19ef50 con 0x7fad6c074eb0 2024-09-19T04:54:36.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad6be00640 1 --2- 172.21.15.123:0/2673174216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad6c074590 0x7fad6c1a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:36.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.428+0000 7fad70c00640 1 --2- 172.21.15.123:0/2673174216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad6c1129e0 0x7fad6c19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:36.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.429+0000 7fad63e00640 1 --2- 172.21.15.123:0/2673174216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad6c074eb0 0x7fad6c1a4b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fad5400e9f0 tx=0x7fad5400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:36.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.429+0000 7fad69600640 1 -- 172.21.15.123:0/2673174216 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5400ccc0 con 0x7fad6c074eb0 2024-09-19T04:54:36.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.429+0000 7fad71600640 1 -- 172.21.15.123:0/2673174216 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6c19f240 con 0x7fad6c074eb0 2024-09-19T04:54:36.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.429+0000 7fad69600640 1 -- 172.21.15.123:0/2673174216 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad54005020 con 0x7fad6c074eb0 2024-09-19T04:54:36.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.429+0000 7fad69600640 1 -- 172.21.15.123:0/2673174216 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad540049e0 con 0x7fad6c074eb0 2024-09-19T04:54:36.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.429+0000 7fad71600640 1 -- 172.21.15.123:0/2673174216 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6c1ab490 con 0x7fad6c074eb0 2024-09-19T04:54:36.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.432+0000 7fad75655740 1 Processor -- start 2024-09-19T04:54:36.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.432+0000 7fad75655740 1 -- start start 2024-09-19T04:54:36.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.432+0000 7fad69600640 1 -- 172.21.15.123:0/2673174216 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fad54005560 con 0x7fad6c074eb0 2024-09-19T04:54:36.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.432+0000 7fad75655740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf1e40 0x55770baf2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.432+0000 7fad75655740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770baf2760 0x55770bafcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.433+0000 7fad75655740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770bafd020 0x55770baff410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.433+0000 7fad75655740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55770ba583e0 con 0x55770baf1e40 2024-09-19T04:54:36.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.433+0000 7fad75655740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55770b9ffd80 con 0x55770baf2760 2024-09-19T04:54:36.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.433+0000 7fad75655740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55770ba543a0 con 0x55770bafd020 2024-09-19T04:54:36.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.433+0000 7fad70c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf1e40 0x55770baf2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.433+0000 7fad70c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf1e40 0x55770baf2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35910/0 (socket says 172.21.15.123:35910) 2024-09-19T04:54:36.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.433+0000 7fad70c00640 1 -- 172.21.15.123:0/1370575747 learned_addr learned my addr 172.21.15.123:0/1370575747 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:36.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.433+0000 7fad6be00640 1 --2- 172.21.15.123:0/1370575747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770baf2760 0x55770bafcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.433+0000 7fad63e00640 1 --2- 172.21.15.123:0/1370575747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770bafd020 0x55770baff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.433+0000 7fad70c00640 1 -- 172.21.15.123:0/1370575747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770bafd020 msgr2=0x55770baff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:36.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.433+0000 7fad70c00640 1 --2- 172.21.15.123:0/1370575747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770bafd020 0x55770baff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.433+0000 7fad70c00640 1 -- 172.21.15.123:0/1370575747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770baf2760 msgr2=0x55770bafcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:36.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.433+0000 7fad70c00640 1 --2- 172.21.15.123:0/1370575747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770baf2760 0x55770bafcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.433+0000 7fad70c00640 1 -- 172.21.15.123:0/1370575747 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55770ba1eae0 con 0x55770baf1e40 2024-09-19T04:54:36.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.433+0000 7fad63e00640 1 --2- 172.21.15.123:0/1370575747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770bafd020 0x55770baff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:54:36.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.434+0000 7fad70c00640 1 --2- 172.21.15.123:0/1370575747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf1e40 0x55770baf2220 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fad64011760 tx=0x7fad64011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3463af9b27adccd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:36.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.434+0000 7fad61200640 1 -- 172.21.15.123:0/1370575747 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64004d20 con 0x55770baf1e40 2024-09-19T04:54:36.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.434+0000 7fad61200640 1 -- 172.21.15.123:0/1370575747 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad6400f9a0 con 0x55770baf1e40 2024-09-19T04:54:36.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.434+0000 7fad61200640 1 -- 172.21.15.123:0/1370575747 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64004590 con 0x55770baf1e40 2024-09-19T04:54:36.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.434+0000 7fad69600640 1 --2- 172.21.15.123:0/2673174216 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fad500a3430 0x7fad500a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.434+0000 7fad69600640 1 -- 172.21.15.123:0/2673174216 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fad540cfca0 con 0x7fad6c074eb0 2024-09-19T04:54:36.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.434+0000 7fad6be00640 1 --2- 172.21.15.123:0/2673174216 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fad500a3430 0x7fad500a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.435+0000 7fad75655740 1 -- 172.21.15.123:0/1370575747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf1e40 msgr2=0x55770baf2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:36.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.435+0000 7fad75655740 1 --2- 172.21.15.123:0/1370575747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf1e40 0x55770baf2220 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fad64011760 tx=0x7fad64011c20 comp rx=0 tx=0).stop 2024-09-19T04:54:36.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.435+0000 7fad75655740 1 -- 172.21.15.123:0/1370575747 shutdown_connections 2024-09-19T04:54:36.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.435+0000 7fad75655740 1 --2- 172.21.15.123:0/1370575747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770bafd020 0x55770baff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.435+0000 7fad75655740 1 --2- 172.21.15.123:0/1370575747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770baf2760 0x55770bafcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.435+0000 7fad75655740 1 --2- 172.21.15.123:0/1370575747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf1e40 0x55770baf2220 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.435+0000 7fad75655740 1 -- 172.21.15.123:0/1370575747 >> 172.21.15.123:0/1370575747 conn(0x55770baee3d0 msgr2=0x55770baee7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:36.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.435+0000 7fad75655740 1 -- 172.21.15.123:0/1370575747 shutdown_connections 2024-09-19T04:54:36.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.435+0000 7fad75655740 1 -- 172.21.15.123:0/1370575747 wait complete. 2024-09-19T04:54:36.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.435+0000 7fad6be00640 1 --2- 172.21.15.123:0/2673174216 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fad500a3430 0x7fad500a58f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55770baef230 tx=0x7fad5c005a20 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:36.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.436+0000 7fad75655740 1 -- start start 2024-09-19T04:54:36.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.436+0000 7fad75655740 1 Processor -- start 2024-09-19T04:54:36.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.436+0000 7fad75655740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770baf1e40 0x55770baf9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.436+0000 7fad75655740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770baf2760 0x55770bafa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.436+0000 7fad75655740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770bafd020 0x55770bafa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.436+0000 7fad75655740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55770baffbf0 con 0x55770bafd020 2024-09-19T04:54:36.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.436+0000 7fad75655740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55770ba130f0 con 0x55770baf2760 2024-09-19T04:54:36.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.436+0000 7fad75655740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55770baffd70 con 0x55770baf1e40 2024-09-19T04:54:36.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.436+0000 7fad75655740 10 client.0 _ll_register_callbacks cb 0x55770b9fabd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:36.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.437+0000 7fad70c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770bafd020 0x55770bafa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.437+0000 7fad63e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770baf2760 0x55770bafa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.437+0000 7fad6be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770baf1e40 0x55770baf9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.437+0000 7fad70c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770bafd020 0x55770bafa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35922/0 (socket says 172.21.15.123:35922) 2024-09-19T04:54:36.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.437+0000 7fad70c00640 1 -- 172.21.15.123:0/3469036689 learned_addr learned my addr 172.21.15.123:0/3469036689 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:36.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.437+0000 7fad63e00640 1 -- 172.21.15.123:0/3469036689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770baf1e40 msgr2=0x55770baf9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:36.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.437+0000 7fad63e00640 1 --2- 172.21.15.123:0/3469036689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770baf1e40 0x55770baf9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.437+0000 7fad63e00640 1 -- 172.21.15.123:0/3469036689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770bafd020 msgr2=0x55770bafa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:36.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.437+0000 7fad63e00640 1 --2- 172.21.15.123:0/3469036689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770bafd020 0x55770bafa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.437+0000 7fad63e00640 1 -- 172.21.15.123:0/3469036689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55770b7c93c0 con 0x55770baf2760 2024-09-19T04:54:36.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.437+0000 7fad63e00640 1 --2- 172.21.15.123:0/3469036689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770baf2760 0x55770bafa490 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fad540cd390 tx=0x7fad5409b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4653896d13a14d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:36.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.437+0000 7fad59600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:54:36.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.438+0000 7fad59600640 1 -- 172.21.15.123:0/3469036689 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5401c040 con 0x55770baf2760 2024-09-19T04:54:36.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.438+0000 7fad59600640 1 -- 172.21.15.123:0/3469036689 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fad540cd610 con 0x55770baf2760 2024-09-19T04:54:36.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.438+0000 7fad75655740 1 -- 172.21.15.123:0/3469036689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad6400f9a0 con 0x55770baf2760 2024-09-19T04:54:36.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.438+0000 7fad59600640 1 -- 172.21.15.123:0/3469036689 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fad540cd810 con 0x55770baf2760 2024-09-19T04:54:36.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.438+0000 7fad75655740 10 client.26945 fetch_fsmap learned FSMap version 41 2024-09-19T04:54:36.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.438+0000 7fad75655740 1 -- 172.21.15.123:0/3469036689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55770bb0bc20 con 0x55770baf2760 2024-09-19T04:54:36.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.438+0000 7fad59600640 1 -- 172.21.15.123:0/3469036689 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad6400f9a0 con 0x55770baf2760 2024-09-19T04:54:36.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.438+0000 7fad59600640 1 -- 172.21.15.123:0/3469036689 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7fad5409f030 con 0x55770baf2760 2024-09-19T04:54:36.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.438+0000 7fad75655740 10 client.26945 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T04:54:36.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.438+0000 7fad75655740 10 client.26945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:36.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.438+0000 7fad75655740 20 client.26945 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:36.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.438+0000 7fad75655740 1 --2- 172.21.15.123:0/3469036689 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55770bb0d130 0x55770bb0f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.438+0000 7fad75655740 4 client.26945 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:36.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.438+0000 7fad75655740 1 -- 172.21.15.123:0/3469036689 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55770ba65950 con 0x55770bb0d130 2024-09-19T04:54:36.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.439+0000 7fad6be00640 1 --2- 172.21.15.123:0/3469036689 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55770bb0d130 0x55770bb0f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.439+0000 7fad6be00640 1 --2- 172.21.15.123:0/3469036689 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55770bb0d130 0x55770bb0f590 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffa078830211f8dc server_cookie=91c1511a8e646236 in_seq=0 out_seq=0 2024-09-19T04:54:36.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.439+0000 7fad59600640 10 client.26945 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:36.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.440+0000 7fad59600640 1 -- 172.21.15.123:0/3469036689 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55770ba65950 con 0x55770bb0d130 2024-09-19T04:54:36.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.440+0000 7fad59600640 10 client.26945 handle_command_reply: tid=0 2024-09-19T04:54:36.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.440+0000 7fad75655740 1 client.26945 shutdown 2024-09-19T04:54:36.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.440+0000 7fad75655740 1 -- 172.21.15.123:0/3469036689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770baf2760 msgr2=0x55770bafa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:36.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.440+0000 7fad75655740 1 --2- 172.21.15.123:0/3469036689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770baf2760 0x55770bafa490 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fad540cd390 tx=0x7fad5409b940 comp rx=0 tx=0).stop 2024-09-19T04:54:36.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.440+0000 7fad75655740 1 -- 172.21.15.123:0/3469036689 shutdown_connections 2024-09-19T04:54:36.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.440+0000 7fad75655740 1 --2- 172.21.15.123:0/3469036689 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55770bb0d130 0x55770bb0f590 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.440+0000 7fad75655740 1 --2- 172.21.15.123:0/3469036689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770bafd020 0x55770bafa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.440+0000 7fad75655740 1 --2- 172.21.15.123:0/3469036689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770baf2760 0x55770bafa490 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.440+0000 7fad75655740 1 --2- 172.21.15.123:0/3469036689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770baf1e40 0x55770baf9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.440+0000 7fad75655740 1 -- 172.21.15.123:0/3469036689 >> 172.21.15.123:0/3469036689 conn(0x55770baee3d0 msgr2=0x55770baf4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:36.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.440+0000 7fad59600640 0 client.26945 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:36.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.440+0000 7fad75655740 1 -- 172.21.15.123:0/3469036689 shutdown_connections 2024-09-19T04:54:36.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.441+0000 7fad75655740 1 -- 172.21.15.123:0/3469036689 wait complete. 2024-09-19T04:54:36.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.441+0000 7fad75655740 20 client.26945 trim_cache size 0 max 16384 2024-09-19T04:54:36.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.455+0000 7fad75655740 1 Processor -- start 2024-09-19T04:54:36.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.455+0000 7fad75655740 1 -- start start 2024-09-19T04:54:36.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.456+0000 7fad75655740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf1e40 0x55770baf48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.456+0000 7fad75655740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770baf2760 0x55770bafbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.456+0000 7fad75655740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770bafd020 0x55770bb14e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.456+0000 7fad75655740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1b3d60 con 0x55770baf1e40 2024-09-19T04:54:36.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.456+0000 7fad75655740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1b5930 con 0x55770bafd020 2024-09-19T04:54:36.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.456+0000 7fad75655740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1b4460 con 0x55770baf2760 2024-09-19T04:54:36.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.456+0000 7fad6be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf1e40 0x55770baf48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.456+0000 7fad6be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf1e40 0x55770baf48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35930/0 (socket says 172.21.15.123:35930) 2024-09-19T04:54:36.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.456+0000 7fad6be00640 1 -- 172.21.15.123:0/1785461538 learned_addr learned my addr 172.21.15.123:0/1785461538 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:36.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.456+0000 7fad63e00640 1 --2- 172.21.15.123:0/1785461538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770baf2760 0x55770bafbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.456+0000 7fad70c00640 1 --2- 172.21.15.123:0/1785461538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770bafd020 0x55770bb14e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.456+0000 7fad6be00640 1 -- 172.21.15.123:0/1785461538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770baf2760 msgr2=0x55770bafbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.456+0000 7fad6be00640 1 --2- 172.21.15.123:0/1785461538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770baf2760 0x55770bafbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.456+0000 7fad6be00640 1 -- 172.21.15.123:0/1785461538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770bafd020 msgr2=0x55770bb14e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.456+0000 7fad6be00640 1 --2- 172.21.15.123:0/1785461538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770bafd020 0x55770bb14e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.457+0000 7fad6be00640 1 -- 172.21.15.123:0/1785461538 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55770baef060 con 0x55770baf1e40 2024-09-19T04:54:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.457+0000 7fad6be00640 1 --2- 172.21.15.123:0/1785461538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf1e40 0x55770baf48f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fad5c006010 tx=0x7fad5c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8aa2f54777cde7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.457+0000 7fad5be00640 1 -- 172.21.15.123:0/1785461538 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c035040 con 0x55770baf1e40 2024-09-19T04:54:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.457+0000 7fad5be00640 1 -- 172.21.15.123:0/1785461538 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad5c056070 con 0x55770baf1e40 2024-09-19T04:54:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.457+0000 7fad5be00640 1 -- 172.21.15.123:0/1785461538 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0035f0 con 0x55770baf1e40 2024-09-19T04:54:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.458+0000 7fad75655740 1 -- 172.21.15.123:0/1785461538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf1e40 msgr2=0x55770baf48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.458+0000 7fad75655740 1 --2- 172.21.15.123:0/1785461538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf1e40 0x55770baf48f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fad5c006010 tx=0x7fad5c03f000 comp rx=0 tx=0).stop 2024-09-19T04:54:36.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.458+0000 7fad75655740 1 -- 172.21.15.123:0/1785461538 shutdown_connections 2024-09-19T04:54:36.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.458+0000 7fad75655740 1 --2- 172.21.15.123:0/1785461538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770bafd020 0x55770bb14e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.458+0000 7fad75655740 1 --2- 172.21.15.123:0/1785461538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770baf2760 0x55770bafbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.458+0000 7fad75655740 1 --2- 172.21.15.123:0/1785461538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf1e40 0x55770baf48f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.458+0000 7fad75655740 1 -- 172.21.15.123:0/1785461538 >> 172.21.15.123:0/1785461538 conn(0x55770baee3d0 msgr2=0x55770bafb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:36.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.458+0000 7fad75655740 1 -- 172.21.15.123:0/1785461538 shutdown_connections 2024-09-19T04:54:36.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.458+0000 7fad75655740 1 -- 172.21.15.123:0/1785461538 wait complete. 2024-09-19T04:54:36.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.458+0000 7fad75655740 1 -- start start 2024-09-19T04:54:36.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.459+0000 7fad75655740 1 Processor -- start 2024-09-19T04:54:36.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.459+0000 7fad75655740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770baf1e40 0x55770bb05610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.459+0000 7fad75655740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf2760 0x55770baf6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.459+0000 7fad75655740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770bafd020 0x55770baf70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.459+0000 7fad75655740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1b5260 con 0x55770baf2760 2024-09-19T04:54:36.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.459+0000 7fad75655740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1b4b60 con 0x55770bafd020 2024-09-19T04:54:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.459+0000 7fad75655740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1b1d60 con 0x55770baf1e40 2024-09-19T04:54:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.459+0000 7fad75655740 10 client.0 _ll_register_callbacks cb 0x7fad5409bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.459+0000 7fad63e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf2760 0x55770baf6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.459+0000 7fad6be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770baf1e40 0x55770bb05610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.459+0000 7fad63e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf2760 0x55770baf6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35936/0 (socket says 172.21.15.123:35936) 2024-09-19T04:54:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.459+0000 7fad63e00640 1 -- 172.21.15.123:0/2381248095 learned_addr learned my addr 172.21.15.123:0/2381248095 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.459+0000 7fad6be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770baf1e40 0x55770bb05610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:35806/0 (socket says 172.21.15.123:35806) 2024-09-19T04:54:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.459+0000 7fad70c00640 1 --2- 172.21.15.123:0/2381248095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770bafd020 0x55770baf70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.460+0000 7fad6be00640 1 -- 172.21.15.123:0/2381248095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770bafd020 msgr2=0x55770baf70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.460+0000 7fad6be00640 1 --2- 172.21.15.123:0/2381248095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770bafd020 0x55770baf70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.460+0000 7fad6be00640 1 -- 172.21.15.123:0/2381248095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf2760 msgr2=0x55770baf6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.460+0000 7fad6be00640 1 --2- 172.21.15.123:0/2381248095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf2760 0x55770baf6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.460+0000 7fad6be00640 1 -- 172.21.15.123:0/2381248095 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55770baf77d0 con 0x55770baf1e40 2024-09-19T04:54:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.460+0000 7fad6be00640 1 --2- 172.21.15.123:0/2381248095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770baf1e40 0x55770bb05610 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fad5c031ba0 tx=0x7fad5c0058b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=581acdf0af233ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.460+0000 7fad59600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:54:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.460+0000 7fad59600640 1 -- 172.21.15.123:0/2381248095 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c05e020 con 0x55770baf1e40 2024-09-19T04:54:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.460+0000 7fad75655740 1 -- 172.21.15.123:0/2381248095 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad5c056070 con 0x55770baf1e40 2024-09-19T04:54:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.460+0000 7fad59600640 1 -- 172.21.15.123:0/2381248095 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fad5c004760 con 0x55770baf1e40 2024-09-19T04:54:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.461+0000 7fad59600640 1 -- 172.21.15.123:0/2381248095 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fad5c069050 con 0x55770baf1e40 2024-09-19T04:54:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.461+0000 7fad75655740 10 client.26974 fetch_fsmap learned FSMap version 41 2024-09-19T04:54:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.461+0000 7fad75655740 1 -- 172.21.15.123:0/2381248095 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55770bb77cf0 con 0x55770baf1e40 2024-09-19T04:54:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.461+0000 7fad59600640 1 -- 172.21.15.123:0/2381248095 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad5c056070 con 0x55770baf1e40 2024-09-19T04:54:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.461+0000 7fad59600640 1 -- 172.21.15.123:0/2381248095 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7fad5c05ab10 con 0x55770baf1e40 2024-09-19T04:54:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.461+0000 7fad75655740 10 client.26974 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T04:54:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.461+0000 7fad75655740 10 client.26974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.461+0000 7fad75655740 20 client.26974 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.461+0000 7fad75655740 1 --2- 172.21.15.123:0/2381248095 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55770bb78470 0x55770bb7a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.461+0000 7fad75655740 4 client.26974 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.461+0000 7fad75655740 1 -- 172.21.15.123:0/2381248095 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55770bae99a0 con 0x55770bb78470 2024-09-19T04:54:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.462+0000 7fad63e00640 1 --2- 172.21.15.123:0/2381248095 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55770bb78470 0x55770bb7a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.462+0000 7fad63e00640 1 --2- 172.21.15.123:0/2381248095 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55770bb78470 0x55770bb7a890 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad3604cfb6165157 server_cookie=19f20e795d60b7e3 in_seq=0 out_seq=0 2024-09-19T04:54:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.462+0000 7fad59600640 10 client.26974 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.462+0000 7fad59600640 1 -- 172.21.15.123:0/2381248095 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55770bae99a0 con 0x55770bb78470 2024-09-19T04:54:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.462+0000 7fad59600640 10 client.26974 handle_command_reply: tid=0 2024-09-19T04:54:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.462+0000 7fad75655740 1 client.26974 shutdown 2024-09-19T04:54:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.463+0000 7fad75655740 1 -- 172.21.15.123:0/2381248095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770baf1e40 msgr2=0x55770bb05610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.463+0000 7fad75655740 1 --2- 172.21.15.123:0/2381248095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770baf1e40 0x55770bb05610 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fad5c031ba0 tx=0x7fad5c0058b0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.463+0000 7fad75655740 1 -- 172.21.15.123:0/2381248095 shutdown_connections 2024-09-19T04:54:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.463+0000 7fad75655740 1 --2- 172.21.15.123:0/2381248095 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55770bb78470 0x55770bb7a890 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.463+0000 7fad75655740 1 --2- 172.21.15.123:0/2381248095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770bafd020 0x55770baf70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.463+0000 7fad75655740 1 --2- 172.21.15.123:0/2381248095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55770baf2760 0x55770baf6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.463+0000 7fad59600640 0 client.26974 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.463+0000 7fad75655740 1 --2- 172.21.15.123:0/2381248095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770baf1e40 0x55770bb05610 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.463+0000 7fad75655740 1 -- 172.21.15.123:0/2381248095 >> 172.21.15.123:0/2381248095 conn(0x55770baee3d0 msgr2=0x55770baed490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.463+0000 7fad75655740 1 -- 172.21.15.123:0/2381248095 shutdown_connections 2024-09-19T04:54:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.463+0000 7fad75655740 1 -- 172.21.15.123:0/2381248095 wait complete. 2024-09-19T04:54:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.463+0000 7fad75655740 20 client.26974 trim_cache size 0 max 16384 2024-09-19T04:54:36.468 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:36.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.464+0000 7fad71600640 1 -- 172.21.15.123:0/2673174216 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fad500a3430 msgr2=0x7fad500a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:36.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.464+0000 7fad71600640 1 --2- 172.21.15.123:0/2673174216 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fad500a3430 0x7fad500a58f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55770baef230 tx=0x7fad5c005a20 comp rx=0 tx=0).stop 2024-09-19T04:54:36.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.464+0000 7fad71600640 1 -- 172.21.15.123:0/2673174216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad6c074eb0 msgr2=0x7fad6c1a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:36.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.464+0000 7fad71600640 1 --2- 172.21.15.123:0/2673174216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad6c074eb0 0x7fad6c1a4b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fad5400e9f0 tx=0x7fad5400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.464+0000 7fad71600640 1 -- 172.21.15.123:0/2673174216 shutdown_connections 2024-09-19T04:54:36.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.464+0000 7fad71600640 1 --2- 172.21.15.123:0/2673174216 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fad500a3430 0x7fad500a58f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.464+0000 7fad71600640 1 --2- 172.21.15.123:0/2673174216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad6c1129e0 0x7fad6c19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.464+0000 7fad71600640 1 --2- 172.21.15.123:0/2673174216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad6c074eb0 0x7fad6c1a4b70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.464+0000 7fad71600640 1 --2- 172.21.15.123:0/2673174216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad6c074590 0x7fad6c1a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:36.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.464+0000 7fad71600640 1 -- 172.21.15.123:0/2673174216 >> 172.21.15.123:0/2673174216 conn(0x7fad6c0fd710 msgr2=0x7fad6c0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:36.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.464+0000 7fad71600640 1 -- 172.21.15.123:0/2673174216 shutdown_connections 2024-09-19T04:54:36.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:36.464+0000 7fad71600640 1 -- 172.21.15.123:0/2673174216 wait complete. 2024-09-19T04:54:37.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:37 smithi183 ceph-mon[35333]: pgmap v244: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:37.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:37 smithi123 ceph-mon[30511]: pgmap v244: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:37.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:37 smithi193 ceph-mon[35592]: pgmap v244: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:38.189 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:54:38.190 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:54:38.561 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:39.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.148+0000 7f25d6c00640 1 -- 172.21.15.123:0/1179701207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d01049a0 msgr2=0x7f25d01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:39.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.148+0000 7f25d6c00640 1 --2- 172.21.15.123:0/1179701207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d01049a0 0x7f25d01089f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009900 tx=0x7f25b802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.148+0000 7f25d6c00640 1 -- 172.21.15.123:0/1179701207 shutdown_connections 2024-09-19T04:54:39.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.148+0000 7f25d6c00640 1 --2- 172.21.15.123:0/1179701207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25d0109120 0x7f25d0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.148+0000 7f25d6c00640 1 --2- 172.21.15.123:0/1179701207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d01049a0 0x7f25d01089f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.148+0000 7f25d6c00640 1 --2- 172.21.15.123:0/1179701207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25d0103ff0 0x7f25d01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.148+0000 7f25d6c00640 1 -- 172.21.15.123:0/1179701207 >> 172.21.15.123:0/1179701207 conn(0x7f25d00ffa90 msgr2=0x7f25d0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:39.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.149+0000 7f25d6c00640 1 -- 172.21.15.123:0/1179701207 shutdown_connections 2024-09-19T04:54:39.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.149+0000 7f25d6c00640 1 -- 172.21.15.123:0/1179701207 wait complete. 2024-09-19T04:54:39.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.149+0000 7f25d6c00640 1 Processor -- start 2024-09-19T04:54:39.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.150+0000 7f25d6c00640 1 -- start start 2024-09-19T04:54:39.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.150+0000 7f25d6c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25d0103ff0 0x7f25d019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.150+0000 7f25d6c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25d01049a0 0x7f25d019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.150+0000 7f25d6c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d0109120 0x7f25d019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.150+0000 7f25d6c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25d0114170 con 0x7f25d0109120 2024-09-19T04:54:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.150+0000 7f25d6c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f25d0113ff0 con 0x7f25d01049a0 2024-09-19T04:54:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.150+0000 7f25d6c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01142f0 con 0x7f25d0103ff0 2024-09-19T04:54:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.150+0000 7f25d4c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d0109120 0x7f25d019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.150+0000 7f25d4c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d0109120 0x7f25d019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35966/0 (socket says 172.21.15.123:35966) 2024-09-19T04:54:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.150+0000 7f25d4c00640 1 -- 172.21.15.123:0/3835911325 learned_addr learned my addr 172.21.15.123:0/3835911325 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.150+0000 7f25c7e00640 1 --2- 172.21.15.123:0/3835911325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25d01049a0 0x7f25d019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.150+0000 7f25cfe00640 1 --2- 172.21.15.123:0/3835911325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25d0103ff0 0x7f25d019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.151+0000 7f25d4c00640 1 -- 172.21.15.123:0/3835911325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25d0103ff0 msgr2=0x7f25d019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.151+0000 7f25d4c00640 1 --2- 172.21.15.123:0/3835911325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25d0103ff0 0x7f25d019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.151+0000 7f25d4c00640 1 -- 172.21.15.123:0/3835911325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25d01049a0 msgr2=0x7f25d019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.151+0000 7f25d4c00640 1 --2- 172.21.15.123:0/3835911325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25d01049a0 0x7f25d019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.151+0000 7f25d4c00640 1 -- 172.21.15.123:0/3835911325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d019f600 con 0x7f25d0109120 2024-09-19T04:54:39.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.151+0000 7f25d4c00640 1 --2- 172.21.15.123:0/3835911325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d0109120 0x7f25d019ef20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f25c800eaa0 tx=0x7f25c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:39.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.151+0000 7f25cd600640 1 -- 172.21.15.123:0/3835911325 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c800cdb0 con 0x7f25d0109120 2024-09-19T04:54:39.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.151+0000 7f25d6c00640 1 -- 172.21.15.123:0/3835911325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d019f8f0 con 0x7f25d0109120 2024-09-19T04:54:39.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.151+0000 7f25cd600640 1 -- 172.21.15.123:0/3835911325 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c8005020 con 0x7f25d0109120 2024-09-19T04:54:39.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.151+0000 7f25cd600640 1 -- 172.21.15.123:0/3835911325 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c80116b0 con 0x7f25d0109120 2024-09-19T04:54:39.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.152+0000 7f25d6c00640 1 -- 172.21.15.123:0/3835911325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d01a71d0 con 0x7f25d0109120 2024-09-19T04:54:39.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.154+0000 7f25cd600640 1 -- 172.21.15.123:0/3835911325 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f25c8005560 con 0x7f25d0109120 2024-09-19T04:54:39.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.155+0000 7f25d95ea740 1 Processor -- start 2024-09-19T04:54:39.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.155+0000 7f25d95ea740 1 -- start start 2024-09-19T04:54:39.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.155+0000 7f25d95ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc963ac0 0x55a1bc963ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.155+0000 7f25d95ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 0x55a1bc96e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.155+0000 7f25d95ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc96eca0 0x55a1bc971090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.155+0000 7f25d95ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a1bc8ca290 con 0x55a1bc963ac0 2024-09-19T04:54:39.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.155+0000 7f25d95ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a1bc871b40 con 0x55a1bc9643e0 2024-09-19T04:54:39.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.155+0000 7f25d95ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a1bc8c6250 con 0x55a1bc96eca0 2024-09-19T04:54:39.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.155+0000 7f25c7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc963ac0 0x55a1bc963ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.155+0000 7f25c7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc963ac0 0x55a1bc963ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35982/0 (socket says 172.21.15.123:35982) 2024-09-19T04:54:39.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.155+0000 7f25c7e00640 1 -- 172.21.15.123:0/3665896350 learned_addr learned my addr 172.21.15.123:0/3665896350 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:39.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.155+0000 7f25cfe00640 1 --2- 172.21.15.123:0/3665896350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 0x55a1bc96e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.156+0000 7f25c7e00640 1 --2- 172.21.15.123:0/3665896350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc96eca0 0x55a1bc971090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.156+0000 7f25cfe00640 1 -- 172.21.15.123:0/3665896350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc96eca0 msgr2=0x55a1bc971090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:39.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.156+0000 7f25cfe00640 1 --2- 172.21.15.123:0/3665896350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc96eca0 0x55a1bc971090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.156+0000 7f25cfe00640 1 -- 172.21.15.123:0/3665896350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc963ac0 msgr2=0x55a1bc963ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:39.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.156+0000 7f25cfe00640 1 --2- 172.21.15.123:0/3665896350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc963ac0 0x55a1bc963ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.156+0000 7f25cfe00640 1 -- 172.21.15.123:0/3665896350 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1bc63b6a0 con 0x55a1bc9643e0 2024-09-19T04:54:39.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.156+0000 7f25c7e00640 1 --2- 172.21.15.123:0/3665896350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc963ac0 0x55a1bc963ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:39.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.156+0000 7f25cd600640 1 --2- 172.21.15.123:0/3835911325 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f25b40a33b0 0x7f25b40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.156+0000 7f25cfe00640 1 --2- 172.21.15.123:0/3665896350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 0x55a1bc96e760 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f25c0005fd0 tx=0x7f25c00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43bff4414f7b29e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:39.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.156+0000 7f25cd600640 1 -- 172.21.15.123:0/3835911325 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f25c80cfec0 con 0x7f25d0109120 2024-09-19T04:54:39.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.157+0000 7f25c7e00640 1 --2- 172.21.15.123:0/3835911325 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f25b40a33b0 0x7f25b40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.157+0000 7f25c5200640 1 -- 172.21.15.123:0/3665896350 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c001c480 con 0x55a1bc9643e0 2024-09-19T04:54:39.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.157+0000 7f25c5200640 1 -- 172.21.15.123:0/3665896350 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c001cb00 con 0x55a1bc9643e0 2024-09-19T04:54:39.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.157+0000 7f25c5200640 1 -- 172.21.15.123:0/3665896350 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0011c20 con 0x55a1bc9643e0 2024-09-19T04:54:39.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.157+0000 7f25d95ea740 1 -- 172.21.15.123:0/3665896350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 msgr2=0x55a1bc96e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:39.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.157+0000 7f25c7e00640 1 --2- 172.21.15.123:0/3835911325 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f25b40a33b0 0x7f25b40a5870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f25b8004650 tx=0x7f25b8009560 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:39.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.157+0000 7f25d95ea740 1 --2- 172.21.15.123:0/3665896350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 0x55a1bc96e760 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f25c0005fd0 tx=0x7f25c00107a0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.158+0000 7f25d95ea740 1 -- 172.21.15.123:0/3665896350 shutdown_connections 2024-09-19T04:54:39.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.158+0000 7f25d95ea740 1 --2- 172.21.15.123:0/3665896350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc96eca0 0x55a1bc971090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.158+0000 7f25d95ea740 1 --2- 172.21.15.123:0/3665896350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 0x55a1bc96e760 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.158+0000 7f25d95ea740 1 --2- 172.21.15.123:0/3665896350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc963ac0 0x55a1bc963ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.158+0000 7f25d95ea740 1 -- 172.21.15.123:0/3665896350 >> 172.21.15.123:0/3665896350 conn(0x55a1bc960050 msgr2=0x55a1bc960430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:39.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.158+0000 7f25d95ea740 1 -- 172.21.15.123:0/3665896350 shutdown_connections 2024-09-19T04:54:39.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.158+0000 7f25d95ea740 1 -- 172.21.15.123:0/3665896350 wait complete. 2024-09-19T04:54:39.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.158+0000 7f25d95ea740 1 -- start start 2024-09-19T04:54:39.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.159+0000 7f25d95ea740 1 Processor -- start 2024-09-19T04:54:39.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.159+0000 7f25d95ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc963ac0 0x55a1bc96e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.159+0000 7f25d95ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 0x55a1bc96b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.159+0000 7f25d95ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc96eca0 0x55a1bc96be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.159+0000 7f25d95ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a1bc971910 con 0x55a1bc96eca0 2024-09-19T04:54:39.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.159+0000 7f25d95ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a1bc884fa0 con 0x55a1bc9643e0 2024-09-19T04:54:39.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.159+0000 7f25d95ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a1bc971a90 con 0x55a1bc963ac0 2024-09-19T04:54:39.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.159+0000 7f25d95ea740 10 client.0 _ll_register_callbacks cb 0x55a1bc5d8ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:39.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.159+0000 7f25c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 0x55a1bc96b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.159+0000 7f25d4c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc96eca0 0x55a1bc96be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.159+0000 7f25c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 0x55a1bc96b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:54884/0 (socket says 172.21.15.123:54884) 2024-09-19T04:54:39.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.159+0000 7f25c7e00640 1 -- 172.21.15.123:0/256179934 learned_addr learned my addr 172.21.15.123:0/256179934 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:39.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.159+0000 7f25cfe00640 1 --2- 172.21.15.123:0/256179934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc963ac0 0x55a1bc96e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.160+0000 7f25c7e00640 1 -- 172.21.15.123:0/256179934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc963ac0 msgr2=0x55a1bc96e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:39.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.160+0000 7f25c7e00640 1 --2- 172.21.15.123:0/256179934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc963ac0 0x55a1bc96e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.160+0000 7f25c7e00640 1 -- 172.21.15.123:0/256179934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc96eca0 msgr2=0x55a1bc96be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:39.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.160+0000 7f25c7e00640 1 --2- 172.21.15.123:0/256179934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc96eca0 0x55a1bc96be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.160+0000 7f25c7e00640 1 -- 172.21.15.123:0/256179934 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1bc96c6c0 con 0x55a1bc9643e0 2024-09-19T04:54:39.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.160+0000 7f25d4c00640 1 --2- 172.21.15.123:0/256179934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc96eca0 0x55a1bc96be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:39.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.160+0000 7f25c7e00640 1 --2- 172.21.15.123:0/256179934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 0x55a1bc96b8f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f25b8000c00 tx=0x7f25b802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbd700fcd1ab9498 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:39.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.160+0000 7f25bd600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:54:39.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.160+0000 7f25bd600640 1 -- 172.21.15.123:0/256179934 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8035040 con 0x55a1bc9643e0 2024-09-19T04:54:39.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.161+0000 7f25bd600640 1 -- 172.21.15.123:0/256179934 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f25b8004e20 con 0x55a1bc9643e0 2024-09-19T04:54:39.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.161+0000 7f25d95ea740 1 -- 172.21.15.123:0/256179934 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25c001cb00 con 0x55a1bc9643e0 2024-09-19T04:54:39.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.161+0000 7f25bd600640 1 -- 172.21.15.123:0/256179934 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f25b80325c0 con 0x55a1bc9643e0 2024-09-19T04:54:39.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.161+0000 7f25d95ea740 10 client.26963 fetch_fsmap learned FSMap version 41 2024-09-19T04:54:39.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.161+0000 7f25d95ea740 1 -- 172.21.15.123:0/256179934 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a1bc97d890 con 0x55a1bc9643e0 2024-09-19T04:54:39.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.161+0000 7f25bd600640 1 -- 172.21.15.123:0/256179934 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c001cb00 con 0x55a1bc9643e0 2024-09-19T04:54:39.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.161+0000 7f25bd600640 1 -- 172.21.15.123:0/256179934 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f25b805d910 con 0x55a1bc9643e0 2024-09-19T04:54:39.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.162+0000 7f25d95ea740 10 client.26963 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T04:54:39.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.162+0000 7f25d95ea740 10 client.26963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:39.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.162+0000 7f25d95ea740 20 client.26963 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:39.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.162+0000 7f25d95ea740 1 --2- 172.21.15.123:0/256179934 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a1bc97eca0 0x55a1bc981100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.162+0000 7f25d95ea740 4 client.26963 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:39.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.162+0000 7f25d95ea740 1 -- 172.21.15.123:0/256179934 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1bc8d7800 con 0x55a1bc97eca0 2024-09-19T04:54:39.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.162+0000 7f25cfe00640 1 --2- 172.21.15.123:0/256179934 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a1bc97eca0 0x55a1bc981100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.163+0000 7f25cfe00640 1 --2- 172.21.15.123:0/256179934 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a1bc97eca0 0x55a1bc981100 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=405f8090fa6663e5 server_cookie=cb9035eda34e52a in_seq=0 out_seq=0 2024-09-19T04:54:39.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.163+0000 7f25bd600640 10 client.26963 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:39.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.163+0000 7f25bd600640 1 -- 172.21.15.123:0/256179934 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1bc8d7800 con 0x55a1bc97eca0 2024-09-19T04:54:39.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.163+0000 7f25bd600640 10 client.26963 handle_command_reply: tid=0 2024-09-19T04:54:39.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.164+0000 7f25d95ea740 1 client.26963 shutdown 2024-09-19T04:54:39.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.164+0000 7f25d95ea740 1 -- 172.21.15.123:0/256179934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 msgr2=0x55a1bc96b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:39.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.164+0000 7f25d95ea740 1 --2- 172.21.15.123:0/256179934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 0x55a1bc96b8f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f25b8000c00 tx=0x7f25b802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.164+0000 7f25d95ea740 1 -- 172.21.15.123:0/256179934 shutdown_connections 2024-09-19T04:54:39.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.164+0000 7f25d95ea740 1 --2- 172.21.15.123:0/256179934 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a1bc97eca0 0x55a1bc981100 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.164+0000 7f25d95ea740 1 --2- 172.21.15.123:0/256179934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc96eca0 0x55a1bc96be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.164+0000 7f25bd600640 0 client.26963 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:39.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.164+0000 7f25d95ea740 1 --2- 172.21.15.123:0/256179934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 0x55a1bc96b8f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.164+0000 7f25d95ea740 1 --2- 172.21.15.123:0/256179934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc963ac0 0x55a1bc96e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.164+0000 7f25d95ea740 1 -- 172.21.15.123:0/256179934 >> 172.21.15.123:0/256179934 conn(0x55a1bc960050 msgr2=0x55a1bc965d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:39.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.164+0000 7f25d95ea740 1 -- 172.21.15.123:0/256179934 shutdown_connections 2024-09-19T04:54:39.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.164+0000 7f25d95ea740 1 -- 172.21.15.123:0/256179934 wait complete. 2024-09-19T04:54:39.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.164+0000 7f25d95ea740 20 client.26963 trim_cache size 0 max 16384 2024-09-19T04:54:39.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.182+0000 7f25d95ea740 1 Processor -- start 2024-09-19T04:54:39.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.182+0000 7f25d95ea740 1 -- start start 2024-09-19T04:54:39.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.182+0000 7f25d95ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc963ac0 0x55a1bc96d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.182+0000 7f25d95ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc9643e0 0x55a1bc9869d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.182+0000 7f25d95ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc96eca0 0x55a1bc986f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.182+0000 7f25d95ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01af9d0 con 0x55a1bc9643e0 2024-09-19T04:54:39.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.182+0000 7f25d95ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01b15a0 con 0x55a1bc96eca0 2024-09-19T04:54:39.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.182+0000 7f25d95ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01b00d0 con 0x55a1bc963ac0 2024-09-19T04:54:39.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.182+0000 7f25c7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc9643e0 0x55a1bc9869d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.182+0000 7f25c7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc9643e0 0x55a1bc9869d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35996/0 (socket says 172.21.15.123:35996) 2024-09-19T04:54:39.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.182+0000 7f25c7e00640 1 -- 172.21.15.123:0/3800024829 learned_addr learned my addr 172.21.15.123:0/3800024829 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:39.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.182+0000 7f25cfe00640 1 --2- 172.21.15.123:0/3800024829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc963ac0 0x55a1bc96d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.182+0000 7f25d4c00640 1 --2- 172.21.15.123:0/3800024829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc96eca0 0x55a1bc986f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.183+0000 7f25c7e00640 1 -- 172.21.15.123:0/3800024829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc963ac0 msgr2=0x55a1bc96d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:39.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.183+0000 7f25c7e00640 1 --2- 172.21.15.123:0/3800024829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc963ac0 0x55a1bc96d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.183+0000 7f25c7e00640 1 -- 172.21.15.123:0/3800024829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc96eca0 msgr2=0x55a1bc986f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:39.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.183+0000 7f25c7e00640 1 --2- 172.21.15.123:0/3800024829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc96eca0 0x55a1bc986f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.183+0000 7f25c7e00640 1 -- 172.21.15.123:0/3800024829 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1bc96c890 con 0x55a1bc9643e0 2024-09-19T04:54:39.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.183+0000 7f25c7e00640 1 --2- 172.21.15.123:0/3800024829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc9643e0 0x55a1bc9869d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009900 tx=0x7f25b8005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978fadf5d58f921e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:39.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.183+0000 7f25bfe00640 1 -- 172.21.15.123:0/3800024829 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8035040 con 0x55a1bc9643e0 2024-09-19T04:54:39.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.183+0000 7f25bfe00640 1 -- 172.21.15.123:0/3800024829 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25b8062020 con 0x55a1bc9643e0 2024-09-19T04:54:39.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.183+0000 7f25bfe00640 1 -- 172.21.15.123:0/3800024829 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8032960 con 0x55a1bc9643e0 2024-09-19T04:54:39.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.184+0000 7f25d95ea740 1 -- 172.21.15.123:0/3800024829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc9643e0 msgr2=0x55a1bc9869d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:39.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.184+0000 7f25d95ea740 1 --2- 172.21.15.123:0/3800024829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc9643e0 0x55a1bc9869d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009900 tx=0x7f25b8005970 comp rx=0 tx=0).stop 2024-09-19T04:54:39.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.184+0000 7f25d95ea740 1 -- 172.21.15.123:0/3800024829 shutdown_connections 2024-09-19T04:54:39.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.184+0000 7f25d95ea740 1 --2- 172.21.15.123:0/3800024829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc96eca0 0x55a1bc986f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.184+0000 7f25d95ea740 1 --2- 172.21.15.123:0/3800024829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc9643e0 0x55a1bc9869d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.184+0000 7f25d95ea740 1 --2- 172.21.15.123:0/3800024829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc963ac0 0x55a1bc96d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.184+0000 7f25d95ea740 1 -- 172.21.15.123:0/3800024829 >> 172.21.15.123:0/3800024829 conn(0x55a1bc960050 msgr2=0x55a1bc966ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:39.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.184+0000 7f25d95ea740 1 -- 172.21.15.123:0/3800024829 shutdown_connections 2024-09-19T04:54:39.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.184+0000 7f25d95ea740 1 -- 172.21.15.123:0/3800024829 wait complete. 2024-09-19T04:54:39.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.185+0000 7f25d95ea740 1 -- start start 2024-09-19T04:54:39.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.185+0000 7f25d95ea740 1 Processor -- start 2024-09-19T04:54:39.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.185+0000 7f25d95ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc963ac0 0x55a1bc976ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.185+0000 7f25d95ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 0x55a1bc977220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.185+0000 7f25d95ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc96eca0 0x55a1bc973cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.185+0000 7f25d95ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01b0ed0 con 0x55a1bc963ac0 2024-09-19T04:54:39.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.185+0000 7f25d95ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01b07d0 con 0x55a1bc9643e0 2024-09-19T04:54:39.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.185+0000 7f25d95ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01ad9d0 con 0x55a1bc96eca0 2024-09-19T04:54:39.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.185+0000 7f25d95ea740 10 client.0 _ll_register_callbacks cb 0x7f25b8005210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:39.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.186+0000 7f25cfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc963ac0 0x55a1bc976ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.186+0000 7f25c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 0x55a1bc977220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.186+0000 7f25cfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc963ac0 0x55a1bc976ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36008/0 (socket says 172.21.15.123:36008) 2024-09-19T04:54:39.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.186+0000 7f25cfe00640 1 -- 172.21.15.123:0/2689663541 learned_addr learned my addr 172.21.15.123:0/2689663541 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:39.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.186+0000 7f25d4c00640 1 --2- 172.21.15.123:0/2689663541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc96eca0 0x55a1bc973cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.186+0000 7f25cfe00640 1 -- 172.21.15.123:0/2689663541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc96eca0 msgr2=0x55a1bc973cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:39.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.186+0000 7f25cfe00640 1 --2- 172.21.15.123:0/2689663541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc96eca0 0x55a1bc973cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.186+0000 7f25cfe00640 1 -- 172.21.15.123:0/2689663541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 msgr2=0x55a1bc977220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:39.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.186+0000 7f25cfe00640 1 --2- 172.21.15.123:0/2689663541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 0x55a1bc977220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.186+0000 7f25cfe00640 1 -- 172.21.15.123:0/2689663541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1bc977930 con 0x55a1bc963ac0 2024-09-19T04:54:39.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.186+0000 7f25c7e00640 1 --2- 172.21.15.123:0/2689663541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 0x55a1bc977220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:39.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.186+0000 7f25cfe00640 1 --2- 172.21.15.123:0/2689663541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc963ac0 0x55a1bc976ce0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f25c001e010 tx=0x7f25c00116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6383485ef46b15 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:39.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.186+0000 7f25bd600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:39.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.187+0000 7f25bd600640 1 -- 172.21.15.123:0/2689663541 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0005450 con 0x55a1bc963ac0 2024-09-19T04:54:39.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.187+0000 7f25bd600640 1 -- 172.21.15.123:0/2689663541 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f25c00055f0 con 0x55a1bc963ac0 2024-09-19T04:54:39.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.187+0000 7f25d95ea740 1 -- 172.21.15.123:0/2689663541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25b8062020 con 0x55a1bc963ac0 2024-09-19T04:54:39.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.187+0000 7f25bd600640 1 -- 172.21.15.123:0/2689663541 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f25c0004020 con 0x55a1bc963ac0 2024-09-19T04:54:39.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.187+0000 7f25d95ea740 10 client.16875 fetch_fsmap learned FSMap version 41 2024-09-19T04:54:39.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.187+0000 7f25d95ea740 1 -- 172.21.15.123:0/2689663541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a1bc9ea000 con 0x55a1bc963ac0 2024-09-19T04:54:39.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.187+0000 7f25bd600640 1 -- 172.21.15.123:0/2689663541 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25b8062020 con 0x55a1bc963ac0 2024-09-19T04:54:39.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.187+0000 7f25bd600640 1 -- 172.21.15.123:0/2689663541 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f25c001c510 con 0x55a1bc963ac0 2024-09-19T04:54:39.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.187+0000 7f25d95ea740 10 client.16875 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T04:54:39.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.187+0000 7f25d95ea740 10 client.16875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:39.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.187+0000 7f25d95ea740 20 client.16875 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:39.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.187+0000 7f25d95ea740 1 --2- 172.21.15.123:0/2689663541 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a1bc9ea490 0x55a1bc9ec8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:39.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.188+0000 7f25d95ea740 4 client.16875 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:54:39.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.188+0000 7f25d95ea740 1 -- 172.21.15.123:0/2689663541 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a1bc8d1d10 con 0x55a1bc9ea490 2024-09-19T04:54:39.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.188+0000 7f25c7e00640 1 --2- 172.21.15.123:0/2689663541 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a1bc9ea490 0x55a1bc9ec8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:39.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.188+0000 7f25c7e00640 1 --2- 172.21.15.123:0/2689663541 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a1bc9ea490 0x55a1bc9ec8b0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=416eedc22dbc49cb server_cookie=d9f98a2533d68f6b in_seq=0 out_seq=0 2024-09-19T04:54:39.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.188+0000 7f25bd600640 10 client.16875 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:39.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.189+0000 7f25bd600640 1 -- 172.21.15.123:0/2689663541 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a1bc8d1d10 con 0x55a1bc9ea490 2024-09-19T04:54:39.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.189+0000 7f25bd600640 10 client.16875 handle_command_reply: tid=0 2024-09-19T04:54:39.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.189+0000 7f25d95ea740 1 client.16875 shutdown 2024-09-19T04:54:39.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.190+0000 7f25d95ea740 1 -- 172.21.15.123:0/2689663541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc963ac0 msgr2=0x55a1bc976ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:39.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.190+0000 7f25d95ea740 1 --2- 172.21.15.123:0/2689663541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc963ac0 0x55a1bc976ce0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f25c001e010 tx=0x7f25c00116e0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.190+0000 7f25d95ea740 1 -- 172.21.15.123:0/2689663541 shutdown_connections 2024-09-19T04:54:39.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.190+0000 7f25d95ea740 1 --2- 172.21.15.123:0/2689663541 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55a1bc9ea490 0x55a1bc9ec8b0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.190+0000 7f25d95ea740 1 --2- 172.21.15.123:0/2689663541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1bc96eca0 0x55a1bc973cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.190+0000 7f25d95ea740 1 --2- 172.21.15.123:0/2689663541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1bc9643e0 0x55a1bc977220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.190+0000 7f25d95ea740 1 --2- 172.21.15.123:0/2689663541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1bc963ac0 0x55a1bc976ce0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.190+0000 7f25bd600640 0 client.16875 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:39.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.190+0000 7f25d95ea740 1 -- 172.21.15.123:0/2689663541 >> 172.21.15.123:0/2689663541 conn(0x55a1bc960050 msgr2=0x55a1bc95e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:39.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.190+0000 7f25d95ea740 1 -- 172.21.15.123:0/2689663541 shutdown_connections 2024-09-19T04:54:39.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.190+0000 7f25d95ea740 1 -- 172.21.15.123:0/2689663541 wait complete. 2024-09-19T04:54:39.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.190+0000 7f25d95ea740 20 client.16875 trim_cache size 0 max 16384 2024-09-19T04:54:39.195 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:54:39.195 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-09-19T04:54:39.195 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "5a0f22c6-a939-471c-abe2-32688c2157f6", 2024-09-19T04:54:39.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-09-19T04:54:39.195 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:54:39.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.191+0000 7f25d6c00640 1 -- 172.21.15.123:0/3835911325 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f25b40a33b0 msgr2=0x7f25b40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:39.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.191+0000 7f25d6c00640 1 --2- 172.21.15.123:0/3835911325 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f25b40a33b0 0x7f25b40a5870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f25b8004650 tx=0x7f25b8009560 comp rx=0 tx=0).stop 2024-09-19T04:54:39.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.191+0000 7f25d6c00640 1 -- 172.21.15.123:0/3835911325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d0109120 msgr2=0x7f25d019ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:39.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.191+0000 7f25d6c00640 1 --2- 172.21.15.123:0/3835911325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d0109120 0x7f25d019ef20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f25c800eaa0 tx=0x7f25c800ef60 comp rx=0 tx=0).stop 2024-09-19T04:54:39.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.191+0000 7f25d6c00640 1 -- 172.21.15.123:0/3835911325 shutdown_connections 2024-09-19T04:54:39.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.191+0000 7f25d6c00640 1 --2- 172.21.15.123:0/3835911325 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f25b40a33b0 0x7f25b40a5870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.191+0000 7f25d6c00640 1 --2- 172.21.15.123:0/3835911325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d0109120 0x7f25d019ef20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.191+0000 7f25d6c00640 1 --2- 172.21.15.123:0/3835911325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25d01049a0 0x7f25d019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.191+0000 7f25d6c00640 1 --2- 172.21.15.123:0/3835911325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25d0103ff0 0x7f25d019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:39.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.191+0000 7f25d6c00640 1 -- 172.21.15.123:0/3835911325 >> 172.21.15.123:0/3835911325 conn(0x7f25d00ffa90 msgr2=0x7f25d0100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:39.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.191+0000 7f25d6c00640 1 -- 172.21.15.123:0/3835911325 shutdown_connections 2024-09-19T04:54:39.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:39.191+0000 7f25d6c00640 1 -- 172.21.15.123:0/3835911325 wait complete. 2024-09-19T04:54:39.764 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:39 smithi123 ceph-mon[30511]: pgmap v245: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:39.764 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:39 smithi123 ceph-mon[30511]: scrub queued for path: / 2024-09-19T04:54:39.764 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:39 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:39.764 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:39 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:54:39.764 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:39 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:39.764 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:39 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:54:39.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:39 smithi183 ceph-mon[35333]: pgmap v245: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:39.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:39 smithi183 ceph-mon[35333]: scrub queued for path: / 2024-09-19T04:54:39.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:39 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:39.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:39 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:54:39.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:39 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:39.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:39 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:54:39.989 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:54:39.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:39 smithi193 ceph-mon[35592]: pgmap v245: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:39.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:39 smithi193 ceph-mon[35592]: scrub queued for path: / 2024-09-19T04:54:39.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:39 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:39.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:39 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:54:39.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:39 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:39.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:39 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:54:40.358 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:40.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.949+0000 7fa90c400640 1 -- 172.21.15.123:0/2242787263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa904109140 msgr2=0x7fa904110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:40.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.949+0000 7fa90c400640 1 --2- 172.21.15.123:0/2242787263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa904109140 0x7fa904110cc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009ca0 tx=0x7fa8fc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.949+0000 7fa90c400640 1 -- 172.21.15.123:0/2242787263 shutdown_connections 2024-09-19T04:54:40.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.949+0000 7fa90c400640 1 --2- 172.21.15.123:0/2242787263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa904109140 0x7fa904110cc0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.949+0000 7fa90c400640 1 --2- 172.21.15.123:0/2242787263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9041049c0 0x7fa904108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.949+0000 7fa90c400640 1 --2- 172.21.15.123:0/2242787263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa904104010 0x7fa9041043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.949+0000 7fa90c400640 1 -- 172.21.15.123:0/2242787263 >> 172.21.15.123:0/2242787263 conn(0x7fa9040ffa50 msgr2=0x7fa904101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:40.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.949+0000 7fa90c400640 1 -- 172.21.15.123:0/2242787263 shutdown_connections 2024-09-19T04:54:40.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.949+0000 7fa90c400640 1 -- 172.21.15.123:0/2242787263 wait complete. 2024-09-19T04:54:40.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.950+0000 7fa90c400640 1 Processor -- start 2024-09-19T04:54:40.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.950+0000 7fa90c400640 1 -- start start 2024-09-19T04:54:40.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.951+0000 7fa90c400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa904104010 0x7fa90419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.951+0000 7fa90c400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9041049c0 0x7fa90419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.951+0000 7fa90c400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa904109140 0x7fa90419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.951+0000 7fa90c400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa904114150 con 0x7fa9041049c0 2024-09-19T04:54:40.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.951+0000 7fa90c400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa904113fd0 con 0x7fa904104010 2024-09-19T04:54:40.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.951+0000 7fa90c400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041142d0 con 0x7fa904109140 2024-09-19T04:54:40.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.951+0000 7fa909000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9041049c0 0x7fa90419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.951+0000 7fa909000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9041049c0 0x7fa90419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36036/0 (socket says 172.21.15.123:36036) 2024-09-19T04:54:40.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.951+0000 7fa909000640 1 -- 172.21.15.123:0/2592476103 learned_addr learned my addr 172.21.15.123:0/2592476103 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:40.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.951+0000 7fa909a00640 1 --2- 172.21.15.123:0/2592476103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa904104010 0x7fa90419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.951+0000 7fa90a400640 1 --2- 172.21.15.123:0/2592476103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa904109140 0x7fa90419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.951+0000 7fa909000640 1 -- 172.21.15.123:0/2592476103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa904109140 msgr2=0x7fa90419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.951+0000 7fa909000640 1 --2- 172.21.15.123:0/2592476103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa904109140 0x7fa90419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.951+0000 7fa909000640 1 -- 172.21.15.123:0/2592476103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa904104010 msgr2=0x7fa90419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.951+0000 7fa909000640 1 --2- 172.21.15.123:0/2592476103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa904104010 0x7fa90419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.951+0000 7fa909000640 1 -- 172.21.15.123:0/2592476103 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90419f5e0 con 0x7fa9041049c0 2024-09-19T04:54:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.952+0000 7fa909000640 1 --2- 172.21.15.123:0/2592476103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9041049c0 0x7fa90419ab70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa8f000e9f0 tx=0x7fa8f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.952+0000 7fa902000640 1 -- 172.21.15.123:0/2592476103 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f000cd80 con 0x7fa9041049c0 2024-09-19T04:54:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.952+0000 7fa902000640 1 -- 172.21.15.123:0/2592476103 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f0010910 con 0x7fa9041049c0 2024-09-19T04:54:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.952+0000 7fa902000640 1 -- 172.21.15.123:0/2592476103 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0018990 con 0x7fa9041049c0 2024-09-19T04:54:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.952+0000 7fa90c400640 1 -- 172.21.15.123:0/2592476103 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90419f8d0 con 0x7fa9041049c0 2024-09-19T04:54:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.952+0000 7fa90c400640 1 -- 172.21.15.123:0/2592476103 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9041a71b0 con 0x7fa9041049c0 2024-09-19T04:54:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.955+0000 7fa902000640 1 -- 172.21.15.123:0/2592476103 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa8f0010ac0 con 0x7fa9041049c0 2024-09-19T04:54:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.955+0000 7fa90ee64740 1 Processor -- start 2024-09-19T04:54:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.956+0000 7fa90ee64740 1 -- start start 2024-09-19T04:54:40.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.956+0000 7fa90ee64740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 0x55c8e818af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.956+0000 7fa90ee64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 0x55c8e8195830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.956+0000 7fa90ee64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e8195d70 0x55c8e8198160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.956+0000 7fa90ee64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c8e80f13e0 con 0x55c8e8195d70 2024-09-19T04:54:40.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.956+0000 7fa90ee64740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c8e8098d80 con 0x55c8e818ab90 2024-09-19T04:54:40.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.956+0000 7fa90ee64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c8e80ed3a0 con 0x55c8e818b4b0 2024-09-19T04:54:40.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.956+0000 7fa90a400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 0x55c8e818af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.956+0000 7fa909a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 0x55c8e8195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.956+0000 7fa90a400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 0x55c8e818af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:54930/0 (socket says 172.21.15.123:54930) 2024-09-19T04:54:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.956+0000 7fa90a400640 1 -- 172.21.15.123:0/1214731205 learned_addr learned my addr 172.21.15.123:0/1214731205 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.956+0000 7fa909a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 0x55c8e8195830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:35906/0 (socket says 172.21.15.123:35906) 2024-09-19T04:54:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.956+0000 7fa909000640 1 --2- 172.21.15.123:0/1214731205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e8195d70 0x55c8e8198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.957+0000 7fa90a400640 1 -- 172.21.15.123:0/1214731205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 msgr2=0x55c8e8195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.957+0000 7fa90a400640 1 --2- 172.21.15.123:0/1214731205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 0x55c8e8195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.957+0000 7fa90a400640 1 -- 172.21.15.123:0/1214731205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e8195d70 msgr2=0x55c8e8198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.957+0000 7fa90a400640 1 --2- 172.21.15.123:0/1214731205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e8195d70 0x55c8e8198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.957+0000 7fa90a400640 1 -- 172.21.15.123:0/1214731205 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8e80b7ae0 con 0x55c8e818ab90 2024-09-19T04:54:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.957+0000 7fa909000640 1 --2- 172.21.15.123:0/1214731205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e8195d70 0x55c8e8198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:54:40.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.957+0000 7fa902000640 1 --2- 172.21.15.123:0/2592476103 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa8ec0a3430 0x7fa8ec0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.957+0000 7fa90a400640 1 --2- 172.21.15.123:0/1214731205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 0x55c8e818af70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009ca0 tx=0x7fa8fc005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6628f25bfbdcd3a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.957+0000 7fa902000640 1 -- 172.21.15.123:0/2592476103 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fa8f00cff50 con 0x7fa9041049c0 2024-09-19T04:54:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.957+0000 7fa8f6a00640 1 -- 172.21.15.123:0/1214731205 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc031db0 con 0x55c8e818ab90 2024-09-19T04:54:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.957+0000 7fa909a00640 1 --2- 172.21.15.123:0/2592476103 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa8ec0a3430 0x7fa8ec0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.958+0000 7fa8f6a00640 1 -- 172.21.15.123:0/1214731205 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8fc036040 con 0x55c8e818ab90 2024-09-19T04:54:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.958+0000 7fa8f6a00640 1 -- 172.21.15.123:0/1214731205 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc004280 con 0x55c8e818ab90 2024-09-19T04:54:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.958+0000 7fa90ee64740 1 -- 172.21.15.123:0/1214731205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 msgr2=0x55c8e818af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.958+0000 7fa909a00640 1 --2- 172.21.15.123:0/2592476103 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa8ec0a3430 0x7fa8ec0a58f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55c8e8187f80 tx=0x7fa8f8005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.958+0000 7fa90ee64740 1 --2- 172.21.15.123:0/1214731205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 0x55c8e818af70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009ca0 tx=0x7fa8fc005f10 comp rx=0 tx=0).stop 2024-09-19T04:54:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.959+0000 7fa90ee64740 1 -- 172.21.15.123:0/1214731205 shutdown_connections 2024-09-19T04:54:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.959+0000 7fa90ee64740 1 --2- 172.21.15.123:0/1214731205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e8195d70 0x55c8e8198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.959+0000 7fa90ee64740 1 --2- 172.21.15.123:0/1214731205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 0x55c8e8195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.959+0000 7fa90ee64740 1 --2- 172.21.15.123:0/1214731205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 0x55c8e818af70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.959+0000 7fa90ee64740 1 -- 172.21.15.123:0/1214731205 >> 172.21.15.123:0/1214731205 conn(0x55c8e8187120 msgr2=0x55c8e8187500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.959+0000 7fa90ee64740 1 -- 172.21.15.123:0/1214731205 shutdown_connections 2024-09-19T04:54:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.959+0000 7fa90ee64740 1 -- 172.21.15.123:0/1214731205 wait complete. 2024-09-19T04:54:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.959+0000 7fa90ee64740 1 -- start start 2024-09-19T04:54:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.959+0000 7fa90ee64740 1 Processor -- start 2024-09-19T04:54:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.960+0000 7fa90ee64740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 0x55c8e8192cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.960+0000 7fa90ee64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e818b4b0 0x55c8e81931f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.960+0000 7fa90ee64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e8195d70 0x55c8e8193730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.960+0000 7fa90ee64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c8e8198860 con 0x55c8e818b4b0 2024-09-19T04:54:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.960+0000 7fa90ee64740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c8e80ac0f0 con 0x55c8e818ab90 2024-09-19T04:54:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.960+0000 7fa90ee64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c8e81989e0 con 0x55c8e8195d70 2024-09-19T04:54:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.960+0000 7fa90ee64740 10 client.0 _ll_register_callbacks cb 0x55c8e809d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.960+0000 7fa909000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e818b4b0 0x55c8e81931f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.960+0000 7fa909a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 0x55c8e8192cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.960+0000 7fa909000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e818b4b0 0x55c8e81931f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36052/0 (socket says 172.21.15.123:36052) 2024-09-19T04:54:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.960+0000 7fa909a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 0x55c8e8192cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:54946/0 (socket says 172.21.15.123:54946) 2024-09-19T04:54:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.960+0000 7fa909000640 1 -- 172.21.15.123:0/4248437209 learned_addr learned my addr 172.21.15.123:0/4248437209 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.960+0000 7fa90a400640 1 --2- 172.21.15.123:0/4248437209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e8195d70 0x55c8e8193730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.960+0000 7fa909a00640 1 -- 172.21.15.123:0/4248437209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e8195d70 msgr2=0x55c8e8193730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:40.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.961+0000 7fa909a00640 1 --2- 172.21.15.123:0/4248437209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e8195d70 0x55c8e8193730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.961+0000 7fa909a00640 1 -- 172.21.15.123:0/4248437209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e818b4b0 msgr2=0x55c8e81931f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:40.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.961+0000 7fa909a00640 1 --2- 172.21.15.123:0/4248437209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e818b4b0 0x55c8e81931f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.961+0000 7fa909a00640 1 -- 172.21.15.123:0/4248437209 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8e7e623c0 con 0x55c8e818ab90 2024-09-19T04:54:40.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.961+0000 7fa909a00640 1 --2- 172.21.15.123:0/4248437209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 0x55c8e8192cb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8012cb0 tx=0x7fa8f8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f6a02bb540b1e93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.961+0000 7fa8eb400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:54:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.961+0000 7fa8eb400640 1 -- 172.21.15.123:0/4248437209 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8009e60 con 0x55c8e818ab90 2024-09-19T04:54:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.961+0000 7fa90ee64740 1 -- 172.21.15.123:0/4248437209 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8fc036040 con 0x55c8e818ab90 2024-09-19T04:54:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.961+0000 7fa8eb400640 1 -- 172.21.15.123:0/4248437209 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fa8f801f640 con 0x55c8e818ab90 2024-09-19T04:54:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.962+0000 7fa8eb400640 1 -- 172.21.15.123:0/4248437209 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa8f801f840 con 0x55c8e818ab90 2024-09-19T04:54:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.962+0000 7fa90ee64740 10 client.26981 fetch_fsmap learned FSMap version 41 2024-09-19T04:54:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.962+0000 7fa90ee64740 1 -- 172.21.15.123:0/4248437209 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c8e81a4610 con 0x55c8e818ab90 2024-09-19T04:54:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.962+0000 7fa8eb400640 1 -- 172.21.15.123:0/4248437209 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8fc036040 con 0x55c8e818ab90 2024-09-19T04:54:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.962+0000 7fa8eb400640 1 -- 172.21.15.123:0/4248437209 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7fa8f8027070 con 0x55c8e818ab90 2024-09-19T04:54:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.962+0000 7fa90ee64740 10 client.26981 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T04:54:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.962+0000 7fa90ee64740 10 client.26981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.962+0000 7fa90ee64740 20 client.26981 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.962+0000 7fa90ee64740 1 --2- 172.21.15.123:0/4248437209 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55c8e81a5ad0 0x55c8e81a7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.962+0000 7fa90ee64740 4 client.26981 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.962+0000 7fa90ee64740 1 -- 172.21.15.123:0/4248437209 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8e80fe950 con 0x55c8e81a5ad0 2024-09-19T04:54:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.963+0000 7fa909000640 1 --2- 172.21.15.123:0/4248437209 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55c8e81a5ad0 0x55c8e81a7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.963+0000 7fa909000640 1 --2- 172.21.15.123:0/4248437209 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55c8e81a5ad0 0x55c8e81a7f70 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90033e18c9b91d7e server_cookie=6701cc3bb20a344 in_seq=0 out_seq=0 2024-09-19T04:54:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.963+0000 7fa8eb400640 10 client.26981 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.964+0000 7fa8eb400640 1 -- 172.21.15.123:0/4248437209 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8e80fe950 con 0x55c8e81a5ad0 2024-09-19T04:54:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.964+0000 7fa8eb400640 10 client.26981 handle_command_reply: tid=0 2024-09-19T04:54:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.964+0000 7fa90ee64740 1 client.26981 shutdown 2024-09-19T04:54:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.964+0000 7fa90ee64740 1 -- 172.21.15.123:0/4248437209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 msgr2=0x55c8e8192cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.964+0000 7fa90ee64740 1 --2- 172.21.15.123:0/4248437209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 0x55c8e8192cb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8012cb0 tx=0x7fa8f8004730 comp rx=0 tx=0).stop 2024-09-19T04:54:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.964+0000 7fa90ee64740 1 -- 172.21.15.123:0/4248437209 shutdown_connections 2024-09-19T04:54:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.964+0000 7fa90ee64740 1 --2- 172.21.15.123:0/4248437209 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55c8e81a5ad0 0x55c8e81a7f70 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.964+0000 7fa90ee64740 1 --2- 172.21.15.123:0/4248437209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e8195d70 0x55c8e8193730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.964+0000 7fa90ee64740 1 --2- 172.21.15.123:0/4248437209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e818b4b0 0x55c8e81931f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.964+0000 7fa90ee64740 1 --2- 172.21.15.123:0/4248437209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 0x55c8e8192cb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.964+0000 7fa8eb400640 0 client.26981 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:40.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.964+0000 7fa90ee64740 1 -- 172.21.15.123:0/4248437209 >> 172.21.15.123:0/4248437209 conn(0x55c8e8187120 msgr2=0x55c8e818ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:40.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.964+0000 7fa90ee64740 1 -- 172.21.15.123:0/4248437209 shutdown_connections 2024-09-19T04:54:40.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.964+0000 7fa90ee64740 1 -- 172.21.15.123:0/4248437209 wait complete. 2024-09-19T04:54:40.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.964+0000 7fa90ee64740 20 client.26981 trim_cache size 0 max 16384 2024-09-19T04:54:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.982+0000 7fa90ee64740 1 Processor -- start 2024-09-19T04:54:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.982+0000 7fa90ee64740 1 -- start start 2024-09-19T04:54:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.982+0000 7fa90ee64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e818ab90 0x55c8e818d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.982+0000 7fa90ee64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 0x55c8e8191580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.982+0000 7fa90ee64740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e8195d70 0x55c8e81860b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.982+0000 7fa90ee64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041afa20 con 0x55c8e818ab90 2024-09-19T04:54:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.982+0000 7fa90ee64740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041b15f0 con 0x55c8e8195d70 2024-09-19T04:54:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.982+0000 7fa90ee64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041b0120 con 0x55c8e818b4b0 2024-09-19T04:54:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.982+0000 7fa909a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e818ab90 0x55c8e818d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.982+0000 7fa909a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e818ab90 0x55c8e818d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36060/0 (socket says 172.21.15.123:36060) 2024-09-19T04:54:40.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.982+0000 7fa909a00640 1 -- 172.21.15.123:0/2286101263 learned_addr learned my addr 172.21.15.123:0/2286101263 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:40.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.983+0000 7fa90a400640 1 --2- 172.21.15.123:0/2286101263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e8195d70 0x55c8e81860b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.983+0000 7fa909000640 1 --2- 172.21.15.123:0/2286101263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 0x55c8e8191580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.983+0000 7fa90a400640 1 -- 172.21.15.123:0/2286101263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 msgr2=0x55c8e8191580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.983+0000 7fa90a400640 1 --2- 172.21.15.123:0/2286101263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 0x55c8e8191580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.983+0000 7fa90a400640 1 -- 172.21.15.123:0/2286101263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e818ab90 msgr2=0x55c8e818d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.983+0000 7fa90a400640 1 --2- 172.21.15.123:0/2286101263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e818ab90 0x55c8e818d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.983+0000 7fa90a400640 1 -- 172.21.15.123:0/2286101263 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8e8187db0 con 0x55c8e8195d70 2024-09-19T04:54:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.983+0000 7fa90a400640 1 --2- 172.21.15.123:0/2286101263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e8195d70 0x55c8e81860b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc005e30 tx=0x7fa8fc02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=954fe2838a032936 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.983+0000 7fa8f6000640 1 -- 172.21.15.123:0/2286101263 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc052070 con 0x55c8e8195d70 2024-09-19T04:54:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.984+0000 7fa8f6000640 1 -- 172.21.15.123:0/2286101263 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8fc036070 con 0x55c8e8195d70 2024-09-19T04:54:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.984+0000 7fa8f6000640 1 -- 172.21.15.123:0/2286101263 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0042a0 con 0x55c8e8195d70 2024-09-19T04:54:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.984+0000 7fa90ee64740 1 -- 172.21.15.123:0/2286101263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e8195d70 msgr2=0x55c8e81860b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.984+0000 7fa90ee64740 1 --2- 172.21.15.123:0/2286101263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e8195d70 0x55c8e81860b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc005e30 tx=0x7fa8fc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.984+0000 7fa90ee64740 1 -- 172.21.15.123:0/2286101263 shutdown_connections 2024-09-19T04:54:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.984+0000 7fa90ee64740 1 --2- 172.21.15.123:0/2286101263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e8195d70 0x55c8e81860b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.984+0000 7fa90ee64740 1 --2- 172.21.15.123:0/2286101263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 0x55c8e8191580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.984+0000 7fa90ee64740 1 --2- 172.21.15.123:0/2286101263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e818ab90 0x55c8e818d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.984+0000 7fa90ee64740 1 -- 172.21.15.123:0/2286101263 >> 172.21.15.123:0/2286101263 conn(0x55c8e8187120 msgr2=0x55c8e81859a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.984+0000 7fa90ee64740 1 -- 172.21.15.123:0/2286101263 shutdown_connections 2024-09-19T04:54:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.984+0000 7fa90ee64740 1 -- 172.21.15.123:0/2286101263 wait complete. 2024-09-19T04:54:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.985+0000 7fa90ee64740 1 -- start start 2024-09-19T04:54:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.985+0000 7fa90ee64740 1 Processor -- start 2024-09-19T04:54:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.985+0000 7fa90ee64740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 0x55c8e818fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.986+0000 7fa90ee64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 0x55c8e8190240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.986+0000 7fa90ee64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e8195d70 0x55c8e8190780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.986+0000 7fa90ee64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041b0f20 con 0x55c8e8195d70 2024-09-19T04:54:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.986+0000 7fa90ee64740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041b0820 con 0x55c8e818ab90 2024-09-19T04:54:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.986+0000 7fa90ee64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041ada20 con 0x55c8e818b4b0 2024-09-19T04:54:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.986+0000 7fa90ee64740 10 client.0 _ll_register_callbacks cb 0x7fa8f80044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.986+0000 7fa909000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 0x55c8e8190240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.986+0000 7fa909000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 0x55c8e8190240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:35922/0 (socket says 172.21.15.123:35922) 2024-09-19T04:54:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.986+0000 7fa909000640 1 -- 172.21.15.123:0/3910810141 learned_addr learned my addr 172.21.15.123:0/3910810141 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.986+0000 7fa90a400640 1 --2- 172.21.15.123:0/3910810141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e8195d70 0x55c8e8190780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.986+0000 7fa909a00640 1 --2- 172.21.15.123:0/3910810141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 0x55c8e818fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.986+0000 7fa909000640 1 -- 172.21.15.123:0/3910810141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 msgr2=0x55c8e818fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.986+0000 7fa909000640 1 --2- 172.21.15.123:0/3910810141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 0x55c8e818fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.986+0000 7fa909000640 1 -- 172.21.15.123:0/3910810141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e8195d70 msgr2=0x55c8e8190780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.986+0000 7fa909000640 1 --2- 172.21.15.123:0/3910810141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e8195d70 0x55c8e8190780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.986+0000 7fa909000640 1 -- 172.21.15.123:0/3910810141 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8e82180c0 con 0x55c8e818b4b0 2024-09-19T04:54:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.987+0000 7fa909000640 1 --2- 172.21.15.123:0/3910810141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 0x55c8e8190240 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00986e0 tx=0x7fa8f00bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5129df3ce358187c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.987+0000 7fa8eb400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:54:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.987+0000 7fa8eb400640 1 -- 172.21.15.123:0/3910810141 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f00a4e40 con 0x55c8e818b4b0 2024-09-19T04:54:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.987+0000 7fa90ee64740 1 -- 172.21.15.123:0/3910810141 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8fc036070 con 0x55c8e818b4b0 2024-09-19T04:54:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.987+0000 7fa8eb400640 1 -- 172.21.15.123:0/3910810141 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fa8f00988f0 con 0x55c8e818b4b0 2024-09-19T04:54:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.987+0000 7fa8eb400640 1 -- 172.21.15.123:0/3910810141 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa8f00b5050 con 0x55c8e818b4b0 2024-09-19T04:54:40.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.987+0000 7fa90ee64740 10 client.27010 fetch_fsmap learned FSMap version 41 2024-09-19T04:54:40.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.987+0000 7fa90ee64740 1 -- 172.21.15.123:0/3910810141 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c8e82193b0 con 0x55c8e818b4b0 2024-09-19T04:54:40.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.987+0000 7fa8eb400640 1 -- 172.21.15.123:0/3910810141 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8fc036070 con 0x55c8e818b4b0 2024-09-19T04:54:40.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.987+0000 7fa8eb400640 1 -- 172.21.15.123:0/3910810141 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7fa8f00a0170 con 0x55c8e818b4b0 2024-09-19T04:54:40.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.988+0000 7fa90ee64740 10 client.27010 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T04:54:40.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.988+0000 7fa90ee64740 10 client.27010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:40.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.988+0000 7fa90ee64740 20 client.27010 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:40.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.988+0000 7fa90ee64740 1 --2- 172.21.15.123:0/3910810141 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55c8e8219ac0 0x55c8e821bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:40.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.988+0000 7fa90ee64740 4 client.27010 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:54:40.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.988+0000 7fa90ee64740 1 -- 172.21.15.123:0/3910810141 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c8e80f8e60 con 0x55c8e8219ac0 2024-09-19T04:54:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.988+0000 7fa909a00640 1 --2- 172.21.15.123:0/3910810141 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55c8e8219ac0 0x55c8e821bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.988+0000 7fa909a00640 1 --2- 172.21.15.123:0/3910810141 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55c8e8219ac0 0x55c8e821bee0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=450a23e2a54c9463 server_cookie=1d5564b0ad360de8 in_seq=0 out_seq=0 2024-09-19T04:54:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.988+0000 7fa8eb400640 10 client.27010 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.989+0000 7fa8eb400640 1 -- 172.21.15.123:0/3910810141 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c8e80f8e60 con 0x55c8e8219ac0 2024-09-19T04:54:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.989+0000 7fa8eb400640 10 client.27010 handle_command_reply: tid=0 2024-09-19T04:54:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.989+0000 7fa90ee64740 1 client.27010 shutdown 2024-09-19T04:54:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.989+0000 7fa90ee64740 1 -- 172.21.15.123:0/3910810141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 msgr2=0x55c8e8190240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.989+0000 7fa90ee64740 1 --2- 172.21.15.123:0/3910810141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 0x55c8e8190240 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00986e0 tx=0x7fa8f00bc040 comp rx=0 tx=0).stop 2024-09-19T04:54:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.989+0000 7fa90ee64740 1 -- 172.21.15.123:0/3910810141 shutdown_connections 2024-09-19T04:54:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.989+0000 7fa90ee64740 1 --2- 172.21.15.123:0/3910810141 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55c8e8219ac0 0x55c8e821bee0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.989+0000 7fa90ee64740 1 --2- 172.21.15.123:0/3910810141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8e8195d70 0x55c8e8190780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.989+0000 7fa90ee64740 1 --2- 172.21.15.123:0/3910810141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8e818b4b0 0x55c8e8190240 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.989+0000 7fa8eb400640 0 client.27010 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.989+0000 7fa90ee64740 1 --2- 172.21.15.123:0/3910810141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e818ab90 0x55c8e818fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.989+0000 7fa90ee64740 1 -- 172.21.15.123:0/3910810141 >> 172.21.15.123:0/3910810141 conn(0x55c8e8187120 msgr2=0x55c8e81861c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:40.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.989+0000 7fa90ee64740 1 -- 172.21.15.123:0/3910810141 shutdown_connections 2024-09-19T04:54:40.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.989+0000 7fa90ee64740 1 -- 172.21.15.123:0/3910810141 wait complete. 2024-09-19T04:54:40.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.989+0000 7fa90ee64740 20 client.27010 trim_cache size 0 max 16384 2024-09-19T04:54:40.995 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:54:40.995 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-09-19T04:54:40.995 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-09-19T04:54:40.996 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:54:40.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.990+0000 7fa90c400640 1 -- 172.21.15.123:0/2592476103 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa8ec0a3430 msgr2=0x7fa8ec0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:40.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.990+0000 7fa90c400640 1 --2- 172.21.15.123:0/2592476103 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa8ec0a3430 0x7fa8ec0a58f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55c8e8187f80 tx=0x7fa8f8005f90 comp rx=0 tx=0).stop 2024-09-19T04:54:40.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.990+0000 7fa90c400640 1 -- 172.21.15.123:0/2592476103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9041049c0 msgr2=0x7fa90419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:40.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.990+0000 7fa90c400640 1 --2- 172.21.15.123:0/2592476103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9041049c0 0x7fa90419ab70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa8f000e9f0 tx=0x7fa8f000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.990+0000 7fa90c400640 1 -- 172.21.15.123:0/2592476103 shutdown_connections 2024-09-19T04:54:40.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.990+0000 7fa90c400640 1 --2- 172.21.15.123:0/2592476103 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa8ec0a3430 0x7fa8ec0a58f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.990+0000 7fa90c400640 1 --2- 172.21.15.123:0/2592476103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa904109140 0x7fa90419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.990+0000 7fa90c400640 1 --2- 172.21.15.123:0/2592476103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9041049c0 0x7fa90419ab70 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.990+0000 7fa90c400640 1 --2- 172.21.15.123:0/2592476103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa904104010 0x7fa90419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:40.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.990+0000 7fa90c400640 1 -- 172.21.15.123:0/2592476103 >> 172.21.15.123:0/2592476103 conn(0x7fa9040ffa50 msgr2=0x7fa904100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:40.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.990+0000 7fa90c400640 1 -- 172.21.15.123:0/2592476103 shutdown_connections 2024-09-19T04:54:40.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:40.990+0000 7fa90c400640 1 -- 172.21.15.123:0/2592476103 wait complete. 2024-09-19T04:54:41.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:41 smithi123 ceph-mon[30511]: pgmap v246: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:54:41.740 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:54:41.740 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:54:41.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:41 smithi183 ceph-mon[35333]: pgmap v246: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:54:41.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:41 smithi193 ceph-mon[35592]: pgmap v246: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:54:42.111 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:42.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.631+0000 7f371fa00640 1 -- 172.21.15.123:0/2410516327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3718101de0 msgr2=0x7f37181021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:42.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.631+0000 7f371fa00640 1 --2- 172.21.15.123:0/2410516327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3718101de0 0x7f37181021c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f370c009860 tx=0x7f370c02a340 comp rx=0 tx=0).stop 2024-09-19T04:54:42.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.632+0000 7f371fa00640 1 -- 172.21.15.123:0/2410516327 shutdown_connections 2024-09-19T04:54:42.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.632+0000 7f371fa00640 1 --2- 172.21.15.123:0/2410516327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3718106f10 0x7f371810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:42.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.632+0000 7f371fa00640 1 --2- 172.21.15.123:0/2410516327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3718102790 0x7f37181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:42.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.632+0000 7f371fa00640 1 --2- 172.21.15.123:0/2410516327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3718101de0 0x7f37181021c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:42.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.632+0000 7f371fa00640 1 -- 172.21.15.123:0/2410516327 >> 172.21.15.123:0/2410516327 conn(0x7f37180fd8c0 msgr2=0x7f37180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:42.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.632+0000 7f371fa00640 1 -- 172.21.15.123:0/2410516327 shutdown_connections 2024-09-19T04:54:42.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.632+0000 7f371fa00640 1 -- 172.21.15.123:0/2410516327 wait complete. 2024-09-19T04:54:42.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.632+0000 7f371fa00640 1 Processor -- start 2024-09-19T04:54:42.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.633+0000 7f371fa00640 1 -- start start 2024-09-19T04:54:42.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.633+0000 7f371fa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3718101de0 0x7f37181984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:42.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.633+0000 7f371fa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3718102790 0x7f37181989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:42.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.633+0000 7f371fa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3718106f10 0x7f371819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:42.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.633+0000 7f371fa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3718111f70 con 0x7f3718106f10 2024-09-19T04:54:42.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.633+0000 7f371fa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3718111df0 con 0x7f3718102790 2024-09-19T04:54:42.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.633+0000 7f371fa00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37181120f0 con 0x7f3718101de0 2024-09-19T04:54:42.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.633+0000 7f371da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3718106f10 0x7f371819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:42.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.633+0000 7f371d000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3718101de0 0x7f37181984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:42.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.633+0000 7f371da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3718106f10 0x7f371819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53600/0 (socket says 172.21.15.123:53600) 2024-09-19T04:54:42.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.633+0000 7f371da00640 1 -- 172.21.15.123:0/731751971 learned_addr learned my addr 172.21.15.123:0/731751971 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:42.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.633+0000 7f371d000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3718101de0 0x7f37181984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42586/0 (socket says 172.21.15.123:42586) 2024-09-19T04:54:42.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.633+0000 7f3717e00640 1 --2- 172.21.15.123:0/731751971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3718102790 0x7f37181989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:42.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.634+0000 7f371da00640 1 -- 172.21.15.123:0/731751971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3718101de0 msgr2=0x7f37181984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:42.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.634+0000 7f371da00640 1 --2- 172.21.15.123:0/731751971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3718101de0 0x7f37181984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:42.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.634+0000 7f371da00640 1 -- 172.21.15.123:0/731751971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3718102790 msgr2=0x7f37181989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:42.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.634+0000 7f371da00640 1 --2- 172.21.15.123:0/731751971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3718102790 0x7f37181989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:42.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.634+0000 7f371da00640 1 -- 172.21.15.123:0/731751971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371819d460 con 0x7f3718106f10 2024-09-19T04:54:42.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.634+0000 7f3717e00640 1 --2- 172.21.15.123:0/731751971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3718102790 0x7f37181989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:42.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.634+0000 7f371da00640 1 --2- 172.21.15.123:0/731751971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3718106f10 0x7f371819cd80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f371000efc0 tx=0x7f371000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:42.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.634+0000 7f3715600640 1 -- 172.21.15.123:0/731751971 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710003970 con 0x7f3718106f10 2024-09-19T04:54:42.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.634+0000 7f3715600640 1 -- 172.21.15.123:0/731751971 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3710004020 con 0x7f3718106f10 2024-09-19T04:54:42.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.634+0000 7f371fa00640 1 -- 172.21.15.123:0/731751971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371819d750 con 0x7f3718106f10 2024-09-19T04:54:42.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.634+0000 7f3715600640 1 -- 172.21.15.123:0/731751971 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371000b9f0 con 0x7f3718106f10 2024-09-19T04:54:42.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.635+0000 7f371fa00640 1 -- 172.21.15.123:0/731751971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37181a5030 con 0x7f3718106f10 2024-09-19T04:54:42.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.636+0000 7f371fa00640 1 -- 172.21.15.123:0/731751971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37181021c0 con 0x7f3718106f10 2024-09-19T04:54:42.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.638+0000 7f3715600640 1 -- 172.21.15.123:0/731751971 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f371001d020 con 0x7f3718106f10 2024-09-19T04:54:42.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.640+0000 7f3715600640 1 --2- 172.21.15.123:0/731751971 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f37000a33b0 0x7f37000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:42.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.640+0000 7f3715600640 1 -- 172.21.15.123:0/731751971 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f37100cfb90 con 0x7f3718106f10 2024-09-19T04:54:42.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.640+0000 7f371d000640 1 --2- 172.21.15.123:0/731751971 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f37000a33b0 0x7f37000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:42.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.641+0000 7f371d000640 1 --2- 172.21.15.123:0/731751971 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f37000a33b0 0x7f37000a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f370c02a930 tx=0x7f370c0056d0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:42.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.644+0000 7f3715600640 1 -- 172.21.15.123:0/731751971 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f371009c510 con 0x7f3718106f10 2024-09-19T04:54:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:42 smithi183 ceph-mon[35333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:54:42.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.854+0000 7f371fa00640 1 -- 172.21.15.123:0/731751971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f371810a150 con 0x7f3718106f10 2024-09-19T04:54:42.857 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:42 smithi123 ceph-mon[30511]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:54:42.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.855+0000 7f3715600640 1 -- 172.21.15.123:0/731751971 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11992 (secure 0 0 0) 0x7f371009be10 con 0x7f3718106f10 2024-09-19T04:54:42.858 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:54:42.858 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":42,"btime":"2024-09-19T04:54:41:509200+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:54:41.177367+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24499,24500,24539,24505,24529]},"id":1}]} 2024-09-19T04:54:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 42 2024-09-19T04:54:42.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.857+0000 7f371fa00640 1 -- 172.21.15.123:0/731751971 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f37000a33b0 msgr2=0x7f37000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:42.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.857+0000 7f371fa00640 1 --2- 172.21.15.123:0/731751971 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f37000a33b0 0x7f37000a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f370c02a930 tx=0x7f370c0056d0 comp rx=0 tx=0).stop 2024-09-19T04:54:42.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.857+0000 7f371fa00640 1 -- 172.21.15.123:0/731751971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3718106f10 msgr2=0x7f371819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:42.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.857+0000 7f371fa00640 1 --2- 172.21.15.123:0/731751971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3718106f10 0x7f371819cd80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f371000efc0 tx=0x7f371000c800 comp rx=0 tx=0).stop 2024-09-19T04:54:42.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.858+0000 7f371fa00640 1 -- 172.21.15.123:0/731751971 shutdown_connections 2024-09-19T04:54:42.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.858+0000 7f371fa00640 1 --2- 172.21.15.123:0/731751971 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f37000a33b0 0x7f37000a5870 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:42.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.858+0000 7f371fa00640 1 --2- 172.21.15.123:0/731751971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3718106f10 0x7f371819cd80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:42.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.858+0000 7f371fa00640 1 --2- 172.21.15.123:0/731751971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3718102790 0x7f37181989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:42.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.858+0000 7f371fa00640 1 --2- 172.21.15.123:0/731751971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3718101de0 0x7f37181984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:42.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.858+0000 7f371fa00640 1 -- 172.21.15.123:0/731751971 >> 172.21.15.123:0/731751971 conn(0x7f37180fd8c0 msgr2=0x7f37180fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:42.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.858+0000 7f371fa00640 1 -- 172.21.15.123:0/731751971 shutdown_connections 2024-09-19T04:54:42.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:42.858+0000 7f371fa00640 1 -- 172.21.15.123:0/731751971 wait complete. 2024-09-19T04:54:42.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:42 smithi193 ceph-mon[35592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:54:43.632 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:54:43.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:43 smithi183 ceph-mon[35333]: pgmap v247: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:54:43.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:43 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/731751971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:54:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:43 smithi123 ceph-mon[30511]: pgmap v247: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:54:43.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:43 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/731751971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:54:43.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:43 smithi193 ceph-mon[35592]: pgmap v247: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:54:43.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:43 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/731751971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:54:44.003 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:44.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.573+0000 7f1d1e200640 1 -- 172.21.15.123:0/2532383538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d18106f20 msgr2=0x7f1d1810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:44.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.573+0000 7f1d1e200640 1 --2- 172.21.15.123:0/2532383538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d18106f20 0x7f1d1810eaa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009ca0 tx=0x7f1d1002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.573+0000 7f1d1e200640 1 -- 172.21.15.123:0/2532383538 shutdown_connections 2024-09-19T04:54:44.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.573+0000 7f1d1e200640 1 --2- 172.21.15.123:0/2532383538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d18106f20 0x7f1d1810eaa0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.573+0000 7f1d1e200640 1 --2- 172.21.15.123:0/2532383538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d181027a0 0x7f1d181067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.573+0000 7f1d1e200640 1 --2- 172.21.15.123:0/2532383538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d18101df0 0x7f1d181021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.573+0000 7f1d1e200640 1 -- 172.21.15.123:0/2532383538 >> 172.21.15.123:0/2532383538 conn(0x7f1d180fd8b0 msgr2=0x7f1d180ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:44.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.573+0000 7f1d1e200640 1 -- 172.21.15.123:0/2532383538 shutdown_connections 2024-09-19T04:54:44.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.573+0000 7f1d1e200640 1 -- 172.21.15.123:0/2532383538 wait complete. 2024-09-19T04:54:44.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.574+0000 7f1d1e200640 1 Processor -- start 2024-09-19T04:54:44.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.575+0000 7f1d1e200640 1 -- start start 2024-09-19T04:54:44.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.575+0000 7f1d1e200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d18101df0 0x7f1d180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.575+0000 7f1d1e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d181027a0 0x7f1d18074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.575+0000 7f1d1e200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d18106f20 0x7f1d18074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.575+0000 7f1d1e200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d18111fc0 con 0x7f1d181027a0 2024-09-19T04:54:44.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.575+0000 7f1d1e200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d18111e40 con 0x7f1d18106f20 2024-09-19T04:54:44.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.575+0000 7f1d1e200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1d18112140 con 0x7f1d18101df0 2024-09-19T04:54:44.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.576+0000 7f1d17400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d18101df0 0x7f1d180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.576+0000 7f1d17400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d18101df0 0x7f1d180744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42610/0 (socket says 172.21.15.123:42610) 2024-09-19T04:54:44.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.576+0000 7f1d17400640 1 -- 172.21.15.123:0/1756230032 learned_addr learned my addr 172.21.15.123:0/1756230032 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:44.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.576+0000 7f1d17e00640 1 --2- 172.21.15.123:0/1756230032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d18106f20 0x7f1d18074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.576+0000 7f1d16a00640 1 --2- 172.21.15.123:0/1756230032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d181027a0 0x7f1d18074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.576+0000 7f1d17400640 1 -- 172.21.15.123:0/1756230032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d18106f20 msgr2=0x7f1d18074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:44.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.576+0000 7f1d17400640 1 --2- 172.21.15.123:0/1756230032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d18106f20 0x7f1d18074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.576+0000 7f1d17400640 1 -- 172.21.15.123:0/1756230032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d181027a0 msgr2=0x7f1d18074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:44.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.576+0000 7f1d17400640 1 --2- 172.21.15.123:0/1756230032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d181027a0 0x7f1d18074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.576+0000 7f1d17400640 1 -- 172.21.15.123:0/1756230032 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d181a6960 con 0x7f1d18101df0 2024-09-19T04:54:44.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.576+0000 7f1d16a00640 1 --2- 172.21.15.123:0/1756230032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d181027a0 0x7f1d18074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:54:44.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.576+0000 7f1d17400640 1 --2- 172.21.15.123:0/1756230032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d18101df0 0x7f1d180744f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800e9f0 tx=0x7f1d0800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:44.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.577+0000 7f1d0fe00640 1 -- 172.21.15.123:0/1756230032 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0800cd80 con 0x7f1d18101df0 2024-09-19T04:54:44.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.577+0000 7f1d1e200640 1 -- 172.21.15.123:0/1756230032 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d181a6c50 con 0x7f1d18101df0 2024-09-19T04:54:44.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.577+0000 7f1d0fe00640 1 -- 172.21.15.123:0/1756230032 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d08005500 con 0x7f1d18101df0 2024-09-19T04:54:44.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.577+0000 7f1d0fe00640 1 -- 172.21.15.123:0/1756230032 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08010690 con 0x7f1d18101df0 2024-09-19T04:54:44.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.577+0000 7f1d1e200640 1 -- 172.21.15.123:0/1756230032 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d181a7190 con 0x7f1d18101df0 2024-09-19T04:54:44.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.580+0000 7f1d0fe00640 1 -- 172.21.15.123:0/1756230032 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f1d080046c0 con 0x7f1d18101df0 2024-09-19T04:54:44.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.580+0000 7f1d20c4d740 1 Processor -- start 2024-09-19T04:54:44.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.580+0000 7f1d20c4d740 1 -- start start 2024-09-19T04:54:44.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.580+0000 7f1d20c4d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5623bc70 0x558b5623c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.580+0000 7f1d20c4d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b5623c590 0x558b56246910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.580+0000 7f1d20c4d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b56246e50 0x558b56249240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.580+0000 7f1d20c4d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558b561a23e0 con 0x558b5623c590 2024-09-19T04:54:44.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.580+0000 7f1d20c4d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558b56149d80 con 0x558b56246e50 2024-09-19T04:54:44.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.580+0000 7f1d20c4d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558b5619e3a0 con 0x558b5623bc70 2024-09-19T04:54:44.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.581+0000 7f1d17400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b5623c590 0x558b56246910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.581+0000 7f1d17400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b5623c590 0x558b56246910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53626/0 (socket says 172.21.15.123:53626) 2024-09-19T04:54:44.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.581+0000 7f1d17400640 1 -- 172.21.15.123:0/3880055033 learned_addr learned my addr 172.21.15.123:0/3880055033 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.581+0000 7f1d17e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5623bc70 0x558b5623c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.581+0000 7f1d16a00640 1 --2- 172.21.15.123:0/3880055033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b56246e50 0x558b56249240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.581+0000 7f1d17e00640 1 -- 172.21.15.123:0/3880055033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b56246e50 msgr2=0x558b56249240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.581+0000 7f1d17e00640 1 --2- 172.21.15.123:0/3880055033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b56246e50 0x558b56249240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.581+0000 7f1d17e00640 1 -- 172.21.15.123:0/3880055033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b5623c590 msgr2=0x558b56246910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.581+0000 7f1d17e00640 1 --2- 172.21.15.123:0/3880055033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b5623c590 0x558b56246910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.581+0000 7f1d17e00640 1 -- 172.21.15.123:0/3880055033 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b56168ae0 con 0x558b5623bc70 2024-09-19T04:54:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.581+0000 7f1d0fe00640 1 --2- 172.21.15.123:0/1756230032 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1cfc0a33b0 0x7f1cfc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.582+0000 7f1d17e00640 1 --2- 172.21.15.123:0/3880055033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5623bc70 0x558b5623c050 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1d10006010 tx=0x7f1d100098b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed48d76add4dcb76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:44.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.582+0000 7f1d17400640 1 --2- 172.21.15.123:0/1756230032 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1cfc0a33b0 0x7f1cfc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.582+0000 7f1d0fe00640 1 -- 172.21.15.123:0/1756230032 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f1d080d0250 con 0x7f1d18101df0 2024-09-19T04:54:44.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.582+0000 7f1d03e00640 1 -- 172.21.15.123:0/3880055033 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10003970 con 0x558b5623bc70 2024-09-19T04:54:44.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.582+0000 7f1d03e00640 1 -- 172.21.15.123:0/3880055033 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d10036040 con 0x558b5623bc70 2024-09-19T04:54:44.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.582+0000 7f1d03e00640 1 -- 172.21.15.123:0/3880055033 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10004260 con 0x558b5623bc70 2024-09-19T04:54:44.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.583+0000 7f1d20c4d740 1 -- 172.21.15.123:0/3880055033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5623bc70 msgr2=0x558b5623c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:44.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.583+0000 7f1d17400640 1 --2- 172.21.15.123:0/1756230032 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1cfc0a33b0 0x7f1cfc0a5870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1d0809c460 tx=0x7f1d08098040 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:44.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.583+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/3880055033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5623bc70 0x558b5623c050 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1d10006010 tx=0x7f1d100098b0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.583+0000 7f1d20c4d740 1 -- 172.21.15.123:0/3880055033 shutdown_connections 2024-09-19T04:54:44.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.583+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/3880055033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b56246e50 0x558b56249240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.583+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/3880055033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b5623c590 0x558b56246910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.583+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/3880055033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5623bc70 0x558b5623c050 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.583+0000 7f1d20c4d740 1 -- 172.21.15.123:0/3880055033 >> 172.21.15.123:0/3880055033 conn(0x558b56238200 msgr2=0x558b562385e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:44.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.583+0000 7f1d20c4d740 1 -- 172.21.15.123:0/3880055033 shutdown_connections 2024-09-19T04:54:44.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.583+0000 7f1d20c4d740 1 -- 172.21.15.123:0/3880055033 wait complete. 2024-09-19T04:54:44.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.583+0000 7f1d20c4d740 1 -- start start 2024-09-19T04:54:44.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.584+0000 7f1d20c4d740 1 Processor -- start 2024-09-19T04:54:44.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.584+0000 7f1d20c4d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623bc70 0x558b56243d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.584+0000 7f1d20c4d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5623c590 0x558b562442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.584+0000 7f1d20c4d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b56246e50 0x558b562447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.584+0000 7f1d20c4d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558b562498a0 con 0x558b56246e50 2024-09-19T04:54:44.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.584+0000 7f1d20c4d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558b5615d0f0 con 0x558b5623bc70 2024-09-19T04:54:44.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.584+0000 7f1d20c4d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558b56249a20 con 0x558b5623c590 2024-09-19T04:54:44.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.584+0000 7f1d20c4d740 10 client.0 _ll_register_callbacks cb 0x558b56144bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:44.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.584+0000 7f1d17e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b56246e50 0x558b562447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.584+0000 7f1d16a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5623c590 0x558b562442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.584+0000 7f1d17e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b56246e50 0x558b562447f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53642/0 (socket says 172.21.15.123:53642) 2024-09-19T04:54:44.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.584+0000 7f1d17e00640 1 -- 172.21.15.123:0/166664738 learned_addr learned my addr 172.21.15.123:0/166664738 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:44.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.585+0000 7f1d17e00640 1 --2- 172.21.15.123:0/166664738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623bc70 0x558b56243d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.585+0000 7f1d16a00640 1 -- 172.21.15.123:0/166664738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623bc70 msgr2=0x558b56243d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:44.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.585+0000 7f1d16a00640 1 --2- 172.21.15.123:0/166664738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623bc70 0x558b56243d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.585+0000 7f1d16a00640 1 -- 172.21.15.123:0/166664738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b56246e50 msgr2=0x558b562447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:44.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.585+0000 7f1d16a00640 1 --2- 172.21.15.123:0/166664738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b56246e50 0x558b562447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.585+0000 7f1d16a00640 1 -- 172.21.15.123:0/166664738 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b55f133c0 con 0x558b5623c590 2024-09-19T04:54:44.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.585+0000 7f1d17e00640 1 --2- 172.21.15.123:0/166664738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b56246e50 0x558b562447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:44.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.585+0000 7f1d16a00640 1 --2- 172.21.15.123:0/166664738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5623c590 0x558b562442b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1d040104d0 tx=0x7f1d04010990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81a8075ff65a50a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:44.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.585+0000 7f1d00c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:54:44.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.585+0000 7f1d00c00640 1 -- 172.21.15.123:0/166664738 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04018070 con 0x558b5623c590 2024-09-19T04:54:44.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.586+0000 7f1d00c00640 1 -- 172.21.15.123:0/166664738 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f1d04014640 con 0x558b5623c590 2024-09-19T04:54:44.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.586+0000 7f1d20c4d740 1 -- 172.21.15.123:0/166664738 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d10036040 con 0x558b5623c590 2024-09-19T04:54:44.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.586+0000 7f1d00c00640 1 -- 172.21.15.123:0/166664738 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1d04024050 con 0x558b5623c590 2024-09-19T04:54:44.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.586+0000 7f1d20c4d740 10 client.27028 fetch_fsmap learned FSMap version 43 2024-09-19T04:54:44.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.586+0000 7f1d20c4d740 1 -- 172.21.15.123:0/166664738 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558b562559c0 con 0x558b5623c590 2024-09-19T04:54:44.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.586+0000 7f1d00c00640 1 -- 172.21.15.123:0/166664738 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d10036040 con 0x558b5623c590 2024-09-19T04:54:44.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.587+0000 7f1d00c00640 1 -- 172.21.15.123:0/166664738 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f1d04012990 con 0x558b5623c590 2024-09-19T04:54:44.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.587+0000 7f1d20c4d740 10 client.27028 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:54:44.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.587+0000 7f1d20c4d740 10 client.27028 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:54:44.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.587+0000 7f1d20c4d740 20 client.27028 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:44.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.587+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/166664738 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x558b56256e70 0x558b56259290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.587+0000 7f1d20c4d740 4 client.27028 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:44.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.587+0000 7f1d20c4d740 1 -- 172.21.15.123:0/166664738 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b561af950 con 0x558b56256e70 2024-09-19T04:54:44.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.587+0000 7f1d17e00640 1 --2- 172.21.15.123:0/166664738 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x558b56256e70 0x558b56259290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.588+0000 7f1d17e00640 1 --2- 172.21.15.123:0/166664738 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x558b56256e70 0x558b56259290 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a5974a9348e77e89 server_cookie=a3b57966af97403e in_seq=0 out_seq=0 2024-09-19T04:54:44.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.588+0000 7f1d00c00640 10 client.27028 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:54:44.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.589+0000 7f1d00c00640 1 -- 172.21.15.123:0/166664738 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b561af950 con 0x558b56256e70 2024-09-19T04:54:44.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.589+0000 7f1d00c00640 10 client.27028 handle_command_reply: tid=0 2024-09-19T04:54:44.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.589+0000 7f1d20c4d740 1 client.27028 shutdown 2024-09-19T04:54:44.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.589+0000 7f1d20c4d740 1 -- 172.21.15.123:0/166664738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5623c590 msgr2=0x558b562442b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:44.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.589+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/166664738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5623c590 0x558b562442b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1d040104d0 tx=0x7f1d04010990 comp rx=0 tx=0).stop 2024-09-19T04:54:44.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.589+0000 7f1d20c4d740 1 -- 172.21.15.123:0/166664738 shutdown_connections 2024-09-19T04:54:44.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.589+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/166664738 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x558b56256e70 0x558b56259290 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.589+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/166664738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b56246e50 0x558b562447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.589+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/166664738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5623c590 0x558b562442b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.589+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/166664738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623bc70 0x558b56243d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.589+0000 7f1d20c4d740 1 -- 172.21.15.123:0/166664738 >> 172.21.15.123:0/166664738 conn(0x558b56238200 msgr2=0x558b5623df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:44.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.589+0000 7f1d00c00640 0 client.27028 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:54:44.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.589+0000 7f1d20c4d740 1 -- 172.21.15.123:0/166664738 shutdown_connections 2024-09-19T04:54:44.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.589+0000 7f1d20c4d740 1 -- 172.21.15.123:0/166664738 wait complete. 2024-09-19T04:54:44.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.589+0000 7f1d20c4d740 20 client.27028 trim_cache size 0 max 16384 2024-09-19T04:54:44.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.603+0000 7f1d20c4d740 1 Processor -- start 2024-09-19T04:54:44.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.603+0000 7f1d20c4d740 1 -- start start 2024-09-19T04:54:44.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.603+0000 7f1d20c4d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623bc70 0x558b5625eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.603+0000 7f1d20c4d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5623c590 0x558b5625f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.603+0000 7f1d20c4d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b56246e50 0x558b56236b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.603+0000 7f1d20c4d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181af930 con 0x558b56246e50 2024-09-19T04:54:44.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.603+0000 7f1d20c4d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181b1470 con 0x558b5623bc70 2024-09-19T04:54:44.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.603+0000 7f1d20c4d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181b0000 con 0x558b5623c590 2024-09-19T04:54:44.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.603+0000 7f1d17e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623bc70 0x558b5625eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.603+0000 7f1d17e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623bc70 0x558b5625eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:53178/0 (socket says 172.21.15.123:53178) 2024-09-19T04:54:44.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.603+0000 7f1d17e00640 1 -- 172.21.15.123:0/804118377 learned_addr learned my addr 172.21.15.123:0/804118377 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:44.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.603+0000 7f1d17e00640 1 --2- 172.21.15.123:0/804118377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b56246e50 0x558b56236b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.603+0000 7f1d16a00640 1 --2- 172.21.15.123:0/804118377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5623c590 0x558b5625f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.604+0000 7f1d17e00640 1 -- 172.21.15.123:0/804118377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5623c590 msgr2=0x558b5625f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:44.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.604+0000 7f1d17e00640 1 --2- 172.21.15.123:0/804118377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5623c590 0x558b5625f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.604+0000 7f1d17e00640 1 -- 172.21.15.123:0/804118377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b56246e50 msgr2=0x558b56236b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:44.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.604+0000 7f1d17e00640 1 --2- 172.21.15.123:0/804118377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b56246e50 0x558b56236b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.604+0000 7f1d17e00640 1 -- 172.21.15.123:0/804118377 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b5623baa0 con 0x558b5623bc70 2024-09-19T04:54:44.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.604+0000 7f1d17e00640 1 --2- 172.21.15.123:0/804118377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623bc70 0x558b5625eb40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1d10005e30 tx=0x7f1d10004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3599179dd55767f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:44.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.604+0000 7f1d03400640 1 -- 172.21.15.123:0/804118377 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10031570 con 0x558b5623bc70 2024-09-19T04:54:44.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.604+0000 7f1d03400640 1 -- 172.21.15.123:0/804118377 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d10036070 con 0x558b5623bc70 2024-09-19T04:54:44.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.604+0000 7f1d03400640 1 -- 172.21.15.123:0/804118377 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1002f610 con 0x558b5623bc70 2024-09-19T04:54:44.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.605+0000 7f1d20c4d740 1 -- 172.21.15.123:0/804118377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623bc70 msgr2=0x558b5625eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:44.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.605+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/804118377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623bc70 0x558b5625eb40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1d10005e30 tx=0x7f1d10004a80 comp rx=0 tx=0).stop 2024-09-19T04:54:44.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.605+0000 7f1d20c4d740 1 -- 172.21.15.123:0/804118377 shutdown_connections 2024-09-19T04:54:44.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.605+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/804118377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b56246e50 0x558b56236b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.605+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/804118377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5623c590 0x558b5625f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.605+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/804118377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623bc70 0x558b5625eb40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.605+0000 7f1d20c4d740 1 -- 172.21.15.123:0/804118377 >> 172.21.15.123:0/804118377 conn(0x558b56238200 msgr2=0x558b56242ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:44.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.605+0000 7f1d20c4d740 1 -- 172.21.15.123:0/804118377 shutdown_connections 2024-09-19T04:54:44.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.605+0000 7f1d20c4d740 1 -- 172.21.15.123:0/804118377 wait complete. 2024-09-19T04:54:44.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.605+0000 7f1d20c4d740 1 -- start start 2024-09-19T04:54:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.606+0000 7f1d20c4d740 1 Processor -- start 2024-09-19T04:54:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.606+0000 7f1d20c4d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b5623bc70 0x558b56240df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.606+0000 7f1d20c4d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623c590 0x558b56241330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.606+0000 7f1d20c4d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b56246e50 0x558b56241870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.606+0000 7f1d20c4d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181b0da0 con 0x558b5623bc70 2024-09-19T04:54:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.606+0000 7f1d20c4d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181b06d0 con 0x558b5623c590 2024-09-19T04:54:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.606+0000 7f1d20c4d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181ad930 con 0x558b56246e50 2024-09-19T04:54:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.606+0000 7f1d20c4d740 10 client.0 _ll_register_callbacks cb 0x7f1d04007520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.606+0000 7f1d17e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b5623bc70 0x558b56240df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.606+0000 7f1d17e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b5623bc70 0x558b56240df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53658/0 (socket says 172.21.15.123:53658) 2024-09-19T04:54:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.606+0000 7f1d17e00640 1 -- 172.21.15.123:0/1674992107 learned_addr learned my addr 172.21.15.123:0/1674992107 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.606+0000 7f1d16a00640 1 --2- 172.21.15.123:0/1674992107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623c590 0x558b56241330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.607+0000 7f1d17e00640 1 --2- 172.21.15.123:0/1674992107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b56246e50 0x558b56241870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.607+0000 7f1d16a00640 1 -- 172.21.15.123:0/1674992107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b56246e50 msgr2=0x558b56241870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:44.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.607+0000 7f1d16a00640 1 --2- 172.21.15.123:0/1674992107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b56246e50 0x558b56241870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.607+0000 7f1d16a00640 1 -- 172.21.15.123:0/1674992107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b5623bc70 msgr2=0x558b56240df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:44.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.607+0000 7f1d16a00640 1 --2- 172.21.15.123:0/1674992107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b5623bc70 0x558b56240df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.607+0000 7f1d16a00640 1 -- 172.21.15.123:0/1674992107 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b56233770 con 0x558b5623c590 2024-09-19T04:54:44.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.607+0000 7f1d17e00640 1 --2- 172.21.15.123:0/1674992107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b5623bc70 0x558b56240df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:44.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.607+0000 7f1d16a00640 1 --2- 172.21.15.123:0/1674992107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623c590 0x558b56241330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1d04000c00 tx=0x7f1d04007400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=256155bc9d02e7a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:44.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.607+0000 7f1d00c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:54:44.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.608+0000 7f1d00c00640 1 -- 172.21.15.123:0/1674992107 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0402c020 con 0x558b5623c590 2024-09-19T04:54:44.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.608+0000 7f1d20c4d740 1 -- 172.21.15.123:0/1674992107 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d10036070 con 0x558b5623c590 2024-09-19T04:54:44.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.608+0000 7f1d00c00640 1 -- 172.21.15.123:0/1674992107 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f1d04012600 con 0x558b5623c590 2024-09-19T04:54:44.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.608+0000 7f1d00c00640 1 -- 172.21.15.123:0/1674992107 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1d04037050 con 0x558b5623c590 2024-09-19T04:54:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.608+0000 7f1d20c4d740 10 client.27008 fetch_fsmap learned FSMap version 43 2024-09-19T04:54:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.608+0000 7f1d20c4d740 1 -- 172.21.15.123:0/1674992107 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558b562c1c50 con 0x558b5623c590 2024-09-19T04:54:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.608+0000 7f1d00c00640 1 -- 172.21.15.123:0/1674992107 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d10036070 con 0x558b5623c590 2024-09-19T04:54:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.608+0000 7f1d00c00640 1 -- 172.21.15.123:0/1674992107 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f1d04014b20 con 0x558b5623c590 2024-09-19T04:54:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.608+0000 7f1d20c4d740 10 client.27008 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:54:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.608+0000 7f1d20c4d740 10 client.27008 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:54:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.608+0000 7f1d20c4d740 20 client.27008 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.608+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/1674992107 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x558b562c2360 0x558b562c4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.609+0000 7f1d20c4d740 4 client.27008 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.609+0000 7f1d20c4d740 1 -- 172.21.15.123:0/1674992107 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b561a9e60 con 0x558b562c2360 2024-09-19T04:54:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.609+0000 7f1d17e00640 1 --2- 172.21.15.123:0/1674992107 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x558b562c2360 0x558b562c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.609+0000 7f1d17e00640 1 --2- 172.21.15.123:0/1674992107 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x558b562c2360 0x558b562c4780 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4247fa85fe02476 server_cookie=ba8d483cb8d0e52e in_seq=0 out_seq=0 2024-09-19T04:54:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.609+0000 7f1d00c00640 10 client.27008 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:54:44.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.610+0000 7f1d00c00640 1 -- 172.21.15.123:0/1674992107 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b561a9e60 con 0x558b562c2360 2024-09-19T04:54:44.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.610+0000 7f1d00c00640 10 client.27008 handle_command_reply: tid=0 2024-09-19T04:54:44.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.610+0000 7f1d20c4d740 1 client.27008 shutdown 2024-09-19T04:54:44.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.610+0000 7f1d20c4d740 1 -- 172.21.15.123:0/1674992107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623c590 msgr2=0x558b56241330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:44.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.610+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/1674992107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623c590 0x558b56241330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1d04000c00 tx=0x7f1d04007400 comp rx=0 tx=0).stop 2024-09-19T04:54:44.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.610+0000 7f1d20c4d740 1 -- 172.21.15.123:0/1674992107 shutdown_connections 2024-09-19T04:54:44.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.610+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/1674992107 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x558b562c2360 0x558b562c4780 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.610+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/1674992107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b56246e50 0x558b56241870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.610+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/1674992107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b5623c590 0x558b56241330 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.610+0000 7f1d20c4d740 1 --2- 172.21.15.123:0/1674992107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b5623bc70 0x558b56240df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.610+0000 7f1d00c00640 0 client.27008 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:54:44.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.610+0000 7f1d20c4d740 1 -- 172.21.15.123:0/1674992107 >> 172.21.15.123:0/1674992107 conn(0x558b56238200 msgr2=0x558b56243d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:44.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.610+0000 7f1d20c4d740 1 -- 172.21.15.123:0/1674992107 shutdown_connections 2024-09-19T04:54:44.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.610+0000 7f1d20c4d740 1 -- 172.21.15.123:0/1674992107 wait complete. 2024-09-19T04:54:44.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.610+0000 7f1d20c4d740 20 client.27008 trim_cache size 0 max 16384 2024-09-19T04:54:44.617 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:44.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.611+0000 7f1d1e200640 1 -- 172.21.15.123:0/1756230032 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1cfc0a33b0 msgr2=0x7f1cfc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:44.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.611+0000 7f1d1e200640 1 --2- 172.21.15.123:0/1756230032 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1cfc0a33b0 0x7f1cfc0a5870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1d0809c460 tx=0x7f1d08098040 comp rx=0 tx=0).stop 2024-09-19T04:54:44.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.611+0000 7f1d1e200640 1 -- 172.21.15.123:0/1756230032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d18101df0 msgr2=0x7f1d180744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:44.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.611+0000 7f1d1e200640 1 --2- 172.21.15.123:0/1756230032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d18101df0 0x7f1d180744f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800e9f0 tx=0x7f1d0800edf0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.611+0000 7f1d1e200640 1 -- 172.21.15.123:0/1756230032 shutdown_connections 2024-09-19T04:54:44.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.611+0000 7f1d1e200640 1 --2- 172.21.15.123:0/1756230032 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1cfc0a33b0 0x7f1cfc0a5870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.611+0000 7f1d1e200640 1 --2- 172.21.15.123:0/1756230032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d18106f20 0x7f1d18074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.611+0000 7f1d1e200640 1 --2- 172.21.15.123:0/1756230032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d181027a0 0x7f1d18074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.611+0000 7f1d1e200640 1 --2- 172.21.15.123:0/1756230032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d18101df0 0x7f1d180744f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.611+0000 7f1d1e200640 1 -- 172.21.15.123:0/1756230032 >> 172.21.15.123:0/1756230032 conn(0x7f1d180fd8b0 msgr2=0x7f1d180feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:44.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.611+0000 7f1d1e200640 1 -- 172.21.15.123:0/1756230032 shutdown_connections 2024-09-19T04:54:44.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:44.611+0000 7f1d1e200640 1 -- 172.21.15.123:0/1756230032 wait complete. 2024-09-19T04:54:45.469 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:54:45.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:45 smithi123 ceph-mon[30511]: pgmap v248: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:45.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:45 smithi123 ceph-mon[30511]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:54:45.841 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:45 smithi183 ceph-mon[35333]: pgmap v248: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:45.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:45 smithi183 ceph-mon[35333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:54:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:45 smithi193 ceph-mon[35592]: pgmap v248: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:45.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:45 smithi193 ceph-mon[35592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:54:46.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.399+0000 7fc0a7000640 1 -- 172.21.15.123:0/4238280411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a0106f10 msgr2=0x7fc0a010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:46.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.399+0000 7fc0a7000640 1 --2- 172.21.15.123:0/4238280411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a0106f10 0x7fc0a010ea90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc098009ca0 tx=0x7fc09802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.399+0000 7fc0a7000640 1 -- 172.21.15.123:0/4238280411 shutdown_connections 2024-09-19T04:54:46.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.399+0000 7fc0a7000640 1 --2- 172.21.15.123:0/4238280411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a0106f10 0x7fc0a010ea90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.399+0000 7fc0a7000640 1 --2- 172.21.15.123:0/4238280411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0a0102790 0x7fc0a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.399+0000 7fc0a7000640 1 --2- 172.21.15.123:0/4238280411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0a0101de0 0x7fc0a01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.399+0000 7fc0a7000640 1 -- 172.21.15.123:0/4238280411 >> 172.21.15.123:0/4238280411 conn(0x7fc0a00fd8c0 msgr2=0x7fc0a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:46.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.399+0000 7fc0a7000640 1 -- 172.21.15.123:0/4238280411 shutdown_connections 2024-09-19T04:54:46.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.399+0000 7fc0a7000640 1 -- 172.21.15.123:0/4238280411 wait complete. 2024-09-19T04:54:46.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.400+0000 7fc0a7000640 1 Processor -- start 2024-09-19T04:54:46.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.400+0000 7fc0a7000640 1 -- start start 2024-09-19T04:54:46.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.400+0000 7fc0a7000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0a0101de0 0x7fc0a019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.400+0000 7fc0a7000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0a0102790 0x7fc0a019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.400+0000 7fc0a7000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a0106f10 0x7fc0a019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.400+0000 7fc0a7000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a0111fc0 con 0x7fc0a0106f10 2024-09-19T04:54:46.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.400+0000 7fc0a7000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a0111e40 con 0x7fc0a0102790 2024-09-19T04:54:46.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.400+0000 7fc0a7000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a0112140 con 0x7fc0a0101de0 2024-09-19T04:54:46.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.401+0000 7fc0a5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a0106f10 0x7fc0a019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.401+0000 7fc0a5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a0106f10 0x7fc0a019eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53674/0 (socket says 172.21.15.123:53674) 2024-09-19T04:54:46.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.401+0000 7fc0a5000640 1 -- 172.21.15.123:0/3138998907 learned_addr learned my addr 172.21.15.123:0/3138998907 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:46.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.401+0000 7fc09fe00640 1 --2- 172.21.15.123:0/3138998907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0a0101de0 0x7fc0a019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.401+0000 7fc09f400640 1 --2- 172.21.15.123:0/3138998907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0a0102790 0x7fc0a019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.401+0000 7fc0a5000640 1 -- 172.21.15.123:0/3138998907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0a0101de0 msgr2=0x7fc0a019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:46.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.401+0000 7fc0a5000640 1 --2- 172.21.15.123:0/3138998907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0a0101de0 0x7fc0a019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.401+0000 7fc0a5000640 1 -- 172.21.15.123:0/3138998907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0a0102790 msgr2=0x7fc0a019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:46.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.401+0000 7fc0a5000640 1 --2- 172.21.15.123:0/3138998907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0a0102790 0x7fc0a019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.401+0000 7fc0a5000640 1 -- 172.21.15.123:0/3138998907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a019f5a0 con 0x7fc0a0106f10 2024-09-19T04:54:46.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.401+0000 7fc0a5000640 1 --2- 172.21.15.123:0/3138998907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a0106f10 0x7fc0a019eec0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc098005e30 tx=0x7fc098003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:46.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.402+0000 7fc09cc00640 1 -- 172.21.15.123:0/3138998907 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098031650 con 0x7fc0a0106f10 2024-09-19T04:54:46.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.402+0000 7fc09cc00640 1 -- 172.21.15.123:0/3138998907 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0980317f0 con 0x7fc0a0106f10 2024-09-19T04:54:46.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.402+0000 7fc09cc00640 1 -- 172.21.15.123:0/3138998907 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09802f9d0 con 0x7fc0a0106f10 2024-09-19T04:54:46.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.402+0000 7fc0a7000640 1 -- 172.21.15.123:0/3138998907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a019f830 con 0x7fc0a0106f10 2024-09-19T04:54:46.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.402+0000 7fc0a7000640 1 -- 172.21.15.123:0/3138998907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a01a7170 con 0x7fc0a0106f10 2024-09-19T04:54:46.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.406+0000 7fc09cc00640 1 -- 172.21.15.123:0/3138998907 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc09802fbf0 con 0x7fc0a0106f10 2024-09-19T04:54:46.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.406+0000 7fc0a9a45740 1 Processor -- start 2024-09-19T04:54:46.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.406+0000 7fc0a9a45740 1 -- start start 2024-09-19T04:54:46.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.406+0000 7fc0a9a45740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b34bc70 0x559a2b34c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.406+0000 7fc0a9a45740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34c590 0x559a2b356910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.406+0000 7fc0a9a45740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 0x559a2b359240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.406+0000 7fc0a9a45740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559a2b2b23e0 con 0x559a2b356e50 2024-09-19T04:54:46.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.406+0000 7fc0a9a45740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559a2b259d80 con 0x559a2b34bc70 2024-09-19T04:54:46.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.406+0000 7fc0a9a45740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559a2b2ae3a0 con 0x559a2b34c590 2024-09-19T04:54:46.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.406+0000 7fc09fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34c590 0x559a2b356910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.406+0000 7fc09f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 0x559a2b359240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.406+0000 7fc09fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34c590 0x559a2b356910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42668/0 (socket says 172.21.15.123:42668) 2024-09-19T04:54:46.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.406+0000 7fc09fe00640 1 -- 172.21.15.123:0/1238024831 learned_addr learned my addr 172.21.15.123:0/1238024831 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:46.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.406+0000 7fc09f400640 1 --2- 172.21.15.123:0/1238024831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b34bc70 0x559a2b34c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.407+0000 7fc09fe00640 1 -- 172.21.15.123:0/1238024831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b34bc70 msgr2=0x559a2b34c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:46.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.407+0000 7fc09fe00640 1 --2- 172.21.15.123:0/1238024831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b34bc70 0x559a2b34c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.407+0000 7fc09fe00640 1 -- 172.21.15.123:0/1238024831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 msgr2=0x559a2b359240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:46.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.407+0000 7fc09fe00640 1 --2- 172.21.15.123:0/1238024831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 0x559a2b359240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.407+0000 7fc09fe00640 1 -- 172.21.15.123:0/1238024831 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a2b278ae0 con 0x559a2b34c590 2024-09-19T04:54:46.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.407+0000 7fc09cc00640 1 --2- 172.21.15.123:0/3138998907 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc0840a3430 0x7fc0840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.407+0000 7fc09fe00640 1 --2- 172.21.15.123:0/1238024831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34c590 0x559a2b356910 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc090009d20 tx=0x7fc090011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6131fc1aacb638 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.407+0000 7fc09cc00640 1 -- 172.21.15.123:0/3138998907 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fc0980f0f00 con 0x7fc0a0106f10 2024-09-19T04:54:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.407+0000 7fc09f400640 1 --2- 172.21.15.123:0/3138998907 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc0840a3430 0x7fc0840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.407+0000 7fc095600640 1 -- 172.21.15.123:0/1238024831 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09001e070 con 0x559a2b34c590 2024-09-19T04:54:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.407+0000 7fc095600640 1 -- 172.21.15.123:0/1238024831 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc090019b10 con 0x559a2b34c590 2024-09-19T04:54:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.408+0000 7fc095600640 1 -- 172.21.15.123:0/1238024831 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09000eb20 con 0x559a2b34c590 2024-09-19T04:54:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.408+0000 7fc0a9a45740 1 -- 172.21.15.123:0/1238024831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34c590 msgr2=0x559a2b356910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.408+0000 7fc09f400640 1 --2- 172.21.15.123:0/3138998907 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc0840a3430 0x7fc0840a58f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x559a2b349580 tx=0x7fc08c004730 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.408+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/1238024831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34c590 0x559a2b356910 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc090009d20 tx=0x7fc090011810 comp rx=0 tx=0).stop 2024-09-19T04:54:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.409+0000 7fc0a9a45740 1 -- 172.21.15.123:0/1238024831 shutdown_connections 2024-09-19T04:54:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.409+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/1238024831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 0x559a2b359240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.409+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/1238024831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34c590 0x559a2b356910 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.409+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/1238024831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b34bc70 0x559a2b34c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.409+0000 7fc0a9a45740 1 -- 172.21.15.123:0/1238024831 >> 172.21.15.123:0/1238024831 conn(0x559a2b348200 msgr2=0x559a2b3485e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.409+0000 7fc0a9a45740 1 -- 172.21.15.123:0/1238024831 shutdown_connections 2024-09-19T04:54:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.409+0000 7fc0a9a45740 1 -- 172.21.15.123:0/1238024831 wait complete. 2024-09-19T04:54:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.409+0000 7fc0a9a45740 1 -- start start 2024-09-19T04:54:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.409+0000 7fc0a9a45740 1 Processor -- start 2024-09-19T04:54:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.410+0000 7fc0a9a45740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 0x559a2b353d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.410+0000 7fc0a9a45740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b34c590 0x559a2b354250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.410+0000 7fc0a9a45740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b356e50 0x559a2b354790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.410+0000 7fc0a9a45740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559a2b359a20 con 0x559a2b34c590 2024-09-19T04:54:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.410+0000 7fc0a9a45740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559a2b26d0f0 con 0x559a2b356e50 2024-09-19T04:54:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.410+0000 7fc0a9a45740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559a2b359ba0 con 0x559a2b34bc70 2024-09-19T04:54:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.410+0000 7fc0a9a45740 10 client.0 _ll_register_callbacks cb 0x559a2b254bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.410+0000 7fc09fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 0x559a2b353d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.410+0000 7fc09fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 0x559a2b353d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42678/0 (socket says 172.21.15.123:42678) 2024-09-19T04:54:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.410+0000 7fc09fe00640 1 -- 172.21.15.123:0/600660554 learned_addr learned my addr 172.21.15.123:0/600660554 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.410+0000 7fc09f400640 1 --2- 172.21.15.123:0/600660554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b34c590 0x559a2b354250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.410+0000 7fc0a5000640 1 --2- 172.21.15.123:0/600660554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b356e50 0x559a2b354790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.410+0000 7fc09fe00640 1 -- 172.21.15.123:0/600660554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b356e50 msgr2=0x559a2b354790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.410+0000 7fc09fe00640 1 --2- 172.21.15.123:0/600660554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b356e50 0x559a2b354790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.410+0000 7fc09fe00640 1 -- 172.21.15.123:0/600660554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b34c590 msgr2=0x559a2b354250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:46.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.410+0000 7fc09fe00640 1 --2- 172.21.15.123:0/600660554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b34c590 0x559a2b354250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.410+0000 7fc09fe00640 1 -- 172.21.15.123:0/600660554 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a2b0233c0 con 0x559a2b34bc70 2024-09-19T04:54:46.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.411+0000 7fc09fe00640 1 --2- 172.21.15.123:0/600660554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 0x559a2b353d10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc090011d10 tx=0x7fc090019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5590c999d338bca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:46.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.411+0000 7fc08a000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:54:46.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.411+0000 7fc08a000640 1 -- 172.21.15.123:0/600660554 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc090027020 con 0x559a2b34bc70 2024-09-19T04:54:46.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.411+0000 7fc0a9a45740 1 -- 172.21.15.123:0/600660554 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc090019b10 con 0x559a2b34bc70 2024-09-19T04:54:46.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.411+0000 7fc08a000640 1 -- 172.21.15.123:0/600660554 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fc090005250 con 0x559a2b34bc70 2024-09-19T04:54:46.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.411+0000 7fc08a000640 1 -- 172.21.15.123:0/600660554 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc09001e070 con 0x559a2b34bc70 2024-09-19T04:54:46.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.411+0000 7fc0a9a45740 10 client.27046 fetch_fsmap learned FSMap version 43 2024-09-19T04:54:46.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.411+0000 7fc0a9a45740 1 -- 172.21.15.123:0/600660554 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559a2b365a90 con 0x559a2b34bc70 2024-09-19T04:54:46.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.412+0000 7fc08a000640 1 -- 172.21.15.123:0/600660554 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc090019b10 con 0x559a2b34bc70 2024-09-19T04:54:46.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.412+0000 7fc08a000640 1 -- 172.21.15.123:0/600660554 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fc09000e980 con 0x559a2b34bc70 2024-09-19T04:54:46.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.412+0000 7fc0a9a45740 10 client.27046 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:54:46.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.412+0000 7fc0a9a45740 10 client.27046 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:54:46.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.412+0000 7fc0a9a45740 20 client.27046 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:46.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.412+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/600660554 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x559a2b366de0 0x559a2b3691d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.412+0000 7fc0a9a45740 4 client.27046 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:46.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.412+0000 7fc0a9a45740 1 -- 172.21.15.123:0/600660554 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a2b2bf950 con 0x559a2b366de0 2024-09-19T04:54:46.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.412+0000 7fc09f400640 1 --2- 172.21.15.123:0/600660554 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x559a2b366de0 0x559a2b3691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.413+0000 7fc09f400640 1 --2- 172.21.15.123:0/600660554 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x559a2b366de0 0x559a2b3691d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b27130160055ec96 server_cookie=e1466493a3d44659 in_seq=0 out_seq=0 2024-09-19T04:54:46.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.413+0000 7fc08a000640 10 client.27046 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:54:46.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.414+0000 7fc08a000640 1 -- 172.21.15.123:0/600660554 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a2b2bf950 con 0x559a2b366de0 2024-09-19T04:54:46.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.414+0000 7fc08a000640 10 client.27046 handle_command_reply: tid=0 2024-09-19T04:54:46.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.414+0000 7fc0a9a45740 1 client.27046 shutdown 2024-09-19T04:54:46.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.414+0000 7fc0a9a45740 1 -- 172.21.15.123:0/600660554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 msgr2=0x559a2b353d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:46.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.414+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/600660554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 0x559a2b353d10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc090011d10 tx=0x7fc090019660 comp rx=0 tx=0).stop 2024-09-19T04:54:46.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.414+0000 7fc0a9a45740 1 -- 172.21.15.123:0/600660554 shutdown_connections 2024-09-19T04:54:46.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.414+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/600660554 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x559a2b366de0 0x559a2b3691d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.414+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/600660554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b356e50 0x559a2b354790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.414+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/600660554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b34c590 0x559a2b354250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.414+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/600660554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 0x559a2b353d10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.414+0000 7fc08a000640 0 client.27046 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:54:46.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.414+0000 7fc0a9a45740 1 -- 172.21.15.123:0/600660554 >> 172.21.15.123:0/600660554 conn(0x559a2b348200 msgr2=0x559a2b34dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:46.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.414+0000 7fc0a9a45740 1 -- 172.21.15.123:0/600660554 shutdown_connections 2024-09-19T04:54:46.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.414+0000 7fc0a9a45740 1 -- 172.21.15.123:0/600660554 wait complete. 2024-09-19T04:54:46.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.415+0000 7fc0a9a45740 20 client.27046 trim_cache size 0 max 16384 2024-09-19T04:54:46.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc0a9a45740 1 Processor -- start 2024-09-19T04:54:46.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc0a9a45740 1 -- start start 2024-09-19T04:54:46.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc0a9a45740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 0x559a2b36ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc0a9a45740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b34c590 0x559a2b36ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc0a9a45740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 0x559a2b34d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc0a9a45740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a01afa80 con 0x559a2b356e50 2024-09-19T04:54:46.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc0a9a45740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a01b15c0 con 0x559a2b34c590 2024-09-19T04:54:46.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc0a9a45740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a01b0150 con 0x559a2b34bc70 2024-09-19T04:54:46.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc0a5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 0x559a2b34d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc09f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b34c590 0x559a2b36ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc09fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 0x559a2b36ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc0a5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 0x559a2b34d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53716/0 (socket says 172.21.15.123:53716) 2024-09-19T04:54:46.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc0a5000640 1 -- 172.21.15.123:0/1314577755 learned_addr learned my addr 172.21.15.123:0/1314577755 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:46.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc09fe00640 1 -- 172.21.15.123:0/1314577755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b34c590 msgr2=0x559a2b36ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:46.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc09fe00640 1 --2- 172.21.15.123:0/1314577755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b34c590 0x559a2b36ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc09fe00640 1 -- 172.21.15.123:0/1314577755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 msgr2=0x559a2b34d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:46.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc09fe00640 1 --2- 172.21.15.123:0/1314577755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 0x559a2b34d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc09fe00640 1 -- 172.21.15.123:0/1314577755 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a2b343770 con 0x559a2b34bc70 2024-09-19T04:54:46.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc09f400640 1 --2- 172.21.15.123:0/1314577755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b34c590 0x559a2b36ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:46.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc0a5000640 1 --2- 172.21.15.123:0/1314577755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 0x559a2b34d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:46.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.429+0000 7fc09fe00640 1 --2- 172.21.15.123:0/1314577755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 0x559a2b36ea30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc09000ef60 tx=0x7fc090005bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3de113121019c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:46.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.430+0000 7fc094c00640 1 -- 172.21.15.123:0/1314577755 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc090027040 con 0x559a2b34bc70 2024-09-19T04:54:46.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.430+0000 7fc094c00640 1 -- 172.21.15.123:0/1314577755 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc090015070 con 0x559a2b34bc70 2024-09-19T04:54:46.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.430+0000 7fc094c00640 1 -- 172.21.15.123:0/1314577755 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc090004290 con 0x559a2b34bc70 2024-09-19T04:54:46.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.430+0000 7fc0a9a45740 1 -- 172.21.15.123:0/1314577755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 msgr2=0x559a2b36ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:46.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.430+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/1314577755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 0x559a2b36ea30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc09000ef60 tx=0x7fc090005bb0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.430+0000 7fc0a9a45740 1 -- 172.21.15.123:0/1314577755 shutdown_connections 2024-09-19T04:54:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.430+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/1314577755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 0x559a2b34d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.430+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/1314577755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b34c590 0x559a2b36ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.430+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/1314577755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 0x559a2b36ea30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.430+0000 7fc0a9a45740 1 -- 172.21.15.123:0/1314577755 >> 172.21.15.123:0/1314577755 conn(0x559a2b348200 msgr2=0x559a2b347380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.431+0000 7fc0a9a45740 1 -- 172.21.15.123:0/1314577755 shutdown_connections 2024-09-19T04:54:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.431+0000 7fc0a9a45740 1 -- 172.21.15.123:0/1314577755 wait complete. 2024-09-19T04:54:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.431+0000 7fc0a9a45740 1 -- start start 2024-09-19T04:54:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.431+0000 7fc0a9a45740 1 Processor -- start 2024-09-19T04:54:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc0a9a45740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 0x559a2b35efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc0a9a45740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b34c590 0x559a2b35f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc0a9a45740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 0x559a2b350520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc0a9a45740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a01b0ef0 con 0x559a2b356e50 2024-09-19T04:54:46.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc0a9a45740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a01b0820 con 0x559a2b34c590 2024-09-19T04:54:46.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc0a9a45740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a01ada80 con 0x559a2b34bc70 2024-09-19T04:54:46.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc0a9a45740 10 client.0 _ll_register_callbacks cb 0x7fc09001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:46.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc09fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 0x559a2b35efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc0a5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 0x559a2b350520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc09fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 0x559a2b35efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42694/0 (socket says 172.21.15.123:42694) 2024-09-19T04:54:46.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc09fe00640 1 -- 172.21.15.123:0/2827700281 learned_addr learned my addr 172.21.15.123:0/2827700281 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:46.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc09f400640 1 --2- 172.21.15.123:0/2827700281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b34c590 0x559a2b35f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc09fe00640 1 -- 172.21.15.123:0/2827700281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b34c590 msgr2=0x559a2b35f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:46.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc09fe00640 1 --2- 172.21.15.123:0/2827700281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b34c590 0x559a2b35f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc09fe00640 1 -- 172.21.15.123:0/2827700281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 msgr2=0x559a2b350520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:46.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc09fe00640 1 --2- 172.21.15.123:0/2827700281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 0x559a2b350520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc09fe00640 1 -- 172.21.15.123:0/2827700281 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a2b350c30 con 0x559a2b34bc70 2024-09-19T04:54:46.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.432+0000 7fc0a5000640 1 --2- 172.21.15.123:0/2827700281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 0x559a2b350520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:46.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.433+0000 7fc09fe00640 1 --2- 172.21.15.123:0/2827700281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 0x559a2b35efd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc09000eee0 tx=0x7fc09000e690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c5e0fe65554f179 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:46.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.433+0000 7fc08a000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:54:46.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.433+0000 7fc08a000640 1 -- 172.21.15.123:0/2827700281 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc090043060 con 0x559a2b34bc70 2024-09-19T04:54:46.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.433+0000 7fc0a9a45740 1 -- 172.21.15.123:0/2827700281 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc090015070 con 0x559a2b34bc70 2024-09-19T04:54:46.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.433+0000 7fc08a000640 1 -- 172.21.15.123:0/2827700281 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fc0900050e0 con 0x559a2b34bc70 2024-09-19T04:54:46.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.433+0000 7fc08a000640 1 -- 172.21.15.123:0/2827700281 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc090027040 con 0x559a2b34bc70 2024-09-19T04:54:46.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.433+0000 7fc0a9a45740 10 client.27052 fetch_fsmap learned FSMap version 43 2024-09-19T04:54:46.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.433+0000 7fc0a9a45740 1 -- 172.21.15.123:0/2827700281 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559a2b3d1400 con 0x559a2b34bc70 2024-09-19T04:54:46.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.434+0000 7fc08a000640 1 -- 172.21.15.123:0/2827700281 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc090015070 con 0x559a2b34bc70 2024-09-19T04:54:46.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.434+0000 7fc08a000640 1 -- 172.21.15.123:0/2827700281 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fc0900037f0 con 0x559a2b34bc70 2024-09-19T04:54:46.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.434+0000 7fc0a9a45740 10 client.27052 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:54:46.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.434+0000 7fc0a9a45740 10 client.27052 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:54:46.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.434+0000 7fc0a9a45740 20 client.27052 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:46.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.434+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/2827700281 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x559a2b3d1a60 0x559a2b3d3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:46.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.434+0000 7fc0a9a45740 4 client.27052 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:46.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.434+0000 7fc0a9a45740 1 -- 172.21.15.123:0/2827700281 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a2b2b9e60 con 0x559a2b3d1a60 2024-09-19T04:54:46.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.434+0000 7fc09f400640 1 --2- 172.21.15.123:0/2827700281 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x559a2b3d1a60 0x559a2b3d3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:46.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.435+0000 7fc09f400640 1 --2- 172.21.15.123:0/2827700281 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x559a2b3d1a60 0x559a2b3d3e50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=19596de41715cb8f server_cookie=33d91ffc1821eb53 in_seq=0 out_seq=0 2024-09-19T04:54:46.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.435+0000 7fc08a000640 10 client.27052 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:54:46.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.435+0000 7fc08a000640 1 -- 172.21.15.123:0/2827700281 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a2b2b9e60 con 0x559a2b3d1a60 2024-09-19T04:54:46.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.435+0000 7fc08a000640 10 client.27052 handle_command_reply: tid=0 2024-09-19T04:54:46.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.435+0000 7fc0a9a45740 1 client.27052 shutdown 2024-09-19T04:54:46.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.435+0000 7fc0a9a45740 1 -- 172.21.15.123:0/2827700281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 msgr2=0x559a2b35efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:46.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.435+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/2827700281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 0x559a2b35efd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc09000eee0 tx=0x7fc09000e690 comp rx=0 tx=0).stop 2024-09-19T04:54:46.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.435+0000 7fc0a9a45740 1 -- 172.21.15.123:0/2827700281 shutdown_connections 2024-09-19T04:54:46.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.435+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/2827700281 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x559a2b3d1a60 0x559a2b3d3e50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.435+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/2827700281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a2b356e50 0x559a2b350520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.435+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/2827700281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a2b34c590 0x559a2b35f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.435+0000 7fc0a9a45740 1 --2- 172.21.15.123:0/2827700281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a2b34bc70 0x559a2b35efd0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.435+0000 7fc0a9a45740 1 -- 172.21.15.123:0/2827700281 >> 172.21.15.123:0/2827700281 conn(0x559a2b348200 msgr2=0x559a2b355570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:46.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.435+0000 7fc08a000640 0 client.27052 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:54:46.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.435+0000 7fc0a9a45740 1 -- 172.21.15.123:0/2827700281 shutdown_connections 2024-09-19T04:54:46.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.435+0000 7fc0a9a45740 1 -- 172.21.15.123:0/2827700281 wait complete. 2024-09-19T04:54:46.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.435+0000 7fc0a9a45740 20 client.27052 trim_cache size 0 max 16384 2024-09-19T04:54:46.444 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:46.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.436+0000 7fc0a7000640 1 -- 172.21.15.123:0/3138998907 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc0840a3430 msgr2=0x7fc0840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:46.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.436+0000 7fc0a7000640 1 --2- 172.21.15.123:0/3138998907 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc0840a3430 0x7fc0840a58f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x559a2b349580 tx=0x7fc08c004730 comp rx=0 tx=0).stop 2024-09-19T04:54:46.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.436+0000 7fc0a7000640 1 -- 172.21.15.123:0/3138998907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a0106f10 msgr2=0x7fc0a019eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:46.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.436+0000 7fc0a7000640 1 --2- 172.21.15.123:0/3138998907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a0106f10 0x7fc0a019eec0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc098005e30 tx=0x7fc098003d00 comp rx=0 tx=0).stop 2024-09-19T04:54:46.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.436+0000 7fc0a7000640 1 -- 172.21.15.123:0/3138998907 shutdown_connections 2024-09-19T04:54:46.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.436+0000 7fc0a7000640 1 --2- 172.21.15.123:0/3138998907 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc0840a3430 0x7fc0840a58f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.436+0000 7fc0a7000640 1 --2- 172.21.15.123:0/3138998907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a0106f10 0x7fc0a019eec0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.436+0000 7fc0a7000640 1 --2- 172.21.15.123:0/3138998907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0a0102790 0x7fc0a019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.436+0000 7fc0a7000640 1 --2- 172.21.15.123:0/3138998907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0a0101de0 0x7fc0a019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:46.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.436+0000 7fc0a7000640 1 -- 172.21.15.123:0/3138998907 >> 172.21.15.123:0/3138998907 conn(0x7fc0a00fd8c0 msgr2=0x7fc0a00ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:46.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.436+0000 7fc0a7000640 1 -- 172.21.15.123:0/3138998907 shutdown_connections 2024-09-19T04:54:46.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:46.436+0000 7fc0a7000640 1 -- 172.21.15.123:0/3138998907 wait complete. 2024-09-19T04:54:47.371 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:54:47.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:47 smithi123 ceph-mon[30511]: pgmap v249: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:47.742 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:47.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:47 smithi183 ceph-mon[35333]: pgmap v249: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:47 smithi193 ceph-mon[35592]: pgmap v249: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:48.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.336+0000 7f9d68a00640 1 -- 172.21.15.123:0/1257469558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d64109120 msgr2=0x7f9d64110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:48.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.336+0000 7f9d68a00640 1 --2- 172.21.15.123:0/1257469558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d64109120 0x7f9d64110ca0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009ca0 tx=0x7f9d5c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.336+0000 7f9d68a00640 1 -- 172.21.15.123:0/1257469558 shutdown_connections 2024-09-19T04:54:48.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.336+0000 7f9d68a00640 1 --2- 172.21.15.123:0/1257469558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d64109120 0x7f9d64110ca0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.336+0000 7f9d68a00640 1 --2- 172.21.15.123:0/1257469558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d641049a0 0x7f9d641089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.336+0000 7f9d68a00640 1 --2- 172.21.15.123:0/1257469558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d64103ff0 0x7f9d641043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.336+0000 7f9d68a00640 1 -- 172.21.15.123:0/1257469558 >> 172.21.15.123:0/1257469558 conn(0x7f9d640ffa90 msgr2=0x7f9d64101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:48.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.336+0000 7f9d68a00640 1 -- 172.21.15.123:0/1257469558 shutdown_connections 2024-09-19T04:54:48.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.336+0000 7f9d68a00640 1 -- 172.21.15.123:0/1257469558 wait complete. 2024-09-19T04:54:48.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.337+0000 7f9d68a00640 1 Processor -- start 2024-09-19T04:54:48.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.337+0000 7f9d68a00640 1 -- start start 2024-09-19T04:54:48.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.338+0000 7f9d68a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d64103ff0 0x7f9d6419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.338+0000 7f9d68a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d641049a0 0x7f9d6419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.338+0000 7f9d68a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d64109120 0x7f9d6419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.338+0000 7f9d68a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641141a0 con 0x7f9d64109120 2024-09-19T04:54:48.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.338+0000 7f9d68a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64114020 con 0x7f9d64103ff0 2024-09-19T04:54:48.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.338+0000 7f9d68a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64114320 con 0x7f9d641049a0 2024-09-19T04:54:48.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.338+0000 7f9d62800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d64109120 0x7f9d6419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.338+0000 7f9d61400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d641049a0 0x7f9d6419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.338+0000 7f9d62800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d64109120 0x7f9d6419ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53738/0 (socket says 172.21.15.123:53738) 2024-09-19T04:54:48.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.338+0000 7f9d62800640 1 -- 172.21.15.123:0/2174959118 learned_addr learned my addr 172.21.15.123:0/2174959118 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:48.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.338+0000 7f9d61e00640 1 --2- 172.21.15.123:0/2174959118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d64103ff0 0x7f9d6419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.339+0000 7f9d61400640 1 -- 172.21.15.123:0/2174959118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d64103ff0 msgr2=0x7f9d6419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:48.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.339+0000 7f9d61400640 1 --2- 172.21.15.123:0/2174959118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d64103ff0 0x7f9d6419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.339+0000 7f9d61400640 1 -- 172.21.15.123:0/2174959118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d64109120 msgr2=0x7f9d6419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:48.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.339+0000 7f9d61400640 1 --2- 172.21.15.123:0/2174959118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d64109120 0x7f9d6419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.339+0000 7f9d61400640 1 -- 172.21.15.123:0/2174959118 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6419f570 con 0x7f9d641049a0 2024-09-19T04:54:48.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.339+0000 7f9d61e00640 1 --2- 172.21.15.123:0/2174959118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d64103ff0 0x7f9d6419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:54:48.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.339+0000 7f9d61400640 1 --2- 172.21.15.123:0/2174959118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d641049a0 0x7f9d6419ab00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000e9f0 tx=0x7f9d5000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:48.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.339+0000 7f9d5aa00640 1 -- 172.21.15.123:0/2174959118 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5000ccc0 con 0x7f9d641049a0 2024-09-19T04:54:48.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.339+0000 7f9d68a00640 1 -- 172.21.15.123:0/2174959118 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6419f860 con 0x7f9d641049a0 2024-09-19T04:54:48.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.339+0000 7f9d5aa00640 1 -- 172.21.15.123:0/2174959118 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d50010910 con 0x7f9d641049a0 2024-09-19T04:54:48.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.339+0000 7f9d5aa00640 1 -- 172.21.15.123:0/2174959118 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50018960 con 0x7f9d641049a0 2024-09-19T04:54:48.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.339+0000 7f9d68a00640 1 -- 172.21.15.123:0/2174959118 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d641a7140 con 0x7f9d641049a0 2024-09-19T04:54:48.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.342+0000 7f9d6b3e0740 1 Processor -- start 2024-09-19T04:54:48.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.343+0000 7f9d6b3e0740 1 -- start start 2024-09-19T04:54:48.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.343+0000 7f9d5aa00640 1 -- 172.21.15.123:0/2174959118 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9d50010ac0 con 0x7f9d641049a0 2024-09-19T04:54:48.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.343+0000 7f9d6b3e0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7ac70 0x559747b7b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.343+0000 7f9d6b3e0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 0x559747b85910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.343+0000 7f9d6b3e0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b85e50 0x559747b88240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.343+0000 7f9d6b3e0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559747ae13e0 con 0x559747b7ac70 2024-09-19T04:54:48.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.343+0000 7f9d6b3e0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559747a88d80 con 0x559747b7b590 2024-09-19T04:54:48.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.343+0000 7f9d6b3e0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559747add3a0 con 0x559747b85e50 2024-09-19T04:54:48.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.343+0000 7f9d62800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7ac70 0x559747b7b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.343+0000 7f9d62800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7ac70 0x559747b7b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53744/0 (socket says 172.21.15.123:53744) 2024-09-19T04:54:48.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.343+0000 7f9d62800640 1 -- 172.21.15.123:0/2986788870 learned_addr learned my addr 172.21.15.123:0/2986788870 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:48.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.343+0000 7f9d61e00640 1 --2- 172.21.15.123:0/2986788870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 0x559747b85910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.344+0000 7f9d61400640 1 --2- 172.21.15.123:0/2986788870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b85e50 0x559747b88240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.344+0000 7f9d62800640 1 -- 172.21.15.123:0/2986788870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b85e50 msgr2=0x559747b88240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:48.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.344+0000 7f9d62800640 1 --2- 172.21.15.123:0/2986788870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b85e50 0x559747b88240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.344+0000 7f9d62800640 1 -- 172.21.15.123:0/2986788870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 msgr2=0x559747b85910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:48.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.344+0000 7f9d62800640 1 --2- 172.21.15.123:0/2986788870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 0x559747b85910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.344+0000 7f9d62800640 1 -- 172.21.15.123:0/2986788870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559747aa7ae0 con 0x559747b7ac70 2024-09-19T04:54:48.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.344+0000 7f9d61400640 1 --2- 172.21.15.123:0/2986788870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b85e50 0x559747b88240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:54:48.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.344+0000 7f9d62800640 1 --2- 172.21.15.123:0/2986788870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7ac70 0x559747b7b050 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c005fb0 tx=0x7f9d5c009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ac8c348fa0c327 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:48.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.344+0000 7f9d4f000640 1 -- 172.21.15.123:0/2986788870 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c031db0 con 0x559747b7ac70 2024-09-19T04:54:48.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.344+0000 7f9d4f000640 1 -- 172.21.15.123:0/2986788870 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d5c036030 con 0x559747b7ac70 2024-09-19T04:54:48.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.344+0000 7f9d4f000640 1 -- 172.21.15.123:0/2986788870 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c004260 con 0x559747b7ac70 2024-09-19T04:54:48.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.345+0000 7f9d5aa00640 1 --2- 172.21.15.123:0/2174959118 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9d480a3430 0x7f9d480a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.345+0000 7f9d5aa00640 1 -- 172.21.15.123:0/2174959118 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f9d500d00e0 con 0x7f9d641049a0 2024-09-19T04:54:48.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.345+0000 7f9d61e00640 1 --2- 172.21.15.123:0/2174959118 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9d480a3430 0x7f9d480a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.345+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/2986788870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7ac70 msgr2=0x559747b7b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:48.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.345+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/2986788870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7ac70 0x559747b7b050 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c005fb0 tx=0x7f9d5c009a10 comp rx=0 tx=0).stop 2024-09-19T04:54:48.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.345+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/2986788870 shutdown_connections 2024-09-19T04:54:48.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.345+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/2986788870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b85e50 0x559747b88240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.345+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/2986788870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 0x559747b85910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.345+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/2986788870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7ac70 0x559747b7b050 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.345+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/2986788870 >> 172.21.15.123:0/2986788870 conn(0x559747b77200 msgr2=0x559747b775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:48.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.346+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/2986788870 shutdown_connections 2024-09-19T04:54:48.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.346+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/2986788870 wait complete. 2024-09-19T04:54:48.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.346+0000 7f9d61e00640 1 --2- 172.21.15.123:0/2174959118 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9d480a3430 0x7f9d480a58f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x559747b78060 tx=0x7f9d54005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:48.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.346+0000 7f9d6b3e0740 1 -- start start 2024-09-19T04:54:48.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.346+0000 7f9d6b3e0740 1 Processor -- start 2024-09-19T04:54:48.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.347+0000 7f9d6b3e0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7ac70 0x559747b82ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.347+0000 7f9d6b3e0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 0x559747b831e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.347+0000 7f9d6b3e0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b85e50 0x559747b83720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.347+0000 7f9d6b3e0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559747b888a0 con 0x559747b7ac70 2024-09-19T04:54:48.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.347+0000 7f9d6b3e0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559747a9c0f0 con 0x559747b7b590 2024-09-19T04:54:48.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.347+0000 7f9d6b3e0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559747b88a20 con 0x559747b85e50 2024-09-19T04:54:48.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.347+0000 7f9d6b3e0740 10 client.0 _ll_register_callbacks cb 0x559747a83bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:48.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.347+0000 7f9d61e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7ac70 0x559747b82ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.347+0000 7f9d61e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7ac70 0x559747b82ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53754/0 (socket says 172.21.15.123:53754) 2024-09-19T04:54:48.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.347+0000 7f9d61e00640 1 -- 172.21.15.123:0/4008095062 learned_addr learned my addr 172.21.15.123:0/4008095062 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:48.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.347+0000 7f9d61400640 1 --2- 172.21.15.123:0/4008095062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 0x559747b831e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.347+0000 7f9d62800640 1 --2- 172.21.15.123:0/4008095062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b85e50 0x559747b83720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.348+0000 7f9d61e00640 1 -- 172.21.15.123:0/4008095062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b85e50 msgr2=0x559747b83720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:48.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.348+0000 7f9d61e00640 1 --2- 172.21.15.123:0/4008095062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b85e50 0x559747b83720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.348+0000 7f9d61e00640 1 -- 172.21.15.123:0/4008095062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 msgr2=0x559747b831e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:48.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.348+0000 7f9d61e00640 1 --2- 172.21.15.123:0/4008095062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 0x559747b831e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.348+0000 7f9d61e00640 1 -- 172.21.15.123:0/4008095062 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597478523c0 con 0x559747b7ac70 2024-09-19T04:54:48.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.348+0000 7f9d61400640 1 --2- 172.21.15.123:0/4008095062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 0x559747b831e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:48.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.348+0000 7f9d61e00640 1 --2- 172.21.15.123:0/4008095062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7ac70 0x559747b82ca0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9d5401fa20 tx=0x7f9d54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b42a29dfa7a3949 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:48.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.348+0000 7f9d43400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:48.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.348+0000 7f9d43400640 1 -- 172.21.15.123:0/4008095062 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5401d620 con 0x559747b7ac70 2024-09-19T04:54:48.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.348+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/4008095062 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d5c036030 con 0x559747b7ac70 2024-09-19T04:54:48.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.348+0000 7f9d43400640 1 -- 172.21.15.123:0/4008095062 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f9d5401d7c0 con 0x559747b7ac70 2024-09-19T04:54:48.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.348+0000 7f9d43400640 1 -- 172.21.15.123:0/4008095062 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9d5401da00 con 0x559747b7ac70 2024-09-19T04:54:48.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.348+0000 7f9d6b3e0740 10 client.16947 fetch_fsmap learned FSMap version 43 2024-09-19T04:54:48.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.348+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/4008095062 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559747b94860 con 0x559747b7ac70 2024-09-19T04:54:48.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.349+0000 7f9d43400640 1 -- 172.21.15.123:0/4008095062 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d5c036030 con 0x559747b7ac70 2024-09-19T04:54:48.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.349+0000 7f9d43400640 1 -- 172.21.15.123:0/4008095062 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f9d54011440 con 0x559747b7ac70 2024-09-19T04:54:48.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.349+0000 7f9d6b3e0740 10 client.16947 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:54:48.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.349+0000 7f9d6b3e0740 10 client.16947 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:54:48.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.349+0000 7f9d6b3e0740 20 client.16947 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:48.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.349+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/4008095062 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x559747b95d20 0x559747b98190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.349+0000 7f9d6b3e0740 4 client.16947 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:48.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.349+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/4008095062 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559747aee950 con 0x559747b95d20 2024-09-19T04:54:48.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.350+0000 7f9d61400640 1 --2- 172.21.15.123:0/4008095062 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x559747b95d20 0x559747b98190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.350+0000 7f9d61400640 1 --2- 172.21.15.123:0/4008095062 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x559747b95d20 0x559747b98190 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20cea4d33a1442f1 server_cookie=a7ce1fb47bd140fe in_seq=0 out_seq=0 2024-09-19T04:54:48.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.350+0000 7f9d43400640 10 client.16947 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:54:48.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.351+0000 7f9d43400640 1 -- 172.21.15.123:0/4008095062 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559747aee950 con 0x559747b95d20 2024-09-19T04:54:48.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.351+0000 7f9d43400640 10 client.16947 handle_command_reply: tid=0 2024-09-19T04:54:48.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.351+0000 7f9d6b3e0740 1 client.16947 shutdown 2024-09-19T04:54:48.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.351+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/4008095062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7ac70 msgr2=0x559747b82ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:48.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.351+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/4008095062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7ac70 0x559747b82ca0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9d5401fa20 tx=0x7f9d54004730 comp rx=0 tx=0).stop 2024-09-19T04:54:48.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.351+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/4008095062 shutdown_connections 2024-09-19T04:54:48.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.351+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/4008095062 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x559747b95d20 0x559747b98190 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.351+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/4008095062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b85e50 0x559747b83720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.351+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/4008095062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 0x559747b831e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.351+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/4008095062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7ac70 0x559747b82ca0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.351+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/4008095062 >> 172.21.15.123:0/4008095062 conn(0x559747b77200 msgr2=0x559747b7cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:48.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.351+0000 7f9d43400640 0 client.16947 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:54:48.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.351+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/4008095062 shutdown_connections 2024-09-19T04:54:48.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.352+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/4008095062 wait complete. 2024-09-19T04:54:48.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.352+0000 7f9d6b3e0740 20 client.16947 trim_cache size 0 max 16384 2024-09-19T04:54:48.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.365+0000 7f9d6b3e0740 1 Processor -- start 2024-09-19T04:54:48.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.365+0000 7f9d6b3e0740 1 -- start start 2024-09-19T04:54:48.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.365+0000 7f9d6b3e0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b7ac70 0x559747b7d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.365+0000 7f9d6b3e0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 0x559747b9da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.365+0000 7f9d6b3e0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b85e50 0x559747b9dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.365+0000 7f9d6b3e0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641af980 con 0x559747b85e50 2024-09-19T04:54:48.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.365+0000 7f9d6b3e0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641b14c0 con 0x559747b7b590 2024-09-19T04:54:48.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.365+0000 7f9d6b3e0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641b0050 con 0x559747b7ac70 2024-09-19T04:54:48.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.366+0000 7f9d61400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 0x559747b9da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.366+0000 7f9d62800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b85e50 0x559747b9dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.366+0000 7f9d61400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 0x559747b9da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:53296/0 (socket says 172.21.15.123:53296) 2024-09-19T04:54:48.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.366+0000 7f9d61400640 1 -- 172.21.15.123:0/158678145 learned_addr learned my addr 172.21.15.123:0/158678145 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:48.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.366+0000 7f9d61e00640 1 --2- 172.21.15.123:0/158678145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b7ac70 0x559747b7d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.366+0000 7f9d61400640 1 -- 172.21.15.123:0/158678145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b7ac70 msgr2=0x559747b7d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:48.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.366+0000 7f9d61400640 1 --2- 172.21.15.123:0/158678145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b7ac70 0x559747b7d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.366+0000 7f9d61400640 1 -- 172.21.15.123:0/158678145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b85e50 msgr2=0x559747b9dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:48.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.366+0000 7f9d61400640 1 --2- 172.21.15.123:0/158678145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b85e50 0x559747b9dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.366+0000 7f9d61400640 1 -- 172.21.15.123:0/158678145 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559747b77030 con 0x559747b7b590 2024-09-19T04:54:48.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.366+0000 7f9d61400640 1 --2- 172.21.15.123:0/158678145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 0x559747b9da60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9d5009c2f0 tx=0x7f9d500bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6df4436e471e80d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:48.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.367+0000 7f9d4e600640 1 -- 172.21.15.123:0/158678145 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d500a4fd0 con 0x559747b7b590 2024-09-19T04:54:48.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.367+0000 7f9d4e600640 1 -- 172.21.15.123:0/158678145 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d5009a660 con 0x559747b7b590 2024-09-19T04:54:48.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.367+0000 7f9d4e600640 1 -- 172.21.15.123:0/158678145 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5009ba10 con 0x559747b7b590 2024-09-19T04:54:48.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.367+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/158678145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 msgr2=0x559747b9da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:48.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.367+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/158678145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 0x559747b9da60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9d5009c2f0 tx=0x7f9d500bc040 comp rx=0 tx=0).stop 2024-09-19T04:54:48.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.367+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/158678145 shutdown_connections 2024-09-19T04:54:48.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.367+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/158678145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b85e50 0x559747b9dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.367+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/158678145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7b590 0x559747b9da60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.367+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/158678145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b7ac70 0x559747b7d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.367+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/158678145 >> 172.21.15.123:0/158678145 conn(0x559747b77200 msgr2=0x559747b84d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:48.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.367+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/158678145 shutdown_connections 2024-09-19T04:54:48.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.367+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/158678145 wait complete. 2024-09-19T04:54:48.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.368+0000 7f9d6b3e0740 1 -- start start 2024-09-19T04:54:48.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.368+0000 7f9d6b3e0740 1 Processor -- start 2024-09-19T04:54:48.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.368+0000 7f9d6b3e0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7ac70 0x559747b899a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.368+0000 7f9d6b3e0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7b590 0x559747b89ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.368+0000 7f9d6b3e0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b85e50 0x559747b8a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.368+0000 7f9d6b3e0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641b0df0 con 0x559747b7b590 2024-09-19T04:54:48.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.368+0000 7f9d6b3e0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641b0720 con 0x559747b7ac70 2024-09-19T04:54:48.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.368+0000 7f9d6b3e0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641ad980 con 0x559747b85e50 2024-09-19T04:54:48.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.369+0000 7f9d6b3e0740 10 client.0 _ll_register_callbacks cb 0x7f9d54009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:48.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.369+0000 7f9d61400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7b590 0x559747b89ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.369+0000 7f9d62800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b85e50 0x559747b8a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.369+0000 7f9d61400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7b590 0x559747b89ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53778/0 (socket says 172.21.15.123:53778) 2024-09-19T04:54:48.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.369+0000 7f9d61400640 1 -- 172.21.15.123:0/643301693 learned_addr learned my addr 172.21.15.123:0/643301693 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:48.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.369+0000 7f9d61e00640 1 --2- 172.21.15.123:0/643301693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7ac70 0x559747b899a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.369+0000 7f9d61400640 1 -- 172.21.15.123:0/643301693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b85e50 msgr2=0x559747b8a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:48.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.369+0000 7f9d61400640 1 --2- 172.21.15.123:0/643301693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b85e50 0x559747b8a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.369+0000 7f9d61400640 1 -- 172.21.15.123:0/643301693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7ac70 msgr2=0x559747b899a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:48.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.369+0000 7f9d61400640 1 --2- 172.21.15.123:0/643301693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7ac70 0x559747b899a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.369+0000 7f9d61400640 1 -- 172.21.15.123:0/643301693 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559747b782b0 con 0x559747b7b590 2024-09-19T04:54:48.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.369+0000 7f9d61400640 1 --2- 172.21.15.123:0/643301693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7b590 0x559747b89ee0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9d5009c060 tx=0x7f9d5009a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66836d92e311553e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:48.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.369+0000 7f9d43400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:48.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.370+0000 7f9d43400640 1 -- 172.21.15.123:0/643301693 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d500cdc60 con 0x559747b7b590 2024-09-19T04:54:48.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.370+0000 7f9d43400640 1 -- 172.21.15.123:0/643301693 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f9d500d20e0 con 0x559747b7b590 2024-09-19T04:54:48.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.370+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/643301693 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d5009a660 con 0x559747b7b590 2024-09-19T04:54:48.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.370+0000 7f9d43400640 1 -- 172.21.15.123:0/643301693 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9d500a4fd0 con 0x559747b7b590 2024-09-19T04:54:48.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.370+0000 7f9d6b3e0740 10 client.16953 fetch_fsmap learned FSMap version 43 2024-09-19T04:54:48.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.370+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/643301693 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559747b72770 con 0x559747b7b590 2024-09-19T04:54:48.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.370+0000 7f9d43400640 1 -- 172.21.15.123:0/643301693 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d5009a660 con 0x559747b7b590 2024-09-19T04:54:48.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.370+0000 7f9d43400640 1 -- 172.21.15.123:0/643301693 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f9d50018cc0 con 0x559747b7b590 2024-09-19T04:54:48.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.370+0000 7f9d6b3e0740 10 client.16953 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:54:48.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.370+0000 7f9d6b3e0740 10 client.16953 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:54:48.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.370+0000 7f9d6b3e0740 20 client.16953 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:48.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.370+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/643301693 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x559747c010f0 0x559747c03510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:48.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.370+0000 7f9d6b3e0740 4 client.16953 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:48.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.370+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/643301693 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559747ae8e60 con 0x559747c010f0 2024-09-19T04:54:48.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.371+0000 7f9d61e00640 1 --2- 172.21.15.123:0/643301693 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x559747c010f0 0x559747c03510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:48.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.371+0000 7f9d61e00640 1 --2- 172.21.15.123:0/643301693 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x559747c010f0 0x559747c03510 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc43b8c93325a71d server_cookie=56b7bc78168f423c in_seq=0 out_seq=0 2024-09-19T04:54:48.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.371+0000 7f9d43400640 10 client.16953 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:54:48.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.371+0000 7f9d43400640 1 -- 172.21.15.123:0/643301693 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559747ae8e60 con 0x559747c010f0 2024-09-19T04:54:48.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.371+0000 7f9d43400640 10 client.16953 handle_command_reply: tid=0 2024-09-19T04:54:48.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.372+0000 7f9d6b3e0740 1 client.16953 shutdown 2024-09-19T04:54:48.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.372+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/643301693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7b590 msgr2=0x559747b89ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:48.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.372+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/643301693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7b590 0x559747b89ee0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9d5009c060 tx=0x7f9d5009a810 comp rx=0 tx=0).stop 2024-09-19T04:54:48.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.372+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/643301693 shutdown_connections 2024-09-19T04:54:48.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.372+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/643301693 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x559747c010f0 0x559747c03510 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.372+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/643301693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559747b85e50 0x559747b8a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.372+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/643301693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559747b7b590 0x559747b89ee0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.372+0000 7f9d6b3e0740 1 --2- 172.21.15.123:0/643301693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559747b7ac70 0x559747b899a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.372+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/643301693 >> 172.21.15.123:0/643301693 conn(0x559747b77200 msgr2=0x559747b75b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:48.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.372+0000 7f9d43400640 0 client.16953 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:54:48.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.372+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/643301693 shutdown_connections 2024-09-19T04:54:48.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.372+0000 7f9d6b3e0740 1 -- 172.21.15.123:0/643301693 wait complete. 2024-09-19T04:54:48.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.372+0000 7f9d6b3e0740 20 client.16953 trim_cache size 0 max 16384 2024-09-19T04:54:48.378 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:48.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.373+0000 7f9d68a00640 1 -- 172.21.15.123:0/2174959118 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9d480a3430 msgr2=0x7f9d480a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:48.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.373+0000 7f9d68a00640 1 --2- 172.21.15.123:0/2174959118 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9d480a3430 0x7f9d480a58f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x559747b78060 tx=0x7f9d54005f90 comp rx=0 tx=0).stop 2024-09-19T04:54:48.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.373+0000 7f9d68a00640 1 -- 172.21.15.123:0/2174959118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d641049a0 msgr2=0x7f9d6419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:48.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.373+0000 7f9d68a00640 1 --2- 172.21.15.123:0/2174959118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d641049a0 0x7f9d6419ab00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000e9f0 tx=0x7f9d5000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.373+0000 7f9d68a00640 1 -- 172.21.15.123:0/2174959118 shutdown_connections 2024-09-19T04:54:48.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.373+0000 7f9d68a00640 1 --2- 172.21.15.123:0/2174959118 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9d480a3430 0x7f9d480a58f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.373+0000 7f9d68a00640 1 --2- 172.21.15.123:0/2174959118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d64109120 0x7f9d6419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.373+0000 7f9d68a00640 1 --2- 172.21.15.123:0/2174959118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d641049a0 0x7f9d6419ab00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.373+0000 7f9d68a00640 1 --2- 172.21.15.123:0/2174959118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d64103ff0 0x7f9d6419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:48.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.373+0000 7f9d68a00640 1 -- 172.21.15.123:0/2174959118 >> 172.21.15.123:0/2174959118 conn(0x7f9d640ffa90 msgr2=0x7f9d641005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:48.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.373+0000 7f9d68a00640 1 -- 172.21.15.123:0/2174959118 shutdown_connections 2024-09-19T04:54:48.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:48.373+0000 7f9d68a00640 1 -- 172.21.15.123:0/2174959118 wait complete. 2024-09-19T04:54:49.198 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:54:49.567 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:49.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:49 smithi183 ceph-mon[35333]: pgmap v250: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:49.864 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:49 smithi123 ceph-mon[30511]: pgmap v250: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:49.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:49 smithi193 ceph-mon[35592]: pgmap v250: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:50.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.124+0000 7fbffb000640 1 -- 172.21.15.123:0/2439858570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbff4102790 msgr2=0x7fbff41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:50.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.124+0000 7fbffb000640 1 --2- 172.21.15.123:0/2439858570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbff4102790 0x7fbff41067e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009900 tx=0x7fbfdc02a410 comp rx=0 tx=0).stop 2024-09-19T04:54:50.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.125+0000 7fbffb000640 1 -- 172.21.15.123:0/2439858570 shutdown_connections 2024-09-19T04:54:50.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.125+0000 7fbffb000640 1 --2- 172.21.15.123:0/2439858570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbff4106f10 0x7fbff410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.125+0000 7fbffb000640 1 --2- 172.21.15.123:0/2439858570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbff4102790 0x7fbff41067e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.125+0000 7fbffb000640 1 --2- 172.21.15.123:0/2439858570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbff4101de0 0x7fbff41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.125+0000 7fbffb000640 1 -- 172.21.15.123:0/2439858570 >> 172.21.15.123:0/2439858570 conn(0x7fbff40fd8c0 msgr2=0x7fbff40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:50.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.125+0000 7fbffb000640 1 -- 172.21.15.123:0/2439858570 shutdown_connections 2024-09-19T04:54:50.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.125+0000 7fbffb000640 1 -- 172.21.15.123:0/2439858570 wait complete. 2024-09-19T04:54:50.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.126+0000 7fbffb000640 1 Processor -- start 2024-09-19T04:54:50.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.126+0000 7fbffb000640 1 -- start start 2024-09-19T04:54:50.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.126+0000 7fbffb000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbff4101de0 0x7fbff419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.126+0000 7fbffb000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbff4102790 0x7fbff419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.126+0000 7fbffb000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbff4106f10 0x7fbff419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.126+0000 7fbffb000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbff4111fc0 con 0x7fbff4106f10 2024-09-19T04:54:50.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.126+0000 7fbffb000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbff4111e40 con 0x7fbff4101de0 2024-09-19T04:54:50.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.126+0000 7fbffb000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbff4112140 con 0x7fbff4102790 2024-09-19T04:54:50.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.126+0000 7fbff3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbff4101de0 0x7fbff419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.126+0000 7fbff3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbff4101de0 0x7fbff419a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:53330/0 (socket says 172.21.15.123:53330) 2024-09-19T04:54:50.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.127+0000 7fbff3e00640 1 -- 172.21.15.123:0/1253066357 learned_addr learned my addr 172.21.15.123:0/1253066357 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:50.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.127+0000 7fbfeb400640 1 --2- 172.21.15.123:0/1253066357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbff4102790 0x7fbff419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.127+0000 7fbff9000640 1 --2- 172.21.15.123:0/1253066357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbff4106f10 0x7fbff419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.127+0000 7fbff3e00640 1 -- 172.21.15.123:0/1253066357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbff4102790 msgr2=0x7fbff419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:50.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.127+0000 7fbff3e00640 1 --2- 172.21.15.123:0/1253066357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbff4102790 0x7fbff419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.127+0000 7fbff3e00640 1 -- 172.21.15.123:0/1253066357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbff4106f10 msgr2=0x7fbff419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:50.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.127+0000 7fbff3e00640 1 --2- 172.21.15.123:0/1253066357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbff4106f10 0x7fbff419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.127+0000 7fbff3e00640 1 -- 172.21.15.123:0/1253066357 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff419f630 con 0x7fbff4101de0 2024-09-19T04:54:50.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.127+0000 7fbff9000640 1 --2- 172.21.15.123:0/1253066357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbff4106f10 0x7fbff419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:50.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.127+0000 7fbff3e00640 1 --2- 172.21.15.123:0/1253066357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbff4101de0 0x7fbff419a680 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400e9f0 tx=0x7fbfe400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:50.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.128+0000 7fbff1600640 1 -- 172.21.15.123:0/1253066357 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe400cd80 con 0x7fbff4101de0 2024-09-19T04:54:50.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.128+0000 7fbffb000640 1 -- 172.21.15.123:0/1253066357 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff419f920 con 0x7fbff4101de0 2024-09-19T04:54:50.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.128+0000 7fbffb000640 1 -- 172.21.15.123:0/1253066357 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff41a7200 con 0x7fbff4101de0 2024-09-19T04:54:50.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.128+0000 7fbff1600640 1 -- 172.21.15.123:0/1253066357 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe4005500 con 0x7fbff4101de0 2024-09-19T04:54:50.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.128+0000 7fbff1600640 1 -- 172.21.15.123:0/1253066357 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4010690 con 0x7fbff4101de0 2024-09-19T04:54:50.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.131+0000 7fbffda41740 1 Processor -- start 2024-09-19T04:54:50.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.131+0000 7fbffda41740 1 -- start start 2024-09-19T04:54:50.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.131+0000 7fbffda41740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90ec70 0x559fef90f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.131+0000 7fbffda41740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90f590 0x559fef919910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.131+0000 7fbffda41740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 0x559fef91c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.131+0000 7fbff1600640 1 -- 172.21.15.123:0/1253066357 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fbfe40046c0 con 0x7fbff4101de0 2024-09-19T04:54:50.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.131+0000 7fbffda41740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559fef8753e0 con 0x559fef90f590 2024-09-19T04:54:50.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.131+0000 7fbffda41740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559fef81cd80 con 0x559fef919e50 2024-09-19T04:54:50.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.131+0000 7fbffda41740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559fef8713a0 con 0x559fef90ec70 2024-09-19T04:54:50.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.132+0000 7fbff3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90f590 0x559fef919910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.132+0000 7fbff9000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90ec70 0x559fef90f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.132+0000 7fbff3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90f590 0x559fef919910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53796/0 (socket says 172.21.15.123:53796) 2024-09-19T04:54:50.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.132+0000 7fbff3e00640 1 -- 172.21.15.123:0/1838810016 learned_addr learned my addr 172.21.15.123:0/1838810016 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:50.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.132+0000 7fbfeb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 0x559fef91c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.132+0000 7fbff9000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90ec70 0x559fef90f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42762/0 (socket says 172.21.15.123:42762) 2024-09-19T04:54:50.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.132+0000 7fbff9000640 1 -- 172.21.15.123:0/1838810016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 msgr2=0x559fef91c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:50.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.132+0000 7fbff9000640 1 --2- 172.21.15.123:0/1838810016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 0x559fef91c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.132+0000 7fbff9000640 1 -- 172.21.15.123:0/1838810016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90f590 msgr2=0x559fef919910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:50.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.132+0000 7fbff9000640 1 --2- 172.21.15.123:0/1838810016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90f590 0x559fef919910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.132+0000 7fbff9000640 1 -- 172.21.15.123:0/1838810016 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fef83bae0 con 0x559fef90ec70 2024-09-19T04:54:50.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.133+0000 7fbff9000640 1 --2- 172.21.15.123:0/1838810016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90ec70 0x559fef90f050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbfec010770 tx=0x7fbfec010c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b327b6591bef62a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:50.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.133+0000 7fbfe9600640 1 -- 172.21.15.123:0/1838810016 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec003970 con 0x559fef90ec70 2024-09-19T04:54:50.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.133+0000 7fbff1600640 1 --2- 172.21.15.123:0/1253066357 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbfd80a3430 0x7fbfd80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.133+0000 7fbfe9600640 1 -- 172.21.15.123:0/1838810016 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfec00e930 con 0x559fef90ec70 2024-09-19T04:54:50.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.133+0000 7fbfe9600640 1 -- 172.21.15.123:0/1838810016 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0045f0 con 0x559fef90ec70 2024-09-19T04:54:50.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.133+0000 7fbff1600640 1 -- 172.21.15.123:0/1253066357 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fbfe40d00d0 con 0x7fbff4101de0 2024-09-19T04:54:50.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.133+0000 7fbff3e00640 1 --2- 172.21.15.123:0/1253066357 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbfd80a3430 0x7fbfd80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.134+0000 7fbffda41740 1 -- 172.21.15.123:0/1838810016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90ec70 msgr2=0x559fef90f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:50.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.134+0000 7fbffda41740 1 --2- 172.21.15.123:0/1838810016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90ec70 0x559fef90f050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbfec010770 tx=0x7fbfec010c30 comp rx=0 tx=0).stop 2024-09-19T04:54:50.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.134+0000 7fbff3e00640 1 --2- 172.21.15.123:0/1253066357 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbfd80a3430 0x7fbfd80a58f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbfe409c050 tx=0x7fbfe40bc040 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:50.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.134+0000 7fbffda41740 1 -- 172.21.15.123:0/1838810016 shutdown_connections 2024-09-19T04:54:50.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.134+0000 7fbffda41740 1 --2- 172.21.15.123:0/1838810016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 0x559fef91c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.134+0000 7fbffda41740 1 --2- 172.21.15.123:0/1838810016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90f590 0x559fef919910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.134+0000 7fbffda41740 1 --2- 172.21.15.123:0/1838810016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90ec70 0x559fef90f050 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.134+0000 7fbffda41740 1 -- 172.21.15.123:0/1838810016 >> 172.21.15.123:0/1838810016 conn(0x559fef90b200 msgr2=0x559fef90b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:50.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.134+0000 7fbffda41740 1 -- 172.21.15.123:0/1838810016 shutdown_connections 2024-09-19T04:54:50.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.134+0000 7fbffda41740 1 -- 172.21.15.123:0/1838810016 wait complete. 2024-09-19T04:54:50.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.135+0000 7fbffda41740 1 -- start start 2024-09-19T04:54:50.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.135+0000 7fbffda41740 1 Processor -- start 2024-09-19T04:54:50.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.135+0000 7fbffda41740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90ec70 0x559fef916d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.135+0000 7fbffda41740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90f590 0x559fef917250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.135+0000 7fbffda41740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 0x559fef917790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.135+0000 7fbffda41740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559fef91ca20 con 0x559fef90ec70 2024-09-19T04:54:50.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.135+0000 7fbffda41740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559fef8300f0 con 0x559fef919e50 2024-09-19T04:54:50.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.135+0000 7fbffda41740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559fef91cba0 con 0x559fef90f590 2024-09-19T04:54:50.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.135+0000 7fbffda41740 10 client.0 _ll_register_callbacks cb 0x559fef817bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:50.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.135+0000 7fbfeb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90f590 0x559fef917250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.136+0000 7fbfeb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90f590 0x559fef917250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:42774/0 (socket says 172.21.15.123:42774) 2024-09-19T04:54:50.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.136+0000 7fbfeb400640 1 -- 172.21.15.123:0/858958801 learned_addr learned my addr 172.21.15.123:0/858958801 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:50.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.135+0000 7fbff9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90ec70 0x559fef916d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.136+0000 7fbff9000640 1 --2- 172.21.15.123:0/858958801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 0x559fef917790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.136+0000 7fbfeb400640 1 -- 172.21.15.123:0/858958801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 msgr2=0x559fef917790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:50.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.136+0000 7fbfeb400640 1 --2- 172.21.15.123:0/858958801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 0x559fef917790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.136+0000 7fbfeb400640 1 -- 172.21.15.123:0/858958801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90ec70 msgr2=0x559fef916d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:50.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.136+0000 7fbfeb400640 1 --2- 172.21.15.123:0/858958801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90ec70 0x559fef916d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.136+0000 7fbfeb400640 1 -- 172.21.15.123:0/858958801 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fef5e63c0 con 0x559fef90f590 2024-09-19T04:54:50.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.136+0000 7fbfeb400640 1 --2- 172.21.15.123:0/858958801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90f590 0x559fef917250 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009900 tx=0x7fbfdc004420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=645177f7aff20330 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:50.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.136+0000 7fbfe2000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:54:50.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.137+0000 7fbfe2000640 1 -- 172.21.15.123:0/858958801 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc035040 con 0x559fef90f590 2024-09-19T04:54:50.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.137+0000 7fbffda41740 1 -- 172.21.15.123:0/858958801 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfec00e930 con 0x559fef90f590 2024-09-19T04:54:50.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.137+0000 7fbfe2000640 1 -- 172.21.15.123:0/858958801 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fbfdc005450 con 0x559fef90f590 2024-09-19T04:54:50.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.137+0000 7fbfe2000640 1 -- 172.21.15.123:0/858958801 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbfdc005650 con 0x559fef90f590 2024-09-19T04:54:50.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.137+0000 7fbffda41740 10 client.27079 fetch_fsmap learned FSMap version 43 2024-09-19T04:54:50.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.137+0000 7fbffda41740 1 -- 172.21.15.123:0/858958801 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559fef928ab0 con 0x559fef90f590 2024-09-19T04:54:50.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.137+0000 7fbfe2000640 1 -- 172.21.15.123:0/858958801 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfec00e930 con 0x559fef90f590 2024-09-19T04:54:50.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.138+0000 7fbfe2000640 1 -- 172.21.15.123:0/858958801 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fbfdc003b50 con 0x559fef90f590 2024-09-19T04:54:50.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.138+0000 7fbffda41740 10 client.27079 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:54:50.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.138+0000 7fbffda41740 10 client.27079 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:54:50.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.138+0000 7fbffda41740 20 client.27079 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:50.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.138+0000 7fbffda41740 1 --2- 172.21.15.123:0/858958801 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x559fef929ec0 0x559fef92c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.138+0000 7fbffda41740 4 client.27079 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:50.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.138+0000 7fbffda41740 1 -- 172.21.15.123:0/858958801 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fef882950 con 0x559fef929ec0 2024-09-19T04:54:50.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.138+0000 7fbff9000640 1 --2- 172.21.15.123:0/858958801 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x559fef929ec0 0x559fef92c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.138+0000 7fbff9000640 1 --2- 172.21.15.123:0/858958801 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x559fef929ec0 0x559fef92c2d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc17f39a20984ee2 server_cookie=2f64d22c4ab23a47 in_seq=0 out_seq=0 2024-09-19T04:54:50.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.138+0000 7fbfe2000640 10 client.27079 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:54:50.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.139+0000 7fbfe2000640 1 -- 172.21.15.123:0/858958801 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559fef882950 con 0x559fef929ec0 2024-09-19T04:54:50.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.139+0000 7fbfe2000640 10 client.27079 handle_command_reply: tid=0 2024-09-19T04:54:50.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.139+0000 7fbffda41740 1 client.27079 shutdown 2024-09-19T04:54:50.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.139+0000 7fbffda41740 1 -- 172.21.15.123:0/858958801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90f590 msgr2=0x559fef917250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:50.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.139+0000 7fbffda41740 1 --2- 172.21.15.123:0/858958801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90f590 0x559fef917250 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009900 tx=0x7fbfdc004420 comp rx=0 tx=0).stop 2024-09-19T04:54:50.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.140+0000 7fbffda41740 1 -- 172.21.15.123:0/858958801 shutdown_connections 2024-09-19T04:54:50.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.140+0000 7fbffda41740 1 --2- 172.21.15.123:0/858958801 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x559fef929ec0 0x559fef92c2d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.140+0000 7fbffda41740 1 --2- 172.21.15.123:0/858958801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 0x559fef917790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.140+0000 7fbffda41740 1 --2- 172.21.15.123:0/858958801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90f590 0x559fef917250 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.140+0000 7fbfe2000640 0 client.27079 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:54:50.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.140+0000 7fbffda41740 1 --2- 172.21.15.123:0/858958801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90ec70 0x559fef916d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.140+0000 7fbffda41740 1 -- 172.21.15.123:0/858958801 >> 172.21.15.123:0/858958801 conn(0x559fef90b200 msgr2=0x559fef90a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:50.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.140+0000 7fbffda41740 1 -- 172.21.15.123:0/858958801 shutdown_connections 2024-09-19T04:54:50.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.140+0000 7fbffda41740 1 -- 172.21.15.123:0/858958801 wait complete. 2024-09-19T04:54:50.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.140+0000 7fbffda41740 20 client.27079 trim_cache size 0 max 16384 2024-09-19T04:54:50.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.153+0000 7fbffda41740 1 Processor -- start 2024-09-19T04:54:50.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.153+0000 7fbffda41740 1 -- start start 2024-09-19T04:54:50.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.154+0000 7fbffda41740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90ec70 0x559fef90b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.154+0000 7fbffda41740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90f590 0x559fef931ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.154+0000 7fbffda41740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 0x559fef9320e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.154+0000 7fbffda41740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbff41afac0 con 0x559fef90f590 2024-09-19T04:54:50.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.154+0000 7fbffda41740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbff41b1600 con 0x559fef919e50 2024-09-19T04:54:50.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.154+0000 7fbffda41740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbff41b0190 con 0x559fef90ec70 2024-09-19T04:54:50.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.154+0000 7fbfeb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90f590 0x559fef931ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.154+0000 7fbff9000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90ec70 0x559fef90b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.154+0000 7fbfeb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90f590 0x559fef931ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53810/0 (socket says 172.21.15.123:53810) 2024-09-19T04:54:50.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.154+0000 7fbfeb400640 1 -- 172.21.15.123:0/2299211497 learned_addr learned my addr 172.21.15.123:0/2299211497 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:50.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.154+0000 7fbff9000640 1 --2- 172.21.15.123:0/2299211497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 0x559fef9320e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.154+0000 7fbfeb400640 1 -- 172.21.15.123:0/2299211497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90ec70 msgr2=0x559fef90b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:50.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.154+0000 7fbfeb400640 1 --2- 172.21.15.123:0/2299211497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90ec70 0x559fef90b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.154+0000 7fbfeb400640 1 -- 172.21.15.123:0/2299211497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 msgr2=0x559fef9320e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:50.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.154+0000 7fbfeb400640 1 --2- 172.21.15.123:0/2299211497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 0x559fef9320e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.154+0000 7fbfeb400640 1 -- 172.21.15.123:0/2299211497 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fef915cc0 con 0x559fef90f590 2024-09-19T04:54:50.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.154+0000 7fbff9000640 1 --2- 172.21.15.123:0/2299211497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90ec70 0x559fef90b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:50.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.155+0000 7fbfeb400640 1 --2- 172.21.15.123:0/2299211497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90f590 0x559fef931ba0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc031720 tx=0x7fbfdc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da3d0d62538e408 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:50.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.155+0000 7fbfe8c00640 1 -- 172.21.15.123:0/2299211497 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc035070 con 0x559fef90f590 2024-09-19T04:54:50.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.155+0000 7fbfe8c00640 1 -- 172.21.15.123:0/2299211497 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfdc041070 con 0x559fef90f590 2024-09-19T04:54:50.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.155+0000 7fbfe8c00640 1 -- 172.21.15.123:0/2299211497 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc053070 con 0x559fef90f590 2024-09-19T04:54:50.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.156+0000 7fbffda41740 1 -- 172.21.15.123:0/2299211497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90f590 msgr2=0x559fef931ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:50.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.156+0000 7fbffda41740 1 --2- 172.21.15.123:0/2299211497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90f590 0x559fef931ba0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc031720 tx=0x7fbfdc02a410 comp rx=0 tx=0).stop 2024-09-19T04:54:50.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.156+0000 7fbffda41740 1 -- 172.21.15.123:0/2299211497 shutdown_connections 2024-09-19T04:54:50.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.156+0000 7fbffda41740 1 --2- 172.21.15.123:0/2299211497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 0x559fef9320e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.156+0000 7fbffda41740 1 --2- 172.21.15.123:0/2299211497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90f590 0x559fef931ba0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.156+0000 7fbffda41740 1 --2- 172.21.15.123:0/2299211497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90ec70 0x559fef90b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.156+0000 7fbffda41740 1 -- 172.21.15.123:0/2299211497 >> 172.21.15.123:0/2299211497 conn(0x559fef90b200 msgr2=0x559fef910f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:50.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.156+0000 7fbffda41740 1 -- 172.21.15.123:0/2299211497 shutdown_connections 2024-09-19T04:54:50.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.156+0000 7fbffda41740 1 -- 172.21.15.123:0/2299211497 wait complete. 2024-09-19T04:54:50.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.156+0000 7fbffda41740 1 -- start start 2024-09-19T04:54:50.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.156+0000 7fbffda41740 1 Processor -- start 2024-09-19T04:54:50.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.157+0000 7fbffda41740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90ec70 0x559fef91df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.157+0000 7fbffda41740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90f590 0x559fef91e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.157+0000 7fbffda41740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 0x559fef99c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.157+0000 7fbffda41740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbff41b0f30 con 0x559fef90ec70 2024-09-19T04:54:50.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.157+0000 7fbffda41740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbff41b0860 con 0x559fef919e50 2024-09-19T04:54:50.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.157+0000 7fbffda41740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbff41adac0 con 0x559fef90f590 2024-09-19T04:54:50.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.157+0000 7fbffda41740 10 client.0 _ll_register_callbacks cb 0x7fbfdc009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:50.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.157+0000 7fbff9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90ec70 0x559fef91df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.157+0000 7fbff9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90ec70 0x559fef91df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53816/0 (socket says 172.21.15.123:53816) 2024-09-19T04:54:50.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.157+0000 7fbff9000640 1 -- 172.21.15.123:0/1542738750 learned_addr learned my addr 172.21.15.123:0/1542738750 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:50.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.157+0000 7fbfeb400640 1 --2- 172.21.15.123:0/1542738750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90f590 0x559fef91e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.158+0000 7fbff9000640 1 --2- 172.21.15.123:0/1542738750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 0x559fef99c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.158+0000 7fbff9000640 1 -- 172.21.15.123:0/1542738750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90f590 msgr2=0x559fef91e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:50.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.158+0000 7fbff9000640 1 --2- 172.21.15.123:0/1542738750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90f590 0x559fef91e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.158+0000 7fbff9000640 1 -- 172.21.15.123:0/1542738750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 msgr2=0x559fef99c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:50.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.158+0000 7fbff9000640 1 --2- 172.21.15.123:0/1542738750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 0x559fef99c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.158+0000 7fbff9000640 1 -- 172.21.15.123:0/1542738750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fef99c8c0 con 0x559fef90ec70 2024-09-19T04:54:50.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.158+0000 7fbff9000640 1 --2- 172.21.15.123:0/1542738750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90ec70 0x559fef91df70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbfec010c70 tx=0x7fbfdc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557e6bc367b18f4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:50.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.158+0000 7fbfe2000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:50.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.158+0000 7fbfe2000640 1 -- 172.21.15.123:0/1542738750 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec025070 con 0x559fef90ec70 2024-09-19T04:54:50.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.158+0000 7fbfe2000640 1 -- 172.21.15.123:0/1542738750 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fbfec005580 con 0x559fef90ec70 2024-09-19T04:54:50.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.158+0000 7fbffda41740 1 -- 172.21.15.123:0/1542738750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfdc041070 con 0x559fef90ec70 2024-09-19T04:54:50.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.159+0000 7fbfe2000640 1 -- 172.21.15.123:0/1542738750 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbfec03f050 con 0x559fef90ec70 2024-09-19T04:54:50.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.159+0000 7fbffda41740 10 client.16971 fetch_fsmap learned FSMap version 43 2024-09-19T04:54:50.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.159+0000 7fbffda41740 1 -- 172.21.15.123:0/1542738750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559fef99da60 con 0x559fef90ec70 2024-09-19T04:54:50.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.159+0000 7fbfe2000640 1 -- 172.21.15.123:0/1542738750 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfdc041070 con 0x559fef90ec70 2024-09-19T04:54:50.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.159+0000 7fbfe2000640 1 -- 172.21.15.123:0/1542738750 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fbfec02e510 con 0x559fef90ec70 2024-09-19T04:54:50.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.159+0000 7fbffda41740 10 client.16971 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:54:50.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.159+0000 7fbffda41740 10 client.16971 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:54:50.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.159+0000 7fbffda41740 20 client.16971 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:50.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.159+0000 7fbffda41740 1 --2- 172.21.15.123:0/1542738750 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x559fef99dfa0 0x559fef9a0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:50.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.159+0000 7fbffda41740 4 client.16971 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:50.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.159+0000 7fbffda41740 1 -- 172.21.15.123:0/1542738750 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559fef87ce60 con 0x559fef99dfa0 2024-09-19T04:54:50.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.160+0000 7fbfeb400640 1 --2- 172.21.15.123:0/1542738750 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x559fef99dfa0 0x559fef9a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:50.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.160+0000 7fbfeb400640 1 --2- 172.21.15.123:0/1542738750 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x559fef99dfa0 0x559fef9a0390 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7ea584d74f33dd server_cookie=1c02da578e8e422d in_seq=0 out_seq=0 2024-09-19T04:54:50.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.160+0000 7fbfe2000640 10 client.16971 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:54:50.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.160+0000 7fbfe2000640 1 -- 172.21.15.123:0/1542738750 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559fef87ce60 con 0x559fef99dfa0 2024-09-19T04:54:50.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.160+0000 7fbfe2000640 10 client.16971 handle_command_reply: tid=0 2024-09-19T04:54:50.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.160+0000 7fbffda41740 1 client.16971 shutdown 2024-09-19T04:54:50.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.160+0000 7fbffda41740 1 -- 172.21.15.123:0/1542738750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90ec70 msgr2=0x559fef91df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:50.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.160+0000 7fbffda41740 1 --2- 172.21.15.123:0/1542738750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90ec70 0x559fef91df70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbfec010c70 tx=0x7fbfdc004040 comp rx=0 tx=0).stop 2024-09-19T04:54:50.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.161+0000 7fbffda41740 1 -- 172.21.15.123:0/1542738750 shutdown_connections 2024-09-19T04:54:50.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.161+0000 7fbffda41740 1 --2- 172.21.15.123:0/1542738750 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x559fef99dfa0 0x559fef9a0390 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.161+0000 7fbffda41740 1 --2- 172.21.15.123:0/1542738750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fef919e50 0x559fef99c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.161+0000 7fbffda41740 1 --2- 172.21.15.123:0/1542738750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fef90f590 0x559fef91e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.161+0000 7fbffda41740 1 --2- 172.21.15.123:0/1542738750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fef90ec70 0x559fef91df70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.161+0000 7fbffda41740 1 -- 172.21.15.123:0/1542738750 >> 172.21.15.123:0/1542738750 conn(0x559fef90b200 msgr2=0x559fef918570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:50.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.161+0000 7fbfe2000640 0 client.16971 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:54:50.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.161+0000 7fbffda41740 1 -- 172.21.15.123:0/1542738750 shutdown_connections 2024-09-19T04:54:50.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.161+0000 7fbffda41740 1 -- 172.21.15.123:0/1542738750 wait complete. 2024-09-19T04:54:50.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.161+0000 7fbffda41740 20 client.16971 trim_cache size 0 max 16384 2024-09-19T04:54:50.167 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:50.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.162+0000 7fbffb000640 1 -- 172.21.15.123:0/1253066357 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbfd80a3430 msgr2=0x7fbfd80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:50.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.162+0000 7fbffb000640 1 --2- 172.21.15.123:0/1253066357 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbfd80a3430 0x7fbfd80a58f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbfe409c050 tx=0x7fbfe40bc040 comp rx=0 tx=0).stop 2024-09-19T04:54:50.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.162+0000 7fbffb000640 1 -- 172.21.15.123:0/1253066357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbff4101de0 msgr2=0x7fbff419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:50.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.162+0000 7fbffb000640 1 --2- 172.21.15.123:0/1253066357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbff4101de0 0x7fbff419a680 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400e9f0 tx=0x7fbfe400edf0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.162+0000 7fbffb000640 1 -- 172.21.15.123:0/1253066357 shutdown_connections 2024-09-19T04:54:50.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.162+0000 7fbffb000640 1 --2- 172.21.15.123:0/1253066357 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fbfd80a3430 0x7fbfd80a58f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.162+0000 7fbffb000640 1 --2- 172.21.15.123:0/1253066357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbff4106f10 0x7fbff419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.162+0000 7fbffb000640 1 --2- 172.21.15.123:0/1253066357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbff4102790 0x7fbff419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.162+0000 7fbffb000640 1 --2- 172.21.15.123:0/1253066357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbff4101de0 0x7fbff419a680 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:50.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.162+0000 7fbffb000640 1 -- 172.21.15.123:0/1253066357 >> 172.21.15.123:0/1253066357 conn(0x7fbff40fd8c0 msgr2=0x7fbff40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:50.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.162+0000 7fbffb000640 1 -- 172.21.15.123:0/1253066357 shutdown_connections 2024-09-19T04:54:50.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:50.162+0000 7fbffb000640 1 -- 172.21.15.123:0/1253066357 wait complete. 2024-09-19T04:54:51.062 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:54:51.428 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:51.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:51 smithi183 ceph-mon[35333]: pgmap v251: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:51.871 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:51 smithi123 ceph-mon[30511]: pgmap v251: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:51.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.982+0000 7fc459e00640 1 -- 172.21.15.123:0/434892483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc454074590 msgr2=0x7fc454074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:51.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.982+0000 7fc459e00640 1 --2- 172.21.15.123:0/434892483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc454074590 0x7fc454074970 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc448009900 tx=0x7fc44802a480 comp rx=0 tx=0).stop 2024-09-19T04:54:51.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.982+0000 7fc459e00640 1 -- 172.21.15.123:0/434892483 shutdown_connections 2024-09-19T04:54:51.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.982+0000 7fc459e00640 1 --2- 172.21.15.123:0/434892483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4541129e0 0x7fc454114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:51.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.982+0000 7fc459e00640 1 --2- 172.21.15.123:0/434892483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc454074eb0 0x7fc4541124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:51.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.982+0000 7fc459e00640 1 --2- 172.21.15.123:0/434892483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc454074590 0x7fc454074970 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:51.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.982+0000 7fc459e00640 1 -- 172.21.15.123:0/434892483 >> 172.21.15.123:0/434892483 conn(0x7fc4540fd710 msgr2=0x7fc4540ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:51.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.982+0000 7fc459e00640 1 -- 172.21.15.123:0/434892483 shutdown_connections 2024-09-19T04:54:51.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.982+0000 7fc459e00640 1 -- 172.21.15.123:0/434892483 wait complete. 2024-09-19T04:54:51.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.983+0000 7fc459e00640 1 Processor -- start 2024-09-19T04:54:51.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.983+0000 7fc459e00640 1 -- start start 2024-09-19T04:54:51.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.984+0000 7fc459e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc454074590 0x7fc4541a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:51.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.984+0000 7fc459e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc454074eb0 0x7fc4541a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:51.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.984+0000 7fc459e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4541129e0 0x7fc45419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:51.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.984+0000 7fc459e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4541183c0 con 0x7fc4541129e0 2024-09-19T04:54:51.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.984+0000 7fc459e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc454118240 con 0x7fc454074eb0 2024-09-19T04:54:51.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.984+0000 7fc459e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc454118540 con 0x7fc454074590 2024-09-19T04:54:51.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.984+0000 7fc458a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc454074590 0x7fc4541a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:51.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.984+0000 7fc458a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc454074590 0x7fc4541a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:45184/0 (socket says 172.21.15.123:45184) 2024-09-19T04:54:51.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.984+0000 7fc458a00640 1 -- 172.21.15.123:0/883810632 learned_addr learned my addr 172.21.15.123:0/883810632 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:51.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.984+0000 7fc459400640 1 --2- 172.21.15.123:0/883810632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4541129e0 0x7fc45419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:51.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.984+0000 7fc453e00640 1 --2- 172.21.15.123:0/883810632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc454074eb0 0x7fc4541a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:51.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.985+0000 7fc458a00640 1 -- 172.21.15.123:0/883810632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc454074eb0 msgr2=0x7fc4541a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:51.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.985+0000 7fc458a00640 1 --2- 172.21.15.123:0/883810632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc454074eb0 0x7fc4541a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:51.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.985+0000 7fc458a00640 1 -- 172.21.15.123:0/883810632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4541129e0 msgr2=0x7fc45419e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:51.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.985+0000 7fc458a00640 1 --2- 172.21.15.123:0/883810632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4541129e0 0x7fc45419e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:51.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.985+0000 7fc458a00640 1 -- 172.21.15.123:0/883810632 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45419ef30 con 0x7fc454074590 2024-09-19T04:54:51.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.985+0000 7fc459400640 1 --2- 172.21.15.123:0/883810632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4541129e0 0x7fc45419e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:51.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.985+0000 7fc458a00640 1 --2- 172.21.15.123:0/883810632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc454074590 0x7fc4541a4610 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc448006010 tx=0x7fc4480042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:51.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.985+0000 7fc451600640 1 -- 172.21.15.123:0/883810632 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44803f030 con 0x7fc454074590 2024-09-19T04:54:51.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.985+0000 7fc459e00640 1 -- 172.21.15.123:0/883810632 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45419f1c0 con 0x7fc454074590 2024-09-19T04:54:51.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.985+0000 7fc451600640 1 -- 172.21.15.123:0/883810632 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc448003ce0 con 0x7fc454074590 2024-09-19T04:54:51.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.985+0000 7fc451600640 1 -- 172.21.15.123:0/883810632 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448035070 con 0x7fc454074590 2024-09-19T04:54:51.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.985+0000 7fc459e00640 1 -- 172.21.15.123:0/883810632 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4541ab470 con 0x7fc454074590 2024-09-19T04:54:51.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.988+0000 7fc45de8b740 1 Processor -- start 2024-09-19T04:54:51.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.988+0000 7fc45de8b740 1 -- start start 2024-09-19T04:54:51.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:51 smithi193 ceph-mon[35592]: pgmap v251: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:51.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.989+0000 7fc451600640 1 -- 172.21.15.123:0/883810632 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc448042030 con 0x7fc454074590 2024-09-19T04:54:51.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.989+0000 7fc45de8b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fbe40 0x55973e0fc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:51.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.989+0000 7fc45de8b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e0fc760 0x55973e106ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:51.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.989+0000 7fc45de8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e107020 0x55973e109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:51.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.989+0000 7fc45de8b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55973e0623e0 con 0x55973e0fc760 2024-09-19T04:54:51.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.989+0000 7fc45de8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55973e009d80 con 0x55973e107020 2024-09-19T04:54:51.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.989+0000 7fc45de8b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55973e05e3a0 con 0x55973e0fbe40 2024-09-19T04:54:51.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.989+0000 7fc459400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fbe40 0x55973e0fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:51.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.989+0000 7fc458a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e0fc760 0x55973e106ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:51.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.989+0000 7fc459400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fbe40 0x55973e0fc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:45192/0 (socket says 172.21.15.123:45192) 2024-09-19T04:54:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.989+0000 7fc459400640 1 -- 172.21.15.123:0/1825004371 learned_addr learned my addr 172.21.15.123:0/1825004371 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.989+0000 7fc453e00640 1 --2- 172.21.15.123:0/1825004371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e107020 0x55973e109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.990+0000 7fc459400640 1 -- 172.21.15.123:0/1825004371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e107020 msgr2=0x55973e109410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.990+0000 7fc459400640 1 --2- 172.21.15.123:0/1825004371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e107020 0x55973e109410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.990+0000 7fc459400640 1 -- 172.21.15.123:0/1825004371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e0fc760 msgr2=0x55973e106ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.990+0000 7fc459400640 1 --2- 172.21.15.123:0/1825004371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e0fc760 0x55973e106ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.990+0000 7fc459400640 1 -- 172.21.15.123:0/1825004371 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55973e028ae0 con 0x55973e0fbe40 2024-09-19T04:54:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.990+0000 7fc458a00640 1 --2- 172.21.15.123:0/1825004371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e0fc760 0x55973e106ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.990+0000 7fc459400640 1 --2- 172.21.15.123:0/1825004371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fbe40 0x55973e0fc220 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc44c011760 tx=0x7fc44c011c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=935603d19ad3b1bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.990+0000 7fc445c00640 1 -- 172.21.15.123:0/1825004371 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c004d20 con 0x55973e0fbe40 2024-09-19T04:54:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.990+0000 7fc445c00640 1 -- 172.21.15.123:0/1825004371 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44c00f9a0 con 0x55973e0fbe40 2024-09-19T04:54:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.990+0000 7fc445c00640 1 -- 172.21.15.123:0/1825004371 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c0044d0 con 0x55973e0fbe40 2024-09-19T04:54:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.991+0000 7fc451600640 1 --2- 172.21.15.123:0/883810632 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc43c0a33b0 0x7fc43c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:51.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.991+0000 7fc451600640 1 -- 172.21.15.123:0/883810632 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fc4480f02b0 con 0x7fc454074590 2024-09-19T04:54:51.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.991+0000 7fc458a00640 1 --2- 172.21.15.123:0/883810632 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc43c0a33b0 0x7fc43c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:51.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.991+0000 7fc45de8b740 1 -- 172.21.15.123:0/1825004371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fbe40 msgr2=0x55973e0fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:51.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.991+0000 7fc45de8b740 1 --2- 172.21.15.123:0/1825004371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fbe40 0x55973e0fc220 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc44c011760 tx=0x7fc44c011c20 comp rx=0 tx=0).stop 2024-09-19T04:54:51.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.991+0000 7fc45de8b740 1 -- 172.21.15.123:0/1825004371 shutdown_connections 2024-09-19T04:54:51.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.991+0000 7fc45de8b740 1 --2- 172.21.15.123:0/1825004371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e107020 0x55973e109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:51.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.991+0000 7fc45de8b740 1 --2- 172.21.15.123:0/1825004371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e0fc760 0x55973e106ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:51.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.991+0000 7fc45de8b740 1 --2- 172.21.15.123:0/1825004371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fbe40 0x55973e0fc220 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.991+0000 7fc45de8b740 1 -- 172.21.15.123:0/1825004371 >> 172.21.15.123:0/1825004371 conn(0x55973e0f83d0 msgr2=0x55973e0f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.992+0000 7fc45de8b740 1 -- 172.21.15.123:0/1825004371 shutdown_connections 2024-09-19T04:54:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.992+0000 7fc45de8b740 1 -- 172.21.15.123:0/1825004371 wait complete. 2024-09-19T04:54:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.992+0000 7fc458a00640 1 --2- 172.21.15.123:0/883810632 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc43c0a33b0 0x7fc43c0a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc4480f3fd0 tx=0x7fc4480f3d60 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.992+0000 7fc45de8b740 1 -- start start 2024-09-19T04:54:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.992+0000 7fc45de8b740 1 Processor -- start 2024-09-19T04:54:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.993+0000 7fc45de8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e0fbe40 0x55973e103ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.993+0000 7fc45de8b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fc760 0x55973e104410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.993+0000 7fc45de8b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e107020 0x55973e104950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.993+0000 7fc45de8b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55973e109bf0 con 0x55973e107020 2024-09-19T04:54:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.993+0000 7fc45de8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55973e01d0f0 con 0x55973e0fbe40 2024-09-19T04:54:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.993+0000 7fc45de8b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55973e109d70 con 0x55973e0fc760 2024-09-19T04:54:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.993+0000 7fc45de8b740 10 client.0 _ll_register_callbacks cb 0x55973e004bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.993+0000 7fc459400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e107020 0x55973e104950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.993+0000 7fc459400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e107020 0x55973e104950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47734/0 (socket says 172.21.15.123:47734) 2024-09-19T04:54:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.993+0000 7fc459400640 1 -- 172.21.15.123:0/3103415405 learned_addr learned my addr 172.21.15.123:0/3103415405 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.993+0000 7fc453e00640 1 --2- 172.21.15.123:0/3103415405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fc760 0x55973e104410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.993+0000 7fc459400640 1 --2- 172.21.15.123:0/3103415405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e0fbe40 0x55973e103ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.994+0000 7fc459400640 1 -- 172.21.15.123:0/3103415405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fc760 msgr2=0x55973e104410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.994+0000 7fc459400640 1 --2- 172.21.15.123:0/3103415405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fc760 0x55973e104410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.994+0000 7fc459400640 1 -- 172.21.15.123:0/3103415405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e0fbe40 msgr2=0x55973e103ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.994+0000 7fc459400640 1 --2- 172.21.15.123:0/3103415405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e0fbe40 0x55973e103ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.994+0000 7fc459400640 1 -- 172.21.15.123:0/3103415405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55973ddd33c0 con 0x55973e107020 2024-09-19T04:54:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.994+0000 7fc459400640 1 --2- 172.21.15.123:0/3103415405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e107020 0x55973e104950 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00f790 tx=0x7fc44c0040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7bd20196433388 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.994+0000 7fc43a000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.994+0000 7fc43a000640 1 -- 172.21.15.123:0/3103415405 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c01e040 con 0x55973e107020 2024-09-19T04:54:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.994+0000 7fc43a000640 1 -- 172.21.15.123:0/3103415405 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fc44c005690 con 0x55973e107020 2024-09-19T04:54:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.994+0000 7fc45de8b740 1 -- 172.21.15.123:0/3103415405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc44c00f9a0 con 0x55973e107020 2024-09-19T04:54:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.994+0000 7fc43a000640 1 -- 172.21.15.123:0/3103415405 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc44c005890 con 0x55973e107020 2024-09-19T04:54:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.994+0000 7fc45de8b740 10 client.16983 fetch_fsmap learned FSMap version 43 2024-09-19T04:54:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.995+0000 7fc45de8b740 1 -- 172.21.15.123:0/3103415405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55973e115c00 con 0x55973e107020 2024-09-19T04:54:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.995+0000 7fc43a000640 1 -- 172.21.15.123:0/3103415405 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44c00f9a0 con 0x55973e107020 2024-09-19T04:54:52.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.995+0000 7fc43a000640 1 -- 172.21.15.123:0/3103415405 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fc44c024030 con 0x55973e107020 2024-09-19T04:54:52.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.995+0000 7fc45de8b740 10 client.16983 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:54:52.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.995+0000 7fc45de8b740 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:52.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.995+0000 7fc45de8b740 20 client.16983 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:52.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.995+0000 7fc45de8b740 1 --2- 172.21.15.123:0/3103415405 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55973e117010 0x55973e119400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:52.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.995+0000 7fc45de8b740 4 client.16983 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.995+0000 7fc45de8b740 1 -- 172.21.15.123:0/3103415405 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55973e06f950 con 0x55973e117010 2024-09-19T04:54:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.996+0000 7fc453e00640 1 --2- 172.21.15.123:0/3103415405 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55973e117010 0x55973e119400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.996+0000 7fc453e00640 1 --2- 172.21.15.123:0/3103415405 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55973e117010 0x55973e119400 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3ea4a68e5d00317 server_cookie=2bf246deae4d0831 in_seq=0 out_seq=0 2024-09-19T04:54:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.996+0000 7fc43a000640 10 client.16983 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.997+0000 7fc43a000640 1 -- 172.21.15.123:0/3103415405 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55973e06f950 con 0x55973e117010 2024-09-19T04:54:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.997+0000 7fc43a000640 10 client.16983 handle_command_reply: tid=0 2024-09-19T04:54:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.997+0000 7fc45de8b740 1 client.16983 shutdown 2024-09-19T04:54:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.997+0000 7fc45de8b740 1 -- 172.21.15.123:0/3103415405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e107020 msgr2=0x55973e104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.997+0000 7fc45de8b740 1 --2- 172.21.15.123:0/3103415405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e107020 0x55973e104950 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00f790 tx=0x7fc44c0040f0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.997+0000 7fc45de8b740 1 -- 172.21.15.123:0/3103415405 shutdown_connections 2024-09-19T04:54:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.997+0000 7fc45de8b740 1 --2- 172.21.15.123:0/3103415405 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55973e117010 0x55973e119400 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.997+0000 7fc45de8b740 1 --2- 172.21.15.123:0/3103415405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e107020 0x55973e104950 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.997+0000 7fc45de8b740 1 --2- 172.21.15.123:0/3103415405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fc760 0x55973e104410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.997+0000 7fc45de8b740 1 --2- 172.21.15.123:0/3103415405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e0fbe40 0x55973e103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.997+0000 7fc45de8b740 1 -- 172.21.15.123:0/3103415405 >> 172.21.15.123:0/3103415405 conn(0x55973e0f83d0 msgr2=0x55973e0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.997+0000 7fc43a000640 0 client.16983 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.997+0000 7fc45de8b740 1 -- 172.21.15.123:0/3103415405 shutdown_connections 2024-09-19T04:54:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.997+0000 7fc45de8b740 1 -- 172.21.15.123:0/3103415405 wait complete. 2024-09-19T04:54:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:51.998+0000 7fc45de8b740 20 client.16983 trim_cache size 0 max 16384 2024-09-19T04:54:52.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.011+0000 7fc45de8b740 1 Processor -- start 2024-09-19T04:54:52.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.011+0000 7fc45de8b740 1 -- start start 2024-09-19T04:54:52.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.011+0000 7fc45de8b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e0fbe40 0x55973e0fe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:52.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.011+0000 7fc45de8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e0fc760 0x55973e1027a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:52.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.011+0000 7fc45de8b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e107020 0x55973e11eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:52.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.011+0000 7fc45de8b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4541b3d20 con 0x55973e0fbe40 2024-09-19T04:54:52.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.011+0000 7fc45de8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4541b5860 con 0x55973e0fc760 2024-09-19T04:54:52.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.011+0000 7fc45de8b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc4541b43f0 con 0x55973e107020 2024-09-19T04:54:52.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.012+0000 7fc459400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e0fbe40 0x55973e0fe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:52.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.012+0000 7fc459400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e0fbe40 0x55973e0fe8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47750/0 (socket says 172.21.15.123:47750) 2024-09-19T04:54:52.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.012+0000 7fc453e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e0fc760 0x55973e1027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:52.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.012+0000 7fc459400640 1 -- 172.21.15.123:0/289583094 learned_addr learned my addr 172.21.15.123:0/289583094 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:52.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.012+0000 7fc459400640 1 --2- 172.21.15.123:0/289583094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e107020 0x55973e11eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:52.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.012+0000 7fc459400640 1 -- 172.21.15.123:0/289583094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e107020 msgr2=0x55973e11eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:52.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.012+0000 7fc459400640 1 --2- 172.21.15.123:0/289583094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e107020 0x55973e11eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.012+0000 7fc459400640 1 -- 172.21.15.123:0/289583094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e0fc760 msgr2=0x55973e1027a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:52.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.012+0000 7fc459400640 1 --2- 172.21.15.123:0/289583094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e0fc760 0x55973e1027a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.012+0000 7fc459400640 1 -- 172.21.15.123:0/289583094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55973e0fccd0 con 0x55973e0fbe40 2024-09-19T04:54:52.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.012+0000 7fc453e00640 1 --2- 172.21.15.123:0/289583094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e0fc760 0x55973e1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:52.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.012+0000 7fc459400640 1 --2- 172.21.15.123:0/289583094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e0fbe40 0x55973e0fe8a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009560 tx=0x7fc44c011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1489667441fb4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:52.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.013+0000 7fc445200640 1 -- 172.21.15.123:0/289583094 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c003d60 con 0x55973e0fbe40 2024-09-19T04:54:52.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.013+0000 7fc445200640 1 -- 172.21.15.123:0/289583094 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44c005a30 con 0x55973e0fbe40 2024-09-19T04:54:52.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.013+0000 7fc445200640 1 -- 172.21.15.123:0/289583094 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c00fe90 con 0x55973e0fbe40 2024-09-19T04:54:52.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.013+0000 7fc45de8b740 1 -- 172.21.15.123:0/289583094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e0fbe40 msgr2=0x55973e0fe8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:52.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.013+0000 7fc45de8b740 1 --2- 172.21.15.123:0/289583094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e0fbe40 0x55973e0fe8a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009560 tx=0x7fc44c011c20 comp rx=0 tx=0).stop 2024-09-19T04:54:52.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.013+0000 7fc45de8b740 1 -- 172.21.15.123:0/289583094 shutdown_connections 2024-09-19T04:54:52.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.013+0000 7fc45de8b740 1 --2- 172.21.15.123:0/289583094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e107020 0x55973e11eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.013+0000 7fc45de8b740 1 --2- 172.21.15.123:0/289583094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e0fc760 0x55973e1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.013+0000 7fc45de8b740 1 --2- 172.21.15.123:0/289583094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e0fbe40 0x55973e0fe8a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.013+0000 7fc45de8b740 1 -- 172.21.15.123:0/289583094 >> 172.21.15.123:0/289583094 conn(0x55973e0f83d0 msgr2=0x55973e105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:52.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.014+0000 7fc45de8b740 1 -- 172.21.15.123:0/289583094 shutdown_connections 2024-09-19T04:54:52.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.014+0000 7fc45de8b740 1 -- 172.21.15.123:0/289583094 wait complete. 2024-09-19T04:54:52.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.014+0000 7fc45de8b740 1 -- start start 2024-09-19T04:54:52.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.014+0000 7fc45de8b740 1 Processor -- start 2024-09-19T04:54:52.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc45de8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e0fbe40 0x55973e10f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:52.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc45de8b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fc760 0x55973e10aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:52.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc45de8b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e107020 0x55973e10af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:52.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc45de8b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4541b5190 con 0x55973e107020 2024-09-19T04:54:52.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc45de8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4541b4ac0 con 0x55973e0fbe40 2024-09-19T04:54:52.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc45de8b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc4541b1d20 con 0x55973e0fc760 2024-09-19T04:54:52.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc45de8b740 10 client.0 _ll_register_callbacks cb 0x7fc4480f3c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:52.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc453e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fc760 0x55973e10aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:52.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc459400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e0fbe40 0x55973e10f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:52.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc453e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fc760 0x55973e10aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:45218/0 (socket says 172.21.15.123:45218) 2024-09-19T04:54:52.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc453e00640 1 -- 172.21.15.123:0/1425877917 learned_addr learned my addr 172.21.15.123:0/1425877917 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:52.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc459400640 1 --2- 172.21.15.123:0/1425877917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e107020 0x55973e10af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:52.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc453e00640 1 -- 172.21.15.123:0/1425877917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e0fbe40 msgr2=0x55973e10f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:52.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc453e00640 1 --2- 172.21.15.123:0/1425877917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e0fbe40 0x55973e10f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc453e00640 1 -- 172.21.15.123:0/1425877917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e107020 msgr2=0x55973e10af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:52.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc453e00640 1 --2- 172.21.15.123:0/1425877917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e107020 0x55973e10af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc453e00640 1 -- 172.21.15.123:0/1425877917 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55973e10b760 con 0x55973e0fc760 2024-09-19T04:54:52.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.015+0000 7fc459400640 1 --2- 172.21.15.123:0/1425877917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e107020 0x55973e10af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:54:52.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.016+0000 7fc453e00640 1 --2- 172.21.15.123:0/1425877917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fc760 0x55973e10aa40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55973e0f9750 tx=0x7fc44000bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7a9b8b9b09a0179 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:52.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.016+0000 7fc43a000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:54:52.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.016+0000 7fc43a000640 1 -- 172.21.15.123:0/1425877917 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440011640 con 0x55973e0fc760 2024-09-19T04:54:52.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.016+0000 7fc43a000640 1 -- 172.21.15.123:0/1425877917 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fc4400117e0 con 0x55973e0fc760 2024-09-19T04:54:52.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.016+0000 7fc45de8b740 1 -- 172.21.15.123:0/1425877917 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc44c005a30 con 0x55973e0fc760 2024-09-19T04:54:52.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.016+0000 7fc43a000640 1 -- 172.21.15.123:0/1425877917 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc440011a00 con 0x55973e0fc760 2024-09-19T04:54:52.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.016+0000 7fc45de8b740 10 client.27103 fetch_fsmap learned FSMap version 43 2024-09-19T04:54:52.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.016+0000 7fc45de8b740 1 -- 172.21.15.123:0/1425877917 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55973e181bd0 con 0x55973e0fc760 2024-09-19T04:54:52.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.017+0000 7fc43a000640 1 -- 172.21.15.123:0/1425877917 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44c005a30 con 0x55973e0fc760 2024-09-19T04:54:52.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.017+0000 7fc43a000640 1 -- 172.21.15.123:0/1425877917 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fc440026410 con 0x55973e0fc760 2024-09-19T04:54:52.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.017+0000 7fc45de8b740 10 client.27103 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:54:52.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.017+0000 7fc45de8b740 10 client.27103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:52.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.017+0000 7fc45de8b740 20 client.27103 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:52.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.017+0000 7fc45de8b740 1 --2- 172.21.15.123:0/1425877917 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55973e182350 0x55973e184770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.017+0000 7fc45de8b740 4 client.27103 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:54:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.017+0000 7fc45de8b740 1 -- 172.21.15.123:0/1425877917 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55973e069e60 con 0x55973e182350 2024-09-19T04:54:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.017+0000 7fc459400640 1 --2- 172.21.15.123:0/1425877917 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55973e182350 0x55973e184770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.018+0000 7fc459400640 1 --2- 172.21.15.123:0/1425877917 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55973e182350 0x55973e184770 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1457aab43db20c32 server_cookie=87e30961756ff19d in_seq=0 out_seq=0 2024-09-19T04:54:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.018+0000 7fc43a000640 10 client.27103 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.018+0000 7fc43a000640 1 -- 172.21.15.123:0/1425877917 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55973e069e60 con 0x55973e182350 2024-09-19T04:54:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.018+0000 7fc43a000640 10 client.27103 handle_command_reply: tid=0 2024-09-19T04:54:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.018+0000 7fc45de8b740 1 client.27103 shutdown 2024-09-19T04:54:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.018+0000 7fc45de8b740 1 -- 172.21.15.123:0/1425877917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fc760 msgr2=0x55973e10aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.018+0000 7fc45de8b740 1 --2- 172.21.15.123:0/1425877917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fc760 0x55973e10aa40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55973e0f9750 tx=0x7fc44000bfd0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.018+0000 7fc45de8b740 1 -- 172.21.15.123:0/1425877917 shutdown_connections 2024-09-19T04:54:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.018+0000 7fc45de8b740 1 --2- 172.21.15.123:0/1425877917 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55973e182350 0x55973e184770 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.018+0000 7fc45de8b740 1 --2- 172.21.15.123:0/1425877917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55973e107020 0x55973e10af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.018+0000 7fc45de8b740 1 --2- 172.21.15.123:0/1425877917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55973e0fc760 0x55973e10aa40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.018+0000 7fc45de8b740 1 --2- 172.21.15.123:0/1425877917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55973e0fbe40 0x55973e10f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.018+0000 7fc43a000640 0 client.27103 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:52.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.018+0000 7fc45de8b740 1 -- 172.21.15.123:0/1425877917 >> 172.21.15.123:0/1425877917 conn(0x55973e0f83d0 msgr2=0x55973e103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:52.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.019+0000 7fc45de8b740 1 -- 172.21.15.123:0/1425877917 shutdown_connections 2024-09-19T04:54:52.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.019+0000 7fc45de8b740 1 -- 172.21.15.123:0/1425877917 wait complete. 2024-09-19T04:54:52.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.019+0000 7fc45de8b740 20 client.27103 trim_cache size 0 max 16384 2024-09-19T04:54:52.024 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:54:52.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.019+0000 7fc459e00640 1 -- 172.21.15.123:0/883810632 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc43c0a33b0 msgr2=0x7fc43c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:52.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.019+0000 7fc459e00640 1 --2- 172.21.15.123:0/883810632 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc43c0a33b0 0x7fc43c0a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc4480f3fd0 tx=0x7fc4480f3d60 comp rx=0 tx=0).stop 2024-09-19T04:54:52.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.019+0000 7fc459e00640 1 -- 172.21.15.123:0/883810632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc454074590 msgr2=0x7fc4541a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:52.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.019+0000 7fc459e00640 1 --2- 172.21.15.123:0/883810632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc454074590 0x7fc4541a4610 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc448006010 tx=0x7fc4480042f0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.019+0000 7fc459e00640 1 -- 172.21.15.123:0/883810632 shutdown_connections 2024-09-19T04:54:52.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.019+0000 7fc459e00640 1 --2- 172.21.15.123:0/883810632 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc43c0a33b0 0x7fc43c0a5870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.019+0000 7fc459e00640 1 --2- 172.21.15.123:0/883810632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4541129e0 0x7fc45419e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.019+0000 7fc459e00640 1 --2- 172.21.15.123:0/883810632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc454074eb0 0x7fc4541a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.019+0000 7fc459e00640 1 --2- 172.21.15.123:0/883810632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc454074590 0x7fc4541a4610 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.019+0000 7fc459e00640 1 -- 172.21.15.123:0/883810632 >> 172.21.15.123:0/883810632 conn(0x7fc4540fd710 msgr2=0x7fc4540ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:52.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.020+0000 7fc459e00640 1 -- 172.21.15.123:0/883810632 shutdown_connections 2024-09-19T04:54:52.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:52.020+0000 7fc459e00640 1 -- 172.21.15.123:0/883810632 wait complete. 2024-09-19T04:54:53.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:53 smithi183 ceph-mon[35333]: pgmap v252: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:53.898 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:54:53.899 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:54:53.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:53 smithi123 ceph-mon[30511]: pgmap v252: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:53.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:53 smithi193 ceph-mon[35592]: pgmap v252: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:54:54.294 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:54.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.879+0000 7f6edea00640 1 -- 172.21.15.123:0/1908514996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ed8108f60 msgr2=0x7f6ed8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:54.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.879+0000 7f6edea00640 1 --2- 172.21.15.123:0/1908514996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ed8108f60 0x7f6ed8110ae0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0009ca0 tx=0x7f6ed002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.879+0000 7f6edea00640 1 -- 172.21.15.123:0/1908514996 shutdown_connections 2024-09-19T04:54:54.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.879+0000 7f6edea00640 1 --2- 172.21.15.123:0/1908514996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ed8108f60 0x7f6ed8110ae0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.879+0000 7f6edea00640 1 --2- 172.21.15.123:0/1908514996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed81047e0 0x7f6ed8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.879+0000 7f6edea00640 1 --2- 172.21.15.123:0/1908514996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed8103e30 0x7f6ed8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.879+0000 7f6edea00640 1 -- 172.21.15.123:0/1908514996 >> 172.21.15.123:0/1908514996 conn(0x7f6ed80ff8d0 msgr2=0x7f6ed8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:54.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.879+0000 7f6edea00640 1 -- 172.21.15.123:0/1908514996 shutdown_connections 2024-09-19T04:54:54.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.880+0000 7f6edea00640 1 -- 172.21.15.123:0/1908514996 wait complete. 2024-09-19T04:54:54.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.881+0000 7f6edea00640 1 Processor -- start 2024-09-19T04:54:54.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.881+0000 7f6edea00640 1 -- start start 2024-09-19T04:54:54.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.881+0000 7f6edea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed8103e30 0x7f6ed819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.881+0000 7f6edea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed81047e0 0x7f6ed819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.881+0000 7f6edea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ed8108f60 0x7f6ed819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.881+0000 7f6edea00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8114010 con 0x7f6ed8108f60 2024-09-19T04:54:54.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.881+0000 7f6edea00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8113e90 con 0x7f6ed81047e0 2024-09-19T04:54:54.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.881+0000 7f6edea00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8114190 con 0x7f6ed8103e30 2024-09-19T04:54:54.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.882+0000 7f6ede000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ed8108f60 0x7f6ed819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.882+0000 7f6ede000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ed8108f60 0x7f6ed819ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47766/0 (socket says 172.21.15.123:47766) 2024-09-19T04:54:54.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.882+0000 7f6ede000640 1 -- 172.21.15.123:0/1116424924 learned_addr learned my addr 172.21.15.123:0/1116424924 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:54.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.882+0000 7f6edd600640 1 --2- 172.21.15.123:0/1116424924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed8103e30 0x7f6ed819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.882+0000 7f6edcc00640 1 --2- 172.21.15.123:0/1116424924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed81047e0 0x7f6ed819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.882+0000 7f6edcc00640 1 -- 172.21.15.123:0/1116424924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed8103e30 msgr2=0x7f6ed819a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:54.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.882+0000 7f6edcc00640 1 --2- 172.21.15.123:0/1116424924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed8103e30 0x7f6ed819a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.882+0000 7f6edcc00640 1 -- 172.21.15.123:0/1116424924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ed8108f60 msgr2=0x7f6ed819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:54.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.882+0000 7f6edcc00640 1 --2- 172.21.15.123:0/1116424924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ed8108f60 0x7f6ed819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.882+0000 7f6edcc00640 1 -- 172.21.15.123:0/1116424924 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed819f3a0 con 0x7f6ed81047e0 2024-09-19T04:54:54.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.882+0000 7f6edd600640 1 --2- 172.21.15.123:0/1116424924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed8103e30 0x7f6ed819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:54.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.883+0000 7f6edcc00640 1 --2- 172.21.15.123:0/1116424924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed81047e0 0x7f6ed819a930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6ec400e9f0 tx=0x7f6ec400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:54.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.883+0000 7f6ed6000640 1 -- 172.21.15.123:0/1116424924 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec400ccc0 con 0x7f6ed81047e0 2024-09-19T04:54:54.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.883+0000 7f6ed6000640 1 -- 172.21.15.123:0/1116424924 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ec4005020 con 0x7f6ed81047e0 2024-09-19T04:54:54.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.883+0000 7f6edea00640 1 -- 172.21.15.123:0/1116424924 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed819f690 con 0x7f6ed81047e0 2024-09-19T04:54:54.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.883+0000 7f6ed6000640 1 -- 172.21.15.123:0/1116424924 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec40049e0 con 0x7f6ed81047e0 2024-09-19T04:54:54.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.883+0000 7f6edea00640 1 -- 172.21.15.123:0/1116424924 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed81a6f70 con 0x7f6ed81047e0 2024-09-19T04:54:54.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.886+0000 7f6ee2a8f740 1 Processor -- start 2024-09-19T04:54:54.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.886+0000 7f6ee2a8f740 1 -- start start 2024-09-19T04:54:54.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.886+0000 7f6ed6000640 1 -- 172.21.15.123:0/1116424924 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6ec4005560 con 0x7f6ed81047e0 2024-09-19T04:54:54.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.886+0000 7f6ee2a8f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387da2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.887+0000 7f6ee2a8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387da25d0 0x55c387dac950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.887+0000 7f6ee2a8f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387dace90 0x55c387daf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.887+0000 7f6ee2a8f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c387d08290 con 0x55c387dace90 2024-09-19T04:54:54.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.887+0000 7f6ee2a8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c387cafb40 con 0x55c387da25d0 2024-09-19T04:54:54.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.887+0000 7f6ee2a8f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c387d04250 con 0x55c387da1cb0 2024-09-19T04:54:54.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.887+0000 7f6ede000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387da2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.887+0000 7f6ede000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387da2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:45236/0 (socket says 172.21.15.123:45236) 2024-09-19T04:54:54.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.887+0000 7f6ede000640 1 -- 172.21.15.123:0/2246049490 learned_addr learned my addr 172.21.15.123:0/2246049490 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:54.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.887+0000 7f6edd600640 1 --2- 172.21.15.123:0/2246049490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387da25d0 0x55c387dac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.887+0000 7f6edcc00640 1 --2- 172.21.15.123:0/2246049490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387dace90 0x55c387daf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.887+0000 7f6ede000640 1 -- 172.21.15.123:0/2246049490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387da25d0 msgr2=0x55c387dac950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:54.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.887+0000 7f6ede000640 1 --2- 172.21.15.123:0/2246049490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387da25d0 0x55c387dac950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.887+0000 7f6ede000640 1 -- 172.21.15.123:0/2246049490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387dace90 msgr2=0x55c387daf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:54.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.887+0000 7f6ede000640 1 --2- 172.21.15.123:0/2246049490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387dace90 0x55c387daf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.887+0000 7f6ede000640 1 -- 172.21.15.123:0/2246049490 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c387a796a0 con 0x55c387da1cb0 2024-09-19T04:54:54.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.888+0000 7f6ede000640 1 --2- 172.21.15.123:0/2246049490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387da2090 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0005fb0 tx=0x7f6ed0009a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2545fed297a209e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:54.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.888+0000 7f6eca600640 1 -- 172.21.15.123:0/2246049490 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0031db0 con 0x55c387da1cb0 2024-09-19T04:54:54.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.888+0000 7f6eca600640 1 -- 172.21.15.123:0/2246049490 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ed0036030 con 0x55c387da1cb0 2024-09-19T04:54:54.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.888+0000 7f6eca600640 1 -- 172.21.15.123:0/2246049490 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed00042a0 con 0x55c387da1cb0 2024-09-19T04:54:54.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.888+0000 7f6ed6000640 1 --2- 172.21.15.123:0/1116424924 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ec00a3430 0x7f6ec00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.888+0000 7f6ed6000640 1 -- 172.21.15.123:0/1116424924 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f6ec40cfbd0 con 0x7f6ed81047e0 2024-09-19T04:54:54.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.888+0000 7f6edd600640 1 --2- 172.21.15.123:0/1116424924 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ec00a3430 0x7f6ec00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.889+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/2246049490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 msgr2=0x55c387da2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:54.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.889+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/2246049490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387da2090 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0005fb0 tx=0x7f6ed0009a10 comp rx=0 tx=0).stop 2024-09-19T04:54:54.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.889+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/2246049490 shutdown_connections 2024-09-19T04:54:54.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.889+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/2246049490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387dace90 0x55c387daf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.889+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/2246049490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387da25d0 0x55c387dac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.889+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/2246049490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387da2090 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.889+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/2246049490 >> 172.21.15.123:0/2246049490 conn(0x55c387d9e240 msgr2=0x55c387d9e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:54.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.889+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/2246049490 shutdown_connections 2024-09-19T04:54:54.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.889+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/2246049490 wait complete. 2024-09-19T04:54:54.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.889+0000 7f6edd600640 1 --2- 172.21.15.123:0/1116424924 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ec00a3430 0x7f6ec00a58f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55c387d9f0a0 tx=0x7f6ecc005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:54.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.890+0000 7f6ee2a8f740 1 -- start start 2024-09-19T04:54:54.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.890+0000 7f6ee2a8f740 1 Processor -- start 2024-09-19T04:54:54.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.890+0000 7f6ee2a8f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387da1cb0 0x55c387dac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.890+0000 7f6ee2a8f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da25d0 0x55c387da9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.890+0000 7f6ee2a8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387dace90 0x55c387da9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.890+0000 7f6ee2a8f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c387daf980 con 0x55c387da1cb0 2024-09-19T04:54:54.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.890+0000 7f6ee2a8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c387cc2fa0 con 0x55c387dace90 2024-09-19T04:54:54.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.890+0000 7f6ee2a8f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c387dafb00 con 0x55c387da25d0 2024-09-19T04:54:54.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.890+0000 7f6ee2a8f740 10 client.0 _ll_register_callbacks cb 0x55c387a16ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:54.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.891+0000 7f6edd600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387da1cb0 0x55c387dac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.891+0000 7f6ede000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387dace90 0x55c387da9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.891+0000 7f6ede000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387dace90 0x55c387da9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:52692/0 (socket says 172.21.15.123:52692) 2024-09-19T04:54:54.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.891+0000 7f6ede000640 1 -- 172.21.15.123:0/51127578 learned_addr learned my addr 172.21.15.123:0/51127578 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:54.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.891+0000 7f6edcc00640 1 --2- 172.21.15.123:0/51127578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da25d0 0x55c387da9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.891+0000 7f6edd600640 1 -- 172.21.15.123:0/51127578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da25d0 msgr2=0x55c387da9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:54.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.891+0000 7f6edd600640 1 --2- 172.21.15.123:0/51127578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da25d0 0x55c387da9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.891+0000 7f6edd600640 1 -- 172.21.15.123:0/51127578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387dace90 msgr2=0x55c387da9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:54.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.891+0000 7f6edd600640 1 --2- 172.21.15.123:0/51127578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387dace90 0x55c387da9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.891+0000 7f6edd600640 1 -- 172.21.15.123:0/51127578 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c387daa780 con 0x55c387da1cb0 2024-09-19T04:54:54.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.891+0000 7f6edd600640 1 --2- 172.21.15.123:0/51127578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387da1cb0 0x55c387dac760 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc012ea0 tx=0x7f6ecc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4e2c74e35b78ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:54.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.891+0000 7f6ebea00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:54.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.892+0000 7f6ebea00640 1 -- 172.21.15.123:0/51127578 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc02e060 con 0x55c387da1cb0 2024-09-19T04:54:54.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.892+0000 7f6ebea00640 1 -- 172.21.15.123:0/51127578 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f6ecc018030 con 0x55c387da1cb0 2024-09-19T04:54:54.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.892+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/51127578 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ed0036030 con 0x55c387da1cb0 2024-09-19T04:54:54.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.892+0000 7f6ebea00640 1 -- 172.21.15.123:0/51127578 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6ecc01e620 con 0x55c387da1cb0 2024-09-19T04:54:54.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.892+0000 7f6ee2a8f740 10 client.17001 fetch_fsmap learned FSMap version 43 2024-09-19T04:54:54.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.892+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/51127578 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c387dbb880 con 0x55c387da1cb0 2024-09-19T04:54:54.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.892+0000 7f6ebea00640 1 -- 172.21.15.123:0/51127578 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ed0036030 con 0x55c387da1cb0 2024-09-19T04:54:54.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.893+0000 7f6ebea00640 1 -- 172.21.15.123:0/51127578 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f6ecc029990 con 0x55c387da1cb0 2024-09-19T04:54:54.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.893+0000 7f6ee2a8f740 10 client.17001 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:54:54.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.893+0000 7f6ee2a8f740 10 client.17001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:54.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.893+0000 7f6ee2a8f740 20 client.17001 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:54.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.893+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/51127578 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55c387dbcd40 0x55c387dbf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.893+0000 7f6ee2a8f740 4 client.17001 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:54.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.893+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/51127578 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c387d15800 con 0x55c387dbcd40 2024-09-19T04:54:54.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.893+0000 7f6edcc00640 1 --2- 172.21.15.123:0/51127578 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55c387dbcd40 0x55c387dbf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.894+0000 7f6edcc00640 1 --2- 172.21.15.123:0/51127578 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55c387dbcd40 0x55c387dbf190 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b62de099a33440ee server_cookie=77a317bd0a4c5698 in_seq=0 out_seq=0 2024-09-19T04:54:54.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.894+0000 7f6ebea00640 10 client.17001 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:54.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.894+0000 7f6ebea00640 1 -- 172.21.15.123:0/51127578 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c387d15800 con 0x55c387dbcd40 2024-09-19T04:54:54.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.894+0000 7f6ebea00640 10 client.17001 handle_command_reply: tid=0 2024-09-19T04:54:54.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.894+0000 7f6ee2a8f740 1 client.17001 shutdown 2024-09-19T04:54:54.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.895+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/51127578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387da1cb0 msgr2=0x55c387dac760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:54.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.895+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/51127578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387da1cb0 0x55c387dac760 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc012ea0 tx=0x7f6ecc004730 comp rx=0 tx=0).stop 2024-09-19T04:54:54.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.895+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/51127578 shutdown_connections 2024-09-19T04:54:54.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.895+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/51127578 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55c387dbcd40 0x55c387dbf190 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.895+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/51127578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387dace90 0x55c387da9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.895+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/51127578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da25d0 0x55c387da9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.895+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/51127578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387da1cb0 0x55c387dac760 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.895+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/51127578 >> 172.21.15.123:0/51127578 conn(0x55c387d9e240 msgr2=0x55c387da3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:54.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.895+0000 7f6ebea00640 0 client.17001 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:54.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.895+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/51127578 shutdown_connections 2024-09-19T04:54:54.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.895+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/51127578 wait complete. 2024-09-19T04:54:54.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.895+0000 7f6ee2a8f740 20 client.17001 trim_cache size 0 max 16384 2024-09-19T04:54:54.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.913+0000 7f6ee2a8f740 1 Processor -- start 2024-09-19T04:54:54.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.913+0000 7f6ee2a8f740 1 -- start start 2024-09-19T04:54:54.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.913+0000 7f6ee2a8f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387da4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.913+0000 7f6ee2a8f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387da25d0 0x55c387dc4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.913+0000 7f6ee2a8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387dace90 0x55c387dc4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.913+0000 7f6ee2a8f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81af840 con 0x55c387da25d0 2024-09-19T04:54:54.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.913+0000 7f6ee2a8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81b1380 con 0x55c387dace90 2024-09-19T04:54:54.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.913+0000 7f6ee2a8f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81aff10 con 0x55c387da1cb0 2024-09-19T04:54:54.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.913+0000 7f6edcc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387da25d0 0x55c387dc4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.913+0000 7f6edd600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387da4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.913+0000 7f6edd600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387da4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:45248/0 (socket says 172.21.15.123:45248) 2024-09-19T04:54:54.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.913+0000 7f6edd600640 1 -- 172.21.15.123:0/1007743600 learned_addr learned my addr 172.21.15.123:0/1007743600 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:54.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.913+0000 7f6ede000640 1 --2- 172.21.15.123:0/1007743600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387dace90 0x55c387dc4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.914+0000 7f6edd600640 1 -- 172.21.15.123:0/1007743600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387dace90 msgr2=0x55c387dc4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:54.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.914+0000 7f6edd600640 1 --2- 172.21.15.123:0/1007743600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387dace90 0x55c387dc4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.914+0000 7f6edd600640 1 -- 172.21.15.123:0/1007743600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387da25d0 msgr2=0x55c387dc4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:54.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.914+0000 7f6edd600640 1 --2- 172.21.15.123:0/1007743600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387da25d0 0x55c387dc4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.914+0000 7f6edd600640 1 -- 172.21.15.123:0/1007743600 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c387da7f60 con 0x55c387da1cb0 2024-09-19T04:54:54.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.914+0000 7f6edcc00640 1 --2- 172.21.15.123:0/1007743600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387da25d0 0x55c387dc4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:54.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.914+0000 7f6edd600640 1 --2- 172.21.15.123:0/1007743600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387da4f50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc004700 tx=0x7f6ecc005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebf237d32698e026 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:54.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.914+0000 7f6ec9c00640 1 -- 172.21.15.123:0/1007743600 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc01e040 con 0x55c387da1cb0 2024-09-19T04:54:54.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.914+0000 7f6ec9c00640 1 -- 172.21.15.123:0/1007743600 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ecc01e1e0 con 0x55c387da1cb0 2024-09-19T04:54:54.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.914+0000 7f6ec9c00640 1 -- 172.21.15.123:0/1007743600 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc01ebb0 con 0x55c387da1cb0 2024-09-19T04:54:54.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.915+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/1007743600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 msgr2=0x55c387da4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:54.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.915+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/1007743600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387da4f50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc004700 tx=0x7f6ecc005d30 comp rx=0 tx=0).stop 2024-09-19T04:54:54.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.915+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/1007743600 shutdown_connections 2024-09-19T04:54:54.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.915+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/1007743600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387dace90 0x55c387dc4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.915+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/1007743600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387da25d0 0x55c387dc4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.915+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/1007743600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387da4f50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.915+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/1007743600 >> 172.21.15.123:0/1007743600 conn(0x55c387d9e240 msgr2=0x55c387da4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:54.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.915+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/1007743600 shutdown_connections 2024-09-19T04:54:54.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.915+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/1007743600 wait complete. 2024-09-19T04:54:54.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.916+0000 7f6ee2a8f740 1 -- start start 2024-09-19T04:54:54.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.916+0000 7f6ee2a8f740 1 Processor -- start 2024-09-19T04:54:54.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.916+0000 7f6ee2a8f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387db0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.916+0000 7f6ee2a8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387da25d0 0x55c387db1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.916+0000 7f6ee2a8f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387dace90 0x55c387db1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.916+0000 7f6ee2a8f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81b0cb0 con 0x55c387dace90 2024-09-19T04:54:54.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.916+0000 7f6ee2a8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81b05e0 con 0x55c387da25d0 2024-09-19T04:54:54.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.916+0000 7f6ee2a8f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81ad840 con 0x55c387da1cb0 2024-09-19T04:54:54.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.916+0000 7f6ee2a8f740 10 client.0 _ll_register_callbacks cb 0x7f6ecc009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:54.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.916+0000 7f6edd600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387db0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.916+0000 7f6edcc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387da25d0 0x55c387db1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.916+0000 7f6edd600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387db0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:45262/0 (socket says 172.21.15.123:45262) 2024-09-19T04:54:54.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.916+0000 7f6edd600640 1 -- 172.21.15.123:0/3892050755 learned_addr learned my addr 172.21.15.123:0/3892050755 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:54.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.917+0000 7f6ede000640 1 --2- 172.21.15.123:0/3892050755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387dace90 0x55c387db1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.917+0000 7f6edcc00640 1 -- 172.21.15.123:0/3892050755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 msgr2=0x55c387db0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:54.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.917+0000 7f6edcc00640 1 --2- 172.21.15.123:0/3892050755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387db0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.917+0000 7f6edcc00640 1 -- 172.21.15.123:0/3892050755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387dace90 msgr2=0x55c387db1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.917+0000 7f6edcc00640 1 --2- 172.21.15.123:0/3892050755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387dace90 0x55c387db1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.917+0000 7f6edcc00640 1 -- 172.21.15.123:0/3892050755 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c387e2f270 con 0x55c387da25d0 2024-09-19T04:54:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.917+0000 7f6edd600640 1 --2- 172.21.15.123:0/3892050755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387db0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:54:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.917+0000 7f6edcc00640 1 --2- 172.21.15.123:0/3892050755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387da25d0 0x55c387db1050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40cd2c0 tx=0x7f6ec40bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f48024f83ff9a644 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.917+0000 7f6ebea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:54:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.917+0000 7f6ebea00640 1 -- 172.21.15.123:0/3892050755 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec401c040 con 0x55c387da25d0 2024-09-19T04:54:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.917+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/3892050755 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ecc01e1e0 con 0x55c387da25d0 2024-09-19T04:54:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.918+0000 7f6ebea00640 1 -- 172.21.15.123:0/3892050755 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f6ec40cd540 con 0x55c387da25d0 2024-09-19T04:54:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.918+0000 7f6ebea00640 1 -- 172.21.15.123:0/3892050755 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6ec40cd740 con 0x55c387da25d0 2024-09-19T04:54:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.918+0000 7f6ee2a8f740 10 client.27098 fetch_fsmap learned FSMap version 43 2024-09-19T04:54:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.918+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/3892050755 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c387e30620 con 0x55c387da25d0 2024-09-19T04:54:54.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.918+0000 7f6ebea00640 1 -- 172.21.15.123:0/3892050755 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ecc01e1e0 con 0x55c387da25d0 2024-09-19T04:54:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.918+0000 7f6ebea00640 1 -- 172.21.15.123:0/3892050755 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f6ec4004bc0 con 0x55c387da25d0 2024-09-19T04:54:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.918+0000 7f6ee2a8f740 10 client.27098 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:54:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.918+0000 7f6ee2a8f740 10 client.27098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.918+0000 7f6ee2a8f740 20 client.27098 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.918+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/3892050755 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55c387e30ca0 0x55c387e330c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.918+0000 7f6ee2a8f740 4 client.27098 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:54:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.918+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/3892050755 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c387dbf520 con 0x55c387e30ca0 2024-09-19T04:54:54.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.918+0000 7f6edd600640 1 --2- 172.21.15.123:0/3892050755 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55c387e30ca0 0x55c387e330c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.919+0000 7f6edd600640 1 --2- 172.21.15.123:0/3892050755 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55c387e30ca0 0x55c387e330c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9869cf923402a2e server_cookie=c030904ae7e32aa9 in_seq=0 out_seq=0 2024-09-19T04:54:54.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.919+0000 7f6ebea00640 10 client.27098 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:54.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.920+0000 7f6ebea00640 1 -- 172.21.15.123:0/3892050755 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c387dbf520 con 0x55c387e30ca0 2024-09-19T04:54:54.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.920+0000 7f6ebea00640 10 client.27098 handle_command_reply: tid=0 2024-09-19T04:54:54.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.920+0000 7f6ee2a8f740 1 client.27098 shutdown 2024-09-19T04:54:54.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.920+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/3892050755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387da25d0 msgr2=0x55c387db1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:54.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.920+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/3892050755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387da25d0 0x55c387db1050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40cd2c0 tx=0x7f6ec40bc040 comp rx=0 tx=0).stop 2024-09-19T04:54:54.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.921+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/3892050755 shutdown_connections 2024-09-19T04:54:54.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.921+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/3892050755 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55c387e30ca0 0x55c387e330c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.921+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/3892050755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c387dace90 0x55c387db1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.921+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/3892050755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c387da25d0 0x55c387db1050 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.921+0000 7f6ee2a8f740 1 --2- 172.21.15.123:0/3892050755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c387da1cb0 0x55c387db0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.921+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/3892050755 >> 172.21.15.123:0/3892050755 conn(0x55c387d9e240 msgr2=0x55c387daba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:54.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.921+0000 7f6ebea00640 0 client.27098 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:54.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.921+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/3892050755 shutdown_connections 2024-09-19T04:54:54.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.921+0000 7f6ee2a8f740 1 -- 172.21.15.123:0/3892050755 wait complete. 2024-09-19T04:54:54.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.921+0000 7f6ee2a8f740 20 client.27098 trim_cache size 0 max 16384 2024-09-19T04:54:54.928 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:54:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-09-19T04:54:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "a21d2bc0-6912-46ea-8920-843d770ccf5d", 2024-09-19T04:54:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-09-19T04:54:54.928 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:54:54.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.922+0000 7f6edea00640 1 -- 172.21.15.123:0/1116424924 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ec00a3430 msgr2=0x7f6ec00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:54.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.922+0000 7f6edea00640 1 --2- 172.21.15.123:0/1116424924 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ec00a3430 0x7f6ec00a58f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55c387d9f0a0 tx=0x7f6ecc005f90 comp rx=0 tx=0).stop 2024-09-19T04:54:54.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.922+0000 7f6edea00640 1 -- 172.21.15.123:0/1116424924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed81047e0 msgr2=0x7f6ed819a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:54.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.922+0000 7f6edea00640 1 --2- 172.21.15.123:0/1116424924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed81047e0 0x7f6ed819a930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6ec400e9f0 tx=0x7f6ec400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.922+0000 7f6edea00640 1 -- 172.21.15.123:0/1116424924 shutdown_connections 2024-09-19T04:54:54.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.922+0000 7f6edea00640 1 --2- 172.21.15.123:0/1116424924 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ec00a3430 0x7f6ec00a58f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.922+0000 7f6edea00640 1 --2- 172.21.15.123:0/1116424924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ed8108f60 0x7f6ed819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.922+0000 7f6edea00640 1 --2- 172.21.15.123:0/1116424924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed81047e0 0x7f6ed819a930 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.922+0000 7f6edea00640 1 --2- 172.21.15.123:0/1116424924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed8103e30 0x7f6ed819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.922+0000 7f6edea00640 1 -- 172.21.15.123:0/1116424924 >> 172.21.15.123:0/1116424924 conn(0x7f6ed80ff8d0 msgr2=0x7f6ed8100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:54.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.922+0000 7f6edea00640 1 -- 172.21.15.123:0/1116424924 shutdown_connections 2024-09-19T04:54:54.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:54.922+0000 7f6edea00640 1 -- 172.21.15.123:0/1116424924 wait complete. 2024-09-19T04:54:55.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:55 smithi123 ceph-mon[30511]: pgmap v253: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:55.769 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:55 smithi123 ceph-mon[30511]: scrub queued for path: / 2024-09-19T04:54:55.769 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:55 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:55.770 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:55 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:54:55.770 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:55 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:55.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:55 smithi183 ceph-mon[35333]: pgmap v253: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:55.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:55 smithi183 ceph-mon[35333]: scrub queued for path: / 2024-09-19T04:54:55.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:55 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:55.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:55 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:54:55.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:55 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:55.854 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:54:55.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:55 smithi193 ceph-mon[35592]: pgmap v253: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:55.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:55 smithi193 ceph-mon[35592]: scrub queued for path: / 2024-09-19T04:54:55.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:55 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:55.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:55 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:54:55.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:55 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:54:56.262 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:54:56.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.810+0000 7f5e5fc00640 1 -- 172.21.15.123:0/2025905003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e58103ff0 msgr2=0x7f5e581043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:56.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.810+0000 7f5e5fc00640 1 --2- 172.21.15.123:0/2025905003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e58103ff0 0x7f5e581043d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009900 tx=0x7f5e4c02a410 comp rx=0 tx=0).stop 2024-09-19T04:54:56.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.810+0000 7f5e5fc00640 1 -- 172.21.15.123:0/2025905003 shutdown_connections 2024-09-19T04:54:56.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.810+0000 7f5e5fc00640 1 --2- 172.21.15.123:0/2025905003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e58109120 0x7f5e58110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.810+0000 7f5e5fc00640 1 --2- 172.21.15.123:0/2025905003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e581049a0 0x7f5e581089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.810+0000 7f5e5fc00640 1 --2- 172.21.15.123:0/2025905003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e58103ff0 0x7f5e581043d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.810+0000 7f5e5fc00640 1 -- 172.21.15.123:0/2025905003 >> 172.21.15.123:0/2025905003 conn(0x7f5e580ffa90 msgr2=0x7f5e58101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:56.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.810+0000 7f5e5fc00640 1 -- 172.21.15.123:0/2025905003 shutdown_connections 2024-09-19T04:54:56.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.810+0000 7f5e5fc00640 1 -- 172.21.15.123:0/2025905003 wait complete. 2024-09-19T04:54:56.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.812+0000 7f5e5fc00640 1 Processor -- start 2024-09-19T04:54:56.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.812+0000 7f5e5fc00640 1 -- start start 2024-09-19T04:54:56.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.812+0000 7f5e5fc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e58103ff0 0x7f5e5819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.812+0000 7f5e5fc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e581049a0 0x7f5e5819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.812+0000 7f5e5fc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e58109120 0x7f5e5819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.812+0000 7f5e5fc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5e58114120 con 0x7f5e58103ff0 2024-09-19T04:54:56.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.812+0000 7f5e5fc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5e58113fa0 con 0x7f5e58109120 2024-09-19T04:54:56.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.812+0000 7f5e5fc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5e581142a0 con 0x7f5e581049a0 2024-09-19T04:54:56.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.813+0000 7f5e5dc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e58109120 0x7f5e5819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.813+0000 7f5e5d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e58103ff0 0x7f5e5819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.813+0000 7f5e5dc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e58109120 0x7f5e5819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:52748/0 (socket says 172.21.15.123:52748) 2024-09-19T04:54:56.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.813+0000 7f5e5dc00640 1 -- 172.21.15.123:0/513479458 learned_addr learned my addr 172.21.15.123:0/513479458 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:56.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.813+0000 7f5e57e00640 1 --2- 172.21.15.123:0/513479458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e581049a0 0x7f5e5819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.813+0000 7f5e5dc00640 1 -- 172.21.15.123:0/513479458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e581049a0 msgr2=0x7f5e5819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:56.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.813+0000 7f5e5dc00640 1 --2- 172.21.15.123:0/513479458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e581049a0 0x7f5e5819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.813+0000 7f5e5dc00640 1 -- 172.21.15.123:0/513479458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e58103ff0 msgr2=0x7f5e5819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:56.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.813+0000 7f5e5dc00640 1 --2- 172.21.15.123:0/513479458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e58103ff0 0x7f5e5819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.813+0000 7f5e5dc00640 1 -- 172.21.15.123:0/513479458 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e5819f590 con 0x7f5e58109120 2024-09-19T04:54:56.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.813+0000 7f5e5dc00640 1 --2- 172.21.15.123:0/513479458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e58109120 0x7f5e5819eeb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000eb10 tx=0x7f5e5000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:56.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.814+0000 7f5e55600640 1 -- 172.21.15.123:0/513479458 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50004b20 con 0x7f5e58109120 2024-09-19T04:54:56.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.814+0000 7f5e5fc00640 1 -- 172.21.15.123:0/513479458 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e5819f880 con 0x7f5e58109120 2024-09-19T04:54:56.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.814+0000 7f5e55600640 1 -- 172.21.15.123:0/513479458 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e50004cc0 con 0x7f5e58109120 2024-09-19T04:54:56.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.814+0000 7f5e55600640 1 -- 172.21.15.123:0/513479458 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50010760 con 0x7f5e58109120 2024-09-19T04:54:56.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.814+0000 7f5e5fc00640 1 -- 172.21.15.123:0/513479458 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e581a7160 con 0x7f5e58109120 2024-09-19T04:54:56.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.817+0000 7f5e6266e740 1 Processor -- start 2024-09-19T04:54:56.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.818+0000 7f5e55600640 1 -- 172.21.15.123:0/513479458 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5e50005020 con 0x7f5e58109120 2024-09-19T04:54:56.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.818+0000 7f5e6266e740 1 -- start start 2024-09-19T04:54:56.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.818+0000 7f5e6266e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b24b90 0x563563b24f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.818+0000 7f5e6266e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b254b0 0x563563b2f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.818+0000 7f5e6266e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b2fd70 0x563563b32160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.818+0000 7f5e6266e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563563a8b3e0 con 0x563563b24b90 2024-09-19T04:54:56.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.818+0000 7f5e6266e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563563a32d80 con 0x563563b254b0 2024-09-19T04:54:56.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.818+0000 7f5e6266e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563563a873a0 con 0x563563b2fd70 2024-09-19T04:54:56.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.819+0000 7f5e57e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b24b90 0x563563b24f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.819+0000 7f5e5d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b254b0 0x563563b2f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.819+0000 7f5e57e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b24b90 0x563563b24f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47840/0 (socket says 172.21.15.123:47840) 2024-09-19T04:54:56.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.819+0000 7f5e57e00640 1 -- 172.21.15.123:0/2052489476 learned_addr learned my addr 172.21.15.123:0/2052489476 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:56.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.819+0000 7f5e57e00640 1 --2- 172.21.15.123:0/2052489476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b2fd70 0x563563b32160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.819+0000 7f5e5d200640 1 -- 172.21.15.123:0/2052489476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b2fd70 msgr2=0x563563b32160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:56.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.819+0000 7f5e5d200640 1 --2- 172.21.15.123:0/2052489476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b2fd70 0x563563b32160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.819+0000 7f5e5d200640 1 -- 172.21.15.123:0/2052489476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b24b90 msgr2=0x563563b24f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:56.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.819+0000 7f5e5d200640 1 --2- 172.21.15.123:0/2052489476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b24b90 0x563563b24f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.819+0000 7f5e5d200640 1 -- 172.21.15.123:0/2052489476 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563563a51ae0 con 0x563563b254b0 2024-09-19T04:54:56.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.819+0000 7f5e57e00640 1 --2- 172.21.15.123:0/2052489476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b2fd70 0x563563b32160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:54:56.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.820+0000 7f5e5d200640 1 --2- 172.21.15.123:0/2052489476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b254b0 0x563563b2f830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c005940 tx=0x7f5e4c005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c175791088ebc733 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:56.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.820+0000 7f5e55600640 1 --2- 172.21.15.123:0/513479458 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5e400a33b0 0x7f5e400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.820+0000 7f5e4a000640 1 -- 172.21.15.123:0/2052489476 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c040070 con 0x563563b254b0 2024-09-19T04:54:56.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.820+0000 7f5e4a000640 1 -- 172.21.15.123:0/2052489476 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e4c031870 con 0x563563b254b0 2024-09-19T04:54:56.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.820+0000 7f5e4a000640 1 -- 172.21.15.123:0/2052489476 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c035040 con 0x563563b254b0 2024-09-19T04:54:56.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.820+0000 7f5e55600640 1 -- 172.21.15.123:0/513479458 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f5e500d0210 con 0x7f5e58109120 2024-09-19T04:54:56.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.820+0000 7f5e57e00640 1 --2- 172.21.15.123:0/513479458 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5e400a33b0 0x7f5e400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.820+0000 7f5e6266e740 1 -- 172.21.15.123:0/2052489476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b254b0 msgr2=0x563563b2f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:56.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.820+0000 7f5e6266e740 1 --2- 172.21.15.123:0/2052489476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b254b0 0x563563b2f830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c005940 tx=0x7f5e4c005be0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.821+0000 7f5e6266e740 1 -- 172.21.15.123:0/2052489476 shutdown_connections 2024-09-19T04:54:56.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.821+0000 7f5e6266e740 1 --2- 172.21.15.123:0/2052489476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b2fd70 0x563563b32160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.821+0000 7f5e6266e740 1 --2- 172.21.15.123:0/2052489476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b254b0 0x563563b2f830 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.821+0000 7f5e6266e740 1 --2- 172.21.15.123:0/2052489476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b24b90 0x563563b24f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.821+0000 7f5e6266e740 1 -- 172.21.15.123:0/2052489476 >> 172.21.15.123:0/2052489476 conn(0x563563b21120 msgr2=0x563563b21500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:56.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.821+0000 7f5e6266e740 1 -- 172.21.15.123:0/2052489476 shutdown_connections 2024-09-19T04:54:56.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.821+0000 7f5e6266e740 1 -- 172.21.15.123:0/2052489476 wait complete. 2024-09-19T04:54:56.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.821+0000 7f5e57e00640 1 --2- 172.21.15.123:0/513479458 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5e400a33b0 0x7f5e400a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x563563b224a0 tx=0x7f5e44009850 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:56.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.821+0000 7f5e6266e740 1 -- start start 2024-09-19T04:54:56.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.821+0000 7f5e6266e740 1 Processor -- start 2024-09-19T04:54:56.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.822+0000 7f5e6266e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b24b90 0x563563b2cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.822+0000 7f5e6266e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b254b0 0x563563b2d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.822+0000 7f5e6266e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b2fd70 0x563563b2d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.822+0000 7f5e6266e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563563b329e0 con 0x563563b2fd70 2024-09-19T04:54:56.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.822+0000 7f5e6266e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563563a460f0 con 0x563563b24b90 2024-09-19T04:54:56.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.822+0000 7f5e6266e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563563b32b60 con 0x563563b254b0 2024-09-19T04:54:56.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.822+0000 7f5e6266e740 10 client.0 _ll_register_callbacks cb 0x563563a379e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:56.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.822+0000 7f5e5d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b24b90 0x563563b2cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.822+0000 7f5e5dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b2fd70 0x563563b2d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.822+0000 7f5e5d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b24b90 0x563563b2cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:52774/0 (socket says 172.21.15.123:52774) 2024-09-19T04:54:56.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.822+0000 7f5e5d200640 1 -- 172.21.15.123:0/1080691898 learned_addr learned my addr 172.21.15.123:0/1080691898 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:56.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.822+0000 7f5e57e00640 1 --2- 172.21.15.123:0/1080691898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b254b0 0x563563b2d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.823+0000 7f5e5d200640 1 -- 172.21.15.123:0/1080691898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b254b0 msgr2=0x563563b2d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:56.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.823+0000 7f5e5d200640 1 --2- 172.21.15.123:0/1080691898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b254b0 0x563563b2d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.823+0000 7f5e5d200640 1 -- 172.21.15.123:0/1080691898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b2fd70 msgr2=0x563563b2d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:56.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.823+0000 7f5e5d200640 1 --2- 172.21.15.123:0/1080691898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b2fd70 0x563563b2d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.823+0000 7f5e5d200640 1 -- 172.21.15.123:0/1080691898 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635637fc3c0 con 0x563563b24b90 2024-09-19T04:54:56.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.823+0000 7f5e5d200640 1 --2- 172.21.15.123:0/1080691898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b24b90 0x563563b2cc60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c005590 tx=0x7f5e4c005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bff674789b24c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:56.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.823+0000 7f5e3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:54:56.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.823+0000 7f5e3ea00640 1 -- 172.21.15.123:0/1080691898 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c040030 con 0x563563b24b90 2024-09-19T04:54:56.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.823+0000 7f5e6266e740 1 -- 172.21.15.123:0/1080691898 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e4c031870 con 0x563563b24b90 2024-09-19T04:54:56.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.823+0000 7f5e3ea00640 1 -- 172.21.15.123:0/1080691898 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f5e4c031670 con 0x563563b24b90 2024-09-19T04:54:56.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.824+0000 7f5e3ea00640 1 -- 172.21.15.123:0/1080691898 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5e4c035070 con 0x563563b24b90 2024-09-19T04:54:56.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.824+0000 7f5e6266e740 10 client.27110 fetch_fsmap learned FSMap version 43 2024-09-19T04:54:56.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.824+0000 7f5e6266e740 1 -- 172.21.15.123:0/1080691898 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563563b3e600 con 0x563563b24b90 2024-09-19T04:54:56.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.824+0000 7f5e3ea00640 1 -- 172.21.15.123:0/1080691898 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e4c031870 con 0x563563b24b90 2024-09-19T04:54:56.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.824+0000 7f5e3ea00640 1 -- 172.21.15.123:0/1080691898 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f5e4c0042d0 con 0x563563b24b90 2024-09-19T04:54:56.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.824+0000 7f5e6266e740 10 client.27110 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:54:56.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.824+0000 7f5e6266e740 10 client.27110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:56.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.824+0000 7f5e6266e740 20 client.27110 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:56.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.824+0000 7f5e6266e740 1 --2- 172.21.15.123:0/1080691898 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563563b3f950 0x563563b41d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.824+0000 7f5e6266e740 4 client.27110 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:54:56.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.824+0000 7f5e6266e740 1 -- 172.21.15.123:0/1080691898 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563563a98950 con 0x563563b3f950 2024-09-19T04:54:56.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.825+0000 7f5e57e00640 1 --2- 172.21.15.123:0/1080691898 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563563b3f950 0x563563b41d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.825+0000 7f5e57e00640 1 --2- 172.21.15.123:0/1080691898 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563563b3f950 0x563563b41d40 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f23bb9089513593 server_cookie=942c7291137bb472 in_seq=0 out_seq=0 2024-09-19T04:54:56.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.825+0000 7f5e3ea00640 10 client.27110 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:56.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.826+0000 7f5e3ea00640 1 -- 172.21.15.123:0/1080691898 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563563a98950 con 0x563563b3f950 2024-09-19T04:54:56.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.826+0000 7f5e3ea00640 10 client.27110 handle_command_reply: tid=0 2024-09-19T04:54:56.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.826+0000 7f5e6266e740 1 client.27110 shutdown 2024-09-19T04:54:56.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.827+0000 7f5e6266e740 1 -- 172.21.15.123:0/1080691898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b24b90 msgr2=0x563563b2cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:56.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.827+0000 7f5e6266e740 1 --2- 172.21.15.123:0/1080691898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b24b90 0x563563b2cc60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c005590 tx=0x7f5e4c005250 comp rx=0 tx=0).stop 2024-09-19T04:54:56.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.827+0000 7f5e6266e740 1 -- 172.21.15.123:0/1080691898 shutdown_connections 2024-09-19T04:54:56.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.827+0000 7f5e6266e740 1 --2- 172.21.15.123:0/1080691898 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563563b3f950 0x563563b41d40 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.827+0000 7f5e6266e740 1 --2- 172.21.15.123:0/1080691898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b2fd70 0x563563b2d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.827+0000 7f5e6266e740 1 --2- 172.21.15.123:0/1080691898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b254b0 0x563563b2d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.827+0000 7f5e3ea00640 0 client.27110 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:56.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.827+0000 7f5e6266e740 1 --2- 172.21.15.123:0/1080691898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b24b90 0x563563b2cc60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.827+0000 7f5e6266e740 1 -- 172.21.15.123:0/1080691898 >> 172.21.15.123:0/1080691898 conn(0x563563b21120 msgr2=0x563563b26e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:56.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.827+0000 7f5e6266e740 1 -- 172.21.15.123:0/1080691898 shutdown_connections 2024-09-19T04:54:56.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.827+0000 7f5e6266e740 1 -- 172.21.15.123:0/1080691898 wait complete. 2024-09-19T04:54:56.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.827+0000 7f5e6266e740 20 client.27110 trim_cache size 0 max 16384 2024-09-19T04:54:56.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.845+0000 7f5e6266e740 1 Processor -- start 2024-09-19T04:54:56.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.845+0000 7f5e6266e740 1 -- start start 2024-09-19T04:54:56.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.845+0000 7f5e6266e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b24b90 0x563563b275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.845+0000 7f5e6266e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b254b0 0x563563b47370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.845+0000 7f5e6266e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b2fd70 0x563563b478b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.845+0000 7f5e6266e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5e581afa10 con 0x563563b24b90 2024-09-19T04:54:56.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.845+0000 7f5e6266e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5e581b1550 con 0x563563b254b0 2024-09-19T04:54:56.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.845+0000 7f5e6266e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5e581b00e0 con 0x563563b2fd70 2024-09-19T04:54:56.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.845+0000 7f5e5d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b24b90 0x563563b275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.845+0000 7f5e5d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b24b90 0x563563b275f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47848/0 (socket says 172.21.15.123:47848) 2024-09-19T04:54:56.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.845+0000 7f5e5d200640 1 -- 172.21.15.123:0/1143389971 learned_addr learned my addr 172.21.15.123:0/1143389971 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:56.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.845+0000 7f5e57e00640 1 --2- 172.21.15.123:0/1143389971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b254b0 0x563563b47370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.845+0000 7f5e5dc00640 1 --2- 172.21.15.123:0/1143389971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b2fd70 0x563563b478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.846+0000 7f5e5d200640 1 -- 172.21.15.123:0/1143389971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b2fd70 msgr2=0x563563b478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:56.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.846+0000 7f5e5d200640 1 --2- 172.21.15.123:0/1143389971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b2fd70 0x563563b478b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.846+0000 7f5e5d200640 1 -- 172.21.15.123:0/1143389971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b254b0 msgr2=0x563563b47370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:56.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.846+0000 7f5e5d200640 1 --2- 172.21.15.123:0/1143389971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b254b0 0x563563b47370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.846+0000 7f5e5d200640 1 -- 172.21.15.123:0/1143389971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563563b222b0 con 0x563563b24b90 2024-09-19T04:54:56.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.846+0000 7f5e5dc00640 1 --2- 172.21.15.123:0/1143389971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b2fd70 0x563563b478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:54:56.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.846+0000 7f5e5d200640 1 --2- 172.21.15.123:0/1143389971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b24b90 0x563563b275f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c02a930 tx=0x7f5e4c005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4895893d8406a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:56.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.846+0000 7f5e49600640 1 -- 172.21.15.123:0/1143389971 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c040040 con 0x563563b24b90 2024-09-19T04:54:56.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.846+0000 7f5e49600640 1 -- 172.21.15.123:0/1143389971 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e4c004240 con 0x563563b24b90 2024-09-19T04:54:56.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.846+0000 7f5e49600640 1 -- 172.21.15.123:0/1143389971 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c0046c0 con 0x563563b24b90 2024-09-19T04:54:56.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.847+0000 7f5e6266e740 1 -- 172.21.15.123:0/1143389971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b24b90 msgr2=0x563563b275f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:56.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.847+0000 7f5e6266e740 1 --2- 172.21.15.123:0/1143389971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b24b90 0x563563b275f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c02a930 tx=0x7f5e4c005340 comp rx=0 tx=0).stop 2024-09-19T04:54:56.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.847+0000 7f5e6266e740 1 -- 172.21.15.123:0/1143389971 shutdown_connections 2024-09-19T04:54:56.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.847+0000 7f5e6266e740 1 --2- 172.21.15.123:0/1143389971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b2fd70 0x563563b478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.847+0000 7f5e6266e740 1 --2- 172.21.15.123:0/1143389971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b254b0 0x563563b47370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.847+0000 7f5e6266e740 1 --2- 172.21.15.123:0/1143389971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b24b90 0x563563b275f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.847+0000 7f5e6266e740 1 -- 172.21.15.123:0/1143389971 >> 172.21.15.123:0/1143389971 conn(0x563563b21120 msgr2=0x563563b2bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:56.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.847+0000 7f5e6266e740 1 -- 172.21.15.123:0/1143389971 shutdown_connections 2024-09-19T04:54:56.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.847+0000 7f5e6266e740 1 -- 172.21.15.123:0/1143389971 wait complete. 2024-09-19T04:54:56.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.847+0000 7f5e6266e740 1 -- start start 2024-09-19T04:54:56.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.847+0000 7f5e6266e740 1 Processor -- start 2024-09-19T04:54:56.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.848+0000 7f5e6266e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b24b90 0x563563b37f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.848+0000 7f5e6266e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b254b0 0x563563b384d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.848+0000 7f5e6266e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b2fd70 0x563563b294c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.848+0000 7f5e6266e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5e581b0e80 con 0x563563b254b0 2024-09-19T04:54:56.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.848+0000 7f5e6266e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5e581b07b0 con 0x563563b24b90 2024-09-19T04:54:56.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.848+0000 7f5e6266e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5e581ada10 con 0x563563b2fd70 2024-09-19T04:54:56.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.848+0000 7f5e6266e740 10 client.0 _ll_register_callbacks cb 0x7f5e4c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:54:56.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.848+0000 7f5e57e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b254b0 0x563563b384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.848+0000 7f5e57e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b254b0 0x563563b384d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47854/0 (socket says 172.21.15.123:47854) 2024-09-19T04:54:56.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.848+0000 7f5e57e00640 1 -- 172.21.15.123:0/3980468726 learned_addr learned my addr 172.21.15.123:0/3980468726 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:54:56.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.848+0000 7f5e5d200640 1 --2- 172.21.15.123:0/3980468726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b24b90 0x563563b37f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.848+0000 7f5e5dc00640 1 --2- 172.21.15.123:0/3980468726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b2fd70 0x563563b294c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.849+0000 7f5e57e00640 1 -- 172.21.15.123:0/3980468726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b2fd70 msgr2=0x563563b294c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:56.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.849+0000 7f5e57e00640 1 --2- 172.21.15.123:0/3980468726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b2fd70 0x563563b294c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.849+0000 7f5e57e00640 1 -- 172.21.15.123:0/3980468726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b24b90 msgr2=0x563563b37f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:54:56.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.849+0000 7f5e57e00640 1 --2- 172.21.15.123:0/3980468726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b24b90 0x563563b37f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.849+0000 7f5e57e00640 1 -- 172.21.15.123:0/3980468726 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563563b29b80 con 0x563563b254b0 2024-09-19T04:54:56.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.849+0000 7f5e57e00640 1 --2- 172.21.15.123:0/3980468726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b254b0 0x563563b384d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009fd0 tx=0x7f5e440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fedc36fe5c1099 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:56.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.849+0000 7f5e3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:54:56.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.849+0000 7f5e3ea00640 1 -- 172.21.15.123:0/3980468726 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4402c430 con 0x563563b254b0 2024-09-19T04:54:56.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.849+0000 7f5e3ea00640 1 -- 172.21.15.123:0/3980468726 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f5e44019030 con 0x563563b254b0 2024-09-19T04:54:56.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.849+0000 7f5e6266e740 1 -- 172.21.15.123:0/3980468726 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e4c004240 con 0x563563b254b0 2024-09-19T04:54:56.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.849+0000 7f5e3ea00640 1 -- 172.21.15.123:0/3980468726 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5e4402c5d0 con 0x563563b254b0 2024-09-19T04:54:56.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.850+0000 7f5e6266e740 10 client.17025 fetch_fsmap learned FSMap version 43 2024-09-19T04:54:56.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.850+0000 7f5e6266e740 1 -- 172.21.15.123:0/3980468726 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563563bb2570 con 0x563563b254b0 2024-09-19T04:54:56.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.850+0000 7f5e3ea00640 1 -- 172.21.15.123:0/3980468726 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e4c004240 con 0x563563b254b0 2024-09-19T04:54:56.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.850+0000 7f5e3ea00640 1 -- 172.21.15.123:0/3980468726 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f5e44034a20 con 0x563563b254b0 2024-09-19T04:54:56.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.850+0000 7f5e6266e740 10 client.17025 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:54:56.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.850+0000 7f5e6266e740 10 client.17025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:54:56.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.850+0000 7f5e6266e740 20 client.17025 populate_metadata read hostname 'smithi123' 2024-09-19T04:54:56.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.850+0000 7f5e6266e740 1 --2- 172.21.15.123:0/3980468726 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563563bb2d20 0x563563bb5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.850+0000 7f5e6266e740 4 client.17025 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:54:56.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.850+0000 7f5e6266e740 1 -- 172.21.15.123:0/3980468726 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563563a92e60 con 0x563563bb2d20 2024-09-19T04:54:56.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.850+0000 7f5e5d200640 1 --2- 172.21.15.123:0/3980468726 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563563bb2d20 0x563563bb5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.851+0000 7f5e5d200640 1 --2- 172.21.15.123:0/3980468726 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563563bb2d20 0x563563bb5140 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2fc6538c4c3e034 server_cookie=73d8013705113df8 in_seq=0 out_seq=0 2024-09-19T04:54:56.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.851+0000 7f5e3ea00640 10 client.17025 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:56.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.851+0000 7f5e3ea00640 1 -- 172.21.15.123:0/3980468726 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563563a92e60 con 0x563563bb2d20 2024-09-19T04:54:56.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.851+0000 7f5e3ea00640 10 client.17025 handle_command_reply: tid=0 2024-09-19T04:54:56.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.851+0000 7f5e6266e740 1 client.17025 shutdown 2024-09-19T04:54:56.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.851+0000 7f5e6266e740 1 -- 172.21.15.123:0/3980468726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b254b0 msgr2=0x563563b384d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:56.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.851+0000 7f5e6266e740 1 --2- 172.21.15.123:0/3980468726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b254b0 0x563563b384d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009fd0 tx=0x7f5e440046c0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.851+0000 7f5e6266e740 1 -- 172.21.15.123:0/3980468726 shutdown_connections 2024-09-19T04:54:56.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.851+0000 7f5e6266e740 1 --2- 172.21.15.123:0/3980468726 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563563bb2d20 0x563563bb5140 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.851+0000 7f5e6266e740 1 --2- 172.21.15.123:0/3980468726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563563b2fd70 0x563563b294c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.851+0000 7f5e6266e740 1 --2- 172.21.15.123:0/3980468726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563563b254b0 0x563563b384d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.851+0000 7f5e6266e740 1 --2- 172.21.15.123:0/3980468726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563563b24b90 0x563563b37f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.851+0000 7f5e6266e740 1 -- 172.21.15.123:0/3980468726 >> 172.21.15.123:0/3980468726 conn(0x563563b21120 msgr2=0x563563b1f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:56.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.852+0000 7f5e3ea00640 0 client.17025 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:54:56.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.852+0000 7f5e6266e740 1 -- 172.21.15.123:0/3980468726 shutdown_connections 2024-09-19T04:54:56.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.852+0000 7f5e6266e740 1 -- 172.21.15.123:0/3980468726 wait complete. 2024-09-19T04:54:56.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.852+0000 7f5e6266e740 20 client.17025 trim_cache size 0 max 16384 2024-09-19T04:54:56.857 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:54:56.857 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:54:56.857 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": { 2024-09-19T04:54:56.858 INFO:teuthology.orchestra.run.smithi123.stdout: "a21d2bc0-6912-46ea-8920-843d770ccf5d": { 2024-09-19T04:54:56.858 INFO:teuthology.orchestra.run.smithi123.stdout: "path": "/", 2024-09-19T04:54:56.858 INFO:teuthology.orchestra.run.smithi123.stdout: "tag": "a21d2bc0-6912-46ea-8920-843d770ccf5d", 2024-09-19T04:54:56.858 INFO:teuthology.orchestra.run.smithi123.stdout: "options": "recursive,force" 2024-09-19T04:54:56.858 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:54:56.858 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:54:56.858 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:54:56.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.852+0000 7f5e5fc00640 1 -- 172.21.15.123:0/513479458 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5e400a33b0 msgr2=0x7f5e400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:56.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.852+0000 7f5e5fc00640 1 --2- 172.21.15.123:0/513479458 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5e400a33b0 0x7f5e400a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x563563b224a0 tx=0x7f5e44009850 comp rx=0 tx=0).stop 2024-09-19T04:54:56.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.852+0000 7f5e5fc00640 1 -- 172.21.15.123:0/513479458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e58109120 msgr2=0x7f5e5819eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:56.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.852+0000 7f5e5fc00640 1 --2- 172.21.15.123:0/513479458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e58109120 0x7f5e5819eeb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000eb10 tx=0x7f5e5000c2d0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.853+0000 7f5e5fc00640 1 -- 172.21.15.123:0/513479458 shutdown_connections 2024-09-19T04:54:56.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.853+0000 7f5e5fc00640 1 --2- 172.21.15.123:0/513479458 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f5e400a33b0 0x7f5e400a5870 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.853+0000 7f5e5fc00640 1 --2- 172.21.15.123:0/513479458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e58109120 0x7f5e5819eeb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.853+0000 7f5e5fc00640 1 --2- 172.21.15.123:0/513479458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e581049a0 0x7f5e5819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.853+0000 7f5e5fc00640 1 --2- 172.21.15.123:0/513479458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e58103ff0 0x7f5e5819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.853+0000 7f5e5fc00640 1 -- 172.21.15.123:0/513479458 >> 172.21.15.123:0/513479458 conn(0x7f5e580ffa90 msgr2=0x7f5e58100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:56.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.853+0000 7f5e5fc00640 1 -- 172.21.15.123:0/513479458 shutdown_connections 2024-09-19T04:54:56.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:54:56.853+0000 7f5e5fc00640 1 -- 172.21.15.123:0/513479458 wait complete. 2024-09-19T04:54:57.710 INFO:tasks.cephfs.filesystem:scrub status for tag:a21d2bc0-6912-46ea-8920-843d770ccf5d - {'path': '/', 'tag': 'a21d2bc0-6912-46ea-8920-843d770ccf5d', 'options': 'recursive,force'} 2024-09-19T04:54:57.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:57 smithi183 ceph-mon[35333]: pgmap v254: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:54:57.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:57 smithi123 ceph-mon[30511]: pgmap v254: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:54:57.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:57 smithi193 ceph-mon[35592]: pgmap v254: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:54:59.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:59 smithi183 ceph-mon[35333]: pgmap v255: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:59.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:54:59 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:54:59.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:59 smithi123 ceph-mon[30511]: pgmap v255: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:59.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:54:59 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:54:59.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:59 smithi193 ceph-mon[35592]: pgmap v255: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:54:59.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:54:59 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:55:01.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:01 smithi183 ceph-mon[35333]: pgmap v256: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:55:01.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:01 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:55:01.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:01 smithi123 ceph-mon[30511]: pgmap v256: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:55:01.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:01 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:55:01.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:01 smithi193 ceph-mon[35592]: pgmap v256: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:55:01.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:01 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:55:02.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:02 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:55:02.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:02 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:55:02.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:02 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:55:02.928 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:02 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:55:02.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:02 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:55:02.929 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:02 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:55:02.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:02 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:55:02.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:02 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:55:02.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:02 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:55:04.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:03 smithi183 ceph-mon[35333]: pgmap v257: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:55:04.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:03 smithi123 ceph-mon[30511]: pgmap v257: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:55:04.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:03 smithi193 ceph-mon[35592]: pgmap v257: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:55:05.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:04 smithi183 ceph-mon[35333]: pgmap v258: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T04:55:05.177 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:04 smithi123 ceph-mon[30511]: pgmap v258: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T04:55:05.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:04 smithi193 ceph-mon[35592]: pgmap v258: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T04:55:07.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:06 smithi183 ceph-mon[35333]: pgmap v259: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T04:55:07.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:06 smithi123 ceph-mon[30511]: pgmap v259: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T04:55:07.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:06 smithi193 ceph-mon[35592]: pgmap v259: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T04:55:09.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:09 smithi123 ceph-mon[30511]: pgmap v260: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T04:55:09.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:09 smithi193 ceph-mon[35592]: pgmap v260: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T04:55:09.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:09 smithi183 ceph-mon[35333]: pgmap v260: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T04:55:11.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:11 smithi123 ceph-mon[30511]: pgmap v261: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T04:55:11.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:11 smithi123 ceph-mon[30511]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T04:55:11.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:11 smithi193 ceph-mon[35592]: pgmap v261: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T04:55:11.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:11 smithi193 ceph-mon[35592]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T04:55:11.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:11 smithi183 ceph-mon[35333]: pgmap v261: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T04:55:11.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:11 smithi183 ceph-mon[35333]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T04:55:13.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:13 smithi123 ceph-mon[30511]: pgmap v263: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T04:55:13.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:13 smithi193 ceph-mon[35592]: pgmap v263: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T04:55:13.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:13 smithi183 ceph-mon[35333]: pgmap v263: 97 pgs: 97 active+clean; 228 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T04:55:15.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:15 smithi123 ceph-mon[30511]: pgmap v264: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:55:15.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:15 smithi193 ceph-mon[35592]: pgmap v264: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:55:15.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:15 smithi183 ceph-mon[35333]: pgmap v264: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:55:17.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:17 smithi123 ceph-mon[30511]: pgmap v265: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:55:17.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:17 smithi123 ceph-mon[30511]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T04:55:17.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:17 smithi193 ceph-mon[35592]: pgmap v265: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:55:17.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:17 smithi193 ceph-mon[35592]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T04:55:17.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:17 smithi183 ceph-mon[35333]: pgmap v265: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:55:17.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:17 smithi183 ceph-mon[35333]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T04:55:19.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:19 smithi123 ceph-mon[30511]: pgmap v267: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T04:55:19.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:19 smithi193 ceph-mon[35592]: pgmap v267: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T04:55:19.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:19 smithi183 ceph-mon[35333]: pgmap v267: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T04:55:21.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:21 smithi123 ceph-mon[30511]: pgmap v268: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 208 B/s wr, 10 op/s 2024-09-19T04:55:21.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:21 smithi193 ceph-mon[35592]: pgmap v268: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 208 B/s wr, 10 op/s 2024-09-19T04:55:21.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:21 smithi183 ceph-mon[35333]: pgmap v268: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 208 B/s wr, 10 op/s 2024-09-19T04:55:23.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:23 smithi123 ceph-mon[30511]: pgmap v269: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:55:23.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:23 smithi193 ceph-mon[35592]: pgmap v269: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:55:23.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:23 smithi183 ceph-mon[35333]: pgmap v269: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:55:25.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:25 smithi123 ceph-mon[30511]: pgmap v270: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:55:25.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:25 smithi193 ceph-mon[35592]: pgmap v270: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:55:25.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:25 smithi183 ceph-mon[35333]: pgmap v270: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:55:27.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:27 smithi123 ceph-mon[30511]: pgmap v271: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:55:27.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:27 smithi193 ceph-mon[35592]: pgmap v271: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:55:27.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:27 smithi183 ceph-mon[35333]: pgmap v271: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:55:27.711 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:55:28.092 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:28.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.678+0000 7fd9f3200640 1 -- 172.21.15.123:0/3031428750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f41019c0 msgr2=0x7fd9f4101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:28.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.678+0000 7fd9f3200640 1 --2- 172.21.15.123:0/3031428750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f41019c0 0x7fd9f4101e20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009930 tx=0x7fd9e002a420 comp rx=0 tx=0).stop 2024-09-19T04:55:28.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.678+0000 7fd9f3200640 1 -- 172.21.15.123:0/3031428750 shutdown_connections 2024-09-19T04:55:28.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.678+0000 7fd9f3200640 1 --2- 172.21.15.123:0/3031428750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9f4102360 0x7fd9f410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.678+0000 7fd9f3200640 1 --2- 172.21.15.123:0/3031428750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f41019c0 0x7fd9f4101e20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.678+0000 7fd9f3200640 1 --2- 172.21.15.123:0/3031428750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9f41079c0 0x7fd9f4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.678+0000 7fd9f3200640 1 -- 172.21.15.123:0/3031428750 >> 172.21.15.123:0/3031428750 conn(0x7fd9f40fd740 msgr2=0x7fd9f40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:28.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.678+0000 7fd9f3200640 1 -- 172.21.15.123:0/3031428750 shutdown_connections 2024-09-19T04:55:28.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.678+0000 7fd9f3200640 1 -- 172.21.15.123:0/3031428750 wait complete. 2024-09-19T04:55:28.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.679+0000 7fd9f3200640 1 Processor -- start 2024-09-19T04:55:28.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.679+0000 7fd9f3200640 1 -- start start 2024-09-19T04:55:28.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.680+0000 7fd9f3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f41019c0 0x7fd9f419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.680+0000 7fd9f3200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9f4102360 0x7fd9f419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.680+0000 7fd9f3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9f41079c0 0x7fd9f419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.680+0000 7fd9f3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f4111dc0 con 0x7fd9f41019c0 2024-09-19T04:55:28.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.680+0000 7fd9f3200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f4111c40 con 0x7fd9f41079c0 2024-09-19T04:55:28.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.680+0000 7fd9f3200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f4111f40 con 0x7fd9f4102360 2024-09-19T04:55:28.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.680+0000 7fd9f1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f41019c0 0x7fd9f419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.680+0000 7fd9f1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f41019c0 0x7fd9f419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33288/0 (socket says 172.21.15.123:33288) 2024-09-19T04:55:28.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.680+0000 7fd9f1e00640 1 -- 172.21.15.123:0/2240637813 learned_addr learned my addr 172.21.15.123:0/2240637813 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:28.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.680+0000 7fd9f1400640 1 --2- 172.21.15.123:0/2240637813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9f4102360 0x7fd9f419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.680+0000 7fd9f2800640 1 --2- 172.21.15.123:0/2240637813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9f41079c0 0x7fd9f419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.681+0000 7fd9f1e00640 1 -- 172.21.15.123:0/2240637813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9f4102360 msgr2=0x7fd9f419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:28.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.681+0000 7fd9f1e00640 1 --2- 172.21.15.123:0/2240637813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9f4102360 0x7fd9f419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.681+0000 7fd9f1e00640 1 -- 172.21.15.123:0/2240637813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9f41079c0 msgr2=0x7fd9f419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:28.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.681+0000 7fd9f1e00640 1 --2- 172.21.15.123:0/2240637813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9f41079c0 0x7fd9f419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.681+0000 7fd9f1e00640 1 -- 172.21.15.123:0/2240637813 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f419f4d0 con 0x7fd9f41019c0 2024-09-19T04:55:28.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.681+0000 7fd9f1e00640 1 --2- 172.21.15.123:0/2240637813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f41019c0 0x7fd9f419a520 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd9e400e9f0 tx=0x7fd9e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:28.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.681+0000 7fd9eaa00640 1 -- 172.21.15.123:0/2240637813 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e400cd80 con 0x7fd9f41019c0 2024-09-19T04:55:28.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.681+0000 7fd9eaa00640 1 -- 172.21.15.123:0/2240637813 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9e4005500 con 0x7fd9f41019c0 2024-09-19T04:55:28.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.681+0000 7fd9eaa00640 1 -- 172.21.15.123:0/2240637813 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4010690 con 0x7fd9f41019c0 2024-09-19T04:55:28.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.681+0000 7fd9f3200640 1 -- 172.21.15.123:0/2240637813 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f419f7c0 con 0x7fd9f41019c0 2024-09-19T04:55:28.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.681+0000 7fd9f3200640 1 -- 172.21.15.123:0/2240637813 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f41a70a0 con 0x7fd9f41019c0 2024-09-19T04:55:28.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.685+0000 7fd9eaa00640 1 -- 172.21.15.123:0/2240637813 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd9e40046c0 con 0x7fd9f41019c0 2024-09-19T04:55:28.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.685+0000 7fd9fbc02740 1 Processor -- start 2024-09-19T04:55:28.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.685+0000 7fd9fbc02740 1 -- start start 2024-09-19T04:55:28.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.685+0000 7fd9fbc02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 0x56368010a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.685+0000 7fd9fbc02740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56368010a670 0x5636801149f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.685+0000 7fd9fbc02740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563680114f30 0x563680117320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.685+0000 7fd9fbc02740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5636800703e0 con 0x56368010a670 2024-09-19T04:55:28.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.685+0000 7fd9fbc02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563680017d80 con 0x563680109d50 2024-09-19T04:55:28.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.685+0000 7fd9fbc02740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56368006c3a0 con 0x563680114f30 2024-09-19T04:55:28.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.685+0000 7fd9f2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 0x56368010a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.686+0000 7fd9f2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 0x56368010a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:33076/0 (socket says 172.21.15.123:33076) 2024-09-19T04:55:28.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.686+0000 7fd9f2800640 1 -- 172.21.15.123:0/3222775015 learned_addr learned my addr 172.21.15.123:0/3222775015 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:28.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.686+0000 7fd9f1e00640 1 --2- 172.21.15.123:0/3222775015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56368010a670 0x5636801149f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.686+0000 7fd9f1400640 1 --2- 172.21.15.123:0/3222775015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563680114f30 0x563680117320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.686+0000 7fd9f2800640 1 -- 172.21.15.123:0/3222775015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563680114f30 msgr2=0x563680117320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:28.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.686+0000 7fd9f2800640 1 --2- 172.21.15.123:0/3222775015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563680114f30 0x563680117320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.686+0000 7fd9f2800640 1 -- 172.21.15.123:0/3222775015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56368010a670 msgr2=0x5636801149f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:28.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.686+0000 7fd9f2800640 1 --2- 172.21.15.123:0/3222775015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56368010a670 0x5636801149f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.686+0000 7fd9f2800640 1 -- 172.21.15.123:0/3222775015 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563680036ae0 con 0x563680109d50 2024-09-19T04:55:28.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.686+0000 7fd9f1e00640 1 --2- 172.21.15.123:0/3222775015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56368010a670 0x5636801149f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:55:28.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.686+0000 7fd9f2800640 1 --2- 172.21.15.123:0/3222775015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 0x56368010a130 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec010370 tx=0x7fd9ec010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d54b209f1dfda server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:28.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.686+0000 7fd9eaa00640 1 --2- 172.21.15.123:0/2240637813 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd9d80a3430 0x7fd9d80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.686+0000 7fd9df400640 1 -- 172.21.15.123:0/3222775015 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec016070 con 0x563680109d50 2024-09-19T04:55:28.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.687+0000 7fd9df400640 1 -- 172.21.15.123:0/3222775015 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ec004d20 con 0x563680109d50 2024-09-19T04:55:28.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.687+0000 7fd9df400640 1 -- 172.21.15.123:0/3222775015 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec005480 con 0x563680109d50 2024-09-19T04:55:28.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.687+0000 7fd9f1e00640 1 --2- 172.21.15.123:0/2240637813 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd9d80a3430 0x7fd9d80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.687+0000 7fd9eaa00640 1 -- 172.21.15.123:0/2240637813 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fd9e40d00d0 con 0x7fd9f41019c0 2024-09-19T04:55:28.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.687+0000 7fd9fbc02740 1 -- 172.21.15.123:0/3222775015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 msgr2=0x56368010a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:28.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.687+0000 7fd9f1e00640 1 --2- 172.21.15.123:0/2240637813 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd9d80a3430 0x7fd9d80a58f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x563680107140 tx=0x7fd9e40bc040 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:28.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.687+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/3222775015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 0x56368010a130 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec010370 tx=0x7fd9ec010830 comp rx=0 tx=0).stop 2024-09-19T04:55:28.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.688+0000 7fd9fbc02740 1 -- 172.21.15.123:0/3222775015 shutdown_connections 2024-09-19T04:55:28.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.688+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/3222775015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563680114f30 0x563680117320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.688+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/3222775015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56368010a670 0x5636801149f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.688+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/3222775015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 0x56368010a130 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.688+0000 7fd9fbc02740 1 -- 172.21.15.123:0/3222775015 >> 172.21.15.123:0/3222775015 conn(0x5636801062e0 msgr2=0x5636801066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:28.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.688+0000 7fd9fbc02740 1 -- 172.21.15.123:0/3222775015 shutdown_connections 2024-09-19T04:55:28.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.688+0000 7fd9fbc02740 1 -- 172.21.15.123:0/3222775015 wait complete. 2024-09-19T04:55:28.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.688+0000 7fd9fbc02740 1 -- start start 2024-09-19T04:55:28.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.688+0000 7fd9fbc02740 1 Processor -- start 2024-09-19T04:55:28.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.689+0000 7fd9fbc02740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563680109d50 0x563680111e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.689+0000 7fd9fbc02740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56368010a670 0x5636801123d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.689+0000 7fd9fbc02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680114f30 0x563680112910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.689+0000 7fd9fbc02740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563680117ba0 con 0x563680109d50 2024-09-19T04:55:28.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.689+0000 7fd9fbc02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56368002b0f0 con 0x563680114f30 2024-09-19T04:55:28.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.689+0000 7fd9fbc02740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563680117d20 con 0x56368010a670 2024-09-19T04:55:28.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.689+0000 7fd9fbc02740 10 client.0 _ll_register_callbacks cb 0x56368001c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:28.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.689+0000 7fd9f2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563680109d50 0x563680111e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.689+0000 7fd9f1400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56368010a670 0x5636801123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.689+0000 7fd9f2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563680109d50 0x563680111e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33304/0 (socket says 172.21.15.123:33304) 2024-09-19T04:55:28.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.689+0000 7fd9f2800640 1 -- 172.21.15.123:0/2608848712 learned_addr learned my addr 172.21.15.123:0/2608848712 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:28.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.689+0000 7fd9f2800640 1 --2- 172.21.15.123:0/2608848712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680114f30 0x563680112910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.689+0000 7fd9f2800640 1 -- 172.21.15.123:0/2608848712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56368010a670 msgr2=0x5636801123d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:28.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.689+0000 7fd9f2800640 1 --2- 172.21.15.123:0/2608848712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56368010a670 0x5636801123d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.689+0000 7fd9f2800640 1 -- 172.21.15.123:0/2608848712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680114f30 msgr2=0x563680112910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:28.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.689+0000 7fd9f2800640 1 --2- 172.21.15.123:0/2608848712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680114f30 0x563680112910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.689+0000 7fd9f2800640 1 -- 172.21.15.123:0/2608848712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56367fde13c0 con 0x563680109d50 2024-09-19T04:55:28.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.690+0000 7fd9f2800640 1 --2- 172.21.15.123:0/2608848712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563680109d50 0x563680111e90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec010fe0 tx=0x7fd9ec0050c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9471c1541ed24217 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:28.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.690+0000 7fd9d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:55:28.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.690+0000 7fd9d3e00640 1 -- 172.21.15.123:0/2608848712 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec016070 con 0x563680109d50 2024-09-19T04:55:28.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.690+0000 7fd9d3e00640 1 -- 172.21.15.123:0/2608848712 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fd9ec0053c0 con 0x563680109d50 2024-09-19T04:55:28.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.690+0000 7fd9fbc02740 1 -- 172.21.15.123:0/2608848712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9ec004d20 con 0x563680109d50 2024-09-19T04:55:28.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.690+0000 7fd9d3e00640 1 -- 172.21.15.123:0/2608848712 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd9ec0055c0 con 0x563680109d50 2024-09-19T04:55:28.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.690+0000 7fd9fbc02740 10 client.17037 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:28.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.690+0000 7fd9fbc02740 1 -- 172.21.15.123:0/2608848712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5636801238d0 con 0x563680109d50 2024-09-19T04:55:28.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.691+0000 7fd9d3e00640 1 -- 172.21.15.123:0/2608848712 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ec004d20 con 0x563680109d50 2024-09-19T04:55:28.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.691+0000 7fd9d3e00640 1 -- 172.21.15.123:0/2608848712 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fd9ec00f040 con 0x563680109d50 2024-09-19T04:55:28.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.691+0000 7fd9fbc02740 10 client.17037 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:28.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.691+0000 7fd9fbc02740 10 client.17037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:55:28.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.691+0000 7fd9fbc02740 20 client.17037 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:28.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.691+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/2608848712 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563680124ce0 0x5636801270d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.691+0000 7fd9fbc02740 4 client.17037 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:55:28.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.691+0000 7fd9fbc02740 1 -- 172.21.15.123:0/2608848712 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56368007d950 con 0x563680124ce0 2024-09-19T04:55:28.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.691+0000 7fd9f1400640 1 --2- 172.21.15.123:0/2608848712 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563680124ce0 0x5636801270d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.692+0000 7fd9f1400640 1 --2- 172.21.15.123:0/2608848712 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563680124ce0 0x5636801270d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=284c518033c4bba2 server_cookie=da8e67e30c56a14a in_seq=0 out_seq=0 2024-09-19T04:55:28.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.692+0000 7fd9d3e00640 10 client.17037 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:28.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.693+0000 7fd9d3e00640 1 -- 172.21.15.123:0/2608848712 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56368007d950 con 0x563680124ce0 2024-09-19T04:55:28.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.693+0000 7fd9d3e00640 10 client.17037 handle_command_reply: tid=0 2024-09-19T04:55:28.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.693+0000 7fd9fbc02740 1 client.17037 shutdown 2024-09-19T04:55:28.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.693+0000 7fd9fbc02740 1 -- 172.21.15.123:0/2608848712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563680109d50 msgr2=0x563680111e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:28.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.693+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/2608848712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563680109d50 0x563680111e90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec010fe0 tx=0x7fd9ec0050c0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.693+0000 7fd9fbc02740 1 -- 172.21.15.123:0/2608848712 shutdown_connections 2024-09-19T04:55:28.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.693+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/2608848712 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563680124ce0 0x5636801270d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.693+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/2608848712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680114f30 0x563680112910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.693+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/2608848712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56368010a670 0x5636801123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.693+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/2608848712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563680109d50 0x563680111e90 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.693+0000 7fd9d3e00640 0 client.17037 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:28.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.693+0000 7fd9fbc02740 1 -- 172.21.15.123:0/2608848712 >> 172.21.15.123:0/2608848712 conn(0x5636801062e0 msgr2=0x56368010c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:28.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.693+0000 7fd9fbc02740 1 -- 172.21.15.123:0/2608848712 shutdown_connections 2024-09-19T04:55:28.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.693+0000 7fd9fbc02740 1 -- 172.21.15.123:0/2608848712 wait complete. 2024-09-19T04:55:28.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.694+0000 7fd9fbc02740 20 client.17037 trim_cache size 0 max 16384 2024-09-19T04:55:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.711+0000 7fd9fbc02740 1 Processor -- start 2024-09-19T04:55:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.711+0000 7fd9fbc02740 1 -- start start 2024-09-19T04:55:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.711+0000 7fd9fbc02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 0x563680113e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.711+0000 7fd9fbc02740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56368010a670 0x56368012a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.711+0000 7fd9fbc02740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563680114f30 0x56368012a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.711+0000 7fd9fbc02740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f41af9c0 con 0x563680114f30 2024-09-19T04:55:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.711+0000 7fd9fbc02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f41b1500 con 0x563680109d50 2024-09-19T04:55:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.711+0000 7fd9fbc02740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f41b0090 con 0x56368010a670 2024-09-19T04:55:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.712+0000 7fd9f2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 0x563680113e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.712+0000 7fd9f1400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56368010a670 0x56368012a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.712+0000 7fd9f2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563680114f30 0x56368012a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.712+0000 7fd9f1400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56368010a670 0x56368012a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:54814/0 (socket says 172.21.15.123:54814) 2024-09-19T04:55:28.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.712+0000 7fd9f1400640 1 -- 172.21.15.123:0/643429477 learned_addr learned my addr 172.21.15.123:0/643429477 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:28.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.712+0000 7fd9f1400640 1 -- 172.21.15.123:0/643429477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 msgr2=0x563680113e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:28.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.712+0000 7fd9f1400640 1 --2- 172.21.15.123:0/643429477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 0x563680113e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.712+0000 7fd9f1400640 1 -- 172.21.15.123:0/643429477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563680114f30 msgr2=0x56368012a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:28.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.712+0000 7fd9f1400640 1 --2- 172.21.15.123:0/643429477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563680114f30 0x56368012a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.712+0000 7fd9f1400640 1 -- 172.21.15.123:0/643429477 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563680114800 con 0x56368010a670 2024-09-19T04:55:28.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.712+0000 7fd9f2800640 1 --2- 172.21.15.123:0/643429477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563680114f30 0x56368012a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:28.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.712+0000 7fd9f1400640 1 --2- 172.21.15.123:0/643429477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56368010a670 0x56368012a0f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009900 tx=0x7fd9e0005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3891eb1705a2719 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:28.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.713+0000 7fd9dea00640 1 -- 172.21.15.123:0/643429477 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e004c070 con 0x56368010a670 2024-09-19T04:55:28.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.713+0000 7fd9dea00640 1 -- 172.21.15.123:0/643429477 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9e0005020 con 0x56368010a670 2024-09-19T04:55:28.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.713+0000 7fd9dea00640 1 -- 172.21.15.123:0/643429477 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0035040 con 0x56368010a670 2024-09-19T04:55:28.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.713+0000 7fd9fbc02740 1 -- 172.21.15.123:0/643429477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56368010a670 msgr2=0x56368012a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:28.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.713+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/643429477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56368010a670 0x56368012a0f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009900 tx=0x7fd9e0005c00 comp rx=0 tx=0).stop 2024-09-19T04:55:28.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.713+0000 7fd9fbc02740 1 -- 172.21.15.123:0/643429477 shutdown_connections 2024-09-19T04:55:28.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.713+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/643429477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563680114f30 0x56368012a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.713+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/643429477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56368010a670 0x56368012a0f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.713+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/643429477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 0x563680113e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.713+0000 7fd9fbc02740 1 -- 172.21.15.123:0/643429477 >> 172.21.15.123:0/643429477 conn(0x5636801062e0 msgr2=0x5636801136f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:28.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.713+0000 7fd9fbc02740 1 -- 172.21.15.123:0/643429477 shutdown_connections 2024-09-19T04:55:28.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.713+0000 7fd9fbc02740 1 -- 172.21.15.123:0/643429477 wait complete. 2024-09-19T04:55:28.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.714+0000 7fd9fbc02740 1 -- start start 2024-09-19T04:55:28.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.714+0000 7fd9fbc02740 1 Processor -- start 2024-09-19T04:55:28.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.714+0000 7fd9fbc02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 0x56368012aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.714+0000 7fd9fbc02740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56368010a670 0x563680127d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.714+0000 7fd9fbc02740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563680114f30 0x563680128510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.714+0000 7fd9fbc02740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f41b0e30 con 0x56368010a670 2024-09-19T04:55:28.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.714+0000 7fd9fbc02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f41b0760 con 0x563680109d50 2024-09-19T04:55:28.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.714+0000 7fd9fbc02740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f41ad9c0 con 0x563680114f30 2024-09-19T04:55:28.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.714+0000 7fd9fbc02740 10 client.0 _ll_register_callbacks cb 0x7fd9ec004ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:28.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.715+0000 7fd9f1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56368010a670 0x563680127d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.715+0000 7fd9f2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 0x56368012aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.715+0000 7fd9f1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56368010a670 0x563680127d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33312/0 (socket says 172.21.15.123:33312) 2024-09-19T04:55:28.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.715+0000 7fd9f1400640 1 -- 172.21.15.123:0/1503007083 learned_addr learned my addr 172.21.15.123:0/1503007083 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:28.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.715+0000 7fd9f2800640 1 --2- 172.21.15.123:0/1503007083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563680114f30 0x563680128510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.715+0000 7fd9f1400640 1 -- 172.21.15.123:0/1503007083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563680114f30 msgr2=0x563680128510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:28.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.715+0000 7fd9f1400640 1 --2- 172.21.15.123:0/1503007083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563680114f30 0x563680128510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.715+0000 7fd9f1400640 1 -- 172.21.15.123:0/1503007083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 msgr2=0x56368012aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:28.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.715+0000 7fd9f1400640 1 --2- 172.21.15.123:0/1503007083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 0x56368012aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.715+0000 7fd9f1400640 1 -- 172.21.15.123:0/1503007083 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56368018e620 con 0x56368010a670 2024-09-19T04:55:28.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.715+0000 7fd9f2800640 1 --2- 172.21.15.123:0/1503007083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 0x56368012aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:28.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.715+0000 7fd9f1400640 1 --2- 172.21.15.123:0/1503007083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56368010a670 0x563680127d80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0000c00 tx=0x7fd9e002ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af52e00d169e2bc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:28.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.715+0000 7fd9d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:55:28.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.716+0000 7fd9d3e00640 1 -- 172.21.15.123:0/1503007083 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0035070 con 0x56368010a670 2024-09-19T04:55:28.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.716+0000 7fd9d3e00640 1 -- 172.21.15.123:0/1503007083 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fd9e0004770 con 0x56368010a670 2024-09-19T04:55:28.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.716+0000 7fd9fbc02740 1 -- 172.21.15.123:0/1503007083 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9e0005020 con 0x56368010a670 2024-09-19T04:55:28.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.716+0000 7fd9d3e00640 1 -- 172.21.15.123:0/1503007083 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd9e004c040 con 0x56368010a670 2024-09-19T04:55:28.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.716+0000 7fd9fbc02740 10 client.17043 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:28.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.716+0000 7fd9fbc02740 1 -- 172.21.15.123:0/1503007083 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56368018f810 con 0x56368010a670 2024-09-19T04:55:28.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.716+0000 7fd9d3e00640 1 -- 172.21.15.123:0/1503007083 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9e0005020 con 0x56368010a670 2024-09-19T04:55:28.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.716+0000 7fd9d3e00640 1 -- 172.21.15.123:0/1503007083 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fd9e0047810 con 0x56368010a670 2024-09-19T04:55:28.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.716+0000 7fd9fbc02740 10 client.17043 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:28.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.716+0000 7fd9fbc02740 10 client.17043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:55:28.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.716+0000 7fd9fbc02740 20 client.17043 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:28.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.716+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/1503007083 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56368018fe90 0x5636801922b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.717+0000 7fd9fbc02740 4 client.17043 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:55:28.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.717+0000 7fd9fbc02740 1 -- 172.21.15.123:0/1503007083 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563680077e60 con 0x56368018fe90 2024-09-19T04:55:28.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.717+0000 7fd9f2800640 1 --2- 172.21.15.123:0/1503007083 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56368018fe90 0x5636801922b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.717+0000 7fd9f2800640 1 --2- 172.21.15.123:0/1503007083 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56368018fe90 0x5636801922b0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35926fe8eb994e9a server_cookie=279a86148547eb59 in_seq=0 out_seq=0 2024-09-19T04:55:28.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.717+0000 7fd9d3e00640 10 client.17043 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:28.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.718+0000 7fd9d3e00640 1 -- 172.21.15.123:0/1503007083 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563680077e60 con 0x56368018fe90 2024-09-19T04:55:28.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.718+0000 7fd9d3e00640 10 client.17043 handle_command_reply: tid=0 2024-09-19T04:55:28.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.718+0000 7fd9fbc02740 1 client.17043 shutdown 2024-09-19T04:55:28.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.718+0000 7fd9fbc02740 1 -- 172.21.15.123:0/1503007083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56368010a670 msgr2=0x563680127d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:28.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.718+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/1503007083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56368010a670 0x563680127d80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0000c00 tx=0x7fd9e002ab60 comp rx=0 tx=0).stop 2024-09-19T04:55:28.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.718+0000 7fd9fbc02740 1 -- 172.21.15.123:0/1503007083 shutdown_connections 2024-09-19T04:55:28.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.718+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/1503007083 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56368018fe90 0x5636801922b0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.718+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/1503007083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563680114f30 0x563680128510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.718+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/1503007083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56368010a670 0x563680127d80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.718+0000 7fd9fbc02740 1 --2- 172.21.15.123:0/1503007083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563680109d50 0x56368012aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.718+0000 7fd9fbc02740 1 -- 172.21.15.123:0/1503007083 >> 172.21.15.123:0/1503007083 conn(0x5636801062e0 msgr2=0x563680105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:28.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.718+0000 7fd9d3e00640 0 client.17043 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:28.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.718+0000 7fd9fbc02740 1 -- 172.21.15.123:0/1503007083 shutdown_connections 2024-09-19T04:55:28.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.718+0000 7fd9fbc02740 1 -- 172.21.15.123:0/1503007083 wait complete. 2024-09-19T04:55:28.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.718+0000 7fd9fbc02740 20 client.17043 trim_cache size 0 max 16384 2024-09-19T04:55:28.725 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:55:28.725 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-09-19T04:55:28.725 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-09-19T04:55:28.725 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:55:28.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.719+0000 7fd9f3200640 1 -- 172.21.15.123:0/2240637813 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd9d80a3430 msgr2=0x7fd9d80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:28.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.719+0000 7fd9f3200640 1 --2- 172.21.15.123:0/2240637813 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd9d80a3430 0x7fd9d80a58f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x563680107140 tx=0x7fd9e40bc040 comp rx=0 tx=0).stop 2024-09-19T04:55:28.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.719+0000 7fd9f3200640 1 -- 172.21.15.123:0/2240637813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f41019c0 msgr2=0x7fd9f419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:28.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.719+0000 7fd9f3200640 1 --2- 172.21.15.123:0/2240637813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f41019c0 0x7fd9f419a520 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd9e400e9f0 tx=0x7fd9e400edf0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.719+0000 7fd9f3200640 1 -- 172.21.15.123:0/2240637813 shutdown_connections 2024-09-19T04:55:28.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.719+0000 7fd9f3200640 1 --2- 172.21.15.123:0/2240637813 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd9d80a3430 0x7fd9d80a58f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.719+0000 7fd9f3200640 1 --2- 172.21.15.123:0/2240637813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9f41079c0 0x7fd9f419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.719+0000 7fd9f3200640 1 --2- 172.21.15.123:0/2240637813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9f4102360 0x7fd9f419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.719+0000 7fd9f3200640 1 --2- 172.21.15.123:0/2240637813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f41019c0 0x7fd9f419a520 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.719+0000 7fd9f3200640 1 -- 172.21.15.123:0/2240637813 >> 172.21.15.123:0/2240637813 conn(0x7fd9f40fd740 msgr2=0x7fd9f410cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:28.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.719+0000 7fd9f3200640 1 -- 172.21.15.123:0/2240637813 shutdown_connections 2024-09-19T04:55:28.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:28.720+0000 7fd9f3200640 1 -- 172.21.15.123:0/2240637813 wait complete. 2024-09-19T04:55:29.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:29 smithi123 ceph-mon[30511]: pgmap v272: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 94 B/s wr, 14 op/s 2024-09-19T04:55:29.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:29 smithi193 ceph-mon[35592]: pgmap v272: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 94 B/s wr, 14 op/s 2024-09-19T04:55:29.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:29 smithi183 ceph-mon[35333]: pgmap v272: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 94 B/s wr, 14 op/s 2024-09-19T04:55:29.692 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:55:29.693 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:55:30.063 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:30.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.600+0000 7f4907600640 1 -- 172.21.15.123:0/2316774900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4900112b90 msgr2=0x7f4900114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:30.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.600+0000 7f4907600640 1 --2- 172.21.15.123:0/2316774900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4900112b90 0x7f4900114f80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009ca0 tx=0x7f48f802a950 comp rx=0 tx=0).stop 2024-09-19T04:55:30.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.601+0000 7f4907600640 1 -- 172.21.15.123:0/2316774900 shutdown_connections 2024-09-19T04:55:30.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.601+0000 7f4907600640 1 --2- 172.21.15.123:0/2316774900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4900112b90 0x7f4900114f80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:30.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.601+0000 7f4907600640 1 --2- 172.21.15.123:0/2316774900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4900074eb0 0x7f4900112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:30.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.601+0000 7f4907600640 1 --2- 172.21.15.123:0/2316774900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4900074590 0x7f4900074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:30.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.601+0000 7f4907600640 1 -- 172.21.15.123:0/2316774900 >> 172.21.15.123:0/2316774900 conn(0x7f49000fd8c0 msgr2=0x7f49000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:30.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.601+0000 7f4907600640 1 -- 172.21.15.123:0/2316774900 shutdown_connections 2024-09-19T04:55:30.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.601+0000 7f4907600640 1 -- 172.21.15.123:0/2316774900 wait complete. 2024-09-19T04:55:30.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.602+0000 7f4907600640 1 Processor -- start 2024-09-19T04:55:30.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.602+0000 7f4907600640 1 -- start start 2024-09-19T04:55:30.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.602+0000 7f4907600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4900074590 0x7f490019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:30.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.602+0000 7f4907600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4900074eb0 0x7f490019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:30.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.602+0000 7f4907600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4900112b90 0x7f49001a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:30.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.602+0000 7f4907600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f49001185a0 con 0x7f4900112b90 2024-09-19T04:55:30.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.602+0000 7f4907600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4900118420 con 0x7f4900074590 2024-09-19T04:55:30.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.602+0000 7f4907600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4900118720 con 0x7f4900074eb0 2024-09-19T04:55:30.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.602+0000 7f4904c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4900074590 0x7f490019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:30.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.603+0000 7f4904c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4900074590 0x7f490019eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:33104/0 (socket says 172.21.15.123:33104) 2024-09-19T04:55:30.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.603+0000 7f4904c00640 1 -- 172.21.15.123:0/2554198542 learned_addr learned my addr 172.21.15.123:0/2554198542 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:30.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.603+0000 7f4905600640 1 --2- 172.21.15.123:0/2554198542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4900112b90 0x7f49001a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:30.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.603+0000 7f48f7e00640 1 --2- 172.21.15.123:0/2554198542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4900074eb0 0x7f490019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:30.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.603+0000 7f4904c00640 1 -- 172.21.15.123:0/2554198542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4900074eb0 msgr2=0x7f490019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:30.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.603+0000 7f4904c00640 1 --2- 172.21.15.123:0/2554198542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4900074eb0 0x7f490019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:30.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.603+0000 7f4904c00640 1 -- 172.21.15.123:0/2554198542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4900112b90 msgr2=0x7f49001a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:30.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.603+0000 7f4904c00640 1 --2- 172.21.15.123:0/2554198542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4900112b90 0x7f49001a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:30.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.603+0000 7f4904c00640 1 -- 172.21.15.123:0/2554198542 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49001a3b70 con 0x7f4900074590 2024-09-19T04:55:30.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.603+0000 7f48f7e00640 1 --2- 172.21.15.123:0/2554198542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4900074eb0 0x7f490019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:30.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.603+0000 7f4905600640 1 --2- 172.21.15.123:0/2554198542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4900112b90 0x7f49001a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:30.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.603+0000 7f4904c00640 1 --2- 172.21.15.123:0/2554198542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4900074590 0x7f490019eb40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f48f000d950 tx=0x7f48f000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:30.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.604+0000 7f48fe000640 1 -- 172.21.15.123:0/2554198542 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0017030 con 0x7f4900074590 2024-09-19T04:55:30.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.604+0000 7f4907600640 1 -- 172.21.15.123:0/2554198542 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49001a3e60 con 0x7f4900074590 2024-09-19T04:55:30.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.604+0000 7f48fe000640 1 -- 172.21.15.123:0/2554198542 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f0004020 con 0x7f4900074590 2024-09-19T04:55:30.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.604+0000 7f48fe000640 1 -- 172.21.15.123:0/2554198542 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f00047a0 con 0x7f4900074590 2024-09-19T04:55:30.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.604+0000 7f4907600640 1 -- 172.21.15.123:0/2554198542 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49001ab6f0 con 0x7f4900074590 2024-09-19T04:55:30.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.605+0000 7f4907600640 1 -- 172.21.15.123:0/2554198542 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4900074970 con 0x7f4900074590 2024-09-19T04:55:30.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.607+0000 7f48fe000640 1 -- 172.21.15.123:0/2554198542 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f48f00041d0 con 0x7f4900074590 2024-09-19T04:55:30.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.608+0000 7f48fe000640 1 --2- 172.21.15.123:0/2554198542 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f48e40a3430 0x7f48e40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:30.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.608+0000 7f48fe000640 1 -- 172.21.15.123:0/2554198542 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f48f00cf630 con 0x7f4900074590 2024-09-19T04:55:30.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.608+0000 7f48f7e00640 1 --2- 172.21.15.123:0/2554198542 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f48e40a3430 0x7f48e40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:30.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.609+0000 7f48f7e00640 1 --2- 172.21.15.123:0/2554198542 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f48e40a3430 0x7f48e40a58f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f48e80095f0 tx=0x7f48e8005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:30.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.611+0000 7f48fe000640 1 -- 172.21.15.123:0/2554198542 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f48f009c030 con 0x7f4900074590 2024-09-19T04:55:30.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.823+0000 7f4907600640 1 -- 172.21.15.123:0/2554198542 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f490019fea0 con 0x7f4900074590 2024-09-19T04:55:30.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.824+0000 7f48fe000640 1 -- 172.21.15.123:0/2554198542 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11990 (secure 0 0 0) 0x7f48f009b930 con 0x7f4900074590 2024-09-19T04:55:30.826 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:55:30.826 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":43,"btime":"2024-09-19T04:54:44:530698+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:54:44.239974+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24529,24505,24539,24500,24499]},"id":1}]} 2024-09-19T04:55:30.826 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 43 2024-09-19T04:55:30.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.826+0000 7f4907600640 1 -- 172.21.15.123:0/2554198542 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f48e40a3430 msgr2=0x7f48e40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:30.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.826+0000 7f4907600640 1 --2- 172.21.15.123:0/2554198542 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f48e40a3430 0x7f48e40a58f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f48e80095f0 tx=0x7f48e8005f90 comp rx=0 tx=0).stop 2024-09-19T04:55:30.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.826+0000 7f4907600640 1 -- 172.21.15.123:0/2554198542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4900074590 msgr2=0x7f490019eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:30.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.826+0000 7f4907600640 1 --2- 172.21.15.123:0/2554198542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4900074590 0x7f490019eb40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f48f000d950 tx=0x7f48f000de10 comp rx=0 tx=0).stop 2024-09-19T04:55:30.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.827+0000 7f4907600640 1 -- 172.21.15.123:0/2554198542 shutdown_connections 2024-09-19T04:55:30.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.827+0000 7f4907600640 1 --2- 172.21.15.123:0/2554198542 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f48e40a3430 0x7f48e40a58f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:30.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.827+0000 7f4907600640 1 --2- 172.21.15.123:0/2554198542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4900112b90 0x7f49001a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:30.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.827+0000 7f4907600640 1 --2- 172.21.15.123:0/2554198542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4900074eb0 0x7f490019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:30.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.827+0000 7f4907600640 1 --2- 172.21.15.123:0/2554198542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4900074590 0x7f490019eb40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:30.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.827+0000 7f4907600640 1 -- 172.21.15.123:0/2554198542 >> 172.21.15.123:0/2554198542 conn(0x7f49000fd8c0 msgr2=0x7f49000ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:30.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.827+0000 7f4907600640 1 -- 172.21.15.123:0/2554198542 shutdown_connections 2024-09-19T04:55:30.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:30.827+0000 7f4907600640 1 -- 172.21.15.123:0/2554198542 wait complete. 2024-09-19T04:55:31.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:31 smithi123 ceph-mon[30511]: pgmap v273: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:31.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:31 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2554198542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:55:31.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:31 smithi193 ceph-mon[35592]: pgmap v273: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:31.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:31 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2554198542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:55:31.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:31 smithi183 ceph-mon[35333]: pgmap v273: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:31.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:31 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2554198542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:55:31.763 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:55:32.133 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:32.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.697+0000 7ffa0a600640 1 -- 172.21.15.123:0/3914503291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04112b90 msgr2=0x7ffa04114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:32.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.697+0000 7ffa0a600640 1 --2- 172.21.15.123:0/3914503291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04112b90 0x7ffa04114f80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009ca0 tx=0x7ff9fc02a950 comp rx=0 tx=0).stop 2024-09-19T04:55:32.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.697+0000 7ffa0a600640 1 -- 172.21.15.123:0/3914503291 shutdown_connections 2024-09-19T04:55:32.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.697+0000 7ffa0a600640 1 --2- 172.21.15.123:0/3914503291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04112b90 0x7ffa04114f80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.697+0000 7ffa0a600640 1 --2- 172.21.15.123:0/3914503291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa04074eb0 0x7ffa04112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.697+0000 7ffa0a600640 1 --2- 172.21.15.123:0/3914503291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa04074590 0x7ffa04074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.697+0000 7ffa0a600640 1 -- 172.21.15.123:0/3914503291 >> 172.21.15.123:0/3914503291 conn(0x7ffa040fd8c0 msgr2=0x7ffa040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:32.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.697+0000 7ffa0a600640 1 -- 172.21.15.123:0/3914503291 shutdown_connections 2024-09-19T04:55:32.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.698+0000 7ffa0a600640 1 -- 172.21.15.123:0/3914503291 wait complete. 2024-09-19T04:55:32.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.698+0000 7ffa0a600640 1 Processor -- start 2024-09-19T04:55:32.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.699+0000 7ffa0a600640 1 -- start start 2024-09-19T04:55:32.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.699+0000 7ffa0a600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa04074590 0x7ffa0419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.699+0000 7ffa0a600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04074eb0 0x7ffa0419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.699+0000 7ffa0a600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa04112b90 0x7ffa041a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.699+0000 7ffa0a600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041185c0 con 0x7ffa04074eb0 2024-09-19T04:55:32.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.699+0000 7ffa0a600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa04118440 con 0x7ffa04112b90 2024-09-19T04:55:32.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.699+0000 7ffa0a600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffa04118740 con 0x7ffa04074590 2024-09-19T04:55:32.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.699+0000 7ff9faa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04074eb0 0x7ffa0419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.699+0000 7ff9faa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04074eb0 0x7ffa0419f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58912/0 (socket says 172.21.15.123:58912) 2024-09-19T04:55:32.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.699+0000 7ff9faa00640 1 -- 172.21.15.123:0/2400702830 learned_addr learned my addr 172.21.15.123:0/2400702830 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:32.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.699+0000 7ffa03e00640 1 --2- 172.21.15.123:0/2400702830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa04112b90 0x7ffa041a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.699+0000 7ffa03400640 1 --2- 172.21.15.123:0/2400702830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa04074590 0x7ffa0419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.700+0000 7ff9faa00640 1 -- 172.21.15.123:0/2400702830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa04074590 msgr2=0x7ffa0419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:32.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.700+0000 7ff9faa00640 1 --2- 172.21.15.123:0/2400702830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa04074590 0x7ffa0419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.700+0000 7ff9faa00640 1 -- 172.21.15.123:0/2400702830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa04112b90 msgr2=0x7ffa041a3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:32.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.700+0000 7ff9faa00640 1 --2- 172.21.15.123:0/2400702830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa04112b90 0x7ffa041a3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.700+0000 7ff9faa00640 1 -- 172.21.15.123:0/2400702830 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa041a3b90 con 0x7ffa04074eb0 2024-09-19T04:55:32.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.700+0000 7ffa03400640 1 --2- 172.21.15.123:0/2400702830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa04074590 0x7ffa0419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:32.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.700+0000 7ff9faa00640 1 --2- 172.21.15.123:0/2400702830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04074eb0 0x7ffa0419f0a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00e9f0 tx=0x7ff9ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:32.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.700+0000 7ffa00c00640 1 -- 172.21.15.123:0/2400702830 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec00cd80 con 0x7ffa04074eb0 2024-09-19T04:55:32.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.700+0000 7ffa00c00640 1 -- 172.21.15.123:0/2400702830 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9ec005500 con 0x7ffa04074eb0 2024-09-19T04:55:32.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.700+0000 7ffa00c00640 1 -- 172.21.15.123:0/2400702830 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec010690 con 0x7ffa04074eb0 2024-09-19T04:55:32.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.700+0000 7ffa0a600640 1 -- 172.21.15.123:0/2400702830 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa041a3e80 con 0x7ffa04074eb0 2024-09-19T04:55:32.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.701+0000 7ffa0a600640 1 -- 172.21.15.123:0/2400702830 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa041ab710 con 0x7ffa04074eb0 2024-09-19T04:55:32.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.704+0000 7ffa0d1e8740 1 Processor -- start 2024-09-19T04:55:32.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.704+0000 7ffa0d1e8740 1 -- start start 2024-09-19T04:55:32.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.704+0000 7ffa00c00640 1 -- 172.21.15.123:0/2400702830 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff9ec0046c0 con 0x7ffa04074eb0 2024-09-19T04:55:32.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.704+0000 7ffa0d1e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4ba050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.704+0000 7ffa0d1e8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4ba590 0x56127d4c4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.704+0000 7ffa0d1e8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4c4e50 0x56127d4c7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.704+0000 7ffa0d1e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56127d4203e0 con 0x56127d4b9c70 2024-09-19T04:55:32.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.704+0000 7ffa0d1e8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56127d3c7d80 con 0x56127d4ba590 2024-09-19T04:55:32.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.704+0000 7ffa0d1e8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56127d41c3a0 con 0x56127d4c4e50 2024-09-19T04:55:32.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.705+0000 7ffa03e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4ba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.705+0000 7ffa03400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4ba590 0x56127d4c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.705+0000 7ffa03e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4ba050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58916/0 (socket says 172.21.15.123:58916) 2024-09-19T04:55:32.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.705+0000 7ffa03e00640 1 -- 172.21.15.123:0/1522185801 learned_addr learned my addr 172.21.15.123:0/1522185801 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:32.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.705+0000 7ff9faa00640 1 --2- 172.21.15.123:0/1522185801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4c4e50 0x56127d4c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.705+0000 7ffa03400640 1 -- 172.21.15.123:0/1522185801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4c4e50 msgr2=0x56127d4c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:32.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.705+0000 7ffa03400640 1 --2- 172.21.15.123:0/1522185801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4c4e50 0x56127d4c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.705+0000 7ffa03400640 1 -- 172.21.15.123:0/1522185801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 msgr2=0x56127d4ba050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:32.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.705+0000 7ffa03400640 1 --2- 172.21.15.123:0/1522185801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4ba050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.705+0000 7ffa03400640 1 -- 172.21.15.123:0/1522185801 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56127d3e6ae0 con 0x56127d4ba590 2024-09-19T04:55:32.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.705+0000 7ffa03e00640 1 --2- 172.21.15.123:0/1522185801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4ba050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:32.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.706+0000 7ffa03400640 1 --2- 172.21.15.123:0/1522185801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4ba590 0x56127d4c4910 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009fd0 tx=0x7ff9f40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a05c7171ab2f41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:32.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.706+0000 7ffa00c00640 1 --2- 172.21.15.123:0/2400702830 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff9e80a3430 0x7ff9e80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.706+0000 7ff9f8c00640 1 -- 172.21.15.123:0/1522185801 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4004030 con 0x56127d4ba590 2024-09-19T04:55:32.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.706+0000 7ff9f8c00640 1 -- 172.21.15.123:0/1522185801 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9f4005500 con 0x56127d4ba590 2024-09-19T04:55:32.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.706+0000 7ff9f8c00640 1 -- 172.21.15.123:0/1522185801 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4004da0 con 0x56127d4ba590 2024-09-19T04:55:32.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.706+0000 7ffa00c00640 1 -- 172.21.15.123:0/2400702830 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7ff9ec0d00d0 con 0x7ffa04074eb0 2024-09-19T04:55:32.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.706+0000 7ffa03e00640 1 --2- 172.21.15.123:0/2400702830 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff9e80a3430 0x7ff9e80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.707+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/1522185801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4ba590 msgr2=0x56127d4c4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:32.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.707+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/1522185801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4ba590 0x56127d4c4910 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009fd0 tx=0x7ff9f40118d0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.707+0000 7ffa03e00640 1 --2- 172.21.15.123:0/2400702830 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff9e80a3430 0x7ff9e80a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009ca0 tx=0x7ff9fc005d80 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:32.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.707+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/1522185801 shutdown_connections 2024-09-19T04:55:32.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.707+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/1522185801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4c4e50 0x56127d4c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.707+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/1522185801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4ba590 0x56127d4c4910 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.707+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/1522185801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4ba050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.707+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/1522185801 >> 172.21.15.123:0/1522185801 conn(0x56127d4b6200 msgr2=0x56127d4b65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:32.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.707+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/1522185801 shutdown_connections 2024-09-19T04:55:32.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.707+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/1522185801 wait complete. 2024-09-19T04:55:32.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.708+0000 7ffa0d1e8740 1 -- start start 2024-09-19T04:55:32.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.708+0000 7ffa0d1e8740 1 Processor -- start 2024-09-19T04:55:32.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.708+0000 7ffa0d1e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4c1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.708+0000 7ffa0d1e8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4ba590 0x56127d4c21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.708+0000 7ffa0d1e8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4c4e50 0x56127d4c2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.708+0000 7ffa0d1e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56127d4c7a20 con 0x56127d4b9c70 2024-09-19T04:55:32.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.708+0000 7ffa0d1e8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56127d3db0f0 con 0x56127d4ba590 2024-09-19T04:55:32.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.708+0000 7ffa0d1e8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56127d4c7ba0 con 0x56127d4c4e50 2024-09-19T04:55:32.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.708+0000 7ffa0d1e8740 10 client.0 _ll_register_callbacks cb 0x56127d3c2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:32.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.709+0000 7ffa03400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4c1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.709+0000 7ffa03400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4c1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58918/0 (socket says 172.21.15.123:58918) 2024-09-19T04:55:32.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.709+0000 7ffa03400640 1 -- 172.21.15.123:0/630611128 learned_addr learned my addr 172.21.15.123:0/630611128 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:32.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.709+0000 7ff9faa00640 1 --2- 172.21.15.123:0/630611128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4ba590 0x56127d4c21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.709+0000 7ffa03e00640 1 --2- 172.21.15.123:0/630611128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4c4e50 0x56127d4c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.709+0000 7ffa03400640 1 -- 172.21.15.123:0/630611128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4c4e50 msgr2=0x56127d4c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:32.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.709+0000 7ffa03400640 1 --2- 172.21.15.123:0/630611128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4c4e50 0x56127d4c2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.709+0000 7ffa03400640 1 -- 172.21.15.123:0/630611128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4ba590 msgr2=0x56127d4c21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:32.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.709+0000 7ffa03400640 1 --2- 172.21.15.123:0/630611128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4ba590 0x56127d4c21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.709+0000 7ffa03400640 1 -- 172.21.15.123:0/630611128 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56127d1913c0 con 0x56127d4b9c70 2024-09-19T04:55:32.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.709+0000 7ffa03400640 1 --2- 172.21.15.123:0/630611128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4c1cb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff9f401b040 tx=0x7ff9f4005140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2dccbbd301866e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:32.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.709+0000 7ff9f1600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:55:32.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.710+0000 7ff9f1600640 1 -- 172.21.15.123:0/630611128 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4028070 con 0x56127d4b9c70 2024-09-19T04:55:32.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.710+0000 7ff9f1600640 1 -- 172.21.15.123:0/630611128 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7ff9f400f5a0 con 0x56127d4b9c70 2024-09-19T04:55:32.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.710+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/630611128 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9f4005500 con 0x56127d4b9c70 2024-09-19T04:55:32.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.710+0000 7ff9f1600640 1 -- 172.21.15.123:0/630611128 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff9f4034050 con 0x56127d4b9c70 2024-09-19T04:55:32.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.710+0000 7ffa0d1e8740 10 client.17061 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:32.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.710+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/630611128 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56127d4d3a10 con 0x56127d4b9c70 2024-09-19T04:55:32.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.710+0000 7ff9f1600640 1 -- 172.21.15.123:0/630611128 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9f4005500 con 0x56127d4b9c70 2024-09-19T04:55:32.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.711+0000 7ff9f1600640 1 -- 172.21.15.123:0/630611128 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7ff9f4019660 con 0x56127d4b9c70 2024-09-19T04:55:32.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.711+0000 7ffa0d1e8740 10 client.17061 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:32.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.711+0000 7ffa0d1e8740 10 client.17061 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:55:32.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.711+0000 7ffa0d1e8740 20 client.17061 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:32.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.711+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/630611128 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x56127d4d4d60 0x56127d4d7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.711+0000 7ffa0d1e8740 4 client.17061 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:55:32.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.711+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/630611128 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56127d42d950 con 0x56127d4d4d60 2024-09-19T04:55:32.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.711+0000 7ff9faa00640 1 --2- 172.21.15.123:0/630611128 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x56127d4d4d60 0x56127d4d7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.712+0000 7ff9faa00640 1 --2- 172.21.15.123:0/630611128 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x56127d4d4d60 0x56127d4d7150 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1f247079a61752dd server_cookie=e1ba932b3941451b in_seq=0 out_seq=0 2024-09-19T04:55:32.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.712+0000 7ff9f1600640 10 client.17061 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:55:32.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.713+0000 7ff9f1600640 1 -- 172.21.15.123:0/630611128 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56127d42d950 con 0x56127d4d4d60 2024-09-19T04:55:32.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.713+0000 7ff9f1600640 10 client.17061 handle_command_reply: tid=0 2024-09-19T04:55:32.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.713+0000 7ffa0d1e8740 1 client.17061 shutdown 2024-09-19T04:55:32.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.713+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/630611128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 msgr2=0x56127d4c1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:32.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.713+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/630611128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4c1cb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff9f401b040 tx=0x7ff9f4005140 comp rx=0 tx=0).stop 2024-09-19T04:55:32.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.713+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/630611128 shutdown_connections 2024-09-19T04:55:32.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.713+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/630611128 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x56127d4d4d60 0x56127d4d7150 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.713+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/630611128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4c4e50 0x56127d4c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.713+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/630611128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4ba590 0x56127d4c21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.713+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/630611128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4c1cb0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.713+0000 7ff9f1600640 0 client.17061 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:55:32.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.713+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/630611128 >> 172.21.15.123:0/630611128 conn(0x56127d4b6200 msgr2=0x56127d4b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:32.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.713+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/630611128 shutdown_connections 2024-09-19T04:55:32.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.713+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/630611128 wait complete. 2024-09-19T04:55:32.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.713+0000 7ffa0d1e8740 20 client.17061 trim_cache size 0 max 16384 2024-09-19T04:55:32.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.727+0000 7ffa0d1e8740 1 Processor -- start 2024-09-19T04:55:32.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.727+0000 7ffa0d1e8740 1 -- start start 2024-09-19T04:55:32.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.727+0000 7ffa0d1e8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4b9c70 0x56127d4dc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.727+0000 7ffa0d1e8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4ba590 0x56127d4dce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.727+0000 7ffa0d1e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4c4e50 0x56127d4b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.727+0000 7ffa0d1e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041b3f80 con 0x56127d4c4e50 2024-09-19T04:55:32.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.727+0000 7ffa0d1e8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041b5ac0 con 0x56127d4b9c70 2024-09-19T04:55:32.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.727+0000 7ffa0d1e8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041b4650 con 0x56127d4ba590 2024-09-19T04:55:32.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.728+0000 7ffa03e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4c4e50 0x56127d4b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.728+0000 7ffa03e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4c4e50 0x56127d4b4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58922/0 (socket says 172.21.15.123:58922) 2024-09-19T04:55:32.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.728+0000 7ffa03e00640 1 -- 172.21.15.123:0/2187386928 learned_addr learned my addr 172.21.15.123:0/2187386928 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:32.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.728+0000 7ffa03400640 1 --2- 172.21.15.123:0/2187386928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4b9c70 0x56127d4dc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.728+0000 7ff9faa00640 1 --2- 172.21.15.123:0/2187386928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4ba590 0x56127d4dce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.728+0000 7ffa03e00640 1 -- 172.21.15.123:0/2187386928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4ba590 msgr2=0x56127d4dce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:32.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.728+0000 7ffa03e00640 1 --2- 172.21.15.123:0/2187386928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4ba590 0x56127d4dce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.728+0000 7ffa03e00640 1 -- 172.21.15.123:0/2187386928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4b9c70 msgr2=0x56127d4dc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:32.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.728+0000 7ffa03e00640 1 --2- 172.21.15.123:0/2187386928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4b9c70 0x56127d4dc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.728+0000 7ffa03e00640 1 -- 172.21.15.123:0/2187386928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56127d4b6030 con 0x56127d4c4e50 2024-09-19T04:55:32.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.728+0000 7ffa03400640 1 --2- 172.21.15.123:0/2187386928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4b9c70 0x56127d4dc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:32.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.728+0000 7ffa03e00640 1 --2- 172.21.15.123:0/2187386928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4c4e50 0x56127d4b4b60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffa04076600 tx=0x7ff9fc042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3976a75353ecd6e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:32.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.729+0000 7ff9f3e00640 1 -- 172.21.15.123:0/2187386928 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc034040 con 0x56127d4c4e50 2024-09-19T04:55:32.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.729+0000 7ff9f3e00640 1 -- 172.21.15.123:0/2187386928 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9fc02fcc0 con 0x56127d4c4e50 2024-09-19T04:55:32.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.729+0000 7ff9f3e00640 1 -- 172.21.15.123:0/2187386928 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0308e0 con 0x56127d4c4e50 2024-09-19T04:55:32.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.729+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/2187386928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4c4e50 msgr2=0x56127d4b4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:32.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.729+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/2187386928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4c4e50 0x56127d4b4b60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffa04076600 tx=0x7ff9fc042000 comp rx=0 tx=0).stop 2024-09-19T04:55:32.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.729+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/2187386928 shutdown_connections 2024-09-19T04:55:32.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.729+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/2187386928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4c4e50 0x56127d4b4b60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.729+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/2187386928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4ba590 0x56127d4dce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.729+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/2187386928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4b9c70 0x56127d4dc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.729+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/2187386928 >> 172.21.15.123:0/2187386928 conn(0x56127d4b6200 msgr2=0x56127d4c3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:32.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.730+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/2187386928 shutdown_connections 2024-09-19T04:55:32.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.730+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/2187386928 wait complete. 2024-09-19T04:55:32.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.730+0000 7ffa0d1e8740 1 -- start start 2024-09-19T04:55:32.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.730+0000 7ffa0d1e8740 1 Processor -- start 2024-09-19T04:55:32.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.731+0000 7ffa0d1e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.731+0000 7ffa0d1e8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4ba590 0x56127d4c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.731+0000 7ffa0d1e8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4c4e50 0x56127d4c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.731+0000 7ffa0d1e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041b53f0 con 0x56127d4b9c70 2024-09-19T04:55:32.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.731+0000 7ffa0d1e8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041b4d20 con 0x56127d4c4e50 2024-09-19T04:55:32.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.731+0000 7ffa0d1e8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041b1f80 con 0x56127d4ba590 2024-09-19T04:55:32.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.731+0000 7ffa0d1e8740 10 client.0 _ll_register_callbacks cb 0x7ff9ec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:32.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.731+0000 7ffa03400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.731+0000 7ffa03400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58932/0 (socket says 172.21.15.123:58932) 2024-09-19T04:55:32.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.731+0000 7ffa03400640 1 -- 172.21.15.123:0/1697678051 learned_addr learned my addr 172.21.15.123:0/1697678051 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:32.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.731+0000 7ff9faa00640 1 --2- 172.21.15.123:0/1697678051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4ba590 0x56127d4c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.731+0000 7ffa03e00640 1 --2- 172.21.15.123:0/1697678051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4c4e50 0x56127d4c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.731+0000 7ffa03400640 1 -- 172.21.15.123:0/1697678051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4ba590 msgr2=0x56127d4c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:32.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.731+0000 7ffa03400640 1 --2- 172.21.15.123:0/1697678051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4ba590 0x56127d4c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.732+0000 7ffa03400640 1 -- 172.21.15.123:0/1697678051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4c4e50 msgr2=0x56127d4c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:32.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.732+0000 7ffa03400640 1 --2- 172.21.15.123:0/1697678051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4c4e50 0x56127d4c8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.732+0000 7ffa03400640 1 -- 172.21.15.123:0/1697678051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56127d4b1770 con 0x56127d4b9c70 2024-09-19T04:55:32.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.732+0000 7ffa03e00640 1 --2- 172.21.15.123:0/1697678051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4c4e50 0x56127d4c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:55:32.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.732+0000 7ffa03400640 1 --2- 172.21.15.123:0/1697678051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4cd560 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4019ab0 tx=0x7ff9f4004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666edcf3d71e2a33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:32.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.732+0000 7ff9f1600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:55:32.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.732+0000 7ff9f1600640 1 -- 172.21.15.123:0/1697678051 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4028040 con 0x56127d4b9c70 2024-09-19T04:55:32.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.732+0000 7ff9f1600640 1 -- 172.21.15.123:0/1697678051 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7ff9f4019660 con 0x56127d4b9c70 2024-09-19T04:55:32.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.732+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/1697678051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9fc02fcc0 con 0x56127d4b9c70 2024-09-19T04:55:32.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.732+0000 7ff9f1600640 1 -- 172.21.15.123:0/1697678051 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff9f4004030 con 0x56127d4b9c70 2024-09-19T04:55:32.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.732+0000 7ffa0d1e8740 10 client.17067 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:32.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.732+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/1697678051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56127d53f750 con 0x56127d4b9c70 2024-09-19T04:55:32.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.733+0000 7ff9f1600640 1 -- 172.21.15.123:0/1697678051 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9fc02fcc0 con 0x56127d4b9c70 2024-09-19T04:55:32.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.733+0000 7ff9f1600640 1 -- 172.21.15.123:0/1697678051 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7ff9f40035f0 con 0x56127d4b9c70 2024-09-19T04:55:32.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.733+0000 7ffa0d1e8740 10 client.17067 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:32.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.733+0000 7ffa0d1e8740 10 client.17067 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:55:32.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.733+0000 7ffa0d1e8740 20 client.17067 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:32.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.733+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/1697678051 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x56127d53fe10 0x56127d542230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:32.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.733+0000 7ffa0d1e8740 4 client.17067 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:55:32.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.733+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/1697678051 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56127d427e60 con 0x56127d53fe10 2024-09-19T04:55:32.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.734+0000 7ff9faa00640 1 --2- 172.21.15.123:0/1697678051 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x56127d53fe10 0x56127d542230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:32.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.734+0000 7ff9faa00640 1 --2- 172.21.15.123:0/1697678051 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x56127d53fe10 0x56127d542230 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4963f72bde624899 server_cookie=c0a223fda74c75dc in_seq=0 out_seq=0 2024-09-19T04:55:32.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.734+0000 7ff9f1600640 10 client.17067 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:55:32.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.734+0000 7ff9f1600640 1 -- 172.21.15.123:0/1697678051 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56127d427e60 con 0x56127d53fe10 2024-09-19T04:55:32.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.734+0000 7ff9f1600640 10 client.17067 handle_command_reply: tid=0 2024-09-19T04:55:32.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.734+0000 7ffa0d1e8740 1 client.17067 shutdown 2024-09-19T04:55:32.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.734+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/1697678051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 msgr2=0x56127d4cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:32.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.734+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/1697678051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4cd560 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4019ab0 tx=0x7ff9f4004a50 comp rx=0 tx=0).stop 2024-09-19T04:55:32.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.735+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/1697678051 shutdown_connections 2024-09-19T04:55:32.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.735+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/1697678051 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x56127d53fe10 0x56127d542230 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.735+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/1697678051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127d4c4e50 0x56127d4c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.735+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/1697678051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127d4ba590 0x56127d4c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.735+0000 7ffa0d1e8740 1 --2- 172.21.15.123:0/1697678051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56127d4b9c70 0x56127d4cd560 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.735+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/1697678051 >> 172.21.15.123:0/1697678051 conn(0x56127d4b6200 msgr2=0x56127d4c0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:32.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.735+0000 7ff9f1600640 0 client.17067 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:55:32.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.735+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/1697678051 shutdown_connections 2024-09-19T04:55:32.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.735+0000 7ffa0d1e8740 1 -- 172.21.15.123:0/1697678051 wait complete. 2024-09-19T04:55:32.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.735+0000 7ffa0d1e8740 20 client.17067 trim_cache size 0 max 16384 2024-09-19T04:55:32.740 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:55:32.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.736+0000 7ffa0a600640 1 -- 172.21.15.123:0/2400702830 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff9e80a3430 msgr2=0x7ff9e80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:32.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.736+0000 7ffa0a600640 1 --2- 172.21.15.123:0/2400702830 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff9e80a3430 0x7ff9e80a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009ca0 tx=0x7ff9fc005d80 comp rx=0 tx=0).stop 2024-09-19T04:55:32.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.736+0000 7ffa0a600640 1 -- 172.21.15.123:0/2400702830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04074eb0 msgr2=0x7ffa0419f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:32.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.736+0000 7ffa0a600640 1 --2- 172.21.15.123:0/2400702830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04074eb0 0x7ffa0419f0a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00e9f0 tx=0x7ff9ec00edf0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.736+0000 7ffa0a600640 1 -- 172.21.15.123:0/2400702830 shutdown_connections 2024-09-19T04:55:32.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.736+0000 7ffa0a600640 1 --2- 172.21.15.123:0/2400702830 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff9e80a3430 0x7ff9e80a58f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.736+0000 7ffa0a600640 1 --2- 172.21.15.123:0/2400702830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa04112b90 0x7ffa041a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.736+0000 7ffa0a600640 1 --2- 172.21.15.123:0/2400702830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04074eb0 0x7ffa0419f0a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.736+0000 7ffa0a600640 1 --2- 172.21.15.123:0/2400702830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa04074590 0x7ffa0419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.736+0000 7ffa0a600640 1 -- 172.21.15.123:0/2400702830 >> 172.21.15.123:0/2400702830 conn(0x7ffa040fd8c0 msgr2=0x7ffa040ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:32.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.736+0000 7ffa0a600640 1 -- 172.21.15.123:0/2400702830 shutdown_connections 2024-09-19T04:55:32.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:32.736+0000 7ffa0a600640 1 -- 172.21.15.123:0/2400702830 wait complete. 2024-09-19T04:55:33.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:33 smithi123 ceph-mon[30511]: pgmap v274: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:33.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:33 smithi193 ceph-mon[35592]: pgmap v274: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:33.597 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:55:33.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:33 smithi183 ceph-mon[35333]: pgmap v274: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:33.962 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:34.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.539+0000 7efe8b600640 1 -- 172.21.15.123:0/601428407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe84109130 msgr2=0x7efe84110cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:34.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.539+0000 7efe8b600640 1 --2- 172.21.15.123:0/601428407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe84109130 0x7efe84110cb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009ca0 tx=0x7efe7c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.539+0000 7efe8b600640 1 -- 172.21.15.123:0/601428407 shutdown_connections 2024-09-19T04:55:34.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.539+0000 7efe8b600640 1 --2- 172.21.15.123:0/601428407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe84109130 0x7efe84110cb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.539+0000 7efe8b600640 1 --2- 172.21.15.123:0/601428407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe841049b0 0x7efe84108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.539+0000 7efe8b600640 1 --2- 172.21.15.123:0/601428407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe84104000 0x7efe841043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.539+0000 7efe8b600640 1 -- 172.21.15.123:0/601428407 >> 172.21.15.123:0/601428407 conn(0x7efe840ffa20 msgr2=0x7efe84101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:34.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.539+0000 7efe8b600640 1 -- 172.21.15.123:0/601428407 shutdown_connections 2024-09-19T04:55:34.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.539+0000 7efe8b600640 1 -- 172.21.15.123:0/601428407 wait complete. 2024-09-19T04:55:34.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.540+0000 7efe8b600640 1 Processor -- start 2024-09-19T04:55:34.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.540+0000 7efe8b600640 1 -- start start 2024-09-19T04:55:34.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.540+0000 7efe8b600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe84104000 0x7efe8419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.540+0000 7efe8b600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe841049b0 0x7efe8419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.540+0000 7efe8b600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe84109130 0x7efe8419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.540+0000 7efe8b600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe84114160 con 0x7efe841049b0 2024-09-19T04:55:34.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.540+0000 7efe8b600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efe84113fe0 con 0x7efe84104000 2024-09-19T04:55:34.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.540+0000 7efe8b600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe841142e0 con 0x7efe84109130 2024-09-19T04:55:34.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.541+0000 7efe7be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe841049b0 0x7efe8419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.541+0000 7efe88c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe84104000 0x7efe8419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.541+0000 7efe7be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe841049b0 0x7efe8419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58942/0 (socket says 172.21.15.123:58942) 2024-09-19T04:55:34.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.541+0000 7efe88c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe84104000 0x7efe8419a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:57034/0 (socket says 172.21.15.123:57034) 2024-09-19T04:55:34.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.541+0000 7efe7be00640 1 -- 172.21.15.123:0/1226040001 learned_addr learned my addr 172.21.15.123:0/1226040001 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:34.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.541+0000 7efe89600640 1 --2- 172.21.15.123:0/1226040001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe84109130 0x7efe8419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.541+0000 7efe88c00640 1 -- 172.21.15.123:0/1226040001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe84109130 msgr2=0x7efe8419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:34.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.541+0000 7efe88c00640 1 --2- 172.21.15.123:0/1226040001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe84109130 0x7efe8419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.541+0000 7efe88c00640 1 -- 172.21.15.123:0/1226040001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe841049b0 msgr2=0x7efe8419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:34.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.541+0000 7efe88c00640 1 --2- 172.21.15.123:0/1226040001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe841049b0 0x7efe8419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.541+0000 7efe88c00640 1 -- 172.21.15.123:0/1226040001 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8419f5b0 con 0x7efe84104000 2024-09-19T04:55:34.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.541+0000 7efe7be00640 1 --2- 172.21.15.123:0/1226040001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe841049b0 0x7efe8419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:34.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.541+0000 7efe88c00640 1 --2- 172.21.15.123:0/1226040001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe84104000 0x7efe8419a600 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efe7400e9f0 tx=0x7efe7400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:34.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.542+0000 7efe82000640 1 -- 172.21.15.123:0/1226040001 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7400ccc0 con 0x7efe84104000 2024-09-19T04:55:34.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.542+0000 7efe8b600640 1 -- 172.21.15.123:0/1226040001 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8419f8a0 con 0x7efe84104000 2024-09-19T04:55:34.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.542+0000 7efe82000640 1 -- 172.21.15.123:0/1226040001 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe74005020 con 0x7efe84104000 2024-09-19T04:55:34.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.542+0000 7efe82000640 1 -- 172.21.15.123:0/1226040001 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe740049e0 con 0x7efe84104000 2024-09-19T04:55:34.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.542+0000 7efe8b600640 1 -- 172.21.15.123:0/1226040001 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe841a7180 con 0x7efe84104000 2024-09-19T04:55:34.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.545+0000 7efe8e091740 1 Processor -- start 2024-09-19T04:55:34.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.545+0000 7efe8e091740 1 -- start start 2024-09-19T04:55:34.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.545+0000 7efe8e091740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157ed3c70 0x558157ed4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.545+0000 7efe8e091740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed4590 0x558157ede910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.545+0000 7efe82000640 1 -- 172.21.15.123:0/1226040001 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7efe74005560 con 0x7efe84104000 2024-09-19T04:55:34.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.545+0000 7efe8e091740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157edee50 0x558157ee1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.545+0000 7efe8e091740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558157e3a3e0 con 0x558157edee50 2024-09-19T04:55:34.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.545+0000 7efe8e091740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558157de1d80 con 0x558157ed3c70 2024-09-19T04:55:34.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.545+0000 7efe8e091740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558157e363a0 con 0x558157ed4590 2024-09-19T04:55:34.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.546+0000 7efe7be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157edee50 0x558157ee1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.546+0000 7efe88c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed4590 0x558157ede910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.546+0000 7efe89600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157ed3c70 0x558157ed4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.546+0000 7efe7be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157edee50 0x558157ee1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58952/0 (socket says 172.21.15.123:58952) 2024-09-19T04:55:34.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.546+0000 7efe7be00640 1 -- 172.21.15.123:0/1815372109 learned_addr learned my addr 172.21.15.123:0/1815372109 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:34.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.546+0000 7efe89600640 1 -- 172.21.15.123:0/1815372109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed4590 msgr2=0x558157ede910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:34.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.546+0000 7efe89600640 1 --2- 172.21.15.123:0/1815372109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed4590 0x558157ede910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.546+0000 7efe89600640 1 -- 172.21.15.123:0/1815372109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157edee50 msgr2=0x558157ee1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:34.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.546+0000 7efe89600640 1 --2- 172.21.15.123:0/1815372109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157edee50 0x558157ee1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.546+0000 7efe89600640 1 -- 172.21.15.123:0/1815372109 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558157e00ae0 con 0x558157ed3c70 2024-09-19T04:55:34.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.546+0000 7efe89600640 1 --2- 172.21.15.123:0/1815372109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157ed3c70 0x558157ed4050 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efe7c006010 tx=0x7efe7c005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b6e8bf91f96976b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:34.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.547+0000 7efe79c00640 1 -- 172.21.15.123:0/1815372109 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c031db0 con 0x558157ed3c70 2024-09-19T04:55:34.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.547+0000 7efe82000640 1 --2- 172.21.15.123:0/1226040001 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efe680a3430 0x7efe680a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.547+0000 7efe79c00640 1 -- 172.21.15.123:0/1815372109 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe7c036040 con 0x558157ed3c70 2024-09-19T04:55:34.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.547+0000 7efe79c00640 1 -- 172.21.15.123:0/1815372109 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0042a0 con 0x558157ed3c70 2024-09-19T04:55:34.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.547+0000 7efe82000640 1 -- 172.21.15.123:0/1226040001 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7efe740cfca0 con 0x7efe84104000 2024-09-19T04:55:34.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.547+0000 7efe88c00640 1 --2- 172.21.15.123:0/1226040001 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efe680a3430 0x7efe680a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.548+0000 7efe8e091740 1 -- 172.21.15.123:0/1815372109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157ed3c70 msgr2=0x558157ed4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:34.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.548+0000 7efe8e091740 1 --2- 172.21.15.123:0/1815372109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157ed3c70 0x558157ed4050 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efe7c006010 tx=0x7efe7c005f10 comp rx=0 tx=0).stop 2024-09-19T04:55:34.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.548+0000 7efe8e091740 1 -- 172.21.15.123:0/1815372109 shutdown_connections 2024-09-19T04:55:34.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.548+0000 7efe8e091740 1 --2- 172.21.15.123:0/1815372109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157edee50 0x558157ee1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.548+0000 7efe8e091740 1 --2- 172.21.15.123:0/1815372109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed4590 0x558157ede910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.548+0000 7efe8e091740 1 --2- 172.21.15.123:0/1815372109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157ed3c70 0x558157ed4050 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.548+0000 7efe8e091740 1 -- 172.21.15.123:0/1815372109 >> 172.21.15.123:0/1815372109 conn(0x558157ed0200 msgr2=0x558157ed05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:34.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.548+0000 7efe88c00640 1 --2- 172.21.15.123:0/1226040001 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efe680a3430 0x7efe680a58f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x558157ed1060 tx=0x7efe740d3750 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:34.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.548+0000 7efe8e091740 1 -- 172.21.15.123:0/1815372109 shutdown_connections 2024-09-19T04:55:34.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.548+0000 7efe8e091740 1 -- 172.21.15.123:0/1815372109 wait complete. 2024-09-19T04:55:34.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.548+0000 7efe8e091740 1 -- start start 2024-09-19T04:55:34.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.549+0000 7efe8e091740 1 Processor -- start 2024-09-19T04:55:34.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.549+0000 7efe8e091740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed3c70 0x558157edbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.549+0000 7efe8e091740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157ed4590 0x558157edc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.549+0000 7efe8e091740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157edee50 0x558157edc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.549+0000 7efe8e091740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558157ee18a0 con 0x558157ed4590 2024-09-19T04:55:34.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.549+0000 7efe8e091740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558157df50f0 con 0x558157edee50 2024-09-19T04:55:34.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.549+0000 7efe8e091740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558157ee1a20 con 0x558157ed3c70 2024-09-19T04:55:34.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.549+0000 7efe8e091740 10 client.0 _ll_register_callbacks cb 0x558157ddcbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:34.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.549+0000 7efe7be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157ed4590 0x558157edc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.549+0000 7efe7be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157ed4590 0x558157edc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58964/0 (socket says 172.21.15.123:58964) 2024-09-19T04:55:34.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.549+0000 7efe89600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157edee50 0x558157edc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.549+0000 7efe7be00640 1 -- 172.21.15.123:0/3846551340 learned_addr learned my addr 172.21.15.123:0/3846551340 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:34.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.550+0000 7efe89600640 1 --2- 172.21.15.123:0/3846551340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed3c70 0x558157edbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.550+0000 7efe7be00640 1 -- 172.21.15.123:0/3846551340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed3c70 msgr2=0x558157edbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:34.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.550+0000 7efe7be00640 1 --2- 172.21.15.123:0/3846551340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed3c70 0x558157edbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.550+0000 7efe7be00640 1 -- 172.21.15.123:0/3846551340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157edee50 msgr2=0x558157edc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:34.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.550+0000 7efe7be00640 1 --2- 172.21.15.123:0/3846551340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157edee50 0x558157edc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.550+0000 7efe7be00640 1 -- 172.21.15.123:0/3846551340 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558157bab3c0 con 0x558157ed4590 2024-09-19T04:55:34.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.550+0000 7efe7be00640 1 --2- 172.21.15.123:0/3846551340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157ed4590 0x558157edc260 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7efe6c012d50 tx=0x7efe6c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2250d39547a30e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:34.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.550+0000 7efe72000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:55:34.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.550+0000 7efe72000640 1 -- 172.21.15.123:0/3846551340 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c004050 con 0x558157ed4590 2024-09-19T04:55:34.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.550+0000 7efe8e091740 1 -- 172.21.15.123:0/3846551340 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe7c036040 con 0x558157ed4590 2024-09-19T04:55:34.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.550+0000 7efe72000640 1 -- 172.21.15.123:0/3846551340 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7efe6c0041f0 con 0x558157ed4590 2024-09-19T04:55:34.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.551+0000 7efe72000640 1 -- 172.21.15.123:0/3846551340 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7efe6c0043f0 con 0x558157ed4590 2024-09-19T04:55:34.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.551+0000 7efe8e091740 10 client.17079 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:34.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.551+0000 7efe8e091740 1 -- 172.21.15.123:0/3846551340 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558157eed920 con 0x558157ed4590 2024-09-19T04:55:34.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.551+0000 7efe72000640 1 -- 172.21.15.123:0/3846551340 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe7c036040 con 0x558157ed4590 2024-09-19T04:55:34.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.551+0000 7efe72000640 1 -- 172.21.15.123:0/3846551340 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7efe6c0255a0 con 0x558157ed4590 2024-09-19T04:55:34.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.551+0000 7efe8e091740 10 client.17079 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:34.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.551+0000 7efe8e091740 10 client.17079 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:55:34.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.551+0000 7efe8e091740 20 client.17079 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:34.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.551+0000 7efe8e091740 1 --2- 172.21.15.123:0/3846551340 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x558157eeed80 0x558157ef11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.551+0000 7efe8e091740 4 client.17079 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:55:34.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.551+0000 7efe8e091740 1 -- 172.21.15.123:0/3846551340 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558157e47950 con 0x558157eeed80 2024-09-19T04:55:34.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.552+0000 7efe89600640 1 --2- 172.21.15.123:0/3846551340 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x558157eeed80 0x558157ef11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.552+0000 7efe89600640 1 --2- 172.21.15.123:0/3846551340 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x558157eeed80 0x558157ef11a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dcef6652de93801b server_cookie=799257df07464868 in_seq=0 out_seq=0 2024-09-19T04:55:34.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.552+0000 7efe72000640 10 client.17079 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:55:34.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.553+0000 7efe72000640 1 -- 172.21.15.123:0/3846551340 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558157e47950 con 0x558157eeed80 2024-09-19T04:55:34.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.553+0000 7efe72000640 10 client.17079 handle_command_reply: tid=0 2024-09-19T04:55:34.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.553+0000 7efe8e091740 1 client.17079 shutdown 2024-09-19T04:55:34.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.554+0000 7efe8e091740 1 -- 172.21.15.123:0/3846551340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157ed4590 msgr2=0x558157edc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:34.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.554+0000 7efe8e091740 1 --2- 172.21.15.123:0/3846551340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157ed4590 0x558157edc260 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7efe6c012d50 tx=0x7efe6c00f790 comp rx=0 tx=0).stop 2024-09-19T04:55:34.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.554+0000 7efe8e091740 1 -- 172.21.15.123:0/3846551340 shutdown_connections 2024-09-19T04:55:34.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.554+0000 7efe8e091740 1 --2- 172.21.15.123:0/3846551340 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x558157eeed80 0x558157ef11a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.554+0000 7efe8e091740 1 --2- 172.21.15.123:0/3846551340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157edee50 0x558157edc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.554+0000 7efe8e091740 1 --2- 172.21.15.123:0/3846551340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157ed4590 0x558157edc260 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.554+0000 7efe8e091740 1 --2- 172.21.15.123:0/3846551340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed3c70 0x558157edbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.554+0000 7efe8e091740 1 -- 172.21.15.123:0/3846551340 >> 172.21.15.123:0/3846551340 conn(0x558157ed0200 msgr2=0x558157ecf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:34.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.554+0000 7efe72000640 0 client.17079 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:55:34.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.554+0000 7efe8e091740 1 -- 172.21.15.123:0/3846551340 shutdown_connections 2024-09-19T04:55:34.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.554+0000 7efe8e091740 1 -- 172.21.15.123:0/3846551340 wait complete. 2024-09-19T04:55:34.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.554+0000 7efe8e091740 20 client.17079 trim_cache size 0 max 16384 2024-09-19T04:55:34.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.567+0000 7efe8e091740 1 Processor -- start 2024-09-19T04:55:34.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.568+0000 7efe8e091740 1 -- start start 2024-09-19T04:55:34.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.568+0000 7efe8e091740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157ed3c70 0x558157edc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.568+0000 7efe8e091740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed4590 0x558157ef6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.568+0000 7efe8e091740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157edee50 0x558157ef6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.568+0000 7efe8e091740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe841af9b0 con 0x558157edee50 2024-09-19T04:55:34.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.568+0000 7efe8e091740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efe841b14f0 con 0x558157ed3c70 2024-09-19T04:55:34.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.568+0000 7efe8e091740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe841b0080 con 0x558157ed4590 2024-09-19T04:55:34.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.568+0000 7efe89600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157edee50 0x558157ef6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.568+0000 7efe89600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157edee50 0x558157ef6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58976/0 (socket says 172.21.15.123:58976) 2024-09-19T04:55:34.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.568+0000 7efe89600640 1 -- 172.21.15.123:0/1193129519 learned_addr learned my addr 172.21.15.123:0/1193129519 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:34.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.568+0000 7efe7be00640 1 --2- 172.21.15.123:0/1193129519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed4590 0x558157ef6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.568+0000 7efe89600640 1 --2- 172.21.15.123:0/1193129519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157ed3c70 0x558157edc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.569+0000 7efe89600640 1 -- 172.21.15.123:0/1193129519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed4590 msgr2=0x558157ef6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:34.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.569+0000 7efe89600640 1 --2- 172.21.15.123:0/1193129519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed4590 0x558157ef6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.569+0000 7efe89600640 1 -- 172.21.15.123:0/1193129519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157ed3c70 msgr2=0x558157edc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:34.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.569+0000 7efe89600640 1 --2- 172.21.15.123:0/1193129519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157ed3c70 0x558157edc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.569+0000 7efe89600640 1 -- 172.21.15.123:0/1193129519 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558157ed43c0 con 0x558157edee50 2024-09-19T04:55:34.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.569+0000 7efe7be00640 1 --2- 172.21.15.123:0/1193129519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed4590 0x558157ef6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:34.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.569+0000 7efe89600640 1 --2- 172.21.15.123:0/1193129519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157edee50 0x558157ef6f90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efe7c031980 tx=0x7efe7c047040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520e89d9a8bf8023 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:34.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.569+0000 7efe79200640 1 -- 172.21.15.123:0/1193129519 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c031bb0 con 0x558157edee50 2024-09-19T04:55:34.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.569+0000 7efe79200640 1 -- 172.21.15.123:0/1193129519 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe7c036070 con 0x558157edee50 2024-09-19T04:55:34.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.569+0000 7efe79200640 1 -- 172.21.15.123:0/1193129519 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0308b0 con 0x558157edee50 2024-09-19T04:55:34.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.570+0000 7efe8e091740 1 -- 172.21.15.123:0/1193129519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157edee50 msgr2=0x558157ef6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:34.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.570+0000 7efe8e091740 1 --2- 172.21.15.123:0/1193129519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157edee50 0x558157ef6f90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efe7c031980 tx=0x7efe7c047040 comp rx=0 tx=0).stop 2024-09-19T04:55:34.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.570+0000 7efe8e091740 1 -- 172.21.15.123:0/1193129519 shutdown_connections 2024-09-19T04:55:34.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.570+0000 7efe8e091740 1 --2- 172.21.15.123:0/1193129519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157edee50 0x558157ef6f90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.570+0000 7efe8e091740 1 --2- 172.21.15.123:0/1193129519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed4590 0x558157ef6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.570+0000 7efe8e091740 1 --2- 172.21.15.123:0/1193129519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157ed3c70 0x558157edc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.570+0000 7efe8e091740 1 -- 172.21.15.123:0/1193129519 >> 172.21.15.123:0/1193129519 conn(0x558157ed0200 msgr2=0x558157edae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:34.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.570+0000 7efe8e091740 1 -- 172.21.15.123:0/1193129519 shutdown_connections 2024-09-19T04:55:34.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.570+0000 7efe8e091740 1 -- 172.21.15.123:0/1193129519 wait complete. 2024-09-19T04:55:34.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.570+0000 7efe8e091740 1 -- start start 2024-09-19T04:55:34.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.571+0000 7efe8e091740 1 Processor -- start 2024-09-19T04:55:34.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.571+0000 7efe8e091740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157ed3c70 0x558157ee6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.571+0000 7efe8e091740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed4590 0x558157ee7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.571+0000 7efe8e091740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157edee50 0x558157ee2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.571+0000 7efe8e091740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe841b0e20 con 0x558157ed3c70 2024-09-19T04:55:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.571+0000 7efe8e091740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efe841b0750 con 0x558157edee50 2024-09-19T04:55:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.571+0000 7efe8e091740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe841ad9b0 con 0x558157ed4590 2024-09-19T04:55:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.571+0000 7efe8e091740 10 client.0 _ll_register_callbacks cb 0x7efe6c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.571+0000 7efe89600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157ed3c70 0x558157ee6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.571+0000 7efe89600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157ed3c70 0x558157ee6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58992/0 (socket says 172.21.15.123:58992) 2024-09-19T04:55:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.571+0000 7efe89600640 1 -- 172.21.15.123:0/3916907352 learned_addr learned my addr 172.21.15.123:0/3916907352 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.571+0000 7efe7be00640 1 --2- 172.21.15.123:0/3916907352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed4590 0x558157ee7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.572+0000 7efe89600640 1 --2- 172.21.15.123:0/3916907352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157edee50 0x558157ee2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.572+0000 7efe89600640 1 -- 172.21.15.123:0/3916907352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed4590 msgr2=0x558157ee7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.572+0000 7efe89600640 1 --2- 172.21.15.123:0/3916907352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed4590 0x558157ee7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.572+0000 7efe89600640 1 -- 172.21.15.123:0/3916907352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157edee50 msgr2=0x558157ee2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.572+0000 7efe89600640 1 --2- 172.21.15.123:0/3916907352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157edee50 0x558157ee2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.572+0000 7efe89600640 1 -- 172.21.15.123:0/3916907352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558157ee2a40 con 0x558157ed3c70 2024-09-19T04:55:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.572+0000 7efe89600640 1 --2- 172.21.15.123:0/3916907352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157ed3c70 0x558157ee6e50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009560 tx=0x7efe7c03d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdd318d1e4edb5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.572+0000 7efe72000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:55:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.572+0000 7efe72000640 1 -- 172.21.15.123:0/3916907352 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c064060 con 0x558157ed3c70 2024-09-19T04:55:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.572+0000 7efe72000640 1 -- 172.21.15.123:0/3916907352 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7efe7c02f610 con 0x558157ed3c70 2024-09-19T04:55:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.572+0000 7efe8e091740 1 -- 172.21.15.123:0/3916907352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe7c036070 con 0x558157ed3c70 2024-09-19T04:55:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.573+0000 7efe72000640 1 -- 172.21.15.123:0/3916907352 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7efe7c057070 con 0x558157ed3c70 2024-09-19T04:55:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.573+0000 7efe8e091740 10 client.17085 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.573+0000 7efe8e091740 1 -- 172.21.15.123:0/3916907352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558157f61d60 con 0x558157ed3c70 2024-09-19T04:55:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.573+0000 7efe72000640 1 -- 172.21.15.123:0/3916907352 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe7c036070 con 0x558157ed3c70 2024-09-19T04:55:34.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.573+0000 7efe72000640 1 -- 172.21.15.123:0/3916907352 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7efe7c050040 con 0x558157ed3c70 2024-09-19T04:55:34.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.573+0000 7efe8e091740 10 client.17085 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:34.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.573+0000 7efe8e091740 10 client.17085 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:55:34.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.573+0000 7efe8e091740 20 client.17085 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:34.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.573+0000 7efe8e091740 1 --2- 172.21.15.123:0/3916907352 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x558157f62100 0x558157f644f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.573+0000 7efe8e091740 4 client.17085 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:55:34.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.573+0000 7efe8e091740 1 -- 172.21.15.123:0/3916907352 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558157e41e60 con 0x558157f62100 2024-09-19T04:55:34.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.574+0000 7efe7be00640 1 --2- 172.21.15.123:0/3916907352 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x558157f62100 0x558157f644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.574+0000 7efe7be00640 1 --2- 172.21.15.123:0/3916907352 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x558157f62100 0x558157f644f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a4b16a8364cc88fd server_cookie=c658debf4faf7c27 in_seq=0 out_seq=0 2024-09-19T04:55:34.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.574+0000 7efe72000640 10 client.17085 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:55:34.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.574+0000 7efe72000640 1 -- 172.21.15.123:0/3916907352 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558157e41e60 con 0x558157f62100 2024-09-19T04:55:34.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.574+0000 7efe72000640 10 client.17085 handle_command_reply: tid=0 2024-09-19T04:55:34.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.574+0000 7efe8e091740 1 client.17085 shutdown 2024-09-19T04:55:34.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.575+0000 7efe8e091740 1 -- 172.21.15.123:0/3916907352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157ed3c70 msgr2=0x558157ee6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:34.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.575+0000 7efe8e091740 1 --2- 172.21.15.123:0/3916907352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157ed3c70 0x558157ee6e50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009560 tx=0x7efe7c03d040 comp rx=0 tx=0).stop 2024-09-19T04:55:34.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.575+0000 7efe8e091740 1 -- 172.21.15.123:0/3916907352 shutdown_connections 2024-09-19T04:55:34.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.575+0000 7efe8e091740 1 --2- 172.21.15.123:0/3916907352 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x558157f62100 0x558157f644f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.575+0000 7efe8e091740 1 --2- 172.21.15.123:0/3916907352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558157edee50 0x558157ee2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.575+0000 7efe8e091740 1 --2- 172.21.15.123:0/3916907352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558157ed4590 0x558157ee7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.575+0000 7efe8e091740 1 --2- 172.21.15.123:0/3916907352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558157ed3c70 0x558157ee6e50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.575+0000 7efe8e091740 1 -- 172.21.15.123:0/3916907352 >> 172.21.15.123:0/3916907352 conn(0x558157ed0200 msgr2=0x558157ed56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:34.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.575+0000 7efe72000640 0 client.17085 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:55:34.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.575+0000 7efe8e091740 1 -- 172.21.15.123:0/3916907352 shutdown_connections 2024-09-19T04:55:34.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.575+0000 7efe8e091740 1 -- 172.21.15.123:0/3916907352 wait complete. 2024-09-19T04:55:34.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.575+0000 7efe8e091740 20 client.17085 trim_cache size 0 max 16384 2024-09-19T04:55:34.580 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:55:34.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.576+0000 7efe8b600640 1 -- 172.21.15.123:0/1226040001 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efe680a3430 msgr2=0x7efe680a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:34.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.576+0000 7efe8b600640 1 --2- 172.21.15.123:0/1226040001 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efe680a3430 0x7efe680a58f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x558157ed1060 tx=0x7efe740d3750 comp rx=0 tx=0).stop 2024-09-19T04:55:34.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.576+0000 7efe8b600640 1 -- 172.21.15.123:0/1226040001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe84104000 msgr2=0x7efe8419a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:34.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.576+0000 7efe8b600640 1 --2- 172.21.15.123:0/1226040001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe84104000 0x7efe8419a600 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efe7400e9f0 tx=0x7efe7400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.576+0000 7efe8b600640 1 -- 172.21.15.123:0/1226040001 shutdown_connections 2024-09-19T04:55:34.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.576+0000 7efe8b600640 1 --2- 172.21.15.123:0/1226040001 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efe680a3430 0x7efe680a58f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.576+0000 7efe8b600640 1 --2- 172.21.15.123:0/1226040001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe84109130 0x7efe8419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.576+0000 7efe8b600640 1 --2- 172.21.15.123:0/1226040001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe841049b0 0x7efe8419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.576+0000 7efe8b600640 1 --2- 172.21.15.123:0/1226040001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe84104000 0x7efe8419a600 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.576+0000 7efe8b600640 1 -- 172.21.15.123:0/1226040001 >> 172.21.15.123:0/1226040001 conn(0x7efe840ffa20 msgr2=0x7efe84100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:34.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.576+0000 7efe8b600640 1 -- 172.21.15.123:0/1226040001 shutdown_connections 2024-09-19T04:55:34.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:34.576+0000 7efe8b600640 1 -- 172.21.15.123:0/1226040001 wait complete. 2024-09-19T04:55:35.342 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:55:35.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:35 smithi123 ceph-mon[30511]: pgmap v275: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:55:35.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:35 smithi193 ceph-mon[35592]: pgmap v275: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:55:35.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:35 smithi183 ceph-mon[35333]: pgmap v275: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:55:35.707 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:36.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.319+0000 7f6ef8800640 1 -- 172.21.15.123:0/1494757359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ef010e3c0 msgr2=0x7f6ef0110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:36.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.319+0000 7f6ef8800640 1 --2- 172.21.15.123:0/1494757359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ef010e3c0 0x7f6ef0110810 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009ca0 tx=0x7f6eec02a950 comp rx=0 tx=0).stop 2024-09-19T04:55:36.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.319+0000 7f6ef8800640 1 -- 172.21.15.123:0/1494757359 shutdown_connections 2024-09-19T04:55:36.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.319+0000 7f6ef8800640 1 --2- 172.21.15.123:0/1494757359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ef010e3c0 0x7f6ef0110810 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.319+0000 7f6ef8800640 1 --2- 172.21.15.123:0/1494757359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ef0074f40 0x7f6ef010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.319+0000 7f6ef8800640 1 --2- 172.21.15.123:0/1494757359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ef0074620 0x7f6ef0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.319+0000 7f6ef8800640 1 -- 172.21.15.123:0/1494757359 >> 172.21.15.123:0/1494757359 conn(0x7f6ef00fd8c0 msgr2=0x7f6ef00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:36.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.319+0000 7f6ef8800640 1 -- 172.21.15.123:0/1494757359 shutdown_connections 2024-09-19T04:55:36.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.319+0000 7f6ef8800640 1 -- 172.21.15.123:0/1494757359 wait complete. 2024-09-19T04:55:36.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.320+0000 7f6ef8800640 1 Processor -- start 2024-09-19T04:55:36.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.320+0000 7f6ef8800640 1 -- start start 2024-09-19T04:55:36.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.320+0000 7f6ef8800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ef0074620 0x7f6ef019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.320+0000 7f6ef8800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ef0074f40 0x7f6ef019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.320+0000 7f6ef8800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ef010e3c0 0x7f6ef019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.320+0000 7f6ef8800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01141c0 con 0x7f6ef0074f40 2024-09-19T04:55:36.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.320+0000 7f6ef8800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef0114040 con 0x7f6ef0074620 2024-09-19T04:55:36.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.320+0000 7f6ef8800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef0114340 con 0x7f6ef010e3c0 2024-09-19T04:55:36.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.321+0000 7f6ef6800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ef010e3c0 0x7f6ef019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.321+0000 7f6ef5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ef0074f40 0x7f6ef019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.321+0000 7f6ef6800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ef010e3c0 0x7f6ef019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:50902/0 (socket says 172.21.15.123:50902) 2024-09-19T04:55:36.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.321+0000 7f6ef6800640 1 -- 172.21.15.123:0/3401133039 learned_addr learned my addr 172.21.15.123:0/3401133039 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:36.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.321+0000 7f6ef5e00640 1 --2- 172.21.15.123:0/3401133039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ef0074620 0x7f6ef019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.321+0000 7f6ef6800640 1 -- 172.21.15.123:0/3401133039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ef0074620 msgr2=0x7f6ef019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:36.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.321+0000 7f6ef6800640 1 --2- 172.21.15.123:0/3401133039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ef0074620 0x7f6ef019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.321+0000 7f6ef6800640 1 -- 172.21.15.123:0/3401133039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ef0074f40 msgr2=0x7f6ef019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:36.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.321+0000 7f6ef6800640 1 --2- 172.21.15.123:0/3401133039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ef0074f40 0x7f6ef019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.321+0000 7f6ef6800640 1 -- 172.21.15.123:0/3401133039 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef019f620 con 0x7f6ef010e3c0 2024-09-19T04:55:36.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.321+0000 7f6ef5e00640 1 --2- 172.21.15.123:0/3401133039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ef0074620 0x7f6ef019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:36.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.321+0000 7f6ef5400640 1 --2- 172.21.15.123:0/3401133039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ef0074f40 0x7f6ef019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:36.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.321+0000 7f6ef6800640 1 --2- 172.21.15.123:0/3401133039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ef010e3c0 0x7f6ef019ef40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009dc0 tx=0x7f6eec003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:36.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.322+0000 7f6eeaa00640 1 -- 172.21.15.123:0/3401133039 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec034040 con 0x7f6ef010e3c0 2024-09-19T04:55:36.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.322+0000 7f6ef8800640 1 -- 172.21.15.123:0/3401133039 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef019f8b0 con 0x7f6ef010e3c0 2024-09-19T04:55:36.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.322+0000 7f6eeaa00640 1 -- 172.21.15.123:0/3401133039 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eec02f940 con 0x7f6ef010e3c0 2024-09-19T04:55:36.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.322+0000 7f6eeaa00640 1 -- 172.21.15.123:0/3401133039 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec0046a0 con 0x7f6ef010e3c0 2024-09-19T04:55:36.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.322+0000 7f6ef8800640 1 -- 172.21.15.123:0/3401133039 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef01a71f0 con 0x7f6ef010e3c0 2024-09-19T04:55:36.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.325+0000 7f6eeaa00640 1 -- 172.21.15.123:0/3401133039 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6eec037020 con 0x7f6ef010e3c0 2024-09-19T04:55:36.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.325+0000 7f6efb2be740 1 Processor -- start 2024-09-19T04:55:36.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.325+0000 7f6efb2be740 1 -- start start 2024-09-19T04:55:36.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.325+0000 7f6efb2be740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc619c70 0x5623cc61a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.325+0000 7f6efb2be740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc61a590 0x5623cc624910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.325+0000 7f6efb2be740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc624e50 0x5623cc627240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.326+0000 7f6efb2be740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5623cc5803e0 con 0x5623cc619c70 2024-09-19T04:55:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.326+0000 7f6efb2be740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5623cc527d80 con 0x5623cc61a590 2024-09-19T04:55:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.326+0000 7f6efb2be740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5623cc57c3a0 con 0x5623cc624e50 2024-09-19T04:55:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.326+0000 7f6ef5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc619c70 0x5623cc61a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.326+0000 7f6ef5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc619c70 0x5623cc61a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59026/0 (socket says 172.21.15.123:59026) 2024-09-19T04:55:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.326+0000 7f6ef5400640 1 -- 172.21.15.123:0/1341396382 learned_addr learned my addr 172.21.15.123:0/1341396382 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.326+0000 7f6ef5e00640 1 --2- 172.21.15.123:0/1341396382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc61a590 0x5623cc624910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.326+0000 7f6ef5400640 1 --2- 172.21.15.123:0/1341396382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc624e50 0x5623cc627240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.326+0000 7f6ef5400640 1 -- 172.21.15.123:0/1341396382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc624e50 msgr2=0x5623cc627240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:36.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.326+0000 7f6ef5400640 1 --2- 172.21.15.123:0/1341396382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc624e50 0x5623cc627240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.326+0000 7f6ef5400640 1 -- 172.21.15.123:0/1341396382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc61a590 msgr2=0x5623cc624910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.326+0000 7f6ef5400640 1 --2- 172.21.15.123:0/1341396382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc61a590 0x5623cc624910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.326+0000 7f6ef5400640 1 -- 172.21.15.123:0/1341396382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623cc546ae0 con 0x5623cc619c70 2024-09-19T04:55:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.326+0000 7f6eeaa00640 1 --2- 172.21.15.123:0/3401133039 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ed80a33b0 0x7f6ed80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.327+0000 7f6eeaa00640 1 -- 172.21.15.123:0/3401133039 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f6eec0f0980 con 0x7f6ef010e3c0 2024-09-19T04:55:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.327+0000 7f6ef5400640 1 --2- 172.21.15.123:0/1341396382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc619c70 0x5623cc61a050 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00138f0 tx=0x7f6ee0009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8547f5cb599641ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.327+0000 7f6ef5e00640 1 --2- 172.21.15.123:0/3401133039 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ed80a33b0 0x7f6ed80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.327+0000 7f6edf000640 1 -- 172.21.15.123:0/1341396382 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0004050 con 0x5623cc619c70 2024-09-19T04:55:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.327+0000 7f6edf000640 1 -- 172.21.15.123:0/1341396382 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee000fe20 con 0x5623cc619c70 2024-09-19T04:55:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.327+0000 7f6edf000640 1 -- 172.21.15.123:0/1341396382 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0003b80 con 0x5623cc619c70 2024-09-19T04:55:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.328+0000 7f6efb2be740 1 -- 172.21.15.123:0/1341396382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc619c70 msgr2=0x5623cc61a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.328+0000 7f6ef5e00640 1 --2- 172.21.15.123:0/3401133039 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ed80a33b0 0x7f6ed80a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x5623cc617060 tx=0x7f6ee4005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.328+0000 7f6efb2be740 1 --2- 172.21.15.123:0/1341396382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc619c70 0x5623cc61a050 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00138f0 tx=0x7f6ee0009be0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.328+0000 7f6efb2be740 1 -- 172.21.15.123:0/1341396382 shutdown_connections 2024-09-19T04:55:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.328+0000 7f6efb2be740 1 --2- 172.21.15.123:0/1341396382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc624e50 0x5623cc627240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.328+0000 7f6efb2be740 1 --2- 172.21.15.123:0/1341396382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc61a590 0x5623cc624910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.328+0000 7f6efb2be740 1 --2- 172.21.15.123:0/1341396382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc619c70 0x5623cc61a050 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.328+0000 7f6efb2be740 1 -- 172.21.15.123:0/1341396382 >> 172.21.15.123:0/1341396382 conn(0x5623cc616200 msgr2=0x5623cc6165e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.328+0000 7f6efb2be740 1 -- 172.21.15.123:0/1341396382 shutdown_connections 2024-09-19T04:55:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.328+0000 7f6efb2be740 1 -- 172.21.15.123:0/1341396382 wait complete. 2024-09-19T04:55:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.328+0000 7f6efb2be740 1 -- start start 2024-09-19T04:55:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.329+0000 7f6efb2be740 1 Processor -- start 2024-09-19T04:55:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.329+0000 7f6efb2be740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc619c70 0x5623cc621ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.329+0000 7f6efb2be740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc61a590 0x5623cc622220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.329+0000 7f6efb2be740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc624e50 0x5623cc622760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.329+0000 7f6efb2be740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5623cc627a20 con 0x5623cc619c70 2024-09-19T04:55:36.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.329+0000 7f6efb2be740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5623cc53b0f0 con 0x5623cc61a590 2024-09-19T04:55:36.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.329+0000 7f6efb2be740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5623cc627ba0 con 0x5623cc624e50 2024-09-19T04:55:36.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.329+0000 7f6efb2be740 10 client.0 _ll_register_callbacks cb 0x5623cc522bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:36.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.329+0000 7f6ef5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc619c70 0x5623cc621ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.329+0000 7f6ef5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc619c70 0x5623cc621ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59028/0 (socket says 172.21.15.123:59028) 2024-09-19T04:55:36.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.329+0000 7f6ef5e00640 1 -- 172.21.15.123:0/545627315 learned_addr learned my addr 172.21.15.123:0/545627315 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:36.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.330+0000 7f6ef5400640 1 --2- 172.21.15.123:0/545627315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc61a590 0x5623cc622220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.330+0000 7f6ef6800640 1 --2- 172.21.15.123:0/545627315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc624e50 0x5623cc622760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.330+0000 7f6ef5e00640 1 -- 172.21.15.123:0/545627315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc624e50 msgr2=0x5623cc622760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:36.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.330+0000 7f6ef5e00640 1 --2- 172.21.15.123:0/545627315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc624e50 0x5623cc622760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.330+0000 7f6ef5e00640 1 -- 172.21.15.123:0/545627315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc61a590 msgr2=0x5623cc622220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:36.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.330+0000 7f6ef5e00640 1 --2- 172.21.15.123:0/545627315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc61a590 0x5623cc622220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.330+0000 7f6ef5e00640 1 -- 172.21.15.123:0/545627315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623cc2f13c0 con 0x5623cc619c70 2024-09-19T04:55:36.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.330+0000 7f6ef5400640 1 --2- 172.21.15.123:0/545627315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc61a590 0x5623cc622220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:36.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.330+0000 7f6ef6800640 1 --2- 172.21.15.123:0/545627315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc624e50 0x5623cc622760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:36.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.330+0000 7f6ef5e00640 1 --2- 172.21.15.123:0/545627315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc619c70 0x5623cc621ce0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009560 tx=0x7f6ee4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72908f46971c3ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:36.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.330+0000 7f6ed3400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:55:36.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.331+0000 7f6ed3400640 1 -- 172.21.15.123:0/545627315 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee402e060 con 0x5623cc619c70 2024-09-19T04:55:36.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.331+0000 7f6ed3400640 1 -- 172.21.15.123:0/545627315 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f6ee4018030 con 0x5623cc619c70 2024-09-19T04:55:36.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.331+0000 7f6efb2be740 1 -- 172.21.15.123:0/545627315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ee000fe20 con 0x5623cc619c70 2024-09-19T04:55:36.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.331+0000 7f6ed3400640 1 -- 172.21.15.123:0/545627315 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6ee400fd20 con 0x5623cc619c70 2024-09-19T04:55:36.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.331+0000 7f6efb2be740 10 client.17097 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:36.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.331+0000 7f6efb2be740 1 -- 172.21.15.123:0/545627315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5623cc633a50 con 0x5623cc619c70 2024-09-19T04:55:36.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.331+0000 7f6ed3400640 1 -- 172.21.15.123:0/545627315 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee000fe20 con 0x5623cc619c70 2024-09-19T04:55:36.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.331+0000 7f6ed3400640 1 -- 172.21.15.123:0/545627315 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f6ee4029b40 con 0x5623cc619c70 2024-09-19T04:55:36.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.332+0000 7f6efb2be740 10 client.17097 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:36.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.332+0000 7f6efb2be740 10 client.17097 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:55:36.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.332+0000 7f6efb2be740 20 client.17097 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:36.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.332+0000 7f6efb2be740 1 --2- 172.21.15.123:0/545627315 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5623cc634f10 0x5623cc637340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.332+0000 7f6efb2be740 4 client.17097 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:55:36.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.332+0000 7f6efb2be740 1 -- 172.21.15.123:0/545627315 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623cc58d950 con 0x5623cc634f10 2024-09-19T04:55:36.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.332+0000 7f6ef5400640 1 --2- 172.21.15.123:0/545627315 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5623cc634f10 0x5623cc637340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.333+0000 7f6ef5400640 1 --2- 172.21.15.123:0/545627315 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5623cc634f10 0x5623cc637340 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=203d7baec810b04f server_cookie=8a9636b96d6f17d2 in_seq=0 out_seq=0 2024-09-19T04:55:36.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.333+0000 7f6ed3400640 10 client.17097 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:55:36.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.333+0000 7f6ed3400640 1 -- 172.21.15.123:0/545627315 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623cc58d950 con 0x5623cc634f10 2024-09-19T04:55:36.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.333+0000 7f6ed3400640 10 client.17097 handle_command_reply: tid=0 2024-09-19T04:55:36.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.334+0000 7f6efb2be740 1 client.17097 shutdown 2024-09-19T04:55:36.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.334+0000 7f6efb2be740 1 -- 172.21.15.123:0/545627315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc619c70 msgr2=0x5623cc621ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:36.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.334+0000 7f6efb2be740 1 --2- 172.21.15.123:0/545627315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc619c70 0x5623cc621ce0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009560 tx=0x7f6ee4004730 comp rx=0 tx=0).stop 2024-09-19T04:55:36.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.334+0000 7f6efb2be740 1 -- 172.21.15.123:0/545627315 shutdown_connections 2024-09-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.334+0000 7f6efb2be740 1 --2- 172.21.15.123:0/545627315 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5623cc634f10 0x5623cc637340 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.334+0000 7f6efb2be740 1 --2- 172.21.15.123:0/545627315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc624e50 0x5623cc622760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.334+0000 7f6efb2be740 1 --2- 172.21.15.123:0/545627315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc61a590 0x5623cc622220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.334+0000 7f6efb2be740 1 --2- 172.21.15.123:0/545627315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc619c70 0x5623cc621ce0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.334+0000 7f6ed3400640 0 client.17097 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.334+0000 7f6efb2be740 1 -- 172.21.15.123:0/545627315 >> 172.21.15.123:0/545627315 conn(0x5623cc616200 msgr2=0x5623cc61bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.334+0000 7f6efb2be740 1 -- 172.21.15.123:0/545627315 shutdown_connections 2024-09-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.334+0000 7f6efb2be740 1 -- 172.21.15.123:0/545627315 wait complete. 2024-09-19T04:55:36.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.334+0000 7f6efb2be740 20 client.17097 trim_cache size 0 max 16384 2024-09-19T04:55:36.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.348+0000 7f6efb2be740 1 Processor -- start 2024-09-19T04:55:36.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.348+0000 7f6efb2be740 1 -- start start 2024-09-19T04:55:36.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.348+0000 7f6efb2be740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc619c70 0x5623cc61c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.348+0000 7f6efb2be740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc61a590 0x5623cc623c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.348+0000 7f6efb2be740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc624e50 0x5623cc63cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.348+0000 7f6efb2be740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01afad0 con 0x5623cc624e50 2024-09-19T04:55:36.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.348+0000 7f6efb2be740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01b1610 con 0x5623cc61a590 2024-09-19T04:55:36.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.348+0000 7f6efb2be740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01b01a0 con 0x5623cc619c70 2024-09-19T04:55:36.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.348+0000 7f6ef6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc624e50 0x5623cc63cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.348+0000 7f6ef6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc624e50 0x5623cc63cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59044/0 (socket says 172.21.15.123:59044) 2024-09-19T04:55:36.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.348+0000 7f6ef6800640 1 -- 172.21.15.123:0/1225163856 learned_addr learned my addr 172.21.15.123:0/1225163856 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:36.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.348+0000 7f6ef5e00640 1 --2- 172.21.15.123:0/1225163856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc619c70 0x5623cc61c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.349+0000 7f6ef5400640 1 --2- 172.21.15.123:0/1225163856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc61a590 0x5623cc623c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.349+0000 7f6ef6800640 1 -- 172.21.15.123:0/1225163856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc619c70 msgr2=0x5623cc61c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:36.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.349+0000 7f6ef6800640 1 --2- 172.21.15.123:0/1225163856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc619c70 0x5623cc61c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.349+0000 7f6ef6800640 1 -- 172.21.15.123:0/1225163856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc61a590 msgr2=0x5623cc623c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:36.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.349+0000 7f6ef6800640 1 --2- 172.21.15.123:0/1225163856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc61a590 0x5623cc623c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.349+0000 7f6ef6800640 1 -- 172.21.15.123:0/1225163856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623cc616030 con 0x5623cc624e50 2024-09-19T04:55:36.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.349+0000 7f6ef6800640 1 --2- 172.21.15.123:0/1225163856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc624e50 0x5623cc63cc10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0bca10 tx=0x7f6eec004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14fc0d610bf0985 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:36.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.349+0000 7f6ede600640 1 -- 172.21.15.123:0/1225163856 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec0da070 con 0x5623cc624e50 2024-09-19T04:55:36.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.349+0000 7f6ede600640 1 -- 172.21.15.123:0/1225163856 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eec0bcd10 con 0x5623cc624e50 2024-09-19T04:55:36.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.349+0000 7f6ede600640 1 -- 172.21.15.123:0/1225163856 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec02c040 con 0x5623cc624e50 2024-09-19T04:55:36.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.350+0000 7f6efb2be740 1 -- 172.21.15.123:0/1225163856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc624e50 msgr2=0x5623cc63cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:36.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.350+0000 7f6efb2be740 1 --2- 172.21.15.123:0/1225163856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc624e50 0x5623cc63cc10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0bca10 tx=0x7f6eec004140 comp rx=0 tx=0).stop 2024-09-19T04:55:36.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.350+0000 7f6efb2be740 1 -- 172.21.15.123:0/1225163856 shutdown_connections 2024-09-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.350+0000 7f6efb2be740 1 --2- 172.21.15.123:0/1225163856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc624e50 0x5623cc63cc10 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.350+0000 7f6efb2be740 1 --2- 172.21.15.123:0/1225163856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc61a590 0x5623cc623c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.350+0000 7f6efb2be740 1 --2- 172.21.15.123:0/1225163856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc619c70 0x5623cc61c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.350+0000 7f6efb2be740 1 -- 172.21.15.123:0/1225163856 >> 172.21.15.123:0/1225163856 conn(0x5623cc616200 msgr2=0x5623cc623540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.350+0000 7f6efb2be740 1 -- 172.21.15.123:0/1225163856 shutdown_connections 2024-09-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.350+0000 7f6efb2be740 1 -- 172.21.15.123:0/1225163856 wait complete. 2024-09-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.350+0000 7f6efb2be740 1 -- start start 2024-09-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.351+0000 7f6efb2be740 1 Processor -- start 2024-09-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.351+0000 7f6efb2be740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc619c70 0x5623cc62d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.351+0000 7f6efb2be740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc61a590 0x5623cc61eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.351+0000 7f6efb2be740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc624e50 0x5623cc61f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.351+0000 7f6efb2be740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01b0f40 con 0x5623cc61a590 2024-09-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.351+0000 7f6efb2be740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01b0870 con 0x5623cc624e50 2024-09-19T04:55:36.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.351+0000 7f6efb2be740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01adad0 con 0x5623cc619c70 2024-09-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.351+0000 7f6efb2be740 10 client.0 _ll_register_callbacks cb 0x7f6ee4009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.351+0000 7f6ef5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc61a590 0x5623cc61eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.351+0000 7f6ef5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc619c70 0x5623cc62d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.351+0000 7f6ef5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc61a590 0x5623cc61eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59056/0 (socket says 172.21.15.123:59056) 2024-09-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.351+0000 7f6ef5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc619c70 0x5623cc62d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:50946/0 (socket says 172.21.15.123:50946) 2024-09-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.351+0000 7f6ef6800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc624e50 0x5623cc61f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.351+0000 7f6ef5400640 1 -- 172.21.15.123:0/4005813008 learned_addr learned my addr 172.21.15.123:0/4005813008 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.352+0000 7f6ef5400640 1 -- 172.21.15.123:0/4005813008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc619c70 msgr2=0x5623cc62d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.352+0000 7f6ef5400640 1 --2- 172.21.15.123:0/4005813008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc619c70 0x5623cc62d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.352+0000 7f6ef5400640 1 -- 172.21.15.123:0/4005813008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc624e50 msgr2=0x5623cc61f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.352+0000 7f6ef5400640 1 --2- 172.21.15.123:0/4005813008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc624e50 0x5623cc61f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.352+0000 7f6ef5400640 1 -- 172.21.15.123:0/4005813008 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623cc611770 con 0x5623cc61a590 2024-09-19T04:55:36.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.352+0000 7f6ef5400640 1 --2- 172.21.15.123:0/4005813008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc61a590 0x5623cc61eae0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000f790 tx=0x7f6ee00037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea88412004b9f5f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.352+0000 7f6ed3400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.352+0000 7f6ed3400640 1 -- 172.21.15.123:0/4005813008 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0013430 con 0x5623cc61a590 2024-09-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.352+0000 7f6efb2be740 1 -- 172.21.15.123:0/4005813008 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eec0bcd10 con 0x5623cc61a590 2024-09-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.352+0000 7f6ed3400640 1 -- 172.21.15.123:0/4005813008 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f6ee00050d0 con 0x5623cc61a590 2024-09-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.353+0000 7f6ed3400640 1 -- 172.21.15.123:0/4005813008 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6ee00052d0 con 0x5623cc61a590 2024-09-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.353+0000 7f6efb2be740 10 client.17103 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.353+0000 7f6efb2be740 1 -- 172.21.15.123:0/4005813008 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5623cc69fae0 con 0x5623cc61a590 2024-09-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.353+0000 7f6ed3400640 1 -- 172.21.15.123:0/4005813008 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eec0bcd10 con 0x5623cc61a590 2024-09-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.353+0000 7f6ed3400640 1 -- 172.21.15.123:0/4005813008 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f6ee002a070 con 0x5623cc61a590 2024-09-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.353+0000 7f6efb2be740 10 client.17103 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.353+0000 7f6efb2be740 10 client.17103 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.353+0000 7f6efb2be740 20 client.17103 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:36.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.353+0000 7f6efb2be740 1 --2- 172.21.15.123:0/4005813008 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5623cc6a0120 0x5623cc6a2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.353+0000 7f6efb2be740 4 client.17103 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:55:36.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.353+0000 7f6efb2be740 1 -- 172.21.15.123:0/4005813008 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623cc587e60 con 0x5623cc6a0120 2024-09-19T04:55:36.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.354+0000 7f6ef5e00640 1 --2- 172.21.15.123:0/4005813008 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5623cc6a0120 0x5623cc6a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.354+0000 7f6ed3400640 10 client.17103 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:55:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.354+0000 7f6ef5e00640 1 --2- 172.21.15.123:0/4005813008 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5623cc6a0120 0x5623cc6a2540 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c41b01cc81c1e05 server_cookie=8fede925d916362a in_seq=0 out_seq=0 2024-09-19T04:55:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.354+0000 7f6ed3400640 1 -- 172.21.15.123:0/4005813008 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623cc587e60 con 0x5623cc6a0120 2024-09-19T04:55:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.354+0000 7f6ed3400640 10 client.17103 handle_command_reply: tid=0 2024-09-19T04:55:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.354+0000 7f6efb2be740 1 client.17103 shutdown 2024-09-19T04:55:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.355+0000 7f6efb2be740 1 -- 172.21.15.123:0/4005813008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc61a590 msgr2=0x5623cc61eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.355+0000 7f6efb2be740 1 --2- 172.21.15.123:0/4005813008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc61a590 0x5623cc61eae0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000f790 tx=0x7f6ee00037c0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.355+0000 7f6efb2be740 1 -- 172.21.15.123:0/4005813008 shutdown_connections 2024-09-19T04:55:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.355+0000 7f6efb2be740 1 --2- 172.21.15.123:0/4005813008 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x5623cc6a0120 0x5623cc6a2540 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.355+0000 7f6efb2be740 1 --2- 172.21.15.123:0/4005813008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623cc624e50 0x5623cc61f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.355+0000 7f6efb2be740 1 --2- 172.21.15.123:0/4005813008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5623cc61a590 0x5623cc61eae0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.355+0000 7f6efb2be740 1 --2- 172.21.15.123:0/4005813008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623cc619c70 0x5623cc62d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.355+0000 7f6efb2be740 1 -- 172.21.15.123:0/4005813008 >> 172.21.15.123:0/4005813008 conn(0x5623cc616200 msgr2=0x5623cc615380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:36.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.355+0000 7f6ed3400640 0 client.17103 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:55:36.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.355+0000 7f6efb2be740 1 -- 172.21.15.123:0/4005813008 shutdown_connections 2024-09-19T04:55:36.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.355+0000 7f6efb2be740 1 -- 172.21.15.123:0/4005813008 wait complete. 2024-09-19T04:55:36.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.355+0000 7f6efb2be740 20 client.17103 trim_cache size 0 max 16384 2024-09-19T04:55:36.360 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.355+0000 7f6ef8800640 1 -- 172.21.15.123:0/3401133039 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ed80a33b0 msgr2=0x7f6ed80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.356+0000 7f6ef8800640 1 --2- 172.21.15.123:0/3401133039 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ed80a33b0 0x7f6ed80a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x5623cc617060 tx=0x7f6ee4005f90 comp rx=0 tx=0).stop 2024-09-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.356+0000 7f6ef8800640 1 -- 172.21.15.123:0/3401133039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ef010e3c0 msgr2=0x7f6ef019ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.356+0000 7f6ef8800640 1 --2- 172.21.15.123:0/3401133039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ef010e3c0 0x7f6ef019ef40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009dc0 tx=0x7f6eec003c10 comp rx=0 tx=0).stop 2024-09-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.356+0000 7f6ef8800640 1 -- 172.21.15.123:0/3401133039 shutdown_connections 2024-09-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.356+0000 7f6ef8800640 1 --2- 172.21.15.123:0/3401133039 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6ed80a33b0 0x7f6ed80a5870 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.356+0000 7f6ef8800640 1 --2- 172.21.15.123:0/3401133039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ef010e3c0 0x7f6ef019ef40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.356+0000 7f6ef8800640 1 --2- 172.21.15.123:0/3401133039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ef0074f40 0x7f6ef019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.356+0000 7f6ef8800640 1 --2- 172.21.15.123:0/3401133039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ef0074620 0x7f6ef019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.356+0000 7f6ef8800640 1 -- 172.21.15.123:0/3401133039 >> 172.21.15.123:0/3401133039 conn(0x7f6ef00fd8c0 msgr2=0x7f6ef00fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.356+0000 7f6ef8800640 1 -- 172.21.15.123:0/3401133039 shutdown_connections 2024-09-19T04:55:36.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:36.356+0000 7f6ef8800640 1 -- 172.21.15.123:0/3401133039 wait complete. 2024-09-19T04:55:37.227 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:55:37.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:37 smithi193 ceph-mon[35592]: pgmap v276: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:37.594 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:37.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:37 smithi183 ceph-mon[35333]: pgmap v276: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:37.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:37 smithi123 ceph-mon[30511]: pgmap v276: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:38.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.190+0000 7fea35c00640 1 -- 172.21.15.123:0/972098098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea30102360 msgr2=0x7fea3010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:38.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.190+0000 7fea35c00640 1 --2- 172.21.15.123:0/972098098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea30102360 0x7fea3010e890 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fea28009c10 tx=0x7fea2802a6f0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.191+0000 7fea35c00640 1 -- 172.21.15.123:0/972098098 shutdown_connections 2024-09-19T04:55:38.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.191+0000 7fea35c00640 1 --2- 172.21.15.123:0/972098098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea30102360 0x7fea3010e890 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.191+0000 7fea35c00640 1 --2- 172.21.15.123:0/972098098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea301019c0 0x7fea30101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.191+0000 7fea35c00640 1 --2- 172.21.15.123:0/972098098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea301079c0 0x7fea30107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.191+0000 7fea35c00640 1 -- 172.21.15.123:0/972098098 >> 172.21.15.123:0/972098098 conn(0x7fea300fd6e0 msgr2=0x7fea300ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:38.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.191+0000 7fea35c00640 1 -- 172.21.15.123:0/972098098 shutdown_connections 2024-09-19T04:55:38.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.191+0000 7fea35c00640 1 -- 172.21.15.123:0/972098098 wait complete. 2024-09-19T04:55:38.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.192+0000 7fea35c00640 1 Processor -- start 2024-09-19T04:55:38.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.192+0000 7fea35c00640 1 -- start start 2024-09-19T04:55:38.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.192+0000 7fea35c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea301019c0 0x7fea300744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.192+0000 7fea35c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea30102360 0x7fea30074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.192+0000 7fea35c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea301079c0 0x7fea30074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.192+0000 7fea35c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea30111dc0 con 0x7fea30102360 2024-09-19T04:55:38.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.192+0000 7fea35c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea30111c40 con 0x7fea301079c0 2024-09-19T04:55:38.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.192+0000 7fea35c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea30111f40 con 0x7fea301019c0 2024-09-19T04:55:38.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.192+0000 7fea2fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea301019c0 0x7fea300744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.192+0000 7fea2f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea30102360 0x7fea30074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.192+0000 7fea2fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea301019c0 0x7fea300744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:50964/0 (socket says 172.21.15.123:50964) 2024-09-19T04:55:38.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.192+0000 7fea2fe00640 1 -- 172.21.15.123:0/4221940999 learned_addr learned my addr 172.21.15.123:0/4221940999 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:38.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.192+0000 7fea35200640 1 --2- 172.21.15.123:0/4221940999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea301079c0 0x7fea30074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.193+0000 7fea2fe00640 1 -- 172.21.15.123:0/4221940999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea301079c0 msgr2=0x7fea30074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:38.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.193+0000 7fea2fe00640 1 --2- 172.21.15.123:0/4221940999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea301079c0 0x7fea30074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.193+0000 7fea2fe00640 1 -- 172.21.15.123:0/4221940999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea30102360 msgr2=0x7fea30074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:38.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.193+0000 7fea2fe00640 1 --2- 172.21.15.123:0/4221940999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea30102360 0x7fea30074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.193+0000 7fea2fe00640 1 -- 172.21.15.123:0/4221940999 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea301a6740 con 0x7fea301019c0 2024-09-19T04:55:38.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.193+0000 7fea2f400640 1 --2- 172.21.15.123:0/4221940999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea30102360 0x7fea30074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:38.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.193+0000 7fea2fe00640 1 --2- 172.21.15.123:0/4221940999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea301019c0 0x7fea300744f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fea2000e9f0 tx=0x7fea2000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:38.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.193+0000 7fea2cc00640 1 -- 172.21.15.123:0/4221940999 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2000ccc0 con 0x7fea301019c0 2024-09-19T04:55:38.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.193+0000 7fea2cc00640 1 -- 172.21.15.123:0/4221940999 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea2000ce60 con 0x7fea301019c0 2024-09-19T04:55:38.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.193+0000 7fea35c00640 1 -- 172.21.15.123:0/4221940999 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea301a6a30 con 0x7fea301019c0 2024-09-19T04:55:38.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.193+0000 7fea2cc00640 1 -- 172.21.15.123:0/4221940999 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20010690 con 0x7fea301019c0 2024-09-19T04:55:38.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.193+0000 7fea35c00640 1 -- 172.21.15.123:0/4221940999 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea301a6f20 con 0x7fea301019c0 2024-09-19T04:55:38.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.196+0000 7fea2cc00640 1 -- 172.21.15.123:0/4221940999 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fea200046c0 con 0x7fea301019c0 2024-09-19T04:55:38.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.196+0000 7fea39c42740 1 Processor -- start 2024-09-19T04:55:38.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.197+0000 7fea39c42740 1 -- start start 2024-09-19T04:55:38.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.197+0000 7fea39c42740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 0x564eecee8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.197+0000 7fea39c42740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee8760 0x564eecef2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.197+0000 7fea39c42740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecef3020 0x564eecef5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.197+0000 7fea39c42740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564eece4e3e0 con 0x564eecef3020 2024-09-19T04:55:38.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.197+0000 7fea39c42740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564eecdf5d80 con 0x564eecee8760 2024-09-19T04:55:38.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.197+0000 7fea39c42740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564eece4a3a0 con 0x564eecee7e40 2024-09-19T04:55:38.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.197+0000 7fea2f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecef3020 0x564eecef5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.197+0000 7fea35200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 0x564eecee8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.197+0000 7fea2fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee8760 0x564eecef2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.197+0000 7fea35200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 0x564eecee8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:50972/0 (socket says 172.21.15.123:50972) 2024-09-19T04:55:38.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.197+0000 7fea2f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecef3020 0x564eecef5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59086/0 (socket says 172.21.15.123:59086) 2024-09-19T04:55:38.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.197+0000 7fea35200640 1 -- 172.21.15.123:0/420944034 learned_addr learned my addr 172.21.15.123:0/420944034 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:38.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.198+0000 7fea2f400640 1 -- 172.21.15.123:0/420944034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 msgr2=0x564eecee8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:38.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.198+0000 7fea2f400640 1 --2- 172.21.15.123:0/420944034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 0x564eecee8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.198+0000 7fea2f400640 1 -- 172.21.15.123:0/420944034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee8760 msgr2=0x564eecef2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:38.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.198+0000 7fea2f400640 1 --2- 172.21.15.123:0/420944034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee8760 0x564eecef2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.198+0000 7fea2f400640 1 -- 172.21.15.123:0/420944034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eece14ae0 con 0x564eecef3020 2024-09-19T04:55:38.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.198+0000 7fea35200640 1 --2- 172.21.15.123:0/420944034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 0x564eecee8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:38.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.198+0000 7fea2fe00640 1 --2- 172.21.15.123:0/420944034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee8760 0x564eecef2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:38.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.198+0000 7fea2cc00640 1 --2- 172.21.15.123:0/4221940999 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fea140a33b0 0x7fea140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.198+0000 7fea2cc00640 1 -- 172.21.15.123:0/4221940999 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fea200d00e0 con 0x7fea301019c0 2024-09-19T04:55:38.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.198+0000 7fea2f400640 1 --2- 172.21.15.123:0/420944034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecef3020 0x564eecef5410 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fea1c005fd0 tx=0x7fea1c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56433ceb57db4af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:38.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.198+0000 7fea35200640 1 --2- 172.21.15.123:0/4221940999 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fea140a33b0 0x7fea140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.198+0000 7fea25600640 1 -- 172.21.15.123:0/420944034 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c01e070 con 0x564eecef3020 2024-09-19T04:55:38.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.198+0000 7fea25600640 1 -- 172.21.15.123:0/420944034 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea1c0035f0 con 0x564eecef3020 2024-09-19T04:55:38.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.198+0000 7fea25600640 1 -- 172.21.15.123:0/420944034 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c00f800 con 0x564eecef3020 2024-09-19T04:55:38.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.199+0000 7fea39c42740 1 -- 172.21.15.123:0/420944034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecef3020 msgr2=0x564eecef5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:38.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.199+0000 7fea35200640 1 --2- 172.21.15.123:0/4221940999 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fea140a33b0 0x7fea140a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fea28004af0 tx=0x7fea28005fd0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:38.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.199+0000 7fea39c42740 1 --2- 172.21.15.123:0/420944034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecef3020 0x564eecef5410 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fea1c005fd0 tx=0x7fea1c0118d0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.199+0000 7fea39c42740 1 -- 172.21.15.123:0/420944034 shutdown_connections 2024-09-19T04:55:38.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.199+0000 7fea39c42740 1 --2- 172.21.15.123:0/420944034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecef3020 0x564eecef5410 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.199+0000 7fea39c42740 1 --2- 172.21.15.123:0/420944034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee8760 0x564eecef2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.199+0000 7fea39c42740 1 --2- 172.21.15.123:0/420944034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 0x564eecee8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.199+0000 7fea39c42740 1 -- 172.21.15.123:0/420944034 >> 172.21.15.123:0/420944034 conn(0x564eecee43d0 msgr2=0x564eecee47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:38.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.199+0000 7fea39c42740 1 -- 172.21.15.123:0/420944034 shutdown_connections 2024-09-19T04:55:38.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.199+0000 7fea39c42740 1 -- 172.21.15.123:0/420944034 wait complete. 2024-09-19T04:55:38.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.200+0000 7fea39c42740 1 -- start start 2024-09-19T04:55:38.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.200+0000 7fea39c42740 1 Processor -- start 2024-09-19T04:55:38.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.201+0000 7fea39c42740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 0x564eeceeffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.201+0000 7fea39c42740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee8760 0x564eecef0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.201+0000 7fea39c42740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecef3020 0x564eecef0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.201+0000 7fea39c42740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564eecef5bf0 con 0x564eecef3020 2024-09-19T04:55:38.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.201+0000 7fea39c42740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564eece090f0 con 0x564eecee8760 2024-09-19T04:55:38.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.201+0000 7fea39c42740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564eecef5d70 con 0x564eecee7e40 2024-09-19T04:55:38.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.201+0000 7fea39c42740 10 client.0 _ll_register_callbacks cb 0x564eecdf0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:38.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.201+0000 7fea35200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecef3020 0x564eecef0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.201+0000 7fea35200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecef3020 0x564eecef0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59096/0 (socket says 172.21.15.123:59096) 2024-09-19T04:55:38.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.201+0000 7fea35200640 1 -- 172.21.15.123:0/3990395096 learned_addr learned my addr 172.21.15.123:0/3990395096 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:38.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.201+0000 7fea2fe00640 1 --2- 172.21.15.123:0/3990395096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 0x564eeceeffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.201+0000 7fea2f400640 1 --2- 172.21.15.123:0/3990395096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee8760 0x564eecef0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.201+0000 7fea2fe00640 1 -- 172.21.15.123:0/3990395096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee8760 msgr2=0x564eecef0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:38.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.201+0000 7fea2fe00640 1 --2- 172.21.15.123:0/3990395096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee8760 0x564eecef0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.201+0000 7fea2fe00640 1 -- 172.21.15.123:0/3990395096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecef3020 msgr2=0x564eecef0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:38.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.201+0000 7fea2fe00640 1 --2- 172.21.15.123:0/3990395096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecef3020 0x564eecef0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.201+0000 7fea2fe00640 1 -- 172.21.15.123:0/3990395096 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eecbbf3c0 con 0x564eecee7e40 2024-09-19T04:55:38.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.202+0000 7fea2fe00640 1 --2- 172.21.15.123:0/3990395096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 0x564eeceeffc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fea200cd7d0 tx=0x7fea2009a870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77d82a20c20cd3b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:38.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.202+0000 7fea1a000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:55:38.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.202+0000 7fea1a000640 1 -- 172.21.15.123:0/3990395096 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2009bcc0 con 0x564eecee7e40 2024-09-19T04:55:38.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.202+0000 7fea39c42740 1 -- 172.21.15.123:0/3990395096 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea1c0035f0 con 0x564eecee7e40 2024-09-19T04:55:38.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.202+0000 7fea1a000640 1 -- 172.21.15.123:0/3990395096 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fea200d2110 con 0x564eecee7e40 2024-09-19T04:55:38.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.202+0000 7fea1a000640 1 -- 172.21.15.123:0/3990395096 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fea200cdc50 con 0x564eecee7e40 2024-09-19T04:55:38.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.202+0000 7fea39c42740 10 client.27226 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:38.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.202+0000 7fea39c42740 1 -- 172.21.15.123:0/3990395096 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564eecf01d20 con 0x564eecee7e40 2024-09-19T04:55:38.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.202+0000 7fea1a000640 1 -- 172.21.15.123:0/3990395096 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea1c0035f0 con 0x564eecee7e40 2024-09-19T04:55:38.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.202+0000 7fea1a000640 1 -- 172.21.15.123:0/3990395096 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fea200b5120 con 0x564eecee7e40 2024-09-19T04:55:38.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.203+0000 7fea39c42740 10 client.27226 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:38.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.203+0000 7fea39c42740 10 client.27226 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:55:38.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.203+0000 7fea39c42740 20 client.27226 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:38.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.203+0000 7fea39c42740 1 --2- 172.21.15.123:0/3990395096 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x564eecf03070 0x564eecf05480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.203+0000 7fea39c42740 4 client.27226 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:55:38.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.203+0000 7fea39c42740 1 -- 172.21.15.123:0/3990395096 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564eece5b950 con 0x564eecf03070 2024-09-19T04:55:38.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.203+0000 7fea2f400640 1 --2- 172.21.15.123:0/3990395096 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x564eecf03070 0x564eecf05480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.203+0000 7fea2f400640 1 --2- 172.21.15.123:0/3990395096 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x564eecf03070 0x564eecf05480 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b52182760174df18 server_cookie=53308a3ee2c78f24 in_seq=0 out_seq=0 2024-09-19T04:55:38.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.203+0000 7fea1a000640 10 client.27226 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:55:38.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.204+0000 7fea1a000640 1 -- 172.21.15.123:0/3990395096 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564eece5b950 con 0x564eecf03070 2024-09-19T04:55:38.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.204+0000 7fea1a000640 10 client.27226 handle_command_reply: tid=0 2024-09-19T04:55:38.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.204+0000 7fea39c42740 1 client.27226 shutdown 2024-09-19T04:55:38.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.204+0000 7fea39c42740 1 -- 172.21.15.123:0/3990395096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 msgr2=0x564eeceeffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:38.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.204+0000 7fea39c42740 1 --2- 172.21.15.123:0/3990395096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 0x564eeceeffc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fea200cd7d0 tx=0x7fea2009a870 comp rx=0 tx=0).stop 2024-09-19T04:55:38.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.205+0000 7fea39c42740 1 -- 172.21.15.123:0/3990395096 shutdown_connections 2024-09-19T04:55:38.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.205+0000 7fea39c42740 1 --2- 172.21.15.123:0/3990395096 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x564eecf03070 0x564eecf05480 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.205+0000 7fea39c42740 1 --2- 172.21.15.123:0/3990395096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecef3020 0x564eecef0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.205+0000 7fea39c42740 1 --2- 172.21.15.123:0/3990395096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee8760 0x564eecef0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.205+0000 7fea1a000640 0 client.27226 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:55:38.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.205+0000 7fea39c42740 1 --2- 172.21.15.123:0/3990395096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 0x564eeceeffc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.205+0000 7fea39c42740 1 -- 172.21.15.123:0/3990395096 >> 172.21.15.123:0/3990395096 conn(0x564eecee43d0 msgr2=0x564eeceea0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:38.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.205+0000 7fea39c42740 1 -- 172.21.15.123:0/3990395096 shutdown_connections 2024-09-19T04:55:38.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.205+0000 7fea39c42740 1 -- 172.21.15.123:0/3990395096 wait complete. 2024-09-19T04:55:38.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.205+0000 7fea39c42740 20 client.27226 trim_cache size 0 max 16384 2024-09-19T04:55:38.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.219+0000 7fea39c42740 1 Processor -- start 2024-09-19T04:55:38.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.219+0000 7fea39c42740 1 -- start start 2024-09-19T04:55:38.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.219+0000 7fea39c42740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 0x564eecf0acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.219+0000 7fea39c42740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecee8760 0x564eecf0b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.219+0000 7fea39c42740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecef3020 0x564eeceed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.219+0000 7fea39c42740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea301af8c0 con 0x564eecee8760 2024-09-19T04:55:38.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.219+0000 7fea39c42740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea301b1400 con 0x564eecef3020 2024-09-19T04:55:38.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.219+0000 7fea39c42740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea301aff90 con 0x564eecee7e40 2024-09-19T04:55:38.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.219+0000 7fea2f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecee8760 0x564eecf0b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.219+0000 7fea35200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecef3020 0x564eeceed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.219+0000 7fea2f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecee8760 0x564eecf0b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59108/0 (socket says 172.21.15.123:59108) 2024-09-19T04:55:38.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.219+0000 7fea2f400640 1 -- 172.21.15.123:0/233808135 learned_addr learned my addr 172.21.15.123:0/233808135 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:38.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.219+0000 7fea2fe00640 1 --2- 172.21.15.123:0/233808135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 0x564eecf0acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.220+0000 7fea2f400640 1 -- 172.21.15.123:0/233808135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 msgr2=0x564eecf0acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:38.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.220+0000 7fea2f400640 1 --2- 172.21.15.123:0/233808135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 0x564eecf0acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.220+0000 7fea2f400640 1 -- 172.21.15.123:0/233808135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecef3020 msgr2=0x564eeceed220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:38.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.220+0000 7fea2f400640 1 --2- 172.21.15.123:0/233808135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecef3020 0x564eeceed220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.220+0000 7fea2f400640 1 -- 172.21.15.123:0/233808135 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eeceee510 con 0x564eecee8760 2024-09-19T04:55:38.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.220+0000 7fea2fe00640 1 --2- 172.21.15.123:0/233808135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 0x564eecf0acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:38.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.220+0000 7fea2f400640 1 --2- 172.21.15.123:0/233808135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecee8760 0x564eecf0b230 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fea1c000c00 tx=0x7fea1c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9cedafe4fdef8f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:38.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.220+0000 7fea24c00640 1 -- 172.21.15.123:0/233808135 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c01e040 con 0x564eecee8760 2024-09-19T04:55:38.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.220+0000 7fea24c00640 1 -- 172.21.15.123:0/233808135 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea1c005b30 con 0x564eecee8760 2024-09-19T04:55:38.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.220+0000 7fea24c00640 1 -- 172.21.15.123:0/233808135 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c003c90 con 0x564eecee8760 2024-09-19T04:55:38.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.221+0000 7fea39c42740 1 -- 172.21.15.123:0/233808135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecee8760 msgr2=0x564eecf0b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:38.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.221+0000 7fea39c42740 1 --2- 172.21.15.123:0/233808135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecee8760 0x564eecf0b230 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fea1c000c00 tx=0x7fea1c009620 comp rx=0 tx=0).stop 2024-09-19T04:55:38.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.221+0000 7fea39c42740 1 -- 172.21.15.123:0/233808135 shutdown_connections 2024-09-19T04:55:38.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.221+0000 7fea39c42740 1 --2- 172.21.15.123:0/233808135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecef3020 0x564eeceed220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.221+0000 7fea39c42740 1 --2- 172.21.15.123:0/233808135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecee8760 0x564eecf0b230 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.221+0000 7fea39c42740 1 --2- 172.21.15.123:0/233808135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecee7e40 0x564eecf0acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.221+0000 7fea39c42740 1 -- 172.21.15.123:0/233808135 >> 172.21.15.123:0/233808135 conn(0x564eecee43d0 msgr2=0x564eecef2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:38.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.221+0000 7fea39c42740 1 -- 172.21.15.123:0/233808135 shutdown_connections 2024-09-19T04:55:38.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.221+0000 7fea39c42740 1 -- 172.21.15.123:0/233808135 wait complete. 2024-09-19T04:55:38.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.221+0000 7fea39c42740 1 -- start start 2024-09-19T04:55:38.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.222+0000 7fea39c42740 1 Processor -- start 2024-09-19T04:55:38.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.222+0000 7fea39c42740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee7e40 0x564eecf66f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.222+0000 7fea39c42740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecee8760 0x564eecf67490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.222+0000 7fea39c42740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecef3020 0x564eecef7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.222+0000 7fea39c42740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea301b0d30 con 0x564eecee8760 2024-09-19T04:55:38.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.222+0000 7fea39c42740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea301b0660 con 0x564eecee7e40 2024-09-19T04:55:38.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.222+0000 7fea39c42740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea301ad8c0 con 0x564eecef3020 2024-09-19T04:55:38.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.222+0000 7fea39c42740 10 client.0 _ll_register_callbacks cb 0x7fea200d3880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:38.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.222+0000 7fea2fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee7e40 0x564eecf66f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.222+0000 7fea2fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee7e40 0x564eecf66f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:57194/0 (socket says 172.21.15.123:57194) 2024-09-19T04:55:38.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.222+0000 7fea2fe00640 1 -- 172.21.15.123:0/2527444272 learned_addr learned my addr 172.21.15.123:0/2527444272 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:38.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.223+0000 7fea2f400640 1 --2- 172.21.15.123:0/2527444272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecee8760 0x564eecf67490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.223+0000 7fea35200640 1 --2- 172.21.15.123:0/2527444272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecef3020 0x564eecef7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.223+0000 7fea2fe00640 1 -- 172.21.15.123:0/2527444272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecef3020 msgr2=0x564eecef7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:38.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.223+0000 7fea2fe00640 1 --2- 172.21.15.123:0/2527444272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecef3020 0x564eecef7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.223+0000 7fea2fe00640 1 -- 172.21.15.123:0/2527444272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecee8760 msgr2=0x564eecf67490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:38.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.223+0000 7fea2fe00640 1 --2- 172.21.15.123:0/2527444272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecee8760 0x564eecf67490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.223+0000 7fea2fe00640 1 -- 172.21.15.123:0/2527444272 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eecef7c30 con 0x564eecee7e40 2024-09-19T04:55:38.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.223+0000 7fea2fe00640 1 --2- 172.21.15.123:0/2527444272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee7e40 0x564eecf66f50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fea20098d30 tx=0x7fea200cdc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb6af8b48a33a2d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:38.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.223+0000 7fea1a000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:55:38.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.223+0000 7fea1a000640 1 -- 172.21.15.123:0/2527444272 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2009ca60 con 0x564eecee7e40 2024-09-19T04:55:38.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.223+0000 7fea1a000640 1 -- 172.21.15.123:0/2527444272 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fea200d3180 con 0x564eecee7e40 2024-09-19T04:55:38.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.223+0000 7fea39c42740 1 -- 172.21.15.123:0/2527444272 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea1c005b30 con 0x564eecee7e40 2024-09-19T04:55:38.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.224+0000 7fea1a000640 1 -- 172.21.15.123:0/2527444272 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fea2009c360 con 0x564eecee7e40 2024-09-19T04:55:38.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.224+0000 7fea39c42740 10 client.27212 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:38.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.224+0000 7fea39c42740 1 -- 172.21.15.123:0/2527444272 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564eecf6e7b0 con 0x564eecee7e40 2024-09-19T04:55:38.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.224+0000 7fea1a000640 1 -- 172.21.15.123:0/2527444272 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea1c005b30 con 0x564eecee7e40 2024-09-19T04:55:38.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.224+0000 7fea1a000640 1 -- 172.21.15.123:0/2527444272 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fea2009b220 con 0x564eecee7e40 2024-09-19T04:55:38.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.224+0000 7fea39c42740 10 client.27212 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:38.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.224+0000 7fea39c42740 10 client.27212 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:55:38.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.224+0000 7fea39c42740 20 client.27212 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:38.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.224+0000 7fea39c42740 1 --2- 172.21.15.123:0/2527444272 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x564eecf6ee70 0x564eecf71260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.224+0000 7fea39c42740 4 client.27212 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:55:38.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.224+0000 7fea39c42740 1 -- 172.21.15.123:0/2527444272 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564eece55e60 con 0x564eecf6ee70 2024-09-19T04:55:38.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.224+0000 7fea2f400640 1 --2- 172.21.15.123:0/2527444272 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x564eecf6ee70 0x564eecf71260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.225+0000 7fea2f400640 1 --2- 172.21.15.123:0/2527444272 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x564eecf6ee70 0x564eecf71260 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e1ff9fc522c83d server_cookie=183dc54d1dd1c9c6 in_seq=0 out_seq=0 2024-09-19T04:55:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.225+0000 7fea1a000640 10 client.27212 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:55:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.225+0000 7fea1a000640 1 -- 172.21.15.123:0/2527444272 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564eece55e60 con 0x564eecf6ee70 2024-09-19T04:55:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.225+0000 7fea1a000640 10 client.27212 handle_command_reply: tid=0 2024-09-19T04:55:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.225+0000 7fea39c42740 1 client.27212 shutdown 2024-09-19T04:55:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.225+0000 7fea39c42740 1 -- 172.21.15.123:0/2527444272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee7e40 msgr2=0x564eecf66f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.225+0000 7fea39c42740 1 --2- 172.21.15.123:0/2527444272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee7e40 0x564eecf66f50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fea20098d30 tx=0x7fea200cdc80 comp rx=0 tx=0).stop 2024-09-19T04:55:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.225+0000 7fea39c42740 1 -- 172.21.15.123:0/2527444272 shutdown_connections 2024-09-19T04:55:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.225+0000 7fea39c42740 1 --2- 172.21.15.123:0/2527444272 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x564eecf6ee70 0x564eecf71260 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.225+0000 7fea39c42740 1 --2- 172.21.15.123:0/2527444272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564eecef3020 0x564eecef7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.225+0000 7fea39c42740 1 --2- 172.21.15.123:0/2527444272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564eecee8760 0x564eecf67490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.225+0000 7fea39c42740 1 --2- 172.21.15.123:0/2527444272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eecee7e40 0x564eecf66f50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.225+0000 7fea1a000640 0 client.27212 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:55:38.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.225+0000 7fea39c42740 1 -- 172.21.15.123:0/2527444272 >> 172.21.15.123:0/2527444272 conn(0x564eecee43d0 msgr2=0x564eecee2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:38.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.226+0000 7fea39c42740 1 -- 172.21.15.123:0/2527444272 shutdown_connections 2024-09-19T04:55:38.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.226+0000 7fea39c42740 1 -- 172.21.15.123:0/2527444272 wait complete. 2024-09-19T04:55:38.302 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:55:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.226+0000 7fea39c42740 20 client.27212 trim_cache size 0 max 16384 2024-09-19T04:55:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.226+0000 7fea35c00640 1 -- 172.21.15.123:0/4221940999 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fea140a33b0 msgr2=0x7fea140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.226+0000 7fea35c00640 1 --2- 172.21.15.123:0/4221940999 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fea140a33b0 0x7fea140a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fea28004af0 tx=0x7fea28005fd0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.226+0000 7fea35c00640 1 -- 172.21.15.123:0/4221940999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea301019c0 msgr2=0x7fea300744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.226+0000 7fea35c00640 1 --2- 172.21.15.123:0/4221940999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea301019c0 0x7fea300744f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fea2000e9f0 tx=0x7fea2000edf0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.226+0000 7fea35c00640 1 -- 172.21.15.123:0/4221940999 shutdown_connections 2024-09-19T04:55:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.226+0000 7fea35c00640 1 --2- 172.21.15.123:0/4221940999 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fea140a33b0 0x7fea140a5870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.226+0000 7fea35c00640 1 --2- 172.21.15.123:0/4221940999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea301079c0 0x7fea30074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.226+0000 7fea35c00640 1 --2- 172.21.15.123:0/4221940999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea30102360 0x7fea30074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.226+0000 7fea35c00640 1 --2- 172.21.15.123:0/4221940999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea301019c0 0x7fea300744f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.226+0000 7fea35c00640 1 -- 172.21.15.123:0/4221940999 >> 172.21.15.123:0/4221940999 conn(0x7fea300fd6e0 msgr2=0x7fea300fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:38.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.227+0000 7fea35c00640 1 -- 172.21.15.123:0/4221940999 shutdown_connections 2024-09-19T04:55:38.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:38.227+0000 7fea35c00640 1 -- 172.21.15.123:0/4221940999 wait complete. 2024-09-19T04:55:39.247 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:55:39.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:39 smithi193 ceph-mon[35592]: pgmap v277: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:55:39.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:39 smithi183 ceph-mon[35333]: pgmap v277: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:55:39.614 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:39.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:39 smithi123 ceph-mon[30511]: pgmap v277: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:55:40.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.159+0000 7f3228600640 1 -- 172.21.15.123:0/3926599800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3220101de0 msgr2=0x7f32201021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:40.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.159+0000 7f3228600640 1 --2- 172.21.15.123:0/3926599800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3220101de0 0x7f32201021c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3214009900 tx=0x7f321402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.160+0000 7f3228600640 1 -- 172.21.15.123:0/3926599800 shutdown_connections 2024-09-19T04:55:40.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.160+0000 7f3228600640 1 --2- 172.21.15.123:0/3926599800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3220106f10 0x7f322010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.160+0000 7f3228600640 1 --2- 172.21.15.123:0/3926599800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3220102790 0x7f32201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.160+0000 7f3228600640 1 --2- 172.21.15.123:0/3926599800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3220101de0 0x7f32201021c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.160+0000 7f3228600640 1 -- 172.21.15.123:0/3926599800 >> 172.21.15.123:0/3926599800 conn(0x7f32200fd8c0 msgr2=0x7f32200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:40.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.160+0000 7f3228600640 1 -- 172.21.15.123:0/3926599800 shutdown_connections 2024-09-19T04:55:40.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.160+0000 7f3228600640 1 -- 172.21.15.123:0/3926599800 wait complete. 2024-09-19T04:55:40.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.161+0000 7f3228600640 1 Processor -- start 2024-09-19T04:55:40.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.161+0000 7f3228600640 1 -- start start 2024-09-19T04:55:40.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.161+0000 7f3228600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3220101de0 0x7f322019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.161+0000 7f3228600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3220102790 0x7f322019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.161+0000 7f3228600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3220106f10 0x7f322019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.161+0000 7f3228600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3220111fc0 con 0x7f3220101de0 2024-09-19T04:55:40.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.161+0000 7f3228600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3220111e40 con 0x7f3220102790 2024-09-19T04:55:40.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.161+0000 7f3228600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3220112140 con 0x7f3220106f10 2024-09-19T04:55:40.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.161+0000 7f3225c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3220101de0 0x7f322019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.161+0000 7f3225c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3220101de0 0x7f322019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59136/0 (socket says 172.21.15.123:59136) 2024-09-19T04:55:40.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.161+0000 7f3225c00640 1 -- 172.21.15.123:0/1451426643 learned_addr learned my addr 172.21.15.123:0/1451426643 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:40.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.162+0000 7f3226600640 1 --2- 172.21.15.123:0/1451426643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3220106f10 0x7f322019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.162+0000 7f3225200640 1 --2- 172.21.15.123:0/1451426643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3220102790 0x7f322019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.162+0000 7f3225c00640 1 -- 172.21.15.123:0/1451426643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3220106f10 msgr2=0x7f322019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:40.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.162+0000 7f3225c00640 1 --2- 172.21.15.123:0/1451426643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3220106f10 0x7f322019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.162+0000 7f3225c00640 1 -- 172.21.15.123:0/1451426643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3220102790 msgr2=0x7f322019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:40.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.162+0000 7f3225c00640 1 --2- 172.21.15.123:0/1451426643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3220102790 0x7f322019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.162+0000 7f3225c00640 1 -- 172.21.15.123:0/1451426643 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322019f600 con 0x7f3220101de0 2024-09-19T04:55:40.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.162+0000 7f3225200640 1 --2- 172.21.15.123:0/1451426643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3220102790 0x7f322019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:55:40.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.162+0000 7f3226600640 1 --2- 172.21.15.123:0/1451426643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3220106f10 0x7f322019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:55:40.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.162+0000 7f3225c00640 1 --2- 172.21.15.123:0/1451426643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3220101de0 0x7f322019a650 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3214004770 tx=0x7f3214003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:40.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.162+0000 7f321e000640 1 -- 172.21.15.123:0/1451426643 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321403f070 con 0x7f3220101de0 2024-09-19T04:55:40.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.162+0000 7f321e000640 1 -- 172.21.15.123:0/1451426643 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3214031ca0 con 0x7f3220101de0 2024-09-19T04:55:40.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.162+0000 7f321e000640 1 -- 172.21.15.123:0/1451426643 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214035030 con 0x7f3220101de0 2024-09-19T04:55:40.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.162+0000 7f3228600640 1 -- 172.21.15.123:0/1451426643 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322019f890 con 0x7f3220101de0 2024-09-19T04:55:40.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.163+0000 7f3228600640 1 -- 172.21.15.123:0/1451426643 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322019fd20 con 0x7f3220101de0 2024-09-19T04:55:40.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.165+0000 7f322afe2740 1 Processor -- start 2024-09-19T04:55:40.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.165+0000 7f321e000640 1 -- 172.21.15.123:0/1451426643 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3214042070 con 0x7f3220101de0 2024-09-19T04:55:40.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.166+0000 7f322afe2740 1 -- start start 2024-09-19T04:55:40.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.166+0000 7f322afe2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3bac70 0x56355d3bb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.166+0000 7f322afe2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3bb590 0x56355d3c5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.166+0000 7f322afe2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3c5e50 0x56355d3c8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.166+0000 7f322afe2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56355d3213e0 con 0x56355d3c5e50 2024-09-19T04:55:40.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.166+0000 7f322afe2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56355d2c8d80 con 0x56355d3bac70 2024-09-19T04:55:40.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.166+0000 7f322afe2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56355d31d3a0 con 0x56355d3bb590 2024-09-19T04:55:40.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.166+0000 7f3226600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3bac70 0x56355d3bb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.166+0000 7f3225c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3bb590 0x56355d3c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.166+0000 7f3226600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3bac70 0x56355d3bb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:57208/0 (socket says 172.21.15.123:57208) 2024-09-19T04:55:40.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.166+0000 7f3226600640 1 -- 172.21.15.123:0/4131152901 learned_addr learned my addr 172.21.15.123:0/4131152901 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:40.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.166+0000 7f3225200640 1 --2- 172.21.15.123:0/4131152901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3c5e50 0x56355d3c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.167+0000 7f3226600640 1 -- 172.21.15.123:0/4131152901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3bb590 msgr2=0x56355d3c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:40.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.167+0000 7f3226600640 1 --2- 172.21.15.123:0/4131152901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3bb590 0x56355d3c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.167+0000 7f3226600640 1 -- 172.21.15.123:0/4131152901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3c5e50 msgr2=0x56355d3c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:40.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.167+0000 7f3226600640 1 --2- 172.21.15.123:0/4131152901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3c5e50 0x56355d3c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.167+0000 7f3226600640 1 -- 172.21.15.123:0/4131152901 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56355d2e7ae0 con 0x56355d3bac70 2024-09-19T04:55:40.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.167+0000 7f3226600640 1 --2- 172.21.15.123:0/4131152901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3bac70 0x56355d3bb050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3218009fd0 tx=0x7f321800f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43c2eb2cd4b10c77 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:40.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.167+0000 7f321e000640 1 --2- 172.21.15.123:0/1451426643 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32080a33b0 0x7f32080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.167+0000 7f3212a00640 1 -- 172.21.15.123:0/4131152901 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218015070 con 0x56355d3bac70 2024-09-19T04:55:40.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.167+0000 7f321e000640 1 -- 172.21.15.123:0/1451426643 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f32140f0170 con 0x7f3220101de0 2024-09-19T04:55:40.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.167+0000 7f3225c00640 1 --2- 172.21.15.123:0/1451426643 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32080a33b0 0x7f32080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.167+0000 7f3212a00640 1 -- 172.21.15.123:0/4131152901 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3218011be0 con 0x56355d3bac70 2024-09-19T04:55:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.167+0000 7f3212a00640 1 -- 172.21.15.123:0/4131152901 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218010c30 con 0x56355d3bac70 2024-09-19T04:55:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.168+0000 7f322afe2740 1 -- 172.21.15.123:0/4131152901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3bac70 msgr2=0x56355d3bb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.168+0000 7f322afe2740 1 --2- 172.21.15.123:0/4131152901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3bac70 0x56355d3bb050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3218009fd0 tx=0x7f321800f7f0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.168+0000 7f3225c00640 1 --2- 172.21.15.123:0/1451426643 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32080a33b0 0x7f32080a5870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f32140ed970 tx=0x7f32140b8040 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.168+0000 7f322afe2740 1 -- 172.21.15.123:0/4131152901 shutdown_connections 2024-09-19T04:55:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.168+0000 7f322afe2740 1 --2- 172.21.15.123:0/4131152901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3c5e50 0x56355d3c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.168+0000 7f322afe2740 1 --2- 172.21.15.123:0/4131152901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3bb590 0x56355d3c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.168+0000 7f322afe2740 1 --2- 172.21.15.123:0/4131152901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3bac70 0x56355d3bb050 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.168+0000 7f322afe2740 1 -- 172.21.15.123:0/4131152901 >> 172.21.15.123:0/4131152901 conn(0x56355d3b7200 msgr2=0x56355d3b75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.168+0000 7f322afe2740 1 -- 172.21.15.123:0/4131152901 shutdown_connections 2024-09-19T04:55:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.168+0000 7f322afe2740 1 -- 172.21.15.123:0/4131152901 wait complete. 2024-09-19T04:55:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.169+0000 7f322afe2740 1 -- start start 2024-09-19T04:55:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.169+0000 7f322afe2740 1 Processor -- start 2024-09-19T04:55:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.169+0000 7f322afe2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bac70 0x56355d3c2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.169+0000 7f322afe2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3bb590 0x56355d3c3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.169+0000 7f322afe2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3c5e50 0x56355d3c3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.169+0000 7f322afe2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56355d3c8a20 con 0x56355d3bac70 2024-09-19T04:55:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.169+0000 7f322afe2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56355d2dc0f0 con 0x56355d3bb590 2024-09-19T04:55:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.169+0000 7f322afe2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56355d3c8ba0 con 0x56355d3c5e50 2024-09-19T04:55:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.169+0000 7f322afe2740 10 client.0 _ll_register_callbacks cb 0x56355d2c3bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.170+0000 7f3226600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bac70 0x56355d3c2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.170+0000 7f3226600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bac70 0x56355d3c2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59162/0 (socket says 172.21.15.123:59162) 2024-09-19T04:55:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.170+0000 7f3226600640 1 -- 172.21.15.123:0/562575572 learned_addr learned my addr 172.21.15.123:0/562575572 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:40.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.170+0000 7f3225200640 1 --2- 172.21.15.123:0/562575572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3bb590 0x56355d3c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.170+0000 7f3226600640 1 --2- 172.21.15.123:0/562575572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3c5e50 0x56355d3c3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.170+0000 7f3226600640 1 -- 172.21.15.123:0/562575572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3c5e50 msgr2=0x56355d3c3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:40.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.170+0000 7f3226600640 1 --2- 172.21.15.123:0/562575572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3c5e50 0x56355d3c3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.170+0000 7f3226600640 1 -- 172.21.15.123:0/562575572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3bb590 msgr2=0x56355d3c3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:40.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.170+0000 7f3226600640 1 --2- 172.21.15.123:0/562575572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3bb590 0x56355d3c3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.170+0000 7f3226600640 1 -- 172.21.15.123:0/562575572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56355d0923c0 con 0x56355d3bac70 2024-09-19T04:55:40.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.170+0000 7f3225200640 1 --2- 172.21.15.123:0/562575572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3bb590 0x56355d3c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:40.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.170+0000 7f3226600640 1 --2- 172.21.15.123:0/562575572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bac70 0x56355d3c2d00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3218000f80 tx=0x7f32180108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ed81a30bffa146 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:40.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.170+0000 7f3207400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:55:40.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.171+0000 7f3207400640 1 -- 172.21.15.123:0/562575572 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218015070 con 0x56355d3bac70 2024-09-19T04:55:40.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.171+0000 7f3207400640 1 -- 172.21.15.123:0/562575572 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f32180109b0 con 0x56355d3bac70 2024-09-19T04:55:40.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.171+0000 7f322afe2740 1 -- 172.21.15.123:0/562575572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3218011be0 con 0x56355d3bac70 2024-09-19T04:55:40.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.171+0000 7f3207400640 1 -- 172.21.15.123:0/562575572 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3218004e50 con 0x56355d3bac70 2024-09-19T04:55:40.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.171+0000 7f322afe2740 10 client.17133 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:40.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.171+0000 7f322afe2740 1 -- 172.21.15.123:0/562575572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56355d3d4a30 con 0x56355d3bac70 2024-09-19T04:55:40.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.171+0000 7f3207400640 1 -- 172.21.15.123:0/562575572 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3218011be0 con 0x56355d3bac70 2024-09-19T04:55:40.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.172+0000 7f3207400640 1 -- 172.21.15.123:0/562575572 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f321801a7f0 con 0x56355d3bac70 2024-09-19T04:55:40.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.172+0000 7f322afe2740 10 client.17133 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:40.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.172+0000 7f322afe2740 10 client.17133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:55:40.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.172+0000 7f322afe2740 20 client.17133 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:40.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.172+0000 7f322afe2740 1 --2- 172.21.15.123:0/562575572 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56355d3d5e40 0x56355d3d8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.172+0000 7f322afe2740 4 client.17133 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:55:40.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.172+0000 7f322afe2740 1 -- 172.21.15.123:0/562575572 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56355d32e950 con 0x56355d3d5e40 2024-09-19T04:55:40.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.172+0000 7f3225200640 1 --2- 172.21.15.123:0/562575572 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56355d3d5e40 0x56355d3d8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.173+0000 7f3225200640 1 --2- 172.21.15.123:0/562575572 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56355d3d5e40 0x56355d3d8230 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd9518501896022 server_cookie=420985a67625de51 in_seq=0 out_seq=0 2024-09-19T04:55:40.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.173+0000 7f3207400640 10 client.17133 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:40.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.174+0000 7f3207400640 1 -- 172.21.15.123:0/562575572 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56355d32e950 con 0x56355d3d5e40 2024-09-19T04:55:40.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.174+0000 7f3207400640 10 client.17133 handle_command_reply: tid=0 2024-09-19T04:55:40.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.174+0000 7f322afe2740 1 client.17133 shutdown 2024-09-19T04:55:40.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.174+0000 7f322afe2740 1 -- 172.21.15.123:0/562575572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bac70 msgr2=0x56355d3c2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:40.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.174+0000 7f322afe2740 1 --2- 172.21.15.123:0/562575572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bac70 0x56355d3c2d00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3218000f80 tx=0x7f32180108b0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.174+0000 7f322afe2740 1 -- 172.21.15.123:0/562575572 shutdown_connections 2024-09-19T04:55:40.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.174+0000 7f322afe2740 1 --2- 172.21.15.123:0/562575572 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56355d3d5e40 0x56355d3d8230 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.174+0000 7f322afe2740 1 --2- 172.21.15.123:0/562575572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3c5e50 0x56355d3c3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.174+0000 7f3207400640 0 client.17133 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:40.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.174+0000 7f322afe2740 1 --2- 172.21.15.123:0/562575572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3bb590 0x56355d3c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.174+0000 7f322afe2740 1 --2- 172.21.15.123:0/562575572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bac70 0x56355d3c2d00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.174+0000 7f322afe2740 1 -- 172.21.15.123:0/562575572 >> 172.21.15.123:0/562575572 conn(0x56355d3b7200 msgr2=0x56355d3b6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:40.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.174+0000 7f322afe2740 1 -- 172.21.15.123:0/562575572 shutdown_connections 2024-09-19T04:55:40.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.174+0000 7f322afe2740 1 -- 172.21.15.123:0/562575572 wait complete. 2024-09-19T04:55:40.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.174+0000 7f322afe2740 20 client.17133 trim_cache size 0 max 16384 2024-09-19T04:55:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.188+0000 7f322afe2740 1 Processor -- start 2024-09-19T04:55:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.189+0000 7f322afe2740 1 -- start start 2024-09-19T04:55:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.189+0000 7f322afe2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bac70 0x56355d3dd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.189+0000 7f322afe2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3bb590 0x56355d3ddda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.189+0000 7f322afe2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3c5e50 0x56355d3c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.189+0000 7f322afe2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32201afb30 con 0x56355d3bac70 2024-09-19T04:55:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.189+0000 7f322afe2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32201b1670 con 0x56355d3c5e50 2024-09-19T04:55:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.189+0000 7f322afe2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32201b0200 con 0x56355d3bb590 2024-09-19T04:55:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.189+0000 7f3226600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bac70 0x56355d3dd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.189+0000 7f3225200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3bb590 0x56355d3ddda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.189+0000 7f3226600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bac70 0x56355d3dd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59178/0 (socket says 172.21.15.123:59178) 2024-09-19T04:55:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.189+0000 7f3226600640 1 -- 172.21.15.123:0/3783094390 learned_addr learned my addr 172.21.15.123:0/3783094390 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.189+0000 7f3226600640 1 --2- 172.21.15.123:0/3783094390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3c5e50 0x56355d3c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.189+0000 7f3225200640 1 -- 172.21.15.123:0/3783094390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3c5e50 msgr2=0x56355d3c2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.189+0000 7f3225200640 1 --2- 172.21.15.123:0/3783094390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3c5e50 0x56355d3c2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.189+0000 7f3225200640 1 -- 172.21.15.123:0/3783094390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bac70 msgr2=0x56355d3dd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:40.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.189+0000 7f3225200640 1 --2- 172.21.15.123:0/3783094390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bac70 0x56355d3dd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.189+0000 7f3225200640 1 -- 172.21.15.123:0/3783094390 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56355d3b7030 con 0x56355d3bb590 2024-09-19T04:55:40.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.190+0000 7f3226600640 1 --2- 172.21.15.123:0/3783094390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bac70 0x56355d3dd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:40.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.190+0000 7f3225200640 1 --2- 172.21.15.123:0/3783094390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3bb590 0x56355d3ddda0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x56355d3d8800 tx=0x7f320c0149f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f728afd543da057 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:40.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.190+0000 7f3212000640 1 -- 172.21.15.123:0/3783094390 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c024070 con 0x56355d3bb590 2024-09-19T04:55:40.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.190+0000 7f3212000640 1 -- 172.21.15.123:0/3783094390 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f320c0047a0 con 0x56355d3bb590 2024-09-19T04:55:40.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.190+0000 7f3212000640 1 -- 172.21.15.123:0/3783094390 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c020880 con 0x56355d3bb590 2024-09-19T04:55:40.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.191+0000 7f322afe2740 1 -- 172.21.15.123:0/3783094390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3bb590 msgr2=0x56355d3ddda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:40.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.191+0000 7f322afe2740 1 --2- 172.21.15.123:0/3783094390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3bb590 0x56355d3ddda0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x56355d3d8800 tx=0x7f320c0149f0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.191+0000 7f322afe2740 1 -- 172.21.15.123:0/3783094390 shutdown_connections 2024-09-19T04:55:40.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.191+0000 7f322afe2740 1 --2- 172.21.15.123:0/3783094390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3c5e50 0x56355d3c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.191+0000 7f322afe2740 1 --2- 172.21.15.123:0/3783094390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3bb590 0x56355d3ddda0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.191+0000 7f322afe2740 1 --2- 172.21.15.123:0/3783094390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bac70 0x56355d3dd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.191+0000 7f322afe2740 1 -- 172.21.15.123:0/3783094390 >> 172.21.15.123:0/3783094390 conn(0x56355d3b7200 msgr2=0x56355d3c1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:40.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.191+0000 7f322afe2740 1 -- 172.21.15.123:0/3783094390 shutdown_connections 2024-09-19T04:55:40.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.191+0000 7f322afe2740 1 -- 172.21.15.123:0/3783094390 wait complete. 2024-09-19T04:55:40.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.192+0000 7f322afe2740 1 -- start start 2024-09-19T04:55:40.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.192+0000 7f322afe2740 1 Processor -- start 2024-09-19T04:55:40.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.192+0000 7f322afe2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3bac70 0x56355d3ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.192+0000 7f322afe2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bb590 0x56355d3ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.192+0000 7f322afe2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3c5e50 0x56355d3c9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.192+0000 7f322afe2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32201b0fa0 con 0x56355d3bb590 2024-09-19T04:55:40.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.192+0000 7f322afe2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32201b08d0 con 0x56355d3c5e50 2024-09-19T04:55:40.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.192+0000 7f322afe2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32201adb30 con 0x56355d3bac70 2024-09-19T04:55:40.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.192+0000 7f322afe2740 10 client.0 _ll_register_callbacks cb 0x7f32140f3b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:40.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.192+0000 7f3225200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bb590 0x56355d3ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.193+0000 7f3225200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bb590 0x56355d3ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59190/0 (socket says 172.21.15.123:59190) 2024-09-19T04:55:40.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.193+0000 7f3225200640 1 -- 172.21.15.123:0/918589137 learned_addr learned my addr 172.21.15.123:0/918589137 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:40.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.193+0000 7f3226600640 1 --2- 172.21.15.123:0/918589137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3bac70 0x56355d3ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.193+0000 7f3226600640 1 --2- 172.21.15.123:0/918589137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3c5e50 0x56355d3c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.193+0000 7f3225200640 1 -- 172.21.15.123:0/918589137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3bac70 msgr2=0x56355d3ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:40.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.193+0000 7f3225200640 1 --2- 172.21.15.123:0/918589137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3bac70 0x56355d3ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.193+0000 7f3225200640 1 -- 172.21.15.123:0/918589137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3c5e50 msgr2=0x56355d3c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:40.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.193+0000 7f3225200640 1 --2- 172.21.15.123:0/918589137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3c5e50 0x56355d3c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.193+0000 7f3225200640 1 -- 172.21.15.123:0/918589137 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56355d3c9f50 con 0x56355d3bb590 2024-09-19T04:55:40.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.193+0000 7f3226600640 1 --2- 172.21.15.123:0/918589137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3bac70 0x56355d3ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:40.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.193+0000 7f3225200640 1 --2- 172.21.15.123:0/918589137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bb590 0x56355d3ce700 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f320c005f60 tx=0x7f320c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76419e4042f481b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:40.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.193+0000 7f3207400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:55:40.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.194+0000 7f3207400640 1 -- 172.21.15.123:0/918589137 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c036060 con 0x56355d3bb590 2024-09-19T04:55:40.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.194+0000 7f3207400640 1 -- 172.21.15.123:0/918589137 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f320c005220 con 0x56355d3bb590 2024-09-19T04:55:40.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.194+0000 7f322afe2740 1 -- 172.21.15.123:0/918589137 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f320c0047a0 con 0x56355d3bb590 2024-09-19T04:55:40.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.194+0000 7f3207400640 1 -- 172.21.15.123:0/918589137 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f320c024040 con 0x56355d3bb590 2024-09-19T04:55:40.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.194+0000 7f322afe2740 10 client.17139 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:40.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.194+0000 7f322afe2740 1 -- 172.21.15.123:0/918589137 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56355d4404c0 con 0x56355d3bb590 2024-09-19T04:55:40.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.194+0000 7f3207400640 1 -- 172.21.15.123:0/918589137 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f320c0047a0 con 0x56355d3bb590 2024-09-19T04:55:40.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.194+0000 7f3207400640 1 -- 172.21.15.123:0/918589137 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f320c011560 con 0x56355d3bb590 2024-09-19T04:55:40.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.195+0000 7f322afe2740 10 client.17139 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:40.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.195+0000 7f322afe2740 10 client.17139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:55:40.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.195+0000 7f322afe2740 20 client.17139 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:40.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.195+0000 7f322afe2740 1 --2- 172.21.15.123:0/918589137 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56355d440b60 0x56355d442f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.195+0000 7f322afe2740 4 client.17139 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:55:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.195+0000 7f322afe2740 1 -- 172.21.15.123:0/918589137 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56355d328e60 con 0x56355d440b60 2024-09-19T04:55:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.195+0000 7f3226600640 1 --2- 172.21.15.123:0/918589137 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56355d440b60 0x56355d442f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.196+0000 7f3207400640 10 client.17139 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.195+0000 7f3226600640 1 --2- 172.21.15.123:0/918589137 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56355d440b60 0x56355d442f80 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2931b4e2584c79 server_cookie=5cdd126d7856c201 in_seq=0 out_seq=0 2024-09-19T04:55:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.196+0000 7f3207400640 1 -- 172.21.15.123:0/918589137 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56355d328e60 con 0x56355d440b60 2024-09-19T04:55:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.196+0000 7f3207400640 10 client.17139 handle_command_reply: tid=0 2024-09-19T04:55:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.196+0000 7f322afe2740 1 client.17139 shutdown 2024-09-19T04:55:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.196+0000 7f322afe2740 1 -- 172.21.15.123:0/918589137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bb590 msgr2=0x56355d3ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.196+0000 7f322afe2740 1 --2- 172.21.15.123:0/918589137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bb590 0x56355d3ce700 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f320c005f60 tx=0x7f320c004ab0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.196+0000 7f322afe2740 1 -- 172.21.15.123:0/918589137 shutdown_connections 2024-09-19T04:55:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.196+0000 7f322afe2740 1 --2- 172.21.15.123:0/918589137 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x56355d440b60 0x56355d442f80 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.196+0000 7f322afe2740 1 --2- 172.21.15.123:0/918589137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56355d3c5e50 0x56355d3c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.196+0000 7f322afe2740 1 --2- 172.21.15.123:0/918589137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56355d3bb590 0x56355d3ce700 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.196+0000 7f3207400640 0 client.17139 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.196+0000 7f322afe2740 1 --2- 172.21.15.123:0/918589137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355d3bac70 0x56355d3ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.196+0000 7f322afe2740 1 -- 172.21.15.123:0/918589137 >> 172.21.15.123:0/918589137 conn(0x56355d3b7200 msgr2=0x56355d3c4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.196+0000 7f322afe2740 1 -- 172.21.15.123:0/918589137 shutdown_connections 2024-09-19T04:55:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.196+0000 7f322afe2740 1 -- 172.21.15.123:0/918589137 wait complete. 2024-09-19T04:55:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.196+0000 7f322afe2740 20 client.17139 trim_cache size 0 max 16384 2024-09-19T04:55:40.216 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:55:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.197+0000 7f3228600640 1 -- 172.21.15.123:0/1451426643 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32080a33b0 msgr2=0x7f32080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.197+0000 7f3228600640 1 --2- 172.21.15.123:0/1451426643 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32080a33b0 0x7f32080a5870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f32140ed970 tx=0x7f32140b8040 comp rx=0 tx=0).stop 2024-09-19T04:55:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.197+0000 7f3228600640 1 -- 172.21.15.123:0/1451426643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3220101de0 msgr2=0x7f322019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.197+0000 7f3228600640 1 --2- 172.21.15.123:0/1451426643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3220101de0 0x7f322019a650 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3214004770 tx=0x7f3214003c10 comp rx=0 tx=0).stop 2024-09-19T04:55:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.197+0000 7f3228600640 1 -- 172.21.15.123:0/1451426643 shutdown_connections 2024-09-19T04:55:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.197+0000 7f3228600640 1 --2- 172.21.15.123:0/1451426643 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f32080a33b0 0x7f32080a5870 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.197+0000 7f3228600640 1 --2- 172.21.15.123:0/1451426643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3220106f10 0x7f322019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.197+0000 7f3228600640 1 --2- 172.21.15.123:0/1451426643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3220102790 0x7f322019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.197+0000 7f3228600640 1 --2- 172.21.15.123:0/1451426643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3220101de0 0x7f322019a650 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.197+0000 7f3228600640 1 -- 172.21.15.123:0/1451426643 >> 172.21.15.123:0/1451426643 conn(0x7f32200fd8c0 msgr2=0x7f32200feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.197+0000 7f3228600640 1 -- 172.21.15.123:0/1451426643 shutdown_connections 2024-09-19T04:55:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:40.198+0000 7f3228600640 1 -- 172.21.15.123:0/1451426643 wait complete. 2024-09-19T04:55:41.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:41 smithi123 ceph-mon[30511]: pgmap v278: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:41.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:41 smithi193 ceph-mon[35592]: pgmap v278: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:41.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:41 smithi183 ceph-mon[35333]: pgmap v278: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:42.045 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:55:42.045 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:55:42.421 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.964+0000 7f079d400640 1 -- 172.21.15.123:0/2723810368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0798074f40 msgr2=0x7f079810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.964+0000 7f079d400640 1 --2- 172.21.15.123:0/2723810368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0798074f40 0x7f079810de80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0784009900 tx=0x7f078402a480 comp rx=0 tx=0).stop 2024-09-19T04:55:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.964+0000 7f079d400640 1 -- 172.21.15.123:0/2723810368 shutdown_connections 2024-09-19T04:55:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.964+0000 7f079d400640 1 --2- 172.21.15.123:0/2723810368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f079810e3c0 0x7f0798110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.964+0000 7f079d400640 1 --2- 172.21.15.123:0/2723810368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0798074f40 0x7f079810de80 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.964+0000 7f079d400640 1 --2- 172.21.15.123:0/2723810368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0798074620 0x7f0798074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:42.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.964+0000 7f079d400640 1 -- 172.21.15.123:0/2723810368 >> 172.21.15.123:0/2723810368 conn(0x7f07980fd8c0 msgr2=0x7f07980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:42.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.964+0000 7f079d400640 1 -- 172.21.15.123:0/2723810368 shutdown_connections 2024-09-19T04:55:42.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.964+0000 7f079d400640 1 -- 172.21.15.123:0/2723810368 wait complete. 2024-09-19T04:55:42.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.965+0000 7f079d400640 1 Processor -- start 2024-09-19T04:55:42.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.965+0000 7f079d400640 1 -- start start 2024-09-19T04:55:42.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.965+0000 7f079d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0798074620 0x7f079819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.965+0000 7f079d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0798074f40 0x7f079819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.965+0000 7f079d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f079810e3c0 0x7f079819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.965+0000 7f079d400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0798114190 con 0x7f0798074f40 2024-09-19T04:55:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.965+0000 7f079d400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0798114010 con 0x7f0798074620 2024-09-19T04:55:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.965+0000 7f079d400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0798114310 con 0x7f079810e3c0 2024-09-19T04:55:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.966+0000 7f0795e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0798074f40 0x7f079819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.966+0000 7f0795e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0798074f40 0x7f079819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40426/0 (socket says 172.21.15.123:40426) 2024-09-19T04:55:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.966+0000 7f0795e00640 1 -- 172.21.15.123:0/1599818623 learned_addr learned my addr 172.21.15.123:0/1599818623 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.966+0000 7f0796800640 1 --2- 172.21.15.123:0/1599818623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0798074620 0x7f079819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.966+0000 7f0797200640 1 --2- 172.21.15.123:0/1599818623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f079810e3c0 0x7f079819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.966+0000 7f0795e00640 1 -- 172.21.15.123:0/1599818623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f079810e3c0 msgr2=0x7f079819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.966+0000 7f0795e00640 1 --2- 172.21.15.123:0/1599818623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f079810e3c0 0x7f079819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.966+0000 7f0795e00640 1 -- 172.21.15.123:0/1599818623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0798074620 msgr2=0x7f079819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.966+0000 7f0795e00640 1 --2- 172.21.15.123:0/1599818623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0798074620 0x7f079819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.966+0000 7f0795e00640 1 -- 172.21.15.123:0/1599818623 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f079819f620 con 0x7f0798074f40 2024-09-19T04:55:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.966+0000 7f0796800640 1 --2- 172.21.15.123:0/1599818623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0798074620 0x7f079819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.967+0000 7f0795e00640 1 --2- 172.21.15.123:0/1599818623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0798074f40 0x7f079819abb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0784004770 tx=0x7f0784003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.967+0000 7f078f400640 1 -- 172.21.15.123:0/1599818623 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078403f070 con 0x7f0798074f40 2024-09-19T04:55:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.967+0000 7f078f400640 1 -- 172.21.15.123:0/1599818623 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0784031ca0 con 0x7f0798074f40 2024-09-19T04:55:42.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.967+0000 7f079d400640 1 -- 172.21.15.123:0/1599818623 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f079819f8b0 con 0x7f0798074f40 2024-09-19T04:55:42.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.967+0000 7f078f400640 1 -- 172.21.15.123:0/1599818623 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784035030 con 0x7f0798074f40 2024-09-19T04:55:42.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.967+0000 7f079d400640 1 -- 172.21.15.123:0/1599818623 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07981a71f0 con 0x7f0798074f40 2024-09-19T04:55:42.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.970+0000 7f079fe7a740 1 Processor -- start 2024-09-19T04:55:42.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.970+0000 7f078f400640 1 -- 172.21.15.123:0/1599818623 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f0784042070 con 0x7f0798074f40 2024-09-19T04:55:42.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.970+0000 7f079fe7a740 1 -- start start 2024-09-19T04:55:42.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.971+0000 7f079fe7a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a15ac0 0x560ab9a15ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:42.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.971+0000 7f079fe7a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a163e0 0x560ab9a20760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:42.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.971+0000 7f079fe7a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a20ca0 0x560ab9a23090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:42.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.971+0000 7f079fe7a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560ab997c290 con 0x560ab9a15ac0 2024-09-19T04:55:42.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.971+0000 7f079fe7a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560ab9923b40 con 0x560ab9a163e0 2024-09-19T04:55:42.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.971+0000 7f079fe7a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560ab9978250 con 0x560ab9a20ca0 2024-09-19T04:55:42.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.971+0000 7f0797200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a15ac0 0x560ab9a15ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:42.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.971+0000 7f0796800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a163e0 0x560ab9a20760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:42.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.971+0000 7f0797200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a15ac0 0x560ab9a15ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40434/0 (socket says 172.21.15.123:40434) 2024-09-19T04:55:42.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.971+0000 7f0797200640 1 -- 172.21.15.123:0/1458883368 learned_addr learned my addr 172.21.15.123:0/1458883368 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:42.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.971+0000 7f0795e00640 1 --2- 172.21.15.123:0/1458883368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a20ca0 0x560ab9a23090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:42.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.972+0000 7f0797200640 1 -- 172.21.15.123:0/1458883368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a20ca0 msgr2=0x560ab9a23090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:42.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.972+0000 7f0797200640 1 --2- 172.21.15.123:0/1458883368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a20ca0 0x560ab9a23090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.972+0000 7f0797200640 1 -- 172.21.15.123:0/1458883368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a163e0 msgr2=0x560ab9a20760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.972+0000 7f0797200640 1 --2- 172.21.15.123:0/1458883368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a163e0 0x560ab9a20760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.972+0000 7f0797200640 1 -- 172.21.15.123:0/1458883368 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab96ed6a0 con 0x560ab9a15ac0 2024-09-19T04:55:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.972+0000 7f0797200640 1 --2- 172.21.15.123:0/1458883368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a15ac0 0x560ab9a15ea0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0790009f30 tx=0x7f07900118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cca60f6a21f2e0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.972+0000 7f078f400640 1 --2- 172.21.15.123:0/1599818623 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f077c0a3430 0x7f077c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.972+0000 7f078f400640 1 -- 172.21.15.123:0/1599818623 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f07840f02c0 con 0x7f0798074f40 2024-09-19T04:55:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.972+0000 7f0783400640 1 -- 172.21.15.123:0/1458883368 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790004d20 con 0x560ab9a15ac0 2024-09-19T04:55:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.972+0000 7f0796800640 1 --2- 172.21.15.123:0/1599818623 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f077c0a3430 0x7f077c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.972+0000 7f0783400640 1 -- 172.21.15.123:0/1458883368 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0790004050 con 0x560ab9a15ac0 2024-09-19T04:55:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.972+0000 7f0783400640 1 -- 172.21.15.123:0/1458883368 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790005280 con 0x560ab9a15ac0 2024-09-19T04:55:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.973+0000 7f079fe7a740 1 -- 172.21.15.123:0/1458883368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a15ac0 msgr2=0x560ab9a15ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.973+0000 7f0796800640 1 --2- 172.21.15.123:0/1599818623 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f077c0a3430 0x7f077c0a58f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x560ab9a12eb0 tx=0x7f0788005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.973+0000 7f079fe7a740 1 --2- 172.21.15.123:0/1458883368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a15ac0 0x560ab9a15ea0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0790009f30 tx=0x7f07900118d0 comp rx=0 tx=0).stop 2024-09-19T04:55:42.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.973+0000 7f079fe7a740 1 -- 172.21.15.123:0/1458883368 shutdown_connections 2024-09-19T04:55:42.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.973+0000 7f079fe7a740 1 --2- 172.21.15.123:0/1458883368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a20ca0 0x560ab9a23090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:42.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.973+0000 7f079fe7a740 1 --2- 172.21.15.123:0/1458883368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a163e0 0x560ab9a20760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:42.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.973+0000 7f079fe7a740 1 --2- 172.21.15.123:0/1458883368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a15ac0 0x560ab9a15ea0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:42.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.973+0000 7f079fe7a740 1 -- 172.21.15.123:0/1458883368 >> 172.21.15.123:0/1458883368 conn(0x560ab9a12050 msgr2=0x560ab9a12430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:42.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.973+0000 7f079fe7a740 1 -- 172.21.15.123:0/1458883368 shutdown_connections 2024-09-19T04:55:42.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.974+0000 7f079fe7a740 1 -- 172.21.15.123:0/1458883368 wait complete. 2024-09-19T04:55:42.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.974+0000 7f079fe7a740 1 -- start start 2024-09-19T04:55:42.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.974+0000 7f079fe7a740 1 Processor -- start 2024-09-19T04:55:42.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.975+0000 7f079fe7a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a15ac0 0x560ab9a20640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:42.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.975+0000 7f079fe7a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a163e0 0x560ab9a1d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:42.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.975+0000 7f079fe7a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a20ca0 0x560ab9a1de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:42.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.975+0000 7f079fe7a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560ab9a23910 con 0x560ab9a163e0 2024-09-19T04:55:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.975+0000 7f079fe7a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560ab9936fa0 con 0x560ab9a20ca0 2024-09-19T04:55:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.975+0000 7f079fe7a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560ab9a23a90 con 0x560ab9a15ac0 2024-09-19T04:55:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.975+0000 7f079fe7a740 10 client.0 _ll_register_callbacks cb 0x560ab968aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.975+0000 7f0795e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a163e0 0x560ab9a1d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.975+0000 7f0795e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a163e0 0x560ab9a1d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40448/0 (socket says 172.21.15.123:40448) 2024-09-19T04:55:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.975+0000 7f0795e00640 1 -- 172.21.15.123:0/23529299 learned_addr learned my addr 172.21.15.123:0/23529299 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.975+0000 7f0796800640 1 --2- 172.21.15.123:0/23529299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a15ac0 0x560ab9a20640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.975+0000 7f0797200640 1 --2- 172.21.15.123:0/23529299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a20ca0 0x560ab9a1de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.975+0000 7f0795e00640 1 -- 172.21.15.123:0/23529299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a15ac0 msgr2=0x560ab9a20640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.975+0000 7f0795e00640 1 --2- 172.21.15.123:0/23529299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a15ac0 0x560ab9a20640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.975+0000 7f0795e00640 1 -- 172.21.15.123:0/23529299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a20ca0 msgr2=0x560ab9a1de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.975+0000 7f0795e00640 1 --2- 172.21.15.123:0/23529299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a20ca0 0x560ab9a1de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.975+0000 7f0795e00640 1 -- 172.21.15.123:0/23529299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ab9a1e6c0 con 0x560ab9a163e0 2024-09-19T04:55:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.976+0000 7f0795e00640 1 --2- 172.21.15.123:0/23529299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a163e0 0x560ab9a1d8f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0784000c00 tx=0x7f07840b8350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f272a631403329a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.976+0000 7f0777e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:55:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.976+0000 7f0777e00640 1 -- 172.21.15.123:0/23529299 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784049050 con 0x560ab9a163e0 2024-09-19T04:55:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.976+0000 7f0777e00640 1 -- 172.21.15.123:0/23529299 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f078403b4f0 con 0x560ab9a163e0 2024-09-19T04:55:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.976+0000 7f079fe7a740 1 -- 172.21.15.123:0/23529299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0790004050 con 0x560ab9a163e0 2024-09-19T04:55:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.976+0000 7f0777e00640 1 -- 172.21.15.123:0/23529299 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f07840bc1f0 con 0x560ab9a163e0 2024-09-19T04:55:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.976+0000 7f079fe7a740 10 client.17151 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.976+0000 7f079fe7a740 1 -- 172.21.15.123:0/23529299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560ab9a2f890 con 0x560ab9a163e0 2024-09-19T04:55:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.977+0000 7f0777e00640 1 -- 172.21.15.123:0/23529299 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0790004050 con 0x560ab9a163e0 2024-09-19T04:55:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.977+0000 7f0777e00640 1 -- 172.21.15.123:0/23529299 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f07840edc30 con 0x560ab9a163e0 2024-09-19T04:55:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.977+0000 7f079fe7a740 10 client.17151 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.977+0000 7f079fe7a740 10 client.17151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:55:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.977+0000 7f079fe7a740 20 client.17151 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.977+0000 7f079fe7a740 1 --2- 172.21.15.123:0/23529299 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x560ab9a30cf0 0x560ab9a33150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:42.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.977+0000 7f079fe7a740 4 client.17151 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:55:42.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.977+0000 7f079fe7a740 1 -- 172.21.15.123:0/23529299 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ab9989800 con 0x560ab9a30cf0 2024-09-19T04:55:42.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.977+0000 7f0796800640 1 --2- 172.21.15.123:0/23529299 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x560ab9a30cf0 0x560ab9a33150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:42.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.978+0000 7f0796800640 1 --2- 172.21.15.123:0/23529299 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x560ab9a30cf0 0x560ab9a33150 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af9bc314f7dce464 server_cookie=c3a719c8108224fe in_seq=0 out_seq=0 2024-09-19T04:55:42.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.978+0000 7f0777e00640 10 client.17151 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:42.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.979+0000 7f0777e00640 1 -- 172.21.15.123:0/23529299 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ab9989800 con 0x560ab9a30cf0 2024-09-19T04:55:42.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.979+0000 7f0777e00640 10 client.17151 handle_command_reply: tid=0 2024-09-19T04:55:42.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.979+0000 7f079fe7a740 1 client.17151 shutdown 2024-09-19T04:55:42.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.979+0000 7f079fe7a740 1 -- 172.21.15.123:0/23529299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a163e0 msgr2=0x560ab9a1d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:42.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.979+0000 7f079fe7a740 1 --2- 172.21.15.123:0/23529299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a163e0 0x560ab9a1d8f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0784000c00 tx=0x7f07840b8350 comp rx=0 tx=0).stop 2024-09-19T04:55:42.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.979+0000 7f079fe7a740 1 -- 172.21.15.123:0/23529299 shutdown_connections 2024-09-19T04:55:42.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.979+0000 7f079fe7a740 1 --2- 172.21.15.123:0/23529299 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x560ab9a30cf0 0x560ab9a33150 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:42.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.979+0000 7f079fe7a740 1 --2- 172.21.15.123:0/23529299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a20ca0 0x560ab9a1de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:42.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.979+0000 7f079fe7a740 1 --2- 172.21.15.123:0/23529299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a163e0 0x560ab9a1d8f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:42.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.979+0000 7f079fe7a740 1 --2- 172.21.15.123:0/23529299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a15ac0 0x560ab9a20640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:42.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.979+0000 7f079fe7a740 1 -- 172.21.15.123:0/23529299 >> 172.21.15.123:0/23529299 conn(0x560ab9a12050 msgr2=0x560ab9a111c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:42.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.979+0000 7f0777e00640 0 client.17151 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:42.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.979+0000 7f079fe7a740 1 -- 172.21.15.123:0/23529299 shutdown_connections 2024-09-19T04:55:42.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.979+0000 7f079fe7a740 1 -- 172.21.15.123:0/23529299 wait complete. 2024-09-19T04:55:42.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.979+0000 7f079fe7a740 20 client.17151 trim_cache size 0 max 16384 2024-09-19T04:55:42.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.997+0000 7f079fe7a740 1 Processor -- start 2024-09-19T04:55:42.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.997+0000 7f079fe7a740 1 -- start start 2024-09-19T04:55:42.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.997+0000 7f079fe7a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a15ac0 0x560ab9a110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:42.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.997+0000 7f079fe7a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a163e0 0x560ab9a28ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.997+0000 7f079fe7a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a20ca0 0x560ab9a29220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.997+0000 7f079fe7a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f07981afac0 con 0x560ab9a15ac0 2024-09-19T04:55:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.997+0000 7f079fe7a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07981b1600 con 0x560ab9a163e0 2024-09-19T04:55:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.997+0000 7f079fe7a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f07981b0190 con 0x560ab9a20ca0 2024-09-19T04:55:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.998+0000 7f0796800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a15ac0 0x560ab9a110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.998+0000 7f0796800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a15ac0 0x560ab9a110b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40462/0 (socket says 172.21.15.123:40462) 2024-09-19T04:55:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.998+0000 7f0796800640 1 -- 172.21.15.123:0/3857552699 learned_addr learned my addr 172.21.15.123:0/3857552699 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.998+0000 7f0797200640 1 --2- 172.21.15.123:0/3857552699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a20ca0 0x560ab9a29220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.998+0000 7f0795e00640 1 --2- 172.21.15.123:0/3857552699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a163e0 0x560ab9a28ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.998+0000 7f0796800640 1 -- 172.21.15.123:0/3857552699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a20ca0 msgr2=0x560ab9a29220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.998+0000 7f0796800640 1 --2- 172.21.15.123:0/3857552699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a20ca0 0x560ab9a29220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.998+0000 7f0796800640 1 -- 172.21.15.123:0/3857552699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a163e0 msgr2=0x560ab9a28ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.998+0000 7f0796800640 1 --2- 172.21.15.123:0/3857552699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a163e0 0x560ab9a28ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.998+0000 7f0796800640 1 -- 172.21.15.123:0/3857552699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab9a1e890 con 0x560ab9a15ac0 2024-09-19T04:55:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.998+0000 7f0795e00640 1 --2- 172.21.15.123:0/3857552699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a163e0 0x560ab9a28ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:55:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.998+0000 7f0796800640 1 --2- 172.21.15.123:0/3857552699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a15ac0 0x560ab9a110b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x560ab9a339c0 tx=0x7f0788017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8add2fa6535411f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:43.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.998+0000 7f0782a00640 1 -- 172.21.15.123:0/3857552699 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078800fe00 con 0x560ab9a15ac0 2024-09-19T04:55:43.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.999+0000 7f0782a00640 1 -- 172.21.15.123:0/3857552699 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f078801d860 con 0x560ab9a15ac0 2024-09-19T04:55:43.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.999+0000 7f0782a00640 1 -- 172.21.15.123:0/3857552699 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0788005480 con 0x560ab9a15ac0 2024-09-19T04:55:43.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.999+0000 7f079fe7a740 1 -- 172.21.15.123:0/3857552699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a15ac0 msgr2=0x560ab9a110b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:43.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.999+0000 7f079fe7a740 1 --2- 172.21.15.123:0/3857552699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a15ac0 0x560ab9a110b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x560ab9a339c0 tx=0x7f0788017000 comp rx=0 tx=0).stop 2024-09-19T04:55:43.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.999+0000 7f079fe7a740 1 -- 172.21.15.123:0/3857552699 shutdown_connections 2024-09-19T04:55:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.999+0000 7f079fe7a740 1 --2- 172.21.15.123:0/3857552699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a20ca0 0x560ab9a29220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.999+0000 7f079fe7a740 1 --2- 172.21.15.123:0/3857552699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a163e0 0x560ab9a28ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.999+0000 7f079fe7a740 1 --2- 172.21.15.123:0/3857552699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a15ac0 0x560ab9a110b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.999+0000 7f079fe7a740 1 -- 172.21.15.123:0/3857552699 >> 172.21.15.123:0/3857552699 conn(0x560ab9a12050 msgr2=0x560ab9a109a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.999+0000 7f079fe7a740 1 -- 172.21.15.123:0/3857552699 shutdown_connections 2024-09-19T04:55:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:42.999+0000 7f079fe7a740 1 -- 172.21.15.123:0/3857552699 wait complete. 2024-09-19T04:55:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.000+0000 7f079fe7a740 1 -- start start 2024-09-19T04:55:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.000+0000 7f079fe7a740 1 Processor -- start 2024-09-19T04:55:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.000+0000 7f079fe7a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a15ac0 0x560ab9a294f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.000+0000 7f079fe7a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a163e0 0x560ab9a278f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.000+0000 7f079fe7a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a20ca0 0x560ab9a27f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.000+0000 7f079fe7a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f07981b0f30 con 0x560ab9a20ca0 2024-09-19T04:55:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.000+0000 7f079fe7a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07981b0860 con 0x560ab9a15ac0 2024-09-19T04:55:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.000+0000 7f079fe7a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f07981adac0 con 0x560ab9a163e0 2024-09-19T04:55:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.000+0000 7f079fe7a740 10 client.0 _ll_register_callbacks cb 0x7f07840f3a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.001+0000 7f0796800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a15ac0 0x560ab9a294f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.001+0000 7f0796800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a15ac0 0x560ab9a294f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:44354/0 (socket says 172.21.15.123:44354) 2024-09-19T04:55:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.001+0000 7f0796800640 1 -- 172.21.15.123:0/2630410435 learned_addr learned my addr 172.21.15.123:0/2630410435 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.001+0000 7f0795e00640 1 --2- 172.21.15.123:0/2630410435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a163e0 0x560ab9a278f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.001+0000 7f0797200640 1 --2- 172.21.15.123:0/2630410435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a20ca0 0x560ab9a27f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.001+0000 7f0796800640 1 -- 172.21.15.123:0/2630410435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a163e0 msgr2=0x560ab9a278f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.001+0000 7f0796800640 1 --2- 172.21.15.123:0/2630410435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a163e0 0x560ab9a278f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.001+0000 7f0796800640 1 -- 172.21.15.123:0/2630410435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a20ca0 msgr2=0x560ab9a27f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.001+0000 7f0796800640 1 --2- 172.21.15.123:0/2630410435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a20ca0 0x560ab9a27f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.001+0000 7f0796800640 1 -- 172.21.15.123:0/2630410435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ab9a286f0 con 0x560ab9a15ac0 2024-09-19T04:55:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.001+0000 7f0796800640 1 --2- 172.21.15.123:0/2630410435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a15ac0 0x560ab9a294f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0788019040 tx=0x7f078800ff10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dc4658e1abead79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:43.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.001+0000 7f0777e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:55:43.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.002+0000 7f0777e00640 1 -- 172.21.15.123:0/2630410435 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078801ddd0 con 0x560ab9a15ac0 2024-09-19T04:55:43.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.002+0000 7f079fe7a740 1 -- 172.21.15.123:0/2630410435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f078801d860 con 0x560ab9a15ac0 2024-09-19T04:55:43.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.002+0000 7f0777e00640 1 -- 172.21.15.123:0/2630410435 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f07880051c0 con 0x560ab9a15ac0 2024-09-19T04:55:43.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.002+0000 7f0777e00640 1 -- 172.21.15.123:0/2630410435 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f07880053c0 con 0x560ab9a15ac0 2024-09-19T04:55:43.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.002+0000 7f079fe7a740 10 client.27248 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:43.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.002+0000 7f079fe7a740 1 -- 172.21.15.123:0/2630410435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560ab9a9cb50 con 0x560ab9a15ac0 2024-09-19T04:55:43.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.003+0000 7f0777e00640 1 -- 172.21.15.123:0/2630410435 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f078801d860 con 0x560ab9a15ac0 2024-09-19T04:55:43.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.003+0000 7f0777e00640 1 -- 172.21.15.123:0/2630410435 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f078802b530 con 0x560ab9a15ac0 2024-09-19T04:55:43.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.003+0000 7f079fe7a740 10 client.27248 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:43.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.003+0000 7f079fe7a740 10 client.27248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:55:43.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.003+0000 7f079fe7a740 20 client.27248 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:43.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.003+0000 7f079fe7a740 1 --2- 172.21.15.123:0/2630410435 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x560ab9a9d250 0x560ab9a9f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:43.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.003+0000 7f079fe7a740 4 client.27248 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:55:43.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.003+0000 7f079fe7a740 1 -- 172.21.15.123:0/2630410435 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560ab9983d10 con 0x560ab9a9d250 2024-09-19T04:55:43.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.003+0000 7f0795e00640 1 --2- 172.21.15.123:0/2630410435 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x560ab9a9d250 0x560ab9a9f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:43.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.004+0000 7f0795e00640 1 --2- 172.21.15.123:0/2630410435 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x560ab9a9d250 0x560ab9a9f670 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5ee1c194f37a19e server_cookie=35baf3ae32bdb81 in_seq=0 out_seq=0 2024-09-19T04:55:43.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.004+0000 7f0777e00640 10 client.27248 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:43.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.005+0000 7f0777e00640 1 -- 172.21.15.123:0/2630410435 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560ab9983d10 con 0x560ab9a9d250 2024-09-19T04:55:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.005+0000 7f0777e00640 10 client.27248 handle_command_reply: tid=0 2024-09-19T04:55:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.005+0000 7f079fe7a740 1 client.27248 shutdown 2024-09-19T04:55:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.005+0000 7f079fe7a740 1 -- 172.21.15.123:0/2630410435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a15ac0 msgr2=0x560ab9a294f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.005+0000 7f079fe7a740 1 --2- 172.21.15.123:0/2630410435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a15ac0 0x560ab9a294f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0788019040 tx=0x7f078800ff10 comp rx=0 tx=0).stop 2024-09-19T04:55:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.005+0000 7f079fe7a740 1 -- 172.21.15.123:0/2630410435 shutdown_connections 2024-09-19T04:55:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.005+0000 7f079fe7a740 1 --2- 172.21.15.123:0/2630410435 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x560ab9a9d250 0x560ab9a9f670 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.005+0000 7f079fe7a740 1 --2- 172.21.15.123:0/2630410435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ab9a20ca0 0x560ab9a27f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.005+0000 7f0777e00640 0 client.27248 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.005+0000 7f079fe7a740 1 --2- 172.21.15.123:0/2630410435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ab9a163e0 0x560ab9a278f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.005+0000 7f079fe7a740 1 --2- 172.21.15.123:0/2630410435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ab9a15ac0 0x560ab9a294f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.005+0000 7f079fe7a740 1 -- 172.21.15.123:0/2630410435 >> 172.21.15.123:0/2630410435 conn(0x560ab9a12050 msgr2=0x560ab9a111c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.005+0000 7f079fe7a740 1 -- 172.21.15.123:0/2630410435 shutdown_connections 2024-09-19T04:55:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.006+0000 7f079fe7a740 1 -- 172.21.15.123:0/2630410435 wait complete. 2024-09-19T04:55:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.006+0000 7f079fe7a740 20 client.27248 trim_cache size 0 max 16384 2024-09-19T04:55:43.011 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:55:43.011 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-09-19T04:55:43.011 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "f00b2f16-8562-4458-b87d-9ca5e33372f8", 2024-09-19T04:55:43.011 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-09-19T04:55:43.011 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:55:43.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.006+0000 7f079d400640 1 -- 172.21.15.123:0/1599818623 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f077c0a3430 msgr2=0x7f077c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:43.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.006+0000 7f079d400640 1 --2- 172.21.15.123:0/1599818623 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f077c0a3430 0x7f077c0a58f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x560ab9a12eb0 tx=0x7f0788005f90 comp rx=0 tx=0).stop 2024-09-19T04:55:43.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.006+0000 7f079d400640 1 -- 172.21.15.123:0/1599818623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0798074f40 msgr2=0x7f079819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:43.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.006+0000 7f079d400640 1 --2- 172.21.15.123:0/1599818623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0798074f40 0x7f079819abb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0784004770 tx=0x7f0784003c10 comp rx=0 tx=0).stop 2024-09-19T04:55:43.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.006+0000 7f079d400640 1 -- 172.21.15.123:0/1599818623 shutdown_connections 2024-09-19T04:55:43.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.006+0000 7f079d400640 1 --2- 172.21.15.123:0/1599818623 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f077c0a3430 0x7f077c0a58f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:43.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.006+0000 7f079d400640 1 --2- 172.21.15.123:0/1599818623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f079810e3c0 0x7f079819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:43.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.006+0000 7f079d400640 1 --2- 172.21.15.123:0/1599818623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0798074f40 0x7f079819abb0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:43.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.006+0000 7f079d400640 1 --2- 172.21.15.123:0/1599818623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0798074620 0x7f079819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:43.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.006+0000 7f079d400640 1 -- 172.21.15.123:0/1599818623 >> 172.21.15.123:0/1599818623 conn(0x7f07980fd8c0 msgr2=0x7f07980fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:43.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.007+0000 7f079d400640 1 -- 172.21.15.123:0/1599818623 shutdown_connections 2024-09-19T04:55:43.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:43.007+0000 7f079d400640 1 -- 172.21.15.123:0/1599818623 wait complete. 2024-09-19T04:55:43.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:43 smithi193 ceph-mon[35592]: pgmap v279: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:43.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:43 smithi123 ceph-mon[30511]: pgmap v279: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:43 smithi183 ceph-mon[35333]: pgmap v279: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:43.845 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:55:44.215 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:44.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:44 smithi193 ceph-mon[35592]: scrub queued for path: / 2024-09-19T04:55:44.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:44 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:55:44.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:44 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:55:44.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:44 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:55:44.521 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:44 smithi123 ceph-mon[30511]: scrub queued for path: / 2024-09-19T04:55:44.521 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:44 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:55:44.521 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:44 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:55:44.522 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:44 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:55:44.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:44 smithi183 ceph-mon[35333]: scrub queued for path: / 2024-09-19T04:55:44.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:44 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:55:44.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:44 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:55:44.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:44 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:55:44.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.770+0000 7ffa59800640 1 -- 172.21.15.123:0/2898433456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa54103ff0 msgr2=0x7ffa541043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:44.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.770+0000 7ffa59800640 1 --2- 172.21.15.123:0/2898433456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa54103ff0 0x7ffa541043d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009900 tx=0x7ffa4402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.770+0000 7ffa59800640 1 -- 172.21.15.123:0/2898433456 shutdown_connections 2024-09-19T04:55:44.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.770+0000 7ffa59800640 1 --2- 172.21.15.123:0/2898433456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa54109120 0x7ffa54110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.770+0000 7ffa59800640 1 --2- 172.21.15.123:0/2898433456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa541049a0 0x7ffa541089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.770+0000 7ffa59800640 1 --2- 172.21.15.123:0/2898433456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa54103ff0 0x7ffa541043d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.770+0000 7ffa59800640 1 -- 172.21.15.123:0/2898433456 >> 172.21.15.123:0/2898433456 conn(0x7ffa540ffa90 msgr2=0x7ffa54101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:44.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.771+0000 7ffa59800640 1 -- 172.21.15.123:0/2898433456 shutdown_connections 2024-09-19T04:55:44.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.771+0000 7ffa59800640 1 -- 172.21.15.123:0/2898433456 wait complete. 2024-09-19T04:55:44.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.772+0000 7ffa59800640 1 Processor -- start 2024-09-19T04:55:44.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.772+0000 7ffa59800640 1 -- start start 2024-09-19T04:55:44.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.772+0000 7ffa59800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa54103ff0 0x7ffa5419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.772+0000 7ffa59800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa541049a0 0x7ffa5419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.772+0000 7ffa59800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa54109120 0x7ffa5419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.772+0000 7ffa59800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa54114150 con 0x7ffa54103ff0 2024-09-19T04:55:44.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.772+0000 7ffa59800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa54113fd0 con 0x7ffa54109120 2024-09-19T04:55:44.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.772+0000 7ffa59800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541142d0 con 0x7ffa541049a0 2024-09-19T04:55:44.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.773+0000 7ffa52a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa54103ff0 0x7ffa5419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.773+0000 7ffa52a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa54103ff0 0x7ffa5419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40492/0 (socket says 172.21.15.123:40492) 2024-09-19T04:55:44.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.773+0000 7ffa52a00640 1 -- 172.21.15.123:0/1773156730 learned_addr learned my addr 172.21.15.123:0/1773156730 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:44.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.773+0000 7ffa53400640 1 --2- 172.21.15.123:0/1773156730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa54109120 0x7ffa5419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.773+0000 7ffa52000640 1 --2- 172.21.15.123:0/1773156730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa541049a0 0x7ffa5419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.773+0000 7ffa52a00640 1 -- 172.21.15.123:0/1773156730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa541049a0 msgr2=0x7ffa5419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:44.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.773+0000 7ffa52a00640 1 --2- 172.21.15.123:0/1773156730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa541049a0 0x7ffa5419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.773+0000 7ffa52a00640 1 -- 172.21.15.123:0/1773156730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa54109120 msgr2=0x7ffa5419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:44.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.773+0000 7ffa52a00640 1 --2- 172.21.15.123:0/1773156730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa54109120 0x7ffa5419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.773+0000 7ffa52a00640 1 -- 172.21.15.123:0/1773156730 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5419f5a0 con 0x7ffa54103ff0 2024-09-19T04:55:44.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.774+0000 7ffa52a00640 1 --2- 172.21.15.123:0/1773156730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa54103ff0 0x7ffa5419a5f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ffa44006010 tx=0x7ffa440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:44.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.774+0000 7ffa4b400640 1 -- 172.21.15.123:0/1773156730 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4403f030 con 0x7ffa54103ff0 2024-09-19T04:55:44.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.774+0000 7ffa4b400640 1 -- 172.21.15.123:0/1773156730 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa44003ce0 con 0x7ffa54103ff0 2024-09-19T04:55:44.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.774+0000 7ffa4b400640 1 -- 172.21.15.123:0/1773156730 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44035070 con 0x7ffa54103ff0 2024-09-19T04:55:44.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.774+0000 7ffa59800640 1 -- 172.21.15.123:0/1773156730 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5419f830 con 0x7ffa54103ff0 2024-09-19T04:55:44.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.774+0000 7ffa59800640 1 -- 172.21.15.123:0/1773156730 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa541a7170 con 0x7ffa54103ff0 2024-09-19T04:55:44.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.777+0000 7ffa5c43a740 1 Processor -- start 2024-09-19T04:55:44.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.777+0000 7ffa4b400640 1 -- 172.21.15.123:0/1773156730 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ffa44042030 con 0x7ffa54103ff0 2024-09-19T04:55:44.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.777+0000 7ffa5c43a740 1 -- start start 2024-09-19T04:55:44.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.778+0000 7ffa5c43a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e59fb90 0x556a0e59ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.778+0000 7ffa5c43a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 0x556a0e5aa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.778+0000 7ffa5c43a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e5aad70 0x556a0e5ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.778+0000 7ffa5c43a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556a0e5063e0 con 0x556a0e5a04b0 2024-09-19T04:55:44.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.778+0000 7ffa5c43a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556a0e4add80 con 0x556a0e59fb90 2024-09-19T04:55:44.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.778+0000 7ffa5c43a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556a0e5023a0 con 0x556a0e5aad70 2024-09-19T04:55:44.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.778+0000 7ffa52a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 0x556a0e5aa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.778+0000 7ffa52a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 0x556a0e5aa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40498/0 (socket says 172.21.15.123:40498) 2024-09-19T04:55:44.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.778+0000 7ffa52a00640 1 -- 172.21.15.123:0/3335569454 learned_addr learned my addr 172.21.15.123:0/3335569454 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:44.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.778+0000 7ffa52000640 1 --2- 172.21.15.123:0/3335569454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e5aad70 0x556a0e5ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.778+0000 7ffa53400640 1 --2- 172.21.15.123:0/3335569454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e59fb90 0x556a0e59ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.779+0000 7ffa53400640 1 -- 172.21.15.123:0/3335569454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e5aad70 msgr2=0x556a0e5ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:44.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.779+0000 7ffa53400640 1 --2- 172.21.15.123:0/3335569454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e5aad70 0x556a0e5ad160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.779+0000 7ffa53400640 1 -- 172.21.15.123:0/3335569454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 msgr2=0x556a0e5aa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:44.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.779+0000 7ffa53400640 1 --2- 172.21.15.123:0/3335569454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 0x556a0e5aa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.779+0000 7ffa53400640 1 -- 172.21.15.123:0/3335569454 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a0e4ccae0 con 0x556a0e59fb90 2024-09-19T04:55:44.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.779+0000 7ffa4b400640 1 --2- 172.21.15.123:0/1773156730 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffa380a33b0 0x7ffa380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.779+0000 7ffa53400640 1 --2- 172.21.15.123:0/3335569454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e59fb90 0x556a0e59ff70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009eb0 tx=0x7ffa4c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2730447186fa88ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:44.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.779+0000 7ffa4b400640 1 -- 172.21.15.123:0/1773156730 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7ffa440f0260 con 0x7ffa54103ff0 2024-09-19T04:55:44.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.779+0000 7ffa52a00640 1 --2- 172.21.15.123:0/1773156730 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffa380a33b0 0x7ffa380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.779+0000 7ffa3fe00640 1 -- 172.21.15.123:0/3335569454 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c016070 con 0x556a0e59fb90 2024-09-19T04:55:44.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.779+0000 7ffa3fe00640 1 -- 172.21.15.123:0/3335569454 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa4c004d20 con 0x556a0e59fb90 2024-09-19T04:55:44.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.779+0000 7ffa3fe00640 1 -- 172.21.15.123:0/3335569454 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c012810 con 0x556a0e59fb90 2024-09-19T04:55:44.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.780+0000 7ffa5c43a740 1 -- 172.21.15.123:0/3335569454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e59fb90 msgr2=0x556a0e59ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:44.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.780+0000 7ffa52a00640 1 --2- 172.21.15.123:0/1773156730 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffa380a33b0 0x7ffa380a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffa44038020 tx=0x7ffa440f3d10 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:44.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.780+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/3335569454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e59fb90 0x556a0e59ff70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009eb0 tx=0x7ffa4c010740 comp rx=0 tx=0).stop 2024-09-19T04:55:44.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.780+0000 7ffa5c43a740 1 -- 172.21.15.123:0/3335569454 shutdown_connections 2024-09-19T04:55:44.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.780+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/3335569454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e5aad70 0x556a0e5ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.780+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/3335569454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 0x556a0e5aa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.780+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/3335569454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e59fb90 0x556a0e59ff70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.780+0000 7ffa5c43a740 1 -- 172.21.15.123:0/3335569454 >> 172.21.15.123:0/3335569454 conn(0x556a0e59c120 msgr2=0x556a0e59c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:44.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.780+0000 7ffa5c43a740 1 -- 172.21.15.123:0/3335569454 shutdown_connections 2024-09-19T04:55:44.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.780+0000 7ffa5c43a740 1 -- 172.21.15.123:0/3335569454 wait complete. 2024-09-19T04:55:44.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.781+0000 7ffa5c43a740 1 -- start start 2024-09-19T04:55:44.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.781+0000 7ffa5c43a740 1 Processor -- start 2024-09-19T04:55:44.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.781+0000 7ffa5c43a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e59fb90 0x556a0e5a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.781+0000 7ffa5c43a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 0x556a0e5a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.781+0000 7ffa5c43a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e5aad70 0x556a0e5a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.781+0000 7ffa5c43a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556a0e5ad9e0 con 0x556a0e5a04b0 2024-09-19T04:55:44.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.781+0000 7ffa5c43a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556a0e4c10f0 con 0x556a0e59fb90 2024-09-19T04:55:44.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.781+0000 7ffa5c43a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556a0e5adb60 con 0x556a0e5aad70 2024-09-19T04:55:44.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.781+0000 7ffa5c43a740 10 client.0 _ll_register_callbacks cb 0x556a0e4b29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:44.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.782+0000 7ffa52000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 0x556a0e5a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.782+0000 7ffa53400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e5aad70 0x556a0e5a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.782+0000 7ffa53400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e5aad70 0x556a0e5a87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:44306/0 (socket says 172.21.15.123:44306) 2024-09-19T04:55:44.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.782+0000 7ffa53400640 1 -- 172.21.15.123:0/4184884040 learned_addr learned my addr 172.21.15.123:0/4184884040 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:44.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.782+0000 7ffa53400640 1 --2- 172.21.15.123:0/4184884040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e59fb90 0x556a0e5a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.782+0000 7ffa53400640 1 -- 172.21.15.123:0/4184884040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e59fb90 msgr2=0x556a0e5a7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:44.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.782+0000 7ffa53400640 1 --2- 172.21.15.123:0/4184884040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e59fb90 0x556a0e5a7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.782+0000 7ffa53400640 1 -- 172.21.15.123:0/4184884040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 msgr2=0x556a0e5a82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:44.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.782+0000 7ffa53400640 1 --2- 172.21.15.123:0/4184884040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 0x556a0e5a82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.782+0000 7ffa53400640 1 -- 172.21.15.123:0/4184884040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a0e2773c0 con 0x556a0e5aad70 2024-09-19T04:55:44.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.782+0000 7ffa52000640 1 --2- 172.21.15.123:0/4184884040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 0x556a0e5a82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:44.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.783+0000 7ffa53400640 1 --2- 172.21.15.123:0/4184884040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e5aad70 0x556a0e5a87f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c004cd0 tx=0x7ffa4c005aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ab05330518e449 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:44.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.783+0000 7ffa3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:55:44.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.783+0000 7ffa3cc00640 1 -- 172.21.15.123:0/4184884040 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c016070 con 0x556a0e5aad70 2024-09-19T04:55:44.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.783+0000 7ffa5c43a740 1 -- 172.21.15.123:0/4184884040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa4c004d20 con 0x556a0e5aad70 2024-09-19T04:55:44.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.783+0000 7ffa3cc00640 1 -- 172.21.15.123:0/4184884040 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7ffa4c003b70 con 0x556a0e5aad70 2024-09-19T04:55:44.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.783+0000 7ffa3cc00640 1 -- 172.21.15.123:0/4184884040 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ffa4c023040 con 0x556a0e5aad70 2024-09-19T04:55:44.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.783+0000 7ffa5c43a740 10 client.27280 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:44.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.783+0000 7ffa5c43a740 1 -- 172.21.15.123:0/4184884040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556a0e5b97b0 con 0x556a0e5aad70 2024-09-19T04:55:44.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.784+0000 7ffa3cc00640 1 -- 172.21.15.123:0/4184884040 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa4c004d20 con 0x556a0e5aad70 2024-09-19T04:55:44.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.784+0000 7ffa3cc00640 1 -- 172.21.15.123:0/4184884040 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7ffa4c00f040 con 0x556a0e5aad70 2024-09-19T04:55:44.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.784+0000 7ffa5c43a740 10 client.27280 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:44.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.784+0000 7ffa5c43a740 10 client.27280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:55:44.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.784+0000 7ffa5c43a740 20 client.27280 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:44.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.784+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/4184884040 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556a0e5bab60 0x556a0e5bcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.784+0000 7ffa5c43a740 4 client.27280 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:55:44.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.784+0000 7ffa5c43a740 1 -- 172.21.15.123:0/4184884040 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a0e513950 con 0x556a0e5bab60 2024-09-19T04:55:44.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.784+0000 7ffa52000640 1 --2- 172.21.15.123:0/4184884040 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556a0e5bab60 0x556a0e5bcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.785+0000 7ffa52000640 1 --2- 172.21.15.123:0/4184884040 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556a0e5bab60 0x556a0e5bcf50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=348b8f169a909933 server_cookie=9829c207da5e95f0 in_seq=0 out_seq=0 2024-09-19T04:55:44.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.785+0000 7ffa3cc00640 10 client.27280 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:44.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.786+0000 7ffa3cc00640 1 -- 172.21.15.123:0/4184884040 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a0e513950 con 0x556a0e5bab60 2024-09-19T04:55:44.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.786+0000 7ffa3cc00640 10 client.27280 handle_command_reply: tid=0 2024-09-19T04:55:44.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.786+0000 7ffa5c43a740 1 client.27280 shutdown 2024-09-19T04:55:44.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.786+0000 7ffa5c43a740 1 -- 172.21.15.123:0/4184884040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e5aad70 msgr2=0x556a0e5a87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:44.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.786+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/4184884040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e5aad70 0x556a0e5a87f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c004cd0 tx=0x7ffa4c005aa0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.786+0000 7ffa5c43a740 1 -- 172.21.15.123:0/4184884040 shutdown_connections 2024-09-19T04:55:44.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.786+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/4184884040 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556a0e5bab60 0x556a0e5bcf50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.786+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/4184884040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e5aad70 0x556a0e5a87f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.786+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/4184884040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 0x556a0e5a82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.786+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/4184884040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e59fb90 0x556a0e5a7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.786+0000 7ffa5c43a740 1 -- 172.21.15.123:0/4184884040 >> 172.21.15.123:0/4184884040 conn(0x556a0e59c120 msgr2=0x556a0e5a1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:44.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.786+0000 7ffa3cc00640 0 client.27280 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:44.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.786+0000 7ffa5c43a740 1 -- 172.21.15.123:0/4184884040 shutdown_connections 2024-09-19T04:55:44.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.786+0000 7ffa5c43a740 1 -- 172.21.15.123:0/4184884040 wait complete. 2024-09-19T04:55:44.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.786+0000 7ffa5c43a740 20 client.27280 trim_cache size 0 max 16384 2024-09-19T04:55:44.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.804+0000 7ffa5c43a740 1 Processor -- start 2024-09-19T04:55:44.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.804+0000 7ffa5c43a740 1 -- start start 2024-09-19T04:55:44.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.804+0000 7ffa5c43a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e59fb90 0x556a0e5a8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.804+0000 7ffa5c43a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 0x556a0e5c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.804+0000 7ffa5c43a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e5aad70 0x556a0e5c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.804+0000 7ffa5c43a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541afaa0 con 0x556a0e5a04b0 2024-09-19T04:55:44.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.804+0000 7ffa5c43a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541b15e0 con 0x556a0e5aad70 2024-09-19T04:55:44.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.804+0000 7ffa5c43a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541b0170 con 0x556a0e59fb90 2024-09-19T04:55:44.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.804+0000 7ffa52000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 0x556a0e5c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.804+0000 7ffa52000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 0x556a0e5c25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40510/0 (socket says 172.21.15.123:40510) 2024-09-19T04:55:44.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.804+0000 7ffa52000640 1 -- 172.21.15.123:0/1797441886 learned_addr learned my addr 172.21.15.123:0/1797441886 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:44.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.804+0000 7ffa53400640 1 --2- 172.21.15.123:0/1797441886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e59fb90 0x556a0e5a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.805+0000 7ffa53400640 1 --2- 172.21.15.123:0/1797441886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e5aad70 0x556a0e5c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.805+0000 7ffa52000640 1 -- 172.21.15.123:0/1797441886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e59fb90 msgr2=0x556a0e5a8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:44.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.805+0000 7ffa52000640 1 --2- 172.21.15.123:0/1797441886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e59fb90 0x556a0e5a8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.805+0000 7ffa52000640 1 -- 172.21.15.123:0/1797441886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e5aad70 msgr2=0x556a0e5c2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:44.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.805+0000 7ffa52000640 1 --2- 172.21.15.123:0/1797441886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e5aad70 0x556a0e5c2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.805+0000 7ffa52000640 1 -- 172.21.15.123:0/1797441886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a0e59cdb0 con 0x556a0e5a04b0 2024-09-19T04:55:44.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.805+0000 7ffa52000640 1 --2- 172.21.15.123:0/1797441886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 0x556a0e5c25e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ffa40005d00 tx=0x7ffa4000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf024fcc05b9b40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:44.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.805+0000 7ffa3f400640 1 -- 172.21.15.123:0/1797441886 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40004220 con 0x556a0e5a04b0 2024-09-19T04:55:44.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.805+0000 7ffa3f400640 1 -- 172.21.15.123:0/1797441886 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa40010e40 con 0x556a0e5a04b0 2024-09-19T04:55:44.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.805+0000 7ffa3f400640 1 -- 172.21.15.123:0/1797441886 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40025400 con 0x556a0e5a04b0 2024-09-19T04:55:44.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.806+0000 7ffa5c43a740 1 -- 172.21.15.123:0/1797441886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 msgr2=0x556a0e5c25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.806+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/1797441886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 0x556a0e5c25e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ffa40005d00 tx=0x7ffa4000b410 comp rx=0 tx=0).stop 2024-09-19T04:55:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.806+0000 7ffa5c43a740 1 -- 172.21.15.123:0/1797441886 shutdown_connections 2024-09-19T04:55:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.806+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/1797441886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e5aad70 0x556a0e5c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.806+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/1797441886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5a04b0 0x556a0e5c25e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.806+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/1797441886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e59fb90 0x556a0e5a8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.806+0000 7ffa5c43a740 1 -- 172.21.15.123:0/1797441886 >> 172.21.15.123:0/1797441886 conn(0x556a0e59c120 msgr2=0x556a0e59b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.806+0000 7ffa5c43a740 1 -- 172.21.15.123:0/1797441886 shutdown_connections 2024-09-19T04:55:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.806+0000 7ffa5c43a740 1 -- 172.21.15.123:0/1797441886 wait complete. 2024-09-19T04:55:44.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.806+0000 7ffa5c43a740 1 -- start start 2024-09-19T04:55:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.807+0000 7ffa5c43a740 1 Processor -- start 2024-09-19T04:55:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.807+0000 7ffa5c43a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e59fb90 0x556a0e5b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.807+0000 7ffa5c43a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e5a04b0 0x556a0e5ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.807+0000 7ffa5c43a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5aad70 0x556a0e5aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.807+0000 7ffa5c43a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541b0f10 con 0x556a0e5aad70 2024-09-19T04:55:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.807+0000 7ffa5c43a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541b0840 con 0x556a0e5a04b0 2024-09-19T04:55:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.807+0000 7ffa5c43a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541adaa0 con 0x556a0e59fb90 2024-09-19T04:55:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.807+0000 7ffa5c43a740 10 client.0 _ll_register_callbacks cb 0x7ffa4c0125b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.807+0000 7ffa53400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5aad70 0x556a0e5aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.807+0000 7ffa53400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5aad70 0x556a0e5aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40514/0 (socket says 172.21.15.123:40514) 2024-09-19T04:55:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.807+0000 7ffa53400640 1 -- 172.21.15.123:0/2332638923 learned_addr learned my addr 172.21.15.123:0/2332638923 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.807+0000 7ffa52000640 1 --2- 172.21.15.123:0/2332638923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e5a04b0 0x556a0e5ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.808+0000 7ffa53400640 1 --2- 172.21.15.123:0/2332638923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e59fb90 0x556a0e5b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.808+0000 7ffa52000640 1 -- 172.21.15.123:0/2332638923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e59fb90 msgr2=0x556a0e5b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.808+0000 7ffa52000640 1 --2- 172.21.15.123:0/2332638923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e59fb90 0x556a0e5b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.808+0000 7ffa52000640 1 -- 172.21.15.123:0/2332638923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5aad70 msgr2=0x556a0e5aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.808+0000 7ffa52000640 1 --2- 172.21.15.123:0/2332638923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5aad70 0x556a0e5aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.808+0000 7ffa52000640 1 -- 172.21.15.123:0/2332638923 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a0e5bd3b0 con 0x556a0e5a04b0 2024-09-19T04:55:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.808+0000 7ffa53400640 1 --2- 172.21.15.123:0/2332638923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5aad70 0x556a0e5aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.808+0000 7ffa52000640 1 --2- 172.21.15.123:0/2332638923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e5a04b0 0x556a0e5ae8a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffa40027040 tx=0x7ffa40003b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e18cfa85d6f67495 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.808+0000 7ffa3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:55:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.808+0000 7ffa3cc00640 1 -- 172.21.15.123:0/2332638923 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40010b60 con 0x556a0e5a04b0 2024-09-19T04:55:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.809+0000 7ffa5c43a740 1 -- 172.21.15.123:0/2332638923 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa40010e40 con 0x556a0e5a04b0 2024-09-19T04:55:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.809+0000 7ffa3cc00640 1 -- 172.21.15.123:0/2332638923 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7ffa40003cd0 con 0x556a0e5a04b0 2024-09-19T04:55:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.809+0000 7ffa3cc00640 1 -- 172.21.15.123:0/2332638923 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ffa4003b050 con 0x556a0e5a04b0 2024-09-19T04:55:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.809+0000 7ffa5c43a740 10 client.27266 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.809+0000 7ffa5c43a740 1 -- 172.21.15.123:0/2332638923 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556a0e625470 con 0x556a0e5a04b0 2024-09-19T04:55:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.809+0000 7ffa3cc00640 1 -- 172.21.15.123:0/2332638923 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa40010e40 con 0x556a0e5a04b0 2024-09-19T04:55:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.809+0000 7ffa3cc00640 1 -- 172.21.15.123:0/2332638923 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7ffa40036030 con 0x556a0e5a04b0 2024-09-19T04:55:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.809+0000 7ffa5c43a740 10 client.27266 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.809+0000 7ffa5c43a740 10 client.27266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:55:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.809+0000 7ffa5c43a740 20 client.27266 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.809+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/2332638923 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556a0e625b00 0x556a0e627f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.809+0000 7ffa5c43a740 4 client.27266 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:55:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.809+0000 7ffa5c43a740 1 -- 172.21.15.123:0/2332638923 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a0e50de60 con 0x556a0e625b00 2024-09-19T04:55:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.810+0000 7ffa53400640 1 --2- 172.21.15.123:0/2332638923 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556a0e625b00 0x556a0e627f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.810+0000 7ffa3cc00640 10 client.27266 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.810+0000 7ffa53400640 1 --2- 172.21.15.123:0/2332638923 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556a0e625b00 0x556a0e627f20 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d51d78b5af4d71b4 server_cookie=a3402b3806acdad5 in_seq=0 out_seq=0 2024-09-19T04:55:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.810+0000 7ffa3cc00640 1 -- 172.21.15.123:0/2332638923 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556a0e50de60 con 0x556a0e625b00 2024-09-19T04:55:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.810+0000 7ffa3cc00640 10 client.27266 handle_command_reply: tid=0 2024-09-19T04:55:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.810+0000 7ffa5c43a740 1 client.27266 shutdown 2024-09-19T04:55:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.811+0000 7ffa5c43a740 1 -- 172.21.15.123:0/2332638923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e5a04b0 msgr2=0x556a0e5ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.811+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/2332638923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e5a04b0 0x556a0e5ae8a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffa40027040 tx=0x7ffa40003b50 comp rx=0 tx=0).stop 2024-09-19T04:55:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.811+0000 7ffa5c43a740 1 -- 172.21.15.123:0/2332638923 shutdown_connections 2024-09-19T04:55:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.811+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/2332638923 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x556a0e625b00 0x556a0e627f20 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.811+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/2332638923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a0e5aad70 0x556a0e5aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.811+0000 7ffa3cc00640 0 client.27266 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.811+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/2332638923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a0e5a04b0 0x556a0e5ae8a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.811+0000 7ffa5c43a740 1 --2- 172.21.15.123:0/2332638923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a0e59fb90 0x556a0e5b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.811+0000 7ffa5c43a740 1 -- 172.21.15.123:0/2332638923 >> 172.21.15.123:0/2332638923 conn(0x556a0e59c120 msgr2=0x556a0e59a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:44.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.811+0000 7ffa5c43a740 1 -- 172.21.15.123:0/2332638923 shutdown_connections 2024-09-19T04:55:44.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.811+0000 7ffa5c43a740 1 -- 172.21.15.123:0/2332638923 wait complete. 2024-09-19T04:55:44.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.811+0000 7ffa5c43a740 20 client.27266 trim_cache size 0 max 16384 2024-09-19T04:55:44.817 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:55:44.817 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-09-19T04:55:44.817 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-09-19T04:55:44.817 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:55:44.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.812+0000 7ffa59800640 1 -- 172.21.15.123:0/1773156730 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffa380a33b0 msgr2=0x7ffa380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:44.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.812+0000 7ffa59800640 1 --2- 172.21.15.123:0/1773156730 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffa380a33b0 0x7ffa380a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffa44038020 tx=0x7ffa440f3d10 comp rx=0 tx=0).stop 2024-09-19T04:55:44.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.812+0000 7ffa59800640 1 -- 172.21.15.123:0/1773156730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa54103ff0 msgr2=0x7ffa5419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:44.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.812+0000 7ffa59800640 1 --2- 172.21.15.123:0/1773156730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa54103ff0 0x7ffa5419a5f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ffa44006010 tx=0x7ffa440042f0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.812+0000 7ffa59800640 1 -- 172.21.15.123:0/1773156730 shutdown_connections 2024-09-19T04:55:44.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.812+0000 7ffa59800640 1 --2- 172.21.15.123:0/1773156730 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffa380a33b0 0x7ffa380a5870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.812+0000 7ffa59800640 1 --2- 172.21.15.123:0/1773156730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa54109120 0x7ffa5419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.812+0000 7ffa59800640 1 --2- 172.21.15.123:0/1773156730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa541049a0 0x7ffa5419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.812+0000 7ffa59800640 1 --2- 172.21.15.123:0/1773156730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa54103ff0 0x7ffa5419a5f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:44.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.812+0000 7ffa59800640 1 -- 172.21.15.123:0/1773156730 >> 172.21.15.123:0/1773156730 conn(0x7ffa540ffa90 msgr2=0x7ffa54100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:44.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.812+0000 7ffa59800640 1 -- 172.21.15.123:0/1773156730 shutdown_connections 2024-09-19T04:55:44.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:44.812+0000 7ffa59800640 1 -- 172.21.15.123:0/1773156730 wait complete. 2024-09-19T04:55:45.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:45 smithi193 ceph-mon[35592]: pgmap v280: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:55:45.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:45 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:55:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:45 smithi183 ceph-mon[35333]: pgmap v280: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:55:45.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:45 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:55:45.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:45 smithi123 ceph-mon[30511]: pgmap v280: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:55:45.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:45 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:55:45.779 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:55:45.779 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:55:46.148 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:46.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.711+0000 7fb66f600640 1 -- 172.21.15.123:0/920918115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb668109120 msgr2=0x7fb668110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:46.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.711+0000 7fb66f600640 1 --2- 172.21.15.123:0/920918115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb668109120 0x7fb668110ca0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb660009ca0 tx=0x7fb66002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:55:46.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.711+0000 7fb66f600640 1 -- 172.21.15.123:0/920918115 shutdown_connections 2024-09-19T04:55:46.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.712+0000 7fb66f600640 1 --2- 172.21.15.123:0/920918115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb668109120 0x7fb668110ca0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:46.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.712+0000 7fb66f600640 1 --2- 172.21.15.123:0/920918115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6681049a0 0x7fb6681089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:46.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.712+0000 7fb66f600640 1 --2- 172.21.15.123:0/920918115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb668103ff0 0x7fb6681043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:46.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.712+0000 7fb66f600640 1 -- 172.21.15.123:0/920918115 >> 172.21.15.123:0/920918115 conn(0x7fb6680ffa90 msgr2=0x7fb668101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:46.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.712+0000 7fb66f600640 1 -- 172.21.15.123:0/920918115 shutdown_connections 2024-09-19T04:55:46.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.712+0000 7fb66f600640 1 -- 172.21.15.123:0/920918115 wait complete. 2024-09-19T04:55:46.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.712+0000 7fb66f600640 1 Processor -- start 2024-09-19T04:55:46.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.713+0000 7fb66f600640 1 -- start start 2024-09-19T04:55:46.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.713+0000 7fb66f600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb668103ff0 0x7fb66819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:46.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.713+0000 7fb66f600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6681049a0 0x7fb66819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:46.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.713+0000 7fb66f600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb668109120 0x7fb66819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:46.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.713+0000 7fb66f600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb668114150 con 0x7fb6681049a0 2024-09-19T04:55:46.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.713+0000 7fb66f600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb668113fd0 con 0x7fb668109120 2024-09-19T04:55:46.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.713+0000 7fb66f600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681142d0 con 0x7fb668103ff0 2024-09-19T04:55:46.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.713+0000 7fb65fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6681049a0 0x7fb66819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.713+0000 7fb65fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6681049a0 0x7fb66819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40538/0 (socket says 172.21.15.123:40538) 2024-09-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.713+0000 7fb65fe00640 1 -- 172.21.15.123:0/3100460531 learned_addr learned my addr 172.21.15.123:0/3100460531 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.713+0000 7fb66cc00640 1 --2- 172.21.15.123:0/3100460531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb668103ff0 0x7fb66819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.713+0000 7fb66d600640 1 --2- 172.21.15.123:0/3100460531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb668109120 0x7fb66819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.714+0000 7fb65fe00640 1 -- 172.21.15.123:0/3100460531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb668103ff0 msgr2=0x7fb66819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.714+0000 7fb65fe00640 1 --2- 172.21.15.123:0/3100460531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb668103ff0 0x7fb66819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.714+0000 7fb65fe00640 1 -- 172.21.15.123:0/3100460531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb668109120 msgr2=0x7fb66819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.714+0000 7fb65fe00640 1 --2- 172.21.15.123:0/3100460531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb668109120 0x7fb66819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.714+0000 7fb65fe00640 1 -- 172.21.15.123:0/3100460531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66819f5e0 con 0x7fb6681049a0 2024-09-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.714+0000 7fb66cc00640 1 --2- 172.21.15.123:0/3100460531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb668103ff0 0x7fb66819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.714+0000 7fb65fe00640 1 --2- 172.21.15.123:0/3100460531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6681049a0 0x7fb66819ab70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb65000d950 tx=0x7fb65000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.714+0000 7fb666000640 1 -- 172.21.15.123:0/3100460531 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650017070 con 0x7fb6681049a0 2024-09-19T04:55:46.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.714+0000 7fb666000640 1 -- 172.21.15.123:0/3100460531 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb650009c30 con 0x7fb6681049a0 2024-09-19T04:55:46.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.714+0000 7fb666000640 1 -- 172.21.15.123:0/3100460531 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65000f6b0 con 0x7fb6681049a0 2024-09-19T04:55:46.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.714+0000 7fb66f600640 1 -- 172.21.15.123:0/3100460531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb66819f8d0 con 0x7fb6681049a0 2024-09-19T04:55:46.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.715+0000 7fb66f600640 1 -- 172.21.15.123:0/3100460531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6681a71b0 con 0x7fb6681049a0 2024-09-19T04:55:46.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.716+0000 7fb66f600640 1 -- 172.21.15.123:0/3100460531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6681043d0 con 0x7fb6681049a0 2024-09-19T04:55:46.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.718+0000 7fb666000640 1 -- 172.21.15.123:0/3100460531 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb6500109d0 con 0x7fb6681049a0 2024-09-19T04:55:46.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.720+0000 7fb666000640 1 --2- 172.21.15.123:0/3100460531 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb64c0a3430 0x7fb64c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:46.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.720+0000 7fb666000640 1 -- 172.21.15.123:0/3100460531 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fb6500cf820 con 0x7fb6681049a0 2024-09-19T04:55:46.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.720+0000 7fb66cc00640 1 --2- 172.21.15.123:0/3100460531 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb64c0a3430 0x7fb64c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:46.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.721+0000 7fb66cc00640 1 --2- 172.21.15.123:0/3100460531 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb64c0a3430 0x7fb64c0a58f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb658009fc0 tx=0x7fb658005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:46.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.724+0000 7fb666000640 1 -- 172.21.15.123:0/3100460531 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb650009de0 con 0x7fb6681049a0 2024-09-19T04:55:46.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.934+0000 7fb66f600640 1 -- 172.21.15.123:0/3100460531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6681a7380 con 0x7fb6681049a0 2024-09-19T04:55:46.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.934+0000 7fb666000640 1 -- 172.21.15.123:0/3100460531 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11990 (secure 0 0 0) 0x7fb65009bb50 con 0x7fb6681049a0 2024-09-19T04:55:46.936 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:55:46.936 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":43,"btime":"2024-09-19T04:54:44:530698+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:54:44.239974+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24529,24505,24539,24500,24499]},"id":1}]} 2024-09-19T04:55:46.937 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 43 2024-09-19T04:55:46.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.937+0000 7fb66f600640 1 -- 172.21.15.123:0/3100460531 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb64c0a3430 msgr2=0x7fb64c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:46.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.937+0000 7fb66f600640 1 --2- 172.21.15.123:0/3100460531 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb64c0a3430 0x7fb64c0a58f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb658009fc0 tx=0x7fb658005f90 comp rx=0 tx=0).stop 2024-09-19T04:55:46.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.937+0000 7fb66f600640 1 -- 172.21.15.123:0/3100460531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6681049a0 msgr2=0x7fb66819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:46.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.937+0000 7fb66f600640 1 --2- 172.21.15.123:0/3100460531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6681049a0 0x7fb66819ab70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb65000d950 tx=0x7fb65000de10 comp rx=0 tx=0).stop 2024-09-19T04:55:46.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.937+0000 7fb66f600640 1 -- 172.21.15.123:0/3100460531 shutdown_connections 2024-09-19T04:55:46.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.937+0000 7fb66f600640 1 --2- 172.21.15.123:0/3100460531 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb64c0a3430 0x7fb64c0a58f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:46.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.937+0000 7fb66f600640 1 --2- 172.21.15.123:0/3100460531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb668109120 0x7fb66819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:46.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.937+0000 7fb66f600640 1 --2- 172.21.15.123:0/3100460531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6681049a0 0x7fb66819ab70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:46.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.937+0000 7fb66f600640 1 --2- 172.21.15.123:0/3100460531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb668103ff0 0x7fb66819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:46.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.937+0000 7fb66f600640 1 -- 172.21.15.123:0/3100460531 >> 172.21.15.123:0/3100460531 conn(0x7fb6680ffa90 msgr2=0x7fb6681005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:46.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.937+0000 7fb66f600640 1 -- 172.21.15.123:0/3100460531 shutdown_connections 2024-09-19T04:55:46.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:46.937+0000 7fb66f600640 1 -- 172.21.15.123:0/3100460531 wait complete. 2024-09-19T04:55:47.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:47 smithi193 ceph-mon[35592]: pgmap v281: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T04:55:47.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:47 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3100460531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:55:47.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:47 smithi183 ceph-mon[35333]: pgmap v281: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T04:55:47.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:47 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3100460531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:55:47.643 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:47 smithi123 ceph-mon[30511]: pgmap v281: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T04:55:47.643 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:47 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3100460531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:55:47.683 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:55:48.054 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:48.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.599+0000 7fa493200640 1 -- 172.21.15.123:0/654989730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4941047e0 msgr2=0x7fa494108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:48.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.599+0000 7fa493200640 1 --2- 172.21.15.123:0/654989730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4941047e0 0x7fa494108830 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009900 tx=0x7fa47c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.599+0000 7fa493200640 1 -- 172.21.15.123:0/654989730 shutdown_connections 2024-09-19T04:55:48.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.599+0000 7fa493200640 1 --2- 172.21.15.123:0/654989730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa494108f60 0x7fa494110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.599+0000 7fa493200640 1 --2- 172.21.15.123:0/654989730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4941047e0 0x7fa494108830 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.599+0000 7fa493200640 1 --2- 172.21.15.123:0/654989730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa494103e30 0x7fa494104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.599+0000 7fa493200640 1 -- 172.21.15.123:0/654989730 >> 172.21.15.123:0/654989730 conn(0x7fa4940ff8d0 msgr2=0x7fa494101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:48.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.599+0000 7fa493200640 1 -- 172.21.15.123:0/654989730 shutdown_connections 2024-09-19T04:55:48.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.599+0000 7fa493200640 1 -- 172.21.15.123:0/654989730 wait complete. 2024-09-19T04:55:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.600+0000 7fa493200640 1 Processor -- start 2024-09-19T04:55:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.600+0000 7fa493200640 1 -- start start 2024-09-19T04:55:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.600+0000 7fa493200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa494103e30 0x7fa49419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.600+0000 7fa493200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4941047e0 0x7fa49419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.600+0000 7fa493200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa494108f60 0x7fa49419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.600+0000 7fa493200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa494114010 con 0x7fa494103e30 2024-09-19T04:55:48.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.600+0000 7fa493200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa494113e90 con 0x7fa4941047e0 2024-09-19T04:55:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.600+0000 7fa493200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa494114190 con 0x7fa494108f60 2024-09-19T04:55:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.601+0000 7fa491e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa494103e30 0x7fa49419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.601+0000 7fa491e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa494103e30 0x7fa49419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40546/0 (socket says 172.21.15.123:40546) 2024-09-19T04:55:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.601+0000 7fa491e00640 1 -- 172.21.15.123:0/414595793 learned_addr learned my addr 172.21.15.123:0/414595793 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.601+0000 7fa489400640 1 --2- 172.21.15.123:0/414595793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4941047e0 0x7fa49419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.601+0000 7fa492800640 1 --2- 172.21.15.123:0/414595793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa494108f60 0x7fa49419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.601+0000 7fa491e00640 1 -- 172.21.15.123:0/414595793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa494108f60 msgr2=0x7fa49419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.601+0000 7fa491e00640 1 --2- 172.21.15.123:0/414595793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa494108f60 0x7fa49419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.601+0000 7fa491e00640 1 -- 172.21.15.123:0/414595793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4941047e0 msgr2=0x7fa49419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.601+0000 7fa491e00640 1 --2- 172.21.15.123:0/414595793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4941047e0 0x7fa49419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.601+0000 7fa491e00640 1 -- 172.21.15.123:0/414595793 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49419f420 con 0x7fa494103e30 2024-09-19T04:55:48.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.601+0000 7fa491e00640 1 --2- 172.21.15.123:0/414595793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa494103e30 0x7fa49419a470 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa48400e9f0 tx=0x7fa48400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.602+0000 7fa48b400640 1 -- 172.21.15.123:0/414595793 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48400cd80 con 0x7fa494103e30 2024-09-19T04:55:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.602+0000 7fa48b400640 1 -- 172.21.15.123:0/414595793 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa484010910 con 0x7fa494103e30 2024-09-19T04:55:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.602+0000 7fa493200640 1 -- 172.21.15.123:0/414595793 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49419f710 con 0x7fa494103e30 2024-09-19T04:55:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.602+0000 7fa48b400640 1 -- 172.21.15.123:0/414595793 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484018990 con 0x7fa494103e30 2024-09-19T04:55:48.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.602+0000 7fa493200640 1 -- 172.21.15.123:0/414595793 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4941a6ff0 con 0x7fa494103e30 2024-09-19T04:55:48.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.606+0000 7fa49b82f740 1 Processor -- start 2024-09-19T04:55:48.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.606+0000 7fa49b82f740 1 -- start start 2024-09-19T04:55:48.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.606+0000 7fa48b400640 1 -- 172.21.15.123:0/414595793 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa484010ac0 con 0x7fa494103e30 2024-09-19T04:55:48.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.606+0000 7fa49b82f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b7ffe40 0x56462b800220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.606+0000 7fa49b82f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 0x56462b80aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.606+0000 7fa49b82f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b80b020 0x56462b80d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.606+0000 7fa49b82f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56462b7663e0 con 0x56462b7ffe40 2024-09-19T04:55:48.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.606+0000 7fa49b82f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56462b70dd80 con 0x56462b800760 2024-09-19T04:55:48.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.606+0000 7fa49b82f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56462b7623a0 con 0x56462b80b020 2024-09-19T04:55:48.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.607+0000 7fa492800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b7ffe40 0x56462b800220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.607+0000 7fa492800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b7ffe40 0x56462b800220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40550/0 (socket says 172.21.15.123:40550) 2024-09-19T04:55:48.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.607+0000 7fa492800640 1 -- 172.21.15.123:0/2625872832 learned_addr learned my addr 172.21.15.123:0/2625872832 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:48.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.607+0000 7fa491e00640 1 --2- 172.21.15.123:0/2625872832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 0x56462b80aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.607+0000 7fa489400640 1 --2- 172.21.15.123:0/2625872832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b80b020 0x56462b80d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.607+0000 7fa492800640 1 -- 172.21.15.123:0/2625872832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b80b020 msgr2=0x56462b80d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:48.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.607+0000 7fa492800640 1 --2- 172.21.15.123:0/2625872832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b80b020 0x56462b80d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.607+0000 7fa492800640 1 -- 172.21.15.123:0/2625872832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 msgr2=0x56462b80aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:48.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.607+0000 7fa492800640 1 --2- 172.21.15.123:0/2625872832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 0x56462b80aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.607+0000 7fa492800640 1 -- 172.21.15.123:0/2625872832 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56462b72cae0 con 0x56462b7ffe40 2024-09-19T04:55:48.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.607+0000 7fa489400640 1 --2- 172.21.15.123:0/2625872832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b80b020 0x56462b80d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:55:48.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.607+0000 7fa492800640 1 --2- 172.21.15.123:0/2625872832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b7ffe40 0x56462b800220 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009f50 tx=0x7fa48c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1567b61044dd64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:48.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.608+0000 7fa483400640 1 -- 172.21.15.123:0/2625872832 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c004d20 con 0x56462b7ffe40 2024-09-19T04:55:48.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.608+0000 7fa483400640 1 -- 172.21.15.123:0/2625872832 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48c004050 con 0x56462b7ffe40 2024-09-19T04:55:48.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.608+0000 7fa483400640 1 -- 172.21.15.123:0/2625872832 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c005280 con 0x56462b7ffe40 2024-09-19T04:55:48.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.608+0000 7fa48b400640 1 --2- 172.21.15.123:0/414595793 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa4780a33b0 0x7fa4780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.609+0000 7fa49b82f740 1 -- 172.21.15.123:0/2625872832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b7ffe40 msgr2=0x56462b800220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:48.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.609+0000 7fa49b82f740 1 --2- 172.21.15.123:0/2625872832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b7ffe40 0x56462b800220 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009f50 tx=0x7fa48c0118d0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.609+0000 7fa48b400640 1 -- 172.21.15.123:0/414595793 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fa4840d0090 con 0x7fa494103e30 2024-09-19T04:55:48.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.609+0000 7fa49b82f740 1 -- 172.21.15.123:0/2625872832 shutdown_connections 2024-09-19T04:55:48.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.609+0000 7fa49b82f740 1 --2- 172.21.15.123:0/2625872832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b80b020 0x56462b80d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.609+0000 7fa49b82f740 1 --2- 172.21.15.123:0/2625872832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 0x56462b80aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.609+0000 7fa49b82f740 1 --2- 172.21.15.123:0/2625872832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b7ffe40 0x56462b800220 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.609+0000 7fa49b82f740 1 -- 172.21.15.123:0/2625872832 >> 172.21.15.123:0/2625872832 conn(0x56462b7fc3d0 msgr2=0x56462b7fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:48.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.609+0000 7fa491e00640 1 --2- 172.21.15.123:0/414595793 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa4780a33b0 0x7fa4780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.609+0000 7fa49b82f740 1 -- 172.21.15.123:0/2625872832 shutdown_connections 2024-09-19T04:55:48.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.609+0000 7fa49b82f740 1 -- 172.21.15.123:0/2625872832 wait complete. 2024-09-19T04:55:48.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.609+0000 7fa491e00640 1 --2- 172.21.15.123:0/414595793 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa4780a33b0 0x7fa4780a5870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x56462b7fd230 tx=0x7fa484098040 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:48.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.610+0000 7fa49b82f740 1 -- start start 2024-09-19T04:55:48.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.610+0000 7fa49b82f740 1 Processor -- start 2024-09-19T04:55:48.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.610+0000 7fa49b82f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b7ffe40 0x56462b807ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.610+0000 7fa49b82f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 0x56462b8083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.610+0000 7fa49b82f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b80b020 0x56462b808920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.610+0000 7fa49b82f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56462b80dc40 con 0x56462b80b020 2024-09-19T04:55:48.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.610+0000 7fa49b82f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56462b7210f0 con 0x56462b800760 2024-09-19T04:55:48.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.610+0000 7fa49b82f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56462b80ddc0 con 0x56462b7ffe40 2024-09-19T04:55:48.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.610+0000 7fa49b82f740 10 client.0 _ll_register_callbacks cb 0x56462b708bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:48.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.610+0000 7fa489400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 0x56462b8083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.611+0000 7fa489400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 0x56462b8083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:44468/0 (socket says 172.21.15.123:44468) 2024-09-19T04:55:48.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.611+0000 7fa489400640 1 -- 172.21.15.123:0/4149935979 learned_addr learned my addr 172.21.15.123:0/4149935979 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:48.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.611+0000 7fa492800640 1 --2- 172.21.15.123:0/4149935979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b7ffe40 0x56462b807ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.611+0000 7fa492800640 1 --2- 172.21.15.123:0/4149935979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b80b020 0x56462b808920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.611+0000 7fa489400640 1 -- 172.21.15.123:0/4149935979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b7ffe40 msgr2=0x56462b807ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:48.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.611+0000 7fa489400640 1 --2- 172.21.15.123:0/4149935979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b7ffe40 0x56462b807ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.611+0000 7fa489400640 1 -- 172.21.15.123:0/4149935979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b80b020 msgr2=0x56462b808920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:48.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.611+0000 7fa489400640 1 --2- 172.21.15.123:0/4149935979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b80b020 0x56462b808920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.611+0000 7fa489400640 1 -- 172.21.15.123:0/4149935979 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56462b4d73c0 con 0x56462b800760 2024-09-19T04:55:48.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.611+0000 7fa489400640 1 --2- 172.21.15.123:0/4149935979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 0x56462b8083e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009230 tx=0x7fa47c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25d48dfdb41a6de1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:48.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.611+0000 7fa477e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:55:48.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.612+0000 7fa477e00640 1 -- 172.21.15.123:0/4149935979 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c035040 con 0x56462b800760 2024-09-19T04:55:48.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.612+0000 7fa49b82f740 1 -- 172.21.15.123:0/4149935979 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa48c004050 con 0x56462b800760 2024-09-19T04:55:48.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.612+0000 7fa477e00640 1 -- 172.21.15.123:0/4149935979 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fa47c005230 con 0x56462b800760 2024-09-19T04:55:48.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.612+0000 7fa477e00640 1 -- 172.21.15.123:0/4149935979 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa47c005430 con 0x56462b800760 2024-09-19T04:55:48.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.612+0000 7fa49b82f740 10 client.27284 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:48.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.612+0000 7fa49b82f740 1 -- 172.21.15.123:0/4149935979 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56462b819c60 con 0x56462b800760 2024-09-19T04:55:48.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.612+0000 7fa477e00640 1 -- 172.21.15.123:0/4149935979 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48c004050 con 0x56462b800760 2024-09-19T04:55:48.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.612+0000 7fa477e00640 1 -- 172.21.15.123:0/4149935979 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fa47c003b40 con 0x56462b800760 2024-09-19T04:55:48.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.613+0000 7fa49b82f740 10 client.27284 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:48.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.613+0000 7fa49b82f740 10 client.27284 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:55:48.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.613+0000 7fa49b82f740 20 client.27284 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:48.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.613+0000 7fa49b82f740 1 --2- 172.21.15.123:0/4149935979 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x56462b81b070 0x56462b81d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.613+0000 7fa49b82f740 4 client.27284 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:55:48.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.613+0000 7fa49b82f740 1 -- 172.21.15.123:0/4149935979 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56462b773950 con 0x56462b81b070 2024-09-19T04:55:48.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.613+0000 7fa492800640 1 --2- 172.21.15.123:0/4149935979 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x56462b81b070 0x56462b81d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.613+0000 7fa492800640 1 --2- 172.21.15.123:0/4149935979 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x56462b81b070 0x56462b81d490 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6fbf7c4049f0a704 server_cookie=f8f9bb09c2ea006a in_seq=0 out_seq=0 2024-09-19T04:55:48.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.613+0000 7fa477e00640 10 client.27284 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:55:48.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.614+0000 7fa477e00640 1 -- 172.21.15.123:0/4149935979 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56462b773950 con 0x56462b81b070 2024-09-19T04:55:48.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.614+0000 7fa477e00640 10 client.27284 handle_command_reply: tid=0 2024-09-19T04:55:48.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.614+0000 7fa49b82f740 1 client.27284 shutdown 2024-09-19T04:55:48.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.614+0000 7fa49b82f740 1 -- 172.21.15.123:0/4149935979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 msgr2=0x56462b8083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:48.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.614+0000 7fa49b82f740 1 --2- 172.21.15.123:0/4149935979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 0x56462b8083e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009230 tx=0x7fa47c004270 comp rx=0 tx=0).stop 2024-09-19T04:55:48.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.615+0000 7fa49b82f740 1 -- 172.21.15.123:0/4149935979 shutdown_connections 2024-09-19T04:55:48.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.615+0000 7fa49b82f740 1 --2- 172.21.15.123:0/4149935979 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x56462b81b070 0x56462b81d490 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.615+0000 7fa49b82f740 1 --2- 172.21.15.123:0/4149935979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b80b020 0x56462b808920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.615+0000 7fa49b82f740 1 --2- 172.21.15.123:0/4149935979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 0x56462b8083e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.615+0000 7fa49b82f740 1 --2- 172.21.15.123:0/4149935979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b7ffe40 0x56462b807ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.615+0000 7fa477e00640 0 client.27284 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:55:48.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.615+0000 7fa49b82f740 1 -- 172.21.15.123:0/4149935979 >> 172.21.15.123:0/4149935979 conn(0x56462b7fc3d0 msgr2=0x56462b802100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:48.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.615+0000 7fa49b82f740 1 -- 172.21.15.123:0/4149935979 shutdown_connections 2024-09-19T04:55:48.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.615+0000 7fa49b82f740 1 -- 172.21.15.123:0/4149935979 wait complete. 2024-09-19T04:55:48.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.615+0000 7fa49b82f740 20 client.27284 trim_cache size 0 max 16384 2024-09-19T04:55:48.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.629+0000 7fa49b82f740 1 Processor -- start 2024-09-19T04:55:48.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.629+0000 7fa49b82f740 1 -- start start 2024-09-19T04:55:48.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.629+0000 7fa49b82f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b7ffe40 0x56462b80a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.629+0000 7fa49b82f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b800760 0x56462b808770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.629+0000 7fa49b82f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b80b020 0x56462b822d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.629+0000 7fa49b82f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa4941af970 con 0x56462b80b020 2024-09-19T04:55:48.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.629+0000 7fa49b82f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa4941b14b0 con 0x56462b7ffe40 2024-09-19T04:55:48.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.629+0000 7fa49b82f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa4941b0040 con 0x56462b800760 2024-09-19T04:55:48.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.630+0000 7fa492800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b80b020 0x56462b822d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.630+0000 7fa492800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b80b020 0x56462b822d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40576/0 (socket says 172.21.15.123:40576) 2024-09-19T04:55:48.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.630+0000 7fa492800640 1 -- 172.21.15.123:0/3030895054 learned_addr learned my addr 172.21.15.123:0/3030895054 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:48.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.630+0000 7fa492800640 1 --2- 172.21.15.123:0/3030895054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b7ffe40 0x56462b80a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.630+0000 7fa489400640 1 --2- 172.21.15.123:0/3030895054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b800760 0x56462b808770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.630+0000 7fa492800640 1 -- 172.21.15.123:0/3030895054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b800760 msgr2=0x56462b808770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:48.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.630+0000 7fa492800640 1 --2- 172.21.15.123:0/3030895054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b800760 0x56462b808770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.630+0000 7fa492800640 1 -- 172.21.15.123:0/3030895054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b7ffe40 msgr2=0x56462b80a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:48.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.630+0000 7fa492800640 1 --2- 172.21.15.123:0/3030895054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b7ffe40 0x56462b80a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.630+0000 7fa492800640 1 -- 172.21.15.123:0/3030895054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56462b800590 con 0x56462b80b020 2024-09-19T04:55:48.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.630+0000 7fa492800640 1 --2- 172.21.15.123:0/3030895054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b80b020 0x56462b822d60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa48c011dd0 tx=0x7fa48c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37169779fc0bf312 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:48.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.631+0000 7fa482a00640 1 -- 172.21.15.123:0/3030895054 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c004d80 con 0x56462b80b020 2024-09-19T04:55:48.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.631+0000 7fa482a00640 1 -- 172.21.15.123:0/3030895054 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48c004560 con 0x56462b80b020 2024-09-19T04:55:48.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.631+0000 7fa482a00640 1 -- 172.21.15.123:0/3030895054 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c0057e0 con 0x56462b80b020 2024-09-19T04:55:48.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.631+0000 7fa49b82f740 1 -- 172.21.15.123:0/3030895054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b80b020 msgr2=0x56462b822d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:48.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.631+0000 7fa49b82f740 1 --2- 172.21.15.123:0/3030895054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b80b020 0x56462b822d60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa48c011dd0 tx=0x7fa48c009500 comp rx=0 tx=0).stop 2024-09-19T04:55:48.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.631+0000 7fa49b82f740 1 -- 172.21.15.123:0/3030895054 shutdown_connections 2024-09-19T04:55:48.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.631+0000 7fa49b82f740 1 --2- 172.21.15.123:0/3030895054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b80b020 0x56462b822d60 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.631+0000 7fa49b82f740 1 --2- 172.21.15.123:0/3030895054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b800760 0x56462b808770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.631+0000 7fa49b82f740 1 --2- 172.21.15.123:0/3030895054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b7ffe40 0x56462b80a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.631+0000 7fa49b82f740 1 -- 172.21.15.123:0/3030895054 >> 172.21.15.123:0/3030895054 conn(0x56462b7fc3d0 msgr2=0x56462b8020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:48.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.631+0000 7fa49b82f740 1 -- 172.21.15.123:0/3030895054 shutdown_connections 2024-09-19T04:55:48.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.632+0000 7fa49b82f740 1 -- 172.21.15.123:0/3030895054 wait complete. 2024-09-19T04:55:48.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.632+0000 7fa49b82f740 1 -- start start 2024-09-19T04:55:48.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.632+0000 7fa49b82f740 1 Processor -- start 2024-09-19T04:55:48.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.632+0000 7fa49b82f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b7ffe40 0x56462b8050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.632+0000 7fa49b82f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 0x56462b8055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa49b82f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b80b020 0x56462b805b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa49b82f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa4941b0de0 con 0x56462b7ffe40 2024-09-19T04:55:48.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa49b82f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa4941b0710 con 0x56462b800760 2024-09-19T04:55:48.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa49b82f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa4941ad970 con 0x56462b80b020 2024-09-19T04:55:48.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa49b82f740 10 client.0 _ll_register_callbacks cb 0x7fa48c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:48.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa489400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 0x56462b8055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa489400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 0x56462b8055f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:44488/0 (socket says 172.21.15.123:44488) 2024-09-19T04:55:48.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa489400640 1 -- 172.21.15.123:0/1658201104 learned_addr learned my addr 172.21.15.123:0/1658201104 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:48.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa492800640 1 --2- 172.21.15.123:0/1658201104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b7ffe40 0x56462b8050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa492800640 1 --2- 172.21.15.123:0/1658201104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b80b020 0x56462b805b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa489400640 1 -- 172.21.15.123:0/1658201104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b80b020 msgr2=0x56462b805b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:48.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa489400640 1 --2- 172.21.15.123:0/1658201104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b80b020 0x56462b805b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa489400640 1 -- 172.21.15.123:0/1658201104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b7ffe40 msgr2=0x56462b8050b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:48.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa489400640 1 --2- 172.21.15.123:0/1658201104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b7ffe40 0x56462b8050b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa489400640 1 -- 172.21.15.123:0/1658201104 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56462b884f30 con 0x56462b800760 2024-09-19T04:55:48.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa492800640 1 --2- 172.21.15.123:0/1658201104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b80b020 0x56462b805b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:55:48.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa489400640 1 --2- 172.21.15.123:0/1658201104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 0x56462b8055f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa47c02a910 tx=0x7fa47c005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8df911fb7e35425f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:48.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.633+0000 7fa477e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:55:48.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.634+0000 7fa477e00640 1 -- 172.21.15.123:0/1658201104 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c035070 con 0x56462b800760 2024-09-19T04:55:48.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.634+0000 7fa477e00640 1 -- 172.21.15.123:0/1658201104 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fa47c031dc0 con 0x56462b800760 2024-09-19T04:55:48.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.634+0000 7fa49b82f740 1 -- 172.21.15.123:0/1658201104 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa48c004560 con 0x56462b800760 2024-09-19T04:55:48.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.634+0000 7fa477e00640 1 -- 172.21.15.123:0/1658201104 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa47c051040 con 0x56462b800760 2024-09-19T04:55:48.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.634+0000 7fa49b82f740 10 client.27290 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:48.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.634+0000 7fa49b82f740 1 -- 172.21.15.123:0/1658201104 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56462b886210 con 0x56462b800760 2024-09-19T04:55:48.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.634+0000 7fa477e00640 1 -- 172.21.15.123:0/1658201104 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48c004560 con 0x56462b800760 2024-09-19T04:55:48.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.635+0000 7fa477e00640 1 -- 172.21.15.123:0/1658201104 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fa47c054070 con 0x56462b800760 2024-09-19T04:55:48.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.635+0000 7fa49b82f740 10 client.27290 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:48.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.635+0000 7fa49b82f740 10 client.27290 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:55:48.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.635+0000 7fa49b82f740 20 client.27290 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:48.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.635+0000 7fa49b82f740 1 --2- 172.21.15.123:0/1658201104 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x56462b8869e0 0x56462b888e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.635+0000 7fa49b82f740 4 client.27290 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:55:48.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.635+0000 7fa49b82f740 1 -- 172.21.15.123:0/1658201104 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56462b76de60 con 0x56462b8869e0 2024-09-19T04:55:48.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.635+0000 7fa492800640 1 --2- 172.21.15.123:0/1658201104 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x56462b8869e0 0x56462b888e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.636+0000 7fa477e00640 10 client.27290 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:55:48.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.636+0000 7fa492800640 1 --2- 172.21.15.123:0/1658201104 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x56462b8869e0 0x56462b888e00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=727a098c8a2b65c8 server_cookie=d9930ba8a37c4985 in_seq=0 out_seq=0 2024-09-19T04:55:48.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.636+0000 7fa477e00640 1 -- 172.21.15.123:0/1658201104 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56462b76de60 con 0x56462b8869e0 2024-09-19T04:55:48.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.636+0000 7fa477e00640 10 client.27290 handle_command_reply: tid=0 2024-09-19T04:55:48.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.636+0000 7fa49b82f740 1 client.27290 shutdown 2024-09-19T04:55:48.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.636+0000 7fa49b82f740 1 -- 172.21.15.123:0/1658201104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 msgr2=0x56462b8055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:48.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.636+0000 7fa49b82f740 1 --2- 172.21.15.123:0/1658201104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 0x56462b8055f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa47c02a910 tx=0x7fa47c005d50 comp rx=0 tx=0).stop 2024-09-19T04:55:48.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.636+0000 7fa49b82f740 1 -- 172.21.15.123:0/1658201104 shutdown_connections 2024-09-19T04:55:48.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.636+0000 7fa49b82f740 1 --2- 172.21.15.123:0/1658201104 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x56462b8869e0 0x56462b888e00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.636+0000 7fa49b82f740 1 --2- 172.21.15.123:0/1658201104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56462b80b020 0x56462b805b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.636+0000 7fa49b82f740 1 --2- 172.21.15.123:0/1658201104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56462b800760 0x56462b8055f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.636+0000 7fa49b82f740 1 --2- 172.21.15.123:0/1658201104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56462b7ffe40 0x56462b8050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.636+0000 7fa477e00640 0 client.27290 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:55:48.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.636+0000 7fa49b82f740 1 -- 172.21.15.123:0/1658201104 >> 172.21.15.123:0/1658201104 conn(0x56462b7fc3d0 msgr2=0x56462b7fb490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:48.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.636+0000 7fa49b82f740 1 -- 172.21.15.123:0/1658201104 shutdown_connections 2024-09-19T04:55:48.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.636+0000 7fa49b82f740 1 -- 172.21.15.123:0/1658201104 wait complete. 2024-09-19T04:55:48.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.636+0000 7fa49b82f740 20 client.27290 trim_cache size 0 max 16384 2024-09-19T04:55:48.642 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:55:48.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.637+0000 7fa493200640 1 -- 172.21.15.123:0/414595793 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa4780a33b0 msgr2=0x7fa4780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:48.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.637+0000 7fa493200640 1 --2- 172.21.15.123:0/414595793 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa4780a33b0 0x7fa4780a5870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x56462b7fd230 tx=0x7fa484098040 comp rx=0 tx=0).stop 2024-09-19T04:55:48.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.637+0000 7fa493200640 1 -- 172.21.15.123:0/414595793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa494103e30 msgr2=0x7fa49419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:48.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.637+0000 7fa493200640 1 --2- 172.21.15.123:0/414595793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa494103e30 0x7fa49419a470 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa48400e9f0 tx=0x7fa48400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.637+0000 7fa493200640 1 -- 172.21.15.123:0/414595793 shutdown_connections 2024-09-19T04:55:48.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.637+0000 7fa493200640 1 --2- 172.21.15.123:0/414595793 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fa4780a33b0 0x7fa4780a5870 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.637+0000 7fa493200640 1 --2- 172.21.15.123:0/414595793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa494108f60 0x7fa49419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.637+0000 7fa493200640 1 --2- 172.21.15.123:0/414595793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4941047e0 0x7fa49419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.637+0000 7fa493200640 1 --2- 172.21.15.123:0/414595793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa494103e30 0x7fa49419a470 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.637+0000 7fa493200640 1 -- 172.21.15.123:0/414595793 >> 172.21.15.123:0/414595793 conn(0x7fa4940ff8d0 msgr2=0x7fa494100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:48.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.637+0000 7fa493200640 1 -- 172.21.15.123:0/414595793 shutdown_connections 2024-09-19T04:55:48.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:48.638+0000 7fa493200640 1 -- 172.21.15.123:0/414595793 wait complete. 2024-09-19T04:55:49.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:49 smithi123 ceph-mon[30511]: pgmap v282: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-09-19T04:55:49.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:49 smithi193 ceph-mon[35592]: pgmap v282: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-09-19T04:55:49.577 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:55:49.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:49 smithi183 ceph-mon[35333]: pgmap v282: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-09-19T04:55:49.943 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:50.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.476+0000 7f70d3200640 1 -- 172.21.15.123:0/3353749822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70d41079c0 msgr2=0x7f70d4107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:50.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.476+0000 7f70d3200640 1 --2- 172.21.15.123:0/3353749822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70d41079c0 0x7f70d4107da0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009930 tx=0x7f70c402a440 comp rx=0 tx=0).stop 2024-09-19T04:55:50.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.476+0000 7f70d3200640 1 -- 172.21.15.123:0/3353749822 shutdown_connections 2024-09-19T04:55:50.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.476+0000 7f70d3200640 1 --2- 172.21.15.123:0/3353749822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d4102360 0x7f70d410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.476+0000 7f70d3200640 1 --2- 172.21.15.123:0/3353749822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d41019c0 0x7f70d4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.476+0000 7f70d3200640 1 --2- 172.21.15.123:0/3353749822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70d41079c0 0x7f70d4107da0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.476+0000 7f70d3200640 1 -- 172.21.15.123:0/3353749822 >> 172.21.15.123:0/3353749822 conn(0x7f70d40fd740 msgr2=0x7f70d40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:50.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.476+0000 7f70d3200640 1 -- 172.21.15.123:0/3353749822 shutdown_connections 2024-09-19T04:55:50.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.476+0000 7f70d3200640 1 -- 172.21.15.123:0/3353749822 wait complete. 2024-09-19T04:55:50.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.477+0000 7f70d3200640 1 Processor -- start 2024-09-19T04:55:50.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.477+0000 7f70d3200640 1 -- start start 2024-09-19T04:55:50.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.477+0000 7f70d3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70d41019c0 0x7f70d419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.477+0000 7f70d3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d4102360 0x7f70d419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.477+0000 7f70d3200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d41079c0 0x7f70d419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.477+0000 7f70d3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70d4111e10 con 0x7f70d41019c0 2024-09-19T04:55:50.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.477+0000 7f70d3200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70d4111c90 con 0x7f70d4102360 2024-09-19T04:55:50.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.477+0000 7f70d3200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70d4111f90 con 0x7f70d41079c0 2024-09-19T04:55:50.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.478+0000 7f70d1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70d41019c0 0x7f70d419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.478+0000 7f70d1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70d41019c0 0x7f70d419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40610/0 (socket says 172.21.15.123:40610) 2024-09-19T04:55:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.478+0000 7f70d1e00640 1 -- 172.21.15.123:0/3322220285 learned_addr learned my addr 172.21.15.123:0/3322220285 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.478+0000 7f70d1400640 1 --2- 172.21.15.123:0/3322220285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d4102360 0x7f70d419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.478+0000 7f70d2800640 1 --2- 172.21.15.123:0/3322220285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d41079c0 0x7f70d419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.478+0000 7f70d1e00640 1 -- 172.21.15.123:0/3322220285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d41079c0 msgr2=0x7f70d419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.478+0000 7f70d1e00640 1 --2- 172.21.15.123:0/3322220285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d41079c0 0x7f70d419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.478+0000 7f70d1e00640 1 -- 172.21.15.123:0/3322220285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d4102360 msgr2=0x7f70d419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.478+0000 7f70d1e00640 1 --2- 172.21.15.123:0/3322220285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d4102360 0x7f70d419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.478+0000 7f70d1e00640 1 -- 172.21.15.123:0/3322220285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d419f4a0 con 0x7f70d41019c0 2024-09-19T04:55:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.479+0000 7f70d1e00640 1 --2- 172.21.15.123:0/3322220285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70d41019c0 0x7f70d419a4f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f70c402a960 tx=0x7f70c4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:50.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.479+0000 7f70caa00640 1 -- 172.21.15.123:0/3322220285 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c403f070 con 0x7f70d41019c0 2024-09-19T04:55:50.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.479+0000 7f70caa00640 1 -- 172.21.15.123:0/3322220285 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c4031c90 con 0x7f70d41019c0 2024-09-19T04:55:50.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.479+0000 7f70d3200640 1 -- 172.21.15.123:0/3322220285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d419f730 con 0x7f70d41019c0 2024-09-19T04:55:50.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.479+0000 7f70caa00640 1 -- 172.21.15.123:0/3322220285 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4035030 con 0x7f70d41019c0 2024-09-19T04:55:50.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.479+0000 7f70d3200640 1 -- 172.21.15.123:0/3322220285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d41a7070 con 0x7f70d41019c0 2024-09-19T04:55:50.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.482+0000 7f70db5fd740 1 Processor -- start 2024-09-19T04:55:50.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.482+0000 7f70db5fd740 1 -- start start 2024-09-19T04:55:50.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.482+0000 7f70caa00640 1 -- 172.21.15.123:0/3322220285 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f70c4042070 con 0x7f70d41019c0 2024-09-19T04:55:50.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.483+0000 7f70db5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edc4e40 0x56351edc5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.483+0000 7f70db5fd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc5760 0x56351edcfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.483+0000 7f70db5fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edd0020 0x56351edd2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.483+0000 7f70db5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56351ed2b3e0 con 0x56351edc4e40 2024-09-19T04:55:50.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.483+0000 7f70db5fd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56351ecd2d80 con 0x56351edc5760 2024-09-19T04:55:50.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.483+0000 7f70db5fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56351ed273a0 con 0x56351edd0020 2024-09-19T04:55:50.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.483+0000 7f70d2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edc4e40 0x56351edc5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.483+0000 7f70d2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edc4e40 0x56351edc5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40620/0 (socket says 172.21.15.123:40620) 2024-09-19T04:55:50.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.483+0000 7f70d1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc5760 0x56351edcfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.483+0000 7f70d2800640 1 -- 172.21.15.123:0/2919063564 learned_addr learned my addr 172.21.15.123:0/2919063564 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:50.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.483+0000 7f70d1400640 1 --2- 172.21.15.123:0/2919063564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edd0020 0x56351edd2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.483+0000 7f70d1400640 1 -- 172.21.15.123:0/2919063564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc5760 msgr2=0x56351edcfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:50.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.483+0000 7f70d1400640 1 --2- 172.21.15.123:0/2919063564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc5760 0x56351edcfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.483+0000 7f70d1400640 1 -- 172.21.15.123:0/2919063564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edc4e40 msgr2=0x56351edc5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:50.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.483+0000 7f70d1400640 1 --2- 172.21.15.123:0/2919063564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edc4e40 0x56351edc5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.483+0000 7f70d1400640 1 -- 172.21.15.123:0/2919063564 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56351ecf1ae0 con 0x56351edd0020 2024-09-19T04:55:50.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.484+0000 7f70d1400640 1 --2- 172.21.15.123:0/2919063564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edd0020 0x56351edd2410 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009bd0 tx=0x7f70c0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d62cd8b4dc05ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:50.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.484+0000 7f70caa00640 1 --2- 172.21.15.123:0/3322220285 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f70b80a33b0 0x7f70b80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.484+0000 7f70bf400640 1 -- 172.21.15.123:0/2919063564 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0004230 con 0x56351edd0020 2024-09-19T04:55:50.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.484+0000 7f70bf400640 1 -- 172.21.15.123:0/2919063564 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c0009e40 con 0x56351edd0020 2024-09-19T04:55:50.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.484+0000 7f70bf400640 1 -- 172.21.15.123:0/2919063564 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0004db0 con 0x56351edd0020 2024-09-19T04:55:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.484+0000 7f70caa00640 1 -- 172.21.15.123:0/3322220285 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f70c40f0480 con 0x7f70d41019c0 2024-09-19T04:55:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.484+0000 7f70d2800640 1 --2- 172.21.15.123:0/3322220285 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f70b80a33b0 0x7f70b80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.485+0000 7f70db5fd740 1 -- 172.21.15.123:0/2919063564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edd0020 msgr2=0x56351edd2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.485+0000 7f70db5fd740 1 --2- 172.21.15.123:0/2919063564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edd0020 0x56351edd2410 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009bd0 tx=0x7f70c0011810 comp rx=0 tx=0).stop 2024-09-19T04:55:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.485+0000 7f70db5fd740 1 -- 172.21.15.123:0/2919063564 shutdown_connections 2024-09-19T04:55:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.485+0000 7f70db5fd740 1 --2- 172.21.15.123:0/2919063564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edd0020 0x56351edd2410 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.485+0000 7f70db5fd740 1 --2- 172.21.15.123:0/2919063564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc5760 0x56351edcfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.485+0000 7f70db5fd740 1 --2- 172.21.15.123:0/2919063564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edc4e40 0x56351edc5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.485+0000 7f70db5fd740 1 -- 172.21.15.123:0/2919063564 >> 172.21.15.123:0/2919063564 conn(0x56351edc13d0 msgr2=0x56351edc17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.485+0000 7f70d2800640 1 --2- 172.21.15.123:0/3322220285 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f70b80a33b0 0x7f70b80a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f70cc005f40 tx=0x7f70cc009600 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.485+0000 7f70db5fd740 1 -- 172.21.15.123:0/2919063564 shutdown_connections 2024-09-19T04:55:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.485+0000 7f70db5fd740 1 -- 172.21.15.123:0/2919063564 wait complete. 2024-09-19T04:55:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.485+0000 7f70db5fd740 1 -- start start 2024-09-19T04:55:50.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.486+0000 7f70db5fd740 1 Processor -- start 2024-09-19T04:55:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.486+0000 7f70db5fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edc4e40 0x56351edccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.486+0000 7f70db5fd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc5760 0x56351edcd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.486+0000 7f70db5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edd0020 0x56351edcd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.486+0000 7f70db5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56351edd2bf0 con 0x56351edd0020 2024-09-19T04:55:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.486+0000 7f70db5fd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56351ece60f0 con 0x56351edc5760 2024-09-19T04:55:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.486+0000 7f70db5fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56351edd2d70 con 0x56351edc4e40 2024-09-19T04:55:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.486+0000 7f70db5fd740 10 client.0 _ll_register_callbacks cb 0x56351eccdbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.487+0000 7f70d2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edd0020 0x56351edcd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.487+0000 7f70d2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edd0020 0x56351edcd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40628/0 (socket says 172.21.15.123:40628) 2024-09-19T04:55:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.487+0000 7f70d2800640 1 -- 172.21.15.123:0/1062499193 learned_addr learned my addr 172.21.15.123:0/1062499193 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:50.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.487+0000 7f70d1400640 1 --2- 172.21.15.123:0/1062499193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc5760 0x56351edcd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.487+0000 7f70d1e00640 1 --2- 172.21.15.123:0/1062499193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edc4e40 0x56351edccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.487+0000 7f70d1e00640 1 -- 172.21.15.123:0/1062499193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc5760 msgr2=0x56351edcd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:50.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.487+0000 7f70d1e00640 1 --2- 172.21.15.123:0/1062499193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc5760 0x56351edcd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.487+0000 7f70d1e00640 1 -- 172.21.15.123:0/1062499193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edd0020 msgr2=0x56351edcd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:50.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.487+0000 7f70d1e00640 1 --2- 172.21.15.123:0/1062499193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edd0020 0x56351edcd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.487+0000 7f70d1e00640 1 -- 172.21.15.123:0/1062499193 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56351ea9c3c0 con 0x56351edc4e40 2024-09-19T04:55:50.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.487+0000 7f70d1e00640 1 --2- 172.21.15.123:0/1062499193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edc4e40 0x56351edccee0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f70c40edb70 tx=0x7f70c4005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8d19fecb17252df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:50.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.487+0000 7f70b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:55:50.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.487+0000 7f70b3e00640 1 -- 172.21.15.123:0/1062499193 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4049070 con 0x56351edc4e40 2024-09-19T04:55:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.488+0000 7f70b3e00640 1 -- 172.21.15.123:0/1062499193 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f70c40eddf0 con 0x56351edc4e40 2024-09-19T04:55:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.488+0000 7f70db5fd740 1 -- 172.21.15.123:0/1062499193 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56351ed38950 con 0x56351edc4e40 2024-09-19T04:55:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.488+0000 7f70b3e00640 1 -- 172.21.15.123:0/1062499193 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f70c40bc2f0 con 0x56351edc4e40 2024-09-19T04:55:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.488+0000 7f70db5fd740 10 client.27319 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.488+0000 7f70db5fd740 1 -- 172.21.15.123:0/1062499193 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56351edd2bf0 con 0x56351edc4e40 2024-09-19T04:55:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.488+0000 7f70b3e00640 1 -- 172.21.15.123:0/1062499193 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56351ed38950 con 0x56351edc4e40 2024-09-19T04:55:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.488+0000 7f70b3e00640 1 -- 172.21.15.123:0/1062499193 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f70c40c4ee0 con 0x56351edc4e40 2024-09-19T04:55:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.488+0000 7f70db5fd740 10 client.27319 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.488+0000 7f70db5fd740 10 client.27319 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:55:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.488+0000 7f70db5fd740 20 client.27319 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.488+0000 7f70db5fd740 1 --2- 172.21.15.123:0/1062499193 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x56351ede0020 0x56351ede2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.488+0000 7f70db5fd740 4 client.27319 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:55:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.488+0000 7f70db5fd740 1 -- 172.21.15.123:0/1062499193 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56351ed32e60 con 0x56351ede0020 2024-09-19T04:55:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.489+0000 7f70d1400640 1 --2- 172.21.15.123:0/1062499193 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x56351ede0020 0x56351ede2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.489+0000 7f70d1400640 1 --2- 172.21.15.123:0/1062499193 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x56351ede0020 0x56351ede2470 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2b70e1eed23f4bf2 server_cookie=eafbca174535f2d1 in_seq=0 out_seq=0 2024-09-19T04:55:50.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.489+0000 7f70b3e00640 10 client.27319 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:55:50.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.490+0000 7f70b3e00640 1 -- 172.21.15.123:0/1062499193 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56351ed32e60 con 0x56351ede0020 2024-09-19T04:55:50.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.490+0000 7f70b3e00640 10 client.27319 handle_command_reply: tid=0 2024-09-19T04:55:50.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.490+0000 7f70db5fd740 1 client.27319 shutdown 2024-09-19T04:55:50.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.490+0000 7f70db5fd740 1 -- 172.21.15.123:0/1062499193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edc4e40 msgr2=0x56351edccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:50.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.490+0000 7f70db5fd740 1 --2- 172.21.15.123:0/1062499193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edc4e40 0x56351edccee0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f70c40edb70 tx=0x7f70c4005dc0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.490+0000 7f70db5fd740 1 -- 172.21.15.123:0/1062499193 shutdown_connections 2024-09-19T04:55:50.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.490+0000 7f70db5fd740 1 --2- 172.21.15.123:0/1062499193 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x56351ede0020 0x56351ede2470 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.490+0000 7f70db5fd740 1 --2- 172.21.15.123:0/1062499193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edd0020 0x56351edcd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.490+0000 7f70db5fd740 1 --2- 172.21.15.123:0/1062499193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc5760 0x56351edcd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.490+0000 7f70db5fd740 1 --2- 172.21.15.123:0/1062499193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edc4e40 0x56351edccee0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.490+0000 7f70db5fd740 1 -- 172.21.15.123:0/1062499193 >> 172.21.15.123:0/1062499193 conn(0x56351edc13d0 msgr2=0x56351edc7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:50.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.490+0000 7f70b3e00640 0 client.27319 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:55:50.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.490+0000 7f70db5fd740 1 -- 172.21.15.123:0/1062499193 shutdown_connections 2024-09-19T04:55:50.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.491+0000 7f70db5fd740 1 -- 172.21.15.123:0/1062499193 wait complete. 2024-09-19T04:55:50.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.491+0000 7f70db5fd740 20 client.27319 trim_cache size 0 max 16384 2024-09-19T04:55:50.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.504+0000 7f70db5fd740 1 Processor -- start 2024-09-19T04:55:50.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.504+0000 7f70db5fd740 1 -- start start 2024-09-19T04:55:50.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.504+0000 7f70db5fd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc4e40 0x56351edc78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.504+0000 7f70db5fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edc5760 0x56351ede7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.504+0000 7f70db5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edd0020 0x56351ede8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.504+0000 7f70db5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70d41af8c0 con 0x56351edd0020 2024-09-19T04:55:50.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.504+0000 7f70db5fd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70d41b1400 con 0x56351edc4e40 2024-09-19T04:55:50.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.504+0000 7f70db5fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70d41aff90 con 0x56351edc5760 2024-09-19T04:55:50.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.505+0000 7f70d2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edd0020 0x56351ede8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.505+0000 7f70d1400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edc5760 0x56351ede7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.505+0000 7f70d2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edd0020 0x56351ede8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40636/0 (socket says 172.21.15.123:40636) 2024-09-19T04:55:50.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.505+0000 7f70d2800640 1 -- 172.21.15.123:0/602801208 learned_addr learned my addr 172.21.15.123:0/602801208 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:50.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.505+0000 7f70d1e00640 1 --2- 172.21.15.123:0/602801208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc4e40 0x56351edc78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.505+0000 7f70d1400640 1 -- 172.21.15.123:0/602801208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc4e40 msgr2=0x56351edc78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:50.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.505+0000 7f70d1400640 1 --2- 172.21.15.123:0/602801208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc4e40 0x56351edc78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.505+0000 7f70d1400640 1 -- 172.21.15.123:0/602801208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edd0020 msgr2=0x56351ede8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:50.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.505+0000 7f70d1400640 1 --2- 172.21.15.123:0/602801208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edd0020 0x56351ede8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.505+0000 7f70d1400640 1 -- 172.21.15.123:0/602801208 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56351edc1200 con 0x56351edc5760 2024-09-19T04:55:50.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.505+0000 7f70d1400640 1 --2- 172.21.15.123:0/602801208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edc5760 0x56351ede7d10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f70c0005890 tx=0x7f70c0009930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65b9e87409b23e1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:50.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.506+0000 7f70bea00640 1 -- 172.21.15.123:0/602801208 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0030070 con 0x56351edc5760 2024-09-19T04:55:50.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.506+0000 7f70bea00640 1 -- 172.21.15.123:0/602801208 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c00043a0 con 0x56351edc5760 2024-09-19T04:55:50.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.506+0000 7f70bea00640 1 -- 172.21.15.123:0/602801208 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c001e040 con 0x56351edc5760 2024-09-19T04:55:50.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.506+0000 7f70db5fd740 1 -- 172.21.15.123:0/602801208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edc5760 msgr2=0x56351ede7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:50.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.506+0000 7f70db5fd740 1 --2- 172.21.15.123:0/602801208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edc5760 0x56351ede7d10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f70c0005890 tx=0x7f70c0009930 comp rx=0 tx=0).stop 2024-09-19T04:55:50.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.506+0000 7f70db5fd740 1 -- 172.21.15.123:0/602801208 shutdown_connections 2024-09-19T04:55:50.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.506+0000 7f70db5fd740 1 --2- 172.21.15.123:0/602801208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edd0020 0x56351ede8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.506+0000 7f70db5fd740 1 --2- 172.21.15.123:0/602801208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edc5760 0x56351ede7d10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.506+0000 7f70db5fd740 1 --2- 172.21.15.123:0/602801208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc4e40 0x56351edc78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.506+0000 7f70db5fd740 1 -- 172.21.15.123:0/602801208 >> 172.21.15.123:0/602801208 conn(0x56351edc13d0 msgr2=0x56351edcef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:50.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.507+0000 7f70db5fd740 1 -- 172.21.15.123:0/602801208 shutdown_connections 2024-09-19T04:55:50.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.507+0000 7f70db5fd740 1 -- 172.21.15.123:0/602801208 wait complete. 2024-09-19T04:55:50.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.507+0000 7f70db5fd740 1 -- start start 2024-09-19T04:55:50.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.507+0000 7f70db5fd740 1 Processor -- start 2024-09-19T04:55:50.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.507+0000 7f70db5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edc4e40 0x56351edc9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.507+0000 7f70db5fd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc5760 0x56351edca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.507+0000 7f70db5fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edd0020 0x56351edca860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.507+0000 7f70db5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70d41b0d30 con 0x56351edc4e40 2024-09-19T04:55:50.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.507+0000 7f70db5fd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70d41b0660 con 0x56351edc5760 2024-09-19T04:55:50.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.507+0000 7f70db5fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70d41ad8c0 con 0x56351edd0020 2024-09-19T04:55:50.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.508+0000 7f70db5fd740 10 client.0 _ll_register_callbacks cb 0x7f70c40f3c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:50.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.508+0000 7f70d1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edc4e40 0x56351edc9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.508+0000 7f70d1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edc4e40 0x56351edc9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40646/0 (socket says 172.21.15.123:40646) 2024-09-19T04:55:50.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.508+0000 7f70d1e00640 1 -- 172.21.15.123:0/3605059607 learned_addr learned my addr 172.21.15.123:0/3605059607 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:50.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.508+0000 7f70d2800640 1 --2- 172.21.15.123:0/3605059607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edd0020 0x56351edca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.508+0000 7f70d1400640 1 --2- 172.21.15.123:0/3605059607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc5760 0x56351edca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.508+0000 7f70d1e00640 1 -- 172.21.15.123:0/3605059607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edd0020 msgr2=0x56351edca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:50.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.508+0000 7f70d1e00640 1 --2- 172.21.15.123:0/3605059607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edd0020 0x56351edca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.508+0000 7f70d1e00640 1 -- 172.21.15.123:0/3605059607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc5760 msgr2=0x56351edca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:50.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.508+0000 7f70d1e00640 1 --2- 172.21.15.123:0/3605059607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc5760 0x56351edca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.508+0000 7f70d1e00640 1 -- 172.21.15.123:0/3605059607 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56351ee49ce0 con 0x56351edc4e40 2024-09-19T04:55:50.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.508+0000 7f70d1400640 1 --2- 172.21.15.123:0/3605059607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc5760 0x56351edca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:55:50.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.508+0000 7f70d1e00640 1 --2- 172.21.15.123:0/3605059607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edc4e40 0x56351edc9de0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f70c40b8c10 tx=0x7f70c403b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23fe899fa8010ed3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:50.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.508+0000 7f70b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:55:50.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.509+0000 7f70b3e00640 1 -- 172.21.15.123:0/3605059607 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4049040 con 0x56351edc4e40 2024-09-19T04:55:50.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.509+0000 7f70b3e00640 1 -- 172.21.15.123:0/3605059607 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f70c40bbe00 con 0x56351edc4e40 2024-09-19T04:55:50.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.509+0000 7f70db5fd740 1 -- 172.21.15.123:0/3605059607 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70c00043a0 con 0x56351edc4e40 2024-09-19T04:55:50.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.509+0000 7f70b3e00640 1 -- 172.21.15.123:0/3605059607 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f70c40bc110 con 0x56351edc4e40 2024-09-19T04:55:50.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.509+0000 7f70db5fd740 10 client.17217 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:50.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.509+0000 7f70db5fd740 1 -- 172.21.15.123:0/3605059607 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56351ee4aec0 con 0x56351edc4e40 2024-09-19T04:55:50.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.509+0000 7f70b3e00640 1 -- 172.21.15.123:0/3605059607 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c00043a0 con 0x56351edc4e40 2024-09-19T04:55:50.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.509+0000 7f70b3e00640 1 -- 172.21.15.123:0/3605059607 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f70c40b8500 con 0x56351edc4e40 2024-09-19T04:55:50.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.509+0000 7f70db5fd740 10 client.17217 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:50.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.509+0000 7f70db5fd740 10 client.17217 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:55:50.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.509+0000 7f70db5fd740 20 client.17217 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:50.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.510+0000 7f70db5fd740 1 --2- 172.21.15.123:0/3605059607 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x56351ee4b450 0x56351ee4d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:50.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.510+0000 7f70db5fd740 4 client.17217 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:55:50.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.510+0000 7f70db5fd740 1 -- 172.21.15.123:0/3605059607 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56351ecaf010 con 0x56351ee4b450 2024-09-19T04:55:50.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.510+0000 7f70d1400640 1 --2- 172.21.15.123:0/3605059607 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x56351ee4b450 0x56351ee4d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:50.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.510+0000 7f70d1400640 1 --2- 172.21.15.123:0/3605059607 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x56351ee4b450 0x56351ee4d840 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cc5891ede44a5ae5 server_cookie=b727010566cee526 in_seq=0 out_seq=0 2024-09-19T04:55:50.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.510+0000 7f70b3e00640 10 client.17217 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:55:50.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.511+0000 7f70b3e00640 1 -- 172.21.15.123:0/3605059607 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56351ecaf010 con 0x56351ee4b450 2024-09-19T04:55:50.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.511+0000 7f70b3e00640 10 client.17217 handle_command_reply: tid=0 2024-09-19T04:55:50.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.511+0000 7f70db5fd740 1 client.17217 shutdown 2024-09-19T04:55:50.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.511+0000 7f70db5fd740 1 -- 172.21.15.123:0/3605059607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edc4e40 msgr2=0x56351edc9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:50.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.511+0000 7f70db5fd740 1 --2- 172.21.15.123:0/3605059607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edc4e40 0x56351edc9de0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f70c40b8c10 tx=0x7f70c403b790 comp rx=0 tx=0).stop 2024-09-19T04:55:50.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.511+0000 7f70db5fd740 1 -- 172.21.15.123:0/3605059607 shutdown_connections 2024-09-19T04:55:50.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.511+0000 7f70db5fd740 1 --2- 172.21.15.123:0/3605059607 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x56351ee4b450 0x56351ee4d840 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.511+0000 7f70db5fd740 1 --2- 172.21.15.123:0/3605059607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56351edd0020 0x56351edca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.511+0000 7f70db5fd740 1 --2- 172.21.15.123:0/3605059607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56351edc5760 0x56351edca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.511+0000 7f70db5fd740 1 --2- 172.21.15.123:0/3605059607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56351edc4e40 0x56351edc9de0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.511+0000 7f70db5fd740 1 -- 172.21.15.123:0/3605059607 >> 172.21.15.123:0/3605059607 conn(0x56351edc13d0 msgr2=0x56351eddae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:50.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.511+0000 7f70b3e00640 0 client.17217 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:55:50.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.511+0000 7f70db5fd740 1 -- 172.21.15.123:0/3605059607 shutdown_connections 2024-09-19T04:55:50.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.511+0000 7f70db5fd740 1 -- 172.21.15.123:0/3605059607 wait complete. 2024-09-19T04:55:50.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.511+0000 7f70db5fd740 20 client.17217 trim_cache size 0 max 16384 2024-09-19T04:55:50.519 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:55:50.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.512+0000 7f70d3200640 1 -- 172.21.15.123:0/3322220285 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f70b80a33b0 msgr2=0x7f70b80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:50.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.512+0000 7f70d3200640 1 --2- 172.21.15.123:0/3322220285 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f70b80a33b0 0x7f70b80a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f70cc005f40 tx=0x7f70cc009600 comp rx=0 tx=0).stop 2024-09-19T04:55:50.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.512+0000 7f70d3200640 1 -- 172.21.15.123:0/3322220285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70d41019c0 msgr2=0x7f70d419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:50.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.512+0000 7f70d3200640 1 --2- 172.21.15.123:0/3322220285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70d41019c0 0x7f70d419a4f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f70c402a960 tx=0x7f70c4003b90 comp rx=0 tx=0).stop 2024-09-19T04:55:50.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.512+0000 7f70d3200640 1 -- 172.21.15.123:0/3322220285 shutdown_connections 2024-09-19T04:55:50.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.512+0000 7f70d3200640 1 --2- 172.21.15.123:0/3322220285 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f70b80a33b0 0x7f70b80a5870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.512+0000 7f70d3200640 1 --2- 172.21.15.123:0/3322220285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d41079c0 0x7f70d419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.512+0000 7f70d3200640 1 --2- 172.21.15.123:0/3322220285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d4102360 0x7f70d419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.512+0000 7f70d3200640 1 --2- 172.21.15.123:0/3322220285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70d41019c0 0x7f70d419a4f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:50.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.512+0000 7f70d3200640 1 -- 172.21.15.123:0/3322220285 >> 172.21.15.123:0/3322220285 conn(0x7f70d40fd740 msgr2=0x7f70d410d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:50.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.512+0000 7f70d3200640 1 -- 172.21.15.123:0/3322220285 shutdown_connections 2024-09-19T04:55:50.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:50.512+0000 7f70d3200640 1 -- 172.21.15.123:0/3322220285 wait complete. 2024-09-19T04:55:51.281 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:55:51.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:51 smithi193 ceph-mon[35592]: pgmap v283: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:51.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:51 smithi183 ceph-mon[35333]: pgmap v283: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:51.650 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:51.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:51 smithi123 ceph-mon[30511]: pgmap v283: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:52.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.177+0000 7f1c72800640 1 -- 172.21.15.123:0/4242472395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c1049a0 msgr2=0x7f1c6c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:52.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.177+0000 7f1c72800640 1 --2- 172.21.15.123:0/4242472395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c1049a0 0x7f1c6c1089f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009900 tx=0x7f1c5402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.177+0000 7f1c72800640 1 -- 172.21.15.123:0/4242472395 shutdown_connections 2024-09-19T04:55:52.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.177+0000 7f1c72800640 1 --2- 172.21.15.123:0/4242472395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c6c109120 0x7f1c6c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.177+0000 7f1c72800640 1 --2- 172.21.15.123:0/4242472395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c1049a0 0x7f1c6c1089f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.177+0000 7f1c72800640 1 --2- 172.21.15.123:0/4242472395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c6c103ff0 0x7f1c6c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.177+0000 7f1c72800640 1 -- 172.21.15.123:0/4242472395 >> 172.21.15.123:0/4242472395 conn(0x7f1c6c0ffa90 msgr2=0x7f1c6c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:52.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.177+0000 7f1c72800640 1 -- 172.21.15.123:0/4242472395 shutdown_connections 2024-09-19T04:55:52.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.177+0000 7f1c72800640 1 -- 172.21.15.123:0/4242472395 wait complete. 2024-09-19T04:55:52.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.178+0000 7f1c72800640 1 Processor -- start 2024-09-19T04:55:52.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.178+0000 7f1c72800640 1 -- start start 2024-09-19T04:55:52.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.178+0000 7f1c72800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c6c103ff0 0x7f1c6c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.178+0000 7f1c72800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c6c1049a0 0x7f1c6c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.178+0000 7f1c72800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c109120 0x7f1c6c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.178+0000 7f1c72800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c114120 con 0x7f1c6c109120 2024-09-19T04:55:52.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.178+0000 7f1c72800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c113fa0 con 0x7f1c6c1049a0 2024-09-19T04:55:52.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.178+0000 7f1c72800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1142a0 con 0x7f1c6c103ff0 2024-09-19T04:55:52.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.179+0000 7f1c62a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c6c1049a0 0x7f1c6c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.179+0000 7f1c62a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c6c1049a0 0x7f1c6c19abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:59644/0 (socket says 172.21.15.123:59644) 2024-09-19T04:55:52.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.179+0000 7f1c6be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c109120 0x7f1c6c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.179+0000 7f1c62a00640 1 -- 172.21.15.123:0/3994846596 learned_addr learned my addr 172.21.15.123:0/3994846596 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:52.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.179+0000 7f1c6b400640 1 --2- 172.21.15.123:0/3994846596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c6c103ff0 0x7f1c6c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.179+0000 7f1c62a00640 1 -- 172.21.15.123:0/3994846596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c6c103ff0 msgr2=0x7f1c6c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:52.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.179+0000 7f1c62a00640 1 --2- 172.21.15.123:0/3994846596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c6c103ff0 0x7f1c6c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.179+0000 7f1c62a00640 1 -- 172.21.15.123:0/3994846596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c109120 msgr2=0x7f1c6c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:52.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.179+0000 7f1c62a00640 1 --2- 172.21.15.123:0/3994846596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c109120 0x7f1c6c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.179+0000 7f1c62a00640 1 -- 172.21.15.123:0/3994846596 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c19f650 con 0x7f1c6c1049a0 2024-09-19T04:55:52.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.179+0000 7f1c6be00640 1 --2- 172.21.15.123:0/3994846596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c109120 0x7f1c6c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:52.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.179+0000 7f1c62a00640 1 --2- 172.21.15.123:0/3994846596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c6c1049a0 0x7f1c6c19abe0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1c54006010 tx=0x7f1c54004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:52.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.180+0000 7f1c68c00640 1 -- 172.21.15.123:0/3994846596 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5403f030 con 0x7f1c6c1049a0 2024-09-19T04:55:52.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.180+0000 7f1c72800640 1 -- 172.21.15.123:0/3994846596 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6c19f8e0 con 0x7f1c6c1049a0 2024-09-19T04:55:52.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.180+0000 7f1c68c00640 1 -- 172.21.15.123:0/3994846596 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c54004440 con 0x7f1c6c1049a0 2024-09-19T04:55:52.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.180+0000 7f1c68c00640 1 -- 172.21.15.123:0/3994846596 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54035070 con 0x7f1c6c1049a0 2024-09-19T04:55:52.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.180+0000 7f1c72800640 1 -- 172.21.15.123:0/3994846596 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c6c1a7220 con 0x7f1c6c1049a0 2024-09-19T04:55:52.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.183+0000 7f1c7523a740 1 Processor -- start 2024-09-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.183+0000 7f1c7523a740 1 -- start start 2024-09-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.183+0000 7f1c68c00640 1 -- 172.21.15.123:0/3994846596 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f1c54042030 con 0x7f1c6c1049a0 2024-09-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.183+0000 7f1c7523a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26e3c70 0x55fcf26e4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.183+0000 7f1c7523a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e4590 0x55fcf26ee910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.183+0000 7f1c7523a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26eee50 0x55fcf26f1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.183+0000 7f1c7523a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fcf264a3e0 con 0x55fcf26e3c70 2024-09-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.183+0000 7f1c7523a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fcf25f1d80 con 0x55fcf26e4590 2024-09-19T04:55:52.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.183+0000 7f1c7523a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fcf26463a0 con 0x55fcf26eee50 2024-09-19T04:55:52.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.184+0000 7f1c6b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e4590 0x55fcf26ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.184+0000 7f1c6b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e4590 0x55fcf26ee910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:59648/0 (socket says 172.21.15.123:59648) 2024-09-19T04:55:52.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.184+0000 7f1c6b400640 1 -- 172.21.15.123:0/1724894153 learned_addr learned my addr 172.21.15.123:0/1724894153 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:52.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.184+0000 7f1c6be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26e3c70 0x55fcf26e4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.184+0000 7f1c62a00640 1 --2- 172.21.15.123:0/1724894153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26eee50 0x55fcf26f1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.184+0000 7f1c6b400640 1 -- 172.21.15.123:0/1724894153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26eee50 msgr2=0x55fcf26f1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:52.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.184+0000 7f1c6b400640 1 --2- 172.21.15.123:0/1724894153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26eee50 0x55fcf26f1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.184+0000 7f1c6b400640 1 -- 172.21.15.123:0/1724894153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26e3c70 msgr2=0x55fcf26e4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:52.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.184+0000 7f1c6b400640 1 --2- 172.21.15.123:0/1724894153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26e3c70 0x55fcf26e4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.184+0000 7f1c6b400640 1 -- 172.21.15.123:0/1724894153 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcf2610ae0 con 0x55fcf26e4590 2024-09-19T04:55:52.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.184+0000 7f1c6be00640 1 --2- 172.21.15.123:0/1724894153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26e3c70 0x55fcf26e4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:52.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.185+0000 7f1c6b400640 1 --2- 172.21.15.123:0/1724894153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e4590 0x55fcf26ee910 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009cf0 tx=0x7f1c5c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1f252aa505b9672 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:52.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.185+0000 7f1c60c00640 1 -- 172.21.15.123:0/1724894153 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c004230 con 0x55fcf26e4590 2024-09-19T04:55:52.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.185+0000 7f1c60c00640 1 -- 172.21.15.123:0/1724894153 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c5c0043d0 con 0x55fcf26e4590 2024-09-19T04:55:52.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.185+0000 7f1c60c00640 1 -- 172.21.15.123:0/1724894153 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c019960 con 0x55fcf26e4590 2024-09-19T04:55:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.185+0000 7f1c68c00640 1 --2- 172.21.15.123:0/3994846596 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1c500a33b0 0x7f1c500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.185+0000 7f1c68c00640 1 -- 172.21.15.123:0/3994846596 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f1c540f00c0 con 0x7f1c6c1049a0 2024-09-19T04:55:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.185+0000 7f1c6be00640 1 --2- 172.21.15.123:0/3994846596 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1c500a33b0 0x7f1c500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.186+0000 7f1c7523a740 1 -- 172.21.15.123:0/1724894153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e4590 msgr2=0x55fcf26ee910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.186+0000 7f1c7523a740 1 --2- 172.21.15.123:0/1724894153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e4590 0x55fcf26ee910 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009cf0 tx=0x7f1c5c011810 comp rx=0 tx=0).stop 2024-09-19T04:55:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.186+0000 7f1c7523a740 1 -- 172.21.15.123:0/1724894153 shutdown_connections 2024-09-19T04:55:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.186+0000 7f1c7523a740 1 --2- 172.21.15.123:0/1724894153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26eee50 0x55fcf26f1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.186+0000 7f1c7523a740 1 --2- 172.21.15.123:0/1724894153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e4590 0x55fcf26ee910 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.186+0000 7f1c7523a740 1 --2- 172.21.15.123:0/1724894153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26e3c70 0x55fcf26e4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.186+0000 7f1c7523a740 1 -- 172.21.15.123:0/1724894153 >> 172.21.15.123:0/1724894153 conn(0x55fcf26e0200 msgr2=0x55fcf26e05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.186+0000 7f1c7523a740 1 -- 172.21.15.123:0/1724894153 shutdown_connections 2024-09-19T04:55:52.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.186+0000 7f1c7523a740 1 -- 172.21.15.123:0/1724894153 wait complete. 2024-09-19T04:55:52.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.186+0000 7f1c6be00640 1 --2- 172.21.15.123:0/3994846596 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1c500a33b0 0x7f1c500a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009fd0 tx=0x7f1c64009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:52.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.186+0000 7f1c7523a740 1 -- start start 2024-09-19T04:55:52.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.187+0000 7f1c7523a740 1 Processor -- start 2024-09-19T04:55:52.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.187+0000 7f1c7523a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e3c70 0x55fcf26ebcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.187+0000 7f1c7523a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26e4590 0x55fcf26ec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.187+0000 7f1c7523a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26eee50 0x55fcf26ec730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.187+0000 7f1c7523a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fcf26f1a20 con 0x55fcf26eee50 2024-09-19T04:55:52.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.187+0000 7f1c7523a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fcf26050f0 con 0x55fcf26e3c70 2024-09-19T04:55:52.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.187+0000 7f1c7523a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fcf26f1ba0 con 0x55fcf26e4590 2024-09-19T04:55:52.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.187+0000 7f1c7523a740 10 client.0 _ll_register_callbacks cb 0x55fcf25ecbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:52.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.188+0000 7f1c6be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26eee50 0x55fcf26ec730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.188+0000 7f1c6b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e3c70 0x55fcf26ebcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.188+0000 7f1c6be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26eee50 0x55fcf26ec730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54592/0 (socket says 172.21.15.123:54592) 2024-09-19T04:55:52.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.188+0000 7f1c6be00640 1 -- 172.21.15.123:0/2132113847 learned_addr learned my addr 172.21.15.123:0/2132113847 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:52.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.188+0000 7f1c62a00640 1 --2- 172.21.15.123:0/2132113847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26e4590 0x55fcf26ec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.188+0000 7f1c6b400640 1 -- 172.21.15.123:0/2132113847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26e4590 msgr2=0x55fcf26ec1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:52.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.188+0000 7f1c6b400640 1 --2- 172.21.15.123:0/2132113847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26e4590 0x55fcf26ec1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.188+0000 7f1c6b400640 1 -- 172.21.15.123:0/2132113847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26eee50 msgr2=0x55fcf26ec730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:52.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.188+0000 7f1c6b400640 1 --2- 172.21.15.123:0/2132113847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26eee50 0x55fcf26ec730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.188+0000 7f1c6b400640 1 -- 172.21.15.123:0/2132113847 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcf23bb3c0 con 0x55fcf26e3c70 2024-09-19T04:55:52.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.188+0000 7f1c6be00640 1 --2- 172.21.15.123:0/2132113847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26eee50 0x55fcf26ec730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:52.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.188+0000 7f1c6b400640 1 --2- 172.21.15.123:0/2132113847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e3c70 0x55fcf26ebcb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009cf0 tx=0x7f1c5c0096f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b42722c21a38d0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:52.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.188+0000 7f1c59600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:55:52.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.189+0000 7f1c59600640 1 -- 172.21.15.123:0/2132113847 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c009ea0 con 0x55fcf26e3c70 2024-09-19T04:55:52.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.189+0000 7f1c7523a740 1 -- 172.21.15.123:0/2132113847 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c5c0043d0 con 0x55fcf26e3c70 2024-09-19T04:55:52.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.189+0000 7f1c59600640 1 -- 172.21.15.123:0/2132113847 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f1c5c005240 con 0x55fcf26e3c70 2024-09-19T04:55:52.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.189+0000 7f1c59600640 1 -- 172.21.15.123:0/2132113847 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1c5c032050 con 0x55fcf26e3c70 2024-09-19T04:55:52.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.189+0000 7f1c7523a740 10 client.27320 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:52.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.189+0000 7f1c7523a740 1 -- 172.21.15.123:0/2132113847 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fcf2706220 con 0x55fcf26e3c70 2024-09-19T04:55:52.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.189+0000 7f1c59600640 1 -- 172.21.15.123:0/2132113847 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c5c0043d0 con 0x55fcf26e3c70 2024-09-19T04:55:52.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.190+0000 7f1c59600640 1 -- 172.21.15.123:0/2132113847 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f1c5c004ba0 con 0x55fcf26e3c70 2024-09-19T04:55:52.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.190+0000 7f1c7523a740 10 client.27320 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:52.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.190+0000 7f1c7523a740 10 client.27320 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:55:52.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.190+0000 7f1c7523a740 20 client.27320 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:52.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.190+0000 7f1c7523a740 1 --2- 172.21.15.123:0/2132113847 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55fcf2707620 0x55fcf2709a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.190+0000 7f1c7523a740 4 client.27320 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:55:52.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.190+0000 7f1c7523a740 1 -- 172.21.15.123:0/2132113847 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcf2657950 con 0x55fcf2707620 2024-09-19T04:55:52.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.190+0000 7f1c62a00640 1 --2- 172.21.15.123:0/2132113847 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55fcf2707620 0x55fcf2709a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.191+0000 7f1c62a00640 1 --2- 172.21.15.123:0/2132113847 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55fcf2707620 0x55fcf2709a10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7de08a1849c9119 server_cookie=1c1cdcdc655d0bba in_seq=0 out_seq=0 2024-09-19T04:55:52.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.191+0000 7f1c59600640 10 client.27320 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:55:52.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.191+0000 7f1c59600640 1 -- 172.21.15.123:0/2132113847 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fcf2657950 con 0x55fcf2707620 2024-09-19T04:55:52.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.191+0000 7f1c59600640 10 client.27320 handle_command_reply: tid=0 2024-09-19T04:55:52.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.192+0000 7f1c7523a740 1 client.27320 shutdown 2024-09-19T04:55:52.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.192+0000 7f1c7523a740 1 -- 172.21.15.123:0/2132113847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e3c70 msgr2=0x55fcf26ebcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:52.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.192+0000 7f1c7523a740 1 --2- 172.21.15.123:0/2132113847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e3c70 0x55fcf26ebcb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009cf0 tx=0x7f1c5c0096f0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.192+0000 7f1c7523a740 1 -- 172.21.15.123:0/2132113847 shutdown_connections 2024-09-19T04:55:52.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.192+0000 7f1c7523a740 1 --2- 172.21.15.123:0/2132113847 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55fcf2707620 0x55fcf2709a10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.192+0000 7f1c7523a740 1 --2- 172.21.15.123:0/2132113847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26eee50 0x55fcf26ec730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.192+0000 7f1c7523a740 1 --2- 172.21.15.123:0/2132113847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26e4590 0x55fcf26ec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.192+0000 7f1c59600640 0 client.27320 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:55:52.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.192+0000 7f1c7523a740 1 --2- 172.21.15.123:0/2132113847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e3c70 0x55fcf26ebcb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.192+0000 7f1c7523a740 1 -- 172.21.15.123:0/2132113847 >> 172.21.15.123:0/2132113847 conn(0x55fcf26e0200 msgr2=0x55fcf26df380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:52.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.192+0000 7f1c7523a740 1 -- 172.21.15.123:0/2132113847 shutdown_connections 2024-09-19T04:55:52.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.192+0000 7f1c7523a740 1 -- 172.21.15.123:0/2132113847 wait complete. 2024-09-19T04:55:52.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.192+0000 7f1c7523a740 20 client.27320 trim_cache size 0 max 16384 2024-09-19T04:55:52.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.205+0000 7f1c7523a740 1 Processor -- start 2024-09-19T04:55:52.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.205+0000 7f1c7523a740 1 -- start start 2024-09-19T04:55:52.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.206+0000 7f1c7523a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26e3c70 0x55fcf270f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.206+0000 7f1c7523a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26e4590 0x55fcf270f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.206+0000 7f1c7523a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26eee50 0x55fcf26deb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.206+0000 7f1c7523a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1ab6d0 con 0x55fcf26e3c70 2024-09-19T04:55:52.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.206+0000 7f1c7523a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1ad210 con 0x55fcf26eee50 2024-09-19T04:55:52.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.206+0000 7f1c7523a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1abda0 con 0x55fcf26e4590 2024-09-19T04:55:52.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.206+0000 7f1c6b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26e3c70 0x55fcf270f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.206+0000 7f1c6be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26eee50 0x55fcf26deb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.206+0000 7f1c62a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26e4590 0x55fcf270f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.206+0000 7f1c6be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26eee50 0x55fcf26deb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:59664/0 (socket says 172.21.15.123:59664) 2024-09-19T04:55:52.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.206+0000 7f1c6be00640 1 -- 172.21.15.123:0/111907117 learned_addr learned my addr 172.21.15.123:0/111907117 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:52.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.206+0000 7f1c62a00640 1 -- 172.21.15.123:0/111907117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26eee50 msgr2=0x55fcf26deb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:52.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.206+0000 7f1c62a00640 1 --2- 172.21.15.123:0/111907117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26eee50 0x55fcf26deb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.206+0000 7f1c62a00640 1 -- 172.21.15.123:0/111907117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26e3c70 msgr2=0x55fcf270f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:52.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.206+0000 7f1c62a00640 1 --2- 172.21.15.123:0/111907117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26e3c70 0x55fcf270f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.207+0000 7f1c62a00640 1 -- 172.21.15.123:0/111907117 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcf26e0030 con 0x55fcf26e4590 2024-09-19T04:55:52.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.207+0000 7f1c62a00640 1 --2- 172.21.15.123:0/111907117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26e4590 0x55fcf270f720 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1c540ed7b0 tx=0x7f1c540f2040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1ffac5209192d99 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:52.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.207+0000 7f1c5be00640 1 -- 172.21.15.123:0/111907117 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54049040 con 0x55fcf26e4590 2024-09-19T04:55:52.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.207+0000 7f1c5be00640 1 -- 172.21.15.123:0/111907117 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c540bc3f0 con 0x55fcf26e4590 2024-09-19T04:55:52.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.207+0000 7f1c5be00640 1 -- 172.21.15.123:0/111907117 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54003cd0 con 0x55fcf26e4590 2024-09-19T04:55:52.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.208+0000 7f1c7523a740 1 -- 172.21.15.123:0/111907117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26e4590 msgr2=0x55fcf270f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:52.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.208+0000 7f1c7523a740 1 --2- 172.21.15.123:0/111907117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26e4590 0x55fcf270f720 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1c540ed7b0 tx=0x7f1c540f2040 comp rx=0 tx=0).stop 2024-09-19T04:55:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.208+0000 7f1c7523a740 1 -- 172.21.15.123:0/111907117 shutdown_connections 2024-09-19T04:55:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.208+0000 7f1c7523a740 1 --2- 172.21.15.123:0/111907117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26eee50 0x55fcf26deb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.208+0000 7f1c7523a740 1 --2- 172.21.15.123:0/111907117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26e4590 0x55fcf270f720 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.208+0000 7f1c7523a740 1 --2- 172.21.15.123:0/111907117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26e3c70 0x55fcf270f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.208+0000 7f1c7523a740 1 -- 172.21.15.123:0/111907117 >> 172.21.15.123:0/111907117 conn(0x55fcf26e0200 msgr2=0x55fcf26edd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.208+0000 7f1c7523a740 1 -- 172.21.15.123:0/111907117 shutdown_connections 2024-09-19T04:55:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.208+0000 7f1c7523a740 1 -- 172.21.15.123:0/111907117 wait complete. 2024-09-19T04:55:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.208+0000 7f1c7523a740 1 -- start start 2024-09-19T04:55:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.208+0000 7f1c7523a740 1 Processor -- start 2024-09-19T04:55:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.209+0000 7f1c7523a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26e3c70 0x55fcf26ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.209+0000 7f1c7523a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e4590 0x55fcf26fae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.209+0000 7f1c7523a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26eee50 0x55fcf26fb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.209+0000 7f1c7523a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1acb40 con 0x55fcf26eee50 2024-09-19T04:55:52.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.209+0000 7f1c7523a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1ac470 con 0x55fcf26e4590 2024-09-19T04:55:52.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.209+0000 7f1c7523a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1a96d0 con 0x55fcf26e3c70 2024-09-19T04:55:52.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.209+0000 7f1c7523a740 10 client.0 _ll_register_callbacks cb 0x7f1c64009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:52.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.209+0000 7f1c6b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26e3c70 0x55fcf26ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.209+0000 7f1c62a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e4590 0x55fcf26fae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.209+0000 7f1c6be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26eee50 0x55fcf26fb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.209+0000 7f1c62a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e4590 0x55fcf26fae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:59672/0 (socket says 172.21.15.123:59672) 2024-09-19T04:55:52.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.209+0000 7f1c62a00640 1 -- 172.21.15.123:0/947248506 learned_addr learned my addr 172.21.15.123:0/947248506 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:52.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.209+0000 7f1c6be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26eee50 0x55fcf26fb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54606/0 (socket says 172.21.15.123:54606) 2024-09-19T04:55:52.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.209+0000 7f1c6b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26e3c70 0x55fcf26ffad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:54400/0 (socket says 172.21.15.123:54400) 2024-09-19T04:55:52.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.210+0000 7f1c62a00640 1 -- 172.21.15.123:0/947248506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26e3c70 msgr2=0x55fcf26ffad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:52.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.210+0000 7f1c62a00640 1 --2- 172.21.15.123:0/947248506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26e3c70 0x55fcf26ffad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.210+0000 7f1c62a00640 1 -- 172.21.15.123:0/947248506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26eee50 msgr2=0x55fcf26fb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:52.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.210+0000 7f1c62a00640 1 --2- 172.21.15.123:0/947248506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26eee50 0x55fcf26fb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.210+0000 7f1c62a00640 1 -- 172.21.15.123:0/947248506 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcf26db770 con 0x55fcf26e4590 2024-09-19T04:55:52.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.210+0000 7f1c6be00640 1 --2- 172.21.15.123:0/947248506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26eee50 0x55fcf26fb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:52.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.210+0000 7f1c62a00640 1 --2- 172.21.15.123:0/947248506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e4590 0x55fcf26fae00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1c54005ab0 tx=0x7f1c540b8750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb1e98e5df194553 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:52.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.210+0000 7f1c59600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:55:52.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.213+0000 7f1c59600640 1 -- 172.21.15.123:0/947248506 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54049070 con 0x55fcf26e4590 2024-09-19T04:55:52.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.213+0000 7f1c59600640 1 -- 172.21.15.123:0/947248506 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f1c540b8a60 con 0x55fcf26e4590 2024-09-19T04:55:52.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.213+0000 7f1c7523a740 1 -- 172.21.15.123:0/947248506 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c540bc3f0 con 0x55fcf26e4590 2024-09-19T04:55:52.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.214+0000 7f1c59600640 1 -- 172.21.15.123:0/947248506 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1c540bbf30 con 0x55fcf26e4590 2024-09-19T04:55:52.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.214+0000 7f1c7523a740 10 client.27326 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:52.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.214+0000 7f1c7523a740 1 -- 172.21.15.123:0/947248506 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fcf2771cb0 con 0x55fcf26e4590 2024-09-19T04:55:52.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.214+0000 7f1c59600640 1 -- 172.21.15.123:0/947248506 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c540bc3f0 con 0x55fcf26e4590 2024-09-19T04:55:52.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.214+0000 7f1c59600640 1 -- 172.21.15.123:0/947248506 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f1c540c4b20 con 0x55fcf26e4590 2024-09-19T04:55:52.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.214+0000 7f1c7523a740 10 client.27326 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:52.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.214+0000 7f1c7523a740 10 client.27326 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:55:52.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.214+0000 7f1c7523a740 20 client.27326 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:52.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.214+0000 7f1c7523a740 1 --2- 172.21.15.123:0/947248506 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55fcf27721c0 0x55fcf27745e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:52.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.214+0000 7f1c7523a740 4 client.27326 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:55:52.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.214+0000 7f1c7523a740 1 -- 172.21.15.123:0/947248506 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fcf2651e60 con 0x55fcf27721c0 2024-09-19T04:55:52.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.214+0000 7f1c6b400640 1 --2- 172.21.15.123:0/947248506 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55fcf27721c0 0x55fcf27745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:52.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.215+0000 7f1c6b400640 1 --2- 172.21.15.123:0/947248506 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55fcf27721c0 0x55fcf27745e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b5b255a74764407c server_cookie=fa80834aac5927c5 in_seq=0 out_seq=0 2024-09-19T04:55:52.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.215+0000 7f1c59600640 10 client.27326 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:55:52.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.215+0000 7f1c59600640 1 -- 172.21.15.123:0/947248506 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fcf2651e60 con 0x55fcf27721c0 2024-09-19T04:55:52.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.215+0000 7f1c59600640 10 client.27326 handle_command_reply: tid=0 2024-09-19T04:55:52.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.215+0000 7f1c7523a740 1 client.27326 shutdown 2024-09-19T04:55:52.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.215+0000 7f1c7523a740 1 -- 172.21.15.123:0/947248506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e4590 msgr2=0x55fcf26fae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:52.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.215+0000 7f1c7523a740 1 --2- 172.21.15.123:0/947248506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e4590 0x55fcf26fae00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1c54005ab0 tx=0x7f1c540b8750 comp rx=0 tx=0).stop 2024-09-19T04:55:52.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.216+0000 7f1c7523a740 1 -- 172.21.15.123:0/947248506 shutdown_connections 2024-09-19T04:55:52.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.216+0000 7f1c7523a740 1 --2- 172.21.15.123:0/947248506 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55fcf27721c0 0x55fcf27745e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.216+0000 7f1c7523a740 1 --2- 172.21.15.123:0/947248506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcf26eee50 0x55fcf26fb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.216+0000 7f1c7523a740 1 --2- 172.21.15.123:0/947248506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcf26e4590 0x55fcf26fae00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.216+0000 7f1c7523a740 1 --2- 172.21.15.123:0/947248506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcf26e3c70 0x55fcf26ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.216+0000 7f1c59600640 0 client.27326 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:55:52.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.216+0000 7f1c7523a740 1 -- 172.21.15.123:0/947248506 >> 172.21.15.123:0/947248506 conn(0x55fcf26e0200 msgr2=0x55fcf26deb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:52.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.216+0000 7f1c7523a740 1 -- 172.21.15.123:0/947248506 shutdown_connections 2024-09-19T04:55:52.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.216+0000 7f1c7523a740 1 -- 172.21.15.123:0/947248506 wait complete. 2024-09-19T04:55:52.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.216+0000 7f1c7523a740 20 client.27326 trim_cache size 0 max 16384 2024-09-19T04:55:52.220 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:55:52.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.216+0000 7f1c72800640 1 -- 172.21.15.123:0/3994846596 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1c500a33b0 msgr2=0x7f1c500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:52.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.216+0000 7f1c72800640 1 --2- 172.21.15.123:0/3994846596 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1c500a33b0 0x7f1c500a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009fd0 tx=0x7f1c64009500 comp rx=0 tx=0).stop 2024-09-19T04:55:52.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.217+0000 7f1c72800640 1 -- 172.21.15.123:0/3994846596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c6c1049a0 msgr2=0x7f1c6c19abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:52.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.217+0000 7f1c72800640 1 --2- 172.21.15.123:0/3994846596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c6c1049a0 0x7f1c6c19abe0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1c54006010 tx=0x7f1c54004040 comp rx=0 tx=0).stop 2024-09-19T04:55:52.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.217+0000 7f1c72800640 1 -- 172.21.15.123:0/3994846596 shutdown_connections 2024-09-19T04:55:52.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.217+0000 7f1c72800640 1 --2- 172.21.15.123:0/3994846596 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f1c500a33b0 0x7f1c500a5870 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.217+0000 7f1c72800640 1 --2- 172.21.15.123:0/3994846596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c109120 0x7f1c6c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.217+0000 7f1c72800640 1 --2- 172.21.15.123:0/3994846596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c6c1049a0 0x7f1c6c19abe0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.217+0000 7f1c72800640 1 --2- 172.21.15.123:0/3994846596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c6c103ff0 0x7f1c6c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:52.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.217+0000 7f1c72800640 1 -- 172.21.15.123:0/3994846596 >> 172.21.15.123:0/3994846596 conn(0x7f1c6c0ffa90 msgr2=0x7f1c6c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:52.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.217+0000 7f1c72800640 1 -- 172.21.15.123:0/3994846596 shutdown_connections 2024-09-19T04:55:52.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:52.217+0000 7f1c72800640 1 -- 172.21.15.123:0/3994846596 wait complete. 2024-09-19T04:55:53.092 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:55:53.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:53 smithi123 ceph-mon[30511]: pgmap v284: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:53.459 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:53.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:53 smithi193 ceph-mon[35592]: pgmap v284: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:53.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:53 smithi183 ceph-mon[35333]: pgmap v284: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:55:54.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.063+0000 7f4696400640 1 -- 172.21.15.123:0/702134285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46901049a0 msgr2=0x7f46901089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:54.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.063+0000 7f4696400640 1 --2- 172.21.15.123:0/702134285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46901049a0 0x7f46901089f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f467c009900 tx=0x7f467c02a410 comp rx=0 tx=0).stop 2024-09-19T04:55:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.064+0000 7f4696400640 1 -- 172.21.15.123:0/702134285 shutdown_connections 2024-09-19T04:55:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.064+0000 7f4696400640 1 --2- 172.21.15.123:0/702134285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4690109120 0x7f4690110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.064+0000 7f4696400640 1 --2- 172.21.15.123:0/702134285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46901049a0 0x7f46901089f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.064+0000 7f4696400640 1 --2- 172.21.15.123:0/702134285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4690103ff0 0x7f46901043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.064+0000 7f4696400640 1 -- 172.21.15.123:0/702134285 >> 172.21.15.123:0/702134285 conn(0x7f46900ffa90 msgr2=0x7f4690101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.064+0000 7f4696400640 1 -- 172.21.15.123:0/702134285 shutdown_connections 2024-09-19T04:55:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.064+0000 7f4696400640 1 -- 172.21.15.123:0/702134285 wait complete. 2024-09-19T04:55:54.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.065+0000 7f4696400640 1 Processor -- start 2024-09-19T04:55:54.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.065+0000 7f4696400640 1 -- start start 2024-09-19T04:55:54.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.065+0000 7f4696400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4690103ff0 0x7f469019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.066+0000 7f4696400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46901049a0 0x7f469019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.066+0000 7f4696400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4690109120 0x7f469019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.066+0000 7f4696400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4690114190 con 0x7f4690103ff0 2024-09-19T04:55:54.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.066+0000 7f4696400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4690114010 con 0x7f46901049a0 2024-09-19T04:55:54.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.066+0000 7f4696400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4690114310 con 0x7f4690109120 2024-09-19T04:55:54.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.066+0000 7f468f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4690103ff0 0x7f469019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.066+0000 7f468f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4690103ff0 0x7f469019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54624/0 (socket says 172.21.15.123:54624) 2024-09-19T04:55:54.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.066+0000 7f468f400640 1 -- 172.21.15.123:0/3953853242 learned_addr learned my addr 172.21.15.123:0/3953853242 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:54.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.066+0000 7f468fe00640 1 --2- 172.21.15.123:0/3953853242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4690109120 0x7f469019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.066+0000 7f468ea00640 1 --2- 172.21.15.123:0/3953853242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46901049a0 0x7f469019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.066+0000 7f468f400640 1 -- 172.21.15.123:0/3953853242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4690109120 msgr2=0x7f469019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.066+0000 7f468f400640 1 --2- 172.21.15.123:0/3953853242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4690109120 0x7f469019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.066+0000 7f468f400640 1 -- 172.21.15.123:0/3953853242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46901049a0 msgr2=0x7f469019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.066+0000 7f468f400640 1 --2- 172.21.15.123:0/3953853242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46901049a0 0x7f469019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.066+0000 7f468f400640 1 -- 172.21.15.123:0/3953853242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469019f5e0 con 0x7f4690103ff0 2024-09-19T04:55:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.067+0000 7f468f400640 1 --2- 172.21.15.123:0/3953853242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4690103ff0 0x7f469019a630 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f468000d770 tx=0x7f468000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.067+0000 7f4687e00640 1 -- 172.21.15.123:0/3953853242 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680017070 con 0x7f4690103ff0 2024-09-19T04:55:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.067+0000 7f4687e00640 1 -- 172.21.15.123:0/3953853242 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4680005500 con 0x7f4690103ff0 2024-09-19T04:55:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.067+0000 7f4687e00640 1 -- 172.21.15.123:0/3953853242 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680004e00 con 0x7f4690103ff0 2024-09-19T04:55:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.067+0000 7f4696400640 1 -- 172.21.15.123:0/3953853242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469019f8d0 con 0x7f4690103ff0 2024-09-19T04:55:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.067+0000 7f4696400640 1 -- 172.21.15.123:0/3953853242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46901a71b0 con 0x7f4690103ff0 2024-09-19T04:55:54.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.070+0000 7f4687e00640 1 -- 172.21.15.123:0/3953853242 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4680004fa0 con 0x7f4690103ff0 2024-09-19T04:55:54.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.071+0000 7f4698e1c740 1 Processor -- start 2024-09-19T04:55:54.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.071+0000 7f4698e1c740 1 -- start start 2024-09-19T04:55:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.071+0000 7f4698e1c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c8fc70 0x55dba7c90050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.071+0000 7f4698e1c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c90590 0x55dba7c9a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.071+0000 7f4698e1c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c9ae50 0x55dba7c9d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.071+0000 7f4698e1c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dba7bf63e0 con 0x55dba7c8fc70 2024-09-19T04:55:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.071+0000 7f4698e1c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dba7b9dd80 con 0x55dba7c9ae50 2024-09-19T04:55:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.071+0000 7f4698e1c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dba7bf23a0 con 0x55dba7c90590 2024-09-19T04:55:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.072+0000 7f468ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c9ae50 0x55dba7c9d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.072+0000 7f468fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c8fc70 0x55dba7c90050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.072+0000 7f468f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c90590 0x55dba7c9a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.072+0000 7f468ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c9ae50 0x55dba7c9d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:59702/0 (socket says 172.21.15.123:59702) 2024-09-19T04:55:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.072+0000 7f468ea00640 1 -- 172.21.15.123:0/3146807619 learned_addr learned my addr 172.21.15.123:0/3146807619 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.072+0000 7f468fe00640 1 -- 172.21.15.123:0/3146807619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c90590 msgr2=0x55dba7c9a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.072+0000 7f468fe00640 1 --2- 172.21.15.123:0/3146807619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c90590 0x55dba7c9a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.072+0000 7f468fe00640 1 -- 172.21.15.123:0/3146807619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c9ae50 msgr2=0x55dba7c9d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.072+0000 7f468fe00640 1 --2- 172.21.15.123:0/3146807619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c9ae50 0x55dba7c9d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.072+0000 7f468fe00640 1 -- 172.21.15.123:0/3146807619 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dba7bbcae0 con 0x55dba7c8fc70 2024-09-19T04:55:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.072+0000 7f4687e00640 1 --2- 172.21.15.123:0/3953853242 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46740a33b0 0x7f46740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.072+0000 7f468fe00640 1 --2- 172.21.15.123:0/3146807619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c8fc70 0x55dba7c90050 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4688009eb0 tx=0x7f46880107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8ece0bd4b91902 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.073+0000 7f4687e00640 1 -- 172.21.15.123:0/3953853242 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f46800cf750 con 0x7f4690103ff0 2024-09-19T04:55:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.073+0000 7f468f400640 1 --2- 172.21.15.123:0/3953853242 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46740a33b0 0x7f46740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.073+0000 7f4684c00640 1 -- 172.21.15.123:0/3146807619 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688003b70 con 0x55dba7c8fc70 2024-09-19T04:55:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.073+0000 7f4684c00640 1 -- 172.21.15.123:0/3146807619 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4688015070 con 0x55dba7c8fc70 2024-09-19T04:55:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.073+0000 7f4684c00640 1 -- 172.21.15.123:0/3146807619 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468801a940 con 0x55dba7c8fc70 2024-09-19T04:55:54.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.074+0000 7f4698e1c740 1 -- 172.21.15.123:0/3146807619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c8fc70 msgr2=0x55dba7c90050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:54.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.074+0000 7f468f400640 1 --2- 172.21.15.123:0/3953853242 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46740a33b0 0x7f46740a5870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55dba7c8d060 tx=0x7f46800d5530 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:54.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.074+0000 7f4698e1c740 1 --2- 172.21.15.123:0/3146807619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c8fc70 0x55dba7c90050 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4688009eb0 tx=0x7f46880107a0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.074+0000 7f4698e1c740 1 -- 172.21.15.123:0/3146807619 shutdown_connections 2024-09-19T04:55:54.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.074+0000 7f4698e1c740 1 --2- 172.21.15.123:0/3146807619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c9ae50 0x55dba7c9d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.074+0000 7f4698e1c740 1 --2- 172.21.15.123:0/3146807619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c90590 0x55dba7c9a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.074+0000 7f4698e1c740 1 --2- 172.21.15.123:0/3146807619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c8fc70 0x55dba7c90050 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.074+0000 7f4698e1c740 1 -- 172.21.15.123:0/3146807619 >> 172.21.15.123:0/3146807619 conn(0x55dba7c8c200 msgr2=0x55dba7c8c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.074+0000 7f4698e1c740 1 -- 172.21.15.123:0/3146807619 shutdown_connections 2024-09-19T04:55:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.074+0000 7f4698e1c740 1 -- 172.21.15.123:0/3146807619 wait complete. 2024-09-19T04:55:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.074+0000 7f4698e1c740 1 -- start start 2024-09-19T04:55:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.075+0000 7f4698e1c740 1 Processor -- start 2024-09-19T04:55:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.075+0000 7f4698e1c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c8fc70 0x55dba7c97d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.075+0000 7f4698e1c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c90590 0x55dba7c982d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.075+0000 7f4698e1c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c9ae50 0x55dba7c98810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.075+0000 7f4698e1c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dba7c9da20 con 0x55dba7c9ae50 2024-09-19T04:55:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.075+0000 7f4698e1c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dba7bb10f0 con 0x55dba7c8fc70 2024-09-19T04:55:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.075+0000 7f4698e1c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dba7c9dba0 con 0x55dba7c90590 2024-09-19T04:55:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.075+0000 7f4698e1c740 10 client.0 _ll_register_callbacks cb 0x55dba7b98bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.076+0000 7f468ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c90590 0x55dba7c982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.076+0000 7f468fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c8fc70 0x55dba7c97d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.076+0000 7f468ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c90590 0x55dba7c982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:54428/0 (socket says 172.21.15.123:54428) 2024-09-19T04:55:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.076+0000 7f468ea00640 1 -- 172.21.15.123:0/658522495 learned_addr learned my addr 172.21.15.123:0/658522495 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.076+0000 7f468fe00640 1 --2- 172.21.15.123:0/658522495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c9ae50 0x55dba7c98810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.076+0000 7f468ea00640 1 -- 172.21.15.123:0/658522495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c8fc70 msgr2=0x55dba7c97d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.076+0000 7f468ea00640 1 --2- 172.21.15.123:0/658522495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c8fc70 0x55dba7c97d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.076+0000 7f468ea00640 1 -- 172.21.15.123:0/658522495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c9ae50 msgr2=0x55dba7c98810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.076+0000 7f468ea00640 1 --2- 172.21.15.123:0/658522495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c9ae50 0x55dba7c98810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.076+0000 7f468ea00640 1 -- 172.21.15.123:0/658522495 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dba79673c0 con 0x55dba7c90590 2024-09-19T04:55:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.076+0000 7f468fe00640 1 --2- 172.21.15.123:0/658522495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c9ae50 0x55dba7c98810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:55:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.076+0000 7f468ea00640 1 --2- 172.21.15.123:0/658522495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c90590 0x55dba7c982d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f467c02aa00 tx=0x7f467c004460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15c80b2d298bdba6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.076+0000 7f4679600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:55:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.077+0000 7f4679600640 1 -- 172.21.15.123:0/658522495 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c035040 con 0x55dba7c90590 2024-09-19T04:55:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.077+0000 7f4679600640 1 -- 172.21.15.123:0/658522495 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f467c005510 con 0x55dba7c90590 2024-09-19T04:55:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.077+0000 7f4698e1c740 1 -- 172.21.15.123:0/658522495 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4688015070 con 0x55dba7c90590 2024-09-19T04:55:54.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.077+0000 7f4679600640 1 -- 172.21.15.123:0/658522495 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f467c0046c0 con 0x55dba7c90590 2024-09-19T04:55:54.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.077+0000 7f4698e1c740 10 client.27355 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:54.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.077+0000 7f4698e1c740 1 -- 172.21.15.123:0/658522495 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55dba7ca9ab0 con 0x55dba7c90590 2024-09-19T04:55:54.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.077+0000 7f4679600640 1 -- 172.21.15.123:0/658522495 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4688015070 con 0x55dba7c90590 2024-09-19T04:55:54.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.077+0000 7f4679600640 1 -- 172.21.15.123:0/658522495 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f467c031b20 con 0x55dba7c90590 2024-09-19T04:55:54.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.077+0000 7f4698e1c740 10 client.27355 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:54.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.077+0000 7f4698e1c740 10 client.27355 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:55:54.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.078+0000 7f4698e1c740 20 client.27355 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:54.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.078+0000 7f4698e1c740 1 --2- 172.21.15.123:0/658522495 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55dba7caaec0 0x55dba7cad300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.078+0000 7f4698e1c740 4 client.27355 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:55:54.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.078+0000 7f4698e1c740 1 -- 172.21.15.123:0/658522495 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dba7c03950 con 0x55dba7caaec0 2024-09-19T04:55:54.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.078+0000 7f468fe00640 1 --2- 172.21.15.123:0/658522495 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55dba7caaec0 0x55dba7cad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.078+0000 7f468fe00640 1 --2- 172.21.15.123:0/658522495 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55dba7caaec0 0x55dba7cad300 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dba118e1ec50491a server_cookie=b9a3edb28c07d8b5 in_seq=0 out_seq=0 2024-09-19T04:55:54.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.078+0000 7f4679600640 10 client.27355 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:55:54.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.079+0000 7f4679600640 1 -- 172.21.15.123:0/658522495 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dba7c03950 con 0x55dba7caaec0 2024-09-19T04:55:54.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.079+0000 7f4679600640 10 client.27355 handle_command_reply: tid=0 2024-09-19T04:55:54.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.079+0000 7f4698e1c740 1 client.27355 shutdown 2024-09-19T04:55:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.079+0000 7f4698e1c740 1 -- 172.21.15.123:0/658522495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c90590 msgr2=0x55dba7c982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.079+0000 7f4698e1c740 1 --2- 172.21.15.123:0/658522495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c90590 0x55dba7c982d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f467c02aa00 tx=0x7f467c004460 comp rx=0 tx=0).stop 2024-09-19T04:55:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.079+0000 7f4698e1c740 1 -- 172.21.15.123:0/658522495 shutdown_connections 2024-09-19T04:55:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.079+0000 7f4698e1c740 1 --2- 172.21.15.123:0/658522495 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55dba7caaec0 0x55dba7cad300 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.079+0000 7f4698e1c740 1 --2- 172.21.15.123:0/658522495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c9ae50 0x55dba7c98810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.079+0000 7f4698e1c740 1 --2- 172.21.15.123:0/658522495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c90590 0x55dba7c982d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.079+0000 7f4698e1c740 1 --2- 172.21.15.123:0/658522495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c8fc70 0x55dba7c97d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.079+0000 7f4679600640 0 client.27355 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:55:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.079+0000 7f4698e1c740 1 -- 172.21.15.123:0/658522495 >> 172.21.15.123:0/658522495 conn(0x55dba7c8c200 msgr2=0x55dba7c8b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.080+0000 7f4698e1c740 1 -- 172.21.15.123:0/658522495 shutdown_connections 2024-09-19T04:55:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.080+0000 7f4698e1c740 1 -- 172.21.15.123:0/658522495 wait complete. 2024-09-19T04:55:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.080+0000 7f4698e1c740 20 client.27355 trim_cache size 0 max 16384 2024-09-19T04:55:54.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.093+0000 7f4698e1c740 1 Processor -- start 2024-09-19T04:55:54.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.093+0000 7f4698e1c740 1 -- start start 2024-09-19T04:55:54.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.094+0000 7f4698e1c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c8fc70 0x55dba7cb2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.094+0000 7f4698e1c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c90590 0x55dba7cb3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.094+0000 7f4698e1c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c9ae50 0x55dba7c94ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.094+0000 7f4698e1c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46901afa30 con 0x55dba7c90590 2024-09-19T04:55:54.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.094+0000 7f4698e1c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46901b1570 con 0x55dba7c9ae50 2024-09-19T04:55:54.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.094+0000 7f4698e1c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f46901b0100 con 0x55dba7c8fc70 2024-09-19T04:55:54.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.094+0000 7f468ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c90590 0x55dba7cb3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.094+0000 7f468fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c8fc70 0x55dba7cb2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.094+0000 7f468ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c90590 0x55dba7cb3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54650/0 (socket says 172.21.15.123:54650) 2024-09-19T04:55:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.094+0000 7f468ea00640 1 -- 172.21.15.123:0/3164360592 learned_addr learned my addr 172.21.15.123:0/3164360592 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.094+0000 7f468fe00640 1 --2- 172.21.15.123:0/3164360592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c9ae50 0x55dba7c94ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.095+0000 7f468ea00640 1 -- 172.21.15.123:0/3164360592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c8fc70 msgr2=0x55dba7cb2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.095+0000 7f468ea00640 1 --2- 172.21.15.123:0/3164360592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c8fc70 0x55dba7cb2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.095+0000 7f468ea00640 1 -- 172.21.15.123:0/3164360592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c9ae50 msgr2=0x55dba7c94ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.095+0000 7f468ea00640 1 --2- 172.21.15.123:0/3164360592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c9ae50 0x55dba7c94ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.095+0000 7f468ea00640 1 -- 172.21.15.123:0/3164360592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dba7c8ce90 con 0x55dba7c90590 2024-09-19T04:55:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.095+0000 7f468ea00640 1 --2- 172.21.15.123:0/3164360592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c90590 0x55dba7cb3110 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f467c031550 tx=0x7f467c0039a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7cf16b8fa7ff5cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.095+0000 7f467be00640 1 -- 172.21.15.123:0/3164360592 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c054030 con 0x55dba7c90590 2024-09-19T04:55:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.095+0000 7f467be00640 1 -- 172.21.15.123:0/3164360592 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f467c041070 con 0x55dba7c90590 2024-09-19T04:55:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.095+0000 7f467be00640 1 -- 172.21.15.123:0/3164360592 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c035030 con 0x55dba7c90590 2024-09-19T04:55:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.096+0000 7f4698e1c740 1 -- 172.21.15.123:0/3164360592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c90590 msgr2=0x55dba7cb3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.096+0000 7f4698e1c740 1 --2- 172.21.15.123:0/3164360592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c90590 0x55dba7cb3110 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f467c031550 tx=0x7f467c0039a0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.096+0000 7f4698e1c740 1 -- 172.21.15.123:0/3164360592 shutdown_connections 2024-09-19T04:55:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.096+0000 7f4698e1c740 1 --2- 172.21.15.123:0/3164360592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c9ae50 0x55dba7c94ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.096+0000 7f4698e1c740 1 --2- 172.21.15.123:0/3164360592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c90590 0x55dba7cb3110 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.096+0000 7f4698e1c740 1 --2- 172.21.15.123:0/3164360592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c8fc70 0x55dba7cb2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.096+0000 7f4698e1c740 1 -- 172.21.15.123:0/3164360592 >> 172.21.15.123:0/3164360592 conn(0x55dba7c8c200 msgr2=0x55dba7c99e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.096+0000 7f4698e1c740 1 -- 172.21.15.123:0/3164360592 shutdown_connections 2024-09-19T04:55:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.096+0000 7f4698e1c740 1 -- 172.21.15.123:0/3164360592 wait complete. 2024-09-19T04:55:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.096+0000 7f4698e1c740 1 -- start start 2024-09-19T04:55:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.097+0000 7f4698e1c740 1 Processor -- start 2024-09-19T04:55:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.097+0000 7f4698e1c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c8fc70 0x55dba7ca3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.097+0000 7f4698e1c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c90590 0x55dba7c9ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.097+0000 7f4698e1c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c9ae50 0x55dba7c9f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.097+0000 7f4698e1c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46901b0ea0 con 0x55dba7c90590 2024-09-19T04:55:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.097+0000 7f4698e1c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46901b07d0 con 0x55dba7c9ae50 2024-09-19T04:55:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.097+0000 7f4698e1c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f46901ada30 con 0x55dba7c8fc70 2024-09-19T04:55:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.097+0000 7f4698e1c740 10 client.0 _ll_register_callbacks cb 0x7f467c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.097+0000 7f468ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c90590 0x55dba7c9ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.097+0000 7f468ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c90590 0x55dba7c9ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54660/0 (socket says 172.21.15.123:54660) 2024-09-19T04:55:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.097+0000 7f468ea00640 1 -- 172.21.15.123:0/2942458102 learned_addr learned my addr 172.21.15.123:0/2942458102 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.097+0000 7f468fe00640 1 --2- 172.21.15.123:0/2942458102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c9ae50 0x55dba7c9f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.097+0000 7f468fe00640 1 --2- 172.21.15.123:0/2942458102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c8fc70 0x55dba7ca3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.098+0000 7f468ea00640 1 -- 172.21.15.123:0/2942458102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c8fc70 msgr2=0x55dba7ca3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.098+0000 7f468ea00640 1 --2- 172.21.15.123:0/2942458102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c8fc70 0x55dba7ca3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.098+0000 7f468ea00640 1 -- 172.21.15.123:0/2942458102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c9ae50 msgr2=0x55dba7c9f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.098+0000 7f468ea00640 1 --2- 172.21.15.123:0/2942458102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c9ae50 0x55dba7c9f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.098+0000 7f468ea00640 1 -- 172.21.15.123:0/2942458102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dba7d14ba0 con 0x55dba7c90590 2024-09-19T04:55:54.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.098+0000 7f468fe00640 1 --2- 172.21.15.123:0/2942458102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c9ae50 0x55dba7c9f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.098+0000 7f468ea00640 1 --2- 172.21.15.123:0/2942458102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c90590 0x55dba7c9ea30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f467c000c00 tx=0x7f467c003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc52ff2308900b31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.098+0000 7f4679600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:55:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.098+0000 7f4679600640 1 -- 172.21.15.123:0/2942458102 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c05c020 con 0x55dba7c90590 2024-09-19T04:55:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.098+0000 7f4679600640 1 -- 172.21.15.123:0/2942458102 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f467c005510 con 0x55dba7c90590 2024-09-19T04:55:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.098+0000 7f4698e1c740 1 -- 172.21.15.123:0/2942458102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f467c041070 con 0x55dba7c90590 2024-09-19T04:55:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.099+0000 7f4679600640 1 -- 172.21.15.123:0/2942458102 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f467c035030 con 0x55dba7c90590 2024-09-19T04:55:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.099+0000 7f4698e1c740 10 client.17253 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.099+0000 7f4698e1c740 1 -- 172.21.15.123:0/2942458102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55dba7d15d60 con 0x55dba7c90590 2024-09-19T04:55:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.099+0000 7f4679600640 1 -- 172.21.15.123:0/2942458102 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f467c041070 con 0x55dba7c90590 2024-09-19T04:55:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.099+0000 7f4679600640 1 -- 172.21.15.123:0/2942458102 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f467c031720 con 0x55dba7c90590 2024-09-19T04:55:54.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.099+0000 7f4698e1c740 10 client.17253 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:54.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.099+0000 7f4698e1c740 10 client.17253 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:55:54.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.099+0000 7f4698e1c740 20 client.17253 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:54.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.099+0000 7f4698e1c740 1 --2- 172.21.15.123:0/2942458102 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55dba7d16470 0x55dba7d18890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:54.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.099+0000 7f4698e1c740 4 client.17253 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:55:54.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.099+0000 7f4698e1c740 1 -- 172.21.15.123:0/2942458102 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dba7bfde60 con 0x55dba7d16470 2024-09-19T04:55:54.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.100+0000 7f468fe00640 1 --2- 172.21.15.123:0/2942458102 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55dba7d16470 0x55dba7d18890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:54.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.100+0000 7f468fe00640 1 --2- 172.21.15.123:0/2942458102 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55dba7d16470 0x55dba7d18890 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b1c41c383d23d55e server_cookie=a965f30422d6061b in_seq=0 out_seq=0 2024-09-19T04:55:54.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.100+0000 7f4679600640 10 client.17253 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:55:54.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.100+0000 7f4679600640 1 -- 172.21.15.123:0/2942458102 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dba7bfde60 con 0x55dba7d16470 2024-09-19T04:55:54.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.100+0000 7f4679600640 10 client.17253 handle_command_reply: tid=0 2024-09-19T04:55:54.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.101+0000 7f4698e1c740 1 client.17253 shutdown 2024-09-19T04:55:54.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.101+0000 7f4698e1c740 1 -- 172.21.15.123:0/2942458102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c90590 msgr2=0x55dba7c9ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:54.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.101+0000 7f4698e1c740 1 --2- 172.21.15.123:0/2942458102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c90590 0x55dba7c9ea30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f467c000c00 tx=0x7f467c003a10 comp rx=0 tx=0).stop 2024-09-19T04:55:54.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.101+0000 7f4698e1c740 1 -- 172.21.15.123:0/2942458102 shutdown_connections 2024-09-19T04:55:54.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.101+0000 7f4698e1c740 1 --2- 172.21.15.123:0/2942458102 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x55dba7d16470 0x55dba7d18890 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.101+0000 7f4698e1c740 1 --2- 172.21.15.123:0/2942458102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dba7c9ae50 0x55dba7c9f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.101+0000 7f4698e1c740 1 --2- 172.21.15.123:0/2942458102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dba7c90590 0x55dba7c9ea30 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.101+0000 7f4698e1c740 1 --2- 172.21.15.123:0/2942458102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dba7c8fc70 0x55dba7ca3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.101+0000 7f4698e1c740 1 -- 172.21.15.123:0/2942458102 >> 172.21.15.123:0/2942458102 conn(0x55dba7c8c200 msgr2=0x55dba7c8ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:54.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.101+0000 7f4679600640 0 client.17253 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:55:54.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.101+0000 7f4698e1c740 1 -- 172.21.15.123:0/2942458102 shutdown_connections 2024-09-19T04:55:54.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.101+0000 7f4698e1c740 1 -- 172.21.15.123:0/2942458102 wait complete. 2024-09-19T04:55:54.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.101+0000 7f4698e1c740 20 client.17253 trim_cache size 0 max 16384 2024-09-19T04:55:54.108 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:55:54.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.102+0000 7f4696400640 1 -- 172.21.15.123:0/3953853242 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46740a33b0 msgr2=0x7f46740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:54.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.102+0000 7f4696400640 1 --2- 172.21.15.123:0/3953853242 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46740a33b0 0x7f46740a5870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55dba7c8d060 tx=0x7f46800d5530 comp rx=0 tx=0).stop 2024-09-19T04:55:54.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.102+0000 7f4696400640 1 -- 172.21.15.123:0/3953853242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4690103ff0 msgr2=0x7f469019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:54.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.102+0000 7f4696400640 1 --2- 172.21.15.123:0/3953853242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4690103ff0 0x7f469019a630 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f468000d770 tx=0x7f468000dc30 comp rx=0 tx=0).stop 2024-09-19T04:55:54.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.102+0000 7f4696400640 1 -- 172.21.15.123:0/3953853242 shutdown_connections 2024-09-19T04:55:54.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.102+0000 7f4696400640 1 --2- 172.21.15.123:0/3953853242 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f46740a33b0 0x7f46740a5870 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.102+0000 7f4696400640 1 --2- 172.21.15.123:0/3953853242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4690109120 0x7f469019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.102+0000 7f4696400640 1 --2- 172.21.15.123:0/3953853242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46901049a0 0x7f469019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.102+0000 7f4696400640 1 --2- 172.21.15.123:0/3953853242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4690103ff0 0x7f469019a630 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:54.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.102+0000 7f4696400640 1 -- 172.21.15.123:0/3953853242 >> 172.21.15.123:0/3953853242 conn(0x7f46900ffa90 msgr2=0x7f4690100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:54.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.102+0000 7f4696400640 1 -- 172.21.15.123:0/3953853242 shutdown_connections 2024-09-19T04:55:54.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:54.102+0000 7f4696400640 1 -- 172.21.15.123:0/3953853242 wait complete. 2024-09-19T04:55:54.928 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:55:55.296 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:55.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:55 smithi193 ceph-mon[35592]: pgmap v285: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-09-19T04:55:55.594 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:55 smithi123 ceph-mon[30511]: pgmap v285: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-09-19T04:55:55.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:55 smithi183 ceph-mon[35333]: pgmap v285: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-09-19T04:55:55.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.853+0000 7ff29e000640 1 -- 172.21.15.123:0/1183566819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff298103ff0 msgr2=0x7ff2981043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:55.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.853+0000 7ff29e000640 1 --2- 172.21.15.123:0/1183566819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff298103ff0 0x7ff2981043d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff288009900 tx=0x7ff28802a410 comp rx=0 tx=0).stop 2024-09-19T04:55:55.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.853+0000 7ff29e000640 1 -- 172.21.15.123:0/1183566819 shutdown_connections 2024-09-19T04:55:55.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.853+0000 7ff29e000640 1 --2- 172.21.15.123:0/1183566819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff298109120 0x7ff298110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.853+0000 7ff29e000640 1 --2- 172.21.15.123:0/1183566819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2981049a0 0x7ff2981089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.853+0000 7ff29e000640 1 --2- 172.21.15.123:0/1183566819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff298103ff0 0x7ff2981043d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.853+0000 7ff29e000640 1 -- 172.21.15.123:0/1183566819 >> 172.21.15.123:0/1183566819 conn(0x7ff2980ffa90 msgr2=0x7ff298101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:55.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.853+0000 7ff29e000640 1 -- 172.21.15.123:0/1183566819 shutdown_connections 2024-09-19T04:55:55.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.853+0000 7ff29e000640 1 -- 172.21.15.123:0/1183566819 wait complete. 2024-09-19T04:55:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.854+0000 7ff29e000640 1 Processor -- start 2024-09-19T04:55:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.854+0000 7ff29e000640 1 -- start start 2024-09-19T04:55:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.854+0000 7ff29e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff298103ff0 0x7ff29819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.854+0000 7ff29e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2981049a0 0x7ff29819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.854+0000 7ff29e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff298109120 0x7ff29819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.854+0000 7ff29e000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff298114120 con 0x7ff2981049a0 2024-09-19T04:55:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.854+0000 7ff29e000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff298113fa0 con 0x7ff298109120 2024-09-19T04:55:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.854+0000 7ff29e000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff2981142a0 con 0x7ff298103ff0 2024-09-19T04:55:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.854+0000 7ff28fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2981049a0 0x7ff29819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.855+0000 7ff297400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff298103ff0 0x7ff29819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.855+0000 7ff297400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff298103ff0 0x7ff29819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:54478/0 (socket says 172.21.15.123:54478) 2024-09-19T04:55:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.855+0000 7ff28fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2981049a0 0x7ff29819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54684/0 (socket says 172.21.15.123:54684) 2024-09-19T04:55:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.855+0000 7ff297400640 1 -- 172.21.15.123:0/2311151201 learned_addr learned my addr 172.21.15.123:0/2311151201 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.855+0000 7ff297e00640 1 --2- 172.21.15.123:0/2311151201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff298109120 0x7ff29819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.855+0000 7ff297400640 1 -- 172.21.15.123:0/2311151201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff298109120 msgr2=0x7ff29819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.855+0000 7ff297400640 1 --2- 172.21.15.123:0/2311151201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff298109120 0x7ff29819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.855+0000 7ff297400640 1 -- 172.21.15.123:0/2311151201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2981049a0 msgr2=0x7ff29819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.855+0000 7ff297400640 1 --2- 172.21.15.123:0/2311151201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2981049a0 0x7ff29819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.855+0000 7ff297400640 1 -- 172.21.15.123:0/2311151201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29819f5d0 con 0x7ff298103ff0 2024-09-19T04:55:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.855+0000 7ff28fe00640 1 --2- 172.21.15.123:0/2311151201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2981049a0 0x7ff29819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.855+0000 7ff297400640 1 --2- 172.21.15.123:0/2311151201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff298103ff0 0x7ff29819a620 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff288006010 tx=0x7ff2880042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.856+0000 7ff294c00640 1 -- 172.21.15.123:0/2311151201 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28803f030 con 0x7ff298103ff0 2024-09-19T04:55:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.856+0000 7ff294c00640 1 -- 172.21.15.123:0/2311151201 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff288003ce0 con 0x7ff298103ff0 2024-09-19T04:55:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.856+0000 7ff29e000640 1 -- 172.21.15.123:0/2311151201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29819f860 con 0x7ff298103ff0 2024-09-19T04:55:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.856+0000 7ff294c00640 1 -- 172.21.15.123:0/2311151201 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288035070 con 0x7ff298103ff0 2024-09-19T04:55:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.856+0000 7ff29e000640 1 -- 172.21.15.123:0/2311151201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2981a71a0 con 0x7ff298103ff0 2024-09-19T04:55:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.859+0000 7ff2a0a9a740 1 Processor -- start 2024-09-19T04:55:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.859+0000 7ff2a0a9a740 1 -- start start 2024-09-19T04:55:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.859+0000 7ff294c00640 1 -- 172.21.15.123:0/2311151201 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff288042030 con 0x7ff298103ff0 2024-09-19T04:55:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.859+0000 7ff2a0a9a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebd8c70 0x55e47ebd9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.859+0000 7ff2a0a9a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebd9590 0x55e47ebe3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.859+0000 7ff2a0a9a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebe3e50 0x55e47ebe6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.859+0000 7ff2a0a9a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e47eb3f3e0 con 0x55e47ebd8c70 2024-09-19T04:55:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.859+0000 7ff2a0a9a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e47eae6d80 con 0x55e47ebe3e50 2024-09-19T04:55:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.859+0000 7ff2a0a9a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e47eb3b3a0 con 0x55e47ebd9590 2024-09-19T04:55:55.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.860+0000 7ff297400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebd9590 0x55e47ebe3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.860+0000 7ff297e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebd8c70 0x55e47ebd9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.860+0000 7ff297400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebd9590 0x55e47ebe3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:54488/0 (socket says 172.21.15.123:54488) 2024-09-19T04:55:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.860+0000 7ff297400640 1 -- 172.21.15.123:0/3819161446 learned_addr learned my addr 172.21.15.123:0/3819161446 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.860+0000 7ff28fe00640 1 --2- 172.21.15.123:0/3819161446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebe3e50 0x55e47ebe6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.860+0000 7ff297400640 1 -- 172.21.15.123:0/3819161446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebe3e50 msgr2=0x55e47ebe6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.860+0000 7ff297400640 1 --2- 172.21.15.123:0/3819161446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebe3e50 0x55e47ebe6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.860+0000 7ff297400640 1 -- 172.21.15.123:0/3819161446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebd8c70 msgr2=0x55e47ebd9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.860+0000 7ff297400640 1 --2- 172.21.15.123:0/3819161446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebd8c70 0x55e47ebd9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.860+0000 7ff297400640 1 -- 172.21.15.123:0/3819161446 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47eb05ae0 con 0x55e47ebd9590 2024-09-19T04:55:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.861+0000 7ff297400640 1 --2- 172.21.15.123:0/3819161446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebd9590 0x55e47ebe3910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff2880ed890 tx=0x7ff2880f8480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3eb9682f020fe8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.861+0000 7ff287e00640 1 -- 172.21.15.123:0/3819161446 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2880f8880 con 0x55e47ebd9590 2024-09-19T04:55:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.861+0000 7ff287e00640 1 -- 172.21.15.123:0/3819161446 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2880fd030 con 0x55e47ebd9590 2024-09-19T04:55:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.861+0000 7ff287e00640 1 -- 172.21.15.123:0/3819161446 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2881013d0 con 0x55e47ebd9590 2024-09-19T04:55:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.861+0000 7ff294c00640 1 --2- 172.21.15.123:0/2311151201 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff27c0a3430 0x7ff27c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.861+0000 7ff294c00640 1 -- 172.21.15.123:0/2311151201 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7ff2880f0260 con 0x7ff298103ff0 2024-09-19T04:55:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.861+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/3819161446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebd9590 msgr2=0x55e47ebe3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.861+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/3819161446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebd9590 0x55e47ebe3910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff2880ed890 tx=0x7ff2880f8480 comp rx=0 tx=0).stop 2024-09-19T04:55:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.861+0000 7ff297e00640 1 --2- 172.21.15.123:0/2311151201 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff27c0a3430 0x7ff27c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.862+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/3819161446 shutdown_connections 2024-09-19T04:55:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.862+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/3819161446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebe3e50 0x55e47ebe6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.862+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/3819161446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebd9590 0x55e47ebe3910 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.862+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/3819161446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebd8c70 0x55e47ebd9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.862+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/3819161446 >> 172.21.15.123:0/3819161446 conn(0x55e47ebd5200 msgr2=0x55e47ebd55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.862+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/3819161446 shutdown_connections 2024-09-19T04:55:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.862+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/3819161446 wait complete. 2024-09-19T04:55:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.862+0000 7ff297e00640 1 --2- 172.21.15.123:0/2311151201 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff27c0a3430 0x7ff27c0a58f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff290005fd0 tx=0x7ff2900096a0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:55.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.863+0000 7ff2a0a9a740 1 -- start start 2024-09-19T04:55:55.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.863+0000 7ff2a0a9a740 1 Processor -- start 2024-09-19T04:55:55.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.863+0000 7ff2a0a9a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebd8c70 0x55e47ebe0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.863+0000 7ff2a0a9a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebe3e50 0x55e47ebe1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.863+0000 7ff2a0a9a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebe1990 0x55e47ebefbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.863+0000 7ff2a0a9a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e47ebe67e0 con 0x55e47ebe3e50 2024-09-19T04:55:55.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.863+0000 7ff2a0a9a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e47eafa0f0 con 0x55e47ebd8c70 2024-09-19T04:55:55.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.863+0000 7ff2a0a9a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e47ebe6960 con 0x55e47ebe1990 2024-09-19T04:55:55.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.863+0000 7ff2a0a9a740 10 client.0 _ll_register_callbacks cb 0x55e47eae1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:55.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.864+0000 7ff28fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebe3e50 0x55e47ebe1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.864+0000 7ff297400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebd8c70 0x55e47ebe0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.864+0000 7ff28fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebe3e50 0x55e47ebe1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54704/0 (socket says 172.21.15.123:54704) 2024-09-19T04:55:55.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.864+0000 7ff28fe00640 1 -- 172.21.15.123:0/1402012681 learned_addr learned my addr 172.21.15.123:0/1402012681 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:55.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.864+0000 7ff297e00640 1 --2- 172.21.15.123:0/1402012681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebe1990 0x55e47ebefbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.864+0000 7ff297400640 1 -- 172.21.15.123:0/1402012681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebe1990 msgr2=0x55e47ebefbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:55.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.864+0000 7ff297400640 1 --2- 172.21.15.123:0/1402012681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebe1990 0x55e47ebefbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.864+0000 7ff297400640 1 -- 172.21.15.123:0/1402012681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebe3e50 msgr2=0x55e47ebe1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:55.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.864+0000 7ff297400640 1 --2- 172.21.15.123:0/1402012681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebe3e50 0x55e47ebe1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.864+0000 7ff297400640 1 -- 172.21.15.123:0/1402012681 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47e8b03c0 con 0x55e47ebd8c70 2024-09-19T04:55:55.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.864+0000 7ff297400640 1 --2- 172.21.15.123:0/1402012681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebd8c70 0x55e47ebe0f10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55e47ebd6450 tx=0x7ff28803b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4772397a1ac272 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:55.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.864+0000 7ff284c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:55:55.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.865+0000 7ff284c00640 1 -- 172.21.15.123:0/1402012681 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288049070 con 0x55e47ebd8c70 2024-09-19T04:55:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.865+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/1402012681 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2880fd030 con 0x55e47ebd8c70 2024-09-19T04:55:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.865+0000 7ff284c00640 1 -- 172.21.15.123:0/1402012681 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7ff2880ba950 con 0x55e47ebd8c70 2024-09-19T04:55:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.865+0000 7ff284c00640 1 -- 172.21.15.123:0/1402012681 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff2880b88f0 con 0x55e47ebd8c70 2024-09-19T04:55:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.865+0000 7ff2a0a9a740 10 client.27356 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.865+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/1402012681 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e47ebe67e0 con 0x55e47ebd8c70 2024-09-19T04:55:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.865+0000 7ff284c00640 1 -- 172.21.15.123:0/1402012681 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2880fd030 con 0x55e47ebd8c70 2024-09-19T04:55:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.865+0000 7ff284c00640 1 -- 172.21.15.123:0/1402012681 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7ff2880f0ca0 con 0x55e47ebd8c70 2024-09-19T04:55:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.865+0000 7ff2a0a9a740 10 client.27356 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.865+0000 7ff2a0a9a740 10 client.27356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:55:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.865+0000 7ff2a0a9a740 20 client.27356 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.865+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/1402012681 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55e47ebf3fc0 0x55e47ebf63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.865+0000 7ff2a0a9a740 4 client.27356 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:55:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.865+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/1402012681 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e47eb4c950 con 0x55e47ebf3fc0 2024-09-19T04:55:55.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.866+0000 7ff28fe00640 1 --2- 172.21.15.123:0/1402012681 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55e47ebf3fc0 0x55e47ebf63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.866+0000 7ff28fe00640 1 --2- 172.21.15.123:0/1402012681 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55e47ebf3fc0 0x55e47ebf63f0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b780ca7ff2e5073 server_cookie=e1af5873f6f66584 in_seq=0 out_seq=0 2024-09-19T04:55:55.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.866+0000 7ff284c00640 10 client.27356 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:55.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.867+0000 7ff284c00640 1 -- 172.21.15.123:0/1402012681 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e47eb4c950 con 0x55e47ebf3fc0 2024-09-19T04:55:55.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.867+0000 7ff284c00640 10 client.27356 handle_command_reply: tid=0 2024-09-19T04:55:55.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.867+0000 7ff2a0a9a740 1 client.27356 shutdown 2024-09-19T04:55:55.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.867+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/1402012681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebd8c70 msgr2=0x55e47ebe0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:55.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.867+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/1402012681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebd8c70 0x55e47ebe0f10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55e47ebd6450 tx=0x7ff28803b460 comp rx=0 tx=0).stop 2024-09-19T04:55:55.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.868+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/1402012681 shutdown_connections 2024-09-19T04:55:55.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.868+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/1402012681 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55e47ebf3fc0 0x55e47ebf63f0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.868+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/1402012681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebe1990 0x55e47ebefbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.868+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/1402012681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebe3e50 0x55e47ebe1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.868+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/1402012681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebd8c70 0x55e47ebe0f10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.868+0000 7ff284c00640 0 client.27356 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:55.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.868+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/1402012681 >> 172.21.15.123:0/1402012681 conn(0x55e47ebd5200 msgr2=0x55e47ebd4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:55.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.868+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/1402012681 shutdown_connections 2024-09-19T04:55:55.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.868+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/1402012681 wait complete. 2024-09-19T04:55:55.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.868+0000 7ff2a0a9a740 20 client.27356 trim_cache size 0 max 16384 2024-09-19T04:55:55.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.881+0000 7ff2a0a9a740 1 Processor -- start 2024-09-19T04:55:55.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.882+0000 7ff2a0a9a740 1 -- start start 2024-09-19T04:55:55.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.882+0000 7ff2a0a9a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebd8c70 0x55e47ebef990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.882+0000 7ff2a0a9a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebe1990 0x55e47ebfbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.882+0000 7ff2a0a9a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebe3e50 0x55e47ebfc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.882+0000 7ff2a0a9a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2981afac0 con 0x55e47ebd8c70 2024-09-19T04:55:55.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.882+0000 7ff2a0a9a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2981b1600 con 0x55e47ebe3e50 2024-09-19T04:55:55.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.882+0000 7ff2a0a9a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff2981b0190 con 0x55e47ebe1990 2024-09-19T04:55:55.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.882+0000 7ff297400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebd8c70 0x55e47ebef990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.882+0000 7ff297400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebd8c70 0x55e47ebef990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54720/0 (socket says 172.21.15.123:54720) 2024-09-19T04:55:55.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.882+0000 7ff297400640 1 -- 172.21.15.123:0/2179471292 learned_addr learned my addr 172.21.15.123:0/2179471292 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:55.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.882+0000 7ff28fe00640 1 --2- 172.21.15.123:0/2179471292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebe1990 0x55e47ebfbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.882+0000 7ff297e00640 1 --2- 172.21.15.123:0/2179471292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebe3e50 0x55e47ebfc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.883+0000 7ff28fe00640 1 -- 172.21.15.123:0/2179471292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebe3e50 msgr2=0x55e47ebfc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.883+0000 7ff28fe00640 1 --2- 172.21.15.123:0/2179471292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebe3e50 0x55e47ebfc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.883+0000 7ff28fe00640 1 -- 172.21.15.123:0/2179471292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebd8c70 msgr2=0x55e47ebef990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.883+0000 7ff28fe00640 1 --2- 172.21.15.123:0/2179471292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebd8c70 0x55e47ebef990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.883+0000 7ff28fe00640 1 -- 172.21.15.123:0/2179471292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47ebd0770 con 0x55e47ebe1990 2024-09-19T04:55:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.883+0000 7ff28fe00640 1 --2- 172.21.15.123:0/2179471292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebe1990 0x55e47ebfbcc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55e47ebf6c00 tx=0x7ff28000b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20c64b9ef204ddf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.883+0000 7ff287400640 1 -- 172.21.15.123:0/2179471292 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280004220 con 0x55e47ebe1990 2024-09-19T04:55:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.883+0000 7ff287400640 1 -- 172.21.15.123:0/2179471292 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2800043c0 con 0x55e47ebe1990 2024-09-19T04:55:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.883+0000 7ff287400640 1 -- 172.21.15.123:0/2179471292 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280004a00 con 0x55e47ebe1990 2024-09-19T04:55:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.884+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/2179471292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebe1990 msgr2=0x55e47ebfbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.884+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/2179471292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebe1990 0x55e47ebfbcc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55e47ebf6c00 tx=0x7ff28000b410 comp rx=0 tx=0).stop 2024-09-19T04:55:55.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.884+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/2179471292 shutdown_connections 2024-09-19T04:55:55.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.884+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/2179471292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebe3e50 0x55e47ebfc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.884+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/2179471292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebe1990 0x55e47ebfbcc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.884+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/2179471292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebd8c70 0x55e47ebef990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.884+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/2179471292 >> 172.21.15.123:0/2179471292 conn(0x55e47ebd5200 msgr2=0x55e47ebe0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:55.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.884+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/2179471292 shutdown_connections 2024-09-19T04:55:55.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.884+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/2179471292 wait complete. 2024-09-19T04:55:55.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.885+0000 7ff2a0a9a740 1 -- start start 2024-09-19T04:55:55.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.885+0000 7ff2a0a9a740 1 Processor -- start 2024-09-19T04:55:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.885+0000 7ff2a0a9a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebd8c70 0x55e47ebe7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.885+0000 7ff2a0a9a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebe1990 0x55e47ebe8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.885+0000 7ff2a0a9a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebe3e50 0x55e47ec664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.885+0000 7ff2a0a9a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2981b0f30 con 0x55e47ebe1990 2024-09-19T04:55:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.885+0000 7ff2a0a9a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2981b0860 con 0x55e47ebe3e50 2024-09-19T04:55:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.885+0000 7ff2a0a9a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff2981adac0 con 0x55e47ebd8c70 2024-09-19T04:55:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.885+0000 7ff2a0a9a740 10 client.0 _ll_register_callbacks cb 0x7ff290009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.886+0000 7ff28fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebe1990 0x55e47ebe8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.886+0000 7ff28fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebe1990 0x55e47ebe8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54722/0 (socket says 172.21.15.123:54722) 2024-09-19T04:55:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.886+0000 7ff28fe00640 1 -- 172.21.15.123:0/3306414825 learned_addr learned my addr 172.21.15.123:0/3306414825 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.886+0000 7ff297400640 1 --2- 172.21.15.123:0/3306414825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebd8c70 0x55e47ebe7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.886+0000 7ff297e00640 1 --2- 172.21.15.123:0/3306414825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebe3e50 0x55e47ec664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.886+0000 7ff28fe00640 1 -- 172.21.15.123:0/3306414825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebd8c70 msgr2=0x55e47ebe7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:55.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.886+0000 7ff28fe00640 1 --2- 172.21.15.123:0/3306414825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebd8c70 0x55e47ebe7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.886+0000 7ff28fe00640 1 -- 172.21.15.123:0/3306414825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebe3e50 msgr2=0x55e47ec664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:55.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.886+0000 7ff28fe00640 1 --2- 172.21.15.123:0/3306414825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebe3e50 0x55e47ec664a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.886+0000 7ff28fe00640 1 -- 172.21.15.123:0/3306414825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47ec669e0 con 0x55e47ebe1990 2024-09-19T04:55:55.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.886+0000 7ff28fe00640 1 --2- 172.21.15.123:0/3306414825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebe1990 0x55e47ebe8470 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff280011630 tx=0x7ff280005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c43e76aeb0e0b2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:55.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.886+0000 7ff284c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:55:55.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.887+0000 7ff284c00640 1 -- 172.21.15.123:0/3306414825 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280032060 con 0x55e47ebe1990 2024-09-19T04:55:55.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.887+0000 7ff284c00640 1 -- 172.21.15.123:0/3306414825 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7ff280005360 con 0x55e47ebe1990 2024-09-19T04:55:55.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.887+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/3306414825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2800043c0 con 0x55e47ebe1990 2024-09-19T04:55:55.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.887+0000 7ff284c00640 1 -- 172.21.15.123:0/3306414825 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff280029040 con 0x55e47ebe1990 2024-09-19T04:55:55.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.887+0000 7ff2a0a9a740 10 client.17271 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:55.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.887+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/3306414825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e47ec67c60 con 0x55e47ebe1990 2024-09-19T04:55:55.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.887+0000 7ff284c00640 1 -- 172.21.15.123:0/3306414825 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2800043c0 con 0x55e47ebe1990 2024-09-19T04:55:55.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.888+0000 7ff284c00640 1 -- 172.21.15.123:0/3306414825 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7ff28002c070 con 0x55e47ebe1990 2024-09-19T04:55:55.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.888+0000 7ff2a0a9a740 10 client.17271 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:55.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.888+0000 7ff2a0a9a740 10 client.17271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:55:55.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.888+0000 7ff2a0a9a740 20 client.17271 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:55.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.888+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/3306414825 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55e47ec68220 0x55e47ec6a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:55.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.888+0000 7ff2a0a9a740 4 client.17271 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:55:55.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.888+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/3306414825 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e47eb46e60 con 0x55e47ec68220 2024-09-19T04:55:55.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.888+0000 7ff297400640 1 --2- 172.21.15.123:0/3306414825 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55e47ec68220 0x55e47ec6a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.888+0000 7ff297400640 1 --2- 172.21.15.123:0/3306414825 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55e47ec68220 0x55e47ec6a640 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=911a932f8b023a9d server_cookie=6619a65ca38d1d36 in_seq=0 out_seq=0 2024-09-19T04:55:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.888+0000 7ff284c00640 10 client.17271 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.889+0000 7ff284c00640 1 -- 172.21.15.123:0/3306414825 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e47eb46e60 con 0x55e47ec68220 2024-09-19T04:55:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.889+0000 7ff284c00640 10 client.17271 handle_command_reply: tid=0 2024-09-19T04:55:55.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.889+0000 7ff2a0a9a740 1 client.17271 shutdown 2024-09-19T04:55:55.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.889+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/3306414825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebe1990 msgr2=0x55e47ebe8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:55.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.889+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/3306414825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebe1990 0x55e47ebe8470 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff280011630 tx=0x7ff280005c60 comp rx=0 tx=0).stop 2024-09-19T04:55:55.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.889+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/3306414825 shutdown_connections 2024-09-19T04:55:55.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.889+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/3306414825 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55e47ec68220 0x55e47ec6a640 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.889+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/3306414825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ebe3e50 0x55e47ec664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.889+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/3306414825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e47ebe1990 0x55e47ebe8470 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.889+0000 7ff2a0a9a740 1 --2- 172.21.15.123:0/3306414825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e47ebd8c70 0x55e47ebe7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.889+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/3306414825 >> 172.21.15.123:0/3306414825 conn(0x55e47ebd5200 msgr2=0x55e47ebd3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:55.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.889+0000 7ff284c00640 0 client.17271 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:55.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.889+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/3306414825 shutdown_connections 2024-09-19T04:55:55.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.889+0000 7ff2a0a9a740 1 -- 172.21.15.123:0/3306414825 wait complete. 2024-09-19T04:55:55.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.889+0000 7ff2a0a9a740 20 client.17271 trim_cache size 0 max 16384 2024-09-19T04:55:55.895 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:55:55.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.890+0000 7ff29e000640 1 -- 172.21.15.123:0/2311151201 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff27c0a3430 msgr2=0x7ff27c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:55.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.890+0000 7ff29e000640 1 --2- 172.21.15.123:0/2311151201 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff27c0a3430 0x7ff27c0a58f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff290005fd0 tx=0x7ff2900096a0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.890+0000 7ff29e000640 1 -- 172.21.15.123:0/2311151201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff298103ff0 msgr2=0x7ff29819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:55.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.890+0000 7ff29e000640 1 --2- 172.21.15.123:0/2311151201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff298103ff0 0x7ff29819a620 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff288006010 tx=0x7ff2880042f0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.890+0000 7ff29e000640 1 -- 172.21.15.123:0/2311151201 shutdown_connections 2024-09-19T04:55:55.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.890+0000 7ff29e000640 1 --2- 172.21.15.123:0/2311151201 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ff27c0a3430 0x7ff27c0a58f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.890+0000 7ff29e000640 1 --2- 172.21.15.123:0/2311151201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff298109120 0x7ff29819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.890+0000 7ff29e000640 1 --2- 172.21.15.123:0/2311151201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2981049a0 0x7ff29819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.890+0000 7ff29e000640 1 --2- 172.21.15.123:0/2311151201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff298103ff0 0x7ff29819a620 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:55.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.890+0000 7ff29e000640 1 -- 172.21.15.123:0/2311151201 >> 172.21.15.123:0/2311151201 conn(0x7ff2980ffa90 msgr2=0x7ff298100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:55.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.890+0000 7ff29e000640 1 -- 172.21.15.123:0/2311151201 shutdown_connections 2024-09-19T04:55:55.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:55.890+0000 7ff29e000640 1 -- 172.21.15.123:0/2311151201 wait complete. 2024-09-19T04:55:57.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:57 smithi193 ceph-mon[35592]: pgmap v286: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:55:57.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:57 smithi183 ceph-mon[35333]: pgmap v286: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:55:57.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:57 smithi123 ceph-mon[30511]: pgmap v286: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:55:57.689 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:55:57.690 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:55:58.073 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:58.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.648+0000 7fd75ca00640 1 -- 172.21.15.123:0/2863195433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd758103e30 msgr2=0x7fd758104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:58.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.648+0000 7fd75ca00640 1 --2- 172.21.15.123:0/2863195433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd758103e30 0x7fd758104210 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd748009900 tx=0x7fd74802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.648+0000 7fd75ca00640 1 -- 172.21.15.123:0/2863195433 shutdown_connections 2024-09-19T04:55:58.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.648+0000 7fd75ca00640 1 --2- 172.21.15.123:0/2863195433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd758108f60 0x7fd758110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.648+0000 7fd75ca00640 1 --2- 172.21.15.123:0/2863195433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7581047e0 0x7fd758108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.648+0000 7fd75ca00640 1 --2- 172.21.15.123:0/2863195433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd758103e30 0x7fd758104210 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.648+0000 7fd75ca00640 1 -- 172.21.15.123:0/2863195433 >> 172.21.15.123:0/2863195433 conn(0x7fd7580ff8d0 msgr2=0x7fd758101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:58.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.648+0000 7fd75ca00640 1 -- 172.21.15.123:0/2863195433 shutdown_connections 2024-09-19T04:55:58.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.648+0000 7fd75ca00640 1 -- 172.21.15.123:0/2863195433 wait complete. 2024-09-19T04:55:58.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.649+0000 7fd75ca00640 1 Processor -- start 2024-09-19T04:55:58.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.649+0000 7fd75ca00640 1 -- start start 2024-09-19T04:55:58.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.650+0000 7fd75ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd758103e30 0x7fd75819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.650+0000 7fd75ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7581047e0 0x7fd75819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.650+0000 7fd75ca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd758108f60 0x7fd75819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.650+0000 7fd75ca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd758113fd0 con 0x7fd7581047e0 2024-09-19T04:55:58.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.650+0000 7fd75ca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd758113e50 con 0x7fd758103e30 2024-09-19T04:55:58.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.650+0000 7fd75ca00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd758114150 con 0x7fd758108f60 2024-09-19T04:55:58.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.650+0000 7fd755e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7581047e0 0x7fd75819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.650+0000 7fd757200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd758108f60 0x7fd75819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.650+0000 7fd757200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd758108f60 0x7fd75819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:54528/0 (socket says 172.21.15.123:54528) 2024-09-19T04:55:58.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.650+0000 7fd757200640 1 -- 172.21.15.123:0/794730911 learned_addr learned my addr 172.21.15.123:0/794730911 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:58.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.650+0000 7fd756800640 1 --2- 172.21.15.123:0/794730911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd758103e30 0x7fd75819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.650+0000 7fd757200640 1 -- 172.21.15.123:0/794730911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd758103e30 msgr2=0x7fd75819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:58.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.650+0000 7fd757200640 1 --2- 172.21.15.123:0/794730911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd758103e30 0x7fd75819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.650+0000 7fd757200640 1 -- 172.21.15.123:0/794730911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7581047e0 msgr2=0x7fd75819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:58.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.650+0000 7fd757200640 1 --2- 172.21.15.123:0/794730911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7581047e0 0x7fd75819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.650+0000 7fd757200640 1 -- 172.21.15.123:0/794730911 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75819f420 con 0x7fd758108f60 2024-09-19T04:55:58.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.650+0000 7fd755e00640 1 --2- 172.21.15.123:0/794730911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7581047e0 0x7fd75819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:58.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.651+0000 7fd757200640 1 --2- 172.21.15.123:0/794730911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd758108f60 0x7fd75819ed40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd750009fd0 tx=0x7fd75000b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:58.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.651+0000 7fd74f400640 1 -- 172.21.15.123:0/794730911 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750017070 con 0x7fd758108f60 2024-09-19T04:55:58.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.651+0000 7fd75ca00640 1 -- 172.21.15.123:0/794730911 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75819f710 con 0x7fd758108f60 2024-09-19T04:55:58.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.651+0000 7fd74f400640 1 -- 172.21.15.123:0/794730911 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd750003e50 con 0x7fd758108f60 2024-09-19T04:55:58.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.651+0000 7fd74f400640 1 -- 172.21.15.123:0/794730911 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7500106c0 con 0x7fd758108f60 2024-09-19T04:55:58.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.651+0000 7fd75ca00640 1 -- 172.21.15.123:0/794730911 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7581a6ff0 con 0x7fd758108f60 2024-09-19T04:55:58.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.654+0000 7fd75fff6740 1 Processor -- start 2024-09-19T04:55:58.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.654+0000 7fd74f400640 1 -- 172.21.15.123:0/794730911 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd750003970 con 0x7fd758108f60 2024-09-19T04:55:58.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.655+0000 7fd75fff6740 1 -- start start 2024-09-19T04:55:58.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.655+0000 7fd75fff6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca3cb0 0x562824ca4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.655+0000 7fd75fff6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca45d0 0x562824cae950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.655+0000 7fd75fff6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824caee90 0x562824cb1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.655+0000 7fd75fff6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562824c0a290 con 0x562824caee90 2024-09-19T04:55:58.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.655+0000 7fd75fff6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562824bb1b40 con 0x562824ca45d0 2024-09-19T04:55:58.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.655+0000 7fd75fff6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562824c06250 con 0x562824ca3cb0 2024-09-19T04:55:58.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.655+0000 7fd755e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca3cb0 0x562824ca4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.655+0000 7fd755e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca3cb0 0x562824ca4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:54542/0 (socket says 172.21.15.123:54542) 2024-09-19T04:55:58.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.655+0000 7fd755e00640 1 -- 172.21.15.123:0/3684124108 learned_addr learned my addr 172.21.15.123:0/3684124108 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:58.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.655+0000 7fd756800640 1 --2- 172.21.15.123:0/3684124108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca45d0 0x562824cae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.655+0000 7fd755e00640 1 --2- 172.21.15.123:0/3684124108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824caee90 0x562824cb1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.656+0000 7fd756800640 1 -- 172.21.15.123:0/3684124108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca3cb0 msgr2=0x562824ca4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:58.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.656+0000 7fd756800640 1 --2- 172.21.15.123:0/3684124108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca3cb0 0x562824ca4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.656+0000 7fd756800640 1 -- 172.21.15.123:0/3684124108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824caee90 msgr2=0x562824cb1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:58.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.656+0000 7fd756800640 1 --2- 172.21.15.123:0/3684124108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824caee90 0x562824cb1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.656+0000 7fd756800640 1 -- 172.21.15.123:0/3684124108 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56282497b6a0 con 0x562824ca45d0 2024-09-19T04:55:58.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.656+0000 7fd74f400640 1 --2- 172.21.15.123:0/794730911 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd73c0a33b0 0x7fd73c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.656+0000 7fd756800640 1 --2- 172.21.15.123:0/3684124108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca45d0 0x562824cae950 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd74802a9e0 tx=0x7fd7480040f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6bec622b28c9be2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:58.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.656+0000 7fd74f400640 1 -- 172.21.15.123:0/794730911 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fd7500cfa40 con 0x7fd758108f60 2024-09-19T04:55:58.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.656+0000 7fd755e00640 1 --2- 172.21.15.123:0/794730911 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd73c0a33b0 0x7fd73c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.656+0000 7fd743400640 1 -- 172.21.15.123:0/3684124108 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748035040 con 0x562824ca45d0 2024-09-19T04:55:58.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.657+0000 7fd743400640 1 -- 172.21.15.123:0/3684124108 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd748005520 con 0x562824ca45d0 2024-09-19T04:55:58.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.657+0000 7fd743400640 1 -- 172.21.15.123:0/3684124108 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7480035f0 con 0x562824ca45d0 2024-09-19T04:55:58.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.657+0000 7fd75fff6740 1 -- 172.21.15.123:0/3684124108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca45d0 msgr2=0x562824cae950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:58.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.657+0000 7fd755e00640 1 --2- 172.21.15.123:0/794730911 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd73c0a33b0 0x7fd73c0a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x562824ca15c0 tx=0x7fd74400b390 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:58.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.657+0000 7fd75fff6740 1 --2- 172.21.15.123:0/3684124108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca45d0 0x562824cae950 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd74802a9e0 tx=0x7fd7480040f0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.657+0000 7fd75fff6740 1 -- 172.21.15.123:0/3684124108 shutdown_connections 2024-09-19T04:55:58.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.657+0000 7fd75fff6740 1 --2- 172.21.15.123:0/3684124108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824caee90 0x562824cb1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.657+0000 7fd75fff6740 1 --2- 172.21.15.123:0/3684124108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca45d0 0x562824cae950 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.657+0000 7fd75fff6740 1 --2- 172.21.15.123:0/3684124108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca3cb0 0x562824ca4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.657+0000 7fd75fff6740 1 -- 172.21.15.123:0/3684124108 >> 172.21.15.123:0/3684124108 conn(0x562824ca0240 msgr2=0x562824ca0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:58.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.657+0000 7fd75fff6740 1 -- 172.21.15.123:0/3684124108 shutdown_connections 2024-09-19T04:55:58.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.657+0000 7fd75fff6740 1 -- 172.21.15.123:0/3684124108 wait complete. 2024-09-19T04:55:58.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.658+0000 7fd75fff6740 1 -- start start 2024-09-19T04:55:58.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.658+0000 7fd75fff6740 1 Processor -- start 2024-09-19T04:55:58.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.658+0000 7fd75fff6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca3cb0 0x562824cae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.658+0000 7fd75fff6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca45d0 0x562824caba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.658+0000 7fd75fff6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824caee90 0x562824cabf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.658+0000 7fd75fff6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562824cb1b00 con 0x562824ca3cb0 2024-09-19T04:55:58.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.658+0000 7fd75fff6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562824bc4fa0 con 0x562824caee90 2024-09-19T04:55:58.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.658+0000 7fd75fff6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562824cb1c80 con 0x562824ca45d0 2024-09-19T04:55:58.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.659+0000 7fd75fff6740 10 client.0 _ll_register_callbacks cb 0x562824918ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:58.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.659+0000 7fd756800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca3cb0 0x562824cae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.659+0000 7fd756800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca3cb0 0x562824cae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54744/0 (socket says 172.21.15.123:54744) 2024-09-19T04:55:58.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.659+0000 7fd756800640 1 -- 172.21.15.123:0/2310211736 learned_addr learned my addr 172.21.15.123:0/2310211736 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:58.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.659+0000 7fd755e00640 1 --2- 172.21.15.123:0/2310211736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca45d0 0x562824caba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.659+0000 7fd757200640 1 --2- 172.21.15.123:0/2310211736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824caee90 0x562824cabf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.659+0000 7fd756800640 1 -- 172.21.15.123:0/2310211736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca45d0 msgr2=0x562824caba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:58.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.659+0000 7fd756800640 1 --2- 172.21.15.123:0/2310211736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca45d0 0x562824caba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.659+0000 7fd756800640 1 -- 172.21.15.123:0/2310211736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824caee90 msgr2=0x562824cabf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:58.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.659+0000 7fd756800640 1 --2- 172.21.15.123:0/2310211736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824caee90 0x562824cabf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.659+0000 7fd756800640 1 -- 172.21.15.123:0/2310211736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562824cac740 con 0x562824ca3cb0 2024-09-19T04:55:58.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.659+0000 7fd757200640 1 --2- 172.21.15.123:0/2310211736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824caee90 0x562824cabf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:55:58.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.660+0000 7fd756800640 1 --2- 172.21.15.123:0/2310211736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca3cb0 0x562824cae770 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd748031550 tx=0x7fd74802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbe1a8f808ee1fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:58.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.660+0000 7fd737e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:55:58.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.660+0000 7fd737e00640 1 -- 172.21.15.123:0/2310211736 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748053060 con 0x562824ca3cb0 2024-09-19T04:55:58.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.660+0000 7fd737e00640 1 -- 172.21.15.123:0/2310211736 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fd748005000 con 0x562824ca3cb0 2024-09-19T04:55:58.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.660+0000 7fd75fff6740 1 -- 172.21.15.123:0/2310211736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd748005520 con 0x562824ca3cb0 2024-09-19T04:55:58.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.660+0000 7fd737e00640 1 -- 172.21.15.123:0/2310211736 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd748035050 con 0x562824ca3cb0 2024-09-19T04:55:58.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.660+0000 7fd75fff6740 10 client.17283 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:58.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.660+0000 7fd75fff6740 1 -- 172.21.15.123:0/2310211736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562824cbd9c0 con 0x562824ca3cb0 2024-09-19T04:55:58.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.660+0000 7fd737e00640 1 -- 172.21.15.123:0/2310211736 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd748005520 con 0x562824ca3cb0 2024-09-19T04:55:58.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.661+0000 7fd737e00640 1 -- 172.21.15.123:0/2310211736 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fd748003c70 con 0x562824ca3cb0 2024-09-19T04:55:58.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.661+0000 7fd75fff6740 10 client.17283 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:58.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.661+0000 7fd75fff6740 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:55:58.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.661+0000 7fd75fff6740 20 client.17283 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:58.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.661+0000 7fd75fff6740 1 --2- 172.21.15.123:0/2310211736 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562824cbed10 0x562824cc1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.661+0000 7fd75fff6740 4 client.17283 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:55:58.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.661+0000 7fd75fff6740 1 -- 172.21.15.123:0/2310211736 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562824c17800 con 0x562824cbed10 2024-09-19T04:55:58.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.661+0000 7fd755e00640 1 --2- 172.21.15.123:0/2310211736 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562824cbed10 0x562824cc1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.662+0000 7fd755e00640 1 --2- 172.21.15.123:0/2310211736 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562824cbed10 0x562824cc1100 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d475ff77cbd9ab1c server_cookie=3988add19275315f in_seq=0 out_seq=0 2024-09-19T04:55:58.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.662+0000 7fd737e00640 10 client.17283 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:58.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.662+0000 7fd737e00640 1 -- 172.21.15.123:0/2310211736 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562824c17800 con 0x562824cbed10 2024-09-19T04:55:58.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.662+0000 7fd737e00640 10 client.17283 handle_command_reply: tid=0 2024-09-19T04:55:58.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.662+0000 7fd75fff6740 1 client.17283 shutdown 2024-09-19T04:55:58.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.663+0000 7fd75fff6740 1 -- 172.21.15.123:0/2310211736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca3cb0 msgr2=0x562824cae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:58.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.663+0000 7fd75fff6740 1 --2- 172.21.15.123:0/2310211736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca3cb0 0x562824cae770 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd748031550 tx=0x7fd74802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.663+0000 7fd75fff6740 1 -- 172.21.15.123:0/2310211736 shutdown_connections 2024-09-19T04:55:58.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.663+0000 7fd75fff6740 1 --2- 172.21.15.123:0/2310211736 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562824cbed10 0x562824cc1100 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.663+0000 7fd75fff6740 1 --2- 172.21.15.123:0/2310211736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824caee90 0x562824cabf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.663+0000 7fd75fff6740 1 --2- 172.21.15.123:0/2310211736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca45d0 0x562824caba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.663+0000 7fd75fff6740 1 --2- 172.21.15.123:0/2310211736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca3cb0 0x562824cae770 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.663+0000 7fd75fff6740 1 -- 172.21.15.123:0/2310211736 >> 172.21.15.123:0/2310211736 conn(0x562824ca0240 msgr2=0x562824ca5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:58.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.663+0000 7fd737e00640 0 client.17283 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:58.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.663+0000 7fd75fff6740 1 -- 172.21.15.123:0/2310211736 shutdown_connections 2024-09-19T04:55:58.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.663+0000 7fd75fff6740 1 -- 172.21.15.123:0/2310211736 wait complete. 2024-09-19T04:55:58.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.663+0000 7fd75fff6740 20 client.17283 trim_cache size 0 max 16384 2024-09-19T04:55:58.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.681+0000 7fd75fff6740 1 Processor -- start 2024-09-19T04:55:58.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.681+0000 7fd75fff6740 1 -- start start 2024-09-19T04:55:58.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.681+0000 7fd75fff6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca0240 0x562824ca6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.681+0000 7fd75fff6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca3cb0 0x562824ca5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.681+0000 7fd75fff6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca45d0 0x562824cc66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.681+0000 7fd75fff6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd7581af810 con 0x562824ca0240 2024-09-19T04:55:58.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.681+0000 7fd75fff6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7581b1350 con 0x562824ca45d0 2024-09-19T04:55:58.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.681+0000 7fd75fff6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd7581afee0 con 0x562824ca3cb0 2024-09-19T04:55:58.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.682+0000 7fd756800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca0240 0x562824ca6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.682+0000 7fd757200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca45d0 0x562824cc66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.682+0000 7fd756800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca0240 0x562824ca6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54758/0 (socket says 172.21.15.123:54758) 2024-09-19T04:55:58.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.682+0000 7fd756800640 1 -- 172.21.15.123:0/1781960000 learned_addr learned my addr 172.21.15.123:0/1781960000 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:58.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.682+0000 7fd755e00640 1 --2- 172.21.15.123:0/1781960000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca3cb0 0x562824ca5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.682+0000 7fd757200640 1 -- 172.21.15.123:0/1781960000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca3cb0 msgr2=0x562824ca5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:58.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.682+0000 7fd757200640 1 --2- 172.21.15.123:0/1781960000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca3cb0 0x562824ca5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.682+0000 7fd757200640 1 -- 172.21.15.123:0/1781960000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca0240 msgr2=0x562824ca6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:58.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.682+0000 7fd757200640 1 --2- 172.21.15.123:0/1781960000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca0240 0x562824ca6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.682+0000 7fd757200640 1 -- 172.21.15.123:0/1781960000 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562824ca13d0 con 0x562824ca45d0 2024-09-19T04:55:58.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.682+0000 7fd757200640 1 --2- 172.21.15.123:0/1781960000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca45d0 0x562824cc66a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd7500cd130 tx=0x7fd75009bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0fed8c7b29c169f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:58.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.683+0000 7fd742a00640 1 -- 172.21.15.123:0/1781960000 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750020040 con 0x562824ca45d0 2024-09-19T04:55:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.683+0000 7fd742a00640 1 -- 172.21.15.123:0/1781960000 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7500a4930 con 0x562824ca45d0 2024-09-19T04:55:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.683+0000 7fd742a00640 1 -- 172.21.15.123:0/1781960000 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75009b420 con 0x562824ca45d0 2024-09-19T04:55:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.683+0000 7fd75fff6740 1 -- 172.21.15.123:0/1781960000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca45d0 msgr2=0x562824cc66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.683+0000 7fd75fff6740 1 --2- 172.21.15.123:0/1781960000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca45d0 0x562824cc66a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd7500cd130 tx=0x7fd75009bd70 comp rx=0 tx=0).stop 2024-09-19T04:55:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.683+0000 7fd75fff6740 1 -- 172.21.15.123:0/1781960000 shutdown_connections 2024-09-19T04:55:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.683+0000 7fd75fff6740 1 --2- 172.21.15.123:0/1781960000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca45d0 0x562824cc66a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.683+0000 7fd75fff6740 1 --2- 172.21.15.123:0/1781960000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca3cb0 0x562824ca5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.683+0000 7fd75fff6740 1 --2- 172.21.15.123:0/1781960000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca0240 0x562824ca6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.683+0000 7fd75fff6740 1 -- 172.21.15.123:0/1781960000 >> 172.21.15.123:0/1781960000 conn(0x562824cac0c0 msgr2=0x562824ca5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.683+0000 7fd75fff6740 1 -- 172.21.15.123:0/1781960000 shutdown_connections 2024-09-19T04:55:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.683+0000 7fd75fff6740 1 -- 172.21.15.123:0/1781960000 wait complete. 2024-09-19T04:55:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.684+0000 7fd75fff6740 1 -- start start 2024-09-19T04:55:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.684+0000 7fd75fff6740 1 Processor -- start 2024-09-19T04:55:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.684+0000 7fd75fff6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca0240 0x562824cb3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.684+0000 7fd75fff6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca3cb0 0x562824cb3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.684+0000 7fd75fff6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca45d0 0x562824d21630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.684+0000 7fd75fff6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd7581b0c80 con 0x562824ca0240 2024-09-19T04:55:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.684+0000 7fd75fff6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7581b05b0 con 0x562824ca3cb0 2024-09-19T04:55:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.684+0000 7fd75fff6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd7581ad810 con 0x562824ca45d0 2024-09-19T04:55:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.684+0000 7fd75fff6740 10 client.0 _ll_register_callbacks cb 0x7fd744012640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:55:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.685+0000 7fd755e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca3cb0 0x562824cb3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.685+0000 7fd755e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca3cb0 0x562824cb3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:59852/0 (socket says 172.21.15.123:59852) 2024-09-19T04:55:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.685+0000 7fd755e00640 1 -- 172.21.15.123:0/3065427189 learned_addr learned my addr 172.21.15.123:0/3065427189 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:55:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.685+0000 7fd756800640 1 --2- 172.21.15.123:0/3065427189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca0240 0x562824cb3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.685+0000 7fd757200640 1 --2- 172.21.15.123:0/3065427189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca45d0 0x562824d21630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.685+0000 7fd755e00640 1 -- 172.21.15.123:0/3065427189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca45d0 msgr2=0x562824d21630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:58.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.685+0000 7fd755e00640 1 --2- 172.21.15.123:0/3065427189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca45d0 0x562824d21630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.685+0000 7fd755e00640 1 -- 172.21.15.123:0/3065427189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca0240 msgr2=0x562824cb3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:55:58.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.685+0000 7fd755e00640 1 --2- 172.21.15.123:0/3065427189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca0240 0x562824cb3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.685+0000 7fd755e00640 1 -- 172.21.15.123:0/3065427189 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562824d21e60 con 0x562824ca3cb0 2024-09-19T04:55:58.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.685+0000 7fd757200640 1 --2- 172.21.15.123:0/3065427189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca45d0 0x562824d21630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:55:58.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.685+0000 7fd755e00640 1 --2- 172.21.15.123:0/3065427189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca3cb0 0x562824cb3d90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd74400be50 tx=0x7fd74400bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf73b66f8dd0ddf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:58.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.685+0000 7fd737e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:55:58.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.686+0000 7fd737e00640 1 -- 172.21.15.123:0/3065427189 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd744012e60 con 0x562824ca3cb0 2024-09-19T04:55:58.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.686+0000 7fd75fff6740 1 -- 172.21.15.123:0/3065427189 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7500a4930 con 0x562824ca3cb0 2024-09-19T04:55:58.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.686+0000 7fd737e00640 1 -- 172.21.15.123:0/3065427189 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fd74401ece0 con 0x562824ca3cb0 2024-09-19T04:55:58.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.686+0000 7fd737e00640 1 -- 172.21.15.123:0/3065427189 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd744005020 con 0x562824ca3cb0 2024-09-19T04:55:58.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.686+0000 7fd75fff6740 10 client.27380 fetch_fsmap learned FSMap version 43 2024-09-19T04:55:58.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.686+0000 7fd75fff6740 1 -- 172.21.15.123:0/3065427189 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562824d2a250 con 0x562824ca3cb0 2024-09-19T04:55:58.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.686+0000 7fd737e00640 1 -- 172.21.15.123:0/3065427189 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7500a4930 con 0x562824ca3cb0 2024-09-19T04:55:58.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.687+0000 7fd737e00640 1 -- 172.21.15.123:0/3065427189 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fd74401d070 con 0x562824ca3cb0 2024-09-19T04:55:58.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.687+0000 7fd75fff6740 10 client.27380 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:55:58.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.687+0000 7fd75fff6740 10 client.27380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:55:58.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.687+0000 7fd75fff6740 20 client.27380 populate_metadata read hostname 'smithi123' 2024-09-19T04:55:58.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.687+0000 7fd75fff6740 1 --2- 172.21.15.123:0/3065427189 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562824caee90 0x562824d2ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.687+0000 7fd75fff6740 4 client.27380 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:55:58.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.687+0000 7fd75fff6740 1 -- 172.21.15.123:0/3065427189 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562824c11d10 con 0x562824caee90 2024-09-19T04:55:58.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.687+0000 7fd756800640 1 --2- 172.21.15.123:0/3065427189 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562824caee90 0x562824d2ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.687+0000 7fd756800640 1 --2- 172.21.15.123:0/3065427189 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562824caee90 0x562824d2ca40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a90b88a9c448406 server_cookie=b69756e0b9370886 in_seq=0 out_seq=0 2024-09-19T04:55:58.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.687+0000 7fd737e00640 10 client.27380 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:58.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.688+0000 7fd737e00640 1 -- 172.21.15.123:0/3065427189 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562824c11d10 con 0x562824caee90 2024-09-19T04:55:58.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.688+0000 7fd737e00640 10 client.27380 handle_command_reply: tid=0 2024-09-19T04:55:58.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.688+0000 7fd75fff6740 1 client.27380 shutdown 2024-09-19T04:55:58.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.689+0000 7fd75fff6740 1 -- 172.21.15.123:0/3065427189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca3cb0 msgr2=0x562824cb3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:58.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.689+0000 7fd75fff6740 1 --2- 172.21.15.123:0/3065427189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca3cb0 0x562824cb3d90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd74400be50 tx=0x7fd74400bb70 comp rx=0 tx=0).stop 2024-09-19T04:55:58.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.689+0000 7fd75fff6740 1 -- 172.21.15.123:0/3065427189 shutdown_connections 2024-09-19T04:55:58.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.689+0000 7fd75fff6740 1 --2- 172.21.15.123:0/3065427189 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562824caee90 0x562824d2ca40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.689+0000 7fd75fff6740 1 --2- 172.21.15.123:0/3065427189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562824ca45d0 0x562824d21630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.689+0000 7fd75fff6740 1 --2- 172.21.15.123:0/3065427189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562824ca3cb0 0x562824cb3d90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.689+0000 7fd75fff6740 1 --2- 172.21.15.123:0/3065427189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562824ca0240 0x562824cb3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.689+0000 7fd75fff6740 1 -- 172.21.15.123:0/3065427189 >> 172.21.15.123:0/3065427189 conn(0x562824cac0c0 msgr2=0x562824c9f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:58.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.689+0000 7fd737e00640 0 client.27380 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:55:58.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.689+0000 7fd75fff6740 1 -- 172.21.15.123:0/3065427189 shutdown_connections 2024-09-19T04:55:58.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.689+0000 7fd75fff6740 1 -- 172.21.15.123:0/3065427189 wait complete. 2024-09-19T04:55:58.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.689+0000 7fd75fff6740 20 client.27380 trim_cache size 0 max 16384 2024-09-19T04:55:58.694 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:55:58.694 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-09-19T04:55:58.694 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "a804f21c-8890-4fde-9faf-3165a358ff93", 2024-09-19T04:55:58.694 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-09-19T04:55:58.694 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:55:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.690+0000 7fd75ca00640 1 -- 172.21.15.123:0/794730911 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd73c0a33b0 msgr2=0x7fd73c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.690+0000 7fd75ca00640 1 --2- 172.21.15.123:0/794730911 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd73c0a33b0 0x7fd73c0a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x562824ca15c0 tx=0x7fd74400b390 comp rx=0 tx=0).stop 2024-09-19T04:55:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.690+0000 7fd75ca00640 1 -- 172.21.15.123:0/794730911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd758108f60 msgr2=0x7fd75819ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.690+0000 7fd75ca00640 1 --2- 172.21.15.123:0/794730911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd758108f60 0x7fd75819ed40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd750009fd0 tx=0x7fd75000b260 comp rx=0 tx=0).stop 2024-09-19T04:55:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.690+0000 7fd75ca00640 1 -- 172.21.15.123:0/794730911 shutdown_connections 2024-09-19T04:55:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.690+0000 7fd75ca00640 1 --2- 172.21.15.123:0/794730911 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd73c0a33b0 0x7fd73c0a5870 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.690+0000 7fd75ca00640 1 --2- 172.21.15.123:0/794730911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd758108f60 0x7fd75819ed40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.690+0000 7fd75ca00640 1 --2- 172.21.15.123:0/794730911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7581047e0 0x7fd75819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.690+0000 7fd75ca00640 1 --2- 172.21.15.123:0/794730911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd758103e30 0x7fd75819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.690+0000 7fd75ca00640 1 -- 172.21.15.123:0/794730911 >> 172.21.15.123:0/794730911 conn(0x7fd7580ff8d0 msgr2=0x7fd758100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:58.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.690+0000 7fd75ca00640 1 -- 172.21.15.123:0/794730911 shutdown_connections 2024-09-19T04:55:58.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:55:58.690+0000 7fd75ca00640 1 -- 172.21.15.123:0/794730911 wait complete. 2024-09-19T04:55:59.416 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:55:59 smithi123 ceph-mon[30511]: pgmap v287: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:55:59.456 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:55:59.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:55:59 smithi193 ceph-mon[35592]: pgmap v287: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:55:59.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:55:59 smithi183 ceph-mon[35333]: pgmap v287: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:55:59.820 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:00.394 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:00 smithi123 ceph-mon[30511]: scrub queued for path: / 2024-09-19T04:56:00.394 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:00 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:56:00.395 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:00 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:56:00.395 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:00 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:56:00.395 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:00 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:56:00.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.390+0000 7fe04a200640 1 -- 172.21.15.123:0/2495620561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe044101de0 msgr2=0x7fe0441021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:00.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.390+0000 7fe04a200640 1 --2- 172.21.15.123:0/2495620561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe044101de0 0x7fe0441021c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe034009900 tx=0x7fe03402a410 comp rx=0 tx=0).stop 2024-09-19T04:56:00.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.390+0000 7fe04a200640 1 -- 172.21.15.123:0/2495620561 shutdown_connections 2024-09-19T04:56:00.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.390+0000 7fe04a200640 1 --2- 172.21.15.123:0/2495620561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe044106f10 0x7fe04410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.390+0000 7fe04a200640 1 --2- 172.21.15.123:0/2495620561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe044102790 0x7fe0441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.390+0000 7fe04a200640 1 --2- 172.21.15.123:0/2495620561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe044101de0 0x7fe0441021c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.390+0000 7fe04a200640 1 -- 172.21.15.123:0/2495620561 >> 172.21.15.123:0/2495620561 conn(0x7fe0440fd8c0 msgr2=0x7fe0440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:00.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.390+0000 7fe04a200640 1 -- 172.21.15.123:0/2495620561 shutdown_connections 2024-09-19T04:56:00.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.390+0000 7fe04a200640 1 -- 172.21.15.123:0/2495620561 wait complete. 2024-09-19T04:56:00.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.391+0000 7fe04a200640 1 Processor -- start 2024-09-19T04:56:00.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.391+0000 7fe04a200640 1 -- start start 2024-09-19T04:56:00.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.391+0000 7fe04a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe044101de0 0x7fe04419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.392+0000 7fe04a200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe044102790 0x7fe04419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.392+0000 7fe04a200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe044106f10 0x7fe04419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.392+0000 7fe04a200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe044111f70 con 0x7fe044101de0 2024-09-19T04:56:00.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.392+0000 7fe04a200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe044111df0 con 0x7fe044106f10 2024-09-19T04:56:00.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.392+0000 7fe04a200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0441120f0 con 0x7fe044102790 2024-09-19T04:56:00.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.392+0000 7fe043400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe044101de0 0x7fe04419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.392+0000 7fe043400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe044101de0 0x7fe04419a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54770/0 (socket says 172.21.15.123:54770) 2024-09-19T04:56:00.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.392+0000 7fe043400640 1 -- 172.21.15.123:0/2966449172 learned_addr learned my addr 172.21.15.123:0/2966449172 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:00.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.392+0000 7fe043e00640 1 --2- 172.21.15.123:0/2966449172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe044106f10 0x7fe04419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.393+0000 7fe03aa00640 1 --2- 172.21.15.123:0/2966449172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe044102790 0x7fe04419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.393+0000 7fe043e00640 1 -- 172.21.15.123:0/2966449172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe044102790 msgr2=0x7fe04419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:00.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.393+0000 7fe043e00640 1 --2- 172.21.15.123:0/2966449172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe044102790 0x7fe04419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.393+0000 7fe043e00640 1 -- 172.21.15.123:0/2966449172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe044101de0 msgr2=0x7fe04419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:00.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.393+0000 7fe043e00640 1 --2- 172.21.15.123:0/2966449172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe044101de0 0x7fe04419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.393+0000 7fe043e00640 1 -- 172.21.15.123:0/2966449172 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04419f5b0 con 0x7fe044106f10 2024-09-19T04:56:00.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.393+0000 7fe043e00640 1 --2- 172.21.15.123:0/2966449172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe044106f10 0x7fe04419eed0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009fd0 tx=0x7fe03c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:00.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.394+0000 7fe040c00640 1 -- 172.21.15.123:0/2966449172 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c00ce20 con 0x7fe044106f10 2024-09-19T04:56:00.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.394+0000 7fe040c00640 1 -- 172.21.15.123:0/2966449172 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe03c005020 con 0x7fe044106f10 2024-09-19T04:56:00.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.394+0000 7fe04a200640 1 -- 172.21.15.123:0/2966449172 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04419f8a0 con 0x7fe044106f10 2024-09-19T04:56:00.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.394+0000 7fe040c00640 1 -- 172.21.15.123:0/2966449172 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c010720 con 0x7fe044106f10 2024-09-19T04:56:00.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.394+0000 7fe04a200640 1 -- 172.21.15.123:0/2966449172 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0441089e0 con 0x7fe044106f10 2024-09-19T04:56:00.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.398+0000 7fe04ccbc740 1 Processor -- start 2024-09-19T04:56:00.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.398+0000 7fe040c00640 1 -- 172.21.15.123:0/2966449172 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe03c005540 con 0x7fe044106f10 2024-09-19T04:56:00.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.398+0000 7fe04ccbc740 1 -- start start 2024-09-19T04:56:00.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.399+0000 7fe04ccbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dccb90 0x562330dccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.399+0000 7fe04ccbc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dcd4b0 0x562330dd7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.399+0000 7fe04ccbc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dd7d70 0x562330dda160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.399+0000 7fe04ccbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562330d333e0 con 0x562330dccb90 2024-09-19T04:56:00.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.399+0000 7fe04ccbc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562330cdad80 con 0x562330dcd4b0 2024-09-19T04:56:00.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.399+0000 7fe04ccbc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562330d2f3a0 con 0x562330dd7d70 2024-09-19T04:56:00.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.399+0000 7fe03aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dccb90 0x562330dccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.399+0000 7fe03aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dccb90 0x562330dccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54774/0 (socket says 172.21.15.123:54774) 2024-09-19T04:56:00.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.399+0000 7fe03aa00640 1 -- 172.21.15.123:0/3296384601 learned_addr learned my addr 172.21.15.123:0/3296384601 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:00.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.399+0000 7fe043400640 1 --2- 172.21.15.123:0/3296384601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dcd4b0 0x562330dd7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.399+0000 7fe03aa00640 1 --2- 172.21.15.123:0/3296384601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dd7d70 0x562330dda160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.399+0000 7fe03aa00640 1 -- 172.21.15.123:0/3296384601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dd7d70 msgr2=0x562330dda160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:00.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.399+0000 7fe03aa00640 1 --2- 172.21.15.123:0/3296384601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dd7d70 0x562330dda160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.399+0000 7fe03aa00640 1 -- 172.21.15.123:0/3296384601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dcd4b0 msgr2=0x562330dd7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:00.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.399+0000 7fe03aa00640 1 --2- 172.21.15.123:0/3296384601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dcd4b0 0x562330dd7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.399+0000 7fe03aa00640 1 -- 172.21.15.123:0/3296384601 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562330cf9ae0 con 0x562330dccb90 2024-09-19T04:56:00.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.400+0000 7fe040c00640 1 --2- 172.21.15.123:0/2966449172 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe0280a33b0 0x7fe0280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.400+0000 7fe03aa00640 1 --2- 172.21.15.123:0/3296384601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dccb90 0x562330dccf70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe02c013ed0 tx=0x7fe02c013e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b198b2087bfdf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.400+0000 7fe040c00640 1 -- 172.21.15.123:0/2966449172 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fe03c0d0310 con 0x7fe044106f10 2024-09-19T04:56:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.400+0000 7fe033e00640 1 -- 172.21.15.123:0/3296384601 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c004020 con 0x562330dccb90 2024-09-19T04:56:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.400+0000 7fe043400640 1 --2- 172.21.15.123:0/2966449172 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe0280a33b0 0x7fe0280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.400+0000 7fe033e00640 1 -- 172.21.15.123:0/3296384601 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe02c008d50 con 0x562330dccb90 2024-09-19T04:56:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.400+0000 7fe033e00640 1 -- 172.21.15.123:0/3296384601 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c003bf0 con 0x562330dccb90 2024-09-19T04:56:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.401+0000 7fe04ccbc740 1 -- 172.21.15.123:0/3296384601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dccb90 msgr2=0x562330dccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.401+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/3296384601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dccb90 0x562330dccf70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe02c013ed0 tx=0x7fe02c013e60 comp rx=0 tx=0).stop 2024-09-19T04:56:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.401+0000 7fe043400640 1 --2- 172.21.15.123:0/2966449172 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe0280a33b0 0x7fe0280a5870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x562330dc9f80 tx=0x7fe034005a20 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.401+0000 7fe04ccbc740 1 -- 172.21.15.123:0/3296384601 shutdown_connections 2024-09-19T04:56:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.401+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/3296384601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dd7d70 0x562330dda160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.401+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/3296384601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dcd4b0 0x562330dd7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.401+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/3296384601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dccb90 0x562330dccf70 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.401+0000 7fe04ccbc740 1 -- 172.21.15.123:0/3296384601 >> 172.21.15.123:0/3296384601 conn(0x562330dc9120 msgr2=0x562330dc9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:00.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.401+0000 7fe04ccbc740 1 -- 172.21.15.123:0/3296384601 shutdown_connections 2024-09-19T04:56:00.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.401+0000 7fe04ccbc740 1 -- 172.21.15.123:0/3296384601 wait complete. 2024-09-19T04:56:00.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.402+0000 7fe04ccbc740 1 -- start start 2024-09-19T04:56:00.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.402+0000 7fe04ccbc740 1 Processor -- start 2024-09-19T04:56:00.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.402+0000 7fe04ccbc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dccb90 0x562330dd4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.402+0000 7fe04ccbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dcd4b0 0x562330dd5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.402+0000 7fe04ccbc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dd7d70 0x562330dd5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.402+0000 7fe04ccbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562330dda9e0 con 0x562330dcd4b0 2024-09-19T04:56:00.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.402+0000 7fe04ccbc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562330cee0f0 con 0x562330dd7d70 2024-09-19T04:56:00.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.402+0000 7fe04ccbc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562330ddab60 con 0x562330dccb90 2024-09-19T04:56:00.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.402+0000 7fe04ccbc740 10 client.0 _ll_register_callbacks cb 0x562330cdf9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:00.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.403+0000 7fe03aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dcd4b0 0x562330dd5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.403+0000 7fe043e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dd7d70 0x562330dd5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.403+0000 7fe043400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dccb90 0x562330dd4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.403+0000 7fe043e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dd7d70 0x562330dd5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:59888/0 (socket says 172.21.15.123:59888) 2024-09-19T04:56:00.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.403+0000 7fe043e00640 1 -- 172.21.15.123:0/2637354303 learned_addr learned my addr 172.21.15.123:0/2637354303 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:00.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.403+0000 7fe043e00640 1 -- 172.21.15.123:0/2637354303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dccb90 msgr2=0x562330dd4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:00.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.403+0000 7fe043e00640 1 --2- 172.21.15.123:0/2637354303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dccb90 0x562330dd4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.403+0000 7fe043e00640 1 -- 172.21.15.123:0/2637354303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dcd4b0 msgr2=0x562330dd5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:00.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.403+0000 7fe043e00640 1 --2- 172.21.15.123:0/2637354303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dcd4b0 0x562330dd5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.403+0000 7fe043e00640 1 -- 172.21.15.123:0/2637354303 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562330aa43c0 con 0x562330dd7d70 2024-09-19T04:56:00.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.403+0000 7fe043e00640 1 --2- 172.21.15.123:0/2637354303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dd7d70 0x562330dd5590 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe03c098f60 tx=0x7fe03c09cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de6183d92eabddf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:00.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.403+0000 7fe030c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:56:00.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.404+0000 7fe030c00640 1 -- 172.21.15.123:0/2637354303 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c0a5200 con 0x562330dd7d70 2024-09-19T04:56:00.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.404+0000 7fe030c00640 1 -- 172.21.15.123:0/2637354303 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fe03c098c00 con 0x562330dd7d70 2024-09-19T04:56:00.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.404+0000 7fe04ccbc740 1 -- 172.21.15.123:0/2637354303 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe02c008d50 con 0x562330dd7d70 2024-09-19T04:56:00.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.404+0000 7fe030c00640 1 -- 172.21.15.123:0/2637354303 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe03c0a4d70 con 0x562330dd7d70 2024-09-19T04:56:00.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.404+0000 7fe04ccbc740 10 client.27392 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:00.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.404+0000 7fe04ccbc740 1 -- 172.21.15.123:0/2637354303 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562330de65b0 con 0x562330dd7d70 2024-09-19T04:56:00.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.404+0000 7fe030c00640 1 -- 172.21.15.123:0/2637354303 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe02c008d50 con 0x562330dd7d70 2024-09-19T04:56:00.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.404+0000 7fe030c00640 1 -- 172.21.15.123:0/2637354303 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fe03c0b5200 con 0x562330dd7d70 2024-09-19T04:56:00.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.404+0000 7fe04ccbc740 10 client.27392 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:00.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.404+0000 7fe04ccbc740 10 client.27392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:56:00.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.404+0000 7fe04ccbc740 20 client.27392 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:00.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.404+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/2637354303 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562330de7900 0x562330de9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.404+0000 7fe04ccbc740 4 client.27392 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:56:00.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.404+0000 7fe04ccbc740 1 -- 172.21.15.123:0/2637354303 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562330d40950 con 0x562330de7900 2024-09-19T04:56:00.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.405+0000 7fe043400640 1 --2- 172.21.15.123:0/2637354303 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562330de7900 0x562330de9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.405+0000 7fe043400640 1 --2- 172.21.15.123:0/2637354303 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562330de7900 0x562330de9d30 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4455e894b6da851e server_cookie=ff1425218ad6f99d in_seq=0 out_seq=0 2024-09-19T04:56:00.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.406+0000 7fe030c00640 10 client.27392 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:00.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.406+0000 7fe030c00640 1 -- 172.21.15.123:0/2637354303 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562330d40950 con 0x562330de7900 2024-09-19T04:56:00.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.406+0000 7fe030c00640 10 client.27392 handle_command_reply: tid=0 2024-09-19T04:56:00.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.406+0000 7fe04ccbc740 1 client.27392 shutdown 2024-09-19T04:56:00.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.407+0000 7fe04ccbc740 1 -- 172.21.15.123:0/2637354303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dd7d70 msgr2=0x562330dd5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:00.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.407+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/2637354303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dd7d70 0x562330dd5590 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe03c098f60 tx=0x7fe03c09cd50 comp rx=0 tx=0).stop 2024-09-19T04:56:00.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.407+0000 7fe04ccbc740 1 -- 172.21.15.123:0/2637354303 shutdown_connections 2024-09-19T04:56:00.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.407+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/2637354303 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562330de7900 0x562330de9d30 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.407+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/2637354303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dd7d70 0x562330dd5590 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.407+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/2637354303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dcd4b0 0x562330dd5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.407+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/2637354303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dccb90 0x562330dd4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.407+0000 7fe04ccbc740 1 -- 172.21.15.123:0/2637354303 >> 172.21.15.123:0/2637354303 conn(0x562330dc9120 msgr2=0x562330dcee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:00.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.407+0000 7fe030c00640 0 client.27392 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:00.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.407+0000 7fe04ccbc740 1 -- 172.21.15.123:0/2637354303 shutdown_connections 2024-09-19T04:56:00.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.407+0000 7fe04ccbc740 1 -- 172.21.15.123:0/2637354303 wait complete. 2024-09-19T04:56:00.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.407+0000 7fe04ccbc740 20 client.27392 trim_cache size 0 max 16384 2024-09-19T04:56:00.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.425+0000 7fe04ccbc740 1 Processor -- start 2024-09-19T04:56:00.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.426+0000 7fe04ccbc740 1 -- start start 2024-09-19T04:56:00.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.426+0000 7fe04ccbc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dccb90 0x562330dd33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.426+0000 7fe04ccbc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dcd4b0 0x562330dd6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.426+0000 7fe04ccbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dd7d70 0x562330def650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.426+0000 7fe04ccbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0441ab680 con 0x562330dd7d70 2024-09-19T04:56:00.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.426+0000 7fe04ccbc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0441ad1c0 con 0x562330dcd4b0 2024-09-19T04:56:00.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.426+0000 7fe04ccbc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0441abd50 con 0x562330dccb90 2024-09-19T04:56:00.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.426+0000 7fe043e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dd7d70 0x562330def650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.426+0000 7fe03aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dcd4b0 0x562330dd6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.426+0000 7fe043e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dd7d70 0x562330def650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54790/0 (socket says 172.21.15.123:54790) 2024-09-19T04:56:00.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.426+0000 7fe043e00640 1 -- 172.21.15.123:0/3278547082 learned_addr learned my addr 172.21.15.123:0/3278547082 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:00.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.426+0000 7fe043400640 1 --2- 172.21.15.123:0/3278547082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dccb90 0x562330dd33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.427+0000 7fe043e00640 1 -- 172.21.15.123:0/3278547082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dccb90 msgr2=0x562330dd33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:00.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.427+0000 7fe043e00640 1 --2- 172.21.15.123:0/3278547082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dccb90 0x562330dd33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.427+0000 7fe043e00640 1 -- 172.21.15.123:0/3278547082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dcd4b0 msgr2=0x562330dd6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:00.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.427+0000 7fe043e00640 1 --2- 172.21.15.123:0/3278547082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dcd4b0 0x562330dd6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.427+0000 7fe043e00640 1 -- 172.21.15.123:0/3278547082 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562330dc8f50 con 0x562330dd7d70 2024-09-19T04:56:00.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.427+0000 7fe043e00640 1 --2- 172.21.15.123:0/3278547082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dd7d70 0x562330def650 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe03c09c290 tx=0x7fe03c0a0ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e462bc518aebe076 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:00.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.427+0000 7fe033400640 1 -- 172.21.15.123:0/3278547082 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c07c030 con 0x562330dd7d70 2024-09-19T04:56:00.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.427+0000 7fe033400640 1 -- 172.21.15.123:0/3278547082 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe03c0a4f10 con 0x562330dd7d70 2024-09-19T04:56:00.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.427+0000 7fe033400640 1 -- 172.21.15.123:0/3278547082 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c0b0c30 con 0x562330dd7d70 2024-09-19T04:56:00.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.428+0000 7fe04ccbc740 1 -- 172.21.15.123:0/3278547082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dd7d70 msgr2=0x562330def650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:00.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.428+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/3278547082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dd7d70 0x562330def650 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe03c09c290 tx=0x7fe03c0a0ac0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.428+0000 7fe04ccbc740 1 -- 172.21.15.123:0/3278547082 shutdown_connections 2024-09-19T04:56:00.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.428+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/3278547082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dd7d70 0x562330def650 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.428+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/3278547082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dcd4b0 0x562330dd6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.428+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/3278547082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dccb90 0x562330dd33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.428+0000 7fe04ccbc740 1 -- 172.21.15.123:0/3278547082 >> 172.21.15.123:0/3278547082 conn(0x562330dc9120 msgr2=0x562330dd6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:00.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.428+0000 7fe04ccbc740 1 -- 172.21.15.123:0/3278547082 shutdown_connections 2024-09-19T04:56:00.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.428+0000 7fe04ccbc740 1 -- 172.21.15.123:0/3278547082 wait complete. 2024-09-19T04:56:00.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.429+0000 7fe04ccbc740 1 -- start start 2024-09-19T04:56:00.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.429+0000 7fe04ccbc740 1 Processor -- start 2024-09-19T04:56:00.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.429+0000 7fe04ccbc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dccb90 0x562330de0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.429+0000 7fe04ccbc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dcd4b0 0x562330ddad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.430+0000 7fe04ccbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dd7d70 0x562330ddb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.430+0000 7fe04ccbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0441acaf0 con 0x562330dd7d70 2024-09-19T04:56:00.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.430+0000 7fe04ccbc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0441ac420 con 0x562330dcd4b0 2024-09-19T04:56:00.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.430+0000 7fe04ccbc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0441a96c0 con 0x562330dccb90 2024-09-19T04:56:00.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.430+0000 7fe04ccbc740 10 client.0 _ll_register_callbacks cb 0x7fe03c09c450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:00.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.430+0000 7fe03aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dcd4b0 0x562330ddad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.430+0000 7fe043400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dccb90 0x562330de0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.430+0000 7fe03aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dcd4b0 0x562330ddad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:59906/0 (socket says 172.21.15.123:59906) 2024-09-19T04:56:00.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.430+0000 7fe043e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dd7d70 0x562330ddb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.430+0000 7fe03aa00640 1 -- 172.21.15.123:0/2060252982 learned_addr learned my addr 172.21.15.123:0/2060252982 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:00.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.430+0000 7fe043400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dccb90 0x562330de0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:54608/0 (socket says 172.21.15.123:54608) 2024-09-19T04:56:00.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.430+0000 7fe043400640 1 -- 172.21.15.123:0/2060252982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dcd4b0 msgr2=0x562330ddad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:00.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.430+0000 7fe043400640 1 --2- 172.21.15.123:0/2060252982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dcd4b0 0x562330ddad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.430+0000 7fe043400640 1 -- 172.21.15.123:0/2060252982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dd7d70 msgr2=0x562330ddb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:00.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.430+0000 7fe043400640 1 --2- 172.21.15.123:0/2060252982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dd7d70 0x562330ddb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.430+0000 7fe043400640 1 -- 172.21.15.123:0/2060252982 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562330dc89d0 con 0x562330dccb90 2024-09-19T04:56:00.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.431+0000 7fe043400640 1 --2- 172.21.15.123:0/2060252982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dccb90 0x562330de0380 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe034009230 tx=0x7fe03403f070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac768d7684ab45d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:00.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.431+0000 7fe030c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:56:00.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.431+0000 7fe030c00640 1 -- 172.21.15.123:0/2060252982 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034056070 con 0x562330dccb90 2024-09-19T04:56:00.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.431+0000 7fe04ccbc740 1 -- 172.21.15.123:0/2060252982 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe03c0a4f10 con 0x562330dccb90 2024-09-19T04:56:00.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.431+0000 7fe030c00640 1 -- 172.21.15.123:0/2060252982 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fe034005240 con 0x562330dccb90 2024-09-19T04:56:00.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.431+0000 7fe030c00640 1 -- 172.21.15.123:0/2060252982 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe03405f050 con 0x562330dccb90 2024-09-19T04:56:00.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.431+0000 7fe04ccbc740 10 client.27415 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:00.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.431+0000 7fe04ccbc740 1 -- 172.21.15.123:0/2060252982 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562330dc9db0 con 0x562330dccb90 2024-09-19T04:56:00.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.431+0000 7fe030c00640 1 -- 172.21.15.123:0/2060252982 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe03c0a4f10 con 0x562330dccb90 2024-09-19T04:56:00.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.432+0000 7fe030c00640 1 -- 172.21.15.123:0/2060252982 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fe034059020 con 0x562330dccb90 2024-09-19T04:56:00.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.432+0000 7fe04ccbc740 10 client.27415 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:00.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.432+0000 7fe04ccbc740 10 client.27415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:56:00.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.432+0000 7fe04ccbc740 20 client.27415 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:00.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.432+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/2060252982 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562330e538f0 0x562330e55d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.432+0000 7fe04ccbc740 4 client.27415 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:56:00.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.432+0000 7fe04ccbc740 1 -- 172.21.15.123:0/2060252982 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562330d3ae60 con 0x562330e538f0 2024-09-19T04:56:00.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.432+0000 7fe03aa00640 1 --2- 172.21.15.123:0/2060252982 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562330e538f0 0x562330e55d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.433+0000 7fe03aa00640 1 --2- 172.21.15.123:0/2060252982 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562330e538f0 0x562330e55d10 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35bb5f70ea012414 server_cookie=a103ab966be00db5 in_seq=0 out_seq=0 2024-09-19T04:56:00.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.433+0000 7fe030c00640 10 client.27415 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:00.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.433+0000 7fe030c00640 1 -- 172.21.15.123:0/2060252982 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562330d3ae60 con 0x562330e538f0 2024-09-19T04:56:00.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.433+0000 7fe030c00640 10 client.27415 handle_command_reply: tid=0 2024-09-19T04:56:00.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.433+0000 7fe04ccbc740 1 client.27415 shutdown 2024-09-19T04:56:00.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.433+0000 7fe04ccbc740 1 -- 172.21.15.123:0/2060252982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dccb90 msgr2=0x562330de0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:00.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.433+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/2060252982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dccb90 0x562330de0380 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe034009230 tx=0x7fe03403f070 comp rx=0 tx=0).stop 2024-09-19T04:56:00.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.433+0000 7fe04ccbc740 1 -- 172.21.15.123:0/2060252982 shutdown_connections 2024-09-19T04:56:00.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.433+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/2060252982 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x562330e538f0 0x562330e55d10 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.433+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/2060252982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562330dd7d70 0x562330ddb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.433+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/2060252982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562330dcd4b0 0x562330ddad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.433+0000 7fe04ccbc740 1 --2- 172.21.15.123:0/2060252982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562330dccb90 0x562330de0380 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.433+0000 7fe030c00640 0 client.27415 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:00.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.433+0000 7fe04ccbc740 1 -- 172.21.15.123:0/2060252982 >> 172.21.15.123:0/2060252982 conn(0x562330dc9120 msgr2=0x562330dd6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:00.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.433+0000 7fe04ccbc740 1 -- 172.21.15.123:0/2060252982 shutdown_connections 2024-09-19T04:56:00.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.433+0000 7fe04ccbc740 1 -- 172.21.15.123:0/2060252982 wait complete. 2024-09-19T04:56:00.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.434+0000 7fe04ccbc740 20 client.27415 trim_cache size 0 max 16384 2024-09-19T04:56:00.438 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:56:00.438 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-09-19T04:56:00.438 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-09-19T04:56:00.439 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:56:00.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.434+0000 7fe04a200640 1 -- 172.21.15.123:0/2966449172 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe0280a33b0 msgr2=0x7fe0280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:00.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.434+0000 7fe04a200640 1 --2- 172.21.15.123:0/2966449172 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe0280a33b0 0x7fe0280a5870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x562330dc9f80 tx=0x7fe034005a20 comp rx=0 tx=0).stop 2024-09-19T04:56:00.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.434+0000 7fe04a200640 1 -- 172.21.15.123:0/2966449172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe044106f10 msgr2=0x7fe04419eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:00.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.434+0000 7fe04a200640 1 --2- 172.21.15.123:0/2966449172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe044106f10 0x7fe04419eed0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009fd0 tx=0x7fe03c00ef10 comp rx=0 tx=0).stop 2024-09-19T04:56:00.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.434+0000 7fe04a200640 1 -- 172.21.15.123:0/2966449172 shutdown_connections 2024-09-19T04:56:00.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.434+0000 7fe04a200640 1 --2- 172.21.15.123:0/2966449172 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe0280a33b0 0x7fe0280a5870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.434+0000 7fe04a200640 1 --2- 172.21.15.123:0/2966449172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe044106f10 0x7fe04419eed0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.434+0000 7fe04a200640 1 --2- 172.21.15.123:0/2966449172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe044102790 0x7fe04419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.434+0000 7fe04a200640 1 --2- 172.21.15.123:0/2966449172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe044101de0 0x7fe04419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.434+0000 7fe04a200640 1 -- 172.21.15.123:0/2966449172 >> 172.21.15.123:0/2966449172 conn(0x7fe0440fd8c0 msgr2=0x7fe0440feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:00.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.434+0000 7fe04a200640 1 -- 172.21.15.123:0/2966449172 shutdown_connections 2024-09-19T04:56:00.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:00.434+0000 7fe04a200640 1 -- 172.21.15.123:0/2966449172 wait complete. 2024-09-19T04:56:00.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:00 smithi193 ceph-mon[35592]: scrub queued for path: / 2024-09-19T04:56:00.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:00 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:56:00.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:00 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:56:00.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:00 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:56:00.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:00 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:56:00.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:00 smithi183 ceph-mon[35333]: scrub queued for path: / 2024-09-19T04:56:00.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:00 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:56:00.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:00 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:56:00.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:00 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:56:00.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:00 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:56:01.196 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:56:01.196 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:56:01.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:01 smithi123 ceph-mon[30511]: pgmap v288: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:01.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:01 smithi193 ceph-mon[35592]: pgmap v288: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:01.566 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:01.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:01 smithi183 ceph-mon[35333]: pgmap v288: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:02.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.126+0000 7fd6db800640 1 -- 172.21.15.123:0/2446202138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6d41027a0 msgr2=0x7fd6d41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:02.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.126+0000 7fd6db800640 1 --2- 172.21.15.123:0/2446202138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6d41027a0 0x7fd6d41067f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009900 tx=0x7fd6c002a410 comp rx=0 tx=0).stop 2024-09-19T04:56:02.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.126+0000 7fd6db800640 1 -- 172.21.15.123:0/2446202138 shutdown_connections 2024-09-19T04:56:02.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.126+0000 7fd6db800640 1 --2- 172.21.15.123:0/2446202138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6d4106f20 0x7fd6d410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:02.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.126+0000 7fd6db800640 1 --2- 172.21.15.123:0/2446202138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6d41027a0 0x7fd6d41067f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:02.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.126+0000 7fd6db800640 1 --2- 172.21.15.123:0/2446202138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6d4101df0 0x7fd6d41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:02.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.126+0000 7fd6db800640 1 -- 172.21.15.123:0/2446202138 >> 172.21.15.123:0/2446202138 conn(0x7fd6d40fd8b0 msgr2=0x7fd6d40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:02.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.126+0000 7fd6db800640 1 -- 172.21.15.123:0/2446202138 shutdown_connections 2024-09-19T04:56:02.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.126+0000 7fd6db800640 1 -- 172.21.15.123:0/2446202138 wait complete. 2024-09-19T04:56:02.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.127+0000 7fd6db800640 1 Processor -- start 2024-09-19T04:56:02.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.128+0000 7fd6db800640 1 -- start start 2024-09-19T04:56:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.128+0000 7fd6db800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6d4101df0 0x7fd6d419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.128+0000 7fd6db800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6d41027a0 0x7fd6d419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.128+0000 7fd6db800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6d4106f20 0x7fd6d419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.128+0000 7fd6db800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d4111f90 con 0x7fd6d41027a0 2024-09-19T04:56:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.128+0000 7fd6db800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d4111e10 con 0x7fd6d4101df0 2024-09-19T04:56:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.128+0000 7fd6db800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d4112110 con 0x7fd6d4106f20 2024-09-19T04:56:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.128+0000 7fd6d3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6d41027a0 0x7fd6d419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.128+0000 7fd6d8e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6d4101df0 0x7fd6d419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.128+0000 7fd6d9800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6d4106f20 0x7fd6d419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.128+0000 7fd6d8e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6d4101df0 0x7fd6d419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:51350/0 (socket says 172.21.15.123:51350) 2024-09-19T04:56:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.128+0000 7fd6d8e00640 1 -- 172.21.15.123:0/796401118 learned_addr learned my addr 172.21.15.123:0/796401118 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.128+0000 7fd6d3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6d41027a0 0x7fd6d419ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56562/0 (socket says 172.21.15.123:56562) 2024-09-19T04:56:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.129+0000 7fd6d8e00640 1 -- 172.21.15.123:0/796401118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6d4106f20 msgr2=0x7fd6d419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.129+0000 7fd6d8e00640 1 --2- 172.21.15.123:0/796401118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6d4106f20 0x7fd6d419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.129+0000 7fd6d8e00640 1 -- 172.21.15.123:0/796401118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6d41027a0 msgr2=0x7fd6d419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.129+0000 7fd6d8e00640 1 --2- 172.21.15.123:0/796401118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6d41027a0 0x7fd6d419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.129+0000 7fd6d8e00640 1 -- 172.21.15.123:0/796401118 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d419f5d0 con 0x7fd6d4101df0 2024-09-19T04:56:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.129+0000 7fd6d8e00640 1 --2- 172.21.15.123:0/796401118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6d4101df0 0x7fd6d419a620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800e9f0 tx=0x7fd6c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.129+0000 7fd6d1600640 1 -- 172.21.15.123:0/796401118 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c800cd80 con 0x7fd6d4101df0 2024-09-19T04:56:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.129+0000 7fd6d1600640 1 -- 172.21.15.123:0/796401118 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6c8010910 con 0x7fd6d4101df0 2024-09-19T04:56:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.129+0000 7fd6d1600640 1 -- 172.21.15.123:0/796401118 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8018990 con 0x7fd6d4101df0 2024-09-19T04:56:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.129+0000 7fd6db800640 1 -- 172.21.15.123:0/796401118 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d419f8c0 con 0x7fd6d4101df0 2024-09-19T04:56:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.130+0000 7fd6db800640 1 -- 172.21.15.123:0/796401118 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d41a71a0 con 0x7fd6d4101df0 2024-09-19T04:56:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.131+0000 7fd6db800640 1 -- 172.21.15.123:0/796401118 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d41021d0 con 0x7fd6d4101df0 2024-09-19T04:56:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.133+0000 7fd6d1600640 1 -- 172.21.15.123:0/796401118 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd6c8010ac0 con 0x7fd6d4101df0 2024-09-19T04:56:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.134+0000 7fd6d1600640 1 --2- 172.21.15.123:0/796401118 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd6bc0a33b0 0x7fd6bc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.134+0000 7fd6d1600640 1 -- 172.21.15.123:0/796401118 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fd6c80d0090 con 0x7fd6d4101df0 2024-09-19T04:56:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.134+0000 7fd6d3e00640 1 --2- 172.21.15.123:0/796401118 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd6bc0a33b0 0x7fd6bc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.135+0000 7fd6d3e00640 1 --2- 172.21.15.123:0/796401118 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd6bc0a33b0 0x7fd6bc0a5870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0000c00 tx=0x7fd6c0005b50 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:02.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.138+0000 7fd6d1600640 1 -- 172.21.15.123:0/796401118 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd6c809c3c0 con 0x7fd6d4101df0 2024-09-19T04:56:02.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.359+0000 7fd6db800640 1 -- 172.21.15.123:0/796401118 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6d41a7370 con 0x7fd6d4101df0 2024-09-19T04:56:02.361 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:02 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:02.362 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:56:02.362 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":43,"btime":"2024-09-19T04:54:44:530698+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:54:44.239974+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24529,24505,24539,24500,24499]},"id":1}]} 2024-09-19T04:56:02.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.360+0000 7fd6d1600640 1 -- 172.21.15.123:0/796401118 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11990 (secure 0 0 0) 0x7fd6c80a06c0 con 0x7fd6d4101df0 2024-09-19T04:56:02.363 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 43 2024-09-19T04:56:02.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.363+0000 7fd6db800640 1 -- 172.21.15.123:0/796401118 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd6bc0a33b0 msgr2=0x7fd6bc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:02.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.363+0000 7fd6db800640 1 --2- 172.21.15.123:0/796401118 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd6bc0a33b0 0x7fd6bc0a5870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0000c00 tx=0x7fd6c0005b50 comp rx=0 tx=0).stop 2024-09-19T04:56:02.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.363+0000 7fd6db800640 1 -- 172.21.15.123:0/796401118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6d4101df0 msgr2=0x7fd6d419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.363+0000 7fd6db800640 1 --2- 172.21.15.123:0/796401118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6d4101df0 0x7fd6d419a620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800e9f0 tx=0x7fd6c800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:56:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.363+0000 7fd6db800640 1 -- 172.21.15.123:0/796401118 shutdown_connections 2024-09-19T04:56:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.363+0000 7fd6db800640 1 --2- 172.21.15.123:0/796401118 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd6bc0a33b0 0x7fd6bc0a5870 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.363+0000 7fd6db800640 1 --2- 172.21.15.123:0/796401118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6d4106f20 0x7fd6d419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.363+0000 7fd6db800640 1 --2- 172.21.15.123:0/796401118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6d41027a0 0x7fd6d419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.363+0000 7fd6db800640 1 --2- 172.21.15.123:0/796401118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6d4101df0 0x7fd6d419a620 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.363+0000 7fd6db800640 1 -- 172.21.15.123:0/796401118 >> 172.21.15.123:0/796401118 conn(0x7fd6d40fd8b0 msgr2=0x7fd6d40feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.363+0000 7fd6db800640 1 -- 172.21.15.123:0/796401118 shutdown_connections 2024-09-19T04:56:02.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:02.363+0000 7fd6db800640 1 -- 172.21.15.123:0/796401118 wait complete. 2024-09-19T04:56:02.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:02 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:02.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:02 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:03.099 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:56:03.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:03 smithi123 ceph-mon[30511]: pgmap v289: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:03.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:03 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/796401118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:56:03.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:03 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:03.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:03 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:03.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:03 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:56:03.471 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:03.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:03 smithi183 ceph-mon[35333]: pgmap v289: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:03.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:03 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/796401118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:56:03.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:03 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:03.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:03 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:03.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:03 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:56:03.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:03 smithi193 ceph-mon[35592]: pgmap v289: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:03.746 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:03 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/796401118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:56:03.746 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:03 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:03.746 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:03 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:03.746 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:03 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:56:04.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.017+0000 7efc6ee00640 1 -- 172.21.15.123:0/2680620088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc681047e0 msgr2=0x7efc68108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:04.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.017+0000 7efc6ee00640 1 --2- 172.21.15.123:0/2680620088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc681047e0 0x7efc68108830 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7efc54009900 tx=0x7efc5402a410 comp rx=0 tx=0).stop 2024-09-19T04:56:04.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.017+0000 7efc6ee00640 1 -- 172.21.15.123:0/2680620088 shutdown_connections 2024-09-19T04:56:04.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.017+0000 7efc6ee00640 1 --2- 172.21.15.123:0/2680620088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc68108f60 0x7efc68110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.017+0000 7efc6ee00640 1 --2- 172.21.15.123:0/2680620088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc681047e0 0x7efc68108830 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.017+0000 7efc6ee00640 1 --2- 172.21.15.123:0/2680620088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc68103e30 0x7efc68104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.017+0000 7efc6ee00640 1 -- 172.21.15.123:0/2680620088 >> 172.21.15.123:0/2680620088 conn(0x7efc680ff910 msgr2=0x7efc68101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:04.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.017+0000 7efc6ee00640 1 -- 172.21.15.123:0/2680620088 shutdown_connections 2024-09-19T04:56:04.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.017+0000 7efc6ee00640 1 -- 172.21.15.123:0/2680620088 wait complete. 2024-09-19T04:56:04.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.018+0000 7efc6ee00640 1 Processor -- start 2024-09-19T04:56:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.018+0000 7efc6ee00640 1 -- start start 2024-09-19T04:56:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.018+0000 7efc6ee00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc68103e30 0x7efc6819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.018+0000 7efc6ee00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc681047e0 0x7efc6819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.018+0000 7efc6ee00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc68108f60 0x7efc6819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.018+0000 7efc6ee00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc68113fc0 con 0x7efc681047e0 2024-09-19T04:56:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.018+0000 7efc6ee00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc68113e40 con 0x7efc68103e30 2024-09-19T04:56:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.018+0000 7efc6ee00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efc68114140 con 0x7efc68108f60 2024-09-19T04:56:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.019+0000 7efc6d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc681047e0 0x7efc6819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.019+0000 7efc6da00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc68103e30 0x7efc6819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.019+0000 7efc6da00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc68103e30 0x7efc6819a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:51364/0 (socket says 172.21.15.123:51364) 2024-09-19T04:56:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.019+0000 7efc6d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc681047e0 0x7efc6819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56582/0 (socket says 172.21.15.123:56582) 2024-09-19T04:56:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.019+0000 7efc6da00640 1 -- 172.21.15.123:0/3237661407 learned_addr learned my addr 172.21.15.123:0/3237661407 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:04.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.019+0000 7efc6e400640 1 --2- 172.21.15.123:0/3237661407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc68108f60 0x7efc6819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.019+0000 7efc6d000640 1 -- 172.21.15.123:0/3237661407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc68108f60 msgr2=0x7efc6819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:04.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.019+0000 7efc6d000640 1 --2- 172.21.15.123:0/3237661407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc68108f60 0x7efc6819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.019+0000 7efc6d000640 1 -- 172.21.15.123:0/3237661407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc68103e30 msgr2=0x7efc6819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:04.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.019+0000 7efc6d000640 1 --2- 172.21.15.123:0/3237661407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc68103e30 0x7efc6819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.019+0000 7efc6d000640 1 -- 172.21.15.123:0/3237661407 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6819f4d0 con 0x7efc681047e0 2024-09-19T04:56:04.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.020+0000 7efc6d000640 1 --2- 172.21.15.123:0/3237661407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc681047e0 0x7efc6819aa60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efc54005940 tx=0x7efc540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:04.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.020+0000 7efc66000640 1 -- 172.21.15.123:0/3237661407 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5403f030 con 0x7efc681047e0 2024-09-19T04:56:04.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.020+0000 7efc66000640 1 -- 172.21.15.123:0/3237661407 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc54003ce0 con 0x7efc681047e0 2024-09-19T04:56:04.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.020+0000 7efc66000640 1 -- 172.21.15.123:0/3237661407 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54035070 con 0x7efc681047e0 2024-09-19T04:56:04.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.020+0000 7efc6ee00640 1 -- 172.21.15.123:0/3237661407 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6819f760 con 0x7efc681047e0 2024-09-19T04:56:04.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.020+0000 7efc6ee00640 1 -- 172.21.15.123:0/3237661407 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc681a70a0 con 0x7efc681047e0 2024-09-19T04:56:04.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.023+0000 7efc66000640 1 -- 172.21.15.123:0/3237661407 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7efc54042030 con 0x7efc681047e0 2024-09-19T04:56:04.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.023+0000 7efc7301e740 1 Processor -- start 2024-09-19T04:56:04.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.023+0000 7efc7301e740 1 -- start start 2024-09-19T04:56:04.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.024+0000 7efc7301e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 0x55b8e8cc6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.024+0000 7efc7301e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 0x55b8e8cd0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.024+0000 7efc7301e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cd1020 0x55b8e8cd3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.024+0000 7efc7301e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b8e8c2c3e0 con 0x55b8e8cc6760 2024-09-19T04:56:04.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.024+0000 7efc7301e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b8e8bd3d80 con 0x55b8e8cc5e40 2024-09-19T04:56:04.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.024+0000 7efc7301e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b8e8c283a0 con 0x55b8e8cd1020 2024-09-19T04:56:04.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.024+0000 7efc6da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 0x55b8e8cd0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.024+0000 7efc6da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 0x55b8e8cd0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56588/0 (socket says 172.21.15.123:56588) 2024-09-19T04:56:04.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.024+0000 7efc6da00640 1 -- 172.21.15.123:0/406637525 learned_addr learned my addr 172.21.15.123:0/406637525 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:04.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.024+0000 7efc6e400640 1 --2- 172.21.15.123:0/406637525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 0x55b8e8cc6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.024+0000 7efc6d000640 1 --2- 172.21.15.123:0/406637525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cd1020 0x55b8e8cd3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.025+0000 7efc6da00640 1 -- 172.21.15.123:0/406637525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cd1020 msgr2=0x55b8e8cd3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:04.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.025+0000 7efc6da00640 1 --2- 172.21.15.123:0/406637525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cd1020 0x55b8e8cd3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.025+0000 7efc6da00640 1 -- 172.21.15.123:0/406637525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 msgr2=0x55b8e8cc6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:04.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.025+0000 7efc6da00640 1 --2- 172.21.15.123:0/406637525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 0x55b8e8cc6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.025+0000 7efc6da00640 1 -- 172.21.15.123:0/406637525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8e8bf2ae0 con 0x55b8e8cc6760 2024-09-19T04:56:04.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.025+0000 7efc6d000640 1 --2- 172.21.15.123:0/406637525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cd1020 0x55b8e8cd3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:56:04.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.025+0000 7efc66000640 1 --2- 172.21.15.123:0/3237661407 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efc500a3430 0x7efc500a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.025+0000 7efc66000640 1 -- 172.21.15.123:0/3237661407 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7efc540f04b0 con 0x7efc681047e0 2024-09-19T04:56:04.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.025+0000 7efc6da00640 1 --2- 172.21.15.123:0/406637525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 0x55b8e8cd0ae0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00bda0 tx=0x7efc5c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c78605dd4547d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:04.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.025+0000 7efc6e400640 1 --2- 172.21.15.123:0/3237661407 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efc500a3430 0x7efc500a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.025+0000 7efc5aa00640 1 -- 172.21.15.123:0/406637525 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c01c680 con 0x55b8e8cc6760 2024-09-19T04:56:04.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.025+0000 7efc5aa00640 1 -- 172.21.15.123:0/406637525 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc5c011920 con 0x55b8e8cc6760 2024-09-19T04:56:04.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.025+0000 7efc5aa00640 1 -- 172.21.15.123:0/406637525 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c01a920 con 0x55b8e8cc6760 2024-09-19T04:56:04.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.026+0000 7efc7301e740 1 -- 172.21.15.123:0/406637525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 msgr2=0x55b8e8cd0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:04.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.026+0000 7efc6e400640 1 --2- 172.21.15.123:0/3237661407 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efc500a3430 0x7efc500a58f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efc60005fd0 tx=0x7efc600096a0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:04.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.026+0000 7efc7301e740 1 --2- 172.21.15.123:0/406637525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 0x55b8e8cd0ae0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00bda0 tx=0x7efc5c0107a0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.026+0000 7efc7301e740 1 -- 172.21.15.123:0/406637525 shutdown_connections 2024-09-19T04:56:04.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.026+0000 7efc7301e740 1 --2- 172.21.15.123:0/406637525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cd1020 0x55b8e8cd3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.026+0000 7efc7301e740 1 --2- 172.21.15.123:0/406637525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 0x55b8e8cd0ae0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.026+0000 7efc7301e740 1 --2- 172.21.15.123:0/406637525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 0x55b8e8cc6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.026+0000 7efc7301e740 1 -- 172.21.15.123:0/406637525 >> 172.21.15.123:0/406637525 conn(0x55b8e8cc23d0 msgr2=0x55b8e8cc27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:04.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.026+0000 7efc7301e740 1 -- 172.21.15.123:0/406637525 shutdown_connections 2024-09-19T04:56:04.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.027+0000 7efc7301e740 1 -- 172.21.15.123:0/406637525 wait complete. 2024-09-19T04:56:04.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.027+0000 7efc7301e740 1 -- start start 2024-09-19T04:56:04.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.027+0000 7efc7301e740 1 Processor -- start 2024-09-19T04:56:04.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc7301e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cc5e40 0x55b8e8ccdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc7301e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 0x55b8e8cce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc7301e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cd1020 0x55b8e8cce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc7301e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b8e8cd3bf0 con 0x55b8e8cc6760 2024-09-19T04:56:04.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc7301e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b8e8be70f0 con 0x55b8e8cd1020 2024-09-19T04:56:04.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc7301e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b8e8cd3d70 con 0x55b8e8cc5e40 2024-09-19T04:56:04.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc7301e740 10 client.0 _ll_register_callbacks cb 0x55b8e8bcebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:04.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc6d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 0x55b8e8cce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc6da00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cc5e40 0x55b8e8ccdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc6da00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cc5e40 0x55b8e8ccdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:44380/0 (socket says 172.21.15.123:44380) 2024-09-19T04:56:04.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc6da00640 1 -- 172.21.15.123:0/2778345581 learned_addr learned my addr 172.21.15.123:0/2778345581 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:04.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc6d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 0x55b8e8cce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56602/0 (socket says 172.21.15.123:56602) 2024-09-19T04:56:04.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc6e400640 1 --2- 172.21.15.123:0/2778345581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cd1020 0x55b8e8cce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc6d000640 1 -- 172.21.15.123:0/2778345581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cc5e40 msgr2=0x55b8e8ccdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:04.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc6d000640 1 --2- 172.21.15.123:0/2778345581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cc5e40 0x55b8e8ccdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc6d000640 1 -- 172.21.15.123:0/2778345581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cd1020 msgr2=0x55b8e8cce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:04.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc6d000640 1 --2- 172.21.15.123:0/2778345581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cd1020 0x55b8e8cce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.028+0000 7efc6d000640 1 -- 172.21.15.123:0/2778345581 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8e899d3c0 con 0x55b8e8cc6760 2024-09-19T04:56:04.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.029+0000 7efc6d000640 1 --2- 172.21.15.123:0/2778345581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 0x55b8e8cce3f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7efc540edba0 tx=0x7efc540b8360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b94a1aa47ca9b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:04.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.029+0000 7efc4f400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:56:04.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.029+0000 7efc4f400640 1 -- 172.21.15.123:0/2778345581 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54049050 con 0x55b8e8cc6760 2024-09-19T04:56:04.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.029+0000 7efc4f400640 1 -- 172.21.15.123:0/2778345581 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7efc5403bc80 con 0x55b8e8cc6760 2024-09-19T04:56:04.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.029+0000 7efc7301e740 1 -- 172.21.15.123:0/2778345581 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc5c011920 con 0x55b8e8cc6760 2024-09-19T04:56:04.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.029+0000 7efc4f400640 1 -- 172.21.15.123:0/2778345581 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7efc540bc320 con 0x55b8e8cc6760 2024-09-19T04:56:04.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.029+0000 7efc7301e740 10 client.17325 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:04.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.029+0000 7efc7301e740 1 -- 172.21.15.123:0/2778345581 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b8e8cdfb30 con 0x55b8e8cc6760 2024-09-19T04:56:04.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.030+0000 7efc4f400640 1 -- 172.21.15.123:0/2778345581 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc5c011920 con 0x55b8e8cc6760 2024-09-19T04:56:04.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.030+0000 7efc4f400640 1 -- 172.21.15.123:0/2778345581 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7efc540ef070 con 0x55b8e8cc6760 2024-09-19T04:56:04.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.030+0000 7efc7301e740 10 client.17325 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:04.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.030+0000 7efc7301e740 10 client.17325 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:56:04.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.030+0000 7efc7301e740 20 client.17325 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:04.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.030+0000 7efc7301e740 1 --2- 172.21.15.123:0/2778345581 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b8e8ce0ef0 0x55b8e8ce3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.030+0000 7efc7301e740 4 client.17325 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:56:04.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.030+0000 7efc7301e740 1 -- 172.21.15.123:0/2778345581 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8e8c39950 con 0x55b8e8ce0ef0 2024-09-19T04:56:04.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.030+0000 7efc6da00640 1 --2- 172.21.15.123:0/2778345581 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b8e8ce0ef0 0x55b8e8ce3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.031+0000 7efc6da00640 1 --2- 172.21.15.123:0/2778345581 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b8e8ce0ef0 0x55b8e8ce3300 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=55b7d32ad83369f4 server_cookie=b7464d4573dfafc6 in_seq=0 out_seq=0 2024-09-19T04:56:04.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.031+0000 7efc4f400640 10 client.17325 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:56:04.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.032+0000 7efc4f400640 1 -- 172.21.15.123:0/2778345581 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8e8c39950 con 0x55b8e8ce0ef0 2024-09-19T04:56:04.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.032+0000 7efc4f400640 10 client.17325 handle_command_reply: tid=0 2024-09-19T04:56:04.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.032+0000 7efc7301e740 1 client.17325 shutdown 2024-09-19T04:56:04.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.032+0000 7efc7301e740 1 -- 172.21.15.123:0/2778345581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 msgr2=0x55b8e8cce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:04.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.032+0000 7efc7301e740 1 --2- 172.21.15.123:0/2778345581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 0x55b8e8cce3f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7efc540edba0 tx=0x7efc540b8360 comp rx=0 tx=0).stop 2024-09-19T04:56:04.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.032+0000 7efc7301e740 1 -- 172.21.15.123:0/2778345581 shutdown_connections 2024-09-19T04:56:04.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.032+0000 7efc7301e740 1 --2- 172.21.15.123:0/2778345581 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b8e8ce0ef0 0x55b8e8ce3300 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.032+0000 7efc7301e740 1 --2- 172.21.15.123:0/2778345581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cd1020 0x55b8e8cce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.032+0000 7efc4f400640 0 client.17325 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:56:04.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.032+0000 7efc7301e740 1 --2- 172.21.15.123:0/2778345581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 0x55b8e8cce3f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.032+0000 7efc7301e740 1 --2- 172.21.15.123:0/2778345581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cc5e40 0x55b8e8ccdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.032+0000 7efc7301e740 1 -- 172.21.15.123:0/2778345581 >> 172.21.15.123:0/2778345581 conn(0x55b8e8cc23d0 msgr2=0x55b8e8cc8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:04.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.032+0000 7efc7301e740 1 -- 172.21.15.123:0/2778345581 shutdown_connections 2024-09-19T04:56:04.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.032+0000 7efc7301e740 1 -- 172.21.15.123:0/2778345581 wait complete. 2024-09-19T04:56:04.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.033+0000 7efc7301e740 20 client.17325 trim_cache size 0 max 16384 2024-09-19T04:56:04.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.046+0000 7efc7301e740 1 Processor -- start 2024-09-19T04:56:04.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.046+0000 7efc7301e740 1 -- start start 2024-09-19T04:56:04.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.047+0000 7efc7301e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 0x55b8e8ccfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.047+0000 7efc7301e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 0x55b8e8ce8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.047+0000 7efc7301e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cd1020 0x55b8e8ce9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.047+0000 7efc7301e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc681af890 con 0x55b8e8cc6760 2024-09-19T04:56:04.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.047+0000 7efc7301e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc681b13d0 con 0x55b8e8cc5e40 2024-09-19T04:56:04.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.047+0000 7efc7301e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efc681aff60 con 0x55b8e8cd1020 2024-09-19T04:56:04.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.047+0000 7efc6da00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 0x55b8e8ccfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.047+0000 7efc6d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 0x55b8e8ce8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.047+0000 7efc6da00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 0x55b8e8ccfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:51402/0 (socket says 172.21.15.123:51402) 2024-09-19T04:56:04.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.047+0000 7efc6da00640 1 -- 172.21.15.123:0/359438347 learned_addr learned my addr 172.21.15.123:0/359438347 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:04.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.047+0000 7efc6e400640 1 --2- 172.21.15.123:0/359438347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cd1020 0x55b8e8ce9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.047+0000 7efc6da00640 1 -- 172.21.15.123:0/359438347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cd1020 msgr2=0x55b8e8ce9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:04.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.047+0000 7efc6da00640 1 --2- 172.21.15.123:0/359438347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cd1020 0x55b8e8ce9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.047+0000 7efc6da00640 1 -- 172.21.15.123:0/359438347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 msgr2=0x55b8e8ce8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:04.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.047+0000 7efc6da00640 1 --2- 172.21.15.123:0/359438347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 0x55b8e8ce8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.047+0000 7efc6da00640 1 -- 172.21.15.123:0/359438347 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8e8cc2200 con 0x55b8e8cc5e40 2024-09-19T04:56:04.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.047+0000 7efc6d000640 1 --2- 172.21.15.123:0/359438347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 0x55b8e8ce8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:04.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.048+0000 7efc6da00640 1 --2- 172.21.15.123:0/359438347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 0x55b8e8ccfe20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efc5c01a530 tx=0x7efc5c01a4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbd9a5496c81d5bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:04.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.048+0000 7efc5a000640 1 -- 172.21.15.123:0/359438347 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c011440 con 0x55b8e8cc5e40 2024-09-19T04:56:04.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.048+0000 7efc5a000640 1 -- 172.21.15.123:0/359438347 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc5c005040 con 0x55b8e8cc5e40 2024-09-19T04:56:04.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.048+0000 7efc5a000640 1 -- 172.21.15.123:0/359438347 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c003850 con 0x55b8e8cc5e40 2024-09-19T04:56:04.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.048+0000 7efc7301e740 1 -- 172.21.15.123:0/359438347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 msgr2=0x55b8e8ccfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:04.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.048+0000 7efc7301e740 1 --2- 172.21.15.123:0/359438347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 0x55b8e8ccfe20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efc5c01a530 tx=0x7efc5c01a4c0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.049+0000 7efc7301e740 1 -- 172.21.15.123:0/359438347 shutdown_connections 2024-09-19T04:56:04.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.049+0000 7efc7301e740 1 --2- 172.21.15.123:0/359438347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cd1020 0x55b8e8ce9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.049+0000 7efc7301e740 1 --2- 172.21.15.123:0/359438347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cc6760 0x55b8e8ce8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.049+0000 7efc7301e740 1 --2- 172.21.15.123:0/359438347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 0x55b8e8ccfe20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.049+0000 7efc7301e740 1 -- 172.21.15.123:0/359438347 >> 172.21.15.123:0/359438347 conn(0x55b8e8cc23d0 msgr2=0x55b8e8ccf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:04.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.049+0000 7efc7301e740 1 -- 172.21.15.123:0/359438347 shutdown_connections 2024-09-19T04:56:04.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.049+0000 7efc7301e740 1 -- 172.21.15.123:0/359438347 wait complete. 2024-09-19T04:56:04.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.049+0000 7efc7301e740 1 -- start start 2024-09-19T04:56:04.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.049+0000 7efc7301e740 1 Processor -- start 2024-09-19T04:56:04.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.050+0000 7efc7301e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 0x55b8e8cd91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.050+0000 7efc7301e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cc6760 0x55b8e8cd96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.050+0000 7efc7301e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cd1020 0x55b8e8cd5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.050+0000 7efc7301e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc681b0d00 con 0x55b8e8cd1020 2024-09-19T04:56:04.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.050+0000 7efc7301e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc681b0630 con 0x55b8e8cc5e40 2024-09-19T04:56:04.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.050+0000 7efc7301e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efc681ad890 con 0x55b8e8cc6760 2024-09-19T04:56:04.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.050+0000 7efc7301e740 10 client.0 _ll_register_callbacks cb 0x7efc54009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:04.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.050+0000 7efc6e400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cd1020 0x55b8e8cd5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.050+0000 7efc6e400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cd1020 0x55b8e8cd5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56634/0 (socket says 172.21.15.123:56634) 2024-09-19T04:56:04.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.050+0000 7efc6e400640 1 -- 172.21.15.123:0/2515381886 learned_addr learned my addr 172.21.15.123:0/2515381886 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:04.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.050+0000 7efc6d000640 1 --2- 172.21.15.123:0/2515381886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cc6760 0x55b8e8cd96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.050+0000 7efc6da00640 1 --2- 172.21.15.123:0/2515381886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 0x55b8e8cd91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.051+0000 7efc6da00640 1 -- 172.21.15.123:0/2515381886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cc6760 msgr2=0x55b8e8cd96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:04.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.051+0000 7efc6da00640 1 --2- 172.21.15.123:0/2515381886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cc6760 0x55b8e8cd96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.051+0000 7efc6da00640 1 -- 172.21.15.123:0/2515381886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cd1020 msgr2=0x55b8e8cd5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:04.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.051+0000 7efc6da00640 1 --2- 172.21.15.123:0/2515381886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cd1020 0x55b8e8cd5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.051+0000 7efc6da00640 1 -- 172.21.15.123:0/2515381886 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8e8cc1c80 con 0x55b8e8cc5e40 2024-09-19T04:56:04.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.051+0000 7efc6d000640 1 --2- 172.21.15.123:0/2515381886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cc6760 0x55b8e8cd96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:04.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.051+0000 7efc6da00640 1 --2- 172.21.15.123:0/2515381886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 0x55b8e8cd91a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efc5c01a530 tx=0x7efc5c005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c39e5fd496e98fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:04.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.051+0000 7efc4f400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:56:04.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.051+0000 7efc4f400640 1 -- 172.21.15.123:0/2515381886 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c011c70 con 0x55b8e8cc5e40 2024-09-19T04:56:04.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.051+0000 7efc7301e740 1 -- 172.21.15.123:0/2515381886 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc5c005040 con 0x55b8e8cc5e40 2024-09-19T04:56:04.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.051+0000 7efc4f400640 1 -- 172.21.15.123:0/2515381886 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7efc5c0051f0 con 0x55b8e8cc5e40 2024-09-19T04:56:04.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.052+0000 7efc4f400640 1 -- 172.21.15.123:0/2515381886 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7efc5c005410 con 0x55b8e8cc5e40 2024-09-19T04:56:04.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.052+0000 7efc7301e740 10 client.27422 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:04.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.052+0000 7efc7301e740 1 -- 172.21.15.123:0/2515381886 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b8e8cc8cb0 con 0x55b8e8cc5e40 2024-09-19T04:56:04.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.052+0000 7efc4f400640 1 -- 172.21.15.123:0/2515381886 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc5c005040 con 0x55b8e8cc5e40 2024-09-19T04:56:04.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.052+0000 7efc4f400640 1 -- 172.21.15.123:0/2515381886 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7efc5c01a8c0 con 0x55b8e8cc5e40 2024-09-19T04:56:04.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.052+0000 7efc7301e740 10 client.27422 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:04.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.052+0000 7efc7301e740 10 client.27422 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:56:04.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.052+0000 7efc7301e740 20 client.27422 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:04.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.052+0000 7efc7301e740 1 --2- 172.21.15.123:0/2515381886 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b8e8d4c650 0x55b8e8d4ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:04.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.052+0000 7efc7301e740 4 client.27422 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:56:04.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.052+0000 7efc7301e740 1 -- 172.21.15.123:0/2515381886 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8e8c33e60 con 0x55b8e8d4c650 2024-09-19T04:56:04.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.053+0000 7efc6d000640 1 --2- 172.21.15.123:0/2515381886 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b8e8d4c650 0x55b8e8d4ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:04.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.053+0000 7efc6d000640 1 --2- 172.21.15.123:0/2515381886 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b8e8d4c650 0x55b8e8d4ea40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=395eacf687e0e0e6 server_cookie=308b1d443830f6bf in_seq=0 out_seq=0 2024-09-19T04:56:04.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.053+0000 7efc4f400640 10 client.27422 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:56:04.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.053+0000 7efc4f400640 1 -- 172.21.15.123:0/2515381886 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8e8c33e60 con 0x55b8e8d4c650 2024-09-19T04:56:04.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.053+0000 7efc4f400640 10 client.27422 handle_command_reply: tid=0 2024-09-19T04:56:04.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.053+0000 7efc7301e740 1 client.27422 shutdown 2024-09-19T04:56:04.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.054+0000 7efc7301e740 1 -- 172.21.15.123:0/2515381886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 msgr2=0x55b8e8cd91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:04.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.054+0000 7efc7301e740 1 --2- 172.21.15.123:0/2515381886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 0x55b8e8cd91a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efc5c01a530 tx=0x7efc5c005fa0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.054+0000 7efc7301e740 1 -- 172.21.15.123:0/2515381886 shutdown_connections 2024-09-19T04:56:04.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.054+0000 7efc7301e740 1 --2- 172.21.15.123:0/2515381886 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x55b8e8d4c650 0x55b8e8d4ea40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.054+0000 7efc7301e740 1 --2- 172.21.15.123:0/2515381886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8e8cd1020 0x55b8e8cd5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.054+0000 7efc7301e740 1 --2- 172.21.15.123:0/2515381886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8e8cc6760 0x55b8e8cd96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.054+0000 7efc7301e740 1 --2- 172.21.15.123:0/2515381886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e8cc5e40 0x55b8e8cd91a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.054+0000 7efc4f400640 0 client.27422 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:56:04.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.054+0000 7efc7301e740 1 -- 172.21.15.123:0/2515381886 >> 172.21.15.123:0/2515381886 conn(0x55b8e8cc23d0 msgr2=0x55b8e8ccc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:04.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.054+0000 7efc7301e740 1 -- 172.21.15.123:0/2515381886 shutdown_connections 2024-09-19T04:56:04.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.054+0000 7efc7301e740 1 -- 172.21.15.123:0/2515381886 wait complete. 2024-09-19T04:56:04.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.054+0000 7efc7301e740 20 client.27422 trim_cache size 0 max 16384 2024-09-19T04:56:04.059 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:56:04.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.055+0000 7efc6ee00640 1 -- 172.21.15.123:0/3237661407 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efc500a3430 msgr2=0x7efc500a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:04.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.055+0000 7efc6ee00640 1 --2- 172.21.15.123:0/3237661407 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efc500a3430 0x7efc500a58f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efc60005fd0 tx=0x7efc600096a0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.055+0000 7efc6ee00640 1 -- 172.21.15.123:0/3237661407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc681047e0 msgr2=0x7efc6819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:04.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.055+0000 7efc6ee00640 1 --2- 172.21.15.123:0/3237661407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc681047e0 0x7efc6819aa60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efc54005940 tx=0x7efc540042f0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.055+0000 7efc6ee00640 1 -- 172.21.15.123:0/3237661407 shutdown_connections 2024-09-19T04:56:04.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.055+0000 7efc6ee00640 1 --2- 172.21.15.123:0/3237661407 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7efc500a3430 0x7efc500a58f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.055+0000 7efc6ee00640 1 --2- 172.21.15.123:0/3237661407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc68108f60 0x7efc6819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.055+0000 7efc6ee00640 1 --2- 172.21.15.123:0/3237661407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc681047e0 0x7efc6819aa60 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.055+0000 7efc6ee00640 1 --2- 172.21.15.123:0/3237661407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc68103e30 0x7efc6819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:04.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.055+0000 7efc6ee00640 1 -- 172.21.15.123:0/3237661407 >> 172.21.15.123:0/3237661407 conn(0x7efc680ff910 msgr2=0x7efc68100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:04.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.055+0000 7efc6ee00640 1 -- 172.21.15.123:0/3237661407 shutdown_connections 2024-09-19T04:56:04.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:04.055+0000 7efc6ee00640 1 -- 172.21.15.123:0/3237661407 wait complete. 2024-09-19T04:56:04.941 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:56:05.312 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:05.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:05 smithi183 ceph-mon[35333]: pgmap v290: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-19T04:56:05.612 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:05 smithi123 ceph-mon[30511]: pgmap v290: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-19T04:56:05.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:05 smithi193 ceph-mon[35592]: pgmap v290: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-19T04:56:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.882+0000 7f6b82200640 1 -- 172.21.15.123:0/3308808540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b7c102790 msgr2=0x7f6b7c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.882+0000 7f6b82200640 1 --2- 172.21.15.123:0/3308808540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b7c102790 0x7f6b7c1067e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009900 tx=0x7f6b6402a410 comp rx=0 tx=0).stop 2024-09-19T04:56:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.882+0000 7f6b82200640 1 -- 172.21.15.123:0/3308808540 shutdown_connections 2024-09-19T04:56:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.882+0000 7f6b82200640 1 --2- 172.21.15.123:0/3308808540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b7c106f10 0x7f6b7c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.882+0000 7f6b82200640 1 --2- 172.21.15.123:0/3308808540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b7c102790 0x7f6b7c1067e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.882+0000 7f6b82200640 1 --2- 172.21.15.123:0/3308808540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b7c101de0 0x7f6b7c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.882+0000 7f6b82200640 1 -- 172.21.15.123:0/3308808540 >> 172.21.15.123:0/3308808540 conn(0x7f6b7c0fd8c0 msgr2=0x7f6b7c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.882+0000 7f6b82200640 1 -- 172.21.15.123:0/3308808540 shutdown_connections 2024-09-19T04:56:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.882+0000 7f6b82200640 1 -- 172.21.15.123:0/3308808540 wait complete. 2024-09-19T04:56:05.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.883+0000 7f6b82200640 1 Processor -- start 2024-09-19T04:56:05.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.884+0000 7f6b82200640 1 -- start start 2024-09-19T04:56:05.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.884+0000 7f6b82200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b7c101de0 0x7f6b7c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.884+0000 7f6b82200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b7c102790 0x7f6b7c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.884+0000 7f6b82200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b7c106f10 0x7f6b7c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.884+0000 7f6b82200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c111f70 con 0x7f6b7c101de0 2024-09-19T04:56:05.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.884+0000 7f6b82200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c111df0 con 0x7f6b7c106f10 2024-09-19T04:56:05.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.884+0000 7f6b82200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1120f0 con 0x7f6b7c102790 2024-09-19T04:56:05.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.884+0000 7f6b7b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b7c101de0 0x7f6b7c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.884+0000 7f6b7b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b7c101de0 0x7f6b7c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56648/0 (socket says 172.21.15.123:56648) 2024-09-19T04:56:05.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.884+0000 7f6b7b400640 1 -- 172.21.15.123:0/794360599 learned_addr learned my addr 172.21.15.123:0/794360599 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:05.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.884+0000 7f6b72a00640 1 --2- 172.21.15.123:0/794360599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b7c102790 0x7f6b7c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.884+0000 7f6b7be00640 1 --2- 172.21.15.123:0/794360599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b7c106f10 0x7f6b7c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.885+0000 7f6b7b400640 1 -- 172.21.15.123:0/794360599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b7c102790 msgr2=0x7f6b7c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:05.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.885+0000 7f6b7b400640 1 --2- 172.21.15.123:0/794360599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b7c102790 0x7f6b7c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.885+0000 7f6b7b400640 1 -- 172.21.15.123:0/794360599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b7c106f10 msgr2=0x7f6b7c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:05.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.885+0000 7f6b7b400640 1 --2- 172.21.15.123:0/794360599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b7c106f10 0x7f6b7c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.885+0000 7f6b7b400640 1 -- 172.21.15.123:0/794360599 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7c19f620 con 0x7f6b7c101de0 2024-09-19T04:56:05.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.885+0000 7f6b7b400640 1 --2- 172.21.15.123:0/794360599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b7c101de0 0x7f6b7c19a670 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00e9f0 tx=0x7f6b6c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:05.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.885+0000 7f6b78c00640 1 -- 172.21.15.123:0/794360599 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c00cd80 con 0x7f6b7c101de0 2024-09-19T04:56:05.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.885+0000 7f6b78c00640 1 -- 172.21.15.123:0/794360599 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b6c005500 con 0x7f6b7c101de0 2024-09-19T04:56:05.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.885+0000 7f6b82200640 1 -- 172.21.15.123:0/794360599 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7c19f910 con 0x7f6b7c101de0 2024-09-19T04:56:05.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.885+0000 7f6b78c00640 1 -- 172.21.15.123:0/794360599 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c010690 con 0x7f6b7c101de0 2024-09-19T04:56:05.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.885+0000 7f6b82200640 1 -- 172.21.15.123:0/794360599 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b7c1a71f0 con 0x7f6b7c101de0 2024-09-19T04:56:05.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.888+0000 7f6b84c4b740 1 Processor -- start 2024-09-19T04:56:05.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.889+0000 7f6b78c00640 1 -- 172.21.15.123:0/794360599 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6b6c0046c0 con 0x7f6b7c101de0 2024-09-19T04:56:05.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.889+0000 7f6b84c4b740 1 -- start start 2024-09-19T04:56:05.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.890+0000 7f6b84c4b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x556065158050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.890+0000 7f6b84c4b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065158590 0x556065162910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.890+0000 7f6b84c4b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065162e50 0x556065165240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.890+0000 7f6b84c4b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5560650be3e0 con 0x556065157c70 2024-09-19T04:56:05.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.890+0000 7f6b84c4b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556065065d80 con 0x556065158590 2024-09-19T04:56:05.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.890+0000 7f6b84c4b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5560650ba3a0 con 0x556065162e50 2024-09-19T04:56:05.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.890+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x556065158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.890+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x556065158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56662/0 (socket says 172.21.15.123:56662) 2024-09-19T04:56:05.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.890+0000 7f6b7be00640 1 -- 172.21.15.123:0/3626488662 learned_addr learned my addr 172.21.15.123:0/3626488662 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:05.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.890+0000 7f6b78c00640 1 --2- 172.21.15.123:0/794360599 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6b600a34b0 0x7f6b600a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.890+0000 7f6b7b400640 1 --2- 172.21.15.123:0/3626488662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065158590 0x556065162910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.890+0000 7f6b72a00640 1 --2- 172.21.15.123:0/3626488662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065162e50 0x556065165240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.891+0000 7f6b78c00640 1 -- 172.21.15.123:0/794360599 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f6b6c0d0100 con 0x7f6b7c101de0 2024-09-19T04:56:05.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.891+0000 7f6b7b400640 1 -- 172.21.15.123:0/3626488662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065162e50 msgr2=0x556065165240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:05.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.891+0000 7f6b7b400640 1 --2- 172.21.15.123:0/3626488662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065162e50 0x556065165240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.891+0000 7f6b7b400640 1 -- 172.21.15.123:0/3626488662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 msgr2=0x556065158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:05.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.891+0000 7f6b7b400640 1 --2- 172.21.15.123:0/3626488662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x556065158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.891+0000 7f6b7b400640 1 -- 172.21.15.123:0/3626488662 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556065084ae0 con 0x556065158590 2024-09-19T04:56:05.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.891+0000 7f6b7be00640 1 --2- 172.21.15.123:0/3626488662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x556065158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:05.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.891+0000 7f6b7be00640 1 --2- 172.21.15.123:0/794360599 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6b600a34b0 0x7f6b600a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.891+0000 7f6b7b400640 1 --2- 172.21.15.123:0/3626488662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065158590 0x556065162910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0cd7f0 tx=0x7f6b6c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b724de934e909a30 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:05.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.891+0000 7f6b6be00640 1 -- 172.21.15.123:0/3626488662 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c09ca80 con 0x556065158590 2024-09-19T04:56:05.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.891+0000 7f6b6be00640 1 -- 172.21.15.123:0/3626488662 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b6c09c380 con 0x556065158590 2024-09-19T04:56:05.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.891+0000 7f6b6be00640 1 -- 172.21.15.123:0/3626488662 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c0d20d0 con 0x556065158590 2024-09-19T04:56:05.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.892+0000 7f6b84c4b740 1 -- 172.21.15.123:0/3626488662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065158590 msgr2=0x556065162910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:05.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.892+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/3626488662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065158590 0x556065162910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0cd7f0 tx=0x7f6b6c004a40 comp rx=0 tx=0).stop 2024-09-19T04:56:05.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.892+0000 7f6b7be00640 1 --2- 172.21.15.123:0/794360599 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6b600a34b0 0x7f6b600a5970 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009fd0 tx=0x7f6b74004a40 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:05.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.892+0000 7f6b84c4b740 1 -- 172.21.15.123:0/3626488662 shutdown_connections 2024-09-19T04:56:05.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.892+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/3626488662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065162e50 0x556065165240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.892+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/3626488662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065158590 0x556065162910 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.892+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/3626488662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x556065158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.892+0000 7f6b84c4b740 1 -- 172.21.15.123:0/3626488662 >> 172.21.15.123:0/3626488662 conn(0x556065154200 msgr2=0x5560651545e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:05.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.892+0000 7f6b84c4b740 1 -- 172.21.15.123:0/3626488662 shutdown_connections 2024-09-19T04:56:05.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.893+0000 7f6b84c4b740 1 -- 172.21.15.123:0/3626488662 wait complete. 2024-09-19T04:56:05.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.893+0000 7f6b84c4b740 1 -- start start 2024-09-19T04:56:05.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.893+0000 7f6b84c4b740 1 Processor -- start 2024-09-19T04:56:05.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b84c4b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065157c70 0x55606515fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b84c4b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065158590 0x5560651602d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b84c4b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065162e50 0x556065160810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b84c4b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5560651659c0 con 0x556065158590 2024-09-19T04:56:05.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b84c4b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5560650790f0 con 0x556065162e50 2024-09-19T04:56:05.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b84c4b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556065165b40 con 0x556065157c70 2024-09-19T04:56:05.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b84c4b740 10 client.0 _ll_register_callbacks cb 0x556065060bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:05.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b72a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065158590 0x5560651602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b7b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065157c70 0x55606515fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b7b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065157c70 0x55606515fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:44444/0 (socket says 172.21.15.123:44444) 2024-09-19T04:56:05.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b72a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065158590 0x5560651602d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56676/0 (socket says 172.21.15.123:56676) 2024-09-19T04:56:05.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b7b400640 1 -- 172.21.15.123:0/2361919112 learned_addr learned my addr 172.21.15.123:0/2361919112 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:05.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b7be00640 1 --2- 172.21.15.123:0/2361919112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065162e50 0x556065160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b7b400640 1 -- 172.21.15.123:0/2361919112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065162e50 msgr2=0x556065160810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:05.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b7b400640 1 --2- 172.21.15.123:0/2361919112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065162e50 0x556065160810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b7b400640 1 -- 172.21.15.123:0/2361919112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065158590 msgr2=0x5560651602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:05.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b7b400640 1 --2- 172.21.15.123:0/2361919112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065158590 0x5560651602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.894+0000 7f6b7b400640 1 -- 172.21.15.123:0/2361919112 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556064e2f3c0 con 0x556065157c70 2024-09-19T04:56:05.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.895+0000 7f6b7b400640 1 --2- 172.21.15.123:0/2361919112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065157c70 0x55606515fd90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c004c00 tx=0x7f6b6c098620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c01c8cda246eaef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:05.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.895+0000 7f6b68c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:56:05.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.895+0000 7f6b68c00640 1 -- 172.21.15.123:0/2361919112 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c0a0730 con 0x556065157c70 2024-09-19T04:56:05.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.895+0000 7f6b84c4b740 1 -- 172.21.15.123:0/2361919112 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b6c09c380 con 0x556065157c70 2024-09-19T04:56:05.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.895+0000 7f6b68c00640 1 -- 172.21.15.123:0/2361919112 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f6b6c0a4d70 con 0x556065157c70 2024-09-19T04:56:05.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.895+0000 7f6b68c00640 1 -- 172.21.15.123:0/2361919112 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6b6c09ca80 con 0x556065157c70 2024-09-19T04:56:05.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.895+0000 7f6b84c4b740 10 client.27448 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:05.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.895+0000 7f6b84c4b740 1 -- 172.21.15.123:0/2361919112 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5560651659c0 con 0x556065157c70 2024-09-19T04:56:05.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.896+0000 7f6b68c00640 1 -- 172.21.15.123:0/2361919112 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b6c09c380 con 0x556065157c70 2024-09-19T04:56:05.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.896+0000 7f6b68c00640 1 -- 172.21.15.123:0/2361919112 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f6b6c09b3d0 con 0x556065157c70 2024-09-19T04:56:05.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.896+0000 7f6b84c4b740 10 client.27448 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:05.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.896+0000 7f6b84c4b740 10 client.27448 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:56:05.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.896+0000 7f6b84c4b740 20 client.27448 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:05.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.896+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/2361919112 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x556065172bd0 0x556065174fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.896+0000 7f6b84c4b740 4 client.27448 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:56:05.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.896+0000 7f6b84c4b740 1 -- 172.21.15.123:0/2361919112 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560650cb950 con 0x556065172bd0 2024-09-19T04:56:05.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.896+0000 7f6b72a00640 1 --2- 172.21.15.123:0/2361919112 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x556065172bd0 0x556065174fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.897+0000 7f6b72a00640 1 --2- 172.21.15.123:0/2361919112 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x556065172bd0 0x556065174fc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a46afabd453de779 server_cookie=4cf0d4e97a925d84 in_seq=0 out_seq=0 2024-09-19T04:56:05.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.897+0000 7f6b68c00640 10 client.27448 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:56:05.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.898+0000 7f6b68c00640 1 -- 172.21.15.123:0/2361919112 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560650cb950 con 0x556065172bd0 2024-09-19T04:56:05.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.898+0000 7f6b68c00640 10 client.27448 handle_command_reply: tid=0 2024-09-19T04:56:05.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.898+0000 7f6b84c4b740 1 client.27448 shutdown 2024-09-19T04:56:05.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.898+0000 7f6b84c4b740 1 -- 172.21.15.123:0/2361919112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065157c70 msgr2=0x55606515fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:05.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.898+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/2361919112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065157c70 0x55606515fd90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c004c00 tx=0x7f6b6c098620 comp rx=0 tx=0).stop 2024-09-19T04:56:05.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.898+0000 7f6b84c4b740 1 -- 172.21.15.123:0/2361919112 shutdown_connections 2024-09-19T04:56:05.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.898+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/2361919112 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x556065172bd0 0x556065174fc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.898+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/2361919112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065162e50 0x556065160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.898+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/2361919112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065158590 0x5560651602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.898+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/2361919112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065157c70 0x55606515fd90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.898+0000 7f6b68c00640 0 client.27448 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:56:05.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.898+0000 7f6b84c4b740 1 -- 172.21.15.123:0/2361919112 >> 172.21.15.123:0/2361919112 conn(0x556065154200 msgr2=0x556065159f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:05.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.898+0000 7f6b84c4b740 1 -- 172.21.15.123:0/2361919112 shutdown_connections 2024-09-19T04:56:05.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.899+0000 7f6b84c4b740 1 -- 172.21.15.123:0/2361919112 wait complete. 2024-09-19T04:56:05.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.899+0000 7f6b84c4b740 20 client.27448 trim_cache size 0 max 16384 2024-09-19T04:56:05.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.912+0000 7f6b84c4b740 1 Processor -- start 2024-09-19T04:56:05.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.912+0000 7f6b84c4b740 1 -- start start 2024-09-19T04:56:05.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.913+0000 7f6b84c4b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x556065153270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.913+0000 7f6b84c4b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065158590 0x55606517a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.913+0000 7f6b84c4b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065162e50 0x55606517ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.913+0000 7f6b84c4b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1afa70 con 0x556065157c70 2024-09-19T04:56:05.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.913+0000 7f6b84c4b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1b15b0 con 0x556065162e50 2024-09-19T04:56:05.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.913+0000 7f6b84c4b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1b0140 con 0x556065158590 2024-09-19T04:56:05.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.913+0000 7f6b7b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x556065153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.913+0000 7f6b7b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x556065153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56682/0 (socket says 172.21.15.123:56682) 2024-09-19T04:56:05.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.913+0000 7f6b7b400640 1 -- 172.21.15.123:0/2764427659 learned_addr learned my addr 172.21.15.123:0/2764427659 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:05.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.913+0000 7f6b72a00640 1 --2- 172.21.15.123:0/2764427659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065158590 0x55606517a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.913+0000 7f6b7be00640 1 --2- 172.21.15.123:0/2764427659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065162e50 0x55606517ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.913+0000 7f6b7b400640 1 -- 172.21.15.123:0/2764427659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065158590 msgr2=0x55606517a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:05.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.913+0000 7f6b7b400640 1 --2- 172.21.15.123:0/2764427659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065158590 0x55606517a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.913+0000 7f6b7b400640 1 -- 172.21.15.123:0/2764427659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065162e50 msgr2=0x55606517ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:05.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.913+0000 7f6b7b400640 1 --2- 172.21.15.123:0/2764427659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065162e50 0x55606517ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.913+0000 7f6b7b400640 1 -- 172.21.15.123:0/2764427659 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556065154030 con 0x556065157c70 2024-09-19T04:56:05.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.914+0000 7f6b7b400640 1 --2- 172.21.15.123:0/2764427659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x556065153270 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0987e0 tx=0x7f6b6c09be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ec2f1d94626e8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:05.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.914+0000 7f6b6b400640 1 -- 172.21.15.123:0/2764427659 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c09c150 con 0x556065157c70 2024-09-19T04:56:05.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.914+0000 7f6b6b400640 1 -- 172.21.15.123:0/2764427659 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b6c0a0730 con 0x556065157c70 2024-09-19T04:56:05.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.914+0000 7f6b6b400640 1 -- 172.21.15.123:0/2764427659 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c09b4a0 con 0x556065157c70 2024-09-19T04:56:05.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.915+0000 7f6b84c4b740 1 -- 172.21.15.123:0/2764427659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 msgr2=0x556065153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:05.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.915+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/2764427659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x556065153270 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0987e0 tx=0x7f6b6c09be10 comp rx=0 tx=0).stop 2024-09-19T04:56:05.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.915+0000 7f6b84c4b740 1 -- 172.21.15.123:0/2764427659 shutdown_connections 2024-09-19T04:56:05.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.915+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/2764427659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065162e50 0x55606517ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.915+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/2764427659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065158590 0x55606517a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.915+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/2764427659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x556065153270 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.915+0000 7f6b84c4b740 1 -- 172.21.15.123:0/2764427659 >> 172.21.15.123:0/2764427659 conn(0x556065154200 msgr2=0x556065152b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:05.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.915+0000 7f6b84c4b740 1 -- 172.21.15.123:0/2764427659 shutdown_connections 2024-09-19T04:56:05.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.915+0000 7f6b84c4b740 1 -- 172.21.15.123:0/2764427659 wait complete. 2024-09-19T04:56:05.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.915+0000 7f6b84c4b740 1 -- start start 2024-09-19T04:56:05.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.916+0000 7f6b84c4b740 1 Processor -- start 2024-09-19T04:56:05.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.916+0000 7f6b84c4b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x5560651675c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.916+0000 7f6b84c4b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065158590 0x556065167b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.916+0000 7f6b84c4b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065162e50 0x55606516abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.916+0000 7f6b84c4b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1b0ee0 con 0x556065157c70 2024-09-19T04:56:05.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.916+0000 7f6b84c4b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1b0810 con 0x556065158590 2024-09-19T04:56:05.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.916+0000 7f6b84c4b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1ada70 con 0x556065162e50 2024-09-19T04:56:05.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.916+0000 7f6b84c4b740 10 client.0 _ll_register_callbacks cb 0x7f6b74009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:05.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.916+0000 7f6b7b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x5560651675c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.916+0000 7f6b72a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065158590 0x556065167b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.916+0000 7f6b7b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x5560651675c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56694/0 (socket says 172.21.15.123:56694) 2024-09-19T04:56:05.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.916+0000 7f6b7b400640 1 -- 172.21.15.123:0/1399940088 learned_addr learned my addr 172.21.15.123:0/1399940088 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:05.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.916+0000 7f6b7be00640 1 --2- 172.21.15.123:0/1399940088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065162e50 0x55606516abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.917+0000 7f6b7b400640 1 -- 172.21.15.123:0/1399940088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065162e50 msgr2=0x55606516abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:05.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.917+0000 7f6b7b400640 1 --2- 172.21.15.123:0/1399940088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065162e50 0x55606516abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.917+0000 7f6b7b400640 1 -- 172.21.15.123:0/1399940088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065158590 msgr2=0x556065167b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:05.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.917+0000 7f6b7b400640 1 --2- 172.21.15.123:0/1399940088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065158590 0x556065167b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.917+0000 7f6b7b400640 1 -- 172.21.15.123:0/1399940088 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55606514f770 con 0x556065157c70 2024-09-19T04:56:05.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.917+0000 7f6b72a00640 1 --2- 172.21.15.123:0/1399940088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065158590 0x556065167b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:05.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.917+0000 7f6b7b400640 1 --2- 172.21.15.123:0/1399940088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x5560651675c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c09b760 tx=0x7f6b6c09b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4822d37aac83512 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:05.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.917+0000 7f6b68c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:56:05.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.917+0000 7f6b68c00640 1 -- 172.21.15.123:0/1399940088 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c09c150 con 0x556065157c70 2024-09-19T04:56:05.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.917+0000 7f6b68c00640 1 -- 172.21.15.123:0/1399940088 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f6b6c09a680 con 0x556065157c70 2024-09-19T04:56:05.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.917+0000 7f6b84c4b740 1 -- 172.21.15.123:0/1399940088 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b6c0a0730 con 0x556065157c70 2024-09-19T04:56:05.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.918+0000 7f6b68c00640 1 -- 172.21.15.123:0/1399940088 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6b6c098620 con 0x556065157c70 2024-09-19T04:56:05.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.918+0000 7f6b84c4b740 10 client.17349 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:05.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.918+0000 7f6b84c4b740 1 -- 172.21.15.123:0/1399940088 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55606514f770 con 0x556065157c70 2024-09-19T04:56:05.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.918+0000 7f6b68c00640 1 -- 172.21.15.123:0/1399940088 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b6c0a0730 con 0x556065157c70 2024-09-19T04:56:05.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.918+0000 7f6b68c00640 1 -- 172.21.15.123:0/1399940088 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f6b6c0b0470 con 0x556065157c70 2024-09-19T04:56:05.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.918+0000 7f6b84c4b740 10 client.17349 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:05.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.918+0000 7f6b84c4b740 10 client.17349 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:56:05.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.918+0000 7f6b84c4b740 20 client.17349 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:05.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.918+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/1399940088 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5560651de350 0x5560651e0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:05.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.918+0000 7f6b84c4b740 4 client.17349 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:56:05.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.918+0000 7f6b84c4b740 1 -- 172.21.15.123:0/1399940088 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560650c5e60 con 0x5560651de350 2024-09-19T04:56:05.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.918+0000 7f6b72a00640 1 --2- 172.21.15.123:0/1399940088 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5560651de350 0x5560651e0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:05.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.919+0000 7f6b72a00640 1 --2- 172.21.15.123:0/1399940088 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5560651de350 0x5560651e0740 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=610b74f8ea2ca51e server_cookie=f291cbacb96231f6 in_seq=0 out_seq=0 2024-09-19T04:56:05.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.919+0000 7f6b68c00640 10 client.17349 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:56:05.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.919+0000 7f6b68c00640 1 -- 172.21.15.123:0/1399940088 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560650c5e60 con 0x5560651de350 2024-09-19T04:56:05.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.919+0000 7f6b68c00640 10 client.17349 handle_command_reply: tid=0 2024-09-19T04:56:05.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.919+0000 7f6b84c4b740 1 client.17349 shutdown 2024-09-19T04:56:05.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.919+0000 7f6b84c4b740 1 -- 172.21.15.123:0/1399940088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 msgr2=0x5560651675c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:05.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.919+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/1399940088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x5560651675c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c09b760 tx=0x7f6b6c09b5f0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.919+0000 7f6b84c4b740 1 -- 172.21.15.123:0/1399940088 shutdown_connections 2024-09-19T04:56:05.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.919+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/1399940088 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5560651de350 0x5560651e0740 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.919+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/1399940088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556065162e50 0x55606516abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.919+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/1399940088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556065158590 0x556065167b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.919+0000 7f6b84c4b740 1 --2- 172.21.15.123:0/1399940088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556065157c70 0x5560651675c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.919+0000 7f6b84c4b740 1 -- 172.21.15.123:0/1399940088 >> 172.21.15.123:0/1399940088 conn(0x556065154200 msgr2=0x556065159ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:05.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.919+0000 7f6b68c00640 0 client.17349 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:56:05.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.920+0000 7f6b84c4b740 1 -- 172.21.15.123:0/1399940088 shutdown_connections 2024-09-19T04:56:05.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.920+0000 7f6b84c4b740 1 -- 172.21.15.123:0/1399940088 wait complete. 2024-09-19T04:56:05.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.920+0000 7f6b84c4b740 20 client.17349 trim_cache size 0 max 16384 2024-09-19T04:56:05.929 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:56:05.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.920+0000 7f6b82200640 1 -- 172.21.15.123:0/794360599 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6b600a34b0 msgr2=0x7f6b600a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:05.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.920+0000 7f6b82200640 1 --2- 172.21.15.123:0/794360599 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6b600a34b0 0x7f6b600a5970 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009fd0 tx=0x7f6b74004a40 comp rx=0 tx=0).stop 2024-09-19T04:56:05.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.920+0000 7f6b82200640 1 -- 172.21.15.123:0/794360599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b7c101de0 msgr2=0x7f6b7c19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:05.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.920+0000 7f6b82200640 1 --2- 172.21.15.123:0/794360599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b7c101de0 0x7f6b7c19a670 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00e9f0 tx=0x7f6b6c00edf0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.921+0000 7f6b82200640 1 -- 172.21.15.123:0/794360599 shutdown_connections 2024-09-19T04:56:05.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.921+0000 7f6b82200640 1 --2- 172.21.15.123:0/794360599 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6b600a34b0 0x7f6b600a5970 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.921+0000 7f6b82200640 1 --2- 172.21.15.123:0/794360599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b7c106f10 0x7f6b7c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.921+0000 7f6b82200640 1 --2- 172.21.15.123:0/794360599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b7c102790 0x7f6b7c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.921+0000 7f6b82200640 1 --2- 172.21.15.123:0/794360599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b7c101de0 0x7f6b7c19a670 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:05.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.921+0000 7f6b82200640 1 -- 172.21.15.123:0/794360599 >> 172.21.15.123:0/794360599 conn(0x7f6b7c0fd8c0 msgr2=0x7f6b7c0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:05.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.921+0000 7f6b82200640 1 -- 172.21.15.123:0/794360599 shutdown_connections 2024-09-19T04:56:05.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:05.921+0000 7f6b82200640 1 -- 172.21.15.123:0/794360599 wait complete. 2024-09-19T04:56:06.760 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:56:07.129 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:07.558 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:07 smithi123 ceph-mon[30511]: pgmap v291: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T04:56:07.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:07 smithi183 ceph-mon[35333]: pgmap v291: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T04:56:07.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.667+0000 7f47e1a00640 1 -- 172.21.15.123:0/1755944427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47dc1049a0 msgr2=0x7f47dc1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:07.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.667+0000 7f47e1a00640 1 --2- 172.21.15.123:0/1755944427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47dc1049a0 0x7f47dc1089f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009900 tx=0x7f47c402a410 comp rx=0 tx=0).stop 2024-09-19T04:56:07.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.667+0000 7f47e1a00640 1 -- 172.21.15.123:0/1755944427 shutdown_connections 2024-09-19T04:56:07.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.667+0000 7f47e1a00640 1 --2- 172.21.15.123:0/1755944427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47dc109120 0x7f47dc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.667+0000 7f47e1a00640 1 --2- 172.21.15.123:0/1755944427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47dc1049a0 0x7f47dc1089f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.667+0000 7f47e1a00640 1 --2- 172.21.15.123:0/1755944427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47dc103ff0 0x7f47dc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.667+0000 7f47e1a00640 1 -- 172.21.15.123:0/1755944427 >> 172.21.15.123:0/1755944427 conn(0x7f47dc0ffa90 msgr2=0x7f47dc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:07.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.667+0000 7f47e1a00640 1 -- 172.21.15.123:0/1755944427 shutdown_connections 2024-09-19T04:56:07.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.667+0000 7f47e1a00640 1 -- 172.21.15.123:0/1755944427 wait complete. 2024-09-19T04:56:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.668+0000 7f47e1a00640 1 Processor -- start 2024-09-19T04:56:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.668+0000 7f47e1a00640 1 -- start start 2024-09-19T04:56:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.669+0000 7f47e1a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47dc103ff0 0x7f47dc19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.669+0000 7f47e1a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47dc1049a0 0x7f47dc19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.669+0000 7f47e1a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47dc109120 0x7f47dc19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.669+0000 7f47e1a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc114140 con 0x7f47dc103ff0 2024-09-19T04:56:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.669+0000 7f47e1a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc113fc0 con 0x7f47dc109120 2024-09-19T04:56:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.669+0000 7f47e1a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc1142c0 con 0x7f47dc1049a0 2024-09-19T04:56:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.669+0000 7f47daa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47dc103ff0 0x7f47dc19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.669+0000 7f47daa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47dc103ff0 0x7f47dc19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56724/0 (socket says 172.21.15.123:56724) 2024-09-19T04:56:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.669+0000 7f47daa00640 1 -- 172.21.15.123:0/2118011745 learned_addr learned my addr 172.21.15.123:0/2118011745 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.669+0000 7f47d3e00640 1 --2- 172.21.15.123:0/2118011745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47dc1049a0 0x7f47dc19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.669+0000 7f47db400640 1 --2- 172.21.15.123:0/2118011745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47dc109120 0x7f47dc19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.670+0000 7f47daa00640 1 -- 172.21.15.123:0/2118011745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47dc1049a0 msgr2=0x7f47dc19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.670+0000 7f47daa00640 1 --2- 172.21.15.123:0/2118011745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47dc1049a0 0x7f47dc19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.670+0000 7f47daa00640 1 -- 172.21.15.123:0/2118011745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47dc109120 msgr2=0x7f47dc19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.670+0000 7f47daa00640 1 --2- 172.21.15.123:0/2118011745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47dc109120 0x7f47dc19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.670+0000 7f47daa00640 1 -- 172.21.15.123:0/2118011745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47dc19f5d0 con 0x7f47dc103ff0 2024-09-19T04:56:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.670+0000 7f47daa00640 1 --2- 172.21.15.123:0/2118011745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47dc103ff0 0x7f47dc19a620 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00e9f0 tx=0x7f47cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.670+0000 7f47d3400640 1 -- 172.21.15.123:0/2118011745 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc00cd80 con 0x7f47dc103ff0 2024-09-19T04:56:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.670+0000 7f47d3400640 1 -- 172.21.15.123:0/2118011745 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47cc005500 con 0x7f47dc103ff0 2024-09-19T04:56:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.670+0000 7f47e1a00640 1 -- 172.21.15.123:0/2118011745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47dc19f8c0 con 0x7f47dc103ff0 2024-09-19T04:56:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.670+0000 7f47d3400640 1 -- 172.21.15.123:0/2118011745 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc010690 con 0x7f47dc103ff0 2024-09-19T04:56:07.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.670+0000 7f47e1a00640 1 -- 172.21.15.123:0/2118011745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47dc1a71a0 con 0x7f47dc103ff0 2024-09-19T04:56:07.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.673+0000 7f47e45d7740 1 Processor -- start 2024-09-19T04:56:07.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.674+0000 7f47e45d7740 1 -- start start 2024-09-19T04:56:07.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.674+0000 7f47d3400640 1 -- 172.21.15.123:0/2118011745 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f47cc0046c0 con 0x7f47dc103ff0 2024-09-19T04:56:07.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.674+0000 7f47e45d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84bc70 0x55584c84c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.674+0000 7f47e45d7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c84c590 0x55584c856910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.674+0000 7f47e45d7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c856e50 0x55584c859240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.674+0000 7f47e45d7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55584c7b23e0 con 0x55584c856e50 2024-09-19T04:56:07.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.674+0000 7f47e45d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55584c759d80 con 0x55584c84bc70 2024-09-19T04:56:07.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.674+0000 7f47e45d7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55584c7ae3a0 con 0x55584c84c590 2024-09-19T04:56:07.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.674+0000 7f47daa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c84c590 0x55584c856910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.674+0000 7f47db400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84bc70 0x55584c84c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.674+0000 7f47db400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84bc70 0x55584c84c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:51482/0 (socket says 172.21.15.123:51482) 2024-09-19T04:56:07.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.674+0000 7f47db400640 1 -- 172.21.15.123:0/1903720881 learned_addr learned my addr 172.21.15.123:0/1903720881 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:07.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.674+0000 7f47d3e00640 1 --2- 172.21.15.123:0/1903720881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c856e50 0x55584c859240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.675+0000 7f47db400640 1 -- 172.21.15.123:0/1903720881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c84c590 msgr2=0x55584c856910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:07.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.675+0000 7f47db400640 1 --2- 172.21.15.123:0/1903720881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c84c590 0x55584c856910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.675+0000 7f47db400640 1 -- 172.21.15.123:0/1903720881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c856e50 msgr2=0x55584c859240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:07.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.675+0000 7f47db400640 1 --2- 172.21.15.123:0/1903720881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c856e50 0x55584c859240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.675+0000 7f47db400640 1 -- 172.21.15.123:0/1903720881 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55584c778ae0 con 0x55584c84bc70 2024-09-19T04:56:07.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.675+0000 7f47db400640 1 --2- 172.21.15.123:0/1903720881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84bc70 0x55584c84c050 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009f90 tx=0x7f47d4010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f2375bcc5807349 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:07.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.675+0000 7f47d3400640 1 --2- 172.21.15.123:0/2118011745 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f47c00a33b0 0x7f47c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.675+0000 7f47cbe00640 1 -- 172.21.15.123:0/1903720881 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4011700 con 0x55584c84bc70 2024-09-19T04:56:07.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.676+0000 7f47cbe00640 1 -- 172.21.15.123:0/1903720881 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d4011d80 con 0x55584c84bc70 2024-09-19T04:56:07.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.676+0000 7f47cbe00640 1 -- 172.21.15.123:0/1903720881 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4019e40 con 0x55584c84bc70 2024-09-19T04:56:07.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.676+0000 7f47d3400640 1 -- 172.21.15.123:0/2118011745 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f47cc0d0100 con 0x7f47dc103ff0 2024-09-19T04:56:07.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.676+0000 7f47daa00640 1 --2- 172.21.15.123:0/2118011745 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f47c00a33b0 0x7f47c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.676+0000 7f47e45d7740 1 -- 172.21.15.123:0/1903720881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84bc70 msgr2=0x55584c84c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:07.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.676+0000 7f47e45d7740 1 --2- 172.21.15.123:0/1903720881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84bc70 0x55584c84c050 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009f90 tx=0x7f47d4010860 comp rx=0 tx=0).stop 2024-09-19T04:56:07.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.676+0000 7f47daa00640 1 --2- 172.21.15.123:0/2118011745 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f47c00a33b0 0x7f47c00a5870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0cd900 tx=0x7f47cc098040 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:07.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.676+0000 7f47e45d7740 1 -- 172.21.15.123:0/1903720881 shutdown_connections 2024-09-19T04:56:07.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.676+0000 7f47e45d7740 1 --2- 172.21.15.123:0/1903720881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c856e50 0x55584c859240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.676+0000 7f47e45d7740 1 --2- 172.21.15.123:0/1903720881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c84c590 0x55584c856910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.676+0000 7f47e45d7740 1 --2- 172.21.15.123:0/1903720881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84bc70 0x55584c84c050 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.676+0000 7f47e45d7740 1 -- 172.21.15.123:0/1903720881 >> 172.21.15.123:0/1903720881 conn(0x55584c848200 msgr2=0x55584c8485e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:07.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.677+0000 7f47e45d7740 1 -- 172.21.15.123:0/1903720881 shutdown_connections 2024-09-19T04:56:07.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.677+0000 7f47e45d7740 1 -- 172.21.15.123:0/1903720881 wait complete. 2024-09-19T04:56:07.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.677+0000 7f47e45d7740 1 -- start start 2024-09-19T04:56:07.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.677+0000 7f47e45d7740 1 Processor -- start 2024-09-19T04:56:07.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.678+0000 7f47e45d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84bc70 0x55584c853cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.678+0000 7f47e45d7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c84c590 0x55584c854230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.678+0000 7f47e45d7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c856e50 0x55584c854770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.678+0000 7f47e45d7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55584c859a20 con 0x55584c856e50 2024-09-19T04:56:07.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.678+0000 7f47e45d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55584c76d0f0 con 0x55584c84bc70 2024-09-19T04:56:07.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.678+0000 7f47e45d7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55584c859ba0 con 0x55584c84c590 2024-09-19T04:56:07.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.678+0000 7f47e45d7740 10 client.0 _ll_register_callbacks cb 0x55584c754bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:07.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.678+0000 7f47db400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84bc70 0x55584c853cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.678+0000 7f47db400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84bc70 0x55584c853cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:51490/0 (socket says 172.21.15.123:51490) 2024-09-19T04:56:07.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.678+0000 7f47db400640 1 -- 172.21.15.123:0/694362382 learned_addr learned my addr 172.21.15.123:0/694362382 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:07.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.678+0000 7f47db400640 1 --2- 172.21.15.123:0/694362382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c856e50 0x55584c854770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.678+0000 7f47d3e00640 1 --2- 172.21.15.123:0/694362382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c84c590 0x55584c854230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.679+0000 7f47d3e00640 1 -- 172.21.15.123:0/694362382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84bc70 msgr2=0x55584c853cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:07.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.679+0000 7f47d3e00640 1 --2- 172.21.15.123:0/694362382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84bc70 0x55584c853cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.679+0000 7f47d3e00640 1 -- 172.21.15.123:0/694362382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c856e50 msgr2=0x55584c854770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:07.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.679+0000 7f47d3e00640 1 --2- 172.21.15.123:0/694362382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c856e50 0x55584c854770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.679+0000 7f47d3e00640 1 -- 172.21.15.123:0/694362382 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55584c5233c0 con 0x55584c84c590 2024-09-19T04:56:07.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.679+0000 7f47db400640 1 --2- 172.21.15.123:0/694362382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c856e50 0x55584c854770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:07.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.679+0000 7f47d3e00640 1 --2- 172.21.15.123:0/694362382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c84c590 0x55584c854230 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f47c4005940 tx=0x7f47c40317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a90465af9b50edca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:07.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.679+0000 7f47c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:56:07.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.679+0000 7f47c8c00640 1 -- 172.21.15.123:0/694362382 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4035040 con 0x55584c84c590 2024-09-19T04:56:07.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.679+0000 7f47c8c00640 1 -- 172.21.15.123:0/694362382 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f47c4004240 con 0x55584c84c590 2024-09-19T04:56:07.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.679+0000 7f47e45d7740 1 -- 172.21.15.123:0/694362382 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47d4011d80 con 0x55584c84c590 2024-09-19T04:56:07.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.679+0000 7f47c8c00640 1 -- 172.21.15.123:0/694362382 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f47c4004440 con 0x55584c84c590 2024-09-19T04:56:07.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.680+0000 7f47e45d7740 10 client.27466 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:07.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.680+0000 7f47e45d7740 1 -- 172.21.15.123:0/694362382 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55584c865a10 con 0x55584c84c590 2024-09-19T04:56:07.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.680+0000 7f47c8c00640 1 -- 172.21.15.123:0/694362382 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d4011d80 con 0x55584c84c590 2024-09-19T04:56:07.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.680+0000 7f47c8c00640 1 -- 172.21.15.123:0/694362382 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f47c404f640 con 0x55584c84c590 2024-09-19T04:56:07.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.680+0000 7f47e45d7740 10 client.27466 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:07.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.680+0000 7f47e45d7740 10 client.27466 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:56:07.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.680+0000 7f47e45d7740 20 client.27466 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:07.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.680+0000 7f47e45d7740 1 --2- 172.21.15.123:0/694362382 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55584c866e20 0x55584c869210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.680+0000 7f47e45d7740 4 client.27466 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:56:07.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.680+0000 7f47e45d7740 1 -- 172.21.15.123:0/694362382 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55584c7bf950 con 0x55584c866e20 2024-09-19T04:56:07.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.681+0000 7f47db400640 1 --2- 172.21.15.123:0/694362382 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55584c866e20 0x55584c869210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.681+0000 7f47db400640 1 --2- 172.21.15.123:0/694362382 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55584c866e20 0x55584c869210 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5edd63feb62bcc9e server_cookie=bafcbdb5e225c59a in_seq=0 out_seq=0 2024-09-19T04:56:07.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.681+0000 7f47c8c00640 10 client.27466 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:56:07.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.682+0000 7f47c8c00640 1 -- 172.21.15.123:0/694362382 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55584c7bf950 con 0x55584c866e20 2024-09-19T04:56:07.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.682+0000 7f47c8c00640 10 client.27466 handle_command_reply: tid=0 2024-09-19T04:56:07.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.683+0000 7f47e45d7740 1 client.27466 shutdown 2024-09-19T04:56:07.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.683+0000 7f47e45d7740 1 -- 172.21.15.123:0/694362382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c84c590 msgr2=0x55584c854230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:07.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.683+0000 7f47e45d7740 1 --2- 172.21.15.123:0/694362382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c84c590 0x55584c854230 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f47c4005940 tx=0x7f47c40317e0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.683+0000 7f47e45d7740 1 -- 172.21.15.123:0/694362382 shutdown_connections 2024-09-19T04:56:07.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.683+0000 7f47e45d7740 1 --2- 172.21.15.123:0/694362382 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55584c866e20 0x55584c869210 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.683+0000 7f47e45d7740 1 --2- 172.21.15.123:0/694362382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c856e50 0x55584c854770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.683+0000 7f47e45d7740 1 --2- 172.21.15.123:0/694362382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c84c590 0x55584c854230 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.683+0000 7f47e45d7740 1 --2- 172.21.15.123:0/694362382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84bc70 0x55584c853cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.683+0000 7f47c8c00640 0 client.27466 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:56:07.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.683+0000 7f47e45d7740 1 -- 172.21.15.123:0/694362382 >> 172.21.15.123:0/694362382 conn(0x55584c848200 msgr2=0x55584c847380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:07.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.683+0000 7f47e45d7740 1 -- 172.21.15.123:0/694362382 shutdown_connections 2024-09-19T04:56:07.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.683+0000 7f47e45d7740 1 -- 172.21.15.123:0/694362382 wait complete. 2024-09-19T04:56:07.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.683+0000 7f47e45d7740 20 client.27466 trim_cache size 0 max 16384 2024-09-19T04:56:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.697+0000 7f47e45d7740 1 Processor -- start 2024-09-19T04:56:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.697+0000 7f47e45d7740 1 -- start start 2024-09-19T04:56:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.697+0000 7f47e45d7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c84bc70 0x55584c8525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.697+0000 7f47e45d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84c590 0x55584c8545c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.697+0000 7f47e45d7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c856e50 0x55584c86eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.697+0000 7f47e45d7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc1af9f0 con 0x55584c84bc70 2024-09-19T04:56:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.697+0000 7f47e45d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc1b1530 con 0x55584c84c590 2024-09-19T04:56:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.697+0000 7f47e45d7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc1b00c0 con 0x55584c856e50 2024-09-19T04:56:07.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.697+0000 7f47db400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c84bc70 0x55584c8525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.697+0000 7f47db400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c84bc70 0x55584c8525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56748/0 (socket says 172.21.15.123:56748) 2024-09-19T04:56:07.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.697+0000 7f47db400640 1 -- 172.21.15.123:0/3898042569 learned_addr learned my addr 172.21.15.123:0/3898042569 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:07.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.697+0000 7f47d3e00640 1 --2- 172.21.15.123:0/3898042569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84c590 0x55584c8545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.698+0000 7f47db400640 1 --2- 172.21.15.123:0/3898042569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c856e50 0x55584c86eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.698+0000 7f47db400640 1 -- 172.21.15.123:0/3898042569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c856e50 msgr2=0x55584c86eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:07.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.698+0000 7f47db400640 1 --2- 172.21.15.123:0/3898042569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c856e50 0x55584c86eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.698+0000 7f47db400640 1 -- 172.21.15.123:0/3898042569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84c590 msgr2=0x55584c8545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:07.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.698+0000 7f47db400640 1 --2- 172.21.15.123:0/3898042569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84c590 0x55584c8545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.698+0000 7f47db400640 1 -- 172.21.15.123:0/3898042569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55584c843770 con 0x55584c84bc70 2024-09-19T04:56:07.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.698+0000 7f47db400640 1 --2- 172.21.15.123:0/3898042569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c84bc70 0x55584c8525c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f47d4019970 tx=0x7f47d40199a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cea7d2cf9d6e761 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:07.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.698+0000 7f47cb400640 1 -- 172.21.15.123:0/3898042569 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4019d20 con 0x55584c84bc70 2024-09-19T04:56:07.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.698+0000 7f47cb400640 1 -- 172.21.15.123:0/3898042569 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d4020040 con 0x55584c84bc70 2024-09-19T04:56:07.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.698+0000 7f47cb400640 1 -- 172.21.15.123:0/3898042569 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4005460 con 0x55584c84bc70 2024-09-19T04:56:07.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.699+0000 7f47e45d7740 1 -- 172.21.15.123:0/3898042569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c84bc70 msgr2=0x55584c8525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:07.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.699+0000 7f47e45d7740 1 --2- 172.21.15.123:0/3898042569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c84bc70 0x55584c8525c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f47d4019970 tx=0x7f47d40199a0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.699+0000 7f47e45d7740 1 -- 172.21.15.123:0/3898042569 shutdown_connections 2024-09-19T04:56:07.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.699+0000 7f47e45d7740 1 --2- 172.21.15.123:0/3898042569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c856e50 0x55584c86eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.699+0000 7f47e45d7740 1 --2- 172.21.15.123:0/3898042569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84c590 0x55584c8545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.699+0000 7f47e45d7740 1 --2- 172.21.15.123:0/3898042569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c84bc70 0x55584c8525c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.699+0000 7f47e45d7740 1 -- 172.21.15.123:0/3898042569 >> 172.21.15.123:0/3898042569 conn(0x55584c848200 msgr2=0x55584c855d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:07.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.699+0000 7f47e45d7740 1 -- 172.21.15.123:0/3898042569 shutdown_connections 2024-09-19T04:56:07.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.699+0000 7f47e45d7740 1 -- 172.21.15.123:0/3898042569 wait complete. 2024-09-19T04:56:07.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.699+0000 7f47e45d7740 1 -- start start 2024-09-19T04:56:07.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.700+0000 7f47e45d7740 1 Processor -- start 2024-09-19T04:56:07.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.700+0000 7f47e45d7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c84bc70 0x55584c85f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.700+0000 7f47e45d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84c590 0x55584c85a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.700+0000 7f47e45d7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c856e50 0x55584c85ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.700+0000 7f47e45d7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc1b0e60 con 0x55584c84bc70 2024-09-19T04:56:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.700+0000 7f47e45d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc1b0790 con 0x55584c84c590 2024-09-19T04:56:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.700+0000 7f47e45d7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc1ad9f0 con 0x55584c856e50 2024-09-19T04:56:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.700+0000 7f47e45d7740 10 client.0 _ll_register_callbacks cb 0x7f47c4005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.700+0000 7f47db400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c84bc70 0x55584c85f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.700+0000 7f47d3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84c590 0x55584c85a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.700+0000 7f47db400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c84bc70 0x55584c85f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56752/0 (socket says 172.21.15.123:56752) 2024-09-19T04:56:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.700+0000 7f47db400640 1 -- 172.21.15.123:0/2125928902 learned_addr learned my addr 172.21.15.123:0/2125928902 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.700+0000 7f47d3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84c590 0x55584c85a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:51504/0 (socket says 172.21.15.123:51504) 2024-09-19T04:56:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.701+0000 7f47db400640 1 --2- 172.21.15.123:0/2125928902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c856e50 0x55584c85ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.701+0000 7f47db400640 1 -- 172.21.15.123:0/2125928902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c856e50 msgr2=0x55584c85ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.701+0000 7f47db400640 1 --2- 172.21.15.123:0/2125928902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c856e50 0x55584c85ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.701+0000 7f47db400640 1 -- 172.21.15.123:0/2125928902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84c590 msgr2=0x55584c85a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:07.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.701+0000 7f47db400640 1 --2- 172.21.15.123:0/2125928902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84c590 0x55584c85a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.701+0000 7f47db400640 1 -- 172.21.15.123:0/2125928902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55584c84eae0 con 0x55584c84bc70 2024-09-19T04:56:07.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.701+0000 7f47d3e00640 1 --2- 172.21.15.123:0/2125928902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84c590 0x55584c85a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:07.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.701+0000 7f47db400640 1 --2- 172.21.15.123:0/2125928902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c84bc70 0x55584c85f5e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009720 tx=0x7f47d4019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ced9259ce9aece server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:07.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.701+0000 7f47c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:56:07.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.701+0000 7f47c8c00640 1 -- 172.21.15.123:0/2125928902 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4003970 con 0x55584c84bc70 2024-09-19T04:56:07.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.701+0000 7f47c8c00640 1 -- 172.21.15.123:0/2125928902 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f47d4003b10 con 0x55584c84bc70 2024-09-19T04:56:07.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.701+0000 7f47e45d7740 1 -- 172.21.15.123:0/2125928902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47d4020040 con 0x55584c84bc70 2024-09-19T04:56:07.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.702+0000 7f47c8c00640 1 -- 172.21.15.123:0/2125928902 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f47d40051e0 con 0x55584c84bc70 2024-09-19T04:56:07.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.702+0000 7f47e45d7740 10 client.17367 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:07.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.702+0000 7f47e45d7740 1 -- 172.21.15.123:0/2125928902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55584c8d1980 con 0x55584c84bc70 2024-09-19T04:56:07.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.702+0000 7f47c8c00640 1 -- 172.21.15.123:0/2125928902 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d4020040 con 0x55584c84bc70 2024-09-19T04:56:07.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.702+0000 7f47c8c00640 1 -- 172.21.15.123:0/2125928902 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f47d4011ad0 con 0x55584c84bc70 2024-09-19T04:56:07.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.702+0000 7f47e45d7740 10 client.17367 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:07.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.702+0000 7f47e45d7740 10 client.17367 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:56:07.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.702+0000 7f47e45d7740 20 client.17367 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:07.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.702+0000 7f47e45d7740 1 --2- 172.21.15.123:0/2125928902 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55584c8d1d50 0x55584c8d4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:07.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.702+0000 7f47e45d7740 4 client.17367 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:56:07.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.702+0000 7f47e45d7740 1 -- 172.21.15.123:0/2125928902 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55584c7b9e60 con 0x55584c8d1d50 2024-09-19T04:56:07.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.703+0000 7f47d3e00640 1 --2- 172.21.15.123:0/2125928902 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55584c8d1d50 0x55584c8d4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:07.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.703+0000 7f47d3e00640 1 --2- 172.21.15.123:0/2125928902 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55584c8d1d50 0x55584c8d4140 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90bd7bbdcd1a5eef server_cookie=bcf5aac8d56b081e in_seq=0 out_seq=0 2024-09-19T04:56:07.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.703+0000 7f47c8c00640 10 client.17367 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:56:07.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.703+0000 7f47c8c00640 1 -- 172.21.15.123:0/2125928902 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55584c7b9e60 con 0x55584c8d1d50 2024-09-19T04:56:07.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.703+0000 7f47c8c00640 10 client.17367 handle_command_reply: tid=0 2024-09-19T04:56:07.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.703+0000 7f47e45d7740 1 client.17367 shutdown 2024-09-19T04:56:07.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.704+0000 7f47e45d7740 1 -- 172.21.15.123:0/2125928902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c84bc70 msgr2=0x55584c85f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:07.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.704+0000 7f47e45d7740 1 --2- 172.21.15.123:0/2125928902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c84bc70 0x55584c85f5e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009720 tx=0x7f47d4019810 comp rx=0 tx=0).stop 2024-09-19T04:56:07.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.704+0000 7f47e45d7740 1 -- 172.21.15.123:0/2125928902 shutdown_connections 2024-09-19T04:56:07.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.704+0000 7f47e45d7740 1 --2- 172.21.15.123:0/2125928902 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55584c8d1d50 0x55584c8d4140 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.704+0000 7f47e45d7740 1 --2- 172.21.15.123:0/2125928902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55584c856e50 0x55584c85ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.704+0000 7f47e45d7740 1 --2- 172.21.15.123:0/2125928902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584c84c590 0x55584c85a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.704+0000 7f47e45d7740 1 --2- 172.21.15.123:0/2125928902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584c84bc70 0x55584c85f5e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.704+0000 7f47e45d7740 1 -- 172.21.15.123:0/2125928902 >> 172.21.15.123:0/2125928902 conn(0x55584c848200 msgr2=0x55584c861c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:07.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.704+0000 7f47c8c00640 0 client.17367 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:56:07.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.704+0000 7f47e45d7740 1 -- 172.21.15.123:0/2125928902 shutdown_connections 2024-09-19T04:56:07.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.704+0000 7f47e45d7740 1 -- 172.21.15.123:0/2125928902 wait complete. 2024-09-19T04:56:07.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.704+0000 7f47e45d7740 20 client.17367 trim_cache size 0 max 16384 2024-09-19T04:56:07.710 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:56:07.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.705+0000 7f47e1a00640 1 -- 172.21.15.123:0/2118011745 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f47c00a33b0 msgr2=0x7f47c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:07.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.705+0000 7f47e1a00640 1 --2- 172.21.15.123:0/2118011745 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f47c00a33b0 0x7f47c00a5870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0cd900 tx=0x7f47cc098040 comp rx=0 tx=0).stop 2024-09-19T04:56:07.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.705+0000 7f47e1a00640 1 -- 172.21.15.123:0/2118011745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47dc103ff0 msgr2=0x7f47dc19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:07.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.705+0000 7f47e1a00640 1 --2- 172.21.15.123:0/2118011745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47dc103ff0 0x7f47dc19a620 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00e9f0 tx=0x7f47cc00edf0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.705+0000 7f47e1a00640 1 -- 172.21.15.123:0/2118011745 shutdown_connections 2024-09-19T04:56:07.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.705+0000 7f47e1a00640 1 --2- 172.21.15.123:0/2118011745 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f47c00a33b0 0x7f47c00a5870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.705+0000 7f47e1a00640 1 --2- 172.21.15.123:0/2118011745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47dc109120 0x7f47dc19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.705+0000 7f47e1a00640 1 --2- 172.21.15.123:0/2118011745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47dc1049a0 0x7f47dc19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.705+0000 7f47e1a00640 1 --2- 172.21.15.123:0/2118011745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47dc103ff0 0x7f47dc19a620 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:07.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.705+0000 7f47e1a00640 1 -- 172.21.15.123:0/2118011745 >> 172.21.15.123:0/2118011745 conn(0x7f47dc0ffa90 msgr2=0x7f47dc100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:07.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.705+0000 7f47e1a00640 1 -- 172.21.15.123:0/2118011745 shutdown_connections 2024-09-19T04:56:07.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:07.705+0000 7f47e1a00640 1 -- 172.21.15.123:0/2118011745 wait complete. 2024-09-19T04:56:07.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:07 smithi193 ceph-mon[35592]: pgmap v291: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T04:56:08.485 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:56:08.853 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:09.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.450+0000 7f4042400640 1 -- 172.21.15.123:0/1477794186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f403c108f60 msgr2=0x7f403c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:09.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.450+0000 7f4042400640 1 --2- 172.21.15.123:0/1477794186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f403c108f60 0x7f403c110ae0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4034009ca0 tx=0x7f403402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.450+0000 7f4042400640 1 -- 172.21.15.123:0/1477794186 shutdown_connections 2024-09-19T04:56:09.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.450+0000 7f4042400640 1 --2- 172.21.15.123:0/1477794186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f403c108f60 0x7f403c110ae0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.450+0000 7f4042400640 1 --2- 172.21.15.123:0/1477794186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f403c1047e0 0x7f403c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.450+0000 7f4042400640 1 --2- 172.21.15.123:0/1477794186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f403c103e30 0x7f403c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.450+0000 7f4042400640 1 -- 172.21.15.123:0/1477794186 >> 172.21.15.123:0/1477794186 conn(0x7f403c0ff8d0 msgr2=0x7f403c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:09.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.450+0000 7f4042400640 1 -- 172.21.15.123:0/1477794186 shutdown_connections 2024-09-19T04:56:09.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.450+0000 7f4042400640 1 -- 172.21.15.123:0/1477794186 wait complete. 2024-09-19T04:56:09.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.451+0000 7f4042400640 1 Processor -- start 2024-09-19T04:56:09.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.451+0000 7f4042400640 1 -- start start 2024-09-19T04:56:09.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.452+0000 7f4042400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f403c103e30 0x7f403c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.452+0000 7f4042400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f403c1047e0 0x7f403c19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.452+0000 7f4042400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f403c108f60 0x7f403c19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.452+0000 7f4042400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f403c113fc0 con 0x7f403c108f60 2024-09-19T04:56:09.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.452+0000 7f4042400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f403c113e40 con 0x7f403c103e30 2024-09-19T04:56:09.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.452+0000 7f4042400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f403c114140 con 0x7f403c1047e0 2024-09-19T04:56:09.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.452+0000 7f4041a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f403c108f60 0x7f403c19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.452+0000 7f403be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f403c1047e0 0x7f403c19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.452+0000 7f4041a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f403c108f60 0x7f403c19ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56774/0 (socket says 172.21.15.123:56774) 2024-09-19T04:56:09.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.452+0000 7f4041a00640 1 -- 172.21.15.123:0/1109916044 learned_addr learned my addr 172.21.15.123:0/1109916044 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:09.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.452+0000 7f4041000640 1 --2- 172.21.15.123:0/1109916044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f403c103e30 0x7f403c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.453+0000 7f4041a00640 1 -- 172.21.15.123:0/1109916044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f403c1047e0 msgr2=0x7f403c19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:09.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.453+0000 7f4041a00640 1 --2- 172.21.15.123:0/1109916044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f403c1047e0 0x7f403c19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.453+0000 7f4041a00640 1 -- 172.21.15.123:0/1109916044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f403c103e30 msgr2=0x7f403c19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:09.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.453+0000 7f4041a00640 1 --2- 172.21.15.123:0/1109916044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f403c103e30 0x7f403c19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.453+0000 7f4041a00640 1 -- 172.21.15.123:0/1109916044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c19f3b0 con 0x7f403c108f60 2024-09-19T04:56:09.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.453+0000 7f403be00640 1 --2- 172.21.15.123:0/1109916044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f403c1047e0 0x7f403c19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:09.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.453+0000 7f4041a00640 1 --2- 172.21.15.123:0/1109916044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f403c108f60 0x7f403c19ecd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4034031a20 tx=0x7f4034031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:09.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.453+0000 7f4039600640 1 -- 172.21.15.123:0/1109916044 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034003b70 con 0x7f403c108f60 2024-09-19T04:56:09.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.453+0000 7f4039600640 1 -- 172.21.15.123:0/1109916044 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4034003d10 con 0x7f403c108f60 2024-09-19T04:56:09.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.453+0000 7f4042400640 1 -- 172.21.15.123:0/1109916044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c19f640 con 0x7f403c108f60 2024-09-19T04:56:09.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.453+0000 7f4039600640 1 -- 172.21.15.123:0/1109916044 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403402fc40 con 0x7f403c108f60 2024-09-19T04:56:09.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.453+0000 7f4042400640 1 -- 172.21.15.123:0/1109916044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c1a6f80 con 0x7f403c108f60 2024-09-19T04:56:09.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.456+0000 7f4039600640 1 -- 172.21.15.123:0/1109916044 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4034004050 con 0x7f403c108f60 2024-09-19T04:56:09.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.456+0000 7f4046671740 1 Processor -- start 2024-09-19T04:56:09.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.457+0000 7f4046671740 1 -- start start 2024-09-19T04:56:09.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.457+0000 7f4046671740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78c8e40 0x555cc78c9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.457+0000 7f4046671740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c9760 0x555cc78d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.457+0000 7f4046671740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78d4020 0x555cc78d6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.457+0000 7f4046671740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555cc782f3e0 con 0x555cc78d4020 2024-09-19T04:56:09.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.457+0000 7f4046671740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555cc77d6d80 con 0x555cc78c9760 2024-09-19T04:56:09.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.457+0000 7f4046671740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555cc782b3a0 con 0x555cc78c8e40 2024-09-19T04:56:09.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.458+0000 7f4041000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c9760 0x555cc78d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.458+0000 7f403be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78c8e40 0x555cc78c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.458+0000 7f4041000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c9760 0x555cc78d3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:51538/0 (socket says 172.21.15.123:51538) 2024-09-19T04:56:09.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.458+0000 7f4041000640 1 -- 172.21.15.123:0/4103838446 learned_addr learned my addr 172.21.15.123:0/4103838446 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:09.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.458+0000 7f403be00640 1 --2- 172.21.15.123:0/4103838446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78d4020 0x555cc78d6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.458+0000 7f4039600640 1 --2- 172.21.15.123:0/1109916044 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f40240a3430 0x7f40240a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.458+0000 7f4041000640 1 -- 172.21.15.123:0/4103838446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78c8e40 msgr2=0x555cc78c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:09.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.458+0000 7f4041000640 1 --2- 172.21.15.123:0/4103838446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78c8e40 0x555cc78c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.458+0000 7f4041000640 1 -- 172.21.15.123:0/4103838446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78d4020 msgr2=0x555cc78d6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:09.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.458+0000 7f4041000640 1 --2- 172.21.15.123:0/4103838446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78d4020 0x555cc78d6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.458+0000 7f4041000640 1 -- 172.21.15.123:0/4103838446 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc77f5ae0 con 0x555cc78c9760 2024-09-19T04:56:09.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.458+0000 7f403be00640 1 --2- 172.21.15.123:0/4103838446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78c8e40 0x555cc78c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:56:09.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.458+0000 7f4039600640 1 -- 172.21.15.123:0/1109916044 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f40340f0f00 con 0x7f403c108f60 2024-09-19T04:56:09.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.458+0000 7f4041a00640 1 --2- 172.21.15.123:0/1109916044 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f40240a3430 0x7f40240a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.458+0000 7f4041000640 1 --2- 172.21.15.123:0/4103838446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c9760 0x555cc78d3ae0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4030009d20 tx=0x7f4030011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c2380836d072116 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:09.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.459+0000 7f402e000640 1 -- 172.21.15.123:0/4103838446 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403001e070 con 0x555cc78c9760 2024-09-19T04:56:09.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.459+0000 7f402e000640 1 -- 172.21.15.123:0/4103838446 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4030019b10 con 0x555cc78c9760 2024-09-19T04:56:09.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.459+0000 7f402e000640 1 -- 172.21.15.123:0/4103838446 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403000eb90 con 0x555cc78c9760 2024-09-19T04:56:09.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.459+0000 7f4046671740 1 -- 172.21.15.123:0/4103838446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c9760 msgr2=0x555cc78d3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:09.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.459+0000 7f4041a00640 1 --2- 172.21.15.123:0/1109916044 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f40240a3430 0x7f40240a58f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f40340b9cf0 tx=0x7f4034047fa0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:09.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.459+0000 7f4046671740 1 --2- 172.21.15.123:0/4103838446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c9760 0x555cc78d3ae0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4030009d20 tx=0x7f4030011810 comp rx=0 tx=0).stop 2024-09-19T04:56:09.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.460+0000 7f4046671740 1 -- 172.21.15.123:0/4103838446 shutdown_connections 2024-09-19T04:56:09.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.460+0000 7f4046671740 1 --2- 172.21.15.123:0/4103838446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78d4020 0x555cc78d6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.460+0000 7f4046671740 1 --2- 172.21.15.123:0/4103838446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c9760 0x555cc78d3ae0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.460+0000 7f4046671740 1 --2- 172.21.15.123:0/4103838446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78c8e40 0x555cc78c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.460+0000 7f4046671740 1 -- 172.21.15.123:0/4103838446 >> 172.21.15.123:0/4103838446 conn(0x555cc78c53d0 msgr2=0x555cc78c57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:09.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.460+0000 7f4046671740 1 -- 172.21.15.123:0/4103838446 shutdown_connections 2024-09-19T04:56:09.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.460+0000 7f4046671740 1 -- 172.21.15.123:0/4103838446 wait complete. 2024-09-19T04:56:09.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.460+0000 7f4046671740 1 -- start start 2024-09-19T04:56:09.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.460+0000 7f4046671740 1 Processor -- start 2024-09-19T04:56:09.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.461+0000 7f4046671740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c8e40 0x555cc78d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.461+0000 7f4046671740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78c9760 0x555cc78d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.461+0000 7f4046671740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78d4020 0x555cc78d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.461+0000 7f4046671740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555cc78d6bf0 con 0x555cc78c9760 2024-09-19T04:56:09.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.461+0000 7f4046671740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555cc77ea0f0 con 0x555cc78c8e40 2024-09-19T04:56:09.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.461+0000 7f4046671740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555cc78d6d70 con 0x555cc78d4020 2024-09-19T04:56:09.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.461+0000 7f4046671740 10 client.0 _ll_register_callbacks cb 0x555cc77d1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:09.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.461+0000 7f4041000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78c9760 0x555cc78d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.461+0000 7f403be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c8e40 0x555cc78d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.461+0000 7f4041000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78c9760 0x555cc78d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56792/0 (socket says 172.21.15.123:56792) 2024-09-19T04:56:09.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.461+0000 7f4041000640 1 -- 172.21.15.123:0/437599580 learned_addr learned my addr 172.21.15.123:0/437599580 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:09.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.461+0000 7f403be00640 1 --2- 172.21.15.123:0/437599580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78d4020 0x555cc78d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.462+0000 7f4041000640 1 -- 172.21.15.123:0/437599580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78d4020 msgr2=0x555cc78d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:09.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.462+0000 7f4041000640 1 --2- 172.21.15.123:0/437599580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78d4020 0x555cc78d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.462+0000 7f4041000640 1 -- 172.21.15.123:0/437599580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c8e40 msgr2=0x555cc78d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:09.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.462+0000 7f4041000640 1 --2- 172.21.15.123:0/437599580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c8e40 0x555cc78d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.462+0000 7f4041000640 1 -- 172.21.15.123:0/437599580 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc75a03c0 con 0x555cc78c9760 2024-09-19T04:56:09.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.462+0000 7f4041000640 1 --2- 172.21.15.123:0/437599580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78c9760 0x555cc78d13e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4030019660 tx=0x7f4030019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f25db61606900a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:09.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.462+0000 7f4022a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:56:09.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.462+0000 7f4022a00640 1 -- 172.21.15.123:0/437599580 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030027020 con 0x555cc78c9760 2024-09-19T04:56:09.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.462+0000 7f4046671740 1 -- 172.21.15.123:0/437599580 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4030019b10 con 0x555cc78c9760 2024-09-19T04:56:09.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.462+0000 7f4022a00640 1 -- 172.21.15.123:0/437599580 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f403000eaf0 con 0x555cc78c9760 2024-09-19T04:56:09.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.463+0000 7f4022a00640 1 -- 172.21.15.123:0/437599580 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4030032050 con 0x555cc78c9760 2024-09-19T04:56:09.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.463+0000 7f4046671740 10 client.17379 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:09.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.463+0000 7f4046671740 1 -- 172.21.15.123:0/437599580 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555cc78e2c20 con 0x555cc78c9760 2024-09-19T04:56:09.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.463+0000 7f4022a00640 1 -- 172.21.15.123:0/437599580 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4030019b10 con 0x555cc78c9760 2024-09-19T04:56:09.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.463+0000 7f4022a00640 1 -- 172.21.15.123:0/437599580 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f40300046c0 con 0x555cc78c9760 2024-09-19T04:56:09.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.463+0000 7f4046671740 10 client.17379 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:09.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.463+0000 7f4046671740 10 client.17379 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:56:09.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.463+0000 7f4046671740 20 client.17379 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:09.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.463+0000 7f4046671740 1 --2- 172.21.15.123:0/437599580 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555cc78e4030 0x555cc78e6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.463+0000 7f4046671740 4 client.17379 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:56:09.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.463+0000 7f4046671740 1 -- 172.21.15.123:0/437599580 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cc783c950 con 0x555cc78e4030 2024-09-19T04:56:09.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.464+0000 7f403be00640 1 --2- 172.21.15.123:0/437599580 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555cc78e4030 0x555cc78e6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.464+0000 7f403be00640 1 --2- 172.21.15.123:0/437599580 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555cc78e4030 0x555cc78e6420 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80af072943445c64 server_cookie=516bec8dc340652a in_seq=0 out_seq=0 2024-09-19T04:56:09.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.464+0000 7f4022a00640 10 client.17379 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:56:09.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.465+0000 7f4022a00640 1 -- 172.21.15.123:0/437599580 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555cc783c950 con 0x555cc78e4030 2024-09-19T04:56:09.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.465+0000 7f4022a00640 10 client.17379 handle_command_reply: tid=0 2024-09-19T04:56:09.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.465+0000 7f4046671740 1 client.17379 shutdown 2024-09-19T04:56:09.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.465+0000 7f4046671740 1 -- 172.21.15.123:0/437599580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78c9760 msgr2=0x555cc78d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:09.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.465+0000 7f4046671740 1 --2- 172.21.15.123:0/437599580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78c9760 0x555cc78d13e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4030019660 tx=0x7f4030019690 comp rx=0 tx=0).stop 2024-09-19T04:56:09.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.466+0000 7f4046671740 1 -- 172.21.15.123:0/437599580 shutdown_connections 2024-09-19T04:56:09.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.466+0000 7f4046671740 1 --2- 172.21.15.123:0/437599580 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555cc78e4030 0x555cc78e6420 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.466+0000 7f4046671740 1 --2- 172.21.15.123:0/437599580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78d4020 0x555cc78d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.466+0000 7f4046671740 1 --2- 172.21.15.123:0/437599580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78c9760 0x555cc78d13e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.466+0000 7f4046671740 1 --2- 172.21.15.123:0/437599580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c8e40 0x555cc78d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.466+0000 7f4022a00640 0 client.17379 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:56:09.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.466+0000 7f4046671740 1 -- 172.21.15.123:0/437599580 >> 172.21.15.123:0/437599580 conn(0x555cc78c53d0 msgr2=0x555cc78cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:09.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.466+0000 7f4046671740 1 -- 172.21.15.123:0/437599580 shutdown_connections 2024-09-19T04:56:09.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.466+0000 7f4046671740 1 -- 172.21.15.123:0/437599580 wait complete. 2024-09-19T04:56:09.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.466+0000 7f4046671740 20 client.17379 trim_cache size 0 max 16384 2024-09-19T04:56:09.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.479+0000 7f4046671740 1 Processor -- start 2024-09-19T04:56:09.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.479+0000 7f4046671740 1 -- start start 2024-09-19T04:56:09.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.480+0000 7f4046671740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c8e40 0x555cc78cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.480+0000 7f4046671740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78c9760 0x555cc78cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.480+0000 7f4046671740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78d4020 0x555cc78ebbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.480+0000 7f4046671740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1af880 con 0x555cc78c9760 2024-09-19T04:56:09.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.480+0000 7f4046671740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1b13c0 con 0x555cc78c8e40 2024-09-19T04:56:09.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.480+0000 7f4046671740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1aff50 con 0x555cc78d4020 2024-09-19T04:56:09.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.480+0000 7f4041000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78c9760 0x555cc78cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.480+0000 7f4041000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78c9760 0x555cc78cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56808/0 (socket says 172.21.15.123:56808) 2024-09-19T04:56:09.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.480+0000 7f4041000640 1 -- 172.21.15.123:0/2151033804 learned_addr learned my addr 172.21.15.123:0/2151033804 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:09.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.480+0000 7f403be00640 1 --2- 172.21.15.123:0/2151033804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c8e40 0x555cc78cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.480+0000 7f403be00640 1 --2- 172.21.15.123:0/2151033804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78d4020 0x555cc78ebbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.480+0000 7f4041000640 1 -- 172.21.15.123:0/2151033804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78d4020 msgr2=0x555cc78ebbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:09.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.480+0000 7f4041000640 1 --2- 172.21.15.123:0/2151033804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78d4020 0x555cc78ebbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.480+0000 7f4041000640 1 -- 172.21.15.123:0/2151033804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c8e40 msgr2=0x555cc78cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:09.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.480+0000 7f4041000640 1 --2- 172.21.15.123:0/2151033804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c8e40 0x555cc78cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.480+0000 7f4041000640 1 -- 172.21.15.123:0/2151033804 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc78c4e70 con 0x555cc78c9760 2024-09-19T04:56:09.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.480+0000 7f403be00640 1 --2- 172.21.15.123:0/2151033804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78d4020 0x555cc78ebbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:56:09.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.481+0000 7f4041000640 1 --2- 172.21.15.123:0/2151033804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78c9760 0x555cc78cf770 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4030005fd0 tx=0x7f4030009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a03df73c077b257 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:09.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.481+0000 7f402d600640 1 -- 172.21.15.123:0/2151033804 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030027040 con 0x555cc78c9760 2024-09-19T04:56:09.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.481+0000 7f402d600640 1 -- 172.21.15.123:0/2151033804 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f403001f040 con 0x555cc78c9760 2024-09-19T04:56:09.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.481+0000 7f402d600640 1 -- 172.21.15.123:0/2151033804 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403001e070 con 0x555cc78c9760 2024-09-19T04:56:09.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.481+0000 7f4046671740 1 -- 172.21.15.123:0/2151033804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78c9760 msgr2=0x555cc78cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:09.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.481+0000 7f4046671740 1 --2- 172.21.15.123:0/2151033804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78c9760 0x555cc78cf770 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4030005fd0 tx=0x7f4030009620 comp rx=0 tx=0).stop 2024-09-19T04:56:09.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.482+0000 7f4046671740 1 -- 172.21.15.123:0/2151033804 shutdown_connections 2024-09-19T04:56:09.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.482+0000 7f4046671740 1 --2- 172.21.15.123:0/2151033804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78d4020 0x555cc78ebbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.482+0000 7f4046671740 1 --2- 172.21.15.123:0/2151033804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78c9760 0x555cc78cf770 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.482+0000 7f4046671740 1 --2- 172.21.15.123:0/2151033804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c8e40 0x555cc78cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.482+0000 7f4046671740 1 -- 172.21.15.123:0/2151033804 >> 172.21.15.123:0/2151033804 conn(0x555cc78c53d0 msgr2=0x555cc78d2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:09.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.482+0000 7f4046671740 1 -- 172.21.15.123:0/2151033804 shutdown_connections 2024-09-19T04:56:09.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.482+0000 7f4046671740 1 -- 172.21.15.123:0/2151033804 wait complete. 2024-09-19T04:56:09.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.482+0000 7f4046671740 1 -- start start 2024-09-19T04:56:09.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.482+0000 7f4046671740 1 Processor -- start 2024-09-19T04:56:09.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f4046671740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c8e40 0x555cc78dc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f4046671740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78c9760 0x555cc78d7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f4046671740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78d4020 0x555cc78d7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f4046671740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1b0cf0 con 0x555cc78d4020 2024-09-19T04:56:09.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f4046671740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1b0620 con 0x555cc78c8e40 2024-09-19T04:56:09.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f4046671740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1ad880 con 0x555cc78c9760 2024-09-19T04:56:09.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f4046671740 10 client.0 _ll_register_callbacks cb 0x7f403001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:09.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f403be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78d4020 0x555cc78d7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f4041000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78c9760 0x555cc78d7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f4041000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78c9760 0x555cc78d7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:44578/0 (socket says 172.21.15.123:44578) 2024-09-19T04:56:09.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f403be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78d4020 0x555cc78d7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56824/0 (socket says 172.21.15.123:56824) 2024-09-19T04:56:09.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f4041000640 1 -- 172.21.15.123:0/2554964633 learned_addr learned my addr 172.21.15.123:0/2554964633 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:09.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f403be00640 1 --2- 172.21.15.123:0/2554964633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c8e40 0x555cc78dc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f4041000640 1 -- 172.21.15.123:0/2554964633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c8e40 msgr2=0x555cc78dc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:09.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f4041000640 1 --2- 172.21.15.123:0/2554964633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c8e40 0x555cc78dc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f4041000640 1 -- 172.21.15.123:0/2554964633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78d4020 msgr2=0x555cc78d7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:09.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f4041000640 1 --2- 172.21.15.123:0/2554964633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78d4020 0x555cc78d7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.483+0000 7f4041000640 1 -- 172.21.15.123:0/2554964633 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc78d8790 con 0x555cc78c9760 2024-09-19T04:56:09.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.484+0000 7f403be00640 1 --2- 172.21.15.123:0/2554964633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78d4020 0x555cc78d7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:09.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.484+0000 7f4041000640 1 --2- 172.21.15.123:0/2554964633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78c9760 0x555cc78d7a20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4030004c00 tx=0x7f4030009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4c2299a1b6da1b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:09.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.484+0000 7f4022a00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:56:09.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.484+0000 7f4022a00640 1 -- 172.21.15.123:0/2554964633 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403003b070 con 0x555cc78c9760 2024-09-19T04:56:09.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.484+0000 7f4046671740 1 -- 172.21.15.123:0/2554964633 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f403001f040 con 0x555cc78c9760 2024-09-19T04:56:09.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.484+0000 7f4022a00640 1 -- 172.21.15.123:0/2554964633 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f403000e780 con 0x555cc78c9760 2024-09-19T04:56:09.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.484+0000 7f4022a00640 1 -- 172.21.15.123:0/2554964633 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4030044050 con 0x555cc78c9760 2024-09-19T04:56:09.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.484+0000 7f4046671740 10 client.27487 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:09.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.484+0000 7f4046671740 1 -- 172.21.15.123:0/2554964633 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555cc7957290 con 0x555cc78c9760 2024-09-19T04:56:09.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.485+0000 7f4022a00640 1 -- 172.21.15.123:0/2554964633 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f403001f040 con 0x555cc78c9760 2024-09-19T04:56:09.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.485+0000 7f4022a00640 1 -- 172.21.15.123:0/2554964633 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f403003e020 con 0x555cc78c9760 2024-09-19T04:56:09.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.485+0000 7f4046671740 10 client.27487 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:09.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.485+0000 7f4046671740 10 client.27487 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:56:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.485+0000 7f4046671740 20 client.27487 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.485+0000 7f4046671740 1 --2- 172.21.15.123:0/2554964633 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555cc7957940 0x555cc7959d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.485+0000 7f4046671740 4 client.27487 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:56:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.485+0000 7f4046671740 1 -- 172.21.15.123:0/2554964633 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555cc7836e60 con 0x555cc7957940 2024-09-19T04:56:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.485+0000 7f403be00640 1 --2- 172.21.15.123:0/2554964633 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555cc7957940 0x555cc7959d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.486+0000 7f403be00640 1 --2- 172.21.15.123:0/2554964633 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555cc7957940 0x555cc7959d60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa11ad75d4e358c1 server_cookie=9fe20db6e45f0b71 in_seq=0 out_seq=0 2024-09-19T04:56:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.486+0000 7f4022a00640 10 client.27487 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:56:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.486+0000 7f4022a00640 1 -- 172.21.15.123:0/2554964633 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555cc7836e60 con 0x555cc7957940 2024-09-19T04:56:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.486+0000 7f4022a00640 10 client.27487 handle_command_reply: tid=0 2024-09-19T04:56:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.486+0000 7f4046671740 1 client.27487 shutdown 2024-09-19T04:56:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.486+0000 7f4046671740 1 -- 172.21.15.123:0/2554964633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78c9760 msgr2=0x555cc78d7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.486+0000 7f4046671740 1 --2- 172.21.15.123:0/2554964633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78c9760 0x555cc78d7a20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4030004c00 tx=0x7f4030009be0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.486+0000 7f4046671740 1 -- 172.21.15.123:0/2554964633 shutdown_connections 2024-09-19T04:56:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.486+0000 7f4046671740 1 --2- 172.21.15.123:0/2554964633 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555cc7957940 0x555cc7959d60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.486+0000 7f4046671740 1 --2- 172.21.15.123:0/2554964633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555cc78d4020 0x555cc78d7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.486+0000 7f4046671740 1 --2- 172.21.15.123:0/2554964633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc78c9760 0x555cc78d7a20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.486+0000 7f4046671740 1 --2- 172.21.15.123:0/2554964633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cc78c8e40 0x555cc78dc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.486+0000 7f4046671740 1 -- 172.21.15.123:0/2554964633 >> 172.21.15.123:0/2554964633 conn(0x555cc78c53d0 msgr2=0x555cc78d0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.486+0000 7f4022a00640 0 client.27487 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:56:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.486+0000 7f4046671740 1 -- 172.21.15.123:0/2554964633 shutdown_connections 2024-09-19T04:56:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.486+0000 7f4046671740 1 -- 172.21.15.123:0/2554964633 wait complete. 2024-09-19T04:56:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.487+0000 7f4046671740 20 client.27487 trim_cache size 0 max 16384 2024-09-19T04:56:09.492 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:56:09.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.487+0000 7f4042400640 1 -- 172.21.15.123:0/1109916044 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f40240a3430 msgr2=0x7f40240a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:09.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.487+0000 7f4042400640 1 --2- 172.21.15.123:0/1109916044 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f40240a3430 0x7f40240a58f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f40340b9cf0 tx=0x7f4034047fa0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.487+0000 7f4042400640 1 -- 172.21.15.123:0/1109916044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f403c108f60 msgr2=0x7f403c19ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:09.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.487+0000 7f4042400640 1 --2- 172.21.15.123:0/1109916044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f403c108f60 0x7f403c19ecd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4034031a20 tx=0x7f4034031a50 comp rx=0 tx=0).stop 2024-09-19T04:56:09.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.487+0000 7f4042400640 1 -- 172.21.15.123:0/1109916044 shutdown_connections 2024-09-19T04:56:09.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.487+0000 7f4042400640 1 --2- 172.21.15.123:0/1109916044 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f40240a3430 0x7f40240a58f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.487+0000 7f4042400640 1 --2- 172.21.15.123:0/1109916044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f403c108f60 0x7f403c19ecd0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.487+0000 7f4042400640 1 --2- 172.21.15.123:0/1109916044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f403c1047e0 0x7f403c19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.487+0000 7f4042400640 1 --2- 172.21.15.123:0/1109916044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f403c103e30 0x7f403c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:09.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.487+0000 7f4042400640 1 -- 172.21.15.123:0/1109916044 >> 172.21.15.123:0/1109916044 conn(0x7f403c0ff8d0 msgr2=0x7f403c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:09.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.487+0000 7f4042400640 1 -- 172.21.15.123:0/1109916044 shutdown_connections 2024-09-19T04:56:09.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:09.487+0000 7f4042400640 1 -- 172.21.15.123:0/1109916044 wait complete. 2024-09-19T04:56:09.502 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:09 smithi123 ceph-mon[30511]: pgmap v292: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-19T04:56:09.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:09 smithi183 ceph-mon[35333]: pgmap v292: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-19T04:56:09.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:09 smithi193 ceph-mon[35592]: pgmap v292: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-19T04:56:10.407 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:56:11.107 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:11.513 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:11 smithi123 ceph-mon[30511]: pgmap v293: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T04:56:11.513 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:11 smithi123 ceph-mon[30511]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T04:56:11.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:11 smithi183 ceph-mon[35333]: pgmap v293: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T04:56:11.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:11 smithi183 ceph-mon[35333]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T04:56:11.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:11 smithi193 ceph-mon[35592]: pgmap v293: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T04:56:11.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:11 smithi193 ceph-mon[35592]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T04:56:11.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.761+0000 7faeab400640 1 -- 172.21.15.123:0/3899644972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea410e3c0 msgr2=0x7faea4110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:11.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.761+0000 7faeab400640 1 --2- 172.21.15.123:0/3899644972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea410e3c0 0x7faea4110810 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009ca0 tx=0x7fae9c02a950 comp rx=0 tx=0).stop 2024-09-19T04:56:11.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.762+0000 7faeab400640 1 -- 172.21.15.123:0/3899644972 shutdown_connections 2024-09-19T04:56:11.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.762+0000 7faeab400640 1 --2- 172.21.15.123:0/3899644972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea410e3c0 0x7faea4110810 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.762+0000 7faeab400640 1 --2- 172.21.15.123:0/3899644972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea4074f40 0x7faea410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.762+0000 7faeab400640 1 --2- 172.21.15.123:0/3899644972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea4074620 0x7faea4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.762+0000 7faeab400640 1 -- 172.21.15.123:0/3899644972 >> 172.21.15.123:0/3899644972 conn(0x7faea40fd8c0 msgr2=0x7faea40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:11.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.762+0000 7faeab400640 1 -- 172.21.15.123:0/3899644972 shutdown_connections 2024-09-19T04:56:11.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.762+0000 7faeab400640 1 -- 172.21.15.123:0/3899644972 wait complete. 2024-09-19T04:56:11.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.763+0000 7faeab400640 1 Processor -- start 2024-09-19T04:56:11.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.763+0000 7faeab400640 1 -- start start 2024-09-19T04:56:11.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.763+0000 7faeab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea4074620 0x7faea410dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.763+0000 7faeab400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea4074f40 0x7faea410adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.763+0000 7faeab400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea410e3c0 0x7faea410b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.763+0000 7faeab400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faea4114150 con 0x7faea4074620 2024-09-19T04:56:11.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.763+0000 7faeab400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faea4113fd0 con 0x7faea4074f40 2024-09-19T04:56:11.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.763+0000 7faeab400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faea41142d0 con 0x7faea410e3c0 2024-09-19T04:56:11.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.763+0000 7faea8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea4074620 0x7faea410dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.763+0000 7faea8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea4074620 0x7faea410dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56840/0 (socket says 172.21.15.123:56840) 2024-09-19T04:56:11.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.763+0000 7faea3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea4074f40 0x7faea410adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.763+0000 7faea8a00640 1 -- 172.21.15.123:0/435172077 learned_addr learned my addr 172.21.15.123:0/435172077 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:11.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.764+0000 7faea9400640 1 --2- 172.21.15.123:0/435172077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea410e3c0 0x7faea410b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.764+0000 7faea8a00640 1 -- 172.21.15.123:0/435172077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea410e3c0 msgr2=0x7faea410b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:11.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.764+0000 7faea8a00640 1 --2- 172.21.15.123:0/435172077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea410e3c0 0x7faea410b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.764+0000 7faea8a00640 1 -- 172.21.15.123:0/435172077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea4074f40 msgr2=0x7faea410adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:11.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.764+0000 7faea8a00640 1 --2- 172.21.15.123:0/435172077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea4074f40 0x7faea410adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.764+0000 7faea8a00640 1 -- 172.21.15.123:0/435172077 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea410bc10 con 0x7faea4074620 2024-09-19T04:56:11.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.764+0000 7faea3e00640 1 --2- 172.21.15.123:0/435172077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea4074f40 0x7faea410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:11.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.764+0000 7faea8a00640 1 --2- 172.21.15.123:0/435172077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea4074620 0x7faea410dd40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fae9800e9f0 tx=0x7fae9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:11.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.764+0000 7faea1600640 1 -- 172.21.15.123:0/435172077 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9800cd80 con 0x7faea4074620 2024-09-19T04:56:11.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.765+0000 7faea1600640 1 -- 172.21.15.123:0/435172077 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae98010910 con 0x7faea4074620 2024-09-19T04:56:11.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.765+0000 7faea1600640 1 -- 172.21.15.123:0/435172077 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98018980 con 0x7faea4074620 2024-09-19T04:56:11.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.765+0000 7faeab400640 1 -- 172.21.15.123:0/435172077 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea41aafb0 con 0x7faea4074620 2024-09-19T04:56:11.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.765+0000 7faeab400640 1 -- 172.21.15.123:0/435172077 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea41ab410 con 0x7faea4074620 2024-09-19T04:56:11.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.768+0000 7faea1600640 1 -- 172.21.15.123:0/435172077 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fae98010ac0 con 0x7faea4074620 2024-09-19T04:56:11.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.768+0000 7faeae01d740 1 Processor -- start 2024-09-19T04:56:11.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.768+0000 7faeae01d740 1 -- start start 2024-09-19T04:56:11.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.768+0000 7faeae01d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c5c70 0x564a337c6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.768+0000 7faeae01d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c6590 0x564a337d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.768+0000 7faeae01d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337d0e50 0x564a337d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.768+0000 7faeae01d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564a3372c3e0 con 0x564a337d0e50 2024-09-19T04:56:11.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.768+0000 7faeae01d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564a336d3d80 con 0x564a337c5c70 2024-09-19T04:56:11.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.768+0000 7faeae01d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564a337283a0 con 0x564a337c6590 2024-09-19T04:56:11.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.769+0000 7faea9400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c5c70 0x564a337c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.769+0000 7faea8a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c6590 0x564a337d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.769+0000 7faea9400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c5c70 0x564a337c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:51584/0 (socket says 172.21.15.123:51584) 2024-09-19T04:56:11.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.769+0000 7faea9400640 1 -- 172.21.15.123:0/3526501575 learned_addr learned my addr 172.21.15.123:0/3526501575 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:11.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.769+0000 7faea3e00640 1 --2- 172.21.15.123:0/3526501575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337d0e50 0x564a337d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.769+0000 7faea3e00640 1 -- 172.21.15.123:0/3526501575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c6590 msgr2=0x564a337d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:11.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.769+0000 7faea3e00640 1 --2- 172.21.15.123:0/3526501575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c6590 0x564a337d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.769+0000 7faea3e00640 1 -- 172.21.15.123:0/3526501575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c5c70 msgr2=0x564a337c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:11.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.769+0000 7faea3e00640 1 --2- 172.21.15.123:0/3526501575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c5c70 0x564a337c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.769+0000 7faea3e00640 1 -- 172.21.15.123:0/3526501575 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a336f2ae0 con 0x564a337d0e50 2024-09-19T04:56:11.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.769+0000 7faea9400640 1 --2- 172.21.15.123:0/3526501575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c5c70 0x564a337c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:11.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.769+0000 7faea1600640 1 --2- 172.21.15.123:0/435172077 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fae8c0a33b0 0x7fae8c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.769+0000 7faea1600640 1 -- 172.21.15.123:0/435172077 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7fae980d0070 con 0x7faea4074620 2024-09-19T04:56:11.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.769+0000 7faea3e00640 1 --2- 172.21.15.123:0/3526501575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337d0e50 0x564a337d3240 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7faea410c640 tx=0x7fae90011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513ac40b0d4fa619 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:11.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.770+0000 7faea9400640 1 --2- 172.21.15.123:0/435172077 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fae8c0a33b0 0x7fae8c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.770+0000 7fae95c00640 1 -- 172.21.15.123:0/3526501575 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90004230 con 0x564a337d0e50 2024-09-19T04:56:11.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.770+0000 7fae95c00640 1 -- 172.21.15.123:0/3526501575 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae900043d0 con 0x564a337d0e50 2024-09-19T04:56:11.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.770+0000 7fae95c00640 1 -- 172.21.15.123:0/3526501575 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae900035f0 con 0x564a337d0e50 2024-09-19T04:56:11.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.771+0000 7faeae01d740 1 -- 172.21.15.123:0/3526501575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337d0e50 msgr2=0x564a337d3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:11.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.771+0000 7faea9400640 1 --2- 172.21.15.123:0/435172077 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fae8c0a33b0 0x7fae8c0a5870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fae9c030040 tx=0x7fae9c005c50 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:11.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.771+0000 7faeae01d740 1 --2- 172.21.15.123:0/3526501575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337d0e50 0x564a337d3240 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7faea410c640 tx=0x7fae90011810 comp rx=0 tx=0).stop 2024-09-19T04:56:11.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.771+0000 7faeae01d740 1 -- 172.21.15.123:0/3526501575 shutdown_connections 2024-09-19T04:56:11.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.771+0000 7faeae01d740 1 --2- 172.21.15.123:0/3526501575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337d0e50 0x564a337d3240 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.771+0000 7faeae01d740 1 --2- 172.21.15.123:0/3526501575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c6590 0x564a337d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.771+0000 7faeae01d740 1 --2- 172.21.15.123:0/3526501575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c5c70 0x564a337c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.771+0000 7faeae01d740 1 -- 172.21.15.123:0/3526501575 >> 172.21.15.123:0/3526501575 conn(0x564a337c2200 msgr2=0x564a337c25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:11.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.771+0000 7faeae01d740 1 -- 172.21.15.123:0/3526501575 shutdown_connections 2024-09-19T04:56:11.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.771+0000 7faeae01d740 1 -- 172.21.15.123:0/3526501575 wait complete. 2024-09-19T04:56:11.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.771+0000 7faeae01d740 1 -- start start 2024-09-19T04:56:11.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.772+0000 7faeae01d740 1 Processor -- start 2024-09-19T04:56:11.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.772+0000 7faeae01d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c5c70 0x564a337cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.772+0000 7faeae01d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c6590 0x564a337ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.772+0000 7faeae01d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337d0e50 0x564a337ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.772+0000 7faeae01d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564a337d3a20 con 0x564a337d0e50 2024-09-19T04:56:11.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.772+0000 7faeae01d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564a336e70f0 con 0x564a337c6590 2024-09-19T04:56:11.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.772+0000 7faeae01d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564a337d3ba0 con 0x564a337c5c70 2024-09-19T04:56:11.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.772+0000 7faeae01d740 10 client.0 _ll_register_callbacks cb 0x564a336cebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:11.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.772+0000 7faea8a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c5c70 0x564a337cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.772+0000 7faea8a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c5c70 0x564a337cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:44624/0 (socket says 172.21.15.123:44624) 2024-09-19T04:56:11.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.772+0000 7faea3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c6590 0x564a337ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.772+0000 7faea8a00640 1 -- 172.21.15.123:0/1143215435 learned_addr learned my addr 172.21.15.123:0/1143215435 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:11.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.772+0000 7faea9400640 1 --2- 172.21.15.123:0/1143215435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337d0e50 0x564a337ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.773+0000 7faea8a00640 1 -- 172.21.15.123:0/1143215435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c6590 msgr2=0x564a337ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:11.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.773+0000 7faea8a00640 1 --2- 172.21.15.123:0/1143215435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c6590 0x564a337ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.773+0000 7faea8a00640 1 -- 172.21.15.123:0/1143215435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337d0e50 msgr2=0x564a337ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:11.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.773+0000 7faea8a00640 1 --2- 172.21.15.123:0/1143215435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337d0e50 0x564a337ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.773+0000 7faea8a00640 1 -- 172.21.15.123:0/1143215435 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a3349d3c0 con 0x564a337c5c70 2024-09-19T04:56:11.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.773+0000 7faea8a00640 1 --2- 172.21.15.123:0/1143215435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c5c70 0x564a337cdce0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fae9800e9c0 tx=0x7fae980986e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3130253bdaf92d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:11.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.773+0000 7fae8a000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:56:11.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.773+0000 7fae8a000640 1 -- 172.21.15.123:0/1143215435 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9809c380 con 0x564a337c5c70 2024-09-19T04:56:11.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.773+0000 7faeae01d740 1 -- 172.21.15.123:0/1143215435 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae900043d0 con 0x564a337c5c70 2024-09-19T04:56:11.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.773+0000 7fae8a000640 1 -- 172.21.15.123:0/1143215435 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7fae980cd9c0 con 0x564a337c5c70 2024-09-19T04:56:11.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.773+0000 7fae8a000640 1 -- 172.21.15.123:0/1143215435 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fae980cdbc0 con 0x564a337c5c70 2024-09-19T04:56:11.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.773+0000 7faeae01d740 10 client.27499 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:11.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.773+0000 7faeae01d740 1 -- 172.21.15.123:0/1143215435 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564a337df9f0 con 0x564a337c5c70 2024-09-19T04:56:11.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.774+0000 7fae8a000640 1 -- 172.21.15.123:0/1143215435 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae900043d0 con 0x564a337c5c70 2024-09-19T04:56:11.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.774+0000 7fae8a000640 1 -- 172.21.15.123:0/1143215435 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fae9809b7d0 con 0x564a337c5c70 2024-09-19T04:56:11.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.774+0000 7faeae01d740 10 client.27499 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:11.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.774+0000 7faeae01d740 10 client.27499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:56:11.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.774+0000 7faeae01d740 20 client.27499 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:11.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.774+0000 7faeae01d740 1 --2- 172.21.15.123:0/1143215435 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x564a337e0e00 0x564a337e3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.774+0000 7faeae01d740 4 client.27499 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:56:11.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.774+0000 7faeae01d740 1 -- 172.21.15.123:0/1143215435 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a33739950 con 0x564a337e0e00 2024-09-19T04:56:11.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.774+0000 7faea3e00640 1 --2- 172.21.15.123:0/1143215435 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x564a337e0e00 0x564a337e3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.775+0000 7faea3e00640 1 --2- 172.21.15.123:0/1143215435 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x564a337e0e00 0x564a337e3210 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6f3709c35020fb7 server_cookie=5827bae8d022a360 in_seq=0 out_seq=0 2024-09-19T04:56:11.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.775+0000 7fae8a000640 10 client.27499 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:11.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.776+0000 7fae8a000640 1 -- 172.21.15.123:0/1143215435 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a33739950 con 0x564a337e0e00 2024-09-19T04:56:11.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.776+0000 7fae8a000640 10 client.27499 handle_command_reply: tid=0 2024-09-19T04:56:11.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.776+0000 7faeae01d740 1 client.27499 shutdown 2024-09-19T04:56:11.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.776+0000 7faeae01d740 1 -- 172.21.15.123:0/1143215435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c5c70 msgr2=0x564a337cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:11.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.776+0000 7faeae01d740 1 --2- 172.21.15.123:0/1143215435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c5c70 0x564a337cdce0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fae9800e9c0 tx=0x7fae980986e0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.776+0000 7faeae01d740 1 -- 172.21.15.123:0/1143215435 shutdown_connections 2024-09-19T04:56:11.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.776+0000 7faeae01d740 1 --2- 172.21.15.123:0/1143215435 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x564a337e0e00 0x564a337e3210 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.776+0000 7faeae01d740 1 --2- 172.21.15.123:0/1143215435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337d0e50 0x564a337ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.776+0000 7fae8a000640 0 client.27499 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:11.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.776+0000 7faeae01d740 1 --2- 172.21.15.123:0/1143215435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c6590 0x564a337ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.776+0000 7faeae01d740 1 --2- 172.21.15.123:0/1143215435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c5c70 0x564a337cdce0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.776+0000 7faeae01d740 1 -- 172.21.15.123:0/1143215435 >> 172.21.15.123:0/1143215435 conn(0x564a337c2200 msgr2=0x564a337c7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:11.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.776+0000 7faeae01d740 1 -- 172.21.15.123:0/1143215435 shutdown_connections 2024-09-19T04:56:11.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.776+0000 7faeae01d740 1 -- 172.21.15.123:0/1143215435 wait complete. 2024-09-19T04:56:11.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.776+0000 7faeae01d740 20 client.27499 trim_cache size 0 max 16384 2024-09-19T04:56:11.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.790+0000 7faeae01d740 1 Processor -- start 2024-09-19T04:56:11.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.790+0000 7faeae01d740 1 -- start start 2024-09-19T04:56:11.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.790+0000 7faeae01d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c5c70 0x564a337c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.790+0000 7faeae01d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337c6590 0x564a337e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.790+0000 7faeae01d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337d0e50 0x564a337e8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.790+0000 7faeae01d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faea41b3f90 con 0x564a337c6590 2024-09-19T04:56:11.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.790+0000 7faeae01d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faea41b5ad0 con 0x564a337d0e50 2024-09-19T04:56:11.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.790+0000 7faeae01d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faea41b4660 con 0x564a337c5c70 2024-09-19T04:56:11.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.790+0000 7faea3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337c6590 0x564a337e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.790+0000 7faea3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337c6590 0x564a337e8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56858/0 (socket says 172.21.15.123:56858) 2024-09-19T04:56:11.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.790+0000 7faea3e00640 1 -- 172.21.15.123:0/3843676881 learned_addr learned my addr 172.21.15.123:0/3843676881 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:11.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.791+0000 7faea8a00640 1 --2- 172.21.15.123:0/3843676881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c5c70 0x564a337c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.791+0000 7faea9400640 1 --2- 172.21.15.123:0/3843676881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337d0e50 0x564a337e8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.791+0000 7faea3e00640 1 -- 172.21.15.123:0/3843676881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c5c70 msgr2=0x564a337c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:11.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.791+0000 7faea3e00640 1 --2- 172.21.15.123:0/3843676881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c5c70 0x564a337c7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.791+0000 7faea3e00640 1 -- 172.21.15.123:0/3843676881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337d0e50 msgr2=0x564a337e8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:11.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.791+0000 7faea3e00640 1 --2- 172.21.15.123:0/3843676881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337d0e50 0x564a337e8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.791+0000 7faea3e00640 1 -- 172.21.15.123:0/3843676881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a337c2030 con 0x564a337c6590 2024-09-19T04:56:11.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.791+0000 7faea9400640 1 --2- 172.21.15.123:0/3843676881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337d0e50 0x564a337e8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:56:11.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.791+0000 7faea3e00640 1 --2- 172.21.15.123:0/3843676881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337c6590 0x564a337e8a80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fae90009ef0 tx=0x7fae90009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f3489bf47c9567 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:11.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.791+0000 7fae95200640 1 -- 172.21.15.123:0/3843676881 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90032070 con 0x564a337c6590 2024-09-19T04:56:11.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.791+0000 7fae95200640 1 -- 172.21.15.123:0/3843676881 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae9001f040 con 0x564a337c6590 2024-09-19T04:56:11.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.792+0000 7fae95200640 1 -- 172.21.15.123:0/3843676881 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9001e040 con 0x564a337c6590 2024-09-19T04:56:11.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.792+0000 7faeae01d740 1 -- 172.21.15.123:0/3843676881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337c6590 msgr2=0x564a337e8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:11.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.792+0000 7faeae01d740 1 --2- 172.21.15.123:0/3843676881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337c6590 0x564a337e8a80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fae90009ef0 tx=0x7fae90009b50 comp rx=0 tx=0).stop 2024-09-19T04:56:11.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.792+0000 7faeae01d740 1 -- 172.21.15.123:0/3843676881 shutdown_connections 2024-09-19T04:56:11.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.792+0000 7faeae01d740 1 --2- 172.21.15.123:0/3843676881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337d0e50 0x564a337e8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.792+0000 7faeae01d740 1 --2- 172.21.15.123:0/3843676881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337c6590 0x564a337e8a80 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.792+0000 7faeae01d740 1 --2- 172.21.15.123:0/3843676881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c5c70 0x564a337c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.792+0000 7faeae01d740 1 -- 172.21.15.123:0/3843676881 >> 172.21.15.123:0/3843676881 conn(0x564a337c2200 msgr2=0x564a337c76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.792+0000 7faeae01d740 1 -- 172.21.15.123:0/3843676881 shutdown_connections 2024-09-19T04:56:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.792+0000 7faeae01d740 1 -- 172.21.15.123:0/3843676881 wait complete. 2024-09-19T04:56:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.793+0000 7faeae01d740 1 -- start start 2024-09-19T04:56:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.793+0000 7faeae01d740 1 Processor -- start 2024-09-19T04:56:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.793+0000 7faeae01d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c5c70 0x564a337d4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.793+0000 7faeae01d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c6590 0x564a337d5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.793+0000 7faeae01d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337d0e50 0x564a3384aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.793+0000 7faeae01d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faea41b5400 con 0x564a337d0e50 2024-09-19T04:56:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.794+0000 7faeae01d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faea41b4d30 con 0x564a337c5c70 2024-09-19T04:56:11.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.794+0000 7faeae01d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faea41b1f90 con 0x564a337c6590 2024-09-19T04:56:11.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.794+0000 7faeae01d740 10 client.0 _ll_register_callbacks cb 0x7fae98009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:11.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.794+0000 7faea3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c6590 0x564a337d5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.794+0000 7faea3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c6590 0x564a337d5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:44652/0 (socket says 172.21.15.123:44652) 2024-09-19T04:56:11.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.794+0000 7faea9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337d0e50 0x564a3384aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.794+0000 7faea3e00640 1 -- 172.21.15.123:0/2883600529 learned_addr learned my addr 172.21.15.123:0/2883600529 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:11.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.794+0000 7faea8a00640 1 --2- 172.21.15.123:0/2883600529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c5c70 0x564a337d4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.794+0000 7faea3e00640 1 -- 172.21.15.123:0/2883600529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c5c70 msgr2=0x564a337d4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.794+0000 7faea3e00640 1 --2- 172.21.15.123:0/2883600529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c5c70 0x564a337d4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.794+0000 7faea3e00640 1 -- 172.21.15.123:0/2883600529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337d0e50 msgr2=0x564a3384aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.794+0000 7faea3e00640 1 --2- 172.21.15.123:0/2883600529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337d0e50 0x564a3384aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.794+0000 7faea3e00640 1 -- 172.21.15.123:0/2883600529 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a337bd770 con 0x564a337c6590 2024-09-19T04:56:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.794+0000 7faea8a00640 1 --2- 172.21.15.123:0/2883600529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c5c70 0x564a337d4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:56:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.794+0000 7faea3e00640 1 --2- 172.21.15.123:0/2883600529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c6590 0x564a337d5400 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fae90005ab0 tx=0x7fae90009d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e1a66d7164315dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.794+0000 7fae8a000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:56:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.795+0000 7fae8a000640 1 -- 172.21.15.123:0/2883600529 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90032070 con 0x564a337c6590 2024-09-19T04:56:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.795+0000 7faeae01d740 1 -- 172.21.15.123:0/2883600529 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae9001f040 con 0x564a337c6590 2024-09-19T04:56:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.795+0000 7fae8a000640 1 -- 172.21.15.123:0/2883600529 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7fae90026c00 con 0x564a337c6590 2024-09-19T04:56:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.795+0000 7fae8a000640 1 -- 172.21.15.123:0/2883600529 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fae90026440 con 0x564a337c6590 2024-09-19T04:56:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.795+0000 7faeae01d740 10 client.27505 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.795+0000 7faeae01d740 1 -- 172.21.15.123:0/2883600529 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564a3384bed0 con 0x564a337c6590 2024-09-19T04:56:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.795+0000 7fae8a000640 1 -- 172.21.15.123:0/2883600529 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae9001f040 con 0x564a337c6590 2024-09-19T04:56:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.796+0000 7fae8a000640 1 -- 172.21.15.123:0/2883600529 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fae90024040 con 0x564a337c6590 2024-09-19T04:56:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.796+0000 7faeae01d740 10 client.27505 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.796+0000 7faeae01d740 10 client.27505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:56:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.796+0000 7faeae01d740 20 client.27505 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.796+0000 7faeae01d740 1 --2- 172.21.15.123:0/2883600529 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x564a3384c3a0 0x564a3384e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.796+0000 7faeae01d740 4 client.27505 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:56:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.796+0000 7faeae01d740 1 -- 172.21.15.123:0/2883600529 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a33733e60 con 0x564a3384c3a0 2024-09-19T04:56:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.796+0000 7faea8a00640 1 --2- 172.21.15.123:0/2883600529 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x564a3384c3a0 0x564a3384e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.796+0000 7faea8a00640 1 --2- 172.21.15.123:0/2883600529 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x564a3384c3a0 0x564a3384e7c0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4386341fb1787b77 server_cookie=8cbd413687161929 in_seq=0 out_seq=0 2024-09-19T04:56:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.796+0000 7fae8a000640 10 client.27505 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.797+0000 7fae8a000640 1 -- 172.21.15.123:0/2883600529 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a33733e60 con 0x564a3384c3a0 2024-09-19T04:56:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.797+0000 7fae8a000640 10 client.27505 handle_command_reply: tid=0 2024-09-19T04:56:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.797+0000 7faeae01d740 1 client.27505 shutdown 2024-09-19T04:56:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.797+0000 7faeae01d740 1 -- 172.21.15.123:0/2883600529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c6590 msgr2=0x564a337d5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.797+0000 7faeae01d740 1 --2- 172.21.15.123:0/2883600529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c6590 0x564a337d5400 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fae90005ab0 tx=0x7fae90009d20 comp rx=0 tx=0).stop 2024-09-19T04:56:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.797+0000 7faeae01d740 1 -- 172.21.15.123:0/2883600529 shutdown_connections 2024-09-19T04:56:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.797+0000 7faeae01d740 1 --2- 172.21.15.123:0/2883600529 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x564a3384c3a0 0x564a3384e7c0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.797+0000 7faeae01d740 1 --2- 172.21.15.123:0/2883600529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a337d0e50 0x564a3384aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.797+0000 7faeae01d740 1 --2- 172.21.15.123:0/2883600529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a337c6590 0x564a337d5400 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.797+0000 7faeae01d740 1 --2- 172.21.15.123:0/2883600529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a337c5c70 0x564a337d4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.797+0000 7fae8a000640 0 client.27505 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.797+0000 7faeae01d740 1 -- 172.21.15.123:0/2883600529 >> 172.21.15.123:0/2883600529 conn(0x564a337c2200 msgr2=0x564a337c1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.797+0000 7faeae01d740 1 -- 172.21.15.123:0/2883600529 shutdown_connections 2024-09-19T04:56:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.797+0000 7faeae01d740 1 -- 172.21.15.123:0/2883600529 wait complete. 2024-09-19T04:56:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.797+0000 7faeae01d740 20 client.27505 trim_cache size 0 max 16384 2024-09-19T04:56:11.803 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:56:11.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.798+0000 7faeab400640 1 -- 172.21.15.123:0/435172077 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fae8c0a33b0 msgr2=0x7fae8c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:11.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.798+0000 7faeab400640 1 --2- 172.21.15.123:0/435172077 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fae8c0a33b0 0x7fae8c0a5870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fae9c030040 tx=0x7fae9c005c50 comp rx=0 tx=0).stop 2024-09-19T04:56:11.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.798+0000 7faeab400640 1 -- 172.21.15.123:0/435172077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea4074620 msgr2=0x7faea410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:11.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.798+0000 7faeab400640 1 --2- 172.21.15.123:0/435172077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea4074620 0x7faea410dd40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fae9800e9f0 tx=0x7fae9800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.798+0000 7faeab400640 1 -- 172.21.15.123:0/435172077 shutdown_connections 2024-09-19T04:56:11.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.798+0000 7faeab400640 1 --2- 172.21.15.123:0/435172077 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fae8c0a33b0 0x7fae8c0a5870 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.798+0000 7faeab400640 1 --2- 172.21.15.123:0/435172077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea410e3c0 0x7faea410b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.798+0000 7faeab400640 1 --2- 172.21.15.123:0/435172077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea4074f40 0x7faea410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.798+0000 7faeab400640 1 --2- 172.21.15.123:0/435172077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea4074620 0x7faea410dd40 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:11.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.798+0000 7faeab400640 1 -- 172.21.15.123:0/435172077 >> 172.21.15.123:0/435172077 conn(0x7faea40fd8c0 msgr2=0x7faea40fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:11.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.798+0000 7faeab400640 1 -- 172.21.15.123:0/435172077 shutdown_connections 2024-09-19T04:56:11.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:11.798+0000 7faeab400640 1 -- 172.21.15.123:0/435172077 wait complete. 2024-09-19T04:56:13.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:13 smithi183 ceph-mon[35333]: pgmap v295: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T04:56:13.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:13 smithi123 ceph-mon[30511]: pgmap v295: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T04:56:13.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:13 smithi193 ceph-mon[35592]: pgmap v295: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T04:56:13.895 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:56:13.895 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:56:14.276 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:14.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.844+0000 7f85d0e00640 1 -- 172.21.15.123:0/3017342345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85cc106f10 msgr2=0x7f85cc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:14.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.844+0000 7f85d0e00640 1 --2- 172.21.15.123:0/3017342345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85cc106f10 0x7f85cc10ea90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009ca0 tx=0x7f85c402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.844+0000 7f85d0e00640 1 -- 172.21.15.123:0/3017342345 shutdown_connections 2024-09-19T04:56:14.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.844+0000 7f85d0e00640 1 --2- 172.21.15.123:0/3017342345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85cc106f10 0x7f85cc10ea90 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.844+0000 7f85d0e00640 1 --2- 172.21.15.123:0/3017342345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85cc102790 0x7f85cc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.844+0000 7f85d0e00640 1 --2- 172.21.15.123:0/3017342345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc101de0 0x7f85cc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.844+0000 7f85d0e00640 1 -- 172.21.15.123:0/3017342345 >> 172.21.15.123:0/3017342345 conn(0x7f85cc0fd8c0 msgr2=0x7f85cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:14.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.845+0000 7f85d0e00640 1 -- 172.21.15.123:0/3017342345 shutdown_connections 2024-09-19T04:56:14.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.845+0000 7f85d0e00640 1 -- 172.21.15.123:0/3017342345 wait complete. 2024-09-19T04:56:14.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.845+0000 7f85d0e00640 1 Processor -- start 2024-09-19T04:56:14.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.846+0000 7f85d0e00640 1 -- start start 2024-09-19T04:56:14.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.846+0000 7f85d0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85cc101de0 0x7f85cc19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.846+0000 7f85d0e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85cc102790 0x7f85cc19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.846+0000 7f85d0e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc106f10 0x7f85cc19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.846+0000 7f85d0e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc111fc0 con 0x7f85cc101de0 2024-09-19T04:56:14.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.846+0000 7f85d0e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc111e40 con 0x7f85cc106f10 2024-09-19T04:56:14.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.846+0000 7f85d0e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc112140 con 0x7f85cc102790 2024-09-19T04:56:14.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.846+0000 7f85c9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85cc101de0 0x7f85cc19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.846+0000 7f85c9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85cc101de0 0x7f85cc19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53420/0 (socket says 172.21.15.123:53420) 2024-09-19T04:56:14.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.846+0000 7f85c9e00640 1 -- 172.21.15.123:0/3172273268 learned_addr learned my addr 172.21.15.123:0/3172273268 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:14.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.846+0000 7f85ca800640 1 --2- 172.21.15.123:0/3172273268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc106f10 0x7f85cc19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.846+0000 7f85c1400640 1 --2- 172.21.15.123:0/3172273268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85cc102790 0x7f85cc19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.847+0000 7f85c9e00640 1 -- 172.21.15.123:0/3172273268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85cc102790 msgr2=0x7f85cc19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:14.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.847+0000 7f85c9e00640 1 --2- 172.21.15.123:0/3172273268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85cc102790 0x7f85cc19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.847+0000 7f85c9e00640 1 -- 172.21.15.123:0/3172273268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc106f10 msgr2=0x7f85cc19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:14.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.847+0000 7f85c9e00640 1 --2- 172.21.15.123:0/3172273268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc106f10 0x7f85cc19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.847+0000 7f85c9e00640 1 -- 172.21.15.123:0/3172273268 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85cc19f600 con 0x7f85cc101de0 2024-09-19T04:56:14.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.847+0000 7f85ca800640 1 --2- 172.21.15.123:0/3172273268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc106f10 0x7f85cc19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:14.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.847+0000 7f85c9e00640 1 --2- 172.21.15.123:0/3172273268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85cc101de0 0x7f85cc19a650 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00e9f0 tx=0x7f85bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:14.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.847+0000 7f85c3400640 1 -- 172.21.15.123:0/3172273268 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc00cd80 con 0x7f85cc101de0 2024-09-19T04:56:14.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.847+0000 7f85c3400640 1 -- 172.21.15.123:0/3172273268 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85bc010910 con 0x7f85cc101de0 2024-09-19T04:56:14.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.847+0000 7f85d0e00640 1 -- 172.21.15.123:0/3172273268 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85cc19f8f0 con 0x7f85cc101de0 2024-09-19T04:56:14.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.847+0000 7f85c3400640 1 -- 172.21.15.123:0/3172273268 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc018990 con 0x7f85cc101de0 2024-09-19T04:56:14.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.848+0000 7f85d0e00640 1 -- 172.21.15.123:0/3172273268 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85cc1a71d0 con 0x7f85cc101de0 2024-09-19T04:56:14.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.851+0000 7f85c3400640 1 -- 172.21.15.123:0/3172273268 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f85bc010ac0 con 0x7f85cc101de0 2024-09-19T04:56:14.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.851+0000 7f85d37bc740 1 Processor -- start 2024-09-19T04:56:14.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.851+0000 7f85d37bc740 1 -- start start 2024-09-19T04:56:14.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.851+0000 7f85d37bc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c67897ac0 0x563c67897ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.851+0000 7f85d37bc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c678983e0 0x563c678a2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.851+0000 7f85d37bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678a2ca0 0x563c678a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.851+0000 7f85d37bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563c677fe290 con 0x563c678a2ca0 2024-09-19T04:56:14.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.851+0000 7f85d37bc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563c677a5b40 con 0x563c67897ac0 2024-09-19T04:56:14.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.851+0000 7f85d37bc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563c677fa250 con 0x563c678983e0 2024-09-19T04:56:14.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.852+0000 7f85c9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c678983e0 0x563c678a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.852+0000 7f85ca800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c67897ac0 0x563c67897ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.852+0000 7f85c1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678a2ca0 0x563c678a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.852+0000 7f85c9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c678983e0 0x563c678a2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53652/0 (socket says 172.21.15.123:53652) 2024-09-19T04:56:14.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.852+0000 7f85ca800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c67897ac0 0x563c67897ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:35064/0 (socket says 172.21.15.123:35064) 2024-09-19T04:56:14.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.852+0000 7f85c9e00640 1 -- 172.21.15.123:0/4003291024 learned_addr learned my addr 172.21.15.123:0/4003291024 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.852+0000 7f85ca800640 1 -- 172.21.15.123:0/4003291024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c678983e0 msgr2=0x563c678a2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.852+0000 7f85ca800640 1 --2- 172.21.15.123:0/4003291024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c678983e0 0x563c678a2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.852+0000 7f85ca800640 1 -- 172.21.15.123:0/4003291024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678a2ca0 msgr2=0x563c678a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.852+0000 7f85ca800640 1 --2- 172.21.15.123:0/4003291024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678a2ca0 0x563c678a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.852+0000 7f85ca800640 1 -- 172.21.15.123:0/4003291024 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c6756f6a0 con 0x563c67897ac0 2024-09-19T04:56:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.852+0000 7f85c1400640 1 --2- 172.21.15.123:0/4003291024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678a2ca0 0x563c678a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.852+0000 7f85c3400640 1 --2- 172.21.15.123:0/3172273268 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85b00a33b0 0x7f85b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.853+0000 7f85ca800640 1 --2- 172.21.15.123:0/4003291024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c67897ac0 0x563c67897ea0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f85c4005e30 tx=0x7f85c4033000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33743680577c3ab9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.853+0000 7f85c3400640 1 -- 172.21.15.123:0/3172273268 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7f85bc0d00b0 con 0x7f85cc101de0 2024-09-19T04:56:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.853+0000 7f85c9e00640 1 --2- 172.21.15.123:0/3172273268 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85b00a33b0 0x7f85b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.853+0000 7f85bb400640 1 -- 172.21.15.123:0/4003291024 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c4003b70 con 0x563c67897ac0 2024-09-19T04:56:14.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.853+0000 7f85bb400640 1 -- 172.21.15.123:0/4003291024 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c4036040 con 0x563c67897ac0 2024-09-19T04:56:14.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.853+0000 7f85bb400640 1 -- 172.21.15.123:0/4003291024 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c40043e0 con 0x563c67897ac0 2024-09-19T04:56:14.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.854+0000 7f85d37bc740 1 -- 172.21.15.123:0/4003291024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c67897ac0 msgr2=0x563c67897ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:14.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.854+0000 7f85d37bc740 1 --2- 172.21.15.123:0/4003291024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c67897ac0 0x563c67897ea0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f85c4005e30 tx=0x7f85c4033000 comp rx=0 tx=0).stop 2024-09-19T04:56:14.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.854+0000 7f85c9e00640 1 --2- 172.21.15.123:0/3172273268 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85b00a33b0 0x7f85b00a5870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f85bc09c0c0 tx=0x7f85bc0d3b60 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:14.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.854+0000 7f85d37bc740 1 -- 172.21.15.123:0/4003291024 shutdown_connections 2024-09-19T04:56:14.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.854+0000 7f85d37bc740 1 --2- 172.21.15.123:0/4003291024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678a2ca0 0x563c678a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.854+0000 7f85d37bc740 1 --2- 172.21.15.123:0/4003291024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c678983e0 0x563c678a2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.854+0000 7f85d37bc740 1 --2- 172.21.15.123:0/4003291024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c67897ac0 0x563c67897ea0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.854+0000 7f85d37bc740 1 -- 172.21.15.123:0/4003291024 >> 172.21.15.123:0/4003291024 conn(0x563c67894050 msgr2=0x563c67894430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:14.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.854+0000 7f85d37bc740 1 -- 172.21.15.123:0/4003291024 shutdown_connections 2024-09-19T04:56:14.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.854+0000 7f85d37bc740 1 -- 172.21.15.123:0/4003291024 wait complete. 2024-09-19T04:56:14.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.854+0000 7f85d37bc740 1 -- start start 2024-09-19T04:56:14.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.855+0000 7f85d37bc740 1 Processor -- start 2024-09-19T04:56:14.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.855+0000 7f85d37bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c67897ac0 0x563c678a2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.855+0000 7f85d37bc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c678983e0 0x563c6789f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.855+0000 7f85d37bc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c678a2ca0 0x563c6789fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.855+0000 7f85d37bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563c678a5790 con 0x563c67897ac0 2024-09-19T04:56:14.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.855+0000 7f85d37bc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563c677b8fa0 con 0x563c678983e0 2024-09-19T04:56:14.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.855+0000 7f85d37bc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563c678a5910 con 0x563c678a2ca0 2024-09-19T04:56:14.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.855+0000 7f85d37bc740 10 client.0 _ll_register_callbacks cb 0x563c6750cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:14.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.855+0000 7f85c1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c678983e0 0x563c6789f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.855+0000 7f85ca800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c67897ac0 0x563c678a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.856+0000 7f85c1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c678983e0 0x563c6789f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:35072/0 (socket says 172.21.15.123:35072) 2024-09-19T04:56:14.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.856+0000 7f85c1400640 1 -- 172.21.15.123:0/2753815103 learned_addr learned my addr 172.21.15.123:0/2753815103 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:14.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.856+0000 7f85ca800640 1 --2- 172.21.15.123:0/2753815103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c678a2ca0 0x563c6789fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.856+0000 7f85c1400640 1 -- 172.21.15.123:0/2753815103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c678a2ca0 msgr2=0x563c6789fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:14.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.856+0000 7f85c1400640 1 --2- 172.21.15.123:0/2753815103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c678a2ca0 0x563c6789fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.856+0000 7f85c1400640 1 -- 172.21.15.123:0/2753815103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c67897ac0 msgr2=0x563c678a2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:14.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.856+0000 7f85c1400640 1 --2- 172.21.15.123:0/2753815103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c67897ac0 0x563c678a2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.856+0000 7f85c1400640 1 -- 172.21.15.123:0/2753815103 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c678a0640 con 0x563c678983e0 2024-09-19T04:56:14.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.856+0000 7f85ca800640 1 --2- 172.21.15.123:0/2753815103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c678a2ca0 0x563c6789fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:56:14.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.856+0000 7f85c1400640 1 --2- 172.21.15.123:0/2753815103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c678983e0 0x563c6789f8b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f85b4012700 tx=0x7f85b4012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9a2ae23a8263223 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:14.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.856+0000 7f85afe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:56:14.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.856+0000 7f85afe00640 1 -- 172.21.15.123:0/2753815103 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b400f8c0 con 0x563c678983e0 2024-09-19T04:56:14.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.857+0000 7f85d37bc740 1 -- 172.21.15.123:0/2753815103 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85c4036040 con 0x563c678983e0 2024-09-19T04:56:14.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.857+0000 7f85afe00640 1 -- 172.21.15.123:0/2753815103 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7f85b400fa60 con 0x563c678983e0 2024-09-19T04:56:14.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.857+0000 7f85afe00640 1 -- 172.21.15.123:0/2753815103 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f85b400fc60 con 0x563c678983e0 2024-09-19T04:56:14.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.857+0000 7f85d37bc740 10 client.27506 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:14.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.857+0000 7f85d37bc740 1 -- 172.21.15.123:0/2753815103 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563c678b1780 con 0x563c678983e0 2024-09-19T04:56:14.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.857+0000 7f85afe00640 1 -- 172.21.15.123:0/2753815103 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c4036040 con 0x563c678983e0 2024-09-19T04:56:14.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.857+0000 7f85afe00640 1 -- 172.21.15.123:0/2753815103 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f85b4010550 con 0x563c678983e0 2024-09-19T04:56:14.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.857+0000 7f85d37bc740 10 client.27506 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:14.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.857+0000 7f85d37bc740 10 client.27506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:56:14.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.857+0000 7f85d37bc740 20 client.27506 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:14.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.857+0000 7f85d37bc740 1 --2- 172.21.15.123:0/2753815103 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563c678b2c40 0x563c678b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.857+0000 7f85d37bc740 4 client.27506 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:56:14.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.857+0000 7f85d37bc740 1 -- 172.21.15.123:0/2753815103 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c6780b800 con 0x563c678b2c40 2024-09-19T04:56:14.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.858+0000 7f85ca800640 1 --2- 172.21.15.123:0/2753815103 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563c678b2c40 0x563c678b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.858+0000 7f85ca800640 1 --2- 172.21.15.123:0/2753815103 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563c678b2c40 0x563c678b5060 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d85c6439d2021530 server_cookie=7f0d27b2f51fdd4f in_seq=0 out_seq=0 2024-09-19T04:56:14.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.858+0000 7f85afe00640 10 client.27506 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:14.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.859+0000 7f85afe00640 1 -- 172.21.15.123:0/2753815103 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c6780b800 con 0x563c678b2c40 2024-09-19T04:56:14.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.859+0000 7f85afe00640 10 client.27506 handle_command_reply: tid=0 2024-09-19T04:56:14.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.859+0000 7f85d37bc740 1 client.27506 shutdown 2024-09-19T04:56:14.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.859+0000 7f85d37bc740 1 -- 172.21.15.123:0/2753815103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c678983e0 msgr2=0x563c6789f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:14.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.859+0000 7f85d37bc740 1 --2- 172.21.15.123:0/2753815103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c678983e0 0x563c6789f8b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f85b4012700 tx=0x7f85b4012bc0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.859+0000 7f85d37bc740 1 -- 172.21.15.123:0/2753815103 shutdown_connections 2024-09-19T04:56:14.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.859+0000 7f85d37bc740 1 --2- 172.21.15.123:0/2753815103 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563c678b2c40 0x563c678b5060 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.859+0000 7f85d37bc740 1 --2- 172.21.15.123:0/2753815103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c678a2ca0 0x563c6789fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.859+0000 7f85d37bc740 1 --2- 172.21.15.123:0/2753815103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c678983e0 0x563c6789f8b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.859+0000 7f85afe00640 0 client.27506 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:14.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.859+0000 7f85d37bc740 1 --2- 172.21.15.123:0/2753815103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c67897ac0 0x563c678a2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.859+0000 7f85d37bc740 1 -- 172.21.15.123:0/2753815103 >> 172.21.15.123:0/2753815103 conn(0x563c67894050 msgr2=0x563c678931c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:14.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.860+0000 7f85d37bc740 1 -- 172.21.15.123:0/2753815103 shutdown_connections 2024-09-19T04:56:14.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.860+0000 7f85d37bc740 1 -- 172.21.15.123:0/2753815103 wait complete. 2024-09-19T04:56:14.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.860+0000 7f85d37bc740 20 client.27506 trim_cache size 0 max 16384 2024-09-19T04:56:14.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.877+0000 7f85d37bc740 1 Processor -- start 2024-09-19T04:56:14.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.877+0000 7f85d37bc740 1 -- start start 2024-09-19T04:56:14.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.878+0000 7f85d37bc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c67897ac0 0x563c678a1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.878+0000 7f85d37bc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c678983e0 0x563c678ba910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.878+0000 7f85d37bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678a2ca0 0x563c678bae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.878+0000 7f85d37bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc1afaa0 con 0x563c678a2ca0 2024-09-19T04:56:14.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.878+0000 7f85d37bc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc1b15e0 con 0x563c678983e0 2024-09-19T04:56:14.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.878+0000 7f85d37bc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc1b0170 con 0x563c67897ac0 2024-09-19T04:56:14.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.878+0000 7f85ca800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678a2ca0 0x563c678bae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.878+0000 7f85c1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c678983e0 0x563c678ba910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.878+0000 7f85ca800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678a2ca0 0x563c678bae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53444/0 (socket says 172.21.15.123:53444) 2024-09-19T04:56:14.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.878+0000 7f85ca800640 1 -- 172.21.15.123:0/1325070149 learned_addr learned my addr 172.21.15.123:0/1325070149 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:14.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.878+0000 7f85ca800640 1 --2- 172.21.15.123:0/1325070149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c67897ac0 0x563c678a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.878+0000 7f85ca800640 1 -- 172.21.15.123:0/1325070149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c67897ac0 msgr2=0x563c678a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:14.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.878+0000 7f85ca800640 1 --2- 172.21.15.123:0/1325070149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c67897ac0 0x563c678a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.878+0000 7f85ca800640 1 -- 172.21.15.123:0/1325070149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c678983e0 msgr2=0x563c678ba910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:14.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.878+0000 7f85ca800640 1 --2- 172.21.15.123:0/1325070149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c678983e0 0x563c678ba910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.878+0000 7f85ca800640 1 -- 172.21.15.123:0/1325070149 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c678b56c0 con 0x563c678a2ca0 2024-09-19T04:56:14.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.879+0000 7f85ca800640 1 --2- 172.21.15.123:0/1325070149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678a2ca0 0x563c678bae50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f85c4033040 tx=0x7f85c402b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c47d962f1169d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:14.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.879+0000 7f85baa00640 1 -- 172.21.15.123:0/1325070149 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c404b040 con 0x563c678a2ca0 2024-09-19T04:56:14.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.879+0000 7f85baa00640 1 -- 172.21.15.123:0/1325070149 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c4036070 con 0x563c678a2ca0 2024-09-19T04:56:14.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.879+0000 7f85baa00640 1 -- 172.21.15.123:0/1325070149 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c40047a0 con 0x563c678a2ca0 2024-09-19T04:56:14.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.880+0000 7f85d37bc740 1 -- 172.21.15.123:0/1325070149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678a2ca0 msgr2=0x563c678bae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:14.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.880+0000 7f85d37bc740 1 --2- 172.21.15.123:0/1325070149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678a2ca0 0x563c678bae50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f85c4033040 tx=0x7f85c402b040 comp rx=0 tx=0).stop 2024-09-19T04:56:14.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.880+0000 7f85d37bc740 1 -- 172.21.15.123:0/1325070149 shutdown_connections 2024-09-19T04:56:14.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.880+0000 7f85d37bc740 1 --2- 172.21.15.123:0/1325070149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678a2ca0 0x563c678bae50 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.880+0000 7f85d37bc740 1 --2- 172.21.15.123:0/1325070149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c678983e0 0x563c678ba910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.880+0000 7f85d37bc740 1 --2- 172.21.15.123:0/1325070149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c67897ac0 0x563c678a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.880+0000 7f85d37bc740 1 -- 172.21.15.123:0/1325070149 >> 172.21.15.123:0/1325070149 conn(0x563c67894050 msgr2=0x563c678a1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:14.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.880+0000 7f85d37bc740 1 -- 172.21.15.123:0/1325070149 shutdown_connections 2024-09-19T04:56:14.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.880+0000 7f85d37bc740 1 -- 172.21.15.123:0/1325070149 wait complete. 2024-09-19T04:56:14.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.880+0000 7f85d37bc740 1 -- start start 2024-09-19T04:56:14.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.881+0000 7f85d37bc740 1 Processor -- start 2024-09-19T04:56:14.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.881+0000 7f85d37bc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c67897ac0 0x563c6789c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.881+0000 7f85d37bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678983e0 0x563c6789cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.881+0000 7f85d37bc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c678a2ca0 0x563c6789d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.881+0000 7f85d37bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc1b0f10 con 0x563c678983e0 2024-09-19T04:56:14.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.881+0000 7f85d37bc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc1b0840 con 0x563c678a2ca0 2024-09-19T04:56:14.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.881+0000 7f85d37bc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc1adaa0 con 0x563c67897ac0 2024-09-19T04:56:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.881+0000 7f85d37bc740 10 client.0 _ll_register_callbacks cb 0x563c678a0810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.881+0000 7f85c1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678983e0 0x563c6789cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.881+0000 7f85ca800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c678a2ca0 0x563c6789d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.881+0000 7f85c1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678983e0 0x563c6789cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53460/0 (socket says 172.21.15.123:53460) 2024-09-19T04:56:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.881+0000 7f85c1400640 1 -- 172.21.15.123:0/2909743796 learned_addr learned my addr 172.21.15.123:0/2909743796 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.882+0000 7f85ca800640 1 --2- 172.21.15.123:0/2909743796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c67897ac0 0x563c6789c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.882+0000 7f85c1400640 1 -- 172.21.15.123:0/2909743796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c67897ac0 msgr2=0x563c6789c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.882+0000 7f85c1400640 1 --2- 172.21.15.123:0/2909743796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c67897ac0 0x563c6789c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.882+0000 7f85c1400640 1 -- 172.21.15.123:0/2909743796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c678a2ca0 msgr2=0x563c6789d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.882+0000 7f85c1400640 1 --2- 172.21.15.123:0/2909743796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c678a2ca0 0x563c6789d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.882+0000 7f85c1400640 1 -- 172.21.15.123:0/2909743796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c6791c8e0 con 0x563c678983e0 2024-09-19T04:56:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.882+0000 7f85c1400640 1 --2- 172.21.15.123:0/2909743796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678983e0 0x563c6789cdc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f85b40126d0 tx=0x7f85b4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fc726635875189 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:14.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.882+0000 7f85afe00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:56:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.882+0000 7f85afe00640 1 -- 172.21.15.123:0/2909743796 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4029070 con 0x563c678983e0 2024-09-19T04:56:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.883+0000 7f85afe00640 1 -- 172.21.15.123:0/2909743796 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7f85b400f790 con 0x563c678983e0 2024-09-19T04:56:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.883+0000 7f85d37bc740 1 -- 172.21.15.123:0/2909743796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85c4036070 con 0x563c678983e0 2024-09-19T04:56:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.883+0000 7f85afe00640 1 -- 172.21.15.123:0/2909743796 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f85b4035050 con 0x563c678983e0 2024-09-19T04:56:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.883+0000 7f85d37bc740 10 client.17421 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.883+0000 7f85d37bc740 1 -- 172.21.15.123:0/2909743796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563c6791db60 con 0x563c678983e0 2024-09-19T04:56:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.883+0000 7f85afe00640 1 -- 172.21.15.123:0/2909743796 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c4036070 con 0x563c678983e0 2024-09-19T04:56:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.883+0000 7f85afe00640 1 -- 172.21.15.123:0/2909743796 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f85b40035f0 con 0x563c678983e0 2024-09-19T04:56:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.883+0000 7f85d37bc740 10 client.17421 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.883+0000 7f85d37bc740 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:56:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.883+0000 7f85d37bc740 20 client.17421 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.883+0000 7f85d37bc740 1 --2- 172.21.15.123:0/2909743796 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563c6791e2e0 0x563c67920700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.883+0000 7f85d37bc740 4 client.17421 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:56:14.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.883+0000 7f85d37bc740 1 -- 172.21.15.123:0/2909743796 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563c67894c00 con 0x563c6791e2e0 2024-09-19T04:56:14.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.884+0000 7f85ca800640 1 --2- 172.21.15.123:0/2909743796 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563c6791e2e0 0x563c67920700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:14.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.884+0000 7f85afe00640 10 client.17421 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:14.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.884+0000 7f85ca800640 1 --2- 172.21.15.123:0/2909743796 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563c6791e2e0 0x563c67920700 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f84f245a467ea280 server_cookie=db9348fb56ef3ceb in_seq=0 out_seq=0 2024-09-19T04:56:14.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.885+0000 7f85afe00640 1 -- 172.21.15.123:0/2909743796 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563c67894c00 con 0x563c6791e2e0 2024-09-19T04:56:14.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.885+0000 7f85afe00640 10 client.17421 handle_command_reply: tid=0 2024-09-19T04:56:14.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.885+0000 7f85d37bc740 1 client.17421 shutdown 2024-09-19T04:56:14.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.886+0000 7f85d37bc740 1 -- 172.21.15.123:0/2909743796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678983e0 msgr2=0x563c6789cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:14.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.886+0000 7f85d37bc740 1 --2- 172.21.15.123:0/2909743796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678983e0 0x563c6789cdc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f85b40126d0 tx=0x7f85b4005d50 comp rx=0 tx=0).stop 2024-09-19T04:56:14.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.886+0000 7f85d37bc740 1 -- 172.21.15.123:0/2909743796 shutdown_connections 2024-09-19T04:56:14.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.886+0000 7f85d37bc740 1 --2- 172.21.15.123:0/2909743796 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x563c6791e2e0 0x563c67920700 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.886+0000 7f85d37bc740 1 --2- 172.21.15.123:0/2909743796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c678a2ca0 0x563c6789d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.886+0000 7f85d37bc740 1 --2- 172.21.15.123:0/2909743796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c678983e0 0x563c6789cdc0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.886+0000 7f85d37bc740 1 --2- 172.21.15.123:0/2909743796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563c67897ac0 0x563c6789c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.886+0000 7f85d37bc740 1 -- 172.21.15.123:0/2909743796 >> 172.21.15.123:0/2909743796 conn(0x563c67894050 msgr2=0x563c67899d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:14.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.886+0000 7f85afe00640 0 client.17421 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:14.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.886+0000 7f85d37bc740 1 -- 172.21.15.123:0/2909743796 shutdown_connections 2024-09-19T04:56:14.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.886+0000 7f85d37bc740 1 -- 172.21.15.123:0/2909743796 wait complete. 2024-09-19T04:56:14.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.886+0000 7f85d37bc740 20 client.17421 trim_cache size 0 max 16384 2024-09-19T04:56:14.890 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:56:14.891 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-09-19T04:56:14.891 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "6c06f028-87b1-46d0-95c1-aac896851c9d", 2024-09-19T04:56:14.891 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-09-19T04:56:14.891 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:56:14.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.887+0000 7f85d0e00640 1 -- 172.21.15.123:0/3172273268 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85b00a33b0 msgr2=0x7f85b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:14.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.887+0000 7f85d0e00640 1 --2- 172.21.15.123:0/3172273268 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85b00a33b0 0x7f85b00a5870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f85bc09c0c0 tx=0x7f85bc0d3b60 comp rx=0 tx=0).stop 2024-09-19T04:56:14.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.887+0000 7f85d0e00640 1 -- 172.21.15.123:0/3172273268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85cc101de0 msgr2=0x7f85cc19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:14.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.887+0000 7f85d0e00640 1 --2- 172.21.15.123:0/3172273268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85cc101de0 0x7f85cc19a650 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00e9f0 tx=0x7f85bc00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.887+0000 7f85d0e00640 1 -- 172.21.15.123:0/3172273268 shutdown_connections 2024-09-19T04:56:14.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.887+0000 7f85d0e00640 1 --2- 172.21.15.123:0/3172273268 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f85b00a33b0 0x7f85b00a5870 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.887+0000 7f85d0e00640 1 --2- 172.21.15.123:0/3172273268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc106f10 0x7f85cc19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.887+0000 7f85d0e00640 1 --2- 172.21.15.123:0/3172273268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85cc102790 0x7f85cc19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.887+0000 7f85d0e00640 1 --2- 172.21.15.123:0/3172273268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85cc101de0 0x7f85cc19a650 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:14.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.887+0000 7f85d0e00640 1 -- 172.21.15.123:0/3172273268 >> 172.21.15.123:0/3172273268 conn(0x7f85cc0fd8c0 msgr2=0x7f85cc0ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:14.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.887+0000 7f85d0e00640 1 -- 172.21.15.123:0/3172273268 shutdown_connections 2024-09-19T04:56:14.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:14.887+0000 7f85d0e00640 1 -- 172.21.15.123:0/3172273268 wait complete. 2024-09-19T04:56:15.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:15 smithi183 ceph-mon[35333]: pgmap v296: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:56:15.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:15 smithi123 ceph-mon[30511]: pgmap v296: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:56:15.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:15 smithi193 ceph-mon[35592]: pgmap v296: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:56:15.755 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:56:16.126 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:16.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:16 smithi123 ceph-mon[30511]: scrub queued for path: / 2024-09-19T04:56:16.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:16 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:56:16.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:16 smithi123 ceph-mon[30511]: scrub summary: active paths [/] 2024-09-19T04:56:16.580 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:16 smithi123 ceph-mon[30511]: scrub summary: idle+waiting paths [/] 2024-09-19T04:56:16.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:16 smithi183 ceph-mon[35333]: scrub queued for path: / 2024-09-19T04:56:16.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:16 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:56:16.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:16 smithi183 ceph-mon[35333]: scrub summary: active paths [/] 2024-09-19T04:56:16.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:16 smithi183 ceph-mon[35333]: scrub summary: idle+waiting paths [/] 2024-09-19T04:56:16.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.689+0000 7f8bd0200640 1 -- 172.21.15.123:0/274077368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc8074590 msgr2=0x7f8bc8074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:16.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.689+0000 7f8bd0200640 1 --2- 172.21.15.123:0/274077368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc8074590 0x7f8bc8074970 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009900 tx=0x7f8bbc02a480 comp rx=0 tx=0).stop 2024-09-19T04:56:16.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.689+0000 7f8bd0200640 1 -- 172.21.15.123:0/274077368 shutdown_connections 2024-09-19T04:56:16.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.689+0000 7f8bd0200640 1 --2- 172.21.15.123:0/274077368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bc8112b90 0x7f8bc8114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.689+0000 7f8bd0200640 1 --2- 172.21.15.123:0/274077368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc8074eb0 0x7f8bc8112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.689+0000 7f8bd0200640 1 --2- 172.21.15.123:0/274077368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc8074590 0x7f8bc8074970 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.689+0000 7f8bd0200640 1 -- 172.21.15.123:0/274077368 >> 172.21.15.123:0/274077368 conn(0x7f8bc80fd8c0 msgr2=0x7f8bc80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:16.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.689+0000 7f8bd0200640 1 -- 172.21.15.123:0/274077368 shutdown_connections 2024-09-19T04:56:16.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.689+0000 7f8bd0200640 1 -- 172.21.15.123:0/274077368 wait complete. 2024-09-19T04:56:16.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.690+0000 7f8bd0200640 1 Processor -- start 2024-09-19T04:56:16.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.690+0000 7f8bd0200640 1 -- start start 2024-09-19T04:56:16.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.690+0000 7f8bd0200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bc8074590 0x7f8bc81a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.690+0000 7f8bd0200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc8074eb0 0x7f8bc81a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.690+0000 7f8bd0200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc8112b90 0x7f8bc819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.690+0000 7f8bd0200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81185c0 con 0x7f8bc8074eb0 2024-09-19T04:56:16.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.690+0000 7f8bd0200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8118440 con 0x7f8bc8112b90 2024-09-19T04:56:16.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.690+0000 7f8bd0200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8118740 con 0x7f8bc8074590 2024-09-19T04:56:16.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.691+0000 7f8bcce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc8074eb0 0x7f8bc81a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.691+0000 7f8bcce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc8074eb0 0x7f8bc81a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53482/0 (socket says 172.21.15.123:53482) 2024-09-19T04:56:16.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.691+0000 7f8bcce00640 1 -- 172.21.15.123:0/2429099298 learned_addr learned my addr 172.21.15.123:0/2429099298 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:16.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.691+0000 7f8bcd800640 1 --2- 172.21.15.123:0/2429099298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bc8074590 0x7f8bc81a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.691+0000 7f8bce200640 1 --2- 172.21.15.123:0/2429099298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc8112b90 0x7f8bc819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.691+0000 7f8bcce00640 1 -- 172.21.15.123:0/2429099298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bc8074590 msgr2=0x7f8bc81a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:16.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.691+0000 7f8bcce00640 1 --2- 172.21.15.123:0/2429099298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bc8074590 0x7f8bc81a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.691+0000 7f8bcce00640 1 -- 172.21.15.123:0/2429099298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc8112b90 msgr2=0x7f8bc819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:16.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.691+0000 7f8bcce00640 1 --2- 172.21.15.123:0/2429099298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc8112b90 0x7f8bc819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.691+0000 7f8bcce00640 1 -- 172.21.15.123:0/2429099298 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc819efe0 con 0x7f8bc8074eb0 2024-09-19T04:56:16.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.691+0000 7f8bce200640 1 --2- 172.21.15.123:0/2429099298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc8112b90 0x7f8bc819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:56:16.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.692+0000 7f8bcce00640 1 --2- 172.21.15.123:0/2429099298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc8074eb0 0x7f8bc81a4c30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400e9f0 tx=0x7f8bb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:16.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.692+0000 7f8bc6000640 1 -- 172.21.15.123:0/2429099298 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb400cd80 con 0x7f8bc8074eb0 2024-09-19T04:56:16.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.692+0000 7f8bc6000640 1 -- 172.21.15.123:0/2429099298 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb4010910 con 0x7f8bc8074eb0 2024-09-19T04:56:16.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.692+0000 7f8bd0200640 1 -- 172.21.15.123:0/2429099298 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc819f2d0 con 0x7f8bc8074eb0 2024-09-19T04:56:16.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.692+0000 7f8bc6000640 1 -- 172.21.15.123:0/2429099298 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4018990 con 0x7f8bc8074eb0 2024-09-19T04:56:16.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.692+0000 7f8bd0200640 1 -- 172.21.15.123:0/2429099298 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc81ab5c0 con 0x7f8bc8074eb0 2024-09-19T04:56:16.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.695+0000 7f8bd2e48740 1 Processor -- start 2024-09-19T04:56:16.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.695+0000 7f8bc6000640 1 -- 172.21.15.123:0/2429099298 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f8bb4010ac0 con 0x7f8bc8074eb0 2024-09-19T04:56:16.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.695+0000 7f8bd2e48740 1 -- start start 2024-09-19T04:56:16.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.696+0000 7f8bd2e48740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c7206b90 0x5563c7206f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.696+0000 7f8bd2e48740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c72074b0 0x5563c7211830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.696+0000 7f8bd2e48740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7211d70 0x5563c7214160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.696+0000 7f8bd2e48740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5563c716d3e0 con 0x5563c7206b90 2024-09-19T04:56:16.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.696+0000 7f8bd2e48740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5563c7114d80 con 0x5563c72074b0 2024-09-19T04:56:16.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.696+0000 7f8bd2e48740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5563c71693a0 con 0x5563c7211d70 2024-09-19T04:56:16.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.696+0000 7f8bce200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c7206b90 0x5563c7206f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.696+0000 7f8bce200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c7206b90 0x5563c7206f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53488/0 (socket says 172.21.15.123:53488) 2024-09-19T04:56:16.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.696+0000 7f8bce200640 1 -- 172.21.15.123:0/647191401 learned_addr learned my addr 172.21.15.123:0/647191401 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:16.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.696+0000 7f8bcd800640 1 --2- 172.21.15.123:0/647191401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c72074b0 0x5563c7211830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.696+0000 7f8bcce00640 1 --2- 172.21.15.123:0/647191401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7211d70 0x5563c7214160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.697+0000 7f8bce200640 1 -- 172.21.15.123:0/647191401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7211d70 msgr2=0x5563c7214160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:16.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.697+0000 7f8bce200640 1 --2- 172.21.15.123:0/647191401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7211d70 0x5563c7214160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.697+0000 7f8bce200640 1 -- 172.21.15.123:0/647191401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c72074b0 msgr2=0x5563c7211830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:16.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.697+0000 7f8bce200640 1 --2- 172.21.15.123:0/647191401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c72074b0 0x5563c7211830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.697+0000 7f8bce200640 1 -- 172.21.15.123:0/647191401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563c7133ae0 con 0x5563c7206b90 2024-09-19T04:56:16.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.697+0000 7f8bcce00640 1 --2- 172.21.15.123:0/647191401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7211d70 0x5563c7214160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:56:16.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.697+0000 7f8bc6000640 1 --2- 172.21.15.123:0/2429099298 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f8bb00a3430 0x7f8bb00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.697+0000 7f8bce200640 1 --2- 172.21.15.123:0/647191401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c7206b90 0x5563c7206f70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009fc0 tx=0x7f8bc0010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e01ff18921e9d32 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:16.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.697+0000 7f8bc6000640 1 -- 172.21.15.123:0/2429099298 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7f8bb40d00c0 con 0x7f8bc8074eb0 2024-09-19T04:56:16.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.697+0000 7f8bcd800640 1 --2- 172.21.15.123:0/2429099298 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f8bb00a3430 0x7f8bb00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.697+0000 7f8bbaa00640 1 -- 172.21.15.123:0/647191401 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0011700 con 0x5563c7206b90 2024-09-19T04:56:16.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.698+0000 7f8bbaa00640 1 -- 172.21.15.123:0/647191401 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc0011d80 con 0x5563c7206b90 2024-09-19T04:56:16.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.698+0000 7f8bbaa00640 1 -- 172.21.15.123:0/647191401 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc001a380 con 0x5563c7206b90 2024-09-19T04:56:16.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.698+0000 7f8bd2e48740 1 -- 172.21.15.123:0/647191401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c7206b90 msgr2=0x5563c7206f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:16.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.698+0000 7f8bcd800640 1 --2- 172.21.15.123:0/2429099298 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f8bb00a3430 0x7f8bb00a58f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc006010 tx=0x7f8bbc000bc0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:16.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.698+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/647191401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c7206b90 0x5563c7206f70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009fc0 tx=0x7f8bc0010860 comp rx=0 tx=0).stop 2024-09-19T04:56:16.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.699+0000 7f8bd2e48740 1 -- 172.21.15.123:0/647191401 shutdown_connections 2024-09-19T04:56:16.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.699+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/647191401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7211d70 0x5563c7214160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.699+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/647191401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c72074b0 0x5563c7211830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.699+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/647191401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c7206b90 0x5563c7206f70 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.699+0000 7f8bd2e48740 1 -- 172.21.15.123:0/647191401 >> 172.21.15.123:0/647191401 conn(0x5563c7203120 msgr2=0x5563c7203500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:16.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.699+0000 7f8bd2e48740 1 -- 172.21.15.123:0/647191401 shutdown_connections 2024-09-19T04:56:16.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.699+0000 7f8bd2e48740 1 -- 172.21.15.123:0/647191401 wait complete. 2024-09-19T04:56:16.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.699+0000 7f8bd2e48740 1 -- start start 2024-09-19T04:56:16.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.699+0000 7f8bd2e48740 1 Processor -- start 2024-09-19T04:56:16.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.700+0000 7f8bd2e48740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7206b90 0x5563c720ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.700+0000 7f8bd2e48740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c72074b0 0x5563c720f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.700+0000 7f8bd2e48740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c7211d70 0x5563c720f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.700+0000 7f8bd2e48740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5563c72149e0 con 0x5563c7211d70 2024-09-19T04:56:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.700+0000 7f8bd2e48740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5563c71280f0 con 0x5563c72074b0 2024-09-19T04:56:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.700+0000 7f8bd2e48740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5563c7214b60 con 0x5563c7206b90 2024-09-19T04:56:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.700+0000 7f8bd2e48740 10 client.0 _ll_register_callbacks cb 0x5563c71199e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.700+0000 7f8bcd800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7206b90 0x5563c720ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.700+0000 7f8bcd800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7206b90 0x5563c720ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53732/0 (socket says 172.21.15.123:53732) 2024-09-19T04:56:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.700+0000 7f8bcd800640 1 -- 172.21.15.123:0/2735920058 learned_addr learned my addr 172.21.15.123:0/2735920058 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.700+0000 7f8bcce00640 1 --2- 172.21.15.123:0/2735920058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c72074b0 0x5563c720f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.700+0000 7f8bce200640 1 --2- 172.21.15.123:0/2735920058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c7211d70 0x5563c720f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.700+0000 7f8bcd800640 1 -- 172.21.15.123:0/2735920058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c72074b0 msgr2=0x5563c720f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.701+0000 7f8bcd800640 1 --2- 172.21.15.123:0/2735920058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c72074b0 0x5563c720f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.701+0000 7f8bcd800640 1 -- 172.21.15.123:0/2735920058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c7211d70 msgr2=0x5563c720f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:16.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.701+0000 7f8bcd800640 1 --2- 172.21.15.123:0/2735920058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c7211d70 0x5563c720f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.701+0000 7f8bcd800640 1 -- 172.21.15.123:0/2735920058 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563c6ede3c0 con 0x5563c7206b90 2024-09-19T04:56:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.701+0000 7f8bcd800640 1 --2- 172.21.15.123:0/2735920058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7206b90 0x5563c720ec40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc033000 tx=0x7f8bbc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8fa93adb206991 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.701+0000 7f8baf400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:56:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.701+0000 7f8baf400640 1 -- 172.21.15.123:0/2735920058 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc04e070 con 0x5563c7206b90 2024-09-19T04:56:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.701+0000 7f8bd2e48740 1 -- 172.21.15.123:0/2735920058 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bc0011d80 con 0x5563c7206b90 2024-09-19T04:56:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.701+0000 7f8baf400640 1 -- 172.21.15.123:0/2735920058 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7f8bbc004b90 con 0x5563c7206b90 2024-09-19T04:56:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.701+0000 7f8baf400640 1 -- 172.21.15.123:0/2735920058 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8bbc05a050 con 0x5563c7206b90 2024-09-19T04:56:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.701+0000 7f8bd2e48740 10 client.27535 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.701+0000 7f8bd2e48740 1 -- 172.21.15.123:0/2735920058 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5563c72206f0 con 0x5563c7206b90 2024-09-19T04:56:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.702+0000 7f8baf400640 1 -- 172.21.15.123:0/2735920058 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc0011d80 con 0x5563c7206b90 2024-09-19T04:56:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.702+0000 7f8baf400640 1 -- 172.21.15.123:0/2735920058 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f8bbc0036a0 con 0x5563c7206b90 2024-09-19T04:56:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.702+0000 7f8bd2e48740 10 client.27535 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.702+0000 7f8bd2e48740 10 client.27535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:56:16.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.702+0000 7f8bd2e48740 20 client.27535 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:16.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.702+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/2735920058 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5563c7221b00 0x5563c7223f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.702+0000 7f8bd2e48740 4 client.27535 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:56:16.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.702+0000 7f8bd2e48740 1 -- 172.21.15.123:0/2735920058 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563c717a950 con 0x5563c7221b00 2024-09-19T04:56:16.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.703+0000 7f8bcce00640 1 --2- 172.21.15.123:0/2735920058 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5563c7221b00 0x5563c7223f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.703+0000 7f8bcce00640 1 --2- 172.21.15.123:0/2735920058 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5563c7221b00 0x5563c7223f60 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6af65a26fb840a4 server_cookie=1a0e75b6ee6b0847 in_seq=0 out_seq=0 2024-09-19T04:56:16.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.703+0000 7f8baf400640 10 client.27535 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:16.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.704+0000 7f8baf400640 1 -- 172.21.15.123:0/2735920058 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563c717a950 con 0x5563c7221b00 2024-09-19T04:56:16.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.704+0000 7f8baf400640 10 client.27535 handle_command_reply: tid=0 2024-09-19T04:56:16.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.704+0000 7f8bd2e48740 1 client.27535 shutdown 2024-09-19T04:56:16.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.704+0000 7f8bd2e48740 1 -- 172.21.15.123:0/2735920058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7206b90 msgr2=0x5563c720ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:16.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.704+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/2735920058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7206b90 0x5563c720ec40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc033000 tx=0x7f8bbc009260 comp rx=0 tx=0).stop 2024-09-19T04:56:16.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.704+0000 7f8bd2e48740 1 -- 172.21.15.123:0/2735920058 shutdown_connections 2024-09-19T04:56:16.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.704+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/2735920058 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5563c7221b00 0x5563c7223f60 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.704+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/2735920058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c7211d70 0x5563c720f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.704+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/2735920058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c72074b0 0x5563c720f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.704+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/2735920058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7206b90 0x5563c720ec40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.704+0000 7f8bd2e48740 1 -- 172.21.15.123:0/2735920058 >> 172.21.15.123:0/2735920058 conn(0x5563c7203120 msgr2=0x5563c7208e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:16.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.704+0000 7f8baf400640 0 client.27535 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:16.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.704+0000 7f8bd2e48740 1 -- 172.21.15.123:0/2735920058 shutdown_connections 2024-09-19T04:56:16.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.704+0000 7f8bd2e48740 1 -- 172.21.15.123:0/2735920058 wait complete. 2024-09-19T04:56:16.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.704+0000 7f8bd2e48740 20 client.27535 trim_cache size 0 max 16384 2024-09-19T04:56:16.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.722+0000 7f8bd2e48740 1 Processor -- start 2024-09-19T04:56:16.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.722+0000 7f8bd2e48740 1 -- start start 2024-09-19T04:56:16.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.722+0000 7f8bd2e48740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7206b90 0x5563c7209640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.722+0000 7f8bd2e48740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c72074b0 0x5563c7229880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.722+0000 7f8bd2e48740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c7211d70 0x5563c7229dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.722+0000 7f8bd2e48740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81b3ea0 con 0x5563c72074b0 2024-09-19T04:56:16.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.722+0000 7f8bd2e48740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81b59e0 con 0x5563c7211d70 2024-09-19T04:56:16.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.722+0000 7f8bd2e48740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81b4570 con 0x5563c7206b90 2024-09-19T04:56:16.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.723+0000 7f8bcce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c72074b0 0x5563c7229880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.723+0000 7f8bcce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c72074b0 0x5563c7229880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53504/0 (socket says 172.21.15.123:53504) 2024-09-19T04:56:16.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.723+0000 7f8bcce00640 1 -- 172.21.15.123:0/894331121 learned_addr learned my addr 172.21.15.123:0/894331121 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:16.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.723+0000 7f8bcd800640 1 --2- 172.21.15.123:0/894331121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7206b90 0x5563c7209640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.723+0000 7f8bce200640 1 --2- 172.21.15.123:0/894331121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c7211d70 0x5563c7229dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.723+0000 7f8bcce00640 1 -- 172.21.15.123:0/894331121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7206b90 msgr2=0x5563c7209640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:16.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.723+0000 7f8bcce00640 1 --2- 172.21.15.123:0/894331121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7206b90 0x5563c7209640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.723+0000 7f8bcce00640 1 -- 172.21.15.123:0/894331121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c7211d70 msgr2=0x5563c7229dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:16.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.723+0000 7f8bcce00640 1 --2- 172.21.15.123:0/894331121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c7211d70 0x5563c7229dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.723+0000 7f8bcce00640 1 -- 172.21.15.123:0/894331121 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563c7206910 con 0x5563c72074b0 2024-09-19T04:56:16.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.723+0000 7f8bcce00640 1 --2- 172.21.15.123:0/894331121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c72074b0 0x5563c7229880 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400c6a0 tx=0x7f8bb40bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fd7e6deafa307f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:16.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.724+0000 7f8bba000640 1 -- 172.21.15.123:0/894331121 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb409c320 con 0x5563c72074b0 2024-09-19T04:56:16.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.724+0000 7f8bba000640 1 -- 172.21.15.123:0/894331121 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb40a06d0 con 0x5563c72074b0 2024-09-19T04:56:16.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.724+0000 7f8bba000640 1 -- 172.21.15.123:0/894331121 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40a4b00 con 0x5563c72074b0 2024-09-19T04:56:16.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.724+0000 7f8bd2e48740 1 -- 172.21.15.123:0/894331121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c72074b0 msgr2=0x5563c7229880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:16.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.724+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/894331121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c72074b0 0x5563c7229880 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400c6a0 tx=0x7f8bb40bc040 comp rx=0 tx=0).stop 2024-09-19T04:56:16.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.724+0000 7f8bd2e48740 1 -- 172.21.15.123:0/894331121 shutdown_connections 2024-09-19T04:56:16.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.724+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/894331121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c7211d70 0x5563c7229dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.724+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/894331121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c72074b0 0x5563c7229880 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.724+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/894331121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7206b90 0x5563c7209640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.724+0000 7f8bd2e48740 1 -- 172.21.15.123:0/894331121 >> 172.21.15.123:0/894331121 conn(0x5563c7203120 msgr2=0x5563c7210cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:16.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.724+0000 7f8bd2e48740 1 -- 172.21.15.123:0/894331121 shutdown_connections 2024-09-19T04:56:16.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.724+0000 7f8bd2e48740 1 -- 172.21.15.123:0/894331121 wait complete. 2024-09-19T04:56:16.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.725+0000 7f8bd2e48740 1 -- start start 2024-09-19T04:56:16.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.725+0000 7f8bd2e48740 1 Processor -- start 2024-09-19T04:56:16.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.725+0000 7f8bd2e48740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c7206b90 0x5563c7215e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.725+0000 7f8bd2e48740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c72074b0 0x5563c7216380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.725+0000 7f8bd2e48740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7211d70 0x5563c7216900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.725+0000 7f8bd2e48740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81b5310 con 0x5563c72074b0 2024-09-19T04:56:16.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.725+0000 7f8bd2e48740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81b4c40 con 0x5563c7206b90 2024-09-19T04:56:16.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.725+0000 7f8bd2e48740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81b1ea0 con 0x5563c7211d70 2024-09-19T04:56:16.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.726+0000 7f8bd2e48740 10 client.0 _ll_register_callbacks cb 0x7f8bbc004410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:16.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.726+0000 7f8bcce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c72074b0 0x5563c7216380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.726+0000 7f8bcd800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c7206b90 0x5563c7215e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.726+0000 7f8bcce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c72074b0 0x5563c7216380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53520/0 (socket says 172.21.15.123:53520) 2024-09-19T04:56:16.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.726+0000 7f8bcce00640 1 -- 172.21.15.123:0/116444382 learned_addr learned my addr 172.21.15.123:0/116444382 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:16.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.726+0000 7f8bce200640 1 --2- 172.21.15.123:0/116444382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7211d70 0x5563c7216900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.726+0000 7f8bcce00640 1 -- 172.21.15.123:0/116444382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7211d70 msgr2=0x5563c7216900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:16.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.726+0000 7f8bcce00640 1 --2- 172.21.15.123:0/116444382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7211d70 0x5563c7216900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.726+0000 7f8bcce00640 1 -- 172.21.15.123:0/116444382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c7206b90 msgr2=0x5563c7215e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:16.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.726+0000 7f8bcce00640 1 --2- 172.21.15.123:0/116444382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c7206b90 0x5563c7215e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.726+0000 7f8bcce00640 1 -- 172.21.15.123:0/116444382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563c728bb40 con 0x5563c72074b0 2024-09-19T04:56:16.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.726+0000 7f8bcd800640 1 --2- 172.21.15.123:0/116444382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c7206b90 0x5563c7215e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:16.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.727+0000 7f8bcce00640 1 --2- 172.21.15.123:0/116444382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c72074b0 0x5563c7216380 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8bb409c2b0 tx=0x7f8bb4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ce69f8e3e7d3a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:16.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.727+0000 7f8baf400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:56:16.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.727+0000 7f8baf400640 1 -- 172.21.15.123:0/116444382 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40a4f90 con 0x5563c72074b0 2024-09-19T04:56:16.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.727+0000 7f8baf400640 1 -- 172.21.15.123:0/116444382 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7f8bb4098990 con 0x5563c72074b0 2024-09-19T04:56:16.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.727+0000 7f8bd2e48740 1 -- 172.21.15.123:0/116444382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb40a06d0 con 0x5563c72074b0 2024-09-19T04:56:16.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.727+0000 7f8baf400640 1 -- 172.21.15.123:0/116444382 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8bb40cda10 con 0x5563c72074b0 2024-09-19T04:56:16.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.727+0000 7f8bd2e48740 10 client.17439 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:16.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.727+0000 7f8bd2e48740 1 -- 172.21.15.123:0/116444382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5563c728ce50 con 0x5563c72074b0 2024-09-19T04:56:16.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.727+0000 7f8baf400640 1 -- 172.21.15.123:0/116444382 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb40a06d0 con 0x5563c72074b0 2024-09-19T04:56:16.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.727+0000 7f8baf400640 1 -- 172.21.15.123:0/116444382 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f8bb4018b30 con 0x5563c72074b0 2024-09-19T04:56:16.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.728+0000 7f8bd2e48740 10 client.17439 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:16.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.728+0000 7f8bd2e48740 10 client.17439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:56:16.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.728+0000 7f8bd2e48740 20 client.17439 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:16.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.728+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/116444382 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5563c728d540 0x5563c728f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:16.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.728+0000 7f8bd2e48740 4 client.17439 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:56:16.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.728+0000 7f8bd2e48740 1 -- 172.21.15.123:0/116444382 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5563c7174e60 con 0x5563c728d540 2024-09-19T04:56:16.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.728+0000 7f8bcd800640 1 --2- 172.21.15.123:0/116444382 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5563c728d540 0x5563c728f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:16.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.728+0000 7f8bcd800640 1 --2- 172.21.15.123:0/116444382 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5563c728d540 0x5563c728f960 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf0033bff08d9a52 server_cookie=d735ceaa3e526a07 in_seq=0 out_seq=0 2024-09-19T04:56:16.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.728+0000 7f8baf400640 10 client.17439 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:16.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.729+0000 7f8baf400640 1 -- 172.21.15.123:0/116444382 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5563c7174e60 con 0x5563c728d540 2024-09-19T04:56:16.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.729+0000 7f8baf400640 10 client.17439 handle_command_reply: tid=0 2024-09-19T04:56:16.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.729+0000 7f8bd2e48740 1 client.17439 shutdown 2024-09-19T04:56:16.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.729+0000 7f8bd2e48740 1 -- 172.21.15.123:0/116444382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c72074b0 msgr2=0x5563c7216380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:16.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.729+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/116444382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c72074b0 0x5563c7216380 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8bb409c2b0 tx=0x7f8bb4005c50 comp rx=0 tx=0).stop 2024-09-19T04:56:16.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.729+0000 7f8bd2e48740 1 -- 172.21.15.123:0/116444382 shutdown_connections 2024-09-19T04:56:16.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.729+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/116444382 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x5563c728d540 0x5563c728f960 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.729+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/116444382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563c7211d70 0x5563c7216900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.729+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/116444382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563c72074b0 0x5563c7216380 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.729+0000 7f8bd2e48740 1 --2- 172.21.15.123:0/116444382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563c7206b90 0x5563c7215e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.729+0000 7f8bd2e48740 1 -- 172.21.15.123:0/116444382 >> 172.21.15.123:0/116444382 conn(0x5563c7203120 msgr2=0x5563c72019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:16.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.729+0000 7f8baf400640 0 client.17439 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:16.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.729+0000 7f8bd2e48740 1 -- 172.21.15.123:0/116444382 shutdown_connections 2024-09-19T04:56:16.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.729+0000 7f8bd2e48740 1 -- 172.21.15.123:0/116444382 wait complete. 2024-09-19T04:56:16.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.729+0000 7f8bd2e48740 20 client.17439 trim_cache size 0 max 16384 2024-09-19T04:56:16.734 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:56:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:56:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": { 2024-09-19T04:56:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "6c06f028-87b1-46d0-95c1-aac896851c9d": { 2024-09-19T04:56:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "path": "/", 2024-09-19T04:56:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "tag": "6c06f028-87b1-46d0-95c1-aac896851c9d", 2024-09-19T04:56:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "options": "recursive,force" 2024-09-19T04:56:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:56:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:56:16.735 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:56:16.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.730+0000 7f8bd0200640 1 -- 172.21.15.123:0/2429099298 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f8bb00a3430 msgr2=0x7f8bb00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:16.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.730+0000 7f8bd0200640 1 --2- 172.21.15.123:0/2429099298 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f8bb00a3430 0x7f8bb00a58f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc006010 tx=0x7f8bbc000bc0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.730+0000 7f8bd0200640 1 -- 172.21.15.123:0/2429099298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc8074eb0 msgr2=0x7f8bc81a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:16.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.730+0000 7f8bd0200640 1 --2- 172.21.15.123:0/2429099298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc8074eb0 0x7f8bc81a4c30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400e9f0 tx=0x7f8bb400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.730+0000 7f8bd0200640 1 -- 172.21.15.123:0/2429099298 shutdown_connections 2024-09-19T04:56:16.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.730+0000 7f8bd0200640 1 --2- 172.21.15.123:0/2429099298 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f8bb00a3430 0x7f8bb00a58f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.730+0000 7f8bd0200640 1 --2- 172.21.15.123:0/2429099298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc8112b90 0x7f8bc819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.730+0000 7f8bd0200640 1 --2- 172.21.15.123:0/2429099298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc8074eb0 0x7f8bc81a4c30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.730+0000 7f8bd0200640 1 --2- 172.21.15.123:0/2429099298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bc8074590 0x7f8bc81a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:16.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.730+0000 7f8bd0200640 1 -- 172.21.15.123:0/2429099298 >> 172.21.15.123:0/2429099298 conn(0x7f8bc80fd8c0 msgr2=0x7f8bc80fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:16.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.730+0000 7f8bd0200640 1 -- 172.21.15.123:0/2429099298 shutdown_connections 2024-09-19T04:56:16.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:16.730+0000 7f8bd0200640 1 -- 172.21.15.123:0/2429099298 wait complete. 2024-09-19T04:56:16.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:16 smithi193 ceph-mon[35592]: scrub queued for path: / 2024-09-19T04:56:16.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:16 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:56:16.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:16 smithi193 ceph-mon[35592]: scrub summary: active paths [/] 2024-09-19T04:56:16.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:16 smithi193 ceph-mon[35592]: scrub summary: idle+waiting paths [/] 2024-09-19T04:56:17.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:17 smithi123 ceph-mon[30511]: pgmap v297: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:56:17.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:17 smithi123 ceph-mon[30511]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T04:56:17.473 INFO:tasks.cephfs.filesystem:scrub status for tag:6c06f028-87b1-46d0-95c1-aac896851c9d - {'path': '/', 'tag': '6c06f028-87b1-46d0-95c1-aac896851c9d', 'options': 'recursive,force'} 2024-09-19T04:56:17.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:17 smithi183 ceph-mon[35333]: pgmap v297: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:56:17.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:17 smithi183 ceph-mon[35333]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T04:56:17.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:17 smithi193 ceph-mon[35592]: pgmap v297: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:56:17.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:17 smithi193 ceph-mon[35592]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T04:56:19.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:19 smithi183 ceph-mon[35333]: pgmap v299: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-19T04:56:19.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:19 smithi123 ceph-mon[30511]: pgmap v299: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-19T04:56:19.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:19 smithi193 ceph-mon[35592]: pgmap v299: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-19T04:56:20.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:20 smithi183 ceph-mon[35333]: scrub summary: idle 2024-09-19T04:56:20.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:20 smithi123 ceph-mon[30511]: scrub summary: idle 2024-09-19T04:56:20.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:20 smithi193 ceph-mon[35592]: scrub summary: idle 2024-09-19T04:56:21.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:21 smithi183 ceph-mon[35333]: pgmap v300: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 211 B/s wr, 10 op/s 2024-09-19T04:56:21.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:21 smithi123 ceph-mon[30511]: pgmap v300: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 211 B/s wr, 10 op/s 2024-09-19T04:56:21.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:21 smithi193 ceph-mon[35592]: pgmap v300: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 211 B/s wr, 10 op/s 2024-09-19T04:56:23.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:23 smithi183 ceph-mon[35333]: pgmap v301: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:56:23.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:23 smithi123 ceph-mon[30511]: pgmap v301: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:56:23.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:23 smithi193 ceph-mon[35592]: pgmap v301: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:56:25.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:25 smithi183 ceph-mon[35333]: pgmap v302: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:56:25.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:25 smithi123 ceph-mon[30511]: pgmap v302: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:56:25.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:25 smithi193 ceph-mon[35592]: pgmap v302: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:56:27.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:27 smithi183 ceph-mon[35333]: pgmap v303: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:56:27.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:27 smithi123 ceph-mon[30511]: pgmap v303: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:56:27.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:27 smithi193 ceph-mon[35592]: pgmap v303: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:56:29.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:29 smithi183 ceph-mon[35333]: pgmap v304: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 94 B/s wr, 14 op/s 2024-09-19T04:56:29.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:29 smithi123 ceph-mon[30511]: pgmap v304: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 94 B/s wr, 14 op/s 2024-09-19T04:56:29.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:29 smithi193 ceph-mon[35592]: pgmap v304: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 94 B/s wr, 14 op/s 2024-09-19T04:56:31.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:31 smithi183 ceph-mon[35333]: pgmap v305: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:31 smithi123 ceph-mon[30511]: pgmap v305: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:31.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:31 smithi193 ceph-mon[35592]: pgmap v305: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:33.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:33 smithi183 ceph-mon[35333]: pgmap v306: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:33.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:33 smithi123 ceph-mon[30511]: pgmap v306: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:33.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:33 smithi193 ceph-mon[35592]: pgmap v306: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:35.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:35 smithi183 ceph-mon[35333]: pgmap v307: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:56:35.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:35 smithi123 ceph-mon[30511]: pgmap v307: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:56:35.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:35 smithi193 ceph-mon[35592]: pgmap v307: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:56:37.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:37 smithi183 ceph-mon[35333]: pgmap v308: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:37.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:37 smithi123 ceph-mon[30511]: pgmap v308: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:37.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:37 smithi193 ceph-mon[35592]: pgmap v308: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:39.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:39 smithi183 ceph-mon[35333]: pgmap v309: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:56:39.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:39 smithi123 ceph-mon[30511]: pgmap v309: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:56:39.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:39 smithi193 ceph-mon[35592]: pgmap v309: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:56:41.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:41 smithi183 ceph-mon[35333]: pgmap v310: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:41.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:41 smithi123 ceph-mon[30511]: pgmap v310: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:41.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:41 smithi193 ceph-mon[35592]: pgmap v310: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:43.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:43 smithi183 ceph-mon[35333]: pgmap v311: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:43.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:43 smithi123 ceph-mon[30511]: pgmap v311: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:43.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:43 smithi193 ceph-mon[35592]: pgmap v311: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:45.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:45 smithi183 ceph-mon[35333]: pgmap v312: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:56:45.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:45 smithi123 ceph-mon[30511]: pgmap v312: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:56:45.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:45 smithi193 ceph-mon[35592]: pgmap v312: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:56:47.474 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:56:47.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:47 smithi183 ceph-mon[35333]: pgmap v313: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:47.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:47 smithi123 ceph-mon[30511]: pgmap v313: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:47.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:47 smithi193 ceph-mon[35592]: pgmap v313: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:47.869 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:48.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.429+0000 7f04f8600640 1 -- 172.21.15.123:0/2864442843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0102790 msgr2=0x7f04f01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:48.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.429+0000 7f04f8600640 1 --2- 172.21.15.123:0/2864442843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0102790 0x7f04f01067e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009900 tx=0x7f04dc02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.429+0000 7f04f8600640 1 -- 172.21.15.123:0/2864442843 shutdown_connections 2024-09-19T04:56:48.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.429+0000 7f04f8600640 1 --2- 172.21.15.123:0/2864442843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04f0106f10 0x7f04f010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.429+0000 7f04f8600640 1 --2- 172.21.15.123:0/2864442843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0102790 0x7f04f01067e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.429+0000 7f04f8600640 1 --2- 172.21.15.123:0/2864442843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04f0101de0 0x7f04f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.429+0000 7f04f8600640 1 -- 172.21.15.123:0/2864442843 >> 172.21.15.123:0/2864442843 conn(0x7f04f00fd8c0 msgr2=0x7f04f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:48.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.429+0000 7f04f8600640 1 -- 172.21.15.123:0/2864442843 shutdown_connections 2024-09-19T04:56:48.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.429+0000 7f04f8600640 1 -- 172.21.15.123:0/2864442843 wait complete. 2024-09-19T04:56:48.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.430+0000 7f04f8600640 1 Processor -- start 2024-09-19T04:56:48.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.430+0000 7f04f8600640 1 -- start start 2024-09-19T04:56:48.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.431+0000 7f04f8600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04f0101de0 0x7f04f019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.431+0000 7f04f8600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0102790 0x7f04f019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.431+0000 7f04f8600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04f0106f10 0x7f04f019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.431+0000 7f04f8600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04f0111fc0 con 0x7f04f0102790 2024-09-19T04:56:48.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.431+0000 7f04f8600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f04f0111e40 con 0x7f04f0106f10 2024-09-19T04:56:48.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.431+0000 7f04f8600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04f0112140 con 0x7f04f0101de0 2024-09-19T04:56:48.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.431+0000 7f04f5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0102790 0x7f04f019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.431+0000 7f04f5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0102790 0x7f04f019ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55656/0 (socket says 172.21.15.123:55656) 2024-09-19T04:56:48.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.431+0000 7f04f5200640 1 -- 172.21.15.123:0/2634979861 learned_addr learned my addr 172.21.15.123:0/2634979861 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:48.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.431+0000 7f04f5c00640 1 --2- 172.21.15.123:0/2634979861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04f0101de0 0x7f04f019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.431+0000 7f04f6600640 1 --2- 172.21.15.123:0/2634979861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04f0106f10 0x7f04f019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.432+0000 7f04f5c00640 1 -- 172.21.15.123:0/2634979861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04f0106f10 msgr2=0x7f04f019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:48.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.432+0000 7f04f5c00640 1 --2- 172.21.15.123:0/2634979861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04f0106f10 0x7f04f019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.432+0000 7f04f5c00640 1 -- 172.21.15.123:0/2634979861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0102790 msgr2=0x7f04f019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:48.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.432+0000 7f04f5c00640 1 --2- 172.21.15.123:0/2634979861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0102790 0x7f04f019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.432+0000 7f04f5c00640 1 -- 172.21.15.123:0/2634979861 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f019f680 con 0x7f04f0101de0 2024-09-19T04:56:48.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.432+0000 7f04f5200640 1 --2- 172.21.15.123:0/2634979861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0102790 0x7f04f019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:48.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.432+0000 7f04f5c00640 1 --2- 172.21.15.123:0/2634979861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04f0101de0 0x7f04f019a6d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f04e400e9f0 tx=0x7f04e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:48.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.432+0000 7f04ee000640 1 -- 172.21.15.123:0/2634979861 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e400cd80 con 0x7f04f0101de0 2024-09-19T04:56:48.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.432+0000 7f04f8600640 1 -- 172.21.15.123:0/2634979861 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f019f970 con 0x7f04f0101de0 2024-09-19T04:56:48.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.432+0000 7f04f8600640 1 -- 172.21.15.123:0/2634979861 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f01a7250 con 0x7f04f0101de0 2024-09-19T04:56:48.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.432+0000 7f04ee000640 1 -- 172.21.15.123:0/2634979861 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e40035f0 con 0x7f04f0101de0 2024-09-19T04:56:48.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.432+0000 7f04ee000640 1 -- 172.21.15.123:0/2634979861 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e4018690 con 0x7f04f0101de0 2024-09-19T04:56:48.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.436+0000 7f04fafed740 1 Processor -- start 2024-09-19T04:56:48.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.436+0000 7f04ee000640 1 -- 172.21.15.123:0/2634979861 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f04e4004020 con 0x7f04f0101de0 2024-09-19T04:56:48.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.436+0000 7f04fafed740 1 -- start start 2024-09-19T04:56:48.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.436+0000 7f04fafed740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07ea9b90 0x55ea07ea9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.436+0000 7f04fafed740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07eaa4b0 0x55ea07eb4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.436+0000 7f04fafed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eb4d70 0x55ea07eb7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.436+0000 7f04fafed740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ea07e103e0 con 0x55ea07ea9b90 2024-09-19T04:56:48.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.436+0000 7f04fafed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ea07db7d80 con 0x55ea07eb4d70 2024-09-19T04:56:48.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.436+0000 7f04fafed740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ea07e0c3a0 con 0x55ea07eaa4b0 2024-09-19T04:56:48.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.436+0000 7f04f6600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07ea9b90 0x55ea07ea9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.437+0000 7f04f6600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07ea9b90 0x55ea07ea9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55666/0 (socket says 172.21.15.123:55666) 2024-09-19T04:56:48.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.437+0000 7f04f6600640 1 -- 172.21.15.123:0/3545205000 learned_addr learned my addr 172.21.15.123:0/3545205000 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:48.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.437+0000 7f04f5200640 1 --2- 172.21.15.123:0/3545205000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eb4d70 0x55ea07eb7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.437+0000 7f04f5c00640 1 --2- 172.21.15.123:0/3545205000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07eaa4b0 0x55ea07eb4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.437+0000 7f04f6600640 1 -- 172.21.15.123:0/3545205000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07eaa4b0 msgr2=0x55ea07eb4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:48.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.437+0000 7f04f6600640 1 --2- 172.21.15.123:0/3545205000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07eaa4b0 0x55ea07eb4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.437+0000 7f04f6600640 1 -- 172.21.15.123:0/3545205000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eb4d70 msgr2=0x55ea07eb7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:48.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.437+0000 7f04f6600640 1 --2- 172.21.15.123:0/3545205000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eb4d70 0x55ea07eb7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.437+0000 7f04f6600640 1 -- 172.21.15.123:0/3545205000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea07dd6ae0 con 0x55ea07ea9b90 2024-09-19T04:56:48.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.437+0000 7f04f5200640 1 --2- 172.21.15.123:0/3545205000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eb4d70 0x55ea07eb7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:48.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.437+0000 7f04f6600640 1 --2- 172.21.15.123:0/3545205000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07ea9b90 0x55ea07ea9f70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009f50 tx=0x7f04e80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b6e728732af25f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:48.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.438+0000 7f04e2a00640 1 -- 172.21.15.123:0/3545205000 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8004d20 con 0x55ea07ea9b90 2024-09-19T04:56:48.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.438+0000 7f04e2a00640 1 -- 172.21.15.123:0/3545205000 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e8004050 con 0x55ea07ea9b90 2024-09-19T04:56:48.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.438+0000 7f04e2a00640 1 -- 172.21.15.123:0/3545205000 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8005280 con 0x55ea07ea9b90 2024-09-19T04:56:48.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.438+0000 7f04ee000640 1 --2- 172.21.15.123:0/2634979861 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f04d80a33b0 0x7f04d80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.438+0000 7f04ee000640 1 -- 172.21.15.123:0/2634979861 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f04e40cffe0 con 0x7f04f0101de0 2024-09-19T04:56:48.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.438+0000 7f04f5c00640 1 --2- 172.21.15.123:0/2634979861 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f04d80a33b0 0x7f04d80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.438+0000 7f04fafed740 1 -- 172.21.15.123:0/3545205000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07ea9b90 msgr2=0x55ea07ea9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:48.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.438+0000 7f04fafed740 1 --2- 172.21.15.123:0/3545205000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07ea9b90 0x55ea07ea9f70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009f50 tx=0x7f04e80118d0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.439+0000 7f04fafed740 1 -- 172.21.15.123:0/3545205000 shutdown_connections 2024-09-19T04:56:48.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.439+0000 7f04fafed740 1 --2- 172.21.15.123:0/3545205000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eb4d70 0x55ea07eb7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.439+0000 7f04fafed740 1 --2- 172.21.15.123:0/3545205000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07eaa4b0 0x55ea07eb4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.439+0000 7f04fafed740 1 --2- 172.21.15.123:0/3545205000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07ea9b90 0x55ea07ea9f70 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.439+0000 7f04fafed740 1 -- 172.21.15.123:0/3545205000 >> 172.21.15.123:0/3545205000 conn(0x55ea07ea6120 msgr2=0x55ea07ea6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:48.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.439+0000 7f04fafed740 1 -- 172.21.15.123:0/3545205000 shutdown_connections 2024-09-19T04:56:48.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.439+0000 7f04f5c00640 1 --2- 172.21.15.123:0/2634979861 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f04d80a33b0 0x7f04d80a5870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55ea07ea6f80 tx=0x7f04e40d3ad0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:48.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.439+0000 7f04fafed740 1 -- 172.21.15.123:0/3545205000 wait complete. 2024-09-19T04:56:48.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.439+0000 7f04fafed740 1 -- start start 2024-09-19T04:56:48.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.439+0000 7f04fafed740 1 Processor -- start 2024-09-19T04:56:48.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.440+0000 7f04fafed740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07ea9b90 0x55ea07eb1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.440+0000 7f04fafed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eaa4b0 0x55ea07eb21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.440+0000 7f04fafed740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07eb4d70 0x55ea07eb26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.440+0000 7f04fafed740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ea07eb79e0 con 0x55ea07eb4d70 2024-09-19T04:56:48.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.440+0000 7f04fafed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ea07dcb0f0 con 0x55ea07eaa4b0 2024-09-19T04:56:48.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.440+0000 7f04fafed740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ea07eb7b60 con 0x55ea07ea9b90 2024-09-19T04:56:48.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.440+0000 7f04fafed740 10 client.0 _ll_register_callbacks cb 0x55ea07dbc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:48.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.440+0000 7f04f5200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eaa4b0 0x55ea07eb21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.440+0000 7f04f6600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07ea9b90 0x55ea07eb1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.440+0000 7f04f5200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eaa4b0 0x55ea07eb21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:50656/0 (socket says 172.21.15.123:50656) 2024-09-19T04:56:48.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.440+0000 7f04f5200640 1 -- 172.21.15.123:0/2063268424 learned_addr learned my addr 172.21.15.123:0/2063268424 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:48.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.440+0000 7f04f6600640 1 --2- 172.21.15.123:0/2063268424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07eb4d70 0x55ea07eb26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.441+0000 7f04f5200640 1 -- 172.21.15.123:0/2063268424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07ea9b90 msgr2=0x55ea07eb1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:48.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.441+0000 7f04f5200640 1 --2- 172.21.15.123:0/2063268424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07ea9b90 0x55ea07eb1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.441+0000 7f04f5200640 1 -- 172.21.15.123:0/2063268424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07eb4d70 msgr2=0x55ea07eb26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:48.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.441+0000 7f04f5200640 1 --2- 172.21.15.123:0/2063268424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07eb4d70 0x55ea07eb26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.441+0000 7f04f5200640 1 -- 172.21.15.123:0/2063268424 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea07b813c0 con 0x55ea07eaa4b0 2024-09-19T04:56:48.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.441+0000 7f04f5200640 1 --2- 172.21.15.123:0/2063268424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eaa4b0 0x55ea07eb21b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f04dc006010 tx=0x7f04dc0035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a0c84842f6fe13d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:48.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.441+0000 7f04d7400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:56:48.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.441+0000 7f04d7400640 1 -- 172.21.15.123:0/2063268424 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc035040 con 0x55ea07eaa4b0 2024-09-19T04:56:48.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.441+0000 7f04fafed740 1 -- 172.21.15.123:0/2063268424 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04e8004050 con 0x55ea07eaa4b0 2024-09-19T04:56:48.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.441+0000 7f04d7400640 1 -- 172.21.15.123:0/2063268424 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f04dc003ce0 con 0x55ea07eaa4b0 2024-09-19T04:56:48.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.442+0000 7f04d7400640 1 -- 172.21.15.123:0/2063268424 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f04dc005020 con 0x55ea07eaa4b0 2024-09-19T04:56:48.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.442+0000 7f04fafed740 10 client.27542 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:48.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.442+0000 7f04fafed740 1 -- 172.21.15.123:0/2063268424 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ea07ecbf00 con 0x55ea07eaa4b0 2024-09-19T04:56:48.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.442+0000 7f04d7400640 1 -- 172.21.15.123:0/2063268424 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e8004050 con 0x55ea07eaa4b0 2024-09-19T04:56:48.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.442+0000 7f04d7400640 1 -- 172.21.15.123:0/2063268424 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f04dc0315b0 con 0x55ea07eaa4b0 2024-09-19T04:56:48.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.442+0000 7f04fafed740 10 client.27542 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:48.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.442+0000 7f04fafed740 10 client.27542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:56:48.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.442+0000 7f04fafed740 20 client.27542 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:48.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.442+0000 7f04fafed740 1 --2- 172.21.15.123:0/2063268424 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ea07ecd310 0x55ea07ecf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.442+0000 7f04fafed740 4 client.27542 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:56:48.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.442+0000 7f04fafed740 1 -- 172.21.15.123:0/2063268424 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea07e1d950 con 0x55ea07ecd310 2024-09-19T04:56:48.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.443+0000 7f04f6600640 1 --2- 172.21.15.123:0/2063268424 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ea07ecd310 0x55ea07ecf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.443+0000 7f04f6600640 1 --2- 172.21.15.123:0/2063268424 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ea07ecd310 0x55ea07ecf730 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e722b50e92b280ea server_cookie=858bc94ea26db0b8 in_seq=0 out_seq=0 2024-09-19T04:56:48.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.443+0000 7f04d7400640 10 client.27542 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:48.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.444+0000 7f04d7400640 1 -- 172.21.15.123:0/2063268424 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea07e1d950 con 0x55ea07ecd310 2024-09-19T04:56:48.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.444+0000 7f04d7400640 10 client.27542 handle_command_reply: tid=0 2024-09-19T04:56:48.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.444+0000 7f04fafed740 1 client.27542 shutdown 2024-09-19T04:56:48.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.444+0000 7f04fafed740 1 -- 172.21.15.123:0/2063268424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eaa4b0 msgr2=0x55ea07eb21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:48.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.444+0000 7f04fafed740 1 --2- 172.21.15.123:0/2063268424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eaa4b0 0x55ea07eb21b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f04dc006010 tx=0x7f04dc0035f0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.444+0000 7f04fafed740 1 -- 172.21.15.123:0/2063268424 shutdown_connections 2024-09-19T04:56:48.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.444+0000 7f04fafed740 1 --2- 172.21.15.123:0/2063268424 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ea07ecd310 0x55ea07ecf730 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.444+0000 7f04fafed740 1 --2- 172.21.15.123:0/2063268424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07eb4d70 0x55ea07eb26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.444+0000 7f04fafed740 1 --2- 172.21.15.123:0/2063268424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eaa4b0 0x55ea07eb21b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.444+0000 7f04d7400640 0 client.27542 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:48.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.444+0000 7f04fafed740 1 --2- 172.21.15.123:0/2063268424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07ea9b90 0x55ea07eb1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.444+0000 7f04fafed740 1 -- 172.21.15.123:0/2063268424 >> 172.21.15.123:0/2063268424 conn(0x55ea07ea6120 msgr2=0x55ea07eabe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:48.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.445+0000 7f04fafed740 1 -- 172.21.15.123:0/2063268424 shutdown_connections 2024-09-19T04:56:48.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.445+0000 7f04fafed740 1 -- 172.21.15.123:0/2063268424 wait complete. 2024-09-19T04:56:48.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.445+0000 7f04fafed740 20 client.27542 trim_cache size 0 max 16384 2024-09-19T04:56:48.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.462+0000 7f04fafed740 1 Processor -- start 2024-09-19T04:56:48.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.462+0000 7f04fafed740 1 -- start start 2024-09-19T04:56:48.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.463+0000 7f04fafed740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07ea9b90 0x55ea07eb3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.463+0000 7f04fafed740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07eaa4b0 0x55ea07ed4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.463+0000 7f04fafed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eb4d70 0x55ea07ed5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.463+0000 7f04fafed740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04f01ab6f0 con 0x55ea07eaa4b0 2024-09-19T04:56:48.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.463+0000 7f04fafed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f04f01ad230 con 0x55ea07eb4d70 2024-09-19T04:56:48.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.463+0000 7f04fafed740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04f01abdc0 con 0x55ea07ea9b90 2024-09-19T04:56:48.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.463+0000 7f04f5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07eaa4b0 0x55ea07ed4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.463+0000 7f04f5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07eaa4b0 0x55ea07ed4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55692/0 (socket says 172.21.15.123:55692) 2024-09-19T04:56:48.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.463+0000 7f04f5200640 1 -- 172.21.15.123:0/3774688910 learned_addr learned my addr 172.21.15.123:0/3774688910 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:48.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.463+0000 7f04f6600640 1 --2- 172.21.15.123:0/3774688910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07ea9b90 0x55ea07eb3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.463+0000 7f04f6600640 1 --2- 172.21.15.123:0/3774688910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eb4d70 0x55ea07ed5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.463+0000 7f04f5200640 1 -- 172.21.15.123:0/3774688910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07ea9b90 msgr2=0x55ea07eb3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:48.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.463+0000 7f04f5200640 1 --2- 172.21.15.123:0/3774688910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07ea9b90 0x55ea07eb3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.463+0000 7f04f5200640 1 -- 172.21.15.123:0/3774688910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eb4d70 msgr2=0x55ea07ed5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:48.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.463+0000 7f04f5200640 1 --2- 172.21.15.123:0/3774688910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eb4d70 0x55ea07ed5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.463+0000 7f04f5200640 1 -- 172.21.15.123:0/3774688910 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea07ea59b0 con 0x55ea07eaa4b0 2024-09-19T04:56:48.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.463+0000 7f04f6600640 1 --2- 172.21.15.123:0/3774688910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eb4d70 0x55ea07ed5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:56:48.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.464+0000 7f04f5200640 1 --2- 172.21.15.123:0/3774688910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07eaa4b0 0x55ea07ed4fe0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f04dc02a9e0 tx=0x7f04dc02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc85c5a072d0170e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:48.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.464+0000 7f04e2000640 1 -- 172.21.15.123:0/3774688910 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc000e60 con 0x55ea07eaa4b0 2024-09-19T04:56:48.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.464+0000 7f04e2000640 1 -- 172.21.15.123:0/3774688910 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04dc005850 con 0x55ea07eaa4b0 2024-09-19T04:56:48.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.464+0000 7f04e2000640 1 -- 172.21.15.123:0/3774688910 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc035030 con 0x55ea07eaa4b0 2024-09-19T04:56:48.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.464+0000 7f04fafed740 1 -- 172.21.15.123:0/3774688910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07eaa4b0 msgr2=0x55ea07ed4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:48.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.464+0000 7f04fafed740 1 --2- 172.21.15.123:0/3774688910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07eaa4b0 0x55ea07ed4fe0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f04dc02a9e0 tx=0x7f04dc02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.465+0000 7f04fafed740 1 -- 172.21.15.123:0/3774688910 shutdown_connections 2024-09-19T04:56:48.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.465+0000 7f04fafed740 1 --2- 172.21.15.123:0/3774688910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eb4d70 0x55ea07ed5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.465+0000 7f04fafed740 1 --2- 172.21.15.123:0/3774688910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07eaa4b0 0x55ea07ed4fe0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.465+0000 7f04fafed740 1 --2- 172.21.15.123:0/3774688910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07ea9b90 0x55ea07eb3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.465+0000 7f04fafed740 1 -- 172.21.15.123:0/3774688910 >> 172.21.15.123:0/3774688910 conn(0x55ea07ea6120 msgr2=0x55ea07eb34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:48.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.465+0000 7f04fafed740 1 -- 172.21.15.123:0/3774688910 shutdown_connections 2024-09-19T04:56:48.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.465+0000 7f04fafed740 1 -- 172.21.15.123:0/3774688910 wait complete. 2024-09-19T04:56:48.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.465+0000 7f04fafed740 1 -- start start 2024-09-19T04:56:48.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.465+0000 7f04fafed740 1 Processor -- start 2024-09-19T04:56:48.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.466+0000 7f04fafed740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07ea9b90 0x55ea07eaeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.466+0000 7f04fafed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eaa4b0 0x55ea07eaf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.466+0000 7f04fafed740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07eb4d70 0x55ea07eaf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.466+0000 7f04fafed740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04f01acb60 con 0x55ea07ea9b90 2024-09-19T04:56:48.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.466+0000 7f04fafed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f04f01ac490 con 0x55ea07eaa4b0 2024-09-19T04:56:48.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.466+0000 7f04fafed740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04f01a9750 con 0x55ea07eb4d70 2024-09-19T04:56:48.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.466+0000 7f04fafed740 10 client.0 _ll_register_callbacks cb 0x7f04dc005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:48.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.466+0000 7f04f6600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07ea9b90 0x55ea07eaeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.466+0000 7f04f6600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07ea9b90 0x55ea07eaeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55698/0 (socket says 172.21.15.123:55698) 2024-09-19T04:56:48.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.466+0000 7f04f6600640 1 -- 172.21.15.123:0/2108240975 learned_addr learned my addr 172.21.15.123:0/2108240975 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:48.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.466+0000 7f04f5200640 1 --2- 172.21.15.123:0/2108240975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eaa4b0 0x55ea07eaf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.466+0000 7f04f6600640 1 --2- 172.21.15.123:0/2108240975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07eb4d70 0x55ea07eaf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.466+0000 7f04f5200640 1 -- 172.21.15.123:0/2108240975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07eb4d70 msgr2=0x55ea07eaf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:48.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.466+0000 7f04f5200640 1 --2- 172.21.15.123:0/2108240975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07eb4d70 0x55ea07eaf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.467+0000 7f04f5200640 1 -- 172.21.15.123:0/2108240975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07ea9b90 msgr2=0x55ea07eaeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:48.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.467+0000 7f04f5200640 1 --2- 172.21.15.123:0/2108240975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07ea9b90 0x55ea07eaeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.467+0000 7f04f5200640 1 -- 172.21.15.123:0/2108240975 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea07f37180 con 0x55ea07eaa4b0 2024-09-19T04:56:48.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.467+0000 7f04f6600640 1 --2- 172.21.15.123:0/2108240975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07ea9b90 0x55ea07eaeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:48.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.467+0000 7f04f5200640 1 --2- 172.21.15.123:0/2108240975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eaa4b0 0x55ea07eaf3e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009850 tx=0x7f04dc005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74008eccd13ded7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:48.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.467+0000 7f04d7400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:56:48.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.467+0000 7f04d7400640 1 -- 172.21.15.123:0/2108240975 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc05c060 con 0x55ea07eaa4b0 2024-09-19T04:56:48.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.467+0000 7f04fafed740 1 -- 172.21.15.123:0/2108240975 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04dc005850 con 0x55ea07eaa4b0 2024-09-19T04:56:48.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.467+0000 7f04d7400640 1 -- 172.21.15.123:0/2108240975 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f04dc0315b0 con 0x55ea07eaa4b0 2024-09-19T04:56:48.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.468+0000 7f04d7400640 1 -- 172.21.15.123:0/2108240975 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f04dc053040 con 0x55ea07eaa4b0 2024-09-19T04:56:48.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.468+0000 7f04fafed740 10 client.27548 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:48.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.468+0000 7f04fafed740 1 -- 172.21.15.123:0/2108240975 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ea07f38450 con 0x55ea07eaa4b0 2024-09-19T04:56:48.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.468+0000 7f04d7400640 1 -- 172.21.15.123:0/2108240975 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04dc005850 con 0x55ea07eaa4b0 2024-09-19T04:56:48.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.468+0000 7f04d7400640 1 -- 172.21.15.123:0/2108240975 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f04dc04e480 con 0x55ea07eaa4b0 2024-09-19T04:56:48.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.468+0000 7f04fafed740 10 client.27548 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:48.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.468+0000 7f04fafed740 10 client.27548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:56:48.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.468+0000 7f04fafed740 20 client.27548 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:48.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.468+0000 7f04fafed740 1 --2- 172.21.15.123:0/2108240975 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ea07f38ab0 0x55ea07f3aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:48.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.468+0000 7f04fafed740 4 client.27548 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:56:48.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.468+0000 7f04fafed740 1 -- 172.21.15.123:0/2108240975 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ea07e17e60 con 0x55ea07f38ab0 2024-09-19T04:56:48.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.469+0000 7f04f6600640 1 --2- 172.21.15.123:0/2108240975 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ea07f38ab0 0x55ea07f3aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:48.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.469+0000 7f04f6600640 1 --2- 172.21.15.123:0/2108240975 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ea07f38ab0 0x55ea07f3aed0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9452831d98f462b6 server_cookie=ad4737ab0815575c in_seq=0 out_seq=0 2024-09-19T04:56:48.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.469+0000 7f04d7400640 10 client.27548 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:48.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.469+0000 7f04d7400640 1 -- 172.21.15.123:0/2108240975 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ea07e17e60 con 0x55ea07f38ab0 2024-09-19T04:56:48.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.469+0000 7f04d7400640 10 client.27548 handle_command_reply: tid=0 2024-09-19T04:56:48.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.469+0000 7f04fafed740 1 client.27548 shutdown 2024-09-19T04:56:48.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.469+0000 7f04fafed740 1 -- 172.21.15.123:0/2108240975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eaa4b0 msgr2=0x55ea07eaf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:48.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.469+0000 7f04fafed740 1 --2- 172.21.15.123:0/2108240975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eaa4b0 0x55ea07eaf3e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009850 tx=0x7f04dc005bd0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04fafed740 1 -- 172.21.15.123:0/2108240975 shutdown_connections 2024-09-19T04:56:48.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04fafed740 1 --2- 172.21.15.123:0/2108240975 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55ea07f38ab0 0x55ea07f3aed0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04fafed740 1 --2- 172.21.15.123:0/2108240975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea07eb4d70 0x55ea07eaf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04d7400640 0 client.27548 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:48.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04fafed740 1 --2- 172.21.15.123:0/2108240975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea07eaa4b0 0x55ea07eaf3e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04fafed740 1 --2- 172.21.15.123:0/2108240975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea07ea9b90 0x55ea07eaeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04fafed740 1 -- 172.21.15.123:0/2108240975 >> 172.21.15.123:0/2108240975 conn(0x55ea07ea6120 msgr2=0x55ea07ea51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:48.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04fafed740 1 -- 172.21.15.123:0/2108240975 shutdown_connections 2024-09-19T04:56:48.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04fafed740 1 -- 172.21.15.123:0/2108240975 wait complete. 2024-09-19T04:56:48.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04fafed740 20 client.27548 trim_cache size 0 max 16384 2024-09-19T04:56:48.478 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:56:48.478 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-09-19T04:56:48.478 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-09-19T04:56:48.478 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:56:48.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04f8600640 1 -- 172.21.15.123:0/2634979861 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f04d80a33b0 msgr2=0x7f04d80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:48.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04f8600640 1 --2- 172.21.15.123:0/2634979861 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f04d80a33b0 0x7f04d80a5870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55ea07ea6f80 tx=0x7f04e40d3ad0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04f8600640 1 -- 172.21.15.123:0/2634979861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04f0101de0 msgr2=0x7f04f019a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:48.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04f8600640 1 --2- 172.21.15.123:0/2634979861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04f0101de0 0x7f04f019a6d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f04e400e9f0 tx=0x7f04e400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04f8600640 1 -- 172.21.15.123:0/2634979861 shutdown_connections 2024-09-19T04:56:48.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04f8600640 1 --2- 172.21.15.123:0/2634979861 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f04d80a33b0 0x7f04d80a5870 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04f8600640 1 --2- 172.21.15.123:0/2634979861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04f0106f10 0x7f04f019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.470+0000 7f04f8600640 1 --2- 172.21.15.123:0/2634979861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0102790 0x7f04f019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.471+0000 7f04f8600640 1 --2- 172.21.15.123:0/2634979861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04f0101de0 0x7f04f019a6d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:48.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.471+0000 7f04f8600640 1 -- 172.21.15.123:0/2634979861 >> 172.21.15.123:0/2634979861 conn(0x7f04f00fd8c0 msgr2=0x7f04f00ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:48.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.471+0000 7f04f8600640 1 -- 172.21.15.123:0/2634979861 shutdown_connections 2024-09-19T04:56:48.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:48.471+0000 7f04f8600640 1 -- 172.21.15.123:0/2634979861 wait complete. 2024-09-19T04:56:49.359 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:56:49.359 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:56:49.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:49 smithi183 ceph-mon[35333]: pgmap v314: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:56:49.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:49 smithi123 ceph-mon[30511]: pgmap v314: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:56:49.729 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:49.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:49 smithi193 ceph-mon[35592]: pgmap v314: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:56:50.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.281+0000 7f44f8a00640 1 -- 172.21.15.123:0/671986704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44f4103e30 msgr2=0x7f44f4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:50.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.281+0000 7f44f8a00640 1 --2- 172.21.15.123:0/671986704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44f4103e30 0x7f44f4104210 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009900 tx=0x7f44e402a410 comp rx=0 tx=0).stop 2024-09-19T04:56:50.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.281+0000 7f44f8a00640 1 -- 172.21.15.123:0/671986704 shutdown_connections 2024-09-19T04:56:50.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.281+0000 7f44f8a00640 1 --2- 172.21.15.123:0/671986704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44f4108f60 0x7f44f4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:50.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.281+0000 7f44f8a00640 1 --2- 172.21.15.123:0/671986704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44f41047e0 0x7f44f4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:50.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.281+0000 7f44f8a00640 1 --2- 172.21.15.123:0/671986704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44f4103e30 0x7f44f4104210 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:50.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.281+0000 7f44f8a00640 1 -- 172.21.15.123:0/671986704 >> 172.21.15.123:0/671986704 conn(0x7f44f40ff8d0 msgr2=0x7f44f4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:50.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.281+0000 7f44f8a00640 1 -- 172.21.15.123:0/671986704 shutdown_connections 2024-09-19T04:56:50.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.281+0000 7f44f8a00640 1 -- 172.21.15.123:0/671986704 wait complete. 2024-09-19T04:56:50.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.282+0000 7f44f8a00640 1 Processor -- start 2024-09-19T04:56:50.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.283+0000 7f44f8a00640 1 -- start start 2024-09-19T04:56:50.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.283+0000 7f44f8a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44f4103e30 0x7f44f419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:50.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.283+0000 7f44f8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44f41047e0 0x7f44f419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:50.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.283+0000 7f44f8a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44f4108f60 0x7f44f419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:50.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.283+0000 7f44f8a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f44f4113fc0 con 0x7f44f41047e0 2024-09-19T04:56:50.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.283+0000 7f44f8a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f44f4113e40 con 0x7f44f4103e30 2024-09-19T04:56:50.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.283+0000 7f44f8a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44f4114140 con 0x7f44f4108f60 2024-09-19T04:56:50.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.283+0000 7f44ebe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44f41047e0 0x7f44f419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:50.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.283+0000 7f44ebe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44f41047e0 0x7f44f419a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55722/0 (socket says 172.21.15.123:55722) 2024-09-19T04:56:50.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.283+0000 7f44ebe00640 1 -- 172.21.15.123:0/2499304252 learned_addr learned my addr 172.21.15.123:0/2499304252 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:50.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.283+0000 7f44f3e00640 1 --2- 172.21.15.123:0/2499304252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44f4108f60 0x7f44f419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:50.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.283+0000 7f44f3400640 1 --2- 172.21.15.123:0/2499304252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44f4103e30 0x7f44f419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:50.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.284+0000 7f44ebe00640 1 -- 172.21.15.123:0/2499304252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44f4108f60 msgr2=0x7f44f419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:50.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.284+0000 7f44ebe00640 1 --2- 172.21.15.123:0/2499304252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44f4108f60 0x7f44f419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:50.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.284+0000 7f44ebe00640 1 -- 172.21.15.123:0/2499304252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44f4103e30 msgr2=0x7f44f419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:50.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.284+0000 7f44ebe00640 1 --2- 172.21.15.123:0/2499304252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44f4103e30 0x7f44f419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:50.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.284+0000 7f44ebe00640 1 -- 172.21.15.123:0/2499304252 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f419f460 con 0x7f44f41047e0 2024-09-19T04:56:50.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.284+0000 7f44ebe00640 1 --2- 172.21.15.123:0/2499304252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44f41047e0 0x7f44f419a9f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00e9f0 tx=0x7f44dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:50.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.284+0000 7f44f0c00640 1 -- 172.21.15.123:0/2499304252 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc00ccc0 con 0x7f44f41047e0 2024-09-19T04:56:50.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.284+0000 7f44f0c00640 1 -- 172.21.15.123:0/2499304252 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44dc00ce60 con 0x7f44f41047e0 2024-09-19T04:56:50.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.284+0000 7f44f0c00640 1 -- 172.21.15.123:0/2499304252 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc010690 con 0x7f44f41047e0 2024-09-19T04:56:50.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.284+0000 7f44f8a00640 1 -- 172.21.15.123:0/2499304252 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f419f750 con 0x7f44f41047e0 2024-09-19T04:56:50.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.284+0000 7f44f8a00640 1 -- 172.21.15.123:0/2499304252 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f41a7030 con 0x7f44f41047e0 2024-09-19T04:56:50.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.286+0000 7f44f8a00640 1 -- 172.21.15.123:0/2499304252 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f4104210 con 0x7f44f41047e0 2024-09-19T04:56:50.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.288+0000 7f44f0c00640 1 -- 172.21.15.123:0/2499304252 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f44dc0046c0 con 0x7f44f41047e0 2024-09-19T04:56:50.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.289+0000 7f44f0c00640 1 --2- 172.21.15.123:0/2499304252 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f44d80a3430 0x7f44d80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:50.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.289+0000 7f44f0c00640 1 -- 172.21.15.123:0/2499304252 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f44dc0d00e0 con 0x7f44f41047e0 2024-09-19T04:56:50.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.289+0000 7f44f3400640 1 --2- 172.21.15.123:0/2499304252 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f44d80a3430 0x7f44d80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:50.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.290+0000 7f44f3400640 1 --2- 172.21.15.123:0/2499304252 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f44d80a3430 0x7f44d80a58f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009230 tx=0x7f44e40058d0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:50.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.293+0000 7f44f0c00640 1 -- 172.21.15.123:0/2499304252 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f44dc09ca60 con 0x7f44f41047e0 2024-09-19T04:56:50.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.505+0000 7f44f8a00640 1 -- 172.21.15.123:0/2499304252 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44f402d7c0 con 0x7f44f41047e0 2024-09-19T04:56:50.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.505+0000 7f44f0c00640 1 -- 172.21.15.123:0/2499304252 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11990 (secure 0 0 0) 0x7f44dc09c360 con 0x7f44f41047e0 2024-09-19T04:56:50.507 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:56:50.508 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":43,"btime":"2024-09-19T04:54:44:530698+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:54:44.239974+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24529,24505,24539,24500,24499]},"id":1}]} 2024-09-19T04:56:50.508 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 43 2024-09-19T04:56:50.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.508+0000 7f44f8a00640 1 -- 172.21.15.123:0/2499304252 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f44d80a3430 msgr2=0x7f44d80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:50.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.508+0000 7f44f8a00640 1 --2- 172.21.15.123:0/2499304252 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f44d80a3430 0x7f44d80a58f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009230 tx=0x7f44e40058d0 comp rx=0 tx=0).stop 2024-09-19T04:56:50.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.508+0000 7f44f8a00640 1 -- 172.21.15.123:0/2499304252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44f41047e0 msgr2=0x7f44f419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:50.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.508+0000 7f44f8a00640 1 --2- 172.21.15.123:0/2499304252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44f41047e0 0x7f44f419a9f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00e9f0 tx=0x7f44dc00edf0 comp rx=0 tx=0).stop 2024-09-19T04:56:50.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.508+0000 7f44f8a00640 1 -- 172.21.15.123:0/2499304252 shutdown_connections 2024-09-19T04:56:50.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.508+0000 7f44f8a00640 1 --2- 172.21.15.123:0/2499304252 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f44d80a3430 0x7f44d80a58f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:50.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.508+0000 7f44f8a00640 1 --2- 172.21.15.123:0/2499304252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44f4108f60 0x7f44f419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:50.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.508+0000 7f44f8a00640 1 --2- 172.21.15.123:0/2499304252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44f41047e0 0x7f44f419a9f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:50.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.508+0000 7f44f8a00640 1 --2- 172.21.15.123:0/2499304252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44f4103e30 0x7f44f419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:50.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.508+0000 7f44f8a00640 1 -- 172.21.15.123:0/2499304252 >> 172.21.15.123:0/2499304252 conn(0x7f44f40ff8d0 msgr2=0x7f44f4100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:50.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.508+0000 7f44f8a00640 1 -- 172.21.15.123:0/2499304252 shutdown_connections 2024-09-19T04:56:50.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:50.508+0000 7f44f8a00640 1 -- 172.21.15.123:0/2499304252 wait complete. 2024-09-19T04:56:51.375 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:56:51.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:51 smithi183 ceph-mon[35333]: pgmap v315: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:51.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:51 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2499304252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:56:51.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:51 smithi123 ceph-mon[30511]: pgmap v315: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:51.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:51 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2499304252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:56:51.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:51 smithi193 ceph-mon[35592]: pgmap v315: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:51.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:51 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2499304252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:56:51.743 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:52.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.276+0000 7f58ad800640 1 -- 172.21.15.123:0/214332985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58a810bfc0 msgr2=0x7f58a810e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:52.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.276+0000 7f58ad800640 1 --2- 172.21.15.123:0/214332985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58a810bfc0 0x7f58a810e460 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f58a0009ca0 tx=0x7f58a002a950 comp rx=0 tx=0).stop 2024-09-19T04:56:52.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.276+0000 7f58ad800640 1 -- 172.21.15.123:0/214332985 shutdown_connections 2024-09-19T04:56:52.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.276+0000 7f58ad800640 1 --2- 172.21.15.123:0/214332985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58a810bfc0 0x7f58a810e460 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.276+0000 7f58ad800640 1 --2- 172.21.15.123:0/214332985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a81034b0 0x7f58a810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.276+0000 7f58ad800640 1 --2- 172.21.15.123:0/214332985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a8102b90 0x7f58a8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.276+0000 7f58ad800640 1 -- 172.21.15.123:0/214332985 >> 172.21.15.123:0/214332985 conn(0x7f58a80fc610 msgr2=0x7f58a80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:52.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.276+0000 7f58ad800640 1 -- 172.21.15.123:0/214332985 shutdown_connections 2024-09-19T04:56:52.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.276+0000 7f58ad800640 1 -- 172.21.15.123:0/214332985 wait complete. 2024-09-19T04:56:52.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.277+0000 7f58ad800640 1 Processor -- start 2024-09-19T04:56:52.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.277+0000 7f58ad800640 1 -- start start 2024-09-19T04:56:52.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.277+0000 7f58ad800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a8102b90 0x7f58a81007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.277+0000 7f58ad800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58a81034b0 0x7f58a8100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.277+0000 7f58ad800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a810bfc0 0x7f58a8101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.277+0000 7f58ad800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8111dc0 con 0x7f58a81034b0 2024-09-19T04:56:52.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.277+0000 7f58ad800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8111c40 con 0x7f58a810bfc0 2024-09-19T04:56:52.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.277+0000 7f58ad800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8111f40 con 0x7f58a8102b90 2024-09-19T04:56:52.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.278+0000 7f58a7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a8102b90 0x7f58a81007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.278+0000 7f589f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58a81034b0 0x7f58a8100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.278+0000 7f589f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58a81034b0 0x7f58a8100d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47328/0 (socket says 172.21.15.123:47328) 2024-09-19T04:56:52.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.278+0000 7f589f400640 1 -- 172.21.15.123:0/3431740435 learned_addr learned my addr 172.21.15.123:0/3431740435 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:52.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.278+0000 7f58a7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a8102b90 0x7f58a81007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:52956/0 (socket says 172.21.15.123:52956) 2024-09-19T04:56:52.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.278+0000 7f58ace00640 1 --2- 172.21.15.123:0/3431740435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a810bfc0 0x7f58a8101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.278+0000 7f589f400640 1 -- 172.21.15.123:0/3431740435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a8102b90 msgr2=0x7f58a81007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:52.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.278+0000 7f589f400640 1 --2- 172.21.15.123:0/3431740435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a8102b90 0x7f58a81007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.278+0000 7f589f400640 1 -- 172.21.15.123:0/3431740435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a810bfc0 msgr2=0x7f58a8101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:52.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.278+0000 7f589f400640 1 --2- 172.21.15.123:0/3431740435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a810bfc0 0x7f58a8101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.278+0000 7f589f400640 1 -- 172.21.15.123:0/3431740435 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a81a6710 con 0x7f58a81034b0 2024-09-19T04:56:52.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.278+0000 7f589f400640 1 --2- 172.21.15.123:0/3431740435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58a81034b0 0x7f58a8100d30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f589000e9f0 tx=0x7f589000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:52.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.279+0000 7f58a5600640 1 -- 172.21.15.123:0/3431740435 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589000ccc0 con 0x7f58a81034b0 2024-09-19T04:56:52.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.279+0000 7f58ad800640 1 -- 172.21.15.123:0/3431740435 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a81a6a00 con 0x7f58a81034b0 2024-09-19T04:56:52.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.279+0000 7f58a5600640 1 -- 172.21.15.123:0/3431740435 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5890010910 con 0x7f58a81034b0 2024-09-19T04:56:52.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.279+0000 7f58a5600640 1 -- 172.21.15.123:0/3431740435 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5890018920 con 0x7f58a81034b0 2024-09-19T04:56:52.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.279+0000 7f58ad800640 1 -- 172.21.15.123:0/3431740435 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a81a6f90 con 0x7f58a81034b0 2024-09-19T04:56:52.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.283+0000 7f58b1a01740 1 Processor -- start 2024-09-19T04:56:52.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.283+0000 7f58b1a01740 1 -- start start 2024-09-19T04:56:52.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.283+0000 7f58a5600640 1 -- 172.21.15.123:0/3431740435 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5890010430 con 0x7f58a81034b0 2024-09-19T04:56:52.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.283+0000 7f58b1a01740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e1ce40 0x555c32e1d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.283+0000 7f58b1a01740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e27ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.284+0000 7f58b1a01740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e28020 0x555c32e2a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.284+0000 7f58b1a01740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555c32d833e0 con 0x555c32e28020 2024-09-19T04:56:52.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.284+0000 7f58b1a01740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555c32d2ad80 con 0x555c32e1d760 2024-09-19T04:56:52.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.284+0000 7f58b1a01740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555c32d7f3a0 con 0x555c32e1ce40 2024-09-19T04:56:52.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.284+0000 7f58a7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e27ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.284+0000 7f58a7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e27ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:45418/0 (socket says 172.21.15.123:45418) 2024-09-19T04:56:52.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.284+0000 7f58a7e00640 1 -- 172.21.15.123:0/3596146561 learned_addr learned my addr 172.21.15.123:0/3596146561 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:52.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.284+0000 7f589f400640 1 --2- 172.21.15.123:0/3596146561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e28020 0x555c32e2a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.284+0000 7f58ace00640 1 --2- 172.21.15.123:0/3596146561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e1ce40 0x555c32e1d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.284+0000 7f58a7e00640 1 -- 172.21.15.123:0/3596146561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e1ce40 msgr2=0x555c32e1d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:52.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.284+0000 7f58a7e00640 1 --2- 172.21.15.123:0/3596146561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e1ce40 0x555c32e1d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.284+0000 7f58a7e00640 1 -- 172.21.15.123:0/3596146561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e28020 msgr2=0x555c32e2a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:52.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.284+0000 7f58a7e00640 1 --2- 172.21.15.123:0/3596146561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e28020 0x555c32e2a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.284+0000 7f58a7e00640 1 -- 172.21.15.123:0/3596146561 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c32d49ae0 con 0x555c32e1d760 2024-09-19T04:56:52.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.285+0000 7f58a7e00640 1 --2- 172.21.15.123:0/3596146561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e27ae0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5898009d20 tx=0x7f5898011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94a4d1b94ef8f023 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:52.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.285+0000 7f58a5600640 1 --2- 172.21.15.123:0/3431740435 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f588c0a3430 0x7f588c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.285+0000 7f589d600640 1 -- 172.21.15.123:0/3596146561 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589801e070 con 0x555c32e1d760 2024-09-19T04:56:52.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.285+0000 7f589d600640 1 -- 172.21.15.123:0/3596146561 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5898019b10 con 0x555c32e1d760 2024-09-19T04:56:52.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.285+0000 7f589d600640 1 -- 172.21.15.123:0/3596146561 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589800ebb0 con 0x555c32e1d760 2024-09-19T04:56:52.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.285+0000 7f58a5600640 1 -- 172.21.15.123:0/3431740435 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f58900cfe70 con 0x7f58a81034b0 2024-09-19T04:56:52.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.285+0000 7f58ace00640 1 --2- 172.21.15.123:0/3431740435 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f588c0a3430 0x7f588c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.286+0000 7f58b1a01740 1 -- 172.21.15.123:0/3596146561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 msgr2=0x555c32e27ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.286+0000 7f58b1a01740 1 --2- 172.21.15.123:0/3596146561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e27ae0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5898009d20 tx=0x7f5898011810 comp rx=0 tx=0).stop 2024-09-19T04:56:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.286+0000 7f58ace00640 1 --2- 172.21.15.123:0/3431740435 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f588c0a3430 0x7f588c0a58f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f58a0005e30 tx=0x7f58a0005c50 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.286+0000 7f58b1a01740 1 -- 172.21.15.123:0/3596146561 shutdown_connections 2024-09-19T04:56:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.286+0000 7f58b1a01740 1 --2- 172.21.15.123:0/3596146561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e28020 0x555c32e2a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.286+0000 7f58b1a01740 1 --2- 172.21.15.123:0/3596146561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e27ae0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.286+0000 7f58b1a01740 1 --2- 172.21.15.123:0/3596146561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e1ce40 0x555c32e1d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.286+0000 7f58b1a01740 1 -- 172.21.15.123:0/3596146561 >> 172.21.15.123:0/3596146561 conn(0x555c32e193d0 msgr2=0x555c32e197b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.286+0000 7f58b1a01740 1 -- 172.21.15.123:0/3596146561 shutdown_connections 2024-09-19T04:56:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.286+0000 7f58b1a01740 1 -- 172.21.15.123:0/3596146561 wait complete. 2024-09-19T04:56:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.287+0000 7f58b1a01740 1 -- start start 2024-09-19T04:56:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.287+0000 7f58b1a01740 1 Processor -- start 2024-09-19T04:56:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.287+0000 7f58b1a01740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e1ce40 0x555c32e24f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.287+0000 7f58b1a01740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e25470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.287+0000 7f58b1a01740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e28020 0x555c32e259b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.287+0000 7f58b1a01740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555c32e2abf0 con 0x555c32e1ce40 2024-09-19T04:56:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.287+0000 7f58b1a01740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555c32d3e0f0 con 0x555c32e1d760 2024-09-19T04:56:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.287+0000 7f58b1a01740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555c32e2ad70 con 0x555c32e28020 2024-09-19T04:56:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.287+0000 7f58b1a01740 10 client.0 _ll_register_callbacks cb 0x555c32d25bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.288+0000 7f58a7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e1ce40 0x555c32e24f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.288+0000 7f58a7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e1ce40 0x555c32e24f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47338/0 (socket says 172.21.15.123:47338) 2024-09-19T04:56:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.288+0000 7f58a7e00640 1 -- 172.21.15.123:0/68306263 learned_addr learned my addr 172.21.15.123:0/68306263 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.288+0000 7f589f400640 1 --2- 172.21.15.123:0/68306263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e25470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.288+0000 7f58ace00640 1 --2- 172.21.15.123:0/68306263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e28020 0x555c32e259b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.288+0000 7f58a7e00640 1 -- 172.21.15.123:0/68306263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e28020 msgr2=0x555c32e259b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.288+0000 7f58a7e00640 1 --2- 172.21.15.123:0/68306263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e28020 0x555c32e259b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.288+0000 7f58a7e00640 1 -- 172.21.15.123:0/68306263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 msgr2=0x555c32e25470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.288+0000 7f58a7e00640 1 --2- 172.21.15.123:0/68306263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e25470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.288+0000 7f58a7e00640 1 -- 172.21.15.123:0/68306263 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c32af43c0 con 0x555c32e1ce40 2024-09-19T04:56:52.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.288+0000 7f589f400640 1 --2- 172.21.15.123:0/68306263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e25470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:52.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.288+0000 7f58a7e00640 1 --2- 172.21.15.123:0/68306263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e1ce40 0x555c32e24f30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5898011d10 tx=0x7f5898019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bddbcfe65d13814 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:52.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.288+0000 7f5896000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:56:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.289+0000 7f5896000640 1 -- 172.21.15.123:0/68306263 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589801e070 con 0x555c32e1ce40 2024-09-19T04:56:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.289+0000 7f58b1a01740 1 -- 172.21.15.123:0/68306263 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5898019b10 con 0x555c32e1ce40 2024-09-19T04:56:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.289+0000 7f5896000640 1 -- 172.21.15.123:0/68306263 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f589800eaf0 con 0x555c32e1ce40 2024-09-19T04:56:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.289+0000 7f5896000640 1 -- 172.21.15.123:0/68306263 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5898031050 con 0x555c32e1ce40 2024-09-19T04:56:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.289+0000 7f58b1a01740 10 client.17475 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.289+0000 7f58b1a01740 1 -- 172.21.15.123:0/68306263 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555c32e36c60 con 0x555c32e1ce40 2024-09-19T04:56:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.289+0000 7f5896000640 1 -- 172.21.15.123:0/68306263 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5898019b10 con 0x555c32e1ce40 2024-09-19T04:56:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.289+0000 7f5896000640 1 -- 172.21.15.123:0/68306263 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f58980046c0 con 0x555c32e1ce40 2024-09-19T04:56:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.290+0000 7f58b1a01740 10 client.17475 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.290+0000 7f58b1a01740 10 client.17475 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:56:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.290+0000 7f58b1a01740 20 client.17475 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.290+0000 7f58b1a01740 1 --2- 172.21.15.123:0/68306263 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x555c32e38010 0x555c32e3a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.290+0000 7f58b1a01740 4 client.17475 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:56:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.290+0000 7f58b1a01740 1 -- 172.21.15.123:0/68306263 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c32d90950 con 0x555c32e38010 2024-09-19T04:56:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.290+0000 7f589f400640 1 --2- 172.21.15.123:0/68306263 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x555c32e38010 0x555c32e3a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.290+0000 7f589f400640 1 --2- 172.21.15.123:0/68306263 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x555c32e38010 0x555c32e3a400 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9e07c7bea978e7db server_cookie=e9729e19220d04d in_seq=0 out_seq=0 2024-09-19T04:56:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.290+0000 7f5896000640 10 client.17475 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:56:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.291+0000 7f5896000640 1 -- 172.21.15.123:0/68306263 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c32d90950 con 0x555c32e38010 2024-09-19T04:56:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.291+0000 7f5896000640 10 client.17475 handle_command_reply: tid=0 2024-09-19T04:56:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.291+0000 7f58b1a01740 1 client.17475 shutdown 2024-09-19T04:56:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.292+0000 7f58b1a01740 1 -- 172.21.15.123:0/68306263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e1ce40 msgr2=0x555c32e24f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.292+0000 7f58b1a01740 1 --2- 172.21.15.123:0/68306263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e1ce40 0x555c32e24f30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5898011d10 tx=0x7f5898019660 comp rx=0 tx=0).stop 2024-09-19T04:56:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.292+0000 7f58b1a01740 1 -- 172.21.15.123:0/68306263 shutdown_connections 2024-09-19T04:56:52.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.292+0000 7f58b1a01740 1 --2- 172.21.15.123:0/68306263 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x555c32e38010 0x555c32e3a400 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.292+0000 7f58b1a01740 1 --2- 172.21.15.123:0/68306263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e28020 0x555c32e259b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.292+0000 7f58b1a01740 1 --2- 172.21.15.123:0/68306263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e25470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.292+0000 7f58b1a01740 1 --2- 172.21.15.123:0/68306263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e1ce40 0x555c32e24f30 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.292+0000 7f58b1a01740 1 -- 172.21.15.123:0/68306263 >> 172.21.15.123:0/68306263 conn(0x555c32e193d0 msgr2=0x555c32e1f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:52.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.292+0000 7f5896000640 0 client.17475 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:56:52.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.292+0000 7f58b1a01740 1 -- 172.21.15.123:0/68306263 shutdown_connections 2024-09-19T04:56:52.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.292+0000 7f58b1a01740 1 -- 172.21.15.123:0/68306263 wait complete. 2024-09-19T04:56:52.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.292+0000 7f58b1a01740 20 client.17475 trim_cache size 0 max 16384 2024-09-19T04:56:52.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.305+0000 7f58b1a01740 1 Processor -- start 2024-09-19T04:56:52.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.305+0000 7f58b1a01740 1 -- start start 2024-09-19T04:56:52.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.306+0000 7f58b1a01740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e1ce40 0x555c32e23de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.306+0000 7f58b1a01740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e3fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.306+0000 7f58b1a01740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e28020 0x555c32e40100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.306+0000 7f58b1a01740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f58a81af960 con 0x555c32e1ce40 2024-09-19T04:56:52.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.306+0000 7f58b1a01740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f58a81b14a0 con 0x555c32e1d760 2024-09-19T04:56:52.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.306+0000 7f58b1a01740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58a81b0030 con 0x555c32e28020 2024-09-19T04:56:52.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.306+0000 7f58a7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e1ce40 0x555c32e23de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.306+0000 7f58a7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e1ce40 0x555c32e23de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47342/0 (socket says 172.21.15.123:47342) 2024-09-19T04:56:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.306+0000 7f58a7e00640 1 -- 172.21.15.123:0/132297406 learned_addr learned my addr 172.21.15.123:0/132297406 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.306+0000 7f589f400640 1 --2- 172.21.15.123:0/132297406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e3fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.306+0000 7f58ace00640 1 --2- 172.21.15.123:0/132297406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e28020 0x555c32e40100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.306+0000 7f589f400640 1 -- 172.21.15.123:0/132297406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e28020 msgr2=0x555c32e40100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.307+0000 7f589f400640 1 --2- 172.21.15.123:0/132297406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e28020 0x555c32e40100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.307+0000 7f589f400640 1 -- 172.21.15.123:0/132297406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e1ce40 msgr2=0x555c32e23de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.307+0000 7f589f400640 1 --2- 172.21.15.123:0/132297406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e1ce40 0x555c32e23de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.307+0000 7f589f400640 1 -- 172.21.15.123:0/132297406 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c32e1d590 con 0x555c32e1d760 2024-09-19T04:56:52.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.307+0000 7f58a7e00640 1 --2- 172.21.15.123:0/132297406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e1ce40 0x555c32e23de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:52.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.307+0000 7f589f400640 1 --2- 172.21.15.123:0/132297406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e3fbc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f58900cd5c0 tx=0x7f58900bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d58db1f13d0e5c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:52.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.307+0000 7f589cc00640 1 -- 172.21.15.123:0/132297406 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589009bb60 con 0x555c32e1d760 2024-09-19T04:56:52.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.307+0000 7f589cc00640 1 -- 172.21.15.123:0/132297406 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f589009c200 con 0x555c32e1d760 2024-09-19T04:56:52.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.307+0000 7f589cc00640 1 -- 172.21.15.123:0/132297406 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589009a590 con 0x555c32e1d760 2024-09-19T04:56:52.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.308+0000 7f58b1a01740 1 -- 172.21.15.123:0/132297406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 msgr2=0x555c32e3fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:52.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.308+0000 7f58b1a01740 1 --2- 172.21.15.123:0/132297406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e3fbc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f58900cd5c0 tx=0x7f58900bc040 comp rx=0 tx=0).stop 2024-09-19T04:56:52.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.308+0000 7f58b1a01740 1 -- 172.21.15.123:0/132297406 shutdown_connections 2024-09-19T04:56:52.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.308+0000 7f58b1a01740 1 --2- 172.21.15.123:0/132297406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e28020 0x555c32e40100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.308+0000 7f58b1a01740 1 --2- 172.21.15.123:0/132297406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e3fbc0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.308+0000 7f58b1a01740 1 --2- 172.21.15.123:0/132297406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e1ce40 0x555c32e23de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.308+0000 7f58b1a01740 1 -- 172.21.15.123:0/132297406 >> 172.21.15.123:0/132297406 conn(0x555c32e193d0 msgr2=0x555c32e236d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.308+0000 7f58b1a01740 1 -- 172.21.15.123:0/132297406 shutdown_connections 2024-09-19T04:56:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.308+0000 7f58b1a01740 1 -- 172.21.15.123:0/132297406 wait complete. 2024-09-19T04:56:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.308+0000 7f58b1a01740 1 -- start start 2024-09-19T04:56:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.309+0000 7f58b1a01740 1 Processor -- start 2024-09-19T04:56:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.309+0000 7f58b1a01740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e1ce40 0x555c32e30340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.309+0000 7f58b1a01740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e30880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.309+0000 7f58b1a01740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e28020 0x555c32e2b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.309+0000 7f58b1a01740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f58a81b0dd0 con 0x555c32e28020 2024-09-19T04:56:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.309+0000 7f58b1a01740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f58a81b0700 con 0x555c32e1d760 2024-09-19T04:56:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.309+0000 7f58b1a01740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58a81ad960 con 0x555c32e1ce40 2024-09-19T04:56:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.309+0000 7f58b1a01740 10 client.0 _ll_register_callbacks cb 0x7f589801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.309+0000 7f58ace00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e28020 0x555c32e2b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.309+0000 7f589f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e30880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.309+0000 7f589f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e30880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:45446/0 (socket says 172.21.15.123:45446) 2024-09-19T04:56:52.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.309+0000 7f589f400640 1 -- 172.21.15.123:0/3858941939 learned_addr learned my addr 172.21.15.123:0/3858941939 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:52.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.309+0000 7f58ace00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e28020 0x555c32e2b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47348/0 (socket says 172.21.15.123:47348) 2024-09-19T04:56:52.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.309+0000 7f58a7e00640 1 --2- 172.21.15.123:0/3858941939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e1ce40 0x555c32e30340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.310+0000 7f589f400640 1 -- 172.21.15.123:0/3858941939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e1ce40 msgr2=0x555c32e30340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:52.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.310+0000 7f589f400640 1 --2- 172.21.15.123:0/3858941939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e1ce40 0x555c32e30340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.310+0000 7f589f400640 1 -- 172.21.15.123:0/3858941939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e28020 msgr2=0x555c32e2b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:52.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.310+0000 7f589f400640 1 --2- 172.21.15.123:0/3858941939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e28020 0x555c32e2b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.310+0000 7f589f400640 1 -- 172.21.15.123:0/3858941939 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c32e2c140 con 0x555c32e1d760 2024-09-19T04:56:52.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.310+0000 7f589f400640 1 --2- 172.21.15.123:0/3858941939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e30880 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f589000c6a0 tx=0x7f5890005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44b24e9919c4504a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:52.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.310+0000 7f5896000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:56:52.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.310+0000 7f5896000640 1 -- 172.21.15.123:0/3858941939 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58900a4dc0 con 0x555c32e1d760 2024-09-19T04:56:52.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.310+0000 7f5896000640 1 -- 172.21.15.123:0/3858941939 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f5890098870 con 0x555c32e1d760 2024-09-19T04:56:52.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.310+0000 7f58b1a01740 1 -- 172.21.15.123:0/3858941939 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f589009c200 con 0x555c32e1d760 2024-09-19T04:56:52.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.311+0000 7f5896000640 1 -- 172.21.15.123:0/3858941939 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5890010720 con 0x555c32e1d760 2024-09-19T04:56:52.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.311+0000 7f58b1a01740 10 client.27572 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:52.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.311+0000 7f58b1a01740 1 -- 172.21.15.123:0/3858941939 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555c32eab000 con 0x555c32e1d760 2024-09-19T04:56:52.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.311+0000 7f5896000640 1 -- 172.21.15.123:0/3858941939 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f589009c200 con 0x555c32e1d760 2024-09-19T04:56:52.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.311+0000 7f5896000640 1 -- 172.21.15.123:0/3858941939 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f58900a0120 con 0x555c32e1d760 2024-09-19T04:56:52.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.311+0000 7f58b1a01740 10 client.27572 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:52.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.311+0000 7f58b1a01740 10 client.27572 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:56:52.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.311+0000 7f58b1a01740 20 client.27572 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:52.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.311+0000 7f58b1a01740 1 --2- 172.21.15.123:0/3858941939 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x555c32eab640 0x555c32eada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:52.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.311+0000 7f58b1a01740 4 client.27572 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:56:52.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.311+0000 7f58b1a01740 1 -- 172.21.15.123:0/3858941939 --> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c32d8ae60 con 0x555c32eab640 2024-09-19T04:56:52.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.312+0000 7f58a7e00640 1 --2- 172.21.15.123:0/3858941939 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x555c32eab640 0x555c32eada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:52.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.312+0000 7f58a7e00640 1 --2- 172.21.15.123:0/3858941939 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x555c32eab640 0x555c32eada60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c4a48b170ac053d8 server_cookie=a9daa7c7fd12933b in_seq=0 out_seq=0 2024-09-19T04:56:52.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.312+0000 7f5896000640 10 client.27572 ms_handle_connect on v2:172.21.15.193:6834/1351624350 2024-09-19T04:56:52.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.312+0000 7f5896000640 1 -- 172.21.15.123:0/3858941939 <== mds.4 v2:172.21.15.193:6834/1351624350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c32d8ae60 con 0x555c32eab640 2024-09-19T04:56:52.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.312+0000 7f5896000640 10 client.27572 handle_command_reply: tid=0 2024-09-19T04:56:52.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.312+0000 7f58b1a01740 1 client.27572 shutdown 2024-09-19T04:56:52.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.312+0000 7f58b1a01740 1 -- 172.21.15.123:0/3858941939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 msgr2=0x555c32e30880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:52.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.312+0000 7f58b1a01740 1 --2- 172.21.15.123:0/3858941939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e30880 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f589000c6a0 tx=0x7f5890005c70 comp rx=0 tx=0).stop 2024-09-19T04:56:52.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.313+0000 7f58b1a01740 1 -- 172.21.15.123:0/3858941939 shutdown_connections 2024-09-19T04:56:52.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.313+0000 7f58b1a01740 1 --2- 172.21.15.123:0/3858941939 >> [v2:172.21.15.193:6834/1351624350,v1:172.21.15.193:6835/1351624350] conn(0x555c32eab640 0x555c32eada60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.313+0000 7f58b1a01740 1 --2- 172.21.15.123:0/3858941939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c32e28020 0x555c32e2b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.313+0000 7f58b1a01740 1 --2- 172.21.15.123:0/3858941939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c32e1d760 0x555c32e30880 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.313+0000 7f5896000640 0 client.27572 ms_handle_reset on v2:172.21.15.193:6834/1351624350 2024-09-19T04:56:52.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.313+0000 7f58b1a01740 1 --2- 172.21.15.123:0/3858941939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c32e1ce40 0x555c32e30340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.313+0000 7f58b1a01740 1 -- 172.21.15.123:0/3858941939 >> 172.21.15.123:0/3858941939 conn(0x555c32e193d0 msgr2=0x555c32e18490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:52.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.313+0000 7f58b1a01740 1 -- 172.21.15.123:0/3858941939 shutdown_connections 2024-09-19T04:56:52.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.313+0000 7f58b1a01740 1 -- 172.21.15.123:0/3858941939 wait complete. 2024-09-19T04:56:52.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.313+0000 7f58b1a01740 20 client.27572 trim_cache size 0 max 16384 2024-09-19T04:56:52.320 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:56:52.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.313+0000 7f58ad800640 1 -- 172.21.15.123:0/3431740435 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f588c0a3430 msgr2=0x7f588c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:52.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.313+0000 7f58ad800640 1 --2- 172.21.15.123:0/3431740435 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f588c0a3430 0x7f588c0a58f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f58a0005e30 tx=0x7f58a0005c50 comp rx=0 tx=0).stop 2024-09-19T04:56:52.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.313+0000 7f58ad800640 1 -- 172.21.15.123:0/3431740435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58a81034b0 msgr2=0x7f58a8100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:52.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.313+0000 7f58ad800640 1 --2- 172.21.15.123:0/3431740435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58a81034b0 0x7f58a8100d30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f589000e9f0 tx=0x7f589000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.314+0000 7f58ad800640 1 -- 172.21.15.123:0/3431740435 shutdown_connections 2024-09-19T04:56:52.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.314+0000 7f58ad800640 1 --2- 172.21.15.123:0/3431740435 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f588c0a3430 0x7f588c0a58f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.314+0000 7f58ad800640 1 --2- 172.21.15.123:0/3431740435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a810bfc0 0x7f58a8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.314+0000 7f58ad800640 1 --2- 172.21.15.123:0/3431740435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58a81034b0 0x7f58a8100d30 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.314+0000 7f58ad800640 1 --2- 172.21.15.123:0/3431740435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a8102b90 0x7f58a81007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:52.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.314+0000 7f58ad800640 1 -- 172.21.15.123:0/3431740435 >> 172.21.15.123:0/3431740435 conn(0x7f58a80fc610 msgr2=0x7f58a80fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:52.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.314+0000 7f58ad800640 1 -- 172.21.15.123:0/3431740435 shutdown_connections 2024-09-19T04:56:52.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:52.314+0000 7f58ad800640 1 -- 172.21.15.123:0/3431740435 wait complete. 2024-09-19T04:56:53.117 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:56:53.487 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:53.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:53 smithi183 ceph-mon[35333]: pgmap v316: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:53.626 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:53 smithi123 ceph-mon[30511]: pgmap v316: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:53.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:53 smithi193 ceph-mon[35592]: pgmap v316: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:54.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.061+0000 7fb6ed000640 1 -- 172.21.15.123:0/4054525313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6e810e3c0 msgr2=0x7fb6e8110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:54.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.061+0000 7fb6ed000640 1 --2- 172.21.15.123:0/4054525313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6e810e3c0 0x7fb6e8110810 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009900 tx=0x7fb6e002a480 comp rx=0 tx=0).stop 2024-09-19T04:56:54.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.062+0000 7fb6ed000640 1 -- 172.21.15.123:0/4054525313 shutdown_connections 2024-09-19T04:56:54.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.062+0000 7fb6ed000640 1 --2- 172.21.15.123:0/4054525313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6e810e3c0 0x7fb6e8110810 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.062+0000 7fb6ed000640 1 --2- 172.21.15.123:0/4054525313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6e8074f40 0x7fb6e810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.062+0000 7fb6ed000640 1 --2- 172.21.15.123:0/4054525313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6e8074620 0x7fb6e8074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.062+0000 7fb6ed000640 1 -- 172.21.15.123:0/4054525313 >> 172.21.15.123:0/4054525313 conn(0x7fb6e80fd8c0 msgr2=0x7fb6e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:54.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.062+0000 7fb6ed000640 1 -- 172.21.15.123:0/4054525313 shutdown_connections 2024-09-19T04:56:54.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.062+0000 7fb6ed000640 1 -- 172.21.15.123:0/4054525313 wait complete. 2024-09-19T04:56:54.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.063+0000 7fb6ed000640 1 Processor -- start 2024-09-19T04:56:54.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.063+0000 7fb6ed000640 1 -- start start 2024-09-19T04:56:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.063+0000 7fb6ed000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6e8074620 0x7fb6e819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.063+0000 7fb6ed000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6e8074f40 0x7fb6e819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.063+0000 7fb6ed000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6e810e3c0 0x7fb6e819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.063+0000 7fb6ed000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8114150 con 0x7fb6e8074f40 2024-09-19T04:56:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.063+0000 7fb6ed000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8113fd0 con 0x7fb6e810e3c0 2024-09-19T04:56:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.063+0000 7fb6ed000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81142d0 con 0x7fb6e8074620 2024-09-19T04:56:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.063+0000 7fb6e5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6e8074f40 0x7fb6e819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.063+0000 7fb6e5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6e8074f40 0x7fb6e819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47364/0 (socket says 172.21.15.123:47364) 2024-09-19T04:56:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.063+0000 7fb6e5400640 1 -- 172.21.15.123:0/451974403 learned_addr learned my addr 172.21.15.123:0/451974403 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.064+0000 7fb6e5e00640 1 --2- 172.21.15.123:0/451974403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6e8074620 0x7fb6e819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.064+0000 7fb6e6800640 1 --2- 172.21.15.123:0/451974403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6e810e3c0 0x7fb6e819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.064+0000 7fb6e5400640 1 -- 172.21.15.123:0/451974403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6e8074620 msgr2=0x7fb6e819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:54.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.064+0000 7fb6e5400640 1 --2- 172.21.15.123:0/451974403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6e8074620 0x7fb6e819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.064+0000 7fb6e5400640 1 -- 172.21.15.123:0/451974403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6e810e3c0 msgr2=0x7fb6e819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:54.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.064+0000 7fb6e5400640 1 --2- 172.21.15.123:0/451974403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6e810e3c0 0x7fb6e819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.064+0000 7fb6e5400640 1 -- 172.21.15.123:0/451974403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e819f600 con 0x7fb6e8074f40 2024-09-19T04:56:54.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.064+0000 7fb6e5400640 1 --2- 172.21.15.123:0/451974403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6e8074f40 0x7fb6e819ab90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400e9f0 tx=0x7fb6d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:54.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.065+0000 7fb6dea00640 1 -- 172.21.15.123:0/451974403 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d400cd80 con 0x7fb6e8074f40 2024-09-19T04:56:54.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.065+0000 7fb6dea00640 1 -- 172.21.15.123:0/451974403 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d4010910 con 0x7fb6e8074f40 2024-09-19T04:56:54.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.065+0000 7fb6dea00640 1 -- 172.21.15.123:0/451974403 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4018990 con 0x7fb6e8074f40 2024-09-19T04:56:54.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.065+0000 7fb6ed000640 1 -- 172.21.15.123:0/451974403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e819f8f0 con 0x7fb6e8074f40 2024-09-19T04:56:54.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.065+0000 7fb6ed000640 1 -- 172.21.15.123:0/451974403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e81a71d0 con 0x7fb6e8074f40 2024-09-19T04:56:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.068+0000 7fb6ef9eb740 1 Processor -- start 2024-09-19T04:56:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.068+0000 7fb6ef9eb740 1 -- start start 2024-09-19T04:56:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.068+0000 7fb6dea00640 1 -- 172.21.15.123:0/451974403 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb6d4010ac0 con 0x7fb6e8074f40 2024-09-19T04:56:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.068+0000 7fb6ef9eb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 0x5555c0e99050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.068+0000 7fb6ef9eb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0e99590 0x5555c0ea3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.068+0000 7fb6ef9eb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0ea3e50 0x5555c0ea6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.068+0000 7fb6ef9eb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5555c0dff3e0 con 0x5555c0e99590 2024-09-19T04:56:54.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.068+0000 7fb6ef9eb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5555c0da6d80 con 0x5555c0e98c70 2024-09-19T04:56:54.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.068+0000 7fb6ef9eb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5555c0dfb3a0 con 0x5555c0ea3e50 2024-09-19T04:56:54.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.069+0000 7fb6e6800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 0x5555c0e99050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.069+0000 7fb6e6800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 0x5555c0e99050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:45486/0 (socket says 172.21.15.123:45486) 2024-09-19T04:56:54.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.069+0000 7fb6e6800640 1 -- 172.21.15.123:0/3534859938 learned_addr learned my addr 172.21.15.123:0/3534859938 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:54.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.069+0000 7fb6e5400640 1 --2- 172.21.15.123:0/3534859938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0ea3e50 0x5555c0ea6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.069+0000 7fb6e5e00640 1 --2- 172.21.15.123:0/3534859938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0e99590 0x5555c0ea3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.069+0000 7fb6e6800640 1 -- 172.21.15.123:0/3534859938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0ea3e50 msgr2=0x5555c0ea6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.069+0000 7fb6e6800640 1 --2- 172.21.15.123:0/3534859938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0ea3e50 0x5555c0ea6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.069+0000 7fb6e6800640 1 -- 172.21.15.123:0/3534859938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0e99590 msgr2=0x5555c0ea3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.069+0000 7fb6e6800640 1 --2- 172.21.15.123:0/3534859938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0e99590 0x5555c0ea3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.069+0000 7fb6e6800640 1 -- 172.21.15.123:0/3534859938 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c0dc5ae0 con 0x5555c0e98c70 2024-09-19T04:56:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.070+0000 7fb6e6800640 1 --2- 172.21.15.123:0/3534859938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 0x5555c0e99050 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0033000 tx=0x7fb6e0005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=738cbb696a96e7c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.070+0000 7fb6dea00640 1 --2- 172.21.15.123:0/451974403 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb6cc0a3430 0x7fb6cc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.070+0000 7fb6d3400640 1 -- 172.21.15.123:0/3534859938 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0040070 con 0x5555c0e98c70 2024-09-19T04:56:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.070+0000 7fb6d3400640 1 -- 172.21.15.123:0/3534859938 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e00317b0 con 0x5555c0e98c70 2024-09-19T04:56:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.070+0000 7fb6d3400640 1 -- 172.21.15.123:0/3534859938 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0035040 con 0x5555c0e98c70 2024-09-19T04:56:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.070+0000 7fb6dea00640 1 -- 172.21.15.123:0/451974403 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fb6d40cfef0 con 0x7fb6e8074f40 2024-09-19T04:56:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.070+0000 7fb6e5e00640 1 --2- 172.21.15.123:0/451974403 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb6cc0a3430 0x7fb6cc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.071+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/3534859938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 msgr2=0x5555c0e99050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.071+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/3534859938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 0x5555c0e99050 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0033000 tx=0x7fb6e0005d50 comp rx=0 tx=0).stop 2024-09-19T04:56:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.071+0000 7fb6e5e00640 1 --2- 172.21.15.123:0/451974403 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb6cc0a3430 0x7fb6cc0a58f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x5555c0e96060 tx=0x7fb6d8009500 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.071+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/3534859938 shutdown_connections 2024-09-19T04:56:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.071+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/3534859938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0ea3e50 0x5555c0ea6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.071+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/3534859938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0e99590 0x5555c0ea3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.071+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/3534859938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 0x5555c0e99050 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.071+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/3534859938 >> 172.21.15.123:0/3534859938 conn(0x5555c0e95200 msgr2=0x5555c0e955e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.071+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/3534859938 shutdown_connections 2024-09-19T04:56:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.071+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/3534859938 wait complete. 2024-09-19T04:56:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.072+0000 7fb6ef9eb740 1 -- start start 2024-09-19T04:56:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.072+0000 7fb6ef9eb740 1 Processor -- start 2024-09-19T04:56:54.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.072+0000 7fb6ef9eb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0e98c70 0x5555c0ea0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.072+0000 7fb6ef9eb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0e99590 0x5555c0ea1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.072+0000 7fb6ef9eb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0ea3e50 0x5555c0ea17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.072+0000 7fb6ef9eb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5555c0ea6a20 con 0x5555c0e98c70 2024-09-19T04:56:54.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.072+0000 7fb6ef9eb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5555c0dba0f0 con 0x5555c0ea3e50 2024-09-19T04:56:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.072+0000 7fb6ef9eb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5555c0ea6ba0 con 0x5555c0e99590 2024-09-19T04:56:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.072+0000 7fb6ef9eb740 10 client.0 _ll_register_callbacks cb 0x5555c0da1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.072+0000 7fb6e5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0e98c70 0x5555c0ea0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.073+0000 7fb6e5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0e98c70 0x5555c0ea0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47386/0 (socket says 172.21.15.123:47386) 2024-09-19T04:56:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.073+0000 7fb6e5e00640 1 -- 172.21.15.123:0/1751709311 learned_addr learned my addr 172.21.15.123:0/1751709311 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.073+0000 7fb6e5400640 1 --2- 172.21.15.123:0/1751709311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0e99590 0x5555c0ea1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.073+0000 7fb6e6800640 1 --2- 172.21.15.123:0/1751709311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0ea3e50 0x5555c0ea17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.073+0000 7fb6e5e00640 1 -- 172.21.15.123:0/1751709311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0e99590 msgr2=0x5555c0ea1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.073+0000 7fb6e5e00640 1 --2- 172.21.15.123:0/1751709311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0e99590 0x5555c0ea1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.073+0000 7fb6e5e00640 1 -- 172.21.15.123:0/1751709311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0ea3e50 msgr2=0x5555c0ea17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.073+0000 7fb6e5e00640 1 --2- 172.21.15.123:0/1751709311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0ea3e50 0x5555c0ea17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.073+0000 7fb6e5e00640 1 -- 172.21.15.123:0/1751709311 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c0b703c0 con 0x5555c0e98c70 2024-09-19T04:56:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.073+0000 7fb6e5e00640 1 --2- 172.21.15.123:0/1751709311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0e98c70 0x5555c0ea0d50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8004040 tx=0x7fb6d8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532eaf61ef3f3440 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.073+0000 7fb6c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:56:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.074+0000 7fb6c7e00640 1 -- 172.21.15.123:0/1751709311 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d800f8b0 con 0x5555c0e98c70 2024-09-19T04:56:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.074+0000 7fb6c7e00640 1 -- 172.21.15.123:0/1751709311 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fb6d800fa50 con 0x5555c0e98c70 2024-09-19T04:56:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.074+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/1751709311 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6e00317b0 con 0x5555c0e98c70 2024-09-19T04:56:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.074+0000 7fb6c7e00640 1 -- 172.21.15.123:0/1751709311 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb6d800fc50 con 0x5555c0e98c70 2024-09-19T04:56:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.074+0000 7fb6ef9eb740 10 client.17493 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.074+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/1751709311 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5555c0eb2ad0 con 0x5555c0e98c70 2024-09-19T04:56:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.074+0000 7fb6c7e00640 1 -- 172.21.15.123:0/1751709311 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e00317b0 con 0x5555c0e98c70 2024-09-19T04:56:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.074+0000 7fb6c7e00640 1 -- 172.21.15.123:0/1751709311 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fb6d8026070 con 0x5555c0e98c70 2024-09-19T04:56:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.075+0000 7fb6ef9eb740 10 client.17493 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:54.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.075+0000 7fb6ef9eb740 10 client.17493 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:56:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.075+0000 7fb6ef9eb740 20 client.17493 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.075+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/1751709311 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5555c0eb3f90 0x5555c0eb6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.075+0000 7fb6ef9eb740 4 client.17493 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:56:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.075+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/1751709311 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555c0e0c950 con 0x5555c0eb3f90 2024-09-19T04:56:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.075+0000 7fb6e5400640 1 --2- 172.21.15.123:0/1751709311 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5555c0eb3f90 0x5555c0eb6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.075+0000 7fb6e5400640 1 --2- 172.21.15.123:0/1751709311 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5555c0eb3f90 0x5555c0eb6430 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=79f92854c1aaecee server_cookie=d654850f3488d152 in_seq=0 out_seq=0 2024-09-19T04:56:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.075+0000 7fb6c7e00640 10 client.17493 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:56:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.076+0000 7fb6c7e00640 1 -- 172.21.15.123:0/1751709311 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555c0e0c950 con 0x5555c0eb3f90 2024-09-19T04:56:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.076+0000 7fb6c7e00640 10 client.17493 handle_command_reply: tid=0 2024-09-19T04:56:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.076+0000 7fb6ef9eb740 1 client.17493 shutdown 2024-09-19T04:56:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.076+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/1751709311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0e98c70 msgr2=0x5555c0ea0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.076+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/1751709311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0e98c70 0x5555c0ea0d50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8004040 tx=0x7fb6d8004ab0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.076+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/1751709311 shutdown_connections 2024-09-19T04:56:54.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.076+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/1751709311 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5555c0eb3f90 0x5555c0eb6430 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.076+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/1751709311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0ea3e50 0x5555c0ea17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.076+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/1751709311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0e99590 0x5555c0ea1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.076+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/1751709311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0e98c70 0x5555c0ea0d50 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.077+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/1751709311 >> 172.21.15.123:0/1751709311 conn(0x5555c0e95200 msgr2=0x5555c0e9af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:54.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.077+0000 7fb6c7e00640 0 client.17493 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:56:54.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.077+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/1751709311 shutdown_connections 2024-09-19T04:56:54.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.077+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/1751709311 wait complete. 2024-09-19T04:56:54.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.077+0000 7fb6ef9eb740 20 client.17493 trim_cache size 0 max 16384 2024-09-19T04:56:54.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.090+0000 7fb6ef9eb740 1 Processor -- start 2024-09-19T04:56:54.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.091+0000 7fb6ef9eb740 1 -- start start 2024-09-19T04:56:54.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.091+0000 7fb6ef9eb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 0x5555c0e9b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.091+0000 7fb6ef9eb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0e99590 0x5555c0e9f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.091+0000 7fb6ef9eb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0ea3e50 0x5555c0ebbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.091+0000 7fb6ef9eb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81afb00 con 0x5555c0ea3e50 2024-09-19T04:56:54.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.091+0000 7fb6ef9eb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81b1640 con 0x5555c0e98c70 2024-09-19T04:56:54.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.091+0000 7fb6ef9eb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81b01d0 con 0x5555c0e99590 2024-09-19T04:56:54.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.091+0000 7fb6e6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0ea3e50 0x5555c0ebbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.091+0000 7fb6e6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0ea3e50 0x5555c0ebbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47388/0 (socket says 172.21.15.123:47388) 2024-09-19T04:56:54.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.091+0000 7fb6e6800640 1 -- 172.21.15.123:0/3858089085 learned_addr learned my addr 172.21.15.123:0/3858089085 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:54.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.091+0000 7fb6e5400640 1 --2- 172.21.15.123:0/3858089085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0e99590 0x5555c0e9f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.091+0000 7fb6e5e00640 1 --2- 172.21.15.123:0/3858089085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 0x5555c0e9b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.092+0000 7fb6e6800640 1 -- 172.21.15.123:0/3858089085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0e99590 msgr2=0x5555c0e9f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:54.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.092+0000 7fb6e6800640 1 --2- 172.21.15.123:0/3858089085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0e99590 0x5555c0e9f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.092+0000 7fb6e6800640 1 -- 172.21.15.123:0/3858089085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 msgr2=0x5555c0e9b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:54.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.092+0000 7fb6e6800640 1 --2- 172.21.15.123:0/3858089085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 0x5555c0e9b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.092+0000 7fb6e6800640 1 -- 172.21.15.123:0/3858089085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c0e962b0 con 0x5555c0ea3e50 2024-09-19T04:56:54.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.092+0000 7fb6e5400640 1 --2- 172.21.15.123:0/3858089085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0e99590 0x5555c0e9f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:54.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.092+0000 7fb6e6800640 1 --2- 172.21.15.123:0/3858089085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0ea3e50 0x5555c0ebbd50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0005430 tx=0x7fb6e0005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523ff70481e60fc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:54.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.092+0000 7fb6d2a00640 1 -- 172.21.15.123:0/3858089085 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0040040 con 0x5555c0ea3e50 2024-09-19T04:56:54.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.092+0000 7fb6d2a00640 1 -- 172.21.15.123:0/3858089085 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e0005500 con 0x5555c0ea3e50 2024-09-19T04:56:54.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.092+0000 7fb6d2a00640 1 -- 172.21.15.123:0/3858089085 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0035040 con 0x5555c0ea3e50 2024-09-19T04:56:54.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.093+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/3858089085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0ea3e50 msgr2=0x5555c0ebbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:54.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.093+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/3858089085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0ea3e50 0x5555c0ebbd50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0005430 tx=0x7fb6e0005460 comp rx=0 tx=0).stop 2024-09-19T04:56:54.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.093+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/3858089085 shutdown_connections 2024-09-19T04:56:54.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.093+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/3858089085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0ea3e50 0x5555c0ebbd50 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.093+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/3858089085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0e99590 0x5555c0e9f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.093+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/3858089085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 0x5555c0e9b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.093+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/3858089085 >> 172.21.15.123:0/3858089085 conn(0x5555c0e95200 msgr2=0x5555c0e94380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:54.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.093+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/3858089085 shutdown_connections 2024-09-19T04:56:54.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.093+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/3858089085 wait complete. 2024-09-19T04:56:54.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.093+0000 7fb6ef9eb740 1 -- start start 2024-09-19T04:56:54.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.094+0000 7fb6ef9eb740 1 Processor -- start 2024-09-19T04:56:54.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.094+0000 7fb6ef9eb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 0x5555c0e9dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.094+0000 7fb6ef9eb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0e99590 0x5555c0e9e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.094+0000 7fb6ef9eb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0ea3e50 0x5555c0e9e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.094+0000 7fb6ef9eb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81b0f70 con 0x5555c0ea3e50 2024-09-19T04:56:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.094+0000 7fb6ef9eb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81b08a0 con 0x5555c0e98c70 2024-09-19T04:56:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.094+0000 7fb6ef9eb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81adb00 con 0x5555c0e99590 2024-09-19T04:56:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.094+0000 7fb6ef9eb740 10 client.0 _ll_register_callbacks cb 0x7fb6d8003ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.094+0000 7fb6e5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 0x5555c0e9dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.094+0000 7fb6e5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 0x5555c0e9dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:45510/0 (socket says 172.21.15.123:45510) 2024-09-19T04:56:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.094+0000 7fb6e5e00640 1 -- 172.21.15.123:0/2951554722 learned_addr learned my addr 172.21.15.123:0/2951554722 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.094+0000 7fb6e6800640 1 --2- 172.21.15.123:0/2951554722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0ea3e50 0x5555c0e9e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.094+0000 7fb6e5400640 1 --2- 172.21.15.123:0/2951554722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0e99590 0x5555c0e9e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.095+0000 7fb6e5e00640 1 -- 172.21.15.123:0/2951554722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0e99590 msgr2=0x5555c0e9e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.095+0000 7fb6e5e00640 1 --2- 172.21.15.123:0/2951554722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0e99590 0x5555c0e9e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.095+0000 7fb6e5e00640 1 -- 172.21.15.123:0/2951554722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0ea3e50 msgr2=0x5555c0e9e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:54.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.095+0000 7fb6e5e00640 1 --2- 172.21.15.123:0/2951554722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0ea3e50 0x5555c0e9e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.095+0000 7fb6e5e00640 1 -- 172.21.15.123:0/2951554722 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c0e94a90 con 0x5555c0e98c70 2024-09-19T04:56:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.095+0000 7fb6e5e00640 1 --2- 172.21.15.123:0/2951554722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 0x5555c0e9dc90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8005c40 tx=0x7fb6d801e430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3d8aa563bcdf41d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.095+0000 7fb6c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:56:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.095+0000 7fb6c7e00640 1 -- 172.21.15.123:0/2951554722 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d80048d0 con 0x5555c0e98c70 2024-09-19T04:56:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.095+0000 7fb6c7e00640 1 -- 172.21.15.123:0/2951554722 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fb6d8004b20 con 0x5555c0e98c70 2024-09-19T04:56:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.095+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/2951554722 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6e0005500 con 0x5555c0e98c70 2024-09-19T04:56:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.096+0000 7fb6c7e00640 1 -- 172.21.15.123:0/2951554722 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb6d8040050 con 0x5555c0e98c70 2024-09-19T04:56:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.096+0000 7fb6ef9eb740 10 client.27590 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.096+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/2951554722 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5555c0e90770 con 0x5555c0e98c70 2024-09-19T04:56:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.096+0000 7fb6c7e00640 1 -- 172.21.15.123:0/2951554722 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e0005500 con 0x5555c0e98c70 2024-09-19T04:56:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.096+0000 7fb6c7e00640 1 -- 172.21.15.123:0/2951554722 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fb6d8037070 con 0x5555c0e98c70 2024-09-19T04:56:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.096+0000 7fb6ef9eb740 10 client.27590 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:54.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.096+0000 7fb6ef9eb740 10 client.27590 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T04:56:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.096+0000 7fb6ef9eb740 20 client.27590 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.096+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/2951554722 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5555c0f1f2f0 0x5555c0f21710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.096+0000 7fb6ef9eb740 4 client.27590 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:56:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.096+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/2951554722 --> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555c0e06e60 con 0x5555c0f1f2f0 2024-09-19T04:56:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.097+0000 7fb6e5400640 1 --2- 172.21.15.123:0/2951554722 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5555c0f1f2f0 0x5555c0f21710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.097+0000 7fb6e5400640 1 --2- 172.21.15.123:0/2951554722 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5555c0f1f2f0 0x5555c0f21710 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6d87428a6027d421 server_cookie=6aa9067153296550 in_seq=0 out_seq=0 2024-09-19T04:56:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.097+0000 7fb6c7e00640 10 client.27590 ms_handle_connect on v2:172.21.15.183:6836/3418533099 2024-09-19T04:56:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.097+0000 7fb6c7e00640 1 -- 172.21.15.123:0/2951554722 <== mds.3 v2:172.21.15.183:6836/3418533099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555c0e06e60 con 0x5555c0f1f2f0 2024-09-19T04:56:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.097+0000 7fb6c7e00640 10 client.27590 handle_command_reply: tid=0 2024-09-19T04:56:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.097+0000 7fb6ef9eb740 1 client.27590 shutdown 2024-09-19T04:56:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.097+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/2951554722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 msgr2=0x5555c0e9dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.097+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/2951554722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 0x5555c0e9dc90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8005c40 tx=0x7fb6d801e430 comp rx=0 tx=0).stop 2024-09-19T04:56:54.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.098+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/2951554722 shutdown_connections 2024-09-19T04:56:54.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.098+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/2951554722 >> [v2:172.21.15.183:6836/3418533099,v1:172.21.15.183:6837/3418533099] conn(0x5555c0f1f2f0 0x5555c0f21710 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.098+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/2951554722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555c0ea3e50 0x5555c0e9e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.098+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/2951554722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5555c0e99590 0x5555c0e9e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.098+0000 7fb6c7e00640 0 client.27590 ms_handle_reset on v2:172.21.15.183:6836/3418533099 2024-09-19T04:56:54.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.098+0000 7fb6ef9eb740 1 --2- 172.21.15.123:0/2951554722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c0e98c70 0x5555c0e9dc90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.098+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/2951554722 >> 172.21.15.123:0/2951554722 conn(0x5555c0e95200 msgr2=0x5555c0eaec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:54.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.098+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/2951554722 shutdown_connections 2024-09-19T04:56:54.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.098+0000 7fb6ef9eb740 1 -- 172.21.15.123:0/2951554722 wait complete. 2024-09-19T04:56:54.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.098+0000 7fb6ef9eb740 20 client.27590 trim_cache size 0 max 16384 2024-09-19T04:56:54.103 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:56:54.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.098+0000 7fb6ed000640 1 -- 172.21.15.123:0/451974403 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb6cc0a3430 msgr2=0x7fb6cc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:54.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.098+0000 7fb6ed000640 1 --2- 172.21.15.123:0/451974403 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb6cc0a3430 0x7fb6cc0a58f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x5555c0e96060 tx=0x7fb6d8009500 comp rx=0 tx=0).stop 2024-09-19T04:56:54.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.099+0000 7fb6ed000640 1 -- 172.21.15.123:0/451974403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6e8074f40 msgr2=0x7fb6e819ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.099+0000 7fb6ed000640 1 --2- 172.21.15.123:0/451974403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6e8074f40 0x7fb6e819ab90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400e9f0 tx=0x7fb6d400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.099+0000 7fb6ed000640 1 -- 172.21.15.123:0/451974403 shutdown_connections 2024-09-19T04:56:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.099+0000 7fb6ed000640 1 --2- 172.21.15.123:0/451974403 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb6cc0a3430 0x7fb6cc0a58f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.099+0000 7fb6ed000640 1 --2- 172.21.15.123:0/451974403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6e810e3c0 0x7fb6e819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.099+0000 7fb6ed000640 1 --2- 172.21.15.123:0/451974403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6e8074f40 0x7fb6e819ab90 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.099+0000 7fb6ed000640 1 --2- 172.21.15.123:0/451974403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6e8074620 0x7fb6e819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.099+0000 7fb6ed000640 1 -- 172.21.15.123:0/451974403 >> 172.21.15.123:0/451974403 conn(0x7fb6e80fd8c0 msgr2=0x7fb6e80fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.099+0000 7fb6ed000640 1 -- 172.21.15.123:0/451974403 shutdown_connections 2024-09-19T04:56:54.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:54.099+0000 7fb6ed000640 1 -- 172.21.15.123:0/451974403 wait complete. 2024-09-19T04:56:54.858 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:56:55.021 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T04:56:55.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.180+0000 7fb198400640 1 -- 172.21.15.123:0/3867617768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190077040 msgr2=0x7fb190112440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.180+0000 7fb198400640 1 --2- 172.21.15.123:0/3867617768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190077040 0x7fb190112440 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009900 tx=0x7fb17c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.180+0000 7fb198400640 1 -- 172.21.15.123:0/3867617768 shutdown_connections 2024-09-19T04:56:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.180+0000 7fb198400640 1 --2- 172.21.15.123:0/3867617768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb190112980 0x7fb190114d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.180+0000 7fb198400640 1 --2- 172.21.15.123:0/3867617768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190077040 0x7fb190112440 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.180+0000 7fb198400640 1 --2- 172.21.15.123:0/3867617768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb190076720 0x7fb190076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.180+0000 7fb198400640 1 -- 172.21.15.123:0/3867617768 >> 172.21.15.123:0/3867617768 conn(0x7fb1900ff9e0 msgr2=0x7fb190101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.180+0000 7fb198400640 1 -- 172.21.15.123:0/3867617768 shutdown_connections 2024-09-19T04:56:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.180+0000 7fb198400640 1 -- 172.21.15.123:0/3867617768 wait complete. 2024-09-19T04:56:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.181+0000 7fb198400640 1 Processor -- start 2024-09-19T04:56:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.181+0000 7fb198400640 1 -- start start 2024-09-19T04:56:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.181+0000 7fb198400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb190076720 0x7fb1901a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.181+0000 7fb198400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190077040 0x7fb1901a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.181+0000 7fb198400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb190112980 0x7fb19019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.181+0000 7fb198400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb190118520 con 0x7fb190077040 2024-09-19T04:56:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.181+0000 7fb198400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901183a0 con 0x7fb190076720 2024-09-19T04:56:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.181+0000 7fb198400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901186a0 con 0x7fb190112980 2024-09-19T04:56:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.182+0000 7fb195000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190077040 0x7fb1901a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.182+0000 7fb195000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190077040 0x7fb1901a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47412/0 (socket says 172.21.15.123:47412) 2024-09-19T04:56:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.182+0000 7fb195000640 1 -- 172.21.15.123:0/1491328572 learned_addr learned my addr 172.21.15.123:0/1491328572 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.182+0000 7fb195a00640 1 --2- 172.21.15.123:0/1491328572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb190076720 0x7fb1901a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.182+0000 7fb196400640 1 --2- 172.21.15.123:0/1491328572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb190112980 0x7fb19019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.182+0000 7fb195a00640 1 -- 172.21.15.123:0/1491328572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb190112980 msgr2=0x7fb19019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.182+0000 7fb195a00640 1 --2- 172.21.15.123:0/1491328572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb190112980 0x7fb19019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.182+0000 7fb195a00640 1 -- 172.21.15.123:0/1491328572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190077040 msgr2=0x7fb1901a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.182+0000 7fb195a00640 1 --2- 172.21.15.123:0/1491328572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190077040 0x7fb1901a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.182+0000 7fb195a00640 1 -- 172.21.15.123:0/1491328572 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19019f000 con 0x7fb190076720 2024-09-19T04:56:55.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.182+0000 7fb195000640 1 --2- 172.21.15.123:0/1491328572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190077040 0x7fb1901a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:55.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.182+0000 7fb195a00640 1 --2- 172.21.15.123:0/1491328572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb190076720 0x7fb1901a46c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb18400e9f0 tx=0x7fb18400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:55.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.183+0000 7fb18e000640 1 -- 172.21.15.123:0/1491328572 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18400ccc0 con 0x7fb190076720 2024-09-19T04:56:55.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.183+0000 7fb198400640 1 -- 172.21.15.123:0/1491328572 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb19019f2f0 con 0x7fb190076720 2024-09-19T04:56:55.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.183+0000 7fb18e000640 1 -- 172.21.15.123:0/1491328572 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb184010910 con 0x7fb190076720 2024-09-19T04:56:55.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.183+0000 7fb18e000640 1 -- 172.21.15.123:0/1491328572 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184018960 con 0x7fb190076720 2024-09-19T04:56:55.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.183+0000 7fb198400640 1 -- 172.21.15.123:0/1491328572 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1901ab510 con 0x7fb190076720 2024-09-19T04:56:55.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.184+0000 7fb198400640 1 -- 172.21.15.123:0/1491328572 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb190077e60 con 0x7fb190076720 2024-09-19T04:56:55.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.186+0000 7fb18e000640 1 -- 172.21.15.123:0/1491328572 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb184010ac0 con 0x7fb190076720 2024-09-19T04:56:55.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.187+0000 7fb18e000640 1 --2- 172.21.15.123:0/1491328572 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb1780a3460 0x7fb1780a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.188+0000 7fb18e000640 1 -- 172.21.15.123:0/1491328572 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fb1840d0110 con 0x7fb190076720 2024-09-19T04:56:55.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.188+0000 7fb195000640 1 --2- 172.21.15.123:0/1491328572 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb1780a3460 0x7fb1780a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.188+0000 7fb195000640 1 --2- 172.21.15.123:0/1491328572 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb1780a3460 0x7fb1780a5920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb1900797b0 tx=0x7fb17c0046c0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:55.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.191+0000 7fb18e000640 1 -- 172.21.15.123:0/1491328572 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb18409c440 con 0x7fb190076720 2024-09-19T04:56:55.242 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:55.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.406+0000 7fb198400640 1 -- 172.21.15.123:0/1491328572 --> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb190062630 con 0x7fb1780a3460 2024-09-19T04:56:55.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.416+0000 7fb18e000640 1 -- 172.21.15.123:0/1491328572 <== mgr.25133 v2:172.21.15.123:6800/1578549821 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fb190062630 con 0x7fb1780a3460 2024-09-19T04:56:55.417 INFO:teuthology.orchestra.run.smithi123.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T04:46:53", "first": "2024-09-19T04:47:00", "last": "2024-09-19T04:56:00", "last_pruned": "2024-09-19T04:56:01", "created_count": 10, "pruned_count": 4, "active": true} 2024-09-19T04:56:55.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.418+0000 7fb198400640 1 -- 172.21.15.123:0/1491328572 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb1780a3460 msgr2=0x7fb1780a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.418+0000 7fb198400640 1 --2- 172.21.15.123:0/1491328572 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb1780a3460 0x7fb1780a5920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb1900797b0 tx=0x7fb17c0046c0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.418+0000 7fb198400640 1 -- 172.21.15.123:0/1491328572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb190076720 msgr2=0x7fb1901a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.418+0000 7fb198400640 1 --2- 172.21.15.123:0/1491328572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb190076720 0x7fb1901a46c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb18400e9f0 tx=0x7fb18400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.418+0000 7fb198400640 1 -- 172.21.15.123:0/1491328572 shutdown_connections 2024-09-19T04:56:55.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.418+0000 7fb198400640 1 --2- 172.21.15.123:0/1491328572 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fb1780a3460 0x7fb1780a5920 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.418+0000 7fb198400640 1 --2- 172.21.15.123:0/1491328572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb190112980 0x7fb19019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.418+0000 7fb198400640 1 --2- 172.21.15.123:0/1491328572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb190077040 0x7fb1901a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.418+0000 7fb198400640 1 --2- 172.21.15.123:0/1491328572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb190076720 0x7fb1901a46c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.418+0000 7fb198400640 1 -- 172.21.15.123:0/1491328572 >> 172.21.15.123:0/1491328572 conn(0x7fb1900ff9e0 msgr2=0x7fb190113580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:55.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.418+0000 7fb198400640 1 -- 172.21.15.123:0/1491328572 shutdown_connections 2024-09-19T04:56:55.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.418+0000 7fb198400640 1 -- 172.21.15.123:0/1491328572 wait complete. 2024-09-19T04:56:55.435 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T04:56:55.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.591+0000 7fe932800640 1 -- 172.21.15.123:0/2610970134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe92c103400 msgr2=0x7fe92c10b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.591+0000 7fe932800640 1 --2- 172.21.15.123:0/2610970134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe92c103400 0x7fe92c10b8e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe918009900 tx=0x7fe91802a480 comp rx=0 tx=0).stop 2024-09-19T04:56:55.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.592+0000 7fe932800640 1 -- 172.21.15.123:0/2610970134 shutdown_connections 2024-09-19T04:56:55.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.592+0000 7fe932800640 1 --2- 172.21.15.123:0/2610970134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe92c10bf00 0x7fe92c10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.592+0000 7fe932800640 1 --2- 172.21.15.123:0/2610970134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe92c103400 0x7fe92c10b8e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.592+0000 7fe932800640 1 --2- 172.21.15.123:0/2610970134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe92c102ae0 0x7fe92c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.592+0000 7fe932800640 1 -- 172.21.15.123:0/2610970134 >> 172.21.15.123:0/2610970134 conn(0x7fe92c0777c0 msgr2=0x7fe92c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:55.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.592+0000 7fe932800640 1 -- 172.21.15.123:0/2610970134 shutdown_connections 2024-09-19T04:56:55.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.592+0000 7fe932800640 1 -- 172.21.15.123:0/2610970134 wait complete. 2024-09-19T04:56:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.592+0000 7fe932800640 1 Processor -- start 2024-09-19T04:56:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.593+0000 7fe932800640 1 -- start start 2024-09-19T04:56:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.593+0000 7fe932800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe92c102ae0 0x7fe92c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.593+0000 7fe932800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe92c103400 0x7fe92c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.593+0000 7fe932800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe92c10bf00 0x7fe92c19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.593+0000 7fe932800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c111d20 con 0x7fe92c10bf00 2024-09-19T04:56:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.593+0000 7fe932800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c111ba0 con 0x7fe92c103400 2024-09-19T04:56:55.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.593+0000 7fe932800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c111ea0 con 0x7fe92c102ae0 2024-09-19T04:56:55.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.593+0000 7fe931e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe92c10bf00 0x7fe92c19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.593+0000 7fe931e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe92c10bf00 0x7fe92c19cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47434/0 (socket says 172.21.15.123:47434) 2024-09-19T04:56:55.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.593+0000 7fe931e00640 1 -- 172.21.15.123:0/687748378 learned_addr learned my addr 172.21.15.123:0/687748378 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:55.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.593+0000 7fe931400640 1 --2- 172.21.15.123:0/687748378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe92c102ae0 0x7fe92c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.593+0000 7fe930a00640 1 --2- 172.21.15.123:0/687748378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe92c103400 0x7fe92c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.594+0000 7fe931e00640 1 -- 172.21.15.123:0/687748378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe92c102ae0 msgr2=0x7fe92c198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.594+0000 7fe931e00640 1 --2- 172.21.15.123:0/687748378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe92c102ae0 0x7fe92c198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.594+0000 7fe931e00640 1 -- 172.21.15.123:0/687748378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe92c103400 msgr2=0x7fe92c1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.594+0000 7fe931e00640 1 --2- 172.21.15.123:0/687748378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe92c103400 0x7fe92c1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.594+0000 7fe931e00640 1 -- 172.21.15.123:0/687748378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c19d250 con 0x7fe92c10bf00 2024-09-19T04:56:55.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.594+0000 7fe931e00640 1 --2- 172.21.15.123:0/687748378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe92c10bf00 0x7fe92c19cb20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe92400efd0 tx=0x7fe92400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:55.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.594+0000 7fe92a000640 1 -- 172.21.15.123:0/687748378 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924003b70 con 0x7fe92c10bf00 2024-09-19T04:56:55.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.594+0000 7fe932800640 1 -- 172.21.15.123:0/687748378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92c19d540 con 0x7fe92c10bf00 2024-09-19T04:56:55.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.594+0000 7fe92a000640 1 -- 172.21.15.123:0/687748378 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe924003d10 con 0x7fe92c10bf00 2024-09-19T04:56:55.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.594+0000 7fe92a000640 1 -- 172.21.15.123:0/687748378 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9240109f0 con 0x7fe92c10bf00 2024-09-19T04:56:55.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.594+0000 7fe932800640 1 -- 172.21.15.123:0/687748378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe92c1a4dd0 con 0x7fe92c10bf00 2024-09-19T04:56:55.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.596+0000 7fe932800640 1 -- 172.21.15.123:0/687748378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe92c102ec0 con 0x7fe92c10bf00 2024-09-19T04:56:55.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.598+0000 7fe92a000640 1 -- 172.21.15.123:0/687748378 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe924004020 con 0x7fe92c10bf00 2024-09-19T04:56:55.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.599+0000 7fe92a000640 1 --2- 172.21.15.123:0/687748378 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9140a33b0 0x7fe9140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.599+0000 7fe92a000640 1 -- 172.21.15.123:0/687748378 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fe9240d00e0 con 0x7fe92c10bf00 2024-09-19T04:56:55.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.599+0000 7fe931400640 1 --2- 172.21.15.123:0/687748378 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9140a33b0 0x7fe9140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.600+0000 7fe931400640 1 --2- 172.21.15.123:0/687748378 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9140a33b0 0x7fe9140a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe920005d00 tx=0x7fe92000b410 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:55.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.603+0000 7fe92a000640 1 -- 172.21.15.123:0/687748378 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe92409cac0 con 0x7fe92c10bf00 2024-09-19T04:56:55.614 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:55 smithi123 ceph-mon[30511]: pgmap v317: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:56:55.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:55 smithi193 ceph-mon[35592]: pgmap v317: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:56:55.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.806+0000 7fe932800640 1 -- 172.21.15.123:0/687748378 --> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fe92c062630 con 0x7fe9140a33b0 2024-09-19T04:56:55.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.814+0000 7fe92a000640 1 -- 172.21.15.123:0/687748378 <== mgr.25133 v2:172.21.15.123:6800/1578549821 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fe92c062630 con 0x7fe9140a33b0 2024-09-19T04:56:55.816 INFO:teuthology.orchestra.run.smithi123.stdout:/ 1m 6m3h 2024-09-19T04:56:55.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.816+0000 7fe932800640 1 -- 172.21.15.123:0/687748378 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9140a33b0 msgr2=0x7fe9140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.816+0000 7fe932800640 1 --2- 172.21.15.123:0/687748378 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9140a33b0 0x7fe9140a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe920005d00 tx=0x7fe92000b410 comp rx=0 tx=0).stop 2024-09-19T04:56:55.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.816+0000 7fe932800640 1 -- 172.21.15.123:0/687748378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe92c10bf00 msgr2=0x7fe92c19cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.816+0000 7fe932800640 1 --2- 172.21.15.123:0/687748378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe92c10bf00 0x7fe92c19cb20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe92400efd0 tx=0x7fe92400c6a0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.817+0000 7fe932800640 1 -- 172.21.15.123:0/687748378 shutdown_connections 2024-09-19T04:56:55.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.817+0000 7fe932800640 1 --2- 172.21.15.123:0/687748378 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe9140a33b0 0x7fe9140a5870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.817+0000 7fe932800640 1 --2- 172.21.15.123:0/687748378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe92c10bf00 0x7fe92c19cb20 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.817+0000 7fe932800640 1 --2- 172.21.15.123:0/687748378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe92c103400 0x7fe92c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.817+0000 7fe932800640 1 --2- 172.21.15.123:0/687748378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe92c102ae0 0x7fe92c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.817+0000 7fe932800640 1 -- 172.21.15.123:0/687748378 >> 172.21.15.123:0/687748378 conn(0x7fe92c0777c0 msgr2=0x7fe92c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:55.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.817+0000 7fe932800640 1 -- 172.21.15.123:0/687748378 shutdown_connections 2024-09-19T04:56:55.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.817+0000 7fe932800640 1 -- 172.21.15.123:0/687748378 wait complete. 2024-09-19T04:56:55.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.831+0000 7fedc6e00640 1 -- 172.21.15.123:0/2257524718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc0104010 msgr2=0x7fedc01043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.831+0000 7fedc6e00640 1 --2- 172.21.15.123:0/2257524718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc0104010 0x7fedc01043f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009ca0 tx=0x7fedb002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.831+0000 7fedc6e00640 1 -- 172.21.15.123:0/2257524718 shutdown_connections 2024-09-19T04:56:55.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.831+0000 7fedc6e00640 1 --2- 172.21.15.123:0/2257524718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedc0109140 0x7fedc0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.831+0000 7fedc6e00640 1 --2- 172.21.15.123:0/2257524718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedc01049c0 0x7fedc0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.831+0000 7fedc6e00640 1 --2- 172.21.15.123:0/2257524718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc0104010 0x7fedc01043f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.831+0000 7fedc6e00640 1 -- 172.21.15.123:0/2257524718 >> 172.21.15.123:0/2257524718 conn(0x7fedc00ffa50 msgr2=0x7fedc0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:55.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.831+0000 7fedc6e00640 1 -- 172.21.15.123:0/2257524718 shutdown_connections 2024-09-19T04:56:55.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.831+0000 7fedc6e00640 1 -- 172.21.15.123:0/2257524718 wait complete. 2024-09-19T04:56:55.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.832+0000 7fedc6e00640 1 Processor -- start 2024-09-19T04:56:55.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.832+0000 7fedc6e00640 1 -- start start 2024-09-19T04:56:55.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedc6e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedc0104010 0x7fedc019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedc6e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedc01049c0 0x7fedc019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedc6e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc0109140 0x7fedc019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedc6e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fedc0114140 con 0x7fedc0109140 2024-09-19T04:56:55.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedc6e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fedc0113fc0 con 0x7fedc0104010 2024-09-19T04:56:55.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedc6e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fedc01142c0 con 0x7fedc01049c0 2024-09-19T04:56:55.836 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-19T04:56:55.848 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-19T04:56:55.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:55 smithi183 ceph-mon[35333]: pgmap v317: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:56:55.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedc0104010 0x7fedc019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedc4e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc0109140 0x7fedc019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedc0104010 0x7fedc019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:45578/0 (socket says 172.21.15.123:45578) 2024-09-19T04:56:55.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedc4e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc0109140 0x7fedc019eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47448/0 (socket says 172.21.15.123:47448) 2024-09-19T04:56:55.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedbfe00640 1 -- 172.21.15.123:0/851328851 learned_addr learned my addr 172.21.15.123:0/851328851 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedbf400640 1 --2- 172.21.15.123:0/851328851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedc01049c0 0x7fedc019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedbf400640 1 -- 172.21.15.123:0/851328851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedc0104010 msgr2=0x7fedc019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedbf400640 1 --2- 172.21.15.123:0/851328851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedc0104010 0x7fedc019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedbf400640 1 -- 172.21.15.123:0/851328851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc0109140 msgr2=0x7fedc019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedbf400640 1 --2- 172.21.15.123:0/851328851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc0109140 0x7fedc019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedbf400640 1 -- 172.21.15.123:0/851328851 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc019f5b0 con 0x7fedc01049c0 2024-09-19T04:56:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedbfe00640 1 --2- 172.21.15.123:0/851328851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedc0104010 0x7fedc019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.833+0000 7fedc4e00640 1 --2- 172.21.15.123:0/851328851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc0109140 0x7fedc019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.834+0000 7fedbf400640 1 --2- 172.21.15.123:0/851328851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedc01049c0 0x7fedc019ab40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7feda800e9f0 tx=0x7feda800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.834+0000 7fedbcc00640 1 -- 172.21.15.123:0/851328851 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda800ccc0 con 0x7fedc01049c0 2024-09-19T04:56:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.834+0000 7fedc6e00640 1 -- 172.21.15.123:0/851328851 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc019f8a0 con 0x7fedc01049c0 2024-09-19T04:56:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.834+0000 7fedbcc00640 1 -- 172.21.15.123:0/851328851 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feda8010910 con 0x7fedc01049c0 2024-09-19T04:56:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.834+0000 7fedbcc00640 1 -- 172.21.15.123:0/851328851 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8018960 con 0x7fedc01049c0 2024-09-19T04:56:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.834+0000 7fedc6e00640 1 -- 172.21.15.123:0/851328851 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc01a7180 con 0x7fedc01049c0 2024-09-19T04:56:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.837+0000 7fedbcc00640 1 -- 172.21.15.123:0/851328851 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7feda8010ac0 con 0x7fedc01049c0 2024-09-19T04:56:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.838+0000 7fedc986d740 1 Processor -- start 2024-09-19T04:56:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.838+0000 7fedc986d740 1 -- start start 2024-09-19T04:56:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.838+0000 7fedc986d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd6ec70 0x55c89dd6f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.838+0000 7fedc986d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6f590 0x55c89dd79910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.838+0000 7fedc986d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd79e50 0x55c89dd7c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.838+0000 7fedc986d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c89dcd53e0 con 0x55c89dd79e50 2024-09-19T04:56:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.838+0000 7fedc986d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c89dc7cd80 con 0x55c89dd6f590 2024-09-19T04:56:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.838+0000 7fedc986d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c89dcd13a0 con 0x55c89dd6ec70 2024-09-19T04:56:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.838+0000 7fedbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6f590 0x55c89dd79910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.838+0000 7fedc4e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd6ec70 0x55c89dd6f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.838+0000 7fedbf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd79e50 0x55c89dd7c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.838+0000 7fedbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6f590 0x55c89dd79910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:45582/0 (socket says 172.21.15.123:45582) 2024-09-19T04:56:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.839+0000 7fedbfe00640 1 -- 172.21.15.123:0/4180436285 learned_addr learned my addr 172.21.15.123:0/4180436285 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.839+0000 7fedc4e00640 1 -- 172.21.15.123:0/4180436285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6f590 msgr2=0x55c89dd79910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.839+0000 7fedc4e00640 1 --2- 172.21.15.123:0/4180436285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6f590 0x55c89dd79910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.839+0000 7fedc4e00640 1 -- 172.21.15.123:0/4180436285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd79e50 msgr2=0x55c89dd7c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.839+0000 7fedc4e00640 1 --2- 172.21.15.123:0/4180436285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd79e50 0x55c89dd7c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.839+0000 7fedc4e00640 1 -- 172.21.15.123:0/4180436285 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c89dc9bae0 con 0x55c89dd6ec70 2024-09-19T04:56:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.839+0000 7fedbfe00640 1 --2- 172.21.15.123:0/4180436285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6f590 0x55c89dd79910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.839+0000 7fedbf400640 1 --2- 172.21.15.123:0/4180436285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd79e50 0x55c89dd7c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.839+0000 7fedbcc00640 1 --2- 172.21.15.123:0/851328851 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7feda40a3430 0x7feda40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.839+0000 7fedc4e00640 1 --2- 172.21.15.123:0/4180436285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd6ec70 0x55c89dd6f050 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009f30 tx=0x7fedb80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37f2dbb972a42ccf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:55.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.839+0000 7fedbcc00640 1 -- 172.21.15.123:0/851328851 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7feda80d0070 con 0x7fedc01049c0 2024-09-19T04:56:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.840+0000 7fedbfe00640 1 --2- 172.21.15.123:0/851328851 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7feda40a3430 0x7feda40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.840+0000 7fedb5600640 1 -- 172.21.15.123:0/4180436285 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb80040f0 con 0x55c89dd6ec70 2024-09-19T04:56:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.840+0000 7fedb5600640 1 -- 172.21.15.123:0/4180436285 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedb8005500 con 0x55c89dd6ec70 2024-09-19T04:56:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.840+0000 7fedb5600640 1 -- 172.21.15.123:0/4180436285 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb8004dd0 con 0x55c89dd6ec70 2024-09-19T04:56:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.840+0000 7fedc986d740 1 -- 172.21.15.123:0/4180436285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd6ec70 msgr2=0x55c89dd6f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.840+0000 7fedbfe00640 1 --2- 172.21.15.123:0/851328851 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7feda40a3430 0x7feda40a58f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fedb0031f80 tx=0x7fedb003d040 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.840+0000 7fedc986d740 1 --2- 172.21.15.123:0/4180436285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd6ec70 0x55c89dd6f050 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009f30 tx=0x7fedb80118d0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.841+0000 7fedc986d740 1 -- 172.21.15.123:0/4180436285 shutdown_connections 2024-09-19T04:56:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.841+0000 7fedc986d740 1 --2- 172.21.15.123:0/4180436285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd79e50 0x55c89dd7c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.841+0000 7fedc986d740 1 --2- 172.21.15.123:0/4180436285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6f590 0x55c89dd79910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.841+0000 7fedc986d740 1 --2- 172.21.15.123:0/4180436285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd6ec70 0x55c89dd6f050 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.841+0000 7fedc986d740 1 -- 172.21.15.123:0/4180436285 >> 172.21.15.123:0/4180436285 conn(0x55c89dd6b200 msgr2=0x55c89dd6b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.841+0000 7fedc986d740 1 -- 172.21.15.123:0/4180436285 shutdown_connections 2024-09-19T04:56:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.841+0000 7fedc986d740 1 -- 172.21.15.123:0/4180436285 wait complete. 2024-09-19T04:56:55.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.841+0000 7fedc986d740 1 -- start start 2024-09-19T04:56:55.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.841+0000 7fedc986d740 1 Processor -- start 2024-09-19T04:56:55.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.842+0000 7fedc986d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6ec70 0x55c89dd76d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.842+0000 7fedc986d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6f590 0x55c89dd772a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.842+0000 7fedc986d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd79e50 0x55c89dd777e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.842+0000 7fedc986d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c89dd7ca20 con 0x55c89dd6f590 2024-09-19T04:56:55.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.842+0000 7fedc986d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c89dc900f0 con 0x55c89dd6ec70 2024-09-19T04:56:55.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.842+0000 7fedc986d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c89dd7cba0 con 0x55c89dd79e50 2024-09-19T04:56:55.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.842+0000 7fedc986d740 10 client.0 _ll_register_callbacks cb 0x55c89dc77bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:55.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.842+0000 7fedbf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6f590 0x55c89dd772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.842+0000 7fedbf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6f590 0x55c89dd772a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47476/0 (socket says 172.21.15.123:47476) 2024-09-19T04:56:55.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.842+0000 7fedbf400640 1 -- 172.21.15.123:0/3814826007 learned_addr learned my addr 172.21.15.123:0/3814826007 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:55.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.842+0000 7fedbfe00640 1 --2- 172.21.15.123:0/3814826007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6ec70 0x55c89dd76d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.842+0000 7fedc4e00640 1 --2- 172.21.15.123:0/3814826007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd79e50 0x55c89dd777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.843+0000 7fedbf400640 1 -- 172.21.15.123:0/3814826007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd79e50 msgr2=0x55c89dd777e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.843+0000 7fedbf400640 1 --2- 172.21.15.123:0/3814826007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd79e50 0x55c89dd777e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.843+0000 7fedbf400640 1 -- 172.21.15.123:0/3814826007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6ec70 msgr2=0x55c89dd76d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.843+0000 7fedbf400640 1 --2- 172.21.15.123:0/3814826007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6ec70 0x55c89dd76d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.843+0000 7fedbf400640 1 -- 172.21.15.123:0/3814826007 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c89da463c0 con 0x55c89dd6f590 2024-09-19T04:56:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.843+0000 7fedbf400640 1 --2- 172.21.15.123:0/3814826007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6f590 0x55c89dd772a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7feda800c6a0 tx=0x7feda8010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725370debee27afa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.843+0000 7fedae000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:56:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.843+0000 7fedae000640 1 -- 172.21.15.123:0/3814826007 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda809c3a0 con 0x55c89dd6f590 2024-09-19T04:56:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.843+0000 7fedae000640 1 -- 172.21.15.123:0/3814826007 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7feda80cd9e0 con 0x55c89dd6f590 2024-09-19T04:56:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.843+0000 7fedc986d740 1 -- 172.21.15.123:0/3814826007 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedb8005500 con 0x55c89dd6f590 2024-09-19T04:56:55.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.843+0000 7fedae000640 1 -- 172.21.15.123:0/3814826007 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7feda80a06a0 con 0x55c89dd6f590 2024-09-19T04:56:55.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.844+0000 7fedc986d740 10 client.17523 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:55.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.844+0000 7fedc986d740 1 -- 172.21.15.123:0/3814826007 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c89dd88a90 con 0x55c89dd6f590 2024-09-19T04:56:55.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.844+0000 7fedae000640 1 -- 172.21.15.123:0/3814826007 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedb8005500 con 0x55c89dd6f590 2024-09-19T04:56:55.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.844+0000 7fedae000640 1 -- 172.21.15.123:0/3814826007 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7feda809b860 con 0x55c89dd6f590 2024-09-19T04:56:55.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.844+0000 7fedc986d740 10 client.17523 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:55.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.844+0000 7fedc986d740 10 client.17523 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:56:55.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.844+0000 7fedc986d740 20 client.17523 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:55.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.844+0000 7fedc986d740 1 --2- 172.21.15.123:0/3814826007 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55c89dd89ea0 0x55c89dd8c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.844+0000 7fedc986d740 4 client.17523 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:56:55.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.844+0000 7fedc986d740 1 -- 172.21.15.123:0/3814826007 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c89dce2950 con 0x55c89dd89ea0 2024-09-19T04:56:55.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.845+0000 7fedbfe00640 1 --2- 172.21.15.123:0/3814826007 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55c89dd89ea0 0x55c89dd8c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.845+0000 7fedbfe00640 1 --2- 172.21.15.123:0/3814826007 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55c89dd89ea0 0x55c89dd8c290 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=744664a621dbc97f server_cookie=5bc22ec7641300e3 in_seq=0 out_seq=0 2024-09-19T04:56:55.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.845+0000 7fedae000640 10 client.17523 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:56:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.846+0000 7fedae000640 1 -- 172.21.15.123:0/3814826007 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c89dce2950 con 0x55c89dd89ea0 2024-09-19T04:56:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.846+0000 7fedae000640 10 client.17523 handle_command_reply: tid=0 2024-09-19T04:56:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.846+0000 7fedc986d740 1 client.17523 shutdown 2024-09-19T04:56:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.846+0000 7fedc986d740 1 -- 172.21.15.123:0/3814826007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6f590 msgr2=0x55c89dd772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.846+0000 7fedc986d740 1 --2- 172.21.15.123:0/3814826007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6f590 0x55c89dd772a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7feda800c6a0 tx=0x7feda8010ee0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.846+0000 7fedc986d740 1 -- 172.21.15.123:0/3814826007 shutdown_connections 2024-09-19T04:56:55.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.846+0000 7fedc986d740 1 --2- 172.21.15.123:0/3814826007 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55c89dd89ea0 0x55c89dd8c290 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.846+0000 7fedc986d740 1 --2- 172.21.15.123:0/3814826007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd79e50 0x55c89dd777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.846+0000 7fedc986d740 1 --2- 172.21.15.123:0/3814826007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6f590 0x55c89dd772a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.846+0000 7fedc986d740 1 --2- 172.21.15.123:0/3814826007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6ec70 0x55c89dd76d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.846+0000 7fedc986d740 1 -- 172.21.15.123:0/3814826007 >> 172.21.15.123:0/3814826007 conn(0x55c89dd6b200 msgr2=0x55c89dd70ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.846+0000 7fedae000640 0 client.17523 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:56:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.846+0000 7fedc986d740 1 -- 172.21.15.123:0/3814826007 shutdown_connections 2024-09-19T04:56:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.847+0000 7fedc986d740 1 -- 172.21.15.123:0/3814826007 wait complete. 2024-09-19T04:56:55.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.847+0000 7fedc986d740 20 client.17523 trim_cache size 0 max 16384 2024-09-19T04:56:55.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.860+0000 7fedc986d740 1 Processor -- start 2024-09-19T04:56:55.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.860+0000 7fedc986d740 1 -- start start 2024-09-19T04:56:55.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.861+0000 7fedc986d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6ec70 0x55c89dd78cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.861+0000 7fedc986d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6f590 0x55c89dd91b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.861+0000 7fedc986d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd79e50 0x55c89dd92080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.861+0000 7fedc986d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fedc01afaa0 con 0x55c89dd6ec70 2024-09-19T04:56:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.861+0000 7fedc986d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fedc01b15e0 con 0x55c89dd6f590 2024-09-19T04:56:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.861+0000 7fedc986d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fedc01b0170 con 0x55c89dd79e50 2024-09-19T04:56:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.861+0000 7fedbfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6ec70 0x55c89dd78cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.861+0000 7fedbfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6ec70 0x55c89dd78cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47482/0 (socket says 172.21.15.123:47482) 2024-09-19T04:56:55.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.861+0000 7fedbfe00640 1 -- 172.21.15.123:0/3872229245 learned_addr learned my addr 172.21.15.123:0/3872229245 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:55.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.861+0000 7fedc4e00640 1 --2- 172.21.15.123:0/3872229245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd79e50 0x55c89dd92080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.861+0000 7fedbf400640 1 --2- 172.21.15.123:0/3872229245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6f590 0x55c89dd91b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.861+0000 7fedbfe00640 1 -- 172.21.15.123:0/3872229245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd79e50 msgr2=0x55c89dd92080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:55.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.862+0000 7fedbfe00640 1 --2- 172.21.15.123:0/3872229245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd79e50 0x55c89dd92080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.862+0000 7fedbfe00640 1 -- 172.21.15.123:0/3872229245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6f590 msgr2=0x55c89dd91b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:55.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.862+0000 7fedbfe00640 1 --2- 172.21.15.123:0/3872229245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6f590 0x55c89dd91b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.862+0000 7fedbfe00640 1 -- 172.21.15.123:0/3872229245 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c89dd66770 con 0x55c89dd6ec70 2024-09-19T04:56:55.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.862+0000 7fedbf400640 1 --2- 172.21.15.123:0/3872229245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6f590 0x55c89dd91b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:56:55.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.862+0000 7fedbfe00640 1 --2- 172.21.15.123:0/3872229245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6ec70 0x55c89dd78cd0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x55c89dd6c060 tx=0x7fedb0041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1ec0adfd7f7a0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:55.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.862+0000 7fedb4c00640 1 -- 172.21.15.123:0/3872229245 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb00307c0 con 0x55c89dd6ec70 2024-09-19T04:56:55.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.862+0000 7fedb4c00640 1 -- 172.21.15.123:0/3872229245 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedb0036040 con 0x55c89dd6ec70 2024-09-19T04:56:55.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.862+0000 7fedb4c00640 1 -- 172.21.15.123:0/3872229245 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb00539d0 con 0x55c89dd6ec70 2024-09-19T04:56:55.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.863+0000 7fedc986d740 1 -- 172.21.15.123:0/3872229245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6ec70 msgr2=0x55c89dd78cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.863+0000 7fedc986d740 1 --2- 172.21.15.123:0/3872229245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6ec70 0x55c89dd78cd0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x55c89dd6c060 tx=0x7fedb0041000 comp rx=0 tx=0).stop 2024-09-19T04:56:55.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.863+0000 7fedc986d740 1 -- 172.21.15.123:0/3872229245 shutdown_connections 2024-09-19T04:56:55.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.863+0000 7fedc986d740 1 --2- 172.21.15.123:0/3872229245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd79e50 0x55c89dd92080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.863+0000 7fedc986d740 1 --2- 172.21.15.123:0/3872229245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd6f590 0x55c89dd91b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.863+0000 7fedc986d740 1 --2- 172.21.15.123:0/3872229245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6ec70 0x55c89dd78cd0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.863+0000 7fedc986d740 1 -- 172.21.15.123:0/3872229245 >> 172.21.15.123:0/3872229245 conn(0x55c89dd6b200 msgr2=0x55c89dd785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:55.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.863+0000 7fedc986d740 1 -- 172.21.15.123:0/3872229245 shutdown_connections 2024-09-19T04:56:55.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.863+0000 7fedc986d740 1 -- 172.21.15.123:0/3872229245 wait complete. 2024-09-19T04:56:55.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.863+0000 7fedc986d740 1 -- start start 2024-09-19T04:56:55.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.863+0000 7fedc986d740 1 Processor -- start 2024-09-19T04:56:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.864+0000 7fedc986d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd6ec70 0x55c89dd81df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.864+0000 7fedc986d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6f590 0x55c89dd82330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.864+0000 7fedc986d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd79e50 0x55c89dd7edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.864+0000 7fedc986d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fedc01b0f10 con 0x55c89dd6f590 2024-09-19T04:56:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.864+0000 7fedc986d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fedc01b0840 con 0x55c89dd79e50 2024-09-19T04:56:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.864+0000 7fedc986d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fedc01adaa0 con 0x55c89dd6ec70 2024-09-19T04:56:55.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.864+0000 7fedc986d740 10 client.0 _ll_register_callbacks cb 0x7feda80d3810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:55.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.864+0000 7fedbf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6f590 0x55c89dd82330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.864+0000 7fedbf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6f590 0x55c89dd82330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47486/0 (socket says 172.21.15.123:47486) 2024-09-19T04:56:55.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.864+0000 7fedbf400640 1 -- 172.21.15.123:0/2558877316 learned_addr learned my addr 172.21.15.123:0/2558877316 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:55.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.864+0000 7fedbfe00640 1 --2- 172.21.15.123:0/2558877316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd6ec70 0x55c89dd81df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.864+0000 7fedc4e00640 1 --2- 172.21.15.123:0/2558877316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd79e50 0x55c89dd7edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.865+0000 7fedbf400640 1 -- 172.21.15.123:0/2558877316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd6ec70 msgr2=0x55c89dd81df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.865+0000 7fedbf400640 1 --2- 172.21.15.123:0/2558877316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd6ec70 0x55c89dd81df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.865+0000 7fedbf400640 1 -- 172.21.15.123:0/2558877316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd79e50 msgr2=0x55c89dd7edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.865+0000 7fedbf400640 1 --2- 172.21.15.123:0/2558877316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd79e50 0x55c89dd7edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.865+0000 7fedbf400640 1 -- 172.21.15.123:0/2558877316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c89dd7f300 con 0x55c89dd6f590 2024-09-19T04:56:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.865+0000 7fedbf400640 1 --2- 172.21.15.123:0/2558877316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6f590 0x55c89dd82330 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7feda8098800 tx=0x7feda809bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a19728e5130a1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.865+0000 7fedae000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:56:55.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.865+0000 7fedae000640 1 -- 172.21.15.123:0/2558877316 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda809c3a0 con 0x55c89dd6f590 2024-09-19T04:56:55.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.865+0000 7fedae000640 1 -- 172.21.15.123:0/2558877316 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7feda80104b0 con 0x55c89dd6f590 2024-09-19T04:56:55.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.865+0000 7fedc986d740 1 -- 172.21.15.123:0/2558877316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedb0036040 con 0x55c89dd6f590 2024-09-19T04:56:55.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.865+0000 7fedae000640 1 -- 172.21.15.123:0/2558877316 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7feda80cdbf0 con 0x55c89dd6f590 2024-09-19T04:56:55.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.865+0000 7fedc986d740 10 client.17529 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:55.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.866+0000 7fedc986d740 1 -- 172.21.15.123:0/2558877316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c89ddf5210 con 0x55c89dd6f590 2024-09-19T04:56:55.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.866+0000 7fedae000640 1 -- 172.21.15.123:0/2558877316 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedb0036040 con 0x55c89dd6f590 2024-09-19T04:56:55.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.866+0000 7fedae000640 1 -- 172.21.15.123:0/2558877316 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7feda80980f0 con 0x55c89dd6f590 2024-09-19T04:56:55.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.866+0000 7fedc986d740 10 client.17529 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:55.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.866+0000 7fedc986d740 10 client.17529 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T04:56:55.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.866+0000 7fedc986d740 20 client.17529 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:55.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.866+0000 7fedc986d740 1 --2- 172.21.15.123:0/2558877316 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55c89ddf57b0 0x55c89ddf7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:55.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.866+0000 7fedc986d740 4 client.17529 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:56:55.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.866+0000 7fedc986d740 1 -- 172.21.15.123:0/2558877316 --> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c89dcdce60 con 0x55c89ddf57b0 2024-09-19T04:56:55.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.866+0000 7fedbfe00640 1 --2- 172.21.15.123:0/2558877316 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55c89ddf57b0 0x55c89ddf7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:55.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.867+0000 7fedbfe00640 1 --2- 172.21.15.123:0/2558877316 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55c89ddf57b0 0x55c89ddf7bd0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9fa10d2d8e835a9 server_cookie=8668d9680191657a in_seq=0 out_seq=0 2024-09-19T04:56:55.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.867+0000 7fedae000640 10 client.17529 ms_handle_connect on v2:172.21.15.193:6836/3444596480 2024-09-19T04:56:55.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.867+0000 7fedae000640 1 -- 172.21.15.123:0/2558877316 <== mds.2 v2:172.21.15.193:6836/3444596480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c89dcdce60 con 0x55c89ddf57b0 2024-09-19T04:56:55.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.867+0000 7fedae000640 10 client.17529 handle_command_reply: tid=0 2024-09-19T04:56:55.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.867+0000 7fedc986d740 1 client.17529 shutdown 2024-09-19T04:56:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.867+0000 7fedc986d740 1 -- 172.21.15.123:0/2558877316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6f590 msgr2=0x55c89dd82330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.867+0000 7fedc986d740 1 --2- 172.21.15.123:0/2558877316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6f590 0x55c89dd82330 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7feda8098800 tx=0x7feda809bdc0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.868+0000 7fedc986d740 1 -- 172.21.15.123:0/2558877316 shutdown_connections 2024-09-19T04:56:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.868+0000 7fedc986d740 1 --2- 172.21.15.123:0/2558877316 >> [v2:172.21.15.193:6836/3444596480,v1:172.21.15.193:6837/3444596480] conn(0x55c89ddf57b0 0x55c89ddf7bd0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.868+0000 7fedc986d740 1 --2- 172.21.15.123:0/2558877316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c89dd79e50 0x55c89dd7edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.868+0000 7fedae000640 0 client.17529 ms_handle_reset on v2:172.21.15.193:6836/3444596480 2024-09-19T04:56:55.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.868+0000 7fedc986d740 1 --2- 172.21.15.123:0/2558877316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c89dd6f590 0x55c89dd82330 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.868+0000 7fedc986d740 1 --2- 172.21.15.123:0/2558877316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c89dd6ec70 0x55c89dd81df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.868+0000 7fedc986d740 1 -- 172.21.15.123:0/2558877316 >> 172.21.15.123:0/2558877316 conn(0x55c89dd6b200 msgr2=0x55c89dd78de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:55.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.868+0000 7fedc986d740 1 -- 172.21.15.123:0/2558877316 shutdown_connections 2024-09-19T04:56:55.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.868+0000 7fedc986d740 1 -- 172.21.15.123:0/2558877316 wait complete. 2024-09-19T04:56:55.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.868+0000 7fedc986d740 20 client.17529 trim_cache size 0 max 16384 2024-09-19T04:56:55.888 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:56:55.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.868+0000 7fedc6e00640 1 -- 172.21.15.123:0/851328851 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7feda40a3430 msgr2=0x7feda40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.868+0000 7fedc6e00640 1 --2- 172.21.15.123:0/851328851 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7feda40a3430 0x7feda40a58f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fedb0031f80 tx=0x7fedb003d040 comp rx=0 tx=0).stop 2024-09-19T04:56:55.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.868+0000 7fedc6e00640 1 -- 172.21.15.123:0/851328851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedc01049c0 msgr2=0x7fedc019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.868+0000 7fedc6e00640 1 --2- 172.21.15.123:0/851328851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedc01049c0 0x7fedc019ab40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7feda800e9f0 tx=0x7feda800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.869+0000 7fedc6e00640 1 -- 172.21.15.123:0/851328851 shutdown_connections 2024-09-19T04:56:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.869+0000 7fedc6e00640 1 --2- 172.21.15.123:0/851328851 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7feda40a3430 0x7feda40a58f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.869+0000 7fedc6e00640 1 --2- 172.21.15.123:0/851328851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc0109140 0x7fedc019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.869+0000 7fedc6e00640 1 --2- 172.21.15.123:0/851328851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedc01049c0 0x7fedc019ab40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.869+0000 7fedc6e00640 1 --2- 172.21.15.123:0/851328851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedc0104010 0x7fedc019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.869+0000 7fedc6e00640 1 -- 172.21.15.123:0/851328851 >> 172.21.15.123:0/851328851 conn(0x7fedc00ffa50 msgr2=0x7fedc0100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.869+0000 7fedc6e00640 1 -- 172.21.15.123:0/851328851 shutdown_connections 2024-09-19T04:56:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:55.869+0000 7fedc6e00640 1 -- 172.21.15.123:0/851328851 wait complete. 2024-09-19T04:56:56.598 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:56 smithi123 ceph-mon[30511]: from='client.27596 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:56:56.598 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:56 smithi123 ceph-mon[30511]: from='client.17511 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:56:56.636 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:56:56.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:56 smithi193 ceph-mon[35592]: from='client.27596 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:56:56.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:56 smithi193 ceph-mon[35592]: from='client.17511 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:56:56.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:56 smithi183 ceph-mon[35333]: from='client.27596 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:56:56.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:56 smithi183 ceph-mon[35333]: from='client.17511 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:56:57.047 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:57.527 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:57 smithi123 ceph-mon[30511]: pgmap v318: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:57.642 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:57 smithi193 ceph-mon[35592]: pgmap v318: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:57.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.635+0000 7f4fc7e00640 1 -- 172.21.15.123:0/2654103798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc8108f60 msgr2=0x7f4fc8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:57.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.635+0000 7f4fc7e00640 1 --2- 172.21.15.123:0/2654103798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc8108f60 0x7f4fc8110ae0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009ca0 tx=0x7f4fc002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.635+0000 7f4fc7e00640 1 -- 172.21.15.123:0/2654103798 shutdown_connections 2024-09-19T04:56:57.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.635+0000 7f4fc7e00640 1 --2- 172.21.15.123:0/2654103798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc8108f60 0x7f4fc8110ae0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.635+0000 7f4fc7e00640 1 --2- 172.21.15.123:0/2654103798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc81047e0 0x7f4fc8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.635+0000 7f4fc7e00640 1 --2- 172.21.15.123:0/2654103798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fc8103e30 0x7f4fc8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.635+0000 7f4fc7e00640 1 -- 172.21.15.123:0/2654103798 >> 172.21.15.123:0/2654103798 conn(0x7f4fc80ff8d0 msgr2=0x7f4fc8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:57.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.635+0000 7f4fc7e00640 1 -- 172.21.15.123:0/2654103798 shutdown_connections 2024-09-19T04:56:57.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.635+0000 7f4fc7e00640 1 -- 172.21.15.123:0/2654103798 wait complete. 2024-09-19T04:56:57.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.636+0000 7f4fc7e00640 1 Processor -- start 2024-09-19T04:56:57.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.636+0000 7f4fc7e00640 1 -- start start 2024-09-19T04:56:57.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.636+0000 7f4fc7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc8103e30 0x7f4fc819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.636+0000 7f4fc7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc81047e0 0x7f4fc819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.636+0000 7f4fc7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fc8108f60 0x7f4fc819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.636+0000 7f4fc7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc8113fc0 con 0x7f4fc81047e0 2024-09-19T04:56:57.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.636+0000 7f4fc7e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc8113e40 con 0x7f4fc8108f60 2024-09-19T04:56:57.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.636+0000 7f4fc7e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc8114140 con 0x7f4fc8103e30 2024-09-19T04:56:57.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.637+0000 7f4fc6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc81047e0 0x7f4fc819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.637+0000 7f4fc6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc8103e30 0x7f4fc819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.637+0000 7f4fc6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc81047e0 0x7f4fc819a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47498/0 (socket says 172.21.15.123:47498) 2024-09-19T04:56:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.637+0000 7f4fc6000640 1 -- 172.21.15.123:0/1870024817 learned_addr learned my addr 172.21.15.123:0/1870024817 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.637+0000 7f4fc7400640 1 --2- 172.21.15.123:0/1870024817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fc8108f60 0x7f4fc819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.637+0000 7f4fc6a00640 1 -- 172.21.15.123:0/1870024817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fc8108f60 msgr2=0x7f4fc819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.637+0000 7f4fc6a00640 1 --2- 172.21.15.123:0/1870024817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fc8108f60 0x7f4fc819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.637+0000 7f4fc6a00640 1 -- 172.21.15.123:0/1870024817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc81047e0 msgr2=0x7f4fc819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.637+0000 7f4fc6a00640 1 --2- 172.21.15.123:0/1870024817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc81047e0 0x7f4fc819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.637+0000 7f4fc6a00640 1 -- 172.21.15.123:0/1870024817 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc819f3b0 con 0x7f4fc8103e30 2024-09-19T04:56:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.637+0000 7f4fc6a00640 1 --2- 172.21.15.123:0/1870024817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc8103e30 0x7f4fc819a400 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800e9f0 tx=0x7f4fb800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.638+0000 7f4fbf400640 1 -- 172.21.15.123:0/1870024817 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800cd80 con 0x7f4fc8103e30 2024-09-19T04:56:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.638+0000 7f4fbf400640 1 -- 172.21.15.123:0/1870024817 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb80035f0 con 0x7f4fc8103e30 2024-09-19T04:56:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.638+0000 7f4fbf400640 1 -- 172.21.15.123:0/1870024817 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8010690 con 0x7f4fc8103e30 2024-09-19T04:56:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.638+0000 7f4fc7e00640 1 -- 172.21.15.123:0/1870024817 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc819f6a0 con 0x7f4fc8103e30 2024-09-19T04:56:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.638+0000 7f4fc7e00640 1 -- 172.21.15.123:0/1870024817 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc81a6f80 con 0x7f4fc8103e30 2024-09-19T04:56:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.641+0000 7f4fbf400640 1 -- 172.21.15.123:0/1870024817 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4fb8003b30 con 0x7f4fc8103e30 2024-09-19T04:56:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.641+0000 7f4fd043b740 1 Processor -- start 2024-09-19T04:56:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.641+0000 7f4fd043b740 1 -- start start 2024-09-19T04:56:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.641+0000 7f4fd043b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40074e40 0x555f40075220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.641+0000 7f4fd043b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40075760 0x555f4007fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.642+0000 7f4fd043b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40080020 0x555f40082410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.642+0000 7f4fd043b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555f3ffdb3e0 con 0x555f40080020 2024-09-19T04:56:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.642+0000 7f4fd043b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555f3ff82d80 con 0x555f40075760 2024-09-19T04:56:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.642+0000 7f4fd043b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555f3ffd73a0 con 0x555f40074e40 2024-09-19T04:56:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.642+0000 7f4fc7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40074e40 0x555f40075220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.642+0000 7f4fc6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40080020 0x555f40082410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.642+0000 7f4fc7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40074e40 0x555f40075220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53152/0 (socket says 172.21.15.123:53152) 2024-09-19T04:56:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.642+0000 7f4fc7400640 1 -- 172.21.15.123:0/3067891988 learned_addr learned my addr 172.21.15.123:0/3067891988 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.642+0000 7f4fc6a00640 1 --2- 172.21.15.123:0/3067891988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40075760 0x555f4007fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.642+0000 7f4fc7400640 1 -- 172.21.15.123:0/3067891988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40075760 msgr2=0x555f4007fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.642+0000 7f4fc7400640 1 --2- 172.21.15.123:0/3067891988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40075760 0x555f4007fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.642+0000 7f4fc7400640 1 -- 172.21.15.123:0/3067891988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40080020 msgr2=0x555f40082410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.642+0000 7f4fc7400640 1 --2- 172.21.15.123:0/3067891988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40080020 0x555f40082410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.642+0000 7f4fc7400640 1 -- 172.21.15.123:0/3067891988 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f3ffa1ae0 con 0x555f40074e40 2024-09-19T04:56:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.643+0000 7f4fbf400640 1 --2- 172.21.15.123:0/1870024817 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f4fac0a3430 0x7f4fac0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.643+0000 7f4fc7400640 1 --2- 172.21.15.123:0/3067891988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40074e40 0x555f40075220 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009ca0 tx=0x7f4fc0005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66c28480be09c04b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.643+0000 7f4fbf400640 1 -- 172.21.15.123:0/1870024817 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f4fb80cfff0 con 0x7f4fc8103e30 2024-09-19T04:56:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.643+0000 7f4fc6a00640 1 --2- 172.21.15.123:0/1870024817 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f4fac0a3430 0x7f4fac0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.643+0000 7f4fb3e00640 1 -- 172.21.15.123:0/3067891988 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0031db0 con 0x555f40074e40 2024-09-19T04:56:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.643+0000 7f4fb3e00640 1 -- 172.21.15.123:0/3067891988 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fc0036040 con 0x555f40074e40 2024-09-19T04:56:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.643+0000 7f4fb3e00640 1 -- 172.21.15.123:0/3067891988 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0004330 con 0x555f40074e40 2024-09-19T04:56:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.644+0000 7f4fd043b740 1 -- 172.21.15.123:0/3067891988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40074e40 msgr2=0x555f40075220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.644+0000 7f4fd043b740 1 --2- 172.21.15.123:0/3067891988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40074e40 0x555f40075220 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009ca0 tx=0x7f4fc0005f10 comp rx=0 tx=0).stop 2024-09-19T04:56:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.644+0000 7f4fc6a00640 1 --2- 172.21.15.123:0/1870024817 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f4fac0a3430 0x7f4fac0a58f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x555f40072230 tx=0x7f4fb80d3ac0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.644+0000 7f4fd043b740 1 -- 172.21.15.123:0/3067891988 shutdown_connections 2024-09-19T04:56:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.644+0000 7f4fd043b740 1 --2- 172.21.15.123:0/3067891988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40080020 0x555f40082410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.644+0000 7f4fd043b740 1 --2- 172.21.15.123:0/3067891988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40075760 0x555f4007fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.644+0000 7f4fd043b740 1 --2- 172.21.15.123:0/3067891988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40074e40 0x555f40075220 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.644+0000 7f4fd043b740 1 -- 172.21.15.123:0/3067891988 >> 172.21.15.123:0/3067891988 conn(0x555f400713d0 msgr2=0x555f400717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.644+0000 7f4fd043b740 1 -- 172.21.15.123:0/3067891988 shutdown_connections 2024-09-19T04:56:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.644+0000 7f4fd043b740 1 -- 172.21.15.123:0/3067891988 wait complete. 2024-09-19T04:56:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.644+0000 7f4fd043b740 1 -- start start 2024-09-19T04:56:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.645+0000 7f4fd043b740 1 Processor -- start 2024-09-19T04:56:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.645+0000 7f4fd043b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40074e40 0x555f4007cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.645+0000 7f4fd043b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40075760 0x555f4007d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.645+0000 7f4fd043b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40080020 0x555f4007d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.645+0000 7f4fd043b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555f40082a70 con 0x555f40075760 2024-09-19T04:56:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.645+0000 7f4fd043b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555f3ff960f0 con 0x555f40074e40 2024-09-19T04:56:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.645+0000 7f4fd043b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555f40082bf0 con 0x555f40080020 2024-09-19T04:56:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.645+0000 7f4fd043b740 10 client.0 _ll_register_callbacks cb 0x555f3ff7dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.645+0000 7f4fc6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40075760 0x555f4007d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.645+0000 7f4fc7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40074e40 0x555f4007cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.645+0000 7f4fc7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40074e40 0x555f4007cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:45640/0 (socket says 172.21.15.123:45640) 2024-09-19T04:56:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.645+0000 7f4fc7400640 1 -- 172.21.15.123:0/4142677166 learned_addr learned my addr 172.21.15.123:0/4142677166 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.645+0000 7f4fc6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40075760 0x555f4007d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47522/0 (socket says 172.21.15.123:47522) 2024-09-19T04:56:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.646+0000 7f4fc7400640 1 --2- 172.21.15.123:0/4142677166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40080020 0x555f4007d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.646+0000 7f4fc6000640 1 -- 172.21.15.123:0/4142677166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40080020 msgr2=0x555f4007d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.646+0000 7f4fc6000640 1 --2- 172.21.15.123:0/4142677166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40080020 0x555f4007d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.646+0000 7f4fc6000640 1 -- 172.21.15.123:0/4142677166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40074e40 msgr2=0x555f4007cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.646+0000 7f4fc6000640 1 --2- 172.21.15.123:0/4142677166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40074e40 0x555f4007cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.646+0000 7f4fc6000640 1 -- 172.21.15.123:0/4142677166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f3fd4c3c0 con 0x555f40075760 2024-09-19T04:56:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.646+0000 7f4fc7400640 1 --2- 172.21.15.123:0/4142677166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40080020 0x555f4007d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:56:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.646+0000 7f4fc6000640 1 --2- 172.21.15.123:0/4142677166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40075760 0x555f4007d470 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x555f40072750 tx=0x7f4fb4012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9e1b27239ce427 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.646+0000 7f4fb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-19T04:56:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.646+0000 7f4fb0c00640 1 -- 172.21.15.123:0/4142677166 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb400f890 con 0x555f40075760 2024-09-19T04:56:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.646+0000 7f4fb0c00640 1 -- 172.21.15.123:0/4142677166 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f4fb400fa30 con 0x555f40075760 2024-09-19T04:56:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.646+0000 7f4fd043b740 1 -- 172.21.15.123:0/4142677166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fc0036040 con 0x555f40075760 2024-09-19T04:56:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.647+0000 7f4fb0c00640 1 -- 172.21.15.123:0/4142677166 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4fb400fc30 con 0x555f40075760 2024-09-19T04:56:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.647+0000 7f4fd043b740 10 client.17541 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.647+0000 7f4fd043b740 1 -- 172.21.15.123:0/4142677166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555f4008eb40 con 0x555f40075760 2024-09-19T04:56:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.647+0000 7f4fb0c00640 1 -- 172.21.15.123:0/4142677166 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fc0036040 con 0x555f40075760 2024-09-19T04:56:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.647+0000 7f4fb0c00640 1 -- 172.21.15.123:0/4142677166 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f4fb4004aa0 con 0x555f40075760 2024-09-19T04:56:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.647+0000 7f4fd043b740 10 client.17541 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.647+0000 7f4fd043b740 10 client.17541 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:56:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.647+0000 7f4fd043b740 20 client.17541 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.647+0000 7f4fd043b740 1 --2- 172.21.15.123:0/4142677166 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555f4008ff50 0x555f40092360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.647+0000 7f4fd043b740 4 client.17541 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:56:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.647+0000 7f4fd043b740 1 -- 172.21.15.123:0/4142677166 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f3ffe8950 con 0x555f4008ff50 2024-09-19T04:56:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.648+0000 7f4fc7400640 1 --2- 172.21.15.123:0/4142677166 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555f4008ff50 0x555f40092360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.648+0000 7f4fb0c00640 10 client.17541 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:56:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.648+0000 7f4fc7400640 1 --2- 172.21.15.123:0/4142677166 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555f4008ff50 0x555f40092360 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2612211448591b9 server_cookie=e0f76872a1fa817e in_seq=0 out_seq=0 2024-09-19T04:56:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.649+0000 7f4fb0c00640 1 -- 172.21.15.123:0/4142677166 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f3ffe8950 con 0x555f4008ff50 2024-09-19T04:56:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.649+0000 7f4fb0c00640 10 client.17541 handle_command_reply: tid=0 2024-09-19T04:56:57.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.649+0000 7f4fd043b740 1 client.17541 shutdown 2024-09-19T04:56:57.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.649+0000 7f4fd043b740 1 -- 172.21.15.123:0/4142677166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40075760 msgr2=0x555f4007d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:57.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.649+0000 7f4fd043b740 1 --2- 172.21.15.123:0/4142677166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40075760 0x555f4007d470 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x555f40072750 tx=0x7f4fb4012bd0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.649+0000 7f4fd043b740 1 -- 172.21.15.123:0/4142677166 shutdown_connections 2024-09-19T04:56:57.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.649+0000 7f4fd043b740 1 --2- 172.21.15.123:0/4142677166 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555f4008ff50 0x555f40092360 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.649+0000 7f4fd043b740 1 --2- 172.21.15.123:0/4142677166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40080020 0x555f4007d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.649+0000 7f4fd043b740 1 --2- 172.21.15.123:0/4142677166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40075760 0x555f4007d470 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.649+0000 7f4fb0c00640 0 client.17541 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:56:57.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.649+0000 7f4fd043b740 1 --2- 172.21.15.123:0/4142677166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40074e40 0x555f4007cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.649+0000 7f4fd043b740 1 -- 172.21.15.123:0/4142677166 >> 172.21.15.123:0/4142677166 conn(0x555f400713d0 msgr2=0x555f400770b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:57.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.649+0000 7f4fd043b740 1 -- 172.21.15.123:0/4142677166 shutdown_connections 2024-09-19T04:56:57.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.650+0000 7f4fd043b740 1 -- 172.21.15.123:0/4142677166 wait complete. 2024-09-19T04:56:57.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.650+0000 7f4fd043b740 20 client.17541 trim_cache size 0 max 16384 2024-09-19T04:56:57.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.663+0000 7f4fd043b740 1 Processor -- start 2024-09-19T04:56:57.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.663+0000 7f4fd043b740 1 -- start start 2024-09-19T04:56:57.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.663+0000 7f4fd043b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40074e40 0x555f400778a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.663+0000 7f4fd043b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40075760 0x555f40097c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.663+0000 7f4fd043b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40080020 0x555f40098170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.663+0000 7f4fd043b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc81af890 con 0x555f40074e40 2024-09-19T04:56:57.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.663+0000 7f4fd043b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc81b13d0 con 0x555f40080020 2024-09-19T04:56:57.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.663+0000 7f4fd043b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc81aff60 con 0x555f40075760 2024-09-19T04:56:57.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.664+0000 7f4fc7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40074e40 0x555f400778a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.664+0000 7f4fc7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40074e40 0x555f400778a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47536/0 (socket says 172.21.15.123:47536) 2024-09-19T04:56:57.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.664+0000 7f4fc7400640 1 -- 172.21.15.123:0/3743098063 learned_addr learned my addr 172.21.15.123:0/3743098063 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:57.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.664+0000 7f4fc6000640 1 --2- 172.21.15.123:0/3743098063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40075760 0x555f40097c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.664+0000 7f4fc7400640 1 --2- 172.21.15.123:0/3743098063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40080020 0x555f40098170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.664+0000 7f4fc6000640 1 -- 172.21.15.123:0/3743098063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40080020 msgr2=0x555f40098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:57.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.664+0000 7f4fc6000640 1 --2- 172.21.15.123:0/3743098063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40080020 0x555f40098170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.664+0000 7f4fc6000640 1 -- 172.21.15.123:0/3743098063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40074e40 msgr2=0x555f400778a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:57.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.664+0000 7f4fc6000640 1 --2- 172.21.15.123:0/3743098063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40074e40 0x555f400778a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.664+0000 7f4fc6000640 1 -- 172.21.15.123:0/3743098063 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f40075590 con 0x555f40075760 2024-09-19T04:56:57.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.664+0000 7f4fc7400640 1 --2- 172.21.15.123:0/3743098063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40074e40 0x555f400778a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:57.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.665+0000 7f4fc6000640 1 --2- 172.21.15.123:0/3743098063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40075760 0x555f40097c30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4025480 tx=0x7f4fb40097b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8de50266346f4093 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:57.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.665+0000 7f4fb3400640 1 -- 172.21.15.123:0/3743098063 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4025500 con 0x555f40075760 2024-09-19T04:56:57.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.665+0000 7f4fb3400640 1 -- 172.21.15.123:0/3743098063 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb40256a0 con 0x555f40075760 2024-09-19T04:56:57.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.665+0000 7f4fb3400640 1 -- 172.21.15.123:0/3743098063 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4005400 con 0x555f40075760 2024-09-19T04:56:57.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.665+0000 7f4fd043b740 1 -- 172.21.15.123:0/3743098063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40075760 msgr2=0x555f40097c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.665+0000 7f4fd043b740 1 --2- 172.21.15.123:0/3743098063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40075760 0x555f40097c30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4025480 tx=0x7f4fb40097b0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.666+0000 7f4fd043b740 1 -- 172.21.15.123:0/3743098063 shutdown_connections 2024-09-19T04:56:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.666+0000 7f4fd043b740 1 --2- 172.21.15.123:0/3743098063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40080020 0x555f40098170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.666+0000 7f4fd043b740 1 --2- 172.21.15.123:0/3743098063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40075760 0x555f40097c30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.666+0000 7f4fd043b740 1 --2- 172.21.15.123:0/3743098063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40074e40 0x555f400778a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.666+0000 7f4fd043b740 1 -- 172.21.15.123:0/3743098063 >> 172.21.15.123:0/3743098063 conn(0x555f400713d0 msgr2=0x555f4007efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.666+0000 7f4fd043b740 1 -- 172.21.15.123:0/3743098063 shutdown_connections 2024-09-19T04:56:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.666+0000 7f4fd043b740 1 -- 172.21.15.123:0/3743098063 wait complete. 2024-09-19T04:56:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.666+0000 7f4fd043b740 1 -- start start 2024-09-19T04:56:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.666+0000 7f4fd043b740 1 Processor -- start 2024-09-19T04:56:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.667+0000 7f4fd043b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40074e40 0x555f40083b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.667+0000 7f4fd043b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40075760 0x555f400840b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.667+0000 7f4fd043b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40080020 0x555f400845f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.667+0000 7f4fd043b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc81b0d00 con 0x555f40080020 2024-09-19T04:56:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.667+0000 7f4fd043b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc81b0630 con 0x555f40074e40 2024-09-19T04:56:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.667+0000 7f4fd043b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc81ad890 con 0x555f40075760 2024-09-19T04:56:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.667+0000 7f4fd043b740 10 client.0 _ll_register_callbacks cb 0x7f4fb4009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.667+0000 7f4fc6000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40075760 0x555f400840b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.667+0000 7f4fc6000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40075760 0x555f400840b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53180/0 (socket says 172.21.15.123:53180) 2024-09-19T04:56:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.667+0000 7f4fc7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40074e40 0x555f40083b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.667+0000 7f4fc6000640 1 -- 172.21.15.123:0/3460632374 learned_addr learned my addr 172.21.15.123:0/3460632374 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.667+0000 7f4fc7400640 1 --2- 172.21.15.123:0/3460632374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40080020 0x555f400845f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.667+0000 7f4fc6000640 1 -- 172.21.15.123:0/3460632374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40074e40 msgr2=0x555f40083b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.667+0000 7f4fc6000640 1 --2- 172.21.15.123:0/3460632374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40074e40 0x555f40083b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.667+0000 7f4fc6000640 1 -- 172.21.15.123:0/3460632374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40080020 msgr2=0x555f400845f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:57.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.667+0000 7f4fc6000640 1 --2- 172.21.15.123:0/3460632374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40080020 0x555f400845f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.667+0000 7f4fc6000640 1 -- 172.21.15.123:0/3460632374 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f400f9c00 con 0x555f40075760 2024-09-19T04:56:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.668+0000 7f4fc6000640 1 --2- 172.21.15.123:0/3460632374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40075760 0x555f400840b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4012710 tx=0x7f4fb4005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16826b363e9039ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.668+0000 7f4fb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:56:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.668+0000 7f4fb0c00640 1 -- 172.21.15.123:0/3460632374 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4025500 con 0x555f40075760 2024-09-19T04:56:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.668+0000 7f4fb0c00640 1 -- 172.21.15.123:0/3460632374 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f4fb400f9a0 con 0x555f40075760 2024-09-19T04:56:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.668+0000 7f4fd043b740 1 -- 172.21.15.123:0/3460632374 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb40256a0 con 0x555f40075760 2024-09-19T04:56:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.668+0000 7f4fb0c00640 1 -- 172.21.15.123:0/3460632374 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4fb4025980 con 0x555f40075760 2024-09-19T04:56:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.668+0000 7f4fd043b740 10 client.27649 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.668+0000 7f4fd043b740 1 -- 172.21.15.123:0/3460632374 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555f400faeb0 con 0x555f40075760 2024-09-19T04:56:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.669+0000 7f4fb0c00640 1 -- 172.21.15.123:0/3460632374 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb40256a0 con 0x555f40075760 2024-09-19T04:56:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.669+0000 7f4fb0c00640 1 -- 172.21.15.123:0/3460632374 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f4fb401c590 con 0x555f40075760 2024-09-19T04:56:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.669+0000 7f4fd043b740 10 client.27649 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.669+0000 7f4fd043b740 10 client.27649 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:56:57.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.669+0000 7f4fd043b740 20 client.27649 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:57.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.669+0000 7f4fd043b740 1 --2- 172.21.15.123:0/3460632374 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555f400fb4e0 0x555f400fd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.669+0000 7f4fd043b740 4 client.27649 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:56:57.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.669+0000 7f4fd043b740 1 -- 172.21.15.123:0/3460632374 --> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f3ffe2e60 con 0x555f400fb4e0 2024-09-19T04:56:57.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.669+0000 7f4fc7400640 1 --2- 172.21.15.123:0/3460632374 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555f400fb4e0 0x555f400fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.670+0000 7f4fc7400640 1 --2- 172.21.15.123:0/3460632374 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555f400fb4e0 0x555f400fd900 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a84e92df5be14b4a server_cookie=3ac8d945c53ee33 in_seq=0 out_seq=0 2024-09-19T04:56:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.670+0000 7f4fb0c00640 10 client.27649 ms_handle_connect on v2:172.21.15.193:6838/4148250455 2024-09-19T04:56:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.670+0000 7f4fb0c00640 1 -- 172.21.15.123:0/3460632374 <== mds.1 v2:172.21.15.193:6838/4148250455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f3ffe2e60 con 0x555f400fb4e0 2024-09-19T04:56:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.670+0000 7f4fb0c00640 10 client.27649 handle_command_reply: tid=0 2024-09-19T04:56:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.670+0000 7f4fd043b740 1 client.27649 shutdown 2024-09-19T04:56:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.670+0000 7f4fd043b740 1 -- 172.21.15.123:0/3460632374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40075760 msgr2=0x555f400840b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.670+0000 7f4fd043b740 1 --2- 172.21.15.123:0/3460632374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40075760 0x555f400840b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4012710 tx=0x7f4fb4005f10 comp rx=0 tx=0).stop 2024-09-19T04:56:57.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.670+0000 7f4fd043b740 1 -- 172.21.15.123:0/3460632374 shutdown_connections 2024-09-19T04:56:57.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.670+0000 7f4fd043b740 1 --2- 172.21.15.123:0/3460632374 >> [v2:172.21.15.193:6838/4148250455,v1:172.21.15.193:6839/4148250455] conn(0x555f400fb4e0 0x555f400fd900 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.670+0000 7f4fd043b740 1 --2- 172.21.15.123:0/3460632374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f40080020 0x555f400845f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.670+0000 7f4fd043b740 1 --2- 172.21.15.123:0/3460632374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f40075760 0x555f400840b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.670+0000 7f4fb0c00640 0 client.27649 ms_handle_reset on v2:172.21.15.193:6838/4148250455 2024-09-19T04:56:57.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.670+0000 7f4fd043b740 1 --2- 172.21.15.123:0/3460632374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f40074e40 0x555f40083b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.670+0000 7f4fd043b740 1 -- 172.21.15.123:0/3460632374 >> 172.21.15.123:0/3460632374 conn(0x555f400713d0 msgr2=0x555f4007ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:57.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.670+0000 7f4fd043b740 1 -- 172.21.15.123:0/3460632374 shutdown_connections 2024-09-19T04:56:57.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.671+0000 7f4fd043b740 1 -- 172.21.15.123:0/3460632374 wait complete. 2024-09-19T04:56:57.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.671+0000 7f4fd043b740 20 client.27649 trim_cache size 0 max 16384 2024-09-19T04:56:57.676 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:56:57.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.671+0000 7f4fc7e00640 1 -- 172.21.15.123:0/1870024817 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f4fac0a3430 msgr2=0x7f4fac0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:57.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.671+0000 7f4fc7e00640 1 --2- 172.21.15.123:0/1870024817 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f4fac0a3430 0x7f4fac0a58f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x555f40072230 tx=0x7f4fb80d3ac0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.671+0000 7f4fc7e00640 1 -- 172.21.15.123:0/1870024817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc8103e30 msgr2=0x7f4fc819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:57.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.671+0000 7f4fc7e00640 1 --2- 172.21.15.123:0/1870024817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc8103e30 0x7f4fc819a400 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800e9f0 tx=0x7f4fb800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.671+0000 7f4fc7e00640 1 -- 172.21.15.123:0/1870024817 shutdown_connections 2024-09-19T04:56:57.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.671+0000 7f4fc7e00640 1 --2- 172.21.15.123:0/1870024817 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f4fac0a3430 0x7f4fac0a58f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.671+0000 7f4fc7e00640 1 --2- 172.21.15.123:0/1870024817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fc8108f60 0x7f4fc819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.671+0000 7f4fc7e00640 1 --2- 172.21.15.123:0/1870024817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc81047e0 0x7f4fc819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.671+0000 7f4fc7e00640 1 --2- 172.21.15.123:0/1870024817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc8103e30 0x7f4fc819a400 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.671+0000 7f4fc7e00640 1 -- 172.21.15.123:0/1870024817 >> 172.21.15.123:0/1870024817 conn(0x7f4fc80ff8d0 msgr2=0x7f4fc8100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:57.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.671+0000 7f4fc7e00640 1 -- 172.21.15.123:0/1870024817 shutdown_connections 2024-09-19T04:56:57.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:57.672+0000 7f4fc7e00640 1 -- 172.21.15.123:0/1870024817 wait complete. 2024-09-19T04:56:57.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:57 smithi183 ceph-mon[35333]: pgmap v318: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:56:58.460 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:56:58.829 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:59.485 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:56:59 smithi123 ceph-mon[30511]: pgmap v319: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-19T04:56:59.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.454+0000 7f65bbe00640 1 -- 172.21.15.123:0/769623832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65bc108f60 msgr2=0x7f65bc110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:59.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.454+0000 7f65bbe00640 1 --2- 172.21.15.123:0/769623832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65bc108f60 0x7f65bc110ae0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009ca0 tx=0x7f65b402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.454+0000 7f65bbe00640 1 -- 172.21.15.123:0/769623832 shutdown_connections 2024-09-19T04:56:59.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.454+0000 7f65bbe00640 1 --2- 172.21.15.123:0/769623832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65bc108f60 0x7f65bc110ae0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.454+0000 7f65bbe00640 1 --2- 172.21.15.123:0/769623832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65bc1047e0 0x7f65bc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.454+0000 7f65bbe00640 1 --2- 172.21.15.123:0/769623832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65bc103e30 0x7f65bc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.454+0000 7f65bbe00640 1 -- 172.21.15.123:0/769623832 >> 172.21.15.123:0/769623832 conn(0x7f65bc0ff910 msgr2=0x7f65bc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:59.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.454+0000 7f65bbe00640 1 -- 172.21.15.123:0/769623832 shutdown_connections 2024-09-19T04:56:59.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.454+0000 7f65bbe00640 1 -- 172.21.15.123:0/769623832 wait complete. 2024-09-19T04:56:59.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.455+0000 7f65bbe00640 1 Processor -- start 2024-09-19T04:56:59.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.455+0000 7f65bbe00640 1 -- start start 2024-09-19T04:56:59.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.456+0000 7f65bbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65bc103e30 0x7f65bc19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.456+0000 7f65bbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65bc1047e0 0x7f65bc19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.456+0000 7f65bbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65bc108f60 0x7f65bc19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.456+0000 7f65bbe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc114010 con 0x7f65bc1047e0 2024-09-19T04:56:59.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.456+0000 7f65bbe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc113e90 con 0x7f65bc108f60 2024-09-19T04:56:59.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.456+0000 7f65bbe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc114190 con 0x7f65bc103e30 2024-09-19T04:56:59.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.456+0000 7f65ba000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65bc1047e0 0x7f65bc19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.456+0000 7f65bb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65bc108f60 0x7f65bc19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.456+0000 7f65ba000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65bc1047e0 0x7f65bc19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47558/0 (socket says 172.21.15.123:47558) 2024-09-19T04:56:59.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.456+0000 7f65ba000640 1 -- 172.21.15.123:0/2128763824 learned_addr learned my addr 172.21.15.123:0/2128763824 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:59.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.456+0000 7f65baa00640 1 --2- 172.21.15.123:0/2128763824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65bc103e30 0x7f65bc19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.457+0000 7f65ba000640 1 -- 172.21.15.123:0/2128763824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65bc103e30 msgr2=0x7f65bc19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:59.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.457+0000 7f65ba000640 1 --2- 172.21.15.123:0/2128763824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65bc103e30 0x7f65bc19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.457+0000 7f65ba000640 1 -- 172.21.15.123:0/2128763824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65bc108f60 msgr2=0x7f65bc19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:59.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.457+0000 7f65ba000640 1 --2- 172.21.15.123:0/2128763824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65bc108f60 0x7f65bc19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.457+0000 7f65ba000640 1 -- 172.21.15.123:0/2128763824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65bc19f4a0 con 0x7f65bc1047e0 2024-09-19T04:56:59.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.457+0000 7f65ba000640 1 --2- 172.21.15.123:0/2128763824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65bc1047e0 0x7f65bc19aa30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f65a800e9f0 tx=0x7f65a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:59.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.457+0000 7f65b3400640 1 -- 172.21.15.123:0/2128763824 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a800cd80 con 0x7f65bc1047e0 2024-09-19T04:56:59.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.457+0000 7f65bbe00640 1 -- 172.21.15.123:0/2128763824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65bc19f790 con 0x7f65bc1047e0 2024-09-19T04:56:59.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.457+0000 7f65b3400640 1 -- 172.21.15.123:0/2128763824 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65a8010910 con 0x7f65bc1047e0 2024-09-19T04:56:59.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.457+0000 7f65bbe00640 1 -- 172.21.15.123:0/2128763824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65bc1a7070 con 0x7f65bc1047e0 2024-09-19T04:56:59.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.457+0000 7f65b3400640 1 -- 172.21.15.123:0/2128763824 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8018910 con 0x7f65bc1047e0 2024-09-19T04:56:59.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.461+0000 7f65c43ec740 1 Processor -- start 2024-09-19T04:56:59.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.462+0000 7f65c43ec740 1 -- start start 2024-09-19T04:56:59.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.462+0000 7f65b3400640 1 -- 172.21.15.123:0/2128763824 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f65a8010ac0 con 0x7f65bc1047e0 2024-09-19T04:56:59.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.462+0000 7f65c43ec740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 0x55919a31f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.462+0000 7f65c43ec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31f760 0x55919a329ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.462+0000 7f65c43ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a32a020 0x55919a32c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.462+0000 7f65c43ec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55919a2853e0 con 0x55919a31f760 2024-09-19T04:56:59.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.462+0000 7f65c43ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55919a22cd80 con 0x55919a32a020 2024-09-19T04:56:59.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.462+0000 7f65c43ec740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55919a2813a0 con 0x55919a31ee40 2024-09-19T04:56:59.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.462+0000 7f65baa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31f760 0x55919a329ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.462+0000 7f65baa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31f760 0x55919a329ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47574/0 (socket says 172.21.15.123:47574) 2024-09-19T04:56:59.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.462+0000 7f65baa00640 1 -- 172.21.15.123:0/1826450303 learned_addr learned my addr 172.21.15.123:0/1826450303 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:59.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.462+0000 7f65bb400640 1 --2- 172.21.15.123:0/1826450303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 0x55919a31f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.462+0000 7f65ba000640 1 --2- 172.21.15.123:0/1826450303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a32a020 0x55919a32c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.463+0000 7f65bb400640 1 -- 172.21.15.123:0/1826450303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a32a020 msgr2=0x55919a32c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:59.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.463+0000 7f65bb400640 1 --2- 172.21.15.123:0/1826450303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a32a020 0x55919a32c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.463+0000 7f65bb400640 1 -- 172.21.15.123:0/1826450303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31f760 msgr2=0x55919a329ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:59.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.463+0000 7f65bb400640 1 --2- 172.21.15.123:0/1826450303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31f760 0x55919a329ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.463+0000 7f65bb400640 1 -- 172.21.15.123:0/1826450303 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55919a24bae0 con 0x55919a31ee40 2024-09-19T04:56:59.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.463+0000 7f65ba000640 1 --2- 172.21.15.123:0/1826450303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a32a020 0x55919a32c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:56:59.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.463+0000 7f65bb400640 1 --2- 172.21.15.123:0/1826450303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 0x55919a31f220 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f65b4005e30 tx=0x7f65b40093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4052c536f30f49b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:59.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.463+0000 7f65a7e00640 1 -- 172.21.15.123:0/1826450303 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4031db0 con 0x55919a31ee40 2024-09-19T04:56:59.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.463+0000 7f65a7e00640 1 -- 172.21.15.123:0/1826450303 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b4036030 con 0x55919a31ee40 2024-09-19T04:56:59.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.464+0000 7f65a7e00640 1 -- 172.21.15.123:0/1826450303 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4004240 con 0x55919a31ee40 2024-09-19T04:56:59.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.464+0000 7f65b3400640 1 --2- 172.21.15.123:0/2128763824 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f65a00a33b0 0x7f65a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.464+0000 7f65b3400640 1 -- 172.21.15.123:0/2128763824 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f65a80d0000 con 0x7f65bc1047e0 2024-09-19T04:56:59.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.464+0000 7f65baa00640 1 --2- 172.21.15.123:0/2128763824 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f65a00a33b0 0x7f65a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.464+0000 7f65c43ec740 1 -- 172.21.15.123:0/1826450303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 msgr2=0x55919a31f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:59.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.464+0000 7f65c43ec740 1 --2- 172.21.15.123:0/1826450303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 0x55919a31f220 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f65b4005e30 tx=0x7f65b40093e0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.464+0000 7f65c43ec740 1 -- 172.21.15.123:0/1826450303 shutdown_connections 2024-09-19T04:56:59.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.464+0000 7f65c43ec740 1 --2- 172.21.15.123:0/1826450303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a32a020 0x55919a32c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.464+0000 7f65c43ec740 1 --2- 172.21.15.123:0/1826450303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31f760 0x55919a329ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.464+0000 7f65c43ec740 1 --2- 172.21.15.123:0/1826450303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 0x55919a31f220 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.464+0000 7f65c43ec740 1 -- 172.21.15.123:0/1826450303 >> 172.21.15.123:0/1826450303 conn(0x55919a31b3d0 msgr2=0x55919a31b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:59.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.465+0000 7f65c43ec740 1 -- 172.21.15.123:0/1826450303 shutdown_connections 2024-09-19T04:56:59.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.465+0000 7f65baa00640 1 --2- 172.21.15.123:0/2128763824 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f65a00a33b0 0x7f65a00a5870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55919a31c230 tx=0x7f65ac005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:59.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.465+0000 7f65c43ec740 1 -- 172.21.15.123:0/1826450303 wait complete. 2024-09-19T04:56:59.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.465+0000 7f65c43ec740 1 -- start start 2024-09-19T04:56:59.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.465+0000 7f65c43ec740 1 Processor -- start 2024-09-19T04:56:59.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65c43ec740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 0x55919a326f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65c43ec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31f760 0x55919a327440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65c43ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a32a020 0x55919a327980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65c43ec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55919a32ca70 con 0x55919a31f760 2024-09-19T04:56:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65c43ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55919a2400f0 con 0x55919a32a020 2024-09-19T04:56:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65c43ec740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55919a32cbf0 con 0x55919a31ee40 2024-09-19T04:56:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65c43ec740 10 client.0 _ll_register_callbacks cb 0x55919a227bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65baa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 0x55919a326f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65ba000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31f760 0x55919a327440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65baa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 0x55919a326f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53214/0 (socket says 172.21.15.123:53214) 2024-09-19T04:56:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65baa00640 1 -- 172.21.15.123:0/4114419204 learned_addr learned my addr 172.21.15.123:0/4114419204 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65bb400640 1 --2- 172.21.15.123:0/4114419204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a32a020 0x55919a327980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65baa00640 1 -- 172.21.15.123:0/4114419204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a32a020 msgr2=0x55919a327980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65baa00640 1 --2- 172.21.15.123:0/4114419204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a32a020 0x55919a327980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65baa00640 1 -- 172.21.15.123:0/4114419204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31f760 msgr2=0x55919a327440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:59.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65baa00640 1 --2- 172.21.15.123:0/4114419204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31f760 0x55919a327440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65baa00640 1 -- 172.21.15.123:0/4114419204 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559199ff63c0 con 0x55919a31ee40 2024-09-19T04:56:59.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.466+0000 7f65ba000640 1 --2- 172.21.15.123:0/4114419204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31f760 0x55919a327440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:59.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.467+0000 7f65baa00640 1 --2- 172.21.15.123:0/4114419204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 0x55919a326f00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f65ac01ea30 tx=0x7f65ac004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81c80b026177853e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:59.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.467+0000 7f65a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-19T04:56:59.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.467+0000 7f65a4c00640 1 -- 172.21.15.123:0/4114419204 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac02e060 con 0x55919a31ee40 2024-09-19T04:56:59.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.467+0000 7f65c43ec740 1 -- 172.21.15.123:0/4114419204 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b4036030 con 0x55919a31ee40 2024-09-19T04:56:59.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.467+0000 7f65a4c00640 1 -- 172.21.15.123:0/4114419204 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f65ac018030 con 0x55919a31ee40 2024-09-19T04:56:59.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.467+0000 7f65a4c00640 1 -- 172.21.15.123:0/4114419204 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f65ac029420 con 0x55919a31ee40 2024-09-19T04:56:59.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.467+0000 7f65c43ec740 10 client.27661 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:59.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.467+0000 7f65c43ec740 1 -- 172.21.15.123:0/4114419204 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55919a338b60 con 0x55919a31ee40 2024-09-19T04:56:59.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.467+0000 7f65a4c00640 1 -- 172.21.15.123:0/4114419204 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b4036030 con 0x55919a31ee40 2024-09-19T04:56:59.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.468+0000 7f65a4c00640 1 -- 172.21.15.123:0/4114419204 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f65ac031610 con 0x55919a31ee40 2024-09-19T04:56:59.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.468+0000 7f65c43ec740 10 client.27661 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:59.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.468+0000 7f65c43ec740 10 client.27661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:56:59.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.468+0000 7f65c43ec740 20 client.27661 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:59.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.468+0000 7f65c43ec740 1 --2- 172.21.15.123:0/4114419204 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55919a339f70 0x55919a33c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.468+0000 7f65c43ec740 4 client.27661 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:56:59.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.468+0000 7f65c43ec740 1 -- 172.21.15.123:0/4114419204 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55919a292950 con 0x55919a339f70 2024-09-19T04:56:59.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.468+0000 7f65ba000640 1 --2- 172.21.15.123:0/4114419204 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55919a339f70 0x55919a33c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.469+0000 7f65ba000640 1 --2- 172.21.15.123:0/4114419204 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55919a339f70 0x55919a33c3c0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=652f55bc788fee06 server_cookie=6bced6e701c40a9a in_seq=0 out_seq=0 2024-09-19T04:56:59.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.469+0000 7f65a4c00640 10 client.27661 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:59.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.470+0000 7f65a4c00640 1 -- 172.21.15.123:0/4114419204 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55919a292950 con 0x55919a339f70 2024-09-19T04:56:59.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.470+0000 7f65a4c00640 10 client.27661 handle_command_reply: tid=0 2024-09-19T04:56:59.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.470+0000 7f65c43ec740 1 client.27661 shutdown 2024-09-19T04:56:59.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.470+0000 7f65c43ec740 1 -- 172.21.15.123:0/4114419204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 msgr2=0x55919a326f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:59.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.470+0000 7f65c43ec740 1 --2- 172.21.15.123:0/4114419204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 0x55919a326f00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f65ac01ea30 tx=0x7f65ac004730 comp rx=0 tx=0).stop 2024-09-19T04:56:59.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.470+0000 7f65c43ec740 1 -- 172.21.15.123:0/4114419204 shutdown_connections 2024-09-19T04:56:59.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.470+0000 7f65c43ec740 1 --2- 172.21.15.123:0/4114419204 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55919a339f70 0x55919a33c3c0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.470+0000 7f65c43ec740 1 --2- 172.21.15.123:0/4114419204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a32a020 0x55919a327980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.470+0000 7f65c43ec740 1 --2- 172.21.15.123:0/4114419204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31f760 0x55919a327440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.470+0000 7f65a4c00640 0 client.27661 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:59.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.470+0000 7f65c43ec740 1 --2- 172.21.15.123:0/4114419204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 0x55919a326f00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.470+0000 7f65c43ec740 1 -- 172.21.15.123:0/4114419204 >> 172.21.15.123:0/4114419204 conn(0x55919a31b3d0 msgr2=0x55919a321100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:59.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.470+0000 7f65c43ec740 1 -- 172.21.15.123:0/4114419204 shutdown_connections 2024-09-19T04:56:59.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.470+0000 7f65c43ec740 1 -- 172.21.15.123:0/4114419204 wait complete. 2024-09-19T04:56:59.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.470+0000 7f65c43ec740 20 client.27661 trim_cache size 0 max 16384 2024-09-19T04:56:59.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.484+0000 7f65c43ec740 1 Processor -- start 2024-09-19T04:56:59.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.484+0000 7f65c43ec740 1 -- start start 2024-09-19T04:56:59.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.484+0000 7f65c43ec740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 0x55919a3267e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.484+0000 7f65c43ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a31f760 0x55919a327990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.484+0000 7f65c43ec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a32a020 0x55919a341c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.484+0000 7f65c43ec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1af8b0 con 0x55919a32a020 2024-09-19T04:56:59.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.484+0000 7f65c43ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1b13f0 con 0x55919a31f760 2024-09-19T04:56:59.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.484+0000 7f65c43ec740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1aff80 con 0x55919a31ee40 2024-09-19T04:56:59.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.484+0000 7f65bb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a32a020 0x55919a341c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.484+0000 7f65bb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a32a020 0x55919a341c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47594/0 (socket says 172.21.15.123:47594) 2024-09-19T04:56:59.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.484+0000 7f65bb400640 1 -- 172.21.15.123:0/2294326834 learned_addr learned my addr 172.21.15.123:0/2294326834 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:59.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.484+0000 7f65ba000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a31f760 0x55919a327990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.484+0000 7f65baa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 0x55919a3267e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.485+0000 7f65ba000640 1 -- 172.21.15.123:0/2294326834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 msgr2=0x55919a3267e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:59.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.485+0000 7f65ba000640 1 --2- 172.21.15.123:0/2294326834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 0x55919a3267e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.485+0000 7f65ba000640 1 -- 172.21.15.123:0/2294326834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a32a020 msgr2=0x55919a341c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:59.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.485+0000 7f65ba000640 1 --2- 172.21.15.123:0/2294326834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a32a020 0x55919a341c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.485+0000 7f65ba000640 1 -- 172.21.15.123:0/2294326834 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55919a31c480 con 0x55919a31f760 2024-09-19T04:56:59.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.485+0000 7f65ba000640 1 --2- 172.21.15.123:0/2294326834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a31f760 0x55919a327990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f65a80cd750 tx=0x7f65a80d2510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58a16921b7762b57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:59.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.485+0000 7f65a7400640 1 -- 172.21.15.123:0/2294326834 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a80a4f50 con 0x55919a31f760 2024-09-19T04:56:59.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.485+0000 7f65a7400640 1 -- 172.21.15.123:0/2294326834 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65a809a690 con 0x55919a31f760 2024-09-19T04:56:59.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.485+0000 7f65a7400640 1 -- 172.21.15.123:0/2294326834 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a80bc310 con 0x55919a31f760 2024-09-19T04:56:59.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.486+0000 7f65c43ec740 1 -- 172.21.15.123:0/2294326834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a31f760 msgr2=0x55919a327990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:59.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.486+0000 7f65c43ec740 1 --2- 172.21.15.123:0/2294326834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a31f760 0x55919a327990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f65a80cd750 tx=0x7f65a80d2510 comp rx=0 tx=0).stop 2024-09-19T04:56:59.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.486+0000 7f65c43ec740 1 -- 172.21.15.123:0/2294326834 shutdown_connections 2024-09-19T04:56:59.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.486+0000 7f65c43ec740 1 --2- 172.21.15.123:0/2294326834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a32a020 0x55919a341c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.486+0000 7f65c43ec740 1 --2- 172.21.15.123:0/2294326834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a31f760 0x55919a327990 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.486+0000 7f65c43ec740 1 --2- 172.21.15.123:0/2294326834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a31ee40 0x55919a3267e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.486+0000 7f65c43ec740 1 -- 172.21.15.123:0/2294326834 >> 172.21.15.123:0/2294326834 conn(0x55919a31b3d0 msgr2=0x55919a3354e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:59.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.486+0000 7f65c43ec740 1 -- 172.21.15.123:0/2294326834 shutdown_connections 2024-09-19T04:56:59.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.486+0000 7f65c43ec740 1 -- 172.21.15.123:0/2294326834 wait complete. 2024-09-19T04:56:59.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.487+0000 7f65c43ec740 1 -- start start 2024-09-19T04:56:59.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.487+0000 7f65c43ec740 1 Processor -- start 2024-09-19T04:56:59.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.488+0000 7f65c43ec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31ee40 0x55919a32dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.488+0000 7f65c43ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a31f760 0x55919a32e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.488+0000 7f65c43ec740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a32a020 0x55919a32e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.488+0000 7f65c43ec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1b0d20 con 0x55919a31ee40 2024-09-19T04:56:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.488+0000 7f65c43ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1b0650 con 0x55919a31f760 2024-09-19T04:56:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.488+0000 7f65c43ec740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1ad8b0 con 0x55919a32a020 2024-09-19T04:56:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.488+0000 7f65c43ec740 10 client.0 _ll_register_callbacks cb 0x7f65ac012eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:56:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.488+0000 7f65ba000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a31f760 0x55919a32e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.488+0000 7f65baa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31ee40 0x55919a32dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.488+0000 7f65ba000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a31f760 0x55919a32e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:45692/0 (socket says 172.21.15.123:45692) 2024-09-19T04:56:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.488+0000 7f65baa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31ee40 0x55919a32dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47610/0 (socket says 172.21.15.123:47610) 2024-09-19T04:56:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.488+0000 7f65ba000640 1 -- 172.21.15.123:0/769796758 learned_addr learned my addr 172.21.15.123:0/769796758 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:56:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.488+0000 7f65bb400640 1 --2- 172.21.15.123:0/769796758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a32a020 0x55919a32e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.489+0000 7f65ba000640 1 -- 172.21.15.123:0/769796758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a32a020 msgr2=0x55919a32e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:59.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.489+0000 7f65ba000640 1 --2- 172.21.15.123:0/769796758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a32a020 0x55919a32e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.489+0000 7f65ba000640 1 -- 172.21.15.123:0/769796758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31ee40 msgr2=0x55919a32dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:56:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.489+0000 7f65ba000640 1 --2- 172.21.15.123:0/769796758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31ee40 0x55919a32dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.489+0000 7f65ba000640 1 -- 172.21.15.123:0/769796758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55919a31c060 con 0x55919a31f760 2024-09-19T04:56:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.489+0000 7f65ba000640 1 --2- 172.21.15.123:0/769796758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a31f760 0x55919a32e2e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f65a80987f0 tx=0x7f65a80056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2be2da2d4538c852 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.489+0000 7f65a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-19T04:56:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.489+0000 7f65a4c00640 1 -- 172.21.15.123:0/769796758 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a80a4cc0 con 0x55919a31f760 2024-09-19T04:56:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.489+0000 7f65a4c00640 1 -- 172.21.15.123:0/769796758 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f65a80b0290 con 0x55919a31f760 2024-09-19T04:56:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.489+0000 7f65c43ec740 1 -- 172.21.15.123:0/769796758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65a809a690 con 0x55919a31f760 2024-09-19T04:56:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.489+0000 7f65a4c00640 1 -- 172.21.15.123:0/769796758 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f65a80b0490 con 0x55919a31f760 2024-09-19T04:56:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.489+0000 7f65c43ec740 10 client.27656 fetch_fsmap learned FSMap version 43 2024-09-19T04:56:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.489+0000 7f65c43ec740 1 -- 172.21.15.123:0/769796758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55919a3a4f40 con 0x55919a31f760 2024-09-19T04:56:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.490+0000 7f65a4c00640 1 -- 172.21.15.123:0/769796758 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65a809a690 con 0x55919a31f760 2024-09-19T04:56:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.490+0000 7f65a4c00640 1 -- 172.21.15.123:0/769796758 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f65a80a0270 con 0x55919a31f760 2024-09-19T04:56:59.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.490+0000 7f65c43ec740 10 client.27656 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:56:59.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.490+0000 7f65c43ec740 10 client.27656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:56:59.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.490+0000 7f65c43ec740 20 client.27656 populate_metadata read hostname 'smithi123' 2024-09-19T04:56:59.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.490+0000 7f65c43ec740 1 --2- 172.21.15.123:0/769796758 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55919a3a54d0 0x55919a3a78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:59.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.490+0000 7f65c43ec740 4 client.27656 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:56:59.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.490+0000 7f65c43ec740 1 -- 172.21.15.123:0/769796758 --> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55919a28ce60 con 0x55919a3a54d0 2024-09-19T04:56:59.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.490+0000 7f65baa00640 1 --2- 172.21.15.123:0/769796758 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55919a3a54d0 0x55919a3a78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:59.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.491+0000 7f65baa00640 1 --2- 172.21.15.123:0/769796758 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55919a3a54d0 0x55919a3a78f0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7104c95e1e6ec9c server_cookie=dd19626266b24b41 in_seq=0 out_seq=0 2024-09-19T04:56:59.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.491+0000 7f65a4c00640 10 client.27656 ms_handle_connect on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:59.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.491+0000 7f65a4c00640 1 -- 172.21.15.123:0/769796758 <== mds.0 v2:172.21.15.183:6838/157697996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55919a28ce60 con 0x55919a3a54d0 2024-09-19T04:56:59.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.491+0000 7f65a4c00640 10 client.27656 handle_command_reply: tid=0 2024-09-19T04:56:59.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.491+0000 7f65c43ec740 1 client.27656 shutdown 2024-09-19T04:56:59.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.491+0000 7f65c43ec740 1 -- 172.21.15.123:0/769796758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a31f760 msgr2=0x55919a32e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:59.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.491+0000 7f65c43ec740 1 --2- 172.21.15.123:0/769796758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a31f760 0x55919a32e2e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f65a80987f0 tx=0x7f65a80056d0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.491+0000 7f65c43ec740 1 -- 172.21.15.123:0/769796758 shutdown_connections 2024-09-19T04:56:59.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.491+0000 7f65c43ec740 1 --2- 172.21.15.123:0/769796758 >> [v2:172.21.15.183:6838/157697996,v1:172.21.15.183:6839/157697996] conn(0x55919a3a54d0 0x55919a3a78f0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.491+0000 7f65c43ec740 1 --2- 172.21.15.123:0/769796758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55919a32a020 0x55919a32e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.491+0000 7f65c43ec740 1 --2- 172.21.15.123:0/769796758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55919a31f760 0x55919a32e2e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.491+0000 7f65c43ec740 1 --2- 172.21.15.123:0/769796758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55919a31ee40 0x55919a32dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.491+0000 7f65a4c00640 0 client.27656 ms_handle_reset on v2:172.21.15.183:6838/157697996 2024-09-19T04:56:59.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.491+0000 7f65c43ec740 1 -- 172.21.15.123:0/769796758 >> 172.21.15.123:0/769796758 conn(0x55919a31b3d0 msgr2=0x55919a3354e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:59.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.491+0000 7f65c43ec740 1 -- 172.21.15.123:0/769796758 shutdown_connections 2024-09-19T04:56:59.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.491+0000 7f65c43ec740 1 -- 172.21.15.123:0/769796758 wait complete. 2024-09-19T04:56:59.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.492+0000 7f65c43ec740 20 client.27656 trim_cache size 0 max 16384 2024-09-19T04:56:59.509 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-09-19T04:56:59.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.492+0000 7f65bbe00640 1 -- 172.21.15.123:0/2128763824 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f65a00a33b0 msgr2=0x7f65a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:59.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.492+0000 7f65bbe00640 1 --2- 172.21.15.123:0/2128763824 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f65a00a33b0 0x7f65a00a5870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55919a31c230 tx=0x7f65ac005f90 comp rx=0 tx=0).stop 2024-09-19T04:56:59.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.492+0000 7f65bbe00640 1 -- 172.21.15.123:0/2128763824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65bc1047e0 msgr2=0x7f65bc19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:59.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.492+0000 7f65bbe00640 1 --2- 172.21.15.123:0/2128763824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65bc1047e0 0x7f65bc19aa30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f65a800e9f0 tx=0x7f65a800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.492+0000 7f65bbe00640 1 -- 172.21.15.123:0/2128763824 shutdown_connections 2024-09-19T04:56:59.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.492+0000 7f65bbe00640 1 --2- 172.21.15.123:0/2128763824 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f65a00a33b0 0x7f65a00a5870 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.492+0000 7f65bbe00640 1 --2- 172.21.15.123:0/2128763824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65bc108f60 0x7f65bc19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.492+0000 7f65bbe00640 1 --2- 172.21.15.123:0/2128763824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65bc1047e0 0x7f65bc19aa30 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.492+0000 7f65bbe00640 1 --2- 172.21.15.123:0/2128763824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65bc103e30 0x7f65bc19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:59.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.492+0000 7f65bbe00640 1 -- 172.21.15.123:0/2128763824 >> 172.21.15.123:0/2128763824 conn(0x7f65bc0ff910 msgr2=0x7f65bc101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:59.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.493+0000 7f65bbe00640 1 -- 172.21.15.123:0/2128763824 shutdown_connections 2024-09-19T04:56:59.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:56:59.493+0000 7f65bbe00640 1 -- 172.21.15.123:0/2128763824 wait complete. 2024-09-19T04:56:59.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:56:59 smithi193 ceph-mon[35592]: pgmap v319: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-19T04:56:59.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:56:59 smithi183 ceph-mon[35333]: pgmap v319: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-19T04:57:00.228 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-19T04:57:00.229 INFO:tasks.fwd_scrub:done joining 2024-09-19T04:57:00.229 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-19T04:57:00.239 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:57:00.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.395+0000 7fd454c00640 1 -- 172.21.15.123:0/72592156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd450103f20 msgr2=0x7fd450104300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:00.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.395+0000 7fd454c00640 1 --2- 172.21.15.123:0/72592156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd450103f20 0x7fd450104300 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd440009900 tx=0x7fd44002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.395+0000 7fd454c00640 1 -- 172.21.15.123:0/72592156 shutdown_connections 2024-09-19T04:57:00.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.395+0000 7fd454c00640 1 --2- 172.21.15.123:0/72592156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd450109070 0x7fd450110bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.395+0000 7fd454c00640 1 --2- 172.21.15.123:0/72592156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4501048d0 0x7fd450108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.395+0000 7fd454c00640 1 --2- 172.21.15.123:0/72592156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd450103f20 0x7fd450104300 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.395+0000 7fd454c00640 1 -- 172.21.15.123:0/72592156 >> 172.21.15.123:0/72592156 conn(0x7fd4500ff9b0 msgr2=0x7fd450101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:00.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.395+0000 7fd454c00640 1 -- 172.21.15.123:0/72592156 shutdown_connections 2024-09-19T04:57:00.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.395+0000 7fd454c00640 1 -- 172.21.15.123:0/72592156 wait complete. 2024-09-19T04:57:00.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.396+0000 7fd454c00640 1 Processor -- start 2024-09-19T04:57:00.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.396+0000 7fd454c00640 1 -- start start 2024-09-19T04:57:00.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.396+0000 7fd454c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd450103f20 0x7fd45019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:00.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.396+0000 7fd454c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd450109070 0x7fd45019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:00.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.397+0000 7fd454c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd45019eec0 0x7fd45019f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:00.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.397+0000 7fd454c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501140b0 con 0x7fd450103f20 2024-09-19T04:57:00.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.397+0000 7fd454c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd450113f30 con 0x7fd450109070 2024-09-19T04:57:00.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.397+0000 7fd454c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd450114230 con 0x7fd45019eec0 2024-09-19T04:57:00.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.397+0000 7fd44de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd450103f20 0x7fd45019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:00.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.397+0000 7fd44de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd450103f20 0x7fd45019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47618/0 (socket says 172.21.15.123:47618) 2024-09-19T04:57:00.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.397+0000 7fd44de00640 1 -- 172.21.15.123:0/4057842542 learned_addr learned my addr 172.21.15.123:0/4057842542 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:57:00.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.397+0000 7fd44d400640 1 --2- 172.21.15.123:0/4057842542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd450109070 0x7fd45019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:00.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.397+0000 7fd44e800640 1 --2- 172.21.15.123:0/4057842542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd45019eec0 0x7fd45019f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:00.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.397+0000 7fd44de00640 1 -- 172.21.15.123:0/4057842542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd45019eec0 msgr2=0x7fd45019f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:00.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.397+0000 7fd44de00640 1 --2- 172.21.15.123:0/4057842542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd45019eec0 0x7fd45019f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.397+0000 7fd44de00640 1 -- 172.21.15.123:0/4057842542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd450109070 msgr2=0x7fd45019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:00.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.397+0000 7fd44de00640 1 --2- 172.21.15.123:0/4057842542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd450109070 0x7fd45019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.397+0000 7fd44de00640 1 -- 172.21.15.123:0/4057842542 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45019f9b0 con 0x7fd450103f20 2024-09-19T04:57:00.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.397+0000 7fd44de00640 1 --2- 172.21.15.123:0/4057842542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd450103f20 0x7fd45019a600 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd440006010 tx=0x7fd440004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:00.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.398+0000 7fd446a00640 1 -- 172.21.15.123:0/4057842542 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44003f030 con 0x7fd450103f20 2024-09-19T04:57:00.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.398+0000 7fd446a00640 1 -- 172.21.15.123:0/4057842542 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd440004440 con 0x7fd450103f20 2024-09-19T04:57:00.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.398+0000 7fd454c00640 1 -- 172.21.15.123:0/4057842542 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45019fc40 con 0x7fd450103f20 2024-09-19T04:57:00.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.398+0000 7fd446a00640 1 -- 172.21.15.123:0/4057842542 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440035070 con 0x7fd450103f20 2024-09-19T04:57:00.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.398+0000 7fd454c00640 1 -- 172.21.15.123:0/4057842542 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4501a7400 con 0x7fd450103f20 2024-09-19T04:57:00.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.399+0000 7fd454c00640 1 -- 172.21.15.123:0/4057842542 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd450104300 con 0x7fd450103f20 2024-09-19T04:57:00.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.401+0000 7fd446a00640 1 -- 172.21.15.123:0/4057842542 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd440042030 con 0x7fd450103f20 2024-09-19T04:57:00.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.402+0000 7fd446a00640 1 --2- 172.21.15.123:0/4057842542 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd4340a33c0 0x7fd4340a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:00.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.402+0000 7fd446a00640 1 -- 172.21.15.123:0/4057842542 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fd4400f0070 con 0x7fd450103f20 2024-09-19T04:57:00.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.402+0000 7fd44d400640 1 --2- 172.21.15.123:0/4057842542 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd4340a33c0 0x7fd4340a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:00.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.403+0000 7fd44d400640 1 --2- 172.21.15.123:0/4057842542 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd4340a33c0 0x7fd4340a5880 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd4500ffe60 tx=0x7fd43c005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:00.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.406+0000 7fd446a00640 1 -- 172.21.15.123:0/4057842542 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd4400bc400 con 0x7fd450103f20 2024-09-19T04:57:00.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.582+0000 7fd454c00640 1 -- 172.21.15.123:0/4057842542 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4501048d0 con 0x7fd450103f20 2024-09-19T04:57:00.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.584+0000 7fd446a00640 1 -- 172.21.15.123:0/4057842542 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+23268 (secure 0 0 0) 0x7fd4400c0700 con 0x7fd450103f20 2024-09-19T04:57:00.586 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:57:00.586 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":95,"fsid":"911fa052-7640-11ef-bceb-c7b262605968","created":"2024-09-19T04:36:21.083333+0000","modified":"2024-09-19T04:56:17.134948+0000","last_up_change":"2024-09-19T04:43:33.818439+0000","last_in_change":"2024-09-19T04:43:10.148272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:39:33.856981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:44:26.956483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:44:29.718625+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":94,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6803","nonce":3631596501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6805","nonce":3631596501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6809","nonce":3631596501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6807","nonce":3631596501}]},"public_addr":"172.21.15.123:6803/3631596501","cluster_addr":"172.21.15.123:6805/3631596501","heartbeat_back_addr":"172.21.15.123:6809/3631596501","heartbeat_front_addr":"172.21.15.123:6807/3631596501","state":["exists","up"]},{"osd":1,"uuid":"a526d4ce-586b-4d66-b720-e868df92fa24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6801","nonce":1259721614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6803","nonce":1259721614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6807","nonce":1259721614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6805","nonce":1259721614}]},"public_addr":"172.21.15.183:6801/1259721614","cluster_addr":"172.21.15.183:6803/1259721614","heartbeat_back_addr":"172.21.15.183:6807/1259721614","heartbeat_front_addr":"172.21.15.183:6805/1259721614","state":["exists","up"]},{"osd":2,"uuid":"3650601a-6259-4b2a-97df-1d47b09b0127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6801","nonce":2708151726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6803","nonce":2708151726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6807","nonce":2708151726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6805","nonce":2708151726}]},"public_addr":"172.21.15.193:6801/2708151726","cluster_addr":"172.21.15.193:6803/2708151726","heartbeat_back_addr":"172.21.15.193:6807/2708151726","heartbeat_front_addr":"172.21.15.193:6805/2708151726","state":["exists","up"]},{"osd":3,"uuid":"67c485ad-136e-4020-950d-9171a3c2de35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6811","nonce":3214806282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6813","nonce":3214806282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6817","nonce":3214806282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6815","nonce":3214806282}]},"public_addr":"172.21.15.123:6811/3214806282","cluster_addr":"172.21.15.123:6813/3214806282","heartbeat_back_addr":"172.21.15.123:6817/3214806282","heartbeat_front_addr":"172.21.15.123:6815/3214806282","state":["exists","up"]},{"osd":4,"uuid":"c9098540-06ab-45a8-9cf9-f90519cc852e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6809","nonce":667618665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6811","nonce":667618665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6815","nonce":667618665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6813","nonce":667618665}]},"public_addr":"172.21.15.183:6809/667618665","cluster_addr":"172.21.15.183:6811/667618665","heartbeat_back_addr":"172.21.15.183:6815/667618665","heartbeat_front_addr":"172.21.15.183:6813/667618665","state":["exists","up"]},{"osd":5,"uuid":"278a8dbd-6492-42d4-b489-1f832350bc6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6809","nonce":2292251639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6811","nonce":2292251639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6815","nonce":2292251639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6813","nonce":2292251639}]},"public_addr":"172.21.15.193:6809/2292251639","cluster_addr":"172.21.15.193:6811/2292251639","heartbeat_back_addr":"172.21.15.193:6815/2292251639","heartbeat_front_addr":"172.21.15.193:6813/2292251639","state":["exists","up"]},{"osd":6,"uuid":"0fda4e39-6adb-4941-a39b-a9a15019088c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6819","nonce":226007499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6821","nonce":226007499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6825","nonce":226007499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6823","nonce":226007499}]},"public_addr":"172.21.15.123:6819/226007499","cluster_addr":"172.21.15.123:6821/226007499","heartbeat_back_addr":"172.21.15.123:6825/226007499","heartbeat_front_addr":"172.21.15.123:6823/226007499","state":["exists","up"]},{"osd":7,"uuid":"5c0a021e-9f3b-4766-9686-0ef24b68cf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6817","nonce":3718518226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6819","nonce":3718518226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6823","nonce":3718518226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6821","nonce":3718518226}]},"public_addr":"172.21.15.183:6817/3718518226","cluster_addr":"172.21.15.183:6819/3718518226","heartbeat_back_addr":"172.21.15.183:6823/3718518226","heartbeat_front_addr":"172.21.15.183:6821/3718518226","state":["exists","up"]},{"osd":8,"uuid":"9b0e67e0-6305-4f81-9211-d41960099a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6817","nonce":3551404118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6819","nonce":3551404118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6823","nonce":3551404118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6821","nonce":3551404118}]},"public_addr":"172.21.15.193:6817/3551404118","cluster_addr":"172.21.15.193:6819/3551404118","heartbeat_back_addr":"172.21.15.193:6823/3551404118","heartbeat_front_addr":"172.21.15.193:6821/3551404118","state":["exists","up"]},{"osd":9,"uuid":"978ef3fa-c6ec-421f-b194-56d44813be5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6827","nonce":1439901149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6829","nonce":1439901149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6833","nonce":1439901149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6831","nonce":1439901149}]},"public_addr":"172.21.15.123:6827/1439901149","cluster_addr":"172.21.15.123:6829/1439901149","heartbeat_back_addr":"172.21.15.123:6833/1439901149","heartbeat_front_addr":"172.21.15.123:6831/1439901149","state":["exists","up"]},{"osd":10,"uuid":"5495e09b-7809-47b0-b7e0-b56c452f9f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6825","nonce":3355349245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6827","nonce":3355349245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6831","nonce":3355349245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6829","nonce":3355349245}]},"public_addr":"172.21.15.183:6825/3355349245","cluster_addr":"172.21.15.183:6827/3355349245","heartbeat_back_addr":"172.21.15.183:6831/3355349245","heartbeat_front_addr":"172.21.15.183:6829/3355349245","state":["exists","up"]},{"osd":11,"uuid":"03046fef-b16b-42f8-9a19-6c0b78cbfe7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6825","nonce":4291617038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6827","nonce":4291617038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6831","nonce":4291617038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6829","nonce":4291617038}]},"public_addr":"172.21.15.193:6825/4291617038","cluster_addr":"172.21.15.193:6827/4291617038","heartbeat_back_addr":"172.21.15.193:6831/4291617038","heartbeat_front_addr":"172.21.15.193:6829/4291617038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:38:39.142529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:05.336911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:30.310967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:59.655973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:25.890576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:50.778689+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:19.063731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:45.424269+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:10.562149+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:40.382368+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:06.297731+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:32.352172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/1532622003":"2024-09-20T04:46:41.775112+0000","172.21.15.123:0/2833919666":"2024-09-20T04:46:41.775112+0000","172.21.15.123:0/2580747029":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/471028951":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:6800/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6800/1532622003":"2024-09-20T04:46:41.775112+0000","172.21.15.123:6801/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/3498196650":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/4040401634":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/2000593686":"2024-09-20T04:36:45.913060+0000","172.21.15.123:6800/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/3339335756":"2024-09-20T04:46:41.775112+0000","172.21.15.123:0/1009496020":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/3244738342":"2024-09-20T04:46:41.775112+0000","172.21.15.123:0/2764863023":"2024-09-20T04:46:41.775112+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:57:00.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.586+0000 7fd454c00640 1 -- 172.21.15.123:0/4057842542 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd4340a33c0 msgr2=0x7fd4340a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:00.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.586+0000 7fd454c00640 1 --2- 172.21.15.123:0/4057842542 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd4340a33c0 0x7fd4340a5880 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd4500ffe60 tx=0x7fd43c005f90 comp rx=0 tx=0).stop 2024-09-19T04:57:00.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.586+0000 7fd454c00640 1 -- 172.21.15.123:0/4057842542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd450103f20 msgr2=0x7fd45019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:00.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.586+0000 7fd454c00640 1 --2- 172.21.15.123:0/4057842542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd450103f20 0x7fd45019a600 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd440006010 tx=0x7fd440004040 comp rx=0 tx=0).stop 2024-09-19T04:57:00.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.587+0000 7fd454c00640 1 -- 172.21.15.123:0/4057842542 shutdown_connections 2024-09-19T04:57:00.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.587+0000 7fd454c00640 1 --2- 172.21.15.123:0/4057842542 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fd4340a33c0 0x7fd4340a5880 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.587+0000 7fd454c00640 1 --2- 172.21.15.123:0/4057842542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd45019eec0 0x7fd45019f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.587+0000 7fd454c00640 1 --2- 172.21.15.123:0/4057842542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd450109070 0x7fd45019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.587+0000 7fd454c00640 1 --2- 172.21.15.123:0/4057842542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd450103f20 0x7fd45019a600 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.587+0000 7fd454c00640 1 -- 172.21.15.123:0/4057842542 >> 172.21.15.123:0/4057842542 conn(0x7fd4500ff9b0 msgr2=0x7fd450100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:00.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.587+0000 7fd454c00640 1 -- 172.21.15.123:0/4057842542 shutdown_connections 2024-09-19T04:57:00.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.587+0000 7fd454c00640 1 -- 172.21.15.123:0/4057842542 wait complete. 2024-09-19T04:57:00.605 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:39:33.856981+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T04:44:26.956483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T04:44:29.718625+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 94, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T04:57:00.606 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T04:57:00.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.764+0000 7f9706600640 1 -- 172.21.15.123:0/1535474754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9700109060 msgr2=0x7f9700110bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:00.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.764+0000 7f9706600640 1 --2- 172.21.15.123:0/1535474754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9700109060 0x7f9700110bb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009ca0 tx=0x7f96f802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.764+0000 7f9706600640 1 -- 172.21.15.123:0/1535474754 shutdown_connections 2024-09-19T04:57:00.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.764+0000 7f9706600640 1 --2- 172.21.15.123:0/1535474754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9700109060 0x7f9700110bb0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.764+0000 7f9706600640 1 --2- 172.21.15.123:0/1535474754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97001048c0 0x7f9700108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.764+0000 7f9706600640 1 --2- 172.21.15.123:0/1535474754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9700103f10 0x7f97001042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.764+0000 7f9706600640 1 -- 172.21.15.123:0/1535474754 >> 172.21.15.123:0/1535474754 conn(0x7f97000ff940 msgr2=0x7f9700101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:00.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.764+0000 7f9706600640 1 -- 172.21.15.123:0/1535474754 shutdown_connections 2024-09-19T04:57:00.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.764+0000 7f9706600640 1 -- 172.21.15.123:0/1535474754 wait complete. 2024-09-19T04:57:00.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.765+0000 7f9706600640 1 Processor -- start 2024-09-19T04:57:00.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.765+0000 7f9706600640 1 -- start start 2024-09-19T04:57:00.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.765+0000 7f9706600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9700103f10 0x7f970019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:00.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.765+0000 7f9706600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9700109060 0x7f970019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:00.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.765+0000 7f9706600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f970019eeb0 0x7f970019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:00.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.765+0000 7f9706600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9700114100 con 0x7f970019eeb0 2024-09-19T04:57:00.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.765+0000 7f9706600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9700113f80 con 0x7f9700103f10 2024-09-19T04:57:00.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.765+0000 7f9706600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9700114280 con 0x7f9700109060 2024-09-19T04:57:00.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.765+0000 7f96ff400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9700103f10 0x7f970019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:00.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.766+0000 7f96ff400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9700103f10 0x7f970019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:45748/0 (socket says 172.21.15.123:45748) 2024-09-19T04:57:00.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.766+0000 7f96ff400640 1 -- 172.21.15.123:0/3045263863 learned_addr learned my addr 172.21.15.123:0/3045263863 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:57:00.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.766+0000 7f96ffe00640 1 --2- 172.21.15.123:0/3045263863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f970019eeb0 0x7f970019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:00.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.766+0000 7f96fea00640 1 --2- 172.21.15.123:0/3045263863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9700109060 0x7f970019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:00.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.766+0000 7f96ff400640 1 -- 172.21.15.123:0/3045263863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9700109060 msgr2=0x7f970019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:00.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.766+0000 7f96ff400640 1 --2- 172.21.15.123:0/3045263863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9700109060 0x7f970019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.766+0000 7f96ff400640 1 -- 172.21.15.123:0/3045263863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f970019eeb0 msgr2=0x7f970019f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:00.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.766+0000 7f96ff400640 1 --2- 172.21.15.123:0/3045263863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f970019eeb0 0x7f970019f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.766+0000 7f96ff400640 1 -- 172.21.15.123:0/3045263863 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970019f9a0 con 0x7f9700103f10 2024-09-19T04:57:00.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.766+0000 7f96fea00640 1 --2- 172.21.15.123:0/3045263863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9700109060 0x7f970019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:57:00.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.766+0000 7f96ff400640 1 --2- 172.21.15.123:0/3045263863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9700103f10 0x7f970019a5f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f96f000e9f0 tx=0x7f96f000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:00.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.766+0000 7f96f7e00640 1 -- 172.21.15.123:0/3045263863 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f000cd80 con 0x7f9700103f10 2024-09-19T04:57:00.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.766+0000 7f96f7e00640 1 -- 172.21.15.123:0/3045263863 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f0005500 con 0x7f9700103f10 2024-09-19T04:57:00.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.766+0000 7f9706600640 1 -- 172.21.15.123:0/3045263863 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970019fc90 con 0x7f9700103f10 2024-09-19T04:57:00.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.767+0000 7f96f7e00640 1 -- 172.21.15.123:0/3045263863 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0010690 con 0x7f9700103f10 2024-09-19T04:57:00.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.767+0000 7f9706600640 1 -- 172.21.15.123:0/3045263863 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97001a7430 con 0x7f9700103f10 2024-09-19T04:57:00.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.768+0000 7f9706600640 1 -- 172.21.15.123:0/3045263863 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97001042f0 con 0x7f9700103f10 2024-09-19T04:57:00.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.770+0000 7f96f7e00640 1 -- 172.21.15.123:0/3045263863 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f96f00046c0 con 0x7f9700103f10 2024-09-19T04:57:00.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.771+0000 7f96f7e00640 1 --2- 172.21.15.123:0/3045263863 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f96e40a33c0 0x7f96e40a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:00.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.771+0000 7f96f7e00640 1 -- 172.21.15.123:0/3045263863 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f96f00d0440 con 0x7f9700103f10 2024-09-19T04:57:00.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.771+0000 7f96fea00640 1 --2- 172.21.15.123:0/3045263863 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f96e40a33c0 0x7f96e40a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:00.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.772+0000 7f96fea00640 1 --2- 172.21.15.123:0/3045263863 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f96e40a33c0 0x7f96e40a5880 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9700102fd0 tx=0x7f96ec005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:00.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.773+0000 7f96f7e00640 1 -- 172.21.15.123:0/3045263863 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f96f0054210 con 0x7f9700103f10 2024-09-19T04:57:00.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.949+0000 7f9706600640 1 -- 172.21.15.123:0/3045263863 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f970010c2d0 con 0x7f9700103f10 2024-09-19T04:57:00.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.949+0000 7f96f7e00640 1 -- 172.21.15.123:0/3045263863 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f96f009cdc0 con 0x7f9700103f10 2024-09-19T04:57:00.951 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-09-19T04:57:00.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.952+0000 7f9706600640 1 -- 172.21.15.123:0/3045263863 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f96e40a33c0 msgr2=0x7f96e40a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:00.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.952+0000 7f9706600640 1 --2- 172.21.15.123:0/3045263863 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f96e40a33c0 0x7f96e40a5880 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9700102fd0 tx=0x7f96ec005f90 comp rx=0 tx=0).stop 2024-09-19T04:57:00.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.952+0000 7f9706600640 1 -- 172.21.15.123:0/3045263863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9700103f10 msgr2=0x7f970019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:00.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.952+0000 7f9706600640 1 --2- 172.21.15.123:0/3045263863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9700103f10 0x7f970019a5f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f96f000e9f0 tx=0x7f96f000edf0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.952+0000 7f9706600640 1 -- 172.21.15.123:0/3045263863 shutdown_connections 2024-09-19T04:57:00.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.952+0000 7f9706600640 1 --2- 172.21.15.123:0/3045263863 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f96e40a33c0 0x7f96e40a5880 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.952+0000 7f9706600640 1 --2- 172.21.15.123:0/3045263863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f970019eeb0 0x7f970019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.952+0000 7f9706600640 1 --2- 172.21.15.123:0/3045263863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9700109060 0x7f970019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.952+0000 7f9706600640 1 --2- 172.21.15.123:0/3045263863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9700103f10 0x7f970019a5f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.952+0000 7f9706600640 1 -- 172.21.15.123:0/3045263863 >> 172.21.15.123:0/3045263863 conn(0x7f97000ff940 msgr2=0x7f9700100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:00.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.952+0000 7f9706600640 1 -- 172.21.15.123:0/3045263863 shutdown_connections 2024-09-19T04:57:00.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:00.952+0000 7f9706600640 1 -- 172.21.15.123:0/3045263863 wait complete. 2024-09-19T04:57:00.970 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T04:57:01.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.126+0000 7f6a38a00640 1 -- 172.21.15.123:0/3047149398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a34103f10 msgr2=0x7f6a341042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:01.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.126+0000 7f6a38a00640 1 --2- 172.21.15.123:0/3047149398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a34103f10 0x7f6a341042f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009900 tx=0x7f6a2402a410 comp rx=0 tx=0).stop 2024-09-19T04:57:01.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.126+0000 7f6a38a00640 1 -- 172.21.15.123:0/3047149398 shutdown_connections 2024-09-19T04:57:01.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.126+0000 7f6a38a00640 1 --2- 172.21.15.123:0/3047149398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a34109060 0x7f6a34110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.126+0000 7f6a38a00640 1 --2- 172.21.15.123:0/3047149398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a341048c0 0x7f6a34108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.126+0000 7f6a38a00640 1 --2- 172.21.15.123:0/3047149398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a34103f10 0x7f6a341042f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.126+0000 7f6a38a00640 1 -- 172.21.15.123:0/3047149398 >> 172.21.15.123:0/3047149398 conn(0x7f6a340ff940 msgr2=0x7f6a34101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:01.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.126+0000 7f6a38a00640 1 -- 172.21.15.123:0/3047149398 shutdown_connections 2024-09-19T04:57:01.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.127+0000 7f6a38a00640 1 -- 172.21.15.123:0/3047149398 wait complete. 2024-09-19T04:57:01.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.127+0000 7f6a38a00640 1 Processor -- start 2024-09-19T04:57:01.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.127+0000 7f6a38a00640 1 -- start start 2024-09-19T04:57:01.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.128+0000 7f6a38a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a34103f10 0x7f6a3419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:01.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.128+0000 7f6a38a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a34109060 0x7f6a3419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:01.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.128+0000 7f6a38a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a3419eef0 0x7f6a3419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:01.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.128+0000 7f6a38a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34114100 con 0x7f6a3419eef0 2024-09-19T04:57:01.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.128+0000 7f6a38a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34113f80 con 0x7f6a34103f10 2024-09-19T04:57:01.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.128+0000 7f6a38a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34114280 con 0x7f6a34109060 2024-09-19T04:57:01.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.128+0000 7f6a31400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a34109060 0x7f6a3419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:01.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.128+0000 7f6a32800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a3419eef0 0x7f6a3419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:01.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.128+0000 7f6a31400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a34109060 0x7f6a3419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:53278/0 (socket says 172.21.15.123:53278) 2024-09-19T04:57:01.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.128+0000 7f6a31400640 1 -- 172.21.15.123:0/302754171 learned_addr learned my addr 172.21.15.123:0/302754171 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:57:01.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.128+0000 7f6a31e00640 1 --2- 172.21.15.123:0/302754171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a34103f10 0x7f6a3419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:01.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.128+0000 7f6a31400640 1 -- 172.21.15.123:0/302754171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a34103f10 msgr2=0x7f6a3419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:01.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.128+0000 7f6a31400640 1 --2- 172.21.15.123:0/302754171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a34103f10 0x7f6a3419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.128+0000 7f6a31400640 1 -- 172.21.15.123:0/302754171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a3419eef0 msgr2=0x7f6a3419f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:01.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.128+0000 7f6a31400640 1 --2- 172.21.15.123:0/302754171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a3419eef0 0x7f6a3419f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.128+0000 7f6a31400640 1 -- 172.21.15.123:0/302754171 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3419f9e0 con 0x7f6a34109060 2024-09-19T04:57:01.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.129+0000 7f6a31400640 1 --2- 172.21.15.123:0/302754171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a34109060 0x7f6a3419ab70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000e9c0 tx=0x7f6a2000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:01.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.129+0000 7f6a2aa00640 1 -- 172.21.15.123:0/302754171 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2000cd80 con 0x7f6a34109060 2024-09-19T04:57:01.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.129+0000 7f6a38a00640 1 -- 172.21.15.123:0/302754171 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3419fcd0 con 0x7f6a34109060 2024-09-19T04:57:01.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.129+0000 7f6a2aa00640 1 -- 172.21.15.123:0/302754171 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a20005500 con 0x7f6a34109060 2024-09-19T04:57:01.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.129+0000 7f6a2aa00640 1 -- 172.21.15.123:0/302754171 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20010690 con 0x7f6a34109060 2024-09-19T04:57:01.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.129+0000 7f6a38a00640 1 -- 172.21.15.123:0/302754171 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a341a74c0 con 0x7f6a34109060 2024-09-19T04:57:01.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.130+0000 7f6a38a00640 1 -- 172.21.15.123:0/302754171 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a341042f0 con 0x7f6a34109060 2024-09-19T04:57:01.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.133+0000 7f6a2aa00640 1 -- 172.21.15.123:0/302754171 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6a200046c0 con 0x7f6a34109060 2024-09-19T04:57:01.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.134+0000 7f6a2aa00640 1 --2- 172.21.15.123:0/302754171 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6a180a33c0 0x7f6a180a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:01.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.134+0000 7f6a2aa00640 1 -- 172.21.15.123:0/302754171 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f6a200d0300 con 0x7f6a34109060 2024-09-19T04:57:01.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.134+0000 7f6a31e00640 1 --2- 172.21.15.123:0/302754171 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6a180a33c0 0x7f6a180a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:01.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.135+0000 7f6a31e00640 1 --2- 172.21.15.123:0/302754171 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6a180a33c0 0x7f6a180a5880 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6a24000c00 tx=0x7f6a24005b50 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:01.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.136+0000 7f6a2aa00640 1 -- 172.21.15.123:0/302754171 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6a20054210 con 0x7f6a34109060 2024-09-19T04:57:01.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.310+0000 7f6a38a00640 1 -- 172.21.15.123:0/302754171 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6a341048c0 con 0x7f6a34109060 2024-09-19T04:57:01.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.311+0000 7f6a2aa00640 1 -- 172.21.15.123:0/302754171 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f6a2009cc80 con 0x7f6a34109060 2024-09-19T04:57:01.313 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-09-19T04:57:01.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.314+0000 7f6a38a00640 1 -- 172.21.15.123:0/302754171 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6a180a33c0 msgr2=0x7f6a180a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:01.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.314+0000 7f6a38a00640 1 --2- 172.21.15.123:0/302754171 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6a180a33c0 0x7f6a180a5880 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6a24000c00 tx=0x7f6a24005b50 comp rx=0 tx=0).stop 2024-09-19T04:57:01.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.314+0000 7f6a38a00640 1 -- 172.21.15.123:0/302754171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a34109060 msgr2=0x7f6a3419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:01.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.314+0000 7f6a38a00640 1 --2- 172.21.15.123:0/302754171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a34109060 0x7f6a3419ab70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000e9c0 tx=0x7f6a2000ee80 comp rx=0 tx=0).stop 2024-09-19T04:57:01.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.314+0000 7f6a38a00640 1 -- 172.21.15.123:0/302754171 shutdown_connections 2024-09-19T04:57:01.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.314+0000 7f6a38a00640 1 --2- 172.21.15.123:0/302754171 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f6a180a33c0 0x7f6a180a5880 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.314+0000 7f6a38a00640 1 --2- 172.21.15.123:0/302754171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a3419eef0 0x7f6a3419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.314+0000 7f6a38a00640 1 --2- 172.21.15.123:0/302754171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a34109060 0x7f6a3419ab70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.314+0000 7f6a38a00640 1 --2- 172.21.15.123:0/302754171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a34103f10 0x7f6a3419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.314+0000 7f6a38a00640 1 -- 172.21.15.123:0/302754171 >> 172.21.15.123:0/302754171 conn(0x7f6a340ff940 msgr2=0x7f6a34100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:01.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.314+0000 7f6a38a00640 1 -- 172.21.15.123:0/302754171 shutdown_connections 2024-09-19T04:57:01.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.314+0000 7f6a38a00640 1 -- 172.21.15.123:0/302754171 wait complete. 2024-09-19T04:57:01.331 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T04:57:01.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.487+0000 7f2a68e00640 1 -- 172.21.15.123:0/3319697855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a641048c0 msgr2=0x7f2a64108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:01.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.487+0000 7f2a68e00640 1 --2- 172.21.15.123:0/3319697855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a641048c0 0x7f2a64108900 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009900 tx=0x7f2a4c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.488+0000 7f2a68e00640 1 -- 172.21.15.123:0/3319697855 shutdown_connections 2024-09-19T04:57:01.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.488+0000 7f2a68e00640 1 --2- 172.21.15.123:0/3319697855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a64109060 0x7f2a64110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.488+0000 7f2a68e00640 1 --2- 172.21.15.123:0/3319697855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a641048c0 0x7f2a64108900 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.488+0000 7f2a68e00640 1 --2- 172.21.15.123:0/3319697855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a64103f10 0x7f2a641042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.488+0000 7f2a68e00640 1 -- 172.21.15.123:0/3319697855 >> 172.21.15.123:0/3319697855 conn(0x7f2a640ff980 msgr2=0x7f2a64101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:01.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.488+0000 7f2a68e00640 1 -- 172.21.15.123:0/3319697855 shutdown_connections 2024-09-19T04:57:01.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.488+0000 7f2a68e00640 1 -- 172.21.15.123:0/3319697855 wait complete. 2024-09-19T04:57:01.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.488+0000 7f2a68e00640 1 Processor -- start 2024-09-19T04:57:01.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.489+0000 7f2a68e00640 1 -- start start 2024-09-19T04:57:01.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.489+0000 7f2a68e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a64103f10 0x7f2a6419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:01.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.489+0000 7f2a68e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a64109060 0x7f2a6419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:01.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.489+0000 7f2a68e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a6419eee0 0x7f2a6419f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:01.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.489+0000 7f2a68e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641140b0 con 0x7f2a64109060 2024-09-19T04:57:01.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.489+0000 7f2a68e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a64113f30 con 0x7f2a64103f10 2024-09-19T04:57:01.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.489+0000 7f2a68e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2a64114230 con 0x7f2a6419eee0 2024-09-19T04:57:01.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.489+0000 7f2a61e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a64103f10 0x7f2a6419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:01.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.489+0000 7f2a5be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a64109060 0x7f2a6419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:01.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.489+0000 7f2a61e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a64103f10 0x7f2a6419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:45800/0 (socket says 172.21.15.123:45800) 2024-09-19T04:57:01.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.489+0000 7f2a61e00640 1 -- 172.21.15.123:0/1683205766 learned_addr learned my addr 172.21.15.123:0/1683205766 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:57:01.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.489+0000 7f2a62800640 1 --2- 172.21.15.123:0/1683205766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a6419eee0 0x7f2a6419f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:01.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.490+0000 7f2a61e00640 1 -- 172.21.15.123:0/1683205766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a6419eee0 msgr2=0x7f2a6419f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:01.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.490+0000 7f2a61e00640 1 --2- 172.21.15.123:0/1683205766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a6419eee0 0x7f2a6419f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.490+0000 7f2a61e00640 1 -- 172.21.15.123:0/1683205766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a64109060 msgr2=0x7f2a6419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:01.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.490+0000 7f2a61e00640 1 --2- 172.21.15.123:0/1683205766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a64109060 0x7f2a6419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.490+0000 7f2a61e00640 1 -- 172.21.15.123:0/1683205766 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6419f9d0 con 0x7f2a64103f10 2024-09-19T04:57:01.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.490+0000 7f2a62800640 1 --2- 172.21.15.123:0/1683205766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a6419eee0 0x7f2a6419f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:57:01.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.490+0000 7f2a61e00640 1 --2- 172.21.15.123:0/1683205766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a64103f10 0x7f2a6419a620 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400e9f0 tx=0x7f2a5400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:01.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.490+0000 7f2a5aa00640 1 -- 172.21.15.123:0/1683205766 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400cd80 con 0x7f2a64103f10 2024-09-19T04:57:01.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.490+0000 7f2a5aa00640 1 -- 172.21.15.123:0/1683205766 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a54010910 con 0x7f2a64103f10 2024-09-19T04:57:01.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.490+0000 7f2a68e00640 1 -- 172.21.15.123:0/1683205766 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6419fcc0 con 0x7f2a64103f10 2024-09-19T04:57:01.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.490+0000 7f2a5aa00640 1 -- 172.21.15.123:0/1683205766 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54018990 con 0x7f2a64103f10 2024-09-19T04:57:01.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.490+0000 7f2a68e00640 1 -- 172.21.15.123:0/1683205766 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a641a74d0 con 0x7f2a64103f10 2024-09-19T04:57:01.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.491+0000 7f2a68e00640 1 -- 172.21.15.123:0/1683205766 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a641042f0 con 0x7f2a64103f10 2024-09-19T04:57:01.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.493+0000 7f2a5aa00640 1 -- 172.21.15.123:0/1683205766 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f2a54010ac0 con 0x7f2a64103f10 2024-09-19T04:57:01.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.494+0000 7f2a5aa00640 1 --2- 172.21.15.123:0/1683205766 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2a480a3440 0x7f2a480a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:01.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.494+0000 7f2a5aa00640 1 -- 172.21.15.123:0/1683205766 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f2a540d00a0 con 0x7f2a64103f10 2024-09-19T04:57:01.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.495+0000 7f2a5be00640 1 --2- 172.21.15.123:0/1683205766 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2a480a3440 0x7f2a480a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:01.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.495+0000 7f2a5be00640 1 --2- 172.21.15.123:0/1683205766 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2a480a3440 0x7f2a480a5900 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c006010 tx=0x7f2a4c000bc0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:01.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.498+0000 7f2a5aa00640 1 -- 172.21.15.123:0/1683205766 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2a5409ca20 con 0x7f2a64103f10 2024-09-19T04:57:01.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.672+0000 7f2a68e00640 1 -- 172.21.15.123:0/1683205766 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2a6410c2d0 con 0x7f2a64103f10 2024-09-19T04:57:01.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.673+0000 7f2a5aa00640 1 -- 172.21.15.123:0/1683205766 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f2a5409c320 con 0x7f2a64103f10 2024-09-19T04:57:01.675 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-09-19T04:57:01.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:01 smithi123 ceph-mon[30511]: pgmap v320: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T04:57:01.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:01 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/4057842542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:57:01.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:01 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3045263863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:57:01.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:01 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/302754171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:57:01.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.675+0000 7f2a68e00640 1 -- 172.21.15.123:0/1683205766 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2a480a3440 msgr2=0x7f2a480a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:01.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.675+0000 7f2a68e00640 1 --2- 172.21.15.123:0/1683205766 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2a480a3440 0x7f2a480a5900 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c006010 tx=0x7f2a4c000bc0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.676+0000 7f2a68e00640 1 -- 172.21.15.123:0/1683205766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a64103f10 msgr2=0x7f2a6419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:01.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.676+0000 7f2a68e00640 1 --2- 172.21.15.123:0/1683205766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a64103f10 0x7f2a6419a620 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400e9f0 tx=0x7f2a5400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.676+0000 7f2a68e00640 1 -- 172.21.15.123:0/1683205766 shutdown_connections 2024-09-19T04:57:01.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.676+0000 7f2a68e00640 1 --2- 172.21.15.123:0/1683205766 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f2a480a3440 0x7f2a480a5900 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.676+0000 7f2a68e00640 1 --2- 172.21.15.123:0/1683205766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a6419eee0 0x7f2a6419f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.676+0000 7f2a68e00640 1 --2- 172.21.15.123:0/1683205766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a64109060 0x7f2a6419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.676+0000 7f2a68e00640 1 --2- 172.21.15.123:0/1683205766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a64103f10 0x7f2a6419a620 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.676+0000 7f2a68e00640 1 -- 172.21.15.123:0/1683205766 >> 172.21.15.123:0/1683205766 conn(0x7f2a640ff980 msgr2=0x7f2a64100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:01.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.676+0000 7f2a68e00640 1 -- 172.21.15.123:0/1683205766 shutdown_connections 2024-09-19T04:57:01.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.676+0000 7f2a68e00640 1 -- 172.21.15.123:0/1683205766 wait complete. 2024-09-19T04:57:01.694 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T04:57:01.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:01 smithi193 ceph-mon[35592]: pgmap v320: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T04:57:01.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:01 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/4057842542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:57:01.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:01 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3045263863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:57:01.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:01 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/302754171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:57:01.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.848+0000 7fc9a3000640 1 -- 172.21.15.123:0/2105180639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc99c1078f0 msgr2=0x7fc99c107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:01.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.848+0000 7fc9a3000640 1 --2- 172.21.15.123:0/2105180639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc99c1078f0 0x7fc99c107cd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc990009930 tx=0x7fc99002a440 comp rx=0 tx=0).stop 2024-09-19T04:57:01.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.848+0000 7fc9a3000640 1 -- 172.21.15.123:0/2105180639 shutdown_connections 2024-09-19T04:57:01.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.848+0000 7fc9a3000640 1 --2- 172.21.15.123:0/2105180639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc99c069d00 0x7fc99c10e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.848+0000 7fc9a3000640 1 --2- 172.21.15.123:0/2105180639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc99c069380 0x7fc99c0697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.848+0000 7fc9a3000640 1 --2- 172.21.15.123:0/2105180639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc99c1078f0 0x7fc99c107cd0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.848+0000 7fc9a3000640 1 -- 172.21.15.123:0/2105180639 >> 172.21.15.123:0/2105180639 conn(0x7fc99c076550 msgr2=0x7fc99c076960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:01.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.848+0000 7fc9a3000640 1 -- 172.21.15.123:0/2105180639 shutdown_connections 2024-09-19T04:57:01.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.848+0000 7fc9a3000640 1 -- 172.21.15.123:0/2105180639 wait complete. 2024-09-19T04:57:01.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:01 smithi183 ceph-mon[35333]: pgmap v320: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T04:57:01.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:01 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/4057842542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:57:01.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:01 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3045263863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:57:01.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:01 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/302754171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:57:01.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.849+0000 7fc9a3000640 1 Processor -- start 2024-09-19T04:57:01.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.849+0000 7fc9a3000640 1 -- start start 2024-09-19T04:57:01.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.849+0000 7fc9a3000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc99c069380 0x7fc99c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:01.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.849+0000 7fc9a3000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc99c069d00 0x7fc99c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:01.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.849+0000 7fc9a3000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc99c1078f0 0x7fc99c19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:01.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.849+0000 7fc9a3000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c111d20 con 0x7fc99c069380 2024-09-19T04:57:01.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.849+0000 7fc9a3000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c111ba0 con 0x7fc99c1078f0 2024-09-19T04:57:01.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.849+0000 7fc9a3000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c111ea0 con 0x7fc99c069d00 2024-09-19T04:57:01.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.849+0000 7fc9a1c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc99c069380 0x7fc99c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:01.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.850+0000 7fc9a1c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc99c069380 0x7fc99c19a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47706/0 (socket says 172.21.15.123:47706) 2024-09-19T04:57:01.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.850+0000 7fc9a1c00640 1 -- 172.21.15.123:0/3219357783 learned_addr learned my addr 172.21.15.123:0/3219357783 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:57:01.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.850+0000 7fc9a1200640 1 --2- 172.21.15.123:0/3219357783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc99c069d00 0x7fc99c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:01.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.850+0000 7fc9a2600640 1 --2- 172.21.15.123:0/3219357783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc99c1078f0 0x7fc99c19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:01.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.850+0000 7fc9a1c00640 1 -- 172.21.15.123:0/3219357783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc99c069d00 msgr2=0x7fc99c19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:01.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.850+0000 7fc9a1c00640 1 --2- 172.21.15.123:0/3219357783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc99c069d00 0x7fc99c19a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.850+0000 7fc9a1c00640 1 -- 172.21.15.123:0/3219357783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc99c1078f0 msgr2=0x7fc99c19ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:01.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.850+0000 7fc9a1c00640 1 --2- 172.21.15.123:0/3219357783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc99c1078f0 0x7fc99c19ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.850+0000 7fc9a1c00640 1 -- 172.21.15.123:0/3219357783 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99c19f3e0 con 0x7fc99c069380 2024-09-19T04:57:01.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.850+0000 7fc9a1200640 1 --2- 172.21.15.123:0/3219357783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc99c069d00 0x7fc99c19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:57:01.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.850+0000 7fc9a1c00640 1 --2- 172.21.15.123:0/3219357783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc99c069380 0x7fc99c19a3f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc99002a960 tx=0x7fc990003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:01.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.850+0000 7fc99a000640 1 -- 172.21.15.123:0/3219357783 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99003f070 con 0x7fc99c069380 2024-09-19T04:57:01.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.851+0000 7fc99a000640 1 -- 172.21.15.123:0/3219357783 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc990031c90 con 0x7fc99c069380 2024-09-19T04:57:01.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.851+0000 7fc9a3000640 1 -- 172.21.15.123:0/3219357783 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99c19f6d0 con 0x7fc99c069380 2024-09-19T04:57:01.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.851+0000 7fc99a000640 1 -- 172.21.15.123:0/3219357783 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990049060 con 0x7fc99c069380 2024-09-19T04:57:01.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.851+0000 7fc9a3000640 1 -- 172.21.15.123:0/3219357783 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99c1a6f60 con 0x7fc99c069380 2024-09-19T04:57:01.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.852+0000 7fc9a3000640 1 -- 172.21.15.123:0/3219357783 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc99c0697c0 con 0x7fc99c069380 2024-09-19T04:57:01.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.853+0000 7fc99a000640 1 -- 172.21.15.123:0/3219357783 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc990042070 con 0x7fc99c069380 2024-09-19T04:57:01.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.855+0000 7fc99a000640 1 --2- 172.21.15.123:0/3219357783 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc9840a33c0 0x7fc9840a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:01.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.855+0000 7fc99a000640 1 -- 172.21.15.123:0/3219357783 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fc9900f06b0 con 0x7fc99c069380 2024-09-19T04:57:01.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.855+0000 7fc9a1200640 1 --2- 172.21.15.123:0/3219357783 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc9840a33c0 0x7fc9840a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:01.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.856+0000 7fc9a1200640 1 --2- 172.21.15.123:0/3219357783 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc9840a33c0 0x7fc9840a5880 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc99c10ebd0 tx=0x7fc988005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:01.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:01.859+0000 7fc99a000640 1 -- 172.21.15.123:0/3219357783 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc9900bc9e0 con 0x7fc99c069380 2024-09-19T04:57:02.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:02.081+0000 7fc9a3000640 1 -- 172.21.15.123:0/3219357783 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fc99c1a7190 con 0x7fc99c069380 2024-09-19T04:57:02.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:02.091+0000 7fc99a000640 1 -- 172.21.15.123:0/3219357783 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+742881 (secure 0 0 0) 0x7fc9900c0ce0 con 0x7fc99c069380 2024-09-19T04:57:02.103 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:57:02.103 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:57:02.104 INFO:teuthology.orchestra.run.smithi123.stdout: "epoch": 26, 2024-09-19T04:57:02.104 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.104 INFO:teuthology.orchestra.run.smithi123.stdout: "active_gid": 25133, 2024-09-19T04:57:02.104 INFO:teuthology.orchestra.run.smithi123.stdout: "active_name": "x", 2024-09-19T04:57:02.104 INFO:teuthology.orchestra.run.smithi123.stdout: "active_addrs": { 2024-09-19T04:57:02.104 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-09-19T04:57:02.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.104 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-09-19T04:57:02.104 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6800", 2024-09-19T04:57:02.104 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1578549821 2024-09-19T04:57:02.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.105 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-09-19T04:57:02.105 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6801", 2024-09-19T04:57:02.105 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1578549821 2024-09-19T04:57:02.105 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.105 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T04:57:02.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.105 INFO:teuthology.orchestra.run.smithi123.stdout: "active_addr": "172.21.15.123:6801/1578549821", 2024-09-19T04:57:02.105 INFO:teuthology.orchestra.run.smithi123.stdout: "active_change": "2024-09-19T04:46:41.775319+0000", 2024-09-19T04:57:02.105 INFO:teuthology.orchestra.run.smithi123.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T04:57:02.105 INFO:teuthology.orchestra.run.smithi123.stdout: "available": true, 2024-09-19T04:57:02.105 INFO:teuthology.orchestra.run.smithi123.stdout: "standbys": [ 2024-09-19T04:57:02.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.105 INFO:teuthology.orchestra.run.smithi123.stdout: "gid": 25130, 2024-09-19T04:57:02.106 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "y", 2024-09-19T04:57:02.106 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_features": 4540701547738038271, 2024-09-19T04:57:02.106 INFO:teuthology.orchestra.run.smithi123.stdout: "available_modules": [ 2024-09-19T04:57:02.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.106 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "alerts", 2024-09-19T04:57:02.106 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.106 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.106 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.106 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-09-19T04:57:02.106 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-09-19T04:57:02.106 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.106 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.107 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.107 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-09-19T04:57:02.107 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.107 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.107 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.107 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T04:57:02.107 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.107 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.107 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.107 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.107 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.107 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.108 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.108 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.108 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.108 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.108 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.108 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.108 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.108 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.108 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.108 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.108 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.108 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.108 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.109 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.109 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.109 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.109 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.109 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.109 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.109 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.109 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.109 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.109 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.109 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.109 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.109 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.110 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.110 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.110 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.110 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.110 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.110 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.110 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.110 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.110 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.110 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.110 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.110 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.110 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.111 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.111 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.111 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.111 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.111 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.111 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.111 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.111 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.111 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.111 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.111 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.111 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.112 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.112 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.112 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.112 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.112 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.112 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.112 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.112 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.112 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.112 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.112 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.112 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.112 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_destination": { 2024-09-19T04:57:02.113 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_destination", 2024-09-19T04:57:02.113 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.113 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.113 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.113 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.113 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.113 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.113 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.113 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email address to send alerts to", 2024-09-19T04:57:02.113 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.113 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.113 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.113 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_from_name": { 2024-09-19T04:57:02.114 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_from_name", 2024-09-19T04:57:02.114 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.114 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.114 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.114 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "Ceph", 2024-09-19T04:57:02.114 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.114 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.114 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.114 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email From: name", 2024-09-19T04:57:02.114 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.114 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.114 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.114 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_host": { 2024-09-19T04:57:02.115 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_host", 2024-09-19T04:57:02.115 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.115 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.115 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.115 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.115 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.115 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.115 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.115 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP server", 2024-09-19T04:57:02.115 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.115 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.115 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.115 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_password": { 2024-09-19T04:57:02.116 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_password", 2024-09-19T04:57:02.116 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.116 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.116 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.116 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.116 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.116 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.116 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.116 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Password to authenticate with", 2024-09-19T04:57:02.116 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.116 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.116 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.117 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_port": { 2024-09-19T04:57:02.117 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_port", 2024-09-19T04:57:02.117 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.117 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.117 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.117 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "465", 2024-09-19T04:57:02.117 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.117 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.117 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.117 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP port", 2024-09-19T04:57:02.117 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.117 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.118 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.118 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_sender": { 2024-09-19T04:57:02.118 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_sender", 2024-09-19T04:57:02.118 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.118 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.118 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.118 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.118 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.118 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.118 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.118 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP envelope sender", 2024-09-19T04:57:02.118 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.118 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.119 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.119 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_ssl": { 2024-09-19T04:57:02.119 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_ssl", 2024-09-19T04:57:02.119 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.119 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.119 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.119 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.119 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.119 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.119 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.119 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T04:57:02.119 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.120 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.120 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.120 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_user": { 2024-09-19T04:57:02.120 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_user", 2024-09-19T04:57:02.120 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.120 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.120 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.120 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.120 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.120 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.120 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.120 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "User to authenticate as", 2024-09-19T04:57:02.121 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.121 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.121 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.121 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.121 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.121 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "balancer", 2024-09-19T04:57:02.121 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.121 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.121 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.121 INFO:teuthology.orchestra.run.smithi123.stdout: "active": { 2024-09-19T04:57:02.121 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "active", 2024-09-19T04:57:02.121 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.122 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.122 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.122 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.122 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.122 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.122 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.122 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T04:57:02.122 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.122 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.122 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.122 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_time": { 2024-09-19T04:57:02.122 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_time", 2024-09-19T04:57:02.122 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.123 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.123 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.123 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0000", 2024-09-19T04:57:02.123 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.123 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.123 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.123 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T04:57:02.123 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:57:02.123 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.123 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.123 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_weekday": { 2024-09-19T04:57:02.123 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_weekday", 2024-09-19T04:57:02.123 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-09-19T04:57:02.124 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.124 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.124 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-09-19T04:57:02.124 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-09-19T04:57:02.124 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-09-19T04:57:02.124 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.124 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T04:57:02.124 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:57:02.124 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.124 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.124 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_max_iterations": { 2024-09-19T04:57:02.124 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_max_iterations", 2024-09-19T04:57:02.125 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-09-19T04:57:02.125 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.125 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.125 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "25", 2024-09-19T04:57:02.125 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-09-19T04:57:02.125 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "250", 2024-09-19T04:57:02.125 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.125 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T04:57:02.125 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.125 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.125 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.125 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_metrics": { 2024-09-19T04:57:02.125 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_metrics", 2024-09-19T04:57:02.125 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.126 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.126 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.126 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T04:57:02.126 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.126 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.126 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.126 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T04:57:02.126 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T04:57:02.126 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.126 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.126 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_step": { 2024-09-19T04:57:02.126 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_step", 2024-09-19T04:57:02.126 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:02.127 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.127 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.127 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.5", 2024-09-19T04:57:02.127 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0.001", 2024-09-19T04:57:02.127 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "0.999", 2024-09-19T04:57:02.127 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.127 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "aggressiveness of optimization", 2024-09-19T04:57:02.127 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T04:57:02.127 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.127 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.127 INFO:teuthology.orchestra.run.smithi123.stdout: "end_time": { 2024-09-19T04:57:02.128 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_time", 2024-09-19T04:57:02.128 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.128 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.128 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.128 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2359", 2024-09-19T04:57:02.128 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.128 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.128 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.128 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T04:57:02.128 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:57:02.128 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.128 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.128 INFO:teuthology.orchestra.run.smithi123.stdout: "end_weekday": { 2024-09-19T04:57:02.129 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_weekday", 2024-09-19T04:57:02.129 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-09-19T04:57:02.129 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.129 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.129 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-09-19T04:57:02.129 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-09-19T04:57:02.129 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-09-19T04:57:02.129 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.129 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T04:57:02.129 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:57:02.129 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.129 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.129 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.130 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.130 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.130 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.130 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.130 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.130 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.130 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.130 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.130 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.130 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.130 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.130 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.130 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.130 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.131 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.131 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.131 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.131 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.131 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.131 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.131 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.131 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.131 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.131 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.131 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.131 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.132 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.132 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.132 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.132 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.132 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.132 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.132 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.132 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.132 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.132 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.132 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.132 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.133 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.133 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.133 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.133 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.133 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.133 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.133 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.133 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.133 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.133 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.133 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.133 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.133 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.134 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.134 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.134 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.134 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.134 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.134 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.134 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.134 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.134 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.134 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.134 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.134 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.134 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.135 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.135 INFO:teuthology.orchestra.run.smithi123.stdout: "min_score": { 2024-09-19T04:57:02.135 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_score", 2024-09-19T04:57:02.135 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:02.135 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.135 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.135 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-09-19T04:57:02.135 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.135 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.135 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.135 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T04:57:02.135 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.136 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.136 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.136 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-09-19T04:57:02.136 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-09-19T04:57:02.136 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.136 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.136 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.136 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "upmap", 2024-09-19T04:57:02.136 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.136 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.136 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.136 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-compat", 2024-09-19T04:57:02.136 INFO:teuthology.orchestra.run.smithi123.stdout: "none", 2024-09-19T04:57:02.137 INFO:teuthology.orchestra.run.smithi123.stdout: "read", 2024-09-19T04:57:02.137 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap", 2024-09-19T04:57:02.137 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap-read" 2024-09-19T04:57:02.137 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.137 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Balancer mode", 2024-09-19T04:57:02.137 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.137 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.137 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.137 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_ids": { 2024-09-19T04:57:02.137 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_ids", 2024-09-19T04:57:02.137 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T04:57:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-09-19T04:57:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-09-19T04:57:02.138 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-09-19T04:57:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T04:57:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_deviation": { 2024-09-19T04:57:02.139 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_deviation", 2024-09-19T04:57:02.140 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.140 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.140 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.140 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-09-19T04:57:02.140 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-09-19T04:57:02.140 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.140 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.140 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T04:57:02.140 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T04:57:02.140 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.140 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.140 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_optimizations": { 2024-09-19T04:57:02.141 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_optimizations", 2024-09-19T04:57:02.141 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-09-19T04:57:02.141 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.141 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.141 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:02.141 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.141 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.141 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.141 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T04:57:02.141 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.141 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.141 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.141 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.142 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.142 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm", 2024-09-19T04:57:02.142 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.142 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.142 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.142 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_down_multiplier": { 2024-09-19T04:57:02.142 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_down_multiplier", 2024-09-19T04:57:02.142 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:02.142 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.142 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.142 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-09-19T04:57:02.143 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.143 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.143 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.143 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T04:57:02.143 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.143 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.143 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.143 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_refresh_rate": { 2024-09-19T04:57:02.143 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_refresh_rate", 2024-09-19T04:57:02.143 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.143 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.143 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.143 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "20", 2024-09-19T04:57:02.144 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.144 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.144 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.144 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T04:57:02.144 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.144 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.144 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.144 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_starting_port": { 2024-09-19T04:57:02.144 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_starting_port", 2024-09-19T04:57:02.144 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.144 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.144 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.144 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4721", 2024-09-19T04:57:02.145 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.145 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.145 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.145 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T04:57:02.145 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.145 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.145 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.145 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_ptrace": { 2024-09-19T04:57:02.145 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_ptrace", 2024-09-19T04:57:02.145 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.146 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.146 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.146 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.146 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.146 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.146 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.146 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T04:57:02.146 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T04:57:02.146 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.146 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.146 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_interval": { 2024-09-19T04:57:02.146 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_interval", 2024-09-19T04:57:02.147 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.147 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.147 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.147 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-09-19T04:57:02.147 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.147 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.147 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.147 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T04:57:02.147 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.147 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.147 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.147 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_memory_target_ratio": { 2024-09-19T04:57:02.147 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T04:57:02.148 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:02.148 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.148 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.148 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.7", 2024-09-19T04:57:02.148 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.148 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.148 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.148 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T04:57:02.148 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.148 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.148 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.148 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm_log_destination": { 2024-09-19T04:57:02.149 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm_log_destination", 2024-09-19T04:57:02.149 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.149 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.149 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.149 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.149 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.149 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.149 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.149 INFO:teuthology.orchestra.run.smithi123.stdout: "file", 2024-09-19T04:57:02.149 INFO:teuthology.orchestra.run.smithi123.stdout: "file,syslog", 2024-09-19T04:57:02.149 INFO:teuthology.orchestra.run.smithi123.stdout: "syslog" 2024-09-19T04:57:02.149 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.149 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T04:57:02.150 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.150 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.150 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.150 INFO:teuthology.orchestra.run.smithi123.stdout: "cgroups_split": { 2024-09-19T04:57:02.150 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cgroups_split", 2024-09-19T04:57:02.150 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.150 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.150 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.150 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.150 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.150 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.151 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.151 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T04:57:02.151 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.151 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.151 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.151 INFO:teuthology.orchestra.run.smithi123.stdout: "config_checks_enabled": { 2024-09-19T04:57:02.151 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_checks_enabled", 2024-09-19T04:57:02.151 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.151 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.151 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.151 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.151 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.151 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.152 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.152 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T04:57:02.152 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.152 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.152 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.152 INFO:teuthology.orchestra.run.smithi123.stdout: "config_dashboard": { 2024-09-19T04:57:02.152 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_dashboard", 2024-09-19T04:57:02.152 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.152 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.152 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.152 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.152 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.153 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.153 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.153 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T04:57:02.153 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.153 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.153 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.153 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_alertmanager": { 2024-09-19T04:57:02.153 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_alertmanager", 2024-09-19T04:57:02.153 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.153 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.153 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.153 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T04:57:02.154 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.154 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.154 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.154 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-09-19T04:57:02.154 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.154 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.154 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.154 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_base": { 2024-09-19T04:57:02.154 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_base", 2024-09-19T04:57:02.154 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.154 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.154 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.154 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T04:57:02.154 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.155 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.155 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.155 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Container image name, without the tag", 2024-09-19T04:57:02.155 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.155 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.155 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.155 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_elasticsearch": { 2024-09-19T04:57:02.155 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_elasticsearch", 2024-09-19T04:57:02.155 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.155 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.155 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.155 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T04:57:02.155 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.155 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.156 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.156 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "elasticsearch container image", 2024-09-19T04:57:02.156 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.156 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.156 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.156 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_grafana": { 2024-09-19T04:57:02.156 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_grafana", 2024-09-19T04:57:02.156 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.156 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.156 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.156 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T04:57:02.156 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.156 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.157 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.157 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-09-19T04:57:02.157 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.157 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.157 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.157 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_haproxy": { 2024-09-19T04:57:02.157 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_haproxy", 2024-09-19T04:57:02.157 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.157 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.157 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.157 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T04:57:02.157 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.158 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.158 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.158 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "HAproxy container image", 2024-09-19T04:57:02.158 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.158 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.158 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.158 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_agent": { 2024-09-19T04:57:02.159 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_agent", 2024-09-19T04:57:02.159 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.159 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.159 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.159 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T04:57:02.159 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.159 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.159 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.159 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger agent container image", 2024-09-19T04:57:02.159 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.159 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.159 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.160 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_collector": { 2024-09-19T04:57:02.160 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_collector", 2024-09-19T04:57:02.160 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.160 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.160 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.160 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T04:57:02.160 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.160 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.160 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.160 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger collector container image", 2024-09-19T04:57:02.160 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.160 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.160 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_query": { 2024-09-19T04:57:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_query", 2024-09-19T04:57:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T04:57:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger query container image", 2024-09-19T04:57:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_keepalived": { 2024-09-19T04:57:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_keepalived", 2024-09-19T04:57:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T04:57:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Keepalived container image", 2024-09-19T04:57:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_loki": { 2024-09-19T04:57:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_loki", 2024-09-19T04:57:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T04:57:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Loki container image", 2024-09-19T04:57:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_node_exporter": { 2024-09-19T04:57:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_node_exporter", 2024-09-19T04:57:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T04:57:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-09-19T04:57:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_nvmeof": { 2024-09-19T04:57:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_nvmeof", 2024-09-19T04:57:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T04:57:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Nvme-of container image", 2024-09-19T04:57:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_prometheus": { 2024-09-19T04:57:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_prometheus", 2024-09-19T04:57:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T04:57:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-09-19T04:57:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_promtail": { 2024-09-19T04:57:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_promtail", 2024-09-19T04:57:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T04:57:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Promtail container image", 2024-09-19T04:57:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_samba": { 2024-09-19T04:57:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_samba", 2024-09-19T04:57:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T04:57:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Samba/SMB container image", 2024-09-19T04:57:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_snmp_gateway": { 2024-09-19T04:57:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_snmp_gateway", 2024-09-19T04:57:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T04:57:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SNMP Gateway container image", 2024-09-19T04:57:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "container_init": { 2024-09-19T04:57:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_init", 2024-09-19T04:57:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T04:57:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "daemon_cache_timeout": { 2024-09-19T04:57:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "daemon_cache_timeout", 2024-09-19T04:57:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-09-19T04:57:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T04:57:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "default_cephadm_command_timeout": { 2024-09-19T04:57:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T04:57:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "900", 2024-09-19T04:57:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T04:57:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "default_registry": { 2024-09-19T04:57:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_registry", 2024-09-19T04:57:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io", 2024-09-19T04:57:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T04:57:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "device_cache_timeout": { 2024-09-19T04:57:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_cache_timeout", 2024-09-19T04:57:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1800", 2024-09-19T04:57:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache device inventory", 2024-09-19T04:57:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "device_enhanced_scan": { 2024-09-19T04:57:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_enhanced_scan", 2024-09-19T04:57:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T04:57:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "facts_cache_timeout": { 2024-09-19T04:57:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "facts_cache_timeout", 2024-09-19T04:57:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-09-19T04:57:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache host facts data", 2024-09-19T04:57:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: "grafana_dashboards_path": { 2024-09-19T04:57:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "grafana_dashboards_path", 2024-09-19T04:57:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T04:57:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T04:57:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: "host_check_interval": { 2024-09-19T04:57:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "host_check_interval", 2024-09-19T04:57:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-09-19T04:57:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to perform a host check", 2024-09-19T04:57:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "hw_monitoring": { 2024-09-19T04:57:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hw_monitoring", 2024-09-19T04:57:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T04:57:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: "inventory_list_all": { 2024-09-19T04:57:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "inventory_list_all", 2024-09-19T04:57:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T04:57:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "log_refresh_metadata": { 2024-09-19T04:57:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_refresh_metadata", 2024-09-19T04:57:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T04:57:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T04:57:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T04:57:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T04:57:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T04:57:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T04:57:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T04:57:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "*", 2024-09-19T04:57:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T04:57:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "max_count_per_host": { 2024-09-19T04:57:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_count_per_host", 2024-09-19T04:57:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of daemons per service per host", 2024-09-19T04:57:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "max_osd_draining_count": { 2024-09-19T04:57:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_osd_draining_count", 2024-09-19T04:57:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T04:57:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "migration_current": { 2024-09-19T04:57:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "migration_current", 2024-09-19T04:57:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "internal - do not modify", 2024-09-19T04:57:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-09-19T04:57:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-09-19T04:57:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "root", 2024-09-19T04:57:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm-package", 2024-09-19T04:57:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "root" 2024-09-19T04:57:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T04:57:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "oob_default_addr": { 2024-09-19T04:57:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "oob_default_addr", 2024-09-19T04:57:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "169.254.1.1", 2024-09-19T04:57:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T04:57:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: "prometheus_alerts_path": { 2024-09-19T04:57:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus_alerts_path", 2024-09-19T04:57:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T04:57:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T04:57:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_insecure": { 2024-09-19T04:57:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_insecure", 2024-09-19T04:57:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T04:57:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_password": { 2024-09-19T04:57:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_password", 2024-09-19T04:57:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T04:57:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_url": { 2024-09-19T04:57:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_url", 2024-09-19T04:57:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T04:57:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_username": { 2024-09-19T04:57:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_username", 2024-09-19T04:57:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T04:57:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: "secure_monitoring_stack": { 2024-09-19T04:57:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "secure_monitoring_stack", 2024-09-19T04:57:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T04:57:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "service_discovery_port": { 2024-09-19T04:57:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "service_discovery_port", 2024-09-19T04:57:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8765", 2024-09-19T04:57:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "cephadm service discovery port", 2024-09-19T04:57:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "ssh_config_file": { 2024-09-19T04:57:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssh_config_file", 2024-09-19T04:57:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T04:57:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "use_agent": { 2024-09-19T04:57:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_agent", 2024-09-19T04:57:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T04:57:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "use_repo_digest": { 2024-09-19T04:57:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_repo_digest", 2024-09-19T04:57:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T04:57:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_failed_host_check": { 2024-09-19T04:57:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_failed_host_check", 2024-09-19T04:57:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T04:57:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_daemons": { 2024-09-19T04:57:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_daemons", 2024-09-19T04:57:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T04:57:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_hosts": { 2024-09-19T04:57:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_hosts", 2024-09-19T04:57:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T04:57:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crash", 2024-09-19T04:57:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "retain_interval": { 2024-09-19T04:57:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retain_interval", 2024-09-19T04:57:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "31536000", 2024-09-19T04:57:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T04:57:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_recent_interval": { 2024-09-19T04:57:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_recent_interval", 2024-09-19T04:57:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1209600", 2024-09-19T04:57:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T04:57:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dashboard", 2024-09-19T04:57:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T04:57:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T04:57:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T04:57:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T04:57:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T04:57:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T04:57:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_ENABLED": { 2024-09-19T04:57:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T04:57:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T04:57:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T04:57:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T04:57:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T04:57:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T04:57:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T04:57:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T04:57:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T04:57:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T04:57:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T04:57:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T04:57:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T04:57:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T04:57:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T04:57:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T04:57:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T04:57:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T04:57:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T04:57:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T04:57:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T04:57:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T04:57:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T04:57:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-09-19T04:57:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T04:57:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T04:57:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_URL": { 2024-09-19T04:57:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_URL", 2024-09-19T04:57:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T04:57:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T04:57:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-09-19T04:57:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T04:57:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T04:57:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T04:57:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T04:57:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T04:57:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T04:57:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T04:57:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T04:57:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T04:57:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T04:57:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T04:57:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T04:57:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T04:57:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T04:57:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T04:57:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T04:57:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T04:57:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T04:57:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T04:57:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T04:57:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T04:57:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T04:57:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T04:57:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T04:57:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T04:57:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T04:57:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T04:57:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T04:57:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T04:57:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T04:57:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T04:57:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T04:57:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T04:57:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T04:57:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T04:57:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8", 2024-09-19T04:57:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T04:57:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T04:57:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "45", 2024-09-19T04:57:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T04:57:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T04:57:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T04:57:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T04:57:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-09-19T04:57:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T04:57:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T04:57:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.258 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.258 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.258 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.258 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T04:57:02.258 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T04:57:02.258 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.258 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.258 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T04:57:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T04:57:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-09-19T04:57:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T04:57:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T04:57:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T04:57:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T04:57:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-09-19T04:57:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "cross_origin_url": { 2024-09-19T04:57:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cross_origin_url", 2024-09-19T04:57:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "crt_file": { 2024-09-19T04:57:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crt_file", 2024-09-19T04:57:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.264 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.264 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.264 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.264 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.264 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.264 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.264 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.264 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.264 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.264 INFO:teuthology.orchestra.run.smithi123.stdout: "debug": { 2024-09-19T04:57:02.264 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "debug", 2024-09-19T04:57:02.264 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.265 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.265 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.265 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.265 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.268 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.269 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.269 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable/disable debug options", 2024-09-19T04:57:02.269 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.269 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.269 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.269 INFO:teuthology.orchestra.run.smithi123.stdout: "jwt_token_ttl": { 2024-09-19T04:57:02.269 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "jwt_token_ttl", 2024-09-19T04:57:02.269 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.269 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.269 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.270 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "28800", 2024-09-19T04:57:02.270 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.270 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.270 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.270 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.270 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.270 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.270 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.270 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-09-19T04:57:02.270 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-09-19T04:57:02.270 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.270 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.271 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.271 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.271 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.271 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.271 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.271 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.271 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.271 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.271 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.271 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.271 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.272 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.272 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.272 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.272 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.272 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.272 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.272 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.272 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.272 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.272 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.272 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.272 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.272 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.273 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.273 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.273 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.273 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.273 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.273 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.273 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.273 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.273 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.273 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.273 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.273 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.274 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.274 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.274 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.274 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.274 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.274 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.274 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.274 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.274 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.274 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.274 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.274 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.274 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.275 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.275 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.275 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.275 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.275 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.275 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.275 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.275 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.275 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.275 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.275 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.275 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.275 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.275 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.276 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.276 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.276 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.276 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.276 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.276 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.276 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.276 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.276 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.276 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.276 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.276 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.277 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.277 INFO:teuthology.orchestra.run.smithi123.stdout: "motd": { 2024-09-19T04:57:02.277 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "motd", 2024-09-19T04:57:02.277 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.277 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.277 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.277 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.277 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.277 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.277 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.277 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "The message of the day", 2024-09-19T04:57:02.277 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.278 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.278 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.278 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.278 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect_resolve_ip_addr": { 2024-09-19T04:57:02.278 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T04:57:02.278 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.278 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.278 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.278 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.278 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.278 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.278 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.278 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.279 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.279 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.279 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.279 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-09-19T04:57:02.279 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-09-19T04:57:02.279 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.279 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.279 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.279 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-09-19T04:57:02.279 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.279 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.279 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.279 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-09-19T04:57:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-09-19T04:57:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8080", 2024-09-19T04:57:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.280 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-09-19T04:57:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-09-19T04:57:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.281 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.282 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.282 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.282 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.282 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl_server_port": { 2024-09-19T04:57:02.282 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl_server_port", 2024-09-19T04:57:02.282 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.282 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.282 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.282 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8443", 2024-09-19T04:57:02.282 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.282 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.282 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.282 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.283 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.283 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.283 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.283 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.283 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-09-19T04:57:02.283 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-09-19T04:57:02.283 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.283 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.283 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.283 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "redirect", 2024-09-19T04:57:02.283 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.283 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.283 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.284 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.284 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect" 2024-09-19T04:57:02.284 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.284 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.284 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.284 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.284 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.284 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-09-19T04:57:02.284 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-09-19T04:57:02.284 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.284 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.284 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.285 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-09-19T04:57:02.285 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-09-19T04:57:02.285 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-09-19T04:57:02.285 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.285 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.285 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.285 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.285 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.285 INFO:teuthology.orchestra.run.smithi123.stdout: "url_prefix": { 2024-09-19T04:57:02.285 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url_prefix", 2024-09-19T04:57:02.285 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.285 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.286 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.286 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.286 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.286 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.286 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.286 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.286 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.286 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.286 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.286 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.286 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.286 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.287 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "devicehealth", 2024-09-19T04:57:02.287 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.287 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.287 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.287 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_monitoring": { 2024-09-19T04:57:02.287 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_monitoring", 2024-09-19T04:57:02.287 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.287 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.287 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.287 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.287 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.287 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.287 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.287 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "monitor device health metrics", 2024-09-19T04:57:02.288 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.288 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.288 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.288 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.288 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.288 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.288 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.288 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.288 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.288 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.288 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.288 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.289 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.289 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.289 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.289 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.289 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.289 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.289 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.289 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.289 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.289 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.289 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.289 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.289 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.290 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.290 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.290 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.290 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.290 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.290 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.290 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.290 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.290 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.290 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.290 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.290 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.290 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.291 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.291 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.291 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.291 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.291 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.291 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.291 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.291 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.291 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.291 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.291 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.291 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.291 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.292 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.292 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.292 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.292 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.292 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.292 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.292 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.292 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.292 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.292 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.292 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.292 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.293 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.293 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.293 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.293 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.293 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.293 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.293 INFO:teuthology.orchestra.run.smithi123.stdout: "mark_out_threshold": { 2024-09-19T04:57:02.293 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mark_out_threshold", 2024-09-19T04:57:02.293 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.293 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.293 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.293 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2419200", 2024-09-19T04:57:02.294 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.294 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.294 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.294 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T04:57:02.294 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.294 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.294 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.294 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": { 2024-09-19T04:57:02.294 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_name", 2024-09-19T04:57:02.294 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.294 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.294 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.294 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "device_health_metrics", 2024-09-19T04:57:02.295 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.295 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.295 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.295 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T04:57:02.295 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.295 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.295 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.295 INFO:teuthology.orchestra.run.smithi123.stdout: "retention_period": { 2024-09-19T04:57:02.295 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retention_period", 2024-09-19T04:57:02.295 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.295 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.295 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.295 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15552000", 2024-09-19T04:57:02.295 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.296 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.296 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.296 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain device health metrics", 2024-09-19T04:57:02.296 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.296 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.296 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.296 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_frequency": { 2024-09-19T04:57:02.296 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_frequency", 2024-09-19T04:57:02.296 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.296 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.296 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.296 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-09-19T04:57:02.297 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.297 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.297 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.297 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T04:57:02.297 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.297 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.297 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.297 INFO:teuthology.orchestra.run.smithi123.stdout: "self_heal": { 2024-09-19T04:57:02.297 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "self_heal", 2024-09-19T04:57:02.297 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.297 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.298 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.298 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.298 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.298 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.298 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.298 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T04:57:02.298 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.298 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.298 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.298 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-09-19T04:57:02.298 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-09-19T04:57:02.298 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.299 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.299 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.299 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-09-19T04:57:02.299 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.299 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.299 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.299 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T04:57:02.299 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.299 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.299 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.299 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_threshold": { 2024-09-19T04:57:02.299 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_threshold", 2024-09-19T04:57:02.299 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.300 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.300 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.300 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7257600", 2024-09-19T04:57:02.300 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.300 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.300 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.300 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T04:57:02.300 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.300 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.300 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.300 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.300 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.301 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "diskprediction_local", 2024-09-19T04:57:02.301 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.301 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.301 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.301 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.301 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.301 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.301 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.301 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.301 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.301 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.301 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.301 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.302 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.302 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.302 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.302 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.302 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.302 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.302 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.302 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.302 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.302 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.302 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.302 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.302 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.303 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.303 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.303 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.303 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.303 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.303 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.303 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.303 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.303 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.303 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.303 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.303 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.304 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.304 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.304 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.304 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.304 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.304 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.304 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.304 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.304 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.304 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.304 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.304 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.304 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.305 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.305 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.305 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.305 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.305 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.305 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.305 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.305 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.305 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.305 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.305 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.305 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.306 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.306 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.306 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.306 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.306 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.306 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.306 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.306 INFO:teuthology.orchestra.run.smithi123.stdout: "predict_interval": { 2024-09-19T04:57:02.306 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predict_interval", 2024-09-19T04:57:02.306 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.306 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.306 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.306 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-09-19T04:57:02.307 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.307 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.307 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.307 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.307 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.307 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.307 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.307 INFO:teuthology.orchestra.run.smithi123.stdout: "predictor_model": { 2024-09-19T04:57:02.307 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predictor_model", 2024-09-19T04:57:02.307 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.307 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.307 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.308 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "prophetstor", 2024-09-19T04:57:02.308 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.308 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.308 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.308 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.308 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.308 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.308 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.308 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-09-19T04:57:02.308 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-09-19T04:57:02.308 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.308 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.309 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.309 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-09-19T04:57:02.309 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.309 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.309 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.309 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.309 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.309 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.309 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.309 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.309 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.310 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "influx", 2024-09-19T04:57:02.310 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": false, 2024-09-19T04:57:02.310 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "influxdb python module not found", 2024-09-19T04:57:02.310 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.310 INFO:teuthology.orchestra.run.smithi123.stdout: "batch_size": { 2024-09-19T04:57:02.310 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "batch_size", 2024-09-19T04:57:02.310 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.310 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.310 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.310 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5000", 2024-09-19T04:57:02.310 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.310 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.310 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.311 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T04:57:02.311 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.311 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.311 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.311 INFO:teuthology.orchestra.run.smithi123.stdout: "database": { 2024-09-19T04:57:02.311 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "database", 2024-09-19T04:57:02.311 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.311 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.311 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.311 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "ceph", 2024-09-19T04:57:02.311 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.311 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.312 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.312 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T04:57:02.312 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.312 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.312 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.312 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": { 2024-09-19T04:57:02.312 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hostname", 2024-09-19T04:57:02.312 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.312 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.312 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.312 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.312 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.313 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.313 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.313 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server hostname", 2024-09-19T04:57:02.313 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.313 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.313 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.313 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-09-19T04:57:02.313 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-09-19T04:57:02.313 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.313 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.313 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.313 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "30", 2024-09-19T04:57:02.314 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "5", 2024-09-19T04:57:02.314 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.314 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.314 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T04:57:02.314 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.314 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.314 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.314 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.314 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.314 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.314 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.314 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.315 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.315 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.315 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.315 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.315 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.315 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.315 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.315 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.315 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.315 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.315 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.315 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.315 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.315 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.316 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.316 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.316 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.316 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.316 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.316 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.316 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.316 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.316 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.316 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.316 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.316 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.317 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.317 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.317 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.317 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.317 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.317 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.317 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.317 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.317 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.317 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.317 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.317 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.318 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.318 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.318 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.318 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.318 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.318 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.318 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.318 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.318 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.318 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.318 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.318 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.319 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.319 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.319 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.319 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.319 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.319 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.319 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.319 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.319 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.319 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.319 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.319 INFO:teuthology.orchestra.run.smithi123.stdout: "password": { 2024-09-19T04:57:02.319 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "password", 2024-09-19T04:57:02.320 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.320 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.320 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.320 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.320 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.320 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.320 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.320 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "password of InfluxDB server user", 2024-09-19T04:57:02.320 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.320 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.320 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.320 INFO:teuthology.orchestra.run.smithi123.stdout: "port": { 2024-09-19T04:57:02.321 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "port", 2024-09-19T04:57:02.321 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.321 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.321 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.321 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8086", 2024-09-19T04:57:02.321 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.321 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.321 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.321 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server port", 2024-09-19T04:57:02.321 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.321 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.321 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.321 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-09-19T04:57:02.322 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-09-19T04:57:02.322 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.322 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.322 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.322 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "false", 2024-09-19T04:57:02.322 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.322 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.322 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.322 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:57:02.322 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.322 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.322 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.322 INFO:teuthology.orchestra.run.smithi123.stdout: "threads": { 2024-09-19T04:57:02.323 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threads", 2024-09-19T04:57:02.323 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.323 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.323 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.323 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-09-19T04:57:02.323 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-09-19T04:57:02.323 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "32", 2024-09-19T04:57:02.323 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.323 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T04:57:02.323 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.323 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.323 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.323 INFO:teuthology.orchestra.run.smithi123.stdout: "username": { 2024-09-19T04:57:02.324 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "username", 2024-09-19T04:57:02.324 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.324 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.324 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.324 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.324 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.324 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.324 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.324 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "username of InfluxDB server user", 2024-09-19T04:57:02.324 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.324 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.324 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.324 INFO:teuthology.orchestra.run.smithi123.stdout: "verify_ssl": { 2024-09-19T04:57:02.325 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "verify_ssl", 2024-09-19T04:57:02.325 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.325 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.325 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.325 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "true", 2024-09-19T04:57:02.325 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.325 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.325 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.325 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:57:02.325 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.325 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.325 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.325 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.326 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.326 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "insights", 2024-09-19T04:57:02.326 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.326 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.326 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.326 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.326 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.326 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.326 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.326 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.327 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.327 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.327 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.327 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.327 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.327 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.327 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.327 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.327 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.327 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.327 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.327 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.327 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.328 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.328 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.328 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.328 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.328 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.328 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.328 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.328 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.328 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.328 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.328 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.328 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.329 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.329 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.329 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.329 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.329 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.329 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.329 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.329 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.329 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.329 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.329 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.329 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.330 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.330 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.330 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.330 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.330 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.330 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.330 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.330 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.330 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.330 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.330 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.330 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.330 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.331 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.331 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.331 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.331 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.331 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.331 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.331 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.331 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.331 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.331 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.331 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.331 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.331 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.332 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "iostat", 2024-09-19T04:57:02.332 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.332 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.332 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.332 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.332 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.332 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.332 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.332 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.332 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.332 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.333 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.333 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.333 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.333 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.333 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.333 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.333 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.333 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.333 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.333 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.333 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.333 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.333 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.334 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.334 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.334 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.334 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.334 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.334 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.334 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.334 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.334 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.334 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.334 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.334 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.334 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.335 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.335 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.335 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.335 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.335 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.335 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.335 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.335 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.335 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.335 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.335 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.335 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.336 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.336 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.336 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.336 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.336 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.336 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.336 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.336 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.336 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.336 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.336 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.336 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.336 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.337 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.337 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.337 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.337 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.337 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.337 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.337 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.337 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.337 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.337 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.337 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "k8sevents", 2024-09-19T04:57:02.337 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.337 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.338 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.338 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_event_retention_days": { 2024-09-19T04:57:02.338 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ceph_event_retention_days", 2024-09-19T04:57:02.338 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.338 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.338 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.338 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7", 2024-09-19T04:57:02.338 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.338 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.338 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.338 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T04:57:02.338 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.338 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.338 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.339 INFO:teuthology.orchestra.run.smithi123.stdout: "config_check_secs": { 2024-09-19T04:57:02.339 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_check_secs", 2024-09-19T04:57:02.339 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.339 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.339 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.339 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:02.339 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "10", 2024-09-19T04:57:02.339 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.339 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.339 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T04:57:02.339 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.340 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.340 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.340 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.340 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.340 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.340 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.340 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.340 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.340 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.340 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.340 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.340 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.340 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.341 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.341 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.341 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.341 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.341 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.341 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.341 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.341 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.341 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.341 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.341 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.341 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.342 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.342 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.342 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.342 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.342 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.342 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.342 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.342 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.342 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.342 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.342 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.343 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.343 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.343 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.343 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.343 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.343 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.343 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.343 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.343 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.343 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.343 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.343 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.343 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.343 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.344 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.344 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.344 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.344 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.344 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.344 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.344 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.344 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.344 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.344 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.344 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.344 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.345 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.345 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.345 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.345 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.345 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.345 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.345 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.345 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.345 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "localpool", 2024-09-19T04:57:02.345 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.346 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.346 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.346 INFO:teuthology.orchestra.run.smithi123.stdout: "failure_domain": { 2024-09-19T04:57:02.346 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "failure_domain", 2024-09-19T04:57:02.346 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.346 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.346 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.346 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "host", 2024-09-19T04:57:02.346 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.346 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.346 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.346 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "failure domain for any created local pool", 2024-09-19T04:57:02.346 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T04:57:02.346 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.347 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.347 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.347 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.347 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.347 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.347 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.347 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.347 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.347 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.347 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.347 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.347 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.347 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.348 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.348 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.348 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.348 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.348 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.348 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.348 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.348 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.348 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.348 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.348 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.348 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.348 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.349 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.349 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.349 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.349 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.349 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.349 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.349 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.349 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.349 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.349 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.349 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.349 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.349 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.350 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.350 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.350 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.350 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.350 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.350 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.350 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.350 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.350 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.350 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.350 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.350 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.350 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.350 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.350 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.351 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.351 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.351 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.351 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.351 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.351 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.351 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.351 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.351 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.351 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.351 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.351 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.352 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.352 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": { 2024-09-19T04:57:02.352 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_size", 2024-09-19T04:57:02.352 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.352 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.352 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.352 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.352 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.352 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.352 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.352 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default min_size for any created local pool", 2024-09-19T04:57:02.352 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T04:57:02.353 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.353 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.353 INFO:teuthology.orchestra.run.smithi123.stdout: "num_rep": { 2024-09-19T04:57:02.353 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "num_rep", 2024-09-19T04:57:02.353 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.353 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.353 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.353 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3", 2024-09-19T04:57:02.353 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.353 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.353 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.353 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default replica count for any created local pool", 2024-09-19T04:57:02.354 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.354 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.354 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.354 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": { 2024-09-19T04:57:02.354 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_num", 2024-09-19T04:57:02.354 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.354 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.354 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.354 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "128", 2024-09-19T04:57:02.354 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.354 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.354 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.355 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T04:57:02.355 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.355 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.355 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.355 INFO:teuthology.orchestra.run.smithi123.stdout: "prefix": { 2024-09-19T04:57:02.355 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prefix", 2024-09-19T04:57:02.355 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.355 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.355 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.355 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.355 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.355 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.356 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.356 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name prefix for any created local pool", 2024-09-19T04:57:02.356 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.356 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.356 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.356 INFO:teuthology.orchestra.run.smithi123.stdout: "subtree": { 2024-09-19T04:57:02.356 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "subtree", 2024-09-19T04:57:02.356 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.356 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.356 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.356 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "rack", 2024-09-19T04:57:02.356 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.356 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.357 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.357 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T04:57:02.357 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T04:57:02.357 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.357 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.357 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.357 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.357 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mds_autoscaler", 2024-09-19T04:57:02.357 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.357 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.357 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.357 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.358 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.358 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.358 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.358 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.358 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.358 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.358 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.358 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.358 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.358 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.358 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.358 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.358 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.359 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.359 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.359 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.359 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.359 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.359 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.359 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.360 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.360 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.360 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.360 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.360 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.360 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.360 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.360 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.360 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.360 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.360 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.360 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.361 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.361 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.361 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.361 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.361 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.361 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.361 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.361 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.361 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.361 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.361 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.361 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.361 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.361 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.362 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.362 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.362 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.362 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.362 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.362 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.362 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.362 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.362 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.362 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.362 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.363 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.363 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.363 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.363 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.363 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.363 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.363 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.363 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.363 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.363 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.363 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirroring", 2024-09-19T04:57:02.363 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.364 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.364 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.364 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.364 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.364 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.364 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.364 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.364 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.364 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.364 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.364 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.364 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.364 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.365 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.365 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.365 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.365 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.365 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.365 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.365 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.365 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.365 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.365 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.365 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.365 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.366 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.366 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.366 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.366 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.366 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.366 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.366 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.366 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.366 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.366 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.366 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.366 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.366 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.367 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.367 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.367 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.367 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.367 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.367 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.367 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.367 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.367 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.367 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.367 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.367 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.367 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.367 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "nfs", 2024-09-19T04:57:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.371 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.371 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.371 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.371 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.371 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.371 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.371 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.371 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.371 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.371 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.371 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.371 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.371 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.372 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.372 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.372 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.372 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.372 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.372 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.372 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.372 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.372 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.372 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.372 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.372 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.373 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.373 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.373 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.373 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.373 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.373 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.373 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.373 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.373 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.373 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.373 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.373 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.374 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.375 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.375 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.375 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.375 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-09-19T04:57:02.375 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.375 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.375 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.375 INFO:teuthology.orchestra.run.smithi123.stdout: "fail_fs": { 2024-09-19T04:57:02.375 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "fail_fs", 2024-09-19T04:57:02.375 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.375 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.375 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.376 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.376 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.376 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.376 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.376 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T04:57:02.376 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.376 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.376 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.376 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.376 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.376 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.376 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.377 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.377 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.377 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.377 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.377 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.377 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.377 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.377 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.377 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.377 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.377 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.377 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.377 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.377 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.378 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.378 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.378 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.378 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.378 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.378 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.378 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.378 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.378 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.378 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.378 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.378 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.379 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.379 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.379 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.379 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.379 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.379 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.379 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.379 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.379 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.379 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.379 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.379 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.379 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.379 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.380 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.380 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.380 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.380 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.380 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.380 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.380 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.380 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.380 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.380 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.380 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.380 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.380 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.381 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.381 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.381 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.381 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.381 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.381 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.381 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.381 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.381 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.381 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator": { 2024-09-19T04:57:02.381 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-09-19T04:57:02.381 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.381 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.382 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.382 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.382 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.382 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.382 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.382 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm", 2024-09-19T04:57:02.382 INFO:teuthology.orchestra.run.smithi123.stdout: "rook", 2024-09-19T04:57:02.382 INFO:teuthology.orchestra.run.smithi123.stdout: "test_orchestrator" 2024-09-19T04:57:02.382 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.382 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Orchestrator backend", 2024-09-19T04:57:02.382 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.382 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.382 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_perf_query", 2024-09-19T04:57:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_support", 2024-09-19T04:57:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.394 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.394 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.394 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.394 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.394 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.394 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.394 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.394 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.394 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.394 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.394 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.394 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.394 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.394 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.395 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_autoscaler", 2024-09-19T04:57:02.395 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.395 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.395 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.395 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.395 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.395 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.395 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.395 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.395 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.395 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.395 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.395 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.395 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.395 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.396 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.396 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.396 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.396 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.396 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.396 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.396 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.396 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.396 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.396 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.396 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.396 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.396 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.397 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.397 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.397 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.397 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.397 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.397 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.397 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.397 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.397 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.397 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.397 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.397 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.398 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.398 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.398 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.398 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.398 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.398 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.398 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.398 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.398 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.398 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.398 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.398 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.398 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.398 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.399 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.399 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.399 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.399 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.399 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.399 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.399 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.399 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.399 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.399 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.399 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.399 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.399 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.399 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.399 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.400 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.400 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.400 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-09-19T04:57:02.400 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-09-19T04:57:02.400 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.400 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.400 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.400 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-09-19T04:57:02.400 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.400 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.400 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.400 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.401 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.401 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.401 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.401 INFO:teuthology.orchestra.run.smithi123.stdout: "threshold": { 2024-09-19T04:57:02.401 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threshold", 2024-09-19T04:57:02.401 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:02.401 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.401 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.401 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-09-19T04:57:02.401 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1.0", 2024-09-19T04:57:02.401 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.401 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.401 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "scaling threshold", 2024-09-19T04:57:02.401 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T04:57:02.402 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.402 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.402 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.402 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.402 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.402 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "progress", 2024-09-19T04:57:02.402 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.402 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.402 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.402 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_pg_recovery_event": { 2024-09-19T04:57:02.402 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_pg_recovery_event", 2024-09-19T04:57:02.402 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.402 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.403 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.403 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.403 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.403 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.403 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.403 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T04:57:02.403 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.403 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.403 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.403 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-09-19T04:57:02.403 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-09-19T04:57:02.403 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.404 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.404 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.404 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.404 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.404 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.404 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.404 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.404 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.404 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.404 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.404 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.404 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.404 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.404 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.405 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.405 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.405 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.405 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.405 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.405 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.405 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.405 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.405 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.405 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.405 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.405 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.405 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.405 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.405 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.406 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.406 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.406 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.406 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.406 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.406 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.406 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.406 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.406 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.406 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.406 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.406 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.406 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.407 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.407 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.407 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.407 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.407 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.407 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.407 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.407 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.407 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.407 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.407 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.407 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.407 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.408 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.408 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.408 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.408 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.408 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.408 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.408 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.408 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.408 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.408 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.408 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.408 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.408 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: "max_completed_events": { 2024-09-19T04:57:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_completed_events", 2024-09-19T04:57:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.410 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "50", 2024-09-19T04:57:02.410 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.410 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.410 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.410 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "number of past completed events to remember", 2024-09-19T04:57:02.410 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.410 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.410 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.410 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-09-19T04:57:02.410 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-09-19T04:57:02.410 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.410 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.410 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.410 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-09-19T04:57:02.411 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.411 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.411 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.411 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long the module is going to sleep", 2024-09-19T04:57:02.411 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.411 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.411 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.411 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.411 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.411 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus", 2024-09-19T04:57:02.411 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.412 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.412 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.412 INFO:teuthology.orchestra.run.smithi123.stdout: "cache": { 2024-09-19T04:57:02.412 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cache", 2024-09-19T04:57:02.412 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.412 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.412 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.412 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.412 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.412 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.412 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.412 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.412 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.412 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.413 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.413 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.413 INFO:teuthology.orchestra.run.smithi123.stdout: "exclude_perf_counters": { 2024-09-19T04:57:02.413 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "exclude_perf_counters", 2024-09-19T04:57:02.413 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.413 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.413 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.413 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.413 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.413 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.413 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.413 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T04:57:02.413 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T04:57:02.414 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.414 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.414 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.414 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.414 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.414 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.414 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.414 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.414 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.414 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.414 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.414 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.414 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.414 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.415 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.415 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.415 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.415 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.415 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.415 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.415 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.415 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.415 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.415 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.415 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.415 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.416 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.416 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.416 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.416 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.416 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.416 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.416 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.416 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.416 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.416 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.416 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.416 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.417 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.417 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.417 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.417 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.417 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.417 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.417 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.417 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.417 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.417 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.417 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.417 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.417 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.418 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.418 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.418 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.418 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.418 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.418 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.418 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.418 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.418 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.418 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.418 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.418 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.418 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.418 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.419 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.419 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.419 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.419 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.419 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.419 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools": { 2024-09-19T04:57:02.419 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools", 2024-09-19T04:57:02.419 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.419 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.419 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.419 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.419 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.420 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.420 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.420 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.420 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.420 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.420 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.420 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.420 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T04:57:02.420 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T04:57:02.420 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.420 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.420 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.420 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "300", 2024-09-19T04:57:02.420 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.421 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.421 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.421 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.421 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.421 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.421 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.421 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_interval": { 2024-09-19T04:57:02.421 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_interval", 2024-09-19T04:57:02.421 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:02.421 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.421 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.421 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15.0", 2024-09-19T04:57:02.421 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.421 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-09-19T04:57:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-09-19T04:57:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-09-19T04:57:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.422 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T04:57:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-09-19T04:57:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-09-19T04:57:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "9283", 2024-09-19T04:57:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.423 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.424 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.424 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T04:57:02.424 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.424 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.424 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.424 INFO:teuthology.orchestra.run.smithi123.stdout: "stale_cache_strategy": { 2024-09-19T04:57:02.424 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stale_cache_strategy", 2024-09-19T04:57:02.424 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.424 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.424 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.424 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "log", 2024-09-19T04:57:02.425 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.425 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.425 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.425 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.425 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.425 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.425 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.425 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-09-19T04:57:02.425 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-09-19T04:57:02.425 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.425 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.425 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.425 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "default", 2024-09-19T04:57:02.426 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.426 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.426 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.426 INFO:teuthology.orchestra.run.smithi123.stdout: "default", 2024-09-19T04:57:02.426 INFO:teuthology.orchestra.run.smithi123.stdout: "error" 2024-09-19T04:57:02.426 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.426 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.426 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.426 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.426 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.426 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-09-19T04:57:02.427 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-09-19T04:57:02.427 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.427 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.427 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.427 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-09-19T04:57:02.427 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-09-19T04:57:02.427 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-09-19T04:57:02.427 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.427 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.427 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.427 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.427 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.427 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.427 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.428 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_support", 2024-09-19T04:57:02.428 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.428 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.428 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.428 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.428 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.428 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.428 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.428 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.428 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.429 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.429 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.429 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.429 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.429 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.429 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.429 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.429 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.429 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.429 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.429 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.429 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.429 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.430 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.430 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.430 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.430 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.430 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.430 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.430 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.430 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.430 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.430 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.430 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.430 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.430 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.431 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.431 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.431 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.431 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.431 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.431 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.431 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.431 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.431 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.431 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.431 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.431 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.432 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.432 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.432 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.432 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.432 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.432 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.432 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.432 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.432 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.432 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.432 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.432 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.432 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.433 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.433 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.433 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.433 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.433 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.433 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.433 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.433 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.433 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.433 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_snap_create": { 2024-09-19T04:57:02.433 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_snap_create", 2024-09-19T04:57:02.434 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.434 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.434 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.434 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:02.434 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.434 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.434 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.434 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.434 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.434 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.434 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.434 INFO:teuthology.orchestra.run.smithi123.stdout: "mirror_snapshot_schedule": { 2024-09-19T04:57:02.434 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T04:57:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "trash_purge_schedule": { 2024-09-19T04:57:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "trash_purge_schedule", 2024-09-19T04:57:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "restful", 2024-09-19T04:57:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_auth": { 2024-09-19T04:57:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_auth", 2024-09-19T04:57:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-09-19T04:57:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-09-19T04:57:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-09-19T04:57:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-09-19T04:57:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-09-19T04:57:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-09-19T04:57:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rgw", 2024-09-19T04:57:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T04:57:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T04:57:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-09-19T04:57:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T04:57:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rook", 2024-09-19T04:57:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "storage_class": { 2024-09-19T04:57:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "storage_class", 2024-09-19T04:57:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "local", 2024-09-19T04:57:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T04:57:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "selftest", 2024-09-19T04:57:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "roption1": { 2024-09-19T04:57:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption1", 2024-09-19T04:57:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "roption2": { 2024-09-19T04:57:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption2", 2024-09-19T04:57:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "xyz", 2024-09-19T04:57:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption1": { 2024-09-19T04:57:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption1", 2024-09-19T04:57:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption2": { 2024-09-19T04:57:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption2", 2024-09-19T04:57:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption3": { 2024-09-19T04:57:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption3", 2024-09-19T04:57:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption4": { 2024-09-19T04:57:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption4", 2024-09-19T04:57:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption5": { 2024-09-19T04:57:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption5", 2024-09-19T04:57:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption6": { 2024-09-19T04:57:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption6", 2024-09-19T04:57:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption7": { 2024-09-19T04:57:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption7", 2024-09-19T04:57:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-09-19T04:57:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "42", 2024-09-19T04:57:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "testkey": { 2024-09-19T04:57:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testkey", 2024-09-19T04:57:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "testlkey": { 2024-09-19T04:57:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testlkey", 2024-09-19T04:57:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "testnewline": { 2024-09-19T04:57:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testnewline", 2024-09-19T04:57:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snap_schedule", 2024-09-19T04:57:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_m_granularity": { 2024-09-19T04:57:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_m_granularity", 2024-09-19T04:57:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T04:57:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "dump_on_update": { 2024-09-19T04:57:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dump_on_update", 2024-09-19T04:57:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "dump database to debug log on update", 2024-09-19T04:57:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stats", 2024-09-19T04:57:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "status", 2024-09-19T04:57:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.495 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.495 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.495 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.495 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.495 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.495 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.495 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.495 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.495 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.495 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.495 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.495 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.496 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.496 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.496 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.496 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.496 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.496 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.496 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.496 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.496 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.496 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.496 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.496 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.496 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.497 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.497 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.497 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.497 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.497 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.497 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.497 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.497 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.497 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.497 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.497 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.497 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.497 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.498 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.498 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.498 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.498 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.498 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.498 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.498 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.498 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telegraf", 2024-09-19T04:57:02.498 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "address": { 2024-09-19T04:57:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "address", 2024-09-19T04:57:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T04:57:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-09-19T04:57:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-09-19T04:57:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15", 2024-09-19T04:57:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telemetry", 2024-09-19T04:57:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_basic": { 2024-09-19T04:57:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_basic", 2024-09-19T04:57:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T04:57:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_crash": { 2024-09-19T04:57:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_crash", 2024-09-19T04:57:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T04:57:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_device": { 2024-09-19T04:57:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_device", 2024-09-19T04:57:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T04:57:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_ident": { 2024-09-19T04:57:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_ident", 2024-09-19T04:57:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T04:57:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_perf": { 2024-09-19T04:57:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_perf", 2024-09-19T04:57:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T04:57:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "contact": { 2024-09-19T04:57:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "contact", 2024-09-19T04:57:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "description": { 2024-09-19T04:57:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "description", 2024-09-19T04:57:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "device_url": { 2024-09-19T04:57:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_url", 2024-09-19T04:57:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T04:57:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-09-19T04:57:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-09-19T04:57:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-09-19T04:57:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-09-19T04:57:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "24", 2024-09-19T04:57:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "8", 2024-09-19T04:57:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "last_opt_revision": { 2024-09-19T04:57:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "last_opt_revision", 2024-09-19T04:57:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1", 2024-09-19T04:57:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard": { 2024-09-19T04:57:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard", 2024-09-19T04:57:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard_description": { 2024-09-19T04:57:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard_description", 2024-09-19T04:57:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "organization": { 2024-09-19T04:57:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "organization", 2024-09-19T04:57:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "proxy": { 2024-09-19T04:57:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "proxy", 2024-09-19T04:57:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "url": { 2024-09-19T04:57:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url", 2024-09-19T04:57:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T04:57:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "test_orchestrator", 2024-09-19T04:57:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "volumes", 2024-09-19T04:57:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_clones": { 2024-09-19T04:57:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_clones", 2024-09-19T04:57:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.540 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4", 2024-09-19T04:57:02.540 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.540 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.540 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.540 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T04:57:02.540 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.540 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.540 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.540 INFO:teuthology.orchestra.run.smithi123.stdout: "periodic_async_work": { 2024-09-19T04:57:02.540 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "periodic_async_work", 2024-09-19T04:57:02.540 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.540 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.541 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.541 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.541 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.541 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.541 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.541 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Periodically check for async work", 2024-09-19T04:57:02.541 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.541 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.541 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.541 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_delay": { 2024-09-19T04:57:02.541 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_delay", 2024-09-19T04:57:02.541 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.542 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.542 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.542 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-09-19T04:57:02.542 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.542 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.542 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.542 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T04:57:02.542 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.542 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.542 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.542 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_no_wait": { 2024-09-19T04:57:02.542 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T04:57:02.542 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.543 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.543 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.543 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.543 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.543 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.543 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.543 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T04:57:02.543 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.543 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.543 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.543 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.543 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.544 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix", 2024-09-19T04:57:02.544 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.544 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.544 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.544 INFO:teuthology.orchestra.run.smithi123.stdout: "discovery_interval": { 2024-09-19T04:57:02.544 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "discovery_interval", 2024-09-19T04:57:02.544 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-09-19T04:57:02.544 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.544 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.544 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "100", 2024-09-19T04:57:02.544 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.544 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.545 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.545 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.545 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.545 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.545 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.545 INFO:teuthology.orchestra.run.smithi123.stdout: "identifier": { 2024-09-19T04:57:02.545 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "identifier", 2024-09-19T04:57:02.545 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.545 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.545 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.545 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.545 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.546 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.546 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.546 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.546 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.546 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.546 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.546 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-09-19T04:57:02.546 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-09-19T04:57:02.546 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.546 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.546 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.546 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-09-19T04:57:02.547 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.547 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.547 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.547 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.547 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.547 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.547 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.547 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.547 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.547 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.547 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.547 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.548 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.548 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.548 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.548 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.548 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.548 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.548 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.548 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.548 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.548 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.548 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.548 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.548 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.549 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.549 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.549 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.549 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.549 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.549 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.549 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.549 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.549 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.549 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.549 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.549 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.550 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.551 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.552 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.552 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.552 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.552 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.552 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.552 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.552 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.552 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.552 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.552 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.552 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_host": { 2024-09-19T04:57:02.552 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_host", 2024-09-19T04:57:02.552 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.553 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.553 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.553 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.553 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.553 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.553 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.553 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.553 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.553 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.553 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.553 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_port": { 2024-09-19T04:57:02.553 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_port", 2024-09-19T04:57:02.554 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.554 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.554 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.554 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10051", 2024-09-19T04:57:02.554 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.554 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.554 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.554 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.554 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.554 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.554 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.554 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_sender": { 2024-09-19T04:57:02.555 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_sender", 2024-09-19T04:57:02.555 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.555 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.555 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.555 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T04:57:02.555 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.555 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.555 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.555 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.555 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.555 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.555 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.555 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.556 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.556 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.556 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T04:57:02.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.556 INFO:teuthology.orchestra.run.smithi123.stdout: "gid": 25138, 2024-09-19T04:57:02.556 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "z", 2024-09-19T04:57:02.556 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_features": 4540701547738038271, 2024-09-19T04:57:02.556 INFO:teuthology.orchestra.run.smithi123.stdout: "available_modules": [ 2024-09-19T04:57:02.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.556 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "alerts", 2024-09-19T04:57:02.556 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.556 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.556 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.556 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-09-19T04:57:02.557 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-09-19T04:57:02.557 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.557 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.557 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.557 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-09-19T04:57:02.557 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.557 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.557 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.557 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T04:57:02.557 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.557 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.557 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.557 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.558 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.558 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.558 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.558 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.558 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.558 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.558 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.558 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.558 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.558 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.558 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.558 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.558 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.558 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.559 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.559 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.559 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.559 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.559 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.559 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.559 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.559 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.559 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.559 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.560 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.560 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.560 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.560 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.560 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.560 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.560 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.560 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.560 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.560 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.561 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.561 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.561 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.561 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.561 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.561 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.561 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.561 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.561 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.561 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.561 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.561 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.561 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.561 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.562 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.562 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.562 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.562 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.562 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.562 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.562 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.562 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.562 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.562 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.562 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.562 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.563 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.563 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.563 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.563 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.563 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.563 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_destination": { 2024-09-19T04:57:02.563 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_destination", 2024-09-19T04:57:02.563 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.563 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.563 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.563 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.563 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.563 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.564 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.564 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email address to send alerts to", 2024-09-19T04:57:02.564 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.564 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.564 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.564 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_from_name": { 2024-09-19T04:57:02.564 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_from_name", 2024-09-19T04:57:02.564 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.564 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.564 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.564 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "Ceph", 2024-09-19T04:57:02.564 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.564 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.565 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.565 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email From: name", 2024-09-19T04:57:02.565 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.565 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.565 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.565 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_host": { 2024-09-19T04:57:02.565 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_host", 2024-09-19T04:57:02.565 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.565 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.565 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.565 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.565 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.565 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.566 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.566 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP server", 2024-09-19T04:57:02.566 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.566 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.566 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.566 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_password": { 2024-09-19T04:57:02.566 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_password", 2024-09-19T04:57:02.566 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.566 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.566 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.566 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.566 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.566 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.566 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Password to authenticate with", 2024-09-19T04:57:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_port": { 2024-09-19T04:57:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_port", 2024-09-19T04:57:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "465", 2024-09-19T04:57:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP port", 2024-09-19T04:57:02.568 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.568 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.568 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.568 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_sender": { 2024-09-19T04:57:02.568 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_sender", 2024-09-19T04:57:02.568 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.568 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.568 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.568 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.568 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.568 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.568 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.568 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP envelope sender", 2024-09-19T04:57:02.568 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.569 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.569 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.569 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_ssl": { 2024-09-19T04:57:02.569 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_ssl", 2024-09-19T04:57:02.569 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.569 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.569 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.569 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.569 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.569 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.569 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.569 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T04:57:02.569 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.570 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.570 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.570 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_user": { 2024-09-19T04:57:02.570 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_user", 2024-09-19T04:57:02.570 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.570 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.570 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.570 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.570 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.570 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.570 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.570 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "User to authenticate as", 2024-09-19T04:57:02.570 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.570 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.571 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.571 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.571 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.571 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "balancer", 2024-09-19T04:57:02.571 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.571 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.571 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.571 INFO:teuthology.orchestra.run.smithi123.stdout: "active": { 2024-09-19T04:57:02.571 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "active", 2024-09-19T04:57:02.571 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.571 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.571 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.572 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.572 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.572 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.572 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.572 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T04:57:02.572 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.572 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.572 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.572 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_time": { 2024-09-19T04:57:02.572 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_time", 2024-09-19T04:57:02.572 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.572 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.573 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.573 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0000", 2024-09-19T04:57:02.573 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.573 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.573 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.573 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T04:57:02.573 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:57:02.573 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.573 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.573 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_weekday": { 2024-09-19T04:57:02.573 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_weekday", 2024-09-19T04:57:02.573 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-09-19T04:57:02.574 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.574 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.574 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-09-19T04:57:02.574 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-09-19T04:57:02.574 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-09-19T04:57:02.574 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.574 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T04:57:02.574 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:57:02.574 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.574 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.574 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_max_iterations": { 2024-09-19T04:57:02.574 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_max_iterations", 2024-09-19T04:57:02.574 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-09-19T04:57:02.575 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.575 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.575 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "25", 2024-09-19T04:57:02.575 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-09-19T04:57:02.575 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "250", 2024-09-19T04:57:02.575 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.575 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T04:57:02.575 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.575 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.575 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.575 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_metrics": { 2024-09-19T04:57:02.575 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_metrics", 2024-09-19T04:57:02.575 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.576 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.576 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.576 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T04:57:02.576 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.576 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.576 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.576 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T04:57:02.576 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T04:57:02.576 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.576 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.576 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_step": { 2024-09-19T04:57:02.576 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_step", 2024-09-19T04:57:02.577 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:02.577 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.577 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.577 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.5", 2024-09-19T04:57:02.577 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0.001", 2024-09-19T04:57:02.577 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "0.999", 2024-09-19T04:57:02.577 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.577 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "aggressiveness of optimization", 2024-09-19T04:57:02.577 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T04:57:02.577 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.577 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.577 INFO:teuthology.orchestra.run.smithi123.stdout: "end_time": { 2024-09-19T04:57:02.578 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_time", 2024-09-19T04:57:02.578 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.578 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.578 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.578 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2359", 2024-09-19T04:57:02.578 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.578 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.578 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.578 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T04:57:02.578 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:57:02.578 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.578 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.578 INFO:teuthology.orchestra.run.smithi123.stdout: "end_weekday": { 2024-09-19T04:57:02.578 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_weekday", 2024-09-19T04:57:02.578 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-09-19T04:57:02.579 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.579 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.579 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-09-19T04:57:02.579 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-09-19T04:57:02.579 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-09-19T04:57:02.579 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.579 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T04:57:02.579 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:57:02.579 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.579 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.579 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.579 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.579 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.579 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.580 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.580 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.580 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.580 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.580 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.580 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.580 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.580 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.580 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.580 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.580 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.580 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.580 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.580 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.580 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.580 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.581 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.581 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.581 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.581 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.581 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.581 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.581 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.581 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.581 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.581 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.581 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.581 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.581 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.582 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.582 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.582 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.582 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.582 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.582 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.582 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.582 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.582 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.582 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.582 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.582 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.582 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.583 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.583 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.583 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.583 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.583 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.583 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.583 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.583 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.583 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.583 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.583 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.583 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.584 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.584 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.584 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.584 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.584 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.584 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.584 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.584 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.584 INFO:teuthology.orchestra.run.smithi123.stdout: "min_score": { 2024-09-19T04:57:02.584 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_score", 2024-09-19T04:57:02.584 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:02.584 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.584 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.584 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-09-19T04:57:02.585 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.585 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.585 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.585 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T04:57:02.585 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.585 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.585 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.585 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-09-19T04:57:02.585 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-09-19T04:57:02.585 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.585 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.585 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.585 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "upmap", 2024-09-19T04:57:02.585 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.586 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.586 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.586 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-compat", 2024-09-19T04:57:02.586 INFO:teuthology.orchestra.run.smithi123.stdout: "none", 2024-09-19T04:57:02.586 INFO:teuthology.orchestra.run.smithi123.stdout: "read", 2024-09-19T04:57:02.586 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap", 2024-09-19T04:57:02.586 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap-read" 2024-09-19T04:57:02.586 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.586 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Balancer mode", 2024-09-19T04:57:02.586 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.586 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.586 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.586 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_ids": { 2024-09-19T04:57:02.586 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_ids", 2024-09-19T04:57:02.587 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.587 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.587 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.587 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.587 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.587 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.587 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.587 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T04:57:02.587 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.587 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.587 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.587 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-09-19T04:57:02.588 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-09-19T04:57:02.588 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.588 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.588 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.588 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-09-19T04:57:02.588 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.588 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.588 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.588 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T04:57:02.588 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.588 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.588 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.588 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_deviation": { 2024-09-19T04:57:02.589 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_deviation", 2024-09-19T04:57:02.589 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.589 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.589 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.589 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-09-19T04:57:02.589 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-09-19T04:57:02.589 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.589 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.589 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T04:57:02.589 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T04:57:02.589 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.589 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.590 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_optimizations": { 2024-09-19T04:57:02.590 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_optimizations", 2024-09-19T04:57:02.590 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-09-19T04:57:02.590 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.590 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.590 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:02.590 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.590 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.590 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.590 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T04:57:02.590 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.590 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.590 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.590 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.591 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.591 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm", 2024-09-19T04:57:02.591 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.591 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.591 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.591 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_down_multiplier": { 2024-09-19T04:57:02.591 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_down_multiplier", 2024-09-19T04:57:02.591 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:02.591 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.591 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.591 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-09-19T04:57:02.591 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.591 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.591 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.592 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T04:57:02.592 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.592 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.592 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.592 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_refresh_rate": { 2024-09-19T04:57:02.592 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_refresh_rate", 2024-09-19T04:57:02.592 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.592 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.592 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.592 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "20", 2024-09-19T04:57:02.592 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.592 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.592 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.593 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T04:57:02.593 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.593 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.593 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.593 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_starting_port": { 2024-09-19T04:57:02.593 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_starting_port", 2024-09-19T04:57:02.593 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.593 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.593 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.593 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4721", 2024-09-19T04:57:02.593 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.593 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.594 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.594 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T04:57:02.594 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.594 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.594 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.594 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_ptrace": { 2024-09-19T04:57:02.594 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_ptrace", 2024-09-19T04:57:02.594 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.594 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.594 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.594 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.594 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.594 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.595 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.595 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T04:57:02.595 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T04:57:02.595 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.595 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.595 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_interval": { 2024-09-19T04:57:02.595 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_interval", 2024-09-19T04:57:02.595 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.595 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.595 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.596 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-09-19T04:57:02.596 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.596 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.596 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.596 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T04:57:02.596 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.596 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.596 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.596 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_memory_target_ratio": { 2024-09-19T04:57:02.596 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T04:57:02.596 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:02.597 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.597 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.597 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.7", 2024-09-19T04:57:02.597 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.597 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.597 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.597 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T04:57:02.597 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.597 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.597 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.597 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm_log_destination": { 2024-09-19T04:57:02.598 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm_log_destination", 2024-09-19T04:57:02.598 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.598 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.598 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.598 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.598 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.598 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.598 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.598 INFO:teuthology.orchestra.run.smithi123.stdout: "file", 2024-09-19T04:57:02.598 INFO:teuthology.orchestra.run.smithi123.stdout: "file,syslog", 2024-09-19T04:57:02.598 INFO:teuthology.orchestra.run.smithi123.stdout: "syslog" 2024-09-19T04:57:02.598 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.598 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T04:57:02.599 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.599 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.599 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.599 INFO:teuthology.orchestra.run.smithi123.stdout: "cgroups_split": { 2024-09-19T04:57:02.599 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cgroups_split", 2024-09-19T04:57:02.599 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.599 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.599 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.599 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.599 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.599 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.599 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.599 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T04:57:02.600 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.600 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.600 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.600 INFO:teuthology.orchestra.run.smithi123.stdout: "config_checks_enabled": { 2024-09-19T04:57:02.600 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_checks_enabled", 2024-09-19T04:57:02.600 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.600 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.600 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.600 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.600 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.600 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.600 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.600 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T04:57:02.601 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.601 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.601 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.601 INFO:teuthology.orchestra.run.smithi123.stdout: "config_dashboard": { 2024-09-19T04:57:02.601 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_dashboard", 2024-09-19T04:57:02.601 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.601 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.601 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.601 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.601 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.601 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.601 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.601 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T04:57:02.602 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.602 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.602 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.602 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_alertmanager": { 2024-09-19T04:57:02.602 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_alertmanager", 2024-09-19T04:57:02.602 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.602 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.602 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.602 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T04:57:02.602 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.602 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.602 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.602 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-09-19T04:57:02.603 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.603 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.603 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.603 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_base": { 2024-09-19T04:57:02.603 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_base", 2024-09-19T04:57:02.603 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.603 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.603 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.603 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T04:57:02.603 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.603 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.603 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.603 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Container image name, without the tag", 2024-09-19T04:57:02.604 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.604 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.604 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.604 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_elasticsearch": { 2024-09-19T04:57:02.604 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_elasticsearch", 2024-09-19T04:57:02.604 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.604 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.604 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.604 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T04:57:02.604 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.604 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.604 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.605 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "elasticsearch container image", 2024-09-19T04:57:02.605 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.605 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.605 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.605 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_grafana": { 2024-09-19T04:57:02.605 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_grafana", 2024-09-19T04:57:02.605 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.605 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.605 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.605 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T04:57:02.605 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.605 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.605 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.605 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-09-19T04:57:02.606 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.606 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.606 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.606 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_haproxy": { 2024-09-19T04:57:02.606 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_haproxy", 2024-09-19T04:57:02.606 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.606 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.606 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.606 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T04:57:02.606 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.606 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.606 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.607 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "HAproxy container image", 2024-09-19T04:57:02.607 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.607 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.607 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.607 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_agent": { 2024-09-19T04:57:02.607 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_agent", 2024-09-19T04:57:02.607 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.607 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.607 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.607 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T04:57:02.607 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.607 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.608 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.608 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger agent container image", 2024-09-19T04:57:02.608 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.608 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.608 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.608 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_collector": { 2024-09-19T04:57:02.608 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_collector", 2024-09-19T04:57:02.608 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.608 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.608 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.608 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T04:57:02.608 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.608 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.609 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.609 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger collector container image", 2024-09-19T04:57:02.609 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.609 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.609 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.609 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_query": { 2024-09-19T04:57:02.609 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_query", 2024-09-19T04:57:02.609 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.609 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.609 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.609 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T04:57:02.609 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.610 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.610 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.610 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger query container image", 2024-09-19T04:57:02.610 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.610 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.610 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.610 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_keepalived": { 2024-09-19T04:57:02.610 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_keepalived", 2024-09-19T04:57:02.610 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.610 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.610 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.611 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T04:57:02.611 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.611 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.611 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.611 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Keepalived container image", 2024-09-19T04:57:02.611 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.611 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.611 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.611 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_loki": { 2024-09-19T04:57:02.611 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_loki", 2024-09-19T04:57:02.611 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.611 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.611 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.611 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T04:57:02.612 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.612 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.612 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.612 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Loki container image", 2024-09-19T04:57:02.612 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.612 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.612 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.612 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_node_exporter": { 2024-09-19T04:57:02.612 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_node_exporter", 2024-09-19T04:57:02.612 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.612 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.612 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.613 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T04:57:02.613 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.613 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.613 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.613 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-09-19T04:57:02.613 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.613 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.613 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.613 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_nvmeof": { 2024-09-19T04:57:02.613 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_nvmeof", 2024-09-19T04:57:02.613 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.613 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.614 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.614 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T04:57:02.614 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.614 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.614 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.614 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Nvme-of container image", 2024-09-19T04:57:02.614 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.614 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.614 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.614 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_prometheus": { 2024-09-19T04:57:02.614 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_prometheus", 2024-09-19T04:57:02.614 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.615 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.615 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.615 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T04:57:02.615 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.615 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.615 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.615 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-09-19T04:57:02.615 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.615 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.615 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.615 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_promtail": { 2024-09-19T04:57:02.615 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_promtail", 2024-09-19T04:57:02.616 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.616 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.616 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.616 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T04:57:02.616 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.616 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.616 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.616 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Promtail container image", 2024-09-19T04:57:02.616 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.616 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.616 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.616 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_samba": { 2024-09-19T04:57:02.617 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_samba", 2024-09-19T04:57:02.617 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.617 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.617 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.617 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T04:57:02.617 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.617 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.617 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.617 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Samba/SMB container image", 2024-09-19T04:57:02.617 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.617 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.617 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.618 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_snmp_gateway": { 2024-09-19T04:57:02.618 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_snmp_gateway", 2024-09-19T04:57:02.618 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.618 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.618 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.618 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T04:57:02.618 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.618 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.618 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.618 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SNMP Gateway container image", 2024-09-19T04:57:02.618 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.618 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.618 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.619 INFO:teuthology.orchestra.run.smithi123.stdout: "container_init": { 2024-09-19T04:57:02.619 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_init", 2024-09-19T04:57:02.619 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.619 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.619 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.619 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.619 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.619 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.619 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.619 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T04:57:02.619 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.619 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.619 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.620 INFO:teuthology.orchestra.run.smithi123.stdout: "daemon_cache_timeout": { 2024-09-19T04:57:02.620 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "daemon_cache_timeout", 2024-09-19T04:57:02.620 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.620 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.620 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.620 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-09-19T04:57:02.620 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.620 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.620 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.620 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T04:57:02.620 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.620 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.621 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.621 INFO:teuthology.orchestra.run.smithi123.stdout: "default_cephadm_command_timeout": { 2024-09-19T04:57:02.621 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T04:57:02.621 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.621 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.621 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.621 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "900", 2024-09-19T04:57:02.621 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.621 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.621 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.621 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T04:57:02.621 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.621 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.622 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.622 INFO:teuthology.orchestra.run.smithi123.stdout: "default_registry": { 2024-09-19T04:57:02.622 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_registry", 2024-09-19T04:57:02.622 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.622 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.622 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.622 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io", 2024-09-19T04:57:02.622 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.622 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.622 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.622 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T04:57:02.622 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.623 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.623 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.623 INFO:teuthology.orchestra.run.smithi123.stdout: "device_cache_timeout": { 2024-09-19T04:57:02.623 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_cache_timeout", 2024-09-19T04:57:02.623 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.623 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.623 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.623 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1800", 2024-09-19T04:57:02.623 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.623 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.623 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.623 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache device inventory", 2024-09-19T04:57:02.623 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.623 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.624 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.624 INFO:teuthology.orchestra.run.smithi123.stdout: "device_enhanced_scan": { 2024-09-19T04:57:02.624 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_enhanced_scan", 2024-09-19T04:57:02.624 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.624 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.624 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.624 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.624 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.624 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.624 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.624 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T04:57:02.624 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.625 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.625 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.625 INFO:teuthology.orchestra.run.smithi123.stdout: "facts_cache_timeout": { 2024-09-19T04:57:02.625 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "facts_cache_timeout", 2024-09-19T04:57:02.625 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.625 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.625 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.625 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-09-19T04:57:02.625 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.625 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.625 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.625 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache host facts data", 2024-09-19T04:57:02.625 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.626 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.626 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.626 INFO:teuthology.orchestra.run.smithi123.stdout: "grafana_dashboards_path": { 2024-09-19T04:57:02.626 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "grafana_dashboards_path", 2024-09-19T04:57:02.626 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.626 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.626 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.626 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T04:57:02.626 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.626 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.626 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.626 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T04:57:02.627 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.627 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.627 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.627 INFO:teuthology.orchestra.run.smithi123.stdout: "host_check_interval": { 2024-09-19T04:57:02.627 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "host_check_interval", 2024-09-19T04:57:02.627 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.627 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.627 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.627 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-09-19T04:57:02.627 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.627 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.627 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.627 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to perform a host check", 2024-09-19T04:57:02.628 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.628 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.628 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.628 INFO:teuthology.orchestra.run.smithi123.stdout: "hw_monitoring": { 2024-09-19T04:57:02.628 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hw_monitoring", 2024-09-19T04:57:02.628 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.628 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.628 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.628 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.628 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.628 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.628 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.629 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T04:57:02.629 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.629 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.629 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.629 INFO:teuthology.orchestra.run.smithi123.stdout: "inventory_list_all": { 2024-09-19T04:57:02.629 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "inventory_list_all", 2024-09-19T04:57:02.629 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.629 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.629 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.629 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.629 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.630 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.630 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.630 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T04:57:02.630 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.630 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.630 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.630 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.630 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.630 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.630 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.630 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.630 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.630 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.631 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.631 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.631 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.631 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.631 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.631 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.631 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.631 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.631 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.631 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.631 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.631 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.632 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.632 INFO:teuthology.orchestra.run.smithi123.stdout: "log_refresh_metadata": { 2024-09-19T04:57:02.632 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_refresh_metadata", 2024-09-19T04:57:02.632 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.632 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.632 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.632 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.632 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.632 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.632 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.632 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T04:57:02.632 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T04:57:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T04:57:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T04:57:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T04:57:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.638 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T04:57:02.638 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T04:57:02.638 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.638 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.638 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.638 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "*", 2024-09-19T04:57:02.638 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.638 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.638 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.638 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T04:57:02.638 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.638 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.638 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.639 INFO:teuthology.orchestra.run.smithi123.stdout: "max_count_per_host": { 2024-09-19T04:57:02.639 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_count_per_host", 2024-09-19T04:57:02.639 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.639 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.639 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.639 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:02.639 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.639 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.639 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.639 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of daemons per service per host", 2024-09-19T04:57:02.639 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.639 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.639 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.640 INFO:teuthology.orchestra.run.smithi123.stdout: "max_osd_draining_count": { 2024-09-19T04:57:02.640 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_osd_draining_count", 2024-09-19T04:57:02.640 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.640 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.640 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.640 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:02.640 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.640 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.640 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.640 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T04:57:02.640 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.640 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.640 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.641 INFO:teuthology.orchestra.run.smithi123.stdout: "migration_current": { 2024-09-19T04:57:02.641 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "migration_current", 2024-09-19T04:57:02.641 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.641 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.641 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.641 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.641 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.641 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.641 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.641 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "internal - do not modify", 2024-09-19T04:57:02.641 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.641 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.641 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.642 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-09-19T04:57:02.642 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-09-19T04:57:02.642 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.642 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.642 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.642 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "root", 2024-09-19T04:57:02.642 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.642 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.642 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.642 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm-package", 2024-09-19T04:57:02.642 INFO:teuthology.orchestra.run.smithi123.stdout: "root" 2024-09-19T04:57:02.642 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.642 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T04:57:02.643 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.643 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.643 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.643 INFO:teuthology.orchestra.run.smithi123.stdout: "oob_default_addr": { 2024-09-19T04:57:02.643 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "oob_default_addr", 2024-09-19T04:57:02.643 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.643 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.643 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.643 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "169.254.1.1", 2024-09-19T04:57:02.643 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.643 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.643 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.644 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T04:57:02.644 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.644 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.644 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.644 INFO:teuthology.orchestra.run.smithi123.stdout: "prometheus_alerts_path": { 2024-09-19T04:57:02.644 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus_alerts_path", 2024-09-19T04:57:02.644 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.644 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.644 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.644 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T04:57:02.644 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.644 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.644 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.645 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T04:57:02.645 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.645 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.645 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.645 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_insecure": { 2024-09-19T04:57:02.645 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_insecure", 2024-09-19T04:57:02.645 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.645 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.645 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.645 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.645 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.645 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.646 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.646 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T04:57:02.646 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.646 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.646 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.646 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_password": { 2024-09-19T04:57:02.646 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_password", 2024-09-19T04:57:02.646 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.646 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.646 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.646 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.646 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.646 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.647 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.647 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T04:57:02.647 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.647 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.647 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.647 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_url": { 2024-09-19T04:57:02.647 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_url", 2024-09-19T04:57:02.647 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.647 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.647 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.647 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.647 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.647 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.647 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.648 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T04:57:02.648 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.648 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.648 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.648 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_username": { 2024-09-19T04:57:02.648 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_username", 2024-09-19T04:57:02.648 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.648 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.648 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.648 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.648 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.648 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.649 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.649 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T04:57:02.649 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.649 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.649 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.649 INFO:teuthology.orchestra.run.smithi123.stdout: "secure_monitoring_stack": { 2024-09-19T04:57:02.649 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "secure_monitoring_stack", 2024-09-19T04:57:02.649 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.649 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.649 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.649 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.649 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.650 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.650 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.650 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T04:57:02.650 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.650 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.650 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.650 INFO:teuthology.orchestra.run.smithi123.stdout: "service_discovery_port": { 2024-09-19T04:57:02.650 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "service_discovery_port", 2024-09-19T04:57:02.650 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.650 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.650 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.650 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8765", 2024-09-19T04:57:02.650 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.651 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.651 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.651 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "cephadm service discovery port", 2024-09-19T04:57:02.651 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.651 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.651 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.651 INFO:teuthology.orchestra.run.smithi123.stdout: "ssh_config_file": { 2024-09-19T04:57:02.651 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssh_config_file", 2024-09-19T04:57:02.651 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.651 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.651 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.651 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.652 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.652 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.652 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.652 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T04:57:02.652 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.652 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.652 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.652 INFO:teuthology.orchestra.run.smithi123.stdout: "use_agent": { 2024-09-19T04:57:02.652 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_agent", 2024-09-19T04:57:02.652 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.652 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.652 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.652 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.653 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.653 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.653 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.653 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T04:57:02.653 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.653 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.653 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.653 INFO:teuthology.orchestra.run.smithi123.stdout: "use_repo_digest": { 2024-09-19T04:57:02.653 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_repo_digest", 2024-09-19T04:57:02.653 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.653 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.653 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.654 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.654 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.654 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.654 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.654 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T04:57:02.654 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.654 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.654 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.654 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_failed_host_check": { 2024-09-19T04:57:02.654 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_failed_host_check", 2024-09-19T04:57:02.654 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.654 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.654 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.655 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.655 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.655 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.655 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.655 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T04:57:02.655 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.655 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.655 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.655 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_daemons": { 2024-09-19T04:57:02.655 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_daemons", 2024-09-19T04:57:02.655 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.655 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.655 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.655 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.656 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.656 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.656 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.656 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T04:57:02.656 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.656 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.656 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.656 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_hosts": { 2024-09-19T04:57:02.656 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_hosts", 2024-09-19T04:57:02.656 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.656 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.656 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.656 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.657 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.657 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.657 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.657 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T04:57:02.657 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.657 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.657 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.657 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.657 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.657 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crash", 2024-09-19T04:57:02.657 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.657 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.658 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.658 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.658 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.658 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.658 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.658 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.658 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.658 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.658 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.658 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.658 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.658 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.658 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.659 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.659 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.659 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.659 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.659 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.659 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.659 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.659 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.659 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.659 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.659 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.659 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.660 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.660 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.660 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.660 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.660 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.660 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.660 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.660 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.660 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.660 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.660 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.661 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.661 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.661 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.661 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.661 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.661 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.661 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.661 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.661 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.661 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.661 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.661 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.661 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.662 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.662 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.662 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.662 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.662 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.662 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.662 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.662 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.662 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.662 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.662 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.662 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.663 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.663 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.663 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.663 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.663 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.663 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.663 INFO:teuthology.orchestra.run.smithi123.stdout: "retain_interval": { 2024-09-19T04:57:02.663 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retain_interval", 2024-09-19T04:57:02.663 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.663 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.663 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.664 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "31536000", 2024-09-19T04:57:02.664 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.664 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.664 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.664 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T04:57:02.664 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.664 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.664 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.664 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_recent_interval": { 2024-09-19T04:57:02.664 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_recent_interval", 2024-09-19T04:57:02.664 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.664 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.665 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.665 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1209600", 2024-09-19T04:57:02.665 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.665 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.665 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.665 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T04:57:02.665 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.665 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.665 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.665 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.665 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.666 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dashboard", 2024-09-19T04:57:02.666 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.666 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.666 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.666 INFO:teuthology.orchestra.run.smithi123.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T04:57:02.666 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T04:57:02.666 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.666 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.666 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.666 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:02.666 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.666 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.666 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.666 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.667 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.667 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.667 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.667 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T04:57:02.667 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T04:57:02.667 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.667 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.667 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.667 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.667 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.667 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.667 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.668 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.668 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.668 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.668 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.668 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T04:57:02.668 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T04:57:02.668 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.668 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.668 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.668 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.668 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.668 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.668 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.669 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.669 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.669 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.669 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.669 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_ENABLED": { 2024-09-19T04:57:02.669 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T04:57:02.669 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.669 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.669 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.669 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.669 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.669 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.670 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.670 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.670 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.670 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.670 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.670 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T04:57:02.670 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T04:57:02.670 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.670 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.670 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.670 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.670 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.670 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.671 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.671 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.671 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.671 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.671 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.671 INFO:teuthology.orchestra.run.smithi123.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T04:57:02.671 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T04:57:02.671 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.671 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.671 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.671 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.671 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.671 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.672 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.672 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.672 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.672 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.672 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.672 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T04:57:02.672 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T04:57:02.672 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.672 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.672 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.672 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.672 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.673 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.673 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.673 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.673 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.673 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.673 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.673 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.673 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T04:57:02.673 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T04:57:02.673 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.673 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.673 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.674 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.674 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.674 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.674 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.674 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.674 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.674 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.674 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.674 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T04:57:02.674 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T04:57:02.674 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.674 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.674 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.675 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.675 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.675 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.675 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.675 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.675 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.675 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.675 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.675 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T04:57:02.675 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T04:57:02.675 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.675 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.675 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.676 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.676 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.676 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.676 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.676 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.676 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.676 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.676 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.676 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T04:57:02.676 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T04:57:02.676 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.676 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.677 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.677 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.677 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.677 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.677 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.677 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.677 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.677 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.677 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.681 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T04:57:02.681 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T04:57:02.681 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.681 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.681 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.681 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.681 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.681 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.682 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.682 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.682 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.682 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.682 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.682 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T04:57:02.682 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T04:57:02.682 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.682 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.682 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.682 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.682 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.682 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.683 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.683 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.683 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.683 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.683 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.683 INFO:teuthology.orchestra.run.smithi123.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T04:57:02.683 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T04:57:02.683 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.683 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.683 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.683 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.683 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T04:57:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T04:57:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-09-19T04:57:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T04:57:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T04:57:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_URL": { 2024-09-19T04:57:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_URL", 2024-09-19T04:57:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T04:57:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T04:57:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-09-19T04:57:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T04:57:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T04:57:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T04:57:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T04:57:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T04:57:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T04:57:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T04:57:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T04:57:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T04:57:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T04:57:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T04:57:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T04:57:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T04:57:02.695 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T04:57:02.695 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.695 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.695 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.695 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.695 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.695 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.695 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.695 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.695 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.695 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.695 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.695 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T04:57:02.695 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T04:57:02.696 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.696 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.696 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.696 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.696 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.696 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.696 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.696 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.696 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.696 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.696 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.696 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T04:57:02.697 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T04:57:02.697 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.697 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.697 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.697 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.697 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.697 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.697 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.697 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.697 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.697 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.697 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.697 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T04:57:02.698 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T04:57:02.698 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.698 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.698 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.698 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.698 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.698 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.698 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.698 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.698 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.698 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.698 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.698 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T04:57:02.699 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T04:57:02.699 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.699 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.699 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.699 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.699 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.699 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.699 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.699 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.699 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.699 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.699 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.699 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.699 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T04:57:02.700 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T04:57:02.700 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.700 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.700 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.700 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.700 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.700 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.700 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.700 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.700 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.700 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.700 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.701 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T04:57:02.701 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T04:57:02.701 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.701 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.701 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.701 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.701 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.701 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.701 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.701 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.701 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.701 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.701 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.701 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T04:57:02.702 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T04:57:02.702 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.702 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.702 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.702 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.702 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.702 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.702 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.702 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.702 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.702 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.702 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.703 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T04:57:02.703 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T04:57:02.703 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.703 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.703 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.703 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T04:57:02.703 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.703 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.703 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.703 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.703 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.703 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.703 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.704 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T04:57:02.704 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T04:57:02.704 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.704 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.704 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.704 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:02.704 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.704 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.704 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.704 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.704 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.704 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.704 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.705 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T04:57:02.705 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T04:57:02.705 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.705 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.705 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.705 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8", 2024-09-19T04:57:02.705 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.705 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.705 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.705 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.705 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.705 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.706 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.706 INFO:teuthology.orchestra.run.smithi123.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T04:57:02.706 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T04:57:02.706 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.706 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.706 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.706 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "45", 2024-09-19T04:57:02.706 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.706 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.706 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.706 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.706 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.706 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.706 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.707 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T04:57:02.707 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T04:57:02.707 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.707 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.707 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.707 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.707 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.707 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.707 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.707 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.707 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.707 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.708 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.708 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T04:57:02.708 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T04:57:02.708 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.708 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.708 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.708 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-09-19T04:57:02.708 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.708 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.708 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.708 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.708 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.708 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.709 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.709 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T04:57:02.709 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T04:57:02.709 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.709 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.709 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.709 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.709 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.709 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.709 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.709 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.709 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.709 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.710 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.710 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T04:57:02.710 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T04:57:02.710 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.710 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.710 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.710 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.710 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.710 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.710 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.710 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.710 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.710 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.711 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.711 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T04:57:02.711 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T04:57:02.711 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.711 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.711 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.711 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-09-19T04:57:02.711 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.711 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.711 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.711 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.711 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.712 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.712 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.712 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T04:57:02.712 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T04:57:02.712 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.712 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.712 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.712 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:02.712 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.712 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.712 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.712 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.712 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.713 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.713 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.713 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T04:57:02.713 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T04:57:02.713 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.713 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.713 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.713 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-09-19T04:57:02.713 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.713 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.713 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.713 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.713 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.714 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.714 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.714 INFO:teuthology.orchestra.run.smithi123.stdout: "cross_origin_url": { 2024-09-19T04:57:02.714 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cross_origin_url", 2024-09-19T04:57:02.714 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.714 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.714 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.714 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.714 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.714 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.714 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.714 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.714 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.715 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.715 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.715 INFO:teuthology.orchestra.run.smithi123.stdout: "crt_file": { 2024-09-19T04:57:02.715 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crt_file", 2024-09-19T04:57:02.715 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.715 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.715 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.715 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.715 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.715 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.715 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.715 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.715 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.716 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.716 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.716 INFO:teuthology.orchestra.run.smithi123.stdout: "debug": { 2024-09-19T04:57:02.716 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "debug", 2024-09-19T04:57:02.716 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.716 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.716 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.716 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.716 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.716 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.716 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.717 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable/disable debug options", 2024-09-19T04:57:02.717 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.717 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.717 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.717 INFO:teuthology.orchestra.run.smithi123.stdout: "jwt_token_ttl": { 2024-09-19T04:57:02.717 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "jwt_token_ttl", 2024-09-19T04:57:02.717 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.717 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.717 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.717 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "28800", 2024-09-19T04:57:02.717 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.717 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.717 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.718 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.718 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.718 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.718 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.718 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-09-19T04:57:02.718 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-09-19T04:57:02.718 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.718 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.718 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.718 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.718 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.718 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.719 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.719 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.719 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.719 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.719 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.719 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.719 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.719 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.719 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.719 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.719 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.719 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.719 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.720 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.720 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.720 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.720 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.720 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.720 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.720 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.720 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.720 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.720 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.720 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.720 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.721 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.721 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.721 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.721 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.721 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.721 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.721 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.721 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.721 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.721 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.721 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.721 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.721 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.722 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.722 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.722 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.722 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.722 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.722 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.722 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.722 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.722 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.722 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.722 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.722 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.722 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.723 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.723 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.723 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.723 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.723 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.723 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.723 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.723 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.723 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.723 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.723 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.723 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.723 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.724 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.724 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.724 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.724 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.724 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.724 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.724 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.724 INFO:teuthology.orchestra.run.smithi123.stdout: "motd": { 2024-09-19T04:57:02.724 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "motd", 2024-09-19T04:57:02.724 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.724 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.724 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.724 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.725 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.725 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.725 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.725 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "The message of the day", 2024-09-19T04:57:02.725 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.725 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.725 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.725 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect_resolve_ip_addr": { 2024-09-19T04:57:02.725 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T04:57:02.725 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.725 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.725 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.726 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.726 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.726 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.726 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.726 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.726 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.726 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.726 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.726 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-09-19T04:57:02.726 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-09-19T04:57:02.726 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.726 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.726 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.726 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-09-19T04:57:02.727 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.727 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.727 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.727 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.727 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.727 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.727 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.727 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-09-19T04:57:02.727 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-09-19T04:57:02.727 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.727 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.727 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.727 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8080", 2024-09-19T04:57:02.728 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.728 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.728 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.728 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.728 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.728 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.728 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.728 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-09-19T04:57:02.728 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-09-19T04:57:02.728 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.728 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.728 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.728 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.729 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.729 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.729 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.729 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.729 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.729 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.729 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.729 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl_server_port": { 2024-09-19T04:57:02.729 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl_server_port", 2024-09-19T04:57:02.729 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.729 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.729 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.729 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8443", 2024-09-19T04:57:02.730 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.730 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.730 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.730 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.730 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.730 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.730 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.730 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-09-19T04:57:02.730 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-09-19T04:57:02.730 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.730 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.730 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.731 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "redirect", 2024-09-19T04:57:02.731 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.731 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.731 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.731 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.731 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect" 2024-09-19T04:57:02.731 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.731 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.731 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.731 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.731 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.731 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-09-19T04:57:02.732 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-09-19T04:57:02.732 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.732 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.732 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.732 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-09-19T04:57:02.732 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-09-19T04:57:02.732 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-09-19T04:57:02.732 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.732 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.732 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.732 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.732 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.733 INFO:teuthology.orchestra.run.smithi123.stdout: "url_prefix": { 2024-09-19T04:57:02.733 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url_prefix", 2024-09-19T04:57:02.733 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.733 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.733 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.733 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.733 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.733 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.733 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.733 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.733 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.733 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.733 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "devicehealth", 2024-09-19T04:57:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_monitoring": { 2024-09-19T04:57:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_monitoring", 2024-09-19T04:57:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.735 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.735 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.735 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.735 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "monitor device health metrics", 2024-09-19T04:57:02.735 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.735 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.735 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.735 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.735 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.735 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.735 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.735 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.735 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.736 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.736 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.736 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.736 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.736 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.736 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.736 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.736 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.736 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.736 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.736 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.736 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.736 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.737 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.737 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.737 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.737 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.737 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.737 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.737 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.737 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.737 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.737 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.737 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.737 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.737 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.737 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.738 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.738 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.738 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.738 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.738 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.738 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.738 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.738 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.738 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.738 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.738 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.738 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.738 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.739 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.739 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.739 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.739 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.739 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.739 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.739 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.739 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.739 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.739 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.739 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.739 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.740 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.740 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.740 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.740 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.740 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.740 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.740 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.740 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.740 INFO:teuthology.orchestra.run.smithi123.stdout: "mark_out_threshold": { 2024-09-19T04:57:02.740 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mark_out_threshold", 2024-09-19T04:57:02.740 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.740 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.740 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.741 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2419200", 2024-09-19T04:57:02.741 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.741 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.741 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.741 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T04:57:02.741 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.741 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.741 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.741 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": { 2024-09-19T04:57:02.741 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_name", 2024-09-19T04:57:02.741 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.741 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.742 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.742 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "device_health_metrics", 2024-09-19T04:57:02.742 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.742 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.742 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.742 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T04:57:02.742 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.742 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.742 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.742 INFO:teuthology.orchestra.run.smithi123.stdout: "retention_period": { 2024-09-19T04:57:02.742 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retention_period", 2024-09-19T04:57:02.742 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.742 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.743 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.743 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15552000", 2024-09-19T04:57:02.743 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.743 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.743 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.743 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain device health metrics", 2024-09-19T04:57:02.743 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.743 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.743 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.743 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_frequency": { 2024-09-19T04:57:02.743 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_frequency", 2024-09-19T04:57:02.743 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.743 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.744 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.744 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-09-19T04:57:02.744 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.744 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.744 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.744 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T04:57:02.744 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.744 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.744 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.744 INFO:teuthology.orchestra.run.smithi123.stdout: "self_heal": { 2024-09-19T04:57:02.744 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "self_heal", 2024-09-19T04:57:02.744 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.745 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.745 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.745 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.745 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.745 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.745 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.745 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T04:57:02.745 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.745 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.745 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.745 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-09-19T04:57:02.745 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-09-19T04:57:02.745 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.746 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.746 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.746 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-09-19T04:57:02.746 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.746 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.746 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.746 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T04:57:02.746 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.746 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.746 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.746 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_threshold": { 2024-09-19T04:57:02.746 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_threshold", 2024-09-19T04:57:02.747 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.747 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.747 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.747 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7257600", 2024-09-19T04:57:02.747 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.747 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.747 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.747 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T04:57:02.747 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.747 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.747 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.747 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.747 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.748 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "diskprediction_local", 2024-09-19T04:57:02.748 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.748 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.748 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.748 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.748 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.748 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.748 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.748 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.748 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.748 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.748 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.748 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.749 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.749 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.749 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.749 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.749 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.749 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.749 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.749 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.749 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.749 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.749 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.749 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.750 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.750 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.750 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.750 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.750 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.750 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.750 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.750 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.750 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.750 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.750 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.750 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.750 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.750 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.751 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.751 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.751 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.751 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.751 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.751 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.751 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.751 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.751 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.751 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.751 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.751 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.751 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.751 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.752 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.752 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.752 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.752 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.752 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.752 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.752 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.752 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.752 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.752 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.752 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.752 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.753 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.753 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.753 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.753 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.753 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.753 INFO:teuthology.orchestra.run.smithi123.stdout: "predict_interval": { 2024-09-19T04:57:02.753 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predict_interval", 2024-09-19T04:57:02.753 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.753 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.753 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.753 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-09-19T04:57:02.753 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.753 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.754 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.754 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.754 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.754 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.754 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.754 INFO:teuthology.orchestra.run.smithi123.stdout: "predictor_model": { 2024-09-19T04:57:02.754 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predictor_model", 2024-09-19T04:57:02.754 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.754 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.754 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.754 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "prophetstor", 2024-09-19T04:57:02.754 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.754 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.755 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.755 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.755 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.755 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.755 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.755 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-09-19T04:57:02.755 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-09-19T04:57:02.755 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.755 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.755 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.755 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-09-19T04:57:02.755 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.755 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.756 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.756 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.756 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.756 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.756 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.756 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.756 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.756 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "influx", 2024-09-19T04:57:02.756 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": false, 2024-09-19T04:57:02.756 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "influxdb python module not found", 2024-09-19T04:57:02.756 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.756 INFO:teuthology.orchestra.run.smithi123.stdout: "batch_size": { 2024-09-19T04:57:02.757 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "batch_size", 2024-09-19T04:57:02.757 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.757 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.757 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.757 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5000", 2024-09-19T04:57:02.757 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.757 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.757 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.757 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T04:57:02.757 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.757 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.757 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.758 INFO:teuthology.orchestra.run.smithi123.stdout: "database": { 2024-09-19T04:57:02.758 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "database", 2024-09-19T04:57:02.758 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.758 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.758 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.758 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "ceph", 2024-09-19T04:57:02.758 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.758 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.758 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.758 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T04:57:02.758 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.758 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.758 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.759 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": { 2024-09-19T04:57:02.759 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hostname", 2024-09-19T04:57:02.759 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.759 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.759 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.759 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.759 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.759 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.759 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.759 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server hostname", 2024-09-19T04:57:02.759 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.759 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.759 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.760 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-09-19T04:57:02.760 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-09-19T04:57:02.760 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.760 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.760 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.760 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "30", 2024-09-19T04:57:02.760 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "5", 2024-09-19T04:57:02.760 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.760 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.760 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T04:57:02.760 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.760 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.760 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.761 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.761 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.761 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.761 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.761 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.761 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.761 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.761 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.761 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.761 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.761 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.761 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.761 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.762 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.762 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.762 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.762 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.762 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.762 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.762 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.762 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.762 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.762 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.762 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.762 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.762 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.763 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.763 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.763 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.763 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.763 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.763 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.763 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.763 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.763 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.763 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.763 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.763 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.763 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.764 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.764 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.764 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.764 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.764 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.764 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.764 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.764 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.764 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.764 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.764 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.764 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.765 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.765 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.765 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.765 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.765 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.765 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.765 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.765 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.765 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.765 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.765 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.765 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.765 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.766 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.766 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.766 INFO:teuthology.orchestra.run.smithi123.stdout: "password": { 2024-09-19T04:57:02.766 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "password", 2024-09-19T04:57:02.766 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.766 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.766 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.766 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.766 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.766 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.766 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.766 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "password of InfluxDB server user", 2024-09-19T04:57:02.766 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.767 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.767 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.767 INFO:teuthology.orchestra.run.smithi123.stdout: "port": { 2024-09-19T04:57:02.767 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "port", 2024-09-19T04:57:02.767 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.767 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.767 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.767 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8086", 2024-09-19T04:57:02.767 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.767 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.767 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.767 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server port", 2024-09-19T04:57:02.767 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.768 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.768 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.768 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-09-19T04:57:02.768 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-09-19T04:57:02.768 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.768 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.768 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.768 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "false", 2024-09-19T04:57:02.768 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.768 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.768 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.768 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:57:02.768 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.769 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.769 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.769 INFO:teuthology.orchestra.run.smithi123.stdout: "threads": { 2024-09-19T04:57:02.769 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threads", 2024-09-19T04:57:02.769 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.769 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.769 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.769 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-09-19T04:57:02.769 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-09-19T04:57:02.769 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "32", 2024-09-19T04:57:02.769 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.769 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T04:57:02.769 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.769 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.770 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.770 INFO:teuthology.orchestra.run.smithi123.stdout: "username": { 2024-09-19T04:57:02.770 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "username", 2024-09-19T04:57:02.770 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.770 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.770 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.770 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.770 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.770 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.770 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.770 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "username of InfluxDB server user", 2024-09-19T04:57:02.770 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.771 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.771 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.771 INFO:teuthology.orchestra.run.smithi123.stdout: "verify_ssl": { 2024-09-19T04:57:02.771 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "verify_ssl", 2024-09-19T04:57:02.771 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.771 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.771 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.771 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "true", 2024-09-19T04:57:02.771 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.771 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.771 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.771 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:57:02.772 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.772 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.772 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.772 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.772 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.772 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "insights", 2024-09-19T04:57:02.772 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.772 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.772 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.772 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.772 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.773 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.773 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.773 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.773 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.773 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.773 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.773 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.773 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.773 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.773 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.773 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.773 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.773 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.773 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.774 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.774 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.774 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.774 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.774 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.774 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.774 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.774 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.774 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.774 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.774 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.774 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.774 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.775 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.775 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.775 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.775 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.775 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.775 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.775 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.775 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.775 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.775 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.775 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.775 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.776 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.776 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.776 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.776 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.776 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.776 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.776 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.776 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.776 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.776 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.776 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.776 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.776 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.777 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.777 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.777 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.777 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.777 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.777 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.777 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.777 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.777 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.777 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.777 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.777 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.777 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.777 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.778 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "iostat", 2024-09-19T04:57:02.778 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.778 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.778 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.778 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.778 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.778 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.778 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.778 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.778 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.778 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.779 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.779 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.779 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.779 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.779 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.779 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.779 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.779 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.779 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.779 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.779 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.779 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.779 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.780 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.780 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.780 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.780 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.780 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.780 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.780 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.780 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.780 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.780 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.780 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.780 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.780 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.781 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.781 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.781 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.781 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.781 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.781 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.781 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.781 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.781 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.781 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.781 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.781 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.781 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.782 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.782 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.782 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.782 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.782 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.782 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.782 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.782 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.782 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.782 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.782 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.782 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.782 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.782 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.783 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.783 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.783 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.783 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.783 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.783 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.783 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.783 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.783 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "k8sevents", 2024-09-19T04:57:02.783 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.783 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.784 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.784 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_event_retention_days": { 2024-09-19T04:57:02.784 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ceph_event_retention_days", 2024-09-19T04:57:02.784 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.784 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.784 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.784 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7", 2024-09-19T04:57:02.784 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.784 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.784 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.784 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T04:57:02.784 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.784 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.784 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.785 INFO:teuthology.orchestra.run.smithi123.stdout: "config_check_secs": { 2024-09-19T04:57:02.785 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_check_secs", 2024-09-19T04:57:02.785 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.785 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.785 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.785 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:02.785 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "10", 2024-09-19T04:57:02.785 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.785 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.785 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T04:57:02.785 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.786 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.786 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.786 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.786 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.786 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.786 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.786 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.786 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.786 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.786 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.786 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.786 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.786 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.787 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.787 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.787 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.787 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.787 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.787 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.787 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.787 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.787 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.787 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.787 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.787 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.787 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.788 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.788 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.788 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.788 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.788 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.788 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.788 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.788 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.788 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.788 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.788 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.788 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.789 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.789 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.789 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.789 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.789 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.789 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.789 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.789 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.789 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.789 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.789 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.789 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.789 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.789 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.790 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.790 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.790 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.790 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.790 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.790 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.790 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.790 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.790 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.790 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.790 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.790 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.790 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.791 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.791 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.791 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.791 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.791 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.791 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "localpool", 2024-09-19T04:57:02.791 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.791 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.791 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.791 INFO:teuthology.orchestra.run.smithi123.stdout: "failure_domain": { 2024-09-19T04:57:02.791 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "failure_domain", 2024-09-19T04:57:02.792 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.792 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.792 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.792 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "host", 2024-09-19T04:57:02.792 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.792 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.792 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.792 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "failure domain for any created local pool", 2024-09-19T04:57:02.792 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T04:57:02.792 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.792 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.792 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.792 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.793 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.793 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.793 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.793 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.793 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.793 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.793 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.793 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.793 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.793 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.793 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.793 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.793 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.794 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.794 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.794 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.794 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.794 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.794 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.794 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.794 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.794 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.794 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.794 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.794 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.794 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.795 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.795 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.795 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.795 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.795 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.795 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.795 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.795 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.795 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.795 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.795 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.795 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.795 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.796 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.796 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.796 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.796 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.796 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.796 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.796 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.796 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.796 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.796 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.796 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.796 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.796 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.797 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.797 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.797 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.797 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.797 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.797 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.797 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.797 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.797 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.797 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.797 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.797 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.797 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": { 2024-09-19T04:57:02.798 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_size", 2024-09-19T04:57:02.798 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.798 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.798 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.798 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.798 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.798 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.798 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.798 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default min_size for any created local pool", 2024-09-19T04:57:02.798 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T04:57:02.798 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.798 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.798 INFO:teuthology.orchestra.run.smithi123.stdout: "num_rep": { 2024-09-19T04:57:02.798 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "num_rep", 2024-09-19T04:57:02.799 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.799 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.799 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.799 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3", 2024-09-19T04:57:02.799 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.799 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.799 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.799 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default replica count for any created local pool", 2024-09-19T04:57:02.799 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.799 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.800 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.800 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": { 2024-09-19T04:57:02.800 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_num", 2024-09-19T04:57:02.800 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.800 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.800 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.800 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "128", 2024-09-19T04:57:02.800 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.800 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.800 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.800 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T04:57:02.800 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.801 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.801 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.801 INFO:teuthology.orchestra.run.smithi123.stdout: "prefix": { 2024-09-19T04:57:02.801 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prefix", 2024-09-19T04:57:02.801 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.801 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.801 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.801 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.801 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.801 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.801 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.801 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name prefix for any created local pool", 2024-09-19T04:57:02.801 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.802 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.802 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.802 INFO:teuthology.orchestra.run.smithi123.stdout: "subtree": { 2024-09-19T04:57:02.802 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "subtree", 2024-09-19T04:57:02.802 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.802 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.802 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.802 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "rack", 2024-09-19T04:57:02.802 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.802 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.802 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.802 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T04:57:02.803 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T04:57:02.803 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.803 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.803 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.803 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.803 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mds_autoscaler", 2024-09-19T04:57:02.803 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.803 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.803 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.803 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.803 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.803 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.804 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.804 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.804 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.804 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.804 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.804 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.804 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.804 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.804 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.804 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.804 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.804 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.804 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.804 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.805 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.805 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.805 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.805 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.805 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.805 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.805 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.805 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.805 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.805 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.805 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.805 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.805 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.806 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.806 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.806 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.806 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.806 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.806 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.806 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.806 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.806 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.806 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.806 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.806 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.806 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.807 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.807 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.807 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.807 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.807 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.807 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.807 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.807 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.807 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.807 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.807 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.807 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.807 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.808 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.808 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.808 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.808 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.808 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.808 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.808 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.808 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.808 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.808 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.808 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.808 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.809 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirroring", 2024-09-19T04:57:02.809 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.809 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.809 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.809 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.809 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.809 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.809 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.809 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.809 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.809 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.809 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.809 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.809 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.810 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.810 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.810 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.810 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.810 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.810 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.810 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.810 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.810 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.810 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.810 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.810 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.810 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.811 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.811 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.811 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.811 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.811 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.811 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.811 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.811 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.811 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.811 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.811 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.811 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.811 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.812 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.812 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.812 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.812 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.812 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.812 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.812 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.812 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.812 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.812 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.812 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.812 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.812 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.812 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.813 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.813 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.813 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.813 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.813 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.813 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.813 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.813 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.813 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.813 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.813 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.814 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.814 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.814 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.814 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.814 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.814 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.814 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.814 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "nfs", 2024-09-19T04:57:02.814 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.814 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.814 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.814 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.815 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.815 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.815 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.815 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.815 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.815 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.815 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.815 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.815 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.815 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.815 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.815 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.815 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.816 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.816 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.816 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.816 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.816 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.816 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.816 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.816 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.816 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.816 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.816 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.816 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.816 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.817 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.817 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.817 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.817 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.817 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.817 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.817 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.817 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.817 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.817 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.817 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.817 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.817 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.818 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.818 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.818 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.818 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.818 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.818 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.818 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.818 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.818 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.818 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.818 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.818 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.818 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.819 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.819 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.819 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.819 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.819 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.819 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.819 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.819 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.819 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.819 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.819 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.819 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.819 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.819 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.820 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.820 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-09-19T04:57:02.820 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.820 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.820 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.820 INFO:teuthology.orchestra.run.smithi123.stdout: "fail_fs": { 2024-09-19T04:57:02.820 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "fail_fs", 2024-09-19T04:57:02.820 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.820 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.820 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.820 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.821 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.821 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.821 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.821 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T04:57:02.821 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.821 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.821 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.821 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.821 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.821 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.821 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.821 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.821 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.821 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.822 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.822 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.822 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.822 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.822 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.822 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.822 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.822 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.822 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.822 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.822 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.822 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.822 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.823 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.823 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.823 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.823 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.823 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.823 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.823 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.823 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.823 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.823 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.823 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.823 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.823 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.824 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.824 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.824 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.824 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.824 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.824 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.824 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.824 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.824 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.824 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.824 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.824 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.824 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.825 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.825 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.825 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.825 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.825 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.825 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.825 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.825 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.825 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.825 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.825 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.825 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.826 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.826 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.826 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.826 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.826 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.826 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.826 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.826 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.826 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator": { 2024-09-19T04:57:02.826 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-09-19T04:57:02.826 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.826 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.826 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.827 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.827 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.827 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.827 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.827 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm", 2024-09-19T04:57:02.827 INFO:teuthology.orchestra.run.smithi123.stdout: "rook", 2024-09-19T04:57:02.827 INFO:teuthology.orchestra.run.smithi123.stdout: "test_orchestrator" 2024-09-19T04:57:02.827 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.827 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Orchestrator backend", 2024-09-19T04:57:02.827 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.827 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.827 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.827 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.828 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.828 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_perf_query", 2024-09-19T04:57:02.828 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.828 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.828 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.828 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.828 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.828 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.828 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.828 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.828 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.829 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.829 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.829 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.829 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.829 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.829 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.829 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.829 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.829 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.829 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.829 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.829 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.829 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.830 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.830 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.830 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.830 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.830 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.830 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.830 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.830 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.830 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.830 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.830 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.830 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.830 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.831 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.831 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.831 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.831 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.831 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.831 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.831 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.831 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.831 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.831 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.831 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.831 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.831 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.832 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.832 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.832 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.832 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.832 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.832 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.832 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.832 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.832 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.832 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.832 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.832 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.833 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.833 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.833 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.833 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.833 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.833 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.833 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.833 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.833 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.833 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.833 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.834 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_support", 2024-09-19T04:57:02.834 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.834 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.834 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.834 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.834 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.834 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.834 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.834 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.834 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.834 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.834 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.834 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.835 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.835 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.835 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.835 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.835 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.835 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.835 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.835 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.835 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.835 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.835 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.835 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.835 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.836 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.836 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.836 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.836 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.836 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.836 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.836 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.836 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.836 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.836 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.836 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.836 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.836 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.837 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.837 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.837 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.837 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.837 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.837 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.837 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.837 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.837 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.837 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.837 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.837 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.837 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.839 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.839 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.839 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.839 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.839 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.839 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.839 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.839 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_autoscaler", 2024-09-19T04:57:02.839 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.839 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.839 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.840 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.840 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.840 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.840 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.840 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.840 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.840 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.840 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.840 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.840 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.840 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.840 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.840 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.840 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.841 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.841 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.841 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.841 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.841 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.841 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.841 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.841 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.841 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.841 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.841 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.841 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.841 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.842 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.842 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.842 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.842 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.842 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.842 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.842 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.842 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.842 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.842 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.842 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.843 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.843 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.843 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.843 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.843 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.843 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.843 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.843 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.843 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.843 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.843 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.843 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.843 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.843 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.844 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.844 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.844 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.844 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.844 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.844 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.844 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.844 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.844 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.844 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.844 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.844 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.845 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.845 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.845 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-09-19T04:57:02.845 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-09-19T04:57:02.845 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.845 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.845 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.845 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-09-19T04:57:02.845 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.845 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.845 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.845 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.846 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.846 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.846 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.846 INFO:teuthology.orchestra.run.smithi123.stdout: "threshold": { 2024-09-19T04:57:02.846 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threshold", 2024-09-19T04:57:02.846 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:02.846 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.846 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.846 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-09-19T04:57:02.846 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1.0", 2024-09-19T04:57:02.846 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.846 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.847 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "scaling threshold", 2024-09-19T04:57:02.847 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T04:57:02.847 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.847 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.847 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.847 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.847 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "progress", 2024-09-19T04:57:02.847 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.847 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.847 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.847 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_pg_recovery_event": { 2024-09-19T04:57:02.847 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_pg_recovery_event", 2024-09-19T04:57:02.848 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.848 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.848 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.848 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.848 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.848 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.848 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.848 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T04:57:02.848 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.848 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.848 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.848 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-09-19T04:57:02.849 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-09-19T04:57:02.849 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.849 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.849 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.849 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.849 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.849 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.849 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.849 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.849 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.849 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.849 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.849 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.850 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.850 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.850 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.850 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.850 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.850 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.850 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.850 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.850 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.850 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.850 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.850 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.850 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.850 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.851 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.851 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.851 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.851 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.851 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.851 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.851 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.851 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.851 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.851 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.851 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.851 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.851 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.852 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.852 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.852 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.852 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.852 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.852 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.852 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.852 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.852 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.852 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.852 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.852 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.853 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.853 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.853 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.853 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.853 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.853 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.853 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.853 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.853 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.853 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.853 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.853 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.853 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.854 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.854 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.854 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.854 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.854 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.854 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.854 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.854 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.854 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.854 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.854 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.854 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.854 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.855 INFO:teuthology.orchestra.run.smithi123.stdout: "max_completed_events": { 2024-09-19T04:57:02.855 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_completed_events", 2024-09-19T04:57:02.855 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.855 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.855 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.855 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "50", 2024-09-19T04:57:02.855 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.855 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.855 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.855 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "number of past completed events to remember", 2024-09-19T04:57:02.855 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.855 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.855 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.856 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-09-19T04:57:02.856 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-09-19T04:57:02.856 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.856 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.856 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.856 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-09-19T04:57:02.856 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.856 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.856 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.856 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long the module is going to sleep", 2024-09-19T04:57:02.856 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.856 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.857 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.857 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.857 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.857 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus", 2024-09-19T04:57:02.857 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.857 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.857 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.857 INFO:teuthology.orchestra.run.smithi123.stdout: "cache": { 2024-09-19T04:57:02.857 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cache", 2024-09-19T04:57:02.857 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.857 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi123.stdout: "exclude_perf_counters": { 2024-09-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "exclude_perf_counters", 2024-09-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.858 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.859 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.859 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.859 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.859 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.859 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.859 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T04:57:02.859 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T04:57:02.859 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.859 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.859 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.859 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.859 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.859 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.860 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.860 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.860 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.860 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.860 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.860 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.860 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.860 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.860 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.860 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.860 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.860 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.860 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.860 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.861 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.861 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.861 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.861 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.861 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.861 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.861 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.861 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.861 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.861 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.861 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.861 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.861 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.862 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.862 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.862 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.862 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.862 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.862 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.862 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.862 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.862 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.862 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.862 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.862 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.862 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.863 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.863 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.863 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.863 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.863 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.863 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.863 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.863 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.863 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.863 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.863 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.863 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.863 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.864 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.864 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.864 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.864 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.864 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.864 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.864 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.864 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.864 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.864 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools": { 2024-09-19T04:57:02.864 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools", 2024-09-19T04:57:02.864 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.865 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.865 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.865 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.865 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.865 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.865 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.865 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.865 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.865 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.865 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.865 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T04:57:02.865 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T04:57:02.865 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.866 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.866 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.866 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "300", 2024-09-19T04:57:02.866 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.866 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.866 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.866 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.866 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.866 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.866 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.866 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_interval": { 2024-09-19T04:57:02.866 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_interval", 2024-09-19T04:57:02.866 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:02.867 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.867 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.867 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15.0", 2024-09-19T04:57:02.867 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.867 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.867 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.867 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.867 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.867 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.867 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.867 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-09-19T04:57:02.867 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-09-19T04:57:02.867 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.868 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.868 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.868 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-09-19T04:57:02.868 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.868 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.868 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.868 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T04:57:02.868 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.868 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.868 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.868 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-09-19T04:57:02.868 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-09-19T04:57:02.869 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.869 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.869 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.869 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "9283", 2024-09-19T04:57:02.869 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.869 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.869 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.869 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T04:57:02.869 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.869 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.869 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.869 INFO:teuthology.orchestra.run.smithi123.stdout: "stale_cache_strategy": { 2024-09-19T04:57:02.870 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stale_cache_strategy", 2024-09-19T04:57:02.870 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.870 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.870 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.870 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "log", 2024-09-19T04:57:02.870 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.870 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.870 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.870 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.870 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.870 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.870 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.870 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-09-19T04:57:02.871 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-09-19T04:57:02.871 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.871 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.871 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.871 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "default", 2024-09-19T04:57:02.871 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.871 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.871 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.871 INFO:teuthology.orchestra.run.smithi123.stdout: "default", 2024-09-19T04:57:02.871 INFO:teuthology.orchestra.run.smithi123.stdout: "error" 2024-09-19T04:57:02.871 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.871 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.871 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.872 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.872 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.872 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-09-19T04:57:02.872 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-09-19T04:57:02.872 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.872 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.872 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.872 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-09-19T04:57:02.872 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-09-19T04:57:02.872 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-09-19T04:57:02.872 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.872 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.873 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.873 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.873 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.873 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.873 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.873 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_support", 2024-09-19T04:57:02.873 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.873 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.873 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.873 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.873 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.873 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.874 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.874 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.874 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.874 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.874 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.874 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.874 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.874 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.874 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.874 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.874 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.874 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.874 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.875 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.875 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.875 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.875 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.875 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.875 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.875 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.875 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.875 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.875 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.875 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.875 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.875 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.876 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.876 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.876 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.876 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.876 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.876 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.876 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.876 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.876 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.876 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.876 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.876 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.877 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.877 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.877 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.877 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.877 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.877 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.877 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.877 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.877 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.877 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.877 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.877 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.878 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.878 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.878 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.878 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.878 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.878 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.878 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.878 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.878 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.878 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.878 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.878 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.878 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.879 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_snap_create": { 2024-09-19T04:57:02.879 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_snap_create", 2024-09-19T04:57:02.879 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.879 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.879 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.879 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:02.879 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.879 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.879 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.879 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.879 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.879 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.879 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.880 INFO:teuthology.orchestra.run.smithi123.stdout: "mirror_snapshot_schedule": { 2024-09-19T04:57:02.880 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T04:57:02.880 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.880 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.880 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.880 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.880 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.880 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.880 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.880 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.880 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.880 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.880 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.881 INFO:teuthology.orchestra.run.smithi123.stdout: "trash_purge_schedule": { 2024-09-19T04:57:02.881 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "trash_purge_schedule", 2024-09-19T04:57:02.881 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.884 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.884 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.884 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.885 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.885 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.885 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.885 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.885 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.885 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.885 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.885 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.885 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.885 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "restful", 2024-09-19T04:57:02.885 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.885 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.886 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.886 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_auth": { 2024-09-19T04:57:02.886 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_auth", 2024-09-19T04:57:02.886 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.886 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.886 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.886 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.886 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.886 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.886 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.886 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.886 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.886 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.886 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.887 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-09-19T04:57:02.887 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-09-19T04:57:02.887 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.887 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.887 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.887 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.887 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.887 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.887 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.887 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.887 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.887 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.888 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.888 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.888 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.888 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.888 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.888 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.888 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.888 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.888 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.888 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.888 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.888 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.889 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.889 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.889 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.889 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.889 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.889 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.889 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.889 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.889 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.889 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.889 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.889 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.889 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.890 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.890 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.890 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.890 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.890 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.890 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.890 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.890 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.890 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.890 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.890 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.890 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.891 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.891 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.891 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.891 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.891 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.891 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.891 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.891 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.891 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.891 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.891 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.891 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.891 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.891 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.892 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.892 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.892 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.892 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.892 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.892 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.892 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.892 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.892 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.892 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.892 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.893 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.893 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.893 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.893 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.893 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.893 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-09-19T04:57:02.893 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-09-19T04:57:02.893 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.893 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.893 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.893 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.893 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.893 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.894 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.894 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.894 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.894 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.894 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.894 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-09-19T04:57:02.894 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-09-19T04:57:02.894 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.894 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.894 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.894 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.894 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.894 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.895 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.895 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.895 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.895 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.895 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.895 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.895 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.895 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rgw", 2024-09-19T04:57:02.895 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.895 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.895 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.896 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.896 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.896 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.896 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.896 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.896 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.896 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.896 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.896 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.896 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.896 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.896 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.896 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.897 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.897 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.897 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.897 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.897 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.897 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.897 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.897 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.897 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.897 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.897 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.897 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.898 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.898 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.898 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.898 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.898 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.898 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.898 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.898 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.898 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.898 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.898 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.898 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.898 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.899 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.899 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.899 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.899 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.899 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.899 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.899 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.899 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.899 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.899 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.899 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.899 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.899 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.899 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.899 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.900 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.900 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.900 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.900 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.900 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.900 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.900 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.900 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.900 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.900 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.900 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.900 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.900 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.901 INFO:teuthology.orchestra.run.smithi123.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T04:57:02.901 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T04:57:02.901 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.901 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.901 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.901 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-09-19T04:57:02.901 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.901 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.901 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.901 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T04:57:02.901 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.901 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.902 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.902 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.902 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.902 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.902 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rook", 2024-09-19T04:57:02.902 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.902 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.902 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.902 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.902 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.902 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.902 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.902 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.903 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.903 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.903 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.903 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.903 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.903 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.903 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.903 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.903 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.903 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.903 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.903 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.903 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.903 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.904 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.904 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.904 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.904 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.904 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.904 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.904 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.904 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.904 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.904 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.904 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.904 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.905 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.905 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.905 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.905 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.905 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.905 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.905 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.905 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.905 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.905 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.905 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.905 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.905 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.906 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.906 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.906 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.906 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.906 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.906 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.906 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.906 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.906 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.906 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.906 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.906 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.906 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.907 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.907 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.907 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.907 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.907 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.907 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.907 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.907 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.907 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.907 INFO:teuthology.orchestra.run.smithi123.stdout: "storage_class": { 2024-09-19T04:57:02.907 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "storage_class", 2024-09-19T04:57:02.907 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.907 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.908 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.908 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "local", 2024-09-19T04:57:02.908 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.908 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.908 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.908 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T04:57:02.908 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.908 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.908 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.908 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.908 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.908 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.908 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "selftest", 2024-09-19T04:57:02.909 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.909 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.909 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.909 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.909 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.909 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.909 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.909 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.909 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.909 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.909 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.909 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.909 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.910 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.910 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.910 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.910 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.910 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.910 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.910 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.910 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.910 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.910 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.910 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.910 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.910 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.911 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.911 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.911 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.911 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.911 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.911 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.911 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.911 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.911 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.911 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.911 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.911 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.911 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.911 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.912 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.912 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.912 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.912 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.912 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.912 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.912 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.912 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.912 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.912 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.912 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.912 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.912 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.912 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.913 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.913 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.913 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.913 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.913 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.913 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.913 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.913 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.913 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.913 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.913 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.913 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.913 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.914 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.914 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.914 INFO:teuthology.orchestra.run.smithi123.stdout: "roption1": { 2024-09-19T04:57:02.914 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption1", 2024-09-19T04:57:02.914 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.914 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.914 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.914 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.914 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.914 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.914 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.914 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.914 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.915 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.915 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.915 INFO:teuthology.orchestra.run.smithi123.stdout: "roption2": { 2024-09-19T04:57:02.915 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption2", 2024-09-19T04:57:02.915 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.915 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.915 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.915 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "xyz", 2024-09-19T04:57:02.915 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.915 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.915 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.915 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.915 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.916 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.916 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.916 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption1": { 2024-09-19T04:57:02.916 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption1", 2024-09-19T04:57:02.916 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.916 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.916 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.916 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.916 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.916 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.916 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.917 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.917 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.917 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.917 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.917 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption2": { 2024-09-19T04:57:02.917 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption2", 2024-09-19T04:57:02.917 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.917 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.917 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.917 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.917 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.917 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.917 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.918 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.918 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.918 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.918 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.918 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption3": { 2024-09-19T04:57:02.918 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption3", 2024-09-19T04:57:02.918 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:02.918 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.918 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.918 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.918 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.918 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.918 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.919 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.919 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.919 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.919 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.919 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption4": { 2024-09-19T04:57:02.919 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption4", 2024-09-19T04:57:02.919 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.919 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.919 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.919 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.919 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.919 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.920 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.920 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.920 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.920 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.920 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.920 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption5": { 2024-09-19T04:57:02.920 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption5", 2024-09-19T04:57:02.920 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.920 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.920 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.920 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.920 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.921 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.921 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.921 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.921 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.921 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.921 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.921 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption6": { 2024-09-19T04:57:02.921 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption6", 2024-09-19T04:57:02.921 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.921 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.921 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.921 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.921 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.922 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.922 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.922 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.922 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.922 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.922 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.922 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption7": { 2024-09-19T04:57:02.922 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption7", 2024-09-19T04:57:02.922 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.922 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.922 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.922 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.923 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-09-19T04:57:02.923 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "42", 2024-09-19T04:57:02.923 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.923 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.923 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.923 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.923 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.923 INFO:teuthology.orchestra.run.smithi123.stdout: "testkey": { 2024-09-19T04:57:02.923 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testkey", 2024-09-19T04:57:02.923 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.923 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.923 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.923 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.924 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.924 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.924 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.924 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.924 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.924 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.924 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.924 INFO:teuthology.orchestra.run.smithi123.stdout: "testlkey": { 2024-09-19T04:57:02.924 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testlkey", 2024-09-19T04:57:02.924 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.924 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.924 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.925 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.925 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.925 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.925 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.925 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.925 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.925 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.925 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.925 INFO:teuthology.orchestra.run.smithi123.stdout: "testnewline": { 2024-09-19T04:57:02.925 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testnewline", 2024-09-19T04:57:02.925 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.925 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.926 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.926 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.926 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.926 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.926 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.926 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.926 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.926 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.926 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.926 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.926 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.926 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snap_schedule", 2024-09-19T04:57:02.927 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.927 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.927 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.927 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_m_granularity": { 2024-09-19T04:57:02.927 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_m_granularity", 2024-09-19T04:57:02.927 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.927 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.927 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.927 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.927 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.927 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.927 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.927 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T04:57:02.927 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.928 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.928 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.928 INFO:teuthology.orchestra.run.smithi123.stdout: "dump_on_update": { 2024-09-19T04:57:02.928 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dump_on_update", 2024-09-19T04:57:02.928 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.928 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.928 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.928 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.928 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.928 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.928 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.928 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "dump database to debug log on update", 2024-09-19T04:57:02.929 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.929 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.929 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.929 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.929 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.929 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.929 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.929 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.929 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.929 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.929 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.929 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.929 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.930 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.930 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.930 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.930 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.930 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.930 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.930 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.930 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.930 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.930 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.930 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.930 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.931 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.931 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.931 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.931 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.931 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.931 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.931 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.931 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.931 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.931 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.931 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.931 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.931 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.932 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.932 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.932 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.932 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.932 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.932 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.932 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.932 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.932 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.932 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.932 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.932 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.932 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.933 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.933 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.933 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.933 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.933 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.933 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.933 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.933 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.933 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.933 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.933 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.933 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.934 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.934 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.934 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.934 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.934 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.934 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.934 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.934 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.934 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stats", 2024-09-19T04:57:02.934 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.934 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.934 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.935 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.935 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.935 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.935 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.935 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.935 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.935 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.935 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.935 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.935 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.935 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.935 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.935 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.935 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.936 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.936 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.936 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.936 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.936 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.936 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.936 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.936 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.936 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.936 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.936 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.936 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.937 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.937 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.937 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.937 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.937 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.937 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.937 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.937 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.937 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.937 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.937 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.937 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.937 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.938 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.938 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.938 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.938 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.938 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.938 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.938 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.938 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.938 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.938 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.938 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.938 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.938 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.938 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.939 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.939 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.939 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.939 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.939 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.939 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.939 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.939 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.939 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.939 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.939 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.939 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.939 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.940 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.940 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.940 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.940 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "status", 2024-09-19T04:57:02.940 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.940 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.940 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.940 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.940 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.940 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.940 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.940 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.941 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.941 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.941 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.941 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.941 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.941 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.941 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.941 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.941 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.941 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.941 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.941 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.941 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.941 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.942 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.942 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.942 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.942 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.942 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.942 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.942 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.942 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.942 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.942 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.942 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.942 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.942 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.943 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.943 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.943 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.943 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.943 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.943 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.943 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.943 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.943 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.943 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.943 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.943 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.943 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.944 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.944 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.944 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.944 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.944 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.944 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.944 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.944 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.944 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.944 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.944 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.944 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.944 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.944 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.945 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.945 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.945 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.945 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.945 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.945 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.945 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.945 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.945 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.945 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.945 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.945 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telegraf", 2024-09-19T04:57:02.946 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.946 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.946 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.946 INFO:teuthology.orchestra.run.smithi123.stdout: "address": { 2024-09-19T04:57:02.946 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "address", 2024-09-19T04:57:02.946 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.946 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.946 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.946 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T04:57:02.946 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.946 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.946 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.946 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.946 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.947 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.947 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.947 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-09-19T04:57:02.947 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-09-19T04:57:02.947 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.947 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.947 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.947 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15", 2024-09-19T04:57:02.947 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.947 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.947 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.947 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.947 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.952 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.952 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.952 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.952 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.952 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.952 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.952 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.952 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.952 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.952 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.952 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.952 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.952 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.953 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.953 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.953 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.953 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.953 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.953 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.953 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telemetry", 2024-09-19T04:57:02.953 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.953 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.953 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.953 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_basic": { 2024-09-19T04:57:02.953 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_basic", 2024-09-19T04:57:02.953 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.954 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.954 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.954 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.954 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.954 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.954 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.954 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T04:57:02.954 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.954 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.954 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.954 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.954 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_crash": { 2024-09-19T04:57:02.954 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_crash", 2024-09-19T04:57:02.955 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.955 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.955 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.955 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.955 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.955 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.955 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.955 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T04:57:02.955 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.955 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.955 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.955 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_device": { 2024-09-19T04:57:02.955 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_device", 2024-09-19T04:57:02.956 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.956 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.956 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.956 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.956 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.956 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.956 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.956 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T04:57:02.956 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.956 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.956 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.956 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_ident": { 2024-09-19T04:57:02.957 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_ident", 2024-09-19T04:57:02.957 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.957 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.957 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.957 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.957 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.957 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.957 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.957 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T04:57:02.957 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.957 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.957 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.957 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_perf": { 2024-09-19T04:57:02.958 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_perf", 2024-09-19T04:57:02.958 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.958 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.958 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.958 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.958 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.958 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.958 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.958 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T04:57:02.958 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.958 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.958 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.959 INFO:teuthology.orchestra.run.smithi123.stdout: "contact": { 2024-09-19T04:57:02.959 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "contact", 2024-09-19T04:57:02.959 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.959 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.959 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.959 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.959 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.959 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.959 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.959 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.959 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.959 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.959 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.960 INFO:teuthology.orchestra.run.smithi123.stdout: "description": { 2024-09-19T04:57:02.960 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "description", 2024-09-19T04:57:02.960 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.960 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.960 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.960 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.960 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.960 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.960 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.960 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.960 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.960 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.960 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.961 INFO:teuthology.orchestra.run.smithi123.stdout: "device_url": { 2024-09-19T04:57:02.961 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_url", 2024-09-19T04:57:02.961 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.961 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.961 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.961 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T04:57:02.961 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.961 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.961 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.961 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.961 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.961 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.961 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.962 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-09-19T04:57:02.962 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-09-19T04:57:02.962 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.962 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.962 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.962 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.962 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.962 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.962 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.962 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.962 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.962 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.962 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.963 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-09-19T04:57:02.963 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-09-19T04:57:02.963 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.963 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.963 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.963 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "24", 2024-09-19T04:57:02.963 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "8", 2024-09-19T04:57:02.963 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.963 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.963 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.963 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.963 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.963 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.964 INFO:teuthology.orchestra.run.smithi123.stdout: "last_opt_revision": { 2024-09-19T04:57:02.964 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "last_opt_revision", 2024-09-19T04:57:02.964 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.964 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.964 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.964 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1", 2024-09-19T04:57:02.964 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.964 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.964 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.964 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.964 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.964 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.965 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.965 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard": { 2024-09-19T04:57:02.965 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard", 2024-09-19T04:57:02.965 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.965 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.965 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.965 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.965 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.965 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.965 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.965 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.965 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.965 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard_description": { 2024-09-19T04:57:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard_description", 2024-09-19T04:57:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.966 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.967 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.968 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.969 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.970 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "organization": { 2024-09-19T04:57:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "organization", 2024-09-19T04:57:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.972 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "proxy": { 2024-09-19T04:57:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "proxy", 2024-09-19T04:57:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.973 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "url": { 2024-09-19T04:57:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url", 2024-09-19T04:57:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T04:57:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.974 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "test_orchestrator", 2024-09-19T04:57:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.975 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.976 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "volumes", 2024-09-19T04:57:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.981 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.982 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.983 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.984 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_clones": { 2024-09-19T04:57:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_clones", 2024-09-19T04:57:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.986 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4", 2024-09-19T04:57:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T04:57:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "periodic_async_work": { 2024-09-19T04:57:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "periodic_async_work", 2024-09-19T04:57:02.987 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Periodically check for async work", 2024-09-19T04:57:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_delay": { 2024-09-19T04:57:02.988 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_delay", 2024-09-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-09-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_no_wait": { 2024-09-19T04:57:02.989 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:02.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix", 2024-09-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "discovery_interval": { 2024-09-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "discovery_interval", 2024-09-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-09-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "100", 2024-09-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.991 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "identifier": { 2024-09-19T04:57:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "identifier", 2024-09-19T04:57:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.992 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-09-19T04:57:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-09-19T04:57:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-09-19T04:57:02.993 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.994 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.995 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.996 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:02.997 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_host": { 2024-09-19T04:57:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_host", 2024-09-19T04:57:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:02.999 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_port": { 2024-09-19T04:57:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_port", 2024-09-19T04:57:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.000 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10051", 2024-09-19T04:57:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_sender": { 2024-09-19T04:57:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_sender", 2024-09-19T04:57:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T04:57:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T04:57:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: "modules": [ 2024-09-19T04:57:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm", 2024-09-19T04:57:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: "dashboard", 2024-09-19T04:57:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: "iostat", 2024-09-19T04:57:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: "nfs", 2024-09-19T04:57:03.003 INFO:teuthology.orchestra.run.smithi123.stdout: "restful", 2024-09-19T04:57:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_schedule" 2024-09-19T04:57:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "available_modules": [ 2024-09-19T04:57:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "alerts", 2024-09-19T04:57:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-09-19T04:57:03.004 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-09-19T04:57:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-09-19T04:57:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T04:57:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.005 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.008 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.011 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_destination": { 2024-09-19T04:57:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_destination", 2024-09-19T04:57:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.012 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email address to send alerts to", 2024-09-19T04:57:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_from_name": { 2024-09-19T04:57:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_from_name", 2024-09-19T04:57:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "Ceph", 2024-09-19T04:57:03.013 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email From: name", 2024-09-19T04:57:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_host": { 2024-09-19T04:57:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_host", 2024-09-19T04:57:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.014 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP server", 2024-09-19T04:57:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_password": { 2024-09-19T04:57:03.015 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_password", 2024-09-19T04:57:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Password to authenticate with", 2024-09-19T04:57:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_port": { 2024-09-19T04:57:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_port", 2024-09-19T04:57:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "465", 2024-09-19T04:57:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP port", 2024-09-19T04:57:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_sender": { 2024-09-19T04:57:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_sender", 2024-09-19T04:57:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.018 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP envelope sender", 2024-09-19T04:57:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_ssl": { 2024-09-19T04:57:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_ssl", 2024-09-19T04:57:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.019 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T04:57:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_user": { 2024-09-19T04:57:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_user", 2024-09-19T04:57:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "User to authenticate as", 2024-09-19T04:57:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "balancer", 2024-09-19T04:57:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "active": { 2024-09-19T04:57:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "active", 2024-09-19T04:57:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.022 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T04:57:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_time": { 2024-09-19T04:57:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_time", 2024-09-19T04:57:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0000", 2024-09-19T04:57:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T04:57:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:57:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_weekday": { 2024-09-19T04:57:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_weekday", 2024-09-19T04:57:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-09-19T04:57:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-09-19T04:57:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-09-19T04:57:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-09-19T04:57:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T04:57:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:57:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_max_iterations": { 2024-09-19T04:57:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_max_iterations", 2024-09-19T04:57:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-09-19T04:57:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "25", 2024-09-19T04:57:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-09-19T04:57:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "250", 2024-09-19T04:57:03.026 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.027 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T04:57:03.027 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.027 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.027 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.027 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_metrics": { 2024-09-19T04:57:03.027 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_metrics", 2024-09-19T04:57:03.027 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.027 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.027 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.027 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T04:57:03.027 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.028 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.028 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.028 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T04:57:03.028 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T04:57:03.028 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.028 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.028 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_step": { 2024-09-19T04:57:03.028 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_step", 2024-09-19T04:57:03.028 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:03.028 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.029 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.029 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.5", 2024-09-19T04:57:03.029 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0.001", 2024-09-19T04:57:03.029 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "0.999", 2024-09-19T04:57:03.029 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.029 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "aggressiveness of optimization", 2024-09-19T04:57:03.029 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T04:57:03.029 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.029 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.029 INFO:teuthology.orchestra.run.smithi123.stdout: "end_time": { 2024-09-19T04:57:03.029 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_time", 2024-09-19T04:57:03.030 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.030 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.030 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.030 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2359", 2024-09-19T04:57:03.030 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.030 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.030 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.030 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T04:57:03.030 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:57:03.030 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.030 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.031 INFO:teuthology.orchestra.run.smithi123.stdout: "end_weekday": { 2024-09-19T04:57:03.031 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_weekday", 2024-09-19T04:57:03.031 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-09-19T04:57:03.031 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.031 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.031 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-09-19T04:57:03.031 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-09-19T04:57:03.031 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-09-19T04:57:03.031 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.031 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T04:57:03.031 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:57:03.032 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.032 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.032 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.032 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.032 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.032 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.032 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.032 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.032 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.032 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.033 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.033 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.033 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.033 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.033 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.033 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.033 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.033 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.033 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.033 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.033 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.033 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.034 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.034 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.034 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.034 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.034 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.034 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.034 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.034 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.034 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.034 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.035 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.035 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.035 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.035 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.035 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.035 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.035 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.035 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.035 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.035 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.035 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.036 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.036 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.036 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.036 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.036 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.036 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.036 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.036 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.036 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.036 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.036 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.037 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.037 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.037 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.037 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.037 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.037 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.037 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.037 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.037 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.037 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.037 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.038 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.038 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.038 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.038 INFO:teuthology.orchestra.run.smithi123.stdout: "min_score": { 2024-09-19T04:57:03.038 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_score", 2024-09-19T04:57:03.038 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:03.038 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.038 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.038 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-09-19T04:57:03.038 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.038 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.039 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.039 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T04:57:03.039 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.039 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.039 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.039 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-09-19T04:57:03.039 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-09-19T04:57:03.039 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.039 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.039 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.039 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "upmap", 2024-09-19T04:57:03.040 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.040 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.040 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.040 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-compat", 2024-09-19T04:57:03.040 INFO:teuthology.orchestra.run.smithi123.stdout: "none", 2024-09-19T04:57:03.040 INFO:teuthology.orchestra.run.smithi123.stdout: "read", 2024-09-19T04:57:03.040 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap", 2024-09-19T04:57:03.040 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap-read" 2024-09-19T04:57:03.040 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.040 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Balancer mode", 2024-09-19T04:57:03.040 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.041 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.041 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.041 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_ids": { 2024-09-19T04:57:03.041 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_ids", 2024-09-19T04:57:03.041 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.041 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.041 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.041 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.041 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.041 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.042 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.042 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T04:57:03.042 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.042 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.042 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.042 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-09-19T04:57:03.042 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-09-19T04:57:03.042 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.042 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.042 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.042 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-09-19T04:57:03.043 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.043 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.043 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.043 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T04:57:03.043 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.043 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.043 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.043 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_deviation": { 2024-09-19T04:57:03.043 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_deviation", 2024-09-19T04:57:03.043 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.044 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.044 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.044 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-09-19T04:57:03.044 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-09-19T04:57:03.044 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.044 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.044 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T04:57:03.044 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T04:57:03.044 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.044 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.045 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_optimizations": { 2024-09-19T04:57:03.045 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_optimizations", 2024-09-19T04:57:03.045 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-09-19T04:57:03.045 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.045 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.045 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:03.045 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.045 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.045 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.045 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T04:57:03.045 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.046 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.046 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.046 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.046 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.046 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm", 2024-09-19T04:57:03.046 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.046 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.046 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.046 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_down_multiplier": { 2024-09-19T04:57:03.047 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_down_multiplier", 2024-09-19T04:57:03.047 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:03.047 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.047 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.047 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-09-19T04:57:03.047 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.047 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.047 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.047 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T04:57:03.047 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.047 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.047 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.048 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_refresh_rate": { 2024-09-19T04:57:03.048 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_refresh_rate", 2024-09-19T04:57:03.048 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.048 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.048 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.048 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "20", 2024-09-19T04:57:03.048 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.048 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.048 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.048 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T04:57:03.049 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.049 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.049 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.049 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_starting_port": { 2024-09-19T04:57:03.049 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_starting_port", 2024-09-19T04:57:03.049 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.049 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.049 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.049 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4721", 2024-09-19T04:57:03.049 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.050 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.050 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.050 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T04:57:03.050 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.050 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.050 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.050 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_ptrace": { 2024-09-19T04:57:03.050 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_ptrace", 2024-09-19T04:57:03.050 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.050 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.050 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.050 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.051 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.051 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.051 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.051 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T04:57:03.051 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T04:57:03.051 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.051 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.051 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_interval": { 2024-09-19T04:57:03.051 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_interval", 2024-09-19T04:57:03.051 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.052 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.052 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.052 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-09-19T04:57:03.052 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.052 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.052 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.052 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T04:57:03.052 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.052 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.052 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.052 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_memory_target_ratio": { 2024-09-19T04:57:03.053 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T04:57:03.053 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:03.053 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.053 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.053 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.7", 2024-09-19T04:57:03.053 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.053 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.053 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.053 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T04:57:03.053 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.053 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.054 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.054 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm_log_destination": { 2024-09-19T04:57:03.054 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm_log_destination", 2024-09-19T04:57:03.054 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.054 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.054 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.054 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.054 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.054 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.054 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.054 INFO:teuthology.orchestra.run.smithi123.stdout: "file", 2024-09-19T04:57:03.055 INFO:teuthology.orchestra.run.smithi123.stdout: "file,syslog", 2024-09-19T04:57:03.055 INFO:teuthology.orchestra.run.smithi123.stdout: "syslog" 2024-09-19T04:57:03.055 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.055 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T04:57:03.055 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.055 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.055 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.055 INFO:teuthology.orchestra.run.smithi123.stdout: "cgroups_split": { 2024-09-19T04:57:03.055 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cgroups_split", 2024-09-19T04:57:03.055 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.055 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.056 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.056 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.056 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.056 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.056 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.056 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T04:57:03.056 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.056 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.056 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.056 INFO:teuthology.orchestra.run.smithi123.stdout: "config_checks_enabled": { 2024-09-19T04:57:03.056 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_checks_enabled", 2024-09-19T04:57:03.057 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.057 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.057 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.057 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.057 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.057 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.057 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.057 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T04:57:03.057 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.057 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.057 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.058 INFO:teuthology.orchestra.run.smithi123.stdout: "config_dashboard": { 2024-09-19T04:57:03.058 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_dashboard", 2024-09-19T04:57:03.058 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.058 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.058 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.058 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.058 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.058 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.058 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.058 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T04:57:03.058 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.058 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.059 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.059 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_alertmanager": { 2024-09-19T04:57:03.059 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_alertmanager", 2024-09-19T04:57:03.059 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.059 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.059 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.059 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T04:57:03.059 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.059 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.059 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.059 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-09-19T04:57:03.060 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.060 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.060 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.060 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_base": { 2024-09-19T04:57:03.060 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_base", 2024-09-19T04:57:03.060 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.060 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.060 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.060 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T04:57:03.060 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.061 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.061 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.061 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Container image name, without the tag", 2024-09-19T04:57:03.061 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.061 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.061 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.061 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_elasticsearch": { 2024-09-19T04:57:03.061 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_elasticsearch", 2024-09-19T04:57:03.061 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.061 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.062 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.062 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T04:57:03.062 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.062 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.062 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.062 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "elasticsearch container image", 2024-09-19T04:57:03.062 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.062 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.062 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.062 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_grafana": { 2024-09-19T04:57:03.063 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_grafana", 2024-09-19T04:57:03.063 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.063 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.063 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.063 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T04:57:03.063 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.063 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.063 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.063 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-09-19T04:57:03.063 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.063 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.064 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.064 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_haproxy": { 2024-09-19T04:57:03.064 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_haproxy", 2024-09-19T04:57:03.064 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.064 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.064 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.064 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T04:57:03.064 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.064 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.064 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.065 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "HAproxy container image", 2024-09-19T04:57:03.065 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.065 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.065 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.065 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_agent": { 2024-09-19T04:57:03.065 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_agent", 2024-09-19T04:57:03.065 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.065 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.065 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.065 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T04:57:03.065 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.066 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.066 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.066 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger agent container image", 2024-09-19T04:57:03.066 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.066 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.066 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.066 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_collector": { 2024-09-19T04:57:03.066 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_collector", 2024-09-19T04:57:03.066 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.066 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.066 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.067 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T04:57:03.067 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.067 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.067 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.067 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger collector container image", 2024-09-19T04:57:03.067 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.067 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.067 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.067 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_query": { 2024-09-19T04:57:03.067 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_query", 2024-09-19T04:57:03.067 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.068 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.068 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.068 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T04:57:03.068 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.068 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.068 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.068 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger query container image", 2024-09-19T04:57:03.068 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.068 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.068 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.069 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_keepalived": { 2024-09-19T04:57:03.069 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_keepalived", 2024-09-19T04:57:03.069 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.069 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.069 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.069 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T04:57:03.069 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.069 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.069 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.069 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Keepalived container image", 2024-09-19T04:57:03.069 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.069 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.070 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.070 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_loki": { 2024-09-19T04:57:03.070 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_loki", 2024-09-19T04:57:03.070 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.070 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.070 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.070 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T04:57:03.070 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.070 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.070 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.070 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Loki container image", 2024-09-19T04:57:03.071 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.071 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.071 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.071 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_node_exporter": { 2024-09-19T04:57:03.071 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_node_exporter", 2024-09-19T04:57:03.071 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.071 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.071 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.071 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T04:57:03.071 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.072 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.072 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.072 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-09-19T04:57:03.072 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.072 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.072 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.072 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_nvmeof": { 2024-09-19T04:57:03.072 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_nvmeof", 2024-09-19T04:57:03.072 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.072 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.072 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.073 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T04:57:03.073 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.073 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.073 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.073 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Nvme-of container image", 2024-09-19T04:57:03.073 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.073 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.073 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.073 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_prometheus": { 2024-09-19T04:57:03.073 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_prometheus", 2024-09-19T04:57:03.073 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.074 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.074 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.074 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T04:57:03.074 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.074 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.074 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.074 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-09-19T04:57:03.074 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.074 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.074 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.074 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_promtail": { 2024-09-19T04:57:03.075 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_promtail", 2024-09-19T04:57:03.075 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.075 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.075 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.075 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T04:57:03.075 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.075 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.075 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.075 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Promtail container image", 2024-09-19T04:57:03.075 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.075 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.075 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.076 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_samba": { 2024-09-19T04:57:03.076 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_samba", 2024-09-19T04:57:03.076 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.076 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.076 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.076 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T04:57:03.076 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.076 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.076 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.076 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Samba/SMB container image", 2024-09-19T04:57:03.077 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.077 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.077 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.077 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_snmp_gateway": { 2024-09-19T04:57:03.077 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_snmp_gateway", 2024-09-19T04:57:03.077 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.077 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.077 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.077 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T04:57:03.077 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.077 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.078 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.078 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SNMP Gateway container image", 2024-09-19T04:57:03.078 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.078 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.078 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.078 INFO:teuthology.orchestra.run.smithi123.stdout: "container_init": { 2024-09-19T04:57:03.078 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_init", 2024-09-19T04:57:03.078 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.078 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.078 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.079 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.079 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.079 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.079 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.079 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T04:57:03.079 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.079 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.079 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.079 INFO:teuthology.orchestra.run.smithi123.stdout: "daemon_cache_timeout": { 2024-09-19T04:57:03.079 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "daemon_cache_timeout", 2024-09-19T04:57:03.079 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.080 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.080 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.080 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-09-19T04:57:03.080 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.080 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.080 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.080 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T04:57:03.080 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.080 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.080 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.080 INFO:teuthology.orchestra.run.smithi123.stdout: "default_cephadm_command_timeout": { 2024-09-19T04:57:03.081 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T04:57:03.081 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.081 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.081 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.081 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "900", 2024-09-19T04:57:03.081 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.081 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.081 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.081 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T04:57:03.081 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.081 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.082 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.082 INFO:teuthology.orchestra.run.smithi123.stdout: "default_registry": { 2024-09-19T04:57:03.082 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_registry", 2024-09-19T04:57:03.082 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.082 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.082 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.082 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io", 2024-09-19T04:57:03.082 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.082 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.082 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.083 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T04:57:03.083 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.083 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.083 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.083 INFO:teuthology.orchestra.run.smithi123.stdout: "device_cache_timeout": { 2024-09-19T04:57:03.083 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_cache_timeout", 2024-09-19T04:57:03.083 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.083 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.083 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.083 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1800", 2024-09-19T04:57:03.084 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.084 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.084 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.084 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache device inventory", 2024-09-19T04:57:03.084 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.084 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.084 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.084 INFO:teuthology.orchestra.run.smithi123.stdout: "device_enhanced_scan": { 2024-09-19T04:57:03.084 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_enhanced_scan", 2024-09-19T04:57:03.084 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.085 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.085 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.085 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.085 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.085 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.085 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.085 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T04:57:03.085 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.085 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.085 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.086 INFO:teuthology.orchestra.run.smithi123.stdout: "facts_cache_timeout": { 2024-09-19T04:57:03.086 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "facts_cache_timeout", 2024-09-19T04:57:03.086 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.086 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.086 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.086 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-09-19T04:57:03.086 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.086 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.086 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.086 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache host facts data", 2024-09-19T04:57:03.086 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.086 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.087 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.087 INFO:teuthology.orchestra.run.smithi123.stdout: "grafana_dashboards_path": { 2024-09-19T04:57:03.087 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "grafana_dashboards_path", 2024-09-19T04:57:03.087 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.087 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.087 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.087 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T04:57:03.087 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.087 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.087 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.087 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T04:57:03.088 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.088 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.088 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.088 INFO:teuthology.orchestra.run.smithi123.stdout: "host_check_interval": { 2024-09-19T04:57:03.088 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "host_check_interval", 2024-09-19T04:57:03.088 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.088 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.088 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.088 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-09-19T04:57:03.088 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.088 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.089 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.089 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to perform a host check", 2024-09-19T04:57:03.089 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.092 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.092 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.092 INFO:teuthology.orchestra.run.smithi123.stdout: "hw_monitoring": { 2024-09-19T04:57:03.092 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hw_monitoring", 2024-09-19T04:57:03.092 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.092 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.092 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.092 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.093 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.093 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.093 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.093 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T04:57:03.093 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.093 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.093 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.093 INFO:teuthology.orchestra.run.smithi123.stdout: "inventory_list_all": { 2024-09-19T04:57:03.093 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "inventory_list_all", 2024-09-19T04:57:03.093 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.094 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.094 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.094 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.094 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.094 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.094 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.094 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T04:57:03.094 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.094 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.094 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.094 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.095 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.095 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.095 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.095 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.095 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.095 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.095 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.095 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.095 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.095 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.095 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.095 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.096 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.096 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.096 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.096 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.096 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.096 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.096 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.096 INFO:teuthology.orchestra.run.smithi123.stdout: "log_refresh_metadata": { 2024-09-19T04:57:03.096 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_refresh_metadata", 2024-09-19T04:57:03.096 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.097 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.097 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.097 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.097 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.097 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.097 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.097 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T04:57:03.097 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.097 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.097 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.097 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.098 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.098 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.098 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.098 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.098 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.098 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.098 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.098 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.098 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T04:57:03.098 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.098 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.098 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.099 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.099 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.099 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.099 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.099 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.099 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.099 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.099 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.099 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.099 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.100 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.100 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.100 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.100 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.100 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.100 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.100 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.100 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.100 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.100 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.100 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.101 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.101 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.101 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.101 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.101 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.101 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.101 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.101 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.101 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.101 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.102 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.102 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.102 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T04:57:03.102 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T04:57:03.102 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.102 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.102 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.102 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.102 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.102 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.103 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.103 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T04:57:03.103 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.103 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.103 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.103 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T04:57:03.103 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T04:57:03.103 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.103 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.103 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.103 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "*", 2024-09-19T04:57:03.104 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.104 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.104 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.104 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T04:57:03.104 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.104 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.104 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.104 INFO:teuthology.orchestra.run.smithi123.stdout: "max_count_per_host": { 2024-09-19T04:57:03.104 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_count_per_host", 2024-09-19T04:57:03.104 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.105 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.105 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.105 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:03.105 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.105 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.105 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.105 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of daemons per service per host", 2024-09-19T04:57:03.105 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.105 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.105 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.105 INFO:teuthology.orchestra.run.smithi123.stdout: "max_osd_draining_count": { 2024-09-19T04:57:03.106 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_osd_draining_count", 2024-09-19T04:57:03.106 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.106 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.106 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.106 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:03.106 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.106 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.106 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.106 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T04:57:03.106 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.106 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.106 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.107 INFO:teuthology.orchestra.run.smithi123.stdout: "migration_current": { 2024-09-19T04:57:03.107 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "migration_current", 2024-09-19T04:57:03.107 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.107 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.107 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.107 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.107 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.107 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.107 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.107 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "internal - do not modify", 2024-09-19T04:57:03.107 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.108 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.108 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.108 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-09-19T04:57:03.108 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-09-19T04:57:03.108 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.108 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.108 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.108 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "root", 2024-09-19T04:57:03.108 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.108 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.109 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.109 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm-package", 2024-09-19T04:57:03.109 INFO:teuthology.orchestra.run.smithi123.stdout: "root" 2024-09-19T04:57:03.109 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.109 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T04:57:03.109 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.109 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.109 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.109 INFO:teuthology.orchestra.run.smithi123.stdout: "oob_default_addr": { 2024-09-19T04:57:03.109 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "oob_default_addr", 2024-09-19T04:57:03.110 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.110 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.110 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.110 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "169.254.1.1", 2024-09-19T04:57:03.110 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.110 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.110 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.110 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T04:57:03.110 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.110 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.110 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.110 INFO:teuthology.orchestra.run.smithi123.stdout: "prometheus_alerts_path": { 2024-09-19T04:57:03.111 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus_alerts_path", 2024-09-19T04:57:03.111 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.111 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.111 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.111 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T04:57:03.111 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.111 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.111 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.111 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T04:57:03.111 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.111 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.111 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.112 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_insecure": { 2024-09-19T04:57:03.112 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_insecure", 2024-09-19T04:57:03.112 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.112 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.112 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.112 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.112 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.112 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.112 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.112 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T04:57:03.113 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.113 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.113 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.113 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_password": { 2024-09-19T04:57:03.113 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_password", 2024-09-19T04:57:03.113 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.113 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.113 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.113 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.113 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.113 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.114 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.114 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T04:57:03.114 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.114 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.114 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.114 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_url": { 2024-09-19T04:57:03.114 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_url", 2024-09-19T04:57:03.114 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.114 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.114 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.115 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.115 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.115 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.115 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.115 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T04:57:03.115 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.115 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.115 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.115 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_username": { 2024-09-19T04:57:03.115 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_username", 2024-09-19T04:57:03.115 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.116 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.116 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.116 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.116 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.116 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.116 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.116 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T04:57:03.116 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.116 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.116 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.117 INFO:teuthology.orchestra.run.smithi123.stdout: "secure_monitoring_stack": { 2024-09-19T04:57:03.117 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "secure_monitoring_stack", 2024-09-19T04:57:03.117 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.117 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.117 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.117 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.117 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.117 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.117 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.117 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T04:57:03.117 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.118 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.118 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.118 INFO:teuthology.orchestra.run.smithi123.stdout: "service_discovery_port": { 2024-09-19T04:57:03.118 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "service_discovery_port", 2024-09-19T04:57:03.118 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.118 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.118 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.118 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8765", 2024-09-19T04:57:03.118 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.118 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.119 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.119 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "cephadm service discovery port", 2024-09-19T04:57:03.119 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.119 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.119 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.119 INFO:teuthology.orchestra.run.smithi123.stdout: "ssh_config_file": { 2024-09-19T04:57:03.119 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssh_config_file", 2024-09-19T04:57:03.119 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.119 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.119 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.119 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.120 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.120 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.120 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.120 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T04:57:03.120 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.120 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.120 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.120 INFO:teuthology.orchestra.run.smithi123.stdout: "use_agent": { 2024-09-19T04:57:03.120 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_agent", 2024-09-19T04:57:03.120 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.121 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.121 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.121 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.121 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.121 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.121 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.121 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T04:57:03.121 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.121 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.121 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.122 INFO:teuthology.orchestra.run.smithi123.stdout: "use_repo_digest": { 2024-09-19T04:57:03.122 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_repo_digest", 2024-09-19T04:57:03.122 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.122 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.122 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.122 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.122 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.122 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.122 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.122 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T04:57:03.122 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.123 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.123 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.123 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_failed_host_check": { 2024-09-19T04:57:03.123 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_failed_host_check", 2024-09-19T04:57:03.123 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.123 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.123 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.123 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.123 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.123 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.123 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.124 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T04:57:03.124 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.124 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.124 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.124 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_daemons": { 2024-09-19T04:57:03.124 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_daemons", 2024-09-19T04:57:03.124 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.124 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.124 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.124 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.125 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.125 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.125 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.125 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T04:57:03.125 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.125 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.125 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.125 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_hosts": { 2024-09-19T04:57:03.125 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_hosts", 2024-09-19T04:57:03.125 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.125 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.126 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.126 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.126 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.126 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.126 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.126 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T04:57:03.126 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.126 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.126 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.126 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.126 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.127 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crash", 2024-09-19T04:57:03.127 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.127 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.127 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.127 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.127 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.127 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.127 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.127 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.127 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.127 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.128 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.128 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.128 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.128 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.128 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.128 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.128 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.128 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.128 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.128 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.129 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.129 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.129 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.129 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.129 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.129 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.129 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.129 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.129 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.129 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.129 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.130 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.130 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.130 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.130 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.130 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.130 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.130 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.130 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.130 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.130 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.131 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.131 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.131 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.131 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.131 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.131 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.131 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.131 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.131 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.131 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.131 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.131 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.132 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.132 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.132 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.132 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.132 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.132 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.132 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.132 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.132 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.132 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.133 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.133 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.133 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.133 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.133 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.133 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.133 INFO:teuthology.orchestra.run.smithi123.stdout: "retain_interval": { 2024-09-19T04:57:03.133 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retain_interval", 2024-09-19T04:57:03.133 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.133 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.134 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.134 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "31536000", 2024-09-19T04:57:03.134 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.134 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.134 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.134 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T04:57:03.134 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.134 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.134 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.134 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_recent_interval": { 2024-09-19T04:57:03.135 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_recent_interval", 2024-09-19T04:57:03.135 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.135 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.135 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.135 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1209600", 2024-09-19T04:57:03.135 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.135 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.135 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.135 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T04:57:03.135 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.135 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.135 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.135 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.136 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.136 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dashboard", 2024-09-19T04:57:03.136 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.136 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.136 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.136 INFO:teuthology.orchestra.run.smithi123.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T04:57:03.136 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T04:57:03.136 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.137 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.137 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.137 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:03.137 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.137 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.137 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.137 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.137 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.137 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.137 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.137 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T04:57:03.138 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T04:57:03.138 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.138 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.138 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.138 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.138 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.138 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.138 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.138 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.138 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.138 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.138 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.139 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T04:57:03.139 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T04:57:03.139 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.139 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.139 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.139 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.139 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.139 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.139 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.139 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.139 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.139 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.140 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.140 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_ENABLED": { 2024-09-19T04:57:03.140 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T04:57:03.140 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.140 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.140 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.140 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.140 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.140 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.140 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.141 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.141 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.141 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.141 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.141 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T04:57:03.141 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T04:57:03.141 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.141 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.141 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.141 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.142 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.142 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.142 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.142 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.142 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.142 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.142 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.142 INFO:teuthology.orchestra.run.smithi123.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T04:57:03.142 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T04:57:03.142 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.142 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.143 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.143 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.143 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.143 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.143 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.143 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.143 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.143 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.143 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.143 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T04:57:03.143 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T04:57:03.144 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.144 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.144 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.144 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.144 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.144 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.144 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.144 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.144 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.144 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.144 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.145 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T04:57:03.145 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T04:57:03.145 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.145 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.145 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.145 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.145 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.145 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.145 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.145 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.145 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.146 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.146 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.146 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T04:57:03.146 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T04:57:03.146 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.146 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.146 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.146 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.146 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.146 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.146 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.147 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.147 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.147 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.147 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.147 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T04:57:03.147 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T04:57:03.147 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.147 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.147 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.147 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.148 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.148 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.148 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.148 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.148 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.148 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.148 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.148 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T04:57:03.148 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T04:57:03.148 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.149 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.149 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.149 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.149 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.149 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.149 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.149 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.149 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.149 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.149 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.149 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T04:57:03.150 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T04:57:03.150 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.150 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.150 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.150 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.150 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.150 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.150 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.150 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.150 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.150 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.151 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.151 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T04:57:03.151 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T04:57:03.151 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.151 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.151 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.151 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.151 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.151 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.151 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.152 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.152 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.152 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.152 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.152 INFO:teuthology.orchestra.run.smithi123.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T04:57:03.152 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T04:57:03.152 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.152 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.152 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.152 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.153 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.153 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.153 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.153 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.153 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.153 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.153 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.153 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T04:57:03.153 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T04:57:03.154 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.154 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.154 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.154 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-09-19T04:57:03.154 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.154 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.154 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.154 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.154 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.154 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.154 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.154 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T04:57:03.155 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T04:57:03.155 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.155 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.155 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.155 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.155 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.155 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.155 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.155 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.155 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.155 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.155 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.156 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_URL": { 2024-09-19T04:57:03.156 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_URL", 2024-09-19T04:57:03.156 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.156 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.156 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.156 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.156 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.156 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.156 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.156 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.157 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.157 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.157 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.157 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T04:57:03.157 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T04:57:03.157 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.157 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.157 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.157 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-09-19T04:57:03.158 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.158 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.158 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.158 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.158 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.158 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.158 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.158 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T04:57:03.158 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T04:57:03.158 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.158 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.159 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.159 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.159 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.159 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.159 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.159 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.159 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.159 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.159 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.159 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T04:57:03.159 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T04:57:03.160 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.160 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.160 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.160 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.160 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.160 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.160 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.160 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.160 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.160 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.160 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.161 INFO:teuthology.orchestra.run.smithi123.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T04:57:03.161 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T04:57:03.161 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.161 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.161 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.161 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.161 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.161 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.161 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.161 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.161 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.162 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.162 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.162 INFO:teuthology.orchestra.run.smithi123.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T04:57:03.162 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T04:57:03.162 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.162 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.162 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.162 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.162 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.162 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.162 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.163 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.163 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.163 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.163 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.163 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T04:57:03.163 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T04:57:03.163 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.163 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.163 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.163 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.163 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.164 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.164 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.164 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.164 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.164 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.164 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.164 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T04:57:03.164 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T04:57:03.164 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.164 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.165 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.165 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.165 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.165 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.165 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.165 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.165 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.165 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.165 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.165 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T04:57:03.165 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T04:57:03.165 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.166 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.166 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.166 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.166 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.166 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.166 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.166 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.166 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.166 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.166 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.166 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T04:57:03.167 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T04:57:03.167 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.167 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.167 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.167 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.167 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.167 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.167 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.167 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.167 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.167 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.167 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.168 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T04:57:03.168 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T04:57:03.168 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.168 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.168 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.168 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.168 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.168 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.168 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.168 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.169 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.169 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.169 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.169 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T04:57:03.169 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T04:57:03.169 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.169 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.169 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.169 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.169 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.169 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.170 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.170 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.170 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.170 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.170 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.170 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T04:57:03.170 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T04:57:03.170 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.170 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.170 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.170 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.171 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.171 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.171 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.171 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.171 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.171 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.171 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.171 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T04:57:03.171 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T04:57:03.171 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.171 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.172 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.172 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.172 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.172 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.172 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.172 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.172 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.172 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.172 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.172 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T04:57:03.173 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T04:57:03.173 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.173 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.173 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.173 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.173 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.173 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.173 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.173 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.173 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.173 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.174 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.174 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T04:57:03.174 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T04:57:03.174 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.174 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.174 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.174 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.174 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.174 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.174 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.174 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.175 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.175 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.175 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.175 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T04:57:03.175 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T04:57:03.175 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.175 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.175 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.175 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T04:57:03.175 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.176 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.176 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.176 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.176 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.176 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.176 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.176 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T04:57:03.176 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T04:57:03.176 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.176 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.177 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.177 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:03.177 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.177 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.177 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.177 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.177 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.177 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.177 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.177 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T04:57:03.178 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T04:57:03.178 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.178 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.178 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.178 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8", 2024-09-19T04:57:03.178 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.178 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.178 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.178 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.178 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.178 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.179 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.179 INFO:teuthology.orchestra.run.smithi123.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T04:57:03.179 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T04:57:03.179 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.179 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.179 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.179 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "45", 2024-09-19T04:57:03.179 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.179 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.179 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.180 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.180 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.180 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.180 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.180 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T04:57:03.180 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T04:57:03.180 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.180 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.180 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.180 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.181 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.181 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.181 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.181 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.181 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.181 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.181 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.181 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T04:57:03.181 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T04:57:03.181 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.182 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.182 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.182 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-09-19T04:57:03.182 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.182 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.182 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.182 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.182 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.182 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.182 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.183 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T04:57:03.183 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T04:57:03.183 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.183 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.183 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.183 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.183 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.183 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.183 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.183 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.183 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.184 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.184 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.184 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T04:57:03.184 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T04:57:03.184 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.184 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.184 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.184 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.184 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.184 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.185 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.185 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.185 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.185 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.185 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.185 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T04:57:03.185 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T04:57:03.185 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.185 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.185 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.186 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-09-19T04:57:03.186 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.186 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.186 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.186 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.186 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.186 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.186 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.186 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T04:57:03.186 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T04:57:03.186 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.187 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.187 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.187 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:03.187 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.187 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.187 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.187 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.187 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.187 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.187 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.187 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T04:57:03.187 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T04:57:03.188 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.188 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.188 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.188 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-09-19T04:57:03.188 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.188 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.188 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.188 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.188 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.188 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.188 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.188 INFO:teuthology.orchestra.run.smithi123.stdout: "cross_origin_url": { 2024-09-19T04:57:03.189 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cross_origin_url", 2024-09-19T04:57:03.189 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.189 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.189 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.189 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.189 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.189 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.189 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.189 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.189 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.189 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.189 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.190 INFO:teuthology.orchestra.run.smithi123.stdout: "crt_file": { 2024-09-19T04:57:03.190 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crt_file", 2024-09-19T04:57:03.190 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.190 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.190 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.190 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.190 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.190 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.190 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.190 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.190 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.190 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.191 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.191 INFO:teuthology.orchestra.run.smithi123.stdout: "debug": { 2024-09-19T04:57:03.191 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "debug", 2024-09-19T04:57:03.191 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.191 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.191 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.191 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.191 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.191 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.191 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.191 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable/disable debug options", 2024-09-19T04:57:03.191 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.192 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.192 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.192 INFO:teuthology.orchestra.run.smithi123.stdout: "jwt_token_ttl": { 2024-09-19T04:57:03.192 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "jwt_token_ttl", 2024-09-19T04:57:03.192 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.192 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.192 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.192 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "28800", 2024-09-19T04:57:03.192 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.192 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.193 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.193 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.193 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.193 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.193 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.193 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-09-19T04:57:03.193 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-09-19T04:57:03.193 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.193 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.193 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.194 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.194 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.194 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.194 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.194 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.194 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.194 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.194 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.194 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.194 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.195 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.195 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.195 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.195 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.195 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.195 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.195 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.195 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.195 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.195 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.195 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.196 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.196 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.196 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.196 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.196 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.196 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.196 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.196 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.196 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.196 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.196 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.197 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.197 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.197 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.197 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.197 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.197 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.197 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.197 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.197 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.197 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.197 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.198 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.198 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.198 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.198 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.198 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.198 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.198 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.198 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.198 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.198 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.198 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.198 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.199 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.199 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.199 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.199 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.199 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.199 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.199 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.199 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.199 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.199 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.199 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.200 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.200 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.200 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.200 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.200 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.200 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.200 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.200 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.200 INFO:teuthology.orchestra.run.smithi123.stdout: "motd": { 2024-09-19T04:57:03.200 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "motd", 2024-09-19T04:57:03.200 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.201 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.201 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.201 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.201 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.201 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.201 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.201 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "The message of the day", 2024-09-19T04:57:03.201 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.201 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.201 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.201 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.201 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect_resolve_ip_addr": { 2024-09-19T04:57:03.202 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T04:57:03.202 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.202 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.202 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.202 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.202 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.202 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.202 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.202 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.202 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.202 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.202 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.203 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.203 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-09-19T04:57:03.203 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-09-19T04:57:03.203 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.203 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.203 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.203 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-09-19T04:57:03.203 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.203 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.203 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.203 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.204 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.204 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.204 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.204 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.204 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-09-19T04:57:03.204 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-09-19T04:57:03.204 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.204 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.204 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.204 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8080", 2024-09-19T04:57:03.204 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.205 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.205 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.205 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.205 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.205 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.205 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.205 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.205 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-09-19T04:57:03.205 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-09-19T04:57:03.205 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.205 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.206 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.206 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.206 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.206 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.206 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.206 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.206 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.206 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.206 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.206 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl_server_port": { 2024-09-19T04:57:03.207 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl_server_port", 2024-09-19T04:57:03.207 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.207 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.207 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.207 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8443", 2024-09-19T04:57:03.207 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.207 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.207 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.207 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.207 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.207 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.208 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.208 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-09-19T04:57:03.208 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-09-19T04:57:03.208 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.208 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.208 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.208 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "redirect", 2024-09-19T04:57:03.208 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.208 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.208 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.208 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.209 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect" 2024-09-19T04:57:03.209 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.209 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.209 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.209 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.209 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.209 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.209 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-09-19T04:57:03.209 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-09-19T04:57:03.209 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.209 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.209 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.209 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-09-19T04:57:03.210 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-09-19T04:57:03.210 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-09-19T04:57:03.210 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.210 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.210 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.210 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.210 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.210 INFO:teuthology.orchestra.run.smithi123.stdout: "url_prefix": { 2024-09-19T04:57:03.210 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url_prefix", 2024-09-19T04:57:03.210 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.210 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.211 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.211 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.211 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.211 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.211 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.211 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.211 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.211 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.211 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.211 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.211 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.212 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.212 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "devicehealth", 2024-09-19T04:57:03.212 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.212 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.212 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.212 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_monitoring": { 2024-09-19T04:57:03.212 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_monitoring", 2024-09-19T04:57:03.212 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.212 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.212 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.213 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.213 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.213 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.213 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.213 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "monitor device health metrics", 2024-09-19T04:57:03.213 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.213 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.213 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.213 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.213 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.213 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.214 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.214 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.214 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.214 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.214 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.214 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.214 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.214 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.214 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.214 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.214 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.215 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.215 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.215 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.215 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.215 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.215 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.215 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.215 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.215 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.215 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.215 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.216 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.216 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.216 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.216 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.216 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.216 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.216 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.216 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.216 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.216 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.217 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.217 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.217 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.217 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.217 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.217 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.217 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.217 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.217 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.217 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.217 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.217 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.218 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.218 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.218 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.218 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.218 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.218 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.218 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.218 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.218 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.218 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.218 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.218 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.219 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.219 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.219 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.219 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.219 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.219 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.219 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.219 INFO:teuthology.orchestra.run.smithi123.stdout: "mark_out_threshold": { 2024-09-19T04:57:03.219 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mark_out_threshold", 2024-09-19T04:57:03.219 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.219 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.220 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.220 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2419200", 2024-09-19T04:57:03.220 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.220 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.220 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.220 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T04:57:03.220 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.220 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.220 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.220 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": { 2024-09-19T04:57:03.220 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_name", 2024-09-19T04:57:03.221 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.221 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.221 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.221 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "device_health_metrics", 2024-09-19T04:57:03.221 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.221 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.221 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.221 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T04:57:03.221 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.221 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.221 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.221 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.222 INFO:teuthology.orchestra.run.smithi123.stdout: "retention_period": { 2024-09-19T04:57:03.222 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retention_period", 2024-09-19T04:57:03.222 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.222 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.222 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.222 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15552000", 2024-09-19T04:57:03.222 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.222 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.222 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.222 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain device health metrics", 2024-09-19T04:57:03.222 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.222 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.223 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.223 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_frequency": { 2024-09-19T04:57:03.223 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_frequency", 2024-09-19T04:57:03.223 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.223 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.223 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.223 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-09-19T04:57:03.223 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.223 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.223 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.223 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T04:57:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "self_heal": { 2024-09-19T04:57:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "self_heal", 2024-09-19T04:57:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T04:57:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-09-19T04:57:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-09-19T04:57:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-09-19T04:57:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T04:57:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_threshold": { 2024-09-19T04:57:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_threshold", 2024-09-19T04:57:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7257600", 2024-09-19T04:57:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T04:57:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "diskprediction_local", 2024-09-19T04:57:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "predict_interval": { 2024-09-19T04:57:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predict_interval", 2024-09-19T04:57:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-09-19T04:57:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "predictor_model": { 2024-09-19T04:57:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predictor_model", 2024-09-19T04:57:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "prophetstor", 2024-09-19T04:57:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-09-19T04:57:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-09-19T04:57:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-09-19T04:57:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "influx", 2024-09-19T04:57:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": false, 2024-09-19T04:57:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "influxdb python module not found", 2024-09-19T04:57:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "batch_size": { 2024-09-19T04:57:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "batch_size", 2024-09-19T04:57:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5000", 2024-09-19T04:57:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T04:57:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: "database": { 2024-09-19T04:57:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "database", 2024-09-19T04:57:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "ceph", 2024-09-19T04:57:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T04:57:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": { 2024-09-19T04:57:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hostname", 2024-09-19T04:57:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server hostname", 2024-09-19T04:57:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-09-19T04:57:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-09-19T04:57:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "30", 2024-09-19T04:57:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "5", 2024-09-19T04:57:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T04:57:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "password": { 2024-09-19T04:57:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "password", 2024-09-19T04:57:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "password of InfluxDB server user", 2024-09-19T04:57:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "port": { 2024-09-19T04:57:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "port", 2024-09-19T04:57:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8086", 2024-09-19T04:57:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server port", 2024-09-19T04:57:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-09-19T04:57:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-09-19T04:57:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "false", 2024-09-19T04:57:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:57:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "threads": { 2024-09-19T04:57:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threads", 2024-09-19T04:57:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-09-19T04:57:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-09-19T04:57:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "32", 2024-09-19T04:57:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T04:57:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "username": { 2024-09-19T04:57:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "username", 2024-09-19T04:57:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "username of InfluxDB server user", 2024-09-19T04:57:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "verify_ssl": { 2024-09-19T04:57:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "verify_ssl", 2024-09-19T04:57:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "true", 2024-09-19T04:57:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:57:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "insights", 2024-09-19T04:57:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "iostat", 2024-09-19T04:57:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "k8sevents", 2024-09-19T04:57:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_event_retention_days": { 2024-09-19T04:57:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ceph_event_retention_days", 2024-09-19T04:57:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7", 2024-09-19T04:57:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T04:57:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: "config_check_secs": { 2024-09-19T04:57:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_check_secs", 2024-09-19T04:57:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "10", 2024-09-19T04:57:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T04:57:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "localpool", 2024-09-19T04:57:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: "failure_domain": { 2024-09-19T04:57:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "failure_domain", 2024-09-19T04:57:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "host", 2024-09-19T04:57:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "failure domain for any created local pool", 2024-09-19T04:57:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T04:57:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": { 2024-09-19T04:57:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_size", 2024-09-19T04:57:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default min_size for any created local pool", 2024-09-19T04:57:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T04:57:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "num_rep": { 2024-09-19T04:57:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "num_rep", 2024-09-19T04:57:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3", 2024-09-19T04:57:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default replica count for any created local pool", 2024-09-19T04:57:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": { 2024-09-19T04:57:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_num", 2024-09-19T04:57:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "128", 2024-09-19T04:57:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T04:57:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "prefix": { 2024-09-19T04:57:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prefix", 2024-09-19T04:57:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name prefix for any created local pool", 2024-09-19T04:57:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "subtree": { 2024-09-19T04:57:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "subtree", 2024-09-19T04:57:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "rack", 2024-09-19T04:57:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T04:57:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T04:57:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mds_autoscaler", 2024-09-19T04:57:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirroring", 2024-09-19T04:57:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "nfs", 2024-09-19T04:57:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-09-19T04:57:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: "fail_fs": { 2024-09-19T04:57:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "fail_fs", 2024-09-19T04:57:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T04:57:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator": { 2024-09-19T04:57:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-09-19T04:57:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm", 2024-09-19T04:57:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "rook", 2024-09-19T04:57:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "test_orchestrator" 2024-09-19T04:57:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Orchestrator backend", 2024-09-19T04:57:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_perf_query", 2024-09-19T04:57:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_support", 2024-09-19T04:57:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_autoscaler", 2024-09-19T04:57:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-09-19T04:57:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-09-19T04:57:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-09-19T04:57:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "threshold": { 2024-09-19T04:57:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threshold", 2024-09-19T04:57:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-09-19T04:57:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1.0", 2024-09-19T04:57:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "scaling threshold", 2024-09-19T04:57:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T04:57:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "progress", 2024-09-19T04:57:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_pg_recovery_event": { 2024-09-19T04:57:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_pg_recovery_event", 2024-09-19T04:57:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T04:57:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-09-19T04:57:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-09-19T04:57:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.351 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.351 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.351 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.351 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.351 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.351 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.351 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.351 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.351 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.351 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.351 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.351 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.352 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.352 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.352 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.352 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.352 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.352 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.352 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.352 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.352 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.353 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.353 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.353 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.353 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.353 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.353 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.353 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.353 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.353 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.353 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.353 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.354 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.354 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.354 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.354 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.354 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.354 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.354 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.354 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.354 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.354 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.354 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.355 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.355 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.355 INFO:teuthology.orchestra.run.smithi123.stdout: "max_completed_events": { 2024-09-19T04:57:03.355 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_completed_events", 2024-09-19T04:57:03.355 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.355 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.355 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.355 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "50", 2024-09-19T04:57:03.355 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.355 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.355 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.356 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "number of past completed events to remember", 2024-09-19T04:57:03.356 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.356 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.356 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.356 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-09-19T04:57:03.356 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-09-19T04:57:03.356 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.356 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.356 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.356 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-09-19T04:57:03.357 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.357 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.357 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.357 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long the module is going to sleep", 2024-09-19T04:57:03.357 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.357 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.357 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.357 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.357 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.357 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus", 2024-09-19T04:57:03.358 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.358 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.358 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.358 INFO:teuthology.orchestra.run.smithi123.stdout: "cache": { 2024-09-19T04:57:03.358 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cache", 2024-09-19T04:57:03.358 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.358 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.358 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.358 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.358 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.358 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.358 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.358 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.359 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.359 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.359 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.359 INFO:teuthology.orchestra.run.smithi123.stdout: "exclude_perf_counters": { 2024-09-19T04:57:03.359 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "exclude_perf_counters", 2024-09-19T04:57:03.359 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.359 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.359 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.359 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.359 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.359 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.360 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.360 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T04:57:03.360 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T04:57:03.360 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.360 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.360 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.360 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.360 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.360 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.361 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.361 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.361 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.361 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.361 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.361 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.361 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.361 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.361 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.361 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.361 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.362 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.362 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.362 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.362 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.362 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.362 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.362 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.362 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.362 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.362 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.363 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.363 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.363 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.363 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.363 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.363 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.363 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.363 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.363 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.363 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.364 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.364 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.364 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.364 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.364 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.364 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.364 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.364 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.364 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.364 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.364 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.365 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.365 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.365 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.365 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.365 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.365 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.365 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.365 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.365 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.365 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.366 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.366 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.366 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.366 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.366 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.366 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.366 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.366 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.366 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.366 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.366 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools": { 2024-09-19T04:57:03.367 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools", 2024-09-19T04:57:03.367 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.367 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.367 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.367 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.367 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.367 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.367 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.367 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.367 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.367 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.367 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.368 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T04:57:03.368 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T04:57:03.368 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.368 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.368 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.368 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "300", 2024-09-19T04:57:03.368 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.368 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.368 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.368 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.368 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.369 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.369 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.369 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_interval": { 2024-09-19T04:57:03.369 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_interval", 2024-09-19T04:57:03.369 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:03.369 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.369 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.369 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15.0", 2024-09-19T04:57:03.369 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.369 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.370 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.370 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.370 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.370 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.370 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.370 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-09-19T04:57:03.370 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-09-19T04:57:03.370 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.370 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.370 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.370 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-09-19T04:57:03.371 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.371 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.371 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.371 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T04:57:03.371 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.371 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.371 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.371 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-09-19T04:57:03.371 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-09-19T04:57:03.371 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.371 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.372 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.372 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "9283", 2024-09-19T04:57:03.372 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.372 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.372 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.372 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T04:57:03.372 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.372 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.372 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.372 INFO:teuthology.orchestra.run.smithi123.stdout: "stale_cache_strategy": { 2024-09-19T04:57:03.372 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stale_cache_strategy", 2024-09-19T04:57:03.373 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.373 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.373 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.373 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "log", 2024-09-19T04:57:03.373 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.373 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.373 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.373 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.373 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.373 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.373 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.374 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-09-19T04:57:03.374 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-09-19T04:57:03.374 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.374 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.374 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.374 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "default", 2024-09-19T04:57:03.374 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.374 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.374 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.374 INFO:teuthology.orchestra.run.smithi123.stdout: "default", 2024-09-19T04:57:03.374 INFO:teuthology.orchestra.run.smithi123.stdout: "error" 2024-09-19T04:57:03.375 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.375 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.375 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.375 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.375 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.375 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-09-19T04:57:03.375 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-09-19T04:57:03.375 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.375 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.376 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.376 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-09-19T04:57:03.376 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-09-19T04:57:03.376 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-09-19T04:57:03.376 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.376 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.376 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.376 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.376 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.376 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.376 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.377 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_support", 2024-09-19T04:57:03.377 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.377 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.377 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.377 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.377 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.377 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.377 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.377 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.378 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.378 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.378 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.378 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.378 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.378 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.378 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.378 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.378 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.378 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.378 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.379 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.379 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.379 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.379 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.379 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.379 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.379 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.379 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.379 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.379 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.380 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.380 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.380 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.380 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.380 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.380 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.380 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.380 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.380 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.380 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.381 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.381 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.381 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.381 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.381 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.381 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.381 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.381 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.381 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.381 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.381 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.382 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.382 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.382 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.382 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.382 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.382 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.382 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.382 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.382 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.382 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.382 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.383 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.383 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.383 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.383 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.383 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.383 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.383 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.383 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.383 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_snap_create": { 2024-09-19T04:57:03.383 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_snap_create", 2024-09-19T04:57:03.384 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.384 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.384 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.384 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-09-19T04:57:03.384 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.384 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.384 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.384 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.384 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.384 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.384 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.385 INFO:teuthology.orchestra.run.smithi123.stdout: "mirror_snapshot_schedule": { 2024-09-19T04:57:03.385 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T04:57:03.385 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.385 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.385 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.385 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.385 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.385 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.385 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.385 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.385 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.386 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.386 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.386 INFO:teuthology.orchestra.run.smithi123.stdout: "trash_purge_schedule": { 2024-09-19T04:57:03.386 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "trash_purge_schedule", 2024-09-19T04:57:03.386 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.386 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.386 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.386 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.386 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.386 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.386 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.387 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.387 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.387 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.387 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.387 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.387 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.387 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "restful", 2024-09-19T04:57:03.387 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.387 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.387 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.388 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_auth": { 2024-09-19T04:57:03.388 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_auth", 2024-09-19T04:57:03.388 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.388 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.388 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.388 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.388 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.388 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.388 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.388 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.388 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.388 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.389 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.389 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-09-19T04:57:03.389 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-09-19T04:57:03.389 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.389 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.389 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.389 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.389 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.389 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.389 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.389 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.390 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.390 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.390 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.390 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.390 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.390 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.390 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.390 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.390 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.390 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.390 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.390 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.391 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.391 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.391 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.391 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.391 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.391 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.391 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.391 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.391 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.391 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.391 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.391 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.391 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.392 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.392 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.392 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.392 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.392 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.392 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.392 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.392 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.392 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.392 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.392 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.392 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.393 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.393 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.393 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.393 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.393 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.393 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.393 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.393 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.393 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.393 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.393 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.394 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.394 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.394 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.394 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.394 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.394 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.394 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.394 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.394 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.394 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.394 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.394 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.395 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.395 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.395 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.395 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.395 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.395 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.395 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.395 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.395 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.395 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.396 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.396 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-09-19T04:57:03.396 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-09-19T04:57:03.396 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.396 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.396 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.396 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.396 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.396 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.396 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.397 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.397 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.397 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.397 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.397 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-09-19T04:57:03.397 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-09-19T04:57:03.397 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.397 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.397 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.397 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.398 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.398 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.398 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.398 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.398 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.398 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.398 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.398 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.398 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.398 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.398 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rgw", 2024-09-19T04:57:03.398 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.399 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.399 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.399 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.399 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.399 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.399 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.399 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.399 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.399 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.399 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.399 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.399 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.399 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.400 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.400 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.400 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.400 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.400 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.400 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.400 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.400 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.400 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.400 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.400 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.401 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.401 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.401 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.401 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.401 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.401 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.401 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.401 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.401 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.401 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.401 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.402 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.402 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.402 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.402 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.402 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.402 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.402 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.402 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.402 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.402 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.403 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.403 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.403 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.403 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.403 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.403 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.403 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.403 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.403 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.403 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.404 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.404 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.404 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.404 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.404 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.404 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.404 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.404 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.404 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.405 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.405 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.405 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.405 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.405 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.405 INFO:teuthology.orchestra.run.smithi123.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T04:57:03.405 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T04:57:03.405 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.405 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.405 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.406 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-09-19T04:57:03.406 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.406 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.406 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.406 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T04:57:03.406 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.406 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.406 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.406 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.406 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.407 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rook", 2024-09-19T04:57:03.407 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.407 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.407 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.407 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.407 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.407 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.407 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.407 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.408 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.408 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.408 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.408 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.408 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.408 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.408 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.408 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.408 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.408 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.408 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.409 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.409 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.409 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.409 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.409 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.409 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.409 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.409 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.409 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.409 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.409 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.410 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.410 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.410 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.410 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.410 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.410 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.410 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.410 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.410 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.410 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.411 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.411 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.411 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.411 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.411 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.411 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.411 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.411 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.411 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.411 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.411 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.411 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.412 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.412 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.412 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.412 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.412 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.412 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.412 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.412 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.412 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.412 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.413 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.413 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.413 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.413 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.413 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.413 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.413 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.413 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.413 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.413 INFO:teuthology.orchestra.run.smithi123.stdout: "storage_class": { 2024-09-19T04:57:03.413 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "storage_class", 2024-09-19T04:57:03.414 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.414 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.414 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.414 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "local", 2024-09-19T04:57:03.414 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.414 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.414 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.414 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T04:57:03.414 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.414 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.414 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.414 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.415 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.415 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.415 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "selftest", 2024-09-19T04:57:03.415 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.415 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.415 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.415 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.415 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.415 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.415 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.416 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.416 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.416 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.416 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.416 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.416 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.416 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.416 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.416 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.416 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.416 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.417 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.417 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.417 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.417 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.417 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.417 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.417 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.417 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.417 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.417 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.418 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.418 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.418 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.418 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.418 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.418 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.418 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.418 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.418 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.418 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.418 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.418 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.419 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.419 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.419 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.419 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.419 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.419 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.419 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.419 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.419 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.419 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.419 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.420 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.420 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.420 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.420 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.420 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.420 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.420 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.420 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.420 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.420 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.420 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.420 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.421 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.421 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.421 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.421 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.421 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.421 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.421 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.421 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.421 INFO:teuthology.orchestra.run.smithi123.stdout: "roption1": { 2024-09-19T04:57:03.422 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption1", 2024-09-19T04:57:03.422 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.422 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.422 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.422 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.422 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.422 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.422 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.422 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.422 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.422 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.423 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.423 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.423 INFO:teuthology.orchestra.run.smithi123.stdout: "roption2": { 2024-09-19T04:57:03.423 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption2", 2024-09-19T04:57:03.423 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.423 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.423 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.423 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "xyz", 2024-09-19T04:57:03.423 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.423 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.423 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.424 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.424 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.424 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.424 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.424 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption1": { 2024-09-19T04:57:03.424 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption1", 2024-09-19T04:57:03.424 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.424 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.424 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.424 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.424 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.425 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.425 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.425 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.425 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.425 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.425 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.425 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption2": { 2024-09-19T04:57:03.425 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption2", 2024-09-19T04:57:03.425 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.425 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.425 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.426 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.426 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.426 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.426 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.426 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.426 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.426 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.426 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.426 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption3": { 2024-09-19T04:57:03.426 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption3", 2024-09-19T04:57:03.427 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-09-19T04:57:03.427 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.427 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.427 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.427 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.427 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.427 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.427 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.427 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.427 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.427 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.428 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption4": { 2024-09-19T04:57:03.428 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption4", 2024-09-19T04:57:03.428 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.428 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.428 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.428 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.428 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.428 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.428 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.428 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.428 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.429 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.429 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.429 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption5": { 2024-09-19T04:57:03.429 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption5", 2024-09-19T04:57:03.429 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.429 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.429 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.429 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.429 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.429 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.430 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.430 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.430 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.430 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.430 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.430 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption6": { 2024-09-19T04:57:03.430 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption6", 2024-09-19T04:57:03.430 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.430 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.430 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.431 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.431 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.431 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.431 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.431 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.431 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.431 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.431 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.431 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption7": { 2024-09-19T04:57:03.431 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption7", 2024-09-19T04:57:03.432 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.432 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.432 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.432 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.432 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-09-19T04:57:03.432 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "42", 2024-09-19T04:57:03.432 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.432 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.432 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.432 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.432 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.433 INFO:teuthology.orchestra.run.smithi123.stdout: "testkey": { 2024-09-19T04:57:03.433 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testkey", 2024-09-19T04:57:03.433 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.433 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.433 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.433 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.433 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.433 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.433 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.433 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.433 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.434 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.434 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.434 INFO:teuthology.orchestra.run.smithi123.stdout: "testlkey": { 2024-09-19T04:57:03.434 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testlkey", 2024-09-19T04:57:03.434 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.434 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.434 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.434 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.434 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.434 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.434 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.435 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.435 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.435 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.435 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.435 INFO:teuthology.orchestra.run.smithi123.stdout: "testnewline": { 2024-09-19T04:57:03.435 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testnewline", 2024-09-19T04:57:03.435 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.435 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.435 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.435 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.435 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.436 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.436 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.436 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.436 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.436 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.436 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.436 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.436 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.436 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snap_schedule", 2024-09-19T04:57:03.436 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.437 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.437 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.437 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_m_granularity": { 2024-09-19T04:57:03.437 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_m_granularity", 2024-09-19T04:57:03.437 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.437 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.437 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.437 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.437 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.437 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.437 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.438 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T04:57:03.438 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.438 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.438 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.438 INFO:teuthology.orchestra.run.smithi123.stdout: "dump_on_update": { 2024-09-19T04:57:03.438 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dump_on_update", 2024-09-19T04:57:03.438 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.438 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.438 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.438 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.439 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.439 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.439 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.439 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "dump database to debug log on update", 2024-09-19T04:57:03.439 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.439 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.439 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.439 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.439 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.439 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.440 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.440 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.440 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.440 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.440 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.440 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.440 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.440 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.440 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.440 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.440 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.441 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.441 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.441 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.441 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.441 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.441 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.441 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.441 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.441 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.441 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.442 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.442 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.442 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.442 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.442 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.442 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.442 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.442 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.442 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.442 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.443 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.443 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.443 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.443 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.443 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.443 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.443 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.443 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.443 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.443 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.443 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.444 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.444 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.444 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.444 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.444 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.444 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.444 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.444 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.444 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.444 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.444 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.445 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.445 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.445 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.445 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.445 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.445 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.445 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.445 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.445 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.445 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.445 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.446 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.446 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stats", 2024-09-19T04:57:03.446 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.446 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.446 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.446 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.446 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.446 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.446 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.446 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.447 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.447 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.447 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.447 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.447 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.447 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.447 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.447 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.447 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.447 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.447 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.448 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.448 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.448 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.448 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.448 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.448 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.448 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.448 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.448 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.448 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.448 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.449 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.449 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.449 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.449 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.449 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.449 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.449 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.449 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.449 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.449 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.450 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.450 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.450 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.450 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.450 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.450 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.450 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.450 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.450 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.450 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.450 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.450 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.451 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.451 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.451 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.451 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.451 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.451 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.451 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.451 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.451 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.451 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.452 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.452 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.452 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.452 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.452 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.452 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.452 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.452 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.452 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.453 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "status", 2024-09-19T04:57:03.453 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.453 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.453 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.453 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.453 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.453 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.453 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.453 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.453 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.453 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.454 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.454 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.454 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.454 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.454 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.454 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.454 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.454 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.454 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.454 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.454 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.454 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.455 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.455 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.455 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.455 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.455 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.455 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.455 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.455 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.455 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.455 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.455 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.455 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.455 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.456 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.456 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.456 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.456 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.456 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.456 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.456 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.456 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.456 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.456 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.456 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.457 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.457 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.457 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.457 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.457 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.457 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.457 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.457 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.457 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.457 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.458 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.458 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.458 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.458 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.458 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.458 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.458 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.458 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.458 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.458 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.458 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.458 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.458 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.459 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.459 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.459 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telegraf", 2024-09-19T04:57:03.459 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.459 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.459 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.459 INFO:teuthology.orchestra.run.smithi123.stdout: "address": { 2024-09-19T04:57:03.459 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "address", 2024-09-19T04:57:03.459 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.460 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.460 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.460 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T04:57:03.460 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.460 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.460 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.460 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.460 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.460 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.460 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.461 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-09-19T04:57:03.461 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-09-19T04:57:03.461 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.461 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.461 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.461 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15", 2024-09-19T04:57:03.461 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.461 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.461 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.461 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.461 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.461 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.462 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.462 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.462 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.462 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.462 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.462 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.462 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.462 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.462 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.462 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.462 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.462 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.463 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.463 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.463 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.463 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.463 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.463 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.463 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.463 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.463 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.463 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.463 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.464 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.464 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.464 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.464 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.464 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.464 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.464 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.464 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.464 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.464 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.464 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.465 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.465 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.465 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.465 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.465 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.465 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.465 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.465 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.465 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.465 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.465 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.466 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.466 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.466 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.466 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.466 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.466 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.466 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.466 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.466 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.466 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.467 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.467 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.467 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.467 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.467 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.467 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.467 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.467 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.467 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.467 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.467 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.467 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.468 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.468 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.468 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telemetry", 2024-09-19T04:57:03.468 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.468 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.468 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.468 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_basic": { 2024-09-19T04:57:03.468 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_basic", 2024-09-19T04:57:03.468 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.468 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.469 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.469 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.469 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.469 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.469 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.469 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T04:57:03.469 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.469 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.469 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.469 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_crash": { 2024-09-19T04:57:03.469 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_crash", 2024-09-19T04:57:03.469 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.470 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.470 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.470 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.470 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.470 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.470 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.470 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T04:57:03.470 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.470 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.470 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.470 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_device": { 2024-09-19T04:57:03.470 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_device", 2024-09-19T04:57:03.471 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.471 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.471 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.471 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.471 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.471 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.471 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.471 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T04:57:03.471 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.471 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.471 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.472 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_ident": { 2024-09-19T04:57:03.472 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_ident", 2024-09-19T04:57:03.472 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.472 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.472 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.472 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.472 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.472 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.472 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.472 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T04:57:03.472 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.473 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.473 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.473 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_perf": { 2024-09-19T04:57:03.473 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_perf", 2024-09-19T04:57:03.473 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.473 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.473 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.473 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.473 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.473 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.474 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.474 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T04:57:03.474 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.474 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.474 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.474 INFO:teuthology.orchestra.run.smithi123.stdout: "contact": { 2024-09-19T04:57:03.474 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "contact", 2024-09-19T04:57:03.474 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.474 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.474 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.474 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.475 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.475 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.475 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.475 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.475 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.475 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.475 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.475 INFO:teuthology.orchestra.run.smithi123.stdout: "description": { 2024-09-19T04:57:03.475 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "description", 2024-09-19T04:57:03.475 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.476 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.476 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.476 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.476 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.476 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.476 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.476 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.476 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.476 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.476 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.476 INFO:teuthology.orchestra.run.smithi123.stdout: "device_url": { 2024-09-19T04:57:03.477 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_url", 2024-09-19T04:57:03.477 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.477 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.477 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.477 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T04:57:03.477 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.477 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.477 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.477 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.477 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.477 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.478 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.478 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-09-19T04:57:03.478 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-09-19T04:57:03.478 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.478 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.478 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.478 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.478 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.478 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.478 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.478 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.479 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.479 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.479 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.479 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-09-19T04:57:03.479 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-09-19T04:57:03.479 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.479 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.479 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.479 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "24", 2024-09-19T04:57:03.479 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "8", 2024-09-19T04:57:03.480 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.480 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.480 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.480 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.480 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.480 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.480 INFO:teuthology.orchestra.run.smithi123.stdout: "last_opt_revision": { 2024-09-19T04:57:03.480 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "last_opt_revision", 2024-09-19T04:57:03.480 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.480 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.481 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.481 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1", 2024-09-19T04:57:03.481 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.481 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.481 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.481 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.481 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.481 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.481 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.481 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard": { 2024-09-19T04:57:03.482 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard", 2024-09-19T04:57:03.482 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.482 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.482 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.482 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.482 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.482 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.482 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.482 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.482 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.482 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.483 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.483 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard_description": { 2024-09-19T04:57:03.483 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard_description", 2024-09-19T04:57:03.483 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.483 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.483 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.483 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.483 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.483 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.484 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.484 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.484 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.484 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.484 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.484 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.484 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.484 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.484 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.484 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.485 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.485 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.485 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.485 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.485 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.485 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.485 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.485 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.485 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.485 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.485 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.485 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.486 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.486 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.486 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.486 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.486 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.486 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.486 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.486 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.486 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.486 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.487 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.487 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.487 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.487 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.487 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.487 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.487 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.487 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.487 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.487 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.488 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.488 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.488 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.488 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.488 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.488 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.488 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.488 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.488 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.488 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.488 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.488 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.488 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.489 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.489 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.489 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.489 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.489 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.489 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.489 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.489 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.489 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.489 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.489 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.490 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.490 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.490 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.490 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.490 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.490 INFO:teuthology.orchestra.run.smithi123.stdout: "organization": { 2024-09-19T04:57:03.490 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "organization", 2024-09-19T04:57:03.490 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.490 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.490 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.490 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.490 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.491 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.491 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.491 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.491 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.491 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.491 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.491 INFO:teuthology.orchestra.run.smithi123.stdout: "proxy": { 2024-09-19T04:57:03.491 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "proxy", 2024-09-19T04:57:03.491 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.491 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.491 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.491 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.491 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.492 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.492 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.492 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.492 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.492 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.492 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.492 INFO:teuthology.orchestra.run.smithi123.stdout: "url": { 2024-09-19T04:57:03.492 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url", 2024-09-19T04:57:03.492 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.492 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.492 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.493 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T04:57:03.493 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.493 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.493 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.493 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.493 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.493 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.493 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.493 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.493 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.494 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "test_orchestrator", 2024-09-19T04:57:03.494 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.494 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.494 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.494 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.494 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.494 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.494 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.494 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.494 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.494 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.495 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.495 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.495 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.495 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.495 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.495 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.495 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.495 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.495 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.495 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.495 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.495 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.496 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.496 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.496 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.496 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.496 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.496 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.496 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.496 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.496 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.496 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.496 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.497 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.497 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.497 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.497 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.497 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.497 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.497 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.497 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.497 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.497 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.497 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.498 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.498 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.498 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.498 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.498 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.498 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.498 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.498 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.498 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.498 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.498 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.498 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.498 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.499 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.499 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.499 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.499 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.499 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.499 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.499 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.499 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.499 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.500 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.500 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.500 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.500 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.500 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.500 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "volumes", 2024-09-19T04:57:03.500 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.500 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.500 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.501 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.501 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.501 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.501 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.501 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.501 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.501 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.501 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.501 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.501 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.501 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.501 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.502 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.502 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.502 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.502 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.502 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.502 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.502 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.502 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.502 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.503 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.503 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.503 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.503 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.503 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.503 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.503 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.503 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.503 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.503 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.503 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.504 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.504 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.504 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.504 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.504 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.504 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.504 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.504 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.504 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.504 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.505 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.505 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.505 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.505 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.505 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.505 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.505 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.505 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.505 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.505 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.505 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.506 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.506 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.506 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.506 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.506 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.506 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.506 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.506 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.506 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.506 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.507 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.507 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.507 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.507 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_clones": { 2024-09-19T04:57:03.507 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_clones", 2024-09-19T04:57:03.507 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.507 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.507 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.507 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4", 2024-09-19T04:57:03.507 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.507 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.508 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.508 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T04:57:03.508 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.508 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.508 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.508 INFO:teuthology.orchestra.run.smithi123.stdout: "periodic_async_work": { 2024-09-19T04:57:03.508 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "periodic_async_work", 2024-09-19T04:57:03.508 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.508 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.508 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.509 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.509 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.509 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.509 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.509 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Periodically check for async work", 2024-09-19T04:57:03.509 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.509 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.509 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.509 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_delay": { 2024-09-19T04:57:03.509 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_delay", 2024-09-19T04:57:03.509 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.510 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.510 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.510 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-09-19T04:57:03.510 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.510 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.510 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.510 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T04:57:03.510 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.510 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.510 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.511 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_no_wait": { 2024-09-19T04:57:03.511 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T04:57:03.511 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.511 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.511 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.511 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-09-19T04:57:03.511 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.511 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.511 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.511 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T04:57:03.511 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.512 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.512 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.512 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.512 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.512 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix", 2024-09-19T04:57:03.512 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-09-19T04:57:03.512 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-09-19T04:57:03.512 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-09-19T04:57:03.512 INFO:teuthology.orchestra.run.smithi123.stdout: "discovery_interval": { 2024-09-19T04:57:03.513 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "discovery_interval", 2024-09-19T04:57:03.513 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-09-19T04:57:03.513 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.513 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.513 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "100", 2024-09-19T04:57:03.513 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.513 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.513 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.513 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.513 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.513 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.513 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.514 INFO:teuthology.orchestra.run.smithi123.stdout: "identifier": { 2024-09-19T04:57:03.514 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "identifier", 2024-09-19T04:57:03.514 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.514 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.514 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.514 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.514 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.514 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.514 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.514 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.515 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.515 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.515 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.515 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-09-19T04:57:03.515 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-09-19T04:57:03.515 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-09-19T04:57:03.515 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.515 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.515 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-09-19T04:57:03.515 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.515 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.515 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.516 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.516 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.516 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.516 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.516 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-09-19T04:57:03.516 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-09-19T04:57:03.516 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.516 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.516 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.516 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.516 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.517 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.517 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.517 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.517 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.517 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.517 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.517 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.517 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.518 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.518 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.518 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.518 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.518 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.518 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-09-19T04:57:03.518 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-09-19T04:57:03.518 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.518 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.518 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.518 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.518 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.519 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.519 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.519 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.519 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.519 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.519 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.519 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-09-19T04:57:03.519 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-09-19T04:57:03.519 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.519 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.519 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.519 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-09-19T04:57:03.519 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.520 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.520 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-09-19T04:57:03.520 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-09-19T04:57:03.520 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-09-19T04:57:03.520 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-09-19T04:57:03.520 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-09-19T04:57:03.520 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-09-19T04:57:03.520 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-09-19T04:57:03.520 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.520 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.520 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.520 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.520 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.521 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-09-19T04:57:03.521 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-09-19T04:57:03.521 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-09-19T04:57:03.521 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.521 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-09-19T04:57:03.522 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-09-19T04:57:03.522 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.522 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.522 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.522 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.522 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.522 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.523 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.523 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_host": { 2024-09-19T04:57:03.523 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_host", 2024-09-19T04:57:03.523 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.524 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.524 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.524 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-09-19T04:57:03.524 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.524 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.524 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.525 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.525 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.525 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.525 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.525 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_port": { 2024-09-19T04:57:03.525 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_port", 2024-09-19T04:57:03.525 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-09-19T04:57:03.526 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.526 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.526 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10051", 2024-09-19T04:57:03.526 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.526 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.526 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.526 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.526 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.527 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.527 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.527 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_sender": { 2024-09-19T04:57:03.527 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_sender", 2024-09-19T04:57:03.527 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-09-19T04:57:03.527 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-09-19T04:57:03.527 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-09-19T04:57:03.528 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T04:57:03.528 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-09-19T04:57:03.528 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-09-19T04:57:03.528 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-09-19T04:57:03.528 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-09-19T04:57:03.528 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-09-19T04:57:03.528 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-09-19T04:57:03.528 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-09-19T04:57:03.529 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.529 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.529 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.529 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.529 INFO:teuthology.orchestra.run.smithi123.stdout: "services": { 2024-09-19T04:57:03.529 INFO:teuthology.orchestra.run.smithi123.stdout: "dashboard": "https://172.21.15.123:8443/" 2024-09-19T04:57:03.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.530 INFO:teuthology.orchestra.run.smithi123.stdout: "always_on_modules": { 2024-09-19T04:57:03.530 INFO:teuthology.orchestra.run.smithi123.stdout: "octopus": [ 2024-09-19T04:57:03.530 INFO:teuthology.orchestra.run.smithi123.stdout: "balancer", 2024-09-19T04:57:03.530 INFO:teuthology.orchestra.run.smithi123.stdout: "crash", 2024-09-19T04:57:03.530 INFO:teuthology.orchestra.run.smithi123.stdout: "devicehealth", 2024-09-19T04:57:03.530 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator", 2024-09-19T04:57:03.530 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscaler", 2024-09-19T04:57:03.530 INFO:teuthology.orchestra.run.smithi123.stdout: "progress", 2024-09-19T04:57:03.531 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_support", 2024-09-19T04:57:03.531 INFO:teuthology.orchestra.run.smithi123.stdout: "status", 2024-09-19T04:57:03.531 INFO:teuthology.orchestra.run.smithi123.stdout: "telemetry", 2024-09-19T04:57:03.531 INFO:teuthology.orchestra.run.smithi123.stdout: "volumes" 2024-09-19T04:57:03.531 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.531 INFO:teuthology.orchestra.run.smithi123.stdout: "pacific": [ 2024-09-19T04:57:03.531 INFO:teuthology.orchestra.run.smithi123.stdout: "balancer", 2024-09-19T04:57:03.531 INFO:teuthology.orchestra.run.smithi123.stdout: "crash", 2024-09-19T04:57:03.531 INFO:teuthology.orchestra.run.smithi123.stdout: "devicehealth", 2024-09-19T04:57:03.531 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator", 2024-09-19T04:57:03.531 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscaler", 2024-09-19T04:57:03.531 INFO:teuthology.orchestra.run.smithi123.stdout: "progress", 2024-09-19T04:57:03.532 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_support", 2024-09-19T04:57:03.532 INFO:teuthology.orchestra.run.smithi123.stdout: "status", 2024-09-19T04:57:03.532 INFO:teuthology.orchestra.run.smithi123.stdout: "telemetry", 2024-09-19T04:57:03.532 INFO:teuthology.orchestra.run.smithi123.stdout: "volumes" 2024-09-19T04:57:03.532 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.532 INFO:teuthology.orchestra.run.smithi123.stdout: "quincy": [ 2024-09-19T04:57:03.532 INFO:teuthology.orchestra.run.smithi123.stdout: "balancer", 2024-09-19T04:57:03.532 INFO:teuthology.orchestra.run.smithi123.stdout: "crash", 2024-09-19T04:57:03.532 INFO:teuthology.orchestra.run.smithi123.stdout: "devicehealth", 2024-09-19T04:57:03.532 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator", 2024-09-19T04:57:03.532 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscaler", 2024-09-19T04:57:03.532 INFO:teuthology.orchestra.run.smithi123.stdout: "progress", 2024-09-19T04:57:03.532 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_support", 2024-09-19T04:57:03.533 INFO:teuthology.orchestra.run.smithi123.stdout: "status", 2024-09-19T04:57:03.533 INFO:teuthology.orchestra.run.smithi123.stdout: "telemetry", 2024-09-19T04:57:03.533 INFO:teuthology.orchestra.run.smithi123.stdout: "volumes" 2024-09-19T04:57:03.533 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.533 INFO:teuthology.orchestra.run.smithi123.stdout: "reef": [ 2024-09-19T04:57:03.533 INFO:teuthology.orchestra.run.smithi123.stdout: "balancer", 2024-09-19T04:57:03.533 INFO:teuthology.orchestra.run.smithi123.stdout: "crash", 2024-09-19T04:57:03.533 INFO:teuthology.orchestra.run.smithi123.stdout: "devicehealth", 2024-09-19T04:57:03.533 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator", 2024-09-19T04:57:03.533 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscaler", 2024-09-19T04:57:03.533 INFO:teuthology.orchestra.run.smithi123.stdout: "progress", 2024-09-19T04:57:03.533 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_support", 2024-09-19T04:57:03.533 INFO:teuthology.orchestra.run.smithi123.stdout: "status", 2024-09-19T04:57:03.533 INFO:teuthology.orchestra.run.smithi123.stdout: "telemetry", 2024-09-19T04:57:03.533 INFO:teuthology.orchestra.run.smithi123.stdout: "volumes" 2024-09-19T04:57:03.534 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T04:57:03.534 INFO:teuthology.orchestra.run.smithi123.stdout: "squid": [ 2024-09-19T04:57:03.534 INFO:teuthology.orchestra.run.smithi123.stdout: "balancer", 2024-09-19T04:57:03.534 INFO:teuthology.orchestra.run.smithi123.stdout: "crash", 2024-09-19T04:57:03.534 INFO:teuthology.orchestra.run.smithi123.stdout: "devicehealth", 2024-09-19T04:57:03.534 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator", 2024-09-19T04:57:03.534 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscaler", 2024-09-19T04:57:03.534 INFO:teuthology.orchestra.run.smithi123.stdout: "progress", 2024-09-19T04:57:03.534 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_support", 2024-09-19T04:57:03.534 INFO:teuthology.orchestra.run.smithi123.stdout: "status", 2024-09-19T04:57:03.534 INFO:teuthology.orchestra.run.smithi123.stdout: "telemetry", 2024-09-19T04:57:03.534 INFO:teuthology.orchestra.run.smithi123.stdout: "volumes" 2024-09-19T04:57:03.534 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T04:57:03.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.535 INFO:teuthology.orchestra.run.smithi123.stdout: "last_failure_osd_epoch": 87, 2024-09-19T04:57:03.535 INFO:teuthology.orchestra.run.smithi123.stdout: "active_clients": [ 2024-09-19T04:57:03.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.535 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "devicehealth", 2024-09-19T04:57:03.535 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-09-19T04:57:03.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.535 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-09-19T04:57:03.535 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-09-19T04:57:03.535 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2980857290 2024-09-19T04:57:03.535 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.535 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T04:57:03.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.535 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "libcephsqlite", 2024-09-19T04:57:03.536 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-09-19T04:57:03.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.536 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-09-19T04:57:03.536 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-09-19T04:57:03.536 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4205149568 2024-09-19T04:57:03.536 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.536 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T04:57:03.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.536 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_support", 2024-09-19T04:57:03.536 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-09-19T04:57:03.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.536 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-09-19T04:57:03.536 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-09-19T04:57:03.537 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2982102291 2024-09-19T04:57:03.537 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.537 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T04:57:03.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.537 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snap_schedule", 2024-09-19T04:57:03.537 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-09-19T04:57:03.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.538 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-09-19T04:57:03.538 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-09-19T04:57:03.538 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 577908330 2024-09-19T04:57:03.538 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.538 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T04:57:03.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.539 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snap_schedule", 2024-09-19T04:57:03.539 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-09-19T04:57:03.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.539 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-09-19T04:57:03.539 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-09-19T04:57:03.540 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 906139912 2024-09-19T04:57:03.540 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.540 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T04:57:03.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:03.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.540 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "volumes", 2024-09-19T04:57:03.541 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-09-19T04:57:03.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T04:57:03.541 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-09-19T04:57:03.541 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-09-19T04:57:03.541 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3199893108 2024-09-19T04:57:03.541 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.542 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T04:57:03.542 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:03.542 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T04:57:03.542 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:57:03.544 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:02 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1683205766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:57:03.545 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:02 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3219357783' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T04:57:03.545 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:02 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1683205766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:57:03.545 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:02 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3219357783' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T04:57:03.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:02.099+0000 7fc9a3000640 1 -- 172.21.15.123:0/3219357783 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc9840a33c0 msgr2=0x7fc9840a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:03.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:02.099+0000 7fc9a3000640 1 --2- 172.21.15.123:0/3219357783 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc9840a33c0 0x7fc9840a5880 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc99c10ebd0 tx=0x7fc988005f90 comp rx=0 tx=0).stop 2024-09-19T04:57:03.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:02.099+0000 7fc9a3000640 1 -- 172.21.15.123:0/3219357783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc99c069380 msgr2=0x7fc99c19a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:03.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:02.099+0000 7fc9a3000640 1 --2- 172.21.15.123:0/3219357783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc99c069380 0x7fc99c19a3f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc99002a960 tx=0x7fc990003b20 comp rx=0 tx=0).stop 2024-09-19T04:57:03.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:02.099+0000 7fc9a3000640 1 -- 172.21.15.123:0/3219357783 shutdown_connections 2024-09-19T04:57:03.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:02.099+0000 7fc9a3000640 1 --2- 172.21.15.123:0/3219357783 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc9840a33c0 0x7fc9840a5880 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:03.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:02.099+0000 7fc9a3000640 1 --2- 172.21.15.123:0/3219357783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc99c1078f0 0x7fc99c19ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:03.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:02.099+0000 7fc9a3000640 1 --2- 172.21.15.123:0/3219357783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc99c069d00 0x7fc99c19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:03.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:02.099+0000 7fc9a3000640 1 --2- 172.21.15.123:0/3219357783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc99c069380 0x7fc99c19a3f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:03.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:02.099+0000 7fc9a3000640 1 -- 172.21.15.123:0/3219357783 >> 172.21.15.123:0/3219357783 conn(0x7fc99c076550 msgr2=0x7fc99c0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:03.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:02.099+0000 7fc9a3000640 1 -- 172.21.15.123:0/3219357783 shutdown_connections 2024-09-19T04:57:03.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:02.099+0000 7fc9a3000640 1 -- 172.21.15.123:0/3219357783 wait complete. 2024-09-19T04:57:03.614 DEBUG:tasks.check_counter:Getting stats from a 2024-09-19T04:57:03.614 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-19T04:57:03.614 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:02 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1683205766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:57:03.615 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:02 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3219357783' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T04:57:03.658 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:03 smithi193 ceph-mon[35592]: pgmap v321: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T04:57:03.658 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:03 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:03.662 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:03 smithi123 ceph-mon[30511]: pgmap v321: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T04:57:03.662 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:03 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:03.664 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:03 smithi183 ceph-mon[35333]: pgmap v321: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T04:57:03.664 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:03 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:04.517 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:05.127 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:57:05.128 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:57:05.128 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 990, 2024-09-19T04:57:05.128 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 963, 2024-09-19T04:57:05.128 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 1964656, 2024-09-19T04:57:05.128 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 365194, 2024-09-19T04:57:05.128 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 5, 2024-09-19T04:57:05.128 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-09-19T04:57:05.128 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.154366089, 2024-09-19T04:57:05.128 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.071139393, 2024-09-19T04:57:05.128 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.065587546, 2024-09-19T04:57:05.129 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.020957117, 2024-09-19T04:57:05.129 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:05.129 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 962, 2024-09-19T04:57:05.129 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.059301491, 2024-09-19T04:57:05.129 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000061643 2024-09-19T04:57:05.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.129 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:05.129 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.129 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.129 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.129 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1716352, 2024-09-19T04:57:05.129 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 130192 2024-09-19T04:57:05.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.130 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:57:05.130 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1377, 2024-09-19T04:57:05.130 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 1531, 2024-09-19T04:57:05.130 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 384202, 2024-09-19T04:57:05.130 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 877763, 2024-09-19T04:57:05.130 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 7, 2024-09-19T04:57:05.130 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 5, 2024-09-19T04:57:05.130 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.130697028, 2024-09-19T04:57:05.130 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.047965192, 2024-09-19T04:57:05.130 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.035632909, 2024-09-19T04:57:05.130 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.036300077, 2024-09-19T04:57:05.130 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:05.131 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1529, 2024-09-19T04:57:05.131 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.081745540, 2024-09-19T04:57:05.131 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000053463 2024-09-19T04:57:05.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.131 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:05.131 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.131 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.131 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.131 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-19T04:57:05.131 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 441728 2024-09-19T04:57:05.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.131 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:57:05.131 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 4, 2024-09-19T04:57:05.132 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 6, 2024-09-19T04:57:05.132 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 1629, 2024-09-19T04:57:05.132 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 1284, 2024-09-19T04:57:05.132 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 5, 2024-09-19T04:57:05.132 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 4, 2024-09-19T04:57:05.132 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.002444346, 2024-09-19T04:57:05.132 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000103366, 2024-09-19T04:57:05.132 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.002050977, 2024-09-19T04:57:05.132 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000104617, 2024-09-19T04:57:05.132 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:05.132 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 4, 2024-09-19T04:57:05.132 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.003458634, 2024-09-19T04:57:05.132 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000864658 2024-09-19T04:57:05.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.133 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:05.133 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.133 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.133 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.133 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:57:05.133 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:57:05.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.133 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-09-19T04:57:05.133 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-09-19T04:57:05.133 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-09-19T04:57:05.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.134 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-09-19T04:57:05.134 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-19T04:57:05.134 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-19T04:57:05.134 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3537, 2024-09-19T04:57:05.134 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.086859907, 2024-09-19T04:57:05.134 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000024557 2024-09-19T04:57:05.134 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:05.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.134 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-09-19T04:57:05.134 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-19T04:57:05.134 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-19T04:57:05.134 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.134 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.134 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.134 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:05.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.135 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-09-19T04:57:05.135 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 0, 2024-09-19T04:57:05.135 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 0, 2024-09-19T04:57:05.135 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-09-19T04:57:05.135 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.135 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.135 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.135 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-09-19T04:57:05.135 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 0, 2024-09-19T04:57:05.135 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-09-19T04:57:05.135 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-09-19T04:57:05.135 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 0, 2024-09-19T04:57:05.136 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2024-09-19T04:57:05.136 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-09-19T04:57:05.136 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 2, 2024-09-19T04:57:05.136 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 0, 2024-09-19T04:57:05.136 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 2, 2024-09-19T04:57:05.136 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 0, 2024-09-19T04:57:05.136 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 2, 2024-09-19T04:57:05.136 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 0, 2024-09-19T04:57:05.136 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 0, 2024-09-19T04:57:05.136 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 0, 2024-09-19T04:57:05.136 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 1, 2024-09-19T04:57:05.136 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 0, 2024-09-19T04:57:05.136 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 0, 2024-09-19T04:57:05.136 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 0, 2024-09-19T04:57:05.137 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 0, 2024-09-19T04:57:05.137 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:57:05.137 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-09-19T04:57:05.137 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2024-09-19T04:57:05.137 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-09-19T04:57:05.137 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-09-19T04:57:05.137 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2024-09-19T04:57:05.137 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2024-09-19T04:57:05.137 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2024-09-19T04:57:05.137 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2024-09-19T04:57:05.137 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-09-19T04:57:05.137 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:57:05.137 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-09-19T04:57:05.137 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 0, 2024-09-19T04:57:05.138 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 0, 2024-09-19T04:57:05.138 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-09-19T04:57:05.138 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:57:05.138 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-09-19T04:57:05.138 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:57:05.138 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:57:05.138 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:57:05.139 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:57:05.139 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:57:05.140 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-09-19T04:57:05.140 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:57:05.140 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:57:05.140 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:57:05.140 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:57:05.140 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:57:05.140 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:57:05.140 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 0, 2024-09-19T04:57:05.140 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:57:05.140 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 0, 2024-09-19T04:57:05.140 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 0 2024-09-19T04:57:05.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.140 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-09-19T04:57:05.141 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2024-09-19T04:57:05.141 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2024-09-19T04:57:05.141 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2024-09-19T04:57:05.141 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 0, 2024-09-19T04:57:05.141 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 0, 2024-09-19T04:57:05.141 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-09-19T04:57:05.141 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-09-19T04:57:05.141 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:57:05.141 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 0, 2024-09-19T04:57:05.141 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 0, 2024-09-19T04:57:05.141 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-09-19T04:57:05.141 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-09-19T04:57:05.141 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-09-19T04:57:05.141 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:57:05.142 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:57:05.142 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-09-19T04:57:05.142 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-09-19T04:57:05.142 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:57:05.142 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:57:05.142 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:57:05.142 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2024-09-19T04:57:05.142 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-09-19T04:57:05.142 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:57:05.142 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-09-19T04:57:05.142 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-09-19T04:57:05.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.142 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-09-19T04:57:05.142 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-09-19T04:57:05.143 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 0, 2024-09-19T04:57:05.143 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-09-19T04:57:05.143 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-09-19T04:57:05.143 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 17, 2024-09-19T04:57:05.143 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-09-19T04:57:05.143 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-09-19T04:57:05.143 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 0, 2024-09-19T04:57:05.143 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-09-19T04:57:05.143 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-09-19T04:57:05.143 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 2, 2024-09-19T04:57:05.143 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 2, 2024-09-19T04:57:05.143 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-09-19T04:57:05.144 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-09-19T04:57:05.144 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-09-19T04:57:05.144 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 4225960, 2024-09-19T04:57:05.144 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 4223355, 2024-09-19T04:57:05.144 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-09-19T04:57:05.144 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.144 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.144 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.144 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 17 2024-09-19T04:57:05.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.144 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-09-19T04:57:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 4, 2024-09-19T04:57:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 19, 2024-09-19T04:57:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 15, 2024-09-19T04:57:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 2, 2024-09-19T04:57:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 16, 2024-09-19T04:57:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 14, 2024-09-19T04:57:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 2, 2024-09-19T04:57:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 17, 2024-09-19T04:57:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 15, 2024-09-19T04:57:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 0, 2024-09-19T04:57:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 0, 2024-09-19T04:57:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 0, 2024-09-19T04:57:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 47552, 2024-09-19T04:57:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 215328 2024-09-19T04:57:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.146 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-09-19T04:57:05.146 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 0, 2024-09-19T04:57:05.146 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2024-09-19T04:57:05.146 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 0, 2024-09-19T04:57:05.146 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 0, 2024-09-19T04:57:05.146 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 0, 2024-09-19T04:57:05.146 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-09-19T04:57:05.146 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.146 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.146 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.146 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-09-19T04:57:05.146 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.146 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.147 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.147 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-09-19T04:57:05.147 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.147 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.147 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.147 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-09-19T04:57:05.147 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.147 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.147 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.147 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-09-19T04:57:05.148 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.148 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.148 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.148 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-09-19T04:57:05.148 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.148 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.148 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.149 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-09-19T04:57:05.149 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.149 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.149 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.149 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-09-19T04:57:05.149 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.149 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.149 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.149 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-09-19T04:57:05.149 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.149 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.150 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.150 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-09-19T04:57:05.150 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.150 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.150 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.150 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-09-19T04:57:05.150 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.150 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.150 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.150 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-09-19T04:57:05.151 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.151 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.151 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.151 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-09-19T04:57:05.151 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.152 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.152 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.152 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-09-19T04:57:05.152 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.152 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.152 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.152 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-09-19T04:57:05.152 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.152 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.152 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.153 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-09-19T04:57:05.153 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.153 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.153 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.153 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-09-19T04:57:05.153 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.153 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.153 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.153 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-09-19T04:57:05.153 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.153 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.154 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.154 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-09-19T04:57:05.154 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.154 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.154 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.154 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-09-19T04:57:05.154 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.154 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.154 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.154 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-09-19T04:57:05.154 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.155 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.155 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.155 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-09-19T04:57:05.155 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.155 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.155 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.155 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-09-19T04:57:05.155 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.155 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.155 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.155 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-09-19T04:57:05.156 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.156 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.156 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.156 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-09-19T04:57:05.156 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.156 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.156 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.156 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-09-19T04:57:05.156 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.156 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.156 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.157 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-09-19T04:57:05.157 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.157 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.157 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.157 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-09-19T04:57:05.157 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.157 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.157 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.157 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-09-19T04:57:05.157 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.158 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.158 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.158 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:57:05.158 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:57:05.158 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-09-19T04:57:05.158 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.158 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.158 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.158 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:05.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.158 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-09-19T04:57:05.158 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-09-19T04:57:05.158 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 3, 2024-09-19T04:57:05.158 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 2, 2024-09-19T04:57:05.159 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2024-09-19T04:57:05.159 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-09-19T04:57:05.159 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 0, 2024-09-19T04:57:05.159 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 0, 2024-09-19T04:57:05.159 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 1742, 2024-09-19T04:57:05.159 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-09-19T04:57:05.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.159 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-09-19T04:57:05.159 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:57:05.159 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-09-19T04:57:05.159 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:57:05.159 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:57:05.159 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:57:05.159 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:57:05.160 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:57:05.160 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:57:05.160 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:57:05.160 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:57:05.160 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:57:05.160 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:57:05.160 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:57:05.160 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:57:05.160 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:57:05.160 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-09-19T04:57:05.160 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:57:05.160 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-09-19T04:57:05.161 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:57:05.161 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:57:05.161 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:57:05.161 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:57:05.161 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:57:05.161 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:57:05.161 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:57:05.161 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-09-19T04:57:05.161 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:57:05.161 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:57:05.161 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:57:05.161 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-09-19T04:57:05.161 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-09-19T04:57:05.161 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-09-19T04:57:05.162 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:57:05.162 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:57:05.162 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:57:05.162 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:57:05.162 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 197490, 2024-09-19T04:57:05.162 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 56, 2024-09-19T04:57:05.162 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:57:05.162 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-09-19T04:57:05.162 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-09-19T04:57:05.163 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-09-19T04:57:05.163 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:57:05.163 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-09-19T04:57:05.163 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:57:05.163 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-09-19T04:57:05.163 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8128, 2024-09-19T04:57:05.163 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 179, 2024-09-19T04:57:05.163 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:57:05.163 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:57:05.163 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-09-19T04:57:05.163 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-09-19T04:57:05.163 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 15136, 2024-09-19T04:57:05.163 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 40, 2024-09-19T04:57:05.164 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-09-19T04:57:05.164 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-09-19T04:57:05.164 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-09-19T04:57:05.164 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-09-19T04:57:05.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.164 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-09-19T04:57:05.164 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-09-19T04:57:05.164 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-09-19T04:57:05.164 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 2114, 2024-09-19T04:57:05.164 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 21, 2024-09-19T04:57:05.164 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 3, 2024-09-19T04:57:05.164 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 2114, 2024-09-19T04:57:05.165 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-09-19T04:57:05.165 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2114, 2024-09-19T04:57:05.165 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.402965201, 2024-09-19T04:57:05.165 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000663654 2024-09-19T04:57:05.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.165 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-09-19T04:57:05.165 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-09-19T04:57:05.165 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2114, 2024-09-19T04:57:05.165 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 2115 2024-09-19T04:57:05.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.165 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 2114, 2024-09-19T04:57:05.165 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 2114, 2024-09-19T04:57:05.165 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 0, 2024-09-19T04:57:05.166 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-09-19T04:57:05.166 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-09-19T04:57:05.166 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 1360, 2024-09-19T04:57:05.166 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 0, 2024-09-19T04:57:05.166 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 752, 2024-09-19T04:57:05.166 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 0, 2024-09-19T04:57:05.166 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 0, 2024-09-19T04:57:05.166 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-09-19T04:57:05.166 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-09-19T04:57:05.166 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-09-19T04:57:05.166 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-09-19T04:57:05.166 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 0, 2024-09-19T04:57:05.166 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-09-19T04:57:05.167 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-09-19T04:57:05.167 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-09-19T04:57:05.167 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 1, 2024-09-19T04:57:05.167 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 0, 2024-09-19T04:57:05.167 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:57:05.167 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-09-19T04:57:05.167 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:57:05.167 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-09-19T04:57:05.167 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-09-19T04:57:05.167 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-09-19T04:57:05.167 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:57:05.167 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-09-19T04:57:05.167 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:57:05.167 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-09-19T04:57:05.168 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-09-19T04:57:05.168 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-09-19T04:57:05.168 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-09-19T04:57:05.168 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-09-19T04:57:05.168 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-09-19T04:57:05.168 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-09-19T04:57:05.168 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-09-19T04:57:05.168 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-09-19T04:57:05.168 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-09-19T04:57:05.168 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-09-19T04:57:05.168 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-09-19T04:57:05.168 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-09-19T04:57:05.168 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-09-19T04:57:05.168 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-09-19T04:57:05.169 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-09-19T04:57:05.169 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-09-19T04:57:05.169 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 95, 2024-09-19T04:57:05.169 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-09-19T04:57:05.169 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 11, 2024-09-19T04:57:05.169 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 6, 2024-09-19T04:57:05.169 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 6, 2024-09-19T04:57:05.169 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-09-19T04:57:05.169 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-09-19T04:57:05.169 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 0, 2024-09-19T04:57:05.169 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 2, 2024-09-19T04:57:05.169 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 0 2024-09-19T04:57:05.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.169 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-09-19T04:57:05.170 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 0, 2024-09-19T04:57:05.170 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:57:05.170 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-09-19T04:57:05.170 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-09-19T04:57:05.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.170 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-09-19T04:57:05.170 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 0 2024-09-19T04:57:05.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.170 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-09-19T04:57:05.170 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-09-19T04:57:05.170 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:57:05.170 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-09-19T04:57:05.170 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 0, 2024-09-19T04:57:05.170 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 0, 2024-09-19T04:57:05.171 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 0, 2024-09-19T04:57:05.171 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-09-19T04:57:05.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.171 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:57:05.171 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-19T04:57:05.171 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-19T04:57:05.171 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-19T04:57:05.171 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 2367, 2024-09-19T04:57:05.171 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 2121581, 2024-09-19T04:57:05.171 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:05.171 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 2367, 2024-09-19T04:57:05.171 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-19T04:57:05.171 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-19T04:57:05.171 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2367, 2024-09-19T04:57:05.172 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 2121581, 2024-09-19T04:57:05.172 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-19T04:57:05.172 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.172 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.172 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.172 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:05.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.172 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-09-19T04:57:05.172 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-19T04:57:05.172 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-19T04:57:05.172 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-19T04:57:05.172 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-19T04:57:05.172 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-19T04:57:05.172 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:05.173 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:05.173 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 2114, 2024-09-19T04:57:05.173 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 31661, 2024-09-19T04:57:05.173 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2114, 2024-09-19T04:57:05.173 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 31661, 2024-09-19T04:57:05.173 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-19T04:57:05.173 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.173 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.173 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.173 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:05.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.173 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-09-19T04:57:05.173 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-19T04:57:05.174 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-09-19T04:57:05.174 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-19T04:57:05.174 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-19T04:57:05.174 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-19T04:57:05.174 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:05.174 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:05.174 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 2114, 2024-09-19T04:57:05.174 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 2114, 2024-09-19T04:57:05.174 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2114, 2024-09-19T04:57:05.174 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 2114, 2024-09-19T04:57:05.174 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-19T04:57:05.174 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.174 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.174 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.175 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:05.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.175 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:57:05.175 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-19T04:57:05.175 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-19T04:57:05.175 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-19T04:57:05.175 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-19T04:57:05.175 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-19T04:57:05.175 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:05.175 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:05.175 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-19T04:57:05.175 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-19T04:57:05.175 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-09-19T04:57:05.175 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-09-19T04:57:05.176 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-19T04:57:05.176 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.176 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.176 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.176 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:05.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:05.176 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x56393f8c0be0": { 2024-09-19T04:57:05.176 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-19T04:57:05.176 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-19T04:57:05.176 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-19T04:57:05.176 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-19T04:57:05.176 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-19T04:57:05.176 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:05.176 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:05.177 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-19T04:57:05.177 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-19T04:57:05.177 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-09-19T04:57:05.177 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-09-19T04:57:05.177 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-19T04:57:05.177 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:05.177 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:05.177 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:05.177 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:05.177 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:05.177 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:57:05.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: pgmap v322: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-09-19T04:57:05.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:05.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:05.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:05.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:05.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.679 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-19T04:57:05.679 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-19T04:57:05.679 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: Unable to set osd_memory_target on smithi193 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T04:57:05.679 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:05 smithi123 ceph-mon[30511]: Unable to set osd_memory_target on smithi183 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T04:57:05.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: pgmap v322: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-09-19T04:57:05.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:05.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:05.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:05.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:05.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-19T04:57:05.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-19T04:57:05.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: Unable to set osd_memory_target on smithi193 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T04:57:05.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:05 smithi193 ceph-mon[35592]: Unable to set osd_memory_target on smithi183 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T04:57:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: pgmap v322: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-09-19T04:57:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:05.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:05.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-19T04:57:05.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-19T04:57:05.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: Unable to set osd_memory_target on smithi193 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T04:57:05.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:05 smithi183 ceph-mon[35333]: Unable to set osd_memory_target on smithi183 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T04:57:05.994 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-19T04:57:05.994 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-19T04:57:05.994 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-19T04:57:05.994 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-19T04:57:05.994 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-19T04:57:05.994 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:57:05.994 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:57:05.994 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-09-19T04:57:05.994 DEBUG:tasks.check_counter:Getting stats from d 2024-09-19T04:57:05.995 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-19T04:57:06.389 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:07.016 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:57:07.017 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:57:07.017 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 2118, 2024-09-19T04:57:07.017 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2120, 2024-09-19T04:57:07.017 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 701103, 2024-09-19T04:57:07.017 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 673531, 2024-09-19T04:57:07.017 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 6, 2024-09-19T04:57:07.017 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2024-09-19T04:57:07.017 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.195538290, 2024-09-19T04:57:07.017 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.061668385, 2024-09-19T04:57:07.017 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.054433221, 2024-09-19T04:57:07.018 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.061391619, 2024-09-19T04:57:07.018 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:07.018 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2118, 2024-09-19T04:57:07.018 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.083771996, 2024-09-19T04:57:07.018 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000039552 2024-09-19T04:57:07.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.018 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:07.018 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.018 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.018 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.018 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:57:07.018 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:57:07.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.019 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:57:07.019 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 258, 2024-09-19T04:57:07.019 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 231, 2024-09-19T04:57:07.019 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 1718268, 2024-09-19T04:57:07.019 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 134090, 2024-09-19T04:57:07.019 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 3, 2024-09-19T04:57:07.019 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2024-09-19T04:57:07.019 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.046315898, 2024-09-19T04:57:07.019 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.015228353, 2024-09-19T04:57:07.019 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.045296551, 2024-09-19T04:57:07.019 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000044828, 2024-09-19T04:57:07.019 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:07.019 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 230, 2024-09-19T04:57:07.019 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.015480435, 2024-09-19T04:57:07.020 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000067306 2024-09-19T04:57:07.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.020 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:07.020 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.020 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.020 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.020 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1717712, 2024-09-19T04:57:07.020 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 133392 2024-09-19T04:57:07.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.020 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:57:07.020 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 7, 2024-09-19T04:57:07.021 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 165, 2024-09-19T04:57:07.021 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 3660, 2024-09-19T04:57:07.021 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 442750, 2024-09-19T04:57:07.021 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 6, 2024-09-19T04:57:07.021 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 4, 2024-09-19T04:57:07.021 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.019849534, 2024-09-19T04:57:07.021 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.011389118, 2024-09-19T04:57:07.021 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.005554565, 2024-09-19T04:57:07.021 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000026509, 2024-09-19T04:57:07.021 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:07.021 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 163, 2024-09-19T04:57:07.021 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.026988610, 2024-09-19T04:57:07.022 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000165574 2024-09-19T04:57:07.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.022 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:07.022 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.022 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.022 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.022 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-19T04:57:07.022 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 442432 2024-09-19T04:57:07.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.022 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-09-19T04:57:07.022 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-09-19T04:57:07.022 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-09-19T04:57:07.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.023 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-09-19T04:57:07.023 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-19T04:57:07.023 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-19T04:57:07.023 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3564, 2024-09-19T04:57:07.023 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.087265096, 2024-09-19T04:57:07.023 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000024485 2024-09-19T04:57:07.023 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:07.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.023 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-09-19T04:57:07.023 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-19T04:57:07.023 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-19T04:57:07.023 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.024 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.024 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.024 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:07.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.024 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-09-19T04:57:07.024 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 0, 2024-09-19T04:57:07.024 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 0, 2024-09-19T04:57:07.024 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-09-19T04:57:07.024 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.024 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.024 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.025 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-09-19T04:57:07.025 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 0, 2024-09-19T04:57:07.025 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-09-19T04:57:07.025 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-09-19T04:57:07.025 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 0, 2024-09-19T04:57:07.025 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2024-09-19T04:57:07.025 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-09-19T04:57:07.025 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 14, 2024-09-19T04:57:07.025 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 0, 2024-09-19T04:57:07.025 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 14, 2024-09-19T04:57:07.025 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 0, 2024-09-19T04:57:07.025 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 14, 2024-09-19T04:57:07.026 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 0, 2024-09-19T04:57:07.026 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 0, 2024-09-19T04:57:07.026 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 0, 2024-09-19T04:57:07.026 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 1, 2024-09-19T04:57:07.026 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 0, 2024-09-19T04:57:07.026 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 0, 2024-09-19T04:57:07.026 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 0, 2024-09-19T04:57:07.026 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 0, 2024-09-19T04:57:07.026 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:57:07.026 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-09-19T04:57:07.026 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2024-09-19T04:57:07.026 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-09-19T04:57:07.026 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-09-19T04:57:07.027 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2024-09-19T04:57:07.027 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2024-09-19T04:57:07.027 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2024-09-19T04:57:07.027 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2024-09-19T04:57:07.027 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-09-19T04:57:07.027 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:57:07.027 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-09-19T04:57:07.027 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 0, 2024-09-19T04:57:07.027 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 0, 2024-09-19T04:57:07.027 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-09-19T04:57:07.027 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:57:07.027 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-09-19T04:57:07.028 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:57:07.028 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:57:07.028 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:57:07.028 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:57:07.028 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:57:07.028 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-09-19T04:57:07.028 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:57:07.028 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:57:07.028 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:57:07.028 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:57:07.028 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:57:07.028 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:57:07.028 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 0, 2024-09-19T04:57:07.029 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:57:07.029 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 0, 2024-09-19T04:57:07.029 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 0 2024-09-19T04:57:07.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.029 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-09-19T04:57:07.029 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2024-09-19T04:57:07.029 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2024-09-19T04:57:07.029 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2024-09-19T04:57:07.029 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 0, 2024-09-19T04:57:07.029 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 0, 2024-09-19T04:57:07.029 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-09-19T04:57:07.029 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-09-19T04:57:07.029 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:57:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 0, 2024-09-19T04:57:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 0, 2024-09-19T04:57:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-09-19T04:57:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-09-19T04:57:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-09-19T04:57:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:57:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:57:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-09-19T04:57:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-09-19T04:57:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:57:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:57:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:57:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2024-09-19T04:57:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-09-19T04:57:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:57:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-09-19T04:57:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-09-19T04:57:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-09-19T04:57:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-09-19T04:57:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 0, 2024-09-19T04:57:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-09-19T04:57:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-09-19T04:57:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 55, 2024-09-19T04:57:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-09-19T04:57:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-09-19T04:57:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 0, 2024-09-19T04:57:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-09-19T04:57:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-09-19T04:57:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 2, 2024-09-19T04:57:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 2, 2024-09-19T04:57:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-09-19T04:57:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-09-19T04:57:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-09-19T04:57:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 4295210, 2024-09-19T04:57:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 4292605, 2024-09-19T04:57:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-09-19T04:57:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 55 2024-09-19T04:57:07.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.033 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-09-19T04:57:07.033 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 4, 2024-09-19T04:57:07.033 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 40, 2024-09-19T04:57:07.033 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 36, 2024-09-19T04:57:07.033 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 2, 2024-09-19T04:57:07.033 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 13, 2024-09-19T04:57:07.033 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 11, 2024-09-19T04:57:07.033 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 14, 2024-09-19T04:57:07.033 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 51, 2024-09-19T04:57:07.033 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 37, 2024-09-19T04:57:07.033 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 0, 2024-09-19T04:57:07.033 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 0, 2024-09-19T04:57:07.034 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 0, 2024-09-19T04:57:07.034 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 47736, 2024-09-19T04:57:07.034 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 215328 2024-09-19T04:57:07.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.034 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-09-19T04:57:07.034 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 0, 2024-09-19T04:57:07.034 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2024-09-19T04:57:07.034 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 0, 2024-09-19T04:57:07.034 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 0, 2024-09-19T04:57:07.034 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 0, 2024-09-19T04:57:07.034 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-09-19T04:57:07.034 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.034 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.035 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.035 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-09-19T04:57:07.035 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.035 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.035 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.035 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-09-19T04:57:07.035 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.035 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.035 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.035 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-09-19T04:57:07.035 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.036 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.036 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.036 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-09-19T04:57:07.036 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.036 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.036 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.036 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-09-19T04:57:07.036 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.036 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.036 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.037 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-09-19T04:57:07.037 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.037 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.037 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.037 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-09-19T04:57:07.037 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.037 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.037 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.037 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-09-19T04:57:07.037 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.037 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.037 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.038 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-09-19T04:57:07.038 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.038 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.038 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.038 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-09-19T04:57:07.038 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.038 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.038 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.038 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-09-19T04:57:07.038 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.039 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.039 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.039 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-09-19T04:57:07.039 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.039 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.040 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.040 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-09-19T04:57:07.040 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.040 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.040 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.040 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-09-19T04:57:07.040 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.040 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.040 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.041 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-09-19T04:57:07.041 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.041 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.041 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.041 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-09-19T04:57:07.041 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.041 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.042 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.042 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-09-19T04:57:07.042 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.042 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.042 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.042 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-09-19T04:57:07.042 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.042 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.042 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.042 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-09-19T04:57:07.042 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.043 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.043 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.043 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-09-19T04:57:07.043 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.043 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.043 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.043 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-09-19T04:57:07.043 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.043 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.043 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.043 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-09-19T04:57:07.044 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.044 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.044 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.044 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-09-19T04:57:07.044 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.044 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.044 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.044 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-09-19T04:57:07.044 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.044 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.044 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.045 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-09-19T04:57:07.045 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.045 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.045 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.045 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-09-19T04:57:07.045 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.045 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.045 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.045 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-09-19T04:57:07.046 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.046 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.046 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.046 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-09-19T04:57:07.046 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.046 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.046 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.046 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:57:07.046 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:57:07.046 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-09-19T04:57:07.046 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-09-19T04:57:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 2, 2024-09-19T04:57:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 4, 2024-09-19T04:57:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 2, 2024-09-19T04:57:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 4, 2024-09-19T04:57:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-09-19T04:57:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 0, 2024-09-19T04:57:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 0, 2024-09-19T04:57:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 2870, 2024-09-19T04:57:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-09-19T04:57:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-09-19T04:57:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:57:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-09-19T04:57:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:57:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:57:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:57:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:57:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:57:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:57:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:57:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:57:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:57:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:57:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:57:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:57:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:57:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-09-19T04:57:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:57:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-09-19T04:57:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:57:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:57:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:57:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:57:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:57:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:57:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:57:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-09-19T04:57:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:57:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:57:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:57:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-09-19T04:57:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-09-19T04:57:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-09-19T04:57:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:57:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:57:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:57:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:57:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 181313, 2024-09-19T04:57:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 47, 2024-09-19T04:57:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:57:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-09-19T04:57:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-09-19T04:57:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-09-19T04:57:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:57:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-09-19T04:57:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:57:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-09-19T04:57:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8128, 2024-09-19T04:57:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 179, 2024-09-19T04:57:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:57:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:57:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-09-19T04:57:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-09-19T04:57:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 23584, 2024-09-19T04:57:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 88, 2024-09-19T04:57:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-09-19T04:57:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-09-19T04:57:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-09-19T04:57:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-09-19T04:57:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-09-19T04:57:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-09-19T04:57:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-09-19T04:57:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 2121, 2024-09-19T04:57:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 21, 2024-09-19T04:57:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 3, 2024-09-19T04:57:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 2121, 2024-09-19T04:57:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-09-19T04:57:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2121, 2024-09-19T04:57:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.262968712, 2024-09-19T04:57:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000595459 2024-09-19T04:57:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-09-19T04:57:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-09-19T04:57:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2121, 2024-09-19T04:57:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 2122 2024-09-19T04:57:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 2121, 2024-09-19T04:57:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 2121, 2024-09-19T04:57:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 0, 2024-09-19T04:57:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-09-19T04:57:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-09-19T04:57:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 1364, 2024-09-19T04:57:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 0, 2024-09-19T04:57:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 755, 2024-09-19T04:57:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 0, 2024-09-19T04:57:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 0, 2024-09-19T04:57:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-09-19T04:57:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-09-19T04:57:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-09-19T04:57:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-09-19T04:57:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 0, 2024-09-19T04:57:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-09-19T04:57:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-09-19T04:57:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-09-19T04:57:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 1, 2024-09-19T04:57:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 0, 2024-09-19T04:57:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:57:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-09-19T04:57:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:57:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-09-19T04:57:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-09-19T04:57:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-09-19T04:57:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:57:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-09-19T04:57:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:57:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-09-19T04:57:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-09-19T04:57:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-09-19T04:57:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-09-19T04:57:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-09-19T04:57:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-09-19T04:57:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-09-19T04:57:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-09-19T04:57:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-09-19T04:57:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-09-19T04:57:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-09-19T04:57:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-09-19T04:57:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-09-19T04:57:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-09-19T04:57:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-09-19T04:57:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-09-19T04:57:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-09-19T04:57:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 95, 2024-09-19T04:57:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-09-19T04:57:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 11, 2024-09-19T04:57:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 4, 2024-09-19T04:57:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 4, 2024-09-19T04:57:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-09-19T04:57:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-09-19T04:57:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 0, 2024-09-19T04:57:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 2, 2024-09-19T04:57:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 0 2024-09-19T04:57:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.059 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-09-19T04:57:07.059 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 0, 2024-09-19T04:57:07.059 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:57:07.059 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-09-19T04:57:07.059 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-09-19T04:57:07.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.059 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-09-19T04:57:07.059 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 0 2024-09-19T04:57:07.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.059 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-09-19T04:57:07.059 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-09-19T04:57:07.059 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:57:07.059 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-09-19T04:57:07.059 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 0, 2024-09-19T04:57:07.060 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 0, 2024-09-19T04:57:07.060 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 0, 2024-09-19T04:57:07.060 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-09-19T04:57:07.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.061 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:57:07.061 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-19T04:57:07.061 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-19T04:57:07.061 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-19T04:57:07.061 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 2379, 2024-09-19T04:57:07.061 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 2192785, 2024-09-19T04:57:07.061 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:07.061 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 2379, 2024-09-19T04:57:07.061 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-19T04:57:07.061 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-19T04:57:07.061 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2379, 2024-09-19T04:57:07.061 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 2192785, 2024-09-19T04:57:07.061 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-19T04:57:07.062 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.062 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.062 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.062 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:07.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.062 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-09-19T04:57:07.063 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-19T04:57:07.063 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-19T04:57:07.063 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-19T04:57:07.063 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-19T04:57:07.063 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-19T04:57:07.063 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:07.063 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:07.063 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 2121, 2024-09-19T04:57:07.063 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 100911, 2024-09-19T04:57:07.063 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2121, 2024-09-19T04:57:07.063 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 100911, 2024-09-19T04:57:07.064 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-19T04:57:07.064 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.064 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.064 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.064 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:07.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.065 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-09-19T04:57:07.065 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-19T04:57:07.065 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-09-19T04:57:07.065 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-19T04:57:07.065 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-19T04:57:07.066 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-19T04:57:07.066 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:07.066 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:07.066 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 2121, 2024-09-19T04:57:07.066 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 2121, 2024-09-19T04:57:07.066 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2121, 2024-09-19T04:57:07.066 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 2121, 2024-09-19T04:57:07.066 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-19T04:57:07.066 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.067 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.067 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.067 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:07.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.067 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:57:07.067 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-19T04:57:07.067 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-19T04:57:07.067 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-19T04:57:07.067 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-19T04:57:07.067 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-19T04:57:07.067 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:07.067 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:07.067 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-19T04:57:07.068 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-19T04:57:07.068 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-09-19T04:57:07.068 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-09-19T04:57:07.068 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-19T04:57:07.068 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.068 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.068 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.068 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:07.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:07.068 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x55bdae3d0be0": { 2024-09-19T04:57:07.068 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-19T04:57:07.068 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-19T04:57:07.068 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-19T04:57:07.069 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-19T04:57:07.069 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-19T04:57:07.069 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:07.069 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:07.069 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-19T04:57:07.069 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-19T04:57:07.069 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-09-19T04:57:07.069 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-09-19T04:57:07.069 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-19T04:57:07.069 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:07.069 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:07.069 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:07.069 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:07.069 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:07.070 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:57:07.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:07 smithi193 ceph-mon[35592]: pgmap v323: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-19T04:57:07.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:07 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:07.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:07 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:07.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:07 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:07 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:07 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:07 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:07 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:07 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.492 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:07 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.492 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:07 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.492 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:07 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:07.492 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:07 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:07.492 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:07 smithi193 ceph-mon[35592]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:07.535 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:07 smithi123 ceph-mon[30511]: pgmap v323: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-19T04:57:07.535 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:07 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:07.535 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:07 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:07.535 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:07 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.535 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:07 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.535 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:07 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.535 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:07 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.535 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:07 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.535 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:07 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.536 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:07 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.536 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:07 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.536 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:07 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:07.536 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:07 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:07.536 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:07 smithi123 ceph-mon[30511]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:07.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:07 smithi183 ceph-mon[35333]: pgmap v323: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-19T04:57:07.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:07 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:07.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:07 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:07.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:07 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:07 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:07 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:07 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:07 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:07 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:07 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:07 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:07.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:07 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:07.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:07 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:07.601 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:07 smithi183 ceph-mon[35333]: from='mgr.25133 ' entity='mgr.x' 2024-09-19T04:57:07.896 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-09-19T04:57:07.897 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-09-19T04:57:07.897 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update=0 2024-09-19T04:57:07.897 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update_receipt=0 2024-09-19T04:57:07.897 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=0 2024-09-19T04:57:07.897 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:57:07.897 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:57:07.897 INFO:tasks.check_counter:Daemon mds.d mds.dir_split=0 2024-09-19T04:57:07.897 DEBUG:tasks.check_counter:Getting stats from g 2024-09-19T04:57:07.897 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-19T04:57:08.265 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:08.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:08 smithi193 ceph-mon[35592]: Adjusting osd_memory_target on smithi123 to 176.4M 2024-09-19T04:57:08.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:08 smithi193 ceph-mon[35592]: Unable to set osd_memory_target on smithi123 to 185036185: error parsing value: Value '185036185' is below minimum 939524096 2024-09-19T04:57:08.529 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:08 smithi123 ceph-mon[30511]: Adjusting osd_memory_target on smithi123 to 176.4M 2024-09-19T04:57:08.529 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:08 smithi123 ceph-mon[30511]: Unable to set osd_memory_target on smithi123 to 185036185: error parsing value: Value '185036185' is below minimum 939524096 2024-09-19T04:57:08.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:08 smithi183 ceph-mon[35333]: Adjusting osd_memory_target on smithi123 to 176.4M 2024-09-19T04:57:08.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:08 smithi183 ceph-mon[35333]: Unable to set osd_memory_target on smithi123 to 185036185: error parsing value: Value '185036185' is below minimum 939524096 2024-09-19T04:57:08.836 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:57:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:57:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 220, 2024-09-19T04:57:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 209, 2024-09-19T04:57:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 1586272, 2024-09-19T04:57:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 128064, 2024-09-19T04:57:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2024-09-19T04:57:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2024-09-19T04:57:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.053833005, 2024-09-19T04:57:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.020495376, 2024-09-19T04:57:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.046985535, 2024-09-19T04:57:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:57:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 208, 2024-09-19T04:57:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.013319450, 2024-09-19T04:57:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000064035 2024-09-19T04:57:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1586272, 2024-09-19T04:57:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 128064 2024-09-19T04:57:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:57:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 2, 2024-09-19T04:57:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 160, 2024-09-19T04:57:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 320, 2024-09-19T04:57:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 185520, 2024-09-19T04:57:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 5, 2024-09-19T04:57:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2024-09-19T04:57:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.018917776, 2024-09-19T04:57:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.012712571, 2024-09-19T04:57:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.003065017, 2024-09-19T04:57:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:57:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 157, 2024-09-19T04:57:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.026463324, 2024-09-19T04:57:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000168556 2024-09-19T04:57:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T04:57:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 185520 2024-09-19T04:57:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:57:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 4, 2024-09-19T04:57:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 4, 2024-09-19T04:57:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 3072, 2024-09-19T04:57:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 448, 2024-09-19T04:57:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2024-09-19T04:57:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-09-19T04:57:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.003672814, 2024-09-19T04:57:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000080283, 2024-09-19T04:57:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.003523704, 2024-09-19T04:57:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:57:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3, 2024-09-19T04:57:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.006221024, 2024-09-19T04:57:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002073674 2024-09-19T04:57:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-19T04:57:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T04:57:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-09-19T04:57:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2024-09-19T04:57:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-09-19T04:57:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-09-19T04:57:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:57:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-09-19T04:57:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:57:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:57:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:57:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:57:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:57:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:57:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:57:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:57:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:57:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:57:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:57:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:57:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:57:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-09-19T04:57:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:57:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-09-19T04:57:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:57:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:57:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:57:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:57:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:57:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:57:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:57:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-09-19T04:57:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:57:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:57:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:57:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-09-19T04:57:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-09-19T04:57:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-09-19T04:57:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:57:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:57:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:57:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:57:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 150008, 2024-09-19T04:57:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 31, 2024-09-19T04:57:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:57:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-09-19T04:57:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-09-19T04:57:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-09-19T04:57:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:57:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-09-19T04:57:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:57:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-09-19T04:57:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2024-09-19T04:57:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2024-09-19T04:57:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:57:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:57:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-09-19T04:57:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-09-19T04:57:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 928, 2024-09-19T04:57:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2024-09-19T04:57:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-09-19T04:57:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-09-19T04:57:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-09-19T04:57:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-09-19T04:57:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:57:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-19T04:57:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-19T04:57:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-19T04:57:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 222, 2024-09-19T04:57:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1555254, 2024-09-19T04:57:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 222, 2024-09-19T04:57:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-19T04:57:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-19T04:57:08.848 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 222, 2024-09-19T04:57:08.848 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1555254, 2024-09-19T04:57:08.848 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-19T04:57:08.848 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:08.848 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:08.848 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:08.848 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:08.848 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:08.848 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:57:09.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:09 smithi193 ceph-mon[35592]: pgmap v324: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-09-19T04:57:09.558 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:09 smithi123 ceph-mon[30511]: pgmap v324: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-09-19T04:57:09.595 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-19T04:57:09.595 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-19T04:57:09.595 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-19T04:57:09.595 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-19T04:57:09.595 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-19T04:57:09.595 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-19T04:57:09.595 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-19T04:57:09.595 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-09-19T04:57:09.595 DEBUG:tasks.check_counter:Getting stats from j 2024-09-19T04:57:09.596 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-19T04:57:09.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:09 smithi183 ceph-mon[35333]: pgmap v324: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-09-19T04:57:09.965 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:10.716 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T04:57:10.716 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:57:10.716 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 4, 2024-09-19T04:57:10.716 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 4, 2024-09-19T04:57:10.716 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 3072, 2024-09-19T04:57:10.716 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 448, 2024-09-19T04:57:10.716 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2024-09-19T04:57:10.716 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2024-09-19T04:57:10.716 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.005815874, 2024-09-19T04:57:10.716 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000078062, 2024-09-19T04:57:10.716 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.003334201, 2024-09-19T04:57:10.717 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:57:10.717 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:10.717 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3, 2024-09-19T04:57:10.717 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.006017208, 2024-09-19T04:57:10.717 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002005736 2024-09-19T04:57:10.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:10.717 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:10.717 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:10.717 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:10.717 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:10.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:10.717 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-19T04:57:10.717 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T04:57:10.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:10.718 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:57:10.718 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 2, 2024-09-19T04:57:10.718 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 164, 2024-09-19T04:57:10.718 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 320, 2024-09-19T04:57:10.718 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 186224, 2024-09-19T04:57:10.718 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 5, 2024-09-19T04:57:10.718 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2024-09-19T04:57:10.718 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.019774339, 2024-09-19T04:57:10.718 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.013340303, 2024-09-19T04:57:10.718 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.003137890, 2024-09-19T04:57:10.718 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:57:10.718 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:10.718 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 161, 2024-09-19T04:57:10.719 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.017702931, 2024-09-19T04:57:10.719 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000109956 2024-09-19T04:57:10.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:10.719 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:10.719 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:10.719 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:10.719 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:10.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:10.719 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T04:57:10.719 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 186224 2024-09-19T04:57:10.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:10.719 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:57:10.719 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 226, 2024-09-19T04:57:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 215, 2024-09-19T04:57:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 1587904, 2024-09-19T04:57:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 131904, 2024-09-19T04:57:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2024-09-19T04:57:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-09-19T04:57:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.051459324, 2024-09-19T04:57:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.021033702, 2024-09-19T04:57:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.046816301, 2024-09-19T04:57:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:57:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 214, 2024-09-19T04:57:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.015042520, 2024-09-19T04:57:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000070292 2024-09-19T04:57:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:10.721 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:10.721 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:10.721 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:10.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:10.721 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1587904, 2024-09-19T04:57:10.721 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 131904 2024-09-19T04:57:10.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:10.721 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-09-19T04:57:10.721 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2024-09-19T04:57:10.721 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-09-19T04:57:10.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:10.721 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-09-19T04:57:10.721 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:57:10.722 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-09-19T04:57:10.722 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:57:10.722 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:57:10.722 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:57:10.722 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:57:10.722 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:57:10.722 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:57:10.722 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:57:10.722 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:57:10.722 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:57:10.722 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:57:10.722 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:57:10.722 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:57:10.722 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:57:10.723 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-09-19T04:57:10.723 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:57:10.723 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-09-19T04:57:10.723 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:57:10.723 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:57:10.723 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:57:10.723 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:57:10.723 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:57:10.723 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:57:10.723 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:57:10.723 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-09-19T04:57:10.723 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:57:10.723 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:57:10.723 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:57:10.724 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-09-19T04:57:10.724 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-09-19T04:57:10.724 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-09-19T04:57:10.724 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:57:10.724 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:57:10.724 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:57:10.724 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:57:10.724 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 148977, 2024-09-19T04:57:10.724 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 30, 2024-09-19T04:57:10.724 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:57:10.724 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-09-19T04:57:10.724 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-09-19T04:57:10.724 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-09-19T04:57:10.724 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:57:10.725 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-09-19T04:57:10.725 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:57:10.725 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-09-19T04:57:10.725 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2024-09-19T04:57:10.725 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2024-09-19T04:57:10.725 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:57:10.725 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:57:10.725 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-09-19T04:57:10.725 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-09-19T04:57:10.725 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 928, 2024-09-19T04:57:10.725 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2024-09-19T04:57:10.725 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-09-19T04:57:10.725 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-09-19T04:57:10.726 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-09-19T04:57:10.726 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-09-19T04:57:10.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T04:57:10.726 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:57:10.726 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-19T04:57:10.726 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-19T04:57:10.726 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-19T04:57:10.726 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 228, 2024-09-19T04:57:10.726 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1556034, 2024-09-19T04:57:10.726 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:10.726 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 228, 2024-09-19T04:57:10.726 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-19T04:57:10.726 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-19T04:57:10.726 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 228, 2024-09-19T04:57:10.727 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1556034, 2024-09-19T04:57:10.727 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-19T04:57:10.727 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-19T04:57:10.727 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-19T04:57:10.727 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-19T04:57:10.727 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:10.727 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T04:57:10.727 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T04:57:11.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:11 smithi123 ceph-mon[30511]: pgmap v325: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T04:57:11.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:11 smithi123 ceph-mon[30511]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T04:57:11.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:11 smithi183 ceph-mon[35333]: pgmap v325: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T04:57:11.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:11 smithi183 ceph-mon[35333]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T04:57:11.616 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-19T04:57:11.616 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-19T04:57:11.617 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-19T04:57:11.617 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-19T04:57:11.617 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-19T04:57:11.617 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-19T04:57:11.617 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-19T04:57:11.617 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-19T04:57:11.617 DEBUG:tasks.check_counter:Getting stats from b 2024-09-19T04:57:11.617 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-19T04:57:11.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:11 smithi193 ceph-mon[35592]: pgmap v325: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T04:57:11.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:11 smithi193 ceph-mon[35592]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T04:57:11.860 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:57:12.743 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-19T04:57:12.743 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:57:12.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 2557, 2024-09-19T04:57:12.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 2076, 2024-09-19T04:57:12.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1060355, 2024-09-19T04:57:12.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 2224587, 2024-09-19T04:57:12.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 28, 2024-09-19T04:57:12.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-09-19T04:57:12.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.234128146, 2024-09-19T04:57:12.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.070341965, 2024-09-19T04:57:12.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.115845536, 2024-09-19T04:57:12.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.051707514, 2024-09-19T04:57:12.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:12.744 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2077, 2024-09-19T04:57:12.744 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.664397599, 2024-09-19T04:57:12.745 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000319883 2024-09-19T04:57:12.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.745 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:12.745 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2146, 2024-09-19T04:57:12.745 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000552138, 2024-09-19T04:57:12.745 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000257 2024-09-19T04:57:12.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.745 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 56464, 2024-09-19T04:57:12.745 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 347184 2024-09-19T04:57:12.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.746 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:57:12.746 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1321, 2024-09-19T04:57:12.746 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1154, 2024-09-19T04:57:12.746 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 2146040, 2024-09-19T04:57:12.746 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 507564, 2024-09-19T04:57:12.746 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 11, 2024-09-19T04:57:12.746 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 7, 2024-09-19T04:57:12.746 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.132416993, 2024-09-19T04:57:12.746 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.051021624, 2024-09-19T04:57:12.746 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.094064489, 2024-09-19T04:57:12.746 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.011207168, 2024-09-19T04:57:12.746 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:12.747 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1153, 2024-09-19T04:57:12.747 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.208523537, 2024-09-19T04:57:12.747 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000180853 2024-09-19T04:57:12.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.747 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:12.747 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1236, 2024-09-19T04:57:12.747 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000248202, 2024-09-19T04:57:12.747 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000200 2024-09-19T04:57:12.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.747 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1800048, 2024-09-19T04:57:12.747 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 160352 2024-09-19T04:57:12.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.748 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:57:12.748 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 6220, 2024-09-19T04:57:12.748 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 3443, 2024-09-19T04:57:12.748 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 5015097, 2024-09-19T04:57:12.748 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 2478971, 2024-09-19T04:57:12.748 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 122, 2024-09-19T04:57:12.748 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 7, 2024-09-19T04:57:12.748 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.425116758, 2024-09-19T04:57:12.748 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.135588298, 2024-09-19T04:57:12.748 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.304039022, 2024-09-19T04:57:12.748 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.045068786, 2024-09-19T04:57:12.748 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:12.749 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3438, 2024-09-19T04:57:12.749 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.219142866, 2024-09-19T04:57:12.749 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000063741 2024-09-19T04:57:12.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.749 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:12.749 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 8931, 2024-09-19T04:57:12.749 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001442598, 2024-09-19T04:57:12.749 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000161 2024-09-19T04:57:12.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.749 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 412448, 2024-09-19T04:57:12.749 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 236640 2024-09-19T04:57:12.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.749 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-19T04:57:12.749 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-19T04:57:12.750 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-19T04:57:12.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.750 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-19T04:57:12.750 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-19T04:57:12.750 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-19T04:57:12.750 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 520, 2024-09-19T04:57:12.750 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.152767678, 2024-09-19T04:57:12.750 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000293783 2024-09-19T04:57:12.750 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:12.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.751 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-19T04:57:12.751 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-19T04:57:12.751 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-19T04:57:12.751 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 36, 2024-09-19T04:57:12.751 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.003745931, 2024-09-19T04:57:12.751 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000104053 2024-09-19T04:57:12.751 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:12.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.751 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-19T04:57:12.751 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 571, 2024-09-19T04:57:12.751 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 487, 2024-09-19T04:57:12.751 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-19T04:57:12.751 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 487, 2024-09-19T04:57:12.751 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.486106861, 2024-09-19T04:57:12.752 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000998166 2024-09-19T04:57:12.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.752 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-19T04:57:12.752 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 84, 2024-09-19T04:57:12.752 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-09-19T04:57:12.752 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-19T04:57:12.752 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 12, 2024-09-19T04:57:12.752 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-09-19T04:57:12.752 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-19T04:57:12.752 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 37, 2024-09-19T04:57:12.752 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 6, 2024-09-19T04:57:12.752 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 13, 2024-09-19T04:57:12.752 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 18, 2024-09-19T04:57:12.752 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 27, 2024-09-19T04:57:12.753 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 411, 2024-09-19T04:57:12.753 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 2, 2024-09-19T04:57:12.753 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 2, 2024-09-19T04:57:12.753 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 3, 2024-09-19T04:57:12.753 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 912, 2024-09-19T04:57:12.753 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 490, 2024-09-19T04:57:12.753 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 82, 2024-09-19T04:57:12.753 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 53, 2024-09-19T04:57:12.753 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:57:12.753 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-19T04:57:12.753 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-09-19T04:57:12.753 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-09-19T04:57:12.753 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-19T04:57:12.754 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 1, 2024-09-19T04:57:12.754 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 298, 2024-09-19T04:57:12.754 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 1, 2024-09-19T04:57:12.754 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 4, 2024-09-19T04:57:12.754 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-19T04:57:12.754 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 195, 2024-09-19T04:57:12.754 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-19T04:57:12.754 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-09-19T04:57:12.754 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-09-19T04:57:12.754 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 6, 2024-09-19T04:57:12.754 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 195, 2024-09-19T04:57:12.754 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-19T04:57:12.754 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:57:12.754 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:57:12.755 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 127, 2024-09-19T04:57:12.755 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 17, 2024-09-19T04:57:12.755 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 110, 2024-09-19T04:57:12.755 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 68, 2024-09-19T04:57:12.755 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 2, 2024-09-19T04:57:12.755 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 138, 2024-09-19T04:57:12.755 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 178, 2024-09-19T04:57:12.755 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:57:12.755 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:57:12.755 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:57:12.755 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 288, 2024-09-19T04:57:12.756 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 11, 2024-09-19T04:57:12.756 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 8, 2024-09-19T04:57:12.756 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 44 2024-09-19T04:57:12.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.756 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-19T04:57:12.756 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 19, 2024-09-19T04:57:12.756 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 463, 2024-09-19T04:57:12.756 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 257, 2024-09-19T04:57:12.756 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 262, 2024-09-19T04:57:12.756 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 40, 2024-09-19T04:57:12.756 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 1, 2024-09-19T04:57:12.756 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-19T04:57:12.756 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:57:12.757 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 10, 2024-09-19T04:57:12.757 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 9, 2024-09-19T04:57:12.757 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-09-19T04:57:12.757 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-19T04:57:12.757 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-19T04:57:12.757 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:57:12.757 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:57:12.757 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-19T04:57:12.757 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-19T04:57:12.757 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:57:12.757 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:57:12.757 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 17, 2024-09-19T04:57:12.757 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 1, 2024-09-19T04:57:12.757 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-19T04:57:12.758 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:57:12.758 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-19T04:57:12.758 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-09-19T04:57:12.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.758 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-19T04:57:12.758 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-19T04:57:12.758 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 229, 2024-09-19T04:57:12.758 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-09-19T04:57:12.758 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-09-19T04:57:12.758 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 229, 2024-09-19T04:57:12.758 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-19T04:57:12.758 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-09-19T04:57:12.759 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 2, 2024-09-19T04:57:12.759 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-09-19T04:57:12.759 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-09-19T04:57:12.759 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-09-19T04:57:12.759 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-09-19T04:57:12.759 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-19T04:57:12.759 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-09-19T04:57:12.759 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-09-19T04:57:12.759 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 5137035, 2024-09-19T04:57:12.759 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-09-19T04:57:12.759 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-19T04:57:12.759 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 142, 2024-09-19T04:57:12.759 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.421807424, 2024-09-19T04:57:12.760 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002970474 2024-09-19T04:57:12.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.760 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-09-19T04:57:12.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.760 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-19T04:57:12.760 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 33, 2024-09-19T04:57:12.760 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 146, 2024-09-19T04:57:12.760 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 113, 2024-09-19T04:57:12.761 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 28, 2024-09-19T04:57:12.761 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 253, 2024-09-19T04:57:12.761 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 225, 2024-09-19T04:57:12.761 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 37, 2024-09-19T04:57:12.761 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 458, 2024-09-19T04:57:12.761 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 421, 2024-09-19T04:57:12.761 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 2, 2024-09-19T04:57:12.761 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 60, 2024-09-19T04:57:12.761 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 58, 2024-09-19T04:57:12.761 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 56060, 2024-09-19T04:57:12.762 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 231712 2024-09-19T04:57:12.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.762 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-19T04:57:12.762 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 633, 2024-09-19T04:57:12.762 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 8, 2024-09-19T04:57:12.762 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 571, 2024-09-19T04:57:12.762 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 37, 2024-09-19T04:57:12.762 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 25, 2024-09-19T04:57:12.762 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-19T04:57:12.762 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 32, 2024-09-19T04:57:12.762 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.050449665, 2024-09-19T04:57:12.762 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001576552 2024-09-19T04:57:12.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.763 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-19T04:57:12.763 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 112, 2024-09-19T04:57:12.763 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.026756095, 2024-09-19T04:57:12.763 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000238893 2024-09-19T04:57:12.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.763 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-19T04:57:12.763 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.763 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.763 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.763 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-19T04:57:12.764 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.764 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.764 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.764 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-19T04:57:12.764 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5, 2024-09-19T04:57:12.764 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000943218, 2024-09-19T04:57:12.764 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000188643 2024-09-19T04:57:12.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.764 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-19T04:57:12.764 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.764 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.764 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.765 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-19T04:57:12.765 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.765 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.765 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.765 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-19T04:57:12.765 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.765 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.765 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.765 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-19T04:57:12.765 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.765 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.766 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.766 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-19T04:57:12.766 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.766 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.766 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.766 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-19T04:57:12.766 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 11, 2024-09-19T04:57:12.766 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001712939, 2024-09-19T04:57:12.766 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000155721 2024-09-19T04:57:12.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.766 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-19T04:57:12.766 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 19, 2024-09-19T04:57:12.767 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.021829935, 2024-09-19T04:57:12.767 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001148943 2024-09-19T04:57:12.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.767 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-19T04:57:12.767 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.767 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.767 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.767 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-19T04:57:12.767 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 11, 2024-09-19T04:57:12.767 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.248207199, 2024-09-19T04:57:12.767 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.022564290 2024-09-19T04:57:12.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.768 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-19T04:57:12.768 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 266, 2024-09-19T04:57:12.768 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.053512562, 2024-09-19T04:57:12.768 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000201175 2024-09-19T04:57:12.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.768 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-19T04:57:12.768 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-09-19T04:57:12.768 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002936483, 2024-09-19T04:57:12.768 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001468241 2024-09-19T04:57:12.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.768 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-19T04:57:12.769 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 10, 2024-09-19T04:57:12.769 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.014315423, 2024-09-19T04:57:12.769 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001431542 2024-09-19T04:57:12.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.769 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-19T04:57:12.769 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.769 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.769 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.769 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-19T04:57:12.769 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.769 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.769 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.770 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-19T04:57:12.770 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-09-19T04:57:12.770 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.021116212, 2024-09-19T04:57:12.770 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.021116212 2024-09-19T04:57:12.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.770 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-19T04:57:12.770 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5, 2024-09-19T04:57:12.770 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.020756390, 2024-09-19T04:57:12.770 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.004151278 2024-09-19T04:57:12.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.770 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-19T04:57:12.771 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.771 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.771 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.771 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-19T04:57:12.771 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-09-19T04:57:12.771 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000715953, 2024-09-19T04:57:12.771 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000715953 2024-09-19T04:57:12.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.771 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-19T04:57:12.771 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.771 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.771 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.772 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-19T04:57:12.772 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.772 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.772 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.772 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-19T04:57:12.772 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.772 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.772 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.772 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-19T04:57:12.772 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 8, 2024-09-19T04:57:12.773 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.019127524, 2024-09-19T04:57:12.773 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002390940 2024-09-19T04:57:12.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.773 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-19T04:57:12.773 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-09-19T04:57:12.773 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000633359, 2024-09-19T04:57:12.773 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000633359 2024-09-19T04:57:12.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.773 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-19T04:57:12.773 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3, 2024-09-19T04:57:12.773 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.003093904, 2024-09-19T04:57:12.773 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001031301 2024-09-19T04:57:12.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.774 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:57:12.774 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:57:12.774 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-19T04:57:12.774 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.774 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.774 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.774 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:12.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.774 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-19T04:57:12.775 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-09-19T04:57:12.775 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 6, 2024-09-19T04:57:12.775 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 4, 2024-09-19T04:57:12.775 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 2, 2024-09-19T04:57:12.775 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-19T04:57:12.775 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 28, 2024-09-19T04:57:12.775 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 14, 2024-09-19T04:57:12.775 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 618, 2024-09-19T04:57:12.775 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-19T04:57:12.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.775 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-19T04:57:12.775 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:57:12.776 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-19T04:57:12.776 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:57:12.776 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:57:12.776 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:57:12.776 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:57:12.776 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:57:12.776 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:57:12.776 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:57:12.776 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:57:12.776 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:57:12.776 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:57:12.776 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:57:12.776 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:57:12.777 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:57:12.777 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-19T04:57:12.777 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:57:12.777 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-19T04:57:12.777 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:57:12.777 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:57:12.777 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:57:12.777 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:57:12.777 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:57:12.777 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:57:12.777 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:57:12.777 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-19T04:57:12.777 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:57:12.777 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:57:12.778 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:57:12.778 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-19T04:57:12.778 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-19T04:57:12.778 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-19T04:57:12.778 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:57:12.778 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:57:12.778 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:57:12.778 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:57:12.778 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 359547, 2024-09-19T04:57:12.778 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 150, 2024-09-19T04:57:12.778 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 1056, 2024-09-19T04:57:12.778 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 12, 2024-09-19T04:57:12.778 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-19T04:57:12.779 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-19T04:57:12.779 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:57:12.779 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-19T04:57:12.779 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:57:12.779 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-19T04:57:12.779 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8128, 2024-09-19T04:57:12.779 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 179, 2024-09-19T04:57:12.779 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:57:12.779 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:57:12.779 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-19T04:57:12.779 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-19T04:57:12.779 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 186095, 2024-09-19T04:57:12.779 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 1084, 2024-09-19T04:57:12.779 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-19T04:57:12.780 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-19T04:57:12.780 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-19T04:57:12.780 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-19T04:57:12.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.780 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-19T04:57:12.780 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-19T04:57:12.780 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-19T04:57:12.780 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 985, 2024-09-19T04:57:12.780 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 965448, 2024-09-19T04:57:12.780 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 37, 2024-09-19T04:57:12.780 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 985, 2024-09-19T04:57:12.780 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-19T04:57:12.780 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 985, 2024-09-19T04:57:12.781 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 20.092459663, 2024-09-19T04:57:12.781 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.020398436 2024-09-19T04:57:12.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.781 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-19T04:57:12.781 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-19T04:57:12.781 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 985, 2024-09-19T04:57:12.781 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1036 2024-09-19T04:57:12.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.781 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 985, 2024-09-19T04:57:12.781 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 232, 2024-09-19T04:57:12.781 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 753, 2024-09-19T04:57:12.781 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-19T04:57:12.781 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-19T04:57:12.781 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 0, 2024-09-19T04:57:12.782 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 11, 2024-09-19T04:57:12.782 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 20, 2024-09-19T04:57:12.782 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 145, 2024-09-19T04:57:12.782 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 37, 2024-09-19T04:57:12.782 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-19T04:57:12.782 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-19T04:57:12.782 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-19T04:57:12.782 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-09-19T04:57:12.782 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 521, 2024-09-19T04:57:12.782 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-19T04:57:12.782 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-19T04:57:12.782 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-19T04:57:12.782 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 212, 2024-09-19T04:57:12.782 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 22, 2024-09-19T04:57:12.783 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:57:12.783 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-19T04:57:12.783 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:57:12.783 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-19T04:57:12.783 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-19T04:57:12.783 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-19T04:57:12.783 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:57:12.783 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-19T04:57:12.783 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:57:12.783 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-19T04:57:12.783 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-19T04:57:12.783 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-19T04:57:12.783 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-19T04:57:12.783 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-19T04:57:12.784 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-19T04:57:12.784 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-19T04:57:12.784 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-19T04:57:12.784 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-19T04:57:12.784 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-19T04:57:12.784 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-19T04:57:12.784 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-19T04:57:12.784 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-19T04:57:12.784 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-19T04:57:12.784 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-19T04:57:12.784 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-19T04:57:12.784 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-19T04:57:12.784 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 96, 2024-09-19T04:57:12.785 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-19T04:57:12.785 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 12, 2024-09-19T04:57:12.785 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 12, 2024-09-19T04:57:12.785 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 12, 2024-09-19T04:57:12.785 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-09-19T04:57:12.785 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-19T04:57:12.785 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 57, 2024-09-19T04:57:12.785 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 0, 2024-09-19T04:57:12.785 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 11 2024-09-19T04:57:12.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.785 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-19T04:57:12.785 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-09-19T04:57:12.785 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:57:12.785 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-19T04:57:12.786 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-19T04:57:12.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.786 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-09-19T04:57:12.786 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-09-19T04:57:12.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.786 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-19T04:57:12.786 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-19T04:57:12.786 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T04:57:12.786 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-19T04:57:12.786 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 6, 2024-09-19T04:57:12.786 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 9, 2024-09-19T04:57:12.786 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 9, 2024-09-19T04:57:12.786 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-19T04:57:12.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.787 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:57:12.787 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:12.787 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-19T04:57:12.787 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:12.787 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 10094, 2024-09-19T04:57:12.787 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 7261773, 2024-09-19T04:57:12.787 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:12.787 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 10094, 2024-09-19T04:57:12.787 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-19T04:57:12.787 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-19T04:57:12.787 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 10094, 2024-09-19T04:57:12.788 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 7261773, 2024-09-19T04:57:12.788 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:12.788 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.788 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.788 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.788 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:12.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.788 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-19T04:57:12.788 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:12.788 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-19T04:57:12.788 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:12.788 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-19T04:57:12.789 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-19T04:57:12.789 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:12.789 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:12.789 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 985, 2024-09-19T04:57:12.789 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 966357, 2024-09-19T04:57:12.789 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 985, 2024-09-19T04:57:12.789 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 966357, 2024-09-19T04:57:12.789 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:12.789 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.790 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.790 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.790 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:12.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.790 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-19T04:57:12.790 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:12.790 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-19T04:57:12.790 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:12.790 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-19T04:57:12.790 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-19T04:57:12.790 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:12.790 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:12.790 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 985, 2024-09-19T04:57:12.790 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 985, 2024-09-19T04:57:12.791 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 985, 2024-09-19T04:57:12.791 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 985, 2024-09-19T04:57:12.791 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:12.791 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.791 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.791 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.791 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:12.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.791 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:57:12.791 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:12.791 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-19T04:57:12.791 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:12.791 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 9, 2024-09-19T04:57:12.792 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 909, 2024-09-19T04:57:12.792 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:12.792 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 9, 2024-09-19T04:57:12.792 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-19T04:57:12.792 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-19T04:57:12.792 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3, 2024-09-19T04:57:12.792 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 909, 2024-09-19T04:57:12.792 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:12.792 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.792 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.792 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.792 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:12.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:12.792 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x560908494960": { 2024-09-19T04:57:12.793 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 112, 2024-09-19T04:57:12.793 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-19T04:57:12.793 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:12.793 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 229, 2024-09-19T04:57:12.793 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 942731, 2024-09-19T04:57:12.793 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:12.793 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 229, 2024-09-19T04:57:12.793 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-19T04:57:12.793 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-19T04:57:12.793 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 142, 2024-09-19T04:57:12.793 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 942619, 2024-09-19T04:57:12.793 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:12.793 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:12.794 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:12.794 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:12.794 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:12.794 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:12.794 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-19T04:57:13.582 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:13 smithi183 ceph-mon[35333]: pgmap v327: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T04:57:13.610 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-09-19T04:57:13.610 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-09-19T04:57:13.610 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=19 2024-09-19T04:57:13.610 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=463 2024-09-19T04:57:13.610 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-19T04:57:13.610 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-09-19T04:57:13.610 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-09-19T04:57:13.610 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-09-19T04:57:13.610 DEBUG:tasks.check_counter:Getting stats from e 2024-09-19T04:57:13.610 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-19T04:57:13.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:13 smithi123 ceph-mon[30511]: pgmap v327: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T04:57:13.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:13 smithi193 ceph-mon[35592]: pgmap v327: 97 pgs: 97 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T04:57:13.859 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:57:14.597 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-19T04:57:14.597 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:57:14.597 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1588, 2024-09-19T04:57:14.597 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1590, 2024-09-19T04:57:14.597 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1413282, 2024-09-19T04:57:14.597 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 504994, 2024-09-19T04:57:14.597 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2024-09-19T04:57:14.597 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-09-19T04:57:14.597 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.134580844, 2024-09-19T04:57:14.597 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.053620941, 2024-09-19T04:57:14.597 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.031873417, 2024-09-19T04:57:14.598 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.036881225, 2024-09-19T04:57:14.598 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:14.598 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1588, 2024-09-19T04:57:14.598 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.051593834, 2024-09-19T04:57:14.598 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000032489 2024-09-19T04:57:14.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.598 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:14.598 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.598 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.598 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.598 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:57:14.598 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:57:14.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.599 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:57:14.599 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 267, 2024-09-19T04:57:14.599 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 238, 2024-09-19T04:57:14.599 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1723736, 2024-09-19T04:57:14.599 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 136154, 2024-09-19T04:57:14.599 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-09-19T04:57:14.599 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-09-19T04:57:14.599 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.045079145, 2024-09-19T04:57:14.599 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.014410838, 2024-09-19T04:57:14.599 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.046101093, 2024-09-19T04:57:14.599 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000112901, 2024-09-19T04:57:14.599 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:14.599 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 238, 2024-09-19T04:57:14.600 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.021599466, 2024-09-19T04:57:14.600 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000090754 2024-09-19T04:57:14.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.600 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:14.600 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.600 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.600 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.600 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1722080, 2024-09-19T04:57:14.600 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 134816 2024-09-19T04:57:14.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.600 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:57:14.601 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 748, 2024-09-19T04:57:14.601 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 910, 2024-09-19T04:57:14.601 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 183907, 2024-09-19T04:57:14.601 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 682797, 2024-09-19T04:57:14.601 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2024-09-19T04:57:14.601 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-09-19T04:57:14.601 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.070978945, 2024-09-19T04:57:14.601 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.024187777, 2024-09-19T04:57:14.601 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.015622084, 2024-09-19T04:57:14.601 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.023960990, 2024-09-19T04:57:14.601 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:14.601 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 907, 2024-09-19T04:57:14.601 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.035332055, 2024-09-19T04:57:14.602 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000038954 2024-09-19T04:57:14.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.602 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:14.602 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.602 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.602 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.602 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T04:57:14.602 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 445568 2024-09-19T04:57:14.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.602 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-19T04:57:14.602 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-19T04:57:14.602 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-19T04:57:14.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.603 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-19T04:57:14.603 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-19T04:57:14.603 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-19T04:57:14.603 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3909, 2024-09-19T04:57:14.603 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.091561710, 2024-09-19T04:57:14.603 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000023423 2024-09-19T04:57:14.603 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:14.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.603 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-19T04:57:14.603 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-19T04:57:14.603 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-19T04:57:14.603 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.603 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.604 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.604 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:14.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.604 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-19T04:57:14.604 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-09-19T04:57:14.604 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-09-19T04:57:14.604 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-19T04:57:14.604 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.604 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.604 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.604 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-19T04:57:14.604 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-09-19T04:57:14.605 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-09-19T04:57:14.605 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-19T04:57:14.605 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-09-19T04:57:14.605 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-09-19T04:57:14.605 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-19T04:57:14.605 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 29, 2024-09-19T04:57:14.605 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 1, 2024-09-19T04:57:14.605 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 28, 2024-09-19T04:57:14.605 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-09-19T04:57:14.605 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 27, 2024-09-19T04:57:14.605 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-09-19T04:57:14.605 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-09-19T04:57:14.605 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-09-19T04:57:14.605 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 2, 2024-09-19T04:57:14.606 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-09-19T04:57:14.606 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-09-19T04:57:14.606 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-09-19T04:57:14.606 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-09-19T04:57:14.606 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:57:14.606 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-19T04:57:14.606 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-09-19T04:57:14.606 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-09-19T04:57:14.606 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-19T04:57:14.606 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-09-19T04:57:14.606 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-09-19T04:57:14.606 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-09-19T04:57:14.606 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-09-19T04:57:14.606 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-19T04:57:14.607 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:57:14.607 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-19T04:57:14.607 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-09-19T04:57:14.607 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-09-19T04:57:14.607 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 6, 2024-09-19T04:57:14.607 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:57:14.607 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-19T04:57:14.607 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:57:14.607 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:57:14.607 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:57:14.607 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:57:14.607 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:57:14.607 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-09-19T04:57:14.607 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:57:14.608 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:57:14.608 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:57:14.608 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:57:14.608 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:57:14.608 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:57:14.608 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-09-19T04:57:14.608 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:57:14.608 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-09-19T04:57:14.608 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-09-19T04:57:14.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.608 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-19T04:57:14.608 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-09-19T04:57:14.608 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-09-19T04:57:14.608 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-09-19T04:57:14.609 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-09-19T04:57:14.609 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-09-19T04:57:14.609 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-09-19T04:57:14.609 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-19T04:57:14.609 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:57:14.609 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-09-19T04:57:14.609 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-09-19T04:57:14.609 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-09-19T04:57:14.609 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-19T04:57:14.609 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-19T04:57:14.609 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:57:14.609 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:57:14.609 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-19T04:57:14.609 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-19T04:57:14.610 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:57:14.610 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:57:14.610 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:57:14.610 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-09-19T04:57:14.610 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-19T04:57:14.610 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:57:14.610 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-19T04:57:14.610 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-09-19T04:57:14.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.610 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-19T04:57:14.610 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-19T04:57:14.610 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-09-19T04:57:14.610 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-09-19T04:57:14.611 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-09-19T04:57:14.611 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 228, 2024-09-19T04:57:14.611 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-19T04:57:14.611 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-09-19T04:57:14.611 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-09-19T04:57:14.611 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-09-19T04:57:14.611 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-09-19T04:57:14.611 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-09-19T04:57:14.611 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-09-19T04:57:14.611 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-19T04:57:14.611 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-09-19T04:57:14.611 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-09-19T04:57:14.611 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 5136923, 2024-09-19T04:57:14.611 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 5136863, 2024-09-19T04:57:14.612 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-19T04:57:14.612 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.612 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.612 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.612 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 228 2024-09-19T04:57:14.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.612 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-19T04:57:14.612 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 15, 2024-09-19T04:57:14.612 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 56, 2024-09-19T04:57:14.612 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 41, 2024-09-19T04:57:14.612 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 10, 2024-09-19T04:57:14.612 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 22, 2024-09-19T04:57:14.613 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 12, 2024-09-19T04:57:14.613 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 29, 2024-09-19T04:57:14.613 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 64, 2024-09-19T04:57:14.613 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 35, 2024-09-19T04:57:14.613 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-09-19T04:57:14.613 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-09-19T04:57:14.613 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-09-19T04:57:14.613 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 52460, 2024-09-19T04:57:14.613 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 215328 2024-09-19T04:57:14.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.613 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-19T04:57:14.613 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-09-19T04:57:14.613 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-09-19T04:57:14.613 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-09-19T04:57:14.614 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-09-19T04:57:14.614 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-09-19T04:57:14.614 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-19T04:57:14.614 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.614 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.614 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.614 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-19T04:57:14.614 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.614 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.614 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.614 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-19T04:57:14.614 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.615 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.615 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.615 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-19T04:57:14.615 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.615 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.615 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.615 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-19T04:57:14.615 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.615 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.615 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.616 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-19T04:57:14.616 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.616 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.616 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.616 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-19T04:57:14.616 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.616 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.616 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.616 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-19T04:57:14.616 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.616 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.616 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.617 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-19T04:57:14.617 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.617 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.617 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.617 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-19T04:57:14.617 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.617 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.617 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.617 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-19T04:57:14.617 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.617 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.618 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.618 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-19T04:57:14.618 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.618 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.618 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.619 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-19T04:57:14.619 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.619 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.619 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.619 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-19T04:57:14.619 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.619 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.619 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.619 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-19T04:57:14.619 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.619 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.620 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.620 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-19T04:57:14.620 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.620 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.620 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.620 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-19T04:57:14.620 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.620 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.620 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.620 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-19T04:57:14.621 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.621 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.621 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.621 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-19T04:57:14.621 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.621 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.621 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.621 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-19T04:57:14.621 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.621 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.621 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.622 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-19T04:57:14.622 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.622 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.622 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.622 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-19T04:57:14.622 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.622 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.622 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.622 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-19T04:57:14.622 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.622 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.623 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.623 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-19T04:57:14.623 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.623 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.623 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.623 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-19T04:57:14.623 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.623 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.623 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.623 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-19T04:57:14.624 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.624 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.624 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.624 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-19T04:57:14.624 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.624 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.624 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.624 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-19T04:57:14.624 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.624 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.624 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.625 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-19T04:57:14.625 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.625 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.625 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.625 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:57:14.625 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:57:14.625 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-19T04:57:14.625 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.625 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.625 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.625 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:14.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.625 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-19T04:57:14.626 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-09-19T04:57:14.626 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 6, 2024-09-19T04:57:14.626 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 4, 2024-09-19T04:57:14.626 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 5, 2024-09-19T04:57:14.626 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-19T04:57:14.626 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-09-19T04:57:14.626 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-09-19T04:57:14.626 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 4054, 2024-09-19T04:57:14.626 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-19T04:57:14.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.626 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-19T04:57:14.626 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:57:14.626 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-19T04:57:14.627 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:57:14.627 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:57:14.627 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:57:14.627 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:57:14.627 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:57:14.627 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:57:14.627 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:57:14.627 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:57:14.627 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:57:14.627 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:57:14.627 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:57:14.627 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:57:14.627 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:57:14.627 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-19T04:57:14.627 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:57:14.628 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-19T04:57:14.628 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:57:14.628 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:57:14.628 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:57:14.628 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:57:14.628 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:57:14.628 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:57:14.628 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:57:14.628 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-19T04:57:14.628 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:57:14.628 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:57:14.628 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:57:14.628 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-19T04:57:14.629 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-19T04:57:14.629 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-19T04:57:14.629 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:57:14.629 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:57:14.629 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:57:14.629 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:57:14.629 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 189630, 2024-09-19T04:57:14.629 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 58, 2024-09-19T04:57:14.629 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:57:14.629 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-19T04:57:14.629 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-19T04:57:14.629 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-19T04:57:14.629 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:57:14.629 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-19T04:57:14.630 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:57:14.630 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-19T04:57:14.630 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8248, 2024-09-19T04:57:14.630 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 182, 2024-09-19T04:57:14.630 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:57:14.630 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:57:14.630 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-19T04:57:14.630 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-19T04:57:14.630 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 95216, 2024-09-19T04:57:14.630 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 543, 2024-09-19T04:57:14.630 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-19T04:57:14.630 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-19T04:57:14.630 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-19T04:57:14.630 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-19T04:57:14.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.631 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-19T04:57:14.631 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-19T04:57:14.631 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-19T04:57:14.631 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 2338, 2024-09-19T04:57:14.631 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 26, 2024-09-19T04:57:14.631 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 4, 2024-09-19T04:57:14.631 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 2338, 2024-09-19T04:57:14.631 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-19T04:57:14.631 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2338, 2024-09-19T04:57:14.631 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.869976647, 2024-09-19T04:57:14.631 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000372102 2024-09-19T04:57:14.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.631 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-19T04:57:14.632 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-19T04:57:14.632 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2338, 2024-09-19T04:57:14.632 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2339 2024-09-19T04:57:14.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.632 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 2338, 2024-09-19T04:57:14.632 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 2338, 2024-09-19T04:57:14.632 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-09-19T04:57:14.632 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-19T04:57:14.632 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-19T04:57:14.632 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 1494, 2024-09-19T04:57:14.632 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-09-19T04:57:14.632 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 841, 2024-09-19T04:57:14.632 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-09-19T04:57:14.632 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-09-19T04:57:14.633 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-19T04:57:14.633 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-19T04:57:14.633 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-19T04:57:14.633 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-09-19T04:57:14.633 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-09-19T04:57:14.633 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-19T04:57:14.633 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-19T04:57:14.633 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-19T04:57:14.633 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 2, 2024-09-19T04:57:14.633 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-09-19T04:57:14.633 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:57:14.633 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-19T04:57:14.633 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:57:14.633 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-19T04:57:14.634 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-19T04:57:14.634 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-19T04:57:14.634 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:57:14.634 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-19T04:57:14.634 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:57:14.634 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-19T04:57:14.634 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-19T04:57:14.634 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-19T04:57:14.634 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-19T04:57:14.634 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-19T04:57:14.634 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-19T04:57:14.634 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-19T04:57:14.634 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-19T04:57:14.634 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-19T04:57:14.635 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-19T04:57:14.635 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-19T04:57:14.635 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-19T04:57:14.635 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-19T04:57:14.635 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-19T04:57:14.635 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-19T04:57:14.635 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-19T04:57:14.635 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-19T04:57:14.635 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 96, 2024-09-19T04:57:14.635 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-19T04:57:14.635 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 12, 2024-09-19T04:57:14.635 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 5, 2024-09-19T04:57:14.635 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 5, 2024-09-19T04:57:14.635 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-09-19T04:57:14.636 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-19T04:57:14.636 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-09-19T04:57:14.636 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-09-19T04:57:14.636 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-09-19T04:57:14.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.636 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-19T04:57:14.636 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-09-19T04:57:14.636 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:57:14.636 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-19T04:57:14.636 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-19T04:57:14.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.636 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-09-19T04:57:14.636 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-09-19T04:57:14.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.637 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-19T04:57:14.637 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-19T04:57:14.637 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:57:14.637 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-19T04:57:14.637 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-09-19T04:57:14.637 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-09-19T04:57:14.637 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-09-19T04:57:14.637 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-19T04:57:14.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.637 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:57:14.637 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:14.637 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-19T04:57:14.637 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:14.637 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 2599, 2024-09-19T04:57:14.638 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 3070730, 2024-09-19T04:57:14.638 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:14.638 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 2599, 2024-09-19T04:57:14.638 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-19T04:57:14.638 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-19T04:57:14.638 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2599, 2024-09-19T04:57:14.638 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3070730, 2024-09-19T04:57:14.638 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:14.638 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.638 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.638 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.638 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:14.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.638 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-19T04:57:14.639 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:14.639 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-19T04:57:14.639 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:14.639 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-19T04:57:14.639 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-19T04:57:14.639 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:14.639 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:14.639 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 2338, 2024-09-19T04:57:14.639 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 942629, 2024-09-19T04:57:14.639 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2338, 2024-09-19T04:57:14.639 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 942629, 2024-09-19T04:57:14.639 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:14.639 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.639 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-19T04:57:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-19T04:57:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-19T04:57:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-19T04:57:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 2338, 2024-09-19T04:57:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 2338, 2024-09-19T04:57:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2338, 2024-09-19T04:57:14.641 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2338, 2024-09-19T04:57:14.641 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:14.641 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.641 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.641 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.641 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:14.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.641 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:57:14.641 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:14.641 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-19T04:57:14.641 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:14.641 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-19T04:57:14.641 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-19T04:57:14.642 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:14.642 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:14.642 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-19T04:57:14.642 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-19T04:57:14.642 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-19T04:57:14.642 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-19T04:57:14.642 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:14.642 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.642 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.642 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.642 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:14.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:14.642 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x56517f344b40": { 2024-09-19T04:57:14.642 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:14.642 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-19T04:57:14.643 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:14.643 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-19T04:57:14.643 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-19T04:57:14.643 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:14.643 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:14.643 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-19T04:57:14.643 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-19T04:57:14.643 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-19T04:57:14.643 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-19T04:57:14.643 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:14.643 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:14.643 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:14.643 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:14.643 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:14.644 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:14.644 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-19T04:57:15.419 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-19T04:57:15.419 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-19T04:57:15.419 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-19T04:57:15.419 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-19T04:57:15.420 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-19T04:57:15.420 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:57:15.420 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:57:15.420 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-19T04:57:15.420 DEBUG:tasks.check_counter:Getting stats from h 2024-09-19T04:57:15.421 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-19T04:57:15.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:15 smithi183 ceph-mon[35333]: pgmap v328: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:57:15.664 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:57:15.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:15 smithi123 ceph-mon[30511]: pgmap v328: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:57:15.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:15 smithi193 ceph-mon[35592]: pgmap v328: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:57:16.212 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-19T04:57:16.212 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:57:16.213 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 4335, 2024-09-19T04:57:16.213 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 2483, 2024-09-19T04:57:16.213 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 4223940, 2024-09-19T04:57:16.213 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 1091222, 2024-09-19T04:57:16.213 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 37, 2024-09-19T04:57:16.213 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2024-09-19T04:57:16.213 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.248479847, 2024-09-19T04:57:16.213 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.095021616, 2024-09-19T04:57:16.213 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.189024042, 2024-09-19T04:57:16.213 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.004644702, 2024-09-19T04:57:16.213 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:16.213 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2483, 2024-09-19T04:57:16.213 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.073604310, 2024-09-19T04:57:16.213 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000029643 2024-09-19T04:57:16.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.214 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:16.214 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6618, 2024-09-19T04:57:16.214 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001029672, 2024-09-19T04:57:16.214 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000155 2024-09-19T04:57:16.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.214 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:57:16.214 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:57:16.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.214 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:57:16.214 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 58, 2024-09-19T04:57:16.214 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 216, 2024-09-19T04:57:16.214 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 36022, 2024-09-19T04:57:16.214 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 510583, 2024-09-19T04:57:16.215 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 12, 2024-09-19T04:57:16.215 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2024-09-19T04:57:16.215 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.017075114, 2024-09-19T04:57:16.215 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.007360796, 2024-09-19T04:57:16.215 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.006607267, 2024-09-19T04:57:16.215 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.001167879, 2024-09-19T04:57:16.215 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:16.215 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 214, 2024-09-19T04:57:16.215 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.034184625, 2024-09-19T04:57:16.215 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000159741 2024-09-19T04:57:16.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.215 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:16.215 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 21, 2024-09-19T04:57:16.215 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000002964, 2024-09-19T04:57:16.215 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000141 2024-09-19T04:57:16.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.216 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-19T04:57:16.216 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 445952 2024-09-19T04:57:16.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.216 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:57:16.216 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 771, 2024-09-19T04:57:16.216 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 776, 2024-09-19T04:57:16.216 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1904374, 2024-09-19T04:57:16.216 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 402958, 2024-09-19T04:57:16.216 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 17, 2024-09-19T04:57:16.216 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 7, 2024-09-19T04:57:16.216 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.080326118, 2024-09-19T04:57:16.216 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.029416209, 2024-09-19T04:57:16.217 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.073565284, 2024-09-19T04:57:16.217 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000462837, 2024-09-19T04:57:16.217 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:16.217 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 773, 2024-09-19T04:57:16.217 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.032110133, 2024-09-19T04:57:16.217 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000041539 2024-09-19T04:57:16.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.217 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:16.217 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 489, 2024-09-19T04:57:16.217 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000163291, 2024-09-19T04:57:16.217 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000333 2024-09-19T04:57:16.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.217 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1741168, 2024-09-19T04:57:16.218 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 146960 2024-09-19T04:57:16.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.218 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-19T04:57:16.218 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-19T04:57:16.218 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-19T04:57:16.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.218 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-19T04:57:16.218 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-19T04:57:16.218 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-19T04:57:16.218 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 124, 2024-09-19T04:57:16.218 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.009583496, 2024-09-19T04:57:16.218 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000077286 2024-09-19T04:57:16.218 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:16.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.219 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-19T04:57:16.219 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-19T04:57:16.219 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-19T04:57:16.219 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 32, 2024-09-19T04:57:16.219 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000717096, 2024-09-19T04:57:16.219 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000022409 2024-09-19T04:57:16.219 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:16.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.219 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-19T04:57:16.219 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 152, 2024-09-19T04:57:16.219 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 18, 2024-09-19T04:57:16.219 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-19T04:57:16.219 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 18, 2024-09-19T04:57:16.219 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.017071069, 2024-09-19T04:57:16.219 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000948392 2024-09-19T04:57:16.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.220 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-19T04:57:16.220 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 134, 2024-09-19T04:57:16.220 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-09-19T04:57:16.220 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-19T04:57:16.220 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 11, 2024-09-19T04:57:16.220 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-09-19T04:57:16.220 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-19T04:57:16.220 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 23, 2024-09-19T04:57:16.220 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 4, 2024-09-19T04:57:16.220 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 9, 2024-09-19T04:57:16.220 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 10, 2024-09-19T04:57:16.220 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 13, 2024-09-19T04:57:16.220 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 126, 2024-09-19T04:57:16.220 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-09-19T04:57:16.220 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-09-19T04:57:16.221 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 4, 2024-09-19T04:57:16.221 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 448, 2024-09-19T04:57:16.221 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 244, 2024-09-19T04:57:16.221 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 133, 2024-09-19T04:57:16.221 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 56, 2024-09-19T04:57:16.221 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:57:16.221 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-19T04:57:16.221 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-09-19T04:57:16.221 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-09-19T04:57:16.221 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-19T04:57:16.221 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 1, 2024-09-19T04:57:16.221 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 10, 2024-09-19T04:57:16.221 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 1, 2024-09-19T04:57:16.221 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 4, 2024-09-19T04:57:16.222 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-19T04:57:16.222 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 16, 2024-09-19T04:57:16.222 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-19T04:57:16.222 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-09-19T04:57:16.222 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-09-19T04:57:16.222 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 6, 2024-09-19T04:57:16.222 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 16, 2024-09-19T04:57:16.222 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-19T04:57:16.222 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:57:16.222 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:57:16.222 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:57:16.222 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:57:16.222 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:57:16.222 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 16, 2024-09-19T04:57:16.223 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:57:16.223 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 128, 2024-09-19T04:57:16.223 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 133, 2024-09-19T04:57:16.223 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:57:16.223 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:57:16.223 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:57:16.223 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 262, 2024-09-19T04:57:16.223 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 7, 2024-09-19T04:57:16.223 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 5, 2024-09-19T04:57:16.223 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 13 2024-09-19T04:57:16.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.223 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-19T04:57:16.223 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 9, 2024-09-19T04:57:16.223 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 485, 2024-09-19T04:57:16.224 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 270, 2024-09-19T04:57:16.224 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 272, 2024-09-19T04:57:16.224 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 9, 2024-09-19T04:57:16.224 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-09-19T04:57:16.224 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-19T04:57:16.224 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:57:16.224 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 5, 2024-09-19T04:57:16.224 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 5, 2024-09-19T04:57:16.224 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-09-19T04:57:16.224 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-19T04:57:16.224 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-19T04:57:16.224 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:57:16.224 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:57:16.224 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-19T04:57:16.224 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-19T04:57:16.225 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:57:16.225 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:57:16.225 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:57:16.225 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 1, 2024-09-19T04:57:16.225 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-19T04:57:16.225 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:57:16.225 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-19T04:57:16.225 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-09-19T04:57:16.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.225 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-19T04:57:16.225 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-19T04:57:16.225 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 39, 2024-09-19T04:57:16.225 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-09-19T04:57:16.225 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-09-19T04:57:16.225 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 39, 2024-09-19T04:57:16.226 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-19T04:57:16.226 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-09-19T04:57:16.226 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 2, 2024-09-19T04:57:16.226 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-09-19T04:57:16.226 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-09-19T04:57:16.226 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-09-19T04:57:16.226 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-09-19T04:57:16.226 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-19T04:57:16.226 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-09-19T04:57:16.226 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-09-19T04:57:16.226 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 4263151, 2024-09-19T04:57:16.226 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-09-19T04:57:16.226 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-19T04:57:16.226 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 21, 2024-09-19T04:57:16.226 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.044926384, 2024-09-19T04:57:16.227 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002139351 2024-09-19T04:57:16.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.227 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-09-19T04:57:16.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.227 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-19T04:57:16.227 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 28, 2024-09-19T04:57:16.227 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 100, 2024-09-19T04:57:16.227 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 72, 2024-09-19T04:57:16.227 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 27, 2024-09-19T04:57:16.227 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 253, 2024-09-19T04:57:16.227 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 226, 2024-09-19T04:57:16.227 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 23, 2024-09-19T04:57:16.227 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 155, 2024-09-19T04:57:16.227 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 132, 2024-09-19T04:57:16.228 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-09-19T04:57:16.228 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 14, 2024-09-19T04:57:16.228 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 14, 2024-09-19T04:57:16.228 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 53060, 2024-09-19T04:57:16.228 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 223520 2024-09-19T04:57:16.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.228 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-19T04:57:16.228 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 163, 2024-09-19T04:57:16.228 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 6, 2024-09-19T04:57:16.228 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 152, 2024-09-19T04:57:16.228 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 12, 2024-09-19T04:57:16.228 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 20, 2024-09-19T04:57:16.228 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-19T04:57:16.229 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-09-19T04:57:16.229 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.006103814, 2024-09-19T04:57:16.229 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.003051907 2024-09-19T04:57:16.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.229 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-19T04:57:16.229 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-09-19T04:57:16.229 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000294642, 2024-09-19T04:57:16.229 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000294642 2024-09-19T04:57:16.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.229 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-19T04:57:16.229 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.229 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.229 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-19T04:57:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-19T04:57:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 8, 2024-09-19T04:57:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001951418, 2024-09-19T04:57:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000243927 2024-09-19T04:57:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-19T04:57:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-19T04:57:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-19T04:57:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-19T04:57:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-19T04:57:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-19T04:57:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-19T04:57:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-19T04:57:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-19T04:57:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-19T04:57:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-19T04:57:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-09-19T04:57:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001067686, 2024-09-19T04:57:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000533843 2024-09-19T04:57:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-19T04:57:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-19T04:57:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-19T04:57:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-19T04:57:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-19T04:57:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-19T04:57:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-19T04:57:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-19T04:57:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-19T04:57:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-19T04:57:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-19T04:57:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-19T04:57:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-19T04:57:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5, 2024-09-19T04:57:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.007653509, 2024-09-19T04:57:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001530701 2024-09-19T04:57:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:57:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:57:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-19T04:57:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-19T04:57:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 1, 2024-09-19T04:57:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 2, 2024-09-19T04:57:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 1, 2024-09-19T04:57:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 1, 2024-09-19T04:57:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-19T04:57:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 8, 2024-09-19T04:57:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 8, 2024-09-19T04:57:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 542, 2024-09-19T04:57:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-19T04:57:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-19T04:57:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:57:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-19T04:57:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:57:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:57:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:57:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:57:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:57:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:57:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:57:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:57:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:57:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:57:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:57:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:57:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:57:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-19T04:57:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:57:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-19T04:57:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:57:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:57:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:57:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:57:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:57:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:57:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:57:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-19T04:57:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:57:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:57:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:57:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-19T04:57:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-19T04:57:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-19T04:57:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:57:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:57:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:57:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:57:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 331045, 2024-09-19T04:57:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 127, 2024-09-19T04:57:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 176, 2024-09-19T04:57:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 2, 2024-09-19T04:57:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-19T04:57:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-19T04:57:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:57:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-19T04:57:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:57:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-19T04:57:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8248, 2024-09-19T04:57:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 182, 2024-09-19T04:57:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:57:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:57:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-19T04:57:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-19T04:57:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 138285, 2024-09-19T04:57:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 474, 2024-09-19T04:57:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-19T04:57:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-19T04:57:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-19T04:57:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-19T04:57:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-19T04:57:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-19T04:57:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-19T04:57:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 113, 2024-09-19T04:57:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 83447, 2024-09-19T04:57:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 33, 2024-09-19T04:57:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 113, 2024-09-19T04:57:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-19T04:57:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 113, 2024-09-19T04:57:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.533985644, 2024-09-19T04:57:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.004725536 2024-09-19T04:57:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-19T04:57:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-19T04:57:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 113, 2024-09-19T04:57:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 152 2024-09-19T04:57:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 113, 2024-09-19T04:57:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 19, 2024-09-19T04:57:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 94, 2024-09-19T04:57:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-19T04:57:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-19T04:57:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 0, 2024-09-19T04:57:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 10, 2024-09-19T04:57:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 3, 2024-09-19T04:57:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 24, 2024-09-19T04:57:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 16, 2024-09-19T04:57:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-19T04:57:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-19T04:57:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-19T04:57:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-09-19T04:57:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 17, 2024-09-19T04:57:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-19T04:57:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-19T04:57:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-19T04:57:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 16, 2024-09-19T04:57:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 20, 2024-09-19T04:57:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:57:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-19T04:57:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:57:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-19T04:57:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-19T04:57:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-19T04:57:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:57:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-19T04:57:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:57:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-19T04:57:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-19T04:57:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-19T04:57:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-19T04:57:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-19T04:57:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-19T04:57:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-19T04:57:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-19T04:57:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-19T04:57:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-19T04:57:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-19T04:57:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-19T04:57:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-19T04:57:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-19T04:57:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-19T04:57:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-19T04:57:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-19T04:57:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 96, 2024-09-19T04:57:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-19T04:57:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 12, 2024-09-19T04:57:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 10, 2024-09-19T04:57:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 10, 2024-09-19T04:57:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-09-19T04:57:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-19T04:57:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 36, 2024-09-19T04:57:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 0, 2024-09-19T04:57:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 10 2024-09-19T04:57:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-19T04:57:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-09-19T04:57:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:57:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-19T04:57:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-19T04:57:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-09-19T04:57:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-09-19T04:57:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-19T04:57:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-19T04:57:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 4, 2024-09-19T04:57:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-19T04:57:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 4, 2024-09-19T04:57:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 5, 2024-09-19T04:57:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 5, 2024-09-19T04:57:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-19T04:57:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:57:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-19T04:57:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 5160, 2024-09-19T04:57:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 5688232, 2024-09-19T04:57:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 5160, 2024-09-19T04:57:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-19T04:57:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-19T04:57:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 5160, 2024-09-19T04:57:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 5688232, 2024-09-19T04:57:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-19T04:57:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-19T04:57:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-19T04:57:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-19T04:57:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 113, 2024-09-19T04:57:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 84080, 2024-09-19T04:57:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 113, 2024-09-19T04:57:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 84080, 2024-09-19T04:57:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-19T04:57:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-19T04:57:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-19T04:57:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-19T04:57:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 113, 2024-09-19T04:57:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 113, 2024-09-19T04:57:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 113, 2024-09-19T04:57:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 113, 2024-09-19T04:57:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:57:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-19T04:57:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 5, 2024-09-19T04:57:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 633, 2024-09-19T04:57:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 5, 2024-09-19T04:57:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-19T04:57:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-19T04:57:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3, 2024-09-19T04:57:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 633, 2024-09-19T04:57:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55dda37d6960": { 2024-09-19T04:57:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-19T04:57:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 39, 2024-09-19T04:57:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 68847, 2024-09-19T04:57:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 39, 2024-09-19T04:57:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-19T04:57:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-19T04:57:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 21, 2024-09-19T04:57:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 68847, 2024-09-19T04:57:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:16.257 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-19T04:57:17.067 INFO:tasks.check_counter:Daemon mds.h mds.exported=1 2024-09-19T04:57:17.067 INFO:tasks.check_counter:Daemon mds.h mds.imported=1 2024-09-19T04:57:17.067 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=9 2024-09-19T04:57:17.067 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=485 2024-09-19T04:57:17.067 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-19T04:57:17.067 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:57:17.067 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:57:17.067 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-09-19T04:57:17.067 DEBUG:tasks.check_counter:Getting stats from k 2024-09-19T04:57:17.068 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-19T04:57:17.316 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:57:17.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:17 smithi183 ceph-mon[35333]: pgmap v329: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:57:17.350 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:17 smithi183 ceph-mon[35333]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T04:57:17.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:17 smithi123 ceph-mon[30511]: pgmap v329: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:57:17.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:17 smithi123 ceph-mon[30511]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T04:57:17.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:17 smithi193 ceph-mon[35592]: pgmap v329: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:57:17.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:17 smithi193 ceph-mon[35592]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T04:57:17.934 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-19T04:57:17.934 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:57:17.934 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1020, 2024-09-19T04:57:17.935 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 991, 2024-09-19T04:57:17.935 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1976480, 2024-09-19T04:57:17.935 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 376846, 2024-09-19T04:57:17.935 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-09-19T04:57:17.935 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-09-19T04:57:17.935 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.126582296, 2024-09-19T04:57:17.935 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.047237712, 2024-09-19T04:57:17.935 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.064938216, 2024-09-19T04:57:17.935 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.021248386, 2024-09-19T04:57:17.935 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:17.935 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 991, 2024-09-19T04:57:17.935 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.052378095, 2024-09-19T04:57:17.935 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000052853 2024-09-19T04:57:17.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.936 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:17.936 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.936 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.936 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.936 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1723472, 2024-09-19T04:57:17.936 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 137392 2024-09-19T04:57:17.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.936 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:57:17.936 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 640, 2024-09-19T04:57:17.936 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 805, 2024-09-19T04:57:17.937 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 157920, 2024-09-19T04:57:17.937 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 650664, 2024-09-19T04:57:17.937 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 7, 2024-09-19T04:57:17.937 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-09-19T04:57:17.937 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.062551035, 2024-09-19T04:57:17.937 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.020767900, 2024-09-19T04:57:17.937 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.014450296, 2024-09-19T04:57:17.937 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.021204980, 2024-09-19T04:57:17.937 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:17.937 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 802, 2024-09-19T04:57:17.937 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.040612562, 2024-09-19T04:57:17.937 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000050639 2024-09-19T04:57:17.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.937 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:17.938 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.938 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.938 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.938 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T04:57:17.938 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 447712 2024-09-19T04:57:17.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.938 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:57:17.938 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 773, 2024-09-19T04:57:17.938 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 775, 2024-09-19T04:57:17.938 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 271110, 2024-09-19T04:57:17.938 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 245815, 2024-09-19T04:57:17.938 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-09-19T04:57:17.939 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-09-19T04:57:17.939 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.056298441, 2024-09-19T04:57:17.939 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.018214939, 2024-09-19T04:57:17.939 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.015750972, 2024-09-19T04:57:17.939 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.017032887, 2024-09-19T04:57:17.939 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:17.939 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 773, 2024-09-19T04:57:17.939 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.021465479, 2024-09-19T04:57:17.939 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000027769 2024-09-19T04:57:17.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.939 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:17.939 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.939 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.940 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.940 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:57:17.940 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:57:17.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.940 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-19T04:57:17.940 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-19T04:57:17.940 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-19T04:57:17.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.940 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-19T04:57:17.940 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-19T04:57:17.940 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-19T04:57:17.940 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3612, 2024-09-19T04:57:17.941 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.089013885, 2024-09-19T04:57:17.941 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000024643 2024-09-19T04:57:17.941 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:17.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.941 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-19T04:57:17.941 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-19T04:57:17.941 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-19T04:57:17.941 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.941 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.941 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.941 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:17.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.941 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-19T04:57:17.942 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-09-19T04:57:17.942 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-09-19T04:57:17.942 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-19T04:57:17.942 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.942 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.942 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.942 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-19T04:57:17.942 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-09-19T04:57:17.942 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-09-19T04:57:17.942 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-19T04:57:17.942 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-09-19T04:57:17.942 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-09-19T04:57:17.942 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-19T04:57:17.943 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 6, 2024-09-19T04:57:17.943 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2024-09-19T04:57:17.943 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 6, 2024-09-19T04:57:17.943 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-09-19T04:57:17.943 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 6, 2024-09-19T04:57:17.943 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-09-19T04:57:17.943 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-09-19T04:57:17.943 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-09-19T04:57:17.943 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 1, 2024-09-19T04:57:17.943 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-09-19T04:57:17.943 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-09-19T04:57:17.943 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-09-19T04:57:17.943 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-09-19T04:57:17.943 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:57:17.944 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-19T04:57:17.944 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-09-19T04:57:17.944 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-09-19T04:57:17.944 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-19T04:57:17.944 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-09-19T04:57:17.944 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-09-19T04:57:17.944 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-09-19T04:57:17.944 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-09-19T04:57:17.944 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-19T04:57:17.944 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:57:17.944 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-19T04:57:17.944 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 0, 2024-09-19T04:57:17.945 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 0, 2024-09-19T04:57:17.945 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-09-19T04:57:17.945 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:57:17.945 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-19T04:57:17.945 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:57:17.945 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:57:17.945 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:57:17.945 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:57:17.945 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:57:17.945 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-09-19T04:57:17.945 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:57:17.945 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:57:17.945 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:57:17.945 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:57:17.946 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:57:17.946 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:57:17.946 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-09-19T04:57:17.946 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:57:17.946 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-09-19T04:57:17.946 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-09-19T04:57:17.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.946 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-19T04:57:17.946 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-09-19T04:57:17.946 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-09-19T04:57:17.946 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-09-19T04:57:17.946 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-09-19T04:57:17.946 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-09-19T04:57:17.947 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-09-19T04:57:17.947 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-19T04:57:17.947 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:57:17.947 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-09-19T04:57:17.947 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-09-19T04:57:17.947 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-09-19T04:57:17.947 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-19T04:57:17.947 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-19T04:57:17.947 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:57:17.947 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:57:17.947 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-19T04:57:17.947 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-19T04:57:17.947 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:57:17.947 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:57:17.948 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:57:17.948 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-09-19T04:57:17.948 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-19T04:57:17.948 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:57:17.948 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-19T04:57:17.948 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-09-19T04:57:17.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.948 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-19T04:57:17.948 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-19T04:57:17.948 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-09-19T04:57:17.948 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-09-19T04:57:17.948 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-09-19T04:57:17.948 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 39, 2024-09-19T04:57:17.949 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-19T04:57:17.949 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-09-19T04:57:17.949 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-09-19T04:57:17.949 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-09-19T04:57:17.949 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-09-19T04:57:17.949 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-09-19T04:57:17.949 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-09-19T04:57:17.949 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-19T04:57:17.949 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-09-19T04:57:17.949 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-09-19T04:57:17.949 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 4263151, 2024-09-19T04:57:17.949 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 4261063, 2024-09-19T04:57:17.949 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-19T04:57:17.949 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.950 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.950 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.950 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 39 2024-09-19T04:57:17.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.950 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-19T04:57:17.950 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 4, 2024-09-19T04:57:17.950 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 26, 2024-09-19T04:57:17.950 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 22, 2024-09-19T04:57:17.950 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 2, 2024-09-19T04:57:17.950 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 14, 2024-09-19T04:57:17.950 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 12, 2024-09-19T04:57:17.950 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 6, 2024-09-19T04:57:17.950 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 28, 2024-09-19T04:57:17.950 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 22, 2024-09-19T04:57:17.951 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-09-19T04:57:17.951 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-09-19T04:57:17.951 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-09-19T04:57:17.951 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 48760, 2024-09-19T04:57:17.951 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 215328 2024-09-19T04:57:17.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.951 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-19T04:57:17.951 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-09-19T04:57:17.951 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-09-19T04:57:17.951 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-09-19T04:57:17.951 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-09-19T04:57:17.951 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-09-19T04:57:17.951 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-19T04:57:17.952 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.952 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.952 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.952 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-19T04:57:17.952 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.952 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.952 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.952 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-19T04:57:17.952 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.952 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.952 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.953 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-19T04:57:17.953 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.953 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.953 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.953 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-19T04:57:17.953 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.953 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.953 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.953 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-19T04:57:17.953 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.953 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.953 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.954 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-19T04:57:17.954 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.954 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.954 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.954 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-19T04:57:17.954 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.954 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.954 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.954 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-19T04:57:17.954 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.955 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.955 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.955 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-19T04:57:17.955 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.955 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.955 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.955 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-19T04:57:17.955 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.955 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.955 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.955 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-19T04:57:17.955 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.956 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.956 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.956 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-19T04:57:17.956 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.957 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.957 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.957 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-19T04:57:17.957 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.957 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.957 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.957 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-19T04:57:17.957 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.957 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.957 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.958 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-19T04:57:17.958 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.958 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.958 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.958 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-19T04:57:17.958 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.958 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.958 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.958 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-19T04:57:17.958 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.958 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.959 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.959 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-19T04:57:17.959 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.959 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.959 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.959 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-19T04:57:17.959 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.959 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.959 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.960 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-19T04:57:17.960 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.960 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.960 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.960 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-19T04:57:17.960 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.960 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.960 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.960 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-19T04:57:17.960 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.960 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.960 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.961 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-19T04:57:17.961 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.961 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.961 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.961 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-19T04:57:17.961 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.961 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.961 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.961 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-19T04:57:17.961 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.962 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.962 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.962 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-19T04:57:17.962 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.962 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.962 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.962 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-19T04:57:17.962 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.962 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.962 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.963 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-19T04:57:17.963 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.963 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.963 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.963 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:57:17.963 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:57:17.963 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-19T04:57:17.963 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.963 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.963 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.963 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:17.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.964 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-19T04:57:17.964 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 1, 2024-09-19T04:57:17.964 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 2, 2024-09-19T04:57:17.964 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 1, 2024-09-19T04:57:17.964 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 2, 2024-09-19T04:57:17.964 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-19T04:57:17.964 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-09-19T04:57:17.964 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-09-19T04:57:17.964 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 3373, 2024-09-19T04:57:17.964 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-19T04:57:17.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.965 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-19T04:57:17.965 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:57:17.965 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-19T04:57:17.965 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:57:17.965 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:57:17.965 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:57:17.965 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:57:17.965 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:57:17.965 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:57:17.965 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:57:17.965 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:57:17.965 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:57:17.965 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:57:17.966 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:57:17.966 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:57:17.966 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:57:17.966 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-19T04:57:17.966 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:57:17.966 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-19T04:57:17.966 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:57:17.966 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:57:17.966 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:57:17.966 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:57:17.966 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:57:17.966 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:57:17.966 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:57:17.967 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-19T04:57:17.967 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:57:17.967 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:57:17.967 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:57:17.967 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-19T04:57:17.967 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-19T04:57:17.967 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-19T04:57:17.967 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:57:17.967 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:57:17.967 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:57:17.967 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:57:17.967 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 189450, 2024-09-19T04:57:17.967 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 53, 2024-09-19T04:57:17.967 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:57:17.968 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-19T04:57:17.968 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-19T04:57:17.968 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-19T04:57:17.968 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:57:17.968 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-19T04:57:17.968 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:57:17.968 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-19T04:57:17.968 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8128, 2024-09-19T04:57:17.968 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 179, 2024-09-19T04:57:17.968 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:57:17.968 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:57:17.968 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-19T04:57:17.968 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-19T04:57:17.969 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 19448, 2024-09-19T04:57:17.969 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 59, 2024-09-19T04:57:17.969 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-19T04:57:17.969 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-19T04:57:17.969 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-19T04:57:17.969 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-19T04:57:17.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.969 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-19T04:57:17.969 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-19T04:57:17.969 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-19T04:57:17.969 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 2164, 2024-09-19T04:57:17.969 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 21, 2024-09-19T04:57:17.969 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 3, 2024-09-19T04:57:17.969 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 2164, 2024-09-19T04:57:17.970 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-19T04:57:17.970 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2164, 2024-09-19T04:57:17.970 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.037972154, 2024-09-19T04:57:17.970 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000479654 2024-09-19T04:57:17.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.970 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-19T04:57:17.970 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-19T04:57:17.970 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2164, 2024-09-19T04:57:17.970 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2165 2024-09-19T04:57:17.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.970 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 2164, 2024-09-19T04:57:17.970 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 2164, 2024-09-19T04:57:17.970 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-09-19T04:57:17.971 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-19T04:57:17.971 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-19T04:57:17.971 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 1388, 2024-09-19T04:57:17.971 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-09-19T04:57:17.971 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 774, 2024-09-19T04:57:17.971 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-09-19T04:57:17.971 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-09-19T04:57:17.971 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-19T04:57:17.971 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-19T04:57:17.971 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-19T04:57:17.971 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-09-19T04:57:17.971 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-09-19T04:57:17.971 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-19T04:57:17.971 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-19T04:57:17.972 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-19T04:57:17.972 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 1, 2024-09-19T04:57:17.972 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-09-19T04:57:17.972 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:57:17.972 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-19T04:57:17.972 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:57:17.972 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-19T04:57:17.972 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-19T04:57:17.972 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-19T04:57:17.972 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:57:17.972 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-19T04:57:17.972 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:57:17.972 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-19T04:57:17.973 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-19T04:57:17.973 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-19T04:57:17.973 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-19T04:57:17.973 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-19T04:57:17.973 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-19T04:57:17.973 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-19T04:57:17.973 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-19T04:57:17.973 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-19T04:57:17.973 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-19T04:57:17.973 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-19T04:57:17.973 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-19T04:57:17.973 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-19T04:57:17.973 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-19T04:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-19T04:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-19T04:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-19T04:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 97, 2024-09-19T04:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-19T04:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 13, 2024-09-19T04:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 5, 2024-09-19T04:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 5, 2024-09-19T04:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-09-19T04:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-19T04:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-09-19T04:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-09-19T04:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-09-19T04:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.975 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-19T04:57:17.975 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-09-19T04:57:17.975 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:57:17.975 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-19T04:57:17.975 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-19T04:57:17.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.975 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-09-19T04:57:17.975 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-09-19T04:57:17.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.975 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-19T04:57:17.975 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-19T04:57:17.975 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:57:17.975 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-19T04:57:17.975 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-09-19T04:57:17.976 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-09-19T04:57:17.976 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-09-19T04:57:17.976 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-19T04:57:17.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.976 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:57:17.976 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:17.976 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-19T04:57:17.976 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:17.976 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 2429, 2024-09-19T04:57:17.976 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 2170411, 2024-09-19T04:57:17.976 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:17.976 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 2429, 2024-09-19T04:57:17.976 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-19T04:57:17.977 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-19T04:57:17.977 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2429, 2024-09-19T04:57:17.977 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2170411, 2024-09-19T04:57:17.977 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:17.977 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.977 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.977 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.977 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:17.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.977 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-19T04:57:17.977 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:17.977 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-19T04:57:17.977 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:17.978 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-19T04:57:17.978 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-19T04:57:17.978 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:17.978 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:17.978 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 2164, 2024-09-19T04:57:17.978 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 68852, 2024-09-19T04:57:17.978 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2164, 2024-09-19T04:57:17.978 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 68852, 2024-09-19T04:57:17.978 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:17.978 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.978 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.978 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.978 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:17.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.979 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-19T04:57:17.979 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:17.979 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-19T04:57:17.979 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:17.979 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-19T04:57:17.979 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-19T04:57:17.979 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:17.979 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:17.979 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 2164, 2024-09-19T04:57:17.980 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 2164, 2024-09-19T04:57:17.980 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2164, 2024-09-19T04:57:17.980 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2164, 2024-09-19T04:57:17.980 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:17.980 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.980 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.980 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.980 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:17.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.980 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:57:17.980 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:17.980 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-19T04:57:17.981 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:17.981 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-19T04:57:17.981 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-19T04:57:17.981 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:17.981 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:17.981 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-19T04:57:17.981 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-19T04:57:17.981 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-19T04:57:17.981 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-19T04:57:17.981 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:17.981 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.981 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.981 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.981 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:17.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-19T04:57:17.982 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55a984204b40": { 2024-09-19T04:57:17.982 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-19T04:57:17.982 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-19T04:57:17.982 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-19T04:57:17.982 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-19T04:57:17.982 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-19T04:57:17.982 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:17.982 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:17.982 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-19T04:57:17.982 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-19T04:57:17.982 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-19T04:57:17.982 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-19T04:57:17.983 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-19T04:57:17.983 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-19T04:57:17.983 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-19T04:57:17.983 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-19T04:57:17.983 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:17.983 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-19T04:57:17.983 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-19T04:57:18.742 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-19T04:57:18.742 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-19T04:57:18.742 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-19T04:57:18.742 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-19T04:57:18.743 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-19T04:57:18.743 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:57:18.743 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:57:18.743 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-09-19T04:57:18.743 DEBUG:tasks.check_counter:Getting stats from c 2024-09-19T04:57:18.743 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-19T04:57:19.003 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:57:19.398 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:19 smithi193 ceph-mon[35592]: pgmap v331: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T04:57:19.533 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-19T04:57:19.533 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:57:19.533 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 76867, 2024-09-19T04:57:19.533 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 136764, 2024-09-19T04:57:19.533 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 23105870, 2024-09-19T04:57:19.533 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 105485805, 2024-09-19T04:57:19.533 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 13, 2024-09-19T04:57:19.533 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 5, 2024-09-19T04:57:19.533 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 4.518563759, 2024-09-19T04:57:19.533 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 2.113435834, 2024-09-19T04:57:19.534 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 2.521693480, 2024-09-19T04:57:19.534 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.549358176, 2024-09-19T04:57:19.534 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:19.534 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 136762, 2024-09-19T04:57:19.534 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 4.227603790, 2024-09-19T04:57:19.534 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000030912 2024-09-19T04:57:19.534 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.534 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:19.534 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 86926, 2024-09-19T04:57:19.534 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.021713519, 2024-09-19T04:57:19.534 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000249 2024-09-19T04:57:19.534 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.534 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1584, 2024-09-19T04:57:19.535 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 1440 2024-09-19T04:57:19.535 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.535 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:57:19.535 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 42687, 2024-09-19T04:57:19.535 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 44563, 2024-09-19T04:57:19.535 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 12365237, 2024-09-19T04:57:19.535 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 103095785, 2024-09-19T04:57:19.535 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 16, 2024-09-19T04:57:19.535 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 7, 2024-09-19T04:57:19.535 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 2.813928944, 2024-09-19T04:57:19.535 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 1.125367062, 2024-09-19T04:57:19.535 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.750159633, 2024-09-19T04:57:19.535 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.798094203, 2024-09-19T04:57:19.535 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:19.536 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 44563, 2024-09-19T04:57:19.536 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.975159351, 2024-09-19T04:57:19.536 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000044322 2024-09-19T04:57:19.536 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.536 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:19.536 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5455, 2024-09-19T04:57:19.536 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001237848, 2024-09-19T04:57:19.536 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000226 2024-09-19T04:57:19.536 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.536 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1749120, 2024-09-19T04:57:19.536 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 146944 2024-09-19T04:57:19.537 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.537 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:57:19.537 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 26047, 2024-09-19T04:57:19.537 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 30125, 2024-09-19T04:57:19.537 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 7409825, 2024-09-19T04:57:19.537 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 32732867, 2024-09-19T04:57:19.537 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 38, 2024-09-19T04:57:19.537 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 8, 2024-09-19T04:57:19.537 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 1.508138783, 2024-09-19T04:57:19.538 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.491179484, 2024-09-19T04:57:19.538 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.508303588, 2024-09-19T04:57:19.538 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.419776438, 2024-09-19T04:57:19.538 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:19.538 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 30122, 2024-09-19T04:57:19.538 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.330293918, 2024-09-19T04:57:19.538 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000044163 2024-09-19T04:57:19.538 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.538 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:19.538 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 11963, 2024-09-19T04:57:19.538 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.002330430, 2024-09-19T04:57:19.538 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000194 2024-09-19T04:57:19.538 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.538 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T04:57:19.539 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 449280 2024-09-19T04:57:19.539 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.539 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-09-19T04:57:19.539 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-09-19T04:57:19.539 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-09-19T04:57:19.539 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.539 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-09-19T04:57:19.539 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-19T04:57:19.539 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-19T04:57:19.539 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 100377, 2024-09-19T04:57:19.539 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 34.743426939, 2024-09-19T04:57:19.539 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000346129 2024-09-19T04:57:19.539 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:19.540 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.540 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-09-19T04:57:19.540 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-19T04:57:19.540 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-19T04:57:19.540 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 12945, 2024-09-19T04:57:19.540 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 3.511494281, 2024-09-19T04:57:19.540 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000271262 2024-09-19T04:57:19.540 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:19.540 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.540 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-09-19T04:57:19.540 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 25204, 2024-09-19T04:57:19.540 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 25160, 2024-09-19T04:57:19.540 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-09-19T04:57:19.540 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 25160, 2024-09-19T04:57:19.541 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 71.324644972, 2024-09-19T04:57:19.541 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002834842 2024-09-19T04:57:19.541 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.541 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-09-19T04:57:19.541 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 44, 2024-09-19T04:57:19.541 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-09-19T04:57:19.541 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-09-19T04:57:19.542 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 220, 2024-09-19T04:57:19.542 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 25, 2024-09-19T04:57:19.542 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 3, 2024-09-19T04:57:19.542 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 10089, 2024-09-19T04:57:19.542 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 261, 2024-09-19T04:57:19.542 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 9754, 2024-09-19T04:57:19.542 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 74, 2024-09-19T04:57:19.542 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 9715, 2024-09-19T04:57:19.542 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 30, 2024-09-19T04:57:19.543 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 1, 2024-09-19T04:57:19.543 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 1, 2024-09-19T04:57:19.543 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 13, 2024-09-19T04:57:19.543 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 27661, 2024-09-19T04:57:19.543 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 26426, 2024-09-19T04:57:19.543 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 42, 2024-09-19T04:57:19.543 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 5, 2024-09-19T04:57:19.543 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:57:19.543 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-09-19T04:57:19.543 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 51, 2024-09-19T04:57:19.543 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-09-19T04:57:19.543 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-09-19T04:57:19.543 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 4, 2024-09-19T04:57:19.544 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 23, 2024-09-19T04:57:19.544 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 4, 2024-09-19T04:57:19.544 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 336, 2024-09-19T04:57:19.544 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-09-19T04:57:19.544 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 20700, 2024-09-19T04:57:19.544 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-09-19T04:57:19.544 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 4, 2024-09-19T04:57:19.544 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 234, 2024-09-19T04:57:19.544 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 6, 2024-09-19T04:57:19.544 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 20700, 2024-09-19T04:57:19.545 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-09-19T04:57:19.545 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:57:19.545 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:57:19.545 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 320, 2024-09-19T04:57:19.545 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:57:19.545 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 320, 2024-09-19T04:57:19.545 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 20380, 2024-09-19T04:57:19.545 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 14, 2024-09-19T04:57:19.545 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 9744, 2024-09-19T04:57:19.545 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 23520, 2024-09-19T04:57:19.545 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 7863, 2024-09-19T04:57:19.545 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:57:19.545 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:57:19.545 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 21585, 2024-09-19T04:57:19.546 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 15495, 2024-09-19T04:57:19.546 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 496, 2024-09-19T04:57:19.546 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 17801 2024-09-19T04:57:19.546 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.546 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-09-19T04:57:19.546 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 803, 2024-09-19T04:57:19.546 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 12, 2024-09-19T04:57:19.546 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 8, 2024-09-19T04:57:19.546 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 10, 2024-09-19T04:57:19.546 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 1013, 2024-09-19T04:57:19.546 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 60, 2024-09-19T04:57:19.546 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-09-19T04:57:19.546 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:57:19.546 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 7250, 2024-09-19T04:57:19.547 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 7187, 2024-09-19T04:57:19.547 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 3, 2024-09-19T04:57:19.547 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-09-19T04:57:19.547 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-09-19T04:57:19.547 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:57:19.547 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:57:19.547 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-09-19T04:57:19.547 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-09-19T04:57:19.547 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:57:19.547 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:57:19.547 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:57:19.547 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 4, 2024-09-19T04:57:19.547 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-09-19T04:57:19.548 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 28, 2024-09-19T04:57:19.548 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-09-19T04:57:19.548 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-09-19T04:57:19.548 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.548 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-09-19T04:57:19.548 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-09-19T04:57:19.548 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 61715, 2024-09-19T04:57:19.548 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-09-19T04:57:19.548 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-09-19T04:57:19.548 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 61715, 2024-09-19T04:57:19.548 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-09-19T04:57:19.548 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-09-19T04:57:19.548 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 85, 2024-09-19T04:57:19.549 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-09-19T04:57:19.549 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-09-19T04:57:19.549 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 85, 2024-09-19T04:57:19.549 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 7, 2024-09-19T04:57:19.549 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-09-19T04:57:19.549 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-09-19T04:57:19.549 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-09-19T04:57:19.549 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 174987825, 2024-09-19T04:57:19.549 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-09-19T04:57:19.549 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-09-19T04:57:19.549 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 24683, 2024-09-19T04:57:19.549 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 57.352543247, 2024-09-19T04:57:19.549 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002323564 2024-09-19T04:57:19.549 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.549 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-09-19T04:57:19.550 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.550 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-09-19T04:57:19.550 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 4881, 2024-09-19T04:57:19.550 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 12088, 2024-09-19T04:57:19.550 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 7207, 2024-09-19T04:57:19.550 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 231, 2024-09-19T04:57:19.550 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 290, 2024-09-19T04:57:19.550 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 59, 2024-09-19T04:57:19.550 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 10089, 2024-09-19T04:57:19.550 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 17402, 2024-09-19T04:57:19.550 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 7313, 2024-09-19T04:57:19.550 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 1, 2024-09-19T04:57:19.550 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 7480, 2024-09-19T04:57:19.550 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 7479, 2024-09-19T04:57:19.551 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 118952, 2024-09-19T04:57:19.551 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 223520 2024-09-19T04:57:19.551 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.551 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-09-19T04:57:19.551 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 44170, 2024-09-19T04:57:19.551 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 26, 2024-09-19T04:57:19.551 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 25204, 2024-09-19T04:57:19.551 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 17, 2024-09-19T04:57:19.551 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 94, 2024-09-19T04:57:19.551 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-09-19T04:57:19.551 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 7173, 2024-09-19T04:57:19.551 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 23.235179980, 2024-09-19T04:57:19.551 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003239255 2024-09-19T04:57:19.551 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.552 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-09-19T04:57:19.552 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 232, 2024-09-19T04:57:19.552 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.024476774, 2024-09-19T04:57:19.552 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.004415848 2024-09-19T04:57:19.552 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.552 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-09-19T04:57:19.552 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.552 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.552 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.552 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.552 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-09-19T04:57:19.552 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3, 2024-09-19T04:57:19.552 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.003722832, 2024-09-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001240944 2024-09-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-09-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 987, 2024-09-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.487695602, 2024-09-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000494119 2024-09-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-09-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-09-19T04:57:19.553 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-09-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000107038, 2024-09-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000107038 2024-09-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-09-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-09-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.554 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-09-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-09-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-09-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 54, 2024-09-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.140068405, 2024-09-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002593859 2024-09-19T04:57:19.555 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.556 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-09-19T04:57:19.556 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.556 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.556 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.556 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.556 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-09-19T04:57:19.556 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.556 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.556 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.556 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.556 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-09-19T04:57:19.556 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 157, 2024-09-19T04:57:19.556 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.055579510, 2024-09-19T04:57:19.557 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000354009 2024-09-19T04:57:19.557 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.557 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-09-19T04:57:19.557 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 462, 2024-09-19T04:57:19.557 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.996108705, 2024-09-19T04:57:19.557 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002156079 2024-09-19T04:57:19.557 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.557 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-09-19T04:57:19.557 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 84, 2024-09-19T04:57:19.557 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.169848731, 2024-09-19T04:57:19.557 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002022008 2024-09-19T04:57:19.557 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.557 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-09-19T04:57:19.558 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.558 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.558 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.558 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.558 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-09-19T04:57:19.558 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.558 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.558 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.558 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.558 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-09-19T04:57:19.558 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 72, 2024-09-19T04:57:19.558 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2.233415736, 2024-09-19T04:57:19.558 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.031019663 2024-09-19T04:57:19.558 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.559 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-09-19T04:57:19.559 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.559 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.559 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.559 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.559 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-09-19T04:57:19.559 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.559 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.559 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.559 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.559 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-09-19T04:57:19.559 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 8793, 2024-09-19T04:57:19.559 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 29.898353943, 2024-09-19T04:57:19.559 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003400244 2024-09-19T04:57:19.560 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.560 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-09-19T04:57:19.560 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.560 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.560 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.560 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.560 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-09-19T04:57:19.560 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 8, 2024-09-19T04:57:19.560 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.002325082, 2024-09-19T04:57:19.560 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000290635 2024-09-19T04:57:19.560 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.560 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-09-19T04:57:19.560 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.560 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.561 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.561 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.561 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-09-19T04:57:19.561 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.561 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.561 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.561 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.561 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-09-19T04:57:19.561 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5, 2024-09-19T04:57:19.561 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.005860381, 2024-09-19T04:57:19.561 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001172076 2024-09-19T04:57:19.561 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.561 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-09-19T04:57:19.561 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 7129, 2024-09-19T04:57:19.562 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 13.071902253, 2024-09-19T04:57:19.562 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001833623 2024-09-19T04:57:19.562 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.562 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:57:19.562 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:57:19.562 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-09-19T04:57:19.562 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.562 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.562 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.562 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:19.562 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.562 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-09-19T04:57:19.562 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 1, 2024-09-19T04:57:19.563 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 2, 2024-09-19T04:57:19.563 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 1, 2024-09-19T04:57:19.563 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 1, 2024-09-19T04:57:19.563 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-09-19T04:57:19.563 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 8257, 2024-09-19T04:57:19.563 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 8257, 2024-09-19T04:57:19.563 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 623, 2024-09-19T04:57:19.563 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-09-19T04:57:19.563 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.563 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-09-19T04:57:19.563 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:57:19.563 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-09-19T04:57:19.563 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:57:19.563 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:57:19.563 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:57:19.564 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:57:19.564 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:57:19.564 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:57:19.564 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:57:19.564 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:57:19.564 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:57:19.564 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:57:19.564 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:57:19.564 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:57:19.564 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:57:19.564 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-09-19T04:57:19.564 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:57:19.564 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-09-19T04:57:19.564 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:57:19.565 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:57:19.565 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:57:19.565 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:57:19.565 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:57:19.565 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:57:19.565 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:57:19.565 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-09-19T04:57:19.565 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:57:19.565 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:57:19.565 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:57:19.565 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-09-19T04:57:19.565 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-09-19T04:57:19.565 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-09-19T04:57:19.565 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:57:19.566 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:57:19.566 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:57:19.566 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:57:19.566 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 984214, 2024-09-19T04:57:19.566 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 4895, 2024-09-19T04:57:19.566 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 1408, 2024-09-19T04:57:19.566 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 16, 2024-09-19T04:57:19.566 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-09-19T04:57:19.566 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-09-19T04:57:19.566 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:57:19.566 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-09-19T04:57:19.566 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:57:19.566 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-09-19T04:57:19.566 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8128, 2024-09-19T04:57:19.566 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 179, 2024-09-19T04:57:19.567 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:57:19.567 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:57:19.567 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-09-19T04:57:19.567 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-09-19T04:57:19.567 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 18888887, 2024-09-19T04:57:19.567 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 194062, 2024-09-19T04:57:19.567 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-09-19T04:57:19.567 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-09-19T04:57:19.567 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-09-19T04:57:19.567 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-09-19T04:57:19.567 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.567 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-09-19T04:57:19.567 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-09-19T04:57:19.567 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-09-19T04:57:19.567 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 89990, 2024-09-19T04:57:19.567 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 175137811, 2024-09-19T04:57:19.568 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 33, 2024-09-19T04:57:19.568 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 89990, 2024-09-19T04:57:19.568 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-09-19T04:57:19.568 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 89990, 2024-09-19T04:57:19.568 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 218.934514040, 2024-09-19T04:57:19.568 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002432876 2024-09-19T04:57:19.568 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.568 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-09-19T04:57:19.568 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-09-19T04:57:19.568 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 89990, 2024-09-19T04:57:19.568 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 90360 2024-09-19T04:57:19.568 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.568 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 89990, 2024-09-19T04:57:19.568 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 20773, 2024-09-19T04:57:19.568 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 69217, 2024-09-19T04:57:19.569 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-09-19T04:57:19.569 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-09-19T04:57:19.569 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 0, 2024-09-19T04:57:19.569 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 10, 2024-09-19T04:57:19.569 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 73, 2024-09-19T04:57:19.569 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 24756, 2024-09-19T04:57:19.569 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 58, 2024-09-19T04:57:19.569 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-09-19T04:57:19.569 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-09-19T04:57:19.569 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-09-19T04:57:19.569 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 9, 2024-09-19T04:57:19.569 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 7309, 2024-09-19T04:57:19.569 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-09-19T04:57:19.569 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-09-19T04:57:19.569 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-09-19T04:57:19.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 20700, 2024-09-19T04:57:19.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 20, 2024-09-19T04:57:19.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:57:19.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-09-19T04:57:19.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:57:19.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-09-19T04:57:19.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-09-19T04:57:19.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-09-19T04:57:19.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:57:19.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-09-19T04:57:19.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:57:19.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 36717, 2024-09-19T04:57:19.570 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-09-19T04:57:19.570 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-09-19T04:57:19.571 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-09-19T04:57:19.571 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-09-19T04:57:19.571 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-09-19T04:57:19.571 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-09-19T04:57:19.571 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-09-19T04:57:19.571 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-09-19T04:57:19.571 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-09-19T04:57:19.571 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-09-19T04:57:19.571 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-09-19T04:57:19.571 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-09-19T04:57:19.571 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-09-19T04:57:19.571 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-09-19T04:57:19.571 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-09-19T04:57:19.571 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-09-19T04:57:19.572 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 97, 2024-09-19T04:57:19.572 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-09-19T04:57:19.572 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 13, 2024-09-19T04:57:19.572 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 12, 2024-09-19T04:57:19.572 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 12, 2024-09-19T04:57:19.572 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-09-19T04:57:19.572 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-09-19T04:57:19.572 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 645, 2024-09-19T04:57:19.572 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 0, 2024-09-19T04:57:19.572 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 63 2024-09-19T04:57:19.572 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.572 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-09-19T04:57:19.572 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-09-19T04:57:19.572 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:57:19.573 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-09-19T04:57:19.573 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-09-19T04:57:19.573 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.573 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-09-19T04:57:19.573 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-09-19T04:57:19.573 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.573 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-09-19T04:57:19.573 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-09-19T04:57:19.573 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 30, 2024-09-19T04:57:19.573 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-09-19T04:57:19.573 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 6, 2024-09-19T04:57:19.573 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 7217, 2024-09-19T04:57:19.573 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 7187, 2024-09-19T04:57:19.573 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-09-19T04:57:19.574 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.574 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:57:19.574 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:19.574 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-19T04:57:19.574 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:19.574 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 145597, 2024-09-19T04:57:19.574 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 30518854, 2024-09-19T04:57:19.574 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:19.574 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 145597, 2024-09-19T04:57:19.574 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-19T04:57:19.574 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-19T04:57:19.574 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 145597, 2024-09-19T04:57:19.574 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 30518854, 2024-09-19T04:57:19.574 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:19.575 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.575 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.575 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.575 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:19.575 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.575 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-09-19T04:57:19.575 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:19.575 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-19T04:57:19.575 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:19.575 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-19T04:57:19.575 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-19T04:57:19.575 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:19.575 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:19.576 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 89990, 2024-09-19T04:57:19.576 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 176007650, 2024-09-19T04:57:19.576 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 89990, 2024-09-19T04:57:19.576 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 176007650, 2024-09-19T04:57:19.576 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:19.576 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.576 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.576 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.576 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:19.576 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.576 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-09-19T04:57:19.576 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:19.576 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-09-19T04:57:19.577 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:19.577 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-19T04:57:19.577 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-19T04:57:19.577 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:19.577 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:19.577 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 89990, 2024-09-19T04:57:19.577 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 89990, 2024-09-19T04:57:19.577 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 89990, 2024-09-19T04:57:19.577 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 89990, 2024-09-19T04:57:19.577 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:19.577 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.577 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.577 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.578 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:19.578 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.578 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:57:19.578 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:19.578 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-19T04:57:19.578 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:19.578 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 7187, 2024-09-19T04:57:19.578 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 869839, 2024-09-19T04:57:19.578 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:19.578 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 7187, 2024-09-19T04:57:19.578 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-19T04:57:19.578 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-19T04:57:19.578 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 73, 2024-09-19T04:57:19.578 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 869839, 2024-09-19T04:57:19.579 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:19.579 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.579 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.579 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.579 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:19.579 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:19.579 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x55774de3c960": { 2024-09-19T04:57:19.579 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:19.579 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-19T04:57:19.579 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:19.579 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 61715, 2024-09-19T04:57:19.579 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 170793521, 2024-09-19T04:57:19.579 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:19.579 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 61715, 2024-09-19T04:57:19.580 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-19T04:57:19.580 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-19T04:57:19.580 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 24683, 2024-09-19T04:57:19.580 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 170793521, 2024-09-19T04:57:19.580 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:19.580 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:19.580 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:19.580 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:19.580 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:19.580 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:19.580 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-19T04:57:19.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:19 smithi183 ceph-mon[35333]: pgmap v331: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T04:57:19.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:19 smithi123 ceph-mon[30511]: pgmap v331: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T04:57:20.499 INFO:tasks.check_counter:Daemon mds.c mds.exported=4 2024-09-19T04:57:20.500 INFO:tasks.check_counter:Daemon mds.c mds.imported=4 2024-09-19T04:57:20.500 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=803 2024-09-19T04:57:20.500 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=12 2024-09-19T04:57:20.500 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-19T04:57:20.500 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:57:20.500 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:57:20.500 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=25 2024-09-19T04:57:20.500 DEBUG:tasks.check_counter:Getting stats from f 2024-09-19T04:57:20.500 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-19T04:57:20.753 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:57:21.372 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-19T04:57:21.372 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:57:21.372 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 904, 2024-09-19T04:57:21.372 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 929, 2024-09-19T04:57:21.372 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1971710, 2024-09-19T04:57:21.373 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 578118, 2024-09-19T04:57:21.373 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 17, 2024-09-19T04:57:21.373 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 5, 2024-09-19T04:57:21.373 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.113666223, 2024-09-19T04:57:21.373 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.040255385, 2024-09-19T04:57:21.373 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.098965375, 2024-09-19T04:57:21.373 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.005725238, 2024-09-19T04:57:21.373 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:21.373 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 928, 2024-09-19T04:57:21.373 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.049529600, 2024-09-19T04:57:21.373 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000053372 2024-09-19T04:57:21.373 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.374 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:21.374 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1145, 2024-09-19T04:57:21.374 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000245408, 2024-09-19T04:57:21.374 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000214 2024-09-19T04:57:21.374 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.374 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1734064, 2024-09-19T04:57:21.374 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 145632 2024-09-19T04:57:21.374 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.374 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:57:21.374 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 3806, 2024-09-19T04:57:21.374 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1925, 2024-09-19T04:57:21.374 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 4026766, 2024-09-19T04:57:21.374 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 873385, 2024-09-19T04:57:21.375 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 38, 2024-09-19T04:57:21.375 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 9, 2024-09-19T04:57:21.375 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.201397813, 2024-09-19T04:57:21.375 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.083517576, 2024-09-19T04:57:21.375 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.154526174, 2024-09-19T04:57:21.375 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000280253, 2024-09-19T04:57:21.375 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:21.375 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1923, 2024-09-19T04:57:21.375 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.051651848, 2024-09-19T04:57:21.375 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000026860 2024-09-19T04:57:21.375 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.375 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:21.376 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5582, 2024-09-19T04:57:21.376 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000800491, 2024-09-19T04:57:21.376 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000143 2024-09-19T04:57:21.376 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.376 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 6016, 2024-09-19T04:57:21.376 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 134416 2024-09-19T04:57:21.376 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.376 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:57:21.376 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 39, 2024-09-19T04:57:21.376 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 169, 2024-09-19T04:57:21.376 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 22516, 2024-09-19T04:57:21.376 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 393065, 2024-09-19T04:57:21.376 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 12, 2024-09-19T04:57:21.377 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 6, 2024-09-19T04:57:21.377 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.011589092, 2024-09-19T04:57:21.377 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.005548969, 2024-09-19T04:57:21.377 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.004121430, 2024-09-19T04:57:21.377 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000676711, 2024-09-19T04:57:21.377 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:21.377 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 167, 2024-09-19T04:57:21.377 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.030071839, 2024-09-19T04:57:21.377 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000180070 2024-09-19T04:57:21.377 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.377 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:21.377 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 19, 2024-09-19T04:57:21.377 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000002709, 2024-09-19T04:57:21.378 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000142 2024-09-19T04:57:21.378 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.378 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T04:57:21.378 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 318224 2024-09-19T04:57:21.378 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.378 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-09-19T04:57:21.378 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-09-19T04:57:21.378 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-09-19T04:57:21.378 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.378 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-09-19T04:57:21.378 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-19T04:57:21.378 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-19T04:57:21.378 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 85, 2024-09-19T04:57:21.379 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.005134641, 2024-09-19T04:57:21.379 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000060407 2024-09-19T04:57:21.379 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:21.379 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.379 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-09-19T04:57:21.379 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-19T04:57:21.379 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-19T04:57:21.379 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 16, 2024-09-19T04:57:21.379 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000260497, 2024-09-19T04:57:21.379 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000016281 2024-09-19T04:57:21.379 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:21.379 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.379 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-09-19T04:57:21.380 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 5, 2024-09-19T04:57:21.380 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 4, 2024-09-19T04:57:21.380 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-09-19T04:57:21.380 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4, 2024-09-19T04:57:21.380 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.003791410, 2024-09-19T04:57:21.380 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000947852 2024-09-19T04:57:21.380 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.380 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-09-19T04:57:21.380 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 1, 2024-09-19T04:57:21.380 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-09-19T04:57:21.380 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-09-19T04:57:21.380 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 11, 2024-09-19T04:57:21.380 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-09-19T04:57:21.380 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-09-19T04:57:21.381 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 23, 2024-09-19T04:57:21.381 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 6, 2024-09-19T04:57:21.381 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 7, 2024-09-19T04:57:21.381 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 10, 2024-09-19T04:57:21.381 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 13, 2024-09-19T04:57:21.381 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 117, 2024-09-19T04:57:21.381 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2024-09-19T04:57:21.381 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2024-09-19T04:57:21.381 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 4, 2024-09-19T04:57:21.381 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 276, 2024-09-19T04:57:21.381 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 219, 2024-09-19T04:57:21.381 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 1, 2024-09-19T04:57:21.381 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 53, 2024-09-19T04:57:21.381 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:57:21.382 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-09-19T04:57:21.382 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-09-19T04:57:21.382 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-09-19T04:57:21.382 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-09-19T04:57:21.382 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 1, 2024-09-19T04:57:21.382 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 2, 2024-09-19T04:57:21.382 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 1, 2024-09-19T04:57:21.382 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 1, 2024-09-19T04:57:21.382 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-09-19T04:57:21.382 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 6, 2024-09-19T04:57:21.382 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-09-19T04:57:21.382 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 4, 2024-09-19T04:57:21.382 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 234, 2024-09-19T04:57:21.382 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 6, 2024-09-19T04:57:21.383 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 6, 2024-09-19T04:57:21.383 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-09-19T04:57:21.383 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:57:21.383 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:57:21.383 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:57:21.383 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:57:21.383 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:57:21.383 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 6, 2024-09-19T04:57:21.383 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:57:21.383 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 2, 2024-09-19T04:57:21.383 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 2, 2024-09-19T04:57:21.383 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:57:21.384 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:57:21.384 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:57:21.384 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 2, 2024-09-19T04:57:21.384 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 1, 2024-09-19T04:57:21.384 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 2, 2024-09-19T04:57:21.384 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 2 2024-09-19T04:57:21.384 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.384 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-09-19T04:57:21.384 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 3, 2024-09-19T04:57:21.384 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 466, 2024-09-19T04:57:21.384 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 259, 2024-09-19T04:57:21.384 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 263, 2024-09-19T04:57:21.384 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 3, 2024-09-19T04:57:21.385 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-09-19T04:57:21.385 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-09-19T04:57:21.385 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:57:21.385 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 1, 2024-09-19T04:57:21.385 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 1, 2024-09-19T04:57:21.385 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-09-19T04:57:21.385 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-09-19T04:57:21.385 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-09-19T04:57:21.385 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:57:21.385 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:57:21.385 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-09-19T04:57:21.385 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-09-19T04:57:21.385 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:57:21.386 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:57:21.386 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:57:21.386 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 1, 2024-09-19T04:57:21.386 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-09-19T04:57:21.386 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:57:21.386 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-09-19T04:57:21.386 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-09-19T04:57:21.386 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.386 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-09-19T04:57:21.386 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-09-19T04:57:21.386 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 17, 2024-09-19T04:57:21.386 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-09-19T04:57:21.386 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-09-19T04:57:21.387 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 17, 2024-09-19T04:57:21.387 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-09-19T04:57:21.387 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-09-19T04:57:21.387 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 2, 2024-09-19T04:57:21.387 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-09-19T04:57:21.387 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-09-19T04:57:21.387 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 2, 2024-09-19T04:57:21.387 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-09-19T04:57:21.387 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-09-19T04:57:21.387 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-09-19T04:57:21.387 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-09-19T04:57:21.387 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 4225960, 2024-09-19T04:57:21.387 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-09-19T04:57:21.387 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-09-19T04:57:21.387 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 15, 2024-09-19T04:57:21.388 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.162925114, 2024-09-19T04:57:21.388 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.010861674 2024-09-19T04:57:21.388 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.388 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-09-19T04:57:21.388 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.388 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-09-19T04:57:21.388 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 28, 2024-09-19T04:57:21.388 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 89, 2024-09-19T04:57:21.388 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 61, 2024-09-19T04:57:21.388 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 27, 2024-09-19T04:57:21.388 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 246, 2024-09-19T04:57:21.388 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 219, 2024-09-19T04:57:21.388 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 23, 2024-09-19T04:57:21.388 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 141, 2024-09-19T04:57:21.388 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 118, 2024-09-19T04:57:21.389 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2024-09-19T04:57:21.389 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 5, 2024-09-19T04:57:21.389 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 5, 2024-09-19T04:57:21.389 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 52812, 2024-09-19T04:57:21.389 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 223520 2024-09-19T04:57:21.389 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.389 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-09-19T04:57:21.389 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 11, 2024-09-19T04:57:21.389 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 6, 2024-09-19T04:57:21.389 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 5, 2024-09-19T04:57:21.389 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 16, 2024-09-19T04:57:21.389 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 15, 2024-09-19T04:57:21.389 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-09-19T04:57:21.390 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.390 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.390 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.390 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.390 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-09-19T04:57:21.390 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-09-19T04:57:21.390 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001034057, 2024-09-19T04:57:21.390 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001034057 2024-09-19T04:57:21.390 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.390 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-09-19T04:57:21.390 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.390 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.391 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.391 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.391 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-09-19T04:57:21.391 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.391 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.391 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.391 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.391 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-09-19T04:57:21.391 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-09-19T04:57:21.391 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000128825, 2024-09-19T04:57:21.391 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000128825 2024-09-19T04:57:21.391 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.391 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-09-19T04:57:21.391 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.392 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.392 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.392 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.392 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-09-19T04:57:21.392 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.392 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.392 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.392 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.392 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-09-19T04:57:21.392 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.392 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.392 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.392 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.393 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-09-19T04:57:21.393 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.393 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.393 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.393 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.393 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-09-19T04:57:21.393 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.393 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.393 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.393 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.393 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-09-19T04:57:21.393 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.393 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.393 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.394 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.394 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-09-19T04:57:21.394 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.394 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.394 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.394 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.395 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-09-19T04:57:21.395 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.395 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.395 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.395 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.395 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-09-19T04:57:21.395 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.395 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.395 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.395 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.395 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-09-19T04:57:21.395 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.395 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.396 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.396 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.396 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-09-19T04:57:21.396 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-09-19T04:57:21.396 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000422203, 2024-09-19T04:57:21.396 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000422203 2024-09-19T04:57:21.396 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.396 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-09-19T04:57:21.396 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.396 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.396 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.396 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.396 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-09-19T04:57:21.397 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.397 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.397 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.397 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.397 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-09-19T04:57:21.397 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.397 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.397 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.397 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.397 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-09-19T04:57:21.397 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.397 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.398 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.398 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.398 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-09-19T04:57:21.398 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.398 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.398 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.398 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.398 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-09-19T04:57:21.398 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.398 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.398 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.398 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.398 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-09-19T04:57:21.398 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.399 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.399 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.399 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.399 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-09-19T04:57:21.399 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.399 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.399 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.399 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.399 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-09-19T04:57:21.399 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.399 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.399 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.399 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.399 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-09-19T04:57:21.399 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.400 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.400 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.400 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.400 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-09-19T04:57:21.400 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.400 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.400 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.400 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.400 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-09-19T04:57:21.400 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.400 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.400 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.400 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.400 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-09-19T04:57:21.401 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-09-19T04:57:21.401 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.002206325, 2024-09-19T04:57:21.401 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002206325 2024-09-19T04:57:21.401 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.401 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:57:21.401 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:57:21.401 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-09-19T04:57:21.401 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.401 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.401 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.401 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:21.401 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.401 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-09-19T04:57:21.402 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 1, 2024-09-19T04:57:21.402 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 3, 2024-09-19T04:57:21.402 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 2, 2024-09-19T04:57:21.402 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 1, 2024-09-19T04:57:21.402 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-09-19T04:57:21.402 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 2, 2024-09-19T04:57:21.402 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 2, 2024-09-19T04:57:21.402 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 2002, 2024-09-19T04:57:21.402 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-09-19T04:57:21.402 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.402 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-09-19T04:57:21.402 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:57:21.403 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-09-19T04:57:21.403 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:57:21.403 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:57:21.403 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:57:21.403 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:57:21.403 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:57:21.403 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:57:21.403 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:57:21.403 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:57:21.403 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:57:21.403 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:57:21.403 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:57:21.404 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:57:21.404 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:57:21.404 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-09-19T04:57:21.404 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:57:21.404 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-09-19T04:57:21.404 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:57:21.404 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:57:21.404 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:57:21.404 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:57:21.404 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:57:21.404 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:57:21.404 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:57:21.404 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-09-19T04:57:21.405 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:57:21.405 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:57:21.405 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:57:21.405 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-09-19T04:57:21.405 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-09-19T04:57:21.405 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-09-19T04:57:21.405 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:57:21.405 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:57:21.405 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:57:21.405 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:57:21.405 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 321315, 2024-09-19T04:57:21.405 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 122, 2024-09-19T04:57:21.405 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:57:21.405 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-09-19T04:57:21.406 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-09-19T04:57:21.406 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-09-19T04:57:21.406 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:57:21.406 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-09-19T04:57:21.406 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:57:21.406 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-09-19T04:57:21.406 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8128, 2024-09-19T04:57:21.406 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 179, 2024-09-19T04:57:21.406 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:57:21.406 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:57:21.406 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-09-19T04:57:21.406 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-09-19T04:57:21.406 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 136741, 2024-09-19T04:57:21.407 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 469, 2024-09-19T04:57:21.407 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-09-19T04:57:21.407 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-09-19T04:57:21.407 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-09-19T04:57:21.407 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-09-19T04:57:21.407 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.407 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-09-19T04:57:21.407 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-09-19T04:57:21.407 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-09-19T04:57:21.407 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 77, 2024-09-19T04:57:21.407 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 44150, 2024-09-19T04:57:21.407 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 32, 2024-09-19T04:57:21.407 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 77, 2024-09-19T04:57:21.408 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-09-19T04:57:21.408 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 77, 2024-09-19T04:57:21.408 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.560991058, 2024-09-19T04:57:21.408 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.007285598 2024-09-19T04:57:21.408 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.408 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-09-19T04:57:21.408 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-09-19T04:57:21.408 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 77, 2024-09-19T04:57:21.408 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 106 2024-09-19T04:57:21.408 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.408 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 77, 2024-09-19T04:57:21.408 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 7, 2024-09-19T04:57:21.408 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 70, 2024-09-19T04:57:21.408 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-09-19T04:57:21.409 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-09-19T04:57:21.409 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 0, 2024-09-19T04:57:21.409 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 10, 2024-09-19T04:57:21.409 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 1, 2024-09-19T04:57:21.409 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 16, 2024-09-19T04:57:21.409 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 11, 2024-09-19T04:57:21.409 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-09-19T04:57:21.409 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-09-19T04:57:21.409 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-09-19T04:57:21.409 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-09-19T04:57:21.409 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 13, 2024-09-19T04:57:21.409 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-09-19T04:57:21.409 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-09-19T04:57:21.409 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-09-19T04:57:21.410 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 6, 2024-09-19T04:57:21.410 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 20, 2024-09-19T04:57:21.410 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:57:21.410 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-09-19T04:57:21.410 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:57:21.410 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-09-19T04:57:21.410 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-09-19T04:57:21.410 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-09-19T04:57:21.410 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:57:21.410 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-09-19T04:57:21.410 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:57:21.410 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-09-19T04:57:21.410 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-09-19T04:57:21.411 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-09-19T04:57:21.411 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-09-19T04:57:21.411 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-09-19T04:57:21.411 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-09-19T04:57:21.411 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-09-19T04:57:21.411 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-09-19T04:57:21.411 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-09-19T04:57:21.411 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-09-19T04:57:21.411 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-09-19T04:57:21.411 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-09-19T04:57:21.411 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-09-19T04:57:21.411 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-09-19T04:57:21.411 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-09-19T04:57:21.412 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-09-19T04:57:21.412 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-09-19T04:57:21.412 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 97, 2024-09-19T04:57:21.412 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-09-19T04:57:21.412 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 13, 2024-09-19T04:57:21.412 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 11, 2024-09-19T04:57:21.412 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 11, 2024-09-19T04:57:21.412 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-09-19T04:57:21.412 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-09-19T04:57:21.412 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 24, 2024-09-19T04:57:21.412 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 0, 2024-09-19T04:57:21.412 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 5 2024-09-19T04:57:21.412 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.413 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-09-19T04:57:21.413 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-09-19T04:57:21.413 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:57:21.413 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-09-19T04:57:21.413 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-09-19T04:57:21.413 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.413 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-09-19T04:57:21.413 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-09-19T04:57:21.413 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.413 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-09-19T04:57:21.413 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-09-19T04:57:21.413 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 1, 2024-09-19T04:57:21.413 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-09-19T04:57:21.414 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 1, 2024-09-19T04:57:21.414 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 1, 2024-09-19T04:57:21.414 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 1, 2024-09-19T04:57:21.414 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-09-19T04:57:21.414 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.414 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:57:21.414 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:21.414 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-19T04:57:21.414 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:21.414 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 4745, 2024-09-19T04:57:21.414 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 5575373, 2024-09-19T04:57:21.414 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:21.414 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 4745, 2024-09-19T04:57:21.415 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-19T04:57:21.415 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-19T04:57:21.415 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 4745, 2024-09-19T04:57:21.415 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 5575373, 2024-09-19T04:57:21.415 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:21.415 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.415 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.415 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.415 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:21.415 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.415 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-09-19T04:57:21.415 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:21.415 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-19T04:57:21.415 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:21.416 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-19T04:57:21.416 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-19T04:57:21.416 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:21.416 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:21.416 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 77, 2024-09-19T04:57:21.416 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 44283, 2024-09-19T04:57:21.416 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 77, 2024-09-19T04:57:21.416 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 44283, 2024-09-19T04:57:21.416 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:21.416 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.416 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.416 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.416 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:21.417 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.417 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-09-19T04:57:21.417 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:21.417 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-09-19T04:57:21.417 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:21.417 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-19T04:57:21.417 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-19T04:57:21.417 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:21.417 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:21.417 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 77, 2024-09-19T04:57:21.417 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 77, 2024-09-19T04:57:21.417 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 77, 2024-09-19T04:57:21.417 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 77, 2024-09-19T04:57:21.418 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:21.418 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.418 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.418 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.418 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:21.418 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.418 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:57:21.418 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:21.418 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-19T04:57:21.418 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:21.418 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 1, 2024-09-19T04:57:21.418 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 133, 2024-09-19T04:57:21.419 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:21.419 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 1, 2024-09-19T04:57:21.419 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-19T04:57:21.419 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-19T04:57:21.419 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1, 2024-09-19T04:57:21.419 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 133, 2024-09-19T04:57:21.419 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:21.419 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.419 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.419 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.419 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:21.419 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:21.419 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x558c689e2960": { 2024-09-19T04:57:21.420 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:21.420 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-19T04:57:21.420 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:21.420 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 17, 2024-09-19T04:57:21.420 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 31656, 2024-09-19T04:57:21.420 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:21.420 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 17, 2024-09-19T04:57:21.420 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-19T04:57:21.420 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-19T04:57:21.420 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 15, 2024-09-19T04:57:21.420 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 31656, 2024-09-19T04:57:21.420 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:21.420 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:21.420 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:21.421 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:21.421 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:21.421 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:21.421 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-19T04:57:21.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:21 smithi183 ceph-mon[35333]: pgmap v332: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 210 B/s wr, 10 op/s 2024-09-19T04:57:21.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:21 smithi123 ceph-mon[30511]: pgmap v332: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 210 B/s wr, 10 op/s 2024-09-19T04:57:21.738 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:21 smithi193 ceph-mon[35592]: pgmap v332: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 210 B/s wr, 10 op/s 2024-09-19T04:57:22.202 INFO:tasks.check_counter:Daemon mds.f mds.exported=1 2024-09-19T04:57:22.202 INFO:tasks.check_counter:Daemon mds.f mds.imported=1 2024-09-19T04:57:22.202 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=3 2024-09-19T04:57:22.202 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=466 2024-09-19T04:57:22.202 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-19T04:57:22.202 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:57:22.202 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:57:22.202 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-19T04:57:22.203 DEBUG:tasks.check_counter:Getting stats from i 2024-09-19T04:57:22.203 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-19T04:57:22.446 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:57:22.948 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-19T04:57:22.948 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:57:22.948 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 4519, 2024-09-19T04:57:22.948 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 2737, 2024-09-19T04:57:22.948 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 4296298, 2024-09-19T04:57:22.948 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 1054882, 2024-09-19T04:57:22.948 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 14, 2024-09-19T04:57:22.949 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 5, 2024-09-19T04:57:22.949 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.261784080, 2024-09-19T04:57:22.949 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.110872625, 2024-09-19T04:57:22.949 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.181129043, 2024-09-19T04:57:22.949 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.006561389, 2024-09-19T04:57:22.949 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:22.949 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2735, 2024-09-19T04:57:22.949 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.079901581, 2024-09-19T04:57:22.949 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000029214 2024-09-19T04:57:22.949 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.949 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:22.949 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6826, 2024-09-19T04:57:22.949 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001160608, 2024-09-19T04:57:22.950 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000170 2024-09-19T04:57:22.950 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.950 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:57:22.950 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:57:22.950 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.950 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:57:22.950 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 384, 2024-09-19T04:57:22.950 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 353, 2024-09-19T04:57:22.950 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1787674, 2024-09-19T04:57:22.950 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 287519, 2024-09-19T04:57:22.950 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 18, 2024-09-19T04:57:22.951 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 6, 2024-09-19T04:57:22.951 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.053118410, 2024-09-19T04:57:22.951 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.016883998, 2024-09-19T04:57:22.951 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.049730322, 2024-09-19T04:57:22.951 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.001481528, 2024-09-19T04:57:22.951 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:22.951 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 352, 2024-09-19T04:57:22.951 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.027288107, 2024-09-19T04:57:22.951 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000077523 2024-09-19T04:57:22.951 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.951 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:22.951 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 40, 2024-09-19T04:57:22.951 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000006081, 2024-09-19T04:57:22.952 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000152 2024-09-19T04:57:22.952 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.952 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1749392, 2024-09-19T04:57:22.952 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 144416 2024-09-19T04:57:22.952 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.952 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:57:22.952 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 83, 2024-09-19T04:57:22.952 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 277, 2024-09-19T04:57:22.952 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 34215, 2024-09-19T04:57:22.952 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 812903, 2024-09-19T04:57:22.952 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 37, 2024-09-19T04:57:22.952 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 10, 2024-09-19T04:57:22.953 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.028533755, 2024-09-19T04:57:22.953 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.009011945, 2024-09-19T04:57:22.953 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.016409554, 2024-09-19T04:57:22.953 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000875582, 2024-09-19T04:57:22.953 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:22.953 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 275, 2024-09-19T04:57:22.953 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.031012086, 2024-09-19T04:57:22.953 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000112771 2024-09-19T04:57:22.953 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.953 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:22.953 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 101, 2024-09-19T04:57:22.953 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000015765, 2024-09-19T04:57:22.953 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000156 2024-09-19T04:57:22.953 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.954 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 7616, 2024-09-19T04:57:22.954 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 464688 2024-09-19T04:57:22.954 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.954 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-09-19T04:57:22.954 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-09-19T04:57:22.954 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-09-19T04:57:22.954 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.954 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-09-19T04:57:22.954 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-19T04:57:22.955 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-19T04:57:22.955 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 127, 2024-09-19T04:57:22.955 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.020982368, 2024-09-19T04:57:22.955 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000165215 2024-09-19T04:57:22.955 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:22.955 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.955 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-09-19T04:57:22.955 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-19T04:57:22.955 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-19T04:57:22.955 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 31, 2024-09-19T04:57:22.955 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001119990, 2024-09-19T04:57:22.955 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000036128 2024-09-19T04:57:22.956 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:22.956 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.956 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-09-19T04:57:22.956 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 159, 2024-09-19T04:57:22.956 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 33, 2024-09-19T04:57:22.956 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-09-19T04:57:22.956 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 33, 2024-09-19T04:57:22.956 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.020368062, 2024-09-19T04:57:22.956 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000617214 2024-09-19T04:57:22.956 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.956 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-09-19T04:57:22.956 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 126, 2024-09-19T04:57:22.957 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-09-19T04:57:22.957 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-09-19T04:57:22.957 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 11, 2024-09-19T04:57:22.957 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-09-19T04:57:22.957 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-09-19T04:57:22.957 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 14, 2024-09-19T04:57:22.957 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 0, 2024-09-19T04:57:22.957 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 1, 2024-09-19T04:57:22.957 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 13, 2024-09-19T04:57:22.957 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 13, 2024-09-19T04:57:22.957 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 150, 2024-09-19T04:57:22.957 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 2, 2024-09-19T04:57:22.957 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 2, 2024-09-19T04:57:22.957 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 3, 2024-09-19T04:57:22.958 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 461, 2024-09-19T04:57:22.958 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 263, 2024-09-19T04:57:22.958 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 124, 2024-09-19T04:57:22.958 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 56, 2024-09-19T04:57:22.958 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:57:22.958 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-09-19T04:57:22.958 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 2, 2024-09-19T04:57:22.958 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-09-19T04:57:22.958 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-09-19T04:57:22.958 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 1, 2024-09-19T04:57:22.958 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 26, 2024-09-19T04:57:22.958 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 1, 2024-09-19T04:57:22.958 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 14, 2024-09-19T04:57:22.958 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-09-19T04:57:22.959 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 65, 2024-09-19T04:57:22.959 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-09-19T04:57:22.959 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 4, 2024-09-19T04:57:22.959 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 234, 2024-09-19T04:57:22.959 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 6, 2024-09-19T04:57:22.959 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 65, 2024-09-19T04:57:22.959 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-09-19T04:57:22.959 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:57:22.959 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:57:22.959 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:57:22.959 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:57:22.959 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:57:22.959 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 65, 2024-09-19T04:57:22.960 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:57:22.960 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 3, 2024-09-19T04:57:22.960 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 15, 2024-09-19T04:57:22.960 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:57:22.960 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:57:22.960 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:57:22.960 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 17, 2024-09-19T04:57:22.960 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 13, 2024-09-19T04:57:22.960 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 3, 2024-09-19T04:57:22.960 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 27 2024-09-19T04:57:22.960 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.960 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-09-19T04:57:22.960 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 6, 2024-09-19T04:57:22.961 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 472, 2024-09-19T04:57:22.961 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 264, 2024-09-19T04:57:22.961 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 264, 2024-09-19T04:57:22.961 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 6, 2024-09-19T04:57:22.961 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-09-19T04:57:22.961 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-09-19T04:57:22.961 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:57:22.961 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 13, 2024-09-19T04:57:22.961 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 13, 2024-09-19T04:57:22.961 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-09-19T04:57:22.961 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-09-19T04:57:22.961 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-09-19T04:57:22.962 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:57:22.962 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:57:22.962 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-09-19T04:57:22.962 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-09-19T04:57:22.962 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:57:22.962 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:57:22.962 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:57:22.962 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 1, 2024-09-19T04:57:22.962 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-09-19T04:57:22.962 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:57:22.962 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-09-19T04:57:22.962 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-09-19T04:57:22.962 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.963 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-09-19T04:57:22.963 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-09-19T04:57:22.963 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 55, 2024-09-19T04:57:22.963 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-09-19T04:57:22.963 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-09-19T04:57:22.963 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 55, 2024-09-19T04:57:22.963 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-09-19T04:57:22.963 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-09-19T04:57:22.963 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 2, 2024-09-19T04:57:22.963 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-09-19T04:57:22.963 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-09-19T04:57:22.963 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 2, 2024-09-19T04:57:22.963 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-09-19T04:57:22.963 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-09-19T04:57:22.964 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-09-19T04:57:22.964 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-09-19T04:57:22.964 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 4295210, 2024-09-19T04:57:22.964 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-09-19T04:57:22.964 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-09-19T04:57:22.964 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 17, 2024-09-19T04:57:22.964 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.035244694, 2024-09-19T04:57:22.964 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002073217 2024-09-19T04:57:22.964 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.964 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-09-19T04:57:22.964 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.964 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-09-19T04:57:22.964 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 19, 2024-09-19T04:57:22.965 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 113, 2024-09-19T04:57:22.965 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 94, 2024-09-19T04:57:22.965 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 17, 2024-09-19T04:57:22.965 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 244, 2024-09-19T04:57:22.965 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 227, 2024-09-19T04:57:22.965 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 14, 2024-09-19T04:57:22.965 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 178, 2024-09-19T04:57:22.965 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 164, 2024-09-19T04:57:22.965 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 2, 2024-09-19T04:57:22.965 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 30, 2024-09-19T04:57:22.965 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 28, 2024-09-19T04:57:22.965 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 52044, 2024-09-19T04:57:22.965 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 223520 2024-09-19T04:57:22.966 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.966 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-09-19T04:57:22.966 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 174, 2024-09-19T04:57:22.966 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 6, 2024-09-19T04:57:22.966 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 159, 2024-09-19T04:57:22.966 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 26, 2024-09-19T04:57:22.966 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 26, 2024-09-19T04:57:22.966 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-09-19T04:57:22.966 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.966 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.966 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.966 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.966 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-09-19T04:57:22.967 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3, 2024-09-19T04:57:22.967 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000536838, 2024-09-19T04:57:22.967 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000178946 2024-09-19T04:57:22.967 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.967 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-09-19T04:57:22.967 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.967 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.967 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.967 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.967 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-09-19T04:57:22.967 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.967 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.967 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.968 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.968 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-09-19T04:57:22.968 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 15, 2024-09-19T04:57:22.968 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.003849843, 2024-09-19T04:57:22.968 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000256656 2024-09-19T04:57:22.968 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.968 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-09-19T04:57:22.968 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.968 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.968 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.968 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.968 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-09-19T04:57:22.968 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-09-19T04:57:22.968 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000132715, 2024-09-19T04:57:22.969 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000132715 2024-09-19T04:57:22.969 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.969 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-09-19T04:57:22.969 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.969 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.969 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.969 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.969 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-09-19T04:57:22.969 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.969 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.969 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.969 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.970 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-09-19T04:57:22.970 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.970 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.970 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.970 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.970 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-09-19T04:57:22.970 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.970 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.970 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.970 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.971 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-09-19T04:57:22.971 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.971 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.971 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.971 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.971 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-09-19T04:57:22.971 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.971 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.971 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.971 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.972 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-09-19T04:57:22.972 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.972 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.972 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.972 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.972 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-09-19T04:57:22.972 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.972 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.972 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.972 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.972 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-09-19T04:57:22.973 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-09-19T04:57:22.973 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000793290, 2024-09-19T04:57:22.973 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000793290 2024-09-19T04:57:22.973 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.973 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-09-19T04:57:22.973 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.973 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.973 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.973 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.973 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-09-19T04:57:22.973 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.973 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.976 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.976 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.976 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-09-19T04:57:22.976 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.976 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.976 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.976 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.976 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-09-19T04:57:22.976 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.976 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.976 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.976 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.977 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-09-19T04:57:22.977 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.977 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.977 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.977 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.977 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-09-19T04:57:22.977 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.977 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.977 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.977 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.977 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-09-19T04:57:22.977 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.978 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.978 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.978 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.978 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-09-19T04:57:22.978 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.978 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.978 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.978 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.978 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-09-19T04:57:22.978 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.978 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.978 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.978 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.979 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-09-19T04:57:22.979 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.979 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.979 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.979 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.979 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-09-19T04:57:22.979 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.979 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.979 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.979 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.979 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-09-19T04:57:22.979 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.979 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.980 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.980 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.980 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-09-19T04:57:22.980 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 13, 2024-09-19T04:57:22.980 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.015055376, 2024-09-19T04:57:22.980 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001158105 2024-09-19T04:57:22.980 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.980 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:57:22.980 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:57:22.980 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-09-19T04:57:22.980 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.980 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.981 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.981 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:22.981 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.981 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-09-19T04:57:22.981 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2024-09-19T04:57:22.981 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 4, 2024-09-19T04:57:22.981 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 2, 2024-09-19T04:57:22.981 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2024-09-19T04:57:22.981 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-09-19T04:57:22.981 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 27, 2024-09-19T04:57:22.981 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 13, 2024-09-19T04:57:22.981 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 621, 2024-09-19T04:57:22.981 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-09-19T04:57:22.981 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.982 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-09-19T04:57:22.982 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:57:22.982 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-09-19T04:57:22.982 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:57:22.982 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:57:22.982 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:57:22.982 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:57:22.982 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:57:22.982 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:57:22.982 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:57:22.982 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:57:22.982 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:57:22.982 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:57:22.982 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:57:22.983 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:57:22.983 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:57:22.983 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-09-19T04:57:22.983 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:57:22.983 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-09-19T04:57:22.983 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:57:22.983 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:57:22.983 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:57:22.983 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:57:22.983 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:57:22.983 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:57:22.983 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:57:22.984 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-09-19T04:57:22.984 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:57:22.984 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:57:22.984 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:57:22.984 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-09-19T04:57:22.984 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-09-19T04:57:22.984 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-09-19T04:57:22.984 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:57:22.984 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:57:22.984 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:57:22.984 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:57:22.984 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 348066, 2024-09-19T04:57:22.984 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 139, 2024-09-19T04:57:22.985 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:57:22.985 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-09-19T04:57:22.985 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-09-19T04:57:22.985 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-09-19T04:57:22.985 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:57:22.985 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-09-19T04:57:22.985 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:57:22.985 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-09-19T04:57:22.986 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8128, 2024-09-19T04:57:22.986 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 179, 2024-09-19T04:57:22.986 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:57:22.986 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:57:22.986 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-09-19T04:57:22.986 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-09-19T04:57:22.986 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 81896, 2024-09-19T04:57:22.986 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 295, 2024-09-19T04:57:22.986 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-09-19T04:57:22.986 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-09-19T04:57:22.987 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-09-19T04:57:22.987 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-09-19T04:57:22.987 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.987 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-09-19T04:57:22.987 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-09-19T04:57:22.987 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-09-19T04:57:22.987 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 151, 2024-09-19T04:57:22.987 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 116234, 2024-09-19T04:57:22.987 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 30, 2024-09-19T04:57:22.987 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 151, 2024-09-19T04:57:22.988 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-09-19T04:57:22.988 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 151, 2024-09-19T04:57:22.988 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.371994087, 2024-09-19T04:57:22.988 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002463537 2024-09-19T04:57:22.988 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.988 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-09-19T04:57:22.988 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-09-19T04:57:22.988 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 151, 2024-09-19T04:57:22.989 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 181 2024-09-19T04:57:22.989 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.989 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 151, 2024-09-19T04:57:22.989 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 66, 2024-09-19T04:57:22.989 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 85, 2024-09-19T04:57:22.989 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-09-19T04:57:22.989 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-09-19T04:57:22.989 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 0, 2024-09-19T04:57:22.989 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 10, 2024-09-19T04:57:22.989 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 1, 2024-09-19T04:57:22.990 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 18, 2024-09-19T04:57:22.990 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 12, 2024-09-19T04:57:22.990 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-09-19T04:57:22.990 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-09-19T04:57:22.990 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-09-19T04:57:22.990 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-09-19T04:57:22.990 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 25, 2024-09-19T04:57:22.990 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-09-19T04:57:22.990 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-09-19T04:57:22.990 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-09-19T04:57:22.990 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 65, 2024-09-19T04:57:22.990 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 20, 2024-09-19T04:57:22.991 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:57:22.991 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-09-19T04:57:22.991 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:57:22.991 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-09-19T04:57:22.991 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-09-19T04:57:22.991 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-09-19T04:57:22.991 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:57:22.991 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-09-19T04:57:22.991 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:57:22.991 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-09-19T04:57:22.991 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-09-19T04:57:22.991 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-09-19T04:57:22.991 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-09-19T04:57:22.992 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-09-19T04:57:22.992 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-09-19T04:57:22.992 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-09-19T04:57:22.992 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-09-19T04:57:22.992 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-09-19T04:57:22.992 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-09-19T04:57:22.992 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-09-19T04:57:22.992 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-09-19T04:57:22.992 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-09-19T04:57:22.992 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-09-19T04:57:22.992 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-09-19T04:57:22.992 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-09-19T04:57:22.992 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-09-19T04:57:22.993 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 97, 2024-09-19T04:57:22.993 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-09-19T04:57:22.993 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 13, 2024-09-19T04:57:22.993 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 11, 2024-09-19T04:57:22.993 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 11, 2024-09-19T04:57:22.993 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-09-19T04:57:22.993 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-09-19T04:57:22.993 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 25, 2024-09-19T04:57:22.993 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 0, 2024-09-19T04:57:22.993 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 5 2024-09-19T04:57:22.993 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.993 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-09-19T04:57:22.994 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-09-19T04:57:22.994 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:57:22.994 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-09-19T04:57:22.994 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-09-19T04:57:22.994 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.994 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-09-19T04:57:22.994 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-09-19T04:57:22.994 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.994 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-09-19T04:57:22.994 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-09-19T04:57:22.994 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T04:57:22.994 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-09-19T04:57:22.994 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 6, 2024-09-19T04:57:22.995 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 13, 2024-09-19T04:57:22.995 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 13, 2024-09-19T04:57:22.995 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-09-19T04:57:22.995 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.995 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:57:22.995 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:22.995 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-19T04:57:22.995 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:22.995 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 4982, 2024-09-19T04:57:22.995 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 5653490, 2024-09-19T04:57:22.995 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:22.995 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 4982, 2024-09-19T04:57:22.995 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-19T04:57:22.996 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-19T04:57:22.996 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 4982, 2024-09-19T04:57:22.996 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 5653490, 2024-09-19T04:57:22.996 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:22.996 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.996 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.996 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.996 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:22.996 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.996 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-09-19T04:57:22.996 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:22.996 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-19T04:57:22.996 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:22.997 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-19T04:57:22.997 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-19T04:57:22.997 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:22.997 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:22.997 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 151, 2024-09-19T04:57:22.997 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 117963, 2024-09-19T04:57:22.997 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 151, 2024-09-19T04:57:22.997 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 117963, 2024-09-19T04:57:22.997 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:22.997 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.997 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.998 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.998 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:22.998 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.998 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-09-19T04:57:22.998 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:22.998 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-09-19T04:57:22.998 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:22.998 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-19T04:57:22.998 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-19T04:57:22.998 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:22.998 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:22.998 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 151, 2024-09-19T04:57:22.998 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 151, 2024-09-19T04:57:22.998 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 151, 2024-09-19T04:57:22.999 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 151, 2024-09-19T04:57:22.999 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:22.999 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:22.999 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:22.999 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:22.999 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:22.999 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:22.999 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:57:22.999 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:22.999 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-19T04:57:22.999 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:22.999 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 13, 2024-09-19T04:57:22.999 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 1729, 2024-09-19T04:57:22.999 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:23.000 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 13, 2024-09-19T04:57:23.000 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-19T04:57:23.000 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-19T04:57:23.000 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1, 2024-09-19T04:57:23.000 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1729, 2024-09-19T04:57:23.000 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:23.000 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:23.000 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:23.000 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:23.000 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:23.000 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:23.000 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x5618ece88960": { 2024-09-19T04:57:23.000 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:23.000 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-19T04:57:23.001 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:23.001 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 55, 2024-09-19T04:57:23.001 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 100906, 2024-09-19T04:57:23.001 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:23.001 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 55, 2024-09-19T04:57:23.001 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-19T04:57:23.001 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-19T04:57:23.001 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 17, 2024-09-19T04:57:23.001 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 100906, 2024-09-19T04:57:23.001 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:23.001 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:23.001 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:23.002 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:23.002 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:23.002 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:23.002 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-19T04:57:23.587 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:23 smithi193 ceph-mon[35592]: pgmap v333: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:57:23.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:23 smithi183 ceph-mon[35333]: pgmap v333: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:57:23.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:23 smithi123 ceph-mon[30511]: pgmap v333: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T04:57:23.764 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-09-19T04:57:23.764 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-09-19T04:57:23.765 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=6 2024-09-19T04:57:23.765 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=472 2024-09-19T04:57:23.765 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-19T04:57:23.765 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:57:23.765 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:57:23.765 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-09-19T04:57:23.765 DEBUG:tasks.check_counter:Getting stats from l 2024-09-19T04:57:23.765 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-19T04:57:24.010 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:57:24.557 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-19T04:57:24.557 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:57:24.557 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 750, 2024-09-19T04:57:24.557 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 752, 2024-09-19T04:57:24.557 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 49360718, 2024-09-19T04:57:24.557 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 238564, 2024-09-19T04:57:24.558 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 8, 2024-09-19T04:57:24.558 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-09-19T04:57:24.558 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.091080125, 2024-09-19T04:57:24.558 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.020682351, 2024-09-19T04:57:24.558 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.042489028, 2024-09-19T04:57:24.558 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.020538231, 2024-09-19T04:57:24.558 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:24.558 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 750, 2024-09-19T04:57:24.558 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.034233660, 2024-09-19T04:57:24.558 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000045644 2024-09-19T04:57:24.558 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.558 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:24.558 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.558 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.559 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.559 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.559 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:57:24.559 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:57:24.559 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.559 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:57:24.559 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 693, 2024-09-19T04:57:24.559 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 858, 2024-09-19T04:57:24.559 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 54702246, 2024-09-19T04:57:24.559 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 670215, 2024-09-19T04:57:24.559 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 8, 2024-09-19T04:57:24.559 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 6, 2024-09-19T04:57:24.559 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.097385155, 2024-09-19T04:57:24.559 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.024617555, 2024-09-19T04:57:24.560 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.042830575, 2024-09-19T04:57:24.560 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.023036006, 2024-09-19T04:57:24.560 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:24.560 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 856, 2024-09-19T04:57:24.560 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.042497642, 2024-09-19T04:57:24.560 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000049646 2024-09-19T04:57:24.560 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.560 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:24.560 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.560 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.560 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.560 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.560 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-19T04:57:24.561 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 451744 2024-09-19T04:57:24.561 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.561 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:57:24.561 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1416, 2024-09-19T04:57:24.561 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1389, 2024-09-19T04:57:24.561 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 69189914, 2024-09-19T04:57:24.561 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 504242, 2024-09-19T04:57:24.561 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 5, 2024-09-19T04:57:24.561 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-09-19T04:57:24.561 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.176590247, 2024-09-19T04:57:24.561 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.049153480, 2024-09-19T04:57:24.561 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.100305150, 2024-09-19T04:57:24.561 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.033477118, 2024-09-19T04:57:24.561 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:57:24.562 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1388, 2024-09-19T04:57:24.562 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.062423897, 2024-09-19T04:57:24.562 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000044973 2024-09-19T04:57:24.562 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.562 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:57:24.562 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.562 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.562 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.562 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.562 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1720944, 2024-09-19T04:57:24.562 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 138224 2024-09-19T04:57:24.563 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.563 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-09-19T04:57:24.563 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-09-19T04:57:24.563 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-09-19T04:57:24.563 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.563 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-09-19T04:57:24.563 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-19T04:57:24.563 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-19T04:57:24.563 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4273, 2024-09-19T04:57:24.563 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.112904626, 2024-09-19T04:57:24.563 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000026422 2024-09-19T04:57:24.563 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:24.564 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.564 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-09-19T04:57:24.564 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-19T04:57:24.564 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-19T04:57:24.564 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.564 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.564 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.564 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:24.564 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.564 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-09-19T04:57:24.565 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 0, 2024-09-19T04:57:24.565 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 0, 2024-09-19T04:57:24.565 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-09-19T04:57:24.565 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.565 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.565 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.565 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.565 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-09-19T04:57:24.565 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 0, 2024-09-19T04:57:24.565 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-09-19T04:57:24.565 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-09-19T04:57:24.565 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 0, 2024-09-19T04:57:24.565 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-09-19T04:57:24.566 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-09-19T04:57:24.566 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 17160, 2024-09-19T04:57:24.566 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 182, 2024-09-19T04:57:24.566 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 16978, 2024-09-19T04:57:24.566 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-09-19T04:57:24.566 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 16899, 2024-09-19T04:57:24.566 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-09-19T04:57:24.566 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2024-09-19T04:57:24.566 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2024-09-19T04:57:24.566 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 1, 2024-09-19T04:57:24.566 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 0, 2024-09-19T04:57:24.566 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 0, 2024-09-19T04:57:24.566 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2024-09-19T04:57:24.566 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 0, 2024-09-19T04:57:24.567 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:57:24.567 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-09-19T04:57:24.567 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-09-19T04:57:24.567 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-09-19T04:57:24.567 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-09-19T04:57:24.567 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-09-19T04:57:24.567 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-09-19T04:57:24.567 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-09-19T04:57:24.567 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-09-19T04:57:24.567 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-09-19T04:57:24.567 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:57:24.567 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-09-19T04:57:24.567 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 0, 2024-09-19T04:57:24.568 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 0, 2024-09-19T04:57:24.568 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-09-19T04:57:24.568 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:57:24.568 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-09-19T04:57:24.568 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:57:24.568 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:57:24.568 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:57:24.568 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:57:24.568 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:57:24.568 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-09-19T04:57:24.568 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:57:24.568 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:57:24.568 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:57:24.569 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:57:24.569 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:57:24.569 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:57:24.569 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2024-09-19T04:57:24.569 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:57:24.569 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-09-19T04:57:24.569 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2024-09-19T04:57:24.569 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.569 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-09-19T04:57:24.569 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-09-19T04:57:24.569 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-09-19T04:57:24.569 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-09-19T04:57:24.569 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 0, 2024-09-19T04:57:24.569 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-09-19T04:57:24.570 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-09-19T04:57:24.570 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-09-19T04:57:24.570 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:57:24.570 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-09-19T04:57:24.570 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-09-19T04:57:24.570 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-09-19T04:57:24.570 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-09-19T04:57:24.570 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-09-19T04:57:24.570 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:57:24.570 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:57:24.570 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-09-19T04:57:24.570 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-09-19T04:57:24.570 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:57:24.570 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:57:24.571 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:57:24.571 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-09-19T04:57:24.571 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-09-19T04:57:24.571 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:57:24.571 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-09-19T04:57:24.571 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-09-19T04:57:24.571 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.571 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-09-19T04:57:24.571 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-09-19T04:57:24.571 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 0, 2024-09-19T04:57:24.571 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-09-19T04:57:24.571 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-09-19T04:57:24.571 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 61715, 2024-09-19T04:57:24.572 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-09-19T04:57:24.572 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-09-19T04:57:24.572 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 0, 2024-09-19T04:57:24.572 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-09-19T04:57:24.572 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-09-19T04:57:24.572 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 85, 2024-09-19T04:57:24.572 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 7, 2024-09-19T04:57:24.572 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-09-19T04:57:24.572 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-09-19T04:57:24.572 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-09-19T04:57:24.572 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 174987825, 2024-09-19T04:57:24.572 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 174983465, 2024-09-19T04:57:24.572 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-09-19T04:57:24.572 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.573 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.573 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.573 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.573 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 61715 2024-09-19T04:57:24.573 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.573 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-09-19T04:57:24.573 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 4876, 2024-09-19T04:57:24.573 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 12087, 2024-09-19T04:57:24.573 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 7211, 2024-09-19T04:57:24.573 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 225, 2024-09-19T04:57:24.573 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 288, 2024-09-19T04:57:24.573 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 63, 2024-09-19T04:57:24.573 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 17160, 2024-09-19T04:57:24.573 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 17199, 2024-09-19T04:57:24.574 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 39, 2024-09-19T04:57:24.574 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2024-09-19T04:57:24.574 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 0, 2024-09-19T04:57:24.574 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2024-09-19T04:57:24.574 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 81460, 2024-09-19T04:57:24.574 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 215328 2024-09-19T04:57:24.574 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.574 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-09-19T04:57:24.574 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 0, 2024-09-19T04:57:24.574 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-09-19T04:57:24.574 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 0, 2024-09-19T04:57:24.574 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 0, 2024-09-19T04:57:24.575 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-09-19T04:57:24.575 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-09-19T04:57:24.575 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.575 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.575 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.575 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.575 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-09-19T04:57:24.575 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.575 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.575 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.575 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.575 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-09-19T04:57:24.576 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.576 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.576 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.576 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.576 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-09-19T04:57:24.576 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.576 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.576 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.576 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.576 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-09-19T04:57:24.576 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.576 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.577 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.577 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.577 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-09-19T04:57:24.577 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.577 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.577 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.577 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.577 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-09-19T04:57:24.577 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.577 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.577 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.577 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.577 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-09-19T04:57:24.577 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.578 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.578 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.578 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.578 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-09-19T04:57:24.578 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.578 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.578 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.578 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.578 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-09-19T04:57:24.578 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.578 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.578 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.578 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.578 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-09-19T04:57:24.579 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.579 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.579 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.579 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.579 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-09-19T04:57:24.579 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.579 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.579 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.579 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.579 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-09-19T04:57:24.579 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.579 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.579 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.579 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.579 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-09-19T04:57:24.580 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.580 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.580 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.580 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.580 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-09-19T04:57:24.580 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.580 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.580 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.580 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.580 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-09-19T04:57:24.580 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.580 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.580 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.580 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.581 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-09-19T04:57:24.581 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.581 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.581 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.581 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.581 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-09-19T04:57:24.581 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.581 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.581 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.581 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.581 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-09-19T04:57:24.581 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.582 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.582 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.582 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.582 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-09-19T04:57:24.582 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.582 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.582 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.582 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.582 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-09-19T04:57:24.582 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.582 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.582 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.583 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.583 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-09-19T04:57:24.583 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.583 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.583 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.583 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.583 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-09-19T04:57:24.583 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.583 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.583 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.583 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.583 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-09-19T04:57:24.584 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.584 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.584 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.584 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.584 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-09-19T04:57:24.584 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.584 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.584 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.584 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.584 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-09-19T04:57:24.584 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.584 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.584 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.584 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.585 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-09-19T04:57:24.585 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.585 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.585 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.585 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.585 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-09-19T04:57:24.585 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.585 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.585 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.585 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.585 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-09-19T04:57:24.585 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.585 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.585 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.586 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.586 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:57:24.586 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:57:24.586 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-09-19T04:57:24.586 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.586 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.586 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.586 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:24.586 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.586 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-09-19T04:57:24.586 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 1, 2024-09-19T04:57:24.586 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 2, 2024-09-19T04:57:24.586 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 1, 2024-09-19T04:57:24.587 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 1, 2024-09-19T04:57:24.587 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-09-19T04:57:24.587 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 0, 2024-09-19T04:57:24.587 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2024-09-19T04:57:24.587 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 2001, 2024-09-19T04:57:24.587 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-09-19T04:57:24.587 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.587 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-09-19T04:57:24.587 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:57:24.587 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-09-19T04:57:24.587 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:57:24.587 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:57:24.588 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:57:24.588 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:57:24.588 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:57:24.588 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:57:24.588 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:57:24.588 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:57:24.588 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:57:24.588 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:57:24.588 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:57:24.588 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:57:24.588 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:57:24.588 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-09-19T04:57:24.588 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:57:24.589 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-09-19T04:57:24.589 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:57:24.589 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:57:24.589 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:57:24.589 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:57:24.589 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:57:24.589 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:57:24.589 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:57:24.589 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-09-19T04:57:24.589 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:57:24.589 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:57:24.589 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:57:24.590 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-09-19T04:57:24.590 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-09-19T04:57:24.590 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-09-19T04:57:24.590 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:57:24.590 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:57:24.590 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:57:24.590 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:57:24.590 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 415278, 2024-09-19T04:57:24.590 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 5047, 2024-09-19T04:57:24.590 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:57:24.590 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-09-19T04:57:24.590 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-09-19T04:57:24.591 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-09-19T04:57:24.591 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:57:24.591 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-09-19T04:57:24.591 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:57:24.591 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-09-19T04:57:24.591 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8128, 2024-09-19T04:57:24.591 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 179, 2024-09-19T04:57:24.591 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:57:24.591 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:57:24.591 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-09-19T04:57:24.591 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-09-19T04:57:24.591 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 23610502, 2024-09-19T04:57:24.592 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 192383, 2024-09-19T04:57:24.592 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-09-19T04:57:24.592 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-09-19T04:57:24.592 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-09-19T04:57:24.592 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-09-19T04:57:24.592 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.592 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-09-19T04:57:24.592 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-09-19T04:57:24.592 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-09-19T04:57:24.592 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 2588, 2024-09-19T04:57:24.592 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 21, 2024-09-19T04:57:24.592 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 3, 2024-09-19T04:57:24.592 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 2588, 2024-09-19T04:57:24.592 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-09-19T04:57:24.593 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2588, 2024-09-19T04:57:24.593 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2.077966906, 2024-09-19T04:57:24.593 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000802923 2024-09-19T04:57:24.593 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.593 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-09-19T04:57:24.593 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-09-19T04:57:24.593 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2588, 2024-09-19T04:57:24.593 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2589 2024-09-19T04:57:24.593 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.593 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 2588, 2024-09-19T04:57:24.593 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 2588, 2024-09-19T04:57:24.593 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 0, 2024-09-19T04:57:24.593 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-09-19T04:57:24.594 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-09-19T04:57:24.594 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 1543, 2024-09-19T04:57:24.594 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 0, 2024-09-19T04:57:24.594 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 1043, 2024-09-19T04:57:24.594 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 0, 2024-09-19T04:57:24.594 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 0, 2024-09-19T04:57:24.594 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-09-19T04:57:24.594 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-09-19T04:57:24.594 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-09-19T04:57:24.594 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-09-19T04:57:24.594 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 0, 2024-09-19T04:57:24.594 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-09-19T04:57:24.594 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-09-19T04:57:24.594 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-09-19T04:57:24.595 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 1, 2024-09-19T04:57:24.595 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 0, 2024-09-19T04:57:24.595 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:57:24.595 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-09-19T04:57:24.595 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:57:24.595 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-09-19T04:57:24.595 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-09-19T04:57:24.595 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-09-19T04:57:24.595 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:57:24.595 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-09-19T04:57:24.595 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:57:24.595 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-09-19T04:57:24.595 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-09-19T04:57:24.595 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-09-19T04:57:24.596 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-09-19T04:57:24.596 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-09-19T04:57:24.596 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-09-19T04:57:24.596 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-09-19T04:57:24.596 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-09-19T04:57:24.596 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-09-19T04:57:24.596 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-09-19T04:57:24.596 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-09-19T04:57:24.596 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-09-19T04:57:24.596 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-09-19T04:57:24.596 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-09-19T04:57:24.596 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-09-19T04:57:24.596 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-09-19T04:57:24.597 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-09-19T04:57:24.597 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 97, 2024-09-19T04:57:24.597 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-09-19T04:57:24.597 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 13, 2024-09-19T04:57:24.597 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 10, 2024-09-19T04:57:24.597 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 10, 2024-09-19T04:57:24.597 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-09-19T04:57:24.597 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-09-19T04:57:24.597 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 0, 2024-09-19T04:57:24.597 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 2, 2024-09-19T04:57:24.598 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 0 2024-09-19T04:57:24.598 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.598 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-09-19T04:57:24.598 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 0, 2024-09-19T04:57:24.598 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:57:24.598 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-09-19T04:57:24.598 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-09-19T04:57:24.598 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.598 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-09-19T04:57:24.598 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-09-19T04:57:24.598 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.598 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-09-19T04:57:24.599 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-09-19T04:57:24.599 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:57:24.599 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-09-19T04:57:24.599 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-09-19T04:57:24.599 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-09-19T04:57:24.599 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-09-19T04:57:24.599 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-09-19T04:57:24.599 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.599 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:57:24.599 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:24.599 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-19T04:57:24.599 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:24.599 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 2855, 2024-09-19T04:57:24.599 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 172979335, 2024-09-19T04:57:24.600 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:24.600 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 2855, 2024-09-19T04:57:24.600 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-19T04:57:24.600 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-19T04:57:24.600 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 2855, 2024-09-19T04:57:24.600 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 172979335, 2024-09-19T04:57:24.600 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:24.600 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.600 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.600 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.600 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:24.600 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.600 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-09-19T04:57:24.601 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:24.601 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-19T04:57:24.601 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:24.601 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-19T04:57:24.601 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-19T04:57:24.601 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:24.601 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:24.601 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 2588, 2024-09-19T04:57:24.601 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 170806604, 2024-09-19T04:57:24.601 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 2588, 2024-09-19T04:57:24.601 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 170806604, 2024-09-19T04:57:24.601 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:24.601 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.601 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.602 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.602 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:24.602 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.602 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-09-19T04:57:24.602 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:24.602 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-09-19T04:57:24.602 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:24.602 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-19T04:57:24.602 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-19T04:57:24.602 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:24.602 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:24.602 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 2588, 2024-09-19T04:57:24.602 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 2588, 2024-09-19T04:57:24.602 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 2588, 2024-09-19T04:57:24.603 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2588, 2024-09-19T04:57:24.603 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:24.603 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.603 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.603 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.603 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:24.603 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.603 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:57:24.603 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:24.603 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-19T04:57:24.603 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:24.603 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-19T04:57:24.603 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-19T04:57:24.603 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:24.604 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:24.604 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-19T04:57:24.604 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-19T04:57:24.604 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-09-19T04:57:24.604 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-09-19T04:57:24.604 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:24.604 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.604 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.604 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.604 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:24.604 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-19T04:57:24.604 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x5567b5b30b40": { 2024-09-19T04:57:24.604 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-19T04:57:24.604 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-19T04:57:24.605 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-19T04:57:24.605 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-19T04:57:24.605 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-19T04:57:24.605 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-19T04:57:24.605 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-19T04:57:24.605 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-19T04:57:24.605 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-19T04:57:24.605 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-09-19T04:57:24.605 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-09-19T04:57:24.605 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-19T04:57:24.605 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-19T04:57:24.605 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-19T04:57:24.605 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-19T04:57:24.605 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:24.606 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-19T04:57:24.606 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-19T04:57:25.255 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-19T04:57:25.255 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-19T04:57:25.255 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-19T04:57:25.255 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-19T04:57:25.256 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-19T04:57:25.256 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:57:25.256 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:57:25.256 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-09-19T04:57:25.256 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-19T04:57:25.267 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-19T04:57:25.268 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:57:25.268 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:57:25.310 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T04:57:25.310 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:57:25.388 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T04:57:25.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:25 smithi193 ceph-mon[35592]: pgmap v334: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 102 B/s wr, 9 op/s 2024-09-19T04:57:25.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:25 smithi183 ceph-mon[35333]: pgmap v334: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 102 B/s wr, 9 op/s 2024-09-19T04:57:25.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:25 smithi123 ceph-mon[30511]: pgmap v334: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 102 B/s wr, 9 op/s 2024-09-19T04:57:25.799 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:26.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.337+0000 7fc75fe00640 1 -- 172.21.15.123:0/3977761165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc758101de0 msgr2=0x7fc7581021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:26.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.337+0000 7fc75fe00640 1 --2- 172.21.15.123:0/3977761165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc758101de0 0x7fc7581021c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009900 tx=0x7fc74c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.337+0000 7fc75fe00640 1 -- 172.21.15.123:0/3977761165 shutdown_connections 2024-09-19T04:57:26.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.337+0000 7fc75fe00640 1 --2- 172.21.15.123:0/3977761165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc758106f10 0x7fc75810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.337+0000 7fc75fe00640 1 --2- 172.21.15.123:0/3977761165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc758102790 0x7fc7581067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.337+0000 7fc75fe00640 1 --2- 172.21.15.123:0/3977761165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc758101de0 0x7fc7581021c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.337+0000 7fc75fe00640 1 -- 172.21.15.123:0/3977761165 >> 172.21.15.123:0/3977761165 conn(0x7fc7580fd8c0 msgr2=0x7fc7580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:26.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.337+0000 7fc75fe00640 1 -- 172.21.15.123:0/3977761165 shutdown_connections 2024-09-19T04:57:26.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.337+0000 7fc75fe00640 1 -- 172.21.15.123:0/3977761165 wait complete. 2024-09-19T04:57:26.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.338+0000 7fc75fe00640 1 Processor -- start 2024-09-19T04:57:26.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.339+0000 7fc75fe00640 1 -- start start 2024-09-19T04:57:26.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.339+0000 7fc75fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc758101de0 0x7fc7580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:26.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.339+0000 7fc75fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc758102790 0x7fc758074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.339+0000 7fc75fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc758106f10 0x7fc758074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.339+0000 7fc75fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc758111fc0 con 0x7fc758102790 2024-09-19T04:57:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.339+0000 7fc75fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc758111e40 con 0x7fc758106f10 2024-09-19T04:57:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.339+0000 7fc75fe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc758112140 con 0x7fc758101de0 2024-09-19T04:57:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.339+0000 7fc75ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc758102790 0x7fc758074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.339+0000 7fc75ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc758102790 0x7fc758074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42908/0 (socket says 172.21.15.123:42908) 2024-09-19T04:57:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.339+0000 7fc75ca00640 1 -- 172.21.15.123:0/3942300733 learned_addr learned my addr 172.21.15.123:0/3942300733 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:57:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.339+0000 7fc75d400640 1 --2- 172.21.15.123:0/3942300733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc758101de0 0x7fc7580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.339+0000 7fc75de00640 1 --2- 172.21.15.123:0/3942300733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc758106f10 0x7fc758074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.340+0000 7fc75d400640 1 -- 172.21.15.123:0/3942300733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc758106f10 msgr2=0x7fc758074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.340+0000 7fc75d400640 1 --2- 172.21.15.123:0/3942300733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc758106f10 0x7fc758074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.340+0000 7fc75d400640 1 -- 172.21.15.123:0/3942300733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc758102790 msgr2=0x7fc758074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.340+0000 7fc75d400640 1 --2- 172.21.15.123:0/3942300733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc758102790 0x7fc758074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.340+0000 7fc75d400640 1 -- 172.21.15.123:0/3942300733 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7581a6900 con 0x7fc758101de0 2024-09-19T04:57:26.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.340+0000 7fc75de00640 1 --2- 172.21.15.123:0/3942300733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc758106f10 0x7fc758074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:57:26.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.340+0000 7fc75d400640 1 --2- 172.21.15.123:0/3942300733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc758101de0 0x7fc7580744f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc74c02a910 tx=0x7fc74c003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:26.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.340+0000 7fc756000640 1 -- 172.21.15.123:0/3942300733 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c03f070 con 0x7fc758101de0 2024-09-19T04:57:26.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.340+0000 7fc75fe00640 1 -- 172.21.15.123:0/3942300733 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7581a6b90 con 0x7fc758101de0 2024-09-19T04:57:26.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.341+0000 7fc756000640 1 -- 172.21.15.123:0/3942300733 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc74c031c90 con 0x7fc758101de0 2024-09-19T04:57:26.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.341+0000 7fc756000640 1 -- 172.21.15.123:0/3942300733 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c035030 con 0x7fc758101de0 2024-09-19T04:57:26.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.341+0000 7fc75fe00640 1 -- 172.21.15.123:0/3942300733 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7581a70c0 con 0x7fc758101de0 2024-09-19T04:57:26.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.342+0000 7fc75fe00640 1 -- 172.21.15.123:0/3942300733 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7581021c0 con 0x7fc758101de0 2024-09-19T04:57:26.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.343+0000 7fc756000640 1 -- 172.21.15.123:0/3942300733 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc74c042070 con 0x7fc758101de0 2024-09-19T04:57:26.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.345+0000 7fc756000640 1 --2- 172.21.15.123:0/3942300733 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc7400a3430 0x7fc7400a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:26.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.345+0000 7fc756000640 1 -- 172.21.15.123:0/3942300733 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fc74c0f0580 con 0x7fc758101de0 2024-09-19T04:57:26.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.345+0000 7fc75ca00640 1 --2- 172.21.15.123:0/3942300733 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc7400a3430 0x7fc7400a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:26.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.346+0000 7fc75ca00640 1 --2- 172.21.15.123:0/3942300733 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc7400a3430 0x7fc7400a58f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc758075f80 tx=0x7fc744005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:26.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.349+0000 7fc756000640 1 -- 172.21.15.123:0/3942300733 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc74c0bcf00 con 0x7fc758101de0 2024-09-19T04:57:26.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.534+0000 7fc75fe00640 1 -- 172.21.15.123:0/3942300733 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc75810a0f0 con 0x7fc758101de0 2024-09-19T04:57:26.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.535+0000 7fc756000640 1 -- 172.21.15.123:0/3942300733 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v97) ==== 81+0+961 (secure 0 0 0) 0x7fc74c0bc800 con 0x7fc758101de0 2024-09-19T04:57:26.538 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:6801/1532622003 2024-09-20T04:46:41.775112+0000 2024-09-19T04:57:26.538 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/2833919666 2024-09-20T04:46:41.775112+0000 2024-09-19T04:57:26.538 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/2580747029 2024-09-20T04:36:45.913060+0000 2024-09-19T04:57:26.538 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/471028951 2024-09-20T04:37:13.257785+0000 2024-09-19T04:57:26.538 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:6801/2065492981 2024-09-20T04:36:45.913060+0000 2024-09-19T04:57:26.538 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:6800/3323203947 2024-09-20T04:37:13.257785+0000 2024-09-19T04:57:26.538 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:6800/1532622003 2024-09-20T04:46:41.775112+0000 2024-09-19T04:57:26.538 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:6801/3323203947 2024-09-20T04:37:13.257785+0000 2024-09-19T04:57:26.538 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/3498196650 2024-09-20T04:37:13.257785+0000 2024-09-19T04:57:26.538 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/4040401634 2024-09-20T04:37:13.257785+0000 2024-09-19T04:57:26.538 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/2000593686 2024-09-20T04:36:45.913060+0000 2024-09-19T04:57:26.539 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:6800/2065492981 2024-09-20T04:36:45.913060+0000 2024-09-19T04:57:26.539 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/3339335756 2024-09-20T04:46:41.775112+0000 2024-09-19T04:57:26.539 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/1009496020 2024-09-20T04:36:45.913060+0000 2024-09-19T04:57:26.539 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/3244738342 2024-09-20T04:46:41.775112+0000 2024-09-19T04:57:26.539 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/2764863023 2024-09-20T04:46:41.775112+0000 2024-09-19T04:57:26.539 INFO:teuthology.orchestra.run.smithi123.stderr:listed 16 entries 2024-09-19T04:57:26.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.537+0000 7fc75fe00640 1 -- 172.21.15.123:0/3942300733 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc7400a3430 msgr2=0x7fc7400a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:26.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.537+0000 7fc75fe00640 1 --2- 172.21.15.123:0/3942300733 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc7400a3430 0x7fc7400a58f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc758075f80 tx=0x7fc744005f90 comp rx=0 tx=0).stop 2024-09-19T04:57:26.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.537+0000 7fc75fe00640 1 -- 172.21.15.123:0/3942300733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc758101de0 msgr2=0x7fc7580744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:26.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.537+0000 7fc75fe00640 1 --2- 172.21.15.123:0/3942300733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc758101de0 0x7fc7580744f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc74c02a910 tx=0x7fc74c003b20 comp rx=0 tx=0).stop 2024-09-19T04:57:26.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.537+0000 7fc75fe00640 1 -- 172.21.15.123:0/3942300733 shutdown_connections 2024-09-19T04:57:26.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.537+0000 7fc75fe00640 1 --2- 172.21.15.123:0/3942300733 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fc7400a3430 0x7fc7400a58f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.537+0000 7fc75fe00640 1 --2- 172.21.15.123:0/3942300733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc758106f10 0x7fc758074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.537+0000 7fc75fe00640 1 --2- 172.21.15.123:0/3942300733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc758102790 0x7fc758074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.537+0000 7fc75fe00640 1 --2- 172.21.15.123:0/3942300733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc758101de0 0x7fc7580744f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.537+0000 7fc75fe00640 1 -- 172.21.15.123:0/3942300733 >> 172.21.15.123:0/3942300733 conn(0x7fc7580fd8c0 msgr2=0x7fc7580fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:26.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.538+0000 7fc75fe00640 1 -- 172.21.15.123:0/3942300733 shutdown_connections 2024-09-19T04:57:26.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:26.538+0000 7fc75fe00640 1 -- 172.21.15.123:0/3942300733 wait complete. 2024-09-19T04:57:27.449 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-19T04:57:27.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:57:27.450 DEBUG:teuthology.orchestra.run.smithi123:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-19T04:57:27.533 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T04:57:27.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:57:27.533 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T04:57:27.534 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:27 smithi123 ceph-mon[30511]: pgmap v335: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 102 B/s wr, 9 op/s 2024-09-19T04:57:27.534 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:27 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3942300733' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:57:27.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:27 smithi183 ceph-mon[35333]: pgmap v335: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 102 B/s wr, 9 op/s 2024-09-19T04:57:27.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:27 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3942300733' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:57:27.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:57:27.612 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-09-19T04:57:27.685 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T04:57:27.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:57:27.685 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:57:27.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:27 smithi193 ceph-mon[35592]: pgmap v335: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 102 B/s wr, 9 op/s 2024-09-19T04:57:27.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:27 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3942300733' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:57:27.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:57:27.786 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-09-19T04:57:28.129 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T04:57:28.140 INFO:tasks.cephadm:Teardown begin 2024-09-19T04:57:28.140 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T04:57:28.140 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:57:28.513 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:29.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.082+0000 7ffaeba00640 1 -- 172.21.15.123:0/1039149441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffae4074620 msgr2=0x7ffae4074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:29.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.082+0000 7ffaeba00640 1 --2- 172.21.15.123:0/1039149441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffae4074620 0x7ffae4074a00 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009900 tx=0x7ffad402a480 comp rx=0 tx=0).stop 2024-09-19T04:57:29.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.082+0000 7ffaeba00640 1 -- 172.21.15.123:0/1039149441 shutdown_connections 2024-09-19T04:57:29.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.082+0000 7ffaeba00640 1 --2- 172.21.15.123:0/1039149441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffae410e3c0 0x7ffae4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:29.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.082+0000 7ffaeba00640 1 --2- 172.21.15.123:0/1039149441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffae4074f40 0x7ffae410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:29.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.082+0000 7ffaeba00640 1 --2- 172.21.15.123:0/1039149441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffae4074620 0x7ffae4074a00 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:29.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.082+0000 7ffaeba00640 1 -- 172.21.15.123:0/1039149441 >> 172.21.15.123:0/1039149441 conn(0x7ffae40fd8c0 msgr2=0x7ffae40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:29.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.082+0000 7ffaeba00640 1 -- 172.21.15.123:0/1039149441 shutdown_connections 2024-09-19T04:57:29.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.082+0000 7ffaeba00640 1 -- 172.21.15.123:0/1039149441 wait complete. 2024-09-19T04:57:29.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.083+0000 7ffaeba00640 1 Processor -- start 2024-09-19T04:57:29.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.083+0000 7ffaeba00640 1 -- start start 2024-09-19T04:57:29.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.083+0000 7ffaeba00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffae4074620 0x7ffae419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:29.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.083+0000 7ffaeba00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffae4074f40 0x7ffae419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:29.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.083+0000 7ffaeba00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffae410e3c0 0x7ffae419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:29.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.083+0000 7ffaeba00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffae4114170 con 0x7ffae4074620 2024-09-19T04:57:29.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.083+0000 7ffaeba00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffae4113ff0 con 0x7ffae4074f40 2024-09-19T04:57:29.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.083+0000 7ffaeba00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41142f0 con 0x7ffae410e3c0 2024-09-19T04:57:29.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.084+0000 7ffae9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffae4074620 0x7ffae419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:29.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.084+0000 7ffae9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffae4074620 0x7ffae419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42924/0 (socket says 172.21.15.123:42924) 2024-09-19T04:57:29.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.084+0000 7ffae9000640 1 -- 172.21.15.123:0/3526085991 learned_addr learned my addr 172.21.15.123:0/3526085991 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:57:29.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.084+0000 7ffadbe00640 1 --2- 172.21.15.123:0/3526085991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffae4074f40 0x7ffae419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:29.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.084+0000 7ffae9a00640 1 --2- 172.21.15.123:0/3526085991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffae410e3c0 0x7ffae419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:29.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.084+0000 7ffae9000640 1 -- 172.21.15.123:0/3526085991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffae410e3c0 msgr2=0x7ffae419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:29.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.084+0000 7ffae9000640 1 --2- 172.21.15.123:0/3526085991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffae410e3c0 0x7ffae419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:29.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.084+0000 7ffae9000640 1 -- 172.21.15.123:0/3526085991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffae4074f40 msgr2=0x7ffae419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:29.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.084+0000 7ffae9000640 1 --2- 172.21.15.123:0/3526085991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffae4074f40 0x7ffae419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:29.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.084+0000 7ffae9000640 1 -- 172.21.15.123:0/3526085991 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae419f620 con 0x7ffae4074620 2024-09-19T04:57:29.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.084+0000 7ffadbe00640 1 --2- 172.21.15.123:0/3526085991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffae4074f40 0x7ffae419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:57:29.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.084+0000 7ffae9000640 1 --2- 172.21.15.123:0/3526085991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffae4074620 0x7ffae419a670 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ffad4006010 tx=0x7ffad4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:29.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.085+0000 7ffae2000640 1 -- 172.21.15.123:0/3526085991 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad403f030 con 0x7ffae4074620 2024-09-19T04:57:29.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.085+0000 7ffae2000640 1 -- 172.21.15.123:0/3526085991 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffad4004440 con 0x7ffae4074620 2024-09-19T04:57:29.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.085+0000 7ffae2000640 1 -- 172.21.15.123:0/3526085991 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4035070 con 0x7ffae4074620 2024-09-19T04:57:29.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.085+0000 7ffaeba00640 1 -- 172.21.15.123:0/3526085991 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae419f8b0 con 0x7ffae4074620 2024-09-19T04:57:29.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.085+0000 7ffaeba00640 1 -- 172.21.15.123:0/3526085991 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae419fd40 con 0x7ffae4074620 2024-09-19T04:57:29.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.086+0000 7ffaeba00640 1 -- 172.21.15.123:0/3526085991 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae4074a00 con 0x7ffae4074620 2024-09-19T04:57:29.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.088+0000 7ffae2000640 1 -- 172.21.15.123:0/3526085991 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ffad4042030 con 0x7ffae4074620 2024-09-19T04:57:29.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.089+0000 7ffae2000640 1 --2- 172.21.15.123:0/3526085991 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffac80a3430 0x7ffac80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:29.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.089+0000 7ffae2000640 1 -- 172.21.15.123:0/3526085991 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7ffad40f0040 con 0x7ffae4074620 2024-09-19T04:57:29.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.089+0000 7ffadbe00640 1 --2- 172.21.15.123:0/3526085991 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffac80a3430 0x7ffac80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:29.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.090+0000 7ffadbe00640 1 --2- 172.21.15.123:0/3526085991 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffac80a3430 0x7ffac80a58f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffae410f820 tx=0x7ffacc005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:29.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.093+0000 7ffae2000640 1 -- 172.21.15.123:0/3526085991 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ffad40bc430 con 0x7ffae4074620 2024-09-19T04:57:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.305+0000 7ffaeba00640 1 -- 172.21.15.123:0/3526085991 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffae4108fb0 con 0x7ffae4074620 2024-09-19T04:57:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.305+0000 7ffae2000640 1 -- 172.21.15.123:0/3526085991 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11990 (secure 0 0 0) 0x7ffad40c0730 con 0x7ffae4074620 2024-09-19T04:57:29.308 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:57:29.308 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":43,"btime":"2024-09-19T04:54:44:530698+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:54:44.239974+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24529,24505,24539,24500,24499]},"id":1}]} 2024-09-19T04:57:29.309 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 43 2024-09-19T04:57:29.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.308+0000 7ffaeba00640 1 -- 172.21.15.123:0/3526085991 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffac80a3430 msgr2=0x7ffac80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:29.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.308+0000 7ffaeba00640 1 --2- 172.21.15.123:0/3526085991 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffac80a3430 0x7ffac80a58f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffae410f820 tx=0x7ffacc005f90 comp rx=0 tx=0).stop 2024-09-19T04:57:29.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.308+0000 7ffaeba00640 1 -- 172.21.15.123:0/3526085991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffae4074620 msgr2=0x7ffae419a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:29.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.308+0000 7ffaeba00640 1 --2- 172.21.15.123:0/3526085991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffae4074620 0x7ffae419a670 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ffad4006010 tx=0x7ffad4004040 comp rx=0 tx=0).stop 2024-09-19T04:57:29.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.308+0000 7ffaeba00640 1 -- 172.21.15.123:0/3526085991 shutdown_connections 2024-09-19T04:57:29.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.308+0000 7ffaeba00640 1 --2- 172.21.15.123:0/3526085991 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7ffac80a3430 0x7ffac80a58f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:29.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.308+0000 7ffaeba00640 1 --2- 172.21.15.123:0/3526085991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffae410e3c0 0x7ffae419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:29.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.308+0000 7ffaeba00640 1 --2- 172.21.15.123:0/3526085991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffae4074f40 0x7ffae419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:29.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.308+0000 7ffaeba00640 1 --2- 172.21.15.123:0/3526085991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffae4074620 0x7ffae419a670 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:29.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.308+0000 7ffaeba00640 1 -- 172.21.15.123:0/3526085991 >> 172.21.15.123:0/3526085991 conn(0x7ffae40fd8c0 msgr2=0x7ffae40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:29.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.308+0000 7ffaeba00640 1 -- 172.21.15.123:0/3526085991 shutdown_connections 2024-09-19T04:57:29.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:29.308+0000 7ffaeba00640 1 -- 172.21.15.123:0/3526085991 wait complete. 2024-09-19T04:57:29.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:29 smithi183 ceph-mon[35333]: pgmap v336: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-09-19T04:57:29.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:29 smithi123 ceph-mon[30511]: pgmap v336: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-09-19T04:57:29.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:29 smithi193 ceph-mon[35592]: pgmap v336: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-09-19T04:57:30.234 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:57:30.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:30 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3526085991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:57:30.602 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:30.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:30 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3526085991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:57:30.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:30 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3526085991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:57:31.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.296+0000 7f9ca4a00640 1 -- 172.21.15.123:0/586931344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ca0108f60 msgr2=0x7f9ca0110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:31.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.296+0000 7f9ca4a00640 1 --2- 172.21.15.123:0/586931344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ca0108f60 0x7f9ca0110ae0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009ca0 tx=0x7f9c9802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:57:31.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.296+0000 7f9ca4a00640 1 -- 172.21.15.123:0/586931344 shutdown_connections 2024-09-19T04:57:31.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.296+0000 7f9ca4a00640 1 --2- 172.21.15.123:0/586931344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ca0108f60 0x7f9ca0110ae0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:31.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.296+0000 7f9ca4a00640 1 --2- 172.21.15.123:0/586931344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ca01047e0 0x7f9ca0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:31.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.296+0000 7f9ca4a00640 1 --2- 172.21.15.123:0/586931344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ca0103e30 0x7f9ca0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:31.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.296+0000 7f9ca4a00640 1 -- 172.21.15.123:0/586931344 >> 172.21.15.123:0/586931344 conn(0x7f9ca00ff910 msgr2=0x7f9ca0101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:31.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.296+0000 7f9ca4a00640 1 -- 172.21.15.123:0/586931344 shutdown_connections 2024-09-19T04:57:31.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.296+0000 7f9ca4a00640 1 -- 172.21.15.123:0/586931344 wait complete. 2024-09-19T04:57:31.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.297+0000 7f9ca4a00640 1 Processor -- start 2024-09-19T04:57:31.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.297+0000 7f9ca4a00640 1 -- start start 2024-09-19T04:57:31.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.297+0000 7f9ca4a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ca0103e30 0x7f9ca019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:31.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.297+0000 7f9ca4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ca01047e0 0x7f9ca019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:31.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.297+0000 7f9ca4a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ca0108f60 0x7f9ca019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:31.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.297+0000 7f9ca4a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca0114010 con 0x7f9ca01047e0 2024-09-19T04:57:31.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.297+0000 7f9ca4a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca0113e90 con 0x7f9ca0103e30 2024-09-19T04:57:31.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.298+0000 7f9ca4a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca0114190 con 0x7f9ca0108f60 2024-09-19T04:57:31.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.298+0000 7f9c97e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ca01047e0 0x7f9ca019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:31.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.298+0000 7f9c97e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ca01047e0 0x7f9ca019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42950/0 (socket says 172.21.15.123:42950) 2024-09-19T04:57:31.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.298+0000 7f9c97e00640 1 -- 172.21.15.123:0/3770663362 learned_addr learned my addr 172.21.15.123:0/3770663362 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:57:31.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.298+0000 7f9c9f200640 1 --2- 172.21.15.123:0/3770663362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ca0108f60 0x7f9ca019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:31.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.298+0000 7f9c9e800640 1 --2- 172.21.15.123:0/3770663362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ca0103e30 0x7f9ca019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:31.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.298+0000 7f9c97e00640 1 -- 172.21.15.123:0/3770663362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ca0108f60 msgr2=0x7f9ca019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:31.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.298+0000 7f9c97e00640 1 --2- 172.21.15.123:0/3770663362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ca0108f60 0x7f9ca019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:31.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.298+0000 7f9c97e00640 1 -- 172.21.15.123:0/3770663362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ca0103e30 msgr2=0x7f9ca019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:31.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.298+0000 7f9c97e00640 1 --2- 172.21.15.123:0/3770663362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ca0103e30 0x7f9ca019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:31.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.298+0000 7f9c97e00640 1 -- 172.21.15.123:0/3770663362 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca019f460 con 0x7f9ca01047e0 2024-09-19T04:57:31.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.299+0000 7f9c97e00640 1 --2- 172.21.15.123:0/3770663362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ca01047e0 0x7f9ca019a9f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800d950 tx=0x7f9c8800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:31.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.299+0000 7f9c97400640 1 -- 172.21.15.123:0/3770663362 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88017030 con 0x7f9ca01047e0 2024-09-19T04:57:31.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.299+0000 7f9c97400640 1 -- 172.21.15.123:0/3770663362 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c88004020 con 0x7f9ca01047e0 2024-09-19T04:57:31.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.299+0000 7f9c97400640 1 -- 172.21.15.123:0/3770663362 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c880047a0 con 0x7f9ca01047e0 2024-09-19T04:57:31.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.299+0000 7f9ca4a00640 1 -- 172.21.15.123:0/3770663362 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca019f750 con 0x7f9ca01047e0 2024-09-19T04:57:31.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.299+0000 7f9ca4a00640 1 -- 172.21.15.123:0/3770663362 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca01a7030 con 0x7f9ca01047e0 2024-09-19T04:57:31.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.300+0000 7f9ca4a00640 1 -- 172.21.15.123:0/3770663362 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca0105310 con 0x7f9ca01047e0 2024-09-19T04:57:31.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.302+0000 7f9c97400640 1 -- 172.21.15.123:0/3770663362 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9c880041d0 con 0x7f9ca01047e0 2024-09-19T04:57:31.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.303+0000 7f9c97400640 1 --2- 172.21.15.123:0/3770663362 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9c840a3430 0x7f9c840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:31.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.303+0000 7f9c97400640 1 -- 172.21.15.123:0/3770663362 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f9c880cf5e0 con 0x7f9ca01047e0 2024-09-19T04:57:31.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.303+0000 7f9c9e800640 1 --2- 172.21.15.123:0/3770663362 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9c840a3430 0x7f9c840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:31.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.304+0000 7f9c9e800640 1 --2- 172.21.15.123:0/3770663362 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9c840a3430 0x7f9c840a58f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9c90005d00 tx=0x7f9c9000b410 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:31.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.308+0000 7f9c97400640 1 -- 172.21.15.123:0/3770663362 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9c8809bf60 con 0x7f9ca01047e0 2024-09-19T04:57:31.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.517+0000 7f9ca4a00640 1 -- 172.21.15.123:0/3770663362 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ca002d7c0 con 0x7f9ca01047e0 2024-09-19T04:57:31.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.518+0000 7f9c97400640 1 -- 172.21.15.123:0/3770663362 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11990 (secure 0 0 0) 0x7f9c8809b860 con 0x7f9ca01047e0 2024-09-19T04:57:31.519 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:57:31.520 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":43,"btime":"2024-09-19T04:54:44:530698+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1654466507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1654466507},{"type":"v1","addr":"172.21.15.123:6835","nonce":1654466507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4146531731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4146531731},{"type":"v1","addr":"172.21.15.123:6839","nonce":4146531731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:44:31.766928+0000","modified":"2024-09-19T04:54:44.239974+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24529,"mds_2":24505,"mds_3":24500,"mds_4":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6841/784762674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":784762674},{"type":"v1","addr":"172.21.15.123:6841","nonce":784762674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2688234655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2688234655},{"type":"v1","addr":"172.21.15.183:6833","nonce":2688234655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/964700250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":964700250},{"type":"v1","addr":"172.21.15.193:6833","nonce":964700250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/3620540498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3620540498},{"type":"v1","addr":"172.21.15.123:6837","nonce":3620540498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/309335638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":309335638},{"type":"v1","addr":"172.21.15.183:6835","nonce":309335638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6835/1351624350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1351624350},{"type":"v1","addr":"172.21.15.193:6835","nonce":1351624350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6837/3418533099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3418533099},{"type":"v1","addr":"172.21.15.183:6837","nonce":3418533099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/3444596480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3444596480},{"type":"v1","addr":"172.21.15.193:6837","nonce":3444596480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6839/4148250455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4148250455},{"type":"v1","addr":"172.21.15.193:6839","nonce":4148250455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/157697996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":157697996},{"type":"v1","addr":"172.21.15.183:6839","nonce":157697996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24529,24505,24539,24500,24499]},"id":1}]} 2024-09-19T04:57:31.520 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 43 2024-09-19T04:57:31.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.520+0000 7f9ca4a00640 1 -- 172.21.15.123:0/3770663362 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9c840a3430 msgr2=0x7f9c840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:31.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.520+0000 7f9ca4a00640 1 --2- 172.21.15.123:0/3770663362 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9c840a3430 0x7f9c840a58f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9c90005d00 tx=0x7f9c9000b410 comp rx=0 tx=0).stop 2024-09-19T04:57:31.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.520+0000 7f9ca4a00640 1 -- 172.21.15.123:0/3770663362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ca01047e0 msgr2=0x7f9ca019a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:31.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.520+0000 7f9ca4a00640 1 --2- 172.21.15.123:0/3770663362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ca01047e0 0x7f9ca019a9f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800d950 tx=0x7f9c8800de10 comp rx=0 tx=0).stop 2024-09-19T04:57:31.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.520+0000 7f9ca4a00640 1 -- 172.21.15.123:0/3770663362 shutdown_connections 2024-09-19T04:57:31.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.520+0000 7f9ca4a00640 1 --2- 172.21.15.123:0/3770663362 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f9c840a3430 0x7f9c840a58f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:31.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.520+0000 7f9ca4a00640 1 --2- 172.21.15.123:0/3770663362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ca0108f60 0x7f9ca019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:31.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.520+0000 7f9ca4a00640 1 --2- 172.21.15.123:0/3770663362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ca01047e0 0x7f9ca019a9f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:31.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.520+0000 7f9ca4a00640 1 --2- 172.21.15.123:0/3770663362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ca0103e30 0x7f9ca019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:31.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.520+0000 7f9ca4a00640 1 -- 172.21.15.123:0/3770663362 >> 172.21.15.123:0/3770663362 conn(0x7f9ca00ff910 msgr2=0x7f9ca0100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:31.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.520+0000 7f9ca4a00640 1 -- 172.21.15.123:0/3770663362 shutdown_connections 2024-09-19T04:57:31.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:31.520+0000 7f9ca4a00640 1 -- 172.21.15.123:0/3770663362 wait complete. 2024-09-19T04:57:31.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:31 smithi183 ceph-mon[35333]: pgmap v337: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T04:57:31.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:31 smithi123 ceph-mon[30511]: pgmap v337: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T04:57:31.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:31 smithi193 ceph-mon[35592]: pgmap v337: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T04:57:32.416 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:57:32.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:32 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/3770663362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:57:32.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:32 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/3770663362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:57:32.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:32 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/3770663362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:57:32.785 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:33.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.344+0000 7f836de00640 1 -- 172.21.15.123:0/3724615163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83681027a0 msgr2=0x7f83681067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:33.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.344+0000 7f836de00640 1 --2- 172.21.15.123:0/3724615163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83681027a0 0x7f83681067f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8354009860 tx=0x7f835402a340 comp rx=0 tx=0).stop 2024-09-19T04:57:33.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.345+0000 7f836de00640 1 -- 172.21.15.123:0/3724615163 shutdown_connections 2024-09-19T04:57:33.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.345+0000 7f836de00640 1 --2- 172.21.15.123:0/3724615163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8368106f20 0x7f836810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:33.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.345+0000 7f836de00640 1 --2- 172.21.15.123:0/3724615163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83681027a0 0x7f83681067f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:33.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.345+0000 7f836de00640 1 --2- 172.21.15.123:0/3724615163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8368101df0 0x7f83681021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:33.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.345+0000 7f836de00640 1 -- 172.21.15.123:0/3724615163 >> 172.21.15.123:0/3724615163 conn(0x7f83680fd8b0 msgr2=0x7f83680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:33.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.345+0000 7f836de00640 1 -- 172.21.15.123:0/3724615163 shutdown_connections 2024-09-19T04:57:33.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.345+0000 7f836de00640 1 -- 172.21.15.123:0/3724615163 wait complete. 2024-09-19T04:57:33.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.346+0000 7f836de00640 1 Processor -- start 2024-09-19T04:57:33.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.346+0000 7f836de00640 1 -- start start 2024-09-19T04:57:33.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.346+0000 7f836de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8368101df0 0x7f83680744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:33.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.346+0000 7f836de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83681027a0 0x7f8368074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:33.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.346+0000 7f836de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8368106f20 0x7f8368074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:33.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.346+0000 7f836de00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8368111f80 con 0x7f8368106f20 2024-09-19T04:57:33.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.346+0000 7f836de00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8368111e00 con 0x7f83681027a0 2024-09-19T04:57:33.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.346+0000 7f836de00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8368112100 con 0x7f8368101df0 2024-09-19T04:57:33.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.346+0000 7f8367400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8368106f20 0x7f8368074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:33.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.346+0000 7f8366a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8368101df0 0x7f83680744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:33.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.346+0000 7f8366a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8368101df0 0x7f83680744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.123:38972/0 (socket says 172.21.15.123:38972) 2024-09-19T04:57:33.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.346+0000 7f8366a00640 1 -- 172.21.15.123:0/2580064269 learned_addr learned my addr 172.21.15.123:0/2580064269 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:57:33.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.347+0000 7f8366000640 1 --2- 172.21.15.123:0/2580064269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83681027a0 0x7f8368074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:33.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.347+0000 7f8366a00640 1 -- 172.21.15.123:0/2580064269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83681027a0 msgr2=0x7f8368074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:33.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.347+0000 7f8366a00640 1 --2- 172.21.15.123:0/2580064269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83681027a0 0x7f8368074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:33.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.347+0000 7f8366a00640 1 -- 172.21.15.123:0/2580064269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8368106f20 msgr2=0x7f8368074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:33.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.347+0000 7f8366a00640 1 --2- 172.21.15.123:0/2580064269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8368106f20 0x7f8368074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:33.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.347+0000 7f8366a00640 1 -- 172.21.15.123:0/2580064269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83681a68a0 con 0x7f8368101df0 2024-09-19T04:57:33.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.347+0000 7f8367400640 1 --2- 172.21.15.123:0/2580064269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8368106f20 0x7f8368074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:57:33.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.347+0000 7f8366a00640 1 --2- 172.21.15.123:0/2580064269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8368101df0 0x7f83680744f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f835800e9c0 tx=0x7f835800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:33.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.347+0000 7f836ca00640 1 -- 172.21.15.123:0/2580064269 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835800cde0 con 0x7f8368101df0 2024-09-19T04:57:33.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.347+0000 7f836de00640 1 -- 172.21.15.123:0/2580064269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83681a6b90 con 0x7f8368101df0 2024-09-19T04:57:33.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.347+0000 7f836ca00640 1 -- 172.21.15.123:0/2580064269 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83580035f0 con 0x7f8368101df0 2024-09-19T04:57:33.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.347+0000 7f836ca00640 1 -- 172.21.15.123:0/2580064269 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358010650 con 0x7f8368101df0 2024-09-19T04:57:33.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.347+0000 7f836de00640 1 -- 172.21.15.123:0/2580064269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83681a70f0 con 0x7f8368101df0 2024-09-19T04:57:33.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.349+0000 7f836de00640 1 -- 172.21.15.123:0/2580064269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83681021d0 con 0x7f8368101df0 2024-09-19T04:57:33.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.350+0000 7f836ca00640 1 -- 172.21.15.123:0/2580064269 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f83580037a0 con 0x7f8368101df0 2024-09-19T04:57:33.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.352+0000 7f836ca00640 1 --2- 172.21.15.123:0/2580064269 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f834c0a3430 0x7f834c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:33.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.352+0000 7f836ca00640 1 -- 172.21.15.123:0/2580064269 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f83580cffe0 con 0x7f8368101df0 2024-09-19T04:57:33.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.352+0000 7f8366000640 1 --2- 172.21.15.123:0/2580064269 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f834c0a3430 0x7f834c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:33.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.352+0000 7f8366000640 1 --2- 172.21.15.123:0/2580064269 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f834c0a3430 0x7f834c0a58f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f835402a930 tx=0x7f83540058d0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:33.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.356+0000 7f836ca00640 1 -- 172.21.15.123:0/2580064269 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f835809c9a0 con 0x7f8368101df0 2024-09-19T04:57:33.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.531+0000 7f836de00640 1 -- 172.21.15.123:0/2580064269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8368109bd0 con 0x7f8368101df0 2024-09-19T04:57:33.533 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:33 smithi123 ceph-mon[30511]: pgmap v338: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T04:57:33.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.533+0000 7f836ca00640 1 -- 172.21.15.123:0/2580064269 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+23268 (secure 0 0 0) 0x7f835809c2a0 con 0x7f8368101df0 2024-09-19T04:57:33.535 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T04:57:33.535 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":97,"fsid":"911fa052-7640-11ef-bceb-c7b262605968","created":"2024-09-19T04:36:21.083333+0000","modified":"2024-09-19T04:57:17.151308+0000","last_up_change":"2024-09-19T04:43:33.818439+0000","last_in_change":"2024-09-19T04:43:10.148272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:39:33.856981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:44:26.956483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:44:29.718625+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4c76e3b-1fbf-4e3f-88ab-a8c71f61d81b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6803","nonce":3631596501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6805","nonce":3631596501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6809","nonce":3631596501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3631596501},{"type":"v1","addr":"172.21.15.123:6807","nonce":3631596501}]},"public_addr":"172.21.15.123:6803/3631596501","cluster_addr":"172.21.15.123:6805/3631596501","heartbeat_back_addr":"172.21.15.123:6809/3631596501","heartbeat_front_addr":"172.21.15.123:6807/3631596501","state":["exists","up"]},{"osd":1,"uuid":"a526d4ce-586b-4d66-b720-e868df92fa24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6801","nonce":1259721614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6803","nonce":1259721614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6807","nonce":1259721614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1259721614},{"type":"v1","addr":"172.21.15.183:6805","nonce":1259721614}]},"public_addr":"172.21.15.183:6801/1259721614","cluster_addr":"172.21.15.183:6803/1259721614","heartbeat_back_addr":"172.21.15.183:6807/1259721614","heartbeat_front_addr":"172.21.15.183:6805/1259721614","state":["exists","up"]},{"osd":2,"uuid":"3650601a-6259-4b2a-97df-1d47b09b0127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6801","nonce":2708151726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6803","nonce":2708151726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6807","nonce":2708151726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2708151726},{"type":"v1","addr":"172.21.15.193:6805","nonce":2708151726}]},"public_addr":"172.21.15.193:6801/2708151726","cluster_addr":"172.21.15.193:6803/2708151726","heartbeat_back_addr":"172.21.15.193:6807/2708151726","heartbeat_front_addr":"172.21.15.193:6805/2708151726","state":["exists","up"]},{"osd":3,"uuid":"67c485ad-136e-4020-950d-9171a3c2de35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6811","nonce":3214806282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6813","nonce":3214806282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6817","nonce":3214806282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3214806282},{"type":"v1","addr":"172.21.15.123:6815","nonce":3214806282}]},"public_addr":"172.21.15.123:6811/3214806282","cluster_addr":"172.21.15.123:6813/3214806282","heartbeat_back_addr":"172.21.15.123:6817/3214806282","heartbeat_front_addr":"172.21.15.123:6815/3214806282","state":["exists","up"]},{"osd":4,"uuid":"c9098540-06ab-45a8-9cf9-f90519cc852e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6809","nonce":667618665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6811","nonce":667618665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6815","nonce":667618665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":667618665},{"type":"v1","addr":"172.21.15.183:6813","nonce":667618665}]},"public_addr":"172.21.15.183:6809/667618665","cluster_addr":"172.21.15.183:6811/667618665","heartbeat_back_addr":"172.21.15.183:6815/667618665","heartbeat_front_addr":"172.21.15.183:6813/667618665","state":["exists","up"]},{"osd":5,"uuid":"278a8dbd-6492-42d4-b489-1f832350bc6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6809","nonce":2292251639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6811","nonce":2292251639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6815","nonce":2292251639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2292251639},{"type":"v1","addr":"172.21.15.193:6813","nonce":2292251639}]},"public_addr":"172.21.15.193:6809/2292251639","cluster_addr":"172.21.15.193:6811/2292251639","heartbeat_back_addr":"172.21.15.193:6815/2292251639","heartbeat_front_addr":"172.21.15.193:6813/2292251639","state":["exists","up"]},{"osd":6,"uuid":"0fda4e39-6adb-4941-a39b-a9a15019088c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6819","nonce":226007499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6821","nonce":226007499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6825","nonce":226007499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":226007499},{"type":"v1","addr":"172.21.15.123:6823","nonce":226007499}]},"public_addr":"172.21.15.123:6819/226007499","cluster_addr":"172.21.15.123:6821/226007499","heartbeat_back_addr":"172.21.15.123:6825/226007499","heartbeat_front_addr":"172.21.15.123:6823/226007499","state":["exists","up"]},{"osd":7,"uuid":"5c0a021e-9f3b-4766-9686-0ef24b68cf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6817","nonce":3718518226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6819","nonce":3718518226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6823","nonce":3718518226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3718518226},{"type":"v1","addr":"172.21.15.183:6821","nonce":3718518226}]},"public_addr":"172.21.15.183:6817/3718518226","cluster_addr":"172.21.15.183:6819/3718518226","heartbeat_back_addr":"172.21.15.183:6823/3718518226","heartbeat_front_addr":"172.21.15.183:6821/3718518226","state":["exists","up"]},{"osd":8,"uuid":"9b0e67e0-6305-4f81-9211-d41960099a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6817","nonce":3551404118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6819","nonce":3551404118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6823","nonce":3551404118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3551404118},{"type":"v1","addr":"172.21.15.193:6821","nonce":3551404118}]},"public_addr":"172.21.15.193:6817/3551404118","cluster_addr":"172.21.15.193:6819/3551404118","heartbeat_back_addr":"172.21.15.193:6823/3551404118","heartbeat_front_addr":"172.21.15.193:6821/3551404118","state":["exists","up"]},{"osd":9,"uuid":"978ef3fa-c6ec-421f-b194-56d44813be5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6827","nonce":1439901149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6829","nonce":1439901149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6833","nonce":1439901149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1439901149},{"type":"v1","addr":"172.21.15.123:6831","nonce":1439901149}]},"public_addr":"172.21.15.123:6827/1439901149","cluster_addr":"172.21.15.123:6829/1439901149","heartbeat_back_addr":"172.21.15.123:6833/1439901149","heartbeat_front_addr":"172.21.15.123:6831/1439901149","state":["exists","up"]},{"osd":10,"uuid":"5495e09b-7809-47b0-b7e0-b56c452f9f22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6825","nonce":3355349245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6827","nonce":3355349245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6831","nonce":3355349245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3355349245},{"type":"v1","addr":"172.21.15.183:6829","nonce":3355349245}]},"public_addr":"172.21.15.183:6825/3355349245","cluster_addr":"172.21.15.183:6827/3355349245","heartbeat_back_addr":"172.21.15.183:6831/3355349245","heartbeat_front_addr":"172.21.15.183:6829/3355349245","state":["exists","up"]},{"osd":11,"uuid":"03046fef-b16b-42f8-9a19-6c0b78cbfe7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6825","nonce":4291617038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6827","nonce":4291617038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6831","nonce":4291617038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":4291617038},{"type":"v1","addr":"172.21.15.193:6829","nonce":4291617038}]},"public_addr":"172.21.15.193:6825/4291617038","cluster_addr":"172.21.15.193:6827/4291617038","heartbeat_back_addr":"172.21.15.193:6831/4291617038","heartbeat_front_addr":"172.21.15.193:6829/4291617038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:38:39.142529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:05.336911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:30.310967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:39:59.655973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:25.890576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:40:50.778689+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:19.063731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:41:45.424269+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:10.562149+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:42:40.382368+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:06.297731+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:43:32.352172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/1532622003":"2024-09-20T04:46:41.775112+0000","172.21.15.123:0/2833919666":"2024-09-20T04:46:41.775112+0000","172.21.15.123:0/2580747029":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/471028951":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6801/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:6800/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:6800/1532622003":"2024-09-20T04:46:41.775112+0000","172.21.15.123:6801/3323203947":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/3498196650":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/4040401634":"2024-09-20T04:37:13.257785+0000","172.21.15.123:0/2000593686":"2024-09-20T04:36:45.913060+0000","172.21.15.123:6800/2065492981":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/3339335756":"2024-09-20T04:46:41.775112+0000","172.21.15.123:0/1009496020":"2024-09-20T04:36:45.913060+0000","172.21.15.123:0/3244738342":"2024-09-20T04:46:41.775112+0000","172.21.15.123:0/2764863023":"2024-09-20T04:46:41.775112+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:57:33.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.536+0000 7f836de00640 1 -- 172.21.15.123:0/2580064269 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f834c0a3430 msgr2=0x7f834c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:33.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.536+0000 7f836de00640 1 --2- 172.21.15.123:0/2580064269 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f834c0a3430 0x7f834c0a58f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f835402a930 tx=0x7f83540058d0 comp rx=0 tx=0).stop 2024-09-19T04:57:33.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.536+0000 7f836de00640 1 -- 172.21.15.123:0/2580064269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8368101df0 msgr2=0x7f83680744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:33.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.536+0000 7f836de00640 1 --2- 172.21.15.123:0/2580064269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8368101df0 0x7f83680744f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f835800e9c0 tx=0x7f835800ee80 comp rx=0 tx=0).stop 2024-09-19T04:57:33.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.536+0000 7f836de00640 1 -- 172.21.15.123:0/2580064269 shutdown_connections 2024-09-19T04:57:33.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.536+0000 7f836de00640 1 --2- 172.21.15.123:0/2580064269 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f834c0a3430 0x7f834c0a58f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:33.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.536+0000 7f836de00640 1 --2- 172.21.15.123:0/2580064269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8368106f20 0x7f8368074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:33.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.536+0000 7f836de00640 1 --2- 172.21.15.123:0/2580064269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83681027a0 0x7f8368074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:33.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.536+0000 7f836de00640 1 --2- 172.21.15.123:0/2580064269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8368101df0 0x7f83680744f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:33.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.536+0000 7f836de00640 1 -- 172.21.15.123:0/2580064269 >> 172.21.15.123:0/2580064269 conn(0x7f83680fd8b0 msgr2=0x7f83680feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:33.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.536+0000 7f836de00640 1 -- 172.21.15.123:0/2580064269 shutdown_connections 2024-09-19T04:57:33.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:33.536+0000 7f836de00640 1 -- 172.21.15.123:0/2580064269 wait complete. 2024-09-19T04:57:33.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:33 smithi183 ceph-mon[35333]: pgmap v338: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T04:57:33.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:33 smithi193 ceph-mon[35592]: pgmap v338: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T04:57:34.395 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-19T04:57:34.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:34 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2580064269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:57:34.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:34 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2580064269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:57:34.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:34 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2580064269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:57:34.763 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:35.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.311+0000 7f5289000640 1 -- 172.21.15.123:0/652670882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5284102360 msgr2=0x7f528410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:35.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.311+0000 7f5289000640 1 --2- 172.21.15.123:0/652670882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5284102360 0x7f528410e890 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f527c009cd0 tx=0x7f527c02a8e0 comp rx=0 tx=0).stop 2024-09-19T04:57:35.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.311+0000 7f5289000640 1 -- 172.21.15.123:0/652670882 shutdown_connections 2024-09-19T04:57:35.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.311+0000 7f5289000640 1 --2- 172.21.15.123:0/652670882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5284102360 0x7f528410e890 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:35.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.311+0000 7f5289000640 1 --2- 172.21.15.123:0/652670882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52841019c0 0x7f5284101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:35.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.311+0000 7f5289000640 1 --2- 172.21.15.123:0/652670882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52841079c0 0x7f5284107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:35.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.311+0000 7f5289000640 1 -- 172.21.15.123:0/652670882 >> 172.21.15.123:0/652670882 conn(0x7f52840fd740 msgr2=0x7f52840ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:35.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.311+0000 7f5289000640 1 -- 172.21.15.123:0/652670882 shutdown_connections 2024-09-19T04:57:35.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.311+0000 7f5289000640 1 -- 172.21.15.123:0/652670882 wait complete. 2024-09-19T04:57:35.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.313+0000 7f5289000640 1 Processor -- start 2024-09-19T04:57:35.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.313+0000 7f5289000640 1 -- start start 2024-09-19T04:57:35.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.313+0000 7f5289000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52841019c0 0x7f5284198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:35.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.313+0000 7f5289000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5284102360 0x7f52841987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:35.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.313+0000 7f5289000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52841079c0 0x7f528419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:35.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.313+0000 7f5289000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5284111e10 con 0x7f52841079c0 2024-09-19T04:57:35.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.313+0000 7f5289000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5284111c90 con 0x7f52841019c0 2024-09-19T04:57:35.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.313+0000 7f5289000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5284111f90 con 0x7f5284102360 2024-09-19T04:57:35.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.313+0000 7f5283e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52841079c0 0x7f528419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:35.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.314+0000 7f5283400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52841019c0 0x7f5284198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:35.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.314+0000 7f5282a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5284102360 0x7f52841987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:35.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.314+0000 7f5283400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52841019c0 0x7f5284198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.123:32894/0 (socket says 172.21.15.123:32894) 2024-09-19T04:57:35.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.314+0000 7f5283400640 1 -- 172.21.15.123:0/2597889666 learned_addr learned my addr 172.21.15.123:0/2597889666 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:57:35.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.314+0000 7f5282a00640 1 -- 172.21.15.123:0/2597889666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52841019c0 msgr2=0x7f5284198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:35.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.314+0000 7f5282a00640 1 --2- 172.21.15.123:0/2597889666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52841019c0 0x7f5284198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:35.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.314+0000 7f5282a00640 1 -- 172.21.15.123:0/2597889666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52841079c0 msgr2=0x7f528419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:35.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.314+0000 7f5282a00640 1 --2- 172.21.15.123:0/2597889666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52841079c0 0x7f528419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:35.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.314+0000 7f5282a00640 1 -- 172.21.15.123:0/2597889666 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528419d240 con 0x7f5284102360 2024-09-19T04:57:35.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.314+0000 7f5283e00640 1 --2- 172.21.15.123:0/2597889666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52841079c0 0x7f528419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:57:35.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.314+0000 7f5283400640 1 --2- 172.21.15.123:0/2597889666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52841019c0 0x7f5284198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:57:35.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.314+0000 7f5282a00640 1 --2- 172.21.15.123:0/2597889666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5284102360 0x7f52841987d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f527000e9f0 tx=0x7f527000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:35.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.315+0000 7f527be00640 1 -- 172.21.15.123:0/2597889666 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527000ccc0 con 0x7f5284102360 2024-09-19T04:57:35.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.315+0000 7f527be00640 1 -- 172.21.15.123:0/2597889666 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5270010910 con 0x7f5284102360 2024-09-19T04:57:35.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.315+0000 7f5289000640 1 -- 172.21.15.123:0/2597889666 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528419d530 con 0x7f5284102360 2024-09-19T04:57:35.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.315+0000 7f527be00640 1 -- 172.21.15.123:0/2597889666 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270018960 con 0x7f5284102360 2024-09-19T04:57:35.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.315+0000 7f5289000640 1 -- 172.21.15.123:0/2597889666 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52841a4e10 con 0x7f5284102360 2024-09-19T04:57:35.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.316+0000 7f5289000640 1 -- 172.21.15.123:0/2597889666 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5284101e20 con 0x7f5284102360 2024-09-19T04:57:35.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.318+0000 7f527be00640 1 -- 172.21.15.123:0/2597889666 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5270010ac0 con 0x7f5284102360 2024-09-19T04:57:35.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.319+0000 7f527be00640 1 --2- 172.21.15.123:0/2597889666 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f52680a3430 0x7f52680a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:35.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.320+0000 7f527be00640 1 -- 172.21.15.123:0/2597889666 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f52700d0070 con 0x7f5284102360 2024-09-19T04:57:35.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.320+0000 7f5283400640 1 --2- 172.21.15.123:0/2597889666 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f52680a3430 0x7f52680a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:35.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.320+0000 7f5283400640 1 --2- 172.21.15.123:0/2597889666 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f52680a3430 0x7f52680a58f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5274011490 tx=0x7f5274005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:35.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.324+0000 7f527be00640 1 -- 172.21.15.123:0/2597889666 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f527009c3a0 con 0x7f5284102360 2024-09-19T04:57:35.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.533+0000 7f5289000640 1 -- 172.21.15.123:0/2597889666 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f528402d7c0 con 0x7f5284102360 2024-09-19T04:57:35.535 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:35 smithi123 ceph-mon[30511]: pgmap v339: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-09-19T04:57:35.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.542+0000 7f527be00640 1 -- 172.21.15.123:0/2597889666 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) ==== 161+0+0 (secure 0 0 0) 0x7f52700a06a0 con 0x7f5284102360 2024-09-19T04:57:35.544 INFO:teuthology.orchestra.run.smithi123.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T04:57:35.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.545+0000 7f5289000640 1 -- 172.21.15.123:0/2597889666 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f52680a3430 msgr2=0x7f52680a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:35.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.545+0000 7f5289000640 1 --2- 172.21.15.123:0/2597889666 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f52680a3430 0x7f52680a58f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5274011490 tx=0x7f5274005f90 comp rx=0 tx=0).stop 2024-09-19T04:57:35.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.545+0000 7f5289000640 1 -- 172.21.15.123:0/2597889666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5284102360 msgr2=0x7f52841987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:35.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.545+0000 7f5289000640 1 --2- 172.21.15.123:0/2597889666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5284102360 0x7f52841987d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f527000e9f0 tx=0x7f527000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:57:35.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.545+0000 7f5289000640 1 -- 172.21.15.123:0/2597889666 shutdown_connections 2024-09-19T04:57:35.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.545+0000 7f5289000640 1 --2- 172.21.15.123:0/2597889666 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f52680a3430 0x7f52680a58f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:35.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.545+0000 7f5289000640 1 --2- 172.21.15.123:0/2597889666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52841079c0 0x7f528419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:35.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.545+0000 7f5289000640 1 --2- 172.21.15.123:0/2597889666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5284102360 0x7f52841987d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:35.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.545+0000 7f5289000640 1 --2- 172.21.15.123:0/2597889666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52841019c0 0x7f5284198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:35.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.545+0000 7f5289000640 1 -- 172.21.15.123:0/2597889666 >> 172.21.15.123:0/2597889666 conn(0x7f52840fd740 msgr2=0x7f52840fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:35.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.546+0000 7f5289000640 1 -- 172.21.15.123:0/2597889666 shutdown_connections 2024-09-19T04:57:35.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:35.546+0000 7f5289000640 1 -- 172.21.15.123:0/2597889666 wait complete. 2024-09-19T04:57:35.580 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:35 smithi193 ceph-mon[35592]: pgmap v339: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-09-19T04:57:35.580 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:35 smithi183 ceph-mon[35333]: pgmap v339: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-09-19T04:57:35.850 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:57:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-b[65203]: ignoring --setuser ceph since I am not root 2024-09-19T04:57:35.850 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:57:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-b[65203]: ignoring --setgroup ceph since I am not root 2024-09-19T04:57:35.850 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:57:35 smithi183 ceph-mds[65207]: -- 172.21.15.183:0/2315209772 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561e531b7c20 con 0x561e53195000 2024-09-19T04:57:35.850 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:57:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-b[65203]: starting mds.b at 2024-09-19T04:57:35.851 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:57:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-e[64601]: ignoring --setuser ceph since I am not root 2024-09-19T04:57:35.851 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:57:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-e[64601]: ignoring --setgroup ceph since I am not root 2024-09-19T04:57:35.851 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:57:35 smithi183 ceph-mds[64605]: -- 172.21.15.183:0/3426453640 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f62ff3c20 con 0x561f62fd1000 2024-09-19T04:57:35.851 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:57:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-e[64601]: starting mds.e at 2024-09-19T04:57:35.851 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:57:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-k[64288]: ignoring --setuser ceph since I am not root 2024-09-19T04:57:35.851 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:57:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-k[64288]: ignoring --setgroup ceph since I am not root 2024-09-19T04:57:35.851 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:57:35 smithi183 ceph-mds[64292]: -- 172.21.15.183:0/2124472516 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b5232d860 con 0x564b5230b000 2024-09-19T04:57:35.851 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:57:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-k[64288]: starting mds.k at 2024-09-19T04:57:35.852 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:57:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-h[64901]: ignoring --setuser ceph since I am not root 2024-09-19T04:57:35.852 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:57:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-h[64901]: ignoring --setgroup ceph since I am not root 2024-09-19T04:57:35.852 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:57:35 smithi183 ceph-mds[64905]: -- 172.21.15.183:0/1531585348 <== mon.0 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5567801ab860 con 0x556780189400 2024-09-19T04:57:35.852 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:57:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-h[64901]: starting mds.h at 2024-09-19T04:57:35.914 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-a[30507]: 2024-09-19T04:57:35.534+0000 7f4233c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:57:35.914 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:57:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-a[70907]: ignoring --setuser ceph since I am not root 2024-09-19T04:57:35.914 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:57:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-a[70907]: ignoring --setgroup ceph since I am not root 2024-09-19T04:57:35.914 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:57:35 smithi123 ceph-mds[70911]: -- 172.21.15.123:0/4033912576 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55798a4c3860 con 0x55798a4a1000 2024-09-19T04:57:35.914 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:57:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-a[70907]: starting mds.a at 2024-09-19T04:57:35.914 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:57:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-d[69697]: ignoring --setuser ceph since I am not root 2024-09-19T04:57:35.914 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:57:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-d[69697]: ignoring --setgroup ceph since I am not root 2024-09-19T04:57:35.914 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:57:35 smithi123 ceph-mds[69707]: -- 172.21.15.123:0/2702586249 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d14e4f860 con 0x563d14e2d000 2024-09-19T04:57:35.915 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:57:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-d[69697]: starting mds.d at 2024-09-19T04:57:35.990 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 04:57:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-c[65683]: ignoring --setuser ceph since I am not root 2024-09-19T04:57:35.991 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 04:57:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-c[65683]: ignoring --setgroup ceph since I am not root 2024-09-19T04:57:35.991 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 04:57:35 smithi193 ceph-mds[65687]: -- 172.21.15.193:0/921245796 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a99de3d860 con 0x55a99de1b400 2024-09-19T04:57:35.991 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 04:57:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-c[65683]: starting mds.c at 2024-09-19T04:57:35.991 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 04:57:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-f[65382]: ignoring --setuser ceph since I am not root 2024-09-19T04:57:35.991 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 04:57:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-f[65382]: ignoring --setgroup ceph since I am not root 2024-09-19T04:57:35.991 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 04:57:35 smithi193 ceph-mds[65386]: -- 172.21.15.193:0/4113403046 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5637f8fe1c20 con 0x5637f8fbe800 2024-09-19T04:57:35.991 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 04:57:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-f[65382]: starting mds.f at 2024-09-19T04:57:35.992 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 04:57:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-i[65988]: ignoring --setuser ceph since I am not root 2024-09-19T04:57:35.992 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 04:57:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-i[65988]: ignoring --setgroup ceph since I am not root 2024-09-19T04:57:35.992 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 04:57:35 smithi193 ceph-mds[65992]: -- 172.21.15.193:0/557856178 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563eaed85c20 con 0x563eaed62800 2024-09-19T04:57:35.992 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 04:57:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-i[65988]: starting mds.i at 2024-09-19T04:57:35.992 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 04:57:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-l[65064]: ignoring --setuser ceph since I am not root 2024-09-19T04:57:35.992 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 04:57:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-l[65064]: ignoring --setgroup ceph since I am not root 2024-09-19T04:57:35.992 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 04:57:35 smithi193 ceph-mds[65068]: -- 172.21.15.193:0/652994482 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560e9ed7f860 con 0x560e9ed5c800 2024-09-19T04:57:35.992 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 04:57:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-l[65064]: starting mds.l at 2024-09-19T04:57:36.312 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T04:57:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:36 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/2597889666' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T04:57:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:36 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T04:57:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:36 smithi183 ceph-mon[35333]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T04:57:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:36 smithi183 ceph-mon[35333]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:57:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:36 smithi183 ceph-mon[35333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T04:57:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:36 smithi183 ceph-mon[35333]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T04:57:36.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:36 smithi183 ceph-mon[35333]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T04:57:36.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:36 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/2597889666' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T04:57:36.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:36 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T04:57:36.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:36 smithi123 ceph-mon[30511]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T04:57:36.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:36 smithi123 ceph-mon[30511]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:57:36.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:36 smithi123 ceph-mon[30511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T04:57:36.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:36 smithi123 ceph-mon[30511]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T04:57:36.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:36 smithi123 ceph-mon[30511]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T04:57:36.684 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:36.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:36 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/2597889666' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T04:57:36.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:36 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T04:57:36.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:36 smithi193 ceph-mon[35592]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T04:57:36.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:36 smithi193 ceph-mon[35592]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:57:36.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:36 smithi193 ceph-mon[35592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T04:57:36.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:36 smithi193 ceph-mon[35592]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T04:57:36.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:36 smithi193 ceph-mon[35592]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T04:57:37.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.255+0000 7f3c51600640 1 -- 172.21.15.123:0/1200006195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c4c076720 msgr2=0x7f3c4c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:37.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.255+0000 7f3c51600640 1 --2- 172.21.15.123:0/1200006195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c4c076720 0x7f3c4c076b00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009900 tx=0x7f3c3c02a400 comp rx=0 tx=0).stop 2024-09-19T04:57:37.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.256+0000 7f3c51600640 1 -- 172.21.15.123:0/1200006195 shutdown_connections 2024-09-19T04:57:37.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.256+0000 7f3c51600640 1 --2- 172.21.15.123:0/1200006195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c4c112a30 0x7f3c4c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:37.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.256+0000 7f3c51600640 1 --2- 172.21.15.123:0/1200006195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c4c077040 0x7f3c4c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:37.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.256+0000 7f3c51600640 1 --2- 172.21.15.123:0/1200006195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c4c076720 0x7f3c4c076b00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:37.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.256+0000 7f3c51600640 1 -- 172.21.15.123:0/1200006195 >> 172.21.15.123:0/1200006195 conn(0x7f3c4c0ffa90 msgr2=0x7f3c4c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:37.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.256+0000 7f3c51600640 1 -- 172.21.15.123:0/1200006195 shutdown_connections 2024-09-19T04:57:37.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.256+0000 7f3c51600640 1 -- 172.21.15.123:0/1200006195 wait complete. 2024-09-19T04:57:37.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.257+0000 7f3c51600640 1 Processor -- start 2024-09-19T04:57:37.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.257+0000 7f3c51600640 1 -- start start 2024-09-19T04:57:37.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.257+0000 7f3c51600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c4c076720 0x7f3c4c1a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:37.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.258+0000 7f3c51600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c4c077040 0x7f3c4c1a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:37.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.258+0000 7f3c51600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c4c112a30 0x7f3c4c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:37.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.258+0000 7f3c51600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c118610 con 0x7f3c4c112a30 2024-09-19T04:57:37.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.258+0000 7f3c51600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c118490 con 0x7f3c4c076720 2024-09-19T04:57:37.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.258+0000 7f3c51600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c118790 con 0x7f3c4c077040 2024-09-19T04:57:37.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.258+0000 7f3c4b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c4c112a30 0x7f3c4c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:37.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.258+0000 7f3c4b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c4c112a30 0x7f3c4c19e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45394/0 (socket says 172.21.15.123:45394) 2024-09-19T04:57:37.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.258+0000 7f3c4b400640 1 -- 172.21.15.123:0/686708613 learned_addr learned my addr 172.21.15.123:0/686708613 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:57:37.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.258+0000 7f3c4aa00640 1 --2- 172.21.15.123:0/686708613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c4c076720 0x7f3c4c1a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:37.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.258+0000 7f3c42000640 1 --2- 172.21.15.123:0/686708613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c4c077040 0x7f3c4c1a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:37.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.258+0000 7f3c4b400640 1 -- 172.21.15.123:0/686708613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c4c077040 msgr2=0x7f3c4c1a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:37.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.258+0000 7f3c4b400640 1 --2- 172.21.15.123:0/686708613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c4c077040 0x7f3c4c1a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:37.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.258+0000 7f3c4b400640 1 -- 172.21.15.123:0/686708613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c4c076720 msgr2=0x7f3c4c1a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:37.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.258+0000 7f3c4b400640 1 --2- 172.21.15.123:0/686708613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c4c076720 0x7f3c4c1a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:37.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.258+0000 7f3c4b400640 1 -- 172.21.15.123:0/686708613 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4c19eff0 con 0x7f3c4c112a30 2024-09-19T04:57:37.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.259+0000 7f3c4b400640 1 --2- 172.21.15.123:0/686708613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c4c112a30 0x7f3c4c19e8c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400eb10 tx=0x7f3c4400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:37.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.259+0000 7f3c43e00640 1 -- 172.21.15.123:0/686708613 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44004b20 con 0x7f3c4c112a30 2024-09-19T04:57:37.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.259+0000 7f3c51600640 1 -- 172.21.15.123:0/686708613 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4c19f2e0 con 0x7f3c4c112a30 2024-09-19T04:57:37.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.259+0000 7f3c43e00640 1 -- 172.21.15.123:0/686708613 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c44004cc0 con 0x7f3c4c112a30 2024-09-19T04:57:37.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.259+0000 7f3c43e00640 1 -- 172.21.15.123:0/686708613 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44010790 con 0x7f3c4c112a30 2024-09-19T04:57:37.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.259+0000 7f3c51600640 1 -- 172.21.15.123:0/686708613 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4c1ab5d0 con 0x7f3c4c112a30 2024-09-19T04:57:37.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.260+0000 7f3c51600640 1 -- 172.21.15.123:0/686708613 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c4c076b00 con 0x7f3c4c112a30 2024-09-19T04:57:37.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.262+0000 7f3c43e00640 1 -- 172.21.15.123:0/686708613 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3c44005020 con 0x7f3c4c112a30 2024-09-19T04:57:37.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.263+0000 7f3c43e00640 1 --2- 172.21.15.123:0/686708613 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f3c300a33b0 0x7f3c300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:37.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.263+0000 7f3c43e00640 1 -- 172.21.15.123:0/686708613 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8899+0+0 (secure 0 0 0) 0x7f3c440d05d0 con 0x7f3c4c112a30 2024-09-19T04:57:37.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.263+0000 7f3c4aa00640 1 --2- 172.21.15.123:0/686708613 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f3c300a33b0 0x7f3c300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:37.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.264+0000 7f3c4aa00640 1 --2- 172.21.15.123:0/686708613 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f3c300a33b0 0x7f3c300a5870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c006010 tx=0x7f3c3c000bc0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:37.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.267+0000 7f3c43e00640 1 -- 172.21.15.123:0/686708613 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3c4409cdd0 con 0x7f3c4c112a30 2024-09-19T04:57:37.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.478+0000 7f3c51600640 1 -- 172.21.15.123:0/686708613 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3c4c1ab860 con 0x7f3c4c112a30 2024-09-19T04:57:37.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.549+0000 7f3c43e00640 1 -- 172.21.15.123:0/686708613 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) ==== 104+0+0 (secure 0 0 0) 0x7f3c4409c6d0 con 0x7f3c4c112a30 2024-09-19T04:57:37.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.552+0000 7f3c51600640 1 -- 172.21.15.123:0/686708613 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f3c300a33b0 msgr2=0x7f3c300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:37.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.552+0000 7f3c51600640 1 --2- 172.21.15.123:0/686708613 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f3c300a33b0 0x7f3c300a5870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c006010 tx=0x7f3c3c000bc0 comp rx=0 tx=0).stop 2024-09-19T04:57:37.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.552+0000 7f3c51600640 1 -- 172.21.15.123:0/686708613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c4c112a30 msgr2=0x7f3c4c19e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:37.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.552+0000 7f3c51600640 1 --2- 172.21.15.123:0/686708613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c4c112a30 0x7f3c4c19e8c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400eb10 tx=0x7f3c4400c2d0 comp rx=0 tx=0).stop 2024-09-19T04:57:37.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.552+0000 7f3c51600640 1 -- 172.21.15.123:0/686708613 shutdown_connections 2024-09-19T04:57:37.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.552+0000 7f3c51600640 1 --2- 172.21.15.123:0/686708613 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7f3c300a33b0 0x7f3c300a5870 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:37.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.552+0000 7f3c51600640 1 --2- 172.21.15.123:0/686708613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c4c112a30 0x7f3c4c19e8c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:37.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.552+0000 7f3c51600640 1 --2- 172.21.15.123:0/686708613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c4c077040 0x7f3c4c1a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:37.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.552+0000 7f3c51600640 1 --2- 172.21.15.123:0/686708613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c4c076720 0x7f3c4c1a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:37.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.552+0000 7f3c51600640 1 -- 172.21.15.123:0/686708613 >> 172.21.15.123:0/686708613 conn(0x7f3c4c0ffa90 msgr2=0x7f3c4c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:37.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.552+0000 7f3c51600640 1 -- 172.21.15.123:0/686708613 shutdown_connections 2024-09-19T04:57:37.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:37.552+0000 7f3c51600640 1 -- 172.21.15.123:0/686708613 wait complete. 2024-09-19T04:57:37.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: pgmap v341: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-19T04:57:37.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.193:6832/2112304879,v1:172.21.15.193:6833/2112304879] up:boot 2024-09-19T04:57:37.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.183:6832/790699989,v1:172.21.15.183:6834/790699989] up:boot 2024-09-19T04:57:37.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.183:6833/167323612,v1:172.21.15.183:6835/167323612] up:boot 2024-09-19T04:57:37.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.193:6834/3350346339,v1:172.21.15.193:6835/3350346339] up:boot 2024-09-19T04:57:37.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.193:6836/2480335003,v1:172.21.15.193:6837/2480335003] up:boot 2024-09-19T04:57:37.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.183:6836/2801316322,v1:172.21.15.183:6837/2801316322] up:boot 2024-09-19T04:57:37.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.183:6838/879374960,v1:172.21.15.183:6839/879374960] up:boot 2024-09-19T04:57:37.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:57:37.741 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.193:6838/3761950110,v1:172.21.15.193:6839/3761950110] up:boot 2024-09-19T04:57:37.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.123:6836/1183786309,v1:172.21.15.123:6840/1183786309] up:boot 2024-09-19T04:57:37.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: mds.? [v2:172.21.15.123:6837/3495700940,v1:172.21.15.123:6841/3495700940] up:boot 2024-09-19T04:57:37.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T04:57:37.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:57:37.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:57:37.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:57:37.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:57:37.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:57:37.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:57:37.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:57:37.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:57:37.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:57:37.742 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:37 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/686708613' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:57:37.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: pgmap v341: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-19T04:57:37.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.193:6832/2112304879,v1:172.21.15.193:6833/2112304879] up:boot 2024-09-19T04:57:37.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.183:6832/790699989,v1:172.21.15.183:6834/790699989] up:boot 2024-09-19T04:57:37.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.183:6833/167323612,v1:172.21.15.183:6835/167323612] up:boot 2024-09-19T04:57:37.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.193:6834/3350346339,v1:172.21.15.193:6835/3350346339] up:boot 2024-09-19T04:57:37.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.193:6836/2480335003,v1:172.21.15.193:6837/2480335003] up:boot 2024-09-19T04:57:37.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.183:6836/2801316322,v1:172.21.15.183:6837/2801316322] up:boot 2024-09-19T04:57:37.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.183:6838/879374960,v1:172.21.15.183:6839/879374960] up:boot 2024-09-19T04:57:37.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:57:37.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.193:6838/3761950110,v1:172.21.15.193:6839/3761950110] up:boot 2024-09-19T04:57:37.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.123:6836/1183786309,v1:172.21.15.123:6840/1183786309] up:boot 2024-09-19T04:57:37.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: mds.? [v2:172.21.15.123:6837/3495700940,v1:172.21.15.123:6841/3495700940] up:boot 2024-09-19T04:57:37.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T04:57:37.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:57:37.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:57:37.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:57:37.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:57:37.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:57:37.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:57:37.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:57:37.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:57:37.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:57:37.852 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:37 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/686708613' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:57:37.893 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: pgmap v341: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-19T04:57:37.893 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.193:6832/2112304879,v1:172.21.15.193:6833/2112304879] up:boot 2024-09-19T04:57:37.893 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.183:6832/790699989,v1:172.21.15.183:6834/790699989] up:boot 2024-09-19T04:57:37.893 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.183:6833/167323612,v1:172.21.15.183:6835/167323612] up:boot 2024-09-19T04:57:37.893 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.193:6834/3350346339,v1:172.21.15.193:6835/3350346339] up:boot 2024-09-19T04:57:37.893 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.193:6836/2480335003,v1:172.21.15.193:6837/2480335003] up:boot 2024-09-19T04:57:37.893 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.183:6836/2801316322,v1:172.21.15.183:6837/2801316322] up:boot 2024-09-19T04:57:37.893 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.183:6838/879374960,v1:172.21.15.183:6839/879374960] up:boot 2024-09-19T04:57:37.893 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:57:37.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.193:6838/3761950110,v1:172.21.15.193:6839/3761950110] up:boot 2024-09-19T04:57:37.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.123:6836/1183786309,v1:172.21.15.123:6840/1183786309] up:boot 2024-09-19T04:57:37.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: mds.? [v2:172.21.15.123:6837/3495700940,v1:172.21.15.123:6841/3495700940] up:boot 2024-09-19T04:57:37.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T04:57:37.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:57:37.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:57:37.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:57:37.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:57:37.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:57:37.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:57:37.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:57:37.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:57:37.895 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: from='mgr.25133 172.21.15.123:0/926924271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:57:37.895 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:37 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/686708613' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:57:38.314 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T04:57:38.682 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:38.820 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:38 smithi123 ceph-mon[30511]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T04:57:38.820 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:38 smithi123 ceph-mon[30511]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T04:57:38.821 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:38 smithi123 ceph-mon[30511]: Cluster is now healthy 2024-09-19T04:57:38.821 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:38 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/686708613' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:57:38.821 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:38 smithi123 ceph-mon[30511]: fsmap 12 up:standby 2024-09-19T04:57:38.841 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:38 smithi183 ceph-mon[35333]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T04:57:38.841 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:38 smithi183 ceph-mon[35333]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T04:57:38.842 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:38 smithi183 ceph-mon[35333]: Cluster is now healthy 2024-09-19T04:57:38.842 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:38 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/686708613' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:57:38.842 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:38 smithi183 ceph-mon[35333]: fsmap 12 up:standby 2024-09-19T04:57:38.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:38 smithi193 ceph-mon[35592]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T04:57:38.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:38 smithi193 ceph-mon[35592]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T04:57:38.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:38 smithi193 ceph-mon[35592]: Cluster is now healthy 2024-09-19T04:57:38.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:38 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/686708613' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:57:38.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:38 smithi193 ceph-mon[35592]: fsmap 12 up:standby 2024-09-19T04:57:39.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.266+0000 7faad1600640 1 -- 172.21.15.123:0/4102382670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faacc101df0 msgr2=0x7faacc1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:39.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.266+0000 7faad1600640 1 --2- 172.21.15.123:0/4102382670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faacc101df0 0x7faacc1021d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7faabc009900 tx=0x7faabc02a410 comp rx=0 tx=0).stop 2024-09-19T04:57:39.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.266+0000 7faad1600640 1 -- 172.21.15.123:0/4102382670 shutdown_connections 2024-09-19T04:57:39.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.266+0000 7faad1600640 1 --2- 172.21.15.123:0/4102382670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faacc106f20 0x7faacc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:39.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.266+0000 7faad1600640 1 --2- 172.21.15.123:0/4102382670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faacc1027a0 0x7faacc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:39.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.266+0000 7faad1600640 1 --2- 172.21.15.123:0/4102382670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faacc101df0 0x7faacc1021d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:39.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.266+0000 7faad1600640 1 -- 172.21.15.123:0/4102382670 >> 172.21.15.123:0/4102382670 conn(0x7faacc0fd8b0 msgr2=0x7faacc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:39.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.267+0000 7faad1600640 1 -- 172.21.15.123:0/4102382670 shutdown_connections 2024-09-19T04:57:39.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.267+0000 7faad1600640 1 -- 172.21.15.123:0/4102382670 wait complete. 2024-09-19T04:57:39.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.268+0000 7faad1600640 1 Processor -- start 2024-09-19T04:57:39.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.268+0000 7faad1600640 1 -- start start 2024-09-19T04:57:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.268+0000 7faad1600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faacc101df0 0x7faacc198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.268+0000 7faad1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faacc1027a0 0x7faacc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.268+0000 7faad1600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faacc106f20 0x7faacc19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.268+0000 7faad1600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faacc111f80 con 0x7faacc1027a0 2024-09-19T04:57:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.268+0000 7faad1600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faacc111e00 con 0x7faacc106f20 2024-09-19T04:57:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.268+0000 7faad1600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faacc112100 con 0x7faacc101df0 2024-09-19T04:57:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.269+0000 7faac3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faacc1027a0 0x7faacc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.269+0000 7faac3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faacc1027a0 0x7faacc198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45410/0 (socket says 172.21.15.123:45410) 2024-09-19T04:57:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.269+0000 7faac3e00640 1 -- 172.21.15.123:0/236758756 learned_addr learned my addr 172.21.15.123:0/236758756 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:57:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.269+0000 7faacaa00640 1 --2- 172.21.15.123:0/236758756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faacc101df0 0x7faacc198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.269+0000 7faacb400640 1 --2- 172.21.15.123:0/236758756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faacc106f20 0x7faacc19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.269+0000 7faac3e00640 1 -- 172.21.15.123:0/236758756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faacc101df0 msgr2=0x7faacc198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.269+0000 7faac3e00640 1 --2- 172.21.15.123:0/236758756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faacc101df0 0x7faacc198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.269+0000 7faac3e00640 1 -- 172.21.15.123:0/236758756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faacc106f20 msgr2=0x7faacc19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.269+0000 7faac3e00640 1 --2- 172.21.15.123:0/236758756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faacc106f20 0x7faacc19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.269+0000 7faac3e00640 1 -- 172.21.15.123:0/236758756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faacc19d3d0 con 0x7faacc1027a0 2024-09-19T04:57:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.269+0000 7faac3e00640 1 --2- 172.21.15.123:0/236758756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faacc1027a0 0x7faacc198960 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7faab400e9f0 tx=0x7faab400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.270+0000 7faac3400640 1 -- 172.21.15.123:0/236758756 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab400cd80 con 0x7faacc1027a0 2024-09-19T04:57:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.270+0000 7faac3400640 1 -- 172.21.15.123:0/236758756 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab4005500 con 0x7faacc1027a0 2024-09-19T04:57:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.270+0000 7faad1600640 1 -- 172.21.15.123:0/236758756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faacc19d6c0 con 0x7faacc1027a0 2024-09-19T04:57:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.270+0000 7faac3400640 1 -- 172.21.15.123:0/236758756 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab4010690 con 0x7faacc1027a0 2024-09-19T04:57:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.270+0000 7faad1600640 1 -- 172.21.15.123:0/236758756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faacc1a4fa0 con 0x7faacc1027a0 2024-09-19T04:57:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.271+0000 7faad1600640 1 -- 172.21.15.123:0/236758756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faacc1021d0 con 0x7faacc1027a0 2024-09-19T04:57:39.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.273+0000 7faac3400640 1 -- 172.21.15.123:0/236758756 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7faab40046c0 con 0x7faacc1027a0 2024-09-19T04:57:39.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.274+0000 7faac3400640 1 --2- 172.21.15.123:0/236758756 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7faab00a33b0 0x7faab00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:39.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.275+0000 7faac3400640 1 -- 172.21.15.123:0/236758756 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8899+0+0 (secure 0 0 0) 0x7faab40d04f0 con 0x7faacc1027a0 2024-09-19T04:57:39.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.275+0000 7faacaa00640 1 --2- 172.21.15.123:0/236758756 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7faab00a33b0 0x7faab00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:39.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.275+0000 7faacaa00640 1 --2- 172.21.15.123:0/236758756 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7faab00a33b0 0x7faab00a5870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faabc009230 tx=0x7faabc0058d0 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:39.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.277+0000 7faac3400640 1 -- 172.21.15.123:0/236758756 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faab4054210 con 0x7faacc1027a0 2024-09-19T04:57:39.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.453+0000 7faad1600640 1 -- 172.21.15.123:0/236758756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7faacc199780 con 0x7faacc1027a0 2024-09-19T04:57:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.552+0000 7faac3400640 1 -- 172.21.15.123:0/236758756 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) ==== 181+0+0 (secure 0 0 0) 0x7faacc199780 con 0x7faacc1027a0 2024-09-19T04:57:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' removed 2024-09-19T04:57:39.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.555+0000 7faad1600640 1 -- 172.21.15.123:0/236758756 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7faab00a33b0 msgr2=0x7faab00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:39.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.555+0000 7faad1600640 1 --2- 172.21.15.123:0/236758756 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7faab00a33b0 0x7faab00a5870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faabc009230 tx=0x7faabc0058d0 comp rx=0 tx=0).stop 2024-09-19T04:57:39.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.555+0000 7faad1600640 1 -- 172.21.15.123:0/236758756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faacc1027a0 msgr2=0x7faacc198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:39.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.555+0000 7faad1600640 1 --2- 172.21.15.123:0/236758756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faacc1027a0 0x7faacc198960 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7faab400e9f0 tx=0x7faab400edf0 comp rx=0 tx=0).stop 2024-09-19T04:57:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.555+0000 7faad1600640 1 -- 172.21.15.123:0/236758756 shutdown_connections 2024-09-19T04:57:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.555+0000 7faad1600640 1 --2- 172.21.15.123:0/236758756 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7faab00a33b0 0x7faab00a5870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.555+0000 7faad1600640 1 --2- 172.21.15.123:0/236758756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faacc106f20 0x7faacc19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.555+0000 7faad1600640 1 --2- 172.21.15.123:0/236758756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faacc1027a0 0x7faacc198960 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.555+0000 7faad1600640 1 --2- 172.21.15.123:0/236758756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faacc101df0 0x7faacc198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.555+0000 7faad1600640 1 -- 172.21.15.123:0/236758756 >> 172.21.15.123:0/236758756 conn(0x7faacc0fd8b0 msgr2=0x7faacc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.556+0000 7faad1600640 1 -- 172.21.15.123:0/236758756 shutdown_connections 2024-09-19T04:57:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:39.556+0000 7faad1600640 1 -- 172.21.15.123:0/236758756 wait complete. 2024-09-19T04:57:39.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:39 smithi183 ceph-mon[35333]: pgmap v342: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-19T04:57:39.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:39 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/236758756' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:57:39.910 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:39 smithi123 ceph-mon[30511]: pgmap v342: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-19T04:57:39.910 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:39 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/236758756' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:57:39.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:39 smithi193 ceph-mon[35592]: pgmap v342: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-19T04:57:39.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:39 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/236758756' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:57:40.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 911fa052-7640-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T04:57:40.698 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:40.836 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:40 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/236758756' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:57:40.837 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:40 smithi123 ceph-mon[30511]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T04:57:40.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:40 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/236758756' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:57:40.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:40 smithi183 ceph-mon[35333]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T04:57:40.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:40 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/236758756' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:57:40.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:40 smithi193 ceph-mon[35592]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T04:57:41.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.334+0000 7fe693400640 1 -- 172.21.15.123:0/3142550308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe68c106f10 msgr2=0x7fe68c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:41.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.334+0000 7fe693400640 1 --2- 172.21.15.123:0/3142550308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe68c106f10 0x7fe68c10ea90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe684009ca0 tx=0x7fe68402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:57:41.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.334+0000 7fe693400640 1 -- 172.21.15.123:0/3142550308 shutdown_connections 2024-09-19T04:57:41.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.334+0000 7fe693400640 1 --2- 172.21.15.123:0/3142550308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe68c106f10 0x7fe68c10ea90 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:41.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.334+0000 7fe693400640 1 --2- 172.21.15.123:0/3142550308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe68c102790 0x7fe68c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:41.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.334+0000 7fe693400640 1 --2- 172.21.15.123:0/3142550308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe68c101de0 0x7fe68c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:41.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.334+0000 7fe693400640 1 -- 172.21.15.123:0/3142550308 >> 172.21.15.123:0/3142550308 conn(0x7fe68c0fd8c0 msgr2=0x7fe68c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:41.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.334+0000 7fe693400640 1 -- 172.21.15.123:0/3142550308 shutdown_connections 2024-09-19T04:57:41.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.334+0000 7fe693400640 1 -- 172.21.15.123:0/3142550308 wait complete. 2024-09-19T04:57:41.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.336+0000 7fe693400640 1 Processor -- start 2024-09-19T04:57:41.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.336+0000 7fe693400640 1 -- start start 2024-09-19T04:57:41.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.336+0000 7fe693400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe68c101de0 0x7fe68c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:41.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.337+0000 7fe693400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe68c102790 0x7fe68c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:41.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.337+0000 7fe693400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe68c106f10 0x7fe68c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:41.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.337+0000 7fe693400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c111fc0 con 0x7fe68c101de0 2024-09-19T04:57:41.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.337+0000 7fe693400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c111e40 con 0x7fe68c106f10 2024-09-19T04:57:41.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.337+0000 7fe693400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c112140 con 0x7fe68c102790 2024-09-19T04:57:41.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.337+0000 7fe690a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe68c101de0 0x7fe68c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:41.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.337+0000 7fe690a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe68c101de0 0x7fe68c198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45432/0 (socket says 172.21.15.123:45432) 2024-09-19T04:57:41.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.337+0000 7fe690a00640 1 -- 172.21.15.123:0/1029139148 learned_addr learned my addr 172.21.15.123:0/1029139148 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T04:57:41.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.337+0000 7fe68be00640 1 --2- 172.21.15.123:0/1029139148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe68c102790 0x7fe68c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:41.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.337+0000 7fe691400640 1 --2- 172.21.15.123:0/1029139148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe68c106f10 0x7fe68c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:41.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.337+0000 7fe690a00640 1 -- 172.21.15.123:0/1029139148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe68c102790 msgr2=0x7fe68c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:41.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.337+0000 7fe690a00640 1 --2- 172.21.15.123:0/1029139148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe68c102790 0x7fe68c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:41.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.337+0000 7fe690a00640 1 -- 172.21.15.123:0/1029139148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe68c106f10 msgr2=0x7fe68c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:41.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.337+0000 7fe690a00640 1 --2- 172.21.15.123:0/1029139148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe68c106f10 0x7fe68c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:41.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.337+0000 7fe690a00640 1 -- 172.21.15.123:0/1029139148 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68c19d3b0 con 0x7fe68c101de0 2024-09-19T04:57:41.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.338+0000 7fe68be00640 1 --2- 172.21.15.123:0/1029139148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe68c102790 0x7fe68c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:57:41.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.338+0000 7fe690a00640 1 --2- 172.21.15.123:0/1029139148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe68c101de0 0x7fe68c198400 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe68000d770 tx=0x7fe68000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:41.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.338+0000 7fe689600640 1 -- 172.21.15.123:0/1029139148 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680017070 con 0x7fe68c101de0 2024-09-19T04:57:41.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.338+0000 7fe689600640 1 -- 172.21.15.123:0/1029139148 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe680005500 con 0x7fe68c101de0 2024-09-19T04:57:41.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.338+0000 7fe693400640 1 -- 172.21.15.123:0/1029139148 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68c19d6a0 con 0x7fe68c101de0 2024-09-19T04:57:41.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.338+0000 7fe689600640 1 -- 172.21.15.123:0/1029139148 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680004e00 con 0x7fe68c101de0 2024-09-19T04:57:41.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.338+0000 7fe693400640 1 -- 172.21.15.123:0/1029139148 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68c1a4f80 con 0x7fe68c101de0 2024-09-19T04:57:41.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.339+0000 7fe693400640 1 -- 172.21.15.123:0/1029139148 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe68c1021c0 con 0x7fe68c101de0 2024-09-19T04:57:41.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.342+0000 7fe689600640 1 -- 172.21.15.123:0/1029139148 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe680004fa0 con 0x7fe68c101de0 2024-09-19T04:57:41.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.343+0000 7fe689600640 1 --2- 172.21.15.123:0/1029139148 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe6740a33b0 0x7fe6740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:41.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.343+0000 7fe689600640 1 -- 172.21.15.123:0/1029139148 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8403+0+0 (secure 0 0 0) 0x7fe6800cfac0 con 0x7fe68c101de0 2024-09-19T04:57:41.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.344+0000 7fe68be00640 1 --2- 172.21.15.123:0/1029139148 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe6740a33b0 0x7fe6740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:41.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.344+0000 7fe68be00640 1 --2- 172.21.15.123:0/1029139148 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe6740a33b0 0x7fe6740a5870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe68c1088e0 tx=0x7fe678005f90 comp rx=0 tx=0).ready entity=mgr.25133 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.345+0000 7fe689600640 1 -- 172.21.15.123:0/1029139148 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe68009be60 con 0x7fe68c101de0 2024-09-19T04:57:41.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.520+0000 7fe693400640 1 -- 172.21.15.123:0/1029139148 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe68c1a5150 con 0x7fe68c101de0 2024-09-19T04:57:41.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.587+0000 7fe689600640 1 -- 172.21.15.123:0/1029139148 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) ==== 169+0+0 (secure 0 0 0) 0x7fe6800a0160 con 0x7fe68c101de0 2024-09-19T04:57:41.589 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' removed 2024-09-19T04:57:41.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.590+0000 7fe693400640 1 -- 172.21.15.123:0/1029139148 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe6740a33b0 msgr2=0x7fe6740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:41.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.590+0000 7fe693400640 1 --2- 172.21.15.123:0/1029139148 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe6740a33b0 0x7fe6740a5870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe68c1088e0 tx=0x7fe678005f90 comp rx=0 tx=0).stop 2024-09-19T04:57:41.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.590+0000 7fe693400640 1 -- 172.21.15.123:0/1029139148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe68c101de0 msgr2=0x7fe68c198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:41.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.590+0000 7fe693400640 1 --2- 172.21.15.123:0/1029139148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe68c101de0 0x7fe68c198400 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe68000d770 tx=0x7fe68000dc30 comp rx=0 tx=0).stop 2024-09-19T04:57:41.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.591+0000 7fe693400640 1 -- 172.21.15.123:0/1029139148 shutdown_connections 2024-09-19T04:57:41.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.591+0000 7fe693400640 1 --2- 172.21.15.123:0/1029139148 >> [v2:172.21.15.123:6800/1578549821,v1:172.21.15.123:6801/1578549821] conn(0x7fe6740a33b0 0x7fe6740a5870 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:41.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.591+0000 7fe693400640 1 --2- 172.21.15.123:0/1029139148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe68c106f10 0x7fe68c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:41.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.591+0000 7fe693400640 1 --2- 172.21.15.123:0/1029139148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe68c102790 0x7fe68c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:41.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.591+0000 7fe693400640 1 --2- 172.21.15.123:0/1029139148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe68c101de0 0x7fe68c198400 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:41.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.591+0000 7fe693400640 1 -- 172.21.15.123:0/1029139148 >> 172.21.15.123:0/1029139148 conn(0x7fe68c0fd8c0 msgr2=0x7fe68c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:41.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.591+0000 7fe693400640 1 -- 172.21.15.123:0/1029139148 shutdown_connections 2024-09-19T04:57:41.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T04:57:41.591+0000 7fe693400640 1 -- 172.21.15.123:0/1029139148 wait complete. 2024-09-19T04:57:41.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:41 smithi183 ceph-mon[35333]: pgmap v344: 65 pgs: 65 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:57:41.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:41 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1029139148' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:57:41.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:41 smithi123 ceph-mon[30511]: pgmap v344: 65 pgs: 65 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:57:41.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:41 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1029139148' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:57:41.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:41 smithi193 ceph-mon[35592]: pgmap v344: 65 pgs: 65 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:57:41.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:41 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1029139148' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:57:42.432 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:57:42.488 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:57:42.522 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:57:42.552 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T04:57:42.552 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T04:57:42.588 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T04:57:42.605 DEBUG:teuthology.orchestra.run.smithi193:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T04:57:42.622 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T04:57:42.623 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-19T04:57:42.623 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mon.a 2024-09-19T04:57:42.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:42 smithi123 ceph-mon[30511]: from='client.? 172.21.15.123:0/1029139148' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:57:42.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:42 smithi123 ceph-mon[30511]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T04:57:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:42 smithi183 ceph-mon[35333]: from='client.? 172.21.15.123:0/1029139148' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:57:42.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:42 smithi183 ceph-mon[35333]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T04:57:42.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:42 smithi123 systemd[1]: Stopping Ceph mon.a for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:57:42.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:42 smithi193 ceph-mon[35592]: from='client.? 172.21.15.123:0/1029139148' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:57:42.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:42 smithi193 ceph-mon[35592]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T04:57:43.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-a[30507]: 2024-09-19T04:57:43.050+0000 7f423aa00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:57:43.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-a[30507]: 2024-09-19T04:57:43.050+0000 7f423aa00640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-19T04:57:43.427 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:43 smithi123 podman[126667]: 2024-09-19 04:57:43.234958953 +0000 UTC m=+0.315510686 container died 16e47864cb79fff026808b31823e5a40efacf6d794b75f0381bbff58605755b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T04:57:43.916 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:43 smithi123 podman[126667]: 2024-09-19 04:57:43.680605438 +0000 UTC m=+0.761157178 container cleanup 16e47864cb79fff026808b31823e5a40efacf6d794b75f0381bbff58605755b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T04:57:43.916 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:43 smithi123 bash[126667]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-a 2024-09-19T04:57:44.168 INFO:journalctl@ceph.mon.a.smithi123.stdout:Sep 19 04:57:43 smithi123 podman[126680]: 2024-09-19 04:57:43.915442124 +0000 UTC m=+0.668431239 container remove 16e47864cb79fff026808b31823e5a40efacf6d794b75f0381bbff58605755b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T04:57:44.174 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mon.a.service' 2024-09-19T04:57:44.326 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:57:44.326 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-19T04:57:44.326 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-19T04:57:44.326 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mon.b 2024-09-19T04:57:44.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:44 smithi183 systemd[1]: Stopping Ceph mon.b for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:57:45.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:44 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-b[35329]: 2024-09-19T04:57:44.662+0000 7fd8637f0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:57:45.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:44 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-b[35329]: 2024-09-19T04:57:44.662+0000 7fd8637f0640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-19T04:57:45.100 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:44 smithi183 podman[70488]: 2024-09-19 04:57:44.877124283 +0000 UTC m=+0.353569253 container died cb785711cbc83b992774f0c9a5c964122c89785c86d6de7b9b783517597bdda5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T04:57:45.540 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:45 smithi183 podman[70488]: 2024-09-19 04:57:45.279505048 +0000 UTC m=+0.755950014 container cleanup cb785711cbc83b992774f0c9a5c964122c89785c86d6de7b9b783517597bdda5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T04:57:45.540 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:45 smithi183 bash[70488]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-b 2024-09-19T04:57:45.768 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mon.b.service' 2024-09-19T04:57:45.790 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:45 smithi183 podman[70503]: 2024-09-19 04:57:45.539569959 +0000 UTC m=+0.662900193 container remove cb785711cbc83b992774f0c9a5c964122c89785c86d6de7b9b783517597bdda5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T04:57:45.790 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:45 smithi183 systemd[1]: ceph-911fa052-7640-11ef-bceb-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-19T04:57:45.790 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:45 smithi183 systemd[1]: Stopped Ceph mon.b for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:57:45.790 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 19 04:57:45 smithi183 systemd[1]: ceph-911fa052-7640-11ef-bceb-c7b262605968@mon.b.service: Consumed 13.807s CPU time. 2024-09-19T04:57:45.824 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:57:45.825 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-19T04:57:45.825 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-19T04:57:45.825 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mon.c 2024-09-19T04:57:46.157 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:45 smithi193 systemd[1]: Stopping Ceph mon.c for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:57:46.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:46 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-c[35588]: 2024-09-19T04:57:46.155+0000 7f5b9babf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:57:46.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:46 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-c[35588]: 2024-09-19T04:57:46.155+0000 7f5b9babf640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-19T04:57:46.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:46 smithi193 podman[71506]: 2024-09-19 04:57:46.391278505 +0000 UTC m=+0.369151804 container died 8e497ed1e57b31272a65c21f7282f0d4fcc100234cc9dd42d0186fa345ff6ecf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-c, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:57:47.151 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:46 smithi193 podman[71506]: 2024-09-19 04:57:46.840588512 +0000 UTC m=+0.818461816 container cleanup 8e497ed1e57b31272a65c21f7282f0d4fcc100234cc9dd42d0186fa345ff6ecf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T04:57:47.151 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 19 04:57:46 smithi193 bash[71506]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mon-c 2024-09-19T04:57:47.307 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mon.c.service' 2024-09-19T04:57:47.351 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:57:47.351 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-19T04:57:47.351 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-19T04:57:47.351 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mgr.x 2024-09-19T04:57:47.677 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:57:47 smithi123 systemd[1]: Stopping Ceph mgr.x for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:57:48.177 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:57:47 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:57:47.772+0000 7f2a8d800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:57:48.177 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:57:47 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x[30735]: 2024-09-19T04:57:47.772+0000 7f2a8d800640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-19T04:57:48.177 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:57:47 smithi123 podman[126789]: 2024-09-19 04:57:47.856270583 +0000 UTC m=+0.220344578 container died 1f7915704579ef2041c10b4d6a7373e7748ba49237add44504e48b1134adf480 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:57:48.564 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:57:48 smithi123 podman[126789]: 2024-09-19 04:57:48.336626661 +0000 UTC m=+0.700700658 container cleanup 1f7915704579ef2041c10b4d6a7373e7748ba49237add44504e48b1134adf480 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-09-19T04:57:48.564 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:57:48 smithi123 bash[126789]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x 2024-09-19T04:57:48.817 INFO:journalctl@ceph.mgr.x.smithi123.stdout:Sep 19 04:57:48 smithi123 podman[126800]: 2024-09-19 04:57:48.563475138 +0000 UTC m=+0.691090130 container remove 1f7915704579ef2041c10b4d6a7373e7748ba49237add44504e48b1134adf480 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T04:57:48.826 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-19T04:57:48.899 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:57:48.899 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-19T04:57:48.899 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-19T04:57:48.899 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mgr.y 2024-09-19T04:57:49.230 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:57:48 smithi183 systemd[1]: Stopping Ceph mgr.y for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:57:49.559 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:57:49 smithi183 podman[70610]: 2024-09-19 04:57:49.230193377 +0000 UTC m=+0.136166746 container died 3fca84f03bdc21b4321b5ff361f4cdf6e9a6fe0c322441e2fe7e960be0ca7e0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T04:57:49.850 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:57:49 smithi183 podman[70610]: 2024-09-19 04:57:49.5592308 +0000 UTC m=+0.465204158 container cleanup 3fca84f03bdc21b4321b5ff361f4cdf6e9a6fe0c322441e2fe7e960be0ca7e0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2) 2024-09-19T04:57:49.850 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:57:49 smithi183 bash[70610]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y 2024-09-19T04:57:49.850 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:57:49 smithi183 podman[70622]: 2024-09-19 04:57:49.786285345 +0000 UTC m=+0.547994404 container remove 3fca84f03bdc21b4321b5ff361f4cdf6e9a6fe0c322441e2fe7e960be0ca7e0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-19T04:57:49.850 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 19 04:57:49 smithi183 systemd[1]: ceph-911fa052-7640-11ef-bceb-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-19T04:57:50.022 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-19T04:57:50.055 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:57:50.055 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-19T04:57:50.056 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-19T04:57:50.056 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mgr.z 2024-09-19T04:57:50.392 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:57:50 smithi193 systemd[1]: Stopping Ceph mgr.z for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:57:50.705 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:57:50 smithi193 podman[71630]: 2024-09-19 04:57:50.391405976 +0000 UTC m=+0.142658055 container died e97a36cb7ef5d14853dc6b4e7abf74cc317c8885a14ace667c06164408c3e9d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-19T04:57:50.990 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:57:50 smithi193 podman[71630]: 2024-09-19 04:57:50.821063141 +0000 UTC m=+0.572315225 container cleanup e97a36cb7ef5d14853dc6b4e7abf74cc317c8885a14ace667c06164408c3e9d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:57:50.991 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:57:50 smithi193 bash[71630]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z 2024-09-19T04:57:51.286 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-19T04:57:51.307 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:57:51 smithi193 podman[71642]: 2024-09-19 04:57:51.047777828 +0000 UTC m=+0.651219060 container remove e97a36cb7ef5d14853dc6b4e7abf74cc317c8885a14ace667c06164408c3e9d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-19T04:57:51.307 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:57:51 smithi193 systemd[1]: ceph-911fa052-7640-11ef-bceb-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-19T04:57:51.307 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:57:51 smithi193 systemd[1]: ceph-911fa052-7640-11ef-bceb-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-19T04:57:51.307 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:57:51 smithi193 systemd[1]: Stopped Ceph mgr.z for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:57:51.307 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 19 04:57:51 smithi193 systemd[1]: ceph-911fa052-7640-11ef-bceb-c7b262605968@mgr.z.service: Consumed 11.509s CPU time. 2024-09-19T04:57:51.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:57:51.319 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-19T04:57:51.319 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T04:57:51.319 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.0 2024-09-19T04:57:51.677 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:57:51 smithi123 systemd[1]: Stopping Ceph osd.0 for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:57:52.177 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:57:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-0[44032]: 2024-09-19T04:57:51.735+0000 7f90e8000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:57:52.177 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:57:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-0[44032]: 2024-09-19T04:57:51.735+0000 7f90e8000640 -1 osd.0 100 *** Got signal Terminated *** 2024-09-19T04:57:52.177 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:57:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-0[44032]: 2024-09-19T04:57:51.735+0000 7f90e8000640 -1 osd.0 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:57:57.677 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:57:57 smithi123 podman[126910]: 2024-09-19 04:57:57.34150197 +0000 UTC m=+5.736230504 container died 8c1269231e5dbb147114b419ff1e2cb0f9304a8e02fa1c8fac42985d518745ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-0, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-19T04:57:57.952 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:57:57 smithi123 podman[126910]: 2024-09-19 04:57:57.808327295 +0000 UTC m=+6.203055815 container cleanup 8c1269231e5dbb147114b419ff1e2cb0f9304a8e02fa1c8fac42985d518745ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T04:57:57.952 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:57:57 smithi123 bash[126910]: ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-0 2024-09-19T04:57:58.427 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:57:58 smithi123 podman[126930]: 2024-09-19 04:57:58.043544056 +0000 UTC m=+0.693981706 container remove 8c1269231e5dbb147114b419ff1e2cb0f9304a8e02fa1c8fac42985d518745ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-19T04:57:58.881 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:57:58 smithi123 podman[127015]: 2024-09-19 04:57:58.495779878 +0000 UTC m=+0.027447610 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:57:58.881 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:57:58 smithi123 podman[127015]: 2024-09-19 04:57:58.63606658 +0000 UTC m=+0.167734318 container create 61a46b462a865f1f58f04b3a4901330ed9d96365744ecbcd944558a28affc6cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T04:57:58.881 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:57:58 smithi123 podman[127015]: 2024-09-19 04:57:58.880419899 +0000 UTC m=+0.412087632 container init 61a46b462a865f1f58f04b3a4901330ed9d96365744ecbcd944558a28affc6cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-0-deactivate, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:57:59.141 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:57:58 smithi123 podman[127015]: 2024-09-19 04:57:58.887441522 +0000 UTC m=+0.419109252 container start 61a46b462a865f1f58f04b3a4901330ed9d96365744ecbcd944558a28affc6cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-19T04:57:59.141 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:57:58 smithi123 podman[127015]: 2024-09-19 04:57:58.963731491 +0000 UTC m=+0.495399213 container attach 61a46b462a865f1f58f04b3a4901330ed9d96365744ecbcd944558a28affc6cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T04:57:59.427 INFO:journalctl@ceph.osd.0.smithi123.stdout:Sep 19 04:57:59 smithi123 podman[127015]: 2024-09-19 04:57:59.141293458 +0000 UTC m=+0.672961179 container died 61a46b462a865f1f58f04b3a4901330ed9d96365744ecbcd944558a28affc6cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-19T04:57:59.871 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T04:57:59.946 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:57:59.947 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T04:57:59.947 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T04:57:59.947 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.1 2024-09-19T04:58:00.240 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:57:59 smithi183 systemd[1]: Stopping Ceph osd.1 for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:58:00.600 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:58:00 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-1[39806]: 2024-09-19T04:58:00.238+0000 7feabb2ad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:58:00.600 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:58:00 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-1[39806]: 2024-09-19T04:58:00.238+0000 7feabb2ad640 -1 osd.1 100 *** Got signal Terminated *** 2024-09-19T04:58:00.600 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:58:00 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-1[39806]: 2024-09-19T04:58:00.238+0000 7feabb2ad640 -1 osd.1 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:58:06.007 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:58:05 smithi183 podman[70749]: 2024-09-19 04:58:05.74024968 +0000 UTC m=+5.610152934 container died a35422581151176f1cf18a1e1609261d4ce2a61dfbe8abbcbefb9d41706b43a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-1, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-19T04:58:06.349 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:58:06 smithi183 podman[70749]: 2024-09-19 04:58:06.088085943 +0000 UTC m=+5.957989202 container cleanup a35422581151176f1cf18a1e1609261d4ce2a61dfbe8abbcbefb9d41706b43a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-1, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T04:58:06.350 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:58:06 smithi183 bash[70749]: ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-1 2024-09-19T04:58:06.757 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:58:06 smithi183 podman[70764]: 2024-09-19 04:58:06.406296412 +0000 UTC m=+0.662012950 container remove a35422581151176f1cf18a1e1609261d4ce2a61dfbe8abbcbefb9d41706b43a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T04:58:07.043 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:58:06 smithi183 podman[70852]: 2024-09-19 04:58:06.659273903 +0000 UTC m=+0.015808932 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:58:07.043 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:58:06 smithi183 podman[70852]: 2024-09-19 04:58:06.782376253 +0000 UTC m=+0.138911323 container create 4bcb4dc0d9bd2383c0031a003f5dde4e73054b3db0840d814397b8658801cffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-19T04:58:07.350 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:58:07 smithi183 podman[70852]: 2024-09-19 04:58:07.043095752 +0000 UTC m=+0.399630776 container init 4bcb4dc0d9bd2383c0031a003f5dde4e73054b3db0840d814397b8658801cffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:58:07.350 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:58:07 smithi183 podman[70852]: 2024-09-19 04:58:07.046775149 +0000 UTC m=+0.403310174 container start 4bcb4dc0d9bd2383c0031a003f5dde4e73054b3db0840d814397b8658801cffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:58:07.350 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:58:07 smithi183 podman[70852]: 2024-09-19 04:58:07.143143512 +0000 UTC m=+0.499678552 container attach 4bcb4dc0d9bd2383c0031a003f5dde4e73054b3db0840d814397b8658801cffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T04:58:07.350 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 19 04:58:07 smithi183 podman[70852]: 2024-09-19 04:58:07.237830945 +0000 UTC m=+0.594365969 container died 4bcb4dc0d9bd2383c0031a003f5dde4e73054b3db0840d814397b8658801cffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:58:07.871 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T04:58:07.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:58:07.910 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T04:58:07.910 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T04:58:07.910 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.2 2024-09-19T04:58:08.240 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:58:07 smithi193 systemd[1]: Stopping Ceph osd.2 for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:58:08.740 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:58:08 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-2[40557]: 2024-09-19T04:58:08.341+0000 7f2c9b1fd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:58:08.740 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:58:08 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-2[40557]: 2024-09-19T04:58:08.341+0000 7f2c9b1fd640 -1 osd.2 100 *** Got signal Terminated *** 2024-09-19T04:58:08.741 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:58:08 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-2[40557]: 2024-09-19T04:58:08.341+0000 7f2c9b1fd640 -1 osd.2 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:58:13.977 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:58:13 smithi193 podman[71770]: 2024-09-19 04:58:13.69901322 +0000 UTC m=+5.595592850 container died 36f82aa19108d31a0382ecb554e8e4559439f838cffcf45028b310be1e8aede2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:58:14.241 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:58:14 smithi193 podman[71770]: 2024-09-19 04:58:14.066957596 +0000 UTC m=+5.963537218 container cleanup 36f82aa19108d31a0382ecb554e8e4559439f838cffcf45028b310be1e8aede2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:58:14.241 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:58:14 smithi193 bash[71770]: ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-2 2024-09-19T04:58:14.722 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:58:14 smithi193 podman[71785]: 2024-09-19 04:58:14.377024226 +0000 UTC m=+0.672952094 container remove 36f82aa19108d31a0382ecb554e8e4559439f838cffcf45028b310be1e8aede2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T04:58:14.991 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:58:14 smithi193 podman[71875]: 2024-09-19 04:58:14.624048527 +0000 UTC m=+0.015512803 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:58:14.991 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:58:14 smithi193 podman[71875]: 2024-09-19 04:58:14.736174171 +0000 UTC m=+0.127638445 container create df402951c0a30b8244e9d9f8f08c783ff9bcbcd1f331e26b484b13a316791618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:58:15.467 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:58:14 smithi193 podman[71875]: 2024-09-19 04:58:14.99710175 +0000 UTC m=+0.388566025 container init df402951c0a30b8244e9d9f8f08c783ff9bcbcd1f331e26b484b13a316791618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-2-deactivate, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:58:15.467 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:58:15 smithi193 podman[71875]: 2024-09-19 04:58:15.000404323 +0000 UTC m=+0.391868615 container start df402951c0a30b8244e9d9f8f08c783ff9bcbcd1f331e26b484b13a316791618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T04:58:15.467 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:58:15 smithi193 podman[71875]: 2024-09-19 04:58:15.10573926 +0000 UTC m=+0.497203554 container attach df402951c0a30b8244e9d9f8f08c783ff9bcbcd1f331e26b484b13a316791618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T04:58:15.467 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 19 04:58:15 smithi193 podman[71875]: 2024-09-19 04:58:15.187991686 +0000 UTC m=+0.579455960 container died df402951c0a30b8244e9d9f8f08c783ff9bcbcd1f331e26b484b13a316791618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T04:58:15.845 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T04:58:15.884 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:58:15.884 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T04:58:15.885 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T04:58:15.885 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.3 2024-09-19T04:58:16.291 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:58:15 smithi123 systemd[1]: Stopping Ceph osd.3 for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:58:16.677 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:58:16 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3[50614]: 2024-09-19T04:58:16.289+0000 7ff887c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:58:16.677 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:58:16 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3[50614]: 2024-09-19T04:58:16.289+0000 7ff887c00640 -1 osd.3 100 *** Got signal Terminated *** 2024-09-19T04:58:16.677 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:58:16 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3[50614]: 2024-09-19T04:58:16.289+0000 7ff887c00640 -1 osd.3 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:58:18.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:17 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:17.908+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:18.490 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:18 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:18.230+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:18.677 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:58:18 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3[50614]: 2024-09-19T04:58:18.398+0000 7ff883600640 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:53.540761+0000 front 2024-09-19T04:57:53.540839+0000 (oldest deadline 2024-09-19T04:58:17.640787+0000) 2024-09-19T04:58:18.991 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:18 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:18.538+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:19.364 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:18 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:18.942+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:19.582 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:19 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:19.275+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:19.677 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:58:19 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3[50614]: 2024-09-19T04:58:19.363+0000 7ff883600640 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:53.540761+0000 front 2024-09-19T04:57:53.540839+0000 (oldest deadline 2024-09-19T04:58:17.640787+0000) 2024-09-19T04:58:19.990 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:19 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:19.580+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:20.390 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:19 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:19.990+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:20.547 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:20 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:20.241+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:20.600 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:20 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:20.104+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:20.677 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:58:20 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3[50614]: 2024-09-19T04:58:20.388+0000 7ff883600640 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:53.540761+0000 front 2024-09-19T04:57:53.540839+0000 (oldest deadline 2024-09-19T04:58:17.640787+0000) 2024-09-19T04:58:20.990 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:20 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:20.546+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:20.996 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:20 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:20.729+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:21.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:20 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:20.995+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:21.515 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:21 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:21.261+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:21.600 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:21 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:21.151+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:21.990 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:21 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:21.514+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:22.029 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:58:21 smithi123 podman[127117]: 2024-09-19 04:58:21.890884213 +0000 UTC m=+5.714699767 container died 880479365d1c11d093f96d8f8b026dc80e16ffa89bdabaf9b464b15f5695a9d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T04:58:22.029 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:21 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:21.738+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:22.395 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:22 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:22.027+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:22.468 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:22 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:22.212+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:22.599 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:22 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:22.125+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:22.677 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:58:22 smithi123 podman[127117]: 2024-09-19 04:58:22.394797793 +0000 UTC m=+6.218613347 container cleanup 880479365d1c11d093f96d8f8b026dc80e16ffa89bdabaf9b464b15f5695a9d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-19T04:58:22.677 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:58:22 smithi123 bash[127117]: ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3 2024-09-19T04:58:22.740 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:22 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:22.467+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:22.989 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:22 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:22.773+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:22.989 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:58:22 smithi123 podman[127134]: 2024-09-19 04:58:22.70507879 +0000 UTC m=+0.807523806 container remove 880479365d1c11d093f96d8f8b026dc80e16ffa89bdabaf9b464b15f5695a9d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T04:58:23.319 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:22 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:22.987+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:23.485 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:23 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:23.226+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:23.600 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:23 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:23.131+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:23.651 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:58:23 smithi123 podman[127217]: 2024-09-19 04:58:23.223023582 +0000 UTC m=+0.027381344 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:58:23.651 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:58:23 smithi123 podman[127217]: 2024-09-19 04:58:23.347578814 +0000 UTC m=+0.151936576 container create ac74f3f4c3400cac4e789bd8117037ee8829b7f091b08a3e474e1a5aec04ad62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T04:58:23.651 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:58:23 smithi123 podman[127217]: 2024-09-19 04:58:23.65039363 +0000 UTC m=+0.454751392 container init ac74f3f4c3400cac4e789bd8117037ee8829b7f091b08a3e474e1a5aec04ad62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:58:23.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:23 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:23.484+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:23.914 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:23 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:23.729+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:23.915 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:58:23 smithi123 podman[127217]: 2024-09-19 04:58:23.657378196 +0000 UTC m=+0.461735960 container start ac74f3f4c3400cac4e789bd8117037ee8829b7f091b08a3e474e1a5aec04ad62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-19T04:58:23.915 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:58:23 smithi123 podman[127217]: 2024-09-19 04:58:23.741757331 +0000 UTC m=+0.546115101 container attach ac74f3f4c3400cac4e789bd8117037ee8829b7f091b08a3e474e1a5aec04ad62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T04:58:24.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:24 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:24.024+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:24.177 INFO:journalctl@ceph.osd.3.smithi123.stdout:Sep 19 04:58:23 smithi123 podman[127217]: 2024-09-19 04:58:23.914273865 +0000 UTC m=+0.718631636 container died ac74f3f4c3400cac4e789bd8117037ee8829b7f091b08a3e474e1a5aec04ad62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:58:24.580 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T04:58:24.600 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:24 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:24.169+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:24.655 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:58:24.656 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T04:58:24.656 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T04:58:24.656 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.4 2024-09-19T04:58:24.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:24 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:24.251+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:24.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:24 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:24.454+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:24.927 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:24 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:24.691+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:24.975 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:58:24 smithi183 systemd[1]: Stopping Ceph osd.4 for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:58:25.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:25 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:25.122+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:25.350 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:58:24 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4[46244]: 2024-09-19T04:58:24.974+0000 7f7b5b22e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:58:25.350 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:58:24 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4[46244]: 2024-09-19T04:58:24.974+0000 7f7b5b22e640 -1 osd.4 100 *** Got signal Terminated *** 2024-09-19T04:58:25.350 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:58:24 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4[46244]: 2024-09-19T04:58:24.974+0000 7f7b5b22e640 -1 osd.4 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:58:25.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:25 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:25.005+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:25.740 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:25 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:25.288+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:25.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:25 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:25.481+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:26.054 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:25 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:25.720+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:26.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:26 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:26.075+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:26.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:26 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:26.052+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:26.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:26 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:26.052+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:26.740 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:26 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:26.524+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:26.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:26 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:26.304+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:27.051 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:26 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:26.757+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:27.350 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:58:27 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4[46244]: 2024-09-19T04:58:27.044+0000 7f7b57844640 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:04.742433+0000 front 2024-09-19T04:58:04.742544+0000 (oldest deadline 2024-09-19T04:58:26.442633+0000) 2024-09-19T04:58:27.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:27 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:27.073+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:27.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:27 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:27.073+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:27.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:27 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:27.049+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:27.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:27 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:27.049+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:27.740 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:27 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:27.517+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:27.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:27 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:27.517+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.796532+0000 front 2024-09-19T04:58:03.796607+0000 (oldest deadline 2024-09-19T04:58:26.696573+0000) 2024-09-19T04:58:27.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:27 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:27.308+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:27.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:27 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:27.308+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:28.054 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:27 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:27.802+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:28.055 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:27 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:27.802+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:28.350 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:58:28 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4[46244]: 2024-09-19T04:58:28.059+0000 7f7b57844640 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:04.742433+0000 front 2024-09-19T04:58:04.742544+0000 (oldest deadline 2024-09-19T04:58:26.442633+0000) 2024-09-19T04:58:28.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:28 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:28.070+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:28.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:28 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:28.070+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:28.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:28 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:28.053+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:28.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:28 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:28.053+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:28.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:28 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:28.283+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:28.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:28 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:28.283+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:28.555 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:28 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:28.531+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:28.990 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:28 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:28.553+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:28.990 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:28 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:28.553+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.796532+0000 front 2024-09-19T04:58:03.796607+0000 (oldest deadline 2024-09-19T04:58:26.696573+0000) 2024-09-19T04:58:29.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:29 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:29.048+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:29.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:29 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:29.048+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:29.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:28 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:28.833+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:29.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:28 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:28.833+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:29.350 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:58:29 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4[46244]: 2024-09-19T04:58:29.062+0000 7f7b57844640 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:04.742433+0000 front 2024-09-19T04:58:04.742544+0000 (oldest deadline 2024-09-19T04:58:26.442633+0000) 2024-09-19T04:58:29.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:29 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:29.084+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:29.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:29 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:29.084+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:29.490 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:29 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:29.239+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:29.490 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:29 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:29.239+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:29.825 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:29 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:29.499+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:29.825 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:29 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:29.560+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:29.825 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:29 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:29.560+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.796532+0000 front 2024-09-19T04:58:03.796607+0000 (oldest deadline 2024-09-19T04:58:26.696573+0000) 2024-09-19T04:58:30.099 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:29 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:29.826+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:30.099 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:29 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:29.826+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:30.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:30 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:30.053+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:30.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:30 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:30.053+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:30.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:30 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:30.097+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:30.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:30 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:30.097+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:30.474 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:30 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:30.189+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:30.474 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:30 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:30.189+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:30.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:30 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:30.473+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:30.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:30 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:30.595+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:30.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:30 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:30.595+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.796532+0000 front 2024-09-19T04:58:03.796607+0000 (oldest deadline 2024-09-19T04:58:26.696573+0000) 2024-09-19T04:58:30.758 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:58:30 smithi183 podman[70955]: 2024-09-19 04:58:30.496541043 +0000 UTC m=+5.646777302 container died 3f895ca06ac409ff882aaa57adef5457561897a27506802b091bcd1a3f654763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T04:58:31.051 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:30 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:30.789+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:31.051 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:30 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:30.789+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:31.100 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:30 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:30.886+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:31.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:31 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:31.007+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:31.101 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:31 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:31.007+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:31.101 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:58:30 smithi183 podman[70955]: 2024-09-19 04:58:30.848371662 +0000 UTC m=+5.998607920 container cleanup 3f895ca06ac409ff882aaa57adef5457561897a27506802b091bcd1a3f654763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:58:31.101 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:58:30 smithi183 bash[70955]: ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4 2024-09-19T04:58:31.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:31 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:31.050+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:31.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:31 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:31.050+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:31.475 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:58:31 smithi183 podman[70969]: 2024-09-19 04:58:31.1246241 +0000 UTC m=+0.626123378 container remove 3f895ca06ac409ff882aaa57adef5457561897a27506802b091bcd1a3f654763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T04:58:31.491 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:31 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:31.154+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:31.491 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:31 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:31.154+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:31.795 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:58:31 smithi183 podman[71055]: 2024-09-19 04:58:31.475322367 +0000 UTC m=+0.100571874 container create 12c906bc280d564e751d5c0f1dbcfa197e99e62742e8e7e498a2bd5544dfc2d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-19T04:58:31.795 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:58:31 smithi183 podman[71055]: 2024-09-19 04:58:31.391061882 +0000 UTC m=+0.016311360 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:58:31.795 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:58:31 smithi183 podman[71055]: 2024-09-19 04:58:31.711077643 +0000 UTC m=+0.336327108 container init 12c906bc280d564e751d5c0f1dbcfa197e99e62742e8e7e498a2bd5544dfc2d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T04:58:31.795 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:58:31 smithi183 podman[71055]: 2024-09-19 04:58:31.714538439 +0000 UTC m=+0.339787906 container start 12c906bc280d564e751d5c0f1dbcfa197e99e62742e8e7e498a2bd5544dfc2d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, ceph=True) 2024-09-19T04:58:31.795 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:58:31 smithi183 podman[71055]: 2024-09-19 04:58:31.794145006 +0000 UTC m=+0.419394486 container attach 12c906bc280d564e751d5c0f1dbcfa197e99e62742e8e7e498a2bd5544dfc2d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T04:58:31.990 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:31 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:31.509+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:31.991 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:31 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:31.597+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:31.991 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:31 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:31.597+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.796532+0000 front 2024-09-19T04:58:03.796607+0000 (oldest deadline 2024-09-19T04:58:26.696573+0000) 2024-09-19T04:58:32.056 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 19 04:58:31 smithi183 podman[71055]: 2024-09-19 04:58:31.898094949 +0000 UTC m=+0.523344426 container died 12c906bc280d564e751d5c0f1dbcfa197e99e62742e8e7e498a2bd5544dfc2d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T04:58:32.056 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:31 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:31.912+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:32.083 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:31 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:31.819+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:32.083 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:31 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:31.819+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:32.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:32 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:32.053+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:32.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:32 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:32.053+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:32.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:32 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:32.081+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:32.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:32 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:32.081+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:32.490 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:32 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:32.105+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:32.490 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:32 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:32.105+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:32.510 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T04:58:32.549 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:58:32.549 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T04:58:32.549 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T04:58:32.549 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.5 2024-09-19T04:58:32.872 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:32 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:32.510+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:32.873 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:32 smithi193 systemd[1]: Stopping Ceph osd.5 for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:58:32.873 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:32 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:32.622+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:32.873 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:32 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:32.622+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.796532+0000 front 2024-09-19T04:58:03.796607+0000 (oldest deadline 2024-09-19T04:58:26.696573+0000) 2024-09-19T04:58:32.873 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:32 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:32.870+0000 7f2df5db9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:58:32.873 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:32 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:32.870+0000 7f2df5db9640 -1 osd.5 100 *** Got signal Terminated *** 2024-09-19T04:58:32.873 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:32 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:32.870+0000 7f2df5db9640 -1 osd.5 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:58:33.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:32 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:32.866+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:33.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:32 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:32.866+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:33.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:33 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:33.068+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:33.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:33 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:33.068+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:33.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:33 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:33.084+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:33.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:33 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:33.084+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:33.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:32 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:32.943+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:33.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:33 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:33.075+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:33.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:33 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:33.075+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:33.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:33 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:33.470+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:33.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:33 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:33.580+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:33.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:33 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:33.580+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.796532+0000 front 2024-09-19T04:58:03.796607+0000 (oldest deadline 2024-09-19T04:58:26.696573+0000) 2024-09-19T04:58:34.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:34 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:34.070+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:34.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:34 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:34.070+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:34.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:33 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:33.850+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:34.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:33 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:33.850+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:34.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:33 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:33.850+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:13.281356+0000 front 2024-09-19T04:58:13.281429+0000 (oldest deadline 2024-09-19T04:58:33.781375+0000) 2024-09-19T04:58:34.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:33 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:33.918+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:34.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:34 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:34.040+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:34.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:34 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:34.040+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:34.440 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:34 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:34.035+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:34.441 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:34 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:34.035+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:34.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:34 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:34.439+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:34.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:34 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:34.580+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:34.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:34 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:34.580+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.796532+0000 front 2024-09-19T04:58:03.796607+0000 (oldest deadline 2024-09-19T04:58:26.696573+0000) 2024-09-19T04:58:34.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:34 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:34.580+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.397437+0000 front 2024-09-19T04:58:11.397399+0000 (oldest deadline 2024-09-19T04:58:33.697520+0000) 2024-09-19T04:58:35.107 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:34 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:34.807+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:35.107 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:34 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:34.807+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:35.107 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:34 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:34.807+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:13.281356+0000 front 2024-09-19T04:58:13.281429+0000 (oldest deadline 2024-09-19T04:58:33.781375+0000) 2024-09-19T04:58:35.107 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:35.021+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:35.107 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:35.021+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:35.107 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:35.021+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:35.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:34 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:34.988+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:35.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:34 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:34.988+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:35.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:34 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:34.964+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:35.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:35.086+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:35.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:35.086+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:35.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:35.086+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:35.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:35.462+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:35.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:35.462+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:35.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:35.583+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:35.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:35.583+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.796532+0000 front 2024-09-19T04:58:03.796607+0000 (oldest deadline 2024-09-19T04:58:26.696573+0000) 2024-09-19T04:58:35.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:35 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:35.583+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.397437+0000 front 2024-09-19T04:58:11.397399+0000 (oldest deadline 2024-09-19T04:58:33.697520+0000) 2024-09-19T04:58:36.071 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:35.852+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:36.072 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:35.852+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:36.072 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:35 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:35.852+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:13.281356+0000 front 2024-09-19T04:58:13.281429+0000 (oldest deadline 2024-09-19T04:58:33.781375+0000) 2024-09-19T04:58:36.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:35.989+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:36.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:35 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:35.989+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:36.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:36 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:36.105+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:36.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:36 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:36.105+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:36.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:36 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:36.105+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:36.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:36 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:36.070+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:36.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:36 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:36.070+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:36.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:36 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:36.070+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:36.448 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:36 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:36.015+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:36.448 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:36 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:36.015+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:36.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:36 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:36.568+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:36.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:36 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:36.568+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.796532+0000 front 2024-09-19T04:58:03.796607+0000 (oldest deadline 2024-09-19T04:58:26.696573+0000) 2024-09-19T04:58:36.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:36 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:36.568+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.397437+0000 front 2024-09-19T04:58:11.397399+0000 (oldest deadline 2024-09-19T04:58:33.697520+0000) 2024-09-19T04:58:36.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:36 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:36.446+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:36.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:36 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:36.446+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:37.104 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:36 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:36.826+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:37.105 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:36 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:36.826+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:37.105 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:36 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:36.826+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:13.281356+0000 front 2024-09-19T04:58:13.281429+0000 (oldest deadline 2024-09-19T04:58:33.781375+0000) 2024-09-19T04:58:37.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:36 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:36.979+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:37.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:36 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:36.979+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:37.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:36 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:36.979+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:37.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:36 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:36.984+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:37.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:36 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:36.984+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:37.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:37 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:37.130+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:37.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:37 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:37.130+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:37.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:37 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:37.130+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:37.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:37 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:37.103+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:37.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:37 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:37.103+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:37.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:37 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:37.103+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:37.740 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:37 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:37.534+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.093774+0000 front 2024-09-19T04:57:54.093690+0000 (oldest deadline 2024-09-19T04:58:17.593565+0000) 2024-09-19T04:58:37.740 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:37 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:37.534+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.796532+0000 front 2024-09-19T04:58:03.796607+0000 (oldest deadline 2024-09-19T04:58:26.696573+0000) 2024-09-19T04:58:37.741 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:37 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5[47021]: 2024-09-19T04:58:37.534+0000 7f2dee3c7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.397437+0000 front 2024-09-19T04:58:11.397399+0000 (oldest deadline 2024-09-19T04:58:33.697520+0000) 2024-09-19T04:58:37.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:37 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:37.403+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:37.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:37 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:37.403+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:38.133 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:37 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:37.831+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:38.133 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:37 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:37.831+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:38.133 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:37 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:37.831+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:13.281356+0000 front 2024-09-19T04:58:13.281429+0000 (oldest deadline 2024-09-19T04:58:33.781375+0000) 2024-09-19T04:58:38.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:37 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:37.963+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:38.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:37 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:37.963+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:38.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:37 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:37.963+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:38.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:37 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:37.939+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:38.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:37 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:37.939+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:38.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:38 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:38.122+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:38.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:38 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:38.122+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:38.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:38 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:38.122+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:38.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:38 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:38.132+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:38.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:38 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:38.132+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:38.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:38 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:38.132+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:38.698 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:38 smithi193 podman[71999]: 2024-09-19 04:58:38.401841925 +0000 UTC m=+5.654627037 container died c917ea986750d3163d02f41a5bce35fbb6a69d834c78094f8ea6decea4f99904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T04:58:38.698 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:38 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:38.375+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:38.698 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:38 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:38.375+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:38.990 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:38 smithi193 podman[71999]: 2024-09-19 04:58:38.79692919 +0000 UTC m=+6.049714286 container cleanup c917ea986750d3163d02f41a5bce35fbb6a69d834c78094f8ea6decea4f99904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T04:58:38.991 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:38 smithi193 bash[71999]: ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5 2024-09-19T04:58:38.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:38 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:38.936+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:38.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:38 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:38.936+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:38.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:38 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:38.936+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:39.090 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:38 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:38.823+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:39.090 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:38 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:38.823+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:39.090 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:38 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:38.823+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:13.281356+0000 front 2024-09-19T04:58:13.281429+0000 (oldest deadline 2024-09-19T04:58:33.781375+0000) 2024-09-19T04:58:39.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:38 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:38.967+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:39.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:38 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:38.967+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:39.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:39 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:39.168+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:39.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:39 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:39.168+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:39.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:39 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:39.168+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:39.408 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:39 smithi193 podman[72014]: 2024-09-19 04:58:39.073452417 +0000 UTC m=+0.672374489 container remove c917ea986750d3163d02f41a5bce35fbb6a69d834c78094f8ea6decea4f99904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T04:58:39.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:39 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:39.089+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:39.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:39 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:39.089+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:39.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:39 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:39.089+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:39.669 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:39 smithi193 podman[72104]: 2024-09-19 04:58:39.407409177 +0000 UTC m=+0.107224560 container create b8d05d709ff6750c0bcfdf29eb74d400e53aa9e19788463b67871abc0e0c750b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:58:39.669 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:39 smithi193 podman[72104]: 2024-09-19 04:58:39.316048315 +0000 UTC m=+0.015863708 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:58:39.670 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:39 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:39.419+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:39.670 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:39 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:39.419+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:39.932 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:39 smithi193 podman[72104]: 2024-09-19 04:58:39.668835239 +0000 UTC m=+0.368650628 container init b8d05d709ff6750c0bcfdf29eb74d400e53aa9e19788463b67871abc0e0c750b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:58:39.932 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:39 smithi193 podman[72104]: 2024-09-19 04:58:39.672456921 +0000 UTC m=+0.372272302 container start b8d05d709ff6750c0bcfdf29eb74d400e53aa9e19788463b67871abc0e0c750b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-19T04:58:39.932 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:39 smithi193 podman[72104]: 2024-09-19 04:58:39.7851536 +0000 UTC m=+0.484969006 container attach b8d05d709ff6750c0bcfdf29eb74d400e53aa9e19788463b67871abc0e0c750b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:58:39.932 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 19 04:58:39 smithi193 podman[72104]: 2024-09-19 04:58:39.861071939 +0000 UTC m=+0.560887319 container died b8d05d709ff6750c0bcfdf29eb74d400e53aa9e19788463b67871abc0e0c750b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:58:40.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:40.040+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:40.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:40.040+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:40.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:40.040+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:40.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:39 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:39.817+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:40.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:39 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:39.817+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:40.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:39 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:39.817+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:13.281356+0000 front 2024-09-19T04:58:13.281429+0000 (oldest deadline 2024-09-19T04:58:33.781375+0000) 2024-09-19T04:58:40.205 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:39 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:39.930+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:40.205 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:39 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:39.930+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:40.205 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:39 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:39.930+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:40.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:39 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:39.997+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:40.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:39 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:39.997+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:40.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:40 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:40.129+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:40.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:40 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:40.129+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:40.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:40 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:40.129+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:40.490 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:40 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:40.412+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:40.490 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:40 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:40.412+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:40.560 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T04:58:40.598 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:58:40.598 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T04:58:40.598 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T04:58:40.598 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.6 2024-09-19T04:58:40.927 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:40 smithi123 systemd[1]: Stopping Ceph osd.6 for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:58:40.927 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:40.799+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:40.927 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:40.799+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:40.928 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:40.799+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:13.281356+0000 front 2024-09-19T04:58:13.281429+0000 (oldest deadline 2024-09-19T04:58:33.781375+0000) 2024-09-19T04:58:41.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:40 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:40.974+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:41.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:40 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:40.974+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:41.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:40 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:40.974+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:41.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:41 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:41.000+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:41.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:41 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:41.000+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:41.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:41 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:41.124+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:41.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:41 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:41.124+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:41.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:41 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:41.124+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:41.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:40.997+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:41.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:40.997+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:41.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:40 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:40.997+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:41.427 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:41 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:41.017+0000 7f8365e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:58:41.427 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:41 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:41.017+0000 7f8365e00640 -1 osd.6 100 *** Got signal Terminated *** 2024-09-19T04:58:41.427 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:41 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:41.017+0000 7f8365e00640 -1 osd.6 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:58:41.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:41 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:41.457+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:41.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:41 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:41.457+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:42.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:41 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:41.960+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:42.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:41 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:41.960+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:42.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:41 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:41.960+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:42.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:41 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:41.782+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:42.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:41 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:41.782+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:42.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:41 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:41.782+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:13.281356+0000 front 2024-09-19T04:58:13.281429+0000 (oldest deadline 2024-09-19T04:58:33.781375+0000) 2024-09-19T04:58:42.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:42 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:42.000+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:42.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:42 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:42.000+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:42.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:42 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:42.127+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:42.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:42 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:42.127+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:42.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:42 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:42.127+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:42.437 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:42 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:42.020+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:42.437 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:42 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:42.020+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:42.437 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:42 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:42.020+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:42.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:42 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:42.436+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:42.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:42 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:42.436+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:43.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:42.765+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:43.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:42.765+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:43.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:42.765+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:13.281356+0000 front 2024-09-19T04:58:13.281429+0000 (oldest deadline 2024-09-19T04:58:33.781375+0000) 2024-09-19T04:58:43.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:42.943+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:43.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:42.943+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:43.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:42.943+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:43.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:42.943+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:58:43.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:43 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:43.001+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:43.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:43 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:43.001+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:43.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:43 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:43.137+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:43.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:43 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:43.137+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:43.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:43 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:43.137+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:43.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:43 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:43.137+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:58:43.405 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:42 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:42.992+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:43.405 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:42 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:42.992+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:43.405 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:42 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:42.992+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:43.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:43 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:43.403+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:43.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:43 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:43.403+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:43.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:43 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:43.403+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:58:44.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:43.756+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:44.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:43.756+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:44.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:43.756+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:13.281356+0000 front 2024-09-19T04:58:13.281429+0000 (oldest deadline 2024-09-19T04:58:33.781375+0000) 2024-09-19T04:58:44.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:43.977+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:44.178 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:43.977+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:44.178 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:43.977+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:44.178 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:43 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:43.977+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:58:44.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:44 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:44.041+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:44.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:44 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:44.041+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:44.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:44 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:44.104+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:44.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:44 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:44.104+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:44.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:44 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:44.104+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:44.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:44 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:44.104+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:58:44.372 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:44 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:44.037+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:44.372 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:44 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:44.037+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:44.372 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:44 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:44.037+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:44.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:44 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:44.370+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:44.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:44 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:44.370+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:44.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:44 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:44.370+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:58:45.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:44 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:44.787+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:45.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:44 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:44.787+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:45.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:44 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:44.787+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:13.281356+0000 front 2024-09-19T04:58:13.281429+0000 (oldest deadline 2024-09-19T04:58:33.781375+0000) 2024-09-19T04:58:45.177 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:44 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:44.787+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.783040+0000 front 2024-09-19T04:58:20.783057+0000 (oldest deadline 2024-09-19T04:58:44.283126+0000) 2024-09-19T04:58:45.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:45 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:45.024+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:45.178 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:45 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:45.024+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:45.178 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:45 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:45.024+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:45.178 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:45 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:45.024+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:58:45.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:45 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:45.049+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:45.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:45 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:45.049+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:45.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:45 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:45.049+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.042583+0000 front 2024-09-19T04:58:18.042550+0000 (oldest deadline 2024-09-19T04:58:45.042800+0000) 2024-09-19T04:58:45.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:45 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:45.097+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:45.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:45 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:45.098+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:45.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:45 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:45.098+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:45.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:45 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:45.098+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:58:45.388 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:44 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:44.995+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:45.389 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:44 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:44.995+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:45.389 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:44 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:44.995+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:45.389 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:44 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:44.995+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:58:45.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:45 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:45.387+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:45.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:45 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:45.387+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:45.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:45 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:45.387+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:58:46.046 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:45 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:45.753+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:55.778707+0000 front 2024-09-19T04:57:55.778741+0000 (oldest deadline 2024-09-19T04:58:19.878837+0000) 2024-09-19T04:58:46.046 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:45 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:45.753+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:05.080497+0000 front 2024-09-19T04:58:05.080483+0000 (oldest deadline 2024-09-19T04:58:27.380304+0000) 2024-09-19T04:58:46.046 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:45 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:45.753+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:13.281356+0000 front 2024-09-19T04:58:13.281429+0000 (oldest deadline 2024-09-19T04:58:33.781375+0000) 2024-09-19T04:58:46.046 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:45 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6[57105]: 2024-09-19T04:58:45.753+0000 7f835c800640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.783040+0000 front 2024-09-19T04:58:20.783057+0000 (oldest deadline 2024-09-19T04:58:44.283126+0000) 2024-09-19T04:58:46.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:46 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:46.019+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:46.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:46 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:46.019+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:46.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:46 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:46.019+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.042583+0000 front 2024-09-19T04:58:18.042550+0000 (oldest deadline 2024-09-19T04:58:45.042800+0000) 2024-09-19T04:58:46.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:46 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:46.132+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:46.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:46 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:46.132+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:46.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:46 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:46.132+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:46.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:46 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:46.132+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:58:46.369 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:46 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:46.016+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:46.369 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:46 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:46.016+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:46.369 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:46 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:46.016+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:46.369 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:46 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:46.016+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:58:46.423 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:46 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:46.052+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:46.423 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:46 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:46.052+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:46.423 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:46 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:46.052+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:46.423 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:46 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:46.052+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:58:46.676 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:46 smithi123 podman[127331]: 2024-09-19 04:58:46.42113695 +0000 UTC m=+5.527127543 container died 4adbf2c8bbf8315aac4e2bf10cda27fb53bc60e9f9b2fe99d3104210ac49f3bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T04:58:46.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:46 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:46.367+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:46.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:46 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:46.367+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:46.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:46 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:46.367+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:58:47.074 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:46 smithi123 podman[127331]: 2024-09-19 04:58:46.86448922 +0000 UTC m=+5.970479813 container cleanup 4adbf2c8bbf8315aac4e2bf10cda27fb53bc60e9f9b2fe99d3104210ac49f3bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:58:47.074 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:46 smithi123 bash[127331]: ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6 2024-09-19T04:58:47.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:46 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:46.983+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:47.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:46 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:46.983+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:47.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:46 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:46.983+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:47.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:46 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:46.983+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:58:47.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:47 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:47.053+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:47.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:47 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:47.053+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:47.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:47 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:47.053+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.042583+0000 front 2024-09-19T04:58:18.042550+0000 (oldest deadline 2024-09-19T04:58:45.042800+0000) 2024-09-19T04:58:47.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:47 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:47.094+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:47.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:47 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:47.094+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:47.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:47 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:47.094+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:47.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:47 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:47.094+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:58:47.427 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:47 smithi123 podman[127349]: 2024-09-19 04:58:47.099967451 +0000 UTC m=+0.671777378 container remove 4adbf2c8bbf8315aac4e2bf10cda27fb53bc60e9f9b2fe99d3104210ac49f3bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True) 2024-09-19T04:58:47.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:47 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:47.072+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:47.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:47 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:47.072+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:47.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:47 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:47.072+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:47.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:47 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:47.072+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:58:47.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:47 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:47.383+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:47.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:47 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:47.383+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:47.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:47 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:47.383+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:58:47.895 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:47 smithi123 podman[127437]: 2024-09-19 04:58:47.532895403 +0000 UTC m=+0.027821042 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:58:47.895 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:47 smithi123 podman[127437]: 2024-09-19 04:58:47.65873019 +0000 UTC m=+0.153655829 container create 9dff61be7b96797454b072addc02d88ae49790c386b43ec2b6b6e53b644bd717 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:58:47.895 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:47 smithi123 podman[127437]: 2024-09-19 04:58:47.894742436 +0000 UTC m=+0.389668075 container init 9dff61be7b96797454b072addc02d88ae49790c386b43ec2b6b6e53b644bd717 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=) 2024-09-19T04:58:48.167 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:47 smithi123 podman[127437]: 2024-09-19 04:58:47.901652462 +0000 UTC m=+0.396578107 container start 9dff61be7b96797454b072addc02d88ae49790c386b43ec2b6b6e53b644bd717 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T04:58:48.167 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:48 smithi123 podman[127437]: 2024-09-19 04:58:48.011171573 +0000 UTC m=+0.506097217 container attach 9dff61be7b96797454b072addc02d88ae49790c386b43ec2b6b6e53b644bd717 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T04:58:48.167 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:48.112+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:48.167 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:48.112+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:48.167 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:48.112+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:48.167 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:48 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:48.112+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:58:48.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:47 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:47.945+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:48.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:47 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:47.945+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:48.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:47 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:47.945+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:48.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:47 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:47.945+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:58:48.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:48 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:48.012+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:48.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:48 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:48.012+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:48.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:48 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:48.012+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.042583+0000 front 2024-09-19T04:58:18.042550+0000 (oldest deadline 2024-09-19T04:58:45.042800+0000) 2024-09-19T04:58:48.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:48 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:48.102+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:48.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:48 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:48.102+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:48.351 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:48 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:48.102+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:48.351 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:48 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:48.102+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:58:48.427 INFO:journalctl@ceph.osd.6.smithi123.stdout:Sep 19 04:58:48 smithi123 podman[127437]: 2024-09-19 04:58:48.166521779 +0000 UTC m=+0.661447428 container died 9dff61be7b96797454b072addc02d88ae49790c386b43ec2b6b6e53b644bd717 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T04:58:48.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:48 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:48.433+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:48.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:48 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:48.433+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:48.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:48 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:48.433+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:58:48.918 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T04:58:48.994 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:58:48.994 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T04:58:48.995 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T04:58:48.995 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.7 2024-09-19T04:58:49.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:48 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:48.915+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:49.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:48 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:48.915+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:49.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:48 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:48.915+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:49.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:48 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:48.915+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:58:49.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:48 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:48.992+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:49.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:48 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:48.992+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:49.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:48 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:48.992+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.042583+0000 front 2024-09-19T04:58:18.042550+0000 (oldest deadline 2024-09-19T04:58:45.042800+0000) 2024-09-19T04:58:49.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:49 smithi183 systemd[1]: Stopping Ceph osd.7 for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:58:49.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:49 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:49.056+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:49.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:49 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:49.056+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:49.351 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:49 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:49.056+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:49.351 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:49 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:49.056+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:58:49.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:49 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:49.129+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:49.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:49 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:49.129+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:49.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:49 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:49.129+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:49.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:49 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:49.129+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:58:49.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:49 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:49.392+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:49.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:49 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:49.392+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:49.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:49 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:49.392+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:58:49.849 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:49 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:49.373+0000 7f5600429640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:58:49.850 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:49 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:49.373+0000 7f5600429640 -1 osd.7 100 *** Got signal Terminated *** 2024-09-19T04:58:49.850 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:49 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:49.373+0000 7f5600429640 -1 osd.7 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:58:50.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:49 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:49.935+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:50.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:49 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:49.935+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:50.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:49 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:49.935+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:50.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:49 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:49.935+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:58:50.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:50 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:50.018+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:50.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:50 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:50.018+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:50.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:50 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:50.018+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.042583+0000 front 2024-09-19T04:58:18.042550+0000 (oldest deadline 2024-09-19T04:58:45.042800+0000) 2024-09-19T04:58:50.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:50 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:50.014+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:50.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:50 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:50.014+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:50.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:50 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:50.014+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:50.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:50 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:50.014+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:58:50.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:50 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:50.111+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:50.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:50 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:50.111+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:50.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:50 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:50.111+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:50.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:50 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:50.111+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:58:50.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:50 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:50.392+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:50.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:50 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:50.392+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:50.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:50 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:50.392+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:58:51.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:50 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:50.911+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:51.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:50 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:50.911+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:51.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:50 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:50.911+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:51.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:50 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:50.911+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:58:51.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:50 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:50.989+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:51.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:50 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:50.989+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:51.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:50 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:50.989+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.042583+0000 front 2024-09-19T04:58:18.042550+0000 (oldest deadline 2024-09-19T04:58:45.042800+0000) 2024-09-19T04:58:51.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:51 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:51.008+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:51.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:51 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:51.008+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:51.351 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:51 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:51.008+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:51.351 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:51 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:51.008+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:58:51.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:51.117+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:51.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:51.117+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:51.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:51.117+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:51.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:51.117+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:58:51.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:51 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:51.117+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:58:51.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:51 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:51.418+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:51.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:51 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:51.418+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:51.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:51 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:51.418+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:58:51.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:51 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:51.418+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:28.542960+0000 front 2024-09-19T04:58:28.542864+0000 (oldest deadline 2024-09-19T04:58:50.842686+0000) 2024-09-19T04:58:52.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:51 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:51.885+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:52.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:51 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:51.885+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:52.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:51 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:51.885+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:52.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:51 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:51.885+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:58:52.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:51 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:51.991+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:52.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:51 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:51.991+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:52.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:51 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:51.991+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.042583+0000 front 2024-09-19T04:58:18.042550+0000 (oldest deadline 2024-09-19T04:58:45.042800+0000) 2024-09-19T04:58:52.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:51 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:51.967+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:52.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:51 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:51.967+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:52.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:51 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:51.967+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:52.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:51 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:51.967+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:58:52.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:52 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:52.103+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:52.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:52 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:52.103+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:52.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:52 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:52.103+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:52.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:52 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:52.103+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:58:52.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:52 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:52.103+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:58:52.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:52 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:52.402+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:52.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:52 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:52.402+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:52.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:52 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:52.402+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:58:52.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:52 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:52.402+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:28.542960+0000 front 2024-09-19T04:58:28.542864+0000 (oldest deadline 2024-09-19T04:58:50.842686+0000) 2024-09-19T04:58:53.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:52 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:52.894+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:53.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:52 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:52.894+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:53.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:52 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:52.894+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:53.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:52 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:52.894+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:58:53.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:53 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:53.001+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:53.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:53 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:53.001+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:53.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:53 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:53.001+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.042583+0000 front 2024-09-19T04:58:18.042550+0000 (oldest deadline 2024-09-19T04:58:45.042800+0000) 2024-09-19T04:58:53.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:52 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:52.940+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:53.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:52 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:52.940+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:53.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:52 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:52.940+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:53.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:52 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:52.940+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:58:53.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:52 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:52.940+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:58:53.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:53 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:53.141+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:53.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:53 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:53.141+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:53.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:53 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:53.141+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:53.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:53 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:53.141+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:58:53.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:53 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:53.141+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:58:53.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:53 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:53.369+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:53.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:53 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:53.369+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:53.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:53 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:53.369+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:58:53.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:53 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:53.369+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:28.542960+0000 front 2024-09-19T04:58:28.542864+0000 (oldest deadline 2024-09-19T04:58:50.842686+0000) 2024-09-19T04:58:54.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:53 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:53.886+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:54.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:53 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:53.886+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:54.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:53 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:53.886+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:54.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:53 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:53.886+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:58:54.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:54 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:54.044+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.638805+0000 front 2024-09-19T04:58:00.638824+0000 (oldest deadline 2024-09-19T04:58:30.639622+0000) 2024-09-19T04:58:54.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:54 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:54.044+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.640183+0000 front 2024-09-19T04:58:10.640225+0000 (oldest deadline 2024-09-19T04:58:35.340194+0000) 2024-09-19T04:58:54.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:54 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:54.044+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.042583+0000 front 2024-09-19T04:58:18.042550+0000 (oldest deadline 2024-09-19T04:58:45.042800+0000) 2024-09-19T04:58:54.350 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:54 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7[52741]: 2024-09-19T04:58:54.044+0000 7f55f8a37640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.644167+0000 front 2024-09-19T04:58:29.644234+0000 (oldest deadline 2024-09-19T04:58:53.744353+0000) 2024-09-19T04:58:54.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:53 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:53.893+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:54.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:53 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:53.893+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:54.351 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:53 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:53.893+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:54.351 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:53 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:53.893+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:58:54.351 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:53 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:53.893+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:58:54.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:54 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:54.118+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:54.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:54 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:54.118+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:54.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:54 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:54.118+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:54.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:54 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:54.118+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:58:54.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:54 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:54.118+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:58:54.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:54 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:54.405+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:54.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:54 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:54.405+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:54.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:54 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:54.405+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:58:54.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:54 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:54.405+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:28.542960+0000 front 2024-09-19T04:58:28.542864+0000 (oldest deadline 2024-09-19T04:58:50.842686+0000) 2024-09-19T04:58:55.100 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:54 smithi183 podman[71157]: 2024-09-19 04:58:54.827001669 +0000 UTC m=+5.644204930 container died 765461ee2d5a5048e67dcd521590d8df8b252bb169a4fb3c71648eaf0dce6dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-19T04:58:55.101 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:54 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:54.860+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:55.101 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:54 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:54.860+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:55.101 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:54 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:54.860+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:55.101 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:54 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:54.860+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:58:55.101 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:54 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:54.860+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:58:55.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:54 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:54.891+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:55.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:54 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:54.891+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:55.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:54 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:54.891+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:55.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:54 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:54.891+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:58:55.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:54 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:54.891+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:58:55.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:55 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:55.139+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:55.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:55 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:55.139+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:55.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:55 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:55.139+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:55.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:55 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:55.139+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:58:55.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:55 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:55.139+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:58:55.500 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:55 smithi183 podman[71157]: 2024-09-19 04:58:55.206863468 +0000 UTC m=+6.024066728 container cleanup 765461ee2d5a5048e67dcd521590d8df8b252bb169a4fb3c71648eaf0dce6dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7, RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T04:58:55.500 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:55 smithi183 bash[71157]: ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7 2024-09-19T04:58:55.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:55 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:55.410+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:55.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:55 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:55.410+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:55.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:55 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:55.410+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:58:55.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:55 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:55.410+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:28.542960+0000 front 2024-09-19T04:58:28.542864+0000 (oldest deadline 2024-09-19T04:58:50.842686+0000) 2024-09-19T04:58:55.827 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:55 smithi183 podman[71171]: 2024-09-19 04:58:55.499785423 +0000 UTC m=+0.668148084 container remove 765461ee2d5a5048e67dcd521590d8df8b252bb169a4fb3c71648eaf0dce6dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-19T04:58:56.100 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:55 smithi183 podman[71261]: 2024-09-19 04:58:55.729610972 +0000 UTC m=+0.015813525 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:58:56.100 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:55 smithi183 podman[71261]: 2024-09-19 04:58:55.858969858 +0000 UTC m=+0.145172413 container create de1a8f83f320497ea7e812d6dbd90fb06f28427a566fdd5302ff3fc584e1831e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T04:58:56.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:55 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:55.829+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:56.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:55 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:55.829+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:56.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:55 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:55.829+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:56.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:55 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:55.829+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:58:56.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:55 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:55.829+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:58:56.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:55 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:55.900+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:56.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:55 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:55.900+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:56.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:55 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:55.900+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:56.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:55 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:55.900+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:58:56.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:55 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:55.900+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:58:56.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:56 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:56.091+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:56.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:56 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:56.091+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:56.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:56 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:56.091+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:56.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:56 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:56.091+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:58:56.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:56 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:56.091+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:58:56.598 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:56 smithi183 podman[71261]: 2024-09-19 04:58:56.115637163 +0000 UTC m=+0.401839715 container init de1a8f83f320497ea7e812d6dbd90fb06f28427a566fdd5302ff3fc584e1831e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T04:58:56.599 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:56 smithi183 podman[71261]: 2024-09-19 04:58:56.118972392 +0000 UTC m=+0.405174946 container start de1a8f83f320497ea7e812d6dbd90fb06f28427a566fdd5302ff3fc584e1831e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T04:58:56.599 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:56 smithi183 podman[71261]: 2024-09-19 04:58:56.203045473 +0000 UTC m=+0.489248028 container attach de1a8f83f320497ea7e812d6dbd90fb06f28427a566fdd5302ff3fc584e1831e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-19T04:58:56.599 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 19 04:58:56 smithi183 podman[71261]: 2024-09-19 04:58:56.308757569 +0000 UTC m=+0.594960124 container died de1a8f83f320497ea7e812d6dbd90fb06f28427a566fdd5302ff3fc584e1831e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-19T04:58:56.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:56 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:56.413+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:56.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:56 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:56.413+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:56.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:56 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:56.413+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:58:56.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:56 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:56.413+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:28.542960+0000 front 2024-09-19T04:58:28.542864+0000 (oldest deadline 2024-09-19T04:58:50.842686+0000) 2024-09-19T04:58:56.937 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T04:58:56.975 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:58:56.975 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T04:58:56.976 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-19T04:58:56.976 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.8 2024-09-19T04:58:57.241 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:57 smithi193 systemd[1]: Stopping Ceph osd.8 for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:58:57.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:56 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:56.938+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:57.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:56 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:56.938+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:57.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:56 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:56.938+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:57.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:56 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:56.938+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:58:57.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:56 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:56.938+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:58:57.274 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:56 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:56.856+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:57.275 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:56 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:56.856+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:57.275 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:56 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:56.856+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:57.275 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:56 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:56.856+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:58:57.275 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:56 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:56.856+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:58:57.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:57 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:57.078+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:57.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:57 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:57.078+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:57.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:57 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:57.078+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:57.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:57 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:57.078+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:58:57.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:57 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:57.078+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:58:57.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:57 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:57.299+0000 7f255d8e9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:58:57.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:57 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:57.299+0000 7f255d8e9640 -1 osd.8 100 *** Got signal Terminated *** 2024-09-19T04:58:57.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:57 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:57.299+0000 7f255d8e9640 -1 osd.8 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:58:57.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:57 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:57.367+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:57.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:57 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:57.367+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:57.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:57 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:57.367+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:58:57.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:57 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:57.367+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:28.542960+0000 front 2024-09-19T04:58:28.542864+0000 (oldest deadline 2024-09-19T04:58:50.842686+0000) 2024-09-19T04:58:58.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:57 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:57.814+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:58.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:57 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:57.814+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:58.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:57 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:57.814+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:58.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:57 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:57.814+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:58:58.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:57 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:57.814+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:58:58.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:57 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:57.940+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:58.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:57 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:57.940+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:58.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:57 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:57.940+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:58.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:57 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:57.940+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:58:58.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:57 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:57.940+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:58:58.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:58 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:58.086+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:58.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:58 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:58.086+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:58.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:58 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:58.086+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:58.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:58 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:58.086+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:58:58.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:58 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:58.086+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:58:58.513 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:58 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:58.406+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:58.513 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:58 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:58.406+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:58.513 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:58 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:58.406+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:58:58.513 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:58 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:58.406+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:28.542960+0000 front 2024-09-19T04:58:28.542864+0000 (oldest deadline 2024-09-19T04:58:50.842686+0000) 2024-09-19T04:58:59.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:58 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:58.831+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:58:59.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:58 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:58.831+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:58:59.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:58 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:58.831+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:58:59.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:58 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:58.831+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:58:59.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:58 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:58.831+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:58:59.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:58 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:58.958+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:58:59.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:58 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:58.958+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:58:59.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:58 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:58.958+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:58:59.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:58 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:58.958+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:58:59.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:58 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:58.958+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:58:59.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:59 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:59.081+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:58:59.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:59 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:59.081+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:58:59.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:59 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:59.081+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:58:59.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:59 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:59.081+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:58:59.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:58:59 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:58:59.081+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:58:59.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:59 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:59.429+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:58:59.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:59 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:59.429+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:58:59.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:59 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:59.429+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:58:59.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:58:59 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:58:59.429+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:28.542960+0000 front 2024-09-19T04:58:28.542864+0000 (oldest deadline 2024-09-19T04:58:50.842686+0000) 2024-09-19T04:59:00.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:59 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:59.789+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:00.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:59 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:59.789+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:00.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:59 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:59.789+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:00.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:59 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:59.789+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:00.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:58:59 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:58:59.789+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:00.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:59 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:59.964+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:00.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:59 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:59.964+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:00.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:59 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:59.964+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:00.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:59 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:59.964+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:00.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:59 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:59.964+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:00.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:58:59 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:58:59.964+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:00.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:00 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:00.047+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:59:00.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:00 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:00.047+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:59:00.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:00 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:00.047+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:59:00.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:00 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:00.047+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:59:00.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:00 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:00.047+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:59:00.428 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:00 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:00.047+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:35.114445+0000 front 2024-09-19T04:58:35.114486+0000 (oldest deadline 2024-09-19T04:58:59.814295+0000) 2024-09-19T04:59:00.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:00 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:59:00.459+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:59:00.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:00 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:59:00.459+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:59:00.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:00 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:59:00.459+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:59:00.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:00 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:59:00.459+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:28.542960+0000 front 2024-09-19T04:58:28.542864+0000 (oldest deadline 2024-09-19T04:58:50.842686+0000) 2024-09-19T04:59:00.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:00 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:59:00.459+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:36.643885+0000 front 2024-09-19T04:58:36.643937+0000 (oldest deadline 2024-09-19T04:58:59.544014+0000) 2024-09-19T04:59:01.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:00 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:00.770+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:01.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:00 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:00.770+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:01.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:00 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:00.770+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:01.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:00 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:00.770+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:01.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:00 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:00.770+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:01.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:00 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:00.932+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:01.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:00 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:00.932+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:01.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:00 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:00.932+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:01.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:00 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:00.932+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:01.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:00 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:00.932+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:01.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:00 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:00.932+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:01.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:01 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:01.012+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:59:01.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:01 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:01.012+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:59:01.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:01 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:01.012+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:59:01.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:01 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:01.012+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:59:01.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:01 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:01.012+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:59:01.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:01 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:01.012+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:35.114445+0000 front 2024-09-19T04:58:35.114486+0000 (oldest deadline 2024-09-19T04:58:59.814295+0000) 2024-09-19T04:59:01.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:01 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:59:01.427+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:02.938678+0000 front 2024-09-19T04:58:02.938813+0000 (oldest deadline 2024-09-19T04:58:27.638660+0000) 2024-09-19T04:59:01.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:01 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:59:01.427+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:11.040150+0000 front 2024-09-19T04:58:11.040204+0000 (oldest deadline 2024-09-19T04:58:35.140286+0000) 2024-09-19T04:59:01.740 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:01 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:59:01.427+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.541833+0000 front 2024-09-19T04:58:18.541868+0000 (oldest deadline 2024-09-19T04:58:43.241827+0000) 2024-09-19T04:59:01.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:01 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:59:01.427+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:28.542960+0000 front 2024-09-19T04:58:28.542864+0000 (oldest deadline 2024-09-19T04:58:50.842686+0000) 2024-09-19T04:59:01.741 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:01 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8[53842]: 2024-09-19T04:59:01.427+0000 7f2559eff640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:36.643885+0000 front 2024-09-19T04:58:36.643937+0000 (oldest deadline 2024-09-19T04:58:59.544014+0000) 2024-09-19T04:59:02.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:01 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:01.810+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:02.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:01 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:01.810+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:02.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:01 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:01.811+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:02.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:01 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:01.811+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:02.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:01 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:01.811+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:02.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:01 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:01.907+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:02.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:01 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:01.907+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:02.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:01 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:01.907+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:02.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:01 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:01.907+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:02.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:01 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:01.907+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:02.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:01 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:01.907+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:02.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:01 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:01.994+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:59:02.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:01 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:01.994+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:59:02.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:01 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:01.994+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:59:02.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:01 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:01.994+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:59:02.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:01 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:01.994+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:59:02.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:01 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:01.994+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:35.114445+0000 front 2024-09-19T04:58:35.114486+0000 (oldest deadline 2024-09-19T04:58:59.814295+0000) 2024-09-19T04:59:02.990 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:02 smithi193 podman[72206]: 2024-09-19 04:59:02.71778495 +0000 UTC m=+5.548176025 container died 4ec0e29819bedbda95f7cf413a28304830d2a999572b2790db47b4021c509d8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T04:59:02.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:02 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:02.861+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:02.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:02 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:02.861+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:02.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:02 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:02.861+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:02.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:02 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:02.861+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:02.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:02 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:02.861+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:02.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:02 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:02.861+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:03.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:02 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:02.775+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:03.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:02 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:02.775+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:03.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:02 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:02.775+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:03.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:02 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:02.775+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:03.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:02 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:02.775+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:03.377 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:03 smithi193 podman[72206]: 2024-09-19 04:59:03.124954544 +0000 UTC m=+5.955345661 container cleanup 4ec0e29819bedbda95f7cf413a28304830d2a999572b2790db47b4021c509d8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T04:59:03.377 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:03 smithi193 bash[72206]: ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8 2024-09-19T04:59:03.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:03 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:03.034+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:59:03.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:03 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:03.034+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:59:03.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:03 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:03.034+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:59:03.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:03 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:03.034+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:59:03.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:03 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:03.034+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:59:03.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:03 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:03.034+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:35.114445+0000 front 2024-09-19T04:58:35.114486+0000 (oldest deadline 2024-09-19T04:58:59.814295+0000) 2024-09-19T04:59:03.687 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:03 smithi193 podman[72234]: 2024-09-19 04:59:03.37640544 +0000 UTC m=+0.655218904 container remove 4ec0e29819bedbda95f7cf413a28304830d2a999572b2790db47b4021c509d8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T04:59:03.938 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:03 smithi193 podman[72323]: 2024-09-19 04:59:03.589030616 +0000 UTC m=+0.015793231 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:59:03.938 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:03 smithi193 podman[72323]: 2024-09-19 04:59:03.694085064 +0000 UTC m=+0.120847711 container create c7ef3ae1fd825c6d03ec6f7e6af636551e214d2f17e90bb454d0d0fb4990bdac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T04:59:03.939 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:03 smithi193 podman[72323]: 2024-09-19 04:59:03.938079351 +0000 UTC m=+0.364841956 container init c7ef3ae1fd825c6d03ec6f7e6af636551e214d2f17e90bb454d0d0fb4990bdac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T04:59:03.939 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:03 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:03.831+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:03.939 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:03 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:03.831+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:03.939 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:03 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:03.831+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:03.939 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:03 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:03.831+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:03.939 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:03 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:03.831+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:03.939 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:03 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:03.831+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:04.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:03 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:03.793+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:04.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:03 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:03.793+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:04.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:03 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:03.793+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:04.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:03 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:03.793+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:04.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:03 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:03.793+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:04.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:03 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:03.793+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:32.246974+0000 front 2024-09-19T04:58:32.246889+0000 (oldest deadline 2024-09-19T04:59:03.447214+0000) 2024-09-19T04:59:04.241 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:03 smithi193 podman[72323]: 2024-09-19 04:59:03.941660899 +0000 UTC m=+0.368423506 container start c7ef3ae1fd825c6d03ec6f7e6af636551e214d2f17e90bb454d0d0fb4990bdac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T04:59:04.241 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:04 smithi193 podman[72323]: 2024-09-19 04:59:04.029815887 +0000 UTC m=+0.456578491 container attach c7ef3ae1fd825c6d03ec6f7e6af636551e214d2f17e90bb454d0d0fb4990bdac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:59:04.241 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 19 04:59:04 smithi193 podman[72323]: 2024-09-19 04:59:04.132211247 +0000 UTC m=+0.558973852 container died c7ef3ae1fd825c6d03ec6f7e6af636551e214d2f17e90bb454d0d0fb4990bdac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:59:04.328 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:04 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:04.021+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:59:04.329 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:04 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:04.021+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:59:04.329 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:04 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:04.021+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:59:04.329 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:04 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:04.021+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:59:04.329 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:04 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:04.021+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:59:04.329 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:04 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:04.021+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:35.114445+0000 front 2024-09-19T04:58:35.114486+0000 (oldest deadline 2024-09-19T04:58:59.814295+0000) 2024-09-19T04:59:04.806 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.8.service' 2024-09-19T04:59:04.847 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:59:04.847 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-19T04:59:04.847 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-19T04:59:04.847 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.9 2024-09-19T04:59:05.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:04 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:04.822+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:05.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:04 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:04.822+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:05.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:04 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:04.822+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:05.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:04 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:04.822+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:05.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:04 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:04.822+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:05.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:04 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:04.822+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:32.246974+0000 front 2024-09-19T04:58:32.246889+0000 (oldest deadline 2024-09-19T04:59:03.447214+0000) 2024-09-19T04:59:05.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:04 smithi123 systemd[1]: Stopping Ceph osd.9 for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:59:05.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:05 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:05.008+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:59:05.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:05 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:05.008+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:59:05.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:05 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:05.008+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:59:05.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:05 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:05.008+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:59:05.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:05 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:05.008+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:59:05.178 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:05 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:05.008+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:35.114445+0000 front 2024-09-19T04:58:35.114486+0000 (oldest deadline 2024-09-19T04:58:59.814295+0000) 2024-09-19T04:59:05.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:04 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:04.788+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:05.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:04 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:04.788+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:05.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:04 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:04.788+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:05.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:04 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:04.788+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:05.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:04 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:04.788+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:05.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:04 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:04.788+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:05.515 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:05 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:05.287+0000 7fb566e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:59:05.516 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:05 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:05.287+0000 7fb566e00640 -1 osd.9 100 *** Got signal Terminated *** 2024-09-19T04:59:05.516 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:05 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:05.287+0000 7fb566e00640 -1 osd.9 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:59:06.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:05 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:05.799+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:06.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:05 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:05.799+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:06.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:05 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:05.799+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:06.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:05 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:05.799+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:06.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:05 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:05.799+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:06.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:05 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:05.799+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:32.246974+0000 front 2024-09-19T04:58:32.246889+0000 (oldest deadline 2024-09-19T04:59:03.447214+0000) 2024-09-19T04:59:06.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:05 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:05.741+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:06.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:05 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:05.741+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:06.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:05 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:05.741+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:06.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:05 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:05.741+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:06.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:05 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:05.741+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:06.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:05 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:05.741+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:06.357 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:05 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:05.983+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:59:06.357 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:05 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:05.983+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:59:06.357 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:05 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:05.983+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:59:06.358 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:05 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:05.983+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:59:06.358 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:05 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:05.983+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:59:06.358 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:05 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:05.983+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:35.114445+0000 front 2024-09-19T04:58:35.114486+0000 (oldest deadline 2024-09-19T04:58:59.814295+0000) 2024-09-19T04:59:06.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:06 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:06.715+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:06.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:06 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:06.715+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:06.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:06 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:06.715+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:06.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:06 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:06.715+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:06.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:06 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:06.715+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:06.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:06 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:06.715+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:07.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:06 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:06.831+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:07.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:06 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:06.831+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:07.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:06 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:06.831+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:07.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:06 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:06.831+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:07.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:06 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:06.831+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:07.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:06 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:06.831+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:32.246974+0000 front 2024-09-19T04:58:32.246889+0000 (oldest deadline 2024-09-19T04:59:03.447214+0000) 2024-09-19T04:59:07.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:06 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:06.948+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:59:07.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:06 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:06.948+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:59:07.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:06 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:06.948+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:59:07.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:06 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:06.948+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:59:07.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:06 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:06.948+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:59:07.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:06 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:06.948+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:35.114445+0000 front 2024-09-19T04:58:35.114486+0000 (oldest deadline 2024-09-19T04:58:59.814295+0000) 2024-09-19T04:59:07.428 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:06 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:06.948+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-09-19T04:58:41.515095+0000 front 2024-09-19T04:58:41.515119+0000 (oldest deadline 2024-09-19T04:59:06.815132+0000) 2024-09-19T04:59:07.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:07 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:07.678+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:07.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:07 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:07.678+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:07.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:07 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:07.678+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:07.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:07 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:07.678+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:07.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:07 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:07.678+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:07.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:07 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:07.678+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:08.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:07 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:07.850+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:08.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:07 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:07.850+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:08.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:07 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:07.850+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:08.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:07 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:07.850+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:08.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:07 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:07.850+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:08.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:07 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:07.850+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:32.246974+0000 front 2024-09-19T04:58:32.246889+0000 (oldest deadline 2024-09-19T04:59:03.447214+0000) 2024-09-19T04:59:08.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:07 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:07.907+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:59:08.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:07 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:07.907+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:59:08.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:07 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:07.907+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:59:08.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:07 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:07.907+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:59:08.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:07 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:07.907+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:59:08.178 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:07 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:07.907+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:35.114445+0000 front 2024-09-19T04:58:35.114486+0000 (oldest deadline 2024-09-19T04:58:59.814295+0000) 2024-09-19T04:59:08.178 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:07 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:07.907+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-09-19T04:58:41.515095+0000 front 2024-09-19T04:58:41.515119+0000 (oldest deadline 2024-09-19T04:59:06.815132+0000) 2024-09-19T04:59:08.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:08 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:08.676+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:08.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:08 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:08.676+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:08.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:08 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:08.676+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:08.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:08 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:08.676+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:08.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:08 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:08.676+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:08.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:08 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:08.676+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:09.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:08 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:08.845+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:09.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:08 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:08.845+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:09.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:08 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:08.845+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:09.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:08 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:08.845+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:09.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:08 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:08.845+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:09.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:08 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:08.845+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:32.246974+0000 front 2024-09-19T04:58:32.246889+0000 (oldest deadline 2024-09-19T04:59:03.447214+0000) 2024-09-19T04:59:09.101 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:08 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:08.845+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-09-19T04:58:43.447920+0000 front 2024-09-19T04:58:43.448063+0000 (oldest deadline 2024-09-19T04:59:08.147642+0000) 2024-09-19T04:59:09.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:08 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:08.908+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:59:09.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:08 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:08.908+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:59:09.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:08 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:08.908+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:59:09.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:08 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:08.908+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:59:09.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:08 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:08.908+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:59:09.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:08 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:08.908+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:35.114445+0000 front 2024-09-19T04:58:35.114486+0000 (oldest deadline 2024-09-19T04:58:59.814295+0000) 2024-09-19T04:59:09.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:08 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:08.908+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-09-19T04:58:41.515095+0000 front 2024-09-19T04:58:41.515119+0000 (oldest deadline 2024-09-19T04:59:06.815132+0000) 2024-09-19T04:59:09.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:09 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:09.647+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:09.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:09 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:09.647+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:09.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:09 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:09.647+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:09.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:09 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:09.647+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:09.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:09 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:09.647+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:09.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:09 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:09.647+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:10.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:09 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:09.921+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.307456+0000 front 2024-09-19T04:57:54.307541+0000 (oldest deadline 2024-09-19T04:58:17.807458+0000) 2024-09-19T04:59:10.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:09 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:09.921+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.609238+0000 front 2024-09-19T04:58:03.609103+0000 (oldest deadline 2024-09-19T04:58:25.909227+0000) 2024-09-19T04:59:10.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:09 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:09.921+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.610593+0000 front 2024-09-19T04:58:10.610705+0000 (oldest deadline 2024-09-19T04:58:34.710343+0000) 2024-09-19T04:59:10.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:09 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:09.921+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:20.011626+0000 front 2024-09-19T04:58:20.011534+0000 (oldest deadline 2024-09-19T04:58:42.911546+0000) 2024-09-19T04:59:10.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:09 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:09.921+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:25.812777+0000 front 2024-09-19T04:58:25.812885+0000 (oldest deadline 2024-09-19T04:58:50.512845+0000) 2024-09-19T04:59:10.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:09 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:09.921+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:35.114445+0000 front 2024-09-19T04:58:35.114486+0000 (oldest deadline 2024-09-19T04:58:59.814295+0000) 2024-09-19T04:59:10.177 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:09 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9[63975]: 2024-09-19T04:59:09.921+0000 7fb55d800640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-09-19T04:58:41.515095+0000 front 2024-09-19T04:58:41.515119+0000 (oldest deadline 2024-09-19T04:59:06.815132+0000) 2024-09-19T04:59:10.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:09 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:09.895+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:10.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:09 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:09.895+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:10.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:09 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:09.895+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:10.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:09 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:09.895+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:10.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:09 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:09.895+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:10.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:09 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:09.895+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:32.246974+0000 front 2024-09-19T04:58:32.246889+0000 (oldest deadline 2024-09-19T04:59:03.447214+0000) 2024-09-19T04:59:10.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:09 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:09.895+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-09-19T04:58:43.447920+0000 front 2024-09-19T04:58:43.448063+0000 (oldest deadline 2024-09-19T04:59:08.147642+0000) 2024-09-19T04:59:10.962 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:10 smithi123 podman[127551]: 2024-09-19 04:59:10.695585524 +0000 UTC m=+5.542699372 container died 3c3100f10775f82ff5536a994babf24b3b137607a0778a2128506ed1c8f97c0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T04:59:10.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:10 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:10.619+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:10.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:10 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:10.619+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:10.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:10 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:10.619+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:10.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:10 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:10.619+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:10.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:10 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:10.619+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:10.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:10 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:10.619+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:11.310 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:11 smithi123 podman[127551]: 2024-09-19 04:59:11.166303261 +0000 UTC m=+6.013417106 container cleanup 3c3100f10775f82ff5536a994babf24b3b137607a0778a2128506ed1c8f97c0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T04:59:11.310 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:11 smithi123 bash[127551]: ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9 2024-09-19T04:59:11.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:10 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:10.851+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:11.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:10 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:10.851+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:11.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:10 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:10.851+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:11.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:10 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:10.851+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:11.350 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:10 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:10.851+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:11.351 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:10 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:10.851+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:32.246974+0000 front 2024-09-19T04:58:32.246889+0000 (oldest deadline 2024-09-19T04:59:03.447214+0000) 2024-09-19T04:59:11.351 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:10 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:10.851+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-09-19T04:58:43.447920+0000 front 2024-09-19T04:58:43.448063+0000 (oldest deadline 2024-09-19T04:59:08.147642+0000) 2024-09-19T04:59:11.677 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:11 smithi123 podman[127577]: 2024-09-19 04:59:11.309322464 +0000 UTC m=+0.599801338 container remove 3c3100f10775f82ff5536a994babf24b3b137607a0778a2128506ed1c8f97c0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-19T04:59:11.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:11 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:11.579+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:11.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:11 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:11.579+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:11.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:11 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:11.579+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:11.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:11 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:11.579+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:11.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:11 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:11.579+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:11.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:11 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:11.579+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:12.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:11 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:11.835+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:12.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:11 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:11.835+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:12.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:11 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:11.835+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:12.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:11 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:11.835+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:12.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:11 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:11.835+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:12.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:11 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:11.835+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:32.246974+0000 front 2024-09-19T04:58:32.246889+0000 (oldest deadline 2024-09-19T04:59:03.447214+0000) 2024-09-19T04:59:12.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:11 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:11.835+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-09-19T04:58:43.447920+0000 front 2024-09-19T04:58:43.448063+0000 (oldest deadline 2024-09-19T04:59:08.147642+0000) 2024-09-19T04:59:12.154 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:11 smithi123 podman[127667]: 2024-09-19 04:59:11.851721626 +0000 UTC m=+0.121808498 container create db5271f2fac1aaed13e0c1fa668f5a8480e1d26d929bb07aa8ecb8c193283ba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:59:12.155 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:11 smithi123 podman[127667]: 2024-09-19 04:59:11.757027867 +0000 UTC m=+0.027114741 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:59:12.155 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:12 smithi123 podman[127667]: 2024-09-19 04:59:12.079528404 +0000 UTC m=+0.349615277 container init db5271f2fac1aaed13e0c1fa668f5a8480e1d26d929bb07aa8ecb8c193283ba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-19T04:59:12.155 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:12 smithi123 podman[127667]: 2024-09-19 04:59:12.086474195 +0000 UTC m=+0.356561069 container start db5271f2fac1aaed13e0c1fa668f5a8480e1d26d929bb07aa8ecb8c193283ba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T04:59:12.155 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:12 smithi123 podman[127667]: 2024-09-19 04:59:12.154127941 +0000 UTC m=+0.424214814 container attach db5271f2fac1aaed13e0c1fa668f5a8480e1d26d929bb07aa8ecb8c193283ba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-19T04:59:12.427 INFO:journalctl@ceph.osd.9.smithi123.stdout:Sep 19 04:59:12 smithi123 podman[127667]: 2024-09-19 04:59:12.339565322 +0000 UTC m=+0.609652196 container died db5271f2fac1aaed13e0c1fa668f5a8480e1d26d929bb07aa8ecb8c193283ba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T04:59:12.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:12 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:12.566+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:12.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:12 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:12.566+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:12.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:12 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:12.566+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:12.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:12 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:12.566+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:12.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:12 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:12.566+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:12.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:12 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:12.566+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:13.061 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.9.service' 2024-09-19T04:59:13.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:12 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:12.796+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:13.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:12 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:12.796+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:13.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:12 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:12.796+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:13.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:12 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:12.796+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:13.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:12 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:12.796+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:13.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:12 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:12.796+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:32.246974+0000 front 2024-09-19T04:58:32.246889+0000 (oldest deadline 2024-09-19T04:59:03.447214+0000) 2024-09-19T04:59:13.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:12 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:12.796+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-09-19T04:58:43.447920+0000 front 2024-09-19T04:58:43.448063+0000 (oldest deadline 2024-09-19T04:59:08.147642+0000) 2024-09-19T04:59:13.137 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:59:13.137 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-19T04:59:13.137 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-19T04:59:13.137 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.10 2024-09-19T04:59:13.450 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:13 smithi183 systemd[1]: Stopping Ceph osd.10 for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:59:13.760 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:13 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:13.449+0000 7f8be3a74640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:59:13.760 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:13 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:13.449+0000 7f8be3a74640 -1 osd.10 100 *** Got signal Terminated *** 2024-09-19T04:59:13.761 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:13 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:13.449+0000 7f8be3a74640 -1 osd.10 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:59:13.761 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:13 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:13.759+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:13.761 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:13 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:13.759+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:13.761 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:13 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:13.759+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:13.761 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:13 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:13.759+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:13.761 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:13 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:13.759+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:13.761 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:13 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:13.759+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:32.246974+0000 front 2024-09-19T04:58:32.246889+0000 (oldest deadline 2024-09-19T04:59:03.447214+0000) 2024-09-19T04:59:13.761 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:13 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:13.759+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-09-19T04:58:43.447920+0000 front 2024-09-19T04:58:43.448063+0000 (oldest deadline 2024-09-19T04:59:08.147642+0000) 2024-09-19T04:59:13.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:13 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:13.568+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:13.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:13 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:13.568+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:13.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:13 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:13.568+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:13.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:13 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:13.568+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:13.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:13 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:13.568+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:13.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:13 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:13.568+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:14.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:14 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:14.519+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:14.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:14 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:14.519+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:14.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:14 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:14.519+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:14.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:14 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:14.519+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:14.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:14 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:14.519+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:14.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:14 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:14.519+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:15.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:14 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:14.801+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:15.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:14 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:14.801+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:15.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:14 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:14.801+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:15.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:14 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:14.801+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:15.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:14 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:14.801+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:15.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:14 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:14.801+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:32.246974+0000 front 2024-09-19T04:58:32.246889+0000 (oldest deadline 2024-09-19T04:59:03.447214+0000) 2024-09-19T04:59:15.101 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:14 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:14.801+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-09-19T04:58:43.447920+0000 front 2024-09-19T04:58:43.448063+0000 (oldest deadline 2024-09-19T04:59:08.147642+0000) 2024-09-19T04:59:15.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:15 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:15.521+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:15.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:15 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:15.521+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:15.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:15 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:15.521+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:15.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:15 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:15.521+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:15.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:15 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:15.521+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:15.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:15 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:15.521+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:16.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:15 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:15.797+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:16.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:15 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:15.797+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:16.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:15 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:15.797+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:16.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:15 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:15.797+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:16.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:15 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:15.797+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:16.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:15 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:15.797+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:32.246974+0000 front 2024-09-19T04:58:32.246889+0000 (oldest deadline 2024-09-19T04:59:03.447214+0000) 2024-09-19T04:59:16.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:15 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:15.797+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-09-19T04:58:43.447920+0000 front 2024-09-19T04:58:43.448063+0000 (oldest deadline 2024-09-19T04:59:08.147642+0000) 2024-09-19T04:59:16.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:16 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:16.554+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:16.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:16 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:16.554+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:16.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:16 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:16.554+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:16.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:16 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:16.554+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:16.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:16 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:16.554+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:16.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:16 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:16.554+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:17.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:16 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:16.842+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:17.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:16 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:16.842+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:17.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:16 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:16.842+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:17.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:16 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:16.842+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:17.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:16 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:16.842+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:17.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:16 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:16.842+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:32.246974+0000 front 2024-09-19T04:58:32.246889+0000 (oldest deadline 2024-09-19T04:59:03.447214+0000) 2024-09-19T04:59:17.101 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:16 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:16.842+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-09-19T04:58:43.447920+0000 front 2024-09-19T04:58:43.448063+0000 (oldest deadline 2024-09-19T04:59:08.147642+0000) 2024-09-19T04:59:17.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:17 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:17.528+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:17.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:17 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:17.528+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:17.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:17 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:17.528+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:17.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:17 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:17.528+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:17.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:17 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:17.528+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:17.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:17 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:17.528+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:18.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:17 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:17.836+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:56.440155+0000 front 2024-09-19T04:57:56.440094+0000 (oldest deadline 2024-09-19T04:58:19.940138+0000) 2024-09-19T04:59:18.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:17 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:17.836+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:03.841858+0000 front 2024-09-19T04:58:03.841995+0000 (oldest deadline 2024-09-19T04:58:26.742064+0000) 2024-09-19T04:59:18.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:17 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:17.836+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:10.743447+0000 front 2024-09-19T04:58:10.743315+0000 (oldest deadline 2024-09-19T04:58:34.143936+0000) 2024-09-19T04:59:18.100 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:17 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:17.836+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.745359+0000 front 2024-09-19T04:58:18.745375+0000 (oldest deadline 2024-09-19T04:58:42.845383+0000) 2024-09-19T04:59:18.101 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:17 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:17.836+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:26.946267+0000 front 2024-09-19T04:58:26.946226+0000 (oldest deadline 2024-09-19T04:58:52.246306+0000) 2024-09-19T04:59:18.101 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:17 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:17.836+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:32.246974+0000 front 2024-09-19T04:58:32.246889+0000 (oldest deadline 2024-09-19T04:59:03.447214+0000) 2024-09-19T04:59:18.101 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:17 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10[59738]: 2024-09-19T04:59:17.836+0000 7f8be008a640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-09-19T04:58:43.447920+0000 front 2024-09-19T04:58:43.448063+0000 (oldest deadline 2024-09-19T04:59:08.147642+0000) 2024-09-19T04:59:18.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:18 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:18.576+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:18.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:18 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:18.576+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:18.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:18 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:18.576+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:18.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:18 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:18.576+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:18.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:18 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:18.576+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:18.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:18 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:18.576+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:19.267 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:18 smithi183 podman[71378]: 2024-09-19 04:59:18.975911383 +0000 UTC m=+5.645388013 container died d0d5b581f1fff30d4037e911c647a1af3efa8d74dec02e2555fa8e1cf305969b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T04:59:19.600 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:19 smithi183 podman[71378]: 2024-09-19 04:59:19.340310632 +0000 UTC m=+6.009787261 container cleanup d0d5b581f1fff30d4037e911c647a1af3efa8d74dec02e2555fa8e1cf305969b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-09-19T04:59:19.600 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:19 smithi183 bash[71378]: ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10 2024-09-19T04:59:19.600 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:19 smithi183 podman[71395]: 2024-09-19 04:59:19.516260022 +0000 UTC m=+0.540421651 container remove d0d5b581f1fff30d4037e911c647a1af3efa8d74dec02e2555fa8e1cf305969b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T04:59:19.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:19 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:19.565+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:19.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:19 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:19.565+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:19.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:19 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:19.565+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:19.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:19 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:19.565+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:19.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:19 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:19.565+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:19.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:19 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:19.565+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:19.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:19 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:19.565+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-19T04:58:49.966906+0000 front 2024-09-19T04:58:49.966959+0000 (oldest deadline 2024-09-19T04:59:19.367275+0000) 2024-09-19T04:59:20.269 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:19 smithi183 podman[71481]: 2024-09-19 04:59:19.856028843 +0000 UTC m=+0.016266219 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:59:20.270 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:19 smithi183 podman[71481]: 2024-09-19 04:59:19.958755422 +0000 UTC m=+0.118992840 container create 0ec55d2dbad7bde06055de07e7f84bcb49dd00b03bf24542cb4f183e090c3c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-19T04:59:20.270 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:20 smithi183 podman[71481]: 2024-09-19 04:59:20.185801562 +0000 UTC m=+0.346038948 container init 0ec55d2dbad7bde06055de07e7f84bcb49dd00b03bf24542cb4f183e090c3c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2) 2024-09-19T04:59:20.270 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:20 smithi183 podman[71481]: 2024-09-19 04:59:20.189046711 +0000 UTC m=+0.349284088 container start 0ec55d2dbad7bde06055de07e7f84bcb49dd00b03bf24542cb4f183e090c3c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T04:59:20.270 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:20 smithi183 podman[71481]: 2024-09-19 04:59:20.269013305 +0000 UTC m=+0.429250686 container attach 0ec55d2dbad7bde06055de07e7f84bcb49dd00b03bf24542cb4f183e090c3c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T04:59:20.600 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 19 04:59:20 smithi183 podman[71481]: 2024-09-19 04:59:20.379063243 +0000 UTC m=+0.539300619 container died 0ec55d2dbad7bde06055de07e7f84bcb49dd00b03bf24542cb4f183e090c3c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-19T04:59:20.976 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.10.service' 2024-09-19T04:59:20.993 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:20 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:20.559+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:20.994 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:20 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:20.559+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:20.994 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:20 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:20.559+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:20.994 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:20 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:20.559+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:20.994 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:20 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:20.559+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:20.994 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:20 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:20.559+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:20.994 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:20 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:20.559+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-19T04:58:49.966906+0000 front 2024-09-19T04:58:49.966959+0000 (oldest deadline 2024-09-19T04:59:19.367275+0000) 2024-09-19T04:59:21.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:59:21.015 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-19T04:59:21.015 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-19T04:59:21.016 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.11 2024-09-19T04:59:21.322 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:21 smithi193 systemd[1]: Stopping Ceph osd.11 for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:59:21.740 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:21 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:21.320+0000 7f89a9670640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:59:21.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:21 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:21.320+0000 7f89a9670640 -1 osd.11 100 *** Got signal Terminated *** 2024-09-19T04:59:21.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:21 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:21.320+0000 7f89a9670640 -1 osd.11 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:59:21.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:21 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:21.509+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:21.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:21 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:21.509+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:21.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:21 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:21.509+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:21.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:21 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:21.509+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:21.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:21 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:21.509+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:21.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:21 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:21.509+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:21.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:21 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:21.509+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-19T04:58:49.966906+0000 front 2024-09-19T04:58:49.966959+0000 (oldest deadline 2024-09-19T04:59:19.367275+0000) 2024-09-19T04:59:22.740 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:22 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:22.483+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:22.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:22 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:22.483+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:22.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:22 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:22.483+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:22.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:22 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:22.483+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:22.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:22 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:22.483+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:22.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:22 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:22.483+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:22.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:22 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:22.483+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-19T04:58:49.966906+0000 front 2024-09-19T04:58:49.966959+0000 (oldest deadline 2024-09-19T04:59:19.367275+0000) 2024-09-19T04:59:23.740 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:23 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:23.456+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:23.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:23 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:23.456+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:23.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:23 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:23.456+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:23.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:23 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:23.456+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:23.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:23 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:23.456+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:23.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:23 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:23.456+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:23.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:23 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:23.456+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-19T04:58:49.966906+0000 front 2024-09-19T04:58:49.966959+0000 (oldest deadline 2024-09-19T04:59:19.367275+0000) 2024-09-19T04:59:24.740 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:24 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:24.484+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:24.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:24 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:24.484+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:24.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:24 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:24.484+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:24.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:24 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:24.484+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:24.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:24 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:24.484+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:24.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:24 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:24.484+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:24.741 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:24 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:24.484+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-19T04:58:49.966906+0000 front 2024-09-19T04:58:49.966959+0000 (oldest deadline 2024-09-19T04:59:19.367275+0000) 2024-09-19T04:59:25.990 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:25 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:25.496+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-09-19T04:57:54.860682+0000 front 2024-09-19T04:57:54.860542+0000 (oldest deadline 2024-09-19T04:58:17.760416+0000) 2024-09-19T04:59:25.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:25 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:25.496+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-19T04:58:00.661486+0000 front 2024-09-19T04:58:00.661469+0000 (oldest deadline 2024-09-19T04:58:26.561511+0000) 2024-09-19T04:59:25.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:25 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:25.496+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-19T04:58:12.363011+0000 front 2024-09-19T04:58:12.362886+0000 (oldest deadline 2024-09-19T04:58:36.463074+0000) 2024-09-19T04:59:25.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:25 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:25.496+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-09-19T04:58:18.764019+0000 front 2024-09-19T04:58:18.764041+0000 (oldest deadline 2024-09-19T04:58:44.664025+0000) 2024-09-19T04:59:25.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:25 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:25.496+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-19T04:58:29.965030+0000 front 2024-09-19T04:58:29.964965+0000 (oldest deadline 2024-09-19T04:58:54.664955+0000) 2024-09-19T04:59:25.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:25 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:25.496+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-19T04:58:34.665375+0000 front 2024-09-19T04:58:34.665416+0000 (oldest deadline 2024-09-19T04:58:59.965481+0000) 2024-09-19T04:59:25.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:25 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:25.496+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-19T04:58:49.966906+0000 front 2024-09-19T04:58:49.966959+0000 (oldest deadline 2024-09-19T04:59:19.367275+0000) 2024-09-19T04:59:25.991 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:25 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11[60371]: 2024-09-19T04:59:25.496+0000 7f89a1c7e640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2024-09-19T04:58:59.367584+0000 front 2024-09-19T04:58:59.367545+0000 (oldest deadline 2024-09-19T04:59:25.267635+0000) 2024-09-19T04:59:26.946 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:26 smithi193 podman[72425]: 2024-09-19 04:59:26.678486569 +0000 UTC m=+5.473035846 container died ace95eee0b0bc89ccd4b2b1acbcee1a0ffb4510ec4722e8e5c61e36db023ce39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T04:59:27.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:27 smithi193 podman[72425]: 2024-09-19 04:59:27.028719267 +0000 UTC m=+5.823268532 container cleanup ace95eee0b0bc89ccd4b2b1acbcee1a0ffb4510ec4722e8e5c61e36db023ce39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T04:59:27.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:27 smithi193 bash[72425]: ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11 2024-09-19T04:59:27.696 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:27 smithi193 podman[72440]: 2024-09-19 04:59:27.355044918 +0000 UTC m=+0.670376033 container remove ace95eee0b0bc89ccd4b2b1acbcee1a0ffb4510ec4722e8e5c61e36db023ce39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T04:59:27.951 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:27 smithi193 podman[72529]: 2024-09-19 04:59:27.599450601 +0000 UTC m=+0.015746997 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:59:27.951 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:27 smithi193 podman[72529]: 2024-09-19 04:59:27.722557669 +0000 UTC m=+0.138854048 container create 1537863a56f42807b48bc5f550ac387e54cebfb3448c9f1967ffd238b70bdff7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T04:59:27.952 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:27 smithi193 podman[72529]: 2024-09-19 04:59:27.951101435 +0000 UTC m=+0.367397832 container init 1537863a56f42807b48bc5f550ac387e54cebfb3448c9f1967ffd238b70bdff7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:59:28.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:27 smithi193 podman[72529]: 2024-09-19 04:59:27.954651782 +0000 UTC m=+0.370948163 container start 1537863a56f42807b48bc5f550ac387e54cebfb3448c9f1967ffd238b70bdff7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T04:59:28.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:28 smithi193 podman[72529]: 2024-09-19 04:59:28.042241569 +0000 UTC m=+0.458537959 container attach 1537863a56f42807b48bc5f550ac387e54cebfb3448c9f1967ffd238b70bdff7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T04:59:28.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 19 04:59:28 smithi193 podman[72529]: 2024-09-19 04:59:28.143145955 +0000 UTC m=+0.559442340 container died 1537863a56f42807b48bc5f550ac387e54cebfb3448c9f1967ffd238b70bdff7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T04:59:28.718 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@osd.11.service' 2024-09-19T04:59:28.755 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:59:28.756 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-19T04:59:28.756 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-19T04:59:28.756 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.a 2024-09-19T04:59:29.155 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:59:28 smithi123 systemd[1]: Stopping Ceph mds.a for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:59:29.426 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:59:29 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-a[70907]: 2024-09-19T04:59:29.152+0000 7fe223c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:59:29.427 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:59:29 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-a[70907]: 2024-09-19T04:59:29.152+0000 7fe223c00640 -1 mds.a *** got signal Terminated *** 2024-09-19T04:59:31.927 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:59:31 smithi123 podman[127769]: 2024-09-19 04:59:31.660180711 +0000 UTC m=+2.617159937 container died 87f469624551f4ec3964683d8d13d595ff02a215d16ee6957ed9f9730a4ff126 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-a, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T04:59:32.410 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:59:32 smithi123 podman[127769]: 2024-09-19 04:59:32.133351118 +0000 UTC m=+3.090330341 container cleanup 87f469624551f4ec3964683d8d13d595ff02a215d16ee6957ed9f9730a4ff126 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T04:59:32.410 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:59:32 smithi123 bash[127769]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-a 2024-09-19T04:59:32.627 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.a.service' 2024-09-19T04:59:32.675 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:59:32 smithi123 podman[127781]: 2024-09-19 04:59:32.409568333 +0000 UTC m=+0.743901308 container remove 87f469624551f4ec3964683d8d13d595ff02a215d16ee6957ed9f9730a4ff126 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:59:32.675 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:59:32 smithi123 systemd[1]: ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.a.service: Deactivated successfully. 2024-09-19T04:59:32.675 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:59:32 smithi123 systemd[1]: Stopped Ceph mds.a for 911fa052-7640-11ef-bceb-c7b262605968. 2024-09-19T04:59:32.675 INFO:journalctl@ceph.mds.a.smithi123.stdout:Sep 19 04:59:32 smithi123 systemd[1]: ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.a.service: Consumed 3.044s CPU time. 2024-09-19T04:59:32.697 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:59:32.698 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-19T04:59:32.698 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-19T04:59:32.698 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.d 2024-09-19T04:59:32.927 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:59:32 smithi123 systemd[1]: Stopping Ceph mds.d for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:59:33.427 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:59:33 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-d[69697]: 2024-09-19T04:59:33.098+0000 7f52f6c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:59:33.427 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:59:33 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-d[69697]: 2024-09-19T04:59:33.098+0000 7f52f6c00640 -1 mds.d *** got signal Terminated *** 2024-09-19T04:59:35.914 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:59:35 smithi123 podman[127893]: 2024-09-19 04:59:35.653366224 +0000 UTC m=+2.674513857 container died fd9818a3f941a28b655f57661934bdc74216d33d3e053c13a8085959bcdde0f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T04:59:36.177 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:59:36 smithi123 podman[127893]: 2024-09-19 04:59:36.062391673 +0000 UTC m=+3.083539306 container cleanup fd9818a3f941a28b655f57661934bdc74216d33d3e053c13a8085959bcdde0f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T04:59:36.177 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:59:36 smithi123 bash[127893]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-d 2024-09-19T04:59:36.535 INFO:journalctl@ceph.mds.d.smithi123.stdout:Sep 19 04:59:36 smithi123 podman[127916]: 2024-09-19 04:59:36.280506748 +0000 UTC m=+0.616905609 container remove fd9818a3f941a28b655f57661934bdc74216d33d3e053c13a8085959bcdde0f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-d, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:59:36.544 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.d.service' 2024-09-19T04:59:36.615 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:59:36.615 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-19T04:59:36.615 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-19T04:59:36.616 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.g 2024-09-19T04:59:36.927 INFO:journalctl@ceph.mds.g.smithi123.stdout:Sep 19 04:59:36 smithi123 systemd[1]: Stopping Ceph mds.g for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:59:37.427 INFO:journalctl@ceph.mds.g.smithi123.stdout:Sep 19 04:59:37 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-g[70566]: 2024-09-19T04:59:37.027+0000 7f4c08a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:59:37.427 INFO:journalctl@ceph.mds.g.smithi123.stdout:Sep 19 04:59:37 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-g[70566]: 2024-09-19T04:59:37.027+0000 7f4c08a00640 -1 mds.g *** got signal Terminated *** 2024-09-19T04:59:41.301 INFO:journalctl@ceph.mds.g.smithi123.stdout:Sep 19 04:59:41 smithi123 podman[128024]: 2024-09-19 04:59:41.037945668 +0000 UTC m=+4.146820245 container died bfae55e6fe727ee0b2fa72c6d6b30be7332ef7dd252d3f06c6bdc88ffa847f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-19T04:59:41.660 INFO:journalctl@ceph.mds.g.smithi123.stdout:Sep 19 04:59:41 smithi123 podman[128024]: 2024-09-19 04:59:41.482817625 +0000 UTC m=+4.591692200 container cleanup bfae55e6fe727ee0b2fa72c6d6b30be7332ef7dd252d3f06c6bdc88ffa847f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-g, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T04:59:41.660 INFO:journalctl@ceph.mds.g.smithi123.stdout:Sep 19 04:59:41 smithi123 bash[128024]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-g 2024-09-19T04:59:41.915 INFO:journalctl@ceph.mds.g.smithi123.stdout:Sep 19 04:59:41 smithi123 podman[128036]: 2024-09-19 04:59:41.659372032 +0000 UTC m=+0.607018652 container remove bfae55e6fe727ee0b2fa72c6d6b30be7332ef7dd252d3f06c6bdc88ffa847f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-g, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T04:59:41.924 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.g.service' 2024-09-19T04:59:41.994 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:59:41.994 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-19T04:59:41.994 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-19T04:59:41.995 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.j 2024-09-19T04:59:42.177 INFO:journalctl@ceph.mds.j.smithi123.stdout:Sep 19 04:59:42 smithi123 systemd[1]: Stopping Ceph mds.j for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:59:42.677 INFO:journalctl@ceph.mds.j.smithi123.stdout:Sep 19 04:59:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-j[69302]: 2024-09-19T04:59:42.406+0000 7faf1e400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:59:42.677 INFO:journalctl@ceph.mds.j.smithi123.stdout:Sep 19 04:59:42 smithi123 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-j[69302]: 2024-09-19T04:59:42.406+0000 7faf1e400640 -1 mds.j *** got signal Terminated *** 2024-09-19T04:59:44.098 INFO:journalctl@ceph.mds.j.smithi123.stdout:Sep 19 04:59:43 smithi123 podman[128143]: 2024-09-19 04:59:43.822219036 +0000 UTC m=+1.551404238 container died 984c514baed29c8198763ac0f40337ebb2008f83dbf070e49b2c956c30e1649e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T04:59:44.427 INFO:journalctl@ceph.mds.j.smithi123.stdout:Sep 19 04:59:44 smithi123 podman[128143]: 2024-09-19 04:59:44.311929767 +0000 UTC m=+2.041114969 container cleanup 984c514baed29c8198763ac0f40337ebb2008f83dbf070e49b2c956c30e1649e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-j, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:59:44.427 INFO:journalctl@ceph.mds.j.smithi123.stdout:Sep 19 04:59:44 smithi123 bash[128143]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-j 2024-09-19T04:59:44.719 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.j.service' 2024-09-19T04:59:44.786 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:59:44.786 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-19T04:59:44.786 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-19T04:59:44.786 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.b 2024-09-19T04:59:45.081 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:59:44 smithi183 systemd[1]: Stopping Ceph mds.b for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:59:45.340 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:59:45 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-b[65203]: 2024-09-19T04:59:45.080+0000 7f0607632640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:59:45.340 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:59:45 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-b[65203]: 2024-09-19T04:59:45.080+0000 7f0607632640 -1 mds.b *** got signal Terminated *** 2024-09-19T04:59:47.874 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:59:47 smithi183 podman[71582]: 2024-09-19 04:59:47.618073547 +0000 UTC m=+2.642315633 container died 711b223d35df9c7e8dc222a60d43d305984c48f35f2156e7de797729fe0d4f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T04:59:48.239 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:59:47 smithi183 podman[71582]: 2024-09-19 04:59:47.971283599 +0000 UTC m=+2.995525723 container cleanup 711b223d35df9c7e8dc222a60d43d305984c48f35f2156e7de797729fe0d4f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-19T04:59:48.239 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 19 04:59:47 smithi183 bash[71582]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-b 2024-09-19T04:59:48.400 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.b.service' 2024-09-19T04:59:48.435 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:59:48.436 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-19T04:59:48.436 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-19T04:59:48.436 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.e 2024-09-19T04:59:48.850 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:59:48 smithi183 systemd[1]: Stopping Ceph mds.e for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:59:48.850 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:59:48 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-e[64601]: 2024-09-19T04:59:48.733+0000 7fb844da5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:59:48.850 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:59:48 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-e[64601]: 2024-09-19T04:59:48.733+0000 7fb844da5640 -1 mds.e *** got signal Terminated *** 2024-09-19T04:59:51.876 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:59:51 smithi183 podman[71717]: 2024-09-19 04:59:51.61661 +0000 UTC m=+2.961216750 container died c61564e5ddbaefbc1effa6bce2c3e4fef5c84476f4484e40b6134db7ea19d41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T04:59:52.251 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:59:51 smithi183 podman[71717]: 2024-09-19 04:59:51.974548739 +0000 UTC m=+3.319155499 container cleanup c61564e5ddbaefbc1effa6bce2c3e4fef5c84476f4484e40b6134db7ea19d41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:59:52.251 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 19 04:59:51 smithi183 bash[71717]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-e 2024-09-19T04:59:52.377 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.e.service' 2024-09-19T04:59:52.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:59:52.411 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-19T04:59:52.411 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-19T04:59:52.411 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.h 2024-09-19T04:59:52.850 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:59:52 smithi183 systemd[1]: Stopping Ceph mds.h for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:59:52.850 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:59:52 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-h[64901]: 2024-09-19T04:59:52.744+0000 7f903d4b6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:59:52.850 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:59:52 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-h[64901]: 2024-09-19T04:59:52.744+0000 7f903d4b6640 -1 mds.h *** got signal Terminated *** 2024-09-19T04:59:55.870 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:59:55 smithi183 podman[71838]: 2024-09-19 04:59:55.602924014 +0000 UTC m=+2.964568815 container died 7bb03127b5ddfbff49f30e57e6d7c2ebd267d7afaccb8b2bdc5e30664d7eb198 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-h, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:59:56.179 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:59:55 smithi183 podman[71838]: 2024-09-19 04:59:55.961137822 +0000 UTC m=+3.322782616 container cleanup 7bb03127b5ddfbff49f30e57e6d7c2ebd267d7afaccb8b2bdc5e30664d7eb198 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T04:59:56.179 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 19 04:59:55 smithi183 bash[71838]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-h 2024-09-19T04:59:56.317 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.h.service' 2024-09-19T04:59:56.351 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:59:56.351 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-19T04:59:56.351 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-19T04:59:56.351 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.k 2024-09-19T04:59:56.850 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:59:56 smithi183 systemd[1]: Stopping Ceph mds.k for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T04:59:56.850 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:59:56 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-k[64288]: 2024-09-19T04:59:56.680+0000 7f5d4dc43640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:59:56.850 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:59:56 smithi183 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-k[64288]: 2024-09-19T04:59:56.680+0000 7f5d4dc43640 -1 mds.k *** got signal Terminated *** 2024-09-19T04:59:59.881 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:59:59 smithi183 podman[71958]: 2024-09-19 04:59:59.609015003 +0000 UTC m=+3.041720045 container died a5492af672c9e3906481622d003c142b60ca68889d0979b85f6548b34ba0c0b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-k, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T05:00:00.238 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:59:59 smithi183 podman[71958]: 2024-09-19 04:59:59.880842706 +0000 UTC m=+3.313547748 container cleanup a5492af672c9e3906481622d003c142b60ca68889d0979b85f6548b34ba0c0b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-k, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T05:00:00.238 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 04:59:59 smithi183 bash[71958]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-k 2024-09-19T05:00:00.238 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 19 05:00:00 smithi183 podman[71973]: 2024-09-19 05:00:00.114997689 +0000 UTC m=+0.508389914 container remove a5492af672c9e3906481622d003c142b60ca68889d0979b85f6548b34ba0c0b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T05:00:00.259 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.k.service' 2024-09-19T05:00:00.291 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:00:00.291 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-19T05:00:00.291 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-19T05:00:00.291 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.c 2024-09-19T05:00:00.605 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 05:00:00 smithi193 systemd[1]: Stopping Ceph mds.c for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T05:00:00.990 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 05:00:00 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-c[65683]: 2024-09-19T05:00:00.603+0000 7f636b9b3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:00:00.991 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 05:00:00 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-c[65683]: 2024-09-19T05:00:00.603+0000 7f636b9b3640 -1 mds.c *** got signal Terminated *** 2024-09-19T05:00:03.931 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 05:00:03 smithi193 podman[72642]: 2024-09-19 05:00:03.623888289 +0000 UTC m=+3.131142499 container died dafb2eeaedcb7a6fc35275f6ee9e349d225d0d1bd39d6d5aec97ca6a546c05bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:00:04.240 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 05:00:04 smithi193 podman[72642]: 2024-09-19 05:00:04.003640862 +0000 UTC m=+3.510895071 container cleanup dafb2eeaedcb7a6fc35275f6ee9e349d225d0d1bd39d6d5aec97ca6a546c05bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:00:04.241 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 19 05:00:04 smithi193 bash[72642]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-c 2024-09-19T05:00:04.437 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.c.service' 2024-09-19T05:00:04.474 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:00:04.475 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-19T05:00:04.475 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-19T05:00:04.475 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.f 2024-09-19T05:00:04.792 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 05:00:04 smithi193 systemd[1]: Stopping Ceph mds.f for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T05:00:05.240 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 05:00:04 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-f[65382]: 2024-09-19T05:00:04.790+0000 7f908b3fd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:00:05.241 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 05:00:04 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-f[65382]: 2024-09-19T05:00:04.790+0000 7f908b3fd640 -1 mds.f *** got signal Terminated *** 2024-09-19T05:00:07.909 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 05:00:07 smithi193 podman[72775]: 2024-09-19 05:00:07.606282516 +0000 UTC m=+2.914469504 container died 0f3bca754953689d7fae833f8e81f711a64503ca4a6bfe6104811f59c01a5fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T05:00:08.235 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 05:00:08 smithi193 podman[72775]: 2024-09-19 05:00:08.016172696 +0000 UTC m=+3.324359683 container cleanup 0f3bca754953689d7fae833f8e81f711a64503ca4a6bfe6104811f59c01a5fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-f, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T05:00:08.235 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 19 05:00:08 smithi193 bash[72775]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-f 2024-09-19T05:00:08.444 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.f.service' 2024-09-19T05:00:08.478 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:00:08.478 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-19T05:00:08.479 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-19T05:00:08.479 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.i 2024-09-19T05:00:08.804 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 05:00:08 smithi193 systemd[1]: Stopping Ceph mds.i for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T05:00:09.240 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 05:00:08 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-i[65988]: 2024-09-19T05:00:08.802+0000 7f8ff63d9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:00:09.240 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 05:00:08 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-i[65988]: 2024-09-19T05:00:08.802+0000 7f8ff63d9640 -1 mds.i *** got signal Terminated *** 2024-09-19T05:00:11.871 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 05:00:11 smithi193 podman[72896]: 2024-09-19 05:00:11.609010857 +0000 UTC m=+2.914292360 container died 00b1c8207448b65c608ee3be07da182ced6fa2d63f1614a61114af18a74789ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-19T05:00:12.233 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 05:00:11 smithi193 podman[72896]: 2024-09-19 05:00:11.969943566 +0000 UTC m=+3.275225115 container cleanup 00b1c8207448b65c608ee3be07da182ced6fa2d63f1614a61114af18a74789ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-i, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T05:00:12.233 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 19 05:00:11 smithi193 bash[72896]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-i 2024-09-19T05:00:12.448 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.i.service' 2024-09-19T05:00:12.482 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:00:12.483 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-19T05:00:12.483 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-19T05:00:12.483 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.l 2024-09-19T05:00:12.795 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 05:00:12 smithi193 systemd[1]: Stopping Ceph mds.l for 911fa052-7640-11ef-bceb-c7b262605968... 2024-09-19T05:00:13.240 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 05:00:12 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-l[65064]: 2024-09-19T05:00:12.832+0000 7fab2f7ef640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:00:13.240 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 05:00:12 smithi193 ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-l[65064]: 2024-09-19T05:00:12.832+0000 7fab2f7ef640 -1 mds.l *** got signal Terminated *** 2024-09-19T05:00:15.924 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 05:00:15 smithi193 podman[73015]: 2024-09-19 05:00:15.609214025 +0000 UTC m=+2.909144128 container died c44754603a223b4e9c2e38c6a5a0800a9c26a0fbc61b3ae888f0eddbbb767ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T05:00:15.924 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 05:00:15 smithi193 podman[73015]: 2024-09-19 05:00:15.924018826 +0000 UTC m=+3.223948917 container cleanup c44754603a223b4e9c2e38c6a5a0800a9c26a0fbc61b3ae888f0eddbbb767ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-l, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:00:16.240 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 05:00:15 smithi193 bash[73015]: ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-l 2024-09-19T05:00:16.240 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 19 05:00:16 smithi193 podman[73027]: 2024-09-19 05:00:16.13375299 +0000 UTC m=+0.517762997 container remove c44754603a223b4e9c2e38c6a5a0800a9c26a0fbc61b3ae888f0eddbbb767ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-911fa052-7640-11ef-bceb-c7b262605968-mds-l, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T05:00:16.350 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-911fa052-7640-11ef-bceb-c7b262605968@mds.l.service' 2024-09-19T05:00:16.382 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:00:16.382 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-19T05:00:16.382 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 911fa052-7640-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T05:00:16.656 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T05:00:20.601 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 911fa052-7640-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T05:00:20.799 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T05:00:23.302 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 911fa052-7640-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T05:00:23.503 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T05:00:25.909 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:00:25.974 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:00:26.007 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:00:26.034 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T05:00:26.037 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911638/remote/smithi123/crash 2024-09-19T05:00:26.038 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/crash -- . 2024-09-19T05:00:26.117 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T05:00:26.118 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now 2024-09-19T05:00:26.119 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911638/remote/smithi183/crash 2024-09-19T05:00:26.120 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/crash -- . 2024-09-19T05:00:26.155 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T05:00:26.155 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2024-09-19T05:00:26.157 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911638/remote/smithi193/crash 2024-09-19T05:00:26.158 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/crash -- . 2024-09-19T05:00:26.197 INFO:teuthology.orchestra.run.smithi193.stderr:tar: /var/lib/ceph/911fa052-7640-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T05:00:26.197 INFO:teuthology.orchestra.run.smithi193.stderr:tar: Error is not recoverable: exiting now 2024-09-19T05:00:26.198 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T05:00:26.198 DEBUG:teuthology.orchestra.run.smithi123:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T05:00:26.284 INFO:tasks.cephadm:Compressing logs... 2024-09-19T05:00:26.284 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T05:00:26.288 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T05:00:26.291 DEBUG:teuthology.orchestra.run.smithi193:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T05:00:26.315 INFO:teuthology.orchestra.run.smithi193.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T05:00:26.315 INFO:teuthology.orchestra.run.smithi193.stderr:: No such file or directory 2024-09-19T05:00:26.315 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T05:00:26.315 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-19T05:00:26.316 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-19T05:00:26.316 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T05:00:26.317 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-19T05:00:26.322 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.audit.log: /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.i.log: /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mon.c.log: /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T05:00:26.322 INFO:teuthology.orchestra.run.smithi193.stderr: 81.8% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T05:00:26.322 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T05:00:26.323 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.log 2024-09-19T05:00:26.323 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.2.log: /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.5.log: /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.log: 90.5% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T05:00:26.323 INFO:teuthology.orchestra.run.smithi193.stderr: 87.3% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T05:00:26.323 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-19T05:00:26.323 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-19T05:00:26.323 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-19T05:00:26.328 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-19T05:00:26.328 INFO:teuthology.orchestra.run.smithi183.stderr:: No such file or directory 2024-09-19T05:00:26.328 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T05:00:26.328 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T05:00:26.328 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T05:00:26.328 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-19T05:00:26.328 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-19T05:00:26.328 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T05:00:26.328 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T05:00:26.328 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T05:00:26.328 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.audit.log: 90.5% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T05:00:26.328 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-19T05:00:26.329 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.cephadm.log: 81.8% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T05:00:26.329 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.log 2024-09-19T05:00:26.329 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.8.log: 91.0% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T05:00:26.333 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mgr.y.log: 91.0% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T05:00:26.338 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.0.log: No such file or directory 2024-09-19T05:00:26.338 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-19T05:00:26.339 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-19T05:00:26.339 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-19T05:00:26.339 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-19T05:00:26.340 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-19T05:00:26.340 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T05:00:26.340 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T05:00:26.341 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-19T05:00:26.341 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-19T05:00:26.342 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.h.log: 87.3% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T05:00:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T05:00:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-19T05:00:26.345 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-19T05:00:26.345 INFO:teuthology.orchestra.run.smithi123.stderr: 84.4% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T05:00:26.346 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mon.a.log: 90.4% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T05:00:26.346 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T05:00:26.347 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.log 2024-09-19T05:00:26.347 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-19T05:00:26.350 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mgr.z.log: 92.0% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-19T05:00:26.351 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.6.log: /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-19T05:00:26.352 INFO:teuthology.orchestra.run.smithi123.stderr: 94.6% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-19T05:00:26.352 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-19T05:00:26.353 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-19T05:00:26.353 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-19T05:00:26.356 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mgr.x.log: 91.0% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T05:00:26.356 INFO:teuthology.orchestra.run.smithi123.stderr: 87.3% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T05:00:26.359 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T05:00:26.360 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T05:00:26.364 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T05:00:26.366 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T05:00:26.371 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-19T05:00:26.375 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T05:00:26.375 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T05:00:26.392 INFO:teuthology.orchestra.run.smithi183.stderr: 94.1% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-19T05:00:26.395 INFO:teuthology.orchestra.run.smithi123.stderr: 94.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T05:00:26.418 INFO:teuthology.orchestra.run.smithi123.stderr: 94.1% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-19T05:00:26.448 INFO:teuthology.orchestra.run.smithi123.stderr: 94.2% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-19T05:00:26.464 INFO:teuthology.orchestra.run.smithi183.stderr: 93.5% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-19T05:00:26.639 INFO:teuthology.orchestra.run.smithi123.stderr: 90.2% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-19T05:00:26.725 INFO:teuthology.orchestra.run.smithi183.stderr: 94.2% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-19T05:00:26.741 INFO:teuthology.orchestra.run.smithi193.stderr: 94.1% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-19T05:00:26.786 INFO:teuthology.orchestra.run.smithi193.stderr: 94.1% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-19T05:00:26.835 INFO:teuthology.orchestra.run.smithi193.stderr: 91.6% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-19T05:00:26.922 INFO:teuthology.orchestra.run.smithi183.stderr: 91.6% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-19T05:00:27.122 INFO:teuthology.orchestra.run.smithi183.stderr: 93.9% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-19T05:00:27.660 INFO:teuthology.orchestra.run.smithi123.stderr: 90.4% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-19T05:00:29.050 INFO:teuthology.orchestra.run.smithi193.stderr: 96.0% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-19T05:00:31.930 INFO:teuthology.orchestra.run.smithi183.stderr: 91.8% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T05:00:32.881 INFO:teuthology.orchestra.run.smithi123.stderr: 91.5% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-19T05:00:32.969 INFO:teuthology.orchestra.run.smithi183.stderr: 92.2% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-19T05:00:33.592 INFO:teuthology.orchestra.run.smithi123.stderr: 91.9% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T05:00:34.435 INFO:teuthology.orchestra.run.smithi193.stderr: 91.8% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-19T05:00:34.765 INFO:teuthology.orchestra.run.smithi183.stderr: 92.1% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T05:00:36.118 INFO:teuthology.orchestra.run.smithi193.stderr: 91.4% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T05:00:36.493 INFO:teuthology.orchestra.run.smithi193.stderr: 91.7% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-19T05:00:36.704 INFO:teuthology.orchestra.run.smithi123.stderr: 91.9% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T05:00:38.224 INFO:teuthology.orchestra.run.smithi193.stderr: 91.9% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T05:00:38.386 INFO:teuthology.orchestra.run.smithi183.stderr: 91.6% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T05:00:38.388 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-09-19T05:00:38.388 INFO:teuthology.orchestra.run.smithi183.stderr:real 0m12.082s 2024-09-19T05:00:38.388 INFO:teuthology.orchestra.run.smithi183.stderr:user 0m33.562s 2024-09-19T05:00:38.389 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m1.113s 2024-09-19T05:00:38.831 INFO:teuthology.orchestra.run.smithi123.stderr: 91.4% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T05:00:38.835 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-19T05:00:38.835 INFO:teuthology.orchestra.run.smithi123.stderr:real 0m12.518s 2024-09-19T05:00:38.835 INFO:teuthology.orchestra.run.smithi123.stderr:user 0m35.480s 2024-09-19T05:00:38.835 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m2.883s 2024-09-19T05:01:05.730 INFO:teuthology.orchestra.run.smithi193.stderr: 93.2% -- replaced with /var/log/ceph/911fa052-7640-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-19T05:01:05.733 INFO:teuthology.orchestra.run.smithi193.stderr: 2024-09-19T05:01:05.733 INFO:teuthology.orchestra.run.smithi193.stderr:real 0m39.427s 2024-09-19T05:01:05.733 INFO:teuthology.orchestra.run.smithi193.stderr:user 1m20.454s 2024-09-19T05:01:05.733 INFO:teuthology.orchestra.run.smithi193.stderr:sys 0m2.512s 2024-09-19T05:01:05.733 INFO:tasks.cephadm:Archiving logs... 2024-09-19T05:01:05.733 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911638/remote/smithi123/log 2024-09-19T05:01:05.734 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T05:01:12.418 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911638/remote/smithi183/log 2024-09-19T05:01:12.419 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T05:01:18.163 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911638/remote/smithi193/log 2024-09-19T05:01:18.164 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T05:01:30.226 INFO:tasks.cephadm:Removing cluster... 2024-09-19T05:01:30.227 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 911fa052-7640-11ef-bceb-c7b262605968 --force 2024-09-19T05:01:30.502 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T05:01:31.075 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 911fa052-7640-11ef-bceb-c7b262605968 --force 2024-09-19T05:01:31.277 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T05:01:31.590 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 911fa052-7640-11ef-bceb-c7b262605968 --force 2024-09-19T05:01:31.793 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 911fa052-7640-11ef-bceb-c7b262605968 2024-09-19T05:01:32.245 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T05:01:32.245 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T05:01:32.285 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T05:01:32.302 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T05:01:32.319 INFO:tasks.cephadm:Teardown complete 2024-09-19T05:01:32.319 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T05:01:32.351 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T05:01:32.351 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T05:01:32.356 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T05:01:32.358 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T05:01:32.482 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T05:01:32.493 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T05:01:32.493 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:01:32.497 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:01:32.500 DEBUG:teuthology.orchestra.run.smithi193:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:01:32.513 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:01:32.515 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:01:32.517 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:01:32.517 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-09-19T05:01:32.517 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv01.front.sepia.ceph.com 3 6 377 19 -33us[ -20us] +/- 63ms 2024-09-19T05:01:32.517 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 +196us[ +209us] +/- 61ms 2024-09-19T05:01:32.517 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 -188us[ -175us] +/- 70ms 2024-09-19T05:01:32.517 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:01:32.518 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:01:32.518 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-09-19T05:01:32.518 INFO:teuthology.orchestra.run.smithi193.stdout:^* hv01.front.sepia.ceph.com 3 6 377 24 -26us[ -10us] +/- 63ms 2024-09-19T05:01:32.518 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 +203us[ +203us] +/- 61ms 2024-09-19T05:01:32.518 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -179us[ -163us] +/- 70ms 2024-09-19T05:01:32.519 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:01:32.527 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:01:32.534 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:01:32.534 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-09-19T05:01:32.534 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 -38us[ -11us] +/- 63ms 2024-09-19T05:01:32.534 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 +216us[ +244us] +/- 61ms 2024-09-19T05:01:32.534 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 -167us[ -167us] +/- 70ms 2024-09-19T05:01:32.534 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:01:32.535 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T05:01:32.546 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T05:01:32.546 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T05:01:32.569 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T05:01:32.707 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T05:01:32.779 DEBUG:teuthology.orchestra.run.smithi193:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T05:01:32.847 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:01:32.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:01:32.908 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:01:32.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:01:32.938 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:01:32.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:01:32.966 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T05:01:32.977 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T05:01:33.003 INFO:teuthology.task.internal:Duration was 1983.706342 seconds 2024-09-19T05:01:33.004 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T05:01:33.037 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T05:01:33.037 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T05:01:33.041 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T05:01:33.044 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T05:01:33.082 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:01:33.083 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:01:33.126 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:01:33.398 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T05:01:33.399 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T05:01:33.399 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T05:01:33.451 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-09-19T05:01:33.451 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T05:01:33.476 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi193.front.sepia.ceph.com 2024-09-19T05:01:33.477 DEBUG:teuthology.orchestra.run.smithi193:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T05:01:33.503 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T05:01:33.504 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T05:01:33.508 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T05:01:33.519 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T05:01:33.570 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T05:01:33.570 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T05:01:33.575 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T05:01:33.586 DEBUG:teuthology.orchestra.run.smithi193:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T05:01:34.872 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T05:01:34.884 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T05:01:34.884 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T05:01:34.941 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T05:01:34.971 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T05:01:35.002 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T05:01:35.013 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:01:35.018 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:01:35.020 DEBUG:teuthology.orchestra.run.smithi193:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:01:35.046 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-09-19T05:01:35.070 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = core 2024-09-19T05:01:35.084 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-09-19T05:01:35.244 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:01:35.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:01:35.278 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:01:35.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:01:35.296 DEBUG:teuthology.orchestra.run.smithi193:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:01:35.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:01:35.313 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T05:01:35.324 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T05:01:35.325 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911638/remote/smithi123 2024-09-19T05:01:35.325 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T05:01:35.415 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911638/remote/smithi183 2024-09-19T05:01:35.415 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T05:01:35.456 DEBUG:teuthology.misc:Transferring archived files from smithi193:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911638/remote/smithi193 2024-09-19T05:01:35.456 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T05:01:35.501 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T05:01:35.501 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T05:01:35.505 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T05:01:35.508 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T05:01:35.549 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T05:01:35.564 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T05:01:35.564 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T05:01:35.589 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T05:01:35.589 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T05:01:35.593 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T05:01:35.596 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T05:01:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 05:01 /home/ubuntu/cephtest 2024-09-19T05:01:35.613 INFO:teuthology.orchestra.run.smithi193.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 05:01 /home/ubuntu/cephtest 2024-09-19T05:01:35.624 INFO:teuthology.orchestra.run.smithi123.stdout: 393754 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 05:01 /home/ubuntu/cephtest 2024-09-19T05:01:35.626 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T05:01:35.637 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T05:01:35.757 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/fsx}} duration: 1983.706342458725 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T05:01:35.757 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T05:01:35.847 INFO:teuthology.run:pass